mgr/balancer: make weekday be compatible with C

The __desc__ field of **begin[end]_weekend** suggests
sunday is the prevailing start of the week, whereas the
Python implementation says no.

See https://stackoverflow.com/questions/21045569/why-is-pythons-weekday-different-from-tm-wday-in-c

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
(cherry picked from commit b9da0925490d4e9e964249c54e7775d9c76b1719)

mgr/balancer: restrict automatic balancing to specific weekdays

We have implemented similar semantics for the scrub behaviour.
Let's try to keep pace with that.

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
(cherry picked from commit 0412a51ac6dfecc1a72b654196ecb1a599dcc94d)

Conflicts:
- path: src/pybind/mgr/balancer/module.py
  comment: 'get_module_option()' -> 'get_config()' for mgr config-key
    'begin_weekday' and 'begin_weekday' should be int, but
    luminous/mimic config-key always return string.

Merge pull request #25720 from ashishkumsingh/wip-36692-luminous

luminous: rbd-mirror: update mirror status when stopping

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #25768 from ashishkumsingh/wip-26913-luminous

luminous: mgr: "balancer execute" only requires read permissions

Reviewed-by: Jan Fajerski <jfajerski@suse.com>

Merge pull request #25955 from pdvian/wip-37901-luminous

luminous: [journal] max journal order is incorrectly set at 64

Reviewed-by: Mykola Golub <mgolub@mirantis.com>

Merge pull request #26000 from pdvian/wip-37926-luminous

luminous: common: fix for broken rbdmap parameter parsing

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #26286 from smithfarm/wip-38186-luminous

luminous: tests: krbd discard no longer guarantees zeroing

Reviewed-by: Ilya Dryomov <idryomov@redhat.com>

Merge pull request #25990 from ukernel/luminous-37760

luminous: mds: handle fragment notify race

Reviewed-by: Venky Shankar <vshankar@redhat.com>

Merge pull request #26005 from ukernel/luminous-37481

luminous: mds: handle state change race

Reviewed-by: Venky Shankar <vshankar@redhat.com>

Merge pull request #26257 from batrick/i38084

luminous: mds: log new client sessions with various metadata

Reviewed-by: Venky Shankar <vshankar@redhat.com>

Merge pull request #26271 from batrick/i38104

luminous: client: session flush does not cause cap release message flush

Reviewed-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Nathan Cutler <ncutler@suse.com>

Merge pull request #26278 from ukernel/luminous-stale-caps

luminous: mds: optimize revoking stale caps

Reviewed-by: Venky Shankar <vshankar@redhat.com>

Merge pull request #26326 from alfredodeza/luminous-rm38229

luminous: doc: pin the version for "breathe" to 4.1.11

Reviewed-by: Andrew Schoen <aschoen@redhat.com>

doc: pin the version for `breathe` to 4.1.11

It has the compatibility needed for Sphinx version 1.6.3

Signed-off-by: Alfredo Deza <adeza@redhat.com>

Merge pull request #26295 from jan--f/25390-luminous

[luminous] ceph-volume: revert partition as disk

Reviewed-by: Andrew Schoen <aschoen@redhat.com>

ceph-volume: add inventory call to functional tests

Signed-off-by: Jan Fajerski <jfajerski@suse.com>
(cherry picked from commit a183a28c5e423c5658a37f7a1d3add9d64c6d683)

ceph-volume: add partition support to Device

Signed-off-by: Jan Fajerski <jfajerski@suse.com>
(cherry picked from commit 05ac680cb37fd15fdeadaea86780bf1b4f2d566a)

Revert "ceph-volume tests.util ensure partitions are listed as top-level keys"

This reverts commit 6dc01772245e557b73551111f7c48cc60aec0893.

Signed-off-by: Jan Fajerski <jfajerski@suse.com>
(cherry picked from commit 3c1c32cbc2736d72295f1bd7921c4a3cd2f4727e)

Revert "ceph-volume util.disk include partitions as top level devices in get_devices"

This reverts commit 7f3c3598491e556ec1b93ea0a54f52144202fab6.
Fixes: http://tracker.ceph.com/issues/37506
Signed-off-by: Jan Fajerski <jfajerski@suse.com>
(cherry picked from commit 50bc44f1c2327ad38504950fe62820a44ead9b14)

qa: update krbd tests for zeroout

Discard no longer guarantees zeroing, use BLKZEROOUT and "fallocate -z"
instead (blkdiscard(8) in xenial doesn't support -z).

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
(cherry picked from commit 04f5b343f9987f0cf67509c2a76ddf7ef46288e5)

Conflicts:
qa/rbd/krbd_whole_object_zeroout.t
qa/suites/krbd/basic/tasks/krbd_whole_object_zeroout.yaml
- the modifications to these files are not relevant for luminous

mds: optimize resuming stale caps

If client doesn't want any cap, there is no need to re-issue stale
caps.

A special case is that client wants some caps, but skipped updating
'wanted'. For this case, client needs to update 'wanted' when stale
session get renewed.

Fixes: http://tracker.ceph.com/issues/38043
Signed-off-by: "Yan, Zheng" <zyan@redhat.com>
(cherry picked from commit e824b3d2024db36789fdf579f0af9bf3bbe55d51)

 Conflicts:
src/client/Client.cc
src/mds/cephfs_features.h

client: avoid unnecessary wakeup when handling RENEWCAPS

Signed-off-by: "Yan, Zheng" <zyan@redhat.com>
(cherry picked from commit c744bc1673ca0b4e50f41516fdf49c8560db073a)

client: set cap->wanted when adding new cap

This avoids unnecessary cap message if cap is added by open/create
request reply.

Signed-off-by: "Yan, Zheng" <zyan@redhat.com>
(cherry picked from commit 314660a46a8a1af97f70b2bac05b2f6fa5d23bc4)

 Conflicts:
src/client/Client.cc
src/client/Client.h

client: don't wakeup cap waiters twice when mds recovered

Both kick_maxsize_requests() and wake_inode_waiters() wake up cap
waiters

Signed-off-by: "Yan, Zheng" <zyan@redhat.com>
(cherry picked from commit 5993a93e606d23ef311831863444294d7c64cc04)

 Conflicts:
src/client/Client.cc
src/client/Client.h

mds: optimize revoking stale caps

For caps that are not being revoked and don't have writeable range
and don't want exclusive caps or file read/write. there is no need
to call Locker::revoke_stale_caps(Capability*). Because these caps
don't need recover and don't affect eval_gather()/try_eval().

Signed-off-by: "Yan, Zheng" <zyan@redhat.com>
(cherry picked from commit cb62030f0468fc04735c1b4cff73da779cb11ad8)

 Conflicts:
src/mds/CInode.cc
src/mds/Capability.h
src/mds/Locker.cc
src/mds/Migrator.cc

mds: put notable caps at the front of session's caps list

Notable Capabilities are ones that are being revoked, ones that
have writeable ranges and ones that want exclusive caps or want
file read/write.

Signed-off-by: "Yan, Zheng" <zyan@redhat.com>
(cherry picked from commit cb6e7184458e64720ad83c266e5e393a80c32697)

 Conflicts:
src/mds/Capability.cc

mds: track if client has writeable range in Capability

Signed-off-by: "Yan, Zheng" <zyan@redhat.com>
(cherry picked from commit 370ae1cb3e5dc07867d80e998082bc514e8fccfd)

 Conflicts:
src/mds/Capability.h
src/mds/Locker.cc
src/mds/MDCache.h
src/mds/Server.cc

mds: add session pointer to Capability

Signed-off-by: "Yan, Zheng" <zyan@redhat.com>
(cherry picked from commit 7c628472a86c6acebb20f0a2504744b10f250587)

 Conflicts:
src/mds/CInode.cc
src/mds/Capability.cc
src/mds/Capability.h
src/mds/Locker.cc

client: skip updating 'wanted' caps if caps are already issued

When reading cached inode that already has Fscr caps, this can avoid
two cap messages (one updats 'wanted' caps, one clears 'wanted' caps).

Signed-off-by: "Yan, Zheng" <zyan@redhat.com>
(cherry picked from commit d20b260ecf3f323c87ad1e6865f87e2381444546)

 Conflicts:
src/client/Client.cc

client: sync 'retain caps' logical from kernel client

The main change is keeping CEPH_CAP_ANY_RD for unused file inodes

Signed-off-by: "Yan, Zheng" <zyan@redhat.com>
(cherry picked from commit 29034396398b1706db625e1abf2e6682be7130a5)

Merge pull request #26207 from vumrao/wip-vumrao-38108

luminous: mgr/DaemonServer: log pgmap usage to cluster log

Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge pull request #26186 from tchaikov/luminous-36737

luminous: build/ops: Allow multi instances of "make tests" on the same machine

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Yuri Weinstein <yweins@redhat.com>
Reviewed-by: Willem Jan Withagen <wjw@digiware.nl>

client: flush cap release on session flush

MDS expects the completed cap release prior to responding to the session flush
for cache drop.

Fixes: http://tracker.ceph.com/issues/38009
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit 59eca7a17816317d5064840745ea5e28d3349d1a)

Conflicts:
src/client/Client.cc

mds: do not log internal journal error

MDS can't reach this code due to assert in C_MDS_session_finish.

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit 0992ac1fa62edb6143efc80a6fea7b4704107705)

Conflicts:
src/mds/Server.cc

mds: clean up log level 2 messages

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit a225c266052d09ffcb066aead83a18dec918aa35)

Conflicts:
src/mds/MDSRank.cc

mds: log session open stats

This is to assist with tracking what clients are connecting and how long it
takes for them to open a session (a rough estimate of how long it takes to
mount).

Example line:

    2018-12-17 17:21:31.682 7f822703c700  2 mds.0.server New client session: addr="127.0.0.1:60552/2307908923",elapsed=0.013384,throttled=0.000071,status="ACCEPTED",root="/"

Fixes: http://tracker.ceph.com/issues/37678
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit c89cd8cac16789ed98dc1487ec17f45adf03dd26)

Conflicts:
src/mds/Server.cc
src/mds/SessionMap.cc
src/mds/SessionMap.h
src/mds/events/ESession.h

cmake: use $CMAKE_BINARY_DIR for default $CEPH_BUILD_VIRTUALENV

to avoid conflict with other `make check` runs in the same host

See-also: http://tracker.ceph.com/issues/36737
Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit a8d6b2170dfa0838dad0d1b7b03fc8cf19e4eeff)

Conflicts:
- drop the change to mgr/*/run-tox.sh, as they don't exist in
luminous.
        - apply the change to ceph-disk/run-tox.sh
- apply the change to ceph-helpers.sh: as it set $PATH using
          $TMPDIR, which is set to /tmp if $TMPDIR is empty.

mgr/DaemonServer: log pgmap usage to cluster log

- change default of mon_cluster_log_file_level to debug
- log to clog->debug()

Fixes: https://tracker.ceph.com/issues/37886
Signed-off-by: Neha Ojha <nojha@redhat.com>
(cherry picked from commit f64a69e5054fb319e9f717bbcb7ddd8541e01c6f)

 Conflicts:
src/common/options.cc
        dropped master .add_service, .add_see_also

12.2.11

test: listen on random port in tests which start ceph-mon

See-also: http://tracker.ceph.com/issues/36737
Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit 94a84b6f5adad75b3cbf956447ee5d9caf1018d2)

test: write log file to current directory

instead of writing log files to '/tmp', write them to current directory.

by default, ctest runs the tests in ${CMAKE_CURRENT_BINARY_DIR}, so, in
this case, the log files are located in build/src/test/. this ensure
that current user has enough previledge to remove the log files.

this also allows multiple instances of 'unittest_log' to run in parallel.

See-also: http://tracker.ceph.com/issues/36737
Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit a13a6f1d0b39a541896591cd04fc1980bb1acfee)

Conflicts:
src/log/test.cc: trivial resolution

Merge pull request #26117 from ceph/backport-luminous-26084

luminous: ceph-volume: mark a device not available if it belongs to ceph-disk

Merge pull request #26114 from alfredodeza/luminous-rm37963

luminous ceph-volume fix TypeError on dmcrypt when using Python3

Reviewed-by: Andrew Schoen <aschoen@redhat.com>

ceph-volume: mark a device not available if it belongs to ceph-disk

The `ceph-volume inventory` command will now show if a device is being
used by ceph-disk and mark it not available if so.

Fixes: https://tracker.ceph.com/issues/24871
Signed-off-by: Andrew Schoen <aschoen@redhat.com>
(cherry picked from commit d14a6b284c44a3f61c0dd8c60ccc474203003653)

ceph-volume tests verify behavior of create_dmcrypt_key in python 3

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 69613cc5a8b2e111e24b75ad5b69f924c89fd75b)

ceph-volume util.encryption ensure int is used for urandom (py3 compat)

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 0ab9fa62d6d46464b16f08d3a5bb3da50f5030e7)

Merge pull request #26088 from ukernel/luminous-37977

luminous: mds: fix infinite loop in OpTracker::check_ops_in_flight

Reviewed-By: Patrick Donelly <pdonelly@redhat.com>

mds: fix infinite loop in OpTracker::check_ops_in_flight

introduced by backport commit 02faf3d
"mds: don't report slow request for blocked filelock request"

Fixes: http://tracker.ceph.com/issues/37977
Signed-off-by: "Yan, Zheng" <zyan@redhat.com>

Revert "mds: fix infinite loop in OpTracker::check_ops_in_flight"

This reverts commit adad2d873a37770e4910b04dfddf92f70964be4a.
Signed-off-by: "Yan, Zheng" <zyan@redhat.com>

Merge pull request #26037 from yuriw/wip-yuriw-p2p-luminous

luminous: qa/tests: changed to `supported' distro and moved all `p2p` suites un…

Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge pull request #25867 from ashishkumsingh/wip-37827-luminous

luminous: mgr: fix crash due to multiple sessions from daemons with same name

qa/tests: changed to `supported' distro and moved all `p2p` suites under the same folder

Signed-off-by: Yuri Weinstein <yweinste@redhat.com>

Merge pull request #26061 from yuriw/wip-yuriw-p2p-luminous_3

qa/tests: changed start point from `luminous` to v12.2.10

Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge pull request #26048 from ukernel/luminous-37977

luminous: mds: fix infinite loop in OpTracker::check_ops_in_flight

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #26036 from noahdesu/luminous-pr26015

luminous: cli: dump osd-fsid as part of osd find <id>

Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge pull request #26030 from alfredodeza/luminous-fix_raw_input

luminous ceph-volume: Adapt code to support Python3

Reviewed-by: Andrew Schoen <aschoen@redhat.com>

qa/tests: changed start point from `luminous` to v12.2.10

Signed-off-by: Yuri Weinstein <yweinste@redhat.com>

Merge pull request #26014 from alfredodeza/luminous-bz1644847

luminous ceph-volume zap devices associated with an OSD ID and/or OSD FSID

Reviewed-by: Andrew Schoen <aschoen@redhat.com>

Merge pull request #25553 from jan--f/25238-luminous

luminous ceph-volume: introduce class hierachy for strategies

Reviewed-by: Alfredo Deza <adeza@redhat.com>

mds: fix infinite loop in OpTracker::check_ops_in_flight

introduced by backport commit 02faf3dc321
"mds: don't report slow request for blocked filelock request"

Fixes: http://tracker.ceph.com/issues/37977
Signed-off-by: "Yan, Zheng" <zyan@redhat.com>

Merge pull request #25307 from trociny/wip-37438-luminous

luminous: crushtool: add --reclassify operation to convert legacy crush maps to use device classes

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #25949 from neha-ojha/wip-36686-luminous

luminous: osd/mon: pg log hard limit with upgrades fixed

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #25804 from ashishkumsingh/wip-37758-luminous

luminous: mds: clean up log messages for standby-replay

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Yan, Zheng <zyan@redhat.com>

Merge pull request #25904 from pdvian/wip-37829-luminous

luminous : client: fix fuse client hang because its pipe to mds is not ok4

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #26011 from batrick/i37953

luminous: qa: test_damage needs to silence MDS_READ_ONLY

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

cli: dump osd-fsid as part of osd find <id>

Dumps the osd-fsid uuid as part of the `osd find <id>` command.
Currently this uuid is only available as part of `osd dump` but
ceph-ansible has a use case to interrogate a single osd without needing
the entire osdmap dump.

Fixes: http://tracker.ceph.com/issues/37966
Signed-off-by: Noah Watkins <noahwatkins@gmail.com>
(cherry picked from commit 4d01b2c96e0ea1122225c30da21bc39c40e15c0e)

doc: pglog_hardlimit flag recommendations

Signed-off-by: Neha Ojha <nojha@redhat.com>

qa/suites/upgrade/jewel-x/stress-split*: require-osd-release luminous after upgrade

Signed-off-by: Neha Ojha <nojha@redhat.com>

include/rados.h: hide CEPH_OSDMAP_PGLOG_HARDLIMIT from ceph -s

Signed-off-by: Neha Ojha <nojha@redhat.com>
(cherry picked from commit dfc292923548fe1e1b5329555dd46949feb96b99)

qa/suites/upgrade/luminous-p2p-stress-split: add split scenario

This commit adds stress-split test cases to test luminous against a point
release of luminous.

Signed-off-by: Neha Ojha <nojha@redhat.com>

qa/suites/upgrade/jewel-x: add pg log settings

- vary pg log lengths
- test pglog_hardlimit flag

These are qa suites changes specific to luminous.

Signed-off-by: Neha Ojha <nojha@redhat.com>

mon/OSDMonitor.cc: make a note about reusing jewel feature bit

For OSD_PGLOG_HARDLIMIT, we have reused a jewel feature bit that was retired
in luminous. Therefore, we need to check the release version for
>= CEPH_RELEASE_LUMINOUS, before using it.

Signed-off-by: Neha Ojha <nojha@redhat.com>
(cherry picked from commit 6abcc20dca0ee5a08a3fe7c560750f904fe3fa65)

Conflicts:
src/mon/OSDMonitor.cc: trivial resolution

mon: add and use OSD_PGLOG_HARDLIMIT feature bit

Signed-off-by: Neha Ojha <nojha@redhat.com>
(cherry picked from commit 6b0a3ded5baabb19809618de16cdf67c925a8e5a)

Conflicts:
src/mon/OSDMonitor.cc: trivial resolution

osd/mon: fix upgrades for pg log hard limit

Signed-off-by: Neha Ojha <nojha@redhat.com>
(cherry picked from commit 6ceeab6e204286148a69eb147fbc4045beddef49)

Conflicts:
src/include/rados.h
src/mon/MonCommands.h
src/mon/OSDMonitor.cc
src/osd/OSDMap.cc
          Luminous does not have CEPH_OSDMAP_NOSNAPTRIM flag.
          In nautilus, CEPH_OSDMAP_PGLOG_HARDLIMIT is set by default,
          which is not the case in luminous.

osd: bring back old calc_trim_to and rename new method

Signed-off-by: Neha Ojha <nojha@redhat.com>
(cherry picked from commit 0536aef761fefda9a86d5bc72a12ca8cab931ccb)

osd/PrimaryLogPG: fix potential pg-log overtrimming

In https://github.com/ceph/ceph/pull/21580 I set a trap to catch some wired
and random segmentfaults and in a recent QA run I was able to observe it was
successfully triggered by one of the test case, see:

```
http://qa-proxy.ceph.com/teuthology/xxg-2018-07-30_05:25:06-rados-wip-hb-peers-distro-basic-smithi/2837916/teuthology.log
```

The root cause is that there might be holes on log versions, thus the
approx_size() method should (almost) always overestimate the actual number of log entries.
As a result, we might be at the risk of overtrimming log entries.

https://github.com/ceph/ceph/pull/18338 reveals a probably easier way
to fix the above problem but unfortunately it also can cause big performance regression
and hence comes this pr..

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
(cherry picked from commit 3654d56985c67d15506fa37b56ef5b0c04e01a65)

Conflicts:
src/osd/PrimaryLogPG.cc: trivial resolution

osd/PrimaryLogPG: avoid dereferencing invalid complete_to

For the auto-repair (EIO caused) case, we will not reinitialize
**complete_to** (because last_complete is equal to last_update!)
and hence there is chance that **complete_to** should aleady
point to **log.end()** before we call recover_got.

We could simply drop it here as we (already) logged the **complete_to**
iterator change in a more compatible way a few lines below.

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
(cherry picked from commit 69a2cc35840939436da09691ca62476d7f599de4)

osd/PrimaryLogPG.cc: limit trimming at can_rollback_to

This change is motivated by the failures seen in the multimds suite,
where we hit assert(s <= can_rollback_to), while trimming the log in ec
pools.

This is due to the fact that we had removed limits on the trim_to value to
address https://tracker.ceph.com/issues/23979.

But, seems that this could be dangerous for ec pools. So, keep the
can_rollback_to limit, while calculating the trim_to value.

Fixes: http://tracker.ceph.com/issues/21416
Signed-off-by: Neha Ojha <nojha@redhat.com>
(cherry picked from commit 4b5c6b88d444e2173e716fe4890717873c8dc8e5)

osd/PGLog.cc: check if complete_to points to log.end()

Signed-off-by: Neha Ojha <nojha@redhat.com>
(cherry picked from commit 630daa1d7d233aa01a2d49bf1bc7b858bf73348a)

src/osd/PG.cc: remove redundant call to trim_log()

This change is motived by the failure tracked in
https://tracker.ceph.com/issues/25198. The failure highlights a case, when a
call to trim_log() after the PG has recovered, races with the previous op,
on a replica OSD. Since the previous operation has not completed, the
last_complete value for that OSD is not valid, when we try to trim the
log. It is also worth noting that the race is due to MOSDPGTrim going through
the strict queue as a peering message vs regular ops going through the
non-strict queue.

During the investigation of this bug, we noticed that, with
https://tracker.ceph.com/issues/23979, we allow pg log trimming to
happen on the primary and replicas, whenever we cross the upper bound of
the pg log. This also ensures that pg log trimming happens while processing
any new op.

Therefore, the function trim_log(), which earlier served the purpose of
trimming logs on the primary and replicas, just before the PG went into
the Recovered state, is no more required. This acted like a last line of
defense to trim logs, when we did not need the logs any more. But, this call
seems redundant now, because, we are limiting the pg log length at all times.

Signed-off-by: Neha Ojha <nojha@redhat.com>
(cherry picked from commit 283b0bde4a52128c1590afe8e5011b266a2e334b)

Conflicts:
src/osd/PG.cc: We do not need the trim_log() call any longer,
        due to the explanation provided in the commit message.

osd/PGLog.cc: use lgeneric_subdout instead of generic_dout

Signed-off-by: Neha Ojha <nojha@redhat.com>
(cherry picked from commit 1b6dafb351b4f80e1e2c9a304f9e1d508ae8bf72)

osd/PGLog: allow pg log trim when complete_to is less than trim_to

Signed-off-by: Neha Ojha <nojha@redhat.com>
(cherry picked from commit a5329ba8dd169e55deaff47d042354e53d8e722d)

Conflicts:
src/osd/PGLog.cc: Now it is possible to have complete_to version
        less than or equal to trim version, because the pg log length upper
        limit is a hard limit, and trim can proceed even when there is
        pending recovery/backfill. So do not complain when this happens.

osd: reset complete_to when trimming the log past it

Signed-off-by: Neha Ojha <nojha@redhat.com>
(cherry picked from commit 38170cdb1b8c3ea7e8b411fabe1fe99abd06cf52)

osd: allow trim() to proceed when there are missing items

Signed-off-by: Neha Ojha <nojha@redhat.com>
(cherry picked from commit de42fee0dff299f4d0377961d05e02fd8f49f21b)

Conflicts:
src/osd/PGLog.cc: The async recovery feature is not present
        in luminous. Remove async recovery requirements from this commit.

osd: handle trim() during backfill

Remove async recovery components: The async recovery feature is not present
in luminous. We do not need commit 22d17fb5aad6ab9d7525d9492c0e96a36d02879e,
which adds a flag to remember async recovery. We have also removed async
recovery requirements from this commit and modified the commit message to
only reflect backfill.

Signed-off-by: Neha Ojha <nojha@redhat.com>
(cherry picked from commit e538c31f0f3133f811a8e478fcb25b575cad66bf)

osd: print pg log length and trim_to

Signed-off-by: Neha Ojha <nojha@redhat.com>
(cherry picked from commit f48584a5b468949c31bffba1b507fb13a8756284)

osd: make calc_trim_to() independent of min_last_complete_ondisk

Signed-off-by: Neha Ojha <nojha@redhat.com>
(cherry picked from commit 1ae5fd32c92ea2f025344c663535d00f71f2cdda)

Conflicts:
src/osd/PrimaryLogPG.cc: min_last_complete_ondisk and
        pg_log.get_can_rollback_to() are no longer the limit of the pg log.
        Make the head of the pg log the new limit for pg log trimming.

Merge pull request #25833 from vshankar/wip-37762

luminous: config: drop config::lock when invoking config observer

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

ceph-volume: Adapt code to support Python3

- raw_input() has been renamed to input() in Python3
- Changed signature of prompt_bool. Variables that are named like built-ins must be named like xxx_ and not _xxx

Fixes: https://tracker.ceph.com/issues/37470
Signed-off-by: Volker Theile <vtheile@suse.com>
(cherry picked from commit fe25a0ea625e75c598f6d0749e7259eef167fa8e)

ceph-volume tests.functional.batch symlink test_zap to all batch scenarios

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit d3c0058911fd0b3e273a7afb1a9f4f6072942d75)

ceph-volume tests.functional.batch add test_zap yaml to tox.ini

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 25843b762953e53c08e6475d1f9dbfd5102a283f)

ceph-volume tests.functional.batch create a separate test_zap playbook

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 6b8f0d483a6203de204fcc605ade536749a9f6c8)

ceph-volume lvm.zap update success message for OSD IDs

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 73bb17f79a5b67b28ceace9dc7dcc2dde38dad5e)

doc/man/ceph-volume add zapping by osd-id examples

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit acbf7c9b2ad2810621d7ed3987e814cd8c2a2f65)

doc/ceph-volume add zapping by osd-id examples

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit f82304f4b5f5ecd865b6a208f894b59cec6386e2)

ceph-volume tests.lvm verify associated lvs by osd id+fsid behavior

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 40a414464727375d5c558532aed3b70766ec5b78)

ceph-volume lvm.zap initial take on zapping by OSD ID, FSID

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 59f6cec6dd0687a2528f75a518bd0d4617c27fe0)

qa: silence read-only WRN for damage testing

Fixes: http://tracker.ceph.com/issues/37944
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit ac302de7b7725c259fee1852e3f402533d69f8cf)

Merge pull request #25719 from ashishkumsingh/wip-37553-luminous

luminous: osdc/Objecter: update op_target_t::paused in _calc_target

Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>

mds: handle state change race

In multi-mds cluster, recovering mds may receive mdsmap that changes
its state after other mds. Furthermore, the recovering mds may receive
messages tiggered by its state change from other mds before it receive
corresponding mdsmap.

Fixes: http://tracker.ceph.com/issues/37594
Signed-off-by: "Yan, Zheng" <zyan@redhat.com>
(cherry picked from commit d3a444473abc98e5ce8121af24538a141a292777)

 Conflicts:
src/mds/Locker.cc
src/mds/MDCache.cc
src/mds/MDSRank.h