Merge pull request #28219 from pdvian/wip-39539-nautilus

nautilus: osd/PG: fix last_complete re-calculation on splitting

Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge pull request #27632 from pdvian/wip-39043-nautilus

nautilus: osd/PGLog: preserve original_crt to check rollbackability

Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Samuel Just <sjust@redhat.com>

Merge pull request #27699 from liewegas/wip-39335-nautilus

nautilus: mgr/BaseMgrModule: run MonCommandCompletion on the finisher

Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>

Merge pull request #27711 from pdvian/wip-39184-nautilus

nautilus: osd: Primary won't automatically repair replica on pulling error

Reviewed-by: David Zafman <dzafman@redhat.com>

Merge pull request #27744 from ashishkumsingh/wip-39432-nautilus

nautilus: osd/PG: discover missing objects when an OSD peers and PG is degraded

Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge pull request #27771 from smithfarm/wip-39419-nautilus

nautilus: mon/Elector: format mon_release correctly

Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge pull request #27839 from pdvian/wip-39219-nautilus

nautilus: osd: FAILED ceph_assert(attrs || !pg_log.get_missing().is_missing(soid) || (it_objects != pg_log.get_log().objects.end() && it_objects->second->op == pg_log_entry_t::LOST_REVERT)) in PrimaryLogPG::get_object_context()

Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge pull request #27876 from liewegas/wip-autoscaler-fixes-nautilus

nautilus: mgr: fix pgp_num adjustments

Reviewed-by: Jan Fajerski <jfajerski@suse.com>
Reviewed-by: Nathan Cutler <ncutler@suse.com>

Merge pull request #28215 from smithfarm/wip-40003-nautilus

nautilus: do_cmake.sh: source not found

Reviewed-by: Lenz Grimmer <lgrimmer@suse.com>
Reviewed-by: Sebastian Wagner <swagner@suse.com>

Merge pull request #28062 from bk201/wip-39664-nautilus

nautilus: mgr/dashboard: incorrect help message for minimum blob size

Reviewed-by: Laura Paduano <lpaduano@suse.com>

osd/PG: fix last_complete re-calculation on splitting

We add hard-limit for pg_logs now, which means we might keep trimming
old log entries irrespective of pg's current missing_set. This as a
result can cause the last_complete pointer moving far ahead of the real
on-disk version (the oldest need of missing_set, for instance) the
corresponding pg should have on splitting:

```
2019-04-19 06:41:52.559247 7efd4725c700 10 osd.2 271 Splitting pg[5.6( v 270'943 lc 0'0 (238'300,270'943] local-lis/les=250/251 n=943 ec=223/223 lis/c 250/223 les/251/224/0 250/271/229) [5,2] r=1 lpr=271 pi=[223,271)/4 crt=270'943 unknown NOTIFY m=518 mbc={}] into 5.16
2019-04-19 06:41:52.561413 7efd4725c700 10 osd.2 pg_epoch: 271 pg[5.6( v 270'943 lc 238'300 (238'300,270'943] local-lis/les=250/251 n=943 ec=223/223 lis/c 250/223 c/f 251/224/0 250/271/229) [5,2] r=1 lpr=271 pi=[223,271)/4 crt=270'943 lcod 0'0 unknown NOTIFY m=261 mbc={}] release_backoffs [MIN,MAX)
```

For the above example, parent's last_complete cursor changed from **0'0** to
**238'300** directly due to the effort of trying to catch up the oldest
log entry changing when splitting was done. However, back into v12.2.9 primary
would still reference shard's last_complete field when trying to figure out all
possible locations of a currently missing object (see PG::MissingLoc::add_source_info):

```c++
  if (oinfo.last_complete < need) {
    if (omissing.is_missing(soid)) {
      ldout(pg->cct, 10) << "search_for_missing " << soid << " " << need
                         << " also missing on osd." << fromosd << dendl;
      continue;
    }
  }
```

Hence a wrongly calculated last_complete could then make primary mis-consider
that a specific shard might have the authoritative object it currently
looking for:

```
2019-04-19 06:41:52.904163 7fd4cfb5a700 10 osd.5 pg_epoch: 271 pg[5.6( v 270'943 lc 238'300 (238'300,270'943] local-lis/les=250/251 n=471 ec=223/223 lis/c 250/223 les/
c/f 251/224/0 250/271/229) [5,2] r=0 lpr=271 pi=[223,271)/4 crt=270'943 lcod 226'77 mlcod 0'0 peering m=16 mbc={}] proc_replica_log for osd.2: 5.6( v 270'943 lc 238'30
0 (238'300,270'943] local-lis/les=250/251 n=471 ec=223/223 lis/c 250/223 les/c/f 251/224/0 250/271/229) log((249'563,270'943], crt=270'943) missing(261 may_include_del
etes = 1)
2019-04-19 06:41:52.904645 7fd4cfb5a700 20 osd.5 pg_epoch: 271 pg[5.6( v 270'943 lc 238'300 (238'300,270'943] local-lis/les=250/251 n=471 ec=223/223 lis/c 250/223 les/
c/f 251/224/0 250/271/229) [5,2] r=0 lpr=271 pi=[223,271)/4 crt=270'943 lcod 226'77 mlcod 0'0 peering m=16 mbc={}]  after missing 5:624c3a7a:::benchmark_data_smithi190
_39968_object1382:head need 226'110 have 0'0
2019-04-19 06:41:53.567820 7fd4d035b700 10 osd.5 pg_epoch: 272 pg[5.6( v 270'943 lc 0'0 (238'300,270'943] local-lis/les=271/272 n=471 ec=223/223 lis/c 250/223 les/c/f
251/224/0 250/271/229) [5,2] r=0 lpr=271 pi=[223,271)/4 crt=270'943 lcod 226'77 mlcod 0'0 unknown m=16 u=13 mbc={255={(1+0)=220,(2+0)=28}}] search_for_missing 5:624c3a
7a:::benchmark_data_smithi190_39968_object1382:head 226'110 is on osd.2
```

note that ```5:624c3a7a:::benchmark_data_smithi190_39968_object1382:head 226'110```
was actually missing on both primary and shard osd.2 whereas primary insisted that
object should exist on shard osd.2!

https://github.com/ceph/ceph/pull/26175 posted an indirect fix
for the above problem by ignoring last_complete when checking the missing set,
but it should generally make more sense to fill in the last_complete field correctly
whenever possible.
Hence coming this additional fix.

Fixes: http://tracker.ceph.com/issues/26958
Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
(cherry picked from commit aad5d47be64ed7feba79f540ec1debc45625a74f)

Merge pull request #27937 from pdvian/wip-39288-nautilus

nautilus: [rbd-mirror] image replayer should periodically flush IO and commit positions

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #27989 from smithfarm/wip-39590-nautilus

nautilus: qa/tasks/rbd_fio: fixed missing delimiter between 'cd' and 'configure'

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #27882 from pdvian/wip-39224-nautilus

nautlius: librbd: do not allow to deep copy migrating image

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #27903 from pdvian/wip-39226-nautilus

nautilus: librbd: support EC data pool images sparsify

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
Reviewed-by: Nathan Cutler <ncutler@suse.com>

Merge pull request #27988 from smithfarm/wip-39541-nautilus

nautilus: tests: rbd: qemu-iotests tests fail under latest Ubuntu kernel

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #28019 from pdvian/wip-39315-nautilus

nautilus: krbd: fix rbd map hang due to udev return subsystem unordered

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #28128 from dillaman/wip-39729-nautilus

nautilus: qa/workunits/rbd: use https protocol for devstack git operations

Reviewed-by: Mykola Golub <mgolub@mirantis.com>

Merge pull request #28132 from dillaman/wip-39195-nautilus

nautilus: librbd: fixed several race conditions related to copyup

Reviewed-by: Mykola Golub <mgolub@mirantis.com>

do_cmake.sh: use bash

Fixes: http://tracker.ceph.com/issues/39981
Signed-off-by: Nathan Cutler <ncutler@suse.com>
(cherry picked from commit ac8d7c5fd4045d79ab6caacc0b72d64866fcc5c8)

Merge pull request #27752 from liewegas/wip-devices-doc-nautilus

nautilus: doc/rados/operations/devices: document device prediction

Reviewed-by: Sébastien Han <seb@redhat.com>

Merge pull request #27488 from sebastian-philipp/wip-39169-nautilus

nautilus: doc/mgr/orchestrator_cli: Rook orch supports mon update

Reviewed-by: Nathan Cutler <ncutler@suse.com>

Merge pull request #28163 from smithfarm/wip-39962-nautilus

nautilus: mgr/dashboard: openssl exception when verifying certificates of HTTPS requests

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Ricardo Dias <rdias@suse.com>

systemd: ceph-mgr: set MemoryDenyWriteExecute to false

Fixes: http://tracker.ceph.com/issues/39628
Signed-off-by: Ricardo Dias <rdias@suse.com>
(cherry picked from commit 1d7506fdce4924fb30bbabc65e26dafa06aab24f)

librbd: copyup read stats were incorrectly tied to child

The parent IO stats should be updated when reading from the parent
image for a copyup.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 9b615c30753ad35636c875696ed951e807fd694c)

librbd: force a copyup object map update if migration completes

If a live-migration copyup was in-progress when the copyup completes,
force the object map to be updated.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit e18e413b602208b38f432f4ba5bcb187fe102cb1)

librbd: cleaned up snap/parent lock release in read-from-parent path

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit b7f6ba710d2855ea8f223d205a55a1e425167afd)

test/librbd: initial unit test for copyup state machine

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit ea4901f539c7404c07e72f5c42fc8abcf840dafa)

librbd: copyup should restart delayed ops against the same object

This avoids the potential for a race condition where an in-flight
copyup is removed from the in-flight copyup list and a subsequent
IO against the same object causes a second in-flight copyup.

Fixes: http://tracker.ceph.com/issues/39021
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 31ed673a7484c8067dd928cabd5bcb923a35dbaf)

librbd: merge copyup object map update states

The object map HEAD and HEAD/snapshot update states have been
simplified and merged into a single state. This also fixes
several potential race conditions and an issue where CoR might
incorrectly mark the HEAD object has exists+dirty.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 5b90dbe0ae4be3dccfc3fbc0c8d5eb1ed7bb3168)

librbd: refactor copyup state machine to match current code style

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 711a070fa086b499eeba9a4c62005456f406689b)

librbd: properly hold snap/parent locks during IO

The ImageCtx::parent pointer was dereferenced without holding the lock
which could lead to a crash. The ImageCtx::migration_info structure
was also dereferenced without holding a lock.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 6dad1acee3f0a4b4b3c88e25c0cfafe21bc066c4)

librbd: empty IO requests should complete in new context

This will avoid the possibility of a lock being held by the caller
and having the IO immediately complete back to the calling state
machine.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit c9ff6cf1945b248b38e83484aa4a22c206da846b)

librbd: deep-copy object copy should register an in-flight op

When handling live migrations, the source image is the parent
image of the destination image. To prevent the parent image from
being closed while a request is in-flight, the object copy
state machine now registers an async operation with the source
image.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 89cfa6c8d328b9294c7f7efd53d82edcec59e66d)

librbd: deep-copy object copy should delay computing object existence

This prevent a potential recursive lock on the destination image's
snap_lock.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 0c37066797737cd7c677b2649dae1e6422869afb)

qa/workunits/rbd: use https protocol for devstack git operations

Fixes: http://tracker.ceph.com/issues/39656
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit fb4f9a8a08edd18b1a23e1be4c1285d0ec4d1de6)

Merge pull request #27740 from ifed01/wip-ifed-fix-39446

nautilus: os/bluestore: fix out-of-bound access in bmap allocator.

Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #27803 from pdvian/wip-39205-nautilus

nautilus: osd: shutdown recovery_request_timer earlier

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #28065 from yuriw/wip-yuriw-fix-distros-nautilus

qa/tests: cleaned up supported distro for nautilus

qa/tests: cleaned up supported disto for nautilus

Signed-off-by: Yuri Weinstein <yweinste@redhat.com>

Merge pull request #27608 from liewegas/wip-38157-nautilus

nautilus: mgr/progress: behave if pgs disappear (due to a racing pg merge)

Reviewed-by: Tim Serong <tserong@suse.com>
Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>

Merge pull request #27654 from neha-ojha/wip-38881-nautilus

nautilus: osd/PG: skip rollforward when !transaction_applied during append_log()

Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>

Merge pull request #27854 from smithfarm/wip-39504-nautilus

nautilus: core: Give recovery for inactive PGs a higher priority

Reviewed-by: David Zafman <dzafman@redhat.com>

Merge pull request #27857 from ashishkumsingh/wip-39465-nautilus

nautilus: msg: output peer address when detecting bad CRCs

Reviewed-by: Greg Farnum <gfarnum@redhat.com>

Merge pull request #27939 from pdvian/wip-39308-nautilus

nautilus: mgr/ActivePyModules: handle_command - fix broken lock

Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>

mgr/dashboard: fix help message for minimum blob size in pool form

A value less than maximum blob size is required.

Fixes: http://tracker.ceph.com/issues/39624
Signed-off-by: Kiefer Chang <kiefer.chang@suse.com>
(cherry picked from commit c8c4ed692a2b6ac8bf2aa9337ed7d683187ff882)

Merge pull request #27642 from LenzGr/nautilus-documentation

nautilus: doc: Backported dashboard documentation changes

Merge pull request #27936 from dillaman/wip-39450-nautilus

nautilus: librbd: re-add support for nautilus clients talking to jewel clusters

Reviewed-by: Mykola Golub <mgolub@mirantis.com>

Merge pull request #28046 from neha-ojha/wip-28036-nautilus

nautilus: doc/dev/osd_internals/async_recovery: update cost calculation

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

doc/dev/osd_internals/async_recovery: update cost calculation

Signed-off-by: Neha Ojha <nojha@redhat.com>
(cherry picked from commit 984073e44d096a3322d90b85da42c593fff94c6d)

Merge pull request #28024 from ricardoasmarques/wip-39630-nautilus

nautilus: mgr/dashboard: iSCSI GET requests should not be logged

Reviewed-by: Tatjana Dehler <tdehler@suse.com>

Merge pull request #27942 from LenzGr/wip-39524-nautilus

nautilus: mgr/dashboard: make auth token work with UTC times only

mgr/dashboard: iSCSI GET requests should not be logged

Fixes: https://tracker.ceph.com/issues/39492
Signed-off-by: Ricardo Marques <rimarques@suse.com>
(cherry picked from commit 93d74cf9b9926c1638f15da6c8e6d3fd68d61831)

Merge pull request #27758 from ricardoasmarques/wip-39452-nautilus

nautilus: mgr/dashboard: iSCSI form is showing a warning

Reviewed-by: Tiago Melo <tmelo@suse.com>

nautilus: mgr/dashboard: Adapt iSCSI discovery auth for read-only users (#27759)

nautilus: mgr/dashboard: Adapt iSCSI discovery auth for read-only users

Reviewed-by: Tiago Melo <tmelo@suse.com>

nautilus: mgr/dashboard: Upgrade to ceph-iscsi config v9 (#28001)

nautilus: mgr/dashboard: Upgrade to ceph-iscsi config v9

Reviewed-by: Ricardo Marques <rimarques@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>

krbd: fix rbd map hang due to udev return subsystem unordered

The order of subsystem returned by udev_device_get_subsystem
might not be same order as adding subsystem by
udev_monitor_filter_add_match_subsystem_devtype. So if block
event is returned first and rbd event is returned next, then
further poll will get nothing back until timed-out.

Fixes: http://tracker.ceph.com/issues/39089
Signed-off-by: Zhi Zhang <zhangz.david@outlook.com>
(cherry picked from commit 42dd1eae630f31a628051049e4007175561f1939)

mgr/dashboard: Support ceph-iscsi config v9

Fixes: https://tracker.ceph.com/issues/39147
Signed-off-by: Ricardo Marques <rimarques@suse.com>
(cherry picked from commit 1b5bf05a2ce0590f034bbbc7ae750ed478f6ffe3)

nautilus: mgr/dashboard: Make all columns sortable (#27889)

nautilus: mgr/dashboard: Make all columns sortable

Reviewed-by: Alfonso Martínez <almartin@redhat.com>

nautilus: mgr/dashboard: Buggy data table search field (#27672)

nautilus: mgr/dashboard: Buggy data table search field

Reviewed-by: Alfonso Martínez <almartin@redhat.com>

Merge pull request #27886 from rhcs-dashboard/39459-nautilus-prometheus-replace-whitespaces

nautilus: mgr/prometheus: replace whitespaces in metrics' names

Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: Nathan Cutler <ncutler@suse.com>

Merge pull request #27673 from Devp00l/wip-39371-nautilus

 nautilus: mgr/dashboard: Localization for date picker module

Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Tiago Melo <tmelo@suse.com>

qa/tasks/rbd_fio: fixed missing delimiter between 'cd' and 'configure'

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 83c11efb98f28c19f70eac04a6a84fe820f968cb)

qa/suites/rbd: add random distro selection to librbd tests

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit d83d598c121e41eb6c6903df794ea2bd15ed4b86)

qa/workunits/rbd: use more recent qemu-iotests that support Bionic

Fixes: http://tracker.ceph.com/issues/24668
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit b184acd218b36b85f5a25678d773c35407a2b745)

PendingReleaseNotes: librbd can open images on Jewel clusters

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

doc: Fix small typo in dashboard documentation

Signed-off-by: Tatjana Dehler <tdehler@suse.com>
(cherry picked from commit 038ef6291d5f5728385683c693046f03184309fa)

doc: Improved the dashboard proxy config section

Added paragraph that explains how standby dashboard instances perform
HTTP redirections to the currently active ceph-mgr node.
Added note about current limitations (see Bug#24662 for details).

Signed-off-by: Lenz Grimmer <lgrimmer@suse.com>
(cherry picked from commit 70a28a3e474177719fef250d0c84d28703946a2d)

doc: dashboard: refined object gateway configuration

As discussed in BUG#39252, the dashboard currently does not
support all possible RGW configuration possibilities when trying
to obtain hostname and port number. Update the docs to better
reflect this.

Signed-off-by: Lenz Grimmer <lgrimmer@suse.com>
(cherry picked from commit 43c56e1a5a92fc876e6edc184459b665d270ce29)

doc: Updated dashboard iSCSI configuration, added labels

Added note about the requirement for the latest ceph-iscsi version
3 to the dashboard documentation. Added some doc references
and replaced some URLs in the iSCSI docs with reST labels instead.

Signed-off-by: Lenz Grimmer <lgrimmer@suse.com>
(cherry picked from commit 71e46887d4b0f1afd7dc8af461879dc0f9440d50)

librbd: re-add support for nautilus clients talking to jewel clusters

We want to support N - 3 client backward compatibility (special case
to support Jewel since it was a LTS release). The "get_snapshot_timestamp"
cls method does not exist in Jewel clusters so librbd should fallback
to excluding the op if it fails.

Note: this is a direct backport to Nautilus since Octopus (master) does not
need to support Jewel-backed clusters

Fixes: http://tracker.ceph.com/issues/39450
Signed-off-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #27923 from Vicente-Cheng/wip-39046-nautilus

nautilus: rgw: update resharding documentation

Reviewed-by: Nathan Cutler <ncutler@suse.com>

Merge pull request #27922 from Vicente-Cheng/wip-38874-nautilus

nautilus: doc: cleanup HTTP Frontends documentation

Reviewed-by: Nathan Cutler <ncutler@suse.com>

mgr/dashboard: auth: fix user lastUpdate timestamp generation

Fixes: http://tracker.ceph.com/issues/39300
Signed-off-by: Ricardo Dias <rdias@suse.com>
(cherry picked from commit 90f6142d6f9ecd712c6c1e7a801a87c346e17855)

mgr/ActivePyModules: handle_command - fix broken lock

by unlocking properly when module is not available.

Fixes: http://tracker.ceph.com/issues/39235
Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
(cherry picked from commit 377de7af5fec521642cc781ac67d555d98ea5e03)

rbd-mirror: periodically flush prior to updating status

This ensure the most up-to-date commit position is included in the
status report.

Fixes: http://tracker.ceph.com/issues/39257
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit af22cd213c90c6a5a6ae9d7bcc44457ef5b01649)

rbd-mirror: simplify image replayer IO / commit position flush path

There is no current user of the callback context provided to the
flush method and there are no longer any reason to have the flush
support methods be virtual.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 50d24c95f3a49dc55d7fc1aa6a5a492f3cdb1331)

Merge pull request #27791 from mogeb/wip-lvmcache-nautilus

nautilus: ceph-volume: add utility functions

Reviewed-by: Alfredo Deza <adeza@redhat.com>

Merge pull request #27687 from dzafman/wip-39389

nautilus: osd: Increase log level of messages which unnecessarily fill up logs

Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed By: Xie Xingguo <xie.xingguo@zte.com.cn>

rgw: updates to resharding documentation

Some minor updates to the resharding documentation with the aim of
clarifying the language and using terms consistently.

Signed-off-by: J. Eric Ivancich <ivancich@redhat.com>
(cherry picked from commit 54b6b650eeeeac0e754bd5860cd53109ba7865bd)

doc/rgw: update civetweb rgw_frontends config example

all of these civetweb options have to be on the same line as rgw_frontends

Fixes: https://tracker.ceph.com/issues/37770
Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit 2f16fcb2fdb8dbd917e804dec97d1be056dc1749)

Merge pull request #27899 from Vicente-Cheng/wip-39051-nautilus

nautilus: doc: add LAZYIO

Reviewed-by: Nathan Cutler <ncutler@suse.com>

Merge pull request #27900 from Vicente-Cheng/wip-39176-nautilus

nautilus: doc: add documentation for "fs set min_compat_client"

Reviewed-by: Nathan Cutler <ncutler@suse.com>

doc: add doc for blacklisting older CephFS clients

Fixes: http://tracker.ceph.com/issues/39130
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit 19372b8492a75ea8423cb9826f4ddaec0a947554)

doc: add doc for cephfs lazyio

Fixes: http://tracker.ceph.com/issues/38729
Signed-off-by: "Yan, Zheng" <zyan@redhat.com>
(cherry picked from commit 8ebd0fb74a0ed06259fc84b13b249756e7f432d3)

librbd: support EC data pool images sparsify

(object truncate or remove)

Fixes: https://tracker.ceph.com/issues/38364
Signed-off-by: Mykola Golub <mgolub@suse.com>
(cherry picked from commit d6bf5ce6f5227c346253ecb40b04f832e8dd7821)

Merge PR #27623 into nautilus

* refs/pull/27623/head:
qa: add crush-node-flags test
mon/OSDMonitor: remove crush node flags too on "crush rm"
mon/OSDMonitor: make per-OSD no{out,down,in,out} flags prior to CRUSH nodes
mon/OSDMonitor: osd add-no{up,down,in,out} - remove state checker
mon/OSDMonitor: respect crush node flags for can_mark_*()
osd/OSDMap: add get_crush_node_flags(int osd)
mon/OSDMonitor: make 'osd {add,rm}-{noin,noout,...}' support crush nodes
osd/OSDMap: raise OSD_FLAGS health alert for crush node flags, too
osd/OSDMap: add flags for crush nodes

Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>

test: osd-recovery-scrub.sh ignore error from kill_daemons()

Another work around for http://tracker.ceph.com/issues/38195

Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit 71d254647ac2019c6973bd58649dbb242651ca7c)

test: Add tests for pool recovery priority conversion

Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit 71d82dbeb9b83fc24511de9dd56110143e3efdfd)

mgr: If the requested OSD is down don't trust osd_stat info

If we have a down AND out OSD it may contains osd_stat with num_pgs == 0.
When all PGs aren't active+clean we need an accurate value or we consider
the osd missing stat info.

Fixes: https://tracker.ceph.com/issues/38930
Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit 97f3b7c2e93f72223ae2c9a904e38bbb30e275da)

mon, mgr: Improve PG count by changing up to up_no_acting in pg_count

By not adding a field to pg_count, we don't have to worry about
versioning or backwards compatibility.  A mixed cluster won't
be any worse than before.

Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit 69eaaaadd00f190f3013abbf8c713b47c0b3e7bc)

osd: Add new object_location_counts to pg_stat_t

Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit 796794b38b49e44dc9112d684a76e87272697043)

Conflicts:
src/osd/osd_types.h
- nautilus does not have 75014ceb1437c5bb48293574ec6f991e4bec64bb

mgr: Improve ok-to-stop by using the avail_no_missing for recovery

Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit 9750061d5d4236aaba156d60790e0b8bcd7cfb64)

osd: Prevent priority from overflowing in the next base level

Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit 42497780db7f1535407a644fca408985456e2fbc)

osd, mon: New pool recovery priority range -10 to 10

Use OSD_POOL_PRIORITY_MAX and OSD_POOL_PRIORITY_MIN constants
Scale legacy priorities if exceeds maximum

Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit 444aa9f9fe0bbbdb34bf2108044598785db6ef65)

osd: For recovery track OSDs that have 0 missing to know how degraded we are

Add backfill priority log message and remove redundanacy to recovery priority

Fixes: https://tracker.ceph.com/issues/39099
Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit f0e7202cffed6e4adf54513891f1deeea2341d81)

Conflicts:
src/osd/osd_types.h
- nautilus does not have 75014ceb1437c5bb48293574ec6f991e4bec64bb

mgr/dashboard: Make all columns sortable

Fixes: https://tracker.ceph.com/issues/39483
Signed-off-by: Stephan Müller <smueller@suse.com>
(cherry picked from commit ba6d0f96dbdd2ccb4824a6377e741a9534d46287)

mgr/prometheus: replace whitespaces in metrics' names

Fixes: https://tracker.ceph.com/issues/39434
Signed-off-by: Alfonso Martínez <almartin@redhat.com>
(cherry picked from commit 843802f43eaa8d0117dd2585b19ac71add3f1024)

librbd: do not allow to deep copy migrating image

Fixes: https://tracker.ceph.com/issues/38661
Signed-off-by: Mykola Golub <mgolub@suse.com>
(cherry picked from commit aeb9192011000bbecc7afa64fb6b7b4e58b5c448)

mgr/DaemonServer: Fix pgp_num throttling min

This was meant to limit the size of any single change to 1/2 of the
max_misplaced (see the comment above!).

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit a0d63c1a1fc673f094bb309635e98ab4c11354e9)