mon/OSDMonitor.cc: better error message about min_size

Signed-off-by: Neha Ojha <nojha@redhat.com>
(cherry picked from commit ce25ac0a9196f90acef82915f122c5a06277993a)

Merge pull request #29440 from neha-ojha/wip-40940-nautilus

nautilus: rocksdb: Updated to v6.1.2

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Mark Nelson <mnelson@redhat.com>

Merge pull request #29439 from neha-ojha/wip-40969-nautilus

nautilus: rocksdb: rocksdb_rmrange related improvements

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #28833 from s0nea/wip-40616-nautilus

nautilus: mgr/dashboard: notify the user about unset 'mon_allow_pool_delete' flag beforehand

Reviewed-by:  Tiago Melo <tmelo@suse.com>
Reviewed-by: Laura Paduano <lpaduano@suse.com>
Reviewed-by: Stephan Müller <smueller@suse.com>

Merge pull request #28889 from bk201/wip-40661-nautilus

nautilus: mgr/dashboard: cephfs multimds graphs stack together

Reviewed-by: Lenz Grimmer <lgrimmer@suse.com>

Merge pull request #28912 from bk201/wip-40685-nautilus

nautilus: mgr/dashboard: Dentries value of MDS daemon in Filesystems page is inconsistent with ceph fs status output

Reviewed-by: Lenz Grimmer <lgrimmer@suse.com>

Merge pull request #28938 from Exotelis/i18n-nautilus

nautilus: mgr/dashboard: Add, update and remove translations

Reviewed-by: Lenz Grimmer <lgrimmer@suse.com>
Reviewed-by:  Tiago Melo <tmelo@suse.com>

Merge pull request #28968 from Devp00l/wip-40699-nautilus

nautilus: mgr/dashboard: Silence Alertmanager alerts

Reviewed-by: Lenz Grimmer <lgrimmer@suse.com>
Reviewed-by:  Tiago Melo <tmelo@suse.com>

Merge pull request #28974 from ricardoasmarques/wip-40723-nautilus

nautilus: mgr/dashboard: Upgrade to ceph-iscsi config v10

Reviewed-by: Lenz Grimmer <lgrimmer@suse.com>
Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #28992 from smithfarm/wip-40691-nautilus

nautilus: build/ops: Module 'dashboard' has failed: No module named routes

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Lenz Grimmer <lgrimmer@suse.com>

Merge pull request #29044 from s0nea/wip-40656-nautilus

nautilus: mgr/dashboard: Changing rgw-api-host does not get effective without disable/enable dashboard mgr module

Reviewed-by: Ricardo Marques <rimarques@suse.com>

Merge pull request #29045 from s0nea/wip-40657-nautilus

nautilus: mgr/dashboard: Display logged in information for each iSCSI client

Reviewed-by: Ricardo Marques <rimarques@suse.com>

Merge pull request #29061 from s0nea/wip-40279-nautilus

nautilus: mgr/dashboard: Optimize the calculation of portal IPs

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #29491 from tspmelo/wip-41075-nautilus

nautilus: mgr/dashboard: Fix e2e failures caused by webdriver version

Reviewed-by: Lenz Grimmer <lgrimmer@suse.com>

Merge PR #29490 into nautilus

* refs/pull/29490/head:
mgr/volumes: set uid/gid of FS client's mount as 0/0
mgr/volumes: add `ceph fs subvolumegroup getpath` command

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge PR #29513 into nautilus

* refs/pull/29513/head:
qa/tasks/ceph_deploy: assume systemd and simplify shutdown wonkiness
qa/tasks/ceph_deploy: do not rely on ceph-create-keys

Reviewed-by: Vasu Kulkarni <vasu@redhat.com>
Reviewed-by: Yuri Weinstein <yweins@redhat.com>

Merge pull request #29506 from jan--f/wip-41137-nautilus

nautilus: ceph-volume prints errors to stdout with --format json

qa/tasks/ceph_deploy: assume systemd and simplify shutdown wonkiness

The 'service' command was succeeding but not actually stopping ceph.  But
we can safely assume systemd now so get rid of the weirdness here.

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 97a45c9c2e8b6f6af51bf387636c9e4eaa228380)

qa/tasks/ceph_deploy: do not rely on ceph-create-keys

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit daeaabf1871a2d3482f97f0def5f9c48da7811e9)

ceph-volume: print most logging messages to stderr

Signed-off-by: Jan Fajerski <jfajerski@suse.com>
Fixes: http://tracker.ceph.com/issues/38548
(cherry picked from commit 1f8f1e95dee88b4bab2d6931e720ed83584d2887)

Merge pull request #29416 from jan--f/wip-41021-nautilus

nautilus: simple: when 'type' file is not present activate fails

Merge pull request #28583 from ukernel/nautilus-40326

nautilus:  mds: change how mds revoke stale caps

Reviewed-by: Venky Shankar <vshankar@redhat.com>

Merge pull request #28609 from batrick/i40324

nautilus: cephfs: ceph_volume_client: d_name needs to be converted to string before using

Reviewed-by: Venky Shankar <vshankar@redhat.com>

Merge pull request #29156 from smithfarm/wip-40839-nautilus

nautilus: cephfs-shell: TypeError in poutput

Reviewed-by: Venky Shankar <vshankar@redhat.com>

Merge pull request #29157 from smithfarm/wip-40842-nautilus

nautilus: cephfs: ceph-fuse: mount does not support the fallocate()

Reviewed-by: Venky Shankar <vshankar@redhat.com>

Merge pull request #29158 from smithfarm/wip-40843-nautilus

nautilus: cephfs-shell: name 'files' is not defined error in do_rm()

Reviewed-by: Venky Shankar <vshankar@redhat.com>

Merge pull request #29186 from xiaoxichen/wip-40874-nautilus

nautilus: cephfs: client: bump ll_ref from int32 to uint64_t

Reviewed-by: Venky Shankar <vshankar@redhat.com>

Merge pull request #29209 from smithfarm/wip-40316-nautilus

nautilus: tests: test_volume_client: print python version correctly

Reviewed-by: Venky Shankar <vshankar@redhat.com>

Merge pull request #29231 from smithfarm/wip-40438-nautilus

nautilus: cephfs: getattr on snap inode stuck

Reviewed-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Yan, Zheng <zyan@redhat.com>

Merge pull request #29233 from smithfarm/wip-40440-nautilus

nautilus: cephfs: mds: cannot switch mds state from standby-replay to active

Reviewed-by: Venky Shankar <vshankar@redhat.com>

Merge pull request #29275 from liewegas/wip-cephfs-meta-pool-priority-nautilus

nautilus: mon: set recovery priority etc on cephfs metadata pool

Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge pull request #29343 from pdvian/wip-40443-nautilus

nautilus: cephfs: client: set snapdir's link count to 1

Reviewed-by: Venky Shankar <vshankar@redhat.com>

Merge pull request #29344 from pdvian/wip-40445-nautilus

nautilus: cephfs: mds: cleanup unneeded client_snap_caps when splitting snap inode

Reviewed-by: Venky Shankar <vshankar@redhat.com>

mgr/dashboard: Fix e2e failures caused by webdriver version

Current version of webdriver-manager was not able to install a webdriver version
compatible with Chrome 76.

This fix could not be cherry-picked because master has diverged with nautilus.

Fixes: https://tracker.ceph.com/issues/41075
Signed-off-by: Tiago Melo <tmelo@suse.com>

mgr/volumes: set uid/gid of FS client's mount as 0/0

... to allow creating subvolumes/subvolume groups within a filesystem.

Fixes: http://tracker.ceph.com/issues/40927
Signed-off-by: Ramana Raja <rraja@redhat.com>
(cherry picked from commit 033d9e743a97534518b817bd10bd27cfbb711a8a)

mgr/volumes: add `ceph fs subvolumegroup getpath` command

... to return the path of a FS subvolume group.

Fixes: http://tracker.ceph.com/issues/40617
Signed-off-by: Ramana Raja <rraja@redhat.com>
(cherry picked from commit 4ac6c2acc2b8d08dd00cf1463a62246bb8128ff4)

Merge pull request #29060 from sebastian-philipp/nautilus-orchestrator_cache

nautilus: mgr/orchestrator: Cache and DeepSea iSCSI + NFS

Reviewed-by: Tim Serong <tserong@suse.com>

Merge pull request #28187 from pdvian/wip-39516-nautilus

nautilus: tests: osd-backfill-space.sh test failed in TEST_backfill_multi_partial()

Reviewed-by: David Zafman <dzafman@redhat.com>

Merge pull request #28573 from smithfarm/wip-40281-nautilus

nautilus: bluestore: 50-100% iops lost due to bluefs_preextend_wal_files = false

Reviewed-by: Igor Fedotov <ifedotov@suse.com>
Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #29007 from smithfarm/wip-40750-nautilus

nautilus: build/ops: rpm: drop SuSEfirewall2

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Tim Serong <tserong@suse.com>

Merge pull request #29102 from tspmelo/wip-40733-nautilus

nautilus: mgr/dashboard: Fix npm vulnerabilities

Reviewed-by: Nathan Cutler <ncutler@suse.com>

Merge pull request #29162 from smithfarm/wip-40837-nautilus

nautilus: bluestore: Set concurrent max_background_compactions in rocksdb to 2

Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge pull request #29204 from smithfarm/wip-40319-nautilus

nautilus: tests: make: *** [hello_world_cpp] Error 127 in rados

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #29391 from pdvian/wip-40625-nautilus

nautilus: osd/OSD: auto mark heartbeat sessions as stale and tear them down

Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>

Merge pull request #29464 from jan--f/wip-41058-nautilus

nautilus: ceph-volume does not recognize wal/db partitions created by ceph-disk

Reviewed-by: Alfredo Deza <adeza@redhat.com>

ceph-volume: fall back to PARTTYPE if PARTLABEL is empty

In some cases ceph-disk does not populate PARTLABEL for wal and db
partitions. This commit adds the assumption that the empty string is a
valid label and falls back to identifying those disks by PARTTYPE.

Fixes: https://tracker.ceph.com/issues/40917
Signed-off-by: Jan Fajerski <jfajerski@suse.com>
(cherry picked from commit 82d2ae7ffdf62ca93c7848fb62e506db4cda893c)

ceph-volume: adjust tests for empty PARTLABEL fields

It seems like ceph-disk sometimes creates OSDs where the wal and db
partitions have no PARTLABEL set. This commit alters tests so that the
empty string is considered a valid label.

Signed-off-by: Jan Fajerski <jfajerski@suse.com>
(cherry picked from commit bb29110c3c6e081568231131c70522dee6d12bb5)

ceph-volume: refactor ceph-disk_member unittests

This adds a parameterized fixture as input to the tests. Should make
writing new tests easier since only fixture parameters need to be added.
In the future we should be able to reuse the fixtures to build other
fixtures and reuse the parametrization.

Signed-off-by: Jan Fajerski <jfajerski@suse.com>
(cherry picked from commit f42287c3416ad09c112e64756f5051c5c13aa344)

Merge pull request #29453 from neha-ojha/wip-41052-nautilus

nautilus: qa/tasks/cbt.py: change port to work with client_endpoints

Reviewed-by: Yuri Weinstein <yweins@redhat.com>

qa/tasks/cbt.py: change port to work with client_endpoints

hack to work with https://github.com/ceph/cbt/pull/160/

Signed-off-by: Neha Ojha <nojha@redhat.com>
(cherry picked from commit ee02416c654581d77d5f976da89b4a33dd1172e3)

Merge pull request #28740 from smithfarm/wip-40546-nautilus

nautilus: common: Keyrings created by ceph auth get are not suitable for ceph auth import

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #28810 from ifed01/wip-ifed-db-stats-nau

nautilus: tools: ceph-kvstore-tool: print db stats

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge pull request #29252 from smithfarm/wip-40180-nautilus

nautilus: tests: qa/standalone/scrub/osd-scrub-snaps.sh sometimes fails

Reviewed-by: David Zafman <dzafman@redhat.com>

Merge pull request #29309 from smithfarm/wip-40216-nautilus

nautilus: rgw_file:  fix invalidation of top-level directories

Reviewed-by: Matt Benjamin <mbenjami@redhat.com>

rocksdb: Updated to v6.1.2

Signed-off-by: Mark Nelson <mnelson@redhat.com>
(cherry picked from commit 5143607e2738f4ccfb02b9be50e4c67d52205390)

rocksdb: enable rocksdb_rmrange=true by default

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 07cc88431a28d831d9d7ffd8b982e7eebdc3841d)

kv: make delete range optional on number of keys

Actually, we may only wanna let the objects with real big number of omap to
use delete range. if not, we find too many tombstones will have side-effects
on performance of kv.

refer: https://github.com/facebook/rocksdb/wiki/DeleteRange-Implementation

Signed-off-by: Zengran Zhang <zhangzengran@sangfor.com.cn>
(cherry picked from commit b039e63c6816ed371c6db975b4cbc7a3d170bb1f)

Merge pull request #29254 from smithfarm/wip-40465-nautilus

nautilus: core: osd beacon sometimes has empty pg list

Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge pull request #28528 from pdvian/wip-39743-nautilus

nautilus: mon: paxos: introduce new reset_pending_committing_finishers for safety

Reviewed-by: Greg Farnum <gfarnum@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge pull request #28575 from smithfarm/wip-40381-nautilus

nautilus: rbd: tests: add "rbd diff" coverage to suite

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #28576 from smithfarm/wip-40382-nautilus

nautilus: tests: expected MON_CLOCK_SKEW but got none

Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge pull request #28870 from pdvian/wip-40235-nautilus

nautilus: rbd/action: fix error getting positional argument

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #29193 from pdvian/wip-40272-nautilus

nautilus: rbd-mirror: link against the specified alloc library

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #29244 from pdvian/wip-40274-nautilus

nautilus: core: librados: move buffer free functions to inline namespace

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #29246 from pdvian/wip-40276-nautilus

nautilus: rbd: librbd: do not unblock IO prior to growing object map during resize

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
Reviewed-by: Mykola Golub <mgolub@mirantis.com>

Merge pull request #29315 from pdvian/wip-40293-nautilus

nautilus: rbd-nbd: sscanf return 0 mean not-match.

Reviewed-by: Mykola Golub <mgolub@mirantis.com>
Reviewed-by: Jason Dillaman <dillaman@redhat.com>

mgr/orchestrator: add --refresh to `ceph orchestrator service ls`

`ceph orchestrator device ls` already supports --refresh to force
a reload, we should support this for `ceph orchestrator service ls`
as well.

Signed-off-by: Tim Serong <tserong@suse.com>
(cherry picked from commit d5e320ae541cabb2bfdfe20e68fa88233999b63c)

mgr/orchestrator: allow listing iscsi services

Signed-off-by: Tim Serong <tserong@suse.com>
(cherry picked from commit e2bebee712a0716c6d009e8f5559fbe247123161)

mgr/deepsea: return ganesha and iscsi endpoint URLs

This updates describe_service() to include nfs and iscsi services
(deepsea internally refers to these as "ganesha" and "igw" roles).
Additionally, if deepsea sets any of container_id, service, version,
rados_config_location, service_url, status or status_desc, these will
come through now too.

This relies on https://github.com/SUSE/DeepSea/pull/1606 for the new
functionality, but if run against an older version of deepsea, it will
continue to operate as it did before, it just won't include
rados_config_location or service_url data.

I've also updated the caching in describe_service() so that if the
cache needs filling, it just requests all services from DeepSea, caches
everything, then does filtering on the results.  Previously, if the
cache was empty and only one service type was requested, the cache
would then *only* include that service and no other items, so subsequent
requests for different service types would return nothing until the
cache expired.

Fixes: https://tracker.ceph.com/issues/39095
Signed-off-by: Tim Serong <tserong@suse.com>
(cherry picked from commit 1af7c08a96ed82b8824e644b2884d963793299b2)

ceph-volume tests: ensure that better heuristics exist for objectstore detection

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit e48ad3ab26fdab72755bc933781db3bfc6001a30)

ceph-volume simple.activate better detect bluestore/filestore when type is not found

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit b6a449e2e3241aaa2c0eb8fa9f74b5d939c97852)

Merge pull request #28962 from ifed01/wip-ifed-kv-prefetch-nau

nautilus: os/bluestore: support RocksDB prefetch in buffered read mode

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #28639 from xiexingguo/wip-pr-28404-for-n

nautilus: osd/OSD: keep synchronizing with mon if stuck at booting

Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge pull request #28891 from ifed01/wip-ifed-fix-alloc-dump-nau

nautilus: os/bluestore: more smart allocator dump when lacking space for bluefs

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #28892 from ifed01/wip-ifed-fix-no-compress-nau

nautilus: os/bluestore: load OSD all compression settings unconditionally.

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #28893 from ifed01/wip-ifed-remove-assert-bs-tool-nau

nautilus: os/bluestore/bluestore-tool: minor fixes around migrate

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #28963 from ifed01/wip-ifed-add-omap-tail-nau

nautilus: os/bluestore: create the tail when first set FLAG_OMAP

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #28966 from liewegas/wip-set-aio-write-max-nautilus

nautilus: os/bluestore: fix >2GB bluefs writes

Reviewed-by: Igor Fedotov <ifedotov@suse.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge pull request #29023 from ifed01/wip-ifed-fix-stuplid-alloc-len0-nau

nautilus: os/bluestore: avoid length overflow in extents returned by Stupid Allocator

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Ilya Dryomov <idryomov@redhat.com>

Merge pull request #29147 from pdvian/wip-40267-nautilus

nautilus: common: OutputDataSocket retakes mutex on error path

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #29168 from b-ranto/wip-n-mgr-metadata

nautilus: mgr: Add mgr metdata to prometheus exporter module

Reviewed-by: Jan Fajerski <jfajerski@suse.com>
Reviewed-by: Paul Cuzner <pcuzner@redhat.com>

Merge pull request #29173 from neha-ojha/wip-40583-nautilus

nautilus:common/options.cc: Lower the default value of osd_deep_scrub_large_omap_object_key_threshold

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: David Zafman <dzafman@redhat.com>

Merge pull request #29194 from pdvian/wip-40273-nautilus

nautilus: mgr: use ipv4 default when ipv6 was disabled

Merge pull request #29207 from smithfarm/wip-40904-nautilus

nautilus: mgr/influx: module fails due to missing close() method

Merge pull request #29227 from ifed01/wip-ifed-slow-ops2-nau

nautilus: bluestore: add slow op detection for collection_listing

Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge pull request #28756 from smithfarm/wip-40231-nautilus

nautilus: core: maybe_remove_pg_upmap can be super inefficient for large clusters

Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>

Merge pull request #28869 from dzafman/wip-backport-40073

nautilus: osd: Fix the way that auto repair triggers after regular scrub

Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge pull request #28993 from smithfarm/wip-40730-nautilus

nautilus: mon: auth mon isn't loading full KeyServerData after restart

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #29115 from liewegas/wip-39693-nautilus

nautilus: osd/PG: fix cleanup of pgmeta-like objects on PG deletion

Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #29140 from liewegas/wip-40441-nautilus

nautilus: msg/async, v2: make the reset_recv_state() unconditional

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

Merge pull request #29141 from liewegas/wip-msg-async-1-nautilus

nautilus: msg/async: avoid unnecessary costly wakeups for outbound messages

Merge pull request #29142 from liewegas/wip-msg-noneed-set-connection-nautilus

nautilus: msg/async: no-need set connection for Message.

Reviewed-by: Ricardo Dias <rdias@suse.com>

Merge pull request #29143 from liewegas/wip-secure-nautilus

nautilus: msg/async: enable secure mode by default, no longer experimental

Reviewed-by: Ricardo Dias <rdias@suse.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge pull request #29159 from smithfarm/wip-40845-nautilus

nautilus: core: MDSMonitor: use stringstream instead of dout for mds repaired

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #29188 from badone/wip-nautilus-lazy-omap-stats-backport-tracker-40744

nautilus: core: lazy omap stat collection

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #28550 from smithfarm/wip-40007-nautilus

nautilus: rgw: fix prefix handling in LCFilter

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #28648 from pdvian/wip-40107-nautilus

nautilus: rgw: Librgw doesn't GC deleted object correctly

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #28712 from pdvian/wip-40134-nautilus

nautilus: rgw: TempURL should not allow PUTs with the X-Object-Manifest.

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #28713 from smithfarm/wip-40505-nautilus

nautilus: rgw: fix miss get ret in STSService::storeARN

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #28714 from smithfarm/wip-40515-nautilus

nautilus: rgw: multisite: DELETE Bucket CORS is not forwarded to master zone

Reviewed-by: Casey Bodley <cbodley@redhat.com>