mgr/nfs: allow multiple addrs for new exports

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit aaa57afd2930b6dafcf4dde4e2d939c428d290f4)

mgr/nfs: fix/finish rgw export

- create the rgw keys for the export
- cleanup the rgw user on export deletion

Fixes: https://tracker.ceph.com/issues/47172
Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit 119555b84d4ae460b59ec74c665a6b215e2ba628)

mgr/nfs/module: clusterid -> cluster_id

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit b54998106330cc8e4754a35b872e470a95568ba9)

mgr/nfs/export: fix export_update_1 to type check

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit 6771de220435d5a8f6ebc7601f0aa451d8e3c68b)

mgr/nfs/cluster: fix type error

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit f9d291b5755520feca393c7fca964de999a6e90a)

mgr/nfs/export: wrap long lines

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit 1cb98a04974da044af81b9ab6c998c6f746c6473)

mgr/nfs: ExportMgr._delete_export only works for cephfs for now

Signed-off-by: Sebastian Wagner <sewagner@redhat.com>
(cherry picked from commit 8ba3b5bf2e0492c73c0982fa1389cb46edbf2475)

mgr/nfs: Remove pool_ns from NFSCluster

Cause this is a singleton instance. Better to explictly pass it along.

Signed-off-by: Sebastian Wagner <sewagner@redhat.com>
(cherry picked from commit e6db0c810187fccbd3d5b77576727c63e3fc1cc3)

mgr/nfs: Remove ExportMgr.rados_namespace

Cause this is a singleton instance. Better to explictly pass it along.

Signed-off-by: Sebastian Wagner <sewagner@redhat.com>
(cherry picked from commit e0a9d9dccf4474253aec43290d54da8f9cd3742d)

mgr/nfs: flake8

Signed-off-by: Sebastian Wagner <sewagner@redhat.com>
(cherry picked from commit a24e912099851a7c77d453afa84154858233e7d8)

Conflicts:
src/pybind/mgr/tox.ini

mgr/nfs: Add type checking

Also introduce RawBlock class

Fixes: https://tracker.ceph.com/issues/50816
Signed-off-by: Sebastian Wagner <sewagner@redhat.com>
(cherry picked from commit 42d01c5ed3da749d2af6d3debffd3e117d18856f)

Conflicts:
src/mypy.ini

mgr/nfs: Add __eq__ method to Export

Signed-off-by: Sebastian Wagner <sewagner@redhat.com>
(cherry picked from commit b3ddff838461be1805612184a6cd8ccbedb03ae5)

mgr/nfs: Add some compatibility to mgr/dashboard

provide default values for `squash` and `security_label`

Signed-off-by: Sebastian Wagner <sewagner@redhat.com>
(cherry picked from commit 42a8d383e6b6da46cc4ec011f491f9221b48ec4d)

mgr/nfs: Fix whitespace handling

keep whitespace in lines starting with %url

Signed-off-by: Sebastian Wagner <sewagner@redhat.com>
(cherry picked from commit e1928d57060ca3e180d6b8fc7fe5b8ee8c4aefad)

mgr/nfs: Copy unit tests from mgr/dashboard

Signed-off-by: Sebastian Wagner <sewagner@redhat.com>
(cherry picked from commit a30328c49677d887c35697aa89619ca1bf6159f4)

mgr/nfs: partially implement rgw export support

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit 377756c9dff1bf690a5be700e99c22d63d0ebad2)

mgr/nfs: abstract FSAL; add RGWFSAL

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit 5c33431ce132ef7fc5db72d0c06cb89c5524c3a1)

mgr/nfs: refactor to merge 'update' and 'import' code

- update must be an existing export
- import can be a new or existing export, and we will either update or
  create as appropriate.

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit 097e4f9aa3b9ed663116381a13eadf279f77b8a2)

mgr/nfs: add 'nfs export import' command

Allow import of JSON to create a new export.  Similar to 'update', which
allows an import of JSON to update an existing export.

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit 778674a9c728b2b04f1b6922005798303475d352)

mgr/nfs: refactor 'nfs export update' and export validation

Move validation to the Export class, so it can check any object in
place.

Refactor update code to extract allowed changes checks from the old
validation class.

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit 2b93356fac49a5bd50a9e4b97314db87ade51666)

mgr/nfs: fix _fetch_export to distinguish between clusters

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit 0eea3d772667a6d9ba4d42427404e8e6730e5df5)

mgr/nfs: move export ganesha conf translation into caller

This belongs in the (one) caller, not the helper.

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit e4b3673262be61ed21178d46cd53ecf7e1969598)

mgr/nfs: name nfs cephfs client key 'nfs.{cluster_id}.{export_id}'

Better than '{cluster_id}{export_id}', which is confusing and ambiguous.

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit 9b1372dc388e5c783208ecc9251598f0985383a8)

mgr/nfs: add --addr to 'nfs export create'

Ganesha exports can have multiple client blocks with addresses and
access modes/squash behavior.  This cannot easily be exposed via a CLI.
However, providing no client address restriction is pretty useful to users
since it allows full access to the mount point with no restrictions.

Add an --addr argument that creates a single client block.  Set the top
export-level access_type to 'none' and squash to 'none' in this case.

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit 137090dac07f95c3af0b09afc19da210b085019b)

mgr/nfs: add --squash to 'nfs export create'

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit 2450447b45110ea98237618eda777e4e9285c6bd)

mgr/nfs/export_utils: include false but non-None items in config

For example, attr_expiration_time = 0 is the default. Include this since
it is not None.

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit fe1155f0151e4619e13eb3df49dc3dd54eb80511)

vstart.sh: enable nfs module

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit 9b9c5bc7a4bebec2b03b7ff4367b8964f02614ae)

Conflicts:
src/vstart.sh

mgr/cephadm: nfs: drop attr_expiration_time from top-level config

This is added on a per-export basis with the same default of 0.

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit 283a8c184e748a0af5a5deedc2d80b855d747924)

mgr/cephadm: remove Dir_Chunk = 0

This breaks RGW exports, and is likely to be beneficial for libcephfs.

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit 9d114e910e953c0adf4ba5d31ab1d17bdfa1f638)

doc/cephfs/nfs: update recommendation for versions

... of Ceph and NFS-Ganesha packages.

Signed-off-by: Ramana Raja <rraja@redhat.com>
(cherry picked from commit e031a5969120c43adc5b20ca004e6540dffd1579)

Conflicts:
doc/cephfs/nfs.rst

mgr/nfs: fix 'nfs export create' argument order

Put path before --readonly so that it can still be positional.

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit ffce51af8e7be9835ff01c8d9a5f378f19d4d746)

qa/tasks/cephfs/test_nfs: fix export create test

Everything after --readonly is non-positional.

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit 8683cccd06077bfbfd39e9e5ca3750078937dc0e)

mgr/k8sevents: fix help strings

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit fd77020e73bb43f254ec8a4ee7093dccd1b63ff9)

pybind/mgr/mgr_module: fix help desc formatting

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit 4bc37ba3b53930589f2fcd3f137b45ea9872f9e6)

mgr/orchestrator: clean up 'orch {daemon add,apply} rgw' args

Make placement the only optional positional.  This means that the "usual"
'orch orch apply <whatever> 3' will do placement=3.

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit 5d8cfb149b34d068b4dd6942e9a38e40aced299f)

Conflicts:
  src/pybind/mgr/orchestrator/module.py

mgr/orchestrator: reformat a few methods

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit 27c2b83a8e4c7efd319fcd6c1b80698f76dea6c4)

doc/cephfs/nfs: remove documented limitation

At the time NFS support was added, this limitation applied.
However, in
 https://github.com/nfs-ganesha/nfs-ganesha/commit/b3d97f8157a131f55d848ff57e46af91b746b944
and
 https://github.com/nfs-ganesha/nfs-ganesha/commit/1cfe7e2df96f9785367ba94d41559140f584a875
we added support for multiple filesystems and started mixing
the fscid into the filehandle.

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit a534f7be3601a18cd7e26af67031f2dfe3fead7b)

Conflicts:
doc/cephfs/nfs.rst

Merge pull request #42939 from batrick/i51832

pacific: mds: META_POP_READDIR, META_POP_FETCH, META_POP_STORE, and cache_hit_rate are not updated

Reviewed-by: Ramana Raja <rraja@redhat.com>

Merge pull request #42938 from batrick/i51932

pacific: mds: MDCache.cc:5319 FAILED ceph_assert(rejoin_ack_gather.count(mds->get_nodeid()))

Reviewed-by: Ramana Raja <rraja@redhat.com>

Merge pull request #42936 from batrick/i51198

pacific: msg: active_connections regression

Reviewed-by: Ramana Raja <rraja@redhat.com>

Merge pull request #42935 from batrick/i51935

pacific: mds: add debugging when rejecting mksnap with EPERM

Reviewed-by: Ramana Raja <rraja@redhat.com>

Merge pull request #42956 from ifed01/wip-ifed-fix-huge-omap-rename-pac

pacific: os/bluestore: cap omap naming scheme upgrade transaction.

Reviewed-by: Adam Kupczyk <akupczyk@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge pull request #42534 from liewegas/use-quay-pacific

pacific: cephadm: use quay, not docker

Reviewed-by: Dimitri Savineau <dsavinea@redhat.com>

Merge pull request #42889 from rhcs-dashboard/wip-52378-pacific

pacific: mgr/dashboard: stats=false not working when listing buckets

Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
Reviewed-by: Pere Diaz Bou <pdiazbou@redhat.com>

Merge pull request #42974 from s0nea/wip-52308-pacific

pacific: mgr/dashboard: don't notify for suppressed alerts

Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
Reviewed-by: Pere Diaz Bou <pdiazbou@redhat.com>
Reviewed-by: Tatjana Dehler <tdehler@suse.com>

Merge pull request #42898 from smithfarm/wip-52380-pacific

pacific: cmake: exclude "grafonnet-lib" target from "all"

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Ken Dreyer <kdreyer@redhat.com>
Reviewed-by: Patrick Seidensal <pnawracay@suse.com>
Reviewed-by: Aashish Sharma <aasharma@redhat.com>

Merge pull request #42897 from ifed01/wip-ifed-fix-spurious-read-errors-warn-pac

pacific: os/bluestore: respect bluestore_warn_on_spurious_read_errors setting

Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge pull request #42888 from callithea/wip-51951-pacific

pacific: Don't persist report data

Reviewed-by: Brad Hubbard <bhubbard@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>

os/bluestore: cap omap naming scheme upgrade transactoin.

We shouldn't use single per-onode transaction for such an upgrade when onode's omap list is huge. This results in similarly sized WAL/SST files which are inefficient, might cause high memory usage and sometimes error-prone.

Fixes: https://tracker.ceph.com/issues/49170
Signed-off-by: Igor Fedotov <ifedotov@suse.com>
(cherry picked from commit e897fa243c1dd38329733b452872616023f14ac8)

Merge pull request #42969 from idryomov/wip-krbd-escape-match-sysattr-pacific

pacific: krbd: escape udev_enumerate_add_match_sysattr values

Reviewed-by: Deepika Upadhyay <dupadhya@redhat.com>

mgr/dashboard: don't notify for suppressed alerts

Fixes: https://tracker.ceph.com/issues/51987
Signed-off-by: Tatjana Dehler <tdehler@suse.com>
(cherry picked from commit e9f316d678dd24e25e82474da95dbfbb72d763b5)

cephadm: quay.io for non-ceph images too

Signed-off-by: Sage Weil <sage@newdream.net>

cephadm,mgr/cephadm: use quay for ceph image

Signed-off-by: Sage Weil <sage@newdream.net>

krbd: escape udev_enumerate_add_match_sysattr values

libudev uses fnmatch(3) for matching attributes, meaning that shell
glob pattern matching is employed instead of literal string matching.
Escape glob metacharacters to suppress pattern matching.

Fixes: https://tracker.ceph.com/issues/52425
Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
(cherry picked from commit 8841029b0a4705825ba394541240b3cb4eb2cf5c)

Merge pull request #42428 from kamoltat/wip-ksirivad-backport-pacific-42036

pacific: mgr/pg_autoscaler: Introduce autoscaler scale-down feature
Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>

PendingReleaseNotes: add autoscaler profile

Added autoscaler profile release notes for
>=16.2.6

Signed-off-by: Kamoltat <ksirivad@redhat.com>
(cherry picked from commit ea094f769f554ff7d5e4a17e76130d00044ef4e4)

doc/rados/operations/placement-groups: added bias + profile

Added documentations on the autoscale profile and bias

Signed-off-by: Kamoltat <ksirivad@redhat.com>
(cherry picked from commit 65fec053e003967aa3ffa5008730661baa46c868)

pybind/mgr/pg_autoscaler: Added PROFILE to autoscale-status

autoscaler now shows the PROFILE status of each pool
in `ceph osd pool autoscale-status`

Signed-off-by: Kamoltat <ksirivad@redhat.com>
(cherry picked from commit 4a2912bbad72b00ae9fe0d892f07ca760e5cefb4)

pybind/mgr/autoscaler: don't scale pools with overlapping roots

In the previous version of get_subtree_resource_status() in
src/pybind/mgr/pg_autoscaler/module.py we ignore overlapping
pools which in some cases if combined with the new `scale-down`
algorithm in https://github.com/ceph/ceph/pull/38805 can cause
some pools to scale up/down to inapproriate amount of pgs.

Therefore, the PR identifies the overlapping roots and prevent the pools
with such roots from scaling. This only happens with `scale-down` profile
as we see no problem with the default `scale-up` profile.

Removed the variable `pool_root` since it is not used anywhere in
the code, it only gets assigned and reassigned

Also included a unit test test_overlapping_roots.py that tests the function
identify_subtrees_and_overlaps() as well as edited test_cal_final_pg_target.py
to account for pools that contain overlapping roots, therefore, those pools
are expected not to scale.

Signed-off-by: Kamoltat <ksirivad@redhat.com>
(cherry picked from commit 16eb42a1d8cef5cf008b04b27d51e13dbd6ec495)

pybind/mgr/pg_autoscaler: add typing annotations

Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit 1b0b32e44cddacbd7acf7024181247f4251c92f9)

Conflicts:
src/mypy.ini - trivial fix

pybind/mgr/pg_autoscaler: do not index pool_root with pool_name

pool_root is indexed by pool_id, and we never index it with pool_name.

Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit 83f841295d86d10aadf3b4dff4e3eaa80fe139c3)

pybind/mgr/pg_autoscaler: do not create a tuple long_desc

long_desc is supposed to be a str, not a tuple of str.

Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit f725e09ae63702097f0eef1395391933083a9d96)

pybind/mgr/pg_autoscaler: extract CrushSubtreeResourceStatus out

as it also serves as part of interface of get_subtree_resource_status(),
not only its internals. to ease adding the type annotations, this class
is promoted out of the class.

Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit 9bb55e84809e17e0aa1cb1aa4ae712a3ea4fffed)

pybind/mgr/pg_autoscaler: define commands using CLICommand

simpler this way

Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit 4efd7400804ff42b5965ad2691223471b4ae38a7)

pybind/mgr/pg_autoscaler: define options using Option

more consistent and less error-prune this way.

Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit b40de19c23b6b49f8d63bb3911bdebe9a2e19255)

pybind/mgr/pg_autoscaler: Added autoscale-profile feature

The autoscaler by default will start out each pool with minimal
pgs and `scale-up` the pgs when there is more usage in each pool.

Users can now use the commands:

`osd pool set autoscale-profile scale-down` to make the pools
start out with a full complement of pgs and only `scale-down`
when usage ratio across the pools are not even.

`osd pool set autoscale-profile scale-up` (by default) to make the pools
start out with minimal pgs and `scale-up` the pgs when there
is more usage in each pool.

Edited KVMonitor.cc file to make the `autoscale_profile` variable
persistent.

Edited tests/test_cal_final_pg_target.py so that it takes into account
the new `profile` argument when calling cal_final_pg_target(). Also,
added some new test cases for when profile is `scale-up`

Renamed tests/test_autoscaler.py to a more appropriate name:
tests/test_cal_ratio.py

Signed-off-by: Kamoltat <ksirivad@redhat.com>
(cherry picked from commit 0e49683b538026e4c51a040950da3dbb9c7e92d5)

mgr/pg_autoscaler: avoid scale-down until there is pressure

The autoscaler will start out with scaling each
pools to have a full complements of pgs from the start
and will only decrease it when pools need more due to
increased usage.

Introduced a unit test that tests only the
function get_final_pg_target_and_ratio() which
deals with the distrubtion of pgs amongst the
pools

Edited workunit script to reflect the change
of how pgs are calculated and distrubted.

Signed-off-by: Kamoltat <ksirivad@redhat.com>
(cherry picked from commit daeb6f6ac0c8f77ae07147f9d1e2ed18d6d8e4cc)

Merge pull request #42914 from vshankar/wip-51544

pacific: mgr/volumes: use dedicated libcephfs handles for subvolume calls and …

Reviewed-by: Ramana Raja <rraja@redhat.com>

Merge pull request #42900 from ajarr/wip-51834-pacific

pacific: mds: create file system with specific ID

Reviewed-by: Ramana Raja <rraja@redhat.com>

mds: update META_POP_READDIR/FETCH/STORE and cache_hit_rate for dump loads

Signed-off-by: Yongseok Oh <yongseok.oh@linecorp.com>
(cherry picked from commit 24e131a58ed255cb8f6845c020ab956450a5fc10)

mds: check rejoin_ack_gather before enter rejoin_gather_finish

Fixes: https://tracker.ceph.com/issues/48422
Signed-off-by: chencan <chen.can2@zte.com.cn>
(cherry picked from commit 88548775330f66226b4e58a34256f1cc5a72221f)

qa/tasks/cephfs/test_sessionmap: reap connections immediately

We have to reap connections promptly for this test to work.

This test was broken indirectly by d51d80b3234e17690061f65dc7e1515f4244a5a3,
which moved the counter decrement to reap time instead of mark_down/stop
time.

The reaping is asynchronous, so allow for a delay in the count change.

Fixes: https://tracker.ceph.com/issues/50622
Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit c8c5071dcd4b0b788f5e924a678095ce5dc1d7f8)

msg/async: configurable threshold for reaping dead connections

It is helpful to set this to 1 for tests.

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit 8129d6bb953015cc05db458afa6aa9b8f5f62614)

Conflicts:
src/common/options/global.yaml.in

mds: add debugging when rejecting mksnap with EPERM

Fixes: https://tracker.ceph.com/issues/51543
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit 92ec79fd57b841c54bd9a5490031f62166ed6ab1)

Merge pull request #42083 from kotreshhr/wip-51411-pacific

pacific: mgr/mgr_util: switch using unshared cephfs connections whenever possible

Reviewed-by: Ramana Raja <rraja@redhat.com>

Merge pull request #42908 from cfsnyder/wip-52330-pacific

pacific: cls/cmpomap: empty values are 0 in U64 comparisons

Reviewed-by: Casey Bodley <cbodley@redhat.com>

mgr/volumes: use dedicated libcephfs handles for subvolume calls and async jobs

Fixes: http://tracker.ceph.com/issues/51271
Signed-off-by: Venky Shankar <vshankar@redhat.com>
(cherry picked from commit cb2883feac1a5c141a3d72120c2320f7a8ffdea8)

Merge pull request #42860 from ideepika/wip-52336-pacific

pacific: mon/PGMap: remove DIRTY field in ceph df detail when cache tiering is not in use

Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge pull request #42844 from aclamk/resharding-column-options-pacific

pacific: kv/RocksDBStore: Add handling of block_cache option for resharding

Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge pull request #42615 from neha-ojha/wip-51830-pacific

pacific: common/options: Set osd_client_message_cap to 256.

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #42300 from callithea/wip-51241-pacific

pacific: debian/control: ceph-mgr-modules-core does not Recommend ceph-mgr-roo…

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #42578 from batrick/i51983

pacific: mon/MDSMonitor: do not pointlessly kill standbys that are incompatible with current CompatSet

Reviewed-by: Ramana Raja <rraja@redhat.com>

Merge pull request #42096 from varshar16/wip-nfs-backport-june

pacific: nfs backport June

Reviewed-by: Ramana Raja <rraja@redhat.com>

Merge pull request #42909 from gregsfortytwo/wip-51290-pacific

pacific: mon: Sanely set the default CRUSH rule when creating pools in stretch…

Reviewed-by: Samuel Just <sjust@redhat.com>

mon: Sanely set the default CRUSH rule when creating pools in stretch mode

If we get a pool create request while in stretch mode that does not explicitly
specify a crush rule, look at the stretch-mode pools and their rules, and
select the most common one.

Also update set_up_stretch_mode.sh to add a few more rules that let me test
this locally.

Fixes: https://tracker.ceph.com/issues/51270
Signed-off-by: Greg Farnum <gfarnum@redhat.com>
(cherry picked from commit 4f9fdca136bde10479fc173c9bcf4288ea960ace)

cls/cmpomap: empty values are 0 in U64 comparisons

previously, when trying to use cmpomap interfaces on an omap key with
an empty value, U64 comparisons would fail to decode with -EIO. so
cmp_set_vals() and cmp_rm_keys() are unable to update or remove such
keys

for backward-compatibility with rgw's data sync error repo, where the
keys used to have empty values, enable these comparisons by treating an
empty value as 0

Fixes: https://tracker.ceph.com/issues/52128
Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit 23339590ca693c6577eb6de3b47103d60ff57a8b)

mds: create file system with specific ID

File system will need to be recreated when monitor databases are lost
and rebuilt. Some applications (e.g., CSI) expect that the recovered
file system have the same ID as before. Allow creating a file system
with a specific ID to help in such scenarios. This can now be done by
the `fs new` command using the argument 'fscid' and 'force' flag.
Newer file systems will no longer have increasing IDs as a corollary.

Fixes: https://tracker.ceph.com/issues/51340
Signed-off-by: Ramana Raja <rraja@redhat.com>
(cherry picked from commit a0a8ba5087f0b82588860cda188dfdb48a964771)

Conflicts:
qa/tasks/cephfs/test_admin.py
        fixed conflicts due tests reorder in master

cmake: exclude "grafonnet-lib" target from "all"

so we don't build this target when running "make", and hence avoid
accessing the internet in a building envronment where the internest
access is not allowed.

Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit 1835fd86ddfe7f00e9ec3124fa1425e53cd7a930)

Conflicts:
monitoring/grafana/dashboards/CMakeLists.txt
- pacific does not have "LOG_DOWNLOAD ON", "LOG_MERGED_STDOUTERR ON", or
  "LOG_OUTPUT_ON_FAILURE ON", but that fact is orthogonal to the substance of
  this backport

Merge pull request #42632 from cfsnyder/wip-51471-pacific

pacific: rgw/sts: correcting the evaluation of session policies

Reviewed-by: Casey Bodley <cbodley@redhat.com>

os/bluestore: respect bluestore_warn_on_spurious_read_errors setting

Fixes: https://tracker.ceph.com/issues/51540
Signed-off-by: Igor Fedotov <ifedotov@suse.com>
(cherry picked from commit e4d44fc79094c36607aab9b98912cdd6bed67019)

mgr/dashboard: stats=false not working when listing buckets

Fixes: https://tracker.ceph.com/issues/51154
Signed-off-by: Avan Thakkar <athakkar@redhat.com>
(cherry picked from commit 95543bb150fc9e458314e62d80667190863caa0b)

qa/tasks/mgr/test_insights: Remove test for persistent checks

This test makes no sense if we are no longer persisting the store.

Signed-off-by: Brad Hubbard <bhubbard@redhat.com>
(cherry picked from commit 32d1cca2d9b606915c590f52d61856ee401fb4fc)

pybind/mgr/insights: Don't persist report data

Don't store health reports in rocksdb.

Fixes: https://tracker.ceph.com/issues/48269
Signed-off-by: Brad Hubbard <bhubbard@redhat.com>
(cherry picked from commit de66522517edd6f7baf19cc0660478502d3c25e8)

mgr/tests: Fix typo in function name

Signed-off-by: Brad Hubbard <bhubbard@redhat.com>
(cherry picked from commit 11268b332f582d658553774f0a9b9662da6c02ef)

Merge pull request #42868 from sebastian-philipp/pacific-backport-42852-42863

pacific: qa/cephadm: centos_8.x_container_tools_3.0.yaml

Reviewed-by: Yuri Weinstein <yweinste@redhat.com>

Merge pull request #42850 from ceph/wip-81356-pacific

pacific: run-make-check.sh: Increase failure output log size

Reviewed-by: Kefu Chai <tchaikov@gmail.com>

qa/distro: Add centos_8.2_container_tools_3.0.yaml

Let's avoid latest kubic stable

Fixes: https://tracker.ceph.com/issues/52279
Signed-off-by: Sebastian Wagner <sewagner@redhat.com>
(cherry picked from commit e436483c77adf7f876533c25a11c14d19a2d32a4)

Conflicts:
qa/suites/orch/cephadm/dashboard/0-distro/centos_8.2_kubic_stable.yaml
qa/suites/orch/cephadm/smoke/distro/centos_8.2_kubic_stable.yaml

qa/cephadm: centos_8.3_container_tools_3.0.yaml

Let's avoid latest stable

Fixes: https://tracker.ceph.com/issues/52279
Signed-off-by: Sebastian Wagner <sewagner@redhat.com>
(cherry picked from commit c6bd968eff96666da7e321fb5787c82dc66ddbb7)

kv/RocksDBStore: Add handling of block_cache option for resharding

Synchronized all situations when we initialize DB to include handling of block_cache option.
Lack of it prevented ability to reshard into specification that we have as default.

Conflicts:
src/kv/RocksDBStore.cc
Trivial conflict, related to gist of the change. No logic involved in resolving.

Fixes: https://tracker.ceph.com/issues/52246
Cherry-picked from: 2d6b20f7aaaf0e6

Signed-off-by: Adam Kupczyk <akupczyk@redhat.com>

doc/rados/op: ceph/df show DIRTY field N/A

DIRTY field should only be available if cache tiering is in use.

Signed-off-by: Deepika Upadhyay <dupadhya@redhat.com>
(cherry picked from commit ba52429897c34a132f2868aad3dc0c79de3db45d)