rgw: fix bucket limit check fill_status warnings

all variables in the calculation where ints, so the warning would
only ever trigger if objects_per_shard == limit_objects and the
percentage for > 100% used a granularity of 100%

Fixes: http://tracker.ceph.com/issues/40255
Signed-off-by: Paul Emmerich <paul.emmerich@croit.io>
(cherry picked from commit 694076419e6f20d1ed6c95b4b1ff94f134a10d4d)

Merge pull request #38620 from smithfarm/wip-48615-octopus

octopus: Do not add sensitive information in Ceph log files

Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #38477 from ifed01/wip-ifed-fast-rm-p1-oct

octopus: osd: optimize PG removal (part1)

Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Adam Kupczyk <akupczyk@redhat.com>

Merge pull request #38353 from ifed01/wip-ifed-fix-statfs-out-oct

octopus: mgr: don't update osd stat which is already out

Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge pull request #38267 from idryomov/wip-relax-preauth-asserts-again-octopus

octopus: msg/async/ProtocolV2: allow rxbuf/txbuf get bigger in testing, again

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

Merge pull request #38165 from rishabh-d-dave/wip-48279-octopus

octopus: qa/mgr: mgr_test_case: raise SkipTest instead of calling skipTest()

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #38098 from trociny/wip-48243-octopus

octopus: bluestore: fix "end reached" check in collection_list_legacy

Reviewed-by: Igor Fedotov <ifedotov@suse.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #38596 from callithea/wip-48551-octopus

octopus: mgr: Fix for dashboard/prometheus failure due to laggy pg state

Reviewed-by: Volker Theile <vtheile@suse.com>
Reviewed-by: Patrick Seidensal <pnawracay@suse.com>

Merge pull request #38526 from s0nea/wip-48538-octopus

octopus: mgr/dashboard: test_standby* (tasks.mgr.test_dashboard.TestDashboard) failed locally

Reviewed-by: Laura Paduano <lpaduano@suse.com>

Merge pull request #38508 from rhcs-dashboard/wip-48515-octopus

octopus: mgr/dashboard: Updating the inbuilt ssl providers error

Reviewed-by: Volker Theile <vtheile@suse.com>
Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Laura Paduano <lpaduano@suse.com>

Merge pull request #38489 from s0nea/wip-47975-octopus

octopus: mgr/dashboard: Add clay plugin support

Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Avan Thakkar <athakkar@redhat.com>

Merge pull request #38469 from s0nea/wip-48474-octopus

octopus: mgr/dashboard: assign flags to single OSDs

Reviewed-by: Kiefer Chang <kiefer.chang@suse.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>

Merge pull request #37960 from callithea/wip-47996-octopus

octopus: monitoring: Use null yaxes min for OSD read latency

Reviewed-by: Patrick Seidensal <pnawracay@suse.com>
Reviewed-by: Avan Thakkar <athakkar@redhat.com>

Merge pull request #37362 from smithfarm/wip-47085-octopus

octopus: mds: dump granular cap info in mds_sessions

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #38553 from batrick/i47095

octopus: mds: provide altrenatives to increase the total cephfs subvolume snapshot counts to greater than the current 400 across a Cephfs volume

Reviewed-by: Yuri Weinstein <yweinste@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #38551 from batrick/i48372

octopus: client: dump which fs is used by client for multiple-fs

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Yuri Weinstein <yweinste@redhat.com>

Merge pull request #38550 from batrick/i48524

octopus: qa: accept timeout argument in run_shell

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #38202 from Vicente-Cheng/wip-48111-octopus

octopus: doc: document MDS cache configuration

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #38095 from kotreshhr/wip-48191-octopus

octopus: mds: throttle cap acquisition via readdir

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #37989 from Vicente-Cheng/wip-48109-octopus

octopus: cephfs: client: increment file position on _read_sync near eof

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #37987 from Vicente-Cheng/wip-48098-octopus

octopus: cephfs: osdc: restart read on truncate/discard

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #37985 from Vicente-Cheng/wip-48096-octopus

octopus: cephfs: mds: move start_files_to_recover() to recovery_done

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #37690 from smithfarm/wip-47659-octopus

octopus: qa: ignore logrotate state rename error

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #37794 from ifed01/wip-ifed-fix-alloc-multi-reg-octopus

octopus: bluestore: provide a different name for fallback allocator

Reviewed-by: Adam Kupczyk <akupczyk@redhat.com>

Merge pull request #37860 from smithfarm/wip-47708-octopus

octopus: bluestore: Support flock retry

Reviewed-by: Igor Fedotov <ifedotov@suse.com>

os/bluestore: fix "end reached" check in collection_list_legacy

To preserve the old bluestore behavior it should compare the
current object with the end using bluestore keys, not oids.

Fixes: https://tracker.ceph.com/issues/48153
Signed-off-by: Mykola Golub <mgolub@suse.com>
(cherry picked from commit e63489f249f9ba3bc9cb1806568f860effd8a0b6)

Merge pull request #38338 from smithfarm/wip-48127-octopus

octopus: rgw: keep syncstopped flag when copying bucket shard headers

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #37766 from smithfarm/wip-47963-octopus

octopus: cephadm: Revert "spec: Podman (temporarily) requires apparmor-abstractions on suse"

Reviewed-by: Michael Fritch <mfritch@suse.com>
Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>

mon, messages: use TOPNSPC::common for crimson compatibility

Signed-off-by: Neha Ojha <nojha@redhat.com>
(cherry picked from commit 9f80520670e72f9b34f362db74acade464766e5a)

Conflicts:
src/messages/MMonCommand.h
- in master, class declaration has "final" qualifier which is missing in octopus
src/messages/MMonCommandAck.h
- in master, class declaration has "final" qualifier which is missing in octopus

Merge pull request #38344 from smithfarm/wip-48341-octopus

octopus: pybind/mgr/rbd_support: delay creation of progress module events

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
Reviewed-by: Mykola Golub <mgolub@mirantis.com>

Merge pull request #38343 from smithfarm/wip-48225-octopus

octopus: librbd: update hidden global config when removing pool config override

Reviewed-by: Mykola Golub <mgolub@mirantis.com>
Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #38342 from smithfarm/wip-48086-octopus

octopus: mgr/rbd_support: store global schedule without localized prefix

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
Reviewed-by: Mykola Golub <mgolub@mirantis.com>

Merge pull request #38341 from smithfarm/wip-48085-octopus

octopus: qa/workunits/rbd: fix permission issue when removing mirror peer

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #38340 from smithfarm/wip-48084-octopus

octopus: global: reexpand conf meta in child process

Reviewed-by: Mykola Golub <mgolub@mirantis.com>
Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #37861 from smithfarm/wip-47892-octopus

octopus: bluestore: attach csum for compressed blobs

Reviewed-by: Igor Fedotov <ifedotov@suse.com>

Merge pull request #37859 from smithfarm/wip-47749-octopus

octopus: build/ops: doc: pass --use-feature=2020-resolver to pip

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge branch 'octopus' into wip-47963-octopus

pybind/mgr/prometheus/module.py: defaultdict for num_by_state

num_by_state[state] += count in get_pg_status method raises KeyError
if pg state is not in PG_STATES list. PG_STATES should be synced with
osd_types.cc:pg_state_string(). But sometimes it is not. After the
KeyError raise mgr metrics are not available at all.

Fixes: https://tracker.ceph.com/issues/46142
Signed-off-by: Alexander Sushko <alexandrsushko@gmail.com>
(cherry picked from commit 3f7ee9cbd335e4b8686688b79ec6110d73a7390e)

 Conflicts:
src/pybind/mgr/prometheus/module.py
 - Resolved import conflict

monitoring: Use null yaxes min for OSD read latency

According to seriesOverrides that negative-Y for read param there shouldn't be a minimum for yaxes

Signed-off-by: Seena Fallah <seenafallah@gmail.com>
(cherry picked from commit 0fd28f646c50e97d36bac2359c4529438fbb0d9f)

mgr/ActivePyModules.cc: don't log config value in get_typed_config

Signed-off-by: Neha Ojha <nojha@redhat.com>
(cherry picked from commit f4b3d6b7692332c329b567d91460b8ada96741e2)

mon/ConfigMonitor.cc: don't log config values in refresh_config

Signed-off-by: Neha Ojha <nojha@redhat.com>
(cherry picked from commit 77932ae49bffd03f905f1039ae4add303738eec6)

mgr: don't add config values to mgr log in config_callback and module_config

The original code has been commented out and left for future debugging
purposes.

Signed-off-by: Neha Ojha <nojha@redhat.com>
(cherry picked from commit 19000fad573138b2b0576c093237c4a708b76020)

mon/ConfigMonitor.cc: don't dump all keys and values in config map

Signed-off-by: Neha Ojha <nojha@redhat.com>
(cherry picked from commit a68b4597785f92a0cb98440030be61f8004e3151)

mon: don't log "config set" and "config-key set" dispatch and finished messages

Signed-off-by: Neha Ojha <nojha@redhat.com>
(cherry picked from commit 4b83dfb1f74e8a59c802ff3c0eb4595f7e763762)

Conflicts:
        src/mon/Monitor.h
- replace one instance of "std::ostringstream ds;" with "stringstream ds;"

messages/MMonCommand, MMonCommandAck: don't log values for "config set" and "config-key set"

This acts like a big hammer to avoid adding sensitive information, like passwords
into mon/mgr/cluster logs when using "config set" and "config-key set" to set keys
whose values should be secure.

Fixes: https://tracker.ceph.com/issues/37503
Signed-off-by: Neha Ojha <nojha@redhat.com>
(cherry picked from commit 3d54660ca1a9a7ae54e884c3181fca17a40d8cd3)

Conflicts:
        src/messages/MMonCommand.h
- change "std::ostringstream ss" to "stringstream ss"
        src/messages/MMonCommandAck.h
- change "std::ostringstream ss" to "stringstream ss"

15.2.8

Merge branch 'CVE-2020-27781-octopus' into octopus

* CVE-2020-27781-octopus:
  tasks/cephfs/test_volume_client: Add tests for authorize/deauthorize
  pybind/ceph_volume_client: Optionally authorize existing auth-ids
  pybind/ceph_volume_client: Preserve existing caps while authorize/deauthorize auth-id
  pybind/ceph_volume_client: Disallow authorize auth_id

Reviewed-by: Ramana Raja <rraja@redhat.com>
Reviewed-by: Kotresh Hiremath Ravishankar <khiremat@redhat.com>

tasks/cephfs/test_volume_client: Add tests for authorize/deauthorize

1. Add testcase for authorizing auth_id which is not added by
   ceph_volume_client
2. Add testcase to test 'allow_existing_id' option
3. Add testcase for deauthorizing auth_id which has got it's caps
   updated out of band

Signed-off-by: Kotresh HR <khiremat@redhat.com>
(cherry picked from commit aa4beb3d993649a696af95cf27150cc460baaf70)

Conflicts:
qa/tasks/cephfs/test_volume_client.py

pybind/ceph_volume_client: Optionally authorize existing auth-ids

Optionally allow authorizing auth-ids not created by ceph_volume_client
via the option 'allow_existing_id'. This can help existing deployers
of manila to disallow/allow authorization of pre-created auth IDs
via a manila driver config that sets 'allow_existing_id' to False/True.

Fixes: https://tracker.ceph.com/issues/48555
Signed-off-by: Kotresh HR <khiremat@redhat.com>
(cherry picked from commit 77b42496e25cbd4af2e80a064ddf26221b53733f)

pybind/ceph_volume_client: Preserve existing caps while authorize/deauthorize auth-id

Authorize/Deauthorize used to overwrite the caps of auth-id which would
end up deleting existing caps. This patch fixes the same by retaining
the existing caps by appending or deleting the new caps as needed.

Fixes: https://tracker.ceph.com/issues/48555
Signed-off-by: Kotresh HR <khiremat@redhat.com>
(cherry picked from commit 47100e528ef77e7e82dc9877424243dc6a7e7533)

pybind/ceph_volume_client: Disallow authorize auth_id

This patch disallow the ceph_volume_client to authorize the auth_id
which is not created by ceph_volume_client. Those auth_ids could be
created by other means for other use cases which should not be modified
by ceph_volume_client.

Fixes: https://tracker.ceph.com/issues/48555
Signed-off-by: Ramana Raja <rraja@redhat.com>
Signed-off-by: Kotresh HR <khiremat@redhat.com>
(cherry picked from commit 3a85d2d04028a323952a31d18cdbefb710be2e2b)

Merge pull request #37973 from callithea/wip-47607-octopus

octopus: mgr/dashboard/api: move/create OSD histogram in separate endpoint

Reviewed-by: Aashish Sharma <aasharma@redhat.com>
Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>

Merge pull request #38405 from callithea/wip-48343-octopus

octopus: mgr/dashboard: Disable sso without python3-saml

Reviewed-by: Ernesto Puerta <epuertat@redhat.com>

Merge pull request #38331 from votdev/wip-48396-octopus

octopus: mgr/dashboard: Disable TLS 1.0 and 1.1

Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Laura Paduano <lpaduano@suse.com>

Merge pull request #38094 from bk201/wip-48226-octopus

octopus: mgr/dashboard: Use pipe instead of calling function within template

Reviewed-by: Laura Paduano <lpaduano@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>

qa/workunits/fs: add test for subvolume

Signed-off-by: "Yan, Zheng" <zyan@redhat.com>
(cherry picked from commit df7e9aa42b97f3203e92387d2ceea5636f5230ad)

mds: don't move inode with nlink > 1 to global snaprealm if it's in subvolume

Fixes: https://tracker.ceph.com/issues/46074
Signed-off-by: "Yan, Zheng" <zyan@redhat.com>
(cherry picked from commit 63da28d6a8be20183a294744892a41f47e8ba24b)

Conflicts: resolution similar to nautilus backport
src/mds/Server.cc

mds: disallow hardlink across subvolume

Signed-off-by: "Yan, Zheng" <zyan@redhat.com>
(cherry picked from commit 55cc3f7d7d27837bfe64ee46444358c0e45d16f6)

mds: disallow across subvolume rename

Signed-off-by: "Yan, Zheng" <zyan@redhat.com>
(cherry picked from commit 53348ab67f0f3e691cfeb6b02d80b868fdb98d89)

mds: disallow creating snapshot on descendent directory of subvolume

Signed-off-by: "Yan, Zheng" <zyan@redhat.com>
(cherry picked from commit 34c6b4437a2e8a49bb76e86d07cd86b740e3f581)

mds: add vxattr that marks/clears subvolume flag

Signed-off-by: "Yan, Zheng" <zyan@redhat.com>
(cherry picked from commit 8d531680cdf48ba43d93becd5a19072844e92c19)

Conflicts: resolutions similar to nautilus backport
src/mds/Server.cc
src/mds/SnapRealm.h
src/mds/snap.h

Merge pull request #37728 from ideepika/wip-octopus-tracker-47915

octopus: qa: drop hammer branch qa tests

Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>

Merge pull request #38345 from smithfarm/wip-48228-octopus

octopus: mon: Log "ceph health detail" periodically in cluster log

Reviewed-by: Neha Ojha <nojha@redhat.com>

client: dump which fs is used by client for multiple-fs

Fixes: https://tracker.ceph.com/issues/48246
Signed-off-by: Zhi Zhang <zhangz.david@outlook.com>
(cherry picked from commit 8daff24c2be2a7c54e569d0abf964986d9a472d4)

Conflicts:
src/client/Client.cc: surrounding changes unrelated

qa: accept timeout argument in run_shell

This was fixed differently in master in a way that's not easy to
backport.

Fixes: https://tracker.ceph.com/issues/48524
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

qa: drop hammer branch qa tests

fixes: https://tracker.ceph.com/issues/47731
Signed-off-by: Deepika Upadhyay <dupadhya@redhat.com>
(cherry picked from commit 7ef18559cbf68c7a09d73c528db600a3c959ac2c)

qa/suites/rados/thrash-old-clients/1-install: remove hammer

Signed-off-by: Neha Ojha <nojha@redhat.com>
(cherry picked from commit c258ff2c15aa295221d9c2c91af573ed9fbaaae7)

mgr/dashboard: test_standby* (tasks.mgr.test_dashboard.TestDashboard) failed locally

Fixes: https://tracker.ceph.com/issues/48449
Signed-off-by: Volker Theile <vtheile@suse.com>
(cherry picked from commit 9d4282e53af6c4a4c48641deeaa5f018781d5006)

mgr/dashboard: Adding the alert bad certificate error to the ssl providers error

upstream tracked in https://github.com/cherrypy/cheroot/pull/348
Fixes: https://tracker.ceph.com/issues/48490
Signed-off-by: Nizamudeen A <nia@redhat.com>
(cherry picked from commit 4cbe89f4db8ed13b2be46f2563c9d9618b0cf52b)

mgr/dashboard: Updating the inbuilt ssl providers error

Fixes: https://tracker.ceph.com/issues/48490
Signed-off-by: Nizamudeen A <nia@redhat.com>
(cherry picked from commit 0c890ea5e8589970c68c8c9ed94e8ac5a6da88ad)

Merge PR #38205 into octopus

* refs/pull/38205/head:
ceph-volume: pass *-slots arguments to LV creation
use extent count for slots conversion instead of free count
ceph-volume: available_lvm: vg space takes precedence

Merge PR #38014 into octopus

* refs/pull/38014/head:
ceph-volume: consume mount opt in simple activate

Reviewed-by: Jan Fajerski <jfajerski@suse.com>
Reviewed-by: Rishabh Dave <ridave@redhat.com>
Reviewed-by: Guillaume Abrioux <gabrioux@redhat.com>

Merge PR #38047 into octopus

* refs/pull/38047/head:
ceph-volume: fix lvm help test
ceph-volume: remove mention of dmcache from docs and help text

Reviewed-by: Rishabh Dave <ridave@redhat.com>
Reviewed-by: Jan Fajerski <jfajerski@suse.com>

Merge PR #38199 into octopus

* refs/pull/38199/head:
ceph-volume: fix test_setup_device_device_name_is_none
ceph-volume: cover devices.lvm.prepare.setup_device
ceph-volume: fix filestore/dmcrypt activate

Reviewed-by: Guillaume Abrioux <gabrioux@redhat.com>
Reviewed-by: Rishabh Dave <ridave@redhat.com>

Merge PR #38280 into octopus

* refs/pull/38280/head:
ceph-volume batch: reject partitions in argparser

Reviewed-by: Guillaume Abrioux <gabrioux@redhat.com>
Reviewed-by: Rishabh Dave <ridave@redhat.com>

Merge PR #38426 into octopus

* refs/pull/38426/head:
ceph-volume: implement the --log-level flag

Reviewed-by: Rishabh Dave <ridave@redhat.com>

mgr/dashboard: Add clay plugin support

The erasure code plugin "clay" is now supported by the dashboard. Now a
clay based profile can be created in the ec profile creation modal
dialog which can be found in the pool form.

The defaults of the plugin are calculated or preselected and shown in the
dashboard, therefore things are made mandatory even if they are not on the
cli, but as they automatically set the user doesn't have to set them,
but sees the defaults instantly before creating the profile.
(This is the same behavior that is used for all other supported
plugins.)

Fixes: https://tracker.ceph.com/issues/44433
Signed-off-by: Stephan Müller <smueller@suse.com>
(cherry picked from commit b3fd05bbc568cb775d25032ce87ea8dbb5106b3a)

Conflicts:
src/pybind/mgr/dashboard/frontend/src/app/shared/api/erasure-code-profile.service.ts
Fixed conflicts because https://github.com/ceph/ceph/pull/34696 has not
been backported to octopus.

mgr/dashboard: assign flags to single OSDs

Add the possibility to assign the flags ['noup',
'nodown', 'noin', 'noout'] to single OSDs.

Fixes: https://tracker.ceph.com/issues/40739
Signed-off-by: Tatjana Dehler <tdehler@suse.com>
(cherry picked from commit 3639332f34344f97ce5bd88d2fb17a659ca281a2)

Conflicts:
src/pybind/mgr/dashboard/controllers/osd.py
src/pybind/mgr/dashboard/frontend/src/app/ceph/cluster/cluster.module.ts
src/pybind/mgr/dashboard/frontend/src/app/ceph/cluster/osd/osd-list/osd-list.component.html
src/pybind/mgr/dashboard/frontend/src/app/ceph/cluster/osd/osd-list/osd-list.component.ts
src/pybind/mgr/dashboard/frontend/src/app/shared/constants/app.constants.ts
src/pybind/mgr/dashboard/openapi.yaml
Fixed conflicts because of missing octopus backports.

ceph-volume: fix lvm help test

ed5ceb0 changed the LVM help code but not the associated test.

Signed-off-by: Dimitri Savineau <dsavinea@redhat.com>
(cherry picked from commit 879ed30984de2b94879959de1c3611083c85bd99)

ceph-volume: remove mention of dmcache from docs and help text

With the introduction of bluestore dmcache is no longer needed and
is no longer supported with `ceph-volume lvm`.

Resolves: rhbz#1876827
Fixes: https://tracker.ceph.com/issues/48039
Signed-off-by: Andrew Schoen <aschoen@redhat.com>
(cherry picked from commit ed5ceb04fc8ff57c5f7e2b5fa5e859c2cdbf2ffd)

doc/osd-config-ref: adjust default value for osd_delete_sleep_hybrid

Signed-off-by: Igor Fedotov <ifedotov@suse.com>
(cherry picked from commit ba9b5471206e1b40ffa91f070cebfd918611d657)

osd: adjust osd_delete_sleep_ssd[_hybrid]

1) Do not differentiate all-flash and hybrid(ssd+hdd) deployments since
they both bound to SSD drive backing DB volume.
2) Drop osd_delete_sleep_hybrid from 2s to 1s
3) Raise osd_delete_sleep_ssd to 1s since 0 is a way too low - having
it that low tend to cause a significant negative imact on OSD's regular
operation.

Signed-off-by: Igor Fedotov <ifedotov@suse.com>
(cherry picked from commit 6e09f0c13a74d6d38522866875a83c6ee505cfa7)

os/bluestore: introduce removal perf counter

Signed-off-by: Igor Fedotov <ifedotov@suse.com>
(cherry picked from commit cc9d58421ef9785802482faa5e68ac25911331e8)

os/bluestore: track kv utilization in kv_sync_thread

Signed-off-by: Igor Fedotov <ifedotov@suse.com>
(cherry picked from commit 0504b9554998005307647744ba82ffbe9ad8c703)

 Conflicts:
src/common/options.cc
src/os/bluestore/BlueStore.cc
 trivial

os/bluestore: perf counters for omap_get_keys[_get_values]

Signed-off-by: Igor Fedotov <ifedotov@suse.com>
(cherry picked from commit 86840616ba62e37cd7bdc04fc0f5d63d2df93525)

osd/pg: use next when calling collection_list for pg removal

Signed-off-by: Igor Fedotov <ifedotov@suse.com>
(cherry picked from commit 7f047005fc72e1f37a45cde2d742bb2eb1e62881)

mgr/dashboard: remove custom class `oa-hr-small`

Signed-off-by: Tatjana Dehler <tdehler@suse.com>
(cherry picked from commit 9a66425fa42634c3e6879cd3503612fd9f714fbe)

mgr/dashboard: remove unnecessary blank

Signed-off-by: Tatjana Dehler <tdehler@suse.com>
(cherry picked from commit 52a9c7610dffa400ce915e7e7253bfd0ad76bed0)

Merge pull request #38155 from sebastian-philipp/octopus-backport-37432-37397-37112-36622-37411-37521-37500-37104-37542-37650-37621-3764

octopus: cephadm batch backport November

Reviewed-by: Michael Fritch <mfritch@suse.com>

doc: document MDS cache configuration

Fixes: https://tracker.ceph.com/issues/48010
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit cb9b2e5815529f2be28a136462c57d86f004657f)

test: add DecayCounter test

In particular, look at the steady state decay. (i.e. a counter that is
continuously "refilled".)

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit ce151e3e472da75d707fb12ab8c7c32cb65c9248)

client: increment file position on _read_sync near eof

Fixes: https://tracker.ceph.com/issues/48076
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit 83f84f601aada043fdece5c733a85230786545cb)

osdc: restart read on truncate/discard

This can be reliably reproduced by reading a large file (~500MB) on a
ceph-fuse mount and then evicting the client.

Now we see in the logs:

2020-10-21T01:46:29.679+0000 7f00beffd700 10 objectcacher.object(10000000000.00000030/head) truncate object[10000000000.00000030/head oset 0x7f00a0006b08 wr 0/0] to 0
2020-10-21T01:46:29.679+0000 7f00beffd700 10 objectcacher.object(10000000000.00000030/head) restarting reads post-truncate
...
2020-10-21T01:46:29.762+0000 7f00bffff700 20 objectcacher.object(10000000000.00000030/head) map_read error bh[ 0x7f0070004e30 1048576~3145728 0x7f008404a720 (0) v 0 error=-108] waiters = {}
2020-10-21T01:46:29.762+0000 7f00bffff700 10 objectcacher readx hit bh bh[ 0x7f0070004e30 1048576~3145728 0x7f008404a720 (0) v 0 error=-108] waiters = {}
2020-10-21T01:46:29.762+0000 7f00bffff700 10 objectcacher readx has all buffers
2020-10-21T01:46:29.762+0000 7f00bffff700 20 objectcacher readx done 0x7f007c00a9b0 -108
...
2020-10-21T01:46:29.763+0000 7f0077fff700  3 client.4497 ll_read 0x7f0080023720 202637312~131072 = -108

Fixes: https://tracker.ceph.com/issues/46434
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit b53390031378a06980fa8e7dae074a7095307c12)

Conflicts:
src/osdc/ObjectCacher.cc
  - keep the specific data type instead using auto

mds : move start_files_to_recover()  to recovery_done

the requests in the queue, named waiting_for_replay, may modify the state of the filelock,
resulting in the wrong lock state when repairing file (start_files_to_recover)

Fixes : https://tracker.ceph.com/issues/46906
Signed-off-by: Simon Gao <simon29rock@gmail.com>
(cherry picked from commit fafb5b4f84e12ba00a68550ffb73fc9bcde867a0)

Merge pull request #38287 from yuriw/wip-yuriw-cleanup-octopus

octopus: qa/tests: added rhel 8.2

Reviewed-by: Rakesh Manohar <rgowdege@redhat.com>

common: do not dup the options when reexpanding

The old code will store all the options, which has `$pid` in them,
in may_reexpand_meta map. And when reexpanding later, the reexpand
code will dup them with a higher priority(CONF_OVERRIDE).

This will be a problem, if the default value has `$pid` and be
stored in the may_reexpand_meta map, and then the code set a new
different value, which may have no `$pid`, from CLI or config file.
The reexpand will override it with the default value always.

This will do not duplicate the options with CONF_OVERRIDE priority
when reexpanding, just refresh them and call the observers. And the
finalize_reexpand_meta() will always be called after the fork() is
done in child processes.

Fixes: https://tracker.ceph.com/issues/48240
Signed-off-by: Xiubo Li <xiubli@redhat.com>
(cherry picked from commit 56ca87ae3b3c341a78a8df8f95d3bf31828c9108)

common/config_proxy: clear the values.changed in _gather_changes()

Signed-off-by: Xiubo Li <xiubli@redhat.com>
(cherry picked from commit 7496be719c3419224ac7fc29d6db4e282b193554)

global: reexpand the conf meta in all the child processes

Especially for the tools or the daemons whose config options need
to expand the '$pid', they will be always expanded with the parent
processes. We need to reexpand them in child processes just after
the fork is done.

Fixes: https://tracker.ceph.com/issues/48240
Signed-off-by: Xiubo Li <xiubli@redhat.com>
(cherry picked from commit 73a48dfe8ed165cb6c8295a88f133acb30e7065e)

Conflicts:
PendingReleaseNotes

ceph-volume: consume mount opt in simple activate

When running ceph-volume simple activate command on a Filestore OSD
then the data device is mounted without any specific options so the
one from the ceph configuration file are ignored.
When deploying Filestore with the lvm subcommand then everything is
fine because the filestore_activate method uses mount_osd which relies
on the mount options defined in the ceph configuration file (if any).

Fixes: https://bugzilla.redhat.com/show_bug.cgi?id=1891557
Fixes: https://tracker.ceph.com/issues/48018
Signed-off-by: Dimitri Savineau <dsavinea@redhat.com>
(cherry picked from commit 1f4301a15df82bf31468d76fbcccc1c5fa192e38)

qa: fix proc exit status check

Fixes: f30a84b6a7a21670d8b5c5a7fe19df70e4371260
Fixes: https://tracker.ceph.com/issues/47677
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit eb95dabc3f5cac18c4efd33f595bbf9799901f33)