octopus: qa: detect timed out failure indicating mount still active

Otherwise, the umount process will fail because the mount still exists
when the mountpoint cleanup (rmdir) is started.

See:

    2020-10-04T22:08:24.448 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules...
    2020-10-04T22:08:24.449 INFO:teuthology.orchestra.run.smithi063:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore'
    2020-10-04T22:08:24.464 INFO:teuthology.orchestra.run.smithi189:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore'
    2020-10-04T22:08:24.482 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules.
    2020-10-04T22:08:24.483 INFO:teuthology.orchestra.run:Running command with timeout 900
    2020-10-04T22:08:24.483 INFO:teuthology.orchestra.run.smithi063:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T
    2020-10-04T22:08:24.483 INFO:teuthology.orchestra.run.smithi063:> ' -- /home/ubuntu/cephtest/mnt.0)
    2020-10-04T22:08:34.550 INFO:teuthology.orchestra.run.smithi063:> sudo logrotate /etc/logrotate.d/ceph-test.conf
    2020-10-04T22:08:34.553 INFO:teuthology.orchestra.run.smithi189:> sudo logrotate /etc/logrotate.d/ceph-test.conf
    2020-10-04T22:09:04.592 INFO:teuthology.orchestra.run.smithi063:> sudo logrotate /etc/logrotate.d/ceph-test.conf
    2020-10-04T22:09:04.596 INFO:teuthology.orchestra.run.smithi189:> sudo logrotate /etc/logrotate.d/ceph-test.conf
    2020-10-04T22:09:34.727 INFO:teuthology.orchestra.run.smithi063:> sudo logrotate /etc/logrotate.d/ceph-test.conf
    2020-10-04T22:09:34.730 INFO:teuthology.orchestra.run.smithi189:> sudo logrotate /etc/logrotate.d/ceph-test.conf
    2020-10-04T22:10:04.815 INFO:teuthology.orchestra.run.smithi063:> sudo logrotate /etc/logrotate.d/ceph-test.conf
    2020-10-04T22:10:04.818 INFO:teuthology.orchestra.run.smithi189:> sudo logrotate /etc/logrotate.d/ceph-test.conf
    2020-10-04T22:10:34.876 INFO:teuthology.orchestra.run.smithi063:> sudo logrotate /etc/logrotate.d/ceph-test.conf
    2020-10-04T22:10:34.880 INFO:teuthology.orchestra.run.smithi189:> sudo logrotate /etc/logrotate.d/ceph-test.conf
    2020-10-04T22:11:04.923 INFO:teuthology.orchestra.run.smithi063:> sudo logrotate /etc/logrotate.d/ceph-test.conf
    2020-10-04T22:11:04.926 INFO:teuthology.orchestra.run.smithi189:> sudo logrotate /etc/logrotate.d/ceph-test.conf
    2020-10-04T22:11:34.996 INFO:teuthology.orchestra.run.smithi063:> sudo logrotate /etc/logrotate.d/ceph-test.conf
    2020-10-04T22:11:35.000 INFO:teuthology.orchestra.run.smithi189:> sudo logrotate /etc/logrotate.d/ceph-test.conf
    2020-10-04T22:12:05.064 INFO:teuthology.orchestra.run.smithi063:> sudo logrotate /etc/logrotate.d/ceph-test.conf
    2020-10-04T22:12:05.067 INFO:teuthology.orchestra.run.smithi189:> sudo logrotate /etc/logrotate.d/ceph-test.conf
    2020-10-04T22:12:35.202 INFO:teuthology.orchestra.run.smithi063:> sudo logrotate /etc/logrotate.d/ceph-test.conf
    2020-10-04T22:12:35.205 INFO:teuthology.orchestra.run.smithi189:> sudo logrotate /etc/logrotate.d/ceph-test.conf
    2020-10-04T22:13:05.316 INFO:teuthology.orchestra.run.smithi063:> sudo logrotate /etc/logrotate.d/ceph-test.conf
    2020-10-04T22:13:05.318 INFO:teuthology.orchestra.run.smithi189:> sudo logrotate /etc/logrotate.d/ceph-test.conf
    2020-10-04T22:13:24.520 INFO:teuthology.orchestra.run.smithi063.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.0': Connection timed out
    2020-10-04T22:13:24.521 DEBUG:teuthology.orchestra.run:got remote process result: 1
    2020-10-04T22:13:24.522 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.0
    2020-10-04T22:13:24.640 INFO:teuthology.orchestra.run:Running command with timeout 300
    2020-10-04T22:13:24.641 INFO:teuthology.orchestra.run.smithi063:> (cd /home/ubuntu/cephtest && exec rm -rf /home/ubuntu/cephtest/mnt.0)
    2020-10-04T22:13:24.688 INFO:teuthology.orchestra.run.smithi063.stderr:rm: cannot remove '/home/ubuntu/cephtest/mnt.0': Is a directory
    2020-10-04T22:13:24.688 DEBUG:teuthology.orchestra.run:got remote process result: 1

From: /ceph/teuthology-archive/pdonnell-2020-10-04_21:51:57-fs-wip-pdonnell-testing-20201004.051319-octopus-distro-basic-smithi/5494771/teuthology.log

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

qa: set rados op timeouts for mds/ceph-fuse

Now that the osdc Objecter obeys updates to these configs, let's use
them to avoid having them block forever on operations that may never
complete (or should complete in a timely manner).

Fixes: https://tracker.ceph.com/issues/47734
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit d060c9a768c6974d3b68e4d408edf78bba9e0e85)

qa: print debug info on mount cleanup

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit f8f607dbd11a5db044bd2981ca1a2fbdae752f10)

Conflicts:
qa/tasks/cephfs/mount.py

        Notes: skip as cleanup is abstract.

qa: remove redundant rmr

The mount.cleanup method will remove the mount point. This `rm -rf` will
always fail (with exit status 0).

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit 8e7a89009c41570e9f87f57d92d519b7c49d7706)

Conflicts:
qa/tasks/cephfs/fuse_mount.py

        Notes: convert to cleanup call.

qa: use null mode to prevent undesired changes to mountpoint

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit 2432871a1ba41223c52bf59fd12f1b8f53c03bda)

Conflicts:
qa/tasks/cephfs/fuse_mount.py

qa: unmount all clients before deleting the file system

Otherwise we have unnecessary timeout waits.

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit bc25bd70f2a357046b300135c1b32f1cf120daa5)

Conflicts:
qa/tasks/cephfs/test_admin.py

        Notes: delete_all_filesystems method moved

osdc: add timeout configs for mons/osds

Have the Objecter track the rados_(mon|osd)_op_timeout configs so that
it can be configured at runtime/startup. This is useful for the
MDS/ceph-fuse so that we can avoid waiting forever for a response from
the Monitors that will never come (statfs on a deleted file system's
pools).

Also: make these configs take a time value rather than double. This is
simpler to deal with in the code and allows time units to be used (e.g.
"5m" for 5 minutes).

Fixes: https://tracker.ceph.com/issues/47734
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit a8a23747aa081d938c9b277ab42507dd506bf6c2)

Conflicts:
src/client/Client.cc
src/librados/RadosClient.cc
src/mds/MDSRank.cc
src/mgr/MgrStandby.cc
src/mon/MonClient.h
src/neorados/RADOSImpl.cc
src/osd/OSD.cc
src/osdc/Objecter.cc
src/osdc/Objecter.h
src/test/mon/test_mon_workloadgen.cc
src/tools/cephfs/MDSUtility.cc

        Notes: different Objecter cons arguments. Added conf obs for
        RadosClient.

common: accept timespan for SaferCond.wait_for

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit 0feabb4ff16e5b97d90718eb677c85699e776fdb)

Merge pull request #37673 from Vicente-Cheng/wip-47815-octopus

octopus: rgw: fix setting of namespace in ordered and unordered bucket listing

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #37672 from Vicente-Cheng/wip-47850-octopus

octopus: rgw/rgw_file: Fix the incorrect lru object eviction

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #37640 from pritha-srivastava/wip-47836-octopus

octopus: rgw: adds code for creating and managing oidc provider entities in rgw and for offline validation of OpenID Connect Access and ID Token.

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #37434 from rhcs-dashboard/wip-47675-octopus

octopus: mgr/dashboard: cluster > manager modules

Merge pull request #37365 from smithfarm/wip-47147-octopus

octopus: qa/tasks/nfs: Test mounting of export created with nfs command

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #37578 from callithea/wip-47770-octopus

Reviewed-by: Nizamudeen A <nia@redhat.com>
Reviewed-by: Sebastian Krah <skrah@suse.com>

Merge pull request #37331 from callithea/wip-47461-octopus

octopus: mgr/dashboard: Update datatable only when necessary

Reviewed-by: Tiago Melo <tmelo@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>

Merge pull request #37367 from smithfarm/wip-47247-octopus

octopus: qa: ignore expected mds failover message

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #37366 from smithfarm/wip-47151-octopus

octopus: pybind/mgr/volumes: add global lock debug

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #37363 from smithfarm/wip-47089-octopus

octopus: mds: standy-replay mds remained in the "resolve" state after resta…

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #37360 from smithfarm/wip-47083-octopus

octopus: mds: fix 'forward loop' when forward_all_requests_to_auth is set

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #37354 from smithfarm/wip-46940-octopus

octopus: mds: place MDSGatherBuilder on the stack

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #37358 from smithfarm/wip-47021-octopus

octopus: cephfs: libcephfs: ignore restoring the open files limit

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #37357 from smithfarm/wip-47018-octopus

octopus: mds: fix kcephfs parse dirfrag's ndist is always 0

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #37356 from smithfarm/wip-47016-octopus

octopus: mds: revert the decode version

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Jos Collin <jcollin@redhat.com>

Merge pull request #37355 from smithfarm/wip-46942-octopus

octopus: mds: resolve SIGSEGV in waiting for uncommitted fragments

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #37353 from smithfarm/wip-46859-octopus

octopus: mds: do not raise "client failing to respond to cap release" when client working set is reasonable

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #37352 from smithfarm/wip-46857-octopus

octopus: qa: add debugging for volumes plugin use of libcephfs

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #37351 from smithfarm/wip-46855-octopus

octopus: cephfs: client: use non-static dirent for thread-safety

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #37350 from smithfarm/wip-46463-octopus

octopus: pybind/cephfs: fix custom exception raised by cephfs.pyx

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #37334 from smithfarm/wip-47087-octopus

octopus: mds: don't recover files after normal session close

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #37249 from Vicente-Cheng/wip-46786-octopus

octopus: cephfs: client: fix extra open ref decrease

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #37248 from Vicente-Cheng/wip-46783-octopus

octopus: mds/CInode: Optimize only pinned by subtrees check

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #37247 from Vicente-Cheng/wip-46637-octopus

octopus: mds: optimize random threshold lookup for dentry load

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #37243 from Vicente-Cheng/wip-46636-octopus

octopus: mds: fix nullptr dereference in MDCache::finish_rollback

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #37240 from Vicente-Cheng/wip-46634-octopus

octopus: mds: fix mds forwarding request 'no_available_op_found'

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge PR #37520 into octopus

* refs/pull/37520/head:
ceph-volume: don't exit before empty report can be printed
PendingReleaseNotes: add note about batch refactor
pybind/mgr/orchestrator: fix preview for new c-v batch output
ceph-volume batch: return valid empty json reports
python-common: do not skip unavailable devices
ceph-volume: pass filter_for_batch as keyword argument
ceph-volume batch: fix very_fast_allocation plan and add tests
ceph-volume: batch: call the right prepare method
python-common drive-groups: allow strings for wal and db size
ceph-volume inventory: add option to filter unwanted devices
ceph-volume: address review comments
ceph-volume: batch: fix size retrieval for lvs
ceph-volume: include encryption in batch report
ceph-volume lvm batch: use namedtuple instead of tuple
ceph-volume: address review comments, mostly tidying, clarification
idempotency must result in the same outcome
ceph-volume: batch test should pass --journal-devices with filestore
ceph-volume: make --journal optional, add --journal-slots
doc: update ceph-volume lvm batch docs
py-common/deployment: drop workarounds for old batch
ceph-volume batch: add deprecation warning for auto behaviour
ceph-volume batch: add ceph.conf mocking to pass tests
ceph-volume batch: use disk.Size for size args
ceph-volume batch: Fix osd_ids passing and improve plan formatting
ceph-volume batch: track rel_size in percent, more tests
ceph-volume batch: improve backwards compatibility
ceph-volume: batch - enable legacy auto behaviour
ceph-volume: batch - major refactor
ceph-volume: Device - available_lvm if 10 extents are free.
ceph-volume: Device - add vg_free property
ceph-volume: prepare/create - size args as Size class
ceph-volume: disk.Size - add cast to bool
ceph-volume: api/lvm - add VolumeGroup.free_percent property
ceph-volume: util.device -  add vg_free_percent property
ceph-volume: api/lvm - query LV units in bytes
ceph-volume: lvm/common - refactor common arg specification
ceph-volume: batch: fix argument help message

Reviewed-by: Rishabh Dave <ridave@redhat.com>

mgr/dashboard: Show warning when replicated size is 1

Issues a warning when creating a replicated pool and the replicated size
is 1. It won't stop the user from creating the pool, but will give the
user a hint that no replication will be created.

Fixes: https://tracker.ceph.com/issues/42404
Signed-off-by: Sebastian Krah <skrah@suse.com>
(cherry picked from commit af1a7dbfe2b0e2e40650d64d4c3247f4961743c2)

 Conflicts:
src/pybind/mgr/dashboard/frontend/src/styles/defaults/_bootstrap-defaults.scss
- The file does not exist in octopus, moving a slightly changed version of the diff
  to src/pybind/mgr/dashboard/frontend/src/styles.scss instead

rgw: remove unused librgw_admin_user

unused and not worth the effort to maintain

Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit c89b59428e66ae6d89d1fd27a4280045c57251cb)

rgw: fix setting of namespace in ordered and unordered bucket listing

The namespace is not always set correctly during bucket listing. This
can, for example, cause the listing of incomplete multipart uploads,
which are in the _multipart_ namespace, to not paginate correctly, and
cause entries to be re-listed.

Signed-off-by: J. Eric Ivancich <ivancich@redhat.com>
(cherry picked from commit bd6f163f366753e8ec42b85a53334f4bf78916bd)

rgw/rgw_file: Fix the incorrect lru object eviction

In func lookup_fh, when RGWFileHandle not be found in fh_cache, it
need to recycle an object and create an new RGWFileHandle. When there
are multi threads use lookup_fh to find and create RGWFileHandle concurrently,
it must to make sure evict lru object from the partiton of fh_cache which new
RGWFileHandle will be inserted to.

Fixes: https://tracker.ceph.com/issues/47235
Signed-off-by: luo rixin <luorixin@huawei.com>
(cherry picked from commit f2097338722d7f2526bb815da47695f2da17fcce)

Merge pull request #37562 from smithfarm/wip-47762-octopus

octopus: mgr/prometheus: add pool compression stats

Reviewed-by: Boris Ranto <branto@redhat.com>

Merge pull request #37504 from smithfarm/wip-arm-isa-l-octopus

octopus: erasure-code: enable isa-l EC for aarch64 platform

Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #37450 from callithea/wip-47621-octopus

octopus: mgr/dashboard: fix security scopes of some NFS-Ganesha endpoints

Reviewed-by: Volker Theile <vtheile@suse.com>
Reviewed-by: Tatjana Dehler <tdehler@suse.com>

Merge pull request #37383 from ukernel/octopus-47608

octopus: mds: reduce memory usage of open file table prefetch #37382

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #37372 from smithfarm/wip-47604-octopus

octopus: mds: fix purge_queue's _calculate_ops is inaccurate

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #37371 from smithfarm/wip-47601-octopus

octopus: mgr/volumes/nfs: Check if orchestrator spec service_id is valid

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #37370 from smithfarm/wip-47260-octopus

octopus: cephfs: client: handle readdir reply without Fs cap

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #37369 from smithfarm/wip-47255-octopus

octopus: cephfs: client: make Client::open() pass proper cap mask to path_walk

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #37368 from smithfarm/wip-47253-octopus

octopus: cephfs: osdc/Journaler: do not call onsafe->complete() if onsafe is 0

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

rgw: adding code for policy evaluation for ops
like getbucketversioning, putbucketversioning etc

Signed-off-by: Pritha Srivastava <prsrivas@redhat.com>
(cherry picked from commit 5f86459381ed9f3c5565d1b74fc064f11b55c275)

rgw: Correcting pool name for oidc provider for older
versions of osd.

Signed-off-by: Pritha Srivastava <prsrivas@redhat.com>
(cherry picked from commit 1e50f8cac9c7e16322ec7d8177d8679ab29cc28a)

rgw: fix for make failure in test folder.

Signed-off-by: Pritha Srivastava <prsrivas@redhat.com>
(cherry picked from commit 47dc509366162121a392651be81010235e8f5154)

rgw: adds code for role session to be used an ARN.

Signed-off-by: Pritha Srivastava <prsrivas@redhat.com>
(cherry picked from commit 039a2b9ce52e3ed2c5422423deaf27c32de3773e)

rgw: adding new iam ops for iam oidc provider
to correct unittest faliure.

Signed-off-by: Pritha Srivastava <prsrivas@redhat.com>
(cherry picked from commit 9f3d772d8061563dd189bb6442d68fff576e162e)

rgw: modifying documentation to include explanation of
open id connect provider related REST APIs and removing
references to token introspection for validating incoming
web token.

Signed-off-by: Pritha Srivastava <prsrivas@redhat.com>
(cherry picked from commit d228d613b00067e1c72f65afdbf90c3cf2fee396)

rgw: adds a commom method to remove prefix(www, http and https)
from a url.

Signed-off-by: Pritha Srivastava <prsrivas@redhat.com>
(cherry picked from commit ca96ae806f79063922f27c3b4628026b9930fd89)

rgw: adds code to check for client id and cert thumbprint
while validating incoming web token in AssumeRoleWithWebIdentity.

Signed-off-by: Pritha Srivastava <prsrivas@redhat.com>
(cherry picked from commit 75b4060c7732bb00ea0811771920658a6492b168)

rgw: Added code for offline OpenId Connect/ OAuth 2.0 tokens
if the token is a JWT.

Signed-off-by: Pritha Srivastava <prsrivas@redhat.com>
(cherry picked from commit 7566664f89be062e0c9f3519dc60b94c8af5e2a4)

rgw: Adding jwt-cpp header only library for JWT validation.
https://github.com/Thalhammer/jwt-cpp

Signed-off-by: Pritha Srivastava <prsrivas@redhat.com>
(cherry picked from commit 8052d6e322dc662ea9cf83431b662a414e11ea1d)

rgw: adds REST APIs for OpenID connect providers manipulation.

This commit implements:
1. CreateOpenIDConnectProvider
2. DeleteOpenIDConnectProvider
3. ListOpenIDConnectProviders
4. GetOpenIDConnectProviders

Signed-off-by: Pritha Srivastava <prsrivas@redhat.com>
(cherry picked from commit aec4815c21dccd3be5766c985488fe4f68a41079)

mds/OpenFileTable: add option that disables dirfrags prefetch

Dirfrags prefetch may speedup rejoin stage of mds recovery. But it may
use lots of memory and cause mds out of memory.

Signed-off-by: "Yan, Zheng" <zyan@redhat.com>
(cherry picked from commit cc19fc624b1ee4d7e3248d1dfc8f89f8879a46bf)

mds: use parent/d_name in cap_reconnect_t to speedup MDCache::open_ino()

Signed-off-by: "Yan, Zheng" <zyan@redhat.com>
(cherry picked from commit 3e7c5b4077b68ec519f9e3e98fea8fd9dcf9cbc3)

client: encode inode's parent/d_name in cap_reconnect_t

Signed-off-by: "Yan, Zheng" <zyan@redhat.com>
(cherry picked from commit 7f43840fd3f9a8f88f809dcad5d3d381e76ba7f3)

mds: explicitly track files with client writeable ranges

Signed-off-by: "Yan, Zheng" <zyan@redhat.com>
(cherry picked from commit 44f67be02b8e8aa8fc69283bf12107c2df3f8277)

 Conflicts:
src/mds/CInode.cc
src/mds/Locker.cc
src/mds/MDCache.cc
src/mds/Server.cc
src/mds/StrayManager.cc

mds: properly set CLIENTWRITEABLE flag for imported caps

Signed-off-by: "Yan, Zheng" <zyan@redhat.com>
(cherry picked from commit 8c56adabb36f2983437fa73f88c00b7d62d0a880)

mds: in-place update projected inode's client ranges

Signed-off-by: "Yan, Zheng" <zyan@redhat.com>
(cherry picked from commit 994017b6155af7fbb5a53f76f6a4525b363ba07f)

 Conflicts:
src/mds/Locker.cc
src/mds/Server.cc

mds: only add inodes that clients want Frw or excl caps to open file table

Signed-off-by: "Yan, Zheng" <zyan@redhat.com>
(cherry picked from commit e6de138f3334adc26cac4c01dfbbdf8a2472679a)

mds/OpenFileTable: reduce anchor map lookup during prefetch

reduce one anchor map lookup for each inode

Signed-off-by: "Yan, Zheng" <zyan@redhat.com>
(cherry picked from commit 1adc21e1b46744fa810b9d29530a04369af303e7)

mds/OpenFileTable: merge dirfrag set into anchor map

Track dirfrags whose child inodes have caps in corresponding dir
inodes' anchor objects.

Signed-off-by: "Yan, Zheng" <zyan@redhat.com>
(cherry picked from commit 12691168dfed4c6cfb97ad3f4841a078b84a1f11)

Merge pull request #37090 from shyukri/wip-46961-octopus

octopus: mgr/dashboard: Improve notification badge

Reviewed-by: Aashish Sharma <aasharma@redhat.com>
Reviewed-by: Laura Paduano <lpaduano@suse.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>

Merge pull request #37300 from alvistack/octopus-aio_graceful_reboot

octopus: systemd: Support Graceful Reboot for AIO Node

Reviewed-by: Deepika Upadhyay <dupadhya@redhat.com>

Merge pull request #37533 from smithfarm/wip-47739-octopus

octopus: mgr/devicehealth: device_health_metrics pool gets created even without any OSDs in the cluster

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge pull request #37225 from smithfarm/wip-47531-octopus

octopus: ceph.in: ignore failures to flush stdout

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #37128 from k0ste/wip-47426-octopus

octopus: mgr/zabbix: indent the output of "zabbix config-show"

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #37048 from trociny/wip-47363-octopus

octopus: bluestore: fix collection_list ordering

Reviewed-by: Igor Fedotov <ifedotov@suse.com>

Merge pull request #37436 from sebastian-philipp/octopus-backport-36800-36894-36929-36995-36946-36432-36903-36766-37046-37071-36980-3693

octopus: cephadm batch backport September (2)

Reviewed-by: Michael Fritch <mfritch@suse.com>
Reviewed-by: Nathan Cutler <ncutler@suse.com>

ceph-volume: don't exit before empty report can be printed

get_plan() called exit in case of an empty plan. This prevented a report
being printed under these circumstances. Avoid exit in this case. Also
adds tests to ensure an empty report is printed.

Fixes: https://tracker.ceph.com/issues/47760
Signed-off-by: Jan Fajerski <jfajerski@suse.com>
(cherry picked from commit 0cc5604843b215709a681fa402145c9fa403b1dd)

PendingReleaseNotes: add note about batch refactor

Signed-off-by: Jan Fajerski <jfajerski@suse.com>
(cherry picked from commit 485f0d797e4b21ffb7ac742f0949e8c0a23d43f2)

 Conflicts:
PendingReleaseNotes
        Sort ceph-volume entry under 15.2.6 heading

pybind/mgr/orchestrator: fix preview for new c-v batch output

Signed-off-by: Jan Fajerski <jfajerski@suse.com>
(cherry picked from commit 996df56a4aebe29e32fd529ae926ae484d240943)

 Conflicts:
src/pybind/mgr/orchestrator/tests/test_orchestrator.py
        drop test_handle_command(), not added by us

Merge PR #37345 into octopus

* refs/pull/37345/head:
ceph-volume: avoid format strings for now

Reviewed-by: Guillaume Abrioux <gabrioux@redhat.com>
Reviewed-by: Rishabh Dave <ridave@redhat.com>

doc: replace luminous with |stable-release|

"luminous" is a magic number in these contexts, and we should use a
constant for representing it.

the "sphinx_substitution_extensions" sphinx extension is introduced for
performing the global subsitution.

Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit 7ca85e77d040e990fc60d7fe99447d3fb1e7f69e)

ceph-volume batch: return valid empty json reports

Fixes: https://tracker.ceph.com/issues/47729
Signed-off-by: Jan Fajerski <jfajerski@suse.com>
(cherry picked from commit ab59269a6ca5bb80c28e94beef0338f23fc10fff)

python-common: do not skip unavailable devices

Fixes: https://tracker.ceph.com/issues/47726
We should assemble the device set based on the drivegroups in
mgr/cephadm (regardless of their availability) and pass it to c-v.

This patch is the missing piece to actually do this.

related changes: https://github.com/ceph/ceph/pull/34740/commits/7d168ad7bdbb6d6d5231a4ae540ab03040b49a38

Signed-off-by: Joshua Schmid <jschmid@suse.de>
(cherry picked from commit b9e3da39193419c3fd606a4cba586bea58c6e961)

ceph-volume: pass filter_for_batch as keyword argument

This PR also removes an unused ctor argument in the Devices class.

Fixes: 7d168ad7bdbb6d6d5231a4ae540ab03040b49a38
Signed-off-by: Jan Fajerski <jfajerski@suse.com>
(cherry picked from commit c5a711e5b6499915b7c2a7b7869f890fa7dc7e2d)

ceph-volume batch: fix very_fast_allocation plan and add tests

Signed-off-by: Jan Fajerski <jfajerski@suse.com>
(cherry picked from commit 8178d5c48ac1a7f3915f0003abab6d625385bd78)

ceph-volume: batch: call the right prepare method

Signed-off-by: Jan Fajerski <jfajerski@suse.com>
(cherry picked from commit e75ef77f23ae07463510ec213ac4007f29cbe2da)

python-common drive-groups: allow strings for wal and db size

This enables user to pass sizes like "10G", which batch now understands.

Signed-off-by: Jan Fajerski <jfajerski@suse.com>
(cherry picked from commit 28e33f3025d03008b46b56989e5d939682b5720d)

ceph-volume inventory: add option to filter unwanted devices

Some device we never want to pass to the batch subcommand. For now this
includes devices that have a partition or are mounted on the machine.
One goal is to filter the root device, so it is not included on a batch
command and thus would contribute to its implicit sizing calculation.

Signed-off-by: Jan Fajerski <jfajerski@suse.com>
(cherry picked from commit 7d168ad7bdbb6d6d5231a4ae540ab03040b49a38)

ceph-volume: address review comments

Signed-off-by: Jan Fajerski <jfajerski@suse.com>
(cherry picked from commit 6f1592a1146529d352184c795aae8ce12f66e554)

ceph-volume: batch: fix size retrieval for lvs

Signed-off-by: Jan Fajerski <jfajerski@suse.com>
(cherry picked from commit 24e4aa1296608ef861d4ea4b6a1892246a53ef76)

ceph-volume: include encryption in batch report

Fixes: https://tracker.ceph.com/issues/44783
Signed-off-by: Jan Fajerski <jfajerski@suse.com>
(cherry picked from commit fce184cf9b2d8f15543e1adee49e1fe6cc17437d)

ceph-volume lvm batch: use namedtuple instead of tuple

Signed-off-by: Jan Fajerski <jfajerski@suse.com>
(cherry picked from commit 98c991fc6fd17b18d5bfbebe4b8febe5ff8fa2f0)

ceph-volume: address review comments, mostly tidying, clarification

Signed-off-by: Jan Fajerski <jfajerski@suse.com>
(cherry picked from commit d0735ce1c90c952a6d2e1b805c1326d13ff7b06c)

idempotency must result in the same outcome

...not should

Co-authored-by: Joshua Schmid <jschmid@suse.de>
Signed-off-by: Jan Fajerski <jfajerski@suse.com>
(cherry picked from commit fcacd0b96ab195e939f6f879b0a0362a06385f9a)

ceph-volume: batch test should pass --journal-devices with filestore

Signed-off-by: Jan Fajerski <jfajerski@suse.com>
(cherry picked from commit 55ba8630176041dce898e8b979b5849f13e01ca5)

ceph-volume: make --journal optional, add --journal-slots

Signed-off-by: Jan Fajerski <jfajerski@suse.com>
(cherry picked from commit 7f766846328aac82d75175ed2c1c0bf3438a99e0)

doc: update ceph-volume lvm batch docs

Signed-off-by: Jan Fajerski <jfajerski@suse.com>
(cherry picked from commit 7695d1ec539f54b7b7cea8d19925f0c320223f03)

py-common/deployment: drop workarounds for old batch

This commit drops two workarounds in drive groups for the old batch
behaviour. OSDs on LVs no longer need to be created individually by
create (related to functional testing) and ceph-volume batch now filters
unavailable drives itself.

Signed-off-by: Jan Fajerski <jfajerski@suse.com>
(cherry picked from commit 0f6f2b46a347c41a15f7ab2d532efb299f226698)

ceph-volume batch: add deprecation warning for auto behaviour

Signed-off-by: Jan Fajerski <jfajerski@suse.com>
(cherry picked from commit 1239f77c8871a45e21bd1e3e81be8bb8854f24da)

ceph-volume batch: add ceph.conf mocking to pass tests

Signed-off-by: Jan Fajerski <jfajerski@suse.com>
(cherry picked from commit eef9dc7a1da6d5dde0d1b02b71301c1d7b7926a9)

ceph-volume batch: use disk.Size for size args

Signed-off-by: Jan Fajerski <jfajerski@suse.com>
(cherry picked from commit 0bc7f7424cdd0a2c5f2cd777467814dbb3959fb4)