qa: do not append file names to dirname

Otherwise the files generated are not actually under the sub-directory!
This is correcting a confusing aspect of the test infrastructure but
doesn't actually require any changes to the tests.

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit ac6c150eb07ddd61c33fc4fad034fee6f5fbad35)

qa: add exception for test timeouts

To make this easier to catch. It is still a RuntimeError so it should
not affect current tests by default.

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit dbc44e5e342605c16e1213120bec0bfec1c0091d)

Merge pull request #36439 from batrick/i46533

nautilus: mds: fix nullptr dereference in MDCache::finish_rollback

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

mds: fix nullptr dereference in MDCache::finish_rollback

introduced by commit 4940ab62e0d
"mds: preserve ESlaveUpdate::OP_PREPARE logevent before doing commit"

Fixes: https://tracker.ceph.com/issues/46533
Signed-off-by: "Yan, Zheng" <zyan@redhat.com>
(cherry picked from commit 6715514aa577842942a1221a0f25924c53e03834)

Merge pull request #35733 from idryomov/wip-msgr21-nautilus

nautilus: New msgr2 crc and secure modes (msgr2.1)

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

Merge pull request #36377 from batrick/i46763

nautilus: qa/tasks: call super class's setUp()

Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Yuri Weinstein <yweinste@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #36339 from smithfarm/wip-46741-nautilus

nautilus: osd: fix crash in _committed_osd_maps if incremental osdmap crc fails

Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Yuri Weinstein <yweinste@redhat.com>

qa/tasks: call super class's setUp()

to address the regression introduced by
87292811215f6ded9a784d3216a910faaef648e2

Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit 7d262db1145a14a73da481acba66e51b5f30e4af)

Merge pull request #36353 from neha-ojha/wip-46731-nautilus

nautilus: qa/*/nautilus-p2p-stress-split: rework objectstore backends

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #36314 from kotreshhr/wip-46641-nautilus

nautilus: qa: Fix name collisions

Reviewed-by: Ramana Raja <rraja@redhat.com>

qa/suites/rados/thrash/crc-failures: randomly inject bad incremental osdmap crc

Signed-off-by: Neha Ojha <nojha@redhat.com>
(cherry picked from commit f9399b508c6584e539e323da858372fa7115be3b)

osd: don't write transaction when inc crc failed

80da5f9a987c6a48b93f25228fdac85890013520 exposed a flaw in how
handle_osd_map falls back to a full osdmap if the crc of an incremental
failed.

If the first message in a map message had a crc error, then the
loop would exit with last < start, which would then cause a null
dereference in _committed_osd_maps.

Fixes: https://tracker.ceph.com/issues/46443
Signed-off-by: Dan van der Ster <daniel.vanderster@cern.ch>
(cherry picked from commit 99463cb523949289384a6210ea250114fbe852f4)

qa/standalone/osd: add bad-inc-map.sh

Test that the osd doesn't crash when it gets a bad incremental osdmap.

Related-to: https://tracker.ceph.com/issues/46443
Signed-off-by: Dan van der Ster <daniel.vanderster@cern.ch>
(cherry picked from commit b550112dba23528655ec150cf9d6d093318abb8c)

qa/*/nautilus-p2p-stress-split: rework objectstore backends

Earlier we were symlinking to qa/objectstore, which causes older versions of
nautilus to fail when trying to use hybrid allocator.

Fixes: https://tracker.ceph.com/issues/46731
Signed-off-by: Neha Ojha <nojha@redhat.com>

Merge pull request #36250 from dillaman/wip-46673

nautilus: librbd: new 'write_zeroes' API methods to suppliment the `discard` APIs

Reviewed-by: Mykola Golub <mgolub@mirantis.com>

Merge pull request #36292 from dzafman/wip-46706-nautilus

nautilus: osd: Cancel in-progress scrubs (not user requested)

Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge pull request #36190 from smithfarm/wip-46458-nautilus

nautilus: build/ops: selinux: allow ceph_t amqp_port_t:tcp_socket

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #36161 from smithfarm/wip-46090-nautilus

nautilus: osd/PG: fix history.same_interval_since of merge target again

Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>
Reviewed-by: Neha Ojha <nojha@redhat.com>

qa: Fix name collisions

Addresses the name collisions of volumes, subvolumes,
clones, subvolume groups and snapshots within the tests.

Fixes: https://tracker.ceph.com/issues/43517
Signed-off-by: Kotresh HR <khiremat@redhat.com>
(cherry picked from commit 3b6f6f1abf1beba54c7b4eb610ae05985cdf42b1)

Conflicts:
    qa/tasks/cephfs/test_volumes.py: Few of the test cases were not
present in nautilus which were present in master, removed them.

Merge pull request #36167 from ajarr/nautilus-mgr-fs-volumes-misc

nautilus: mgr/fs/volumes misc fixes

Reviewed-by: Kotresh HR <khiremat@redhat.com>

Merge pull request #36294 from ajarr/wip-46470

nautilus: client: release the client_lock before copying data in read

Reviewed-by: Ramana Raja <rraja@redhat.com>

Merge pull request #36215 from vshankar/wip-46388

nautilus: client: introduce timeout for client shutdown

Reviewed-by: Ramana Raja <rraja@redhat.com>

Merge pull request #36180 from ajarr/wip-46464-nautilus

nautilus: pybind/cephfs: fix custom exception raised by cephfs.pyx

Reviewed-by: Ramana Raja <rraja@redhat.com>

Merge pull request #36179 from smithfarm/wip-46523-nautilus

nautilus: mds: fix hang issue when accessing a file under a lost parent directory

Reviewed-by: Ramana Raja <rraja@redhat.com>

Merge pull request #36178 from smithfarm/wip-46521-nautilus

nautilus: mds: do not submit omap_rm_keys if the dir is the basedir of merge.

Reviewed-by: Ramana Raja <rraja@redhat.com>

Merge pull request #36177 from smithfarm/wip-46517-nautilus

nautilus: cephfs: client: fix directory inode can not call release callback

Reviewed-by: Ramana Raja <rraja@redhat.com>
Reviewed-by: Jeff Layton <jlayton@redhat.com>

Merge pull request #36175 from smithfarm/wip-46474-nautilus

nautilus: mds: make threshold for MDS_TRIM configurable

Merge pull request #36173 from smithfarm/wip-46409-nautilus

nautilus: cephfs: client: fix setxattr for 0 size value (NULL value)

Reviewed-by: Brad Hubbard <bhubbard@redhat.com>

Merge pull request #36172 from smithfarm/wip-46310-nautilus

nautilus: tests: qa/tasks/cephfs/cephfs_test_case.py: skip cleaning the core dumps when in program case

Reviewed-by: Brad Hubbard <bhubbard@redhat.com>

Merge pull request #36171 from smithfarm/wip-46200-nautilus

nautilus: tests: Revert "Revert "qa/suites/rados/mgr/tasks/module_selftest: whitelist …

Reviewed-by: Ramana Raja <rraja@redhat.com>
Reviewed-by: Brad Hubbard <bhubbard@redhat.com>

Merge pull request #36170 from smithfarm/wip-46189-nautilus

nautilus: mds: reset heartbeat in EMetaBlob replay

Reviewed-by: Ramana Raja <rraja@redhat.com>

Merge pull request #36169 from smithfarm/wip-46187-nautilus

nautilus: cephfs: client: fix snap directory atime

Reviewed-by: Ramana Raja <rraja@redhat.com>

Merge pull request #36270 from yuriw/wip-yuriw-nautilus-p2p-nautilus

nautilus: qa/tests: changed rbd-python.yaml workunit to v14.2.10

Reviewed-by: Neha Ojha <nojha@redhat.com>

nautilus: qa/test_volumes: Modify tests not to use json for 'ceph fs status'

Signed-off-by: Kotresh HR <khiremat@redhat.com>

nautilus: mgr/volumes: use MgrModule's mon_command()

... instead of 'check_mon_command' which is not in nautilus, and not
compatible with python2 and early versions of py3.

Signed-off-by: Ramana Raja <rraja@redhat.com>

nautilus: mgr/volumes: fix import typing error

... for python versions earlier than 3.5.

Signed-off-by: Ramana Raja <rraja@redhat.com>

pybind/mgr/volumes: fix volume search for pools

Loop logic would bail out if it first sees any file system that does not
match the volume it's looking for.

Fixes: https://tracker.ceph.com/issues/46277
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit be74a81e19ef3b5e8204d52e6a3f7f0c57d49854)

mgr/volumes: Validate mon_allow_pool_delete before volume deletion

Volume deletion wasn't validating mon_allow_pool_delete config
before destroying volume metadata. Hence when mon_allow_pool_delete
is set to false, it was deleting metadata but failed to delete pool
resulting in inconsistent state. This patch validates the config
before going ahead with deletion.

Fixes: https://tracker.ceph.com/issues/45662
Signed-off-by: Kotresh HR <khiremat@redhat.com>
(cherry picked from commit e770bb9075d931913847a572d121e02a2e349ca8)

qa: manage config changes through mons

This provides a generic framework for modifying Ceph configuration
changes in tests through the monitors rather than the asok interface or
local ceph.conf changes. Any changes are reverted during test teardown.

A future patch will convert existing tests manipulating the local
ceph.conf or admin socket.

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit 87292811215f6ded9a784d3216a910faaef648e2)

mgr/volumes: Fix pool removal on volume deletion

While volume deletion, the associated pools are not always
removed. The pools are removed only if the volume is created
using mgr plugin and not if created with custom osd pools.
This is because mgr plugin generates pool names with specific
pattern. Both create and delete volume relies on it. This
patch fixes the issue by identifying the pools of the volume
without relying on the pattern.

Fixes: https://tracker.ceph.com/issues/45910
Signed-off-by: Kotresh HR <khiremat@redhat.com>
(cherry picked from commit d07ea8db471b1a1d9082756e1cb775d5543b1307)

Conflicts:
src/pybind/mgr/volumes/fs/operations/volume.py:
- In nautilus, fs volume create doesn't have placement arg
src/pybind/mgr/volumes/fs/volume.py:
- In nautilus, VolumeClient code not moved to mgr_util.py

client: release the client_lock before copying data in read

Fixes: https://tracker.ceph.com/issues/46025
Signed-off-by: Chencan <chen.can2@zte.com.cn>
(cherry picked from commit db292da7d5571ed3addedfb4c5ab8c61fe9bb5e8)

Conflicts:
       src/client/Client.cc
- in nautilus, not switched bufferlist::copy() to iterators

Merge pull request #35500 from ifed01/wip-ifed-hybrid-alloc-nau

nautilus: os/bluestore: implement Hybrid allocator

Reviewed-by: akupczyk@redhat.com

qa/objectstore: test hybrid allocator instead of avl one.

Signed-off-by: Igor Fedotov <ifedotov@suse.com>
(cherry picked from commit 0f639740a3b99d0600f6508d90d426ca12501a03)

os/bluestore: set hybrid allocator as a default for bluefs/bluestore

Signed-off-by: Igor Fedotov <ifedotov@suse.com>
(cherry picked from commit cacc7391dca779c7e6e3b57e6d1753a53b8c49b9)

 Conflicts:
src/test/objectstore/store_test.cc
 trivial

os/bluestore: improve adjustent extents merge in hybrid allocatoc

Bitmap counterpart is inspected for adjustent free extents when
releasing extent and trying to add it to AVL tree.

Signed-off-by: Igor Fedotov <ifedotov@suse.com>
(cherry picked from commit c27490e3d641f2b8ea65759fd79640e5f6dd398c)

os/bluestore: remove HybridAllocator::init_add_free method.

It's a 100% replica of AvlAllocator one which is inherited.

Signed-off-by: Igor Fedotov <ifedotov@suse.com>
(cherry picked from commit cccbc62528e03f4e49882137342adfea20b9a0f2)

os/bluestore: configure hybrid allocator threshold to fallback.

Signed-off-by: Igor Fedotov <ifedotov@suse.com>
(cherry picked from commit b682d9a8e8d3f41d55ad10d2fb3deca8edf8540a)

 Conflicts:
src/common/options.cc
 trivial

os/bluestore: add hybrid (avl+bitmap) allocator

Signed-off-by: Igor Fedotov <ifedotov@suse.com>
(cherry picked from commit c90eee34955734a5d4d2d4f0d1ff04b275374fb1)

 Conflicts:
src/os/CMakeLists.txt
src/test/objectstore/Allocator_bench.cc
src/test/objectstore/Allocator_test.cc
src/test/objectstore/CMakeLists.txt
  GTEST_SKIP not available, +trivial

os/bluestore: do not require min_length to be power of two for bmap
allocator.

Signed-off-by: Igor Fedotov <ifedotov@suse.com>
(cherry picked from commit 21532045efeb8d357518686df84a529ee3d32a60)

 Conflicts:
src/os/bluestore/fastbmap_allocator_impl.h
 variable rename

os/bluestore: extend AvlAllocator interface.

Signed-off-by: Igor Fedotov <ifedotov@suse.com>
(cherry picked from commit cccf94da463aa15370fe674bda24d168d331ceca)

tests/test_bluestore_types: show AvlAllocator's range_seg_t size.

Extend corresponding test case to be aware of the struct's sizeof.

Signed-off-by: Igor Fedotov <ifedotov@suse.com>
(cherry picked from commit 719cd7d9423d1d16e3005cc0b807699ed24533de)

os/bluestore: make AvlAllocator overridable.

Signed-off-by: Igor Fedotov <ifedotov@suse.com>
(cherry picked from commit 59c0aa5eb40957364ddb0e9d62dffa3449991249)

os/bluestore: intdoduce spillover mechanics to AvlAllocator.

This to be utilized in AvlAllocator's descendant class(es) to be able
to limit amount of entries allocator tracks. Hence bringing a cap for
RAM used by the allocator.

Signed-off-by: Igor Fedotov <ifedotov@suse.com>
(cherry picked from commit efedcc077c7ed4e1e5ea26ae989fa3d6cbd287d4)

os/bluestore: avoid constant name replica in bitmap allocator.
This was introduced by Adam's https://github.com/ceph/ceph/pull/29425

Signed-off-by: Igor Fedotov <ifedotov@suse.com>
(cherry picked from commit 683f7bab5a571cc3a6c7106fdb1d4017cc843c5d)

os/bluestore: implement original allocator fragmentation inspection.

Using admin socket interface: bluestore allocator fragmentation
[block|bluefs-db|bluefs-wal|bluefs-slow]

Signed-off-by: Igor Fedotov <ifedotov@suse.com>
(cherry picked from commit 11b5feb01d273365c32ec6419faead629fb8c20b)

os/bluestore: simplify Allocator::get_fragmentation() signature
Remove alloc_unit parameter.

Signed-off-by: Igor Fedotov <ifedotov@suse.com>
(cherry picked from commit 3a96db924968b65afed5fc1355acbf185455c5c4)

os/bluestore: StupidAllocator stucks on 4GB allocations

Signed-off-by: Igor Fedotov <ifedotov@suse.com>
(cherry picked from commit 1a91a67bf2fab64d27a000b3c62d3d7beac1aa75)

 Conflicts:
src/test/objectstore/Allocator_test.cc
 Previous gtest version

test/allocator_test: additional coverage for get_fragmentation_score

Tests the function results agains the same cases as we do for
get_fragmentation.

Signed-off-by: Igor Fedotov <ifedotov@suse.com>
(cherry picked from commit 6ae213181433dd2ae392d8eed85c2606b9c4bb54)

os/bluestore: Adds shutdown in destructor of AvlAllocator

Adds shutdown. Fixes incorrect mempool accounting.

Signed-off-by: Adam Kupczyk <akupczyk@redhat.com>
(cherry picked from commit ea95309f478428b6e4b3af93b19459f6b5cc162d)

 Conflicts:
src/test/objectstore/Allocator_bench.cc
 trivial

qa: add AvlAllocator test script

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
(cherry picked from commit 29512a70d45628ff4efcd4fd45b5d689b21b01d8)

src/test/objectstore/Allocator_{test,bench}.cc: test AvlAllocator too

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
(cherry picked from commit 97dfbc9c4816027d766731f0facf1ffa03455491)

 Conflicts:
src/test/objectstore/Allocator_bench.cc
src/test/objectstore/Allocator_test.cc
 GTEST_SKIP not present, +trivial

os/bluestore: implement avl-extent-based allocator

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit adbc79a8ac28115c692a9fe6d4f6c098a7c7a5ea)

 Conflicts:
src/common/options.cc
 non-backported options

Merge pull request #36290 from neha-ojha/wip-fio-nautilus

nautilus: qa/suites/rados/perf: run on ubuntu only

Reviewed-by: Yuri Weinstein <yweinste@redhat.com>

qa/suites/rados/perf: run on ubuntu only

This change is specific to nautilus since fio tests have been failing
on centos and rhel 7.8 due to dependency issues.

Signed-off-by: Neha Ojha <nojha@redhat.com>

test: Check for interuption of scrubs with nosrub/nodeep_scrub

Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit 365e48d6ec71cec7c85acbf918927a20856d55da)

Conflicts:
qa/standalone/scrub/osd-scrub-test.sh (The scrub/deep_scrub trigger
doesn't use ceph tell)

Merge pull request #36074 from neha-ojha/wip-46184-nautilus

nautilus: mgr/DaemonServer.cc: make 'config show' on fsid work

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #35713 from dzafman/wip-46116

nautilus: tools: Add statfs operation to ceph-objecstore-tool

Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge pull request #35464 from bk201/wip-45837-nautilus

nautilus: monitoring: fixing some issues in RBD detail dashboard

Reviewed-by: Stephan Müller <smueller@suse.com>
Reviewed-by: Laura Paduano <lpaduano@suse.com>

Merge pull request #35237 from smithfarm/wip-45677-nautilus

nautilus: qa/workunits: Remove redundant Xenial cmake3 requirements

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Brad Hubbard <bhubbard@redhat.com>

Merge pull request #35841 from Vicente-Cheng/wip-46191-nautilus

nautilus: mds: fix filelock state when Fc is issued

Reviewed-by: Ramana Raja <rraja@redhat.com>
Reviewed-by: Yan, Zheng <zyan@redhat.com>

osd: Cancel in-progress scrubs (not user requested)

This change adds new scrubber.req_scrub to track user
requested scrubs, deep_scrub or repair.

Fixes: https://tracker.ceph.com/issues/46275
Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit 33749cc3c39131d7abed9b8c14064dbfaa87f3a2)

Conflicts:
PendingReleaseNotes (trivial)
src/osd/PG.cc (Due to code re-arrangement changes add manually)

osd: Arrange code so that it is clearer should not cause any change

Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit ef9ab850d87a84ce42c2c8df1c9b502b2fb04989)

test: mon-last-epoch-clean.sh fixed to avoid shell globbing

Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit f272768802648f2cb4d519c4c790ab219ad43f9f)

qa/tests: changed rbd-python.yaml workunit to v14.2.10

Signed-off-by: Yuri Weinstein <yweinste@redhat.com>

selinux: add amqp and soundd types to ceph.te

Signed-off-by: Thomas Serlin <tserlin@redhat.com>
(cherry picked from commit c3c5129748d2580ec95ef11609b124eef4dc29ff)

Conflicts:
selinux/ceph.te

selinux: allow ceph_t amqp_port_t:tcp_socket

allow ceph_t amqp_port_t:tcp_socket name_connect;
allow ceph_t soundd_port_t:tcp_socket name_connect;

Required for running RabbitMQ

(soundd_port_t) for running RabbitMQ on port 8000

Fixes: https://bugzilla.redhat.com/show_bug.cgi?id=1854083
Signed-off-by: Kaleb S. KEITHLEY <kkeithle@redhat.com>
(cherry picked from commit 05c523185b2d5ddd9e10f425c7e1f1ee1e409ba2)

Merge pull request #36186 from smithfarm/wip-45930-nautilus

nautilus: rgw: Add support wildcard subuser for bucket policy

Reviewed-by: J. Eric Ivancich <ivancich@redhat.com>

Merge pull request #36163 from ivancich/nautilus-bucket-listing-clear-results

nautilus: rgw: cls_bucket_list_(un)ordered should clear results collection

Reviewed-by: Casey Bodley <cbodley@redhat.com>

rbd: use new 'write_zeroes' API to ensure zeroed extents will be zeroed

Fixes: https://tracker.ceph.com/issues/45986
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit f319ecfbec3486917824eab84ab76347319b8bde)
(cherry picked from commit f53e43c6b1c520733ffd2c506a958fb7c4a45f9c)

pybind/rbd: add new write_zeroes/aio_write_zeroes API methods

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit c27d744a339d3644b756ee496445909f63c8903e)
(cherry picked from commit 6a884184b680b89e2b46ad001651e07e58514eb0)

librbd: add 'write_zeroes' public C/C++ API methods

Unlike the existing 'discard' option which is more of a hint to
attempt to release space, the new 'write_zeroes' APIs will ensure
that the entire provided extent is fully zeroed.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit ae6dd86b22e928dc23a385faf41cf76b0d293576)

Conflicts:
src/librbd/api/Io.h/cc: logic exists in ImageRequestWQ files
src/librbd/librbd.cc: trivial resolution due missing api::Io
src/test/librbd/test_librbd.cc: trivial resolution
(cherry picked from commit 0e3add97ff1f919d81a1c644b150672cbd691c84)

Conflicts:
        src/librbd/io/ImageRequestWQ.cc: different locks and no overlap guard
        src/test/librbd/test_librbd.cc: trivial resolution

Merge pull request #35601 from smithfarm/wip-46012-nautilus

nautilus: qa/vstart_runner: update vstart_runner.LocalRemote.sh

Reviewed-by: Ramana Raja <rraja@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Kyr Shatskyy <kyrylo.shatskyy@suse.com>
Reviewed-by: Jos Collin <jcollin@redhat.com>
Reviewed-by: Rishabh Dave <ridave@redhat.com>

Merge pull request #35401 from smithfarm/wip-45854-nautilus

nautilus: tools/cephfs: don't bind to public_addr

Reviewed-by: Ramana Raja <rraja@redhat.com>

Merge pull request #34461 from Vicente-Cheng/wip-44487-nautilus

nautilus: qa: add upgrade test for volume upgrade from legacy

Reviewed-by: Ramana Raja <rraja@redhat.com>

Merge pull request #36166 from ShyamsundarR/wip-46527-nautilus

nautilus: mgr/volumes: Deprecate protect/unprotect CLI calls for subvolume snapshots

Reviewed-by: Ramana Raja <rraja@redhat.com>

Merge pull request #36193 from smithfarm/wip-46476-nautilus

nautilus: rgw/iam: correcting the result of get role policy.

Reviewed-by: J. Eric Ivancich <ivancich@redhat.com>

Merge pull request #36192 from smithfarm/wip-46472-nautilus

nautilus: rgw: stop realm reloader before store shutdown

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #36191 from smithfarm/wip-46468-nautilus

nautilus: rgw: radoslist incomplete multipart uploads fix marker progression

Reviewed-by: J. Eric Ivancich <ivancich@redhat.com>

Merge pull request #36189 from smithfarm/wip-46004-nautilus

nautilus: rgw: add check for index entry's existing when adding bucket stats during bucket reshard.

Reviewed-by: J. Eric Ivancich <ivancich@redhat.com>

Merge pull request #36187 from smithfarm/wip-45932-nautilus

nautilus: rgw: Add subuser to OPA request

Reviewed-by: J. Eric Ivancich <ivancich@redhat.com>

Merge pull request #36185 from smithfarm/wip-45927-nautilus

nautilus: rgw: fix loop problem with swift stat on account.

Reviewed-by: J. Eric Ivancich <ivancich@redhat.com>

Merge pull request #36184 from smithfarm/wip-45925-nautilus

nautilus: rgw: add quota enforcement to CopyObj

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #36165 from ivancich/wip-45923-nautilus

nautilus: rgw: bucket list/stats truncates for user w/ >1000 buckets

Reviewed-by: Casey Bodley <cbodley@redhat.com>

rgw: cls_bucket_list_(un)ordered should clear results collection

Each call to cls_bucket_list_(un)ordered should have an empty
collection to populate with results. Rather than rely on the caller to
insure this, it's more reliable to have these functions do the clear.

Additionally in some cases, a reserve call was added to the collection
to pre-allocate the space needed for the expected number of
results. This will potentially result in fewer re-allocations plus
copies.

Signed-off-by: J. Eric Ivancich <ivancich@redhat.com>
(cherry picked from commit c52039a9e8a49f3a29e2d2ee80e50d7b1b441842)

Conflicts:
    src/rgw/rgw_admin.cc
    src/rgw/rgw_rados.cc

client: introduce timeout for client shutdown

Client::shutdown() could indefinitely wait when tearing down
MDS sessions if an MDS is unreachable during shutdown, but a
valid session existed, i.e., Client::mount() was successfull.

These failures were initially observed in mgr/volumes tests in
teuthology and lately during rados/mgr selftests where manager
plugins do not respawn as volumes plugin waits for cleaning up
libcephfs handles via Client::shutdown().

Fixes: http://tracker.ceph.com/issues/44276
Signed-off-by: Venky Shankar <vshankar@redhat.com>
(cherry picked from commit 7c716be4b94d7640d051d427f5d7e97fefe5baf2)

 Conflicts:
src/client/Client.cc
src/client/Client.h
src/common/options.cc

Condition variables in nautilus use WaitInterval() rather
than wait_for(). Client::_closed_mds_session() does not accept
errno parameter in nautilus -- so adjust for that. Also, fixup
config option conflict.

mgr/volumes: Remove enum usage in code for Py2 support

SubvolumeFeatures uses an enum for the feature list, this is
replaced with a constant instead.

Signed-off-by: Shyamsundar Ranganathan <srangana@redhat.com>

rgw/iam: correcting the result of get role policy.

Fixes: https://tracker.ceph.com/issues/45671
Signed-off-by: Pritha Srivastava <prsrivas@redhat.com>
(cherry picked from commit 9e1ae9de9905587a227e50d0aff6533e290a8f87)

rgw: add quota enforcement to CopyObj

Fixes: https://tracker.ceph.com/issues/45382
Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit 70d397541cc8c0a68b3e420a78fec8f0f66f06b0)

Conflicts:
    src/rgw/rgw_op.cc
- nautilus uses plain "store->" where master has "store->getRados()->"
- nautilus store->get_object_state takes 6 arguments instead of 7 in
  master

rgw: stop realm reloader before store shutdown

otherwise the two could race to destroy the store

Fixes: https://tracker.ceph.com/issues/45969
Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit 7787dfb4926dcc2baa7530b620ae8cfccf90bc30)

Conflicts:
src/rgw/rgw_main.cc
- RGWRealmWatcher realm_watcher(...) line looks different in nautilus

rgw: radoslist incomplete multipart uploads fix marker progression

When the radosgw-admin radoslist was running, the portion where it
lists the incomplete multipart uploads did not handle the marker
correctly between calls to the ordered list op. This fixes those
issues and does some additional clean-up.

Signed-off-by: J. Eric Ivancich <ivancich@redhat.com>
(cherry picked from commit 653848da4ae67a45a5d72b0daece86de939f5e15)