Merge pull request #29016 from ivancich/mimic-abort-multipart-fix

mimic: rgw: abort multipart fix

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #29021 from smithfarm/wip-39074-mimic

mimic: rgw: multisite: data sync loops back to the start of the datalog after reaching the end

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #29139 from ivancich/mimic-fix-resharding-race

mimic: rgw: fix race b/w bucket reshard and ops waiting on reshard completion

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #29017 from smithfarm/wip-38080-mimic

mimic: multisite: overwrites in versioning-suspended buckets fail to sync

Reviewed-by: Casey Bodley <cbodley@redhat.com>

rgw: add RGWRados::repair_olh() to reconstruct olh attrs

Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit 1309e2d50ee8118afe1138cd6677d84438edb55a)

Conflicts:
    src/rgw/rgw_rados.cc
- no rgw_bl_str() in mimic
- mimic does not have a4d3977cfacc5eb0de16618f2c957bb417505597

rgw: add RGWRados::bi_get_olh() to read/decode olh entry

Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit b8d7f0ad1f882b999e0fd7f3e45f8579599714cf)

Conflicts:
    src/rgw/rgw_rados.cc
- bufferlist::cbegin()

rgw: bi_get() takes bucket_info

and calls the bucket_info overload of BucketShard::init() to avoid
reading the bucket info again

Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit a15e8736340c916120fef1f3a5ff1b982cf7c837)

rgw: add BucketShard::init overload for bucket_info,obj

Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit c6c445f6074df113a28adcae89248b909d031f8e)

test/rgw: add multisite test_version_suspended_incremental_sync

Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit 81fa354c1dadf4f957592f20fe9e02acd70671e0)

Conflicts:
src/test/rgw/rgw_multi/tests.py

rgw: fetch_remote_obj filters out olh attrs

each zone manages its own olh log, so we don't want any of the source
zone's olh attributes. in one case (sync of an object overwrite in a
versioning-suspended bucket), these olh attributes were being written
directly to the head object and leading to link_olh errors due to the
olh_tag mismatch

Fixes: http://tracker.ceph.com/issues/37792
Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit a4195fc55fb76f0a2576d5ff0b2c99ad12fd7a49)

Merge pull request #29020 from smithfarm/wip-39047-mimic

mimic: rgw: update resharding documentation

Reviewed-by: J. Eric Ivancich <ivancich@redhat.com>

Merge pull request #29014 from smithfarm/wip-37891-mimic

mimic: doc: wrong value of usage log default in logging section

Reviewed-by: Casey Bodley <cbodley@redhat.com>

rgw: fix race b/w bucket reshard and ops waiting on reshard completion

A previous commit (f84f70d4) added functionality to clean up old
bucket instances and bucket shards once sharding completed
successfully. However the existing code that allowed a bucket
operation to wait until resharding completed was relying on a field in
the old bucket instance to know the bucket instance id of the new
bucket instance. This created a race condition as to whether the
clean-up or read of the bucket instance id would occur first.

This solution rereads the bucket entry point object when resharding
completes to determine the bucket instance id of the new bucket
thereby avoiding the race.

Signed-off-by: J. Eric Ivancich <ivancich@redhat.com>
(cherry picked from commit 26b793680805721cac86f64cdd234b273bf33be4)

Conflicts:
        src/rgw/rgw_rados.cc: equivalent changes made to rgw_reshard.cc

rgw: correct comment describing function try_refresh_bucket_info

Signed-off-by: J. Eric Ivancich <ivancich@redhat.com>
(cherry picked from commit d0894e6f6c062907796d10b524088c2abf7e4252)

rgw: fix log messages using "proceeding" to "processing"

Signed-off-by: J. Eric Ivancich <ivancich@redhat.com>
(cherry picked from commit 1c2aa902cb2bec690387ace8eb6b8afb5678373f)

Merge pull request #28206 from pdvian/wip-39475-mimic

mimic: common/util: handle long lines in /proc/cpuinfo

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #28217 from smithfarm/wip-40005-mimic

mimic: build/ops: do_cmake.sh: source not found

Reviewed-by: Lenz Grimmer <lgrimmer@suse.com>
Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>

Merge pull request #28741 from smithfarm/wip-40547-mimic

mimic: common: Keyrings created by ceph auth get are not suitable for ceph auth import

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #28894 from ifed01/wip-ifed-fix-no-compress-mimic

mimic: os/bluestore: load OSD all compression settings unconditionally.

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #28967 from liewegas/wip-set-aio-write-max-mimic

mimic: os/bluestore: fix >2GB bluefs writes

Reviewed-by: Igor Fedotov <ifedotov@suse.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge pull request #28970 from neha-ojha/wip-40720-mimic

mimic: os/bluestore: default to bitmap allocator for bluestore/bluefs

Reviewed-by: Igor Fedotov <ifedotov@suse.com>
Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #28995 from smithfarm/wip-38751-mimic

mimic: core: should report EINVAL in ErasureCode::parse() if m<=0

Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge pull request #28220 from pdvian/wip-39513-mimic

mimic: osd: take heartbeat_lock when calling heartbeat()

Reviewed-by: David Zafman <dzafman@redhat.com>

Merge pull request #26499 from k0ste/mimic_backports2

mimic: mgr/balancer: restrict automatic balancing to specific weekdays

Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>

Merge pull request #27986 from pdvian/wip-39311-mimic

mimic: crushtool crash on Fedora 28 and newer

Reviewed-by: Brad Hubbard <bhubbard@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #28089 from dzafman/wip-39720

mimic: osd: Include dups in copy_after() and copy_up_to()

Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge pull request #28097 from pdvian/wip-39374-mimic

mimic: osd/OSD.cc: make osd bench description consistent with parameters

Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge pull request #28142 from pdvian/wip-39422-mimic

mimic: osd/OSDMap: Replace get_out_osds with get_out_existing_osds

Reviewed-by: Brad Hubbard <bhubbard@redhat.com>

Merge pull request #28165 from pdvian/wip-39458-mimic

mimic: mgr/prometheus: replace whitespaces in metrics' names

Reviewed-by: Jan Fajerski <jfajerski@suse.com>

Merge pull request #28201 from dzafman/wip-38341

mimic: osd: pg stuck in backfill_wait with plenty of disk space

Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge pull request #29074 from ceph/backport-mimic-26397

mimic: ceph-volume: update testing playbook 'deploy.yml'

Reviewed-by: Alfredo Deza <adeza@redhat.com>

Merge pull request #27949 from smithfarm/wip-37613-mimic

mimic: rpm: missing dependency on python34-ceph-argparse from python34-cephfs (and others?)

Reviewed-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #28090 from pdvian/wip-39233-mimic

mimic: cephfs: mount.ceph.c: do not pass nofail to the kernel

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #28186 from pdvian/wip-39472-mimic

mimic: mds: fix SnapRealm::resolve_snapname for long name

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Yan, Zheng <zyan@redhat.com>

Merge pull request #28274 from pdvian/wip-39669-mimic

mimic: mds: output lock state in format dump

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #28337 from pdvian/wip-39679-mimic

mimic: cephfs: pybind: added lseek()

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #28381 from pdvian/wip-39689-mimic

mimic: mds: check dir fragment to split dir if mkdir makes it oversized.

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #28501 from smithfarm/wip-40168-mimic

mimic: cephfs: client: ceph.dir.rctime xattr value incorrectly prefixes 09 to the nanoseconds component

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: David Disseldorp <ddiss@suse.de>

Merge pull request #28579 from ukernel/mimic-40342

mimic: mds: fix corner case of replaying open sessions

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #28650 from pdvian/wip-40042-mimic

mimic: mds: avoid sending too many osd requests at once after mds restarts

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Yan, Zheng <zyan@redhat.com>

Merge pull request #28796 from shenhang/mimic

mimic: mds: destroy reconnect msg when it is from non-existent session to avoid memory leak

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Yan, Zheng <zyan@redhat.com>

ceph-volume: fix deploy playbook for ansible~=2.6,<2.7

When testing the lumionus and mimic branch the import_role
ansible feature does not work as expected. Variables created by one
role is not available to the next role called when using
import_role. To fix this, we need to use the roles keyword instead.

This commit is only applies to luminous and mimic and should
only be applied there.

Signed-off-by: Andrew Schoen <aschoen@redhat.com>
(cherry picked from commit b619988fbfafa7568e0ec51cfb56336a72460d44)

Merge pull request #28887 from pdvian/wip-40262-mimic

mimic: rgw: rgw_file: all directories are virtual with respect to contents

Reviewed-by: Matt Benjamin <mbenjami@redhat.com>

Merge pull request #29018 from smithfarm/wip-38776-mimic

mimic: rgw: support delimiter longer then one symbol

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #28073 from pdvian/wip-39229-mimic

mimic: rgw_file: save etag and acl info in setattr

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #28757 from smithfarm/wip-38747-mimic

mimic: rgw: non-existent mdlog failures logged at level 0

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #28310 from pdvian/wip-39674-mimic

mimic: qa/workunits/rbd: wait for rbd-nbd unmap to complete

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #28815 from trociny/wip-40593-mimic

mimic: rbd-mirror: handle duplicates in image sync throttler queue

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #28852 from jdurgin/wip-18096-mimic

mimic: osd: report omap/data/metadata usage

Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge pull request #27971 from smithfarm/wip-39593-mimic

mimic: build/ops: Cython 0.29 removed support for subinterpreters: raises ImportError: Interpreter change detected ...

Reviewed-by: Tim Serong <tserong@suse.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #28208 from votdev/fix_vstart_runner

mimic: qa: Fix issues in vstart runner

Reviewed-by: Rishabh Dave <ridave@redhat.com>
Reviewed-by: Nathan Cutler <ncutler@suse.com>

Merge pull request #28954 from bk201/wip-40707

mimic: mgr/dashboard: Fix run-frontend-e2e-tests.sh

Reviewed-by: Lenz Grimmer <lgrimmer@suse.com>
Reviewed-by: Laura Paduano <lpaduano@suse.com>

ceph-volume: update deploy.yml

update deploy.yml accordingly with recent changes in ceph-ansible.

Fixes: ceph/ceph-ansible#3602
Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
(cherry picked from commit da13da67bc1e8ba062c61a716eea485ae462db48)

rgw: minor code clean-up

Adding some const correctness and reformatting.

Signed-off-by: J. Eric Ivancich <ivancich@redhat.com>
(cherry picked from commit fb862909202c12c4bd908a0efe140c0f0ef0bd5a)

rgw: allow multipart upload abort to proceed

Cleaning up after aborting unfinished multipart uploads is a
best-effort process. Due to the history of the multipart upload, not
all components may be found and the process needs to be robust to
that.

Presently, if a component is not found after the final call to
list_bucket_multiparts, the error propogates to the caller, but not if
the component is not found in an earlier call. This change does not
propogate that error up ever.

Additional logging is also added to multipart-related functions.

Signed-off-by: J. Eric Ivancich <ivancich@redhat.com>
(cherry picked from commit 51aff62ac7cbb4653e0de667dccfa46795ffd8cb)

rgw: always generate after delimiter char to skip directory

the following code also need after_delim_s to skip directory after list
1000 objects

Signed-off-by: Tianshan Qu <tianshan@xsky.com>
(cherry picked from commit 8c6678387b00c862d71cffd6fed6ca6c236e3fb7)

rgw: fix list bucket with start maker and delimiter '/' will miss next object with char '0'

'/' + 1 = '0', and the start marker will be skipped, so it will miss the
'0' object.

Fixes: http://tracker.ceph.com/issues/39989
Signed-off-by: Tianshan Qu <tianshan@xsky.com>
(cherry picked from commit 43b5264c7d61ae3be56d68f131b544edee307a46)

Conflicts:
src/rgw/rgw_rados.cc
- mimic does not have "unsigned" in the line "unsigned char e = delim.back();"

rgw: data sync checks empty next_marker for datalog

RGWReadRemoteDataLogShardCR tracks the marker and next_marker
separately, because next_marker will be empty when it reaches the end

this allows RGWDataSyncShardCR to avoid clearing its sync_marker and
restarting datalog listing from the beginning

Fixes: http://tracker.ceph.com/issues/39033
Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit 2ff63778342282ab5c676529f6e7d90f30e6da1a)

rgw: fix up indentation in incremental data sync

Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit 666bfe3a909ec83a3067398e7896527f99283e0a)

Conflicts:
src/rgw/rgw_data_sync.cc

rgw: updates to resharding documentation

Some minor updates to the resharding documentation with the aim of
clarifying the language and using terms consistently.

Signed-off-by: J. Eric Ivancich <ivancich@redhat.com>
(cherry picked from commit 54b6b650eeeeac0e754bd5860cd53109ba7865bd)

Conflicts:
doc/radosgw/dynamicresharding.rst
- refrain from backporting changes to sections that aren't present in mimic

rgw: prefix-delimiter listing: support >1 character delimiter

Fix prefix and CommonPrefix extraction logic in
RGWRados::Bucket::List::list_objects_ordered so as to permit
arbitrary-length string delimiters.

Fixes: https://tracker.ceph.com/issues/24821
Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
(cherry picked from commit e3c1ea244234aace7368d5a5ee95af2f6a529b00)

doc: rgw: fix the default value of usage log setting

Fixes: http://tracker.ceph.com/issues/37856
Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>
(cherry picked from commit b21c96ec69ec65f359473a0f193c716925086828)

Merge pull request #28123 from dillaman/wip-39461-mimic

mimic: rbd-mirror: clear out bufferlist prior to listing mirror images

Reviewed-by: Mykola Golub <mgolub@mirantis.com>

Merge pull request #28133 from dillaman/wip-38441-mimic

mimic: librbd: ensure compare-and-write doesn't skip compare after copyup

Reviewed-by: Mykola Golub <mgolub@mirantis.com>

Merge pull request #28770 from dillaman/wip-40573-mimic

mimic: librbd: properly track in-flight flush requests

Reviewed-by: Mykola Golub <mgolub@mirantis.com>

Merge pull request #28814 from trociny/wip-40461-mimic

mimic: journal: properly advance read offset after skipping invalid range

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

qa/standalone/erasure-code/test-erasure-code: adjust test to avoid m=0

_DD is k=2 m=0, which we don't allow.  Switch it to cDD.

I confess I don't fully understand why this was _DD to begin with, but
I'm pretty sure mapping is there to control the order of results so that
it can be mapped to the CRUSH rule output sanely, and the coding portion
is not relevant to the test.

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 52d5797c3d3d74dd88664cfe85e5d551e7b334a6)

erasure-code: ensure m >= 1

Fixes: http://tracker.ceph.com/issues/38682
Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit ab0a6528266ab30d442f351de97f5475399ff278)

Conflicts:
src/erasure-code/clay/ErasureCodeClay.cc
- file does not exist in mimic

mon/OSDMonitor: set ec min_size to k + min(1, m - 1)

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 7f49be2104f0a5df894a53e200c431c527781f9c)

deb,rpm,do_cmake: switch to cmake3

Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit af2c91ace6c1082925f118d145937ae55fa780d4)

cmake: remove cython 0.29's subinterpreter check during install

Commit 3bde34af8a removed cython 0.29's subinterpreter check when
building the various python modules during `make`, but unforunately
they're *rebuilt* during `make install`, with the rebuild overwriting
the original build.  The original fix was of course missing from the
install stage...

Fixes: https://tracker.ceph.com/issues/38788
Signed-off-by: Tim Serong <tserong@suse.com>
(cherry picked from commit 108462e3fb0e814c98ab32f02b79cf3ea1247692)

cmake: remove cython 0.29's subinterpreter check

cython 0.29 introduced a check which prevents multiple python
subinterpreters from loading the same module:

https://github.com/cython/cython/commit/7e27c7c

Unfortunately, this completely breaks ceph-mgr.  Until we can
figure out a better long term solution, this commit removes
cython's subinterpreter check, via some careful abuse of the
C preprocessor.

This works because when cython is invoked, it first generates
some C code, then compiles it.  We know it's going to generate
C code including:

  int __Pyx_check_single_interpreter(void) { ... }

and:

  if (__Pyx_check_single_interpreter())
      return NULL;

So, we can do the following:

  #define void0 dead_function(void)
  #define __Pyx_check_single_interpreter(ARG)=ARG ## 0

This replaces the call to __Pyx_check_single_interpreter()
with a literal 0, removing the subinterpreter check.

The void0 dead_function(void) thing is necessary because
the __Pyx_check_single_interpreter() macro also clobbers
that function definition, so we need to make sure it's
replaced with something that works as a function definition.

Fixes: https://tracker.ceph.com/issues/37472
Signed-off-by: Tim Serong <tserong@suse.com>
(cherry picked from commit 3bde34af8a84d76744887b2cf10ee4ef36e3925b)

Conflicts:
cmake/modules/Distutils.cmake
- mimic doesn't have the immediately preceding ccache stuff

Merge pull request #28161 from pdvian/wip-39426-mimic

mimic: mgr: deadlock

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>

Merge pull request #28326 from cbodley/wip-40088

mimic: common: parse ISO 8601 datetime format

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #28399 from xiexingguo/wip-fudege-for-m

mimic: mgr/balancer: fix fudge

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #28645 from ifed01/wip-ifed-fix-40080-mimic

mimic: os/bluestore: fix duplicate allocations in bmap allocator

Reviewed-by: Sage Weil <sage@redhat.com>

os/bluestore: default to bitmap allocator for bluestore/bluefs

Signed-off-by: Igor Fedotov <ifedotov@suse.com>
(cherry picked from commit 231b7dd9c5dc1d22e93a8f81d07e772a6615f9f0)

ceph_test_objectstore: add very_large_write test

Write 2GB to verify the blockdevice aio splitting.

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 1974f011137976edf277833d68bdcc4ff22779b0)

- fixed set_val/get_val usage

os/bluestore: fix aio pwritev lost data problem.

On Linux, write() (and similar system calls) will transfer at most
0x7ffff000 (2,147,479,552) bytes, it will cap data if aio pwritev
more than 0x7ffff000, so we have the split the data to more aio submit.

Signed-off-by: kungf <yang.wang@easystack.cn>
(cherry picked from commit 4d33114a40d5ae0d541c36175977ca22789a3b88)

# Conflicts:
# src/os/bluestore/KernelDevice.cc
- fd_direct vs choose_fd()

Merge pull request #28948 from ceph/backport-mimic-28866

mimic: ceph-volume broken assertion errors after pytest changes

Reviewed-by: Alfredo Deza <adeza@redhat.com>

Merge pull request #28947 from ceph/backport-mimic-28836

mimic: ceph-volume tests add a sleep in tox for slow OSDs after booting

Reviewed-by: Alfredo Deza <adeza@redhat.com>

mgr/dashboard: Fix run-frontend-e2e-tests.sh

e2e tests were always returning positive result, even when they failed.

Fixed problem with jq, where it was failing when it was not receiving a string.

Fixes: http://tracker.ceph.com/issues/40707
Signed-off-by: Kiefer Chang <kiefer.chang@suse.com>
Signed-off-by: Tiago Melo <tmelo@suse.com>
(cherry picked from commit ad293e1961065ea4b92244c8f059dff0ef73a016)

Conflicts:
src/pybind/mgr/dashboard/run-frontend-e2e-tests.sh

ceph-volume api.lvm catch IndexError when parsing dmmapper output

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit bfff70d07a1ccf7fb11a96dd7c63335cfc9b5e26)

ceph-volume tests update to use error.value instead of str(error)

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 484be7ffb1e0865ff840090a73036567a1f4f5d2)

ceph-volume tests add a sleep in tox for slow OSDs after booting

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit b398f99f810d97c94af6d943aed4c11a5fd69c19)

Merge pull request #28930 from ceph/backport-mimic-26739

mimic: ceph-volume: replace testinfra command with py.test

Reviewed-by: Alfredo Deza

Merge pull request #28209 from votdev/add_tenant_to_bucket

mimic: rgw: Return tenant field in bucket_stats function

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #28249 from pdvian/wip-39532-mimic

mimic: rgw: resharding of a versioned bucket causes a bucket stats discrepancy

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #28348 from smithfarm/wip-39179-mimic

mimic: rgw: remove_olh_pending_entries() does not limit the number of xattrs to remove

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #28409 from cbodley/wip-40147

mimic: rgw: fix bucket may redundantly list keys after BI_PREFIX_CHAR

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #28422 from pdvian/wip-39697-mimic

mimic: rgw: return ERR_NO_SUCH_BUCKET early while evaluating bucket policy

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #28483 from pdvian/wip-39734-mimic

mimic: rgw: multisite: mismatch of bucket creation times from List Buckets

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #28668 from smithfarm/wip-40348-mimic

mimic: tests: use curl in wait_for_radosgw() in util/rgw.py

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #28711 from pdvian/wip-40133-mimic

mimic: rgw: TempURL should not allow PUTs with the X-Object-Manifest.

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #28716 from smithfarm/wip-40507-mimic

mimic: rgw: conditionally allow builtin users with non-unique email addresses

Reviewed-by: Casey Bodley <cbodley@redhat.com>

ceph-volume tests.simple replace testinfra command for py.test

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit e90ebc3264004cab3286895c843db43f20a4840b)

ceph-volume tests.lvm replace testinfra command for py.test

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit c4597fc63195c654b960f747f9d50a558fa719c0)

ceph-volume tests.batch replace testinfra command for py.test

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit dc7501af69ccbdabb5e6a77a91696454855201e9)

rbd-mirror: handle duplicates in image sync throttler queue

Fixes: http://tracker.ceph.com/issues/40519
Signed-off-by: Mykola Golub <mgolub@suse.com>
(cherry picked from commit 2f35ab70a59fc22c02c98e5aa80a1e6cde29a226)

Conflicts:
    src/test/rbd_mirror/test_mock_ImageSyncThrottler.cc:
       no g_ceph_context arg for MockImageSyncThrottler constructor

os/bluestore: load OSD all compression settings unconditionally.

Inititally we don't load them if OSD compression mode is set to none.
But we might need them when pool's compression is enabled but some
settings aren't set.

Fixes: http://tracker.ceph.com/issues/40480
Signed-off-by: Igor Fedotov <ifedotov@suse.com>
(cherry picked from commit 187509ebf6bf913b65f0171013bf7c20279dadb4)