ceph-volume tests create a test file for checking unicode output

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 79196354aca3b3d30c7de1e2ac8b5a3c06ee8330)

ceph-volume tests create a shell test for functional unicode

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 595e492783283359b613d199a75f3e7495bfb9ed)

ceph-volume tests verify new logging fallback and encodings in terminal

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit ee18ebc078b2d380b88f3726737184f34345e2f3)

ceph-volume terminal remove unicode stream handler

This caused problems in environments where stderr was redirected, since
stderr sets the encoding to None. Getting it back again allows
everything to work correctly, and keeps all the current unit tests
passing

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit fe66e01f6a4b19bb15695537713a629a951b28ce)

ceph-volume: instantiate the new terminal logger in main()

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit bfb8422e262dcce49c50610a3977bfe1edc2a435)

ceph-volume create a logger for the terminal

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 95b16b516c0736839fe50f12ecbeb7cd62c67335)

ceph-volume create a new tox.ini for shell-based tests

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 4612ab3ae62a3e8e1cc9e51489feab0c938296d3)

Merge pull request #30188 from smithfarm/wip-41332-mimic

mimic: build/ops: rpm: always build ceph-test package

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #30227 from smithfarm/wip-41530-mimic

mimic: doc: default values for mon_health_to_clog_* were flipped

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #30244 from smithfarm/wip-41701-mimic

mimic: build/ops: install-deps.sh: install `python*-devel` for python*rpm-macros

Reviewed-by: Kefu Chai <kchai@redhat.com>

install-deps.sh: install `python*-devel` for python*rpm-macros

in 087ea813, we installed '*rpm-macros' for the macros, so we can have
access to the latest python packaging related macros for preparing the
build dependencies.

but we could run into https://bugs.centos.org/view.php?id=16379, if
we already have an old version of python-devel installed. as the newer
version of python-rpm-macros conflicts with it.

it was a chicken-and-egg problem, as we don't know the exact name of
*rpm-macros packages. that's why we chose to install all of them. but
we have to upgrade the existing python-devel package to resolve the
conflict. but the since there is no python3-devel in RHEL7/CentOS7,
what they have is python36-devel. so we have to hardwire the
`%{python3_pkgversion}` to "36" even before we have access to this
macro, and upgrade the python36-devel package beforehand. but this
renders installing the rpm-macro package less useful -- we intend to
use the macro offered by the package to figure out "36".

as a workaround, we pretend that we know the "main" version of python3
in current RHEL/CentOS. and always install python36-devel for
python-rpm-macros. as the former requires the latter.

once all python3*-devel on all builders are upgraded, we will be safe
to install '*rpm-macros' again without installing python36-devel first.
by then, we could revert this change, or continue installing
python36-devel until the distro bumps up the "main" python version to 3.7

Fixes: https://tracker.ceph.com/issues/41603
Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit 0b2be2b6710695867197ee84e411bdf0a843a06a)

Conflicts:
install-deps.sh
- VERSION_ID in mimic, MAJOR_VERSION in master

doc: default values for mon_health_to_clog_* were flipped

On a freshly installed nautilus cluster (i.e. 14.2.2), the default values are:

mon_health_to_clog_interval = 3600
mon_health_to_clog_tick_interval = 60.000000

Fixes: https://tracker.ceph.com/issues/41403
Signed-off-by: James McClune <jmcclune@mcclunetechnologies.net>
(cherry picked from commit 820ada1e58c3713b7de8a4ce669185d26860b84b)

rpm: always build ceph-test package

Fixes: https://tracker.ceph.com/issues/41296
Signed-off-by: Nathan Cutler <ncutler@suse.com>
(cherry picked from commit b9c744301051aa637f948be074be7b53c26cb504)

Merge pull request #30128 from smithfarm/wip-41645-mimic

mimic: build/ops: install-deps.sh: Remove CR repo

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Brad Hubbard <bhubbard@redhat.com>

install-deps.sh: Remove CR repo

Remove the continuous release repos for CentOS and Virtuozzo 7 as they
should no longer be needed since http://tracker.ceph.com/issues/13997 is
no longer relevant and the newer versions of selinux packages pulled in
by the build system are causing problems for systems without CR repos
enabled.

Fixes: https://tracker.ceph.com/issues/37335
Signed-off-by: Alfredo Deza <adeza@redhat.com>
Signed-off-by: Brad Hubbard <bhubbard@redhat.com>
(cherry picked from commit f39d9b7d1f4f2b63233802211a9ed0e641446ca1)

Merge pull request #29900 from alfredodeza/mimic-rm41378

mimic ceph-volume tests set the noninteractive flag for Debian

Reviewed-by: Andrew Schoen <aschoen@redhat.com>

ceph-volume tests set the noninteractive flag for Debian, to avoid prompts in apt

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 89231c9a60a46ba0691f787efbfe5ab35f05c679)

Merge pull request #29770 from alfredodeza/mimic-bz-1738379

mimic ceph-volume: use the OSD identifier when reporting success

Reviewed-by: Jan Fajerski <jfajerski@suse.com>

ceph-volume devices.lvm zap use the identifier to report success

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit d0d5750c1ba508152c1d5111f2003a039a27145f)

ceph-volume tests check success message when zapping

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 88c7429a60fdab6222d14132bea93b9bf773c720)

Merge pull request #29661 from jan--f/wip-41246-mimic

mimic: simple functional tests drop test for lvm zap

Reviewed-by: Alfredo Deza <adeza@redhat.com>

Merge pull request #29688 from jan--f/wip-41298-mimic

mimic: batch functional idempotency test fails since message is now on stderr

Reviewed-by: Alfredo Deza <adeza@redhat.com>

Merge pull request #29618 from pdvian/wip-40943-mimic

mimic: mon/OSDMonitor.cc: better error message about min_size

Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge pull request #29699 from pdvian/wip-41086-mimic

mimic: Change default for bluestore_fsck_on_mount_deep as false

Reviewed-by: Neha Ojha <nojha@redhat.com>

common/options.cc: common/options.cc: change default value of bluestore_fsck_on_umount_deep to false

Signed-off-by: Neha Ojha <nojha@redhat.com>
(cherry picked from commit 8c86c3a5b4bdca258016019f63359f52aa8cd84b)

common/options.cc: change default value of bluestore_fsck_on_mount_deep to false

Fixes: https://tracker.ceph.com/issues/41017
Signed-off-by: Neha Ojha <nojha@redhat.com>
(cherry picked from commit 7c03f088f1004d4e26b6bdea8fa58f8455f1d6a2)

Merge pull request #29289 from smithfarm/wip-40497-mimic

mimic: doc: Object Gateway multisite document read-only argument error

Reviewed-by: Casey Bodley <cbodley@redhat.com>

ceph-volume: fix batch functional tests, idempotent test must check stderr

Fixes: https://tracker.ceph.com/issues/41295
Signed-off-by: Jan Fajerski <jfajerski@suse.com>
(cherry picked from commit 88807110f3014e7a2a39c37ae5c64bb3ab107d80)

ceph-volume: don't try to test lvm zap on simple tests

Fixes: https://tracker.ceph.com/issues/41245
Signed-off-by: Jan Fajerski <jfajerski@suse.com>
(cherry picked from commit 081109859a06f1b2ff6652dce07da4ddf6fb258e)

Merge pull request #29631 from jan--f/backport-mimic-26826

ceph-volume: more mimic backports

Merge pull request #29602 from jan--f/wip-41201-mimic

mimic: ceph-volume prints log messages to stdout

ceph-volume: make systemctl.get_running_osds resilient to garbage output

Makes systemd.systemctl.get_running_osds more resilient to output
that doesn't match what we're expecting to parse. This also handles
the case of a nonzero exit status by retuning an empty list.

Signed-off-by: Andrew Schoen <aschoen@redhat.com>
(cherry picked from commit f3707f1ad43513c2a6df17e9dce756533c3b4a1d)

ceph-volume: remove obsoleted simple unit test

This test is no longer valid because `simple scan`
without any arguments will now attempt to scan all
running OSDs.

Signed-off-by: Andrew Schoen <aschoen@redhat.com>
(cherry picked from commit b59dcd01c2dd386ca1b63eb66ed243afffb80211)

ceph-volume: tox tests should ignore the tests/functional directory

Without this the functional testinfra tests try to run when we
are intending to only run the unit tests.

Signed-off-by: Andrew Schoen <aschoen@redhat.com>
(cherry picked from commit 1881f386d8207ee8c10f0863c4a1abf18cf3c5d9)

ceph-volume: use `simple scan` and `simple activate --all` in tests

In a couple of the functional tests for simple we should run `simple
scan` without any arguments and `simple activate --all` to increase
the test coverage. We want to test pointing at a diretory and scanning
all running OSDs.

Signed-off-by: Andrew Schoen <aschoen@redhat.com>
(cherry picked from commit 87c33f149a83bb1175bfb473b226e8c46e4e2a6e)

ceph-volume: update docs and help menu for `simple scan`

Signed-off-by: Andrew Schoen <aschoen@redhat.com>
(cherry picked from commit f7943ef6007fb05d83602bc22cbacf3bb67d8625)

ceph-volume: adds unit tests for systemd.systemctl.get_running_osd_ids

Signed-off-by: Andrew Schoen <aschoen@redhat.com>
(cherry picked from commit 7cb098b6899542c875d911d28771a4c450d0bb3a)

ceph-volume: update `simple scan` to scan all running OSDs

If no argument is passed to `ceph-volume simple scan` it will
inspect any running osds and scan them if they were created by
ceph-disk.

Signed-off-by: Andrew Schoen <aschoen@redhat.com>
(cherry picked from commit 28e454ec30f867ae1229c66d263bcd608654553f)

ceph-volume: add systemd.systemctl.get_running_osd_ids

This is used to retrieve a list of osd ids from the currently
running ceph-osd systemd units.

Signed-off-by: Andrew Schoen <aschoen@redhat.com>
(cherry picked from commit 15d20e29a2f2d74caf05dfec559e30941c7a46ab)

ceph-volume tests.functional.simple add zap by osd id to xenial

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit f53f67ca3d25deb82c20d31df418a19948ba6f64)

ceph-volume tests.functional.simple add zap by osd id to centos7

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 8955ec14566c48a08bb4826256b422aa29a2c16e)

Conflicts:
src/ceph-volume/ceph_volume/tests/functional/simple/centos7/filestore/activate/test.yml
        remove with_items key

mon/OSDMonitor.cc: better error message about min_size

Signed-off-by: Neha Ojha <nojha@redhat.com>
(cherry picked from commit ce25ac0a9196f90acef82915f122c5a06277993a)

ceph-volume: never log to stdout, use stderr instead

We should never print log messages to stdout, as this should be reserved
for output of ceph-volume.

Fixes: https://tracker.ceph.com/issues/41158
Signed-off-by: Jan Fajerski <jfajerski@suse.com>
(cherry picked from commit b8d6dcbe9f803c96c0af68da54f1262e9b6a9e77)

ceph-volume: terminal: encode unicode when writing to stdout

python determins the encoding of stdout and stderr based on the LC_CTYPE
and PYTHONIOENCODING env variable, by default, python3's sys.stdout uses
'utf-8' as its encoding, so it will be able to write unicode string even
the stdout is not attached to a tty device. but when it comes to
python2, it will default to ascii if neither of these variabls is set.
so, if we are writing unicode using `_Write` in an environment where
LC_CTYPE and/or PYTHONIOENCODING are using non UTF-8 encoding, it chokes
by raising `UnicodeEncodeError` exception.

in this change, we add a wrapper around `_Write._writer` so it is able
to write unicode string in such a non-unicode-friendly environment.

for more info related the encoding of stdout and stderr, see
https://docs.python.org/3/using/cmdline.html#envvar-PYTHONIOENCODING .

Signed-off-by: Alfredo Deza <adeza@redhat.com>
Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit 77912c0c71874a23785d3bdd109ad61e4f4c0e28)

Merge pull request #29507 from jan--f/wip-41138-mimic

mimic: ceph-volume prints errors to stdout with --format json

ceph-volume: print most logging messages to stderr

Signed-off-by: Jan Fajerski <jfajerski@suse.com>
Fixes: http://tracker.ceph.com/issues/38548
(cherry picked from commit 1f8f1e95dee88b4bab2d6931e720ed83584d2887)

 Conflicts:
src/pybind/mgr/test_orchestrator/module.py
        removed

Merge pull request #29417 from jan--f/wip-41022-mimic

mimic: simple: when 'type' file is not present activate fails

Merge pull request #29255 from smithfarm/wip-39274-mimic

mimic: rgw: S3 policy evaluated incorrectly

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #29256 from smithfarm/wip-39415-mimic

mimic: rgw: multisite: period pusher gets 403 Forbidden against other zonegroups

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #29262 from smithfarm/wip-39748-mimic

mimic: rgw: Add support for --bypass-gc flag of radosgw-admin bucket rm command in RGW Multi-site

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #29267 from smithfarm/wip-40128-mimic

mimic: rgw: Swift interface: server side copy fails if object name contains '?'

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #29268 from smithfarm/wip-40136-mimic

mimic: rgw: the Multi-Object Delete operation of S3 API wrongly handles the Code response element

Reviewed-by: Matt Benjamin <mbenjami@redhat.com>
Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #29270 from smithfarm/wip-40141-mimic

mimic: rgw: multisite: 'radosgw-admin bucket sync status' should call syncs_from(source.name) instead of id

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #29273 from smithfarm/wip-40144-mimic

mimic: rgw: Multisite sync corruption for large multipart obj

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #29277 from smithfarm/wip-40225-mimic

mimic: rgw_file: include tenant when hashing bucket names

Reviewed-by: Matt Benjamin <mbenjami@redhat.com>

Merge pull request #29279 from smithfarm/wip-40351-mimic

mimic: rgw/OutputDataSocket: append_output(buffer::list&) says it will (but does not) discard output at data_max_backlog

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #29284 from smithfarm/wip-40354-mimic

mimic: rgw: multisite: RGWListBucketIndexesCR for data full sync needs pagination

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #29285 from smithfarm/wip-40357-mimic

mimic: rgw: provide admin-friendly reshard status output

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #29288 from smithfarm/wip-40360-mimic

mimic: rgw: set null version object issues

Reviewed-by: Casey Bodley <cbodley@redhat.com>

mimic: ceph-volume broken assertion errors after pytest changes

This fixes a backported test that was not fixed on a backport. Original
fix: https://github.com/ceph/ceph/pull/28948. Backport:
https://github.com/ceph/ceph/pull/29243.

Signed-off-by: Jan Fajerski <jfajerski@suse.com>

Merge pull request #29463 from jan--f/wip-41056-mimic

mimic: ceph-volume does not recognize wal/db partitions created by ceph-disk

Reviewed-by: Alfredo Deza <adeza@redhat.com>

ceph-volume: fall back to PARTTYPE if PARTLABEL is empty

In some cases ceph-disk does not populate PARTLABEL for wal and db
partitions. This commit adds the assumption that the empty string is a
valid label and falls back to identifying those disks by PARTTYPE.

Fixes: https://tracker.ceph.com/issues/40917
Signed-off-by: Jan Fajerski <jfajerski@suse.com>
(cherry picked from commit 82d2ae7ffdf62ca93c7848fb62e506db4cda893c)

ceph-volume: adjust tests for empty PARTLABEL fields

It seems like ceph-disk sometimes creates OSDs where the wal and db
partitions have no PARTLABEL set. This commit alters tests so that the
empty string is considered a valid label.

Signed-off-by: Jan Fajerski <jfajerski@suse.com>
(cherry picked from commit bb29110c3c6e081568231131c70522dee6d12bb5)

ceph-volume: refactor ceph-disk_member unittests

This adds a parameterized fixture as input to the tests. Should make
writing new tests easier since only fixture parameters need to be added.
In the future we should be able to reuse the fixtures to build other
fixtures and reuse the parametrization.

Signed-off-by: Jan Fajerski <jfajerski@suse.com>
(cherry picked from commit f42287c3416ad09c112e64756f5051c5c13aa344)

ceph-volume tests: ensure that better heuristics exist for objectstore detection

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit e48ad3ab26fdab72755bc933781db3bfc6001a30)

ceph-volume simple.activate better detect bluestore/filestore when type is not found

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit b6a449e2e3241aaa2c0eb8fa9f74b5d939c97852)

Merge pull request #29029 from liewegas/wip-mgr-crash-py3-mimic

mimic: mgr/crash: fix python3 invalid syntax problems

Reviewed-by: Nathan Cutler <ncutler@suse.com>

Merge pull request #29150 from pdvian/wip-40269-mimic

mimic: build/ops: make "patch" build dependency explicit

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #29197 from smithfarm/wip-38963-mimic

mimic: mgr: DaemonServer::handle_conf_change - broken locking

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Mykola Golub <mgolub@mirantis.com>

Merge pull request #29201 from smithfarm/wip-40268-mimic

mimic: common: data race in OutputDataSocket

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #29217 from smithfarm/wip-39692-mimic

mimic: bluestore: _txc_add_transaction error (39) Directory not empty not handled on operation 21 (op 1, counting from 0)

Reviewed-by: Igor Fedotov <ifedotov@suse.com>

Merge pull request #29243 from jan--f/mimic-c-v-extend-batch

mimic: c-v extend batch

ceph-volume: create two NVME drives for testing

Signed-off-by: Andrew Schoen <aschoen@redhat.com>
Resolves: bz#1650306
(cherry picked from commit 893b61b256bcff142dbd90b777b2c9df9e032e9d)

ceph-volume: set block_db_size to 9GB in centos7 mixed type test

This is to enable testing of the --block-db-size flag

Signed-off-by: Andrew Schoen <aschoen@redhat.com>
(cherry picked from commit eab962c6ef68c588674bdac6fc4a86c84714f300)

ceph-volume: use two NVME drives in centos7 mixed_type tests

Signed-off-by: Andrew Schoen <aschoen@redhat.com>
(cherry picked from commit 934c32acb318aac9aff3c800cdbf99bbdee04d06)

Merge pull request #29350 from jan--f/wip-40977-mimic

mimic: ceph-volume: missing string substitution when reporting mounts

Reviewed-by: Alfredo Deza <adeza@redhat.com>

ceph-volume:util: Use proper param substition

Fixes: https://tracker.ceph.com/issues/25030
Signed-off-by: Shyukri Shyukriev <shshyukriev@suse.com>
(cherry picked from commit 654660cd89fe9d93087d390805651c860bf4e862)

Merge pull request #26991 from ceph/backport-mimic-26957

mimic: ceph-volume: look for rotational data in lsblk

Reviewed-by: Alfredo Deza <adeza@redhat.com>

ceph-volume: look for rotational data in lsblk

Also ask lsblk if a device is rotational if no information
is found in /sys/block, default to True if nothing is found.

Resolves: https://bugzilla.redhat.com/show_bug.cgi?id=1666822

Signed-off-by: Andrew Schoen <aschoen@redhat.com>
(cherry picked from commit de748f9aef894b247622ee5dc49c1fbc31097ad0)

Merge pull request #28918 from smithfarm/wip-40222-mimic

mimic: mds: reset heartbeat during long-running loops in recovery

Reviewed-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Yan, Zheng <zyan@redhat.com>

Merge pull request #29187 from xiaoxichen/wip-40875-mimic

mimic: cephfs: client: bump ll_ref from int32 to uint64_t

Reviewed-by: Venky Shankar <vshankar@redhat.com>

Merge pull request #29200 from smithfarm/wip-39685-mimic

mimic: cephfs: ceph-fuse: client hang because its bad session PipeConnection to mds

Reviewed-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Yan, Zheng <zyan@redhat.com>

Merge pull request #29208 from smithfarm/wip-40317-mimic

mimic: tests: test_volume_client: print python version correctly

Reviewed-by: Venky Shankar <vshankar@redhat.com>

Merge pull request #29210 from smithfarm/wip-38099-mimic

mimic: mds: remove cache drop admin socket command

Reviewed-by: Venky Shankar <vshankar@redhat.com>

Merge pull request #29211 from smithfarm/wip-38687-mimic

mimic: kcephfs TestClientLimits.test_client_pin fails with client caps fell below min

Reviewed-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Yan, Zheng <zyan@redhat.com>

Merge pull request #27028 from mrandybu/mimic-alt

mimic: ceph_detect_init: Add support for ALT Linux

Reviewed-by: Yuri Weinstein <yweinste@redhat.com>

Merge pull request #28210 from votdev/fix_rgw_issue

mimic: mgr/dashboard: Fix various RGW issues

Reviewed-by:  Tiago Melo <tmelo@suse.com>

Merge pull request #28911 from bk201/wip-40660-mimic

mimic: mgr/dashboard: cephfs multimds graphs stack together

Reviewed-by: Ricardo Dias <rdias@suse.com>

Merge pull request #29024 from ifed01/wip-ifed-fix-len0-mimic

mimic: os/bluestore: avoid length overflow in extents returned by Stupid Allocator

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Ilya Dryomov <idryomov@redhat.com>

Merge pull request #29174 from neha-ojha/wip-40583-mimic

mimic: common/options.cc: Lower the default value of osd_deep_scrub_large_omap_object_key_threshold

Reviewed-by: David Zafman <dzafman@redhat.com>
Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #29189 from badone/wip-mimic-lazy-omap-stats-backport-tracker-38552

mimic: core: lazy omap stat collection

Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #29196 from smithfarm/wip-37749-mimic

mimic: tools: Add clear-data-digest command to objectstore tool

Reviewed-by: David Zafman <dzafman@redhat.com>

test/rgw: test truncated reads in RGWHTTPManager

Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit c669cf293f02b3c68a4c539d7accf7ddf35cb1ee)

Conflicts:
    src/test/rgw/test_http_manager.cc
- RGWHTTP::process() only takes one argument in mimic

doc: fixed --read-only argument value in multisite doc

Fixes: https://tracker.ceph.com/issues/40458
Signed-off-by: Chenjiong Deng <dengchenjiong@umcloud.com>
(cherry picked from commit c65832c1ff03e802f98cb7f4bda620e1e515264f)

rgw: set null version object acl issues

1.set null version object acl will create empty index
RGWRados::set_attrs did not clear instance, so index prepare, complete got instance=null,
which lead to empty index 1000_<obj>_i_null.
there is no harm to create empty index, but listomapkeys to find that key.

2.if object is exist with versioned key, we can set none exists null version object
order:
1) enable bucket version
2) put obj
3) disable bucket version
4) set versoned_id=null acl will succeed which should not

Fixes: http://tracker.ceph.com/issues/36763
Signed-off-by: Tianshan Qu <tianshan@xsky.com>
(cherry picked from commit ca6e5c89e8d6da498da15f0eea4bcd4d271b44e9)

rgw: provide admin friendly reshard status output

Signed-off-by: Mark Kogan <mkogan@redhat.com>
(cherry picked from commit ba4167c2f30fbb868482d48aaada6388f58de920)

rgw: update the "radosgw-admin reshard status"
command documentation with expected output examples

Fixes: http://tracker.ceph.com/issues/37615
Signed-off-by: Mark Kogan <mkogan@redhat.com>
(cherry picked from commit 8565ec0db69223ae71766ba000eecc0d2f7d14c2)

rgw/multisite:RGWListBucketIndexesCR for data full sync pagination

RGWListBucketIndexesCR uses the 'marker' from remote zone's RGWOp_Metadata_List
admin api to list all keys for data sync.

Fixes http://tracker.ceph.com/issues/39551

Signed-off-by: Shilpa Jagannath <smanjara@redhat.com>
(cherry picked from commit 708ba6dcc90525c8c4316760e01ee9b1e6da21b7)

Conflicts:
src/rgw/rgw_data_sync.cc
- trivial

Merge pull request #29016 from ivancich/mimic-abort-multipart-fix

mimic: rgw: abort multipart fix

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #29021 from smithfarm/wip-39074-mimic

mimic: rgw: multisite: data sync loops back to the start of the datalog after reaching the end

Reviewed-by: Casey Bodley <cbodley@redhat.com>