Merge pull request #28648 from pdvian/wip-40107-nautilus

nautilus: rgw: Librgw doesn't GC deleted object correctly

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #28712 from pdvian/wip-40134-nautilus

nautilus: rgw: TempURL should not allow PUTs with the X-Object-Manifest.

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #28713 from smithfarm/wip-40505-nautilus

nautilus: rgw: fix miss get ret in STSService::storeARN

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #28714 from smithfarm/wip-40515-nautilus

nautilus: rgw: multisite: DELETE Bucket CORS is not forwarded to master zone

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #28715 from smithfarm/wip-40508-nautilus

nautilus: rgw: conditionally allow builtin users with non-unique email addresses

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #28728 from joke-lee/nautilus

nautilus: rgw, Policy should be url_decode when assume_role

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #28729 from joke-lee/nautilus-backport-28172

nautilus: rgw: fix rgw crash and set correct error code

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #28735 from smithfarm/wip-40125-nautilus

nautilus: rgw: hadoop-s3a suite failing with more ansible errors

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #28736 from smithfarm/wip-40129-nautilus

nautilus: rgw: Swift interface: server side copy fails if object name contains "?"

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #28737 from smithfarm/wip-40137-nautilus

nautilus: rgw: the Multi-Object Delete operation of S3 API wrongly handles the Code response element

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #28739 from smithfarm/wip-40142-nautilus

nautilus: rgw: multisite: 'radosgw-admin bucket sync status' should call syncs_from(source.name) instead of id

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #28751 from pdvian/wip-40150-nautilus

nautilus: rgw: add 'GET /admin/realm?list' api to list realms

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #28854 from pdvian/wip-40226-nautilus

nautilus: rgw_file: include tenant when hashing bucket names

Reviewed-by: Matt Benjamin <mbenjami@redhat.com>

Merge pull request #28886 from pdvian/wip-40263-nautilus

nautilus: rgw_file: all directories are virtual with respect to contents

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #29154 from smithfarm/wip-40591-nautilus

nautilus: rgw: deleting bucket can fail when it contains unfinished multipart uploads

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #29163 from smithfarm/wip-40851-nautilus

nautilus: rgw: multisite: radosgw-admin commands should not modify metadata on a non-master zone

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #29205 from smithfarm/wip-40760-nautilus

nautilus: rgw: Save an unnecessary copy of RGWEnv

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #29265 from smithfarm/wip-40124-nautilus

nautilus: rgw: hadoop-s3a suite failing with more ansible errors

Merge pull request #29286 from smithfarm/wip-40355-nautilus

nautilus: rgw: provide admin-friendly reshard status output

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #29287 from smithfarm/wip-40358-nautilus

nautilus: rgw: set null version object issues

Merge pull request #29310 from smithfarm/wip-40349-nautilus

nautilus: rgw/OutputDataSocket: append_output(buffer::list&) says it will (but does not) discard output at data_max_backlog

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #29311 from smithfarm/wip-40352-nautilus

nautilus: rgw: multisite: RGWListBucketIndexesCR for data full sync needs pagination

Merge pull request #29313 from smithfarm/wip-40512-nautilus

nautilus: rgw: Put LC doesn't clear existing lifecycle

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #29325 from smithfarm/wip-40450-nautilus

nautilus: core: s3tests-test-readwrite failed in rados run (Connection refused)

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #28549 from smithfarm/wip-39749-nautilus

nautilus: rgw: multisite: fix --bypass-gc flag for 'radosgw-admin bucket rm'

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #29040 from dcasier/wip-40100-nautilus

nautilus: ceph-volume: lvm.activate: Return an error if WAL/DB devices absent

Reviewed-by: Jan Fajerski <jfajerski@suse.com>
Reviewed-by: Alfredo Deza <adeza@redhat.com>

qa/rgw: update default port in perl workunits

Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit 1643879218638915073d5310b859a94d10ffeac6)

qa/rgw: extra s3tests tasks use rgw endpoint configuration

Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit 09e992ff01b4ce286540e1230a30df67103f5968)

rgw_lc: use a new bl while encoding RGW_ATTR_LC

The current code seems to append the value to the bucket attr which isn't what
we want

Fixes: https://tracker.ceph.com/issues/39654
Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>
(cherry picked from commit e23af175aff1d80a7857406f7c17f5a609ad7e5e)

rgw/multisite:RGWListBucketIndexesCR for data full sync pagination

RGWListBucketIndexesCR uses the 'marker' from remote zone's RGWOp_Metadata_List
admin api to list all keys for data sync.

Fixes http://tracker.ceph.com/issues/39551

Signed-off-by: Shilpa Jagannath <smanjara@redhat.com>
(cherry picked from commit 708ba6dcc90525c8c4316760e01ee9b1e6da21b7)

rgw/OutputDataSocket: actually discard data on full buffer

A dout message in OutputDataSocket::append_output() states that
data will be dropped when appending would cause data_max_backlog
to be exceeded--but the method appends it anyway.

Log output discards at level 0, as messages will be lost.  Suppress
repeated warnings mod 100.  Switch to vector.

Fixes: http://tracker.ceph.com/issues/40178
Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
(cherry picked from commit c806b825dae649829de8847d36cb21ffd2bbee8e)

Merge pull request #29260 from jan--f/wip-40921-nautilus

nautilus: ceph-volume: missing string substitution when reporting mounts

Reviewed-by: Alfredo Deza <adeza@redhat.com>

rgw: set null version object acl issues

1.set null version object acl will create empty index
RGWRados::set_attrs did not clear instance, so index prepare, complete got instance=null,
which lead to empty index 1000_<obj>_i_null.
there is no harm to create empty index, but listomapkeys to find that key.

2.if object is exist with versioned key, we can set none exists null version object
order:
1) enable bucket version
2) put obj
3) disable bucket version
4) set versoned_id=null acl will succeed which should not

Fixes: http://tracker.ceph.com/issues/36763
Signed-off-by: Tianshan Qu <tianshan@xsky.com>
(cherry picked from commit ca6e5c89e8d6da498da15f0eea4bcd4d271b44e9)

rgw: provide admin friendly reshard status output

Signed-off-by: Mark Kogan <mkogan@redhat.com>
(cherry picked from commit ba4167c2f30fbb868482d48aaada6388f58de920)

rgw: update the "radosgw-admin reshard status"
command documentation with expected output examples

Fixes: http://tracker.ceph.com/issues/37615
Signed-off-by: Mark Kogan <mkogan@redhat.com>
(cherry picked from commit 8565ec0db69223ae71766ba000eecc0d2f7d14c2)

qa/rgw: add dnsmasq back to s3a-hadoop

Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit 8c8a706b6f4cb3b2a5b89aa1eb06b7e47909852d)

qa/rgw: remove ceph-ansible from s3a-hadoop suite

Fixes: http://tracker.ceph.com/issues/39706
Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit 0fc2c8ecee2b6233292b9fd1325347fd0fdf9171)

Conflicts:
qa/tasks/s3a_hadoop.py
- mimic lacks the "if hadoop_ver == 'trunk'" conditional block

qa/rgw: use default ports (80 or 443) unless overridden

Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit 3ff5f0c2406e40d1024d8152e0ac9400302a757a)

qa/rgw: rgw task can override --rgw-dns-name on the command line

the value for rgw_dns_name isn't known until a machine is assigned, so
it can't be set via 'override: conf:'. add a per-client config option
to the rgw task so it can add the endpoint's hostname and/or s3website
hostname on the radosgw command line

Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit 658e5932fb79e2d33b73363b6ce76ff299809e16)

qa/rgw: allow rgw client config to override port

Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit 921faebb723c11686bf790ca424c952a786f358a)

ceph-volume:util: Use proper param substition

Fixes: https://tracker.ceph.com/issues/25030
Signed-off-by: Shyukri Shyukriev <shshyukriev@suse.com>
(cherry picked from commit 654660cd89fe9d93087d390805651c860bf4e862)

Merge pull request #28958 from smithfarm/wip-40710-nautilus

nautilus: doc: Document more cache modes

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #28919 from LenzGr/nautilus-documentation

nautilus: doc: Improved dashboard feature overview

Reviewed-by: Tatjana Dehler <tdehler@suse.com>
Reviewed-by: Ricardo Marques <rimarques@suse.com>
Reviewed-by: Tiago Melo <tmelo@suse.com>
Reviewed-by: Patrick Nawracay <pnawracay@suse.com>
Reviewed-by: Laura Paduano <lpaduano@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>

Merge pull request #28738 from smithfarm/wip-40140-nautilus

nautilus: doc: steps to disable metadata_heap on existing rgw zones

Reviewed-by: Casey Bodley <cbodley@redhat.com>

rgw: Save an unnecessary copy of RGWEnv

fixes: https://tracker.ceph.com/issues/40183

Signed-off-by: Mark Kogan <mkogan@redhat.com>
(cherry picked from commit 0693ec241ec40b3b7c21e9182b99213024b41049)

Merge pull request #29137 from yuriw/wip-yuriw-40832-nautilus

qa/tests: added 14.2.2 to the mix

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #28768 from dzafman/wip-40265

nautilus: osd: Don't randomize deep scrubs when noscrub set

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #29050 from rhcs-dashboard/fix-40768-nautilus

nautilus: mgr/dashboard: switch ng2-toastr to ngx-toastr

Reviewed-by:  Tiago Melo <tmelo@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>
Reviewed-by: Yuri Weinstein <yweinste@redhat.com>

qa/tests: added 14.2.2 to the mix

Fixes: http://tracker.ceph.com/issues/40251
Fixes: http://tracker.ceph.com/issues/40832
Signed-off-by: Yuri Weinstein <yweinste@redhat.com>

Merge pull request #28230 from ivancich/nautilus-wip-rgw-admin-unordered

nautilus: rgw: allow radosgw-admin to list bucket w --allow-unordered

Reviewed-by: Matt Benjamin <mbenjami@redhat.com>

Merge pull request #28769 from dillaman/wip-40572-nautilus

nautilus: librbd: properly track in-flight flush requests

Reviewed-by: Mykola Golub <mgolub@mirantis.com>

Merge pull request #28816 from trociny/wip-40462-nautilus

nautilus: journal: properly advance read offset after skipping invalid range

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #28817 from trociny/wip-40594-nautilus

nautilus: rbd-mirror: handle duplicates in image sync throttler queue

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #28937 from smithfarm/wip-systemd-dep-suse-nautilus

nautilus: ceph.spec.in: Drop systemd BuildRequires in case of building for SUSE

Reviewed-by: Kefu Chai <kchai@redhat.com>

Added single check to avoid duplication. Included few more commands.

Signed-off-by: Shilpa Jagannath <smanjara@redhat.com>
(cherry picked from commit 14c3b4b3ebb6ef2fc44d24f8020beb2977c8e46e)

rgw: Fail radosgw-admin commands on non-master zone that modify metadata
but with an option to override, allowing changes only on the local zone.

Fixes: http://tracker.ceph.com/issues/39548
Signed-off-by: Shilpa Jagannath <smanjara@redhat.com>
(cherry picked from commit 79d884eb018cd2fea9ee9e763d0339b3e8626e6d)

rgw: minor code clean-up

Adding some const correctness and reformatting.

Signed-off-by: J. Eric Ivancich <ivancich@redhat.com>
(cherry picked from commit fb862909202c12c4bd908a0efe140c0f0ef0bd5a)

rgw: allow multipart upload abort to proceed

Cleaning up after aborting unfinished multipart uploads is a
best-effort process. Due to the history of the multipart upload, not
all components may be found and the process needs to be robust to
that.

Presently, if a component is not found after the final call to
list_bucket_multiparts, the error propogates to the caller, but not if
the component is not found in an earlier call. This change does not
propogate that error up ever.

Additional logging is also added to multipart-related functions.

Signed-off-by: J. Eric Ivancich <ivancich@redhat.com>
(cherry picked from commit 51aff62ac7cbb4653e0de667dccfa46795ffd8cb)

Merge PR #29079 into nautilus

* refs/pull/29079/head:
client: do not return EEXIST for mkdirs
pybind/mgr/volumes: print exceptions in purge thread
pybind/mgr/volumes: refactor trash readdir
pybind/mgr/volumes: use existing client provided recursive mkdir
pybind/mgr/volumes: cleanup fs removal
pybind/mgr/subvolumes: use bytes for paths
pybind/mgr/volumes: remove unused property
test: cleanup removing all subvolumes before removing subvolume group
mgr / volumes: wrap rmtree() call within try..except block
mgr / volumes: use negative error codes everywhere
test: add basic purge queue validation test
mgr / volumes: schedule purge job for volumes on init
mgr / volumes: purge queue for async subvolume delete
mgr / volumes: maintain connection pool for fs volumes
mgr/volumes: do not import unused module

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

14.2.2

mgr/dashboard: Switch ng2-toastr for ngx-toastr

Fixes: https://tracker.ceph.com/issues/40768
Signed-off-by: Tiago Melo <tmelo@suse.com>
(cherry picked from commit 67f5e5b18b71ccabcaa8d9e35c57630d17395b06)
Signed-off-by: Ernesto Puerta <epuertat@redhat.com>
Required manual (trivial) fixing of the following files:
Conflicts:
      src/pybind/mgr/dashboard/frontend/package.json
      src/pybind/mgr/dashboard/frontend/src/app/ceph/cluster/prometheus/alert-list/alert-list.component.spec.ts
      src/pybind/mgr/dashboard/frontend/src/app/ceph/cluster/prometheus/silence-form/silence-form.component.spec.ts
      src/pybind/mgr/dashboard/frontend/src/app/ceph/cluster/prometheus/silence-list/silence-list.component.spec.ts
      src/pybind/mgr/dashboard/frontend/src/app/ceph/pool/pool-list/pool-list.component.spec.ts
      src/pybind/mgr/dashboard/frontend/src/app/shared/services/prometheus-notification.service.spec.ts

client: do not return EEXIST for mkdirs

Behavior should be similar to `mkdir -p`.

Introduced-by: 26905ca82776e8119a347a9a2e8d65c272c7ed62
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit b049eb9bbcdaef547c3559e9dbe5a3ca272f60ec)

pybind/mgr/volumes: print exceptions in purge thread

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit ecf85023954f464c328671c897d610c8ebc8c789)

pybind/mgr/volumes: refactor trash readdir

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit e71e9187ec69ce80e9d7a2fc2c58946dd3db88e1)

pybind/mgr/volumes: use existing client provided recursive mkdir

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit e9c314abd06e2d91a58cbd681978a3abb6134952)

pybind/mgr/volumes: cleanup fs removal

In Nautilus, the simplest sequence is:

    fs fail name
    fs rm name --yes-i-really-mean-it

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit 7cc4b72305a895e99ae79e6a4d09f45c23a6257c)

pybind/mgr/subvolumes: use bytes for paths

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit 3d63cd947f55a8508f972ee249ea6a04836429ae)

pybind/mgr/volumes: remove unused property

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit f7987f5fcba322879046e86720f1db55d076d881)

test: cleanup removing all subvolumes before removing subvolume group

Test `test_subvolume_create_with_desired_mode_in_group()` creates three
subvolume in a subvolume group. During cleanup, it only removed two of
the three subvolumes. This causes failure when removing the subvolume
group since it's not empty.

Signed-off-by: Venky Shankar <vshankar@redhat.com>
(cherry picked from commit ae470a64ecf8ded8bbe3dbe02b5a470fecb7e2c3)

mgr / volumes: wrap rmtree() call within try..except block

This will be invoked by purge threads, so be sure to catch all
exceptions and return proper error codes appropriately.

Signed-off-by: Venky Shankar <vshankar@redhat.com>
(cherry picked from commit 04547c92c1424c3c17aa84c49bfbfe0ffe4ac781)

mgr / volumes: use negative error codes everywhere

cephfs python binding returns positive error code. mgr/volumes
incorrectly does error code checks assuming the error codes to
be negative.

this was not an issue till now since mgr/volumes mostly does a
`raise VolumeException()` for the most part followed by the exception
being displayed to the operator (one exception is catching cephfs
ObjectNotFound error, in which case -errno.ENOENT is returned
(and checked whereever required)).

Signed-off-by: Venky Shankar <vshankar@redhat.com>
(cherry picked from commit dd569e22e8d1b6287207fb2d1a4269d0bef20784)

test: add basic purge queue validation test

.. and since we have async subvolume deletes now, check
trash directory for emptiness in other tests.

Fixes: http://tracker.ceph.com/issues/40036
Signed-off-by: Venky Shankar <vshankar@redhat.com>
(cherry picked from commit aec1d90b3611fec19556c9e4c639cb8abf90e4b6)

mgr / volumes: schedule purge job for volumes on init

So as to kickstart leftover subvolume purges.

Signed-off-by: Venky Shankar <vshankar@redhat.com>
(cherry picked from commit 06e44be9986e7598f557d9646d7f93c9db4bc89d)

mgr / volumes: purge queue for async subvolume delete

Support asynchronous subvolume deletes by handing off the delete
operation to a dedicated set of threads. A subvolume delete operation
renames the subvolume (subdirectory) to a unique trash path entry
and signals the set of worker threads to pick up entries from the
trash directory for background removal.

This commit implements a `thread pool` strategy as a class mixin.

Signed-off-by: Venky Shankar <vshankar@redhat.com>
(cherry picked from commit 483a2141fe8c9a58bc25a544412cdf5b047ad772)

mgr / volumes: maintain connection pool for fs volumes

Right now every [sub]volume call does a connect/disconnect to the
cephfs filesystem. This is unnecessary and can be optimized by
caching the filesystem handle in a connection pool and (re)using
the handle for subsequent [sub]volume operations.

This would be useful for implementing features such as purge queue
for asynchronous subvolume deletes.

Signed-off-by: Venky Shankar <vshankar@redhat.com>
(cherry picked from commit 5c41e949af9acabd612b0644de0603e374b4b42a)

mgr/volumes: do not import unused module

cephfs is not used in this module, so drop its `import`

Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit bc13b7941dc06072d103dd50e00f8089dfacfb13)

Merge PR #28589 into nautilus

* refs/pull/28589/head:
msg/async/ProtocolV[12]: add ms_learn_addr_from_peer

Reviewed-by: Sage Weil <sage@redhat.com>

Merge PR #28646 into nautilus

* refs/pull/28646/head:
os/bluestore: BitmapAllocator::allocate: log new extents only
os/bluestore: distinguis request and results in bitmap alloc log
os/bluestore: fix duplicate allocations in bmap allocator

Reviewed-by: Sage Weil <sage@redhat.com>

Merge PR #29032 into nautilus

* refs/pull/29032/head:
qa/tasks/mgr/dashboard/test_health: fix test_full_health test
mon: use per-pool stats only when all OSDs are reporting
osd: report whether we have per-pool stats
osd/osd_types: osd_stat_t: include num_per_pool_osds member

Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge pull request #29028 from ceph/backport-nautilus-28060

nautilus: ceph-volume: use the Device.rotational property instead of sys_api

Reviewed-by: Andrew Schoen <aschoen@redhat.com>

ceph-volume: lvm.activate: Return an error if WAL/DB devices absent

Fixes: https://tracker.ceph.com/issues/40100
Signed-off-by: David Casier <david.casier@aevoo.fr>
(cherry picked from commit de439a0a236e4752e64a0cd63b7ce1ad3278eff3)

qa/tasks/mgr/dashboard/test_health: fix test_full_health test

New fields!

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit e0a66c564ee0ec72c20d891ca695d1df76f58692)

mon: use per-pool stats only when all OSDs are reporting

Previously, we would start using the per-pool stat sums as soon as *any*
OSDs were reporting.  For a legacy cluster, that meant that as soon as
one bluestore instance is updated or one new bluestore OSD is created, the
usage stats per pool would become useless.

Instead, only use the per-pool stats once *all* OSDs are reporting the new
values.  This mostly aligns with the health warning when one more bluestore
OSDs are not reporting; once they are update the warning goes away.  (That
does not factor in fielstore OSDs, though; all OSDs need to be new *and*
bluestore.)

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 5dcb6d81bbc2a4c4d0da4a33d9f6bbba5065a1ad)

# Conflicts:
# src/messages/MGetPoolStatsReply.h
- ctor arguments are all weird
# src/mon/PGMap.h
- lots of std:: everywhre
# src/osdc/Objecter.h
- std:: everywhere

osd: report whether we have per-pool stats

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 5f99db7efaa3ec929ca6125709a7838fb1ca773a)

osd/osd_types: osd_stat_t: include num_per_pool_osds member

Count OSDs that are reporting per-pool stats.

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 6a0dc2bd35448fcfe3dc67fd6fa0b3e6dbce622b)

Merge pull request #29022 from smithfarm/wip-40762-nautilus

nautilus: rgw: list bucket with start marker and delimiter will miss next object with char '0'

Reviewed-by: Matt Benjamin <mbenjami@redhat.com>

ceph-volume: skip missing interpreters when running tox tests

This will skip tests for versions of python that don't exist on the node
that's running the tests.

Signed-off-by: Andrew Schoen <aschoen@redhat.com>
(cherry picked from commit f1db0a864814d9b81192d89b6eebb6c624ddec3b)

ceph-volume: use the Device.rotational property instead of sys_api

Resolves: https://bugzilla.redhat.com/show_bug.cgi?id=1666822

Signed-off-by: Andrew Schoen <aschoen@redhat.com>
(cherry picked from commit 8ebff475dfa136c88101ef7364820fe3c5cee547)

rgw: always generate after delimiter char to skip directory

the following code also need after_delim_s to skip directory after list
1000 objects

Signed-off-by: Tianshan Qu <tianshan@xsky.com>
(cherry picked from commit 8c6678387b00c862d71cffd6fed6ca6c236e3fb7)

rgw: fix list bucket with start maker and delimiter '/' will miss next object with char '0'

'/' + 1 = '0', and the start marker will be skipped, so it will miss the
'0' object.

Fixes: http://tracker.ceph.com/issues/39989
Signed-off-by: Tianshan Qu <tianshan@xsky.com>
(cherry picked from commit 43b5264c7d61ae3be56d68f131b544edee307a46)

Conflicts:
src/rgw/rgw_rados.cc
- nautilus lacks "unsigned" in the line "unsigned char e = delim.back();"

Merge pull request #27723 from ceph/backport-nautilus-26957

nautilus: ceph-volume: look for rotational data in lsblk

Reviewed-by: Alfredo Deza <adeza@redhat.com>

doc: cover more cache modes in rados/operations/cache-tiering.rst

Fixes: http://tracker.ceph.com/issues/14153
Signed-off-by: Nathan Cutler <ncutler@suse.com>
(cherry picked from commit 31f29d796a221438ac7632a0eb81ba72b449bf8b)

ceph-volume: look for rotational data in lsblk

Also ask lsblk if a device is rotational if no information
is found in /sys/block, default to True if nothing is found.

Resolves: https://bugzilla.redhat.com/show_bug.cgi?id=1666822

Signed-off-by: Andrew Schoen <aschoen@redhat.com>
(cherry picked from commit de748f9aef894b247622ee5dc49c1fbc31097ad0)

Packaging: Drop systemd BuildRequires in case of building for SUSE

There is already pkgconfig(systemd) present, which, in SUSE's packaging,
translates to the same package (systemd.pc is shipped as part of the main
systemd package). Not explicitly mentioning 'systemd' as package name allows
the openSUSE Build Service though to find shortcuts by using the
bootstrap packages, i.e. systemd-mini (ABI/API complete).

Signed-off-by: Dominique Leuenberger <dimstar@opensuse.org>
(cherry picked from commit b48ea142de50ea7b0242e7f478578b9ee8e31b8f)

Merge pull request #28923 from ceph/backport-nautilus-28294

nautilus: tests: pass --ssh-config to pytest to resolve hosts when connecting

Reviewed-by: Alfredo Deza <adeza@redhat.com>

Merge pull request #28925 from ceph/backport-nautilus-28866

nautilus: ceph-volume broken assertion errors after pytest changes

Reviewed-by: Alfredo Deza <adeza@redhat.com>

Merge pull request #28924 from ceph/backport-nautilus-28836

nautilus: ceph-volume tests add a sleep in tox for slow OSDs after booting

ceph-volume api.lvm catch IndexError when parsing dmmapper output

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit bfff70d07a1ccf7fb11a96dd7c63335cfc9b5e26)

ceph-volume tests update to use error.value instead of str(error)

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 484be7ffb1e0865ff840090a73036567a1f4f5d2)

ceph-volume tests add a sleep in tox for slow OSDs after booting

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit b398f99f810d97c94af6d943aed4c11a5fd69c19)