13.2.10

Merge pull request #34715 from ceph/mimic-rgw-post-obj-tagging

rgw: check for tagging element in POST Obj requests

Reviewed-By: Casey Bodley <cbodley@redhat.com>
Reviewed-By: Matt Benjamin <mbenjami@redhat.com>

rgw: check for tagging element in POST Obj requests

Check for null element when reading the tagging field from POST obj XML

Fixes: https://tracker.ceph.com/issues/44967
Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>

Merge pull request #34521 from smithfarm/wip-45057-mimic

mimic: qa/suites/rados/rest: don't pass empty dict as data arg

Reviewed-by: Brad Hubbard <bhubbard@redhat.com>

Merge pull request #34370 from smithfarm/wip-44083-mimic

mimic: qa/suites/rados/multimon/tasks/mon_clock_with_skews: disable ntpd etc

Reviewed-by: Neha Ojha <nojha@redhat.com>

13.2.9

qa/suites/rados/rest: Don't pass empty dict as data arg

Passing an empty 'args' dict as a data argument when calling
requests.get somehow confuses the transaction, causing it to fail. Pass
'None' instead.

Fixes: https://tracker.ceph.com/issues/43720
Signed-off-by: Brad Hubbard <bhubbard@redhat.com>
(cherry picked from commit e20e6fc3295da836fb5f098d46d6190ab28195d5)

Merge pull request #34504 from ceph/wip-mimic-rgw-fixes

Reviewed-By: Yuri Weinstein <yuriw@redhat.com>
Reviewed-By: Casey Bodley <cbodley@redhat.com>
Reviewed-By: Nathan Cutler <ncutler@suse.com>

Merge pull request #34458 from badone/wip-44983-mimic

mimic: selinux: Allow ceph-mgr access to httpd dir

Reviewed-by: Boris Ranto <branto@redhat.com>
Reviewed-by: Yuri Weinstein <yweinste@redhat.com>

Merge pull request #34393 from tchaikov/mimic-cbt

mimic: qa/suites: use "mimic" branch for cbt based testing

Reviewed-by: Yuri Weinstein <yweinste@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge pull request #34479 from jan--f/wip-45004-mimic

mimic: ceph-volume/batch: check lvs list before access

Reviewed-by: Yuri Weinstein <yweinste@redhat.com>

Merge pull request #34476 from jan--f/wip-45000-mimic

mimic: ceph-volume/batch: return success when all devices are filtered

Reviewed-by: Yuri Weinstein <yweinste@redhat.com>

ceph-volume/batch: check lvs list before access

Fixes: https://tracker.ceph.com/issues/44989
Signed-off-by: Jan Fajerski <jfajerski@suse.com>
(cherry picked from commit c6dcf3161b2d69b88acdda473b85c9dd2c8433e3)

rgw: reject control characters in response-header actions

S3 GetObject permits overriding response header values, but those inputs
need to be validated to insure only characters that are valid in an HTTP
header value are present.

Credit: Initial vulnerability discovery by William Bowling (@wcbowling)
Credit: Further vulnerability discovery by Robin H. Johnson <rjohnson@digitalocean.com>
Signed-off-by: Robin H. Johnson <rjohnson@digitalocean.com>

rgw: EPERM to ERR_INVALID_REQUEST

As per Robin's comments and S3 spec

Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>

rgw: reject unauthenticated response-header actions

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
Reviewed-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit d8dd5e513c0c62bbd7d3044d7e2eddcd897bd400)

ceph-volume/batch: return success when all devices are filtered

batch should only return an error if some (but not all) devices are
filtered. When only some devices are filtered the resulting osd layout
could look very different from what a user expects. If all devies are
filtered just return success.

Fixes: https://tracker.ceph.com/issues/44994
Signed-off-by: Jan Fajerski <jfajerski@suse.com>
(cherry picked from commit cb3eade3d00fdcdb34c4156f3c488bbb8747cd6f)

Merge pull request #34444 from jan--f/wip-44973-mimic

mimic: devices/simple/scan: Fix string in log statement

Reviewed-by: Yuri Weinstein <yweinste@redhat.com>

devices/simple/scan: Fix string in log statement

A multiline string enclosed by () gained a comma, making it into a
tuple, which has no format() method.

Fixes: https://tracker.ceph.com/issues/44949
Signed-off-by: Jan Fajerski <jfajerski@suse.com>
(cherry picked from commit 292ae0305db3e48a87db45bcc7ca8cb837a08ead)

selinux: Allow ceph-mgr access to httpd dir

ceph-mgr loads modules which require read access and this causes a
denial on el7.

Fixes: https://tracker.ceph.com/issues/44216
Signed-off-by: Brad Hubbard <bhubbard@redhat.com>
(cherry picked from commit 35a7fc8249337c3c59f0c561632abf578f5d20fc)

Merge pull request #34221 from ceph/wip-yuriw-clients-upgrades-mimic-octopus

qa/tests: initial check in for client-upgrade-mimic-octopus

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #34379 from cbodley/wip-44898

qa/swift: fix py3 backport regression

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Kyr Shatskyy <kyrylo.shatskyy@suse.com>

qa/suites: use "mimic" branch for cbt based testing

cbt is using features like the f string introduced by python3.6, so we
need either stop testing on ubuntu 16.04, which came with python3.5, or
specify the a cbt branch which support python3.5. to address this issue
once and for all, it's simpler just pin the branch to "mimic" when
performing cbt based tests.

this change is not cherry-picked from master, as in master, we only
tests on distros which offers python3.6 and up.

Signed-off-by: Kefu Chai <kchai@redhat.com>

qa/tests: initial check in for client-upgrade-mimic-octopus

Signed-off-by: Yuri Weinstein <yweinste@redhat.com>

qa/swift: fix py3 backport regression

this applies only to the mimic branch because it corrects a mimic
backport

Fixes: https://tracker.ceph.com/issues/44898
Signed-off-by: Casey Bodley <cbodley@redhat.com>

qa/suites/rados/multimon/tasks/mon_clock_with_skews: whitelist MOST_DOWN

The skewed clock makes some mons miss elections.

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 08b6a2bc008c006ff815dcb85bfb24b73072c7ab)

qa/suites/rados/multimon/tasks/mon_clock_with_skews: disable ntpd etc

Fixes: https://tracker.ceph.com/issues/43889
Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 9f2a854b175f156d4ab7fba955aff515052c9d93)

qa/suites/rados/multimon: skew clocks 2s (< paxos lease)

If the leader is the one with the accurate clock, it can still
form quorum, but if the leader has the skewed clock, all other mons appear
skewed from its perspective and no quorum is formed.  This leads to
intermittent failures, depending on the non-deterministic teuthology
deployment order and how the mon IPs sort.

Fix by reducing the skew.  This is enough skew to trigger a warning, but
not enough that it will break quorum.  This ensures that the parts of the
teuthology test that issue random mon commands won't fail (e.g., 'ceph osd
dump').

Fixes: http://tracker.ceph.com/issues/40112
Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit fd00136eb3b181ca34937fc7ea99427338ae6069)

qa/suites/rados/multimon: create_rbd_pool: false

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 8d137b9345afd7e77f788e7142ff28a7f422c4f9)

qa/suites/rados/multimon: no osds when skewing clock

Sometimes the clock skew prevents the mon quorum from making progress and
processing the osd boot messages.

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 1991faaafe6e450896377bb69c04fbce84b1f989)

Merge pull request #34193 from tchaikov/mimic-cbt-py3

mimic: qa/tasks/cbt: install python3 deps

Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Yuri Weinstein <yweinste@redhat.com>

qa/tasks/cbt: install python3 deps

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 299375852a4b3210cf5b95d3559d69080f14339f)

Conflict: qa/tasks/cbt.py: change python3-yaml to python36-PyYAML,
  as in EL7, python3-* packages are still named python36-*

Merge pull request #33327 from smithfarm/wip-43257-mimic

mimic: common/config: update values when they are removed via mon

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge pull request #31875 from smithfarm/wip-42258-mimic

mimic: doc, qa: remove invalid option mon_pg_warn_max_per_osd

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #32911 from smithfarm/wip-43502-mimic

mimic: mount.ceph: give a hint message when no mds is up or cluster is laggy

Reviewed-by: Ilya Dryomov <idryomov@redhat.com>
Reviewed-by: Jeff Layton <jlayton@redhat.com>

Merge pull request #33999 from shyukri/wip-42323-mimic

mimic: qa: get rid of iterkeys for py3 compatibility

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #33598 from trociny/wip-44262-mimic

mimic: rbd-mirror: improve detection of blacklisted state

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #33966 from smithfarm/wip-44465-mimic

mimic: rgw: get barbican secret key request maybe return error code

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #33986 from toabctl/mimic-testtools

mimic: ceph-disk,ceph-detect-init: Pin testtools test requirement for old py versions

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Nathan Cutler <ncutler@suse.com>
Reviewed-by: Laura Paduano <lpaduano@suse.com>

qa: get rid of iterkeys for py3 compatibility

Fixes: https://tracker.ceph.com/issues/42287
Signed-off-by: Kyr Shatskyy <kyrylo.shatskyy@suse.com>
(cherry picked from commit 5f95b532aa26249cecea4586166399a0bf10ad47)

Ensured all iterkeys -> keys under qa/tasks
Removed ceph_crash / install_packages methods by PR comments

ceph-detect-init & ceph-disk: Pin testtools test requirement

When running make-check on python 3.4, the test requirements
installation fails with:

ERROR: testtools requires Python '>=2.7, !=3.0.*, !=3.1.*, !=3.2.*, \
  !=3.3.*, !=3.4.*' but the running Python is 3.4.6

This happens because there is a new testtols release on pypi from Mar
14, 2020. Pinning to an older version fixes this.

This is not needed for Master, Octopus and Nautilus  given that python
3.4 is already EOL[1] before Nautilus got released.

[1] https://www.python.org/downloads/release/python-3410/

Signed-off-by: Thomas Bechtold <tbechtold@suse.com>

Merge pull request #33594 from neha-ojha/wip-44022-mimic

mimic: osd/PeeringState.h: ignore MLogRec in Peering/GetInfo

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>

Merge pull request #33748 from cbodley/wip-44445

mimic: rgw: making implicit_tenants backwards compatible

Reviewed-by: Casey Bodley <cbodley@redhat.com>

get barbican secret key request return error code

Signed-off-by: Richard Bai(白学余) <baixueyu@inspur.com>
(cherry picked from commit fbe2be57474df43996dd45bf04d1a1137a02c729)

Conflicts:
src/rgw/rgw_http_client.cc
- mimic uses Mutex::Locker instead of std::lock_guard

rgw: making implicit_tenants backwards compatible.

In jewel, "rgw keystone implicit tenants" only applied to swift. As of
luminous), this option applies to s3 also.
Sites that used this feature with jewel now have outstanding data that
depends on the old behavior.

The fix here is to expand "rgw keystone implicit tenants" so that it
can be set to any of "none", "all", "s3" or "swift" (also 0=false=none,
1=true=all).  When set to "s3" or "swift", the actual id lookup
is also partitioned.

Formerly "rgw keystone implicit tenants" was a legacy opt.
This change converts it to the new style of option,
including support for dynamically changing it.

Fixes: http://tracker.ceph.com/issues/24348
Signed-off-by: Marcus Watts <mwatts@redhat.com>
(cherry picked from commit a28a38f6e91da3abe59c34fad0e059eeaf29a65f)
(cherry picked from commit d2bdea8e7bc8eae64e4b72f581e424a9a6871aed)

Conflicts:
  - forward port from luminous was easier than backport from master

Merge pull request #31384 from less-is-morr/wip-42159-mimic

mimic: cephfs: osdc/objecter: Fix last_sent in scientific format and add age to ops

Reviewed-by: Milind Changire <mchangir@redhat.com>

Merge pull request #31379 from smithfarm/wip-42143-mimic

mimic: mds: split the dir if the op makes it oversized, because some ops maybe in flight

Reviewed-by: Milind Changire <mchangir@redhat.com>

Merge pull request #31284 from ukernel/mimic-42156

mimic: mds: fix infinite loop in Locker::file_update_finish

Reviewed-by: Milind Changire <mchangir@redhat.com>

Merge pull request #31283 from ukernel/mimic-41114

mimic: client: more precise CEPH_CLIENT_CAPS_PENDING_CAPSNAP

Reviewed-by: Milind Changire <mchangir@redhat.com>

Merge pull request #31282 from ukernel/mimic-38643

mimic: mds: avoid calling clientreplay_done() prematurely

Reviewed-by: Milind Changire <mchangir@redhat.com>

Merge pull request #31281 from ukernel/mimic-37906

mimic: tools/cephfs: make 'cephfs-data-scan scan_links' reconstruct snaptable

Reviewed-by: Milind Changire <mchangir@redhat.com>

Merge pull request #31381 from smithfarm/wip-42148-mimic

mimic: mds: mds returns -5(EIO) error when the deleted file does not exist

Reviewed-by: huanwen ren <ren.huanwen@zte.com.cn>

Merge pull request #32757 from batrick/i43347-mimic

mimic: mds: fix assert(omap_num_objs <= MAX_OBJECTS) of OpenFileTable

Reviewed-by: Nathan Cutler <ncutler@suse.com>

Merge pull request #31380 from smithfarm/wip-42146-mimic

mimic: client: remove Inode.dir_contacts field and handle bad whence value to llseek gracefully

Reviewed-by: Varsha Rao <varao@redhat.com>

Merge pull request #30110 from smithfarm/wip-40494-mimic

mimic: cephfs: test_volume_client: declare only one default for python version

Reviewed-by: Venky Shankar <vshankar@redhat.com>

Merge pull request #33626 from jan--f/wip-38472-notracker-mimic

mimic: ceph-volume: don't create osd['block.db'] by default

ceph-volume: don't create osd['block.db'] by default

Signed-off-by: Jan Fajerski <jfajerski@suse.com>
(cherry picked from commit f648ba1d9a1ef12fa8dc35101ef485726c002987)

Merge pull request #33622 from jan--f/wip-32828-notracker-mimic

mimic: ceph-volume: add db and wal support to raw mode

Merge pull request #33623 from jan--f/wip-43872-mimic-encore

mimic: ceph-volume: batch bluestore fix create_lvs call

ceph-volume: batch bluestore fix create_lvs call

Fixes: https://tracker.ceph.com/issues/43844
Signed-off-by: Jan Fajerski <jfajerski@suse.com>
(cherry picked from commit df18497bc9eaf1922e5c885e8cc124e439c59364)

ceph-volume: add db and wal support to raw mode

Using the raw mode, the OSD preparation can now be called with extra
arguments to specify a block device for either rocksdb db or wal.

The code expects the device to be available and no validation will be
performed. Also, the entire block device will be consumed.

The goal is run this on Kubernetes with Rook and OSD are running on PVC.
Users will request PVC from a certain size, the request will be
acknowledged and a block will be created, then attached to the machine.
Later on consumed by Rook's preparation job.

Signed-off-by: Sébastien Han <seb@redhat.com>
(cherry picked from commit c8f23c7182e2681f2bc756776a890995aca1efd2)

ceph-volume: fix raw list

There is no json anymore, just plain output.

Signed-off-by: Sébastien Han <seb@redhat.com>
(cherry picked from commit e99eb8cc95c5f50ec13170f202226c3581e3f5e2)

ceph-volume: remove unnecessary comment

This comment is not relevant since we will never bring systemd support
to the raw mode.

Signed-off-by: Sébastien Han <seb@redhat.com>
(cherry picked from commit af3f5fded95e4dcc0dfa2a0b2eec5fb2c4252b6c)

ceph-volume: remove osd-id and osd-fsid flag

These are not used since we only rely on the device path so removing
them to avid confusion.

Signed-off-by: Sébastien Han <seb@redhat.com>
(cherry picked from commit 525957be98a0a6ff742f6bae4450f291591f516f)

ceph-volume: remove "create" call from raw mode

This mode is intented to work when the cluster is running on Kubernetes
and OSDs are running on PVC.
So we always have a "prepare" pod and an "activate" pod.

Signed-off-by: Sébastien Han <seb@redhat.com>
(cherry picked from commit c7a2ecc59394a9eb323a19ff233a2e3dfe04a018)

Merge pull request #33615 from jan--f/wip-44154-mimic

mimic: ceph-volume: don't remove vg twice when zapping filestore

Merge pull request #33617 from jan--f/wip-44326-mimic

mimic: ceph-volume: lvm: get_device_vgs() filter by provided prefix

Merge pull request #33618 from jan--f/wip-44332-mimic

mimic: ceph-volume: strip _dmcrypt suffix in simple scan json output

ceph-volume: strip _dmcrypt suffix in simple scan json output

LUKS encrypted OSDs name their block* files with a _dmcrypt suffix.
activate fails on json files like this. Stripping this suffix in scan
fixes this.

Fixes: https://tracker.ceph.com/issues/43966
Signed-off-by: Jan Fajerski <jfajerski@suse.com>
(cherry picked from commit 2ddf76d118d77659c590ea076d34ce9a8e351a86)

ceph-volume: don't create Volume/VolumeGroup objects with empty name

Signed-off-by: Jan Fajerski <jfajerski@suse.com>
(cherry picked from commit 235b19eb66b9ba9beab99508f2fdf7b6afe26f31)

ceph-volume: get_device_lvs filter by provided prefix

Fixes: https://tracker.ceph.com/issues/44124
Signed-off-by: Jan Fajerski <jfajerski@suse.com>
(cherry picked from commit ac43d37f581510d1854e76afb634a142ad9a0fc0)

ceph-volume: lvm.get_device_vgs: don't create VolumeGroup with empty name

Signed-off-by: Jan Fajerski <jfajerski@suse.com>
(cherry picked from commit 3c4c8c5e4a242985389c8d1c85b45cac89b2dc89)

ceph-volume: lvm: get_device_vgs() filter by provided prefix

Fixes: https://tracker.ceph.com/issues/44246
Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
(cherry picked from commit b48caf4873c11690d36a88760504f98b46101995)

ceph-volume: don't remove vg twice when zapping filestore

Signed-off-by: Jan Fajerski <jfajerski@suse.com>
Fixes: https://tracker.ceph.com/issues/44149
(cherry picked from commit bccdf6eafaf851d5092bb99d61edd44cd36d9dd2)

Merge pull request #33610 from jan--f/wip-44134-mimic

mimic: ceph-volume: avoid calling zap_lv with a LV-less VG

Merge pull request #33611 from jan--f/wip-44151-mimic

mimic: ceph-volume: pass journal_size as Size not string

Merge pull request #33609 from jan--f/wip-44111-mimic

mimic: ceph-volume: use get_device_vgs in has_common_vg

Merge pull request #33606 from jan--f/wip-44036-mimic

mimic: ceph-volume: finer grained availability notion in inventory

Merge pull request #33607 from jan--f/wip-44048-mimic

mimic: ceph-volume: skip osd creation when already done

Merge pull request #33608 from jan--f/wip-44110-mimic

mimic: ceph-volume: fix is_ceph_device for lvm batch

Merge pull request #33605 from jan--f/wip-44227-mimic

mimic: ceph-volume: silence 'ceph-bluestore-tool' failures

Merge pull request #33489 from shyukri/wip-44073

mimic: ceph-volume: skip missing interpreters when running tox tests

ceph-volume: skip missing interpreters when running tox tests

This will skip tests for versions of python that don't exist on the node
that's running the tests.

Signed-off-by: Andrew Schoen <aschoen@redhat.com>
(cherry picked from commit f1db0a864814d9b81192d89b6eebb6c624ddec3b)

ceph-volume: pass journal_size as Size not string

Fixes: https://tracker.ceph.com/issues/44148
Signed-off-by: Jan Fajerski <jfajerski@suse.com>
(cherry picked from commit 49f6e6d600aae6310f941c6635408d496b0ff2b9)

ceph-volume: avoid calling zap_lv with a LV-less VG

Fixes: https://tracker.ceph.com/issues/44125
Signed-off-by: Jan Fajerski <jfajerski@suse.com>
(cherry picked from commit ad0dea53b8585d8397bf3069b1b39f13b6e0a8ce)

ceph-volume: use get_device_vgs in has_common_vg

Fixes: https://tracker.ceph.com/issues/44099
Signed-off-by: Jan Fajerski <jfajerski@suse.com>
(cherry picked from commit 2c5a8c3b4066dd2aca47d719c9723850ce5f96fc)

ceph-volume: add is_ceph_device unit tests

Signed-off-by: Jan Fajerski <jfajerski@suse.com>
(cherry picked from commit 60d80636e4708761287197c534347f82e307c603)

ceph-volume: fix is_ceph_device for lvm batch

This is a regression introduced by 634a709

The lvm batch command fails to prepare the OSDs on the created LV.
When using lvm batch, the LV/VG are created prior the OSD prepare.
During that creation, multiple tags are set with null value.

$ lvs -o lv_tags --noheadings
  ceph.cluster_fsid=null,ceph.osd_fsid=null,ceph.osd_id=null,ceph.type=null

Since we call is_ceph_device which returns True if the ceph.osd_id LVM
tag exists but doesn't test the value then we raise an execption.

When the tag value is set to 'null' then we can consider that the device
isn't part of the ceph cluster (because not yet prepared).

Closes: https://tracker.ceph.com/issues/44069
Signed-off-by: Dimitri Savineau <dsavinea@redhat.com>
(cherry picked from commit a82582364c7b65a4a5e2673e3886acd6d2066130)

ceph-volume: add unit test test_safe_prepare_osd_already_created

This commit adds a new unit test
`test_safe_prepare_osd_already_created()` in order to test when
`is_ceph_device()` returns `True` `RuntimeError` is well raised.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
(cherry picked from commit ccf92d718d6f363f6ca3b7e8499b68beb8b30c06)

ceph-volume: skip osd creation when already done

When rerunning ceph-volume lvm create on a device already prepared and
activated, ceph-volume should skip the creation.

This is a regression introduced by bb4de1a3fc238eaf9f717dc59c6bdf338ef6d657

Fixes: https://tracker.ceph.com/issues/43981
Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
(cherry picked from commit 634a709b9c6802c5b12e2d45c2f43181b297adfb)

ceph-volume: add available property in target specific flavors

This adds two properties available_[lvm,raw] to device (and thus inventory).
The goal is to have different notions of availability based on the
intended use case. For example finding LVM structures make a drive
unavailable for the raw mode, but might be available for the lvm mode.

Fixes: https://tracker.ceph.com/issues/43400
Signed-off-by: Jan Fajerski <jfajerski@suse.com>
(cherry picked from commit 233ccff24006082766b52a94b7c46cdf5b7cd929)

Merge pull request #33603 from jan--f/wip-31700-notracker-mimic

mimic: ceph-volume: refactor listing.py + fixes

ceph-volume: silence 'ceph-bluestore-tool' failures

If 'ceph-bluestore-tool' fails on a device, the json output of the list
command will be messed up. Ignoring stderr of that command fixes this.

Signed-off-by: Sébastien Han <seb@redhat.com>
(cherry picked from commit e8828fe76a0c78786e53a1bd63c3fc574e36a6f0)

Merge pull request #33580 from jan--f/wip-notracker-c-v-raw-mimic

mimic: ceph-volume: add raw mode

ceph-volume: fix various lvm list issues

A single report on a non-lvm device now works.
Format was cleaned up, report lvm journal,wal, db only once.

Fixes: https://tracker.ceph.com/issues/44009
Signed-off-by: Jan Fajerski <jfajerski@suse.com>
(cherry picked from commit 000bf2ffff57701952e2aa1a67a04e519c4d07a6)

ceph-volume: add get_device_lvs to easily retrieve all lvs per device

Also drop the sep argument from get_lvs and siblings, unused.
Introduce LV_CMD_OPTIONS to unify options to lvs.

Signed-off-by: Jan Fajerski <jfajerski@suse.com>
(cherry picked from commit ffe5b5732a46bd5ff888696adbfe53a38c982448)

ceph-volume: fix lvm list

17957d9beb42a04b8f180ccb7ba07d43179a41d3 introduced a regression in `lvm
list`.

When passing a vg/lv path for generating a single report, it fails
because the filter used in the `lvs` command isn't right. It uses the lv
name instead of the vg name because `os.path.basename(device)` is used
while it should be `os.path.dirname(device)`

Fixes: https://tracker.ceph.com/issues/43969
Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
(cherry picked from commit 0179fed3ab624830ba77349531763c3e116c82e5)

ceph-volume: delete test_lvs_list_is_created_just_once

lisitng.py doesn't call api.Volumes anymore. Therefore, this test is
redundant.

Signed-off-by: Rishabh Dave <ridave@redhat.com>
(cherry picked from commit 665ed2424b7bef4160289f0135acc015f8ea9980)

ceph-volume: update tests since listing.py got heavily modified

Signed-off-by: Rishabh Dave <ridave@redhat.com>
(cherry picked from commit d1ae6d1a8b495adfc0c512f08359e0db1590272d)

ceph-volume: refactor devices/lvm/listing.py

Get rid of duplicate and redundant code and use get_lvs, get_vgs and
get_pvs to simplify the module as much as possible.

Signed-off-by: Rishabh Dave <ridave@redhat.com>
(cherry picked from commit d02bd7dd581a4bd4041eb397fae540a18f16a88b)