14.2.8

Merge PR #33569 into nautilus

* refs/pull/33569/head:
mgr/volumes: unregister job upon async threads exception

Reviewed-by: Ramana Raja <rraja@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

mgr/volumes: unregister job upon async threads exception

If the async threads hit a temporary exception the job is
never unregistered and therefore gets skipped by the async
threads on subsequent scans.

Patrick hit this in nautilus when one of the purge threads
hit an exception when trying to log a message. The trash
entry was never picked up again by the purge threads.

Fixes: http://tracker.ceph.com/issues/44315
Signed-off-by: Venky Shankar <vshankar@redhat.com>
(cherry picked from commit 46476ef2e290bd15af2fec2410cb4f3f86b27cd2)

Merge pull request #33346 from yaarith/backport-nautilus-pr-32903

nautilus: mgr/devicehealth: fix telemetry stops sending device reports after 48 hours

Reviewed-by: Sage Weil <sage@redhat.com>

Merge PR #33526 into nautilus

* refs/pull/33526/head:
test: verify purge queue w/ large number of subvolumes
test: pass timeout argument to mount::wait_for_dir_empty()
mgr/volumes: access volume in lockless mode when fetching async job

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

test: verify purge queue w/ large number of subvolumes

Fixes: http://tracker.ceph.com/issues/44282
Signed-off-by: Venky Shankar <vshankar@redhat.com>
(cherry picked from commit 92b20089369b0d549c8c337a60bb93ae24c7b66a)

test: pass timeout argument to mount::wait_for_dir_empty()

Signed-off-by: Venky Shankar <vshankar@redhat.com>
(cherry picked from commit 5ec09a228ca2b726da3ce79fc01e07911051326a)

mgr/volumes: access volume in lockless mode when fetching async job

Saw a deadlock when deleting lot of subvolumes -- purge threads were
stuck in accessing global lock for volume access. This can happen
when there is a concurrent remove (which renames and signals the
purge threads) and a purge thread is just about to scan the trash
directory for entries.

For the fix, purge threads fetches entries by accessing the volume
in lockless mode. This is safe from functionality point-of-view as
the rename and directory scan is correctly handled by the filesystem.
Worst case the purge thread would pick up the trash entry on next
scan, never leaving a stale trash entry.

Signed-off-by: Venky Shankar <vshankar@redhat.com>
(cherry picked from commit 808a1ce1f96f6dfd3472156ce5087372da4c1314)

Merge PR #33498 into nautilus

* refs/pull/33498/head:
mgr: drop reference to msg on return

Reviewed-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Sage Weil <sage@redhat.com>

mgr: drop reference to msg on return

Caused by backport commit cb48be5a69fe6482cbe3bff1b53ba090e077de0d which
did not account for the explicit drop of the message reference, only in
Nautilus-.

Fixes: https://tracker.ceph.com/issues/44245
Fixes: cb48be5a69fe6482cbe3bff1b53ba090e077de0d
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #33470 from neha-ojha/wip-mgsr2-order-nautilus

nautilus: qa/suites/upgrade/mimic-x/stress-split: fix msgr2 vs nautilus ordering

qa/suites/upgrade/mimic-x/stress-split: fix msgr2 vs nautilus ordering

This was done for octopus in 8283ea9f587fb1136b4e9fa9d8a0435852fb948a,
but not for nautilus

Signed-off-by: Neha Ojha <nojha@redhat.com>

Merge pull request #33378 from badone/wip-badone-testing

nautilus: qa/ceph-ansible: ansible-version and ceph_ansible

Reviewed-by: Yuri Weinstein <yweinste@redhat.com>

nautilus: qa/ceph-ansible: ansible-version and ceph_ansible

Upgrade to 2.8.1 and stable-4.0 respectively

Signed-off-by: Brad Hubbard <bhubbard@redhat.com>

mgr/devicehealth: fix telemetry stops sending device reports after 48 hours

Telemetry module fetches device metrics which were scraped in the last
"telemetry interval"*2 (=48 hours by default) by calling
_get_device_metrics() with min_sample. _get_device_metrics() fetches the
metrics from omap and breaks on the first one that is older than
min_sample. But because it fetched in ascending order (from oldest to
newest) it was breaking on the first one it received, if it was older
than the interval above. We need to pass min_sample to get_omap_vals()
so it will start fetching from that value.

Fixes: https://tracker.ceph.com/issues/43837
Signed-off-by: Yaarit Hatuka <yaarit@redhat.com>
(cherry picked from commit 5f7e4a980a73e8cacb2c9bde47d822a32fb8c440)

mgr/devicehealth: factor _get_device_metrics out of show_device_metrics

Add the min_sample lower-bound argument too

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 7be5c1323b3814e2634d5cd66d45cab5a77df680)
Conflicts: had to be backported to enable backporting of
https://github.com/ceph/ceph/pull/32903
Backport tracker: https://tracker.ceph.com/issues/43873

Merge pull request #33278 from smithfarm/wip-44085-nautilus

nautilus: ceph-monstore-tool: correct the key for storing mgr_command_descs

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #33277 from smithfarm/wip-43722-nautilus

nautilus: common/bl: fix the dangling last_p issue.

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #33276 from smithfarm/wip-44082-nautilus

nautilus: qa/suites/rados/multimon/tasks/mon_clock_with_skews: disable ntpd etc

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #32844 from smithfarm/wip-43239-nautilus

nautilus: mgr/DaemonServer: fix 'osd ok-to-stop' for EC pools

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: David Zafman <dzafman@redhat.com>

Merge pull request #33337 from jan--f/wip-44153-nautilus

nautilus: ceph-volume: don't remove vg twice when zapping filestore

Merge pull request #33334 from jan--f/wip-44152-nautilus

nautilus: ceph-volume: pass journal_size as Size not string

ceph-volume: don't remove vg twice when zapping filestore

Signed-off-by: Jan Fajerski <jfajerski@suse.com>
Fixes: https://tracker.ceph.com/issues/44149
(cherry picked from commit bccdf6eafaf851d5092bb99d61edd44cd36d9dd2)

ceph-volume: pass journal_size as Size not string

Fixes: https://tracker.ceph.com/issues/44148
Signed-off-by: Jan Fajerski <jfajerski@suse.com>
(cherry picked from commit 49f6e6d600aae6310f941c6635408d496b0ff2b9)

Merge pull request #33301 from jan--f/wip-43871-nautilus-failed-cp

nautilus: ceph-volume: batch bluestore fix create_lvs call

Merge pull request #33297 from jan--f/wip-44135-nautilus

nautilus: ceph-volume: avoid calling zap_lv with a LV-less VG

ceph-volume: batch bluestore fix create_lvs call

Fixes: https://tracker.ceph.com/issues/43844
Signed-off-by: Jan Fajerski <jfajerski@suse.com>
(cherry picked from commit df18497bc9eaf1922e5c885e8cc124e439c59364)

ceph-volume: avoid calling zap_lv with a LV-less VG

Fixes: https://tracker.ceph.com/issues/44125
Signed-off-by: Jan Fajerski <jfajerski@suse.com>
(cherry picked from commit ad0dea53b8585d8397bf3069b1b39f13b6e0a8ce)

Merge pull request #33151 from shyukri/wip-43877-nautilus

nautilus: rgw: fix one part of the bulk delete(RGWDeleteMultiObj_ObjStore_S3) fails but no error messages

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #33149 from shyukri/wip-43874-nautilus

nautilus: rgw: maybe coredump when reload operator happened

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #33008 from smithfarm/wip-43922-nautilus

nautilus: rgw_file: avoid string::front() on empty path

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #33152 from shyukri/wip-43879-nautilus

nautilus: mon: Don't put session during feature change

Reviewed-by: David Zafman <dzafman@redhat.com>

Merge pull request #33095 from k0ste/wip-43979-nautilus

nautilus: mgr/telemetry: check get_metadata return val

Reviewed-by: David Zafman <dzafman@redhat.com>

Merge pull request #32908 from smithfarm/wip-43821-nautilus

nautilus: mon/Session: only index osd ids >= 0

Merge pull request #33170 from k0ste/wip-43727-nautilus

nautilus: mgr/pg_autoscaler: calculate pool_pg_target using pool size

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #33168 from k0ste/wip-44057-nautilus

nautilus: mgr/telemetry: split entity_name only once (handle ids with dots)

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #33157 from shyukri/wip-43924-nautilus

nautilus: mgr/prometheus: report per-pool pg states

Reviewed-by: Jan Fajerski <jfajerski@suse.com>

Merge pull request #33155 from shyukri/wip-43916-nautilus

nautilus: mon/ConfigMonitor: only propose if leader

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #33147 from shyukri/wip-43989-nautilus

nautilus: osd: Allow 64-char hostname to be added as the "host" in CRUSH

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge pull request #33142 from shyukri/wip-44000-nautilus

nautilus: mon/MgrMonitor.cc: warn about missing mgr in a cluster with osds

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #33082 from k0ste/wip-43974-nautilus

nautilus: mgr/telemetry: anonymizing smartctl report itself

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #32948 from yaarith/wip-telemetry-serial-nautilus-yh

nautilus: mgr/telemetry: fix device serial number anonymization

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge pull request #33007 from smithfarm/wip-43928-nautilus

nautilus: mon: elector: return after triggering a new election

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #32931 from smithfarm/wip-43819-nautilus

nautilus: mgr/pg_autoscaler: default to pg_num[_min] = 32

Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Laura Paduano <lpaduano@suse.com>

Merge pull request #32905 from smithfarm/wip-43731-nautilus

nautilus: crush/CrushWrapper: behave with empty weight vector

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #32856 from zhengchengyao/nautilus_no_mon_update

nautilus: mon/ConfigMonitor: fix handling of NO_MON_UPDATE settings

Reviewed-by: Nathan Cutler <ncutler@suse.com>
Reviewed-by: Sage Weil <sage@redhat.com>

doc: update mondb recovery script

to note that we also need to add mgr's key to monitor's keyring

Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit 75f4765f2ffe795dba85540b8aa1675ba9de28e4)

ceph-monstore-tool: correct the key for storing mgr_command_descs

Fixes: https://tracker.ceph.com/issues/43582
Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit a5bfeca64f3204142fda5320b4fd403df4b5f532)

ceph-monstore-tool: rename mon-ids in initial monmap

when ceph-mon starts, it checks to see if it's listed in the monmap, if
not it complains
```
no public_addr or public_network specified, and mon.a not present in
monmap or ceph.conf.
```
then bails out. normally, the monitor will try to rename its name in
monmap when performing "mkfs", but in our case, we are merely using the
"mkfs" monmap for passing the monmap built by ceph-monstore-tools, and
we don't actually go through the "mkfs" process. so, ceph-mon won't
rename when booting up.

in this change, user is allowed to specify the mon-ids in command line
when rebuilding mondb, the default mon-ids would be a,b,c,... if not
specified.

Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit 4b3df5a850db054928f9fcc6ef0a160a05a2ffa9)

common/bl: fix the dangling last_p issue.

Fixes: https://tracker.ceph.com/issues/43646
Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
(cherry picked from commit 8198332f3afb5de748dfa6e3349fefbf7e9ed137)

Conflicts:
src/test/bufferlist.cc

qa/suites/rados/multimon/tasks/mon_clock_with_skews: whitelist MOST_DOWN

The skewed clock makes some mons miss elections.

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 08b6a2bc008c006ff815dcb85bfb24b73072c7ab)

qa/suites/rados/multimon/tasks/mon_clock_with_skews: disable ntpd etc

Fixes: https://tracker.ceph.com/issues/43889
Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 9f2a854b175f156d4ab7fba955aff515052c9d93)

Merge pull request #33254 from jan--f/wip-44112-nautilus

nautilus: ceph-volume: use get_device_vgs in has_common_vg

Merge pull request #33253 from jan--f/wip-44109-nautilus

nautilus: ceph-volume: fix is_ceph_device for lvm batch

Merge pull request #33240 from jan--f/wip-44035-nautilus

nautilus: ceph-volume: finer grained availability notion in inventory.

Merge pull request #33239 from jan--f/wip-43984-nautilus

nautilus: ceph-volume: fix has_bluestore_label() function

ceph-volume: use get_device_vgs in has_common_vg

Fixes: https://tracker.ceph.com/issues/44099
Signed-off-by: Jan Fajerski <jfajerski@suse.com>
(cherry picked from commit 2c5a8c3b4066dd2aca47d719c9723850ce5f96fc)

ceph-volume: add is_ceph_device unit tests

Signed-off-by: Jan Fajerski <jfajerski@suse.com>
(cherry picked from commit 60d80636e4708761287197c534347f82e307c603)

ceph-volume: fix is_ceph_device for lvm batch

This is a regression introduced by 634a709

The lvm batch command fails to prepare the OSDs on the created LV.
When using lvm batch, the LV/VG are created prior the OSD prepare.
During that creation, multiple tags are set with null value.

$ lvs -o lv_tags --noheadings
  ceph.cluster_fsid=null,ceph.osd_fsid=null,ceph.osd_id=null,ceph.type=null

Since we call is_ceph_device which returns True if the ceph.osd_id LVM
tag exists but doesn't test the value then we raise an execption.

When the tag value is set to 'null' then we can consider that the device
isn't part of the ceph cluster (because not yet prepared).

Closes: https://tracker.ceph.com/issues/44069
Signed-off-by: Dimitri Savineau <dsavinea@redhat.com>
(cherry picked from commit a82582364c7b65a4a5e2673e3886acd6d2066130)

Merge pull request #33242 from jan--f/wip-44047-nautilus

nautilus: ceph-volume: skip osd creation when already done

Merge pull request #32919 from smithfarm/wip-43780-nautilus

nautilus: cephfs: qa: ignore trimmed cache items for dead cache drop

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #33183 from smithfarm/wip-43846-nautilus

nautilus: rgw: update the hash source for multipart entries during resharding

Reviewed-by: J. Eric Ivancich <ivancich@redhat.com>

Merge pull request #33115 from batrick/i43790

nautilus: RuntimeError: Files in flight high water is unexpectedly low (0 / 6)

Reviewed-by: Ramana Raja <rraja@redhat.com>

Merge pull request #32921 from smithfarm/wip-43784-nautilus

nautilus: mds/OpenFileTable: match MAX_ITEMS_PER_OBJ to osd_deep_scrub_large_omap_object_key_threshold

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #32918 from smithfarm/wip-43777-nautilus

nautilus: cephfs: qa: save MDS epoch barrier

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #32917 from smithfarm/wip-43733-nautilus

nautilus: cephfs: qa: ignore slow ops for ffsb workunit

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #32756 from batrick/i43347

nautilus: mds: fix assert(omap_num_objs <= MAX_OBJECTS) of OpenFileTable

Reviewed-by: Ramana Raja <rraja@redhat.com>

Merge pull request #31905 from batrick/i43046

nautilus: mgr: "mds metadata" to setup new DaemonState races with fsmap

Reviewed-by: Ramana Raja <rraja@redhat.com>

Merge pull request #32916 from smithfarm/wip-43729-nautilus

nautilus: cephfs: client: Add is_dir() check before changing directory

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Jeff Layton <jlayton@redhat.com>

Merge pull request #32807 from smithfarm/wip-43770-nautilus

nautilus: mount.ceph: remove arbitrary limit on size of name= option

Reviewed-by: Ilya Dryomov <idryomov@redhat.com>
Reviewed-by: Jeff Layton <jlayton@redhat.com>
Reviewed-by: Ramana Raja <rraja@redhat.com>

Merge pull request #32910 from smithfarm/wip-43503-nautilus

nautilus: mount.ceph: give a hint message when no mds is up or cluster is laggy

Reviewed-by: Jeff Layton <jlayton@redhat.com>
Reviewed-by: Ilya Dryomov <idryomov@redhat.com>

Merge pull request #33122 from ajarr/wip-ajarr-mgr-volumes-nautilus

mgr/volumes: misc fix and feature enhancements

Reviewed-by: Venky Shankar <vshankar@redhat.com>

qa/standalone/misc/ok-to-stop: improve test

Make sure PGs peer (simply flushing state to mon isn't enough).

Fixes: https://tracker.ceph.com/issues/43721
Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 76ea774c109cc565c0b85feed40f7f29105029d3)

qa/standalone/ceph-helpers: add wait_for_peered

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 78ec6aec90c7d0eb8b017c7f3d34e376b7f6713f)

mgr/DaemonServer: fix 'osd ok-to-stop' for EC pools

We need to pay attention to account for CRUSH_ITEM_NONE entries in the
EC PG acting set.

Fixes: https://tracker.ceph.com/issues/43151
Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 66690ea3143ac5097f7c7f3118f2d00fed30cc4b)

Conflicts:
        qa/standalone/misc/ok-to-stop.sh
- nautilus "ceph osd pool create" CLI command takes a pg_num argument

ceph-volume: add unit test test_safe_prepare_osd_already_created

This commit adds a new unit test
`test_safe_prepare_osd_already_created()` in order to test when
`is_ceph_device()` returns `True` `RuntimeError` is well raised.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
(cherry picked from commit ccf92d718d6f363f6ca3b7e8499b68beb8b30c06)

ceph-volume: skip osd creation when already done

When rerunning ceph-volume lvm create on a device already prepared and
activated, ceph-volume should skip the creation.

This is a regression introduced by bb4de1a3fc238eaf9f717dc59c6bdf338ef6d657

Fixes: https://tracker.ceph.com/issues/43981
Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
(cherry picked from commit 634a709b9c6802c5b12e2d45c2f43181b297adfb)

ceph-volume: add available property in target specific flavors

This adds two properties available_[lvm,raw] to device (and thus inventory).
The goal is to have different notions of availability based on the
intended use case. For example finding LVM structures make a drive
unavailable for the raw mode, but might be available for the lvm mode.

Fixes: https://tracker.ceph.com/issues/43400
Signed-off-by: Jan Fajerski <jfajerski@suse.com>
(cherry picked from commit 233ccff24006082766b52a94b7c46cdf5b7cd929)

ceph-volume: remove stderr in has_bluestore_label()

We don't want to generate this log when a call to
`has_bluestore_label()` fails.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
(cherry picked from commit 7f8371c32b3f613b0d094f6f26ffbeb19ea0d25b)

ceph-volume: fix has_bluestore_label() function

When using vg/lv, this function throws an error like following:

```
 stderr: unable to read label for test_group/data-lv2: (2) No such file or directory
 stderr: 2020-02-04T21:03:32.153+0000 7fe091af4200 -1 bluestore(test_group/data-lv2) _read_bdev_label failed to open test_group/data-lv2: (2) No such file or directory
```

using `self.abspath` fixes this error.

Fixes: https://tracker.ceph.com/issues/43970
Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
(cherry picked from commit 148069a20fef79ef8fe510f06879a0de02987eac)

Merge pull request #33238 from jan--f/wip-31700-notracker-nautilus

nautilus: ceph-volume: refactor listing.py + fixes

ceph-volume: fix various lvm list issues

A single report on a non-lvm device now works.
Format was cleaned up, report lvm journal,wal, db only once.

Fixes: https://tracker.ceph.com/issues/44009
Signed-off-by: Jan Fajerski <jfajerski@suse.com>
(cherry picked from commit 000bf2ffff57701952e2aa1a67a04e519c4d07a6)

ceph-volume: add get_device_lvs to easily retrieve all lvs per device

Also drop the sep argument from get_lvs and siblings, unused.
Introduce LV_CMD_OPTIONS to unify options to lvs.

Signed-off-by: Jan Fajerski <jfajerski@suse.com>
(cherry picked from commit ffe5b5732a46bd5ff888696adbfe53a38c982448)

ceph-volume: fix lvm list

17957d9beb42a04b8f180ccb7ba07d43179a41d3 introduced a regression in `lvm
list`.

When passing a vg/lv path for generating a single report, it fails
because the filter used in the `lvs` command isn't right. It uses the lv
name instead of the vg name because `os.path.basename(device)` is used
while it should be `os.path.dirname(device)`

Fixes: https://tracker.ceph.com/issues/43969
Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
(cherry picked from commit 0179fed3ab624830ba77349531763c3e116c82e5)

ceph-volume: delete test_lvs_list_is_created_just_once

lisitng.py doesn't call api.Volumes anymore. Therefore, this test is
redundant.

Signed-off-by: Rishabh Dave <ridave@redhat.com>
(cherry picked from commit 665ed2424b7bef4160289f0135acc015f8ea9980)

ceph-volume: update tests since listing.py got heavily modified

Signed-off-by: Rishabh Dave <ridave@redhat.com>
(cherry picked from commit d1ae6d1a8b495adfc0c512f08359e0db1590272d)

ceph-volume: refactor devices/lvm/listing.py

Get rid of duplicate and redundant code and use get_lvs, get_vgs and
get_pvs to simplify the module as much as possible.

Signed-off-by: Rishabh Dave <ridave@redhat.com>
(cherry picked from commit d02bd7dd581a4bd4041eb397fae540a18f16a88b)

ceph-volume: add new method in api/lvm.py

The method determines whether given LV is managed by Ceph or not.

Signed-off-by: Rishabh Dave <ridave@redhat.com>
(cherry picked from commit 876244b6ab2cf1fbd724fd33966501a3366c6d3f)

ceph-volume: add helper methods to get only first LVM devs

These convenience methods shortens following phrase to
"lv = get_first_lv()" -

lvs = get_lvs()
if len(lvs) >= 1:
lvs = lv[0]

These methods do the same things as above phrase internall. Rewrite
listing.py to use these new helper methods.

Signed-off-by: Rishabh Dave <ridave@redhat.com>
(cherry picked from commit 17957d9beb42a04b8f180ccb7ba07d43179a41d3)

ceph-volume: filter based on tags for api.lvm.get_* methods

get_pvs, get_vgs and get_lvs must accept tags and filter volumes based
on tags.

Signed-off-by: Rishabh Dave <ridave@redhat.com>
(cherry picked from commit fb1390964fdfd10815ab4a4302ad454169bc0d5f)

Merge pull request #33232 from jan--f/wip-43871-nautilus

nautilus: ceph-volume: batch bluestore fix create_lvs call

Merge pull request #33231 from jan--f/wip-43849-nautilus

nautilus: ceph-volume: add sizing arguments to prepare

mgr/volumes: fix py2 compat issue

Fix the following issue seen while upstream teuthology testing,
 File "/usr/share/ceph/mgr/volumes/fs/operations/versions/subvolume_base.py", line 98, in load_config
   self.metadata_mgr = MetadataManager(self.fs, self.legacy_config_path, 0o640)
 File "/usr/share/ceph/mgr/volumes/fs/operations/versions/subvolume_base.py", line 73, in legacy_config_path
   meta_config = "{0}.meta".format(m.digest().hex())
 AttributeError: 'str' object has no attribute 'hex'

This issue is not observed in master/octopus, as it only supports
py3.

Signed-off-by: Ramana Raja <rraja@redhat.com>

mgr/volumes: type convert uid and gid to int

This fix is only needed in nautilus, and the issue
was observed during upstream teuthology testing.

   File "/usr/share/ceph/mgr/volumes/fs/async_cloner.py", line 114, in cptree
     copy_file(fs_handle, d_full_src, d_full_dst, mo, st.st_uid, st.st_gid)
   File "/usr/share/ceph/mgr/volumes/fs/fs_util.py", line 97, in copy_file
     fs.chown(dst, uid, gid)
   File "cephfs.pyx", line 855, in cephfs.LibCephFS.chown
      TypeError: uid must be an int

The issue wasn't observed in master/octopus teuthology
testing.

Signed-off-by: Ramana Raja <rraja@redhat.com>

test: add subvolume clone tests

Signed-off-by: Venky Shankar <vshankar@redhat.com>
(cherry picked from commit b5970ff80d9d203e6289770cbb8f51cc6a2c5938)

mgr/volumes: allow force removal of incomplete failed clones

Signed-off-by: Venky Shankar <vshankar@redhat.com>
(cherry picked from commit 451be112603f970de3cb0ca9048c88a98c034188)

mgr/volumes: asynchronous cloner module

Signed-off-by: Venky Shankar <vshankar@redhat.com>
(cherry picked from commit 4f09568b012cef24d2075733f9d81064790fe4e6)

mgr/volumes: purge thread uses new async interface

This also makes `_cancel_jobs()` thread safe, which was not the
case earlier (with `_cancel_purge_job()`) -- this also makes the
code simpler by sharing the lock betweent two condition variables.

Signed-off-by: Venky Shankar <vshankar@redhat.com>
(cherry picked from commit f16cc1e8eebb0868a1e07c25f8d8e4656b11b7bf)

mgr/volumes: fetch oldest clone entry

Signed-off-by: Venky Shankar <vshankar@redhat.com>
(cherry picked from commit 7ad14cf09b447fcdbfe76ac739971be8609aa862)

mgr/volumes: add clone specific commands

Signed-off-by: Venky Shankar <vshankar@redhat.com>
(cherry picked from commit b2145b73d7640851c27ac4bb09fd98e146de953d)