ceph-volume: batch bluestore fix create_lvs call

Fixes: https://tracker.ceph.com/issues/43844
Signed-off-by: Jan Fajerski <jfajerski@suse.com>
(cherry picked from commit df18497bc9eaf1922e5c885e8cc124e439c59364)

Merge pull request #33151 from shyukri/wip-43877-nautilus

nautilus: rgw: fix one part of the bulk delete(RGWDeleteMultiObj_ObjStore_S3) fails but no error messages

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #33149 from shyukri/wip-43874-nautilus

nautilus: rgw: maybe coredump when reload operator happened

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #33008 from smithfarm/wip-43922-nautilus

nautilus: rgw_file: avoid string::front() on empty path

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #33152 from shyukri/wip-43879-nautilus

nautilus: mon: Don't put session during feature change

Reviewed-by: David Zafman <dzafman@redhat.com>

Merge pull request #33095 from k0ste/wip-43979-nautilus

nautilus: mgr/telemetry: check get_metadata return val

Reviewed-by: David Zafman <dzafman@redhat.com>

Merge pull request #32908 from smithfarm/wip-43821-nautilus

nautilus: mon/Session: only index osd ids >= 0

Merge pull request #33170 from k0ste/wip-43727-nautilus

nautilus: mgr/pg_autoscaler: calculate pool_pg_target using pool size

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #33168 from k0ste/wip-44057-nautilus

nautilus: mgr/telemetry: split entity_name only once (handle ids with dots)

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #33157 from shyukri/wip-43924-nautilus

nautilus: mgr/prometheus: report per-pool pg states

Reviewed-by: Jan Fajerski <jfajerski@suse.com>

Merge pull request #33155 from shyukri/wip-43916-nautilus

nautilus: mon/ConfigMonitor: only propose if leader

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #33147 from shyukri/wip-43989-nautilus

nautilus: osd: Allow 64-char hostname to be added as the "host" in CRUSH

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge pull request #33142 from shyukri/wip-44000-nautilus

nautilus: mon/MgrMonitor.cc: warn about missing mgr in a cluster with osds

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #33082 from k0ste/wip-43974-nautilus

nautilus: mgr/telemetry: anonymizing smartctl report itself

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #32948 from yaarith/wip-telemetry-serial-nautilus-yh

nautilus: mgr/telemetry: fix device serial number anonymization

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge pull request #33007 from smithfarm/wip-43928-nautilus

nautilus: mon: elector: return after triggering a new election

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #32931 from smithfarm/wip-43819-nautilus

nautilus: mgr/pg_autoscaler: default to pg_num[_min] = 32

Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Laura Paduano <lpaduano@suse.com>

Merge pull request #32905 from smithfarm/wip-43731-nautilus

nautilus: crush/CrushWrapper: behave with empty weight vector

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #32856 from zhengchengyao/nautilus_no_mon_update

nautilus: mon/ConfigMonitor: fix handling of NO_MON_UPDATE settings

Reviewed-by: Nathan Cutler <ncutler@suse.com>
Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #33254 from jan--f/wip-44112-nautilus

nautilus: ceph-volume: use get_device_vgs in has_common_vg

Merge pull request #33253 from jan--f/wip-44109-nautilus

nautilus: ceph-volume: fix is_ceph_device for lvm batch

Merge pull request #33240 from jan--f/wip-44035-nautilus

nautilus: ceph-volume: finer grained availability notion in inventory.

Merge pull request #33239 from jan--f/wip-43984-nautilus

nautilus: ceph-volume: fix has_bluestore_label() function

ceph-volume: use get_device_vgs in has_common_vg

Fixes: https://tracker.ceph.com/issues/44099
Signed-off-by: Jan Fajerski <jfajerski@suse.com>
(cherry picked from commit 2c5a8c3b4066dd2aca47d719c9723850ce5f96fc)

ceph-volume: add is_ceph_device unit tests

Signed-off-by: Jan Fajerski <jfajerski@suse.com>
(cherry picked from commit 60d80636e4708761287197c534347f82e307c603)

ceph-volume: fix is_ceph_device for lvm batch

This is a regression introduced by 634a709

The lvm batch command fails to prepare the OSDs on the created LV.
When using lvm batch, the LV/VG are created prior the OSD prepare.
During that creation, multiple tags are set with null value.

$ lvs -o lv_tags --noheadings
  ceph.cluster_fsid=null,ceph.osd_fsid=null,ceph.osd_id=null,ceph.type=null

Since we call is_ceph_device which returns True if the ceph.osd_id LVM
tag exists but doesn't test the value then we raise an execption.

When the tag value is set to 'null' then we can consider that the device
isn't part of the ceph cluster (because not yet prepared).

Closes: https://tracker.ceph.com/issues/44069
Signed-off-by: Dimitri Savineau <dsavinea@redhat.com>
(cherry picked from commit a82582364c7b65a4a5e2673e3886acd6d2066130)

Merge pull request #33242 from jan--f/wip-44047-nautilus

nautilus: ceph-volume: skip osd creation when already done

Merge pull request #32919 from smithfarm/wip-43780-nautilus

nautilus: cephfs: qa: ignore trimmed cache items for dead cache drop

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #33183 from smithfarm/wip-43846-nautilus

nautilus: rgw: update the hash source for multipart entries during resharding

Reviewed-by: J. Eric Ivancich <ivancich@redhat.com>

Merge pull request #33115 from batrick/i43790

nautilus: RuntimeError: Files in flight high water is unexpectedly low (0 / 6)

Reviewed-by: Ramana Raja <rraja@redhat.com>

Merge pull request #32921 from smithfarm/wip-43784-nautilus

nautilus: mds/OpenFileTable: match MAX_ITEMS_PER_OBJ to osd_deep_scrub_large_omap_object_key_threshold

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #32918 from smithfarm/wip-43777-nautilus

nautilus: cephfs: qa: save MDS epoch barrier

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #32917 from smithfarm/wip-43733-nautilus

nautilus: cephfs: qa: ignore slow ops for ffsb workunit

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #32756 from batrick/i43347

nautilus: mds: fix assert(omap_num_objs <= MAX_OBJECTS) of OpenFileTable

Reviewed-by: Ramana Raja <rraja@redhat.com>

Merge pull request #31905 from batrick/i43046

nautilus: mgr: "mds metadata" to setup new DaemonState races with fsmap

Reviewed-by: Ramana Raja <rraja@redhat.com>

Merge pull request #32916 from smithfarm/wip-43729-nautilus

nautilus: cephfs: client: Add is_dir() check before changing directory

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Jeff Layton <jlayton@redhat.com>

Merge pull request #32807 from smithfarm/wip-43770-nautilus

nautilus: mount.ceph: remove arbitrary limit on size of name= option

Reviewed-by: Ilya Dryomov <idryomov@redhat.com>
Reviewed-by: Jeff Layton <jlayton@redhat.com>
Reviewed-by: Ramana Raja <rraja@redhat.com>

Merge pull request #32910 from smithfarm/wip-43503-nautilus

nautilus: mount.ceph: give a hint message when no mds is up or cluster is laggy

Reviewed-by: Jeff Layton <jlayton@redhat.com>
Reviewed-by: Ilya Dryomov <idryomov@redhat.com>

Merge pull request #33122 from ajarr/wip-ajarr-mgr-volumes-nautilus

mgr/volumes: misc fix and feature enhancements

Reviewed-by: Venky Shankar <vshankar@redhat.com>

ceph-volume: add unit test test_safe_prepare_osd_already_created

This commit adds a new unit test
`test_safe_prepare_osd_already_created()` in order to test when
`is_ceph_device()` returns `True` `RuntimeError` is well raised.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
(cherry picked from commit ccf92d718d6f363f6ca3b7e8499b68beb8b30c06)

ceph-volume: skip osd creation when already done

When rerunning ceph-volume lvm create on a device already prepared and
activated, ceph-volume should skip the creation.

This is a regression introduced by bb4de1a3fc238eaf9f717dc59c6bdf338ef6d657

Fixes: https://tracker.ceph.com/issues/43981
Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
(cherry picked from commit 634a709b9c6802c5b12e2d45c2f43181b297adfb)

ceph-volume: add available property in target specific flavors

This adds two properties available_[lvm,raw] to device (and thus inventory).
The goal is to have different notions of availability based on the
intended use case. For example finding LVM structures make a drive
unavailable for the raw mode, but might be available for the lvm mode.

Fixes: https://tracker.ceph.com/issues/43400
Signed-off-by: Jan Fajerski <jfajerski@suse.com>
(cherry picked from commit 233ccff24006082766b52a94b7c46cdf5b7cd929)

ceph-volume: remove stderr in has_bluestore_label()

We don't want to generate this log when a call to
`has_bluestore_label()` fails.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
(cherry picked from commit 7f8371c32b3f613b0d094f6f26ffbeb19ea0d25b)

ceph-volume: fix has_bluestore_label() function

When using vg/lv, this function throws an error like following:

```
 stderr: unable to read label for test_group/data-lv2: (2) No such file or directory
 stderr: 2020-02-04T21:03:32.153+0000 7fe091af4200 -1 bluestore(test_group/data-lv2) _read_bdev_label failed to open test_group/data-lv2: (2) No such file or directory
```

using `self.abspath` fixes this error.

Fixes: https://tracker.ceph.com/issues/43970
Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
(cherry picked from commit 148069a20fef79ef8fe510f06879a0de02987eac)

Merge pull request #33238 from jan--f/wip-31700-notracker-nautilus

nautilus: ceph-volume: refactor listing.py + fixes

ceph-volume: fix various lvm list issues

A single report on a non-lvm device now works.
Format was cleaned up, report lvm journal,wal, db only once.

Fixes: https://tracker.ceph.com/issues/44009
Signed-off-by: Jan Fajerski <jfajerski@suse.com>
(cherry picked from commit 000bf2ffff57701952e2aa1a67a04e519c4d07a6)

ceph-volume: add get_device_lvs to easily retrieve all lvs per device

Also drop the sep argument from get_lvs and siblings, unused.
Introduce LV_CMD_OPTIONS to unify options to lvs.

Signed-off-by: Jan Fajerski <jfajerski@suse.com>
(cherry picked from commit ffe5b5732a46bd5ff888696adbfe53a38c982448)

ceph-volume: fix lvm list

17957d9beb42a04b8f180ccb7ba07d43179a41d3 introduced a regression in `lvm
list`.

When passing a vg/lv path for generating a single report, it fails
because the filter used in the `lvs` command isn't right. It uses the lv
name instead of the vg name because `os.path.basename(device)` is used
while it should be `os.path.dirname(device)`

Fixes: https://tracker.ceph.com/issues/43969
Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
(cherry picked from commit 0179fed3ab624830ba77349531763c3e116c82e5)

ceph-volume: delete test_lvs_list_is_created_just_once

lisitng.py doesn't call api.Volumes anymore. Therefore, this test is
redundant.

Signed-off-by: Rishabh Dave <ridave@redhat.com>
(cherry picked from commit 665ed2424b7bef4160289f0135acc015f8ea9980)

ceph-volume: update tests since listing.py got heavily modified

Signed-off-by: Rishabh Dave <ridave@redhat.com>
(cherry picked from commit d1ae6d1a8b495adfc0c512f08359e0db1590272d)

ceph-volume: refactor devices/lvm/listing.py

Get rid of duplicate and redundant code and use get_lvs, get_vgs and
get_pvs to simplify the module as much as possible.

Signed-off-by: Rishabh Dave <ridave@redhat.com>
(cherry picked from commit d02bd7dd581a4bd4041eb397fae540a18f16a88b)

ceph-volume: add new method in api/lvm.py

The method determines whether given LV is managed by Ceph or not.

Signed-off-by: Rishabh Dave <ridave@redhat.com>
(cherry picked from commit 876244b6ab2cf1fbd724fd33966501a3366c6d3f)

ceph-volume: add helper methods to get only first LVM devs

These convenience methods shortens following phrase to
"lv = get_first_lv()" -

lvs = get_lvs()
if len(lvs) >= 1:
lvs = lv[0]

These methods do the same things as above phrase internall. Rewrite
listing.py to use these new helper methods.

Signed-off-by: Rishabh Dave <ridave@redhat.com>
(cherry picked from commit 17957d9beb42a04b8f180ccb7ba07d43179a41d3)

ceph-volume: filter based on tags for api.lvm.get_* methods

get_pvs, get_vgs and get_lvs must accept tags and filter volumes based
on tags.

Signed-off-by: Rishabh Dave <ridave@redhat.com>
(cherry picked from commit fb1390964fdfd10815ab4a4302ad454169bc0d5f)

Merge pull request #33232 from jan--f/wip-43871-nautilus

nautilus: ceph-volume: batch bluestore fix create_lvs call

Merge pull request #33231 from jan--f/wip-43849-nautilus

nautilus: ceph-volume: add sizing arguments to prepare

mgr/volumes: fix py2 compat issue

Fix the following issue seen while upstream teuthology testing,
 File "/usr/share/ceph/mgr/volumes/fs/operations/versions/subvolume_base.py", line 98, in load_config
   self.metadata_mgr = MetadataManager(self.fs, self.legacy_config_path, 0o640)
 File "/usr/share/ceph/mgr/volumes/fs/operations/versions/subvolume_base.py", line 73, in legacy_config_path
   meta_config = "{0}.meta".format(m.digest().hex())
 AttributeError: 'str' object has no attribute 'hex'

This issue is not observed in master/octopus, as it only supports
py3.

Signed-off-by: Ramana Raja <rraja@redhat.com>

mgr/volumes: type convert uid and gid to int

This fix is only needed in nautilus, and the issue
was observed during upstream teuthology testing.

   File "/usr/share/ceph/mgr/volumes/fs/async_cloner.py", line 114, in cptree
     copy_file(fs_handle, d_full_src, d_full_dst, mo, st.st_uid, st.st_gid)
   File "/usr/share/ceph/mgr/volumes/fs/fs_util.py", line 97, in copy_file
     fs.chown(dst, uid, gid)
   File "cephfs.pyx", line 855, in cephfs.LibCephFS.chown
      TypeError: uid must be an int

The issue wasn't observed in master/octopus teuthology
testing.

Signed-off-by: Ramana Raja <rraja@redhat.com>

test: add subvolume clone tests

Signed-off-by: Venky Shankar <vshankar@redhat.com>
(cherry picked from commit b5970ff80d9d203e6289770cbb8f51cc6a2c5938)

mgr/volumes: allow force removal of incomplete failed clones

Signed-off-by: Venky Shankar <vshankar@redhat.com>
(cherry picked from commit 451be112603f970de3cb0ca9048c88a98c034188)

mgr/volumes: asynchronous cloner module

Signed-off-by: Venky Shankar <vshankar@redhat.com>
(cherry picked from commit 4f09568b012cef24d2075733f9d81064790fe4e6)

mgr/volumes: purge thread uses new async interface

This also makes `_cancel_jobs()` thread safe, which was not the
case earlier (with `_cancel_purge_job()`) -- this also makes the
code simpler by sharing the lock betweent two condition variables.

Signed-off-by: Venky Shankar <vshankar@redhat.com>
(cherry picked from commit f16cc1e8eebb0868a1e07c25f8d8e4656b11b7bf)

mgr/volumes: fetch oldest clone entry

Signed-off-by: Venky Shankar <vshankar@redhat.com>
(cherry picked from commit 7ad14cf09b447fcdbfe76ac739971be8609aa862)

mgr/volumes: add clone specific commands

Signed-off-by: Venky Shankar <vshankar@redhat.com>
(cherry picked from commit b2145b73d7640851c27ac4bb09fd98e146de953d)

mgr/volumes: interface for fetching cloned subvolume status

Signed-off-by: Venky Shankar <vshankar@redhat.com>
(cherry picked from commit fa3c56f55273eca5be0a4a9d9fc93aab3dacda2b)

mgr/volumes: add protect/unprotect and snap clone interface

Signed-off-by: Venky Shankar <vshankar@redhat.com>
(cherry picked from commit 8d68f1af3a12ce096e52024aa710e03e0006ba0c)

mgr/volumes: handle transient subvolume states

Signed-off-by: Venky Shankar <vshankar@redhat.com>
(cherry picked from commit 461909be2edd2f691263785469e5fc90a82448e8)

mgr/volumes: interface for creating a cloned subvolume

Signed-off-by: Venky Shankar <vshankar@redhat.com>
(cherry picked from commit 7089808bf82321712676803dd4bfdd62cf5fc00e)

mgr/volumes: get/set property for subvolume mode/uid/gid

This will be required when creating a clone as the clone would
inherit source subvolumes creation mode and uid/gid.

Signed-off-by: Venky Shankar <vshankar@redhat.com>
(cherry picked from commit f02b1e7e546609a4f4287a27b3495a09e287888b)

mgr/volumes: module to track pending clone operations

Signed-off-by: Venky Shankar <vshankar@redhat.com>
(cherry picked from commit 53187795838ccee4b470f3a08aeaf6e474975408)

mgr/volumes: add operation state machine table

... and fetch creation state from state machine table.

Signed-off-by: Venky Shankar <vshankar@redhat.com>
(cherry picked from commit 46f29bf3b4fbb86aeefc940d47eadd6634b5a1ec)

mgr/volumes: fail removing subvolume with snapshots

Fixes: http://tracker.ceph.com/issues/43645
Signed-off-by: Venky Shankar <vshankar@redhat.com>
(cherry picked from commit c158a1334207966c9727eb1e9e2eaab756e5e08b)

mgr/volumes: remove stale subvolume module

this was lying around post versioning changes.

Signed-off-by: Venky Shankar <vshankar@redhat.com>
(cherry picked from commit 5595476af7fe4dc2dfdeae17d5507d1d8eeeb3f5)

test: auto-upgrade subvolume test

Fixes: https://tracker.ceph.com/issues/43349
Signed-off-by: Venky Shankar <vshankar@redhat.com>
(cherry picked from commit 03ee966b6c92b43a4f38f35351baf9ebc0126bbc)

mgr/volumes: tie everything together to implement versioned subvolumes

apart from the new way of provisioning subvolumes, this makes heavy
use of context manager for volumes, groups and subvolumes.

this change classifies volumes, groups and subvolumes to be treated
as filesystem dentries and inodes. a "volume" can be thought as a
dentry with "groups" as it's entries (inodes). likewise, a "group"
is a dentry again with "subvolumes" as entries (inodes). this is
built into the access mechanism as follows:

      with open_volume(...) as fs_handle:
          with open_gorup(fs_handle, ...) as group:
              with open_subvolume(group, ...) as subvolume:
                  # call subvolume object API
                  path = subvolume.getpath()

this way, lot of redundant checks such as verifying if a volume or
group exist before accessing a subvolume is built right into the
access mechanism, plus, an added bonus of simple error handling.

Signed-off-by: Venky Shankar <vshankar@redhat.com>
(cherry picked from commit 9b87bd72093eaecda5e2ffc3ed32e482431f9842)

mgr/volumes: provide subvolume create/remove/open APIs

create_subvolume() creates a subvolume with the max version known
to the plugin. open_subvolume() performs version discovery by
using loader stub and returns a subvoule object.

Signed-off-by: Venky Shankar <vshankar@redhat.com>
(cherry picked from commit 8a649148025620991a67ed761153f18594f5268c)

mgr/volumes: implement subvolume based on subvolume template

subvolume base class implements common routines/helpers and
initializes a metadata manager. later, when v2 subvolume version
is implemented, the metadata manager would be used to persist
subvolume metadata in ceph filesystem. this would allow flexible
metadata management when complex subvolume features are added.

typically, a subvolume would be implemented by subclassing the
subvolume base class and the subvolume template -- instantiating
this would be called a "subvolume object".

with this commit, current subvolume topology is maintained. but
we introduce the concept of subvolume versions. a loader stub
loads available "versions" of subvolumes. right now, the only
available version is v1. since backward compatibility needs to
be maintained for existing subvolumes, the loader API allows
version discovery w/ auto upgradation to the most recent version.

Signed-off-by: Venky Shankar <vshankar@redhat.com>
(cherry picked from commit 97170d72b764ed26e93b45e8e2e7ecc2368a2a04)

mgr/volumes: implement subvolume group based on group template

Signed-off-by: Venky Shankar <vshankar@redhat.com>
(cherry picked from commit 3eccd618ecce13052e31d07437ee2d09ae98e5c4)

mgr/volumes: implement trash as a subvolume group

Signed-off-by: Venky Shankar <vshankar@redhat.com>
(cherry picked from commit 0e3c48e39b2f366877fb38aa79438b0f7c5ed075)

mgr/volumes: snapshot util module

Signed-off-by: Venky Shankar <vshankar@redhat.com>
(cherry picked from commit b30f0cb18543d7bcd9161b4e39d163425df19a27)

mgr/volumes: template for implementing groups and subvolumes

Signed-off-by: Venky Shankar <vshankar@redhat.com>
(cherry picked from commit 74f349fe1550f4e5a67760582628436a828a99ce)

mgr/volumes: implement filesystem volume module

Signed-off-by: Venky Shankar <vshankar@redhat.com>
(cherry picked from commit f9ae6e38600bada917e9d7c953e736c5c45e7064)

mgr/volumes: lock module to serialize volume operations

Signed-off-by: Venky Shankar <vshankar@redhat.com>
(cherry picked from commit bc89d05358404039938d260da1a7f0f6a4953e00)

mgr/volumes: introduce volume specification module

unlike existing subvolume specification, this is just a
minimal set of globally available configurations. bulk
of other configurations will be moved to the respective
entity modules (subsequent commits).

Signed-off-by: Venky Shankar <vshankar@redhat.com>
(cherry picked from commit 0039b5d90142c9eedd23af3c5dbf2c21bcb56823)

mgr/volumes: add fs_util helper module

helpers for various filesystem querying routines, utils
for creating/removing filesystem, pool and MDSs.

Signed-off-by: Venky Shankar <vshankar@redhat.com>
(cherry picked from commit 6682c7736f4d6462aefd5387ea9f34f97ce28264)

mgr/volumes: drop obsolete comment in _cmd_fs_volume_create

This is fixed already.
Now the pool names are:
cephfs.<volume name>.meta
cephfs.<volume name>.data

Signed-off-by: Jos Collin <jcollin@redhat.com>
(cherry picked from commit ffda5f651106d2d0466a3376d0dab7291720b38c)

mgr/volumes: cleanup on fs create error

* clean up on fs create error
* drop unnecessary check in create_pool

Signed-off-by: Jos Collin <jcollin@redhat.com>
(cherry picked from commit 171c37504f71df95ec10fb294229abbc09d36e47)

mgr/volumes: move up 'confirm' validation

Instead of checking if the --yes-i-really-mean-it
flag was set _after_ removing the MDS daemon, we
need to check if before starting any removal operation.

Fixes: https://tracker.ceph.com/issues/42931
Signed-off-by: Joshua Schmid <jschmid@suse.de>
(cherry picked from commit c1db7f8b42f238f5875386a71d30bc34ae29171c)

mgr/volumes: remove unsed variable

Signed-off-by: Joshua Schmid <jschmid@suse.de>
(cherry picked from commit 2f705ee69bbad22c1744905276527c3515877caf)

mgr/volumes: guard volume delete by waiting for pending ops

Signed-off-by: Venky Shankar <vshankar@redhat.com>
(cherry picked from commit 968f67576e10fcb486819cfe3e5b44586d0d7b84)

mgr/volumes: cleanup libcephfs handles when stopping

Signed-off-by: Venky Shankar <vshankar@redhat.com>
(cherry picked from commit 2eb0c503047bdf46b0a884df96819cba8979b624)

mgr/volumes: refactor dir handle cleanup

introduce with statement in rmtree. This change
simplifies the code's handling of directory cleanup.

Signed-off-by: Jos Collin <jcollin@redhat.com>
(cherry picked from commit 9e27cd1aac1169b2cf547cc4adb38c7ac8409dd1)

mgr/volumes: cleanup leftovers from earlier purge job implementation

... which was not fully implemented anyway, so just remove the
boilerplates.

Signed-off-by: Venky Shankar <vshankar@redhat.com>
(cherry picked from commit 45c22bdcea4e23d101e27c748536eecf65dc79e7)

qa/tasks: Nothing to clean up if the volume was not created

There are only 2 cases which needs cleanup:
1. The volume is successfully created
2. The volume is successfully created but create_mds fails

In either case, we could do a 'volume rm'.

Signed-off-by: Jos Collin <jcollin@redhat.com>
(cherry picked from commit 67e43f466826f121d7649da33954b5b6a5454101)

qa/tasks: Fix the volume ls in test_volume_rm

Signed-off-by: Jos Collin <jcollin@redhat.com>
(cherry picked from commit 691352413e2b8ada25b587c1287a47087c3b9385)

qa/tasks: tests for 'fs volume create' and 'fs volume ls'

Fixes: https://tracker.ceph.com/issues/42872
Signed-off-by: Jos Collin <jcollin@redhat.com>
(cherry picked from commit 5e998bd9b5d0a6d819d87f74b9769a4086b886fe)

qa/tasks: remove subvolume, subvolumegroup and their snapshots with --force

* tests 'fs subvolume rm --force'
* tests 'fs subvolume snapshot rm --force'
* tests 'fs subvolumegroup rm --force'
* tests 'fs subvolumegroup snapshot rm --force'

Signed-off-by: Jos Collin <jcollin@redhat.com>
(cherry picked from commit 42c135d85a01213b6abf0b3b86d549e025e7325d)

qa/tasks: Fix the commands success

* Raised RuntimeException when the commands, which were expected to fail succeed.
* Dropped some commands with --force remove commands, as it is unnecessary.

Signed-off-by: Jos Collin <jcollin@redhat.com>
(cherry picked from commit c8f6a25ca4f5c4ec870bfed81d640ea2aa5cc7bb)

qa/tasks: Fix raises that doesn't re-raise

* Fixed raises that doesn't re-raise
* Dropped some commands with --force remove commands, as it is unnecessary.

Signed-off-by: Jos Collin <jcollin@redhat.com>
(cherry picked from commit 992d8b6a74400133405db69f1da94c98cf47fed6)

test: use distinct subvolume/group/snapshot names

Fixes: http://tracker.ceph.com/issues/42646
Signed-off-by: Venky Shankar <vshankar@redhat.com>
(cherry picked from commit b9cff8af8e8f423c8e2b58de2b4da48842223cf3)