doc/rbd: s/guess/xml/ for codeblock lexer

this change silences the warning of

```
doc/rbd/qemu-rbd.rst:174: WARNING: Pygments lexer name 'guess' is not
known
```

see http://pygments.org/docs/lexers/, we should use "xml" for XML .

Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit df226da996e468d2707b08eb012d54b4e37ffdc6)

Merge pull request #30775 from smithfarm/wip-41979-mimic

mimic: rgw: fix list versions starts with version_id=null

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #30868 from smithfarm/wip-41324-mimic

mimic: rgw: datalog/mdlog trim commands loop until done

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #30980 from smithfarm/wip-41496-mimic

mimic: rgw: fix the bug of rgw not doing necessary checking to website configuration

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #30891 from smithfarm/wip-41715-mimic

mimic: rgw: fix refcount tags to match and update object's idtag

Reviewed-by: J. Eric Ivancich <ivancich@redhat.com>

Merge pull request #30977 from theanalyst/wip-41570-mimic

mimic: rgw: asio: check the remote endpoint before processing requests

Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: Yuri Weinstein <yweinste@redhat.com>

Merge pull request #30713 from smithfarm/wip-40258-mimic

mimic: cmake: detect armv8 crc and crypto feature using CHECK_C_COMPILER_FLAG

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #30893 from smithfarm/wip-41964-mimic

mimic: tools/rados: list objects in a pg

Reviewed-by: Vikhyat Umrao <vikhyat@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge pull request #30898 from smithfarm/wip-42128-mimic

mimic: osd/OSDMap: do not trust partially simplified pg_upmap_item

Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>

Merge pull request #30903 from smithfarm/wip-42154-mimic

mimic: mon/OSDMonitor: trim not-longer-exist failure reporters

Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #30924 from vumrao/wip-vumrao-42240

mimic: osd/PG: Add PG to large omap log message

Reviewed-by: Brad Hubbard <bhubbard@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge pull request #30846 from wido/mimic-42116

mimic: mgr/telemetry: Ignore crashes in report when module not enabled

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #30895 from smithfarm/wip-42036-mimic

mimic: osd/PeeringState: recover_got - add special handler for empty log

Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>
Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge pull request #30901 from smithfarm/wip-42137-mimic

mimic: osd: Remove unused osdmap flags full, nearfull from output

Reviewed-by: David Zafman <dzafman@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>

Merge pull request #30916 from smithfarm/wip-41457-mimic

mimic: osd: merge replica log on primary need according to replica log's crt

Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge pull request #30982 from tchaikov/wip-mimic-42362

mimic: build/ops: python3-cephfs should provide python36-cephfs

Reviewed-by: Nathan Cutler <ncutler@suse.com>

Merge pull request #30991 from smithfarm/wip-37520-mimic-revert

mimic: msg: Revert "msg/async: do not trigger RESETSESSION from connect fault during connection phase"

Reviewed-by: Ricardo Dias <rdias@suse.com>
Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>

Merge pull request #30808 from jan--f/wip-42233-mimic

mimic: ceph-volume: VolumeGroups.filter shouldn't purge itself

Merge pull request #30936 from smithfarm/wip-42130-mimic

mimic: doc/ceph-fuse: mention -k option in ceph-fuse man page

Reviewed-by: Rishabh Dave <ridave@redhat.com>

Merge pull request #30806 from jan--f/wip-42235-mimic

mimic: ceph-volume: PVolumes.filter shouldn't purge itself

Revert "msg/async: do not trigger RESETSESSION from connect fault during connection phase"

This reverts commit 00b163564c6cafd4edf54d470cc708eab9dae10e.

Xie Xingguo and Ricardo Dias looked at this, and both agreed that the bug
was caused with the reduction of states of the V1 protocol during its
refactoring. In other words, the bug is not present in mimic.

Signed-off-by: Nathan Cutler <ncutler@suse.com>

ceph.spec.in: provide python2-<modname>

to be consistent with other python2 packages, and their python3
counterparts

the `python_provide` macro is offered by `python-rpm-macros` package,
which is in turn required by python*-devel

Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit fb6290b1fab0bd8a73043f2d68210c010e2fb425)

ceph.spec.in: use python_provide macro

our python3 bindings are now named `python3-<modname>` after python3 is
now maintained by RHEL/CentOS instead EPEL. to help the users using
`python36-<modname>`, we should "Provide" `python36-<modname>`.

the `python_provide` macro is offered by `python-rpm-macros` package,
which is in turn required by python*-devel. and we do install
`python36-devel` in install-deps.sh, and install `python3-devel` in
ceph-*build/build/setup_rpm

see also
https://docs.fedoraproject.org/en-US/packaging-guidelines/Python/#_provides

Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit 50b19e673d8200306c3e36d1abaec414a3d336b9)

rgw: fix checking index_doc_suffix when getting effective key

Currently, if the index_doc_suffix is empty which is leaded
by the IndexDocument field is not configurated or is set to
empty string during enabling bucket website function, the
rgw will crash when accessing the static website through the
S3Website enabled RGW instance.

In actually, we have add the necessary checking in the commit
355f392ad26631f44dac250296e96f421d86fb8f, but double checking
in here is better.

Signed-off-by: Enming Zhang <enming.zhang@umcloud.com>
(cherry picked from commit c96f415dafe176b1b8d10ff9456d13fb76c79baa)

Conflicts:
src/rgw/rgw_rest_s3.cc
- ldpp_dout

rgw: fix the bug of rgw not doing necessary checking to website configuration

Fixes: http://tracker.ceph.com/issues/40678
Signed-off-by: Enming Zhang <enming.zhang@umcloud.com>
(cherry picked from commit 355f392ad26631f44dac250296e96f421d86fb8f)

Conflicts:
src/rgw/rgw_rest_s3.cc
- 3275dffa45ae08d3818562d2d11a8d1c0afa326b is not being backported

Merge pull request #29224 from smithfarm/wip-39223-mimic

mimic: mds: behind on trimming and [dentry] was purgeable but no longer is!

Reviewed-by: Venky Shankar <vshankar@redhat.com>

Merge pull request #29232 from smithfarm/wip-40439-mimic

mimic: mds: cannot switch mds state from standby-replay to active

Reviewed-by: Venky Shankar <vshankar@redhat.com>

Merge pull request #29479 from xiaoxichen/wip-41001

mimic: cephfs: client: unlink dentry for inode with llref=0

Reviewed-by: Venky Shankar <vshankar@redhat.com>

Merge pull request #29218 from smithfarm/wip-38709-mimic

mimic: tests: kclient unmount hangs after file system goes down

Reviewed-by: Venky Shankar <vshankar@redhat.com>

Merge pull request #29220 from smithfarm/wip-39210-mimic

mimic: mds: mds_cap_revoke_eviction_timeout is not used to initialize Server::cap_revoke_eviction_timeout

Reviewed-by: Venky Shankar <vshankar@redhat.com>

Merge pull request #29222 from smithfarm/wip-39212-mimic

mimic: cephfs: MDSTableServer.cc: 83: FAILED assert(version == tid)

Reviewed-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Yan, Zheng <zyan@redhat.com>

Merge pull request #29223 from smithfarm/wip-39215-mimic

mimic: mds: there is an assertion when calling Beacon::shutdown()

Reviewed-by: Venky Shankar <vshankar@redhat.com>

Merge pull request #29228 from smithfarm/wip-40219-mimic

mimic: tests: cephfs: TestMisc.test_evict_client fails

Reviewed-by: Venky Shankar <vshankar@redhat.com>

Merge pull request #29230 from smithfarm/wip-40437-mimic

mimic: cephfs: getattr on snap inode stuck

Reviewed-by: Venky Shankar <vshankar@redhat.com>

Merge pull request #30796 from dillaman/wip-36122-mimic

mimic: librbd: properly handle potential object map failures

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #30828 from dillaman/wip-41882-mimic

mimic: rbd-mirror: cannot restore deferred deletion mirrored images

Reviewed-by: Mykola Golub <mgolub@mirantis.com>

rgw: asio: check the remote endpoint before processing requests

`socket.remote_endpoint()` can throw exceptions corresponding to errors in the
`getpeername` syscall, make sure these are handled.

Fixes: CVE-2019-10222, https://tracker.ceph.com/issues/40018
Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>
(cherry picked from commit caa653196856ecdf50519a9a33195d5c4e3372af)

 Conflicts:
src/rgw/rgw_asio_frontend.cc
conflicts due to optional-yield-ctx changes in master

Merge pull request #30213 from smithfarm/wip-41449-mimic

mimic: core: mon: C_AckMarkedDown has not handled the Callback Arguments

Reviewed-by: David Zafman <dzafman@redhat.com>

Merge pull request #30150 from neha-ojha/wip-40769-mimic

mimic: bluestore: common/options: Set concurrent bluestore rocksdb compactions to 2

Reviewed-by: Igor Fedotov <ifedotov@suse.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #30069 from smithfarm/wip-40124-mimic

mimic: qa/rgw: don't use ceph-ansible in s3a-hadoop suite

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #30133 from smithfarm/wip-40850-mimic

mimic: rgw/multisite: Don't allow certain radosgw-admin commands to run on non-master zone

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #29203 from smithfarm/wip-40320-mimic

mimic: tests: make: *** [hello_world_cpp] Error 127 in rados

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #30219 from vumrao/wip-vumrao-bluefs-shared-alloc-with-log-level-change-mimic

mimic: os/bluestore: apply shared_alloc_size to shared device with log level change

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #30260 from smithfarm/wip-41723-mimic

mimic: build/ops: fix build fail related to PYTHON_EXECUTABLE variable

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #30355 from pdvian/wip-41765-mimic

mimic: build/ops: ceph.spec.in: reserve 2500MB per build job

Reviewed-by: Nathan Cutler <ncutler@suse.com>

Merge pull request #30672 from smithfarm/wip-37520-mimic

mimic: msg/async: do not trigger RESETSESSION from connect fault during connection phase

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #30784 from smithfarm/wip-41918-mimic

mimic: core: osd: scrub error on big objects; make bluestore refuse to start on big objects

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: David Zafman <dzafman@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>

doc/ceph-fuse: mention -k option in ceph-fuse man page

Fixes: https://tracker.ceph.com/issues/42044
Signed-off-by: Rishabh Dave <ridave@redhat.com>
(cherry picked from commit a96a32bd2ef15e963bdb8c46e45edcb83e2361bb)

tools/rados: call pool_lookup() after rados is connected
This commit fixes a segmentation fault when using --pgid
option in rados ls command in combination with --pool/-p option.
The reason for the crash was that we can not use the rados object
before connecting it with the cluster using rados.init_with_context().

Fixes: https://tracker.ceph.com/issues/41875
Signed-off-by: Vikhyat Umrao <vumrao@redhat.com>
(cherry picked from commit 56b6dd9bd7a50371062128122e6a899f2c79c5f5)

Conflicts:
src/tools/rados/rados.cc

tools/rados: always call rados.shutdown() before exit()

When connected to ceph cluster, if call exit(1) directly, will cause the finisher
thread segmentation fault as follows,

Caught signal (Segmentation fault) **in thread 7f45377fe700 thread_name:fn_anonymous

run 'rados mapext' without other arguments could easily get it. This patch fixes it
by calling rados.shutdown() before exit()

Fixes: http://tracker.ceph.com/issues/36732
Signed-off-by: Li Wang <laurence.liwang@gmail.com>
(cherry picked from commit 2c149262888c50beb9f480a6cd78f77fd5920d1c)

Conflicts:
src/tools/rados/rados.cc
- revert "std::make_unique" change that, in master, should have been included in
  3804f52b109558591d699609749a789b5cfc72b9 (which is not being backported)

tools/rados: allow list objects in a pg

When the cluster is not healthy, normally only a very few pgs encounter
problems, and the operator wants to know which objects may potentially
get lost.

Signed-off-by: Li Wang <laurence.liwang@gmail.com>
(cherry picked from commit 825c7b9729b2410a116e123d4b21210c646704d4)

Conflicts:
src/tools/rados/rados.cc
- 6e4db7b0a41f8d56c8725ea27352e85a13273888 is not being backported to mimic

tools/rados: add --pgid in help

This commit 825c7b9729b2410a116e123d4b21210c646704d4 brought
the support for --pgid in rados ls but was not in help.
Fixes: https://tracker.ceph.com/issues/41876
Signed-off-by: Vikhyat Umrao <vumrao@redhat.com>
(cherry picked from commit ae9a24eaba330fd3f6946063c2bc926f58c93772)

mon/OSDMonitor: trim no-longer-exist failure reporters

remove the report from failure_info in OSDMonitor::check_failure(), if
the reporter does not exist in the osdmap any longer. otherwise, we will
run into assert() failure when trying to lookup the osd in osdmap using
`osdmap.get_xinfo()`.

Fixes: https://tracker.ceph.com/issues/41680
Signed-off-by: NancySu05 <su_nan@inspur.com>
(cherry picked from commit 99ccc166ede5f296b86abfe9db0a3d597100edde)

Conflicts:
src/mon/OSDMonitor.cc
- ceph_assert, g_conf()

osd/PG: Add PG to large omap log message

Fixes: https://tracker.ceph.com/issues/41359
Signed-off-by: Brad Hubbard <bhubbard@redhat.com>
(cherry picked from commit a71d51bfa154a278a350e6dbeb842fb3585b74f5)

osd: merge replica log on primary need according to replica log's crt

Fixes: https://tracker.ceph.com/issues/41194
Signed-off-by: Zengran Zhang <zhangzengran@sangfor.com.cn>
(cherry picked from commit ac387844e12edc9a1f481bdd1ba307ac4dc60e82)

Conflicts:
src/osd/PGLog.h
- ceph_assert

osd: Remove unused osdmap flags full, nearfull from output

Fixes: https://tracker.ceph.com/issues/42015
Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit d9c645d7d8c7355c8aaf3ff9121f2b8fc3b1290a)

osd/OSDMap: do not trust partially simplified pg_upmap_item

If we simplified a partially no-op pg_upmap_item, we shall still
continue to verify that the remaining part is valid.
The bug is introduced by 02e5499b350bcd7d9eac98b2072052a9a4a1f535,
before which we always validate the correctness of a pg_upmap_item
before trying to cancel or simplify it.

Fixes: https://tracker.ceph.com/issues/42052
Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
(cherry picked from commit 4196b13283144de966eeba40e6765f10b254dac6)

osd/PeeringState: recover_got - add special handler for empty log

If we use a small osd_min[max]_pg_log_entries setting, say, 10, and
do splitting, a child pg may only be able to keep some divergent log
entries in the list. Any follow-up peering cycle on that child pg
would then rewind all the divergent log entries and lead to an empty
log list, which as a result could reliably fire the

  ceph_assert(info.last_complete == info.last_update)

assert if we start to recover any of the missing divergent objects
later.

Fixes: https://tracker.ceph.com/issues/41816
Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
(cherry picked from commit a5e9e9099865be7a3d90c58cf3f3fc50ed6eed6d)

Conflicts:
src/osd/PeeringState.cc
- file does not exist in the stable branch: made the changes manually to
  src/osd/PrimaryLogPG.cc

rgw: make rollback refcount tag match

When the refcount does a "get", the tag ends with a null character,
but during a rollback, when the tag is "put" it does not end with a
null character, so the same tag is not being manipulated and the
refcounts are therefore off, preventing future deletion of an rgw
object. This adds the null character to the "put" operation.

This change was originally submitted by lltlo <120680451@qq.com> but
wasn't signed-off.

Signed-off-by: J. Eric Ivancich <ivancich@redhat.com>
(cherry picked from commit a0fee94982d89f0210e1c526605831105cd90ef4)

Conflicts:
src/rgw/rgw_rados.cc

rgw: make sure object's idtag is updated when available

This change was originally submitted by lltlo <120680451@qq.com> but
had some formatting issues and wasn't signed-off.

Signed-off-by: J. Eric Ivancich <ivancich@redhat.com>
(cherry picked from commit 7d32b082f826181b605ae02ebc914fba9ac0d449)

Conflicts:
src/rgw/rgw_rados.cc

Merge pull request #29258 from smithfarm/wip-39741-mimic

mimic: rgw: swift object expiry fails when a bucket reshards

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #29483 from pdvian/wip-40761-mimic

mimic: rgw: Save an unnecessary copy of RGWEnv

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #29500 from pdvian/wip-40847-mimic

mimic: rgw: Don't crash on copy when metadata directive not supplied

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #30073 from smithfarm/wip-40517-mimic

mimic: rgw: perfcounters: add gc retire counter

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #30077 from smithfarm/wip-40599-mimic

mimic: rgw_file: fix readdir eof() calc--caller stop implies !eof and introduce fast S3 Unix stats (immutable)

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #29276 from smithfarm/wip-40215-mimic

mimic: rgw_file:  fix invalidation of top-level directories

Reviewed-by: Matt Benjamin <mbenjami@redhat.com>

Merge pull request #29984 from pdvian/wip-41110-mimic

mimic: rgw: fix drain handles error when deleting bucket with bypass-gc option

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #30074 from smithfarm/wip-40539-mimic

mimic: cls/rgw: keep issuing bilog trim ops after reset

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #30105 from smithfarm/wip-41120-mimic

mimic: rgw: permit rgw-admin to populate user info by access-key

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #30130 from smithfarm/wip-40629-mimic

mimic: rgw: data/bilogs are trimmed when no peers are reading them

Reviewed-by: Casey Bodley <cbodley@redhat.com>

radosgw-admin: 'mdlog trim' loops until done

Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit 91eef6712ad29d6c15907f1663e6143fc4144dec)

Conflicts: store->time_log_trim() instead of svc.cls->timelog.trim()
src/rgw/rgw_metadata.cc no null_yield

radosgw-admin: 'datalog trim' takes shard-id and loops until done

Fixes: https://tracker.ceph.com/issues/41045
Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit b9af497b2ae445b858fbdf5b7c311770714a58da)

Conflicts: store->data_log instead of store->svc()->datalog_rados
src/rgw/rgw_admin.cc
src/rgw/rgw_bucket.cc no null_yield
src/rgw/services/svc_datalog_rados.cc (not present in luminous)
src/rgw/services/svc_datalog_rados.h

Merge pull request #30233 from smithfarm/wip-41129-mimic

mimic: qa: use hard_reset to reboot kclient

Reviewed-by: Venky Shankar <vshankar@redhat.com>

Merge pull request #30234 from smithfarm/wip-40444-mimic

mimic: mds: cleanup unneeded client_snap_caps when splitting snap inode

Reviewed-by: Venky Shankar <vshankar@redhat.com>

Merge pull request #30235 from smithfarm/wip-40844-mimic

mimic: mon/MDSMonitor: use stringstream instead of dout for mds repaired

Reviewed-by: Venky Shankar <vshankar@redhat.com>

Merge pull request #30236 from smithfarm/wip-40853-mimic

mimic: cephfs: test_volume_client: fix test_put_object_versioned()

Reviewed-by: Rishabh Dave <ridave@redhat.com>
Reviewed-by: Venky Shankar <vshankar@redhat.com>

Merge pull request #30238 from smithfarm/wip-40896-mimic

mimic: cephfs: ceph_volume_client: convert string to bytes object

Reviewed-by: Rishabh Dave <ridave@redhat.com>
Reviewed-by: Venky Shankar <vshankar@redhat.com>

Merge pull request #30239 from smithfarm/wip-40899-mimic

mimic: mds: evict an unresponsive client only when another client wants its caps

Reviewed-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Rishabh Dave <ridave@redhat.com>

Merge pull request #30240 from smithfarm/wip-41466-mimic

mimic: cephfs: mount.ceph: properly handle -o strictatime

Reviewed-by: Venky Shankar <vshankar@redhat.com>

Merge pull request #30241 from smithfarm/wip-41487-mimic

mimic: cephfs: client: return -EIO when sync file which unsafe reqs have been dropped

Reviewed-by: Venky Shankar <vshankar@redhat.com>

Merge pull request #30417 from pdvian/wip-41852-mimic

mimic: mds: make MDSIOContextBase delete itself when shutting down

Reviewed-by: Venky Shankar <vshankar@redhat.com>

Merge pull request #30443 from pdvian/wip-41856-mimic

mimic: cephfs: client: nfs-ganesha with cephfs client, removing dir reports not empty

Reviewed-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Jeff Layton <jlayton@redhat.com>

mgr/telemetry: Ignore crashes in report when module not enabled

The crash module is not guaranteerd to be enabled and this will render
the telemetry module useless:

  Error EINVAL: Traceback (most recent call last):
    File "/usr/lib/ceph/mgr/telemetry/module.py", line 325, in handle_command
      report = self.compile_report()
    File "/usr/lib/ceph/mgr/telemetry/module.py", line 291, in compile_report
      report['crashes'] = self.gather_crashinfo()
    File "/usr/lib/ceph/mgr/telemetry/module.py", line 214, in gather_crashinfo
      errno, crashids, err = self.remote('crash', 'do_ls', '', '')
    File "/usr/lib/ceph/mgr/mgr_module.py", line 845, in remote
      args, kwargs)
  ImportError: Module not found

We can safely ignore this error and just continue without the crash information.

Fixes: https://tracker.ceph.com/issues/42116
Signed-off-by: Wido den Hollander <wido@42on.com>

rbd-mirror: prevent restored trash images from being deleted after delay

The image deleter wasn't verifying whether or not an image was still in the trash
prior to deleting the image. This not only would incorrectly remove any restored
images but it will also leave the image id object and entry within the directory.

Fixes: https://tracker.ceph.com/issues/41780
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit f091a31d5252bba76598fffdc997275ca531621d)

Conflicts:
src/test/rbd_mirror/image_deleter/test_mock_TrashRemoveRequest.cc: removed state test cases
src/tools/rbd_mirror/image_deleter/TrashRemoveRequest.h/cc: removed state validation

rbd-mirror: renamed RemoveRequest state machine to TrashRemoveRequest

This better matches the current behavior where the images are only
removed from the trash.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 55daa8e1e28f457a897070adfabf1583093aadd3)

Conflicts:
src/test/rbd_mirror/CMakeLists.txt: trivial resolution
src/tools/rbd_mirror/ImageDeleter.cc: trivial resolution
src/tools/rbd_mirror/image_deleter/TrashRemoveRequest.cc: trivial resolution

rbd-mirror: set image as primary when moving to trash

This will allow the the image to be restored and re-mirrored if
desired.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 73d4577d1b9a9bff087b555c9de5005d1120a0ea)

Conflicts:
src/test/rbd_mirror/image_deleter/test_mock_TrashMoveRequest.cc: trivial resolution

librbd: reuse async trash remove state machine

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

librbd: async trash remove state machine

Signed-off-by: Mykola Golub <mgolub@suse.com>
(cherry picked from commit 8455d6611c48ae8a721d307d157b64d8a7041abe)

Conflicts:
src/librbd/trash/RemoveRequest.cc/h: removed set state calls
src/test/librbd/trash/test_mock_RemoveRequest.cc: removed set state test cases

Merge pull request #29609 from jtlayton/wip-40162-mimic

mimic: cephfs: client: fix bad error handling in _lookup_parent

Reviewed-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Yan, Zheng <zyan@redhat.com>

Merge pull request #29751 from pdvian/wip-41088-mimic

mimic: qa: sleep briefly after resetting kclient

Reviewed-by: Venky Shankar <vshankar@redhat.com>

Merge pull request #29812 from pdvian/wip-41094-mimic

mimic: qa: ignore expected MDS_CLIENT_LATE_RELEASE warning

Reviewed-by: Venky Shankar <vshankar@redhat.com>

Merge pull request #29833 from pdvian/wip-41097-mimic

mimic: cephfs: avoid map been inserted by mistake

Reviewed-by: Venky Shankar <vshankar@redhat.com>

Merge pull request #29915 from pdvian/wip-41100-mimic

mimic: cephfs: fix a memory leak

Reviewed-by: Venky Shankar <vshankar@redhat.com>

Merge pull request #29940 from pdvian/wip-41108-mimic

mimic: mds: delay exporting directory whose pin value exceeds max rank id

Reviewed-by: Venky Shankar <vshankar@redhat.com>

Merge pull request #30108 from smithfarm/wip-40442-mimic

mimic: cephfs: client: set snapdir's link count to 1

Reviewed-by: Venky Shankar <vshankar@redhat.com>

Merge pull request #30228 from smithfarm/wip-40841-mimic

mimic: cephfs: client: support the fallocate() when fuse version >= 2.9

Reviewed-by: Venky Shankar <vshankar@redhat.com>

ceph-volume: update tests since VolumeGroups.filter returns a list

VolumeGroups.filter returns VolumeGroups object that contains VGs
matching the filter. Update the tests to hold the list returned by the
Volumes.filter() call.

Signed-off-by: Rishabh Dave <ridave@redhat.com>
(cherry picked from commit da58d239a0e067cd3ea5fd26aac24a2551b871c1)

ceph-volume: VolumeGroups.filter shouldn't purge itself

VolumeGroups.filter remove VGs from the list that do no match filter.
Instead of doing that, return a new list that contains VGs that match
the fiter so that VolumeGroups object held by code calling it is not
modified.

Fixes: https://tracker.ceph.com/issues/42171
Signed-off-by: Rishabh Dave <ridave@redhat.com>
(cherry picked from commit 028757813282f764ebcce05572f9e4b76ea4e552)

ceph-volume: allow creating empty VolumeGroup objects

Signed-off-by: Rishabh Dave <ridave@redhat.com>
(cherry picked from commit 2dc4f7de96e51c8117b719640ec4c09388a1412e)

ceph-volume: update tests since PVolumes.filter returns a list

...returns a list of filtered PVs instead of removing the items that do
not match the filters from itself.

Signed-off-by: Rishabh Dave <ridave@redhat.com>
(cherry picked from commit 38ccfe089a86f86b6e48b9a60286f26bb2444596)