Merge pull request #30979 from smithfarm/wip-41464-mimic

mimic: tools: ceph-objectstore-tool: update-mon-db: do not fail if incmap is missing

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge pull request #31017 from smithfarm/wip-40896-mimic-revert

mimic: cephfs: Revert "ceph_volume_client: convert string to bytes object"

Reviewed-by: Venky Shankar <vshankar@redhat.com>

Merge pull request #29219 from smithfarm/wip-38875-mimic

mimic: mds: high debug logging with many subtrees is slow

Reviewed-by: Venky Shankar <vshankar@redhat.com>

Merge pull request #30932 from smithfarm/wip-42034-mimic

mimic: cephfs: client: EINVAL may be returned when offset is 0.

Reviewed-by: Venky Shankar <vshankar@redhat.com>

Merge pull request #30933 from smithfarm/wip-42038-mimic

mimic: cephfs: client: _readdir_cache_cb() may use the readdir_cache already clear

Reviewed-by: Venky Shankar <vshankar@redhat.com>

Merge pull request #31090 from smithfarm/wip-42416-mimic

mimic: doc/rbd: s/guess/xml/ for codeblock lexer

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

doc/rbd: s/guess/xml/ for codeblock lexer

this change silences the warning of

```
doc/rbd/qemu-rbd.rst:174: WARNING: Pygments lexer name 'guess' is not
known
```

see http://pygments.org/docs/lexers/, we should use "xml" for XML .

Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit df226da996e468d2707b08eb012d54b4e37ffdc6)

Merge pull request #30775 from smithfarm/wip-41979-mimic

mimic: rgw: fix list versions starts with version_id=null

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #30868 from smithfarm/wip-41324-mimic

mimic: rgw: datalog/mdlog trim commands loop until done

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #30980 from smithfarm/wip-41496-mimic

mimic: rgw: fix the bug of rgw not doing necessary checking to website configuration

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #30891 from smithfarm/wip-41715-mimic

mimic: rgw: fix refcount tags to match and update object's idtag

Reviewed-by: J. Eric Ivancich <ivancich@redhat.com>

Merge pull request #30977 from theanalyst/wip-41570-mimic

mimic: rgw: asio: check the remote endpoint before processing requests

Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: Yuri Weinstein <yweinste@redhat.com>

Merge pull request #30713 from smithfarm/wip-40258-mimic

mimic: cmake: detect armv8 crc and crypto feature using CHECK_C_COMPILER_FLAG

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #30893 from smithfarm/wip-41964-mimic

mimic: tools/rados: list objects in a pg

Reviewed-by: Vikhyat Umrao <vikhyat@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge pull request #30898 from smithfarm/wip-42128-mimic

mimic: osd/OSDMap: do not trust partially simplified pg_upmap_item

Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>

Merge pull request #30903 from smithfarm/wip-42154-mimic

mimic: mon/OSDMonitor: trim not-longer-exist failure reporters

Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #30924 from vumrao/wip-vumrao-42240

mimic: osd/PG: Add PG to large omap log message

Reviewed-by: Brad Hubbard <bhubbard@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge pull request #30846 from wido/mimic-42116

mimic: mgr/telemetry: Ignore crashes in report when module not enabled

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #30895 from smithfarm/wip-42036-mimic

mimic: osd/PeeringState: recover_got - add special handler for empty log

Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>
Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge pull request #30901 from smithfarm/wip-42137-mimic

mimic: osd: Remove unused osdmap flags full, nearfull from output

Reviewed-by: David Zafman <dzafman@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>

Merge pull request #30916 from smithfarm/wip-41457-mimic

mimic: osd: merge replica log on primary need according to replica log's crt

Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge pull request #30982 from tchaikov/wip-mimic-42362

mimic: build/ops: python3-cephfs should provide python36-cephfs

Reviewed-by: Nathan Cutler <ncutler@suse.com>

Merge pull request #30991 from smithfarm/wip-37520-mimic-revert

mimic: msg: Revert "msg/async: do not trigger RESETSESSION from connect fault during connection phase"

Reviewed-by: Ricardo Dias <rdias@suse.com>
Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>

Revert "ceph_volume_client: make UTF-8 encoding explicit"

This reverts commit ddb8cfa072cc19fb6cb61128b4fe4b8ffe1a1742.

Signed-off-by: Nathan Cutler <ncutler@suse.com>

Revert "pybind/ceph_volume_client: allow volume_client to"

This reverts commit 1ab18fcf056ef8e4fcb9ed27e64ee4f8a75866dc.

Signed-off-by: Nathan Cutler <ncutler@suse.com>

Revert "ceph_volume_client: convert string to bytes object"

This reverts commit e081e97190b58d3a09377275f6d484bf9020fe51.

Signed-off-by: Nathan Cutler <ncutler@suse.com>

Revert "ceph_volume_client: don't convert None to str object"

This reverts commit e085982def42738d181240c7b6e97b30f6bc0cf8.

Signed-off-by: Nathan Cutler <ncutler@suse.com>

Merge pull request #30808 from jan--f/wip-42233-mimic

mimic: ceph-volume: VolumeGroups.filter shouldn't purge itself

Merge pull request #30936 from smithfarm/wip-42130-mimic

mimic: doc/ceph-fuse: mention -k option in ceph-fuse man page

Reviewed-by: Rishabh Dave <ridave@redhat.com>

Merge pull request #30806 from jan--f/wip-42235-mimic

mimic: ceph-volume: PVolumes.filter shouldn't purge itself

Revert "msg/async: do not trigger RESETSESSION from connect fault during connection phase"

This reverts commit 00b163564c6cafd4edf54d470cc708eab9dae10e.

Xie Xingguo and Ricardo Dias looked at this, and both agreed that the bug
was caused with the reduction of states of the V1 protocol during its
refactoring. In other words, the bug is not present in mimic.

Signed-off-by: Nathan Cutler <ncutler@suse.com>

ceph.spec.in: provide python2-<modname>

to be consistent with other python2 packages, and their python3
counterparts

the `python_provide` macro is offered by `python-rpm-macros` package,
which is in turn required by python*-devel

Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit fb6290b1fab0bd8a73043f2d68210c010e2fb425)

ceph.spec.in: use python_provide macro

our python3 bindings are now named `python3-<modname>` after python3 is
now maintained by RHEL/CentOS instead EPEL. to help the users using
`python36-<modname>`, we should "Provide" `python36-<modname>`.

the `python_provide` macro is offered by `python-rpm-macros` package,
which is in turn required by python*-devel. and we do install
`python36-devel` in install-deps.sh, and install `python3-devel` in
ceph-*build/build/setup_rpm

see also
https://docs.fedoraproject.org/en-US/packaging-guidelines/Python/#_provides

Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit 50b19e673d8200306c3e36d1abaec414a3d336b9)

rgw: fix checking index_doc_suffix when getting effective key

Currently, if the index_doc_suffix is empty which is leaded
by the IndexDocument field is not configurated or is set to
empty string during enabling bucket website function, the
rgw will crash when accessing the static website through the
S3Website enabled RGW instance.

In actually, we have add the necessary checking in the commit
355f392ad26631f44dac250296e96f421d86fb8f, but double checking
in here is better.

Signed-off-by: Enming Zhang <enming.zhang@umcloud.com>
(cherry picked from commit c96f415dafe176b1b8d10ff9456d13fb76c79baa)

Conflicts:
src/rgw/rgw_rest_s3.cc
- ldpp_dout

rgw: fix the bug of rgw not doing necessary checking to website configuration

Fixes: http://tracker.ceph.com/issues/40678
Signed-off-by: Enming Zhang <enming.zhang@umcloud.com>
(cherry picked from commit 355f392ad26631f44dac250296e96f421d86fb8f)

Conflicts:
src/rgw/rgw_rest_s3.cc
- 3275dffa45ae08d3818562d2d11a8d1c0afa326b is not being backported

Merge pull request #29224 from smithfarm/wip-39223-mimic

mimic: mds: behind on trimming and [dentry] was purgeable but no longer is!

Reviewed-by: Venky Shankar <vshankar@redhat.com>

Merge pull request #29232 from smithfarm/wip-40439-mimic

mimic: mds: cannot switch mds state from standby-replay to active

Reviewed-by: Venky Shankar <vshankar@redhat.com>

Merge pull request #29479 from xiaoxichen/wip-41001

mimic: cephfs: client: unlink dentry for inode with llref=0

Reviewed-by: Venky Shankar <vshankar@redhat.com>

ceph-objectstore-tool: update-mon-db: do not fail if incmap is missing

there is chance that we could use an OSD which does not have incmap of a
certain epoch for rebuilding the monstore. and since OSD does not read
and store the incmap if the MOSDMap message already has the fullmap of
that fullmap, and if an OSD does not have previous fullmap, monitor
will just send it the fullmao. so it's not unusual that an OSD has
a fullmap of some epoch without corresponding incmap.

Fixes: https://tracker.ceph.com/issues/41177
Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit 2e2414b3df97b22ccc54500830c24b28597ce75f)

Merge pull request #29218 from smithfarm/wip-38709-mimic

mimic: tests: kclient unmount hangs after file system goes down

Reviewed-by: Venky Shankar <vshankar@redhat.com>

Merge pull request #29220 from smithfarm/wip-39210-mimic

mimic: mds: mds_cap_revoke_eviction_timeout is not used to initialize Server::cap_revoke_eviction_timeout

Reviewed-by: Venky Shankar <vshankar@redhat.com>

Merge pull request #29222 from smithfarm/wip-39212-mimic

mimic: cephfs: MDSTableServer.cc: 83: FAILED assert(version == tid)

Reviewed-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Yan, Zheng <zyan@redhat.com>

Merge pull request #29223 from smithfarm/wip-39215-mimic

mimic: mds: there is an assertion when calling Beacon::shutdown()

Reviewed-by: Venky Shankar <vshankar@redhat.com>

Merge pull request #29228 from smithfarm/wip-40219-mimic

mimic: tests: cephfs: TestMisc.test_evict_client fails

Reviewed-by: Venky Shankar <vshankar@redhat.com>

Merge pull request #29230 from smithfarm/wip-40437-mimic

mimic: cephfs: getattr on snap inode stuck

Reviewed-by: Venky Shankar <vshankar@redhat.com>

Merge pull request #30796 from dillaman/wip-36122-mimic

mimic: librbd: properly handle potential object map failures

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #30828 from dillaman/wip-41882-mimic

mimic: rbd-mirror: cannot restore deferred deletion mirrored images

Reviewed-by: Mykola Golub <mgolub@mirantis.com>

rgw: asio: check the remote endpoint before processing requests

`socket.remote_endpoint()` can throw exceptions corresponding to errors in the
`getpeername` syscall, make sure these are handled.

Fixes: CVE-2019-10222, https://tracker.ceph.com/issues/40018
Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>
(cherry picked from commit caa653196856ecdf50519a9a33195d5c4e3372af)

 Conflicts:
src/rgw/rgw_asio_frontend.cc
conflicts due to optional-yield-ctx changes in master

Merge pull request #30213 from smithfarm/wip-41449-mimic

mimic: core: mon: C_AckMarkedDown has not handled the Callback Arguments

Reviewed-by: David Zafman <dzafman@redhat.com>

Merge pull request #30150 from neha-ojha/wip-40769-mimic

mimic: bluestore: common/options: Set concurrent bluestore rocksdb compactions to 2

Reviewed-by: Igor Fedotov <ifedotov@suse.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>

mds: check earlier if directories are already exported

Signed-off-by: Rishabh Dave <ridave@redhat.com>
(cherry picked from commit 46fb90734f371b5467d81cae25c06f8a487a3041)

Conflicts:
    src/mds/MDCache.cc
- g_conf-> instead of g_conf()->

mds: dont print auth trees if they are too many

Signed-off-by: Rishabh Dave <ridave@redhat.com>
(cherry picked from commit 13c4152c8b1756a871f817d4b503396b6c9cc81d)

Conflicts:
src/mds/MDBalancer.cc
- g_conf
- authsubs in method MDBalancer::handle_export_pins() is redeclared from set
  to vector in master branch. Removed vector declaration as per the luminous
  backport.

mds: dont print subtrees if they are too many/big

Also, add an argument to force print subtrees and let
MDBalancer::tick() always print subtrees.

Fixes: https://tracker.ceph.com/issues/37726
Signed-off-by: Rishabh Dave <ridave@redhat.com>
(cherry picked from commit 50d28ec4f842670fb96ab3b0d2b37a9c2e282236)

Conflicts:
    src/mds/MDCache.cc
- g_conf

Merge pull request #30069 from smithfarm/wip-40124-mimic

mimic: qa/rgw: don't use ceph-ansible in s3a-hadoop suite

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #30133 from smithfarm/wip-40850-mimic

mimic: rgw/multisite: Don't allow certain radosgw-admin commands to run on non-master zone

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #29203 from smithfarm/wip-40320-mimic

mimic: tests: make: *** [hello_world_cpp] Error 127 in rados

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #30219 from vumrao/wip-vumrao-bluefs-shared-alloc-with-log-level-change-mimic

mimic: os/bluestore: apply shared_alloc_size to shared device with log level change

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #30260 from smithfarm/wip-41723-mimic

mimic: build/ops: fix build fail related to PYTHON_EXECUTABLE variable

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #30355 from pdvian/wip-41765-mimic

mimic: build/ops: ceph.spec.in: reserve 2500MB per build job

Reviewed-by: Nathan Cutler <ncutler@suse.com>

Merge pull request #30672 from smithfarm/wip-37520-mimic

mimic: msg/async: do not trigger RESETSESSION from connect fault during connection phase

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #30784 from smithfarm/wip-41918-mimic

mimic: core: osd: scrub error on big objects; make bluestore refuse to start on big objects

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: David Zafman <dzafman@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>

doc/ceph-fuse: mention -k option in ceph-fuse man page

Fixes: https://tracker.ceph.com/issues/42044
Signed-off-by: Rishabh Dave <ridave@redhat.com>
(cherry picked from commit a96a32bd2ef15e963bdb8c46e45edcb83e2361bb)

client: _readdir_cache_cb() may use the readdir_cache already clear

Calling function _readdir_cache_cb() means to get dir information from the cache, but in the while loop,
the contents of readdir_cache are not static.

For example, _getattr() will trigger insert_trace to update readdir_cache, such as:
1. insert_trace()->clear_dir_complete_and_ordered() or
2. insert_trace()->add_update_inode()->add_update_cap()-> check_cap_issue()->clear_dir_complete_and_ordered()
So, there is a possibility that dn is empty after the _getattr() call.

Fixes: http://tracker.ceph.com/issues/41148
Signed-off-by: huanwen ren <ren.huanwen@zte.com.cn>
(cherry picked from commit 404c06784e46016703104db59ea73f36d09b578b)

client:EINVAL may be returned when offset is 0 ,Loff_t pos = f->pos should be the best.
Fixes:https://tracker.ceph.com/issues/41837
Signed-off-by: wenpengLi <liwenpeng@inspur.com>
(cherry picked from commit 0572e54318cebdfa83d67d47be9a6d110489e808)

tools/rados: call pool_lookup() after rados is connected
This commit fixes a segmentation fault when using --pgid
option in rados ls command in combination with --pool/-p option.
The reason for the crash was that we can not use the rados object
before connecting it with the cluster using rados.init_with_context().

Fixes: https://tracker.ceph.com/issues/41875
Signed-off-by: Vikhyat Umrao <vumrao@redhat.com>
(cherry picked from commit 56b6dd9bd7a50371062128122e6a899f2c79c5f5)

Conflicts:
src/tools/rados/rados.cc

tools/rados: always call rados.shutdown() before exit()

When connected to ceph cluster, if call exit(1) directly, will cause the finisher
thread segmentation fault as follows,

Caught signal (Segmentation fault) **in thread 7f45377fe700 thread_name:fn_anonymous

run 'rados mapext' without other arguments could easily get it. This patch fixes it
by calling rados.shutdown() before exit()

Fixes: http://tracker.ceph.com/issues/36732
Signed-off-by: Li Wang <laurence.liwang@gmail.com>
(cherry picked from commit 2c149262888c50beb9f480a6cd78f77fd5920d1c)

Conflicts:
src/tools/rados/rados.cc
- revert "std::make_unique" change that, in master, should have been included in
  3804f52b109558591d699609749a789b5cfc72b9 (which is not being backported)

tools/rados: allow list objects in a pg

When the cluster is not healthy, normally only a very few pgs encounter
problems, and the operator wants to know which objects may potentially
get lost.

Signed-off-by: Li Wang <laurence.liwang@gmail.com>
(cherry picked from commit 825c7b9729b2410a116e123d4b21210c646704d4)

Conflicts:
src/tools/rados/rados.cc
- 6e4db7b0a41f8d56c8725ea27352e85a13273888 is not being backported to mimic

tools/rados: add --pgid in help

This commit 825c7b9729b2410a116e123d4b21210c646704d4 brought
the support for --pgid in rados ls but was not in help.
Fixes: https://tracker.ceph.com/issues/41876
Signed-off-by: Vikhyat Umrao <vumrao@redhat.com>
(cherry picked from commit ae9a24eaba330fd3f6946063c2bc926f58c93772)

mon/OSDMonitor: trim no-longer-exist failure reporters

remove the report from failure_info in OSDMonitor::check_failure(), if
the reporter does not exist in the osdmap any longer. otherwise, we will
run into assert() failure when trying to lookup the osd in osdmap using
`osdmap.get_xinfo()`.

Fixes: https://tracker.ceph.com/issues/41680
Signed-off-by: NancySu05 <su_nan@inspur.com>
(cherry picked from commit 99ccc166ede5f296b86abfe9db0a3d597100edde)

Conflicts:
src/mon/OSDMonitor.cc
- ceph_assert, g_conf()

osd/PG: Add PG to large omap log message

Fixes: https://tracker.ceph.com/issues/41359
Signed-off-by: Brad Hubbard <bhubbard@redhat.com>
(cherry picked from commit a71d51bfa154a278a350e6dbeb842fb3585b74f5)

osd: merge replica log on primary need according to replica log's crt

Fixes: https://tracker.ceph.com/issues/41194
Signed-off-by: Zengran Zhang <zhangzengran@sangfor.com.cn>
(cherry picked from commit ac387844e12edc9a1f481bdd1ba307ac4dc60e82)

Conflicts:
src/osd/PGLog.h
- ceph_assert

osd: Remove unused osdmap flags full, nearfull from output

Fixes: https://tracker.ceph.com/issues/42015
Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit d9c645d7d8c7355c8aaf3ff9121f2b8fc3b1290a)

osd/OSDMap: do not trust partially simplified pg_upmap_item

If we simplified a partially no-op pg_upmap_item, we shall still
continue to verify that the remaining part is valid.
The bug is introduced by 02e5499b350bcd7d9eac98b2072052a9a4a1f535,
before which we always validate the correctness of a pg_upmap_item
before trying to cancel or simplify it.

Fixes: https://tracker.ceph.com/issues/42052
Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
(cherry picked from commit 4196b13283144de966eeba40e6765f10b254dac6)

osd/PeeringState: recover_got - add special handler for empty log

If we use a small osd_min[max]_pg_log_entries setting, say, 10, and
do splitting, a child pg may only be able to keep some divergent log
entries in the list. Any follow-up peering cycle on that child pg
would then rewind all the divergent log entries and lead to an empty
log list, which as a result could reliably fire the

  ceph_assert(info.last_complete == info.last_update)

assert if we start to recover any of the missing divergent objects
later.

Fixes: https://tracker.ceph.com/issues/41816
Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
(cherry picked from commit a5e9e9099865be7a3d90c58cf3f3fc50ed6eed6d)

Conflicts:
src/osd/PeeringState.cc
- file does not exist in the stable branch: made the changes manually to
  src/osd/PrimaryLogPG.cc

rgw: make rollback refcount tag match

When the refcount does a "get", the tag ends with a null character,
but during a rollback, when the tag is "put" it does not end with a
null character, so the same tag is not being manipulated and the
refcounts are therefore off, preventing future deletion of an rgw
object. This adds the null character to the "put" operation.

This change was originally submitted by lltlo <120680451@qq.com> but
wasn't signed-off.

Signed-off-by: J. Eric Ivancich <ivancich@redhat.com>
(cherry picked from commit a0fee94982d89f0210e1c526605831105cd90ef4)

Conflicts:
src/rgw/rgw_rados.cc

rgw: make sure object's idtag is updated when available

This change was originally submitted by lltlo <120680451@qq.com> but
had some formatting issues and wasn't signed-off.

Signed-off-by: J. Eric Ivancich <ivancich@redhat.com>
(cherry picked from commit 7d32b082f826181b605ae02ebc914fba9ac0d449)

Conflicts:
src/rgw/rgw_rados.cc

Merge pull request #29258 from smithfarm/wip-39741-mimic

mimic: rgw: swift object expiry fails when a bucket reshards

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #29483 from pdvian/wip-40761-mimic

mimic: rgw: Save an unnecessary copy of RGWEnv

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #29500 from pdvian/wip-40847-mimic

mimic: rgw: Don't crash on copy when metadata directive not supplied

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #30073 from smithfarm/wip-40517-mimic

mimic: rgw: perfcounters: add gc retire counter

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #30077 from smithfarm/wip-40599-mimic

mimic: rgw_file: fix readdir eof() calc--caller stop implies !eof and introduce fast S3 Unix stats (immutable)

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #29276 from smithfarm/wip-40215-mimic

mimic: rgw_file:  fix invalidation of top-level directories

Reviewed-by: Matt Benjamin <mbenjami@redhat.com>

Merge pull request #29984 from pdvian/wip-41110-mimic

mimic: rgw: fix drain handles error when deleting bucket with bypass-gc option

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #30074 from smithfarm/wip-40539-mimic

mimic: cls/rgw: keep issuing bilog trim ops after reset

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #30105 from smithfarm/wip-41120-mimic

mimic: rgw: permit rgw-admin to populate user info by access-key

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #30130 from smithfarm/wip-40629-mimic

mimic: rgw: data/bilogs are trimmed when no peers are reading them

Reviewed-by: Casey Bodley <cbodley@redhat.com>

radosgw-admin: 'mdlog trim' loops until done

Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit 91eef6712ad29d6c15907f1663e6143fc4144dec)

Conflicts: store->time_log_trim() instead of svc.cls->timelog.trim()
src/rgw/rgw_metadata.cc no null_yield

radosgw-admin: 'datalog trim' takes shard-id and loops until done

Fixes: https://tracker.ceph.com/issues/41045
Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit b9af497b2ae445b858fbdf5b7c311770714a58da)

Conflicts: store->data_log instead of store->svc()->datalog_rados
src/rgw/rgw_admin.cc
src/rgw/rgw_bucket.cc no null_yield
src/rgw/services/svc_datalog_rados.cc (not present in luminous)
src/rgw/services/svc_datalog_rados.h

Merge pull request #30233 from smithfarm/wip-41129-mimic

mimic: qa: use hard_reset to reboot kclient

Reviewed-by: Venky Shankar <vshankar@redhat.com>

Merge pull request #30234 from smithfarm/wip-40444-mimic

mimic: mds: cleanup unneeded client_snap_caps when splitting snap inode

Reviewed-by: Venky Shankar <vshankar@redhat.com>

Merge pull request #30235 from smithfarm/wip-40844-mimic

mimic: mon/MDSMonitor: use stringstream instead of dout for mds repaired

Reviewed-by: Venky Shankar <vshankar@redhat.com>

Merge pull request #30236 from smithfarm/wip-40853-mimic

mimic: cephfs: test_volume_client: fix test_put_object_versioned()

Reviewed-by: Rishabh Dave <ridave@redhat.com>
Reviewed-by: Venky Shankar <vshankar@redhat.com>

Merge pull request #30238 from smithfarm/wip-40896-mimic

mimic: cephfs: ceph_volume_client: convert string to bytes object

Reviewed-by: Rishabh Dave <ridave@redhat.com>
Reviewed-by: Venky Shankar <vshankar@redhat.com>

Merge pull request #30239 from smithfarm/wip-40899-mimic

mimic: mds: evict an unresponsive client only when another client wants its caps

Reviewed-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Rishabh Dave <ridave@redhat.com>

Merge pull request #30240 from smithfarm/wip-41466-mimic

mimic: cephfs: mount.ceph: properly handle -o strictatime

Reviewed-by: Venky Shankar <vshankar@redhat.com>

Merge pull request #30241 from smithfarm/wip-41487-mimic

mimic: cephfs: client: return -EIO when sync file which unsafe reqs have been dropped

Reviewed-by: Venky Shankar <vshankar@redhat.com>

Merge pull request #30417 from pdvian/wip-41852-mimic

mimic: mds: make MDSIOContextBase delete itself when shutting down

Reviewed-by: Venky Shankar <vshankar@redhat.com>

Merge pull request #30443 from pdvian/wip-41856-mimic

mimic: cephfs: client: nfs-ganesha with cephfs client, removing dir reports not empty

Reviewed-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Jeff Layton <jlayton@redhat.com>

mgr/telemetry: Ignore crashes in report when module not enabled

The crash module is not guaranteerd to be enabled and this will render
the telemetry module useless:

  Error EINVAL: Traceback (most recent call last):
    File "/usr/lib/ceph/mgr/telemetry/module.py", line 325, in handle_command
      report = self.compile_report()
    File "/usr/lib/ceph/mgr/telemetry/module.py", line 291, in compile_report
      report['crashes'] = self.gather_crashinfo()
    File "/usr/lib/ceph/mgr/telemetry/module.py", line 214, in gather_crashinfo
      errno, crashids, err = self.remote('crash', 'do_ls', '', '')
    File "/usr/lib/ceph/mgr/mgr_module.py", line 845, in remote
      args, kwargs)
  ImportError: Module not found

We can safely ignore this error and just continue without the crash information.

Fixes: https://tracker.ceph.com/issues/42116
Signed-off-by: Wido den Hollander <wido@42on.com>

rbd-mirror: prevent restored trash images from being deleted after delay

The image deleter wasn't verifying whether or not an image was still in the trash
prior to deleting the image. This not only would incorrectly remove any restored
images but it will also leave the image id object and entry within the directory.

Fixes: https://tracker.ceph.com/issues/41780
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit f091a31d5252bba76598fffdc997275ca531621d)

Conflicts:
src/test/rbd_mirror/image_deleter/test_mock_TrashRemoveRequest.cc: removed state test cases
src/tools/rbd_mirror/image_deleter/TrashRemoveRequest.h/cc: removed state validation