crimson/seastore: improve metrics about onode and lba trees

Absolute value:
* tree_depth
By transaction source:
* tree_inserts_committed
* tree_inserts_invalidated
* tree_erases_committed
* tree_erase_invalidated

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>

Merge pull request #42885 from cyx1231st/wip-seastar-tree-metrics

crimson/seastore: add metrics about onode and lba trees

Reviewed-by: Samuel Just <sjust@redhat.com>

Merge pull request #42907 from rzarzynski/wip-crimson-smphack

 crimson/osd: implicitly append '--smp 1' when invoked without it

Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Chunmei Liu <chunmei.liu@intel.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

crimson/osd: implicitly append '--smp 1' when invoked without it

This commit is basically a hack supposed to fulfil the obligation
of crimson being a drop-in replacement for the classical OSD
we already made by packaging it under `/usr/bin/ceph-osd`.
The discussion whether the interface-exactness should be continued
or not is out of scope of the commit; it's supposed just to handle
the issue unveiled by the Rook integration effort: `crimson-osd`
is unable to `--mkfs` because Seastar, if not restricted by passing
`--smp N`, considers all CPU cores available in the system when
allocating resources. This leads to the following error:

```
ERROR 2021-08-24 14:17:32,105 [shard 5] seastar - Could not setup Async I/O: Resource temporarily unavailable. The most common cause is not enough request capacity in /proc/sys/fs/aio-max-nr. Try increasing that number or reducing the amount of logical CPUs available for your application
```

This hack will need to be dropped when integrating multi-reactor
support in crimson.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

Merge pull request #42910 from zdover23/doc-2021-08-25-doc-dev-config-verb-agreement-typo

doc/dev/config: fix verb disagreement

Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge pull request #42713 from ronen-fr/wip-ronenf-explicit-config

common: removing the explicit attribute from a public copy constructor

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

doc/dev/config: fix verb disagreement

This PR changes the following sentence:

   The other is to register an observer
   that will be called every time the
   relevant configuration values changes.

This sentence has been rewritten so that the
verb agrees with its subject, like this:

   The other is to register an observer
   that will be called every time the
   relevant configuration values change.

Signed-off-by: Zac Dover <zac.dover@gmail.com>

Merge pull request #42806 from zdover23/wip-doc-2021-08-17-rbd-nomad-3-of-x

doc: rewrite configure ceph-csi to "conf. nomad"

Reviewed-by: Etienne Menguy <etienne.menguy@croit.io>
Reviewed-by: Ilya Dryomov <idryomov@gmail.com>

Merge pull request #42845 from tchaikov/wip-51483

common/options: validate see-also

Reviewed-by: Deepika Upadhyay <dupadhya@redhat.com>

doc: rewrite configure ceph-csi to "conf. nomad"

This PR rewrites the sections
     - Configure Ceph-CSI
     - Configure Nomad

in the rbd-nomad.rst Chapter of the RBD
Guide.

Signed-off-by: Zac Dover <zac.dover@gmail.com>

Merge pull request #42857 from dsavineau/cephadm_tcmalloc_ceph_daemons_only

cephadm: Set tcmalloc env var for Ceph daemons

Reviewed-by: Sebastian Wagner <sewagner@redhat.com>

Merge pull request #42833 from oleeander/cephadm_openntpd

cephadm: check for openntpd.service as time sync service

Reviewed-by: Dimitri Savineau <dsavinea@redhat.com>
Reviewed-by: Sebastian Wagner <sewagner@redhat.com>

Merge pull request #42824 from pcuzner/cephadm-add-ports

cephadm:Add listening ports to gather-facts output

Reviewed-by: Adam King <adking@redhat.com>
Reviewed-by: Dimitri Savineau <dsavinea@redhat.com>

Merge pull request #42699 from sebastian-philipp/cephadm-duplicate-ips

cephadm: list-networks: Avoid duplicated IPs

Reviewed-by: Francesco Pantano <fpantano@redhat.com>
Reviewed-by: Michael Fritch <mfritch@suse.com>
Reviewed-by: Sage Weil <sage@newdream.net>

Merge pull request #42405 from Daniel-Pivonka/cephadm_rm_offline_host

mgr/cephadm: add ability to remove offline host

Reviewed-by: Adam King <adking@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #42805 from zdover23/wip-doc-2021-08-17-rbd-nomad-2-of-x

doc: improve "Create a Pool" in rbd-nomad.rst

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Ilya Dryomov <idryomov@gmail.com>

cmake: s/Python_EXECUTABLE/Python3_EXECUTABLE/

as FindPython3.cmake only sets Python3_EXECUTABLE for us, we should
stick with Python3_EXECUTABLE instead of Python_EXECUTABLE.

Signed-off-by: Kefu Chai <kchai@redhat.com>

common/options: validate see-also

y2c.py is like a compiler which translates .yaml to .cc and .h files,
it does not have access to all .yaml files. to validate the dangling
see-also issue, we need to do this with a "linker".

in this change, validate-options.py is introduced to check if any of
option name included by the see-also property is valid.

Fixes: https://tracker.ceph.com/issues/51483
Signed-off-by: Kefu Chai <kchai@redhat.com>

doc: improve "Create a Pool" in rbd-nomad.rst

This PR improves the English in the "Create
a Pool" section of the "RBD & Nomad Integration"
chapter of the RBD Guide.

Signed-off-by: Zac Dover <zac.dover@gmail.com>

Merge pull request #42826 from tchaikov/wip-test-fair-mutex

test/common/test_fair_mutex: increase NR_ROUNDS from 256 to 512

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #42881 from wjwithagen/wjw-fix-vstart-mgr

src/vstart.sh: put modules option for MGR in the [mgr] block.

Reviewed-by: Kefu Chai <tchaikov@gmail.com>

Merge pull request #42882 from cyx1231st/wip-seastar-cleanup

crimson/common: minor cleanups to futures

Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Chunmei Liu <chunmei.liu@intel.com>
Reviewed-by: Xuehan Xu <xxhdx1985126@gmail.com>
Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

Merge pull request #42802 from Rethan/fix-rbd-bench-unit

rbd: avoid overflow of ios and clarify io-size limit for bench

Reviewed-by: Ilya Dryomov <idryomov@gmail.com>

Merge pull request #42841 from xxhdx1985126/wip-52305

crimson/common: keep ref count of crimson::interruptible::interrupt_cond

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Yingxin Cheng <yingxin.cheng@intel.com>

rbd: avoid overflow of ios and clarify io-size limit for bench

When doing rbd bench, we record done ios to print progress, current it's unsigned.
Suppose we do a bench of io-size 512B and io-total 4T, that means a total number of
8G ios which causes an overflow.
And we don't support io-size greater than 4G, so change help message.

Signed-off-by: haoyixing <haoyixing@kuaishou.com>

crimson/interruptible_future: remove unused code

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>

crimson/common: remove unused futurize::apply()

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>

Merge PR #42371 into master

* refs/pull/42371/head:
mgr/volumes: Fix a race during clone cancel
mgr/volumes: Fail subvolume removal if it's in progress

Reviewed-by: Ramana Raja <rraja@redhat.com>
Reviewed-by: Venky Shankar <vshankar@redhat.com>

Merge pull request #42884 from ideepika/wip-remove-unused-submodules

.gitmodules: remove thrift submodule

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

Merge pull request #41335 from noonedeadpunk/doc/stretch_mon

doc/rados/operations/stretch-mode: Adjust description of mon.e placement

Reviewed-by: Greg Farnum <gfarnum@redhat.com>

Merge pull request #42053 from TRYTOBE8TME/wip-rgw-dpp-4

rgw: DPP addition 5

Reviewed-by: Ali Maredia <amaredia@redhat.com>

Merge pull request #42439 from ifed01/wip-ifed-more-errors-shared-blob-repair

os/bluestore: fix additional errors during missed shared blob repair.

Reviewed-by: Adam Kupczyk <akupczyk@redhat.com>

Merge pull request #41569 from MrFreezeex/rbd-mirror-snap-perf-dump

rbd-mirror: add perf counters to snapshot replayer

Merge pull request #42871 from tchaikov/wip-cmake-exclude-grafonnet-lib

cmake: exclude "grafonnet-lib" target from "all"

Reviewed-by: Nathan Cutler <ncutler@suse.com>
Reviewed-by: Patrick Seidensal <pseidensal@suse.com>
Reviewed-by: Deepika Upadhyay <dupadhya@redhat.com>
Reviewed-by: Aashish Sharma <aasharma@redhat.com>

Merge pull request #42434 from zenomri/wip-omri-rgw-tracing

[WIP] -  RGW tracing implementation

Merge pull request #41380 from Devp00l/wip-rgw-multisite-doc

doc: Improve RGW multisite documentation

Reviewed-by: Volker Theile <vtheile@suse.com>
Reviewed-by: Alex Settle <asettle@suse.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #42864 from aaryanporwal/vrt-doc

doc/dev: mgr/dashboard: Visual regression tests for ceph dashboard

Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
Reviewed-by: Pere Diaz Bou <pdiazbou@redhat.com>

src/vstart.sh: put modules option for MGR in the [mgr] block.

Signed-off-by: Willem Jan Withagen <wjw@digiware.nl>

Merge pull request #42739 from Matan-B/lua-install-packages-fix

rgw: fix install packages function

Merge pull request #42865 from rhcs-dashboard/fix-rgw-bucket-list-api

mgr/dashboard: stats=false not working when listing buckets

Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Pere Diaz Bou <pdiazbou@redhat.com>

Merge pull request #42051 from melissa-kun-li/asyncssh

mgr/cephadm: replace execnet and remoto with asyncssh

Reviewed-by: Adam King <adking@redhat.com>
Reviewed-by: Dimitri Savineau <dsavinea@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #42804 from zdover23/wip-doc-2021-08-17-rbd-nomad-1-of-x

doc: improve RBD "Block Devices and Nomad"

Reviewed-by: Etienne Menguy <etienne.menguy@croit.io>
Reviewed-by: Ilya Dryomov <idryomov@gmail.com>

Merge pull request #42879 from tchaikov/wip-seastar

seastar: pick up change silencing gcc warning

Reviewed-by: Yingxin Cheng <yingxin.cheng@intel.com>

mgr/dashboard: stats=false not working when listing buckets

Fixes: https://tracker.ceph.com/issues/51154
Signed-off-by: Avan Thakkar <athakkar@redhat.com>

crimson/common: all interruptible future invocations should have already had interrupt condition in place

Signed-off-by: Xuehan Xu <xxhdx1985126@gmail.com>

crimson/common: correct comment error about interruptible future

Signed-off-by: Xuehan Xu <xxhdx1985126@gmail.com>

test/crimson: add interruptible async unit test

Signed-off-by: Xuehan Xu <xxhdx1985126@gmail.com>

crimson/common: keep ref count of crimson::interruptible::interrupt_cond

Currently, interrupt conditionss are transfered between inner and outer continuation
chains via a tls interrupt_cond variable. This simple strategy leads to problem when it
comes to mixing normal future/continuation procedures and seastar::thread. When seastar::async()
is called, the reactor can directly invoke the passed functor and lead to two different
scenarios:

1.if a seastar::get/yield() inside the passed lambda, the interrupt_cond should be erased at the
end of the continuation execution when it is yielded back;
2.otherwise, the interrupt_cond should be not erased.

There can be so many possible sequences of yielding of several different fibers that we can hardly
judge at the end of the continuation execution whether there was a yielding during the current
execution, which means we can't be able to know whether the tls interrupt_cond should be erased.
There could be other scenarios where the current strategy fails. To end this kind of issues
once and for all, we involve the ref counting mechinary.

Fixes: https://tracker.ceph.com/issues/52305
Signed-off-by: Xuehan Xu <xxhdx1985126@gmail.com>

cephadm: Add test_command_list_networks

Signed-off-by: Sebastian Wagner <sewagner@redhat.com>

cephadm: test_networks: Move tests from test_cephadm

Signed-off-by: Sebastian Wagner <sewagner@redhat.com>

cephadm: list-networks: Avoid duplicated IPs

Fixes: https://tracker.ceph.com/issues/52083
Signed-off-by: Sebastian Wagner <sewagner@redhat.com>

cephadm: Fix TestCephVolume.test_noop

Fixes `PermissionError: [Errno 13] Permission denied: '/var/lib/ceph/760b...5a5fbb'`

Signed-off-by: Sebastian Wagner <sewagner@redhat.com>

doc/dev: mgr/dashboard: Visual regression tests for ceph dashboard

Fixes: https://tracker.ceph.com/issues/52370
Signed-off-by: Aaryan Porwal <aaryanporwal2233@gmail.com>

Merge pull request #41405 from ideepika/wip-rbd-update-feature-test

test/librbd: add unit tests for rbd update features

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Mykola Golub <mgolub@suse.com>

crimson/onode-staged-tree: suppress warnings about unused variable

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>

crimson/seastore: add metrics about onode and lba trees

* tree_depth
* tree_inserts_committed
* tree_inserts_invalidated
* tree_erases_committed
* tree_erase_invalidated

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>

crimson/common: trace TLS crimson::interruptible::interrupt_cond<InterruptCond> modifications

Signed-off-by: Xuehan Xu <xxhdx1985126@gmail.com>

crimson/errorator: remove unecessary template param

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>

Merge pull request #42532 from ivancich/wip-multisite-conflict-detection

rgw-multisite: metadata conflict not computed correctly

Reviewed-by: Casey Bodley <cbodley@redhat.com>

rgw-multisite: metadata conflict not computed correctly

The former logic with a conditional based on `++i == 0` would never
execute. So this uses a boolean to differentiate the first from other
iterations and tries to clarify the code through commenting and an
explicit declaration. Additionally a warning is eliminated by
initializing a variable.

Signed-off-by: J. Eric Ivancich <ivancich@redhat.com>

Merge pull request #42792 from tchaikov/wip-librbd-cleanups

librbd: trade a map<> for a plain array

Reviewed-by: Ilya Dryomov <idryomov@gmail.com>

seastar: pick up change silencing gcc warning

silences compiler warning from src/net/tls.cc

Signed-off-by: Kefu Chai <tchaikov@gmail.com>

Merge pull request #42855 from josephsawaya/rook-label

.github/labeler: add rook label to PRs related to Rook

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Sebastian Wagner <sewagner@redhat.com>

.gitmodules: remove thrift submodule

with 80e82686ebafe36fca6dfd21cb32e63ced94d5cd we now use thrift as a
distro based dependency, hence we no longer need it as a submodule.

Signed-off-by: Deepika Upadhyay <dupadhya@redhat.com>

mgr/cephadm: replace instances of remoto/execnet with asyncssh

Fixes: https://tracker.ceph.com/issues/44676
Signed-off-by: Melissa Li <li.melissa.kun@gmail.com>

mgr/cephadm: rewrite test_etc_ceph to apply to asyncssh

Fixes: https://tracker.ceph.com/issues/44676
Signed-off-by: Melissa Li <li.melissa.kun@gmail.com>

mgr/cephadm: remove test_stale_connections

remove test_stale_connections because it applies to remoto

Fixes: https://tracker.ceph.com/issues/44676
Signed-off-by: Melissa Li <li.melissa.kun@gmail.com>

mgr/cephadm: add test_offline, edit with_cephadm_module

add `event_loop` and `tkey` object to with_cephadm_module, and create MockEventLoopThread in fixtures.py to test async functions of ssh.py.
rewrite test_offline to be compatible with asyncssh

Fixes: https://tracker.ceph.com/issues/44676
Signed-off-by: Melissa Li <li.melissa.kun@gmail.com>

mgr/cephadm: simplify _deploy_cephadm_binary, remove _deploy_cephadm_binary_conn

_deploy_cephadm_binary can be simplified to call write_remote_file in ssh.py, and _deploy_cephadm_binary_conn is no longer needed

Fixes: https://tracker.ceph.com/issues/44676
Signed-off-by: Melissa Li <li.melissa.kun@gmail.com>

mgr/cephadm: use _remote_connection (ssh.py), _execute_command, _check_execute_command in _run_cephadm

remove _get_connection from module.py and _remote_connection in serve.py, replacing with _remote_connection in ssh.py.
also, replace remoto.process.check with _execute_command and _check_execute_command in ssh.py

Fixes: https://tracker.ceph.com/issues/44676
Signed-off-by: Melissa Li <li.melissa.kun@gmail.com>

mgr/cephadm: remove remotes.py, replace old _write_remote_file in serve.py with write_remote_file in ssh.py

remove remotes.py because it is specific to execnet/remoto.
_write_remote_file in ssh.py now fulfills the function of write_file in remotes.py and the old _write_remote_file in serve.py

Fixes: https://tracker.ceph.com/issues/44676
Signed-off-by: Melissa Li <li.melissa.kun@gmail.com>

mgr/cephadm: remove _executable_path from module.py

Fixes: https://tracker.ceph.com/issues/44676
Signed-off-by: Melissa Li <li.melissa.kun@gmail.com>

mgr/cephadm: make synchronous wrapper functions of the async functions in ssh.py

Fixes: https://tracker.ceph.com/issues/44676
Signed-off-by: Melissa Li <li.melissa.kun@gmail.com>

mgr/cephadm: create thread to start event loop for ssh.py, and return results of the async functions with get_result

The EventLoopThread class starts a thread and an event loop which runs forever. Coroutines are scheduled on the event loop by the `get_result` method which uses `run_coroutine_threadsafe` to return a concurrent.futures.Future, and ultimately the result with .result()

Fixes: https://tracker.ceph.com/issues/44676
Signed-off-by: Melissa Li <li.melissa.kun@gmail.com>

mgr/cephadm: move _reconfig_ssh, _reset_con, and _reset_cons to ssh.py

Fixes: https://tracker.ceph.com/issues/44676
Signed-off-by: Melissa Li <li.melissa.kun@gmail.com>

mgr/cephadm: create async function _write_remote_file to write files on remote host

_write_remote_file uses _check_execute_command in ssh.py which calls _execute_command which uses shlex quote. Thus, any commands with an int will need to be transformed into a str because shlex quote does not take int objects

Fixes: https://tracker.ceph.com/issues/44676
Signed-off-by: Melissa Li <li.melissa.kun@gmail.com>

mgr/cephadm: execute commands run over ssh via asyncssh

_execute_command will run commands over ssh using the asyncssh `run` method: https://asyncssh.readthedocs.io/en/latest/api.html#asyncssh.SSHClientConnection.run
_check_execute_command will check the output of _execute_command and raise OrchestratorError if command fails on the remote host.
All commands run over ssh are prepended with sudo in `_execute_command` and shell-escaped with shlex quote.
If the cached ssh connection is closed or broken, the connection object will be removed from the cache, added to the `offline_hosts`, and an OrchestratorError will be raised. On the next call, the connection object will attempt to be recreated.
Exceptions involving asyncssh methods should be handled otherwise errors like TypeError: __init__() missing 1 required positional argument: 'reason' could occur due to the asyncssh error interacting with `raise_if_exception`

Fixes: https://tracker.ceph.com/issues/44676
Signed-off-by: Melissa Li <li.melissa.kun@gmail.com>

mgr/cephadm: create and cache asyncssh connection objects, and handle asyncssh connection errors

Create asyncssh connection object in async `_remote_connection` function and cache in `self.cons`
Create a handler for asyncssh log redirection and output ssh log if a connection error occurs
Disable asyncssh logger from propagating because the asyncssh info messages are verbose

Fixes: https://tracker.ceph.com/issues/44676
Signed-off-by: Melissa Li <li.melissa.kun@gmail.com>

cmake: exclude "grafonnet-lib" target from "all"

so we don't build this target when running "make", and hence avoid
accessing the internet in a building envronment where the internest
access is not allowed.

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #42813 from wjwithagen/wjw-fix-hexdump-output

blk: start 1st line of hexdump() on a new line

Reviewed-by: Kefu Chai <tchaikov@gmail.com>

Merge pull request #42856 from dang/wip-dang-zipper-manifest

RGW - Zipper - Remove RGWManifest from API

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #41522 from Dreamyu20xx/wip-50967

rgw: Get the correct initial value(pos) in raw_obj_to_obj function when zoneid contains ‘_ ’character.

Reviewed-by: J. Eric Ivancich <ivancich@redhat.com>

Merge pull request #42723 from liewegas/fix-rgw-warning-2

rgw: default auth_client_required=cephx

Reviewed-by: Matt Benjamin <mbenjamin@redhat.com>
Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #42803 from joke-lee/sts-memory-leak

rgw: fix sts memory leak

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #42683 from cbodley/wip-52027

rgw: change order of xml elements in ListRoles response

Reviewed-by: Pritha Srivastava <prsrivas@redhat.com>

Merge pull request #42740 from cbodley/wip-52128

cls/cmpomap: empty values are 0 in U64 comparisons

Reviewed-by: Adam C. Emerson <aemerson@redhat.com>
Reviewed-by: Yuval Lifshitz <ylifshit@redhat.com>

.github/labeler: add rook label to PRs related to Rook

To make it easier to look for PRs directly related to the Rook orchestrator.

Signed-off-by: Joseph Sawaya <jsawaya@redhat.com>

Merge pull request #42863 from sebastian-philipp/centos_8.2_container_tools_3.0.yaml

qa/distro: Add centos_8.2_container_tools_3.0.yaml

Reviewed-by: Patrick Seidensal <pseidensal@suse.com>

qa/distro: Add centos_8.2_container_tools_3.0.yaml

Let's avoid latest kubic stable

Fixes: https://tracker.ceph.com/issues/52279
Signed-off-by: Sebastian Wagner <sewagner@redhat.com>

Merge pull request #42842 from ideepika/wip-werror-testing

run-make-check.sh: fix run-make-check.sh skipping cmake options

Signed-off-by: Deepika Upadhyay <dupadhya@redhat.com>

blk: start 1st line of hexdump() on a new line

Otherwise the fist line looks rather strange

```
2021-08-19T23:42:33.604+0200 4f60700 40 bdev:kd:965(0x4fab180 /usr/local/src/wip.bluestore-test/build/dev/osd0/block.db) data: 00000000  56 f9 b8 f8 1c 00 01 01  1a 6c 65 76 65 6c 64 62  |V........leveldb|
00000010  2e 42 79 74 65 77 69 73  65 43 6f 6d 70 61 72 61  |.BytewiseCompara|
00000020  74 6f 72 98 af 58 a6 02  00 01 02 00 d2 c7 3c 95  |tor..X........<.|
00000030  06 00 01 09 00 03 04 04  00 6b 93 6d c5 2b 00 01  |.........k.m.+..|
00000040  01 1a 6c 65 76 65 6c 64  62 2e 42 79 74 65 77 69  |..leveldb.Bytewi|
.....
```

versus new:
```
2021-08-19T23:42:33.604+0200 4f60700 40 bdev:kd:965(0x4fab180 /usr/local/src/wip.bluestore-test/build/dev/osd0/block.db) data:
00000000  56 f9 b8 f8 1c 00 01 01  1a 6c 65 76 65 6c 64 62  |V........leveldb|
00000010  2e 42 79 74 65 77 69 73  65 43 6f 6d 70 61 72 61  |.BytewiseCompara|
00000020  74 6f 72 98 af 58 a6 02  00 01 02 00 d2 c7 3c 95  |tor..X........<.|
00000030  06 00 01 09 00 03 04 04  00 6b 93 6d c5 2b 00 01  |.........k.m.+..|
00000040  01 1a 6c 65 76 65 6c 64  62 2e 42 79 74 65 77 69  |..leveldb.Bytewi|
.....
```

Signed-off-by: Willem Jan Withagen <wjw@digiware.nl>

Merge pull request #42852 from sebastian-philipp/centos_8.3_container_tools_3.0.yaml

qa/cephadm: centos_8.3_container_tools_3.0.yaml

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #41950 from zenomri/rgw-crypt-dpp

src/rgw : ldout and lderr calls were replaced by ldpp_dout calls

Reviewed-by: Ali Maredia <amaredia@redhat.com>

cephadm: Set tcmalloc env var for Ceph daemons

We don't need to set the TCMALLOC_MAX_TOTAL_THREAD_CACHE_BYTES environment
variable for non Ceph daemons.

Fixes: https://tracker.ceph.com/issues/52334
Signed-off-by: Dimitri Savineau <dsavinea@redhat.com>

RGW - Zipper - Remove StatOp

As part of removing RGWObjManifest from the Zipper API, we need to
remove StatOp.  It's only used by orphan list, which is part of the
RadosStore.

Signed-off-by: Daniel Gryniewicz <dang@redhat.com>

RGW - Zipper - Remove WriteOp

As part of removing RGWObjManifest from the Zipper API, we need to remove
WriteOp.  Fortunately, with the multipart upload changes, it's no longer
needed outside the RadosStore.

Signed-off-by: Daniel Gryniewicz <dang@redhat.com>

Merge pull request #42851 from dang/wip-dang-zipper-raw_obj

RGW - Zipper remove unused files

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #42834 from josephsawaya/rook-orch-qa

qa/tasks/rook: add `ceph orch device ls` command to the rook qa task

Merge pull request #42757 from josephsawaya/wip-mgr-rook-osd-creation

mgr/rook: Rook orchestrator OSD creation using `ceph orch apply osd`

qa/cephadm: centos_8.3_container_tools_3.0.yaml

Let's avoid latest stable

Fixes: https://tracker.ceph.com/issues/52279
Signed-off-by: Sebastian Wagner <sewagner@redhat.com>