test/crimson/cbt: increase stddev variance to 2.0

as per Mark requests. we've being observing

```
03:55:21 - WARNING  - cbt      - prefill/incerta04.front.sepia.ceph.com/1: iops_stddev: (or (less) (near 0.50)):: 434.037/69.2122  => rejected
...
04:37:52 - WARNING  - cbt      - rand/incerta04.front.sepia.ceph.com/0: iops_stddev: (or (less) (near 0.50)):: 3830.51/2473.17  => rejected
...
04:37:52 - WARNING  - cbt      - rand/incerta04.front.sepia.ceph.com/1: iops_avg: (or (greater) (near 0.05)):: 53916.0/57375.0  => rejected
```

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #34813 from rzarzynski/wip-osd-fix-typo-pglog

crimson/osd, osd: fix typos in PG Log-related comments

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #34791 from rzarzynski/wip-crimson-backfill-pgtemp-ref

crimson/osd: fix missed ref in ShardServices::send_pg_temp().

Reviewed-by: Kefu Chai <kchai@redhat.com>

osd: s/log.bottom/log.tail/ in osd_types.h.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

osd: fix typo in PGLog.cc.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

Merge pull request #34753 from trociny/wip-44938-fixup

qa/workunits/rbd: retry the addition of a mirror pool peer

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #34699 from yison/rwl-image-writeback-cache-seq5

rbd/cache: Replicated Write Log core codes - aio_read

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

crimson/osd: fix missed ref in ShardServices::send_pg_temp().

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

qa/workunits/rbd: fix peer_add helper

We need to temporary disable "exit on error" mode so it does not
abort when `rbd mirror pool peer add` returns "already exists"
error code.

Signed-off-by: Mykola Golub <mgolub@suse.com>

Merge pull request #34700 from lixiaoy1/early_finish

librbd: Return read request early

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #34285 from tspmelo/wip-cypress

Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Stephan Müller <smueller@suse.com>

Merge pull request #30024 from vumrao/wip-vumrao-41061

radosgw-admin: add support for --bucket-id in bucket stats command

Reviewed-by: Casey Bodley <cbodley@redhat.com>

mgr/dashboard: E2E: Add navigateEdit method

This method should be use to navigate to the edit page.

Signed-off-by: Tiago Melo <tmelo@suse.com>

mgr/dashboard: Replace Protractor with Cypress

Fixes: https://tracker.ceph.com/issues/44812
Signed-off-by: Tiago Melo <tmelo@suse.com>

Merge pull request #34429 from xxhdx1985126/wip-crimson-recovery-pr

crimson: pglog based recovery ---- part 1

Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #34656 from rhcs-dashboard/44935-custom-cherryPy-header

Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Nizamudeen <nia@redhat.com>
Reviewed-by: Stephan Müller <smueller@suse.com>
Reviewed-by: Tatjana Dehler <tdehler@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>

Merge pull request #34748 from neha-ojha/wip-jewel-cleanup

qa/suites/rados: jewel cleanup

Reviewed-by: Brad Hubbard <bhubbard@redhat.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #34602 from neha-ojha/wip-45075

qa/standalone/scrub/osd-scrub-repair.sh: fix race in TEST_auto_repair_bluestore_failed

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #34595 from rishabh-d-dave/fs-move-run_shell

qa/cephfs: move run_shell to mount.py and add methods for negative testing

Reviewed-by: Jeff Layton <jlayton@redhat.com>

librbd: Return read request early

Finish read requests if the total length is 0.

Signed-off-by: Li, Xiaoyan <xiaoyan.li@intel.com>

Merge pull request #34705 from bk201/wip-45149

mgr/dashboard: fixing RBD purge error in backend

Reviewed-by: Tiago Melo <tmelo@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>
Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Laura Paduano <lpaduano@suse.com>

Merge pull request #34752 from tchaikov/fix-test-perf-counters-mgr-get

mgr/dashboard: fix API tests to be py3 compatible

Reviewed-by: Xiubo Li <xiubli@redhat.com>

Merge pull request #34292 from cbodley/wip-bufferlist-comparison

bufferlist: use iterators for comparison operators

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #34090 from trociny/wip-ceph-erasure-code-tool

tools/erasure-code: new tool to encode/decode files

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #34683 from ifed01/wip-ifed-fix-freelist-label

os/bluestore: fix freelist's meta usage when bdev label is unsupported

Reviewed-by: Adam Kupczyk <akucpzyk@redhat.com>

Merge pull request #34588 from ifed01/wip-ifed-lower-spinner-mas

os/bluestore:  set bluestore_min_alloc_size to 4K for spinners

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Mark Nelson <mnelson@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge pull request #34006 from aclamk/wip-rocksdb-sharding

kv/RocksDBStore,os/blustore: shard rocksdb database using column families

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #33790 from bstillwell/add_conf_option_zstd_compression_level

compressor: Add a config option to specify Zstd compression level

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #34635 from tchaikov/wip-mgr-py3

mgr: drop python2 support

Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>

librbd: add aio_read test cases

Signed-off-by: Peterson, Scott <scott.d.peterson@intel.com>
Signed-off-by: Li, Xiaoyan <xiaoyan.li@intel.com>
Signed-off-by: Lu, Yuan <yuan.y.lu@intel.com>
Signed-off-by: Chamarthy, Mahati <mahati.chamarthy@intel.com>

librbd: add aio_read

Signed-off-by: Peterson, Scott <scott.d.peterson@intel.com>
Signed-off-by: Li, Xiaoyan <xiaoyan.li@intel.com>
Signed-off-by: Lu, Yuan <yuan.y.lu@intel.com>
Signed-off-by: Chamarthy, Mahati <mahati.chamarthy@intel.com>

crimson: prevent PG::schedule_event_after from creating daggling future

Signed-off-by: Xuehan Xu <xxhdx1985126@163.com>

crimson/osdmap_gate: sort the blockers in ascending order of map epoch

When the osdmap_gate got a new map, it's supposed to release blocked ops on
all previous maps section of which is represented by [waiting_peering.begin(),
waiting_peering.upper_bound(epoch)]. So it's essentail that waiting_peering is
ordered in ascending order of the map epoch

Signed-off-by: Xuehan Xu <xxhdx1985126@163.com>

crimson: keep track of clean_regions/dirty_regions of modifications

Signed-off-by: Xuehan Xu <xxhdx1985126@163.com>

crimson: add osd interface for recovery messages

Signed-off-by: Xuehan Xu <xxhdx1985126@163.com>

crimson: add recovery operations oriented request ---- "RecoverySubRequest"

Signed-off-by: Xuehan Xu <xxhdx1985126@163.com>

crimson: add pg log based recovery machinery

Signed-off-by: Xuehan Xu <xxhdx1985126@163.com>

crimson/os: add readv method to futurized_store

Signed-off-by: Xuehan Xu <xxhdx1985126@163.com>

crimson: add background recovery request and necessary interfaces for recover/backfill

Signed-off-by: Samuel Just <sjust@redhat.com>
Signed-off-by: Xuehan Xu <xxhdx1985126@163.com>

osd: add some clean ups

Signed-off-by: Samuel Just <sjust@redhat.com>

crimson/osd/osd_operation: add support for AggregateBlocker

Signed-off-by: Samuel Just <sjust@redhat.com>

crimson/osd: introduce scheduler implementations and operation throttler

Signed-off-by: Samuel Just <sjust@redhat.com>
Signed-off-by: Xuehan Xu <xxhdx1985126@163.com>

crimson/osd: wire up local and remote async reservers

Signed-off-by: Samuel Just <sjust@redhat.com>

common/AsyncReserver: template out the finisher type

Signed-off-by: Samuel Just <sjust@redhat.com>

osd/PeeringState: eliminate unused cct from PGPool

Signed-off-by: Samuel Just <sjust@redhat.com>

osd/PrimaryLogPG: move earliest_backfill to PeeringState

We'll want this in crimson as well.

Signed-off-by: Samuel Just <sjust@redhat.com>

librbd: add WriteLogMap test cases

Signed-off-by: Peterson, Scott <scott.d.peterson@intel.com>
Signed-off-by: Li, Xiaoyan <xiaoyan.li@intel.com>
Signed-off-by: Lu, Yuan <yuan.y.lu@intel.com>
Signed-off-by: Chamarthy, Mahati <mahati.chamarthy@intel.com>

librbd: add LogMap class

Signed-off-by: Peterson, Scott <scott.d.peterson@intel.com>
Signed-off-by: Li, Xiaoyan <xiaoyan.li@intel.com>
Signed-off-by: Lu, Yuan <yuan.y.lu@intel.com>
Signed-off-by: Chamarthy, Mahati <mahati.chamarthy@intel.com>

librbd: add ReadRequest to handle read

Signed-off-by: Peterson, Scott <scott.d.peterson@intel.com>
Signed-off-by: Li, Xiaoyan <xiaoyan.li@intel.com>
Signed-off-by: Lu, Yuan <yuan.y.lu@intel.com>
Signed-off-by: Chamarthy, Mahati <mahati.chamarthy@intel.com>

Merge pull request #34732 from lxbsz/fs_fix_new

qa: misc fixes to run for cephfs qa test

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #31912 from zjcmszh/cmpext

pybind/rados: add Ioctx::cmpext()

Reviewed-by: Deepika Upadhyay <dupadhya@redhat.com>

test/pybind/test_rados.py: test Ioctx::cmpext()
Signed-off-by: Zhang Jiao <zhangjiao@cmss.chinamobile.com>

pybind/rados: add Ioctx:cmpext()
Signed-off-by: Zhang Jiao <zhangjiao@cmss.chinamobile.com>

Merge pull request #34578 from lxbsz/volume_client_py3

qa/volume_client: be python3 compatible

Reviewed-by: Ramana Raja <rraja@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

qa/tasks/cephfs: switch to StringIO to fix byte-like errors

Signed-off-by: Xiubo Li <xiubli@redhat.com>

qa/cephfs: switch to mount_wait() to avoid possible umounted assert

For the fuse mount, it sometime will need to wait to a moment to let
the client get ready.

Signed-off-by: Xiubo Li <xiubli@redhat.com>

qa/tasks/cephfs/fuse_mount: fix possible chmod 1777 error

INFO:teuthology.orchestra.run.smithi13a2:> (cd /home/ubuntu/cephtest && exec sudo chmod 1777 /home/ubuntu/cephtest/mnt.2)
INFO:teuthology.orchestra.run.smithi132.stderr:chmod: changing permissions of '/home/ubuntu/cephtest/mnt.2': Permission denied
DEBUG:teuthology.orchestra.run:got remote process result: 1

Here just wait and rety for 10 times.

Signed-off-by: Xiubo Li <xiubli@redhat.com>

qa/tasks/vstart_runner: remove the stdin str type check

For the "ceph-dencoder" it will use the byte type data.

Signed-off-by: Xiubo Li <xiubli@redhat.com>

qa/cephfs: fix AssertionError: CommandFailedError not raised under py3

Signed-off-by: Xiubo Li <xiubli@redhat.com>

qa/workunits/rbd: retry the addition of a mirror pool peer

fb4311f5 has fixed this for setup, but "remove mirroring pool"
test needs fixing too.

Fixes: https://tracker.ceph.com/issues/44938
Signed-off-by: Mykola Golub <mgolub@suse.com>

Merge pull request #34697 from dillaman/wip-44596

rbd-mirror: skip snapshot image-sync for newly created/cloned images

Reviewed-by: Mykola Golub <mgolub@suse.com>

Merge pull request #34751 from tchaikov/wip-doc-fix

doc/releases/mimic.rst: add title underline

Reviewed-by: Brad Hubbard <bhubbard@redhat.com>

Merge pull request #34741 from tchaikov/wip-teuthology-cronjobs-mimic

qa/tests: pass "-t 2" to teuthology-suite when testing mimic

Reviewed-by: Yuri Weinstein <yweinste@redhat.com>

mgr/dashboard: fix API tests

* test_perf_counters_mgr_get
* test_selftest_cluster_log

Fixes: https://tracker.ceph.com/issues/45170
Fixes: https://tracker.ceph.com/issues/45246
Signed-off-by: Alfonso Martínez <almartin@redhat.com>

qa/tasks/vstart_runner.py: use StringIO for capturing text

to be consistent with 8bfe977854686149c74de3ebaed7b571d43404dd

Fixes: https://tracker.ceph.com/issues/45246
Signed-off-by: Kefu Chai <kchai@redhat.com>

doc/releases/general.rst: add mising link targets

they are referenced by 60c825486916d0b0b43248a0966d2ecb90f352df

Signed-off-by: Kefu Chai <kchai@redhat.com>

doc/releases: silence sphinx warnings

* mimic.rst: add title underline

to address sphinx warning like:
```
WARNING: Title underline too short.
```

* octopus.rst: use consistent underline

to address following sphinx warning:
```
doc/releases/octopus.rst:49: WARNING: Title level inconsistent:
```

Signed-off-by: Kefu Chai <kchai@redhat.com>

qa/suites/rados/thrash-old-clients/1-install: remove hammer

Signed-off-by: Neha Ojha <nojha@redhat.com>

qa/suites/rados/thrash/workloads: use default mon osd initial require min compat client

Signed-off-by: Neha Ojha <nojha@redhat.com>

qa/suites/rados/thrash-old-clients/1-install: remove jewel

Since we've changed mon_osd_initial_require_min_compat_client to
luminous in 986c271b756a6b3f472ec6848fc0d6680c45cf5b, we can remove jewel from
mix.

Fixes: https://tracker.ceph.com/issues/45242
Signed-off-by: Neha Ojha <nojha@redhat.com>

Merge pull request #34698 from matthewoliver/cephadm_check_pool_exists

cephadm: Check for pool existence for iscsi And NFS

Reviewed-by: Michael Fritch <mfritch@suse.com>
Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>

Merge pull request #34676 from sebastian-philipp/cephadm-inventory-class

mgr/cephadm: add Inventory class

Merge pull request #34695 from mgfritch/cephadm-iscsi-container-image

mgr/cephadm: allow config for an iscsi `container_image`

Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>

Merge pull request #34718 from mgfritch/cephadm-mgr-is-available

cephadm: catch Exception during is_mgr_available

Reviewed-by: Joshua Schmid <jschmid@suse.de>

Merge pull request #34619 from sebastian-philipp/qa-adopt-specific-commit

qa/cephadm: make test_adoption checkout a specific commit

Merge pull request #34693 from sebastian-philipp/cephadm-warn-master-improvement

cephadm: improve warn message

Reviewed-by: Michael Fritch <mfritch@suse.com>

Merge pull request #34675 from sebastian-philipp/cephadm-service-rm-feedback

mgr/cephadm: remove_service: Feedback, if spec not found

Reviewed-by: Joshua Schmid <jschmid@suse.de>

qa/tests: pass "-t 2" to teuthology-suite when testing mimic

it's an oversight in 21dacc823be184718584c5fd3b647e87b3eeba42

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #34142 from rhcs-dashboard/wip-44667-fix-reload-issues

Reviewed-by: Tiago Melo <tmelo@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>

Merge pull request #34730 from theanalyst/docs/release-updates-04-20

docs: release updates for nautilus,mimic,octopus

Reviewed-By: Nathan Cutler <ncutler@suse.com>

os/bluestore: Enable sharding by default

Signed-off-by: Adam Kupczyk <akupczyk@redhat.com>

Merge pull request #34673 from sebastian-philipp/cephadm-man

doc/man: add cephadm

Reviewed-by: Alexandra Settle <asettle@suse.com>
Reviewed-by: Michael Fritch <mfritch@suse.com>
Reviewed-by: Zac Dover <zac.dover@gmail.com>

Merge pull request #34725 from lxbsz/_type_equality_funcs

qa/tasks/cephfs/test_scrub: fix self.assertEqual no attribute '_type_equality_funcs'

Reviewed-by: Kefu Chai <kchai@redhat.com>

doc: update release.yml with octopus, nautilus & mimic

Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>

doc: releases: add release notes for v14.2.9

Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>

Merge pull request #34726 from lxbsz/dict_key_error

qa/tasks/cephfs/cephfs_test_case: switch 'dict_keys' object to list

Reviewed-by: Kefu Chai <kchai@redhat.com>

doc: release notes for v13.2.10 mimic

Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>

doc: releases: release notes for v13.2.9 mimic

Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>

doc: releases: v15.2.1 release notes

Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>

qa/tasks/cephfs/test_volumes: to make sure the size is int type

Fix "Error EINVAL: invalid size option '4915200.0'".

In pytho2 int/2 will get a int type result, but in python3 it maybe
a float type.

Fixes: https://tracker.ceph.com/issues/45247
Signed-off-by: Xiubo Li <xiubli@redhat.com>

qa/tasks/cephfs/mount: to make sure that the count/seek are int type

To fix "stderr:dd: invalid number: '5.0'".

Fixes: https://tracker.ceph.com/issues/45247
Signed-off-by: Xiubo Li <xiubli@redhat.com>

qa/tasks/cephfs/test_scrub: fix self.assertEqual no attribute '_type_equality_funcs'

Signed-off-by: Xiubo Li <xiubli@redhat.com>

qa/tasks/cephfs/cephfs_test_case: switch 'dict_keys' object to list

Fix the "TypeError: 'dict_keys' object does not support indexing" under
py3.

Signed-off-by: Xiubo Li <xiubli@redhat.com>

Merge pull request #33769 from lxbsz/qa_vstart_runner_os

qa/vstart_runner.py: add os and arch properties support

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

qa/tasks/vstart_runner: make remote.sh return str as default

To address the issue in https://github.com/ceph/teuthology/pull/1459.

Signed-off-by: Xiubo Li <xiubli@redhat.com>

qa/vstart_runner.py: add os and arch properties support

Fixes: https://tracker.ceph.com/issues/44456
Signed-off-by: Xiubo Li <xiubli@redhat.com>

Merge pull request #34685 from lxbsz/vstart_runner

qa/tasks/cephfs/mount: switch to StringIO to fix TypeErrors

Reviewed-by: Kiefer Chang <kiefer.chang@suse.com>
Reviewed-by: Laura Paduano <lpaduano@suse.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

qa/tasks/cephfs/mount: switch to StringIO to fix TypeErrors

TypeError: cannot use a string pattern on a bytes-like object

Fixes: https://tracker.ceph.com/issues/45175
Signed-off-by: Xiubo Li <xiubli@redhat.com>

qa/vstart_runner: remove the debug info

Fixes: https://tracker.ceph.com/issues/45175
Signed-off-by: Xiubo Li <xiubli@redhat.com>

cephadm: Check for pool existance for iscsi And NFS

Currently both iscsi and NFS require pools to be specified when they are
deployed. However, we don't actaully check these pools exist. Leading to
broken containers.

This patch uses the rados client that is part of the mgrmodule to check
that the specified pool exists. As we need to check in 2 different
daemons, a helper method:

   def _check_pool_exists(self, pool, service_name):

was added to `cephadm/module.py`.

Fixes: https://tracker.ceph.com/issues/45161
Signed-off-by: Matthew Oliver <moliver@suse.com>