qa/suites: clean up client-upgrade-octopus-pacific test

- fix .qa symlinks
- rename nautilus-client-x.yaml to octopus-client-x.yaml
- fix typos and remove stale comment
- remove 2-features permutation (it doesn't do anything useful as the
  workunit is run with RBD_FEATURES environment variable set and those
  features are explicitly passed to RBD.create and RBD.clone calls;
  the net effect is that the exact same job is run twice)

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>

Merge pull request #45282 from ceph/wip-yuri-octopus-clients

qa/tests: added upgrade-clients/client-upgrade-octopus-quincy tests

Reviewed-by: Mykola Golub <mgolub@suse.com>
Reviewed-by: Ilya Dryomov <idryomov@gmail.com>

qa/tests: added upgrade-clients/client-upgrade-octopus-quincy tests

Signed-off-by: Yuri Weinstein <yweinste@redhat.com>

Merge pull request #45261 from idryomov/wip-cmake-disable-dpdk-warnings-octopus

octopus: cmake: pass RTE_DEVEL_BUILD=n when building dpdk

Reviewed-by: Kefu Chai <tchaikov@gmail.com>

cmake: pass RTE_DEVEL_BUILD=n when building dpdk

ceph is still using the Makefile based building system for building
DPDK. and DPDK enables -Werror if RTE_DEVEL_BUILD is 'y' which is
enabled by default when the dpdk is built from a git repo.

but newer GCC is more picky than the older versions, to prevent
the possible FTBFS when we switch to newer GCC for building old
branches whose dpdk submodule might be include the changes addressing
those warnings. let's just disable this option.

the only effect of this option is to add -Werror to CFLAGS. but
the building warnings from DPDK is not our focus when developing
Ceph in the most cases. so it should be fine.

see also
https://github.com/ceph/dpdk/blob/eac901ce29be559b1bb5c5da33fe2bf5c0b4bfd6/doc/build-sdk-quick.txt#L18

Signed-off-by: Kefu Chai <tchaikov@gmail.com>
(cherry picked from commit 91a616b26e830e85962200d0bac86bee7e022892)

Conflicts:
cmake/modules/BuildDPDK.cmake [ commit d3c315703ae6 ("cmake:
  pass -Wunused-but-set-variable when building dpdk") not in
  octopus ]

Merge pull request #45169 from pponnuvel/wip-54382-octopus

octopus: rbd-mirror: make mirror properly detect pool replayer needs restart

Reviewed-by: Ilya Dryomov <idryomov@redhat.com>
Reviewed-by: Deepika Upadhyay <dupadhya@redhat.com>
Reviewed-by: Sunny Kumar <sunkumar@redhat.com>

Merge pull request #45147 from guits/wip-54400-octopus

octopus: ceph-volume: abort when passed devices have partitions

Merge pull request #44800 from kotreshhr/wip-53947-octopus

octopus: mgr/volumes: A few volumes plugin backport

Reviewed-by: Venky Shankar vshankar@redhat.com

Merge pull request #44624 from lxbsz/wip-53865

octopus: mds: directly return just after responding the link request

Reviewed-by: Jeff Layton <jlayton@redhat.com>

Merge pull request #44976 from vshankar/wip-54242

octopus: mds: ignore unknown client op when tracking op latency

Reviewed-by: Xiubo Li <xiubli@redhat.com>
Reviewed-by: Jeff Layton <jlayton@redhat.com>

Merge pull request #45206 from idryomov/wip-rbd-qemu-iotests-8stream-octopus

octopus: backport qemu-iotests fixup for centos stream 8

Reviewed-by: Deepika Upadhyay <dupadhya@redhat.com>

Merge pull request #44883 from guits/wip-54126-octopus

octopus: ceph-volume: fix error 'KeyError' with inventory

Merge pull request #44768 from guits/wip-54008-octopus

octopus: ceph-volume: fix tags dict output in `lvm list`

ceph-volume: abort when passed devices have partitions

ceph-volume doesn't prevent from using db and/or wal devices
with existing partitions on them.
This can lead to a data loss situation.

Fixes: https://tracker.ceph.com/issues/54376
Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
(cherry picked from commit 75c91a8c6f37a38d69d5da8b1e7d49d9c636230b)

Merge pull request #44806 from mkogan1/wip-52900-octopus

octopus rgw: under fips, set flag to allow md5 in select rgw ops

Reviewed-by: Matt Benjamin <mbenjamin@redhat.com>

Merge pull request #43656 from poulpreben/backport-object-lock-retain-date-iso8601

octopus: rgw: Dump Object Lock Retain Date as ISO 8601

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #45110 from ljflores/wip-54351-octopus

octopus: mgr/dashboard: dashboard turns telemetry off when configuring report

Merge pull request #45076 from chrisphoffman/wip-54297-octopus

octopus: cls/rbd: GroupSnapshotNamespace comparator violates ordering rules

Reviewed-by: Ilya Dryomov <idryomov@redhat.com>

Merge pull request #45071 from idryomov/wip-qemu-task-rbd-package-octopus

octopus: qa/tasks/qemu: make sure block-rbd.so is installed

Reviewed-by: Deepika Upadhyay <dupadhya@redhat.com>

Merge pull request #45019 from trociny/wip-47427-octopus

octopus: librbd: track complete async operation requests

Reviewed-by: Ilya Dryomov <idryomov@redhat.com>

Merge pull request #45006 from sunnyku/wip-54169-octopus

octopus: mgr/rbd_support: fix schedule remove

Reviewed-by: Ilya Dryomov <idryomov@redhat.com>

Merge pull request #45009 from idryomov/wip-rbd-help-positional-optional-octopus

octopus: rbd: mark optional positional arguments as such in help output

Reviewed-by: Sunny Kumar <sunkumar@redhat.com>
Reviewed-by: Mykola Golub <mgolub@mirantis.com>

Merge pull request #45004 from idryomov/wip-54128-octopus

octopus: krbd: return error when no initial monitor address found

Reviewed-by: Deepika Upadhyay <dupadhya@redhat.com>

Merge pull request #45001 from idryomov/wip-krbd-rxbounce-option-octopus

octopus: rbd: recognize rxbounce map option

Reviewed-by: Deepika Upadhyay <dupadhya@redhat.com>

Merge pull request #45000 from idryomov/wip-52522-octopus

octopus: librbd: fix use-after-free on ictx in list_descendants()

Reviewed-by: Mykola Golub <mgolub@mirantis.com>

Merge pull request #44992 from idryomov/wip-writesame-fua-octopus

octopus: librbd: honor FUA op flag for write_same() in write-around cache

Reviewed-by: Sunny Kumar <sunkumar@redhat.com>

15.2.16

workunits/rbd: remove lsb_release

The lsb_release utility brings in a lot of other dependencies. Remove
it from the RBD workunit script.

Signed-off-by: Ken Dreyer <kdreyer@redhat.com>
(cherry picked from commit 65f4d6eb3664a4cc6120031679b8368cbc02a4a5)

qa/workunits/rbd: use xenial version of qemu-iotests for centos stream 8

It is already used for centos 8(.3) and rhel 8(.4).

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
(cherry picked from commit df96b85b58822b0f1a327b9d543eba4170312dc5)

mgr/volumes: Fix clone uid/gid mismatch

This is the regression caused by commit 18b85c53a.
The 'set_attrs' function sets the uid/gid of the
group to the subvolume if uid/gid is not passed.
The attrs of the clone should match the source
snapshot. Hence, don't use the 'set_attrs'
function to set only the quota attrs for the
clone.

Fixes: https://tracker.ceph.com/issues/54066
Signed-off-by: Kotresh HR <khiremat@redhat.com>
(cherry picked from commit b3c9e6b50cf4264538e4c41d19e7ebb8b2900c3a)

qa: Add tests snapshot clone failure with quota

Fixes: https://tracker.ceph.com/issues/53848
Signed-off-by: Kotresh HR <khiremat@redhat.com>
(cherry picked from commit 7c0d31e52cea90e65152996024cabfa8a8fd299f)

Conflicts:
  qa/tasks/cephfs/test_volumes.py: The commit 103c7bdc70ca is
   not backported

mgr/volumes: Fix subvoume snapshot clone failure

Problem:
The subvolume snapshot clone fails if the quota on the source
has exceeded. Since the quota is not strictly enforced at the
byte range, this is a possibility.

Cause:
The quota on the clone is set prior to copying the data
from the source. Hence the quota mostly get enforced before
copying the entire data from the source resulting in the
clone failure.

Solution:
Enforce quota on the clone after the data is copied.

Fixes: https://tracker.ceph.com/issues/53848
Signed-off-by: Kotresh HR <khiremat@redhat.com>
(cherry picked from commit 18b85c53af36d89a8c53b40cfc44fe06816a9733)

mgr/volumes: Improve debugging, add clone failure logs

Fixes: https://tracker.ceph.com/issues/53848
Signed-off-by: Kotresh HR <khiremat@redhat.com>
(cherry picked from commit 29ad638773715c92a0c77f10731bd69167e4ce80)

mgr/volumes: use dedicated libcephfs handles for subvolume calls and async jobs

Fixes: http://tracker.ceph.com/issues/51271
Signed-off-by: Venky Shankar <vshankar@redhat.com>
(cherry picked from commit cb2883feac1a5c141a3d72120c2320f7a8ffdea8)

Conflicts:
  src/pybind/mgr/volumes/fs/async_cloner.py: The commit cf2a1ad65120 is
not backported
  src/pybind/mgr/volumes/fs/async_job.py: The commit cf2a1ad65120 is not
backported

mgr/volumes: Add config to insert delay at the beginning of the clone

Added the config 'delay_snapshot_clone' to insert delay at the beginning
of the clone to avoid races in tests. The default value is set to 0.

Fixes: https://tracker.ceph.com/issues/48231
Signed-off-by: Kotresh HR <khiremat@redhat.com>
(cherry picked from commit 7588f985054282d2cff7f3582e995584b1fd20f8)

Conflicts:
 qa/tasks/cephfs/test_volumes.py: Conflicts due to tests ordering
 src/pybind/mgr/volumes/fs/volume.py: The commit e308bf898955 is not
backported
 src/pybind/mgr/volumes/module.py: The commit f002c6ce4033 is not
backported

mgr_util: move is_stopping from VolumeClient to CephfsClient

Signed-off-by: Jan Fajerski <jfajerski@suse.com>
(cherry picked from commit 8c94c21d098ac31708dda66cd35ccb0da5d1fd75)

mgr_util: rename ConnectionPool -> CephfsConnectionPool

Signed-off-by: Jan Fajerski <jfajerski@suse.com>
(cherry picked from commit 51f6f64c123533bd2d33e3ec768af63a757a1d2a)

mgr_util: add CephfsClient implementation

This pulls parts of the VolumesClient implementation into mgr_util to
make the CephFS specific pieces available to other mgr modules. To
reduce code duplication the VolumeClient now extends the CephfsClient
class to add the volume specific methods.

Signed-off-by: Jan Fajerski <jfajerski@suse.com>
(cherry picked from commit a44de38b61d598fb0512ea48da0de4179d39b804)

src/pybind/mgr/mgr_util.py
src/pybind/mgr/tox.ini
src/pybind/mgr/volumes/fs/operations/volume.py
src/pybind/mgr/volumes/fs/volume.py
  Trivial conflicts because ofthe order of backports to octopus

rbd-mirror: make mirror properly detect pool replayer needs restart

When a PoolReplayer detects remote pool metadata change it
sets "stopping" flag expecting the Mirror will restart it.

Although setting "stopping" flag makes the PoolReplayer::run
thread to terminate, the thread's is_started function will still
return true until join is called (and reset the thread id).

This made impossible for the Mirror to detect (by calling
PoolReplayer::is_running) that the PoolReplayer needed restart.

Fixes: https://tracker.ceph.com/issues/54258
Signed-off-by: Mykola Golub <mgolub@suse.com>
(cherry picked from commit ad4a2990b87834fe4ae8c9111547d071aa6e75e5)

mgr/dashboard: Contact Info should be visible only when Ident channel is checked

Fixes:https://tracker.ceph.com/issues/54133
Signed-off-by: Sarthak0702 <sarthak.0702@gmail.com>
(cherry picked from commit 15211a6378a6fee9316f79ba0b27821891527c38)

 Conflicts:
src/pybind/mgr/dashboard/frontend/src/app/ceph/cluster/telemetry/telemetry.component.ts
- `this.loading` used in Octopus instead of `this.loadingReady()`

mgr/dashboard: telemetry activate: show ident fields when checked

Signed-off-by: Aaryan Porwal <aaryanporwal2233@gmail.com>
(cherry picked from commit ad5b3f200529fc0bc511ce99eed338afcaef6a62)

mgr/dashboard: dashboard turns telemetry off when configuring report

Signed-off-by: Sarthak0702 <sarthak.0702@gmail.com>
(cherry picked from commit 97c57adf8565756dbf24f3c46ed3916303903fb7)

Conflicts:
src/pybind/mgr/dashboard/frontend/src/app/ceph/cluster/telemetry/telemetry.component.ts
- `this.i18n()` was used in Octopus instead of `$localize`

cls/rbd: GroupSnapshotNamespace comparator violates ordering rules

For

  GroupSnapshotNamespace a(1, "group-1", "snap-2");
  GroupSnapshotNamespace b(1, "group-2", "snap-1");

both a < b and b < a evaluate to true.  This violates STL strict weak
ordering requirements which is a problem because GroupSnapshotNamespace
is used as a key in std::map (ictx->snap_ids at least), etc.

Fixes: https://tracker.ceph.com/issues/49792
Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
(cherry picked from commit 830e72ab9d66c8f5703ea27da5249b02dd16ccd0)

qa/suites/rbd: make sure block-rbd.so is installed

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
(cherry picked from commit 8f0fd0af3da8581c47dc916303615264714a0489)

qa/tasks/qemu: make sure block-rbd.so is installed

Fixes: https://tracker.ceph.com/issues/54286
Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
(cherry picked from commit 525ff61cfc8516b4d7bed6f819b00a0b6cb7be0a)

ceph-volume: fix error 'KeyError' with inventory

The tag ceph.cluster_name is always set at the end.
The only way it could be absent was if the osd prepare
has been interrupted between [1] and [2].

[1] https://github.com/ceph/ceph/blob/v14.2.11/src/ceph-volume/ceph_volume/devices/lvm/strategies/bluestore.py#L355-L387
[2] https://github.com/ceph/ceph/blob/v14.2.11/src/ceph-volume/ceph_volume/devices/lvm/prepare.py

Although the code received tremendous changes meantime
and this error shouldn't show up again, we need to handle
the case where this tag wouldn't have been set.

Fixes: https://tracker.ceph.com/issues/44356
Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
(cherry picked from commit 691660c42eaa568a754670e093c512aa041d1479)

Merge pull request #44986 from badone/wip-octopus-ceph-ansible-move-to-stream

octopus: qa/ceph-ansible: Move to Centos Stream

Reviewed-by: Yuri Weinstein <yweinste@redhat.com>

librbd: reset complete async request expiration time

Signed-off-by: Mykola Golub <mgolub@suse.com>
(cherry picked from commit 5c3ba95b4d1208868db1d00d1ba06ed086e1b268)

Conflicts:
src/librbd/ImageWatcher.cc (no quiesce requests in octopus,
            AsyncRequestId == operator is not supported, use !=)

Merge pull request #44791 from guits/wip-54023-octopus

octopus: ceph-volume: improve mpath devices support

Reviewed-by: Yuri Weinstein <yweinste@redhat.com>

ceph-volume: fix typo in tests

This fixes 2 typo in ceph-volume tests.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
(cherry picked from commit b07bd3e0e17021e0cf9773f916fad954f12254ed)

doc/ceph-volume: fix a typo

This fixes a typo in ceph-volume documentation.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
(cherry picked from commit 5d0a3cee5d7021dafd1e166e17946689b4bb90b7)

ceph-volume: add a test `test_mpath_device_is_device`

This test checks that Device.is_device() returns True for a mpath device.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
(cherry picked from commit 0280ff6df09bc26107bc97446e9d5c18fbc582e9)

ceph-volume: improve mpath devices support

ee8887f4c0ff4f91117f31b621b95c8d08019130 was intended for adding
mpath devices support in ceph-volume but it has missed the lvm batch scenario.
This also fixes the zapping of mpath devices prepared with `ceph-volume raw`

Fixes: https://tracker.ceph.com/issues/52908
Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
(cherry picked from commit 601ff7ed0a3ba5172b6bd886ca8ba2bd4d9e655a)

Merge pull request #44974 from guits/wip-54245-octopus

octopus: ceph-volume: honour osd_dmcrypt_key_size option

librbd: track complete async operation return code

Signed-off-by: Mykola Golub <mgolub@suse.com>
(cherry picked from commit 96bc20445afb0b2579a79b54a7854bc4a23f5b62)

Conflicts:
src/librbd/ImageWatcher.cc (no quiesce requests in octopus)

librbd: track complete async operation requests

to prevent duplicate maintenance operations due to RPC hiccups.

Fixes: https://tracker.ceph.com/issues/46803
Signed-off-by: Mykola Golub <mgolub@suse.com>
(cherry picked from commit 07fbc4b71df450655dec046c10e919dbfde989ba)

Conflicts:
src/librbd/ImageWatcher.cc (no quiesce requests in octopus)

rbd: mark optional positional arguments as such in help output

Currently at least five commands have optional positional arguments.

Overloading po::value<std::string>()->default_value("") for this
is a bit sneaky but nothing better fits into the existing Shell.cc
framework.

Note that strictly speaking "[<interval>] [<start-time>]" should be
"[<interval> [<start-time>]]" but we aren't doing that here because
"ceph" command doesn't do it either.

Fixes: https://tracker.ceph.com/issues/54191
Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
(cherry picked from commit cb0df397aae552adc80713ca0d59ed1ebfd3b1be)

rbd: ensure the help printer doesn't print past the end of the line

When long command names and long optional names are combined,
it's possible for the help text to be printed beyond the 80
character limit.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 51478deab22178d01db7fa8632ba72071b4dfc38)

qa/workunits/rbd: improve schedule add/remove cli test

This patch adds few tests to cover schedule add/remove with invalid
inputs.

Signed-off-by: Sunny Kumar <sunkumar@redhat.com>
(cherry picked from commit a9312d4777a82d8f2d8766a011f10952f84d3f27)

mgr/rbd_support: fix schedule remove

Issue:

If we provide a random string in the schedule remove
command the entire schedule at specified level gets
removed.

Fixes: https://tracker.ceph.com/issues/53250
Signed-off-by: Sunny Kumar <sunkumar@redhat.com>
(cherry picked from commit 1b62447071a900b9fa7d856617cb7db9e030f91e)

krbd: return error when no initial monitor address found

Since we filter monitor addresses based on ms_mode, check that at
least one address was found.

Otherwise, we mismatch arguments when calling sysfs/add_single_major
which emits a misleading error message to dmesg:

  libceph: resolve 'name=user1' (ret=-3): failed
  libceph: parse_ips bad ip 'name=user1,key=client.user1'

Fixes: https://tracker.ceph.com/issues/54128
Signed-off-by: Burt Holzman <burt@fnal.gov>
(cherry picked from commit 0076ffc86e043af7aedc127df8661eaf87fc1c58)

qa/suites/krbd: add legacy+rxbounce and crc+rxbounce coverage

For basic, rbd and rbd-nomount subsuites, replace legacy and crc
facets with "legacy or legacy+rxbounce" and "crc or crc+rxbounce"
facets (chosen at random).

For fsx, singleton and thrash subsuites, add legacy+rxbounce and
crc+rxbounce facets and drop prefer-crc facet.  The expected behaviour
of the latter depends on cluster configuration and should be tested
separately.

The total number of jobs remains the same.

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
(cherry picked from commit fbf8c1d68be60ab294719113edbd7f459a755c15)

qa: krbd rxbounce test

Lives in its own directory since ms_mode doesn't need to be permuted
here.

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
(cherry picked from commit 95d30b534ef65207168397dd25ca7213c8290568)

rbd: recognize rxbounce map option

Fixes: https://tracker.ceph.com/issues/54063
Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
(cherry picked from commit 8d2a456d7055cfb64e6bb9927187e2240b8c4d2a)

librbd: report correct error for ictx->state->close()

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
(cherry picked from commit 22903c3965b15245b464540ccfb3404bf45f0627)

librbd: fix use-after-free on ictx in list_descendants()

Ictx is deleted when "ictx->state->open()" and "ictx->state->close()"
fail, and then "lderr(ictx->cct)" crashes.

Fixes: https://tracker.ceph.com/issues/52522
Signed-off-by: Wang ShuaiChao <wangshuaich@chinatelecom.cn>
(cherry picked from commit fa5d61ee5144f67cba53d54d36013614183e53a3)

Merge pull request #44978 from ifed01/wip-ifed-clist-pend-bug-oct

octopus:  os/bluestore: list obj which equals to pend

Reviewed-by: Mykola Golub <mgolub@mirantis.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>

librbd: honor FUA op flag for write_same() in write-around cache

WriteAroundObjectDispatch::write_same() should pass op_flags through
to dispatch_io() so that it can bypass the cache if needed.

Fixes: https://tracker.ceph.com/issues/52956
Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
(cherry picked from commit 0dcea098cf4d51bed31d8646dc3b533514c08a72)

os/bluestore: list obj which equals to pend

otherwise we could have failures like

scrub : stat mismatch, got 3/4 objects, 1/2 clones, 3/4 dirty, 3/4 omap, 0/0 pinned, 0/0 hit_set_archive, 0/0 whiteouts, 49/56 bytes, 0/0 manifest objects, 0/0 hit_set_archive bytes."

where the numbers of scrubbed object, clones, dirty and omap are always
less than the total number of corresponding numbers, if the PG contains
object(s) whose hash happens to be 0xffffffff.

in this change, if the calculated hash of the upper bound is greater
than the maximum possible number represented by uint32_t, in addition to
setting the hash of the upper bound hobj to 0xffffffff, we also set the
nspace of hobj of the upper bound to "\xff", so that the upper bound
is greater than an hobj whose hash happens to be 0xfffffff. please note,
the nspace of "\xff" is not an ascii string, so it's not likely to be
less than a real-world nspace of an hobj.

with this new *greater* upper bound, we are able to include the previous
missing hobj when listing the objects in a PG. so the scrub won't be
annoyed when the number of objects does not match.

Fixes: https://tracker.ceph.com/issues/52705
Signed-off-by: Mykola Golub <mykola.golub@clyso.com>
Signed-off-by: Kefu Chai <tchaikov@gmail.com>
(cherry picked from commit ffab13bcd9006c1f961a24b8016df9d1fe06ba1d)

 Conflicts:
src/os/bluestore/BlueStore.cc
 - get_coll_range function signature alignment

Merge pull request #44614 from ifed01/wip-ifed-fix-ram-gridy-fsck-oct

octopus: os/bluestore: make shared blob fsck much less RAM-greedy.

Reviewed-by: Adam Kupczyk <akupczyk@redhat.com>

os/bluestore: use scope_guard to log latency

simpler this way, and avoid using `goto`.

Signed-off-by: Kefu Chai <tchaikov@gmail.com>
(cherry picked from commit 715a83822ebc1a3d102d1ec13323b69db0600719)

ceph-volume/activate: load the config from lv tag

When `ceph-volume lvm trigger` is called with an OSD where the tag
`ceph.cluster_name` is not 'ceph', it fails.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
(cherry picked from commit 5ac1ec65cb2a582b2ae550202cc9911f993943f2)

ceph-volume/tests: use centos/stream8 images

Since recent move from CentOS 8 to CentOS Stream 8, let's do the same here.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
(cherry picked from commit 2b793952bbac7973b97d245c282165daadeabb51)

ceph-volume/tests: add tests in util/encryption.py

this adds some unit tests in order to cover `luks_format()` and `luks_open()`
in `util/encryption.py`.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
(cherry picked from commit db48850745f218e08cf53ae2d8edf3428f2b4010)

ceph-volume: honour osd_dmcrypt_key_size option

ceph-volume doesn't honour osd_dmcrypt_key_size.
It means the default size is always applied.

It also changes the default value in `get_key_size_from_conf()`

From cryptsetup manpage:

> For XTS mode you can optionally set a key size of 512 bits with the -s option.

Using more than 512bits will end up with the following error message:

```
Key size in XTS mode must be 256 or 512 bits.
```

Fixes: https://tracker.ceph.com/issues/54006
Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
(cherry picked from commit 47c33179f9a15ae95cc1579a421be89378602656)

mds: ignore unknown client op when tracking op latency

Server::handle_client_request() ignores unknown client operation
by returning -ENOTSUPP, however, Server::perf_gather_op_latency()
aborts on unknown client op, thereby causing -ENOTSUPP to never
reach the client. ceph_abort() seems unnecessary here.

Note, we could have invoked Server::perf_gather_op_latency()
when the return value to client is not -ENOTSUPP, however,
a valid client operation *might* just return -ENOTSUPP in
some cases.

@mchangir ran into this with his getvxattr op changes (PR #42001).

Signed-off-by: Venky Shankar <vshankar@redhat.com>
(cherry picked from commit 2f4060b8c41004d10d9a64676ccd847f6e1304dd)

octopus: qa/ceph-ansible: Move to Centos Stream

Centos 8 is eol and its package repos no longer exist.

Signed-off-by: Brad Hubbard <bhubbard@redhat.com>

Merge pull request #44860 from aclamk/wip-53392-octopus

octopus: Fix data corruption in bluefs truncate()

Reviewed-by: Igor Fedotov <ifedotov@suse.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge pull request #44929 from adk3798/octopus-cephadm-qa-centos-8-stream

octopus: qa/suites/rados/cephadm: use centos 8.stream

Reviewed-by: Yuri Weinstein <yweinste@redhat.com>

Merge pull request #44864 from cbodley/wip-54089

octopus: qa: remove centos8 from supported distros

Reviewed-by: Neha Ojha <nojha@redhat.com>

qa/suites/rados/cephadm: remove centos 8.2, 8.3

Signed-off-by: Adam King <adking@redhat.com>

qa/suites/orch/cephadm: add 8.stream + container_tools

Signed-off-by: Sage Weil <sage@newdream.net>
Conflicts:
qa/suites/rados/cephadm/upgrade/1-start-distro/1-start-centos_8.stream_container-tools.yaml

qa/rgw: rgw/verify no longer pins centos 8.0

the symlink rgw/verify/centos_latest.yaml already selects centos

Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit 0fad609d4dca01335abda6c48ae2663a8fd15494)

qa/distros: remove duplicate centos_8.stream.yaml from supported

Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit 3b71b41190bbb0af5026babc82266541b6398e92)

qa/distros: centos_8.yaml is now a symlink to centos_8.stream.yaml

Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit 0f4e51f05f9b340fe6128b46ea4601ecf01625d2)

qa/distro/supported: add centos 8.stream

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit 577cbd162ed63bcee9f027776d72d569d9adf93b)

os/bluestore/bluefs: Fix data corruption in truncate()

It is possible to create condition in which a BlueFS contains file that is corrupted.
It can happen when BlueFS replay log is on device A and we just wrote to device B and truncated file.

Scenario:
1) write to file h1 on SLOW device
2) flush h1 (initiate transfer, but no fdatasync yet)
3) truncate h1
4) write to file h2 on DB
5) fsync h2 (forces replay log to be written, after fdatasync to DB)
6) poweroff

Fixes: https://tracker.ceph.com/issues/53129
Signed-off-by: Adam Kupczyk <akupczyk@redhat.com>
(cherry picked from commit 49b7b44b3b5c94ee401562e603999e2b3bd8f9a2)

os/objectstore/test: Add test for data corruption in file truncation

Test for https://tracker.ceph.com/issues/53129

Signed-off-by: Adam Kupczyk <akupczyk@redhat.com>
(cherry picked from commit 1f7771d4a77ebb271b939b3893d7607d964796f0)

Merge pull request #44726 from cfsnyder/wip-53704-octopus

octopus: osdc: add set_error in BufferHead, when split set_error to right

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Venky Shankar vshankar@redhat.com

rgw: under fips, set flag to allow md5 in select rgw ops - for review

the ovrrides for review and relevant md5 usage:
```
src/rgw/rgw_crypt.cc:975:      unsigned char key_hash_res[CEPH_CRYPTO_MD5_DIGESTSIZE];
    int rgw_s3_prepare_encrypt(...)
        crypt_http_responses["x-amz-server-side-encryption-customer-algorithm"] = "AES256";
        crypt_http_responses["x-amz-server-side-encryption-customer-key-MD5"] = std::string(keymd5);
                                                           ~~~~~~~~~~~~~~~~

src/rgw/rgw_crypt.cc:1225:    uint8_t key_hash_res[CEPH_CRYPTO_MD5_DIGESTSIZE];
    int rgw_s3_prepare_decrypt(...)
        crypt_http_responses["x-amz-server-side-encryption-customer-algorithm"] = "AES256";
        crypt_http_responses["x-amz-server-side-encryption-customer-key-MD5"] = keymd5;
                                                           ~~~~~~~~~~~~~~~~

src/rgw/rgw_keystone.cc:40:  unsigned char m[CEPH_CRYPTO_MD5_DIGESTSIZE];
        void TokenCache::add_admin(...)
  rgw_get_token_id(token.token.id, admin_token_id);
                                   ~~~~~~~~~~~~~~ md5
  add_locked(admin_token_id, token);

        void TokenCache::add_barbican(...)
  rgw_get_token_id(token.token.id, barbican_token_id);
                                   ~~~~~~~~~~~~~~~~~ md5
  add_locked(barbican_token_id, token);
```

Signed-off-by: Mark Kogan <mkogan@redhat.com>
(cherry picked from commit 551e0c8f38f3f646dbfb5fbfde51d3107ca90cc6)

rgw: under fips, set flag to allow md5 in select rgw ops

Fixes: https://tracker.ceph.com/issues/52900
Signed-off-by: Mark Kogan <mkogan@redhat.com>
(cherry picked from commit a5df0cfcbe9b04832095f1f8ef0ce9f178888efd)

Merge pull request #44728 from cfsnyder/wip-51826-octopus

octopus: qa/run-tox-mgr-dashboard: Do not write to /tmp/test_sanitize_password…

Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: kevinzs2048 <NOT@FOUND>
Reviewed-by: Yuri Weinstein <yweins@redhat.com>

Merge pull request #44544 from cfsnyder/wip-53660-octopus

octopus: mon: prevent new sessions during shutdown

Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge pull request #44720 from cfsnyder/wip-53495-octopus

octopus: mgr: fix locking for MetadataUpdate::finish

Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge pull request #44700 from cfsnyder/wip-53943-octopus

octopus: mon/OSDMonitor: avoid null dereference if stats are not available

Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge pull request #44753 from idryomov/wip-rbd-mirror-delprop-races-octopus

octopus: rbd-mirror: fix races in snapshot-based mirroring deletion propagation

Reviewed-by: Mykola Golub <mgolub@mirantis.com>
Reviewed-by: Arthur Outhenin-Chalandre <arthur.outhenin-chalandre@cern.ch>

Merge pull request #44741 from idryomov/wip-rbd-switch-arguments-fix-octopus

octopus: rbd: add missing switch arguments for recognition by get_command_spec()

Reviewed-by: Mykola Golub <mgolub@mirantis.com>

Merge pull request #44722 from cfsnyder/wip-53534-octopus

octopus: mon/MgrStatMonitor: do not spam subscribers (mgr) with service_map

Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge pull request #44724 from cfsnyder/wip-53609-octopus

octopus: os/bluestore: avoid premature onode release.

Reviewed-by: Igor Fedotov <ifedotov@suse.com>

Merge pull request #44730 from cfsnyder/wip-53850-octopus

octopus: rgwlc:  remove lc entry on bucket delete

Reviewed-by: Casey Bodley <cbodley@redhat.com>