ceph-volume: abort when passed devices have partitions

ceph-volume doesn't prevent from using db and/or wal devices
with existing partitions on them.
This can lead to a data loss situation.

Fixes: https://tracker.ceph.com/issues/54376
Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
(cherry picked from commit 75c91a8c6f37a38d69d5da8b1e7d49d9c636230b)

Merge pull request #44806 from mkogan1/wip-52900-octopus

octopus rgw: under fips, set flag to allow md5 in select rgw ops

Reviewed-by: Matt Benjamin <mbenjamin@redhat.com>

Merge pull request #43656 from poulpreben/backport-object-lock-retain-date-iso8601

octopus: rgw: Dump Object Lock Retain Date as ISO 8601

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #45110 from ljflores/wip-54351-octopus

octopus: mgr/dashboard: dashboard turns telemetry off when configuring report

Merge pull request #45076 from chrisphoffman/wip-54297-octopus

octopus: cls/rbd: GroupSnapshotNamespace comparator violates ordering rules

Reviewed-by: Ilya Dryomov <idryomov@redhat.com>

Merge pull request #45071 from idryomov/wip-qemu-task-rbd-package-octopus

octopus: qa/tasks/qemu: make sure block-rbd.so is installed

Reviewed-by: Deepika Upadhyay <dupadhya@redhat.com>

Merge pull request #45019 from trociny/wip-47427-octopus

octopus: librbd: track complete async operation requests

Reviewed-by: Ilya Dryomov <idryomov@redhat.com>

Merge pull request #45006 from sunnyku/wip-54169-octopus

octopus: mgr/rbd_support: fix schedule remove

Reviewed-by: Ilya Dryomov <idryomov@redhat.com>

Merge pull request #45009 from idryomov/wip-rbd-help-positional-optional-octopus

octopus: rbd: mark optional positional arguments as such in help output

Reviewed-by: Sunny Kumar <sunkumar@redhat.com>
Reviewed-by: Mykola Golub <mgolub@mirantis.com>

Merge pull request #45004 from idryomov/wip-54128-octopus

octopus: krbd: return error when no initial monitor address found

Reviewed-by: Deepika Upadhyay <dupadhya@redhat.com>

Merge pull request #45001 from idryomov/wip-krbd-rxbounce-option-octopus

octopus: rbd: recognize rxbounce map option

Reviewed-by: Deepika Upadhyay <dupadhya@redhat.com>

Merge pull request #45000 from idryomov/wip-52522-octopus

octopus: librbd: fix use-after-free on ictx in list_descendants()

Reviewed-by: Mykola Golub <mgolub@mirantis.com>

Merge pull request #44992 from idryomov/wip-writesame-fua-octopus

octopus: librbd: honor FUA op flag for write_same() in write-around cache

Reviewed-by: Sunny Kumar <sunkumar@redhat.com>

15.2.16

mgr/dashboard: Contact Info should be visible only when Ident channel is checked

Fixes:https://tracker.ceph.com/issues/54133
Signed-off-by: Sarthak0702 <sarthak.0702@gmail.com>
(cherry picked from commit 15211a6378a6fee9316f79ba0b27821891527c38)

 Conflicts:
src/pybind/mgr/dashboard/frontend/src/app/ceph/cluster/telemetry/telemetry.component.ts
- `this.loading` used in Octopus instead of `this.loadingReady()`

mgr/dashboard: telemetry activate: show ident fields when checked

Signed-off-by: Aaryan Porwal <aaryanporwal2233@gmail.com>
(cherry picked from commit ad5b3f200529fc0bc511ce99eed338afcaef6a62)

mgr/dashboard: dashboard turns telemetry off when configuring report

Signed-off-by: Sarthak0702 <sarthak.0702@gmail.com>
(cherry picked from commit 97c57adf8565756dbf24f3c46ed3916303903fb7)

Conflicts:
src/pybind/mgr/dashboard/frontend/src/app/ceph/cluster/telemetry/telemetry.component.ts
- `this.i18n()` was used in Octopus instead of `$localize`

cls/rbd: GroupSnapshotNamespace comparator violates ordering rules

For

  GroupSnapshotNamespace a(1, "group-1", "snap-2");
  GroupSnapshotNamespace b(1, "group-2", "snap-1");

both a < b and b < a evaluate to true.  This violates STL strict weak
ordering requirements which is a problem because GroupSnapshotNamespace
is used as a key in std::map (ictx->snap_ids at least), etc.

Fixes: https://tracker.ceph.com/issues/49792
Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
(cherry picked from commit 830e72ab9d66c8f5703ea27da5249b02dd16ccd0)

qa/suites/rbd: make sure block-rbd.so is installed

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
(cherry picked from commit 8f0fd0af3da8581c47dc916303615264714a0489)

qa/tasks/qemu: make sure block-rbd.so is installed

Fixes: https://tracker.ceph.com/issues/54286
Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
(cherry picked from commit 525ff61cfc8516b4d7bed6f819b00a0b6cb7be0a)

Merge pull request #44986 from badone/wip-octopus-ceph-ansible-move-to-stream

octopus: qa/ceph-ansible: Move to Centos Stream

Reviewed-by: Yuri Weinstein <yweinste@redhat.com>

librbd: reset complete async request expiration time

Signed-off-by: Mykola Golub <mgolub@suse.com>
(cherry picked from commit 5c3ba95b4d1208868db1d00d1ba06ed086e1b268)

Conflicts:
src/librbd/ImageWatcher.cc (no quiesce requests in octopus,
            AsyncRequestId == operator is not supported, use !=)

Merge pull request #44791 from guits/wip-54023-octopus

octopus: ceph-volume: improve mpath devices support

Reviewed-by: Yuri Weinstein <yweinste@redhat.com>

ceph-volume: fix typo in tests

This fixes 2 typo in ceph-volume tests.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
(cherry picked from commit b07bd3e0e17021e0cf9773f916fad954f12254ed)

doc/ceph-volume: fix a typo

This fixes a typo in ceph-volume documentation.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
(cherry picked from commit 5d0a3cee5d7021dafd1e166e17946689b4bb90b7)

ceph-volume: add a test `test_mpath_device_is_device`

This test checks that Device.is_device() returns True for a mpath device.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
(cherry picked from commit 0280ff6df09bc26107bc97446e9d5c18fbc582e9)

ceph-volume: improve mpath devices support

ee8887f4c0ff4f91117f31b621b95c8d08019130 was intended for adding
mpath devices support in ceph-volume but it has missed the lvm batch scenario.
This also fixes the zapping of mpath devices prepared with `ceph-volume raw`

Fixes: https://tracker.ceph.com/issues/52908
Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
(cherry picked from commit 601ff7ed0a3ba5172b6bd886ca8ba2bd4d9e655a)

Merge pull request #44974 from guits/wip-54245-octopus

octopus: ceph-volume: honour osd_dmcrypt_key_size option

librbd: track complete async operation return code

Signed-off-by: Mykola Golub <mgolub@suse.com>
(cherry picked from commit 96bc20445afb0b2579a79b54a7854bc4a23f5b62)

Conflicts:
src/librbd/ImageWatcher.cc (no quiesce requests in octopus)

librbd: track complete async operation requests

to prevent duplicate maintenance operations due to RPC hiccups.

Fixes: https://tracker.ceph.com/issues/46803
Signed-off-by: Mykola Golub <mgolub@suse.com>
(cherry picked from commit 07fbc4b71df450655dec046c10e919dbfde989ba)

Conflicts:
src/librbd/ImageWatcher.cc (no quiesce requests in octopus)

rbd: mark optional positional arguments as such in help output

Currently at least five commands have optional positional arguments.

Overloading po::value<std::string>()->default_value("") for this
is a bit sneaky but nothing better fits into the existing Shell.cc
framework.

Note that strictly speaking "[<interval>] [<start-time>]" should be
"[<interval> [<start-time>]]" but we aren't doing that here because
"ceph" command doesn't do it either.

Fixes: https://tracker.ceph.com/issues/54191
Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
(cherry picked from commit cb0df397aae552adc80713ca0d59ed1ebfd3b1be)

rbd: ensure the help printer doesn't print past the end of the line

When long command names and long optional names are combined,
it's possible for the help text to be printed beyond the 80
character limit.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 51478deab22178d01db7fa8632ba72071b4dfc38)

qa/workunits/rbd: improve schedule add/remove cli test

This patch adds few tests to cover schedule add/remove with invalid
inputs.

Signed-off-by: Sunny Kumar <sunkumar@redhat.com>
(cherry picked from commit a9312d4777a82d8f2d8766a011f10952f84d3f27)

mgr/rbd_support: fix schedule remove

Issue:

If we provide a random string in the schedule remove
command the entire schedule at specified level gets
removed.

Fixes: https://tracker.ceph.com/issues/53250
Signed-off-by: Sunny Kumar <sunkumar@redhat.com>
(cherry picked from commit 1b62447071a900b9fa7d856617cb7db9e030f91e)

krbd: return error when no initial monitor address found

Since we filter monitor addresses based on ms_mode, check that at
least one address was found.

Otherwise, we mismatch arguments when calling sysfs/add_single_major
which emits a misleading error message to dmesg:

  libceph: resolve 'name=user1' (ret=-3): failed
  libceph: parse_ips bad ip 'name=user1,key=client.user1'

Fixes: https://tracker.ceph.com/issues/54128
Signed-off-by: Burt Holzman <burt@fnal.gov>
(cherry picked from commit 0076ffc86e043af7aedc127df8661eaf87fc1c58)

qa/suites/krbd: add legacy+rxbounce and crc+rxbounce coverage

For basic, rbd and rbd-nomount subsuites, replace legacy and crc
facets with "legacy or legacy+rxbounce" and "crc or crc+rxbounce"
facets (chosen at random).

For fsx, singleton and thrash subsuites, add legacy+rxbounce and
crc+rxbounce facets and drop prefer-crc facet.  The expected behaviour
of the latter depends on cluster configuration and should be tested
separately.

The total number of jobs remains the same.

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
(cherry picked from commit fbf8c1d68be60ab294719113edbd7f459a755c15)

qa: krbd rxbounce test

Lives in its own directory since ms_mode doesn't need to be permuted
here.

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
(cherry picked from commit 95d30b534ef65207168397dd25ca7213c8290568)

rbd: recognize rxbounce map option

Fixes: https://tracker.ceph.com/issues/54063
Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
(cherry picked from commit 8d2a456d7055cfb64e6bb9927187e2240b8c4d2a)

librbd: report correct error for ictx->state->close()

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
(cherry picked from commit 22903c3965b15245b464540ccfb3404bf45f0627)

librbd: fix use-after-free on ictx in list_descendants()

Ictx is deleted when "ictx->state->open()" and "ictx->state->close()"
fail, and then "lderr(ictx->cct)" crashes.

Fixes: https://tracker.ceph.com/issues/52522
Signed-off-by: Wang ShuaiChao <wangshuaich@chinatelecom.cn>
(cherry picked from commit fa5d61ee5144f67cba53d54d36013614183e53a3)

Merge pull request #44978 from ifed01/wip-ifed-clist-pend-bug-oct

octopus:  os/bluestore: list obj which equals to pend

Reviewed-by: Mykola Golub <mgolub@mirantis.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>

librbd: honor FUA op flag for write_same() in write-around cache

WriteAroundObjectDispatch::write_same() should pass op_flags through
to dispatch_io() so that it can bypass the cache if needed.

Fixes: https://tracker.ceph.com/issues/52956
Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
(cherry picked from commit 0dcea098cf4d51bed31d8646dc3b533514c08a72)

os/bluestore: list obj which equals to pend

otherwise we could have failures like

scrub : stat mismatch, got 3/4 objects, 1/2 clones, 3/4 dirty, 3/4 omap, 0/0 pinned, 0/0 hit_set_archive, 0/0 whiteouts, 49/56 bytes, 0/0 manifest objects, 0/0 hit_set_archive bytes."

where the numbers of scrubbed object, clones, dirty and omap are always
less than the total number of corresponding numbers, if the PG contains
object(s) whose hash happens to be 0xffffffff.

in this change, if the calculated hash of the upper bound is greater
than the maximum possible number represented by uint32_t, in addition to
setting the hash of the upper bound hobj to 0xffffffff, we also set the
nspace of hobj of the upper bound to "\xff", so that the upper bound
is greater than an hobj whose hash happens to be 0xfffffff. please note,
the nspace of "\xff" is not an ascii string, so it's not likely to be
less than a real-world nspace of an hobj.

with this new *greater* upper bound, we are able to include the previous
missing hobj when listing the objects in a PG. so the scrub won't be
annoyed when the number of objects does not match.

Fixes: https://tracker.ceph.com/issues/52705
Signed-off-by: Mykola Golub <mykola.golub@clyso.com>
Signed-off-by: Kefu Chai <tchaikov@gmail.com>
(cherry picked from commit ffab13bcd9006c1f961a24b8016df9d1fe06ba1d)

 Conflicts:
src/os/bluestore/BlueStore.cc
 - get_coll_range function signature alignment

Merge pull request #44614 from ifed01/wip-ifed-fix-ram-gridy-fsck-oct

octopus: os/bluestore: make shared blob fsck much less RAM-greedy.

Reviewed-by: Adam Kupczyk <akupczyk@redhat.com>

os/bluestore: use scope_guard to log latency

simpler this way, and avoid using `goto`.

Signed-off-by: Kefu Chai <tchaikov@gmail.com>
(cherry picked from commit 715a83822ebc1a3d102d1ec13323b69db0600719)

ceph-volume/activate: load the config from lv tag

When `ceph-volume lvm trigger` is called with an OSD where the tag
`ceph.cluster_name` is not 'ceph', it fails.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
(cherry picked from commit 5ac1ec65cb2a582b2ae550202cc9911f993943f2)

ceph-volume/tests: use centos/stream8 images

Since recent move from CentOS 8 to CentOS Stream 8, let's do the same here.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
(cherry picked from commit 2b793952bbac7973b97d245c282165daadeabb51)

ceph-volume/tests: add tests in util/encryption.py

this adds some unit tests in order to cover `luks_format()` and `luks_open()`
in `util/encryption.py`.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
(cherry picked from commit db48850745f218e08cf53ae2d8edf3428f2b4010)

ceph-volume: honour osd_dmcrypt_key_size option

ceph-volume doesn't honour osd_dmcrypt_key_size.
It means the default size is always applied.

It also changes the default value in `get_key_size_from_conf()`

From cryptsetup manpage:

> For XTS mode you can optionally set a key size of 512 bits with the -s option.

Using more than 512bits will end up with the following error message:

```
Key size in XTS mode must be 256 or 512 bits.
```

Fixes: https://tracker.ceph.com/issues/54006
Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
(cherry picked from commit 47c33179f9a15ae95cc1579a421be89378602656)

octopus: qa/ceph-ansible: Move to Centos Stream

Centos 8 is eol and its package repos no longer exist.

Signed-off-by: Brad Hubbard <bhubbard@redhat.com>

Merge pull request #44860 from aclamk/wip-53392-octopus

octopus: Fix data corruption in bluefs truncate()

Reviewed-by: Igor Fedotov <ifedotov@suse.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge pull request #44929 from adk3798/octopus-cephadm-qa-centos-8-stream

octopus: qa/suites/rados/cephadm: use centos 8.stream

Reviewed-by: Yuri Weinstein <yweinste@redhat.com>

Merge pull request #44864 from cbodley/wip-54089

octopus: qa: remove centos8 from supported distros

Reviewed-by: Neha Ojha <nojha@redhat.com>

qa/suites/rados/cephadm: remove centos 8.2, 8.3

Signed-off-by: Adam King <adking@redhat.com>

qa/suites/orch/cephadm: add 8.stream + container_tools

Signed-off-by: Sage Weil <sage@newdream.net>
Conflicts:
qa/suites/rados/cephadm/upgrade/1-start-distro/1-start-centos_8.stream_container-tools.yaml

qa/rgw: rgw/verify no longer pins centos 8.0

the symlink rgw/verify/centos_latest.yaml already selects centos

Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit 0fad609d4dca01335abda6c48ae2663a8fd15494)

qa/distros: remove duplicate centos_8.stream.yaml from supported

Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit 3b71b41190bbb0af5026babc82266541b6398e92)

qa/distros: centos_8.yaml is now a symlink to centos_8.stream.yaml

Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit 0f4e51f05f9b340fe6128b46ea4601ecf01625d2)

qa/distro/supported: add centos 8.stream

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit 577cbd162ed63bcee9f027776d72d569d9adf93b)

os/bluestore/bluefs: Fix data corruption in truncate()

It is possible to create condition in which a BlueFS contains file that is corrupted.
It can happen when BlueFS replay log is on device A and we just wrote to device B and truncated file.

Scenario:
1) write to file h1 on SLOW device
2) flush h1 (initiate transfer, but no fdatasync yet)
3) truncate h1
4) write to file h2 on DB
5) fsync h2 (forces replay log to be written, after fdatasync to DB)
6) poweroff

Fixes: https://tracker.ceph.com/issues/53129
Signed-off-by: Adam Kupczyk <akupczyk@redhat.com>
(cherry picked from commit 49b7b44b3b5c94ee401562e603999e2b3bd8f9a2)

os/objectstore/test: Add test for data corruption in file truncation

Test for https://tracker.ceph.com/issues/53129

Signed-off-by: Adam Kupczyk <akupczyk@redhat.com>
(cherry picked from commit 1f7771d4a77ebb271b939b3893d7607d964796f0)

Merge pull request #44726 from cfsnyder/wip-53704-octopus

octopus: osdc: add set_error in BufferHead, when split set_error to right

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Venky Shankar vshankar@redhat.com

rgw: under fips, set flag to allow md5 in select rgw ops - for review

the ovrrides for review and relevant md5 usage:
```
src/rgw/rgw_crypt.cc:975:      unsigned char key_hash_res[CEPH_CRYPTO_MD5_DIGESTSIZE];
    int rgw_s3_prepare_encrypt(...)
        crypt_http_responses["x-amz-server-side-encryption-customer-algorithm"] = "AES256";
        crypt_http_responses["x-amz-server-side-encryption-customer-key-MD5"] = std::string(keymd5);
                                                           ~~~~~~~~~~~~~~~~

src/rgw/rgw_crypt.cc:1225:    uint8_t key_hash_res[CEPH_CRYPTO_MD5_DIGESTSIZE];
    int rgw_s3_prepare_decrypt(...)
        crypt_http_responses["x-amz-server-side-encryption-customer-algorithm"] = "AES256";
        crypt_http_responses["x-amz-server-side-encryption-customer-key-MD5"] = keymd5;
                                                           ~~~~~~~~~~~~~~~~

src/rgw/rgw_keystone.cc:40:  unsigned char m[CEPH_CRYPTO_MD5_DIGESTSIZE];
        void TokenCache::add_admin(...)
  rgw_get_token_id(token.token.id, admin_token_id);
                                   ~~~~~~~~~~~~~~ md5
  add_locked(admin_token_id, token);

        void TokenCache::add_barbican(...)
  rgw_get_token_id(token.token.id, barbican_token_id);
                                   ~~~~~~~~~~~~~~~~~ md5
  add_locked(barbican_token_id, token);
```

Signed-off-by: Mark Kogan <mkogan@redhat.com>
(cherry picked from commit 551e0c8f38f3f646dbfb5fbfde51d3107ca90cc6)

rgw: under fips, set flag to allow md5 in select rgw ops

Fixes: https://tracker.ceph.com/issues/52900
Signed-off-by: Mark Kogan <mkogan@redhat.com>
(cherry picked from commit a5df0cfcbe9b04832095f1f8ef0ce9f178888efd)

Merge pull request #44728 from cfsnyder/wip-51826-octopus

octopus: qa/run-tox-mgr-dashboard: Do not write to /tmp/test_sanitize_password…

Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: kevinzs2048 <NOT@FOUND>
Reviewed-by: Yuri Weinstein <yweins@redhat.com>

Merge pull request #44544 from cfsnyder/wip-53660-octopus

octopus: mon: prevent new sessions during shutdown

Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge pull request #44720 from cfsnyder/wip-53495-octopus

octopus: mgr: fix locking for MetadataUpdate::finish

Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge pull request #44700 from cfsnyder/wip-53943-octopus

octopus: mon/OSDMonitor: avoid null dereference if stats are not available

Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge pull request #44753 from idryomov/wip-rbd-mirror-delprop-races-octopus

octopus: rbd-mirror: fix races in snapshot-based mirroring deletion propagation

Reviewed-by: Mykola Golub <mgolub@mirantis.com>
Reviewed-by: Arthur Outhenin-Chalandre <arthur.outhenin-chalandre@cern.ch>

Merge pull request #44741 from idryomov/wip-rbd-switch-arguments-fix-octopus

octopus: rbd: add missing switch arguments for recognition by get_command_spec()

Reviewed-by: Mykola Golub <mgolub@mirantis.com>

Merge pull request #44722 from cfsnyder/wip-53534-octopus

octopus: mon/MgrStatMonitor: do not spam subscribers (mgr) with service_map

Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge pull request #44724 from cfsnyder/wip-53609-octopus

octopus: os/bluestore: avoid premature onode release.

Reviewed-by: Igor Fedotov <ifedotov@suse.com>

Merge pull request #44730 from cfsnyder/wip-53850-octopus

octopus: rgwlc:  remove lc entry on bucket delete

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #44167 from cfsnyder/wip-53290-octopus

octopus: rgw: fix `bi put` not using right bucket index shard

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #44585 from vumrao/wip-vumrao-53877

octopus: osd/PeeringState: separate history's pruub from pg's

Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge pull request #43446 from k0ste/wip-52849-octopus

octopus: mgr: Add check to prevent mgr from crashing

Reviewed-by: Venky Shankar vshankar@redhat.com

Merge pull request #43438 from trociny/wip-52833-octopus

octopus: osd: re-cache peer_bytes on every peering state activate

Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge pull request #42677 from callithea/wip-51423-octopus

octopus: mgr: set debug_mgr=2/5 (so INFO goes to mgr log by default)

Reviewed-by: Yuri Weinstein <yweinste@redhat.com>

Merge pull request #44766 from guits/wip-53954-octopus

octopus: ceph-volume: don't use MultiLogger in find_executable_on_host()

ceph-volume: don't use MultiLogger in find_executable_on_host()

This generates a lot of unnecessary messages on the terminal.

Fixes: https://tracker.ceph.com/issues/53934
Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
(cherry picked from commit 3be55621600be3ebc9c70295a3a351dab426b3a3)

Merge pull request #44757 from guits/wip-53917-octopus

octopus: ceph-volume: fix regression introcuded via #43536

Merge pull request #44709 from guits/wip-53961-octopus

octopus: ceph-volume: show RBD devices as not available

test/rbd_mirror: drop redundant MockJournaler instances

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
(cherry picked from commit 303d3ede48a088b947cb99f6fe1b400a6b0871be)

rbd-mirror: fix races in snapshot-based mirroring deletion propagation

When remote image is deleted, rbd-mirror can encounter three cases:

  1) no remote image id
  2) no remote mirror metadata
  3) MIRROR_IMAGE_STATE_DISABLING in remote mirror metadata

Commit d4c66ac5c615 ("rbd-mirror: fix issue with snapshot-based
mirroring deletion propagation") fixed case 1.  Cases 2 and 3 remained
broken because for both of them finalize_snapshot_state_builder() would
populate not only remote_mirror_peer_uuid but also remote_image_id,
thus disabling ENOLINK logic in handle_prepare_remote_image() and
handle_bootstrap().  Commit ff60aec2d9ef ("rbd-mirror: fix bootstrap
sequence while the image is removed") touched on case 3, but it made
a difference only for journal-based mirroring.

Stop calling finalize_snapshot_state_builder() on errors.  Instead,
align with journal-based mirroring by filling remote_mirror_peer_uuid
together with remote_mirror_uuid.

Fixes: https://tracker.ceph.com/issues/53963
Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
(cherry picked from commit d634a1df5b19d61955f2f94c7cc29bd4f3b678c8)

rbd-mirror: don't default replay_requires_remote_image() implementation

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
(cherry picked from commit ccfbf3e97ed1f50df0adcbec812f1b11fe22cace)

rbd-mirror: untangle StateBuilder::is_linked() overloads

Make it clear that the local image non-primariness is asserted
independent of the mode; avoid the default implementation being
overridden but still relied on by both modes.

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
(cherry picked from commit f49fa483ec6cdc19b4d60debefbb21bf65b7a385)

rbd-mirror: drop redundant initialization of StateBuilder members

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
(cherry picked from commit baf57925abdee287cfa0aefc5ba2f602dac8c25e)

rbd: add missing switch arguments for recognition by get_command_spec()

Currently this

  $ rbd --all children img

doesn't work, while this

  $ rbd children --all img

or this

  $ rbd children img --all

does.  The issue is that -a/--all isn't on the list of known switch
arguments.  The "rbd children" example may seem contrived but for more
complicated commands such as "rbd device map" mixing switches and
positional arguments occurs naturally:

  $ rbd device --device-type nbd --options try-netlink --show-cookie map img

Fixes: https://tracker.ceph.com/issues/53935
Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
(cherry picked from commit e1b4811bc324236892e43a6bb841d6278fe1584e)

Conflicts:
src/tools/rbd/ArgumentTypes.h [ snapshot quiesce support
  not in octopus ]
src/tools/rbd/action/Device.cc [ nbd cookie support not in
  octopus ]
src/tools/rbd/action/Migration.cc [ import-only migration
  not supported in octopus ]
src/tools/rbd/action/Wnbd.cc [ wnbd support not in octopus ]

Merge pull request #44689 from MrFreezeex/wip-53937-octopus

octopus: cls/journal: skip disconnected clients when calculating min_commit_position

Reviewed-by: Ilya Dryomov <idryomov@redhat.com>
Reviewed-by: Mykola Golub <mgolub@mirantis.com>

Merge pull request #43806 from sunnyku/wip-53027-octopus

octopus: librbd/object_map: rbd diff between two snapshots lists entire image content

Reviewed-by: Ilya Dryomov <idryomov@redhat.com>

rgwlc:  remove lc entry on bucket delete

Buckets with lifecycle policies installed have a state entry that
must also be deleted when the bucket is removed.

Fixes: https://tracker.ceph.com/issues/46728
N.b., should really be generic, not specific to the RADOS store, but
there doesn't seem to be a clean model for implementing generic side
effects in Zipper, currently.

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
(cherry picked from commit cc1e812a003e2af74fe0c69ccae08dd7aa68bbe0)

Conflicts:
src/rgw/rgw_sal_rados.cc

Cherry-pick notes:
- Code from rgw_sal_rados.cc existed in rgw_sal.cc in Octopus

qa/run-tox-mgr-dashboard: Do not write to /tmp/test_sanitize_password.txt file

To allow running multiple instances of the same tests.

Fixes: https://tracker.ceph.com/issues/51792
Signed-off-by: Kevin Zhao <kevin.zhao@linaro.org>
(cherry picked from commit d04ef800abd671a564795eba198ca976619b4cc7)

osdc: add set_error in BufferHead, when split set_error to right

Fixes: https://tracker.ceph.com/issues/53227
Signed-off-by: jiawd <jiawendong@xtaotech.com>
(cherry picked from commit dba751ac0c0e9c8276a59ea3337b31fc71e26bf0)

os/bluestore: avoid premature onode release.

This was observed when onode's removal is followed by reading
and the latter causes object release before the removal is finalized.
The root cause is an improper 'pinned' state assessment in Onode::get

More detailed overview is:
At some point Onode::get() might face the case when nref == 2 and pinned = true
which means parallel incomplete put is running on the onode - ref count is
decremented but pinned state is still unmodified (and even lock hasn't been
acquired yet).
This might finally result in two puts racing over the same onode with nref == 2
which finally results in a premature onode release:
  // nref =3, pinned = 1
  // Thread 1                   Thread 2
  //   o->put()                   o->get()
  //   --nref(n = 2, pinned=1)
  //                              nref++ (n=3, pinned = 1)
  //                              return
  //                              ...
  //                              o->put()
  //                              --nref(n = 2)
  //                              pinned = 0,
  //                              --nref(n = 1)
  //                              ocs->_unpin_and_rm(o) -> o->put()
  //                                ...
  //                                --nref(n = 0)
  //                                release o
  //  o->c->get_onode_cache()
  //  FAULT!
  //
The suggested fix is to introduce additional atomic counter tracking
running put() functions. And permit onode release when both regular
nref and put_nref are both equal to zero.

Fixes: https://tracker.ceph.com/issues/53002
Signed-off-by: Igor Fedotov <igor.fedotov@croit.io>
(cherry picked from commit 96f0efe6d5307a55bea32f7216ef9511da0c5a47)

mon/MgrStatMonitor: do not spam subscribers (mgr) with service_map

We are comparing the monmap epoch to the service_map epoch!

Fixes: https://tracker.ceph.com/issues/53479
Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit d0f5ed13567f54b1bdedc388568d2afc5922ab70)

Conflicts:
src/mon/MgrStatMonitor.cc

Cherry-pick notes:
- MgrStatMonitor mon member was a pointer in Octopus

mgr: fix locking for MetadataUpdate::finish

We need to hold the DaemonState lock here since we are both reading and
writing its content.

Fixes: https://tracker.ceph.com/issues/53393
Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit 5096685cd623de71a7c45a667e1dd776357fd997)

Conflicts:
src/mgr/Mgr.cc

Cherry-pick notes:
- state variable was declared outside of if condition in Octopus

Merge pull request #44320 from guits/wip-53618-octopus

octopus: ceph-volume: make it possible to skip needs_root()

ceph-volume: filter RBD devices from the device inventory

Avoid running `blkid` or deploying OSDs on RBD devices by ensuring they
do not appear in the `ceph-volume inventory`

Fixes: https://tracker.ceph.com/issues/53846
Signed-off-by: Michael Fritch <mfritch@suse.com>
(cherry picked from commit 47325ec3ec5ce1d53c5eae2952f631e95b7135fe)

Merge pull request #44673 from idryomov/wip-diff-iterate-parent-tests-octopus

octopus: backport diff-iterate include_parent tests

Reviewed-by: Deepika Upadhyay <dupadhya@redhat.com>

Merge pull request #44595 from idryomov/wip-xfstests-qemu-cert-octopus

octopus: qa/run_xfstests_qemu.sh: stop reporting success without actually running any tests

Reviewed-by: Deepika Upadhyay <dupadhya@redhat.com>