qa: Wait for mirror daemon restart before getting new rados_inst

After blocklisted/failed, wait for the mirror daemon restart
which is after 30 seconds timeout and then check for the new rados_inst.

Fixes: https://tracker.ceph.com/issues/64927
Signed-off-by: Jos Collin <jcollin@redhat.com>
(cherry picked from commit a9a56919ff95d2face436da2ca08fd26580fadb9)

cephfs_mirror: Fixed negative seconds

Fixes: https://tracker.ceph.com/issues/64927
Fixes: https://tracker.ceph.com/issues/51964
Fixes: https://tracker.ceph.com/issues/63931
Fixes: https://tracker.ceph.com/issues/63089
Signed-off-by: Jos Collin <jcollin@redhat.com>
(cherry picked from commit 96c3f8f33d789a065c7ed418679afd53faffa3d8)

cephfs_mirror: Add ErrorListener to maintain blocklisted/failed timestamp in FSMirror

Have FSMirror register a listener with InstanceWatcher/MirrorWatcher which would get invoked when the mirror daemon is blocklisted or failed.
Thus FSMirror can maintain the last blocklisted/failed timestamp and use that for restarting the mirror daemon.

Fixes: https://tracker.ceph.com/issues/64927
Fixes: https://tracker.ceph.com/issues/51964
Fixes: https://tracker.ceph.com/issues/63931
Fixes: https://tracker.ceph.com/issues/63089
Signed-off-by: Jos Collin <jcollin@redhat.com>
(cherry picked from commit 77ec7bfde7a349b0e06b34ecdf328996c7642d43)

Merge pull request #57678 from batrick/wip-66188-squid

squid: mds: use regular dispatch for processing metrics

Merge pull request #58476 from rhcs-dashboard/wip-66872-squid

squid: mgr/dashboard: nfs export enhancement for CEPHFS

Reviewed-by: afreen23 <NOT@FOUND>

Merge pull request #58593 from rhcs-dashboard/wip-66785-squid

squid: mgr/dashboard: add RGW lifecycle management

Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Ankush Behl <cloudbehl@gmail.com>

mgr/dashboard: nfs export enhancement for CEPHFS

Fixes: https://tracker.ceph.com/issues/66718
Signed-off-by: Avan Thakkar <athakkar@redhat.com>
(cherry picked from commit 69a45db9c3cc9ed4ac36e7f2d6db78bd7940530b)

Merge pull request #58553 from adk3798/squid-cephadm-cert-store

squid: mgr/cephadm: Cert Store + use secure_channel for grpc requests

Reviewed-by: Nizamudeen A <nia@redhat.com>

Merge pull request #58347 from idryomov/wip-66570-squid

squid: librbd: make diff-iterate in fast-diff mode aware of encryption

Reviewed-by: Ramana Raja <rraja@redhat.com>

Merge pull request #57610 from vshankar/wip-65988-squid

squid: qa: ignore cluster warnings generated from forward-scrub task

Merge pull request #57918 from batrick/wip-66386-squid

squid: qa: disable mon_warn_on_pool_no_app in fs suite

Merge pull request #57922 from batrick/wip-66385-squid

squid: qa: correct json lookup for new `lock path` output

Merge pull request #58155 from vshankar/wip-66325-squid

squid: qa: enable debug logs for fs:cephadm:multivolume subsuite

Merge pull request #58233 from batrick/wip-66593-squid

squid: tools/cephfs: recover alternate_name of dentries from journal

Merge pull request #58237 from batrick/wip-66619-squid

squid: mds: initialize epoch for quiescedb

Merge pull request #58252 from joscollin/wip-66674-squid

squid: qa: enhance per-client labelled perf counters test

Merge pull request #58294 from batrick/wip-66623-squid

squid: mds: do remove the cap when seqs equal or larger than last issue

Merge pull request #58317 from batrick/wip-66585-squid

squid: mds: QuiesceDbManager: mark next retry event during bootstrap

Merge pull request #58318 from batrick/wip-66572-squid

squid: mds/quiesce-db: calculate_quiesce_map: aggregate quiesce roots' TTL as `max`

Merge pull request #58319 from batrick/wip-66542-squid

squid: mds: dump formatter even for errors

librbd: make diff-iterate in fast-diff mode aware of encryption

diff-iterate wasn't updated when librbd was being prepared to support
encryption in commit 8d6a47933269 ("librbd: add crypto image dispatch
layer").  This is even noted in [1]:

> The two places I skipped for now are DiffIterate and TrimRequest.

CryptoImageDispatch has since been removed, but diff-iterate in
fast-diff mode is still unaware of encryption and just assumes that all
offsets are raw.  This means that the callback gets invoked with
incorrect image offsets when encryption is loaded.  For example, for
a LUKS1-formatted image with some data at offsets 0 and 20971520,
diff-iterate with encryption loaded reports

  0~4194304
  4194304~4194304
  25165824~4194304

instead of

  0~4194304
  20971520~4194304

as "exists".  For any piece of code that is using diff-iterate to
optimize block-by-block processing (e.g. copy an encrypted source image
to a differently-encrypted destination image), this is fatal: it would
skip processing block 20971520 which has data and instead process block
25165824 which doesn't have any data and was to be skipped, producing
a corrupted destination image.

[1] https://github.com/ceph/ceph/pull/37935#issue-735278403

Fixes: https://tracker.ceph.com/issues/66570
Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
(cherry picked from commit cdeb0efce3f9f857ad6d5b7ff3965f3292cb571a)

test/librbd: extend DiffIterateTest.DiffIterateDeterministic{,PP}

Currently we are laying data only at the beginning of an object.
Extend the skeletons to write to three different offsets in the middle
and also at the end of the object.

Separately, make C and C++ API test variants slightly different in
terms of offsets being targeted to not go through exactly the same
scenario twice.

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
(cherry picked from commit da69d1fdbf69cf37ba8b75ea9a230634bb25ae76)

test/librbd: refactor DiffIterateTest.DiffIterateDeterministic{,PP}

In preparation for extending coverage, extract test logic into a couple
of parametrized helpers.

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
(cherry picked from commit 1df87998dc30be6e3b4c90f5833539c7ccc70911)

Merge pull request #58541 from idryomov/wip-66886-squid

squid: qa: account for rbd_trash object in krbd_data_pool.sh + related ceph{,adm} task fixes

Reviewed-by: Ramana Raja <rraja@redhat.com>

Merge pull request #58212 from idryomov/wip-66418-squid

squid: librbd: diff-iterate shouldn't crash on an empty byte range

Reviewed-by: Mykola Golub <mgolub@suse.com>

Merge pull request #58206 from idryomov/wip-66300-squid

squid: librbd: disallow group snap rollback if memberships don't match

Reviewed-by: Ramana Raja <rraja@redhat.com>

Merge pull request #58575 from zdover23/wip-doc-2024-07-14-backport-58451-to-squid

squid: doc/dev: edit "Principles for format change"

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>

Merge pull request #58385 from smanjara/wip-squid-pr-56874

[squid] qa/multisite: stabilize multisite testing

Reviewed-by: Casey Bodley <cbodley@redhat.com>

mgr/dashboard: add RGW lifecycle management

Fixes: https://tracker.ceph.com/issues/50327
Signed-off-by: Pedro Gonzalez Gomez <pegonzal@redhat.com>
(cherry picked from commit b54efd44e8d72d9b56fe4c69a96d53934de1df2a)

 Conflicts:
src/pybind/mgr/dashboard/frontend/src/app/shared/pipes/pipes.module.ts

Merge pull request #58578 from zdover23/wip-doc-2024-07-15-backport-58574-to-squid

squid: doc/cephfs: edit "is mount helper present"

Merge pull request #58001 from amathuria/wip-66421-squid

squid: osd/scrub: Change scrub cost to average object size

Reviewed-by: Sridhar Seshasayee <sseshasa@redhat.com>

Merge pull request #57618 from sajibreadd/wip-66143-squid

squid: osd: CEPH_OSD_OP_FLAG_BYPASS_CLEAN_CACHE flag is passed from ECBackend

Reviewed-by: Samuel Just <sjust@redhat.com>

Merge pull request #57195 from k0ste/wip-64591-squid

squid: os/bluestore: fix the problem of l_bluefs_log_compactions double recording

Reviewed-by: Igor Fedotov <ifedotov@suse.com>

doc/cephfs: edit "is mount helper present"

Edit the section called "Is mount helper present?", the title of which
prior to this commit was "Is mount helper is present?". Other small
disambiguating improvements have been made to the text in the section.
An unselectable prompt has been added before a command.

Signed-off-by: Zac Dover <zac.dover@proton.me>
(cherry picked from commit 0ebfb3eda8e2fc88662530bbddc84f2efcc4e13c)

doc/dev: edit "Principles for format change"

Improve "Principles for format change" in doc/dev/encoding.rst. This
commit started as a response to Anthony D'Atri's suggestion here: https://github.com/ceph/ceph/pull/58299/files#r1656985564

Review of this section suggested to me that certain minor English usage
improvements would be of benefit. The numbered lists in this section
could still be made a bit clearer.

Co-authored-by: Anthony D'Atri <anthony.datri@gmail.com>
Signed-off-by: Zac Dover <zac.dover@proton.me>
(cherry picked from commit 570797e5588b67b8c72e5297b61f84d9aa48dc45)

Merge pull request #58071 from yuvalif/wip-66497-squid

squid: rgw/admin/notification: add command to dump notifications

Reviewed-by: Daniel Gryniewicz <dang@redhat.com>

Merge pull request #58538 from cbodley/wip-66913-squid

squid: rgw/datalog: fix LazyFIFO race fix

Reviewed-by: Adam Emerson <aemerson@redhat.com>

Merge pull request #58545 from smanjara/wip-66919-squid

squid: rgw/multisite: allow PutACL replication.

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #58550 from idryomov/wip-test-cookie-squid

squid: test/librbd: clean up unused TEST_COOKIE variable

Reviewed-by: Ramana Raja <rraja@redhat.com>

Merge pull request #58477 from afreen23/wip-backport-nvmeof-service

Batch backport of nvmeof service configuration

Reviewed-by: Pedro Gonzalez Gomez <pegonzal@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>

mgr/dashboard: fall back to server_cert

if root_ca is not found

Signed-off-by: Nizamudeen A <nia@redhat.com>
(cherry picked from commit 0a393ca9bde714524da369caaef1c097472f791c)
(cherry picked from commit 5a89eae68c1b4004832a6c2dfe3f1f54fd72c3a1)

mgr/cephadm: create OrchSecretNotFound exception type

This exception type is made to handle the formatting
of errors where we try to find a cert/key in the
cert/key store and can't

Signed-off-by: Adam King <adking@redhat.com>
(cherry picked from commit 91e7328b033cec89641b53ec29cef14715970aec)

mgr/dashboard: use secure_channel for grpc requests

Store the certificates to config-key stores and then later on used by
dashboard to set-up the secure_channel for grpc nvmeof requests

By storing the certificates we can ensure that the dashboard nvmeof apis
will be configurable even if the deployments are not cephadm based

Signed-off-by: Nizamudeen A <nia@redhat.com>
(cherry picked from commit 0a393ca9bde714524da369caaef1c097472f791c)
(cherry picked from commit ebcb198894347d1a21ab4cd81fea985aca454b29)

Conflicts:
src/pybind/mgr/dashboard/services/orchestrator.py

mgr/cephadm: fix nvmeof conf unit test

This needed changes to reflect changes made
to the conf to not have the certs stored at a
relative path and the addition of the root ca
cert

Signed-off-by: Adam King <adking@redhat.com>
(cherry picked from commit 91883b9efc396779dfb61494e9aa140251f22b54)
(cherry picked from commit ccdfe8e4606eeb9b25c35e104cb2ad4ccd9eb5c1)

mgr/cephadm: add get cert/key commands

In order to be able to grab certs/keys stored
in the new CertKeyStore class

Signed-off-by: Adam King <adking@redhat.com>
(cherry picked from commit 2dd0ce91a53a898b8b3635f8943cc2d0e39b4ec1)
(cherry picked from commit f357f9b1afb5a417f541bfe0cc69d9777accf136)

mgr/cephadm: integrate nvmeof spec certs/keys into cert/key store

Now that we're taking actual certs/keys in the spec, they
should go into the cert/key store with the others

Signed-off-by: Adam King <adking@redhat.com>
(cherry picked from commit 3b1ec5851902c6b672db2ea81ae7eebcb1002101)
(cherry picked from commit 5f24d366af30bf8a4334e208daa9be2fcf024f3b)

python-common/service_spec: add root_ca_cert to nvmeof spec

Also improves the error messaging around when spec/key
attributes are missing when enable_auth is set to true

Signed-off-by: Adam King <adking@redhat.com>
(cherry picked from commit 9212914be65fe3adde2108f5a2cfd2587d17c0ff)
(cherry picked from commit f7c46fa3adf71631a1cea372841e4d74f09094ed)

mgr/cephadm: allow passing client/server cert/key in nvmeof spec

Before this patch the client/server cert/key fields were
just filepaths that told the nvmeof gw daemon where to look
for the cert/key. There's not much reason why users would
care where in the nvmeof gw container the cert goes. It's more
useful to use these fields as a way to pass the certs/keys
to the daemon and then just hardcode where in the container
we'll place the certs/keys

Signed-off-by: Adam King <adking@redhat.com>
(cherry picked from commit e9fca39092348e6c08022341116875e831c175f0)
(cherry picked from commit e2e6aeb40acc98070e0e2c4a0056e42458e9f4f1)

mgr/cephadm: add migration for certs/keys to cert store

Signed-off-by: Adam King <adking@redhat.com>
(cherry picked from commit ab77c8dc714e84b4a0055821f800a458acc6a051)

mgr/cephadm: move ingress cert to cert store

In an effort to try and organize our cert handling
and provide cert management features in the future

Signed-off-by: Adam King <adking@redhat.com>
(cherry picked from commit 621f2e036c6bd4acb2f551fab94421d4e4788c97)

mgr/cephadm: store iscsi cert and key in cert store

In an effort to organize our handling of certs and
offer cert management functionality in the future

Signed-off-by: Adam King <adking@redhat.com>
(cherry picked from commit a7e05c2f5acd4ec683056cc163320a28481f8269)

mgr/cephadm: move rgw frontend ssl cert to cert store

In an effort to organize our certs and in the future
add cert management features.

Signed-off-by: Adam King <adking@redhat.com>
(cherry picked from commit 900cc4467ed2ac6ebc98649b6e3101c86a538bc7)

mgr/cephadm: move node-exporter cert/key to cert store

We weren't storing this before, but we want to
add some cert management stuff later, so it's
good to start doing so

Signed-off-by: Adam King <adking@redhat.com>
(cherry picked from commit 59a7c43f5d3674b8d8a42c2ada66710a08ab0138)

mgr/cephadm: move prometheus daemon cert to cert store

We weren't storing this before, but since we want to
add more options for cert management, it's good to start
doing so

Signed-off-by: Adam King <adking@redhat.com>
(cherry picked from commit 255e50827828b464654072c410007923697058ef)

mgr/cephadm: move alertmanager crt/key to cert store

We weren't actually even storing these before, but
given we want to be able to offer some more cert
management options in the future, it's good to start
doing so

Signed-off-by: Adam King <adking@redhat.com>
(cherry picked from commit 6ae8c5ae645851dbb6981ede9ae3d0b4dd8e511e)

mgr/cephadm: add cert-store ls commands

To see what certs cephadm has stored in
its internal cert store

Signed-off-by: Adam King <adking@redhat.com>
(cherry picked from commit 16417abe39e6a1c009c373b77c734ceae915b2a4)

mgr/cephadm: move grafana cert/key to cert/key store

In an effort to try and better track what certs/keys
we manage with cephadm

Signed-off-by: Adam King <adking@redhat.com>
(cherry picked from commit bb7e715320e41f5d6b6291769e2b6d230eec74cc)

mgr/cephadm: move agent endpoint root cert/key to cert store

Trying to move all certs/keys cephadm manages to
a central location

Signed-off-by: Adam King <adking@redhat.com>
(cherry picked from commit 2e583e47c5e69d104476f9350db3020d7ffae0e6)

mgr/cephadm: move service discovery cert/key to cert store

Trying to move all of the certs/keys cephadm manages
into one place.

Signed-off-by: Adam King <adking@redhat.com>
(cherry picked from commit 078b694b99ad5e2362790ba21aeb784cb5e8ccdb)

mgr/cephadm: add a Cert/Key tracking/storage class

The idea is to move storage/handling of certs from any misc.
spot in the mgr/cephadm codebase into a single class. This will
make it much easier to do things with the certs we have in
the future.

Signed-off-by: Adam King <adking@redhat.com>
(cherry picked from commit 67814772458eb2cb3311ae81bd3a111ee9069a6c)

Merge pull request #58521 from cbodley/wip-66897-squid

squid: install-deps.sh, do_cmake.sh: almalinux is another el flavour

Reviewed-by: Kyr Shatskyy <kyrylo.shatskyy@gmail.com>
Reviewed-by: Dan van der Ster <dan.vanderster@clyso.com>

test/ librbd: clean up unused 'TEST_COOKIE'

Clean them up because
```
/home/jenkins-build/build/workspace/ceph-pull-requests-arm64/src/test/librbd/exclusive_lock/test_mock_PreAcquireRequest.cc:47:26: error: unused variable 'TEST_COOKIE' [-Werror,-Wunused-const-variable]
static const std::string TEST_COOKIE("auto 123");
                         ^
1 error generated.
```

Ref: https://jenkins.ceph.com/job/ceph-pull-requests-arm64/59242/consoleFull#-5120798632a811ea2-3e7b-466b-84b4-d13df7e35809

Signed-off-by: Rongqi Sun <sunrongqi@huawei.com>
(cherry picked from commit 49869222c816023f696ff2a42898e2d9f3f54215)

qa/multisite: add object set acl test

Signed-off-by: Shilpa Jagannath <smanjara@redhat.com>
(cherry picked from commit d18b722557af8fc082d578547fa7be07a8ffc49e)

rgw/multisite: allow modified object acls to be replicated

Signed-off-by: Shilpa Jagannath <smanjara@redhat.com>
(cherry picked from commit f5f6194c07b40c50913e956e06aaa3709d4d53aa)

Merge pull request #58479 from afreen23/wip-backport-nfs

Batch backport NFS route separation for cephfs and rgw

Reviewed-by: Pedro Gonzalez Gomez <pegonzal@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>

qa/tasks/cephadm: drop --cluster and sudo in create_rbd_pool()

The cluster (name) is already specified in the arguments passed to
_shell() and this command doesn't need privileges.

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
(cherry picked from commit 26cee25873d5c91c704326fd0b708c53b353dd11)

qa/tasks/cephadm: don't wait for OSDs in create_rbd_pool()

This fails because teuthology.wait_until_osds_up() wants to use
adjust-ulimits wrapper which isn't available in "cephadm shell"
environment.  The whole thing is also redundant because cephadm task
is supposed to wait for OSDs to come up earlier, in ceph_osds().

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
(cherry picked from commit 0bda782952a81820752acdbf0b3ab9dbed56fb64)

qa/tasks: initialize rbd pool instead of just tagging it

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
(cherry picked from commit fcc440da9aaa6585872e830a5e39aefce1df82fe)

qa: krbd_data_pool.sh: account for rbd_trash object

... created since commit 0bcb5cfce9a2 ("librbd/image: create rbd_trash
object during RBD pool initialization").

Fixes: https://tracker.ceph.com/issues/66886
Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
(cherry picked from commit f8e5aecfc75bf7d32523e7a8d2d6034b434d216a)

rgw/datalog: fix LazyFIFO race fix

initialize `fifo_tmp` to prevent the race on member variable `fifo` that
was identified in 8fa844383f9c22e758f39ecdda74f70de054ad68

Fixes: https://tracker.ceph.com/issues/66880
Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit 5dfbaac15f0ffb3dfe84ae304643d1cf4eee892d)

Merge pull request #58255 from rhcs-dashboard/wip-66254-squid

squid: mgr/dashboard: rgw bucket replication management

Reviewed-by: Pedro Gonzalez Gomez <pegonzal@redhat.com>
Reviewed-by: Avan Thakkar <athakkar@redhat.com>

Merge pull request #58244 from rhcs-dashboard/wip-66425-squid

squid: mgr/dashboard: fix edit bucket failing in other selected gateways

Reviewed-by: Pedro Gonzalez Gomez <pegonzal@redhat.com>
Reviewed-by: nmunet <NOT@FOUND>

Merge pull request #58337 from rhcs-dashboard/wip-66737-squid

squid: mgr/dashboard: fix clone async validators with different groups

Reviewed-by: nmunet <NOT@FOUND>

Merge pull request #58436 from cbodley/wip-66839-squid

squid: doc/rgw: update s3 authentication

Reviewed-by: Zac Dover <zac.dover@proton.me>
Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>

install-deps.sh, do_cmake.sh: almalinux is another el flavour

Signed-off-by: Dan van der Ster <dan.vanderster@clyso.com>
(cherry picked from commit 6e85f76385c3cffd860a8a3054bf14ce85a43b59)

Merge pull request #57971 from adk3798/squid-adjust-bootstrap-config-priorities

squid: cephadm: the user config is preferred during the bootstrap

Reviewed-by: John Mulligan <jmulligan@redhat.com>

mgr/dashboard: fix nav urls for cephfs and nfs

Fixes: https://tracker.ceph.com/issues/66697
Signed-off-by: Nizamudeen A <nia@redhat.com>
(cherry picked from commit e8d0e76b63079f8d16c470030bda105637e1bb1e)

 Conflicts:
src/pybind/mgr/dashboard/frontend/src/app/core/navigation/navigation/navigation.component.html

mgr/dashboard: Fix NFS routing

Fixes https://tracker.ceph.com/issues/65310

The NFS tab in object and File nav uses same route due to which both
gets activated when one of them is clicked.
Hence, this PR separates the routing for Object and File nav.
Object-> NFS: /rgw/nfs
File-> NFS: /cephfs/nfs

Both routes use same NFS List and Form component but under different
routes as mentioned above.

Changes summary
- updated route for File from "/fs" to "/cephfs/<any_other_sub_route>"
  to support both fs and nfs tabs. Since using `/fs` and `/fs/nfs` will
activate both paths and it will be an undesirable user experience.
- `getFsalRouteFromPath` helper function to set the storage backend from
  route.
- removed `stoarge-backend` field from nfs form as now route decides teh
  storage backend
- breadcrumbs redirect to respective navs
- updated e2e tests
- updated unit tests
- changes list page of object-> nfs page to say Bucket instead of Path

Signed-off-by: Afreen <afreen23.git@gmail.com>
(cherry picked from commit 97359d6b1ee8133afcddf1f540b147c26723e24e)

Conflicts:
src/pybind/mgr/dashboard/frontend/src/app/core/navigation/navigation/navigation.component.html

mgr/dashboard: fix service form count bugs

Fixes https://tracker.ceph.com/issues/66608

- for services which do not have a count set default count to be null, otherwise the previous selected service's count is used which is wrong
- make count null when label is selected for placement

Signed-off-by: Afreen Misbah <afreen23.git@gmail.com>
(cherry picked from commit 27a8b2f2883fcc1fc4128b3e8995a7819a5ae51e)

Conflicts:
src/pybind/mgr/dashboard/frontend/src/app/ceph/cluster/services/service-form/service-form.component.html
src/pybind/mgr/dashboard/frontend/src/app/core/navigation/navigation/navigation.component.html

mgr/dashboard: fix service page e2e tests

- service page now uses defaults value for the placement count due to which mds test failing
- in test we pass "1" while "2" which is the default count for mds is already populated, making it 21 and causing unable to create mds service

Fixes: https://tracker.ceph.com/issues/66540
Signed-off-by: Afreen Misbah <afreen23.git@gmail.com>
(cherry picked from commit 5a718229dc43953f140a16342f19523a1c075863)
(cherry picked from commit 1da0a4f8cb234ef84729c24cc9a82397361e9534)

Conflicts:
src/pybind/mgr/dashboard/frontend/cypress/e2e/pools/pools.e2e-spec.ts
src/pybind/mgr/dashboard/frontend/cypress/e2e/pools/pools.po.ts

mgr/dashboard: Configure NVMe/TCP

Fixes https://tracker.ceph.com/issues/63686

- creation of Nvme-oF/TCP service
- deletion of Nvme-oF/TCP service
- edit/update Nvme-oF/TCP service
- added unit tests for Nvme-oF/TCP service
- changed Id -> Service Name
- added prefix of service type in service name (similar to <client.> in
  fs access)
- service name and pool are required fields for nvmeof
- placement count now takes default value as mentioned in cephadm
- slight refactors
- prepopulate serviceId for each service type setServiceId()
- in case serviceId is same as servcie type then do not add create service name with<servicetype>.<setrviceid> format

Signed-off-by: Afreen <afreen23.git@gmail.com>
(cherry picked from commit c6cf91766c45cf766c1d5a3851c2e956a80ac9ee)

Merge pull request #58488 from zdover23/wip-doc-2024-07-10-backport-58484-to-squid

squid: doc: Correct link to "Device management"

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>

Merge pull request #58468 from zdover23/wip-doc-2024-07-09-backport-58454-to-squid

squid: doc/cephfs: fix typo

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>

doc: Correct link to "Device management"

The current link goes to `cephadm/services/rados/operations/devices` which is a 404, it instead needs to be `rados/operations/devices/`.

Signed-off-by: Matthew Vernon <mvernon@wikimedia.org>
(cherry picked from commit d1a7cc4978adacd4e11df2d3e0886b191b118cc1)

Merge pull request #58443 from cbodley/wip-66847-squid

squid: rgw: fix multipart get part when count==1

Reviewed-by: Shilpa Jagannath <smanjara@redhat.com>

Merge pull request #58089 from ronen-fr/wip-rf-57865-squid

squid: osd/scrub: allow new scrubs while reserving

Reviewed-by: Samuel Just <sjust@redhat.com>

Merge pull request #57623 from sajibreadd/wip-65937-squid

squid: os/bluestore: set rocksdb iterator bounds for Bluestore::_collection_list()

Reviewed-by: Igor Fedotov <ifedotov@suse.com>

Merge pull request #57382 from pdvian/wip-65890-squid

squid: mgr: integrate optracker in ceph-mgr

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
Reviewed-by: Michael Kidd <linuxkidd@gmail.com>

Merge pull request #56200 from ifed01/wip-ifed-fix-64443-squi

squid: test/store_test: fix DeferredWrite test when prefer_deferred_size=0

Reviewed-by: Adam Kupczyk <akupczyk@redhat.com>
Reviewed-by: Pere Diaz Bou <pdiazbou@redhat.com>

doc/cephfs: fix typo

blanacer -> balancer

Signed-off-by: spdfnet <32593931+spdfnet@users.noreply.github.com>
(cherry picked from commit d94f90b757189e269882523178a401cd4411a946)

Merge pull request #58072 from yuvalif/wip-66499-squid

squid: rgw: do not assert on thread name setting failures

Reviewed-by: Daniel Gryniewicz <dang@redhat.com>

Merge pull request #57784 from yuvalif/wip-66288-squid

squid: rgw/kafka: do not destroy the connection on errors

Reviewed-by: Casey Bodley <cbodley@redhat.com>

rgw/admin/notification: add command to dump notifications

command will dump pending notifications from a persistent queue in JSON
format:

radosgw-admin topic dump --topic <name>

Fixes: https://tracker.ceph.com/issues/66404
Signed-off-by: Yuval Lifshitz <ylifshit@ibm.com>
(cherry picked from commit e63e663dc9095589f4fe28d541fd5178c033bffd)

Conflicts:
src/rgw/driver/rados/rgw_notify.cc
        src/test/rgw/bucket_notification/test_bn.py

Merge pull request #57996 from kchheda3/wip-66443-squid

squid:  rgw/notification: Store the value of `persistent_queue` for existing topics and continue commiting events for all topics subscribed to given bucket

Reviewed-by: Casey Bodley <cbodley@redhat.com>

rgw: fix multipart get part when count==1

the RGWObjManifest for multipart uploads is subtly different when
there's only a single part. in that case, get_cur_part_id() for the
final rule returns 1 where it otherwise returns (parts_count + 1)

this caused two problems:
* we returned a parts_count of 0 instead 1, and
* the do-while loop got stuck in an infinite loop expecting the last
  rule's part id to be higher than the requested part id

Fixes: https://tracker.ceph.com/issues/66705
Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit 562e0ce9baac53e714d91a46b4f0114b82827026)

Merge pull request #58420 from guits/wip-66830-squid

squid: qa: fix log errors for cephadm tests

doc/rgw: update s3 authentication

authentication.rst described the steps to generate a v2 signature,
without reference to aws docs. replace that with sections that reference
aws docs for v2 and v4 signatures. list which values of the request
header x-amz-content-sha256 are supported for v4

Fixes: https://tracker.ceph.com/issues/42888
Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit 05cab64b9ce3662f0d3e9673dfbbb5d42906caf7)

Merge pull request #58431 from zdover23/wip-doc-2024-07-05-backport-58368-to-squid

squid: doc/rados: document manually passing search domain

doc/rados: document manually passing search domain

Document how to manually pass the search domain to "mon_dns_srv_name" in
doc/rados/configuration/mon-lookup-dns.rst.

This commit is made in response to a request by Lander Duncan that was made on the [ceph-users] mailing list, and can be seen here: https://lists.ceph.io/hyperkitty/list/ceph-users@ceph.io/thread/F7V4CWLIYCAJ4JXI2JLNY6QPCFPR4SLA/

Co-authored-by: Anthony D'Atri <anthony.datri@gmail.com>
Signed-off-by: Zac Dover <zac.dover@proton.me>
(cherry picked from commit 98938a0312dd0c8e0b293ed9aa2e0760cc9619fa)

Merge pull request #58422 from AliMasarweh/wip-66772-squid

squid: RGW|Bucket notification: reload realm correctly

Reviewed-by: yuvalif<ylifshit@redhat.com>