qa/standalone/scrub: increase status updates frequency

To prevent test timeouts.
Also - remove a failing assertion on a specific 'pg query'
output, as it is not central to the test.

Fixes: https://tracker.ceph.com/issues/61385
(cherry picked from commit 6850bce554330d09f476d4e2aa8ac6d5e3f8bf12)

Conflict resolution:
- removed references to perf counters (not part of the original fix)
- related conf values in TEST_scrub_test - not removed (as directly
  related)

Signed-off-by: Ronen Friedman <rfriedma@redhat.com>

Merge pull request #59952 from rhcs-dashboard/wip-68200-reef

reef: mgr/dashboard: Cloning subvolume not listing _nogroup if no subvolume

Reviewed-by: Afreen Misbah <afreen23.git@gmail.com>

Merge pull request #59940 from zdover23/wip-doc-2024-09-24-backport-59854-to-reef

reef: doc: update nfs doc for Kerberos setup of ganesha in Ceph

Reviewed-by: Avan Thakkar <athakkar@redhat.com>

mgr/dashboard: Cloning subvolume not listing _nogroup subvolumegroup if there are no subvols in _nogroup
Fixes: https://tracker.ceph.com/issues/67891
Signed-off-by: Dnyaneshwari talwekar <dtalweka@redhat.com>
(cherry picked from commit 5c6c4a07d8dcd7bde46057310fbd1c5580a0da2f)

 Conflicts:
src/pybind/mgr/dashboard/frontend/src/app/ceph/cephfs/cephfs-subvolume-snapshots-list/cephfs-subvolume-snapshots-list.component.ts

doc: update nfs doc for Kerberos setup of ganesha in Ceph

Signed-off-by: Avan Thakkar <athakkar@redhat.com>
(cherry picked from commit 0f6039f9afa223ad70601d4dc36fc99f6fd78b49)

Merge pull request #59792 from ronen-fr/wip-rf-59636-to-reef

reef: osd/scrub: decrease default deep scrub chunk size

Reviewed-by: Sridhar Seshasayee <sseshasa@redhat.com>

Merge pull request #59908 from zdover23/wip-doc-2024-09-21-backport-59904-to-reef

reef: doc: Update "Getting Started" to link to start not install

doc: Update "Getting Started" to link to start not install

Currently the "Getting Started" link points at the install page, not the "Intro to Ceph" section, which seems more appropriate.

If you're happy with this, can it get backported to the still-supported releases, please?

Signed-off-by: Matthew Vernon <mvernon@wikimedia.org>
(cherry picked from commit d257eacb05a463c93872ae8aa8210806affda4ec)

Merge pull request #59877 from aaSharma14/wip-68145-reef

reef: mgr/dashboard: Zone details showing incorrect data for data pool values and compression info for Storage Classes

Reviewed-by: Nizamudeen A <nia@redhat.com>

Merge pull request #59872 from zdover23/wip-doc-2024-09-19-backport-59856-to-reef

reef: doc/rados: add confval directives to health-checks

Merge pull request #59852 from rhcs-dashboard/reef-remove-orch-required-decorator

Reef: mgr/dashboard: remove orch required decorator from host UI router (list)

Reviewed-by: Pedro Gonzalez Gomez <pegonzal@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>

Merge pull request #59876 from rhcs-dashboard/wip-68143-reef

reef: mgr/dashboard: Wrong(half) uid is observed in dashboard

Reviewed-by: Pedro Gonzalez Gomez <pegonzal@redhat.com>

mgr/dashboard: Zone details showing incorrect data for data pool values and compression info for Storage Classes

Fixes: https://tracker.ceph.com/issues/67103
Signed-off-by: Aashish Sharma <aasharma@redhat.com>
(cherry picked from commit 76af1c0b1be562d38f3d6323351b7b7d0462af48)

mgr/dashboard: Wrong(half) uid is observed in dashboard

Fixes: https://tracker.ceph.com/issues/67850
Signed-off-by: Dnyaneshwari Talwekar <dtalweka@redhat.com>
(cherry picked from commit 7ca0d71eada6d7a0d8cce5b6e80d328c453ef7df)

doc/rados: add confval directives to health-checks

Add confval directives to doc/rados/operations/health-checks.rst, as
requested by Anthony D'Atri here: https://github.com/ceph/ceph/pull/59635#pullrequestreview-2286205705

Signed-off-by: Zac Dover <zac.dover@proton.me>
(cherry picked from commit a159821ddfcecaa75f5a92af7e22ea198d82a8db)

Merge pull request #59835 from zdover23/wip-doc-2024-09-17-backport-59788-to-reef

reef: doc/README.md: create selectable commands

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>

mgr/dashboard: remove orch required decorator from host UI router (list)

https://tracker.ceph.com/issues/68101

Signed-off-by: Naman Munet <nmunet@redhat.com>
(cherry picked from commit 21454d08484cefefef7803c250d4d0511ae4415b)

doc/README.md: create selectable commands

An indentation of five spaces relative to the previous line creates a
command that is copyable with a single mouse click. This commit adds
those copyabale commands to the procedure in the section "Building
Ceph".

Signed-off-by: Zac Dover <zac.dover@proton.me>
(cherry picked from commit 36c620b04f3a88562d75955b46cc1864de60ce70)

Merge pull request #59761 from rhcs-dashboard/reef-hosts-selection-show-only-10-entries-fix

Reef: mgr/dashboard: service form hosts selection only show up to 10 entries

Reviewed-by: afreen23 <NOT@FOUND>

Merge pull request #56819 from cbodley/wip-65410-reef

reef: qa/rgw: barbican uses branch stable/2023.1

Merge pull request #59803 from zdover23/wip-doc-2024-09-16-backport-59801-to-reef

reef: doc/rados: add osd_deep_scrub_interval setting operation

doc/rados: add osd_deep_scrub_interval setting operation

Add a second method of changing the value of osd_deep_scrub_interval to
remedy the condition indicated by the "PGs not deep-scrubbed in time"
warning.

This procedure was developed by Eugen Block, and is at the time of this
commit available on his blog at
https://heiterbiswolkig.blogs.nde.ag/2024/09/06/pgs-not-deep-scrubbed-in-time/

Co-authored-by: Eugen Block <eblock@nde.ag>
Signed-off-by: Zac Dover <zac.dover@proton.me>
(cherry picked from commit f57e99e1731af3affee1c299bb9a79852e31f5ea)

Merge pull request #59795 from zdover23/wip-doc-2024-09-15-backport-59613-to-reef

reef: doc/rados: add link to messenger v2 info in mon-lookup-dns.rst

Merge pull request #59799 from zdover23/wip-doc-2024-09-15-backport-59024-to-reef

reef: doc/README.md - add ordered list

doc/README.md - add ordered list

Add ordered-list formatting to the "Building Ceph" section of README.md.

Signed-off-by: Zac Dover <zac.dover@proton.me>
(cherry picked from commit 1f21057e4166f2ea9f51ee037eb8e234c1390cfb)

doc/rados: add link to messenger v2 info in mon-lookup-dns.rst

Add a link to the page about Messenger v2 to the end of
doc/rados/configuration/mon-lookup-dns.rst.

Fixes: https://tracker.ceph.com/issues/58752
Signed-off-by: Zac Dover <zac.dover@proton.me>
(cherry picked from commit 81f9d064f714f42ff27bf620df526f87b393402b)

osd/scrub: decrease default deep scrub chunk size

The previous default of 25 objects per chunk proved to take too long
(many hundreds of milliseconds) on a busy cluster. As the scrubber
locks all objects in the chunk for the duration, a large chunk size
can cause a significant impact on the client ops' latencies.

Fixes: https://tracker.ceph.com/issues/68057
Signed-off-by: Ronen Friedman <rfriedma@redhat.com>
(cherry picked from commit 8c2ed94e9cbc7b1ed84b034a65b126e16b18a596)

Merge pull request #59701 from zdover23/wip-doc-2024-09-10-backport-59675-to-reef

reef: doc/README.md: improve formatting

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>

Merge pull request #59638 from zdover23/wip-doc-2024-09-06-backport-59630-to-reef

reef: doc/README.md: edit "Build Prerequisites"

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>

Merge pull request #59768 from zdover23/wip-doc-2024-09-13-backport-59438-to-reef

reef: doc/rgw/d3n: pass cache dir volume to extra_container_args

doc/rgw/d3n: pass cache dir volume to extra_container_args

Fixes: https://tracker.ceph.com/issues/66492
Signed-off-by: Mark Kogan <mkogan@ibm.com>
(cherry picked from commit 4594e38bf19a8f73191904f5f373a3707cddd2f3)

Merge pull request #59757 from zdover23/wip-doc-2024-09-12-backport-59753-to-reef

reef: doc/install: Keep the name field of the created user consistent with …

mgr/dashboard: service form hosts selection only show up to 10 entries

exposed new UIRouter API to get all hosts

Fixes: https://tracker.ceph.com/issues/67413
Signed-off-by: Naman Munet <nmunet@redhat.com>
(cherry picked from commit ec3e4abeeb7feebd996681a7d8d4e950de5cf518)

 Conflicts:
src/pybind/mgr/dashboard/frontend/src/app/ceph/block/nvmeof-listeners-form/nvmeof-listeners-form.component.ts (file deleted)
src/pybind/mgr/dashboard/frontend/src/app/ceph/cephfs/cephfs-form/cephfs-form.component.ts (resolved conflicts)

doc/install: Keep the name field of the created user consistent with the node name in the Start RADOSGW service command

If the user name does not match the name of the node that started the RADOSGW service, this will cause confusion for those who are new to ceph. Because they can't start the radosgw service normally as shown in the tutorial.

Signed-off-by: hejindong <wodeshijie3394@126.com>
(cherry picked from commit 128e6451d8c4764025d4a2565f15ec175c857bb2)

Merge pull request #59734 from zdover23/wip-doc-2024-09-11-backport-59635-to-reef

reef: doc/rados: add "pgs not deep scrubbed in time" info

Merge pull request #59655 from zdover23/wip-doc-2024-09-08-backport-59628-to-reef

reef: doc/dev: add "activate latest release" RTD step

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>

doc/rados: add "pgs not deep scrubbed in time" info

Add a procedure to doc/rados/operations/health-warnings.rst that
explains how to remedy the "X PGs not deep-scrubbed in time" health
warning.

This procedure was developed by Eugen Block, and is at the time of this
commit available on his blog at
https://heiterbiswolkig.blogs.nde.ag/2024/09/06/pgs-not-deep-scrubbed-in-time/

Co-authored-by: Eugen Block <eblock@nde.ag>
Signed-off-by: Zac Dover <zac.dover@proton.me>
(cherry picked from commit d620a51c306d1ebcdfc974dc0b85dc46cfde8d40)

Merge pull request #59683 from zdover23/wip-doc-2024-09-10-backport-59677-to-reef

reef: doc/man: edit ceph-bluestore-tool.rst

doc/README.md: improve formatting

Improve the formatting of the section "Building Ceph" in the file
README.md.

Signed-off-by: Zac Dover <zac.dover@proton.me>
(cherry picked from commit 7171f73c81d603f8ff62198fc1e5f50c20a0aa05)

doc/man: edit ceph-bluestore-tool.rst

Edit the section "bluefs-bdev-migrate" in
doc/man/8/ceph-bluestore-tool.rst to add the information that this
operation expands the target storage by updating its size label, making
"bluefs-bdev-expand" unnecessary.

Improve the subject-verb agreement in this section, and supply some
absent definite articles.

Co-authored-by: Peter Gervai <grin@drop.grin.hu>
Signed-off-by: Zac Dover <zac.dover@proton.me>
(cherry picked from commit 6b34707f827b2b197f53fe2e430d173b30b81401)

Merge pull request #59454 from adk3798/wip-65418-reef

reef: cephadm: fix host-maintenance command always exiting with a failure

Reviewed-by: Guillaume Abrioux <gabrioux@ibm.com>

Merge pull request #59664 from zdover23/wip-doc-2024-09-09-backport-59642-to-reef

reef: doc/cephadm: add malformed-JSON removal instructions

Merge pull request #59451 from adk3798/wip-65379-reef

reef: mgr/cephadm: make client-keyring deploying ceph.conf optional

Reviewed-by: Guillaume Abrioux <gabrioux@ibm.com>

Merge pull request #59453 from adk3798/wip-65416-reef

reef: cephadm: add idmap.conf to nfs sample file

Reviewed-by: Guillaume Abrioux <gabrioux@ibm.com>

Merge pull request #59458 from adk3798/wip-65949-reef

reef: mgr/pybind/object_format: fix json-pretty being marked invalid

Reviewed-by: Guillaume Abrioux <gabrioux@ibm.com>

Merge pull request #59459 from adk3798/wip-65952-reef

reef: mgr/cephadm: cleanup iscsi and nvmeof keyrings upon daemon removal

Reviewed-by: Guillaume Abrioux <gabrioux@ibm.com>

doc/cephadm: add malformed-JSON removal instructions

Add instructions for removing malformed JSON that causes the cephadm
module to fail to start, which in turn makes the orchestrator fail to
start.

Signed-off-by: Zac Dover <zac.dover@proton.me>
(cherry picked from commit 93d90ea358cf17b08e502caf8529061a10008fad)

Merge pull request #59601 from idryomov/wip-67752-reef

reef: rbd: amend "rbd {group,} rename" and "rbd mirror pool" command descriptions

Reviewed-by: Ramana Raja <rraja@redhat.com>

doc/dev: add "activate latest release" RTD step

Add a step to the section called "First Stable Release" that explains
how to activate the latest release in Read the Docs.

Signed-off-by: Zac Dover <zac.dover@proton.me>
(cherry picked from commit e7bd4bcdd074a3defae5866a6c035559bf210271)

Merge pull request #59262 from k0ste/wip-63920-reef

reef: ceph-volume: Fix unbound var in disk.get_devices()

Merge pull request #59201 from k0ste/wip-67528-reef

reef: debian pkg: record python3-packaging dependency for ceph-volume

Merge pull request #58687 from kamoltat/wip-ksirivad-backport-reef-57906

reef: mon/ElectionLogic: tie-breaker mon ignore proposal from marked down mon
Reviewed-by: Samuel Just <sjust@redhat.com>

Merge pull request #59463 from adk3798/wip-67733-reef

reef: qa/distros: reinstall nvme-cli on centos 9 nodes

doc/README.md: edit "Build Prerequisites"

Make the sentences in the "Build Prerequisites" section of README.md
more direct.

Signed-off-by: Zac Dover <zac.dover@proton.me>
(cherry picked from commit 0b277fe3809c2eae7bfced002ac85f7e1629fa30)

ceph-volume: add "packaging" to install_requires

in 0985e201, "packaging" was introduced as a runtime dependency of
ceph-volume, and `ceph.spec.in` was updated accordingly to note
this new dependency. but the debian packaging was not updated.

in 80edcd40, the missing dependency was added to debian/control as
one of ceph-volume's runtime dependency.

but dh_python3 is able to figure out the dependencies by reading
the egg's metadata of the ceph-volume python module. and as a
python project, ceph-volume is using its `setup.py` for
tracking its dependencies.

so in order to be more consistent, and keep all of its dependencies
in one place, let's move this dependency to setup.py . as the
packagings in both distros are able to figure the dependencies
from egg-info.

see also
- https://manpages.debian.org/testing/dh-python/dh_python3.1.en.html#dependencies
- https://docs.fedoraproject.org/en-US/packaging-guidelines/Python_201x/#_automatically_generated_dependencies

Signed-off-by: Kefu Chai <tchaikov@gmail.com>
(cherry picked from commit 729fd8e25ff2bfbcf99790d6cd08489d1c4e2ede)

debian pkg: record python3-packaging dependency for ceph-volume

Since commit 0985e201342 ("ceph-volume: use 'no workqueue' options
with dmcrypt") the python "packaging" module is used to parse the
cryptsetup version output, but the debian packaging was not updated to
record that new dependency.

So simply record this in the d/control file, adding a <pkg>.requires
file seemed to not really winning us anything here.

Fixes: https://tracker.ceph.com/issues/67290
Fixes: 0985e201342fa53c014a811156aed661b4b8f994
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
(cherry picked from commit 80edcd40e8092d9fb3b45c1a6c7f9b7f4f37b69e)

rbd: mention namespace in "rbd mirror pool" command descriptions

Commit 5e64748927d0 ("doc/rbd: add namespace information for mirror
commands") did this for the man page, update the built-in help as well.

The "by default" bit in the description of "rbd mirror pool enable" and
"rbd mirror pool disable" commands is specific to pool mode which is in
turn specific to journal-based mirroring, so it's removed.

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
(cherry picked from commit 7e02946a3f60da0dcc99e0b4a4b6b3d359db1c15)

rbd: clarify "rbd {group,} rename" command description

Make it clearer that, despite a full image or group spec being taken
for source and destination, an image or a group can be renamed only
within its pool or namespace.

Rename across pools or namespaces within the same pool is unsupported.

Fixes: https://tracker.ceph.com/issues/67752
Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
(cherry picked from commit 5f064a52707af58fd6007734a768899aeb5a32fc)

Merge pull request #59583 from zdover23/wip-doc-2024-09-04-backport-59546-to-reef

reef: doc/rados/operations: Improve health-checks.rst

Merge pull request #59586 from zdover23/wip-doc-2024-09-04-backport-59577-to-reef

reef: doc/mds: improve wording

doc/mds: improve wording

Signed-off-by: Piotr Parczewski <piotr@stackhpc.com>
(cherry picked from commit 332804bad58c892d01d2d2da557e42104365ef8a)

doc/rados/operations: Improve health-checks.rst

Signed-off-by: Anthony D'Atri <anthonyeleven@users.noreply.github.com>
(cherry picked from commit 2aa82539ede8ade095ad0b22e973eac538354711)

Merge pull request #59560 from zdover23/wip-doc-2024-09-02-backport-59556-to-reef

reef: doc: Correct link to Prometheus docs

doc: Correct link to Prometheus docs

The link is to the `#http_sd_config` anchor in the prometheus config docs; that link only works without the trailing `/`.

This correction would ideally get backported to at least reef & squid.

Signed-off-by: Matthew Vernon <mvernon@wikimedia.org>
(cherry picked from commit 84a30ba6b94b34806faac8217ccaa299c9ee68d6)

Merge pull request #59549 from zdover23/wip-doc-2024-09-01-backport-59544-to-reef

reef: doc: update tests-integration-testing-teuthology-workflow.rst

doc: update tests-integration-testing-teuthology-workflow.rst

* add "Infrastructure" section.

* move "Naming the ceph-ci branch" section under
   "Getting binaries - Build Ceph". Also mention
   about centos9-only trick.

* in "Teuthology Archives", mention about developer
   playground machines and ceph log files.

Signed-off-by: Vallari Agrawal <val.agl002@gmail.com>
(cherry picked from commit 9bfcb8e17db8c61e523e10856d12b237433d831a)

Merge pull request #59541 from zdover23/wip-doc-2024-08-31-backport-59528-to-reef

reef: doc/ceph-volume: add spillover fix procedure

doc/ceph-volume: add spillover fix procedure

Add a procedure that explains how, after an upgrade, to move bytes that
have spilled over to a relatively slow device back to the faster device.

This procedure was developed by Chris Dunlop on the [ceph-users] mailing
list, here: https://lists.ceph.io/hyperkitty/list/ceph-users@ceph.io/message/POPUFSZGXR3P2RPYPJ4WJ4HGHZ3QESF6/

Eugen Block requested the addition of this procedure to the
documentation on 30 Aug 2024.

Co-authored-by: Anthony D'Atri <anthony.datri@gmail.com>
Signed-off-by: Zac Dover <zac.dover@proton.me>
(cherry picked from commit 98618aaa1c8b786c7d240a210b62cc737fdb048d)

Merge pull request #59462 from adk3798/wip-66428-reef

reef: mgr/cephadm: make SMB and NVMEoF upgrade last in staggered upgrade

Reviewed-by: Guillaume Abrioux <gabrioux@ibm.com>

Merge pull request #59461 from adk3798/wip-66426-reef

reef: cephadm: CephExporter doesn't bind to IPv6 in dual stack

Reviewed-by: Guillaume Abrioux <gabrioux@ibm.com>

Merge pull request #59460 from adk3798/wip-65969-reef

reef: mgr/cephadm: make setting --cgroups=split configurable for adopted daemons

Reviewed-by: Guillaume Abrioux <gabrioux@ibm.com>

Merge pull request #59455 from adk3798/wip-65723-reef

reef: cephadm: have agent check for errors before json loading mgr response

Reviewed-by: Guillaume Abrioux <gabrioux@ibm.com>

Merge pull request #57519 from asm0deuz/backport_PR54158

reef: cephadm: added check for `--skip-firewalld` to section on adding explicit Ports to firewalld

Reviewed-by: Adam King <adking@redhat.com

Merge pull request #57234 from adk3798/wip-65763-reef

reef: mgr/cephadm: set OSD cap for NVMEoF daemon to "profile rbd"

Reviewed-by: Guillaume Abrioux <gabrioux@ibm.com>

Merge pull request #56909 from adk3798/wip-65383-reef

reef: mgr/cephadm: Allows enabling NFS Ganesha NLM

Reviewed-by: Guillaume Abrioux <gabrioux@ibm.com>

Merge pull request #56490 from adk3798/wip-64991-reef

reef: cephadm: fix `cephadm shell --name <daemon-name>` for stopped/failed daemon

Reviewed-by: John Mulligan <jmulligan@redhat.com>

Merge pull request #59268 from k0ste/wip-64671-reef

reef: qa/tasks/ceph_manager.py: Rewrite test_pool_min_size
Reviewed-by: Kamoltat Sirivadhna <ksirivad@redhat.com>

qa/distros: reinstall nvme-cli on centos 9 nodes

To work around a potential linking issue between
nvme-cli ad libnvme that prevents nvme-cli from
correctly generating a hostnqn, causing

nvme_fabrics: found same hostid edb4e426-766f-44c6-b127-da2a5b7446ef but different hostnqn hostnqn

messages in dmesg and the inability to setup nvme
loop devices

Fixes: https://tracker.ceph.com/issues/67684
Signed-off-by: Adam King <adking@redhat.com>
(cherry picked from commit 4e5f269c01dd94741fbddaa7f76ebe4ac155e650)

mgr/cephadm: make SMB and NVMEoF upgrade last in staggered upgrade

This needs to happen as some work on the NVMEoF side (still unmerged
as of writing this) will make the NVMEoF daemon dependent on the mon.
Prior to this patch, in a staggered upgrade, all daemons not using the
ceph image were upgraded after the mgr since we typically only care
about the default image changing or potential changes to how we handle
our systemd units which only needs the mgr to be upgraded to be applied.
This NVMEoF dependency on the mon changes this and we can no longer
upgrade it directly after the mgr. This patch changes it so the NVMEoF
daemon is instead upgraded after all ceph image daemons have been
upgraded in a staggered upgrade scenario. Non-staggered upgrades
are unaffected as the NVMEoF daemon was already upgraded near the
end in that scenario. The SMB dameon has no reason it needs to be
upgraded later, but it's in the (small) pool of daemons that don't
use the ceph image and aren't for monitoring, so it's been affected
by this as well.

NOTE: This is a bit of an ugly patch imo and shows that a refactoring
of the upgrade code is likely required. Hopefully this patch is more
of a stopgap until that larger effort can be made

Fixes: https://tracker.ceph.com/issues/65809
Signed-off-by: Adam King <adking@redhat.com>
(cherry picked from commit 5e7a3c2147d87c1fc5be71acbadedefb70e024bf)

Fix CephExporter protocol bind logic

In a dual stack configuration ceph-exporter binds to ipv4 only and the metrics fail in ipv6

Signed-off-by: Mouratidis Theofilos <mtheofilos@gmail.com>
(cherry picked from commit 110bc665078fe19c31e3680c4197587e69e4e751)

Conflicts:
src/cephadm/cephadmlib/daemons/ceph.py

cephadm: Adding support to pass --no-cgroups-split flag when adopting legacy daemons

Signed-off-by: Gilad Sid <sid.gilad@gmail.com>
(cherry picked from commit 20ffd4d6e330095c8cf2816a36f61bd950e213a5)

mgr/cephadm: clean up nvmeof keyring when removing daemon

The parent class' (CephService) post_remove handles
keyring cleanup

Fixes: https://tracker.ceph.com/issues/65717
Signed-off-by: Adam King <adking@redhat.com>
(cherry picked from commit 04065e34d74421fd1f85b52b87efaf357fe88505)

Conflicts:
src/pybind/mgr/cephadm/services/nvmeof.py

mgr/cephadm: clean up iscsi auth keyring when removing daemon

The parent class (CephService) handles the cleanup of
the auth keyring

Fixes: https://tracker.ceph.com/issues/65717
Signed-off-by: Adam King <adking@redhat.com>
(cherry picked from commit f7e8ee74d5bfb941a5f67d0e03c3845f5cdb1525)

mgr/pybind/object_format: fix json-pretty being marked invalid

without this patch you'd get

```
[ceph: root@vm-00 /]# ceph nfs cluster info --format json-pretty

Error EINVAL: Unknown format name: json-pretty
```

this seems to be because valid formats are checked using
the class

```
class Format(str, enum.Enum):
    plain = "plain"
    json = "json"
    json_pretty = "json-pretty"
    yaml = "yaml"
    xml_pretty = "xml-pretty"
    xml = "xml"
```

and then

```
set(str(v) for v in Format.__members__)
```

but that resolves to

```
{'yaml', 'json_pretty', 'plain', 'xml', 'json', 'xml_pretty'}
```

and so json-pretty is marked as invalid. Note that it's also
impossible to pass json_pretty as the format as core ceph
blocks it with

invalid choice: 'json_pretty' (choose from 'json', 'json-pretty', 'xml', 'xml-pretty', 'plain', 'yaml')

Fixes: https://tracker.ceph.com/issues/65554
Signed-off-by: Adam King <adking@redhat.com>
(cherry picked from commit cd988a01a0edf13882527f5526f1793d4dece437)

cephadm: have agent check for errors before json loading mgr response

Currently, since it tries to json.loads the response
payload before checking the return code, if there was
an error it fails with

Failed to send metadata to mgr: the JSON object must be str, bytes or bytearray, not ConnectionRefusedError

which is masking the actual failure.

Also adds more context to the RuntimeError raised

Fixes: https://tracker.ceph.com/issues/65553
Signed-off-by: Adam King <adking@redhat.com>
(cherry picked from commit 287bd34eec09815602700747c62e0a709e6e5ff0)

cephadm: fix host-maintenance command always exiting with a failure

The host-maintenance command would always fail because
command_maintenance always returns a string. This string is passed to
sys.exit and thus always gets printed and causes a non-zero exit code.
Fix the command line behavior by renaming the original function and
adding a new command_maintenance that prints the string and returns an
int like other command_* functions do.

Fixes: https://tracker.ceph.com/issues/65122
Signed-off-by: John Mulligan <jmulligan@redhat.com>
(cherry picked from commit 0a137b140e47d02b03d133adb55e62aa0de7b984)

cephadm: add idmap.conf to nfs sample file

Otherwise deployment using this sample will fail with
"ERROR: required file missing from config-json: idmap.conf"

Fixes: https://tracker.ceph.com/issues/65155
Signed-off-by: Adam King <adking@redhat.com>
(cherry picked from commit 79b1dd5b30f8556aacc9a7536366694ba901795e)

doc/cephadm: document new --no-ceph-conf flag for `orch client-keyring set`

Signed-off-by: Adam King <adking@redhat.com>
(cherry picked from commit 2741d9471c959f07f9b55c52114ea9d71916345e)

mgr/cephadm: make client-keyring deploying ceph.conf optional

There are cases where users would like to manage their own
ceph.conf but still have cephadm deploy the client keyrings,
so this is being added to facilitate that.

Fixes: https://tracker.ceph.com/issues/65335
Signed-off-by: Adam King <adking@redhat.com>
(cherry picked from commit 0dab95eb4fffb493edc3e542a6613bdb5332a670)

mgr/cephadm: rename ceph_admin_key -> client_key in _calc_client_files

I think this variable name is misleading, as we use this same
logic for all client-keyrings that can be setup, not just
the client.admin keyring

Signed-off-by: Adam King <adking@redhat.com>
(cherry picked from commit 9e2327f0c5f0c103e2db8c7d86ceddd177179df9)

Merge pull request #59431 from zdover23/wip-doc-2024-08-26-backport-59428-to-reef

reef: doc/cephadm: how to get exact size_spec from device

doc/cephadm: how to get exact size_spec from device

Add instructions for retrieving the exact size of block devices.

Fixes: https://tracker.ceph.com/issues/66754
Signed-off-by: Zac Dover <zac.dover@proton.me>
(cherry picked from commit d00d1b52d50b5575d918c3be7b7a8249ef31f0a8)

Merge pull request #59425 from zdover23/wip-doc-2024-08-25-backport-59418-to-reef

reef: doc/glossary: add "object storage"

doc/glossary: add "object storage"

Add a (very basic) definition of object storage.

Signed-off-by: Zac Dover <zac.dover@proton.me>
(cherry picked from commit 43057b88538e868b817acb04d5e6c4e95b4c716e)

Merge pull request #57625 from sajibreadd/wip-65938-reef

reef: os/bluestore: set rocksdb iterator bounds for Bluestore::_collection_list()

Reviewed-by: Igor Fedotov <ifedotov@suse.com>

Merge pull request #57621 from sajibreadd/wip-66145-reef

reef: osd: CEPH_OSD_OP_FLAG_BYPASS_CLEAN_CACHE flag is passed from ECBackend

Reviewed-by: Samuel Just <sjust@redhat.com>

Merge pull request #55110 from k0ste/wip-63977-reef

reef: mgr/BaseMgrModule: Optimize CPython Call in Finish Function

Reviewed-by: Nitzan Mordechai <nmordech@redhat.com>

Merge pull request #53269 from YiteGu/backport-always-generate-random-nonce

reef: msg: always generate random nonce; don't try to reuse PID

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

Merge pull request #59411 from adk3798/wip-67682-reef

reef: mgr/cephadm: add "original_weight" parameter to OSD class

Reviewed-by: John Mulligan <jmulligan@redhat.com>

Merge pull request #59381 from zdover23/wip-doc-2024-08-21-backport-59348-to-reef

reef: doc/rados: document unfound object cache-tiering scenario

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>