orch: Fix `daemon add osd` boolean param handling

- Correct usage examples for 'ceph orch daemon add osd'
- Fix boolean argument handling for drive group specifications

Fixes: https://tracker.ceph.com/issues/68045
Signed-off-by: Guillaume Abrioux <gabrioux@ibm.com>
(cherry picked from commit 45327462e0949638b37f66f9cceff2ff5ef07242)

orch: refactor boolean handling in drive group spec

The intent of 42721c03ee6f was to address an issue where boolean
parameters weren't handled correctly.

I noticed that a parameter (`tpm2`) was missed, which made me realize
that maintaining a list of these boolean parameters is necessary.

To simplify things, we should only accept `"true"` or `"false"` (in any case),
allowing us to avoid the need to maintain a list of boolean parameters.

This change introduces a `list_drive_group_spec_bool_arg` to store boolean
arguments related to drive group specifications, simplifying the validation
process for boolean values by directly checking if the values are 'true' or 'false'.

Fixes: https://tracker.ceph.com/issues/68045
Signed-off-by: Guillaume Abrioux <gabrioux@ibm.com>
(cherry picked from commit e3d8a37ef6e31eaf69671cec6ee5b1ed11ca267d)

Merge pull request #59672 from rishabh-d-dave/wip-67961-squid

squid: cephfs,mon: fix bugs related to updating MDS caps

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #59813 from yaarith/squid-release-checklist-telemetry

squid: doc/dev/release-checklist: check telemetry validation

Reviewed-by:Anthony D'Atri <anthony.datri@gmail.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge pull request #58923 from NitzanMordhai/wip-67234-squid

squid: corpus: update submodule with mark cls_rgw_reshard_entry forward_inco…

Reviewed-by: Casey Bodley <cbodley@redhat.com>

cephfs: disallow removing root_squash via "fs authorize" cmd

Removing root_squasn from MDS auth caps through "fs authorize" command
should not be allowed as this command it not allowed to/meant for
removing caps.

Fixes: https://tracker.ceph.com/issues/65808
Signed-off-by: Rishabh Dave <ridave@redhat.com>
(cherry picked from commit c6e2c97c6e9cbf1e37c53d5d490d65091205928c)

Conflict:

- qa/tasks/cephfs/test_admin.py
  Test test_idem_unaffected_root_squash (which was fixed by this commit)
  was disabled on the main branch since it was buggy. But that wasn't
  the case with squid branch.

Merge pull request #59834 from zdover23/wip-doc-2024-09-17-backport-59788-to-squid

squid: doc/README.md: create selectable commands

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>

Merge pull request #59819 from afreen23/wip-backport-batch

squid: mgr/dashboard: Nvme mTLS support and service name changes

Reviewed-by: Pedro Gonzalez Gomez <pegonzal@redhat.com>

Merge pull request #59825 from rhcs-dashboard/squid-rgw-multisite-sync-policy-edit-zones-fix

Squid: mgr/dashboard: RGW multisite sync remove zones fix

Reviewed-by: afreen23 <NOT@FOUND>

Merge pull request #59827 from adk3798/squid-update-latest-stable-release

squid: cephadm: update default image and latest stable release

Reviewed-by: Guillaume Abrioux <gabrioux@ibm.com>

mgr/dashboard: Add mTLS support

- enables mTLS support from dashboard
- adds unit tests related to mTLS support
- can enable mTLS
- can disable mTLS
- inlcuded refactoring from prev commit

Fixes https://tracker.ceph.com/issues/66416

Signed-off-by: Afreen Misbah <afreen23.git@gmail.com>
(cherry picked from commit 1d01d0407ee08c3da338a323a168c454ecdd3bdf)

Conflicts:
src/pybind/mgr/dashboard/frontend/src/app/ceph/cluster/services/service-form/service-form.component.html

modified:   src/app/shared/models/service.interface.ts

modified:   src/app/ceph/cluster/services/service-form/service-form.component.html
modified:   src/app/ceph/cluster/services/service-form/service-form.component.spec.ts
modified:   src/app/ceph/cluster/services/service-form/service-form.component.ts
modified:   src/app/ceph/pool/pool-details/pool-details.component.spec.ts

Merge pull request #59811 from afreen23/wip-68079-squid

squid: mgr/dashboard: mgr/dashboard: Select no device by default in EC profile

Reviewed-by: Pedro Gonzalez Gomez <pegonzal@redhat.com>

squid: cephadm: update latest stable release and default image

Signed-off-by: Adam King <adking@redhat.com>

doc/README.md: create selectable commands

An indentation of five spaces relative to the previous line creates a
command that is copyable with a single mouse click. This commit adds
those copyabale commands to the procedure in the section "Building
Ceph".

Signed-off-by: Zac Dover <zac.dover@proton.me>
(cherry picked from commit 36c620b04f3a88562d75955b46cc1864de60ce70)

mgr/dashboard: RGW multisite sync remove zones fix

Fixes: https://tracker.ceph.com/issues/66945
Signed-off-by: Naman Munet <nmunet@redhat.com>
(cherry picked from commit 5eee7e8105167dc4f0542f62d041fbcb143611d0)

mgr/dashboard: Adding group and pool name to service name

- Pre-populating the service name field with the format `nvmeof.<pool_name>.<group_name>`.
- This can be changed by user but by default this value will be there.
- This will help user to quickly fill form and proceed hence improving usability.
- cephadm also uses this format as of now this convention so it will make UI aligned with CLI experience

- updates unit tests to improve coverage

- hides`count` values as that is not needed for 'nvmeof' only hosts and labels required.

Fixes https://tracker.ceph.com/issues/68065

Signed-off-by: Afreen Misbah <afreen23.git@gmail.com>
(cherry picked from commit e0e467a7a2011c991e487ab848750b59d79adc3f)

Merge pull request #59751 from rhcs-dashboard/wip-68041-squid

squid: mgr/dashboard: add gw_groups support to nvmeof api

Reviewed-by: Afreen Misbah <afreen23.git@gmail.com>

Merge pull request #59722 from aaSharma14/wip-67990-squid

squid: mgr/dashboard: use grafana server instead of grafana-server in grafana 10.4.0

Reviewed-by: Afreen Misbah <afreen23.git@gmail.com>

doc/dev/release-checklist: check telemetry validation

Signed-off-by: Yaarit Hatuka <yaarit@redhat.com>

Merge pull request #59760 from rhcs-dashboard/squid-hosts-selection-show-only-10-entries-fix

Squid: mgr/dashboard: service form hosts selection only show up to 10 entries

Reviewed-by: Afreen Misbah <afreen23.git@gmail.com>

Merge pull request #59798 from zdover23/wip-doc-2024-09-15-backport-59024-to-squid

squid: doc/README.md - add ordered list

mgr/dashboard: mgr/dashboard: Select no device by default in EC profile

Fixes https://tracker.ceph.com/issues/67853

When EC pools are created with device class specified, the pools are created with just 1 PG and autoscaler does not work.
PG autoscaler not working on a cluster where pools have multiple overlapping roots is a known issue, and bug is raised for same :>

Issue documented already : https://docs.ceph.com/en/reef/rados/operations/placement-groups/#viewing-pg-scaling-recommendations

Also renames "let ceph decide" option to "All devices" in crush rule and ec profile component.
Updates unit tests for ec profile modal
Signed-off-by: Afreen Misbah <afreen23.git@gmail.com>
(cherry picked from commit 4af51349e5e1d21c200b6bf7db81fa18eb163a61)

Merge pull request #59802 from zdover23/wip-doc-2024-09-16-backport-59801-to-squid

squid: doc/rados: add osd_deep_scrub_interval setting operation

doc/rados: add osd_deep_scrub_interval setting operation

Add a second method of changing the value of osd_deep_scrub_interval to
remedy the condition indicated by the "PGs not deep-scrubbed in time"
warning.

This procedure was developed by Eugen Block, and is at the time of this
commit available on his blog at
https://heiterbiswolkig.blogs.nde.ag/2024/09/06/pgs-not-deep-scrubbed-in-time/

Co-authored-by: Eugen Block <eblock@nde.ag>
Signed-off-by: Zac Dover <zac.dover@proton.me>
(cherry picked from commit f57e99e1731af3affee1c299bb9a79852e31f5ea)

Merge pull request #59794 from zdover23/wip-doc-2024-09-15-backport-59613-to-squid

squid: doc/rados: add link to messenger v2 info in mon-lookup-dns.rst

doc/README.md - add ordered list

Add ordered-list formatting to the "Building Ceph" section of README.md.

Signed-off-by: Zac Dover <zac.dover@proton.me>
(cherry picked from commit 1f21057e4166f2ea9f51ee037eb8e234c1390cfb)

doc/rados: add link to messenger v2 info in mon-lookup-dns.rst

Add a link to the page about Messenger v2 to the end of
doc/rados/configuration/mon-lookup-dns.rst.

Fixes: https://tracker.ceph.com/issues/58752
Signed-off-by: Zac Dover <zac.dover@proton.me>
(cherry picked from commit 81f9d064f714f42ff27bf620df526f87b393402b)

Merge pull request #59785 from zdover23/wip-doc-2024-09-14-backport-59732-to-squid

squid: doc/README.md: improve formatting

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>

doc/README.md: improve formatting

Improve the formatting in the section "Building Ceph" in the file
README.md.

Signed-off-by: Zac Dover <zac.dover@proton.me>
(cherry picked from commit b9ca3957303989bbba9301cbbd18ba8faa0b8168)

Merge pull request #59767 from zdover23/wip-doc-2024-09-13-backport-59438-to-squid

squid: doc/rgw/d3n: pass cache dir volume to extra_container_args

Merge pull request #59718 from afreen23/wip-67994-squid

squid: mgr/dashboard: Update nvmeof microcopies

Reviewed-by: Afreen Misbah <afreen23.git@gmail.com>

doc/rgw/d3n: pass cache dir volume to extra_container_args

Fixes: https://tracker.ceph.com/issues/66492
Signed-off-by: Mark Kogan <mkogan@ibm.com>
(cherry picked from commit 4594e38bf19a8f73191904f5f373a3707cddd2f3)

Merge pull request #59756 from zdover23/wip-doc-2024-09-12-backport-59753-to-squid

squid: doc/install: Keep the name field of the created user consistent with …

doc/install: Keep the name field of the created user consistent with the node name in the Start RADOSGW service command

If the user name does not match the name of the node that started the RADOSGW service, this will cause confusion for those who are new to ceph. Because they can't start the radosgw service normally as shown in the tutorial.

Signed-off-by: hejindong <wodeshijie3394@126.com>
(cherry picked from commit 128e6451d8c4764025d4a2565f15ec175c857bb2)

mgr/dashboard: service form hosts selection only show up to 10 entries

exposed new UIRouter API to get all hosts

Fixes: https://tracker.ceph.com/issues/67413
Signed-off-by: Naman Munet <nmunet@redhat.com>
(cherry picked from commit ec3e4abeeb7feebd996681a7d8d4e950de5cf518)

mgr/dashboard: expose gw_groups list api

also if the cephadm is available, make the api smart enough to
understand the running daemons and get its gateway_addr instead of
taking the first one in the list so that the HA will be there for the UI
and API as well

Fixes: https://tracker.ceph.com/issues/67774
Signed-off-by: Nizamudeen A <nia@redhat.com>
(cherry picked from commit c8434d67d08882ac88c579c6e199f59f05a31699)

mgr/dashboard: add gw_groups support to nvmeof api

Introduces a gw_group query-param to gateway and subsystem requests.

Modified the dashboard config to support adding more than one gateway to
the list.

Fixes: https://tracker.ceph.com/issues/67774
Signed-off-by: Nizamudeen A <nia@redhat.com>
(cherry picked from commit 47f57d434d1690cc3c29dd048ec4cdedf618571f)

Merge pull request #59717 from afreen23/wip-67993-squid

squid: mgr/dashboard: Increase maximum namespace count to 1024

Reviewed-by: Nizamudeen A <nia@redhat.com>

Merge pull request #59721 from aaSharma14/wip-67996-squid

squid: mgr/dashboard: Rephrase dedicated pool helper in rbd create form

Reviewed-by: Pedro Gonzalez Gomez <pegonzal@redhat.com>

Merge pull request #59733 from zdover23/wip-doc-2024-09-11-backport-59635-to-squid

squid: doc/rados: add "pgs not deep scrubbed in time" info

Merge pull request #59700 from zdover23/wip-doc-2024-09-10-backport-59675-to-squid

squid: doc/README.md: improve formatting

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>

doc/rados: add "pgs not deep scrubbed in time" info

Add a procedure to doc/rados/operations/health-warnings.rst that
explains how to remedy the "X PGs not deep-scrubbed in time" health
warning.

This procedure was developed by Eugen Block, and is at the time of this
commit available on his blog at
https://heiterbiswolkig.blogs.nde.ag/2024/09/06/pgs-not-deep-scrubbed-in-time/

Co-authored-by: Eugen Block <eblock@nde.ag>
Signed-off-by: Zac Dover <zac.dover@proton.me>
(cherry picked from commit d620a51c306d1ebcdfc974dc0b85dc46cfde8d40)

Merge pull request #59693 from rhcs-dashboard/squid-wrong-uid

Squid: mgr/dashboard: Wrong(half) uid is observed in dashboard when user created via cli contains $ in its name

Reviewed-by: Afreen Misbah <afreen23.git@gmail.com>

mgr/dashboard: use grafana server instead of grafana-server in grafana
10.4.0

The grafana-server command is deprecated in grafana v10.4.0. It is
advised to use grafan server in place of it

Fixes: https://tracker.ceph.com/issues/65479
Signed-off-by: Aashish Sharma <aasharma@redhat.com>
(cherry picked from commit ba7b6ec215701d17295b751835d29b18950be960)

mgr/dashboard: Rephrase dedicated pool helper in rbd create form

Fixes: https://tracker.ceph.com/issues/67893
Signed-off-by: Aashish Sharma <aasharma@redhat.com>
(cherry picked from commit a94a0ad634d3fdf637b74666a8a8758afc50b9e0)

Conflicts:
src/pybind/mgr/dashboard/frontend/src/app/ceph/block/rbd-form/rbd-form.component.html (conflicts with carbon changes)

mgr/dashboard: Update nvmeof microcopies

- As per the review from the stakeholders , updating the microcopies in nvmeof.
- in task service file changed casing of `Service` to align with rest.

Fixes https://tracker.ceph.com/issues/67728

Signed-off-by: Afreen Misbah <afreen23.git@gmail.com>
(cherry picked from commit 5d1ee140d0514348faa1db323a18c69fcb8950e6)

mgr/dashboard: Increase maximum namespace count to 1024

Fixes https://tracker.ceph.com/issues/67871

The maximum namespace count will be 1024 for subsystems now.

Signed-off-by: Afreen Misbah <afreen23.git@gmail.com>
(cherry picked from commit 12c7c6522bac5a0e669ceabaa044e6a7cd479f88)

Merge pull request #59682 from zdover23/wip-doc-2024-09-10-backport-59677-to-squid

squid: doc/man: edit ceph-bluestore-tool.rst

doc/README.md: improve formatting

Improve the formatting of the section "Building Ceph" in the file
README.md.

Signed-off-by: Zac Dover <zac.dover@proton.me>
(cherry picked from commit 7171f73c81d603f8ff62198fc1e5f50c20a0aa05)

mgr/dashboard: Wrong(half) uid is observed in dashboard

Fixes: https://tracker.ceph.com/issues/67850
Signed-off-by: Dnyaneshwari Talwekar <dtalweka@redhat.com>

doc/man: edit ceph-bluestore-tool.rst

Edit the section "bluefs-bdev-migrate" in
doc/man/8/ceph-bluestore-tool.rst to add the information that this
operation expands the target storage by updating its size label, making
"bluefs-bdev-expand" unnecessary.

Improve the subject-verb agreement in this section, and supply some
absent definite articles.

Co-authored-by: Peter Gervai <grin@drop.grin.hu>
Signed-off-by: Zac Dover <zac.dover@proton.me>
(cherry picked from commit 6b34707f827b2b197f53fe2e430d173b30b81401)

Merge pull request #58383 from adk3798/squid-fix-secure-monitoring-stack-check

squid: mgr/cephadm: redeploy when some dependency daemon is add/removed

Reviewed-by: Redouane Kachach <rkachach@ibm.com>

Merge pull request #59138 from k0ste/wip-67110-squid

squid: mgr/rgw: Adding a retry config while calling zone_create()

Reviewed-by: Adam King <adking@redhat.com>

Merge pull request #59473 from adk3798/wip-67695-squid

squid: mgr/orchestrator: fix encrypted flag handling in orch daemon add osd

Reviewed-by: Guillaume Abrioux <gabrioux@ibm.com>

Merge pull request #59663 from zdover23/wip-doc-2024-09-09-backport-59642-to-squid

squid: doc/cephadm: add malformed-JSON removal instructions

mds/MDSAuthCaps: allow updating root_squash in entity's MDS cap

Command "ceph fs authorize" can update fsname, path and perm/spec in a
MDS cap but it can't update "root_squash" in it. Add code to allow this.

Specifically, "MDSAuthCaps::merge_one_cap_grant()" doesn't check if
the value for "root_squash" needs an update while attempting to merge
the new MDS cap into the list of MDS caps already held by an entity. It
should do so to ensure correct behaviour.

Fixes: https://tracker.ceph.com/issues/64417
Signed-off-by: Rishabh Dave <ridave@redhat.com>
(cherry picked from commit f6bb5797c1ed3534355ffc6a12dc5f3b08d99f58)

mds/MDSAuthCaps: allow adding multiple caps via "fs auth" cmd

When multiple caps are passed to "ceph fs authorize" command, the
command hangs because the MON crashes due to this command. Underlying
code assumes that only one cap can be passed at a time by the user
through the "ceph fs authorize" command. "ceph_assert()" checks if only
one cap is passed. If not, the code crashes.

Fixes: https://tracker.ceph.com/issues/64127
Signed-off-by: Rishabh Dave <ridave@redhat.com>
(cherry picked from commit 56e0f73afe5fdebf38af339efea0dfde65a4de87)

qa/cephfs: minor adjustment in test_admin.TestFsAuthorize

Move related methods ("_remount_and_run_tests()" and "_remount()")
closer to each other and move "tearDown()" to the bottom.

Signed-off-by: Rishabh Dave <ridave@redhat.com>
(cherry picked from commit f33cc3303454a5688a2a568caa4067ba4a8dc847)

mon/AuthMonitor: update MDS caps even if other caps need no update

When entity's MDS caps are to be updated but MON and OSD caps do not
require an update, in certains cases, the update doesn't happen.

src/mon/AuthMonitor.cc receives boolean values for MON, OSD, and MDS cap
indicating whether or not each of them requires an update. However,
instead of updating entity's keyring when anyone of these caps require
an update, AuthMontitor updates only when the last one of these needs an
update. Thus, if MDS cap needs an update but it is not last one to be
considered (among MON, OSD and MDS caps), update doesn't happen.

Fixes: https://tracker.ceph.com/issues/64182
Signed-off-by: Rishabh Dave <ridave@redhat.com>
(cherry picked from commit 1e90cbfb39868f3df7f564e540f15f08147cb85b)

qa/cephfs: handle case where mntpt is '/' in caps_helper.py

Signed-off-by: Rishabh Dave <ridave@redhat.com>
(cherry picked from commit 7626bd5d37b6f24fe60840c225ef917356f8dfc4)

Merge pull request #59596 from aaSharma14/wip-67792-squid

squid: mgr/dashboard: Zone details showing incorrect data for data pool values and compression info for Storage Classes

Reviewed-by: Nizamudeen A <nia@redhat.com>

Merge pull request #59597 from aaSharma14/wip-66559-squid

squid: mgr/dashboard: Show which daemons failed in CEPHADM_FAILED_DAEMON healthcheck

Reviewed-by: Nizamudeen A <nia@redhat.com>

Merge pull request #58912 from lxbsz/wip-67122

squid: mds/QuiesceDbManager: get requested state of members before iterating…

Reviewed-by: Venky Shankar <vshankar@redhat.com>

Merge pull request #58984 from joscollin/wip-65223-squid

squid: cephfs_mirror: use snapdiff api for incremental syncing

Reviewed-by: Venky Shankar <vshankar@redhat.com>

Merge pull request #59068 from joscollin/wip-67382-squid

squid: qa/suites: drop --show-reachable=yes from fs:valgrind tests

Reviewed-by: Venky Shankar <vshankar@redhat.com>

Merge pull request #59070 from joscollin/wip-67387-squid

squid: cephfs_mirror: provide metrics for last successful snapshot sync

Reviewed-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>

doc/cephadm: add malformed-JSON removal instructions

Add instructions for removing malformed JSON that causes the cephadm
module to fail to start, which in turn makes the orchestrator fail to
start.

Signed-off-by: Zac Dover <zac.dover@proton.me>
(cherry picked from commit 93d90ea358cf17b08e502caf8529061a10008fad)

Merge pull request #59654 from zdover23/wip-doc-2024-09-08-backport-59628-to-squid

squid: doc/dev: add "activate latest release" RTD step

doc/dev: add "activate latest release" RTD step

Add a step to the section called "First Stable Release" that explains
how to activate the latest release in Read the Docs.

Signed-off-by: Zac Dover <zac.dover@proton.me>
(cherry picked from commit e7bd4bcdd074a3defae5866a6c035559bf210271)

Merge pull request #59637 from zdover23/wip-doc-2024-09-06-backport-59630-to-squid

squid: doc/README.md: edit "Build Prerequisites"

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>

Merge pull request #59623 from guits/wip-67924-squid

squid: ceph-volume: switch over to new disk sorting behavior

doc/README.md: edit "Build Prerequisites"

Make the sentences in the "Build Prerequisites" section of README.md
more direct.

Signed-off-by: Zac Dover <zac.dover@proton.me>
(cherry picked from commit 0b277fe3809c2eae7bfced002ac85f7e1629fa30)

Merge pull request #59598 from guits/wip-67892-squid

squid: ceph-volume: fix generic activation with raw osds

Merge pull request #59622 from guits/wip-67923-squid

squid: ceph-volume: pass self.osd_id to create_id() call

Merge pull request #59202 from k0ste/wip-67529-squid

squid: debian pkg: record python3-packaging dependency for ceph-volume

Merge pull request #59602 from idryomov/wip-67752-squid

squid: rbd: amend "rbd {group,} rename" and "rbd mirror pool" command descriptions

Reviewed-by: Ramana Raja <rraja@redhat.com>

ceph-volume: switch over to new disk sorting behavior by default

The automatic disk sorting behavior is deprecated.
The new behavior should be the default since Ceph Pacific but it
got pushed back multiple times.

Fixes: https://tracker.ceph.com/issues/67497
Signed-off-by: Guillaume Abrioux <gabrioux@ibm.com>
(cherry picked from commit be725040bdff367c29262becdea269416b326b8f)

ceph-volume: pass self.osd_id to create_id() call

This fixes a regression
(most likely introduced with 04c93a1ed42)

Current (broken) behavior:
```
[root@devel1-1 /]# ceph osd tree
ID  CLASS  WEIGHT  TYPE NAME     STATUS     REWEIGHT  PRI-AFF
-1              0  root default
 0              0  osd.0         destroyed         0  1.00000
[root@devel1-1 /]# ceph-volume raw prepare --bluestore --data /dev/vdb --osd-id 0
Running command: /usr/bin/ceph-authtool --gen-print-key
Running command: /usr/bin/ceph-authtool --gen-print-key
Running command: /usr/bin/ceph --cluster ceph --name client.bootstrap-osd --keyring /var/lib/ceph/bootstrap-osd/ceph.keyring -i - osd new 543e0268-ea66-451f-a2f6-95ae60ab6146
Running command: /usr/bin/mount -t tmpfs tmpfs /var/lib/ceph/osd/ceph-1
Running command: /usr/sbin/restorecon /var/lib/ceph/osd/ceph-1
Running command: /usr/bin/chown -R ceph:ceph /dev/vdb
Running command: /usr/bin/ln -s /dev/vdb /var/lib/ceph/osd/ceph-1/block
Running command: /usr/bin/ceph --cluster ceph --name client.bootstrap-osd --keyring /var/lib/ceph/bootstrap-osd/ceph.keyring mon getmap -o /var/lib/ceph/osd/ceph-1/activate.monmap
 stderr: got monmap epoch 1
--> Creating keyring file for osd.1
Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-1/keyring
Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-1/
Running command: /usr/bin/ceph-osd --cluster ceph --osd-objectstore bluestore --mkfs -i 1 --monmap /var/lib/ceph/osd/ceph-1/activate.monmap --keyfile - --osd-data /var/lib/ceph/osd/ceph-1/ --osd-uuid 543e0268-ea66-451f-a2f6-95ae60ab6146 --setuser ceph --setgroup ceph
 stderr: 2024-09-04T11:46:54.283+0000 7f0027d2b640 -1 bluestore(/var/lib/ceph/osd/ceph-1//block) _read_bdev_label unable to decode label /var/lib/ceph/osd/ceph-1//block at offset 102: void bluestore_bdev_label_t::decode(ceph::buffer::v15_2_0::list::const_iterator&) decode past end of struct encoding: Malformed input [buffer:3]
 stderr: 2024-09-04T11:46:54.283+0000 7f0027d2b640 -1 bluestore(/var/lib/ceph/osd/ceph-1//block) _read_bdev_label unable to decode label /var/lib/ceph/osd/ceph-1//block at offset 102: void bluestore_bdev_label_t::decode(ceph::buffer::v15_2_0::list::const_iterator&) decode past end of struct encoding: Malformed input [buffer:3]
 stderr: 2024-09-04T11:46:54.283+0000 7f0027d2b640 -1 bluestore(/var/lib/ceph/osd/ceph-1//block) _read_bdev_label unable to decode label /var/lib/ceph/osd/ceph-1//block at offset 102: void bluestore_bdev_label_t::decode(ceph::buffer::v15_2_0::list::const_iterator&) decode past end of struct encoding: Malformed input [buffer:3]
 stderr: 2024-09-04T11:46:54.283+0000 7f0027d2b640 -1 bluestore(/var/lib/ceph/osd/ceph-1/) _read_fsid unparsable uuid
--> ceph-volume raw dmcrypt prepare successful for: /dev/vdb
[root@devel1-1 /]# ceph osd tree
ID  CLASS  WEIGHT  TYPE NAME     STATUS     REWEIGHT  PRI-AFF
-1              0  root default
 0              0  osd.0         destroyed         0  1.00000
 1              0  osd.1              down   1.00000  1.00000
```

A new osd id has been generated while we want to reuse the existing one (0 in this case).

Expected (fixed) behavior:

```
[root@devel1-1 /]# ceph osd tree
ID  CLASS  WEIGHT  TYPE NAME     STATUS     REWEIGHT  PRI-AFF
-1              0  root default
 0              0  osd.0         destroyed         0  1.00000
[root@devel1-1 /]# ceph-volume raw prepare --bluestore --data /dev/vdb --osd-id 0
Running command: /usr/bin/ceph-authtool --gen-print-key
Running command: /usr/bin/ceph-authtool --gen-print-key
Running command: /usr/bin/ceph --cluster ceph --name client.bootstrap-osd --keyring /var/lib/ceph/bootstrap-osd/ceph.keyring osd tree -f json
Running command: /usr/bin/ceph --cluster ceph --name client.bootstrap-osd --keyring /var/lib/ceph/bootstrap-osd/ceph.keyring -i - osd new c87841fb-f56f-40a2-a653-62bb092e8e95 0
Running command: /usr/bin/mount -t tmpfs tmpfs /var/lib/ceph/osd/ceph-0
Running command: /usr/sbin/restorecon /var/lib/ceph/osd/ceph-0
Running command: /usr/bin/chown -R ceph:ceph /dev/vdb
Running command: /usr/bin/ln -s /dev/vdb /var/lib/ceph/osd/ceph-0/block
Running command: /usr/bin/ceph --cluster ceph --name client.bootstrap-osd --keyring /var/lib/ceph/bootstrap-osd/ceph.keyring mon getmap -o /var/lib/ceph/osd/ceph-0/activate.monmap
 stderr: got monmap epoch 1
--> Creating keyring file for osd.0
Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-0/keyring
Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-0/
Running command: /usr/bin/ceph-osd --cluster ceph --osd-objectstore bluestore --mkfs -i 0 --monmap /var/lib/ceph/osd/ceph-0/activate.monmap --keyfile - --osd-data /var/lib/ceph/osd/ceph-0/ --osd-uuid c87841fb-f56f-40a2-a653-62bb092e8e95 --setuser ceph --setgroup ceph
 stderr: 2024-09-04T13:07:39.125+0000 7ff02baaa640 -1 bluestore(/var/lib/ceph/osd/ceph-0//block) _read_bdev_label unable to decode label /var/lib/ceph/osd/ceph-0//block at offset 102: void bluestore_bdev_label_t::decode(ceph::buffer::v15_2_0::list::const_iterator&) decode past end of struct encoding: Malformed input [buffer:3]
 stderr: 2024-09-04T13:07:39.125+0000 7ff02baaa640 -1 bluestore(/var/lib/ceph/osd/ceph-0//block) _read_bdev_label unable to decode label /var/lib/ceph/osd/ceph-0//block at offset 102: void bluestore_bdev_label_t::decode(ceph::buffer::v15_2_0::list::const_iterator&) decode past end of struct encoding: Malformed input [buffer:3]
 stderr: 2024-09-04T13:07:39.126+0000 7ff02baaa640 -1 bluestore(/var/lib/ceph/osd/ceph-0//block) _read_bdev_label unable to decode label /var/lib/ceph/osd/ceph-0//block at offset 102: void bluestore_bdev_label_t::decode(ceph::buffer::v15_2_0::list::const_iterator&) decode past end of struct encoding: Malformed input [buffer:3]
 stderr: 2024-09-04T13:07:39.126+0000 7ff02baaa640 -1 bluestore(/var/lib/ceph/osd/ceph-0/) _read_fsid unparsable uuid
--> ceph-volume raw dmcrypt prepare successful for: /dev/vdb
[root@devel1-1 /]# ceph osd tree
ID  CLASS  WEIGHT  TYPE NAME     STATUS  REWEIGHT  PRI-AFF
-1              0  root default
 0              0  osd.0           down         0  1.00000
```

Fixes: https://tracker.ceph.com/issues/67898
Signed-off-by: Guillaume Abrioux <gabrioux@ibm.com>
(cherry picked from commit 5fc36f6463c247ca108b8f123135dd15aae5f13b)

Merge pull request #59492 from ljflores/wip-bz-2305110-squid

squid: osd: fix require_min_compat_client handling for msr rules

qa/tasks/ceph_manager: set-require-min-compat-client to squid for msr profiles

Signed-off-by: Samuel Just <sjust@redhat.com>
(cherry picked from commit 4f9289e11ae4118f7d26bb154d04288c8fd87a4b)

rbd: mention namespace in "rbd mirror pool" command descriptions

Commit 5e64748927d0 ("doc/rbd: add namespace information for mirror
commands") did this for the man page, update the built-in help as well.

The "by default" bit in the description of "rbd mirror pool enable" and
"rbd mirror pool disable" commands is specific to pool mode which is in
turn specific to journal-based mirroring, so it's removed.

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
(cherry picked from commit 7e02946a3f60da0dcc99e0b4a4b6b3d359db1c15)

rbd: clarify "rbd {group,} rename" command description

Make it clearer that, despite a full image or group spec being taken
for source and destination, an image or a group can be renamed only
within its pool or namespace.

Rename across pools or namespaces within the same pool is unsupported.

Fixes: https://tracker.ceph.com/issues/67752
Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
(cherry picked from commit 5f064a52707af58fd6007734a768899aeb5a32fc)

Merge pull request #59582 from zdover23/wip-doc-2024-09-04-backport-59546-to-squid

squid: doc/rados/operations: Improve health-checks.rst

Merge pull request #59585 from zdover23/wip-doc-2024-09-04-backport-59577-to-squid

squid: doc/mds: improve wording

ceph-volume: fix generic activation with raw osds

Typical failure:

```
Running command: /usr/bin/ceph-authtool --gen-print-key
Running command: /usr/bin/ceph-authtool --gen-print-key
--> Failed to activate via raw: activate() takes 1 positional argument but 5 were given
Running command: /usr/bin/ceph-authtool --gen-print-key
Running command: /usr/bin/ceph-authtool --gen-print-key
--> Failed to activate via LVM: could not find osd.0 with osd_fsid 1e764f4a-db4b-4b41-86eb-468efe4c3f44
--> Failed to activate via simple: 'Namespace' object has no attribute 'json_config'
--> Failed to activate any OSD(s)
```

04c93a1ed42 seems to have broken it.
This commit fixes it.

Fixes: https://tracker.ceph.com/issues/67873
Signed-off-by: Guillaume Abrioux <gabrioux@ibm.com>
(cherry picked from commit 710747031b78948d4a373036f5b8c41269b5a399)

mgr/dashboard: Show which daemons failed in CEPHADM_FAILED_DAEMON
healthcheck

Fixes: https://tracker.ceph.com/issues/63792
Signed-off-by: Aashish Sharma <aasharma@redhat.com>
(cherry picked from commit 5a44b395d4fd86f51fe7431a8f8f539ee57eeee7)

mgr/dashboard: Zone details showing incorrect data for data pool values and compression info for Storage Classes

Fixes: https://tracker.ceph.com/issues/67103
Signed-off-by: Aashish Sharma <aasharma@redhat.com>
(cherry picked from commit 76af1c0b1be562d38f3d6323351b7b7d0462af48)

ceph-volume: add "packaging" to install_requires

in 0985e201, "packaging" was introduced as a runtime dependency of
ceph-volume, and `ceph.spec.in` was updated accordingly to note
this new dependency. but the debian packaging was not updated.

in 80edcd40, the missing dependency was added to debian/control as
one of ceph-volume's runtime dependency.

but dh_python3 is able to figure out the dependencies by reading
the egg's metadata of the ceph-volume python module. and as a
python project, ceph-volume is using its `setup.py` for
tracking its dependencies.

so in order to be more consistent, and keep all of its dependencies
in one place, let's move this dependency to setup.py . as the
packagings in both distros are able to figure the dependencies
from egg-info.

see also
- https://manpages.debian.org/testing/dh-python/dh_python3.1.en.html#dependencies
- https://docs.fedoraproject.org/en-US/packaging-guidelines/Python_201x/#_automatically_generated_dependencies

Signed-off-by: Kefu Chai <tchaikov@gmail.com>
(cherry picked from commit 729fd8e25ff2bfbcf99790d6cd08489d1c4e2ede)

debian pkg: record python3-packaging dependency for ceph-volume

Since commit 0985e201342 ("ceph-volume: use 'no workqueue' options
with dmcrypt") the python "packaging" module is used to parse the
cryptsetup version output, but the debian packaging was not updated to
record that new dependency.

So simply record this in the d/control file, adding a <pkg>.requires
file seemed to not really winning us anything here.

Fixes: https://tracker.ceph.com/issues/67290
Fixes: 0985e201342fa53c014a811156aed661b4b8f994
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
(cherry picked from commit 80edcd40e8092d9fb3b45c1a6c7f9b7f4f37b69e)

Merge pull request #59446 from afreen23/wip-67724-squid

squid: mgr/dashboard: Add group field in nvmeof service form

Reviewed-by: Aashish Sharma <aasharma@redhat.com>
Reviewed-by: Ankush Behl <cloudbehl@gmail.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>

doc/mds: improve wording

Signed-off-by: Piotr Parczewski <piotr@stackhpc.com>
(cherry picked from commit 332804bad58c892d01d2d2da557e42104365ef8a)

doc/rados/operations: Improve health-checks.rst

Signed-off-by: Anthony D'Atri <anthonyeleven@users.noreply.github.com>
(cherry picked from commit 2aa82539ede8ade095ad0b22e973eac538354711)

Merge pull request #59578 from zdover23/wip-doc-2024-09-03-backport-59545-to-squid

squid: doc/radosgw: Improve config-ref.rst

doc/radosgw: Improve config-ref.rst

Signed-off-by: Anthony D'Atri <anthonyeleven@users.noreply.github.com>
(cherry picked from commit 8e743fc603310a802b8553af5e73266a9caf6554)

mgr/dashboard: Fixed bug in service form

The commit in main went through this PR https://github.com/ceph/ceph/pull/59496/commits/6c25635be8c2ec0c2f637c4fe6952f781c1ad12c

This PR wont be backported to squid hence adding the change in separate commit.

Signed-off-by: Afreen Misbah <afreen23.git@gmail.com>

Merge pull request #59559 from zdover23/wip-doc-2024-09-02-backport-59556-to-squid

squid: doc: Correct link to Prometheus docs

doc: Correct link to Prometheus docs

The link is to the `#http_sd_config` anchor in the prometheus config docs; that link only works without the trailing `/`.

This correction would ideally get backported to at least reef & squid.

Signed-off-by: Matthew Vernon <mvernon@wikimedia.org>
(cherry picked from commit 84a30ba6b94b34806faac8217ccaa299c9ee68d6)

Merge pull request #59548 from zdover23/wip-doc-2024-09-01-backport-59544-to-squid

squid: doc: update tests-integration-testing-teuthology-workflow.rst

Reviewed-by: Vallari Agrawal <val.agl002@gmail.com>

doc: update tests-integration-testing-teuthology-workflow.rst

* add "Infrastructure" section.

* move "Naming the ceph-ci branch" section under
   "Getting binaries - Build Ceph". Also mention
   about centos9-only trick.

* in "Teuthology Archives", mention about developer
   playground machines and ceph log files.

Signed-off-by: Vallari Agrawal <val.agl002@gmail.com>
(cherry picked from commit 9bfcb8e17db8c61e523e10856d12b237433d831a)

Merge pull request #59464 from sajibreadd/wip-67037-squid

squid: os/bluestore: Warning added for slow operations and stalled read