qa/rgw: fix ignorelist entry for 'HEALTH_WARN 1 pool(s) full'

Fixes: https://tracker.ceph.com/issues/70712
Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit a5fd9bca04f92da73f7c0f811dc67191fd41d06c)

qa/rgw: suppress "pool(s) full" cluster warning for rados-pool-quota

Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit c29f531a972606e09c01596dadb1501f77d39f46)

qa/rgw: test_rgw_rados_pool_quota expects InsufficientCapacity

Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit 9ff036e7ed426458442d46f8c15c3a26a20cb73d)

rgw/s3: map ENOSPC to 507 InsufficientCapacity

Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit 7a06e84146871db1f885a7b5993aa1e06c3693a0)

Conflicts:
src/rgw/rgw_common.cc missing account-related error mappings

rgw: rgw_init_ioctx() adds set_pool_full_try()

add librados::OPERATION_FULL_TRY globally so we never hang waiting for
space when the pool reaches its quota limit

Fixes: https://tracker.ceph.com/issues/69842
Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit 9d351a76ab1cb917b2af7c497814deeee02ead07)

rgw: RGWRadosRemoveCR uses rgw_init_ioctx()

avoid direct calls to ioctx_create() so rgw_init_ioctx() can set
appropriate flags globally

Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit 38929f4e17a624745103bf6a977bed007308a551)

qa/.../test_rgw_rados_pool_quota: ignore cluster warnings

> cluster [WRN] pool 'default.rgw.buckets.data' is full (reached quota's max_objects: 1)" in cluster log
> cluster [WRN] Health check failed: 1 pool(s) full (POOL_FULL)" in cluster log

Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit b7ee4a3d50b52d1ad25621b42586df476827da15)

qa/.../test_rgw_rados_pool_quota: add test for deleting objects with pool at quota

Signed-off-by: Samuel Just <sjust@redhat.com>
(cherry picked from commit 66b782fcda4b715b3505b22c3bf1165c0cec3a9f)

rgw/rados: use set_pool_full_try() when deleting tail objects

Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit 2efa41d860b70ff58a8eefbd6c47b4eb833a7f70)

Conflicts:
src/rgw/driver/rados/rgw_rados.cc missing 0b7b2333984e2ac8a32304336345696f3b6091b5, f39c5f50544822da8f2f79dcc4586b14583392e2

rgw/rados: use set_pool_full_try() when deleting head objects

Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit 2b700c6d00209cf582cf3f6fc428342a95bd1952)

Conflicts:
src/rgw/driver/rados/rgw_rados.cc missing 451b70dedb9975a605458b2dae83de61a107c936

Merge pull request #61595 from yuvalif/wip-63630-reef

reef: rgw/test/kafka: let consumer read events from the beginning

Reviewed-by: J. Eric Ivancich <ivancich@redhat.com>

Merge pull request #61973 from rhcs-dashboard/wip-70122-reef

reef: mgr/dashboard: disable deleting bucket with objects

Reviewed-by: Afreen Misbah <afreen@ibm.com>

Merge pull request #62078 from zdover23/wip-doc-2025-03-03-backport-62076-to-reef

reef: doc/rados/operations: Clarify stretch mode vs device class

doc/rados/operations: Clarify stretch mode vs device class

Signed-off-by: Anthony D'Atri <anthonyeleven@users.noreply.github.com>
(cherry picked from commit 75be0272e8469ed214302b8f354bed675cdcaed6)

Merge pull request #61403 from ronen-fr/wip-rf-61289-reef

reef: common: fix md_config_cacher_t

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

Merge pull request #62057 from zdover23/wip-doc-2025-02-28-backport-61626-to-reef

reef: doc/rados: improve pg_num/pgp_num info

doc/rados: improve pg_num/pgp_num info

Improve the guidance around setting pg_num, and clear up confusion
around whether pgp_num should be set manually or, indeed, if it even can
be set manually.

This PR was raised in response to Mark Schouten's email here: https://lists.ceph.io/hyperkitty/list/ceph-users@ceph.io/thread/CBDJTLTTIEZVG7GVZBX37UAWGYNSSMPD/

Co-authored-by: Anthony D'Atri <anthony.datri@gmail.com>
Signed-off-by: Zac Dover <zac.dover@proton.me>
(cherry picked from commit c43e7337212fe38e8db63d00345fa9858b3cb10a)

mgr/dashboard: disable deleting bucket with objects

Fixes: https://tracker.ceph.com/issues/70078
Signed-off-by: Naman Munet <naman.munet@ibm.com>
(cherry picked from commit 11677c29ee6ee60d9191edfdbfbe37b5308eb45e)

 Conflicts:
src/pybind/mgr/dashboard/frontend/src/app/ceph/rgw/rgw-bucket-list/rgw-bucket-list.component.ts

Merge pull request #61980 from rhcs-dashboard/wip-70139-reef

reef: mgr/dashboard: critical confirmation modal changes

Reviewed-by: Afreen Misbah <afreen@ibm.com>

Merge PR #57190 into reef

* refs/pull/57190/head:
pybind/mgr/mgr_module: turn off all automatic transactions
pybind/mgr: disable sqlite3/python autocommit
qa/tasks/mgr: add tests for sqlite autocommit
qa/tasks/vstart_runner: run daemons in foreground
qa/tasks/vstart_runner: add missing poll method
qa/suites/rados/mgr: add cli/devicehealth tasks
qa: reorganize mgr unit tests
qa: use position-independent link
qa: add missing terminating newline
pybind/mgr: add killpoint for sqlite3 database setup
mgr: allow specifying module option level
mon/MgrMonitor: promote standby when unsetting down flag
mon/MgrMonitor: only drop active if exists

Reviewed-by: Laura Flores <lflores@redhat.com>

Merge pull request #61831 from idryomov/wip-69911-reef

reef: librbd: fix mirror image status summary in a namespace

Reviewed-by: Ilya Dryomov <idryomov@redhat.com>

Merge pull request #61916 from adk3798/wip-68158-reef

reef: cephadm: Support Docker Live Restore

Reviewed-by: Anthony D'Atri <anthonyeleven@users.noreply.github.com>
Reviewed-by: John Mulligan <jmulligan@redhat.com>

Merge pull request #62005 from zdover23/wip-doc-2025-02-26-backport-62001-to-reef

reef: doc: fix incorrect radosgw-admin subcommand

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>

doc: fix incorrect radosgw-admin subcommand

Signed-off-by: Toshikuni Fukaya <toshikuni-fukaya@cybozu.co.jp>
(cherry picked from commit c3b7d6f19360af90ead61cd5ef8bc73ca1c7fe8f)

pybind/mgr/mgr_module: turn off all automatic transactions

I misunderstood autocommit=False in prior patches. The sqlite3 binding will
still create transactions automatically which confused newer bindings using
autocommit.

So, turn off automatic transaction management completely to maintain backwards
compatibility.

Fixes: https://tracker.ceph.com/issues/69912
Signed-off-by: Patrick Donnelly <pdonnell@ibm.com>
(cherry picked from commit df49652987019d5eeec31c86332d8e69995d931a)

pybind/mgr: disable sqlite3/python autocommit

SQLite3 and python's binding were both committing transactions at unintended
points. Turn it off and stop using executescript.

Fixes: https://tracker.ceph.com/issues/65494
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit 6eacfe96625e272fdb9094ab8753e70ec163c04d)

qa/tasks/mgr: add tests for sqlite autocommit

That autocommit is properly turned off and that commits via context managers
work as expected.

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit fb82b6d35a734cbf3c27e4dbc5e6eb2eceb30759)

qa/tasks/vstart_runner: run daemons in foreground

This mirrors teuthology and makes it possible to check the exit status of a
daemon.

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit e2e2144a56bc71de739e7bc531b253a04a122b00)

qa/tasks/vstart_runner: add missing poll method

Otherwise you cannot use LocalDaemon.check_status.

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit 9748d0c465f88a5118bf3c694fc5e33116229d12)

qa/suites/rados/mgr: add cli/devicehealth tasks

These should have been part of the commit adding the tests.

Fixes: 9ebcbdbed0fbaf37d60fbd8db321742ee5721d8c
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit 440f25e1eca4c3ffbe5c8451cfa0bae0502362da)

qa: reorganize mgr unit tests

Refactor common tasks and allow loading mgrmodules before unittests start.

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit 2f48dc9a0034f28d06640fb6eb375aaf30d6b87d)

qa: use position-independent link

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit 1749edd668060d8453933c5dbfc779f27640043c)

qa: add missing terminating newline

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit 8ac4bbc682b2e77a6402c10eca821b1c7412d631)

pybind/mgr: add killpoint for sqlite3 database setup

To catch unintentional autocommit by sqlite3.

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit fdd0dde62bbc4d84c6d77247315a5cf8eabfa13c)

mgr: allow specifying module option level

Some are for development purposes and should be filtered out by the dashboard.

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit 0d94eebb0dd1b3e2afdde92e1c1051affded46f2)

mon/MgrMonitor: promote standby when unsetting down flag

In the same epoch, to simplify testing.

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit e0441402d6691f68c3ea95ce60b6e3ba101a0a1e)

mon/MgrMonitor: only drop active if exists

This avoids an assertion in ::drop_active.

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit 3fca1187460cdd05c08879b19840d78d787da9b0)

Merge pull request #61953 from zdover23/wip-doc-2025-02-21-backport-61948-to-reef

reef: doc/cephadm/services: Re-improve osd.rst

Merge pull request #61923 from adk3798/wip-69821-reef

reef: mgr/cephadm: create OSD daemon deploy specs through make_daemon_spec

Reviewed-by: John Mulligan <jmulligan@redhat.com>

Merge pull request #59457 from adk3798/wip-65725-reef

reef: python-common: handle "anonymous_access: false" in to_json of Grafana spec

Reviewed-by: John Mulligan <jmulligan@redhat.com>

Merge pull request #61405 from cbodley/wip-69183-reef

reef: Revert "rgw/auth: Fix the return code returned by AuthStrategy,"

Reviewed-by: J. Eric Ivancich <ivancich@redhat.com>

Merge pull request #59125 from k0ste/wip-64796-reef

reef: rgw: compatibility issues on BucketPublicAccessBlock

Reviewed-by: Daniel Gryniewicz <dang@redhat.com>

Merge pull request #61915 from adk3798/wip-68137-reef

reef: mgr/cephadm: add command to expose systemd units of all daemons

Reviewed-by: John Mulligan <jmulligan@redhat.com>

Merge pull request #61924 from adk3798/wip-70088-reef

reef: mgr/cephadm: use double quotes for NFSv4 RecoveryBackend in ganesha conf

Reviewed-by: Avan Thakkar <athakkar@redhat.com>

Merge pull request #61922 from adk3798/wip-69711-reef

reef: cephadm: rgw: allow specifying the ssl_certificate by filepath

Reviewed-by: John Mulligan <jmulligan@redhat.com>

Merge pull request #61921 from adk3798/wip-69515-reef

reef: mgr/cephadm: mgr orchestrator module raise exception if there is trailing tab in yaml file

Reviewed-by: John Mulligan <jmulligan@redhat.com>

Merge pull request #61919 from adk3798/wip-68822-reef

reef: mgr/cephadm: ceph orch host drain command to return error for invalid hostname

Reviewed-by: John Mulligan <jmulligan@redhat.com>

Merge pull request #61918 from adk3798/wip-68783-reef

reef: cephadm: make bootstrap default to "global" section for public_network setting

Reviewed-by: John Mulligan <jmulligan@redhat.com>

Merge pull request #61917 from adk3798/wip-68647-reef

reef: qa/cephadm: wait a bit before checking rgw daemons upgraded w/ `ceph versions`

Reviewed-by: John Mulligan <jmulligan@redhat.com>

Merge pull request #61914 from adk3798/wip-68081-reef

reef: orch: refactor boolean handling in drive group spec

Reviewed-by: John Mulligan <jmulligan@redhat.com>

Merge pull request #61904 from adk3798/reef-cephadm-vrrp-typo

reef: mgr/cephadm: fix typo with vrrp_interfaces in keepalive setup

Reviewed-by: Redouane Kachach <rkachach@ibm.com>

Merge pull request #61717 from adk3798/wip-67111-reef

reef: mgr/rgw: Adding a retry config while calling zone_create()

Reviewed-by: Redouane Kachach <rkachach@ibm.com>

Merge pull request #60474 from guits/wip-68555-reef

reef: cephadm: pull container images from quay.io

Reviewed-by: Adam King <adking@redhat.com>
Reviewed-by: Afreen Misbah <afreen@ibm.com>

Merge pull request #61913 from phlogistonjohn/wip-70090-reef

reef: build-with-container fixes exec bit, dnf cache dir option

Reviewed-by: Adam King <adking@redhat.com>

mgr/dashboard: renaming critical-confirmation-modal to delete-confirmation-modal and keeping default deletion impact as medium

Fixes: https://tracker.ceph.com/issues/69628
Signed-off-by: Naman Munet <naman.munet@ibm.com>
(cherry picked from commit 6866d6a3fe72a62b5239267d2efa42729c9aa4a4)

 Conflicts:
src/pybind/mgr/dashboard/frontend/src/app/ceph/block/mirroring/pool-list/pool-list.component.ts
src/pybind/mgr/dashboard/frontend/src/app/ceph/block/nvmeof-initiators-list/nvmeof-initiators-list.component.ts
src/pybind/mgr/dashboard/frontend/src/app/ceph/block/nvmeof-listeners-list/nvmeof-listeners-list.component.ts
src/pybind/mgr/dashboard/frontend/src/app/ceph/block/nvmeof-namespaces-list/nvmeof-namespaces-list.component.ts
src/pybind/mgr/dashboard/frontend/src/app/ceph/block/nvmeof-subsystems/nvmeof-subsystems.component.ts
src/pybind/mgr/dashboard/frontend/src/app/ceph/block/rbd-list/rbd-list.component.ts
src/pybind/mgr/dashboard/frontend/src/app/ceph/block/rbd-namespace-list/rbd-namespace-list.component.ts
src/pybind/mgr/dashboard/frontend/src/app/ceph/block/rbd-snapshot-list/rbd-snapshot-list.component.ts
src/pybind/mgr/dashboard/frontend/src/app/ceph/cephfs/cephfs-directories/cephfs-directories.component.ts
src/pybind/mgr/dashboard/frontend/src/app/ceph/cephfs/cephfs-list/cephfs-list.component.ts
src/pybind/mgr/dashboard/frontend/src/app/ceph/cephfs/cephfs-subvolume-group/cephfs-subvolume-group.component.ts
src/pybind/mgr/dashboard/frontend/src/app/ceph/cephfs/cephfs-subvolume-list/cephfs-subvolume-list.component.ts
src/pybind/mgr/dashboard/frontend/src/app/ceph/cephfs/cephfs-subvolume-snapshots-list/cephfs-subvolume-snapshots-list.component.ts
src/pybind/mgr/dashboard/frontend/src/app/ceph/cluster/configuration/configuration-form/configuration-form.component.ts
src/pybind/mgr/dashboard/frontend/src/app/ceph/cluster/hosts/hosts.component.ts
src/pybind/mgr/dashboard/frontend/src/app/ceph/cluster/multi-cluster/multi-cluster-list/multi-cluster-list.component.ts
src/pybind/mgr/dashboard/frontend/src/app/ceph/cluster/osd/osd-list/osd-list.component.spec.ts
src/pybind/mgr/dashboard/frontend/src/app/ceph/cluster/osd/osd-list/osd-list.component.ts
src/pybind/mgr/dashboard/frontend/src/app/ceph/cluster/services/services.component.ts
src/pybind/mgr/dashboard/frontend/src/app/ceph/cluster/upgrade/upgrade-progress/upgrade-progress.component.ts
src/pybind/mgr/dashboard/frontend/src/app/ceph/nfs/nfs-list/nfs-list.component.ts
src/pybind/mgr/dashboard/frontend/src/app/ceph/pool/pool-list/pool-list.component.ts
src/pybind/mgr/dashboard/frontend/src/app/ceph/rgw/rgw-multisite-details/rgw-multisite-details.component.ts
src/pybind/mgr/dashboard/frontend/src/app/ceph/rgw/rgw-multisite-sync-policy-details/rgw-multisite-sync-policy-details.component.ts
src/pybind/mgr/dashboard/frontend/src/app/ceph/rgw/rgw-multisite-sync-policy/rgw-multisite-sync-policy.component.ts
src/pybind/mgr/dashboard/frontend/src/app/ceph/rgw/rgw-storage-class-list/rgw-storage-class-list.component.ts
src/pybind/mgr/dashboard/frontend/src/app/ceph/rgw/rgw-user-accounts/rgw-user-accounts.component.ts
src/pybind/mgr/dashboard/frontend/src/app/ceph/smb/smb-cluster-list/smb-cluster-list.component.ts
src/pybind/mgr/dashboard/frontend/src/app/shared/components/delete-confirmation-modal/delete-confirmation-modal.component.html
src/pybind/mgr/dashboard/frontend/src/app/shared/components/delete-confirmation-modal/delete-confirmation-modal.component.spec.ts
src/pybind/mgr/dashboard/frontend/src/app/shared/components/delete-confirmation-modal/delete-confirmation-modal.component.ts
src/pybind/mgr/dashboard/frontend/src/app/shared/datatable/crud-table/crud-table.component.ts

mgr/dashboard: Add confirmation textbox for resource name on delete action

Before:
=====
User was able to delete a single or multiple critical resources like (  images, snapshots, subvolumes, subvolume-groups, pools, hosts , OSDs, buckets, file system, services ) by just clicking on a checkbox.

After:
=====
User now has to type the resource name that they are deleting in the textbox on the delete modal, and then only they will be able to delete the critical resource.
Also from now onwards multiple selection for deletions of critical resources is not possible. Hence, user can delete only single resource at a time. On the other side, non-critical resources can be deleted in one go.

fixes: https://tracker.ceph.com/issues/69628

Signed-off-by: Naman Munet <naman.munet@ibm.com>
(cherry picked from commit ed222df10900198707d7708518e02af5c06c912b)

Merge pull request #61750 from NitzanMordhai/wip-69888-reef

reef: workunit/dencoder: dencoder test forward incompat fix

Merge pull request #59286 from k0ste/wip-65947-reef

reef: msg/async: Encode message once features are set

Merge pull request #61959 from idryomov/wip-70041-reef

reef: test/rbd_mirror: clear Namespace::s_instance at the end of a test

Reviewed-by: Vinay Bhaskar Varada <vvarada@redhat.com>

Merge pull request #61957 from idryomov/wip-63798-reef

reef: test/rbd_mirror: flush watch/notify callbacks in TestImageReplayer

Reviewed-by: Vinay Bhaskar Varada <vvarada@redhat.com>

test/rbd_mirror: clear Namespace::s_instance at the end of a test

TestMockPoolReplayer.Namespaces and NamespacesError tests leave behind
a dangling pointer to a stack-allocated MockNamespace which leads to an
easily reproducible use-after-free and segfault when tests are shuffled.

Fixes: https://tracker.ceph.com/issues/70041
Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
(cherry picked from commit c0f5359b7b47bb424d700f69a2af43ab81fa87c2)

test/rbd_mirror: flush watch/notify callbacks in TestImageReplayer

TestImageReplayer establishes its own (i.e. outside of the SUT code)
watch on the header of the remote image to be able to synchronize the
execution of the test with certain notifications.  This watch is
established before the remote image is opened and is teared down until
after the remote image is closed but while the image replayer is still
running.  The flush that is part of image close sequence thus isn't
guaranteed to cover all callbacks, especially for snapshot-based
mirroring where UnlinkPeerRequest spawned from Replayer::unlink_peer()
generates a notification on the remote image for each completed unlink.
Since TestImageReplayer further immediately deletes C_WatchCtx, pretty
much any test can segfault when C_WatchCtx::handle_notify() is invoked
by TestWatchNotify infrastructure.  Because it's a virtual method, the
segfault often involves a completely bogus instruction pointer:

  fn_anonymous[3176255]: segfault at b ip 000000000000000b sp 00007fffbceba7a8 error 14 in unittest_rbd_mirror[555555554000+96a000]
  Code: Bad RIP value.

Fixes: https://tracker.ceph.com/issues/63798
Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
(cherry picked from commit 5d111e23b89228d75daeada65067765333eefb73)

Merge pull request #61947 from adk3798/reef-fix-valid-addr-py311

reef: python-common: fix valid_addr on python 3.11

Reviewed-by: John Mulligan <jmulligan@redhat.com>

doc/cephadm/services: Re-improve osd.rst

Signed-off-by: Anthony D'Atri <anthonyeleven@users.noreply.github.com>

python-common: fix valid_addr on python 3.11

The behavior on python 3.11 regarding IPv4 addresses in bracket has
changed:
```
$ python3.8 -c 'from urllib.parse import urlparse; urlparse("http://[192.168.0.1]")'

[john@edfu ~]$ python3.11 -c 'from urllib.parse import urlparse; urlparse("http://[192.168.0.1]")'
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/usr/lib64/python3.11/urllib/parse.py", line 395, in urlparse
    splitresult = urlsplit(url, scheme, allow_fragments)
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib64/python3.11/urllib/parse.py", line 500, in urlsplit
    _check_bracketed_host(bracketed_host)
  File "/usr/lib64/python3.11/urllib/parse.py", line 448, in
_check_bracketed_host
    raise ValueError(f"An IPv4 address cannot be in brackets")
ValueError: An IPv4 address cannot be in brackets
```

This breaks the test in test_valid_addr that asserts that function
valid_addr returns the string "IPv4 address wrapped in brackets is
invalid".
Move the step that checks for brackets and dots above the urllib
check so that the function continues returning the expected string.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
(cherry picked from commit ef72af7d720f8811368f25cb6f613ccaf5dcd55d)

Merge pull request #61932 from zdover23/wip-doc-2025-02-20-backport-61785-to-reef

reef: doc: Upgrade and unpin some python versions

Merge pull request #61927 from zdover23/wip-doc-2025-02-20-backport-61883-to-reef

reef: doc/start: Mention RGW in Intro to Ceph

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>

doc: Upgrade and unpin some python versions

https://access.redhat.com/security/cve/CVE-2022-34749

https://bugzilla.redhat.com/show_bug.cgi?id=2255448

Fixes https://github.com/ceph/ceph/pull/44222

Signed-off-by: David Galloway <david.galloway@ibm.com>
(cherry picked from commit 4e2924433612263ae0dee3fc11d9ffa37a1c0346)

doc/start: Mention RGW in Intro to Ceph

Signed-off-by: Anthony D'Atri <anthonyeleven@users.noreply.github.com>
(cherry picked from commit 4a6e9b0de6b899c09fcb40aa73ed3edddfdecba9)

mgr/cephadm: use double quotes for NFSv4 RecoveryBackend in ganesha conf

This came directly from someone on the ganesha team. We've actually had
this use single quotes for a long time (at least since mid 2020) but I
believe recent feature work on the ganesha side exposed the issue

ganesha.nfsd-2[main] config_errs_to_log :CONFIG :WARN :Config File (/etc/ganesha/ganesha.conf:13): Unknown token ('rados_cluster')

Fixes: https://tracker.ceph.com/issues/69930
Signed-off-by: Adam King <adking@redhat.com>
(cherry picked from commit 4f5aa60c5d7605444b76485a473749d06ee65694)

mgr/cephadm: create OSD daemon deploy specs through make_daemon_spec

That function handles setting up the extra container/entrypoint
args for the daemon during initial deployment. Having the
CephadmDaemonDeploySpec made directly in the OSD deployment
workflow means initial deployments of OSDs won't have the
extra container/entrypoint args from the spec

Fixes: https://tracker.ceph.com/issues/69734
Signed-off-by: Adam King <adking@redhat.com>
(cherry picked from commit 8e5db6e09ce251b7d1caf9e19666e52c3480752a)

Allow passing ssl_certificate as an rgw_frontend_extra_arg.

Fixes: https://tracker.ceph.com/issues/69567
Signed-off-by: Alexander Hussein-Kershaw <alexhus@microsoft.com>
fix markup

(cherry picked from commit 154307e817a0646e7171906d34353c6af0e82dee)

Conflicts:
src/pybind/mgr/cephadm/services/cephadmservice.py

mgr/cephadm: mgr orchestrator module raise exception if there is trailing tab in yaml file

Fixes: https://tracker.ceph.com/issues/69192
Signed-off-by: Shweta Bhosale <Shweta.Bhosale1@ibm.com>
(cherry picked from commit dfa632b42558278d26cabb88248aa7ae8ba8fcfc)

mgr/cephadm: orch host drain command does not return error for invalid hostname

Fixes: https://tracker.ceph.com/issues/68681
Signed-off-by: Shweta Bhosale <Shweta.Bhosale1@ibm.com>
(cherry picked from commit ab8d7095979322613e5cfcdd6f96befc299b408e)

cephadm: make bootstrap default to "global" section for public_network setting

Fixes: https://tracker.ceph.com/issues/68212
Signed-off-by: Adam King <adking@redhat.com>
(cherry picked from commit 3ec029703254c6f9c9b778fb6a09202631a719aa)

qa/cephadm: wait a bit before checking rgw daemons upgraded w/ `ceph versions`

As this seems to take a little bit to be updated and the tests end
up failing despite the rgw daemons actually being upgraded successfully

Fixes: https://tracker.ceph.com/issues/67758
Signed-off-by: Adam King <adking@redhat.com>
(cherry picked from commit b9f63e12572c34fdc617dc8c12339051ae829bde)

cephadm: Support Docker Live Restore

Currently with Docker Live Restore [1] enabled and while restarting
Docker Engine - all Ceph container images will get restarted,
while the feature allows restarting docker.service without
containers downtime.

This is due to Requires=docker.service in systemd units templates,
which mandates that on docker.service restart - the ceph container
systemd units will be restarted as well.

Reworking Requires= to Wants= that is a weaker version of the former,
see [2].

Leaving After= entries, because they should allow systemd to correctly
order the startup (first docker, then ceph containers).

[1]: https://docs.docker.com/engine/daemon/live-restore/
[2]: https://www.freedesktop.org/software/systemd/man/latest/systemd.unit.html#Wants=

Fixes: https://tracker.ceph.com/issues/68028
Signed-off-by: Michal Nasiadka <mnasiadka@gmail.com>
(cherry picked from commit e0f77686523337f37e3ddbbe40eaa92c68947195)

Conflicts:
src/cephadm/cephadmlib/templates/ceph.service.j2
src/cephadm/cephadmlib/templates/init_ctr.service.j2
src/cephadm/cephadmlib/templates/sidecar.service.j2
src/cephadm/tests/test_unit_file.py

mgr/cephadm: add command to list systemd units of dameons

Fixes: https://tracker.ceph.com/issues/64299
Signed-off-by: Adam King <adking@redhat.com>
(cherry picked from commit 447f8818becc13bc1d05181564bb5c0f1d19eb98)

mgr/cephadm: add systemd unit to stored info about daemons

We already collect this in `cephadm ls` but don't
store it currently. Storing it will allow us to expose
it to users who want that information.

Signed-off-by: Adam King <adking@redhat.com>
(cherry picked from commit c939c95265cf17e7ab5ad60f95b5bbaabfd32ef1)

orch: refactor boolean handling in drive group spec

The intent of 42721c03ee6f was to address an issue where boolean
parameters weren't handled correctly.

I noticed that a parameter (`tpm2`) was missed, which made me realize
that maintaining a list of these boolean parameters is necessary.

To simplify things, we should only accept `"true"` or `"false"` (in any case),
allowing us to avoid the need to maintain a list of boolean parameters.

This change introduces a `list_drive_group_spec_bool_arg` to store boolean
arguments related to drive group specifications, simplifying the validation
process for boolean values by directly checking if the values are 'true' or 'false'.

Fixes: https://tracker.ceph.com/issues/68045
Signed-off-by: Guillaume Abrioux <gabrioux@ibm.com>
(cherry picked from commit e3d8a37ef6e31eaf69671cec6ee5b1ed11ca267d)

script: allow relative &  ~ paths in build-with-container dnf cache arg

Signed-off-by: John Mulligan <jmulligan@redhat.com>
(cherry picked from commit cf0ed0a2859204fab3cbd871d2e56c1a86fa6a53)

script: set execute bit on build-with-container.py

Signed-off-by: John Mulligan <jmulligan@redhat.com>
(cherry picked from commit f7165966277636b926692aece82977dad3bd1784)

mgr/rgw: Adding a retry config while calling zone_create()

Fixes https://tracker.ceph.com/issues/66750

Signed-off-by: Kritik Sachdeva <sachdevakritik.440@gmail.com>
(cherry picked from commit 480253eaea553b3827a9d584ba8b45a32c845386)

Conflicts:
src/pybind/mgr/rgw/module.py

Merge pull request #61721 from adk3798/wip-68018-reef

reef: cephadm: emit warning if daemon's image is not to be used

Reviewed-by: Redouane Kachach <rkachach@redhat.com>

Merge pull request #61720 from adk3798/wip-67694-reef

reef: mgr/orchestrator: fix encrypted flag handling in orch daemon add osd

Reviewed-by: John Mulligan <jmulligan@redhat.com>

Merge pull request #61719 from adk3798/wip-67498-reef

reef: mgr/nfs: scrape nfs monitoring endpoint

Reviewed-by: John Mulligan <jmulligan@redhat.com>

Merge pull request #61718 from adk3798/wip-67463-reef

reef: qa/upgrade: fix checks to make sure upgrade is still in progress

Reviewed-by: John Mulligan <jmulligan@redhat.com>

Merge pull request #61716 from adk3798/wip-66959-reef

reef: cephadm: turn off cgroups_split setting  when bootstrapping with --no-cgroups-split

Reviewed-by: John Mulligan <jmulligan@redhat.com>

Merge pull request #61715 from adk3798/wip-66891-reef

reef: mgr/rgw: fix setting rgw realm token in secondary site rgw spec

Reviewed-by: Redouane Kachach <rkachach@redhat.com>

Merge pull request #61714 from adk3798/wip-66788-reef

reef: cephadm: disable ms_bind_ipv4 if we will enable ms_bind_ipv6

Reviewed-by: Redouane Kachach <rkachach@redhat.com>

Merge pull request #61683 from phlogistonjohn/wip-69856-reef

reef: containerized build tools [V2]

Reviewed-by: Adam King <adking@redhat.com>

Merge pull request #61713 from adk3798/wip-66647-reef

reef: mgr/rgw: fix error handling in rgw zone create

Reviewed-by: Redouane Kachach <rkachach@ibm.com>

Merge pull request #61712 from adk3798/wip-66529-reef

reef: cephadm: fix apparmor profiles with spaces in the names

Reviewed-by: John Mulligan <jmulligan@redhat.com>

Merge pull request #61711 from adk3798/wip-66476-reef

reef: qa/suites: add "mon down" log variations to ignorelist

Reviewed-by: John Mulligan <jmulligan@redhat.com>

Merge pull request #61151 from stackhpc/reef-keepalived-security-context

reef: cephadm/services/ingress: configure security user in keepalived template

Reviewed-by: Adam King <adking@redhat.com>

Merge pull request #61027 from adk3798/wip-69186-reef

reef: qa/tasks/nvme_loop: update task to work with new nvme list format

Reviewed-by: Laura Flores <lflores@ibm.com>

Merge pull request #60350 from aaSharma14/wip-68547-reef

reef: mgr/cephadm: Update multi-site configs before deploying  daemons on rgw service create

Reviewed-by: Adam King <adking@redhat.com>

Merge pull request #59947 from aaSharma14/wip-68202-reef

reef: mgr/cephadm: use host address while updating rgw zone endpoints

Reviewed-by: Adam King <adking@redhat.com>

Merge pull request #56714 from adk3798/reef-test-cephadm-correct-bootstrap-image

reef: qa/cephadm: use reef image as default for test_cephadm workunit

Reviewed-by: John Mulligan <jmulligan@redhat.com>

Merge pull request #61881 from yuvalif/wip-yuval-bucket-notif-test

reef: fix issue with bucket notification test

Reviewed-by: Yuri Weinstein <yuriw@redhat.com>
Reviewed-by: Adam C. Emerson <aemerson@redhat.com>