rbd-mirror: default <-> non-default namespace remapping

These changes allow the pool to be configured to disable mirroring on
the default namespace if required (init-only mode). It also allows the
default namespace to be mirrored to a non-default namespace on the
remote pool.

Co-authored-by: Ilya Dryomov <idryomov@gmail.com>
Signed-off-by: N Balachandran <nibalach@redhat.com>
Signed-off-by: Ilya Dryomov <idryomov@gmail.com>

Merge pull request #61899 from rhcs-dashboard/disable-deleting-bucket-with-objects

mgr/dashboard: disable deleting bucket with objects

Reviewed-by: Aashish Sharma <aasharma@redhat.com>

Merge pull request #61893 from laimis9133/main

mgr/dashboard: update openapi specification for /api/cluster/user/export

Reviewed-by: Anthony D Atri <anthony.datri@gmail.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>

Merge pull request #60872 from ronen-fr/wip-rf-repaironly

osd/scrub: remove config option osd_repair_during_recovery
Reviewed-by: Samuel Just <sjust@redhat.com>

Merge pull request #61950 from anthonyeleven/improve-rgw-yaml

src/common/options: Improve descriptions in rgw.yaml.in

doc: fix syntax (squash this)

Signed-off-by: Zac Dover <zac.dover@gmail.com>

doc: fixup (squash this)

Remove a colon from a yaml field that will not accept a colon.

Signed-off-by: Zac Dover <zac.dover@proton.me>

Merge pull request #61948 from anthonyeleven/osd-rst-reimprove

doc/cephadm/services: Re-improve osd.rst

Reviewed-by: Mike Perez <thingee@gmail.com>

mgr/dashboard: update openapi specification for /api/cluster/user/export

This commit updates the openapi specification for /api/cluster/user/export endpoint to reflect the current api behaviour.

Fixes: https://tracker.ceph.com/issues/70043
Signed-off-by: Laimis Juzeliunas <laimis.juzeliunas@oxylabs.io>

src/common/options: Improve descriptions in rgw.yaml.in

Signed-off-by: Anthony D'Atri <anthonyeleven@users.noreply.github.com>

doc/cephadm/services: Re-improve osd.rst

Signed-off-by: Anthony D'Atri <anthonyeleven@users.noreply.github.com>

Merge pull request #61870 from afreen23/backward-compatibility

mgr/dashboard: Make API backward compatible with size param change

Reviewed-by: Afreen Misbah <afreen@ibm.com>

Merge pull request #61880 from idryomov/wip-70041

test/rbd_mirror: clear Namespace::s_instance at the end of a test

Reviewed-by: Vinay Bhaskar Varada <vvarada@redhat.com>

Merge pull request #61841 from MaxKellermann/librbd__unordered_map

include/unordered_{map,set}: remove, use `std` directly instead

Reviewed-by: Ronen Friedman <rfriedma@redhat.com>
Reviewed-by: Adam Emerson <aemerson@redhat.com>
Reviewed-by: Ilya Dryomov <idryomov@gmail.com>

Merge pull request #61315 from rhcs-dashboard/smb-join_auth-usersgroups-form-actions

mgr/dashboard: add actions for smb join-auth and usersgroups resources mgmt also updates smb cluster form

Reviewed-by: Afreen Misbah <afreen@ibm.com>

Merge pull request #59615 from xxhdx1985126/wip-63844-67916

crimson/osd/recovery_backend: always set the primary osd's BackfillInterval::version to the last_update before the backfill scan

Reviewed-by: Samuel Just <sjust@redhat.com>

Merge pull request #61225 from rhcs-dashboard/rgw-ratelimit-integration1

mgr/dashboard: Rgw ratelimit feature for user and bucket

Reviewed-by: Aashish Sharma <aasharma@redhat.com>
Reviewed-by: Ankush Behl <cloudbehl@gmail.com>
Reviewed-by: Mark Nelson <mnelson@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
Reviewed-by: Naman Munet <nmunet@redhat.com>

qa/standalone/scrub: fix osd-scrub-repair()

following changes to 'scrub while recovering' logic, the
'what type of scrubs are allowed to run when recovering'
tests were broken, and are now fixed.

Signed-off-by: Ronen Friedman <rfriedma@redhat.com>

qa/standalone/scrub: expand standard_scrub_cluster()

Expand scrub-helpers.sh::standard_scrub_cluster() to cater for
non-standard (i.e. - not "3") pool default size. Also - to
allow clients to specify a "nopool" pool name, preventing the
automatic creation of a pool.

Signed-off-by: Ronen Friedman <rfriedma@redhat.com>

osd/scrub: remove config option osd_repair_during_recovery

The option was used to allow repair to be explicitly ordered by the
operator, even if the Primary OSD was in the middle of a recovery.
This is no longer necessary, as the code now accepts both repair and
scrub operator requests, even during recovery.

Signed-off-by: Ronen Friedman <rfriedma@redhat.com>

mgr/dashboard: add actions to create, edit and delete smb join-auth and usersgroups resources

Add join-auth and usersgroups resources management and improve the way to select those from the smb cluster form using a dropdown.
Add option to navigate to join-auth/usersgroups resource create form from smb form
Add some additional fixes left over from previous work, such as adding helper texts or adding missing smb cluster form fields

Fixes: https://tracker.ceph.com/issues/69483
Signed-off-by: Pedro Gonzalez Gomez <pegonzal@redhat.com>

Merge pull request #61812 from ronen-fr/wip-rf-fstab

qa/standalone: do not use /etc/fstab as an always-there bytes source
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Ilya Dryomov <idryomov@gmail.com>
Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

Merge pull request #61353 from rhcs-dashboard/smb-share-delete

mgr/dashboard: add smb delete share action

Reviewed-by: Afreen Misbah <afreen@ibm.com>

mgr/dashboard: disable deleting bucket with objects

Fixes: https://tracker.ceph.com/issues/70078
Signed-off-by: Naman Munet <naman.munet@ibm.com>

include/unordered_{map,set}: remove, use `std` directly instead

This is only an alias for the `std` classes anyway.  This effectively
reverts commit 8e8672047ccc which added these aliases, probably
because C++11 was new and not all compilers supported it properly, but
that was 12 years ago and we're onl C++20 meanwhile.

Signed-off-by: Max Kellermann <max.kellermann@ionos.com>

Merge pull request #61906 from nbalacha/wip-nbalacha-70075

librbd: fix a crash in get_rollback_snap_id

mgr/dashboard: Make API backward compatible with size param change

- We changed the api with size and now its an issue for other folks utilizing the API
- Hence making it compatible to work backwards as well

Fixes https://tracker.ceph.com/issues/70020

Signed-off-by: Afreen Misbah <afreen@ibm.com>

mgr/dashboard: add smb share delete action

Fixes: https://tracker.ceph.com/issues/69521
Signed-off-by: Pedro Gonzalez Gomez <pegonzal@redhat.com>

Merge pull request #60610 from rzarzynski/wip-no-getval-on-osd-stats

osd: avoid costly md_config_t::get_val<>() when preparing stats

Merge pull request #61408 from adk3798/cephadm-nvmeof-more-group-validation

mgr/cephadm: validate no duplicate groups and group vs. service id for nvmeof

Reviewed-by: John Mulligan <jmulligan@redhat.com>

Merge pull request #61778 from adk3798/cephadm-double-quotes-nfs-recovery-backend

mgr/cephadm: use double quotes for NFSv4 RecoveryBackend in ganesha conf

Reviewed-by: John Mulligan <jmulligan@redhat.com>

Merge pull request #61694 from rkachach/fix_issue_69863

mgr/cephadm: reconfig rgw and ingress daemons when certificate changes

Reviewed-by: Adam King <adking@redhat.com>

Merge pull request #61890 from zhscn/fix-clang-19

crimson: fix compile errors in clang 19

Reviewed-by: Matan Breizman <mbreizma@redhat.com>

Merge pull request #61765 from Kushal-deb/fix_orch_ls_to_display_smb_ports

cephadm: ceph orch ls to Display SMB Service Ports

Reviewed-by: Adam King <adking@redhat.com>
Reviewed-by: John Mulligan <jmulligan@redhat.com>

Merge pull request #61749 from rhcs-dashboard/sqlite-recursive-error-fix

mgr/smb: add threading lock for SqliteStore

Reviewed-by: John Mulligan <jmulligan@redhat.com>

Merge pull request #61833 from Kushal-deb/add_v4v6_flag_to_bind_port

cephadm: ingress- Add v4v6 flag to HAProxy bind directive

Reviewed-by: Adam King <adking@redhat.com>

librbd: fix a crash in get_rollback_snap_id

get_rollback_snap_id() did not check if the snapshot it was
accessing was a mirror snapshot, causing it to crash if it wasn't.

Fixes: https://tracker.ceph.com/issues/70075
Signed-off-by: N Balachandran <nithya.balachandran@ibm.com>

Merge pull request #61722 from adk3798/stop-orch-on-cleanup

qa/cephadm: stop cephadm mgr module during cleanup

Reviewed-by: Redouane Kachach <rkachach@ibm.com>

Merge pull request #61663 from adk3798/vrrp_interfaces_typo

mgr/cephadm: fix typo with vrrp_interfaces in keepalive setup

Reviewed-by: Redouane Kachach <rkachach@ibm.com>

Merge pull request #61658 from ShwetaBhosale1/fix_issue_69808_numric_node_id_for_rados_kv_block

mgr/nfs: nodeid should be numeric for RADOS_KV block in ganesha.conf file

Reviewed-by: Adam King <adking@redhat.com>

Merge pull request #61587 from Kushal-deb/display_comma_separated_ports_for_RGW_services

mgr/cephadm: Display comma separated ports for RGW services

Reviewed-by: Adam King <adking@redhat.com>

Merge pull request #61564 from Sodani/shsodani_profile

mgr/smb: Add profiling configuration to global definition

Reviewed-by: Anoop C S <anoopcs@samba.org>
Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: John Mulligan <jmulligan@redhat.com>

Merge pull request #61372 from adk3798/cephadm-orch-pause-test-failure

qa/cephadm: wait between starting and restarting osd in test_daemon_restart test

Reviewed-by: John Mulligan <jmulligan@redhat.com>

Merge pull request #60606 from Naveenaidu/naveen/telemetry-add-stretch-cluster

mgr/telemetry: add stretch cluster data

qa/standalone: do not use /etc/fstab as an always-there bytes source

Multiple tests use /etc/fstab when a small data file is
required as input. After all, as some comments say:
    # something that is always there

Alas - it's not always there. Not in containers.

Replacing with a newly-created temporary file filled with
random bytes.
For completeness - replacing similar references to
/etc/resolv.conf (as a source for random objects) in
the standalone tests, too.

Signed-off-by: Ronen Friedman <rfriedma@redhat.com>

Merge pull request #61853 from ArbitCode/wip-rgw-raja-remove-clientid-oidc-provider

rgw/iam: add RemoveClientIDFromOpenIDConnectProvider

mgr/dashboard: Rgw ratelimit feature for user and bucket

Fixes: https://tracker.ceph.com/issues/69233
Signed-off-by: Achint Kaur<ackaur@redhat.com>
Signed-off-by: Achintk1491 <achintk1491@gmail.com>

crimson: fix -Wmissing-template-arg-list-after-template-kw

See: https://github.com/llvm/llvm-project/commit/f46d1463b835560d90ad3ac02b63c771e4ebe566
```
common/interruptible_future.h:1182:16: error: a template argument list is expected after a name prefixed by the template keyword [-Wmissing-template-arg-list-after-template-kw]
 1182 |     ).template handle_interruption(std::move(efunc));
      |                ^
1 error generated.
```

Signed-off-by: Matan Breizman <mbreizma@redhat.com>

crimson/common/operation: fix decltype usage

```
ceph/src/crimson/common/operation.cc
In file included from /home/matan/ceph/src/crimson/common/operation.cc:4:
/home/matan/ceph/src/crimson/common/operation.h:312:31: error: no member named 'events' in 'AggregateBlockingEvent<T>'
  312 |       typename decltype(event.events)::iterator iter;
      |                         ~~~~~ ^
1 error generated.
```

Signed-off-by: Matan Breizman <mbreizma@redhat.com>

crimson/tools/perf_crimson_msgr:  fix unused-lamda-capture

Signed-off-by: Matan Breizman <mbreizma@redhat.com>

crimson/osd: move the destructor of OpsExecuter to ops_executer.cc

Signed-off-by: Zhang Song <zhangsong02@qianxin.com>

Merge pull request #60286 from salieri11/igolikov-wip-bug-64611

mds: enforce usage of host error in cephfs, use errorcode32_t in MClientReply message
Fixes: https://tracker.ceph.com/issues/64611

Merge pull request #61690 from rhcs-dashboard/bucket-tiering

mgr/dashboard: bucket lifecycle policy management

Reviewed-by: Aashish Sharma <aasharma@redhat.com>

Merge pull request #60205 from Rethan/bluestore-aio-submit-retries

os/bluestore: make retry_max and initial_delay configurable for aio s…

rgw/iam: add RemoveClientIDFromOpenIDConnectProvider

Signed-off-by: Raja Sharma <raja@ibm.com>
Fixes : https://tracker.ceph.com/issues/70015

Merge pull request #61347 from xxhdx1985126/wip-seastore-tree-pointer-abstract

crimson/os/seastore: refactor fixed-kv-btree

Reviewed-by: Yingxin Cheng <yingxin.cheng@intel.com>

Merge pull request #61837 from anthonyeleven/osd-svc

doc/cephadm/services: Improve osd.rst

Merge pull request #61883 from anthonyeleven/rgw-into-intro

doc/start: Mention RGW in Intro to Ceph

doc/cephadm/services: Improve osd.rst

Signed-off-by: Anthony D'Atri <anthonyeleven@users.noreply.github.com>

Merge pull request #61867 from anuradhagadge/doc_ca_cert_with_fsid

doc/mgr: Add root CA cert instructions to rgw.rst

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>

doc/start: Mention RGW in Intro to Ceph

Signed-off-by: Anthony D'Atri <anthonyeleven@users.noreply.github.com>

Merge pull request #61762 from liu-chunmei/compress_extentmap

os/bluestore: fix do_write_v2 extentmap compress issue

do_write_v2 insert extentmap by aligned offset which is adjusted in writer.do_write, if use original offset will cause extentmap can't do compress.

Reviewed-by: Adam Kupczyk <akupczyk@ibm.com>

test/rbd_mirror: clear Namespace::s_instance at the end of a test

TestMockPoolReplayer.Namespaces and NamespacesError tests leave behind
a dangling pointer to a stack-allocated MockNamespace which leads to an
easily reproducible use-after-free and segfault when tests are shuffled.

Fixes: https://tracker.ceph.com/issues/70041
Signed-off-by: Ilya Dryomov <idryomov@gmail.com>

mgr/dashboard: add bucket tiering option to create lifecycle policy

Fixes: https://tracker.ceph.com/issues/69649
Signed-off-by: Naman Munet <naman.munet@ibm.com>

Merge pull request #61478 from rhcs-dashboard/critical-confirmation-modal-changes

mgr/dashboard: Add confirmation textbox for resource name on delete action

Reviewed-by: Afreen Misbah <afreen@ibm.com>

Merge pull request #61871 from rhcs-dashboard/fix-70019-main

mgr/dashboard: fix Python xmlsec build issue

Reviewed-by: @afreen23
Reviewed-by: @aaSharma14

Merge pull request #61847 from idryomov/wip-63798

test/rbd_mirror: flush watch/notify callbacks in TestImageReplayer

Reviewed-by: Vinay Bhaskar Varada <vvarada@redhat.com>

Merge pull request #61756 from dang/wip-dang-68336

RGW - Fix POSIX bucket notify test

doc/mgr: edit grammar and formatting of rgw.rst

Improve the grammar and correct the formatting of the "Upgrading root ca
certificates" procedure that was added to the documentation in https://github.com/ceph/ceph/pull/61867

Fixes: https://tracker.ceph.com/issues/70014
Signed-off-by: Zac Dover <zac.dover@proton.me>

Merge pull request #61214 from aclamk/wip-aclamk-bs-fragmentation-health

os/bluestore: Add health warning for bluestore fragmentation

Merge pull request #58752 from Matan-B/wip-matanb-replicated-backend-50608

doc/mgr: Add root CA cert instructions to rgw.rst

Add documentation for adding fs_id in root_ca_cert upgrade path

Fixes: https://tracker.ceph.com/issues/70014
Signed-off-by: Anuradha Gadge <anuradha.gadge@ibm.com>

mds: use system error codes instead of CEPHFS_EXX codes

Signed-off-by: Igor Golikov <igolikov@ibm.com>
Fixes: https://tracker.ceph.com/issues/64611

mgr/dashboard: fix Python xmlsec build issue

It disables the specific pinning to a Python xmlsec version.

Fixes: https://tracker.ceph.com/issues/70019
Fixes: 37e633384b7c18893bc0e30a8f3967f6496da00b
Fixes: https://github.com/ceph/ceph/pull/61089
Signed-off-by: Ernesto Puerta <epuertat@redhat.com>

mds,client: change result field handling in the MClientReply message

Signed-off-by: Igor Golikov <igolikov@ibm.com>
Fixes: https://tracker.ceph.com/issues/64611

mds: prepare infra to replace CEPHFS_EXXX codes to system codes

Signed-off-by: Igor Golikov <igolikov@ibm.com>
Fixes: https://tracker.ceph.com/issues/64611

pybind/cephfs: switch CEPHFS_E error codes to system error codes

Signed-off-by: Igor Golikov <igolikov@ibm.com>
Fixes: https://tracker.ceph.com/issues/64611

PendingReleaseNotes: add note on client/mds error codes

Signed-off-by: Igor Golikov <igolikov@ibm.com>
Fixes: https://tracker.ceph.com/issues/64611

mgr/dashboard: remaning critical-confirmation-modal to delete-confirmation-modal and keeping default demetion impact as medium

Fixes: https://tracker.ceph.com/issues/69628
Signed-off-by: Naman Munet <naman.munet@ibm.com>

mgr/dashboard: Add confirmation textbox for resource name on delete action

Before:
=====
User was able to delete a single or multiple critical resources like (  images, snapshots, subvolumes, subvolume-groups, pools, hosts , OSDs, buckets, file system, services ) by just clicking on a checkbox.

After:
=====
User now has to type the resource name that they are deleting in the textbox on the delete modal, and then only they will be able to delete the critical resource.
Also from now onwards multiple selection for deletions of critical resources is not possible. Hence, user can delete only single resource at a time. On the other side, non-critical resources can be deleted in one go.

fixes: https://tracker.ceph.com/issues/69628

Signed-off-by: Naman Munet <naman.munet@ibm.com>

Merge pull request #61836 from myoungwon/fix-crimson-osd-invalid-argument-print

crimson/osd: remove invalid an argument when printing log

Reviewed-by: Matan Breizman <mbreizma@redhat.com>

osd/ReplicatedBackend: assert non-trivial obc existence

Added to help debug: https://tracker.ceph.com/issues/50608

More information: https://gist.github.com/Matan-B/24b49e9b734259f06edf17fdfea5d24d

Signed-off-by: Matan Breizman <mbreizma@redhat.com>

Merge pull request #61469 from ronen-fr/wip-rf-assert-perf

common/perf-counters: replace 'assert' with 'ceph_assert'

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

Merge pull request #61606 from rhcs-dashboard/create-storage-class

mgr/dashboard: RGW - Create Storage Class

Reviewed-by: Afreen Misbah <afreen@ibm.com>

Merge pull request #61793 from MaxKellermann/rados__includes

Add missing librbd includes

Reviewed-by: Ilya Dryomov <idryomov@gmail.com>

Merge pull request #61814 from gbregman/main

mgr/cephadm/nvmeof: Add QOS timeslice field to NVMeOF configuration

Merge pull request #47189 from pdvian/wip-crush-df

crush/CrushWrapper: Merge weights if rule contains multiple take

Reviewed-by: Kamoltat Sirivadhna <ksirivad@redhat.com>
Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

Merge pull request #61752 from afreen23/wip-nvme-ns

Add namespace counter in nvmeof namepsaces

Reviewed-by: Afreen Misbah <afreen@ibm.com>

Merge PR #60983 into main

* refs/pull/60983/head:
script: ceph-backport no longer launches firefox

Reviewed-by: Patrick Donnelly <pdonnell@ibm.com>
Reviewed-by: John Mulligan <jmulligan@redhat.com>

Merge PR #61552 into main

* refs/pull/61552/head:
script/ceph-debug-docker.sh: fix manifest unknown error

Reviewed-by: Patrick Donnelly <pdonnell@ibm.com>

Merge PR #61776 into main

* refs/pull/61776/head:
ceph-backport.sh: add squid to known milestones

Reviewed-by: Patrick Donnelly <pdonnell@ibm.com>
Reviewed-by: John Mulligan <jmulligan@redhat.com>

Merge PR #61813 into main

* refs/pull/61813/head:
common: add missing mds op names

Reviewed-by: Greg Farnum <gfarnum@redhat.com>

RGW - Fix POSIX bucket notify test

The BucketCacheFixtureInotify1.List2Inotify1 test was failing due to the
count being wrong.  Fix the count, and flesh out the test case to test
the cache for deleting and adding.  Also, move to per-test fixture, so
tests can't interfere with eachother due to inotify latency.

Fixes: https://tracker.ceph.com/issues/68336
Signed-off-by: Daniel Gryniewicz <dang@fprintf.net>

Merge pull request #61785 from ceph/cve-34749-fix

doc: Upgrade and unpin some python versions

mgr/dashboard: RGW - Create Storage Class

Fixes: https://tracker.ceph.com/issues/69750
Signed-off-by: Dnyaneshwari Talwekar <dtalweka@redhat.com>

mgr/cephadm/nvmeof: Add QOS timeslice field to NVMeOF configuration
Fixes https://tracker.ceph.com/issues/69952

Signed-off-by: Gil Bregman <gbregman@il.ibm.com>

test/rbd_mirror: flush watch/notify callbacks in TestImageReplayer

TestImageReplayer establishes its own (i.e. outside of the SUT code)
watch on the header of the remote image to be able to synchronize the
execution of the test with certain notifications.  This watch is
established before the remote image is opened and is teared down until
after the remote image is closed but while the image replayer is still
running.  The flush that is part of image close sequence thus isn't
guaranteed to cover all callbacks, especially for snapshot-based
mirroring where UnlinkPeerRequest spawned from Replayer::unlink_peer()
generates a notification on the remote image for each completed unlink.
Since TestImageReplayer further immediately deletes C_WatchCtx, pretty
much any test can segfault when C_WatchCtx::handle_notify() is invoked
by TestWatchNotify infrastructure.  Because it's a virtual method, the
segfault often involves a completely bogus instruction pointer:

  fn_anonymous[3176255]: segfault at b ip 000000000000000b sp 00007fffbceba7a8 error 14 in unittest_rbd_mirror[555555554000+96a000]
  Code: Bad RIP value.

Fixes: https://tracker.ceph.com/issues/63798
Signed-off-by: Ilya Dryomov <idryomov@gmail.com>

Merge pull request #61702 from MaxKellermann/bluestore_includes

os/bluestore: include cleanup

Reviewed-by: Igor Fedotov <ifedotov@suse.com>

Merge pull request #61808 from ronen-fr/wip-rf-store-ids

osd/scrub: cleanup scrub-store IDs generation

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

Merge pull request #61588 from rhcs-dashboard/smb-share-form-create

mgr/dashboard: SMB Create Share

Reviewed-by: Afreen Misbah <afreen@ibm.com>