Merge PR #63636 into main

* refs/pull/63636/head:

Reviewed-by: Christopher Hoffman <choffman@redhat.com>
Reviewed-by: Dhairya Parmar <dparmar@redhat.com>

Merge pull request #65015 from connorfawcett/pg-autoscale-threshold-cmd

mgr/pg_autoscaler: Add 'osd pool get threshold' command which returns the current threshold value

Merge pull request #64788 from rhcs-dashboard/acl-mapping

mgr/dashboard:RGW- Storage Class ACL Mapping

Reviewed-by: Afreen Misbah <afreen@ibm.com>

Merge PR #64958 into main

* refs/pull/64958/head:

Reviewed-by: Patrick Donnelly <pdonnell@ibm.com>
Reviewed-by: Venky Shankar <vshankar@redhat.com>

Merge PR #64967 into main

* refs/pull/64967/head:

Reviewed-by: Anthony D Atri <anthony.datri@gmail.com>
Reviewed-by: Venky Shankar <vshankar@redhat.com>

Merge pull request #65436 from gbregman/main

mgr/cephadm/nvmeof: Add fields for prometheus frequency to NVMEOF spec file

test/libcephfs: validate asynchronous write and fsync executing concurrently

This synthetic reproducer does three things:

- setup a client mount with a configuration to delay write operations and
  initiates a write operation via a thread.
- a thread that invokes asynchronous fsync
- a thread that invokes setxattr for the client to track early replies

Without the fix[0], the test reproduces the following crash:

```
/home/vshankar/ceph/src/client/Client.cc: In function 'void Client::put_request(MetaRequest*)' thread 7f7210ff9640 time 2025-06-03T09:34:45.634974+0000
/home/vshankar/ceph/src/client/Client.cc: 2290: FAILED ceph_assert(request->ref >= 1)
 ceph version 20.3.0-673-gdd152807f7e (dd152807f7e7f7a82df6cfc0159f5fc65f60ecd5) tentacle (dev - Debug)
 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x119) [0x7f72222ebb98]
 2: (ceph::__ceph_assert_fail(ceph::assert_data const&)+0x17) [0x7f72222ebedc]
 3: /home/vshankar/ceph/build/lib/libcephfs.so.2(+0x6a075) [0x7f7222e6a075]
 4: /home/vshankar/ceph/build/lib/libcephfs.so.2(+0xb8289) [0x7f7222eb8289]
 5: /home/vshankar/ceph/build/lib/libcephfs.so.2(+0xee951) [0x7f7222eee951]
 6: /home/vshankar/ceph/build/lib/libcephfs.so.2(+0xf167c) [0x7f7222ef167c]
 7: (Context::complete(int)+0x9) [0x7f7222e5949d]
 8: /home/vshankar/ceph/build/lib/libcephfs.so.2(+0x16a853) [0x7f7222f6a853]
 9: /home/vshankar/ceph/build/lib/libcephfs.so.2(+0xa7cc5) [0x7f7222ea7cc5]
 10: /home/vshankar/ceph/build/lib/libcephfs.so.2(+0xf128d) [0x7f7222ef128d]
 11: /home/vshankar/ceph/build/lib/libcephfs.so.2(+0x16e09d) [0x7f7222f6e09d]
 12: (Context::complete(int)+0x9) [0x7f7222e5949d]
 13: /home/vshankar/ceph/build/lib/libcephfs.so.2(+0x6d108) [0x7f7222e6d108]
 14: (Context::complete(int)+0x9) [0x7f7222e5949d]
 15: (Finisher::finisher_thread_entry()+0x665) [0x7f722226fdc1]
 16: (Finisher::FinisherThread::entry()+0xd) [0x7f7222270ddf]
 17: (Thread::entry_wrapper()+0x2f) [0x7f72222b88f5]
 18: (Thread::_entry_func(void*)+0x9) [0x7f72222b8907]
 19: /lib64/libc.so.6(+0x89e92) [0x7f7221089e92]
 20: /lib64/libc.so.6(+0x10ef20) [0x7f722110ef20]
[1]    2162689 IOT instruction (core dumped)  ./bin/ceph_test_libcephfs --gtest_filter=LibCephFS.ConcurrentWriteAndFsync
```

[0]: https://github.com/ceph/ceph/pull/63619

Fixes: http://tracker.ceph.com/issues/71515
Signed-off-by: Venky Shankar <vshankar@redhat.com>

client: catch buggy reference count drop for MetaRequest

With the prior commit that introduces a synthetic delay in write
operation so as to write a test reproducer which would interleave
asynchronous fsync and an operation that makes the MDS send a early
reply to the client (therefore, having the client track the early
replied response for an inode in Inode::unsafe_ops). Now, this is
enough to trick the client into the code path that causes a buggy
reference drop for the request (MetaRequest), but, hitting the
_exact_ crash backtrace requires the request to be a in various
[x]list's.

This last bit is tricky to synthetically massage in the test. So,
in order to catch the buggy reference drop, it would suffice to
assert on the reference count dropping to less than zero (0).

Signed-off-by: Venky Shankar <vshankar@redhat.com>

client: synthetically delay write operation

To allow the client to hold Fb caps for an extended period of
time, to allow an asynchronous fsync to intervene and block, so
as to hunt [0].

[0]: https://tracker.ceph.com/issues/71510

Signed-off-by: Venky Shankar <vshankar@redhat.com>

client: log unsafe operation count (for debugging)

Signed-off-by: Venky Shankar <vshankar@redhat.com>

libcephfs/client: asynchronous fsync interface

Mostly for writing test for hunting [0].

[0]: https://tracker.ceph.com/issues/71510

Signed-off-by: Venky Shankar <vshankar@redhat.com>

Merge pull request #64999 from rishabh-d-dave/fs-pyx-chown

cephfs.pyx: handle when UID/GID passed to chown() is -1

Reviewed-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #64927 from NitzanMordhai/wip-nitzan-suites-ignore-still-running-cephadm-osds-suites

suites/rados/cephadm: typo in ignotr list for still running message

Merge pull request #65431 from afreen23/doc-release-notes

Update dashboard Pending release notes

Reviewed-by: Anthony D Atri <anthony.datri@gmail.com>
Reviewed-by: Yuri Weinstein <yweinste@redhat.com>

Merge pull request #65419 from ljflores/wip-tracker-72897

doc/rados/operations: add kernel client procedure to read balancer documentation

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>
Reviewed-by: Ilya Dryomov <idryomov@gmail.com>

Merge pull request #64837 from rzarzynski/wip-bug-72412

osd: stop scrub_purged_snaps() from ignoring osd_beacon_report_interval

mgr/cephadm/nvmeof: Add fields for prometheus frequency to NVMEOF spec file.

Fixes: https://tracker.ceph.com/issues/72805
Signed-off-by: Gil Bregman <gbregman@il.ibm.com>

Merge pull request #65384 from rhcs-dashboard/fix-72868-main

mgr/dashboard: fix RGW Bucket Notification Dashboard units

Reviewed-by: Afreen Misbah <afreen@ibm.com>
Reviewed-by: Ankush Behl <cloudbehl@gmail.com>

doc: wq!Update dashboard Pending release notes

- added for tentacle
- moved the inccorect notes added in tentacke to umbrella

Signed-off-by: Afreen Misbah <afreen@ibm.com>

doc/rados/operations: add kernel client procedure to read balancer documentation

As of now, the kernel client does not support `pg-upmap-primary`. I have
added some troubleshooting steps to help users who are unable to
mount images and filesystems with the kernel client while using `pg-upmap-primary`.

Once the feature is supported by the kernel client, users will be able
to perform mounts along with `pg-upmap-primary`.

Fixes: https://tracker.ceph.com/issues/72897
Signed-off-by: Laura Flores <lflores@ibm.com>

Merge pull request #62747 from bill-scales/issue70844

test: add replica pool support to ceph_test_rados_io_sequence

Reviewed-by: Ronen Friedman <rfriedma@redhat.com>
Reviewed-by: Connor Fawcett <connorfa@uk.ibm.com>

Merge pull request #65281 from nbalacha/wip-nbalacha-72740

rgw/logging: fixes data loss during rollover

Merge pull request #65231 from anthonyeleven/improve-osd-dot-cc

src/osd: Improve message in OSD.cc

rgw/logging: fixes data loss during rollover

Multiple threads attempting to roll over the same log object can result
in the creation of numerous orphan tail objects, each with a single record.
This occurs when a NULL RGWObjVersionTracker is used during the creation of
a new logging object. These records are inaccessible, leading to data loss,
which is particularly critical in Journal mode.
Furthermore, valid log tail objects may be added to the Garbage Collection (GC)
list, exacerbating data loss.

Fixes: https://tracker.ceph.com/issues/72740
Signed-off-by: N Balachandran <nithya.balachandran@ibm.com>

Merge pull request #65421 from ronen-fr/wip-rf-ec72898

osd/scrub: clear m_ec_digest_map between objects

Reviewed-by: Jon Bailey <jonathan.bailey1@ibm.com>

Merge pull request #62106 from rkachach/fix_certmgr_v2

Add cephadm-signed certificate support for all services

Reviewed-by: John Mulligan <jmulligan@redhat.com>

test: add replica pool support to ceph_test_rados_io_sequence

Make 'ceph_test_rados_io_sequenece --pool rbd' work, replica
pools don't have an erausre code profile and do not have the
ec_allow_overwrites or ec_allow_optimizations flags

Fixes: https://tracker.ceph.com/issues/70844
Signed-off-by: Bill Scales <bill_scales@uk.ibm.com>
Signed-off-by: Jon Bailey <jonathan.bailey1@ibm.com>

common: Added values to json::OSDPoolGetReply

OSDPoolGetReply actually returns a lot more values than what is currently supplied. These have been added in as optionals (as they can not be give as well) so its possible to query them to find out if they exist and use them if they do.

Signed-off-by: Jon Bailey <jonathan.bailey1@ibm.com>

osd/scrub: clear m_ec_digest_map between objects

Fixing a bug introduced by commit 4c61079e931
("caluculate EC digest map size only once").

Fixes: https://tracker.ceph.com/issues/72897
Signed-off-by: Ronen Friedman <rfriedma@redhat.com>

Merge pull request #65150 from rkachach/fix_issue_nvmeof_prometheus

mgr/cepahdm: fixing nvmeof scraping Prometheus config generation

Reviewed-by: Kushal Deb <Kushal.Deb@ibm.com>

Merge pull request #65338 from ronen-fr/wip-rf-be-st3

osd/scrub: modify OMAP stats collection

Reviewed-by: Jon Bailey <jonathan.bailey1@ibm.com>
Reviewed-by: Samuel Just <sjust@redhat.com>

Merge pull request #65193 from tchaikov/wip-osd-scrub-fix-buffer-overflow

osd/scrub: fix heap-buffer-overflow when checking digest emptiness

Reviewed-by: Jon Bailey <jonathan.bailey1@ibm.com>
Reviewed-by: Ronen Friedman <rfriedma@redhat.com>

mgr/cepahdm: fixing nvmeof scraping Prometheus config generation

Signed-off-by: Redouane Kachach <rkachach@ibm.com>

doc/cephadm: updating certmgr docs to reflects new changes

Signed-off-by: Redouane Kachach <rkachach@ibm.com>

mgr/cephadm: Adding RGW migration for the new certmgr certs format

Signed-off-by: Redouane Kachach <rkachach@ibm.com>

mgr/cephadm: removing call to previous method to storing all certs

Signed-off-by: Redouane Kachach <rkachach@ibm.com>

mgr/cephadm: fixing nvmeof section in cert_mgr UT + new UT

Signed-off-by: Redouane Kachach <rkachach@ibm.com>

mgr/cepahdm: adapting UT after all the changes

Signed-off-by: Redouane Kachach <rkachach@ibm.com>

mgr/cephadm: using 5 years for service-discovery internal certs

Signed-off-by: Redouane Kachach <rkachach@ibm.com>

mgr/cephadm: adding self-signed certifiactes support for nvmeof svc

Signed-off-by: Redouane Kachach <rkachach@ibm.com>

mgr/cepahdm: adapting node-proxy service to use the new cert mgmt

Signed-off-by: Redouane Kachach <rkachach@ibm.com>

mgr/cepahdm: adapting agent service to use the new cert mgmt

Signed-off-by: Redouane Kachach <rkachach@ibm.com>

mgr/cepahdm: adapting node-exporter service to use the new cert mgmt

Signed-off-by: Redouane Kachach <rkachach@ibm.com>

mgr/cepahdm: adapting Prometheus service to use the new cert mgmt

Signed-off-by: Redouane Kachach <rkachach@ibm.com>

mgr/cepahdm: adapting alertmanager service to use the new cert mgmt

Signed-off-by: Redouane Kachach <rkachach@ibm.com>

mgr/cepahdm: adapting oauth2-proxy service to use the new cert mgmt

Signed-off-by: Redouane Kachach <rkachach@ibm.com>

mgr/cepahdm: adapting Grafana service to use the new cert mgmt

Signed-off-by: Redouane Kachach <rkachach@ibm.com>

mgr/cephadm: adapting mgmt-gateway service to use the new cert mgmt

Signed-off-by: Redouane Kachach <rkachach@ibm.com>

mgr/cephadm: adapting iscsi service to use the new cert mgmt

Signed-off-by: Redouane Kachach <rkachach@ibm.com>

mgr/cephadm: adapting ingress svc to use the new cert mgmt approach

Signed-off-by: Redouane Kachach <rkachach@ibm.com>

mgr/cephadm: adding automated certificates mgmt for cephadmservice

Signed-off-by: Redouane Kachach <rkachach@ibm.com>

mgr/cepahdm: adding new filtring options and certificate ref checks

Signed-off-by: Redouane Kachach <rkachach@ibm.com>

mgr/cephadm: adding support to prevent editing readonly certs

Signed-off-by: Redouane Kachach <rkachach@ibm.com>

mgr/cephadm: adding enahanced support for self-signed certs

Signed-off-by: Redouane Kachach <rkachach@ibm.com>

mgr/cepahdm: add support for custom duration when generating certs

Signed-off-by: Redouane Kachach <rkachach@ibm.com>

mgr/cepahdm: adding support to extract ips and fqdns from cert

Signed-off-by: Redouane Kachach <rkachach@ibm.com>

mgr/cephadm: adapting python-common UT to the spec changes

Signed-off-by: Redouane Kachach <rkachach@ibm.com>

mgr/cephadm: adding new spec generic fields fo ssl/certifiactes

Signed-off-by: Redouane Kachach <rkachach@ibm.com>

osd/scrub: reinstate one-warning-per-chunk behaviour

Modify collect_omap_stats() to guarantee that only
one 'large omap entry' warning message is logged
per chunk, thus maintaining the existing behaviour.

Unlike the existing behaviour - all 'large omap'
entries are counted.

Signed-off-by: Ronen Friedman <rfriedma@redhat.com>

osd/scrub: modify OMAP stats collection

Replace the separate all-chunk loop with object-by-object
handling. Use the selected authoritative version of the object
as the info source.

Signed-off-by: Ronen Friedman <rfriedma@redhat.com>

Merge pull request #64859 from ronen-fr/wip-rf-72426movedref

osd/scrub: avoid using moved-from auth_n_errs

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
Reviewed-by: Alex Ainscow <aainscow@uk.ibm.com>
Reviewed-by: Laura Flores <lflores@redhat.com>

Merge pull request #64963 from edwinzrodriguez/ceph-wip-72452

test/rgw: Refactor CORS configuration handling for S3

Merge pull request #64801 from JoshuaGabriel/wip-jblanch/rgw-reindex

rgw: auto-create missing bucket index shards during reindex operations

Merge pull request #65414 from ceph/fix-api-tests

pybind/mgr/dashboard: Use teuthology's actual requirements

pybind/mgr/dashboard: Use teuthology's actual requirements

Signed-off-by: David Galloway <david.galloway@ibm.com>

Merge pull request #65011 from irq0/pr/fix-barbican-deprecated-call

rgw: Replace deprecated Barbican payload endpoint with alternative

Merge pull request #65376 from phlogistonjohn/jjm-bwc-npmdir-fix

build-with-container: ensure npm dir is set up before configure

Merge pull request #65062 from adamemerson/wip-redirect-error

common/async: improved `redirect_error`, `run_coro`, and tests

Reviewed-by: Anthony D'Atri anthony.datri@gmail.com
Reviewed-by: Casey Bodley cbodley@redhat.com
Reviewed-by: Jesse F. Williamson jfw@ibm.com

Merge pull request #64997 from AliMasarweh/wip-alimasa-72398

RGW | fixed enqueueing the overwritten object for gc

Reviewed-by: Daniel Gryniewicz <dang@redhat.com>

Merge pull request #64977 from MaxKellermann/includes3

Add missing includes to many subsystems

Reviewed-by: Adam Emerson <aemerson@redhat.com>
Reviewed-by: Ilya Dryomov <idryomov@redhat.com>

mgr/dashboard: fix RGW Bucket Notification Dashboard units

Fixes: https://tracker.ceph.com/issues/72868
Signed-off-by: Aashish Sharma <aasharma@redhat.com>

Merge pull request #65120 from leiwen2025/wip-add-rv64-high-precision-counter-support

common/Cycles: Add high-precision counter support for riscv64

Reviewed-by: Kefu Chai <tchaikov@gmail.com>

Merge pull request #65325 from zdover23/wip-doc-2025-09-02-cephfs-troubleshooting-consistent-cache

doc/cephfs: edit troubleshooting.rst

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>
Reviewed-by: Venky Shankar <vshankar@redhat.com>

Merge pull request #65215 from athanatos/sjust/wip-store-bench-3

crimson: further improvements to store-bench

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Matan Breizman <mbreizma@redhat.com>

Merge pull request #65016 from jamiepryde/isal-auto-switch-to-cauchy

erasure-code: Use ISA-L cauchy if reed_sol_van does not support the specified K and M values

Merge pull request #64845 from ljflores/wip-tracker-70468

qa/suites/orch/cephadm: ignore intentional OSD_DOWN warning from test_host_drain task

Merge pull request #65371 from cbodley/wip-72361

cmake: remove _FORTIFY_SOURCE define

build-with-container: ensure npm dir is set up before configure

When the npm cache path option is passed the npm cache dir is passed
to all container `run` commands, ensure the dir has been created
before the first container command (configure) is used.

Signed-off-by: John Mulligan <jmulligan@redhat.com>

Merge pull request #64863 from leonidc/cleanup_pending_map

nvmeofgw: cleanup pending map upon monitor restart

Reviewed-by: Samuel Just <sjust@redhat.com>

Merge pull request #64764 from badone/wip-tracker-72337-mgr-drop-daemonstateindex-lock

mgr/DaemonState: Minimise time we hold the DaemonStateIndex lock

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

cmake: remove _FORTIFY_SOURCE define

according to `dpkg-buildflags`, ubuntu 24 raised this value to
`-D_FORTIFY_SOURCE=3` which causes `error: "_FORTIFY_SOURCE" redefined`
compilation failures because Ceph itself adds `-D_FORTIFY_SOURCE=2`

`_FORTIFY_SOURCE` is a hardening option. both our rpm and debian builds
already specify that via environment variables, so Ceph's cmake should
leave it alone

Fixes: https://tracker.ceph.com/issues/72361
Signed-off-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #65313 from zmc/debs-nproc

make-debs.sh: Avoid OOM errors during build

Merge PR #64957 into main

* refs/pull/64957/head:
os: Page class: disable move operations and improve custom delete operator for proper memory management

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@ibm.com>

Merge PR #64627 into main

* refs/pull/64627/head:

Reviewed-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Dhairya Parmar <dparmar@redhat.com>

Merge pull request #65061 from zhscn/wip-replay-backref-retire

crimson/os/seastore: add retire alloc remarks for backref extents to skip replay invalid deltas

Reviewed-by: Myoungwon Oh <ohmyoungwon@gmail.com>
Reviewed-by: Matan Breizman <mbreizma@redhat.com>
Reviewed-by: Xuehan Xu <xuxuehan@qianxin.com>

doc/cephfs: edit troubleshooting.rst

Update the "Disconnected+Remounted FS" section in
doc/cephfs/troubleshooting.rst, as suggested by Venky Shankar in https://github.com/ceph/ceph/pull/65129/files#r2312903062

Signed-off-by: Zac Dover <zac.dover@proton.me>

osd/scrub: avoid using moved-from auth_n_errs

... in ScrubBackend::inconsistents()

Existing code was using a reference to a moved-from
`auth_and_obj_errs_t` object, which is undefined behavior.

The change here was tested locally. No noticeable performance
degradation was observed, and the code is now more robust.

Fixes: https://tracker.ceph.com/issues/72426
Signed-off-by: Ronen Friedman <rfriedma@redhat.com>

Merge pull request #64553 from jamiepryde/teuthology-ec-op-on-off

qa: Add a config setting to the teuthology erasure code tests so that we can test with EC optimizations on and/or off

Merge pull request #64458 from linuxkidd/add-bluestore-frag-warning-desc

mon/PGmap: Add warn description for bluestore fragmentation

Merge pull request #64995 from ifed01/wip-ifed-fix-snapdiff-fragment

mds: fix snapdiff result fragmentation

Reviewed-by: Venky Shankar <vshankar@redhat.com>

Merge pull request #65316 from dmick/uadk-ftbfs

Fix uadk build (arm64 only) on debian (conflict with DESTDIR)

Merge PR #65331 into main

* refs/pull/65331/head:
script/redmine-upkeep: fix formatting of comment

Reviewed-by: Afreen Misbah <afreen@ibm.com>

Merge pull request #65101 from spuiuk/smb_lazy_init

mgr/smb: use lazy_init to create the rados store

Reviewed-by: John Mulligan <jmulligan@redhat.com>
Reviewed-by: Avan Thakkar <athakkar@redhat.com>

Merge pull request #64603 from Naveenaidu/wip-naveen-fix-70972

qa/suites/rados/thrash-old-clients: Add OSD warnings to ignore list

Merge pull request #63687 from samarahu/wip-samarahu-standalone-user

Reviewed-by: Daniel Gryniewicz <dang@redhat.com>
Reviewed-by: Matt Benjamin <mbenjamin@redhat.com>

Merge pull request #64717 from abitdrag/tracker_72265

auth: msgr2 can return incorrect allowed_modes through AuthBadMethodFrame

Reviewed-by: Ilya Dryomov <idryomov@gmail.com>
Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

script/redmine-upkeep: fix formatting of comment

The update log link would appear as preformatted text.

Signed-off-by: Patrick Donnelly <pdonnell@ibm.com>

Merge pull request #64550 from kotreshhr/multifs-authcaps

Fix multifs auth caps check

Reviewed-by: Patrick Donnelly <pdonnell@ibm.com>
Reviewed-by: Venky Shankar <vshankar@redhat.com>

Merge pull request #64273 from tchaikov/wip-ceph-mutex-atomic

common/mutex_debug: fix arm64 SIGBUS/segfault due to data race

Merge pull request #64696 from rzarzynski/wip-bug-71966

tools: make rados ls aware about the locator concept

Reviewed-by: Matan Breizman <mbreizma@redhat.com>