rgw/logging: fixes data loss during rollover

Multiple threads attempting to roll over the same log object can result
in the creation of numerous orphan tail objects, each with a single record.
This occurs when a NULL RGWObjVersionTracker is used during the creation of
a new logging object. These records are inaccessible, leading to data loss,
which is particularly critical in Journal mode.
Furthermore, valid log tail objects may be added to the Garbage Collection (GC)
list, exacerbating data loss.

Fixes: https://tracker.ceph.com/issues/72740
Signed-off-by: N Balachandran <nithya.balachandran@ibm.com>

Merge pull request #65421 from ronen-fr/wip-rf-ec72898

osd/scrub: clear m_ec_digest_map between objects

Reviewed-by: Jon Bailey <jonathan.bailey1@ibm.com>

Merge pull request #62106 from rkachach/fix_certmgr_v2

Add cephadm-signed certificate support for all services

Reviewed-by: John Mulligan <jmulligan@redhat.com>

osd/scrub: clear m_ec_digest_map between objects

Fixing a bug introduced by commit 4c61079e931
("caluculate EC digest map size only once").

Fixes: https://tracker.ceph.com/issues/72897
Signed-off-by: Ronen Friedman <rfriedma@redhat.com>

Merge pull request #65150 from rkachach/fix_issue_nvmeof_prometheus

mgr/cepahdm: fixing nvmeof scraping Prometheus config generation

Reviewed-by: Kushal Deb <Kushal.Deb@ibm.com>

Merge pull request #65338 from ronen-fr/wip-rf-be-st3

osd/scrub: modify OMAP stats collection

Reviewed-by: Jon Bailey <jonathan.bailey1@ibm.com>
Reviewed-by: Samuel Just <sjust@redhat.com>

Merge pull request #65193 from tchaikov/wip-osd-scrub-fix-buffer-overflow

osd/scrub: fix heap-buffer-overflow when checking digest emptiness

Reviewed-by: Jon Bailey <jonathan.bailey1@ibm.com>
Reviewed-by: Ronen Friedman <rfriedma@redhat.com>

mgr/cepahdm: fixing nvmeof scraping Prometheus config generation

Signed-off-by: Redouane Kachach <rkachach@ibm.com>

doc/cephadm: updating certmgr docs to reflects new changes

Signed-off-by: Redouane Kachach <rkachach@ibm.com>

mgr/cephadm: Adding RGW migration for the new certmgr certs format

Signed-off-by: Redouane Kachach <rkachach@ibm.com>

mgr/cephadm: removing call to previous method to storing all certs

Signed-off-by: Redouane Kachach <rkachach@ibm.com>

mgr/cephadm: fixing nvmeof section in cert_mgr UT + new UT

Signed-off-by: Redouane Kachach <rkachach@ibm.com>

mgr/cepahdm: adapting UT after all the changes

Signed-off-by: Redouane Kachach <rkachach@ibm.com>

mgr/cephadm: using 5 years for service-discovery internal certs

Signed-off-by: Redouane Kachach <rkachach@ibm.com>

mgr/cephadm: adding self-signed certifiactes support for nvmeof svc

Signed-off-by: Redouane Kachach <rkachach@ibm.com>

mgr/cepahdm: adapting node-proxy service to use the new cert mgmt

Signed-off-by: Redouane Kachach <rkachach@ibm.com>

mgr/cepahdm: adapting agent service to use the new cert mgmt

Signed-off-by: Redouane Kachach <rkachach@ibm.com>

mgr/cepahdm: adapting node-exporter service to use the new cert mgmt

Signed-off-by: Redouane Kachach <rkachach@ibm.com>

mgr/cepahdm: adapting Prometheus service to use the new cert mgmt

Signed-off-by: Redouane Kachach <rkachach@ibm.com>

mgr/cepahdm: adapting alertmanager service to use the new cert mgmt

Signed-off-by: Redouane Kachach <rkachach@ibm.com>

mgr/cepahdm: adapting oauth2-proxy service to use the new cert mgmt

Signed-off-by: Redouane Kachach <rkachach@ibm.com>

mgr/cepahdm: adapting Grafana service to use the new cert mgmt

Signed-off-by: Redouane Kachach <rkachach@ibm.com>

mgr/cephadm: adapting mgmt-gateway service to use the new cert mgmt

Signed-off-by: Redouane Kachach <rkachach@ibm.com>

mgr/cephadm: adapting iscsi service to use the new cert mgmt

Signed-off-by: Redouane Kachach <rkachach@ibm.com>

mgr/cephadm: adapting ingress svc to use the new cert mgmt approach

Signed-off-by: Redouane Kachach <rkachach@ibm.com>

mgr/cephadm: adding automated certificates mgmt for cephadmservice

Signed-off-by: Redouane Kachach <rkachach@ibm.com>

mgr/cepahdm: adding new filtring options and certificate ref checks

Signed-off-by: Redouane Kachach <rkachach@ibm.com>

mgr/cephadm: adding support to prevent editing readonly certs

Signed-off-by: Redouane Kachach <rkachach@ibm.com>

mgr/cephadm: adding enahanced support for self-signed certs

Signed-off-by: Redouane Kachach <rkachach@ibm.com>

mgr/cepahdm: add support for custom duration when generating certs

Signed-off-by: Redouane Kachach <rkachach@ibm.com>

mgr/cepahdm: adding support to extract ips and fqdns from cert

Signed-off-by: Redouane Kachach <rkachach@ibm.com>

mgr/cephadm: adapting python-common UT to the spec changes

Signed-off-by: Redouane Kachach <rkachach@ibm.com>

mgr/cephadm: adding new spec generic fields fo ssl/certifiactes

Signed-off-by: Redouane Kachach <rkachach@ibm.com>

osd/scrub: reinstate one-warning-per-chunk behaviour

Modify collect_omap_stats() to guarantee that only
one 'large omap entry' warning message is logged
per chunk, thus maintaining the existing behaviour.

Unlike the existing behaviour - all 'large omap'
entries are counted.

Signed-off-by: Ronen Friedman <rfriedma@redhat.com>

osd/scrub: modify OMAP stats collection

Replace the separate all-chunk loop with object-by-object
handling. Use the selected authoritative version of the object
as the info source.

Signed-off-by: Ronen Friedman <rfriedma@redhat.com>

Merge pull request #64859 from ronen-fr/wip-rf-72426movedref

osd/scrub: avoid using moved-from auth_n_errs

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
Reviewed-by: Alex Ainscow <aainscow@uk.ibm.com>
Reviewed-by: Laura Flores <lflores@redhat.com>

Merge pull request #64963 from edwinzrodriguez/ceph-wip-72452

test/rgw: Refactor CORS configuration handling for S3

Merge pull request #64801 from JoshuaGabriel/wip-jblanch/rgw-reindex

rgw: auto-create missing bucket index shards during reindex operations

Merge pull request #65414 from ceph/fix-api-tests

pybind/mgr/dashboard: Use teuthology's actual requirements

pybind/mgr/dashboard: Use teuthology's actual requirements

Signed-off-by: David Galloway <david.galloway@ibm.com>

Merge pull request #65011 from irq0/pr/fix-barbican-deprecated-call

rgw: Replace deprecated Barbican payload endpoint with alternative

Merge pull request #65376 from phlogistonjohn/jjm-bwc-npmdir-fix

build-with-container: ensure npm dir is set up before configure

Merge pull request #65062 from adamemerson/wip-redirect-error

common/async: improved `redirect_error`, `run_coro`, and tests

Reviewed-by: Anthony D'Atri anthony.datri@gmail.com
Reviewed-by: Casey Bodley cbodley@redhat.com
Reviewed-by: Jesse F. Williamson jfw@ibm.com

Merge pull request #64997 from AliMasarweh/wip-alimasa-72398

RGW | fixed enqueueing the overwritten object for gc

Reviewed-by: Daniel Gryniewicz <dang@redhat.com>

Merge pull request #64977 from MaxKellermann/includes3

Add missing includes to many subsystems

Reviewed-by: Adam Emerson <aemerson@redhat.com>
Reviewed-by: Ilya Dryomov <idryomov@redhat.com>

Merge pull request #65120 from leiwen2025/wip-add-rv64-high-precision-counter-support

common/Cycles: Add high-precision counter support for riscv64

Reviewed-by: Kefu Chai <tchaikov@gmail.com>

Merge pull request #65325 from zdover23/wip-doc-2025-09-02-cephfs-troubleshooting-consistent-cache

doc/cephfs: edit troubleshooting.rst

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>
Reviewed-by: Venky Shankar <vshankar@redhat.com>

Merge pull request #65215 from athanatos/sjust/wip-store-bench-3

crimson: further improvements to store-bench

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Matan Breizman <mbreizma@redhat.com>

Merge pull request #65016 from jamiepryde/isal-auto-switch-to-cauchy

erasure-code: Use ISA-L cauchy if reed_sol_van does not support the specified K and M values

Merge pull request #64845 from ljflores/wip-tracker-70468

qa/suites/orch/cephadm: ignore intentional OSD_DOWN warning from test_host_drain task

Merge pull request #65371 from cbodley/wip-72361

cmake: remove _FORTIFY_SOURCE define

build-with-container: ensure npm dir is set up before configure

When the npm cache path option is passed the npm cache dir is passed
to all container `run` commands, ensure the dir has been created
before the first container command (configure) is used.

Signed-off-by: John Mulligan <jmulligan@redhat.com>

Merge pull request #64863 from leonidc/cleanup_pending_map

nvmeofgw: cleanup pending map upon monitor restart

Reviewed-by: Samuel Just <sjust@redhat.com>

Merge pull request #64764 from badone/wip-tracker-72337-mgr-drop-daemonstateindex-lock

mgr/DaemonState: Minimise time we hold the DaemonStateIndex lock

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

cmake: remove _FORTIFY_SOURCE define

according to `dpkg-buildflags`, ubuntu 24 raised this value to
`-D_FORTIFY_SOURCE=3` which causes `error: "_FORTIFY_SOURCE" redefined`
compilation failures because Ceph itself adds `-D_FORTIFY_SOURCE=2`

`_FORTIFY_SOURCE` is a hardening option. both our rpm and debian builds
already specify that via environment variables, so Ceph's cmake should
leave it alone

Fixes: https://tracker.ceph.com/issues/72361
Signed-off-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #65313 from zmc/debs-nproc

make-debs.sh: Avoid OOM errors during build

Merge PR #64957 into main

* refs/pull/64957/head:
os: Page class: disable move operations and improve custom delete operator for proper memory management

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@ibm.com>

Merge PR #64627 into main

* refs/pull/64627/head:

Reviewed-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Dhairya Parmar <dparmar@redhat.com>

Merge pull request #65061 from zhscn/wip-replay-backref-retire

crimson/os/seastore: add retire alloc remarks for backref extents to skip replay invalid deltas

Reviewed-by: Myoungwon Oh <ohmyoungwon@gmail.com>
Reviewed-by: Matan Breizman <mbreizma@redhat.com>
Reviewed-by: Xuehan Xu <xuxuehan@qianxin.com>

doc/cephfs: edit troubleshooting.rst

Update the "Disconnected+Remounted FS" section in
doc/cephfs/troubleshooting.rst, as suggested by Venky Shankar in https://github.com/ceph/ceph/pull/65129/files#r2312903062

Signed-off-by: Zac Dover <zac.dover@proton.me>

osd/scrub: avoid using moved-from auth_n_errs

... in ScrubBackend::inconsistents()

Existing code was using a reference to a moved-from
`auth_and_obj_errs_t` object, which is undefined behavior.

The change here was tested locally. No noticeable performance
degradation was observed, and the code is now more robust.

Fixes: https://tracker.ceph.com/issues/72426
Signed-off-by: Ronen Friedman <rfriedma@redhat.com>

Merge pull request #64553 from jamiepryde/teuthology-ec-op-on-off

qa: Add a config setting to the teuthology erasure code tests so that we can test with EC optimizations on and/or off

Merge pull request #64458 from linuxkidd/add-bluestore-frag-warning-desc

mon/PGmap: Add warn description for bluestore fragmentation

Merge pull request #64995 from ifed01/wip-ifed-fix-snapdiff-fragment

mds: fix snapdiff result fragmentation

Reviewed-by: Venky Shankar <vshankar@redhat.com>

Merge pull request #65316 from dmick/uadk-ftbfs

Fix uadk build (arm64 only) on debian (conflict with DESTDIR)

Merge PR #65331 into main

* refs/pull/65331/head:
script/redmine-upkeep: fix formatting of comment

Reviewed-by: Afreen Misbah <afreen@ibm.com>

Merge pull request #65101 from spuiuk/smb_lazy_init

mgr/smb: use lazy_init to create the rados store

Reviewed-by: John Mulligan <jmulligan@redhat.com>
Reviewed-by: Avan Thakkar <athakkar@redhat.com>

Merge pull request #64603 from Naveenaidu/wip-naveen-fix-70972

qa/suites/rados/thrash-old-clients: Add OSD warnings to ignore list

Merge pull request #63687 from samarahu/wip-samarahu-standalone-user

Reviewed-by: Daniel Gryniewicz <dang@redhat.com>
Reviewed-by: Matt Benjamin <mbenjamin@redhat.com>

Merge pull request #64717 from abitdrag/tracker_72265

auth: msgr2 can return incorrect allowed_modes through AuthBadMethodFrame

Reviewed-by: Ilya Dryomov <idryomov@gmail.com>
Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

script/redmine-upkeep: fix formatting of comment

The update log link would appear as preformatted text.

Signed-off-by: Patrick Donnelly <pdonnell@ibm.com>

Merge pull request #64550 from kotreshhr/multifs-authcaps

Fix multifs auth caps check

Reviewed-by: Patrick Donnelly <pdonnell@ibm.com>
Reviewed-by: Venky Shankar <vshankar@redhat.com>

Merge pull request #64273 from tchaikov/wip-ceph-mutex-atomic

common/mutex_debug: fix arm64 SIGBUS/segfault due to data race

Merge pull request #64696 from rzarzynski/wip-bug-71966

tools: make rados ls aware about the locator concept

Reviewed-by: Matan Breizman <mbreizma@redhat.com>

Merge PR #64663 into main

* refs/pull/64663/head:

Reviewed-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@ibm.com>

Merge pull request #65079 from rhcs-dashboard/img-summary-mirroring

mgr/dashboard: expose image summary API

Merge pull request #65240 from knrt10/test-crimson-librbd-reenable

qa/suites/crimson-rados: run full librbd test suite

Reviewed-by: Matan Breizman <mbreizma@redhat.com>

Merge pull request #64554 from jamiepryde/teuthology-ec-more-configs

qa: Add cauchy and 16K chunk size profiles to rados erasure code teuthology suites

Merge pull request #64577 from rzarzynski/wip-ec-shuffling-for-crimson

osd: shuffle FastEC code needed by crimson from ECBackend to ECCommon

Merge pull request #65192 from tchaikov/wip-cmake-silence-lsan

cmake: disable LeakSanitizer suppression output to fix CLI tests

Reviewed-by: Casey Bodley <cbodley@redhat.com>

cmake/modules/Builduadk.cmake: fix build

DESTDIR should not be set when building.

Fixes: https://tracker.ceph.com/issues/72722
Signed-off-by: Dan Mick <dan.mick@redhat.com>

cmake/modules/Builduadk.cmake: fix tabs/spaces

Signed-off-by: Dan Mick <dan.mick@redhat.com>

Merge PR #65276 into main

* refs/pull/65276/head:
.github/workflows: run redmine-upkeep for release branches

Reviewed-by: Afreen Misbah <afreen@ibm.com>

make-debs.sh: Avoid OOM errors during build

This uses the logic from: https://github.com/ceph/ceph-build/blob/f074f1ce7fc05a5ac69e06fa9fa7ae45858b0110/scripts/build_utils.sh#L1411-L1435

Signed-off-by: Zack Cerza <zack@cerza.org>
CEPH-BUILD-JOB: ceph-dev-pipeline
ARCHS: x86_64
DISTROS: jammy noble
FLAVORS: default

Merge pull request #65285 from rhcs-dashboard/table-actions

mgr/dashboard: add customizations to table-actions

Reviewed-by: Afreen Misbah <afreen@ibm.com>

Merge pull request #65246 from afreen23/remove-old-dash

Drop support for older version of dashboard

Reviewed-by: Nizamudeen A <nia@redhat.com>

Merge pull request #63641 from anushruti10116/add-bucket-notification-dashboard

mgr/dashboard: Adding RGW Bucket Notification Dashboard for Grafana

osd/scrub: fix heap-buffer-overflow when checking digest emptiness

Replace unsafe string construction with bufferlist::length() to avoid
reading beyond buffer boundaries.

In commit 92ccbff1, we introduced a bug when checking if a digest was
empty by constructing a std::string from bufferlist:

```
std::string(digest.second.c_str()).empty()
```

This is unsafe because bufferlist data is not guaranteed to be null-
terminated. The std::string constructor searches for a null terminator
and may read beyond the bufferlist's allocated memory, causing a
heap-buffer-overflow detected by AddressSanitizer:

```
==66092==ERROR: AddressSanitizer: heap-buffer-overflow on address 0x7e0c65215004 at pc 0x7fbc6e27c597 bp 0x7ffe29fb6100 sp 0x7ffe29fb58b8
READ of size 5 at 0x7e0c65215004 thread T0
    #0 0x7fbc6e27c596 in strlen /usr/src/debug/gcc/gcc/libsanitizer/sanitizer_common/sanitizer_common_interceptors.inc:425
    #1 0x562c75fad91a in std::char_traits<char>::length(char const*) /usr/include/c++/15.2.1/bits/char_traits.h:393
    #2 0x562c75fb4222 in std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >::basic_string<std::allocator<char> >(char const*, std::allocator<char> const&) /usr/include/c++/15.2.1/bits/b
asic_string.h:713
    #3 0x562c761b81ae in operator() /home/kefu/dev/ceph/src/osd/scrubber/scrub_backend.cc:1300
    #4 0x562c761d7d53 in operator()<mini_flat_map<shard_id_t, ceph::buffer::v15_2_0::list, signed char>::_iterator<false> > /usr/include/c++/15.2.1/bits/predefined_ops.h:318
    #5 0x562c761d789c in __find_if<mini_flat_map<shard_id_t, ceph::buffer::v15_2_0::list, signed char>::_iterator<false>, __gnu_cxx::__ops::_Iter_pred<ScrubBackend::match_in_shards(const hobject_t&, auth_selection_
t&, inconsistent_obj_wrapper&, std::stringstream&)::<lambda(const std::pair<const shard_id_t, ceph::buffer::v15_2_0::list&>&)> > > /usr/include/c++/15.2.1/bits/stl_algobase.h:2095
    #6 0x562c761d72b2 in find_if<mini_flat_map<shard_id_t, ceph::buffer::v15_2_0::list, signed char>::_iterator<false>, ScrubBackend::match_in_shards(const hobject_t&, auth_selection_t&, inconsistent_obj_wrapper&,
std::stringstream&)::<lambda(const std::pair<const shard_id_t, ceph::buffer::v15_2_0::list&>&)> > /usr/include/c++/15.2.1/bits/stl_algo.h:3921
    #7 0x562c761d5f6f in none_of<mini_flat_map<shard_id_t, ceph::buffer::v15_2_0::list, signed char>::_iterator<false>, ScrubBackend::match_in_shards(const hobject_t&, auth_selection_t&, inconsistent_obj_wrapper&,
std::stringstream&)::<lambda(const std::pair<const shard_id_t, ceph::buffer::v15_2_0::list&>&)> > /usr/include/c++/15.2.1/bits/stl_algo.h:431
    #8 0x562c761d4a50 in any_of<mini_flat_map<shard_id_t, ceph::buffer::v15_2_0::list, signed char>::_iterator<false>, ScrubBackend::match_in_shards(const hobject_t&, auth_selection_t&, inconsistent_obj_wrapper&, s
td::stringstream&)::<lambda(const std::pair<const shard_id_t, ceph::buffer::v15_2_0::list&>&)> > /usr/include/c++/15.2.1/bits/stl_algo.h:450
    #9 0x562c761bb84b in ScrubBackend::match_in_shards(hobject_t const&, auth_selection_t&, inconsistent_obj_wrapper&, std::__cxx11::basic_stringstream<char, std::char_traits<char>, std::allocator<char> >&) /home/k
efu/dev/ceph/src/osd/scrubber/scrub_backend.cc:1297
    #10 0x562c761b4282 in ScrubBackend::compare_obj_in_maps[abi:cxx11](hobject_t const&) /home/kefu/dev/ceph/src/osd/scrubber/scrub_backend.cc:941
    #11 0x562c761d44af in operator()<hobject_t> /home/kefu/dev/ceph/src/osd/scrubber/scrub_backend.cc:887
    #12 0x562c761d4836 in for_each<std::_Rb_tree_const_iterator<hobject_t>, ScrubBackend::compare_smaps()::<lambda(const auto:422&)> > /usr/include/c++/15.2.1/bits/stl_algo.h:3798
    #13 0x562c761b3259 in ScrubBackend::compare_smaps() /home/kefu/dev/ceph/src/osd/scrubber/scrub_backend.cc:884
    #14 0x562c761a478d in ScrubBackend::update_authoritative() /home/kefu/dev/ceph/src/osd/scrubber/scrub_backend.cc:315`
```

Fix by using bufferlist::length() which tells if the given buffer is
empty instead of converting the buffer content to a string.

Signed-off-by: Kefu Chai <tchaikov@gmail.com>

Merge pull request #65284 from pritha-srivastava/wip-rgw-d4n-centos9-build-failure

rgw/d4n: removing fmt related code as it is

Reviewed-by: Casey Bodley <cbodley@redhat.com>

mgr/dashboard: Update release notes and doc

-  for removal of old dashoard support

Signed-off-by: Afreen Misbah <afreen@ibm.com>

Merge pull request #65176 from phlogistonjohn/jjm-cephadm-build-el10rpm

cephadm: fix building rpm-sourced cephadm zippapp on el10

Reviewed-by: Adam King <adking@redhat.com>

rgw/d4n: removing fmt related code as it is
causing centos 9 build failure and is debug
only code.

Signed-off-by: Pritha Srivastava <prsrivas@redhat.com>

mgr/dashboard: add customizations to table-actions

https://tracker.ceph.com/issues/72764

Signed-off-by: Naman Munet <naman.munet@ibm.com>

Merge pull request #65029 from guits/drop-udevadm-subprocess-calls

ceph-volume: drop udevadm subprocess calls

Reviewed-by: Adam King <adking@redhat.com>

Merge pull request #64955 from rhcs-dashboard/bug-2314829-about-panel

mgr/dashboard: About panel showing other icons in background while open

Reviewed-by: Afreen Misbah <afreen@ibm.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
Reviewed-by: Dnyaneshwari Talwekar <dtalwekar@redhat.com>

Merge pull request #65146 from rhcs-dashboard/fix-72656-main

mgr/dashboard: Fix duplicate selection on multi-select in table component

Reviewed-by: Afreen Misbah <afreen@ibm.com>

common/CommandTable.h: add missing includes

Signed-off-by: Max Kellermann <max.kellermann@ionos.com>

common/ceph_time: add missing include

Signed-off-by: Max Kellermann <max.kellermann@ionos.com>

crush/CrushWrapper: add missing include

Signed-off-by: Max Kellermann <max.kellermann@ionos.com>

msg_types: add missing include

Signed-off-by: Max Kellermann <max.kellermann@ionos.com>