Yael Azulay [Thu, 21 May 2026 11:38:28 +0000 (14:38 +0300)]
cephadm: keep mgr ports in sync when modules are enabled/disabled
ceph orch ps - was showing stale or incorrect port information for `mgr` daemons after enabling or disabling mgr modules (e.g., dashboard, prometheus).
This fix ensures the displayed ports always reflect the actual active module endpoints.
Code Fixes:
-----------
Changes in src/pybind/mgr/cephadm/services/cephadmservice.py:
- Extract and refactor _get_mgr_service_ports:
The logic for reading active mgr module ports was inline inside prepare_create, using check_mon_command({'prefix': 'mgr services'}). It was extracted into a static helper method and refactored to read directly from mgr.get('mgr_map')['services'] instead -- which is faster and consistent with how get_dependencies reads the same data. The method is now shared by both prepare_create (existing use) and get_dependencies (new use).
- Extract `_get_mgr_service_ports`:
The logic for reading active mgr module ports from `mgr_map['services']` was inline inside `prepare_create`. It was extracted into a static helper method so it can be reused by both `prepare_create` (existing use) and `get_dependencies` (new use). Returns the port numbers currently registered by active mgr modules (e.g., `[8443, 9283]`).
- Override `get_dependencies` in `MgrService`:
The base class returns `[]` for all Ceph daemons. The serve loop compares `get_dependencies()` output against `last_deps` (saved after the previous reconfig). If they differ, a reconfig is triggered. By overriding this method in `MgrService`, we make the serve loop aware of mgr port changes. Returns a sorted list such as `['port:8443', 'port:9283', 'sd_port:8765']`.
- Override `generate_config` in `MgrService`:
After a reconfig completes, the result of `generate_config` is saved as `last_deps`. The inherited implementation returns `[]` as deps. If `[]` is saved but `get_dependencies` returns `['port:8443', 'sd_port:8765']`, they would always differ, causing an infinite reconfig loop. This override ensures that what is saved as `last_deps` matches what `get_dependencies` will return in the next iteration.
Changes in `src/pybind/mgr/cephadm/serve.py`:
- Update mgr cache on reconfig:
The existing logic skipped the cache update for all Ceph daemon reconfigs. A new `elif` branch for mgr reconfigs fetches the existing cached `DaemonDescription`, updates only its `ports` field, and saves it back. This makes `ceph orch ps` reflect the correct ports immediately after a reconfig, without showing a misleading "starting" status.
Changes in `src/cephadm/cephadm.py`:
- Update `unit.meta` on reconfig:
`unit.meta` is a per-daemon file on the host that stores metadata including ports. It is read periodically by `cephadm ls` and fed back into the orchestrator cache. Previously it was only written on initial deployment, so every periodic cache refresh would overwrite the corrected ports with stale data from `unit.meta`. This change updates the `ports` field in `unit.meta` during reconfig, ensuring the cache refresh confirms the correct state rather than reverting it.
Changes to src/pybind/mgr/cephadm/tests/services/test_mgr.py:
- Updated _prepare helper:
The original mocked check_mon_command to feed services data to prepare_create. Since our _get_mgr_service_ports now reads from mgr.get('mgr_map') instead of check_mon_command, the mock was replaced with mock_store_set('_ceph_get', 'mgr_map', ...) which sets the data in the right place.
- Added test_get_dependencies_changes_when_module_enabled:
A new test that verifies get_dependencies returns different dep strings before and after a module is enabled. This directly tests the core sync mechanism -- the serve loop compares get_dependencies output against last_deps to detect port changes and trigger a reconfig.
to run new tests:
cd src/pybind/mgr
tox -e py3 -- cephadm/tests/services/test_mgr.py -v
Ilya Dryomov [Fri, 26 Jun 2026 10:56:27 +0000 (12:56 +0200)]
script/ptl-tool: fix redmine_linkage_correct case in _get_active_issues()
If redmine_linkage_correct is True, _get_active_issues() is supposed to
ignore issues with "Multiple Source PRs" category. However that doesn't
happen because the category string is compared against (category, text)
pair.
Sun Yuechi [Thu, 25 Jun 2026 11:50:36 +0000 (19:50 +0800)]
cmake: build fio from upstream 3.42
fio-3.42 is the latest stable release. We switched to the ceph/fio fork in 10baab3fc8293b8c30ca90a4acd76f70d011f1b5 to pick up a clang build fix; that
fix has since landed upstream, so switch back. This also gives riscv64
proper arch support (arch-riscv64.h, added in fio-3.36), which the fork's
fio-3.27-cxx lacks.
Avan Thakkar [Thu, 11 Jun 2026 09:55:33 +0000 (15:25 +0530)]
mgr/smb: add RGW credentials resource and store in priv_stor
Refactored RGW credential management to use dedicated RGWCredential
resources instead of embedding credentials directly in Share resources.
This will help to reuse the same credential across multiple shares using
same user.
RGW credentials must not appear in the public RADOS store, which any
client with pool caps can read. Instead, write a config:merge stub
containing only the credential fields to the private mon config-key
store, and pass its URI to the container alongside the primary config
URI via extra_config_uris.
- Public config: ceph_rgw:access_key / secret_access_key are empty strings
- Private stub: stored under smb/config/<cluster_id>/config.smb.rgw,
contains only the real credential values via config:merge
- extra_config_uris: new Config field carrying supplementary URIs
appended to SAMBACC_CONFIG after the primary config_uri
- Add rgw.py module for RGW integration
- Implement fetch_rgw_credentials() to get user credentials from radosgw-admin
- Implement validate_rgw_bucket() to check bucket existence
- Handle bucket owner lookup when user_id not provided
- Add proper error handling and logging
This provides the foundation for RGW authentication in SMB shares,
similar to the NFS RGW export implementation.
mgr/smb: Add RGWStorage resource class and update Share to support RGW
- Add RGWStorage component class for RGW bucket storage backend
- Include fields: bucket, user_id, access_key_id, secret_access_key
- Add password conversion support for credentials
- Update Share class to accept optional rgw field and added convert
defination
- Update Share validation to support both cephfs and rgw backends
- Ensure exactly one storage backend (cephfs or rgw) is specified
This is the first step in implementing RGW storage support for SMB
shares, similar to the existing NFS RGW export functionality.
rbd-mirror: Remove old non-primary demoted image snapshots on the local cluster
When an image is demoted on primary cluster and later promoted again,
a non-primary demoted image snapshot is created on the peer (secondary)
cluster. Each such promote/demote cycle on same cluster results in an
additional non-primary demoted snapshot being created on peer.
As a result, repeated promote/demote cycles on the same cluster can
lead to accumulation of stale non-primary demoted snapshots on the
secondary cluster. These snapshots are not removed immediately because
cleanup is only triggered when peer (secondary) is promoted and then
demoted.
The proposed changes ensures that such snapshots are proactively identified
and removed, preventing unbounded buildup. Also added test coverage to
verify cleanup behavior.
Ronen Friedman [Thu, 11 Jun 2026 12:32:34 +0000 (12:32 +0000)]
crimson/osd: return ENOENT from EC getxattr for non-existent objects
PGBackend::getxattr() for EC pools reads from the attr_cache and
returns ENODATA when an attr is not found, without distinguishing
between "object exists but attr not set" and "object doesn't exist".
This causes cls methods like cls_refcount_get() to proceed as if the
object exists, enter the EC RMW pipeline, and hang.
Add an os.exists check in the EC path so non-existent objects return
ENOENT, matching the behavior of the replicated path which goes to
the object store and gets ENOENT directly.
Avan Thakkar [Wed, 24 Jun 2026 11:15:51 +0000 (16:45 +0530)]
mgr: fix MgrModuleRecoverDB to honor its retry budget on db cleanup failure
Both close_db() and open_db() during retry were unguarded, so a failure
in either escaped the decorator immediately instead of retrying up to
MAX_DBCLEANUP_RETRIES.
Wrapped both in the same try/except, and logged the
failed attempt.
Sun Yuechi [Sun, 14 Jun 2026 12:43:50 +0000 (20:43 +0800)]
test/encoding: run readable.sh with a sliding window of jobs
The old scheduler spawned nproc jobs, then waited for the whole batch to
finish before starting the next, so cores that finished early sat idle
waiting on the slowest type in the batch. Keep a sliding window instead:
once the window is full, reap a single finished job before spawning the
next, which keeps the cores busy when per-type runtimes vary widely.
While here, pass vdir/arversion into test_object explicitly instead of
reading the loop globals, and give each job its own result file instead
of reusing slot-numbered files that the sliding window would otherwise
hand out to overlapping jobs.
Do not mark an upgrade complete until daemons in the filtered scope
match the target image. Limited (--limit) upgrades still complete
when remaining_count reaches zero.
Sun Yuechi [Tue, 23 Jun 2026 06:16:38 +0000 (14:16 +0800)]
test/bufferlist: shrink benchmark iterations under ASan
The pure-timing *_bench/BenchAlloc loops repeat a stateless path millions of
times. Under ASan that adds no memory-error coverage but inflates the run from
minutes to hours, so cut the iteration counts when built with ASan.
Sun Yuechi [Tue, 23 Jun 2026 10:18:25 +0000 (18:18 +0800)]
test/bufferlist: avoid BenchDeref iterator overrun past end
Drive the deref loop by get_remaining() >= step instead of comparing to
end(), so a total length that is not a multiple of step no longer lets
`iter += step` run past the end and throw "End of buffer".
The original counts were always a multiple of step, so the overrun only
showed up once the following commit shrinks the rounds under ASan -- a
false alarm rather than a real regression, which is why it went unnoticed
until now.
Collect FCM stats locally from NVMe drives (vendor log page 0xCA)
and expose them via node-proxy, the cephadm agent, and
`ceph orch hardware status --category fcm`.
Introduce a node backend that aggregates Redfish data with
node-local collectors, since FCM metrics are not available
from the BMC.
Kefu Chai [Tue, 23 Jun 2026 07:46:05 +0000 (15:46 +0800)]
cmake/boost: build libboost_context with BOOST_USE_ASAN
The ceph-api job failed under ASan: radosgw-admin aborted at startup with a
heap-buffer-overflow in boost.context's fiber resume().
==1155842==ERROR: AddressSanitizer: heap-buffer-overflow ... READ of size 8
#0 boost::context::detail::fiber_activation_record::resume() fiber_ucontext.hpp:153
#10 RGWSI_Notify::do_start(optional_yield, DoutPrefixProvider const*) svc_notify.cc:261
0x... is located 8 bytes after 1048-byte region allocated by
boost::context::detail::fiber_activation_record_initializer()
fiber_activation_record carries three extra members (fake_stack, stack_bottom,
stack_size) only when BOOST_USE_ASAN is defined. Under WITH_ASAN we build
boost.context with context-impl=ucontext and give consumers BOOST_USE_ASAN and
BOOST_USE_UCONTEXT through Boost::context's INTERFACE_COMPILE_DEFINITIONS, but
we never pass BOOST_USE_ASAN to b2. So libboost_context, which compiles
fiber_activation_record_initializer() and allocates the record, uses the
smaller no-ASan layout, while consumers that include the header (here rgw's
RGWSI_Notify::do_start via boost::asio::spawn) compile resume() with the larger
layout and read stack_bottom/stack_size past the allocation.
Pass define=BOOST_USE_ASAN to b2 so the library and its consumers agree on the
struct layout.
Afreen Misbah [Tue, 23 Jun 2026 13:30:22 +0000 (19:00 +0530)]
mgr/dashboard: fix bind address regression from CherryPy isolation
The CherryPy isolation refactor (PR #67227) accidentally changed the
dashboard bind address from wildcard (*:8443) to mon_ip:8443. The
get_mgr_ip() replacement was originally only for URI generation, but
the refactor passed the mutated address to CherryPyMgr.mount() as the
actual socket bind address.
This breaks the management gateway when its VIP is not on the same
interface as mon_ip, as the dashboard becomes unreachable on other
interfaces.
Preserve the original wildcard address for binding and only use
get_mgr_ip() for the advertised URI. Add regression test to prevent
future confusion between bind_addr and server_addr.
The python binaries on some CI images and dev boxes ship stripped, so the
allocator frames in an interpreter-shutdown leak come through unsymbolised
(/usr/bin/python3.13+0x...) and the function-name matches above cannot apply.
leak:python3.10 already handled this for 3.10, and a stale comment claimed 3.12
does not leak.
Add leak:python3.12 and leak:python3.13, mirroring the 3.10 entry, so the
interpreter globals are suppressed whatever CPython the build uses.
Kefu Chai [Tue, 23 Jun 2026 08:14:19 +0000 (16:14 +0800)]
ceph.in: load asan/lsan suppressions on WITH_ASAN builds
bin/ceph from a WITH_ASAN build aborts at exit, with LeakSanitizer reporting
CPython and Cython module-init allocations as leaks:
==2577940==ERROR: LeakSanitizer: detected memory leaks
Direct leak ... in PyObject_Malloc (/usr/bin/python3.12+...)
#4 __pyx_pymod_exec_rados rados_processed.c
SUMMARY: AddressSanitizer: 32113 byte(s) leaked in 30 allocation(s).
These are interpreter globals that live for the process lifetime. qa/lsan.supp
already suppresses them, but bin/ceph never loaded it: vstart.sh sets
LSAN_OPTIONS for the daemons it spawns, while a bin/ceph invoked separately
(ceph-api runs ./bin/ceph fsid once vstart.sh returns) inherits none and exits
non-zero. It stayed hidden until radosgw-admin stopped crashing in vstart and
the run reached that call.
ceph.in already re-execs with the ASan runtime preloaded under WITH_ASAN. Set
ASAN_OPTIONS and LSAN_OPTIONS first, from the CEPH_ASAN_OPTIONS and
CEPH_LSAN_OPTIONS that CMake also feeds add_ceph_test(), so the re-exec'd
interpreter starts with the suppressions loaded. Use setdefault so a value
from the caller still wins.
Kefu Chai [Tue, 23 Jun 2026 08:14:19 +0000 (16:14 +0800)]
cmake: factor the ASan/LSan test options into cache variables
add_ceph_test() spelled out the suppression-file paths and sanitizer flags
inline. bin/ceph needs the same options, so lift them into CEPH_ASAN_OPTIONS
and CEPH_LSAN_OPTIONS and have add_ceph_test() consume those. The environment
the tests run with is unchanged.
projects / ceph.git / log