mgr/cephadm: don't treat unknown daemon state as error
cephadm ls may transiently report state='unknown' for daemons that are
still starting up. The state mapping in _refresh_host_daemons mapped
'unknown' to DaemonDescriptionStatus.error, which caused
update_failed_daemon_health_check to fire CEPHADM_FAILED_DAEMON for
daemons that were simply mid-startup. The warning clears on the next
refresh once the daemon reaches a real state, but it causes spurious
QA failures across many test suites.
Map 'unknown' to DaemonDescriptionStatus.unknown instead — the enum
value already exists and is handled by to_str() and the dashboard.
get_error_daemons() only checks for .error, so unknown daemons no
longer trigger the health warning.
qa/suites/rbd/valgrind: pin to centos_9.stream instead of rpm_latest
This used to be the case before commit d4b977afdc59 ("qa/distros:
rename centos_latest.yaml to rpm_latest.yaml") and subsequent changes
to enable Rocky 10. When paired with valgrind, python_api_tests* jobs
fail persistently on Ubuntu and Rocky, see [1]. Switch back until that
is resolved (or for as long as centos_9.stream remains in the mix).
Matan Breizman [Mon, 8 Jun 2026 12:45:20 +0000 (15:45 +0300)]
crimson/osd: fix CyanStore boot with crimson OSD
- Remove dead cyan_store.h includes from pg.cc and shard_services.cc.
- Fix CyanStore::mkfs to write type="cyanstore" instead of "memstore"
- Handle missing device class in OSD::start: stores that return an empty
device class (e.g CyanStore) now fall back to is_rotational=false with
a warning instead of throwing
Before optimizing the onode lookup paths we should measure the existing
cost if each possible optimization.
The per-node search is already binary search(see search_result_bs_t
binary_search). Possible costs are:
- per-level node traversal
- STAGE_STRING ns/oid memcmp (rbd has long shared oid prefixes)
Adding the following stats to be exposed as seastar-metrics (not logs).
Keep the new sampling per-comparison overhead out unless needed for
measurment runs.
Candidate optimization leads from the numbers:
* cross-node string/prefix dedup (marked as TODOs in stage.h) this could
result in cheaper compares for shared (rbd) prefixes.
* shrink onode_layout_t - marked as onode.h TODO.
stop inlining oi and ss to get more objects per leaf.
Sun Yuechi [Mon, 6 Jul 2026 08:06:09 +0000 (01:06 -0700)]
test/subprocess: add missing space in SubshellKilled subcommand
9abf2388f24 rewrote "sh -c cat" as SHELL "-c cat", which concatenates
to "/bin/sh-c cat", so the subshell tries to execute a non-existent
"/bin/sh-c". The test still passed because the SIGTERM sent by kill()
usually wins the race against the shell startup; under CI load the
shell occasionally prints
/bin/sh: line 1: /bin/sh-c: No such file or directory
into the stderr pipe first, ASSERT_TRUE(buf.empty()) fails, and the
skipped join() turns this into a SIGABRT in ~SubProcess().
monitoring/ceph-mixin: scope test queries per dashboard
get_dashboards_data() keeps every query in a single map keyed by
"<panel title>-<legendFormat>", but that id is not unique: several
dashboards have a panel titled "IOPS", "OSDs" or "Throughput". The
dashboard read last overwrites the earlier entry, so one query shadows
another and never gets tested. glob() walks the files in filesystem
order, so which query survives, and whether the test passes, depends on
readdir.
run-tox-promql-query-test hit this in "Test IOPS Read"
(ceph-cluster.feature): the scenario feeds ceph_osd_op_r but the id
resolved to a CephFS pool panel.
pool_id is "UNSET VARIABLE" because the scenario does not set $mdatapool,
and no ceph_pool_rd series were fed, so the query returns nil. It only
fails when readdir returns cephfsdashboard after ceph-cluster-advanced,
so the earlier change that walked nested rows and made the winner
deterministic did not fix it; it fixed which query wins, not that the
wrong one can win.
Key the queries per dashboard (data['queries'][<dashboard>][<id>]) and
have each .feature name its dashboard in a Background step. The lookup is
confined to that dashboard, so a title/legend used elsewhere no longer
shadows it. A duplicate within one dashboard is still an error, unless it
is in a collapsed row.
ceph-cluster.feature covers ceph-cluster-advanced; the rest map to one
dashboard each.
Ronen Friedman [Tue, 30 Jun 2026 12:55:33 +0000 (12:55 +0000)]
crimson/os/alienstore: make CnLog::_flush non-blocking
CnLog::_flush() calls alien::submit_to(...).wait(), which blocks the
log flusher thread until the seastar reactor finishes outputting the
log entries. When the reactor writes to stderr and the pipe buffer is
full (e.g. two OSDs on the same host both dumping at debug level 20),
the reactor itself blocks on the write() syscall. Meanwhile BlueStore
threads filling the log buffer past m_max_new block in submit_entry()
waiting for the flusher - which is waiting for the reactor - which is
blocked on I/O. The kv thread cannot fire commit callbacks while it is
blocked on logging, so store transactions (including osdmap persistence)
hang indefinitely.
Fix by moving the entries into the lambda capture so _flush() can
return immediately without waiting.
This is a corrected version of the approach in commit 511af83e274
("crimson/os/alienstore/alien_log: _flush concurrently", PR #55262),
which was reverted in 4d52d1d22b7 because its lambda captured the
EntryVector by reference - the caller would clear/reuse the vector
before the lambda finished, causing a "pure virtual method called"
crash (https://tracker.ceph.com/issues/64140). The fix here avoids
that by swapping the entries out of the caller's vector and moving
them into the lambda's capture, so the lambda owns them for their
full lifetime.
Fixes: https://tracker.ceph.com/issues/77826
Fixes (failure of the 1st attempt): https://tracker.ceph.com/issues/64140 Assisted-by: Claude <claude.ai> Signed-off-by: Ronen Friedman <rfriedma@redhat.com>
mgr/cephadm: Add documentation for RGW keyring capability dependencies
This serves as a reminder for maintainers to verify whether capability
changes in the RGW service should also be applied to SMB and NFS
services to maintain consistency across services that use librgw.
mgr/dashboard: support clone_by_snap_id flag in clone API for group snapshots
The clone API endpoint POST /api/block/image/{image_spec}/snap/{snapshot_name}/clone
only accepted a snapshot name, which uses rbd_clone3 (clone by name). This only works
for snapshots in the user namespace.
Group snapshots (with .group prefix, in RBD_SNAP_NAMESPACE_TYPE_GROUP namespace)
require rbd_clone4 (clone by snap ID). The rbd Python binding already supports this:
passing an int for p_snapshot triggers rbd_clone4.
Add an optional clone_by_snap_id boolean flag to the clone endpoint. When set, the
path parameter is treated as a numeric snap ID and passed directly as an int to
rbd.RBD().clone(). Clone format 2 is enforced because this flag is intended for
non-user namespace snapshots which cannot be protected (a prerequisite for clone
format 1).
Usage:
POST /api/block/image/{image_spec}/snap/{snap_id}/clone
{"clone_by_snap_id": true, ...}
Fixes: https://tracker.ceph.com/issues/77875 Signed-off-by: Imran Imtiaz <imran.imtiaz@uk.ibm.com> Assisted-by: IBM Bob
mgr/smb: Allow pool creation for empty RGW buckets
When an RGW bucket is empty, SMB clients cannot create directories,
files, or upload data because the default.rgw.buckets.data pool does
not yet exist. The previous CephX capabilities ('mon allow r') did not
permit pool creation during the first write operation.
The fix changes the mon capability from 'allow r' to 'allow *' to
enable pool creation on the first write to an empty RGW bucket. Once
the pool exists, subsequent operations succeed normally.
Sun Yuechi [Thu, 2 Jul 2026 09:26:59 +0000 (02:26 -0700)]
common/crc32c: stop using gp/tp as scratch in RISC-V Zbc CRC32C
crc32c_zbc's fold-by-four loop used gp (x3) and tp (x4) as scratch for
two folding temporaries. Save/restore around the loop is not enough:
they are ABI-reserved, so a signal delivered mid-loop runs its handler
with a corrupted tp, and the first TLS access there faults.
On riscv64 this reliably crashes the crimson/seastore unittests
unittest-transaction-manager and unittest-omap-manager, where seastar's
stall-detector timer fires often: the process dies with SIGSEGV in the
linker's TLS path with garbage in gp/tp.
Fixes: https://tracker.ceph.com/issues/77904 Signed-off-by: Sun Yuechi <sunyuechi@iscas.ac.cn>
vstart.sh: only skip fs volume create when there's no mgr to run it
The mgr volumes module handles fs volume ls/create, not the monitor, so
without a mgr (CEPH_NUM_MGR=0) the "wait for volume module to load" loop
spun forever. Open-coding fs new unconditionally, my first attempt,
changed pool creation for every vstart user and dropped the wait, so
cephfs tests calling `fs volume ...` right after vstart.sh returns could
race a module still mid-load.
create_fs_volume() branches on CEPH_NUM_MGR instead. With a mgr it still
calls fs volume ls/create. Without one it runs the plain mon commands fs
volume create wraps: osd pool create cephfs.<name>.{meta,data}, then fs
new <name> <meta> <data>. The wait stays at its one call site above the
per-filesystem loop, so it runs once regardless of CEPH_NUM_FS, not once
per filesystem.
Verified against a local vstart: mgr present, fs volume ls polls once
then creates CEPH_NUM_FS=2 filesystems with no repeated poll;
CEPH_NUM_MGR=0, no poll, fs new creates 'a' directly, HEALTH_OK.
crimson/osd: skip store flush for empty peering transactions
dispatch_context_transaction submits a real empty transaction to
AlienStore even when the peering context has no store ops. This
goes through the AlienCollection lock and thread pool slot machinery.
Under load from broadcast_map_to_pgs (many PGAdvanceMap operations
running concurrently with recovery), some of these empty transactions
get stuck inside AlienStore's pipeline and never reach BlueStore,
causing PGAdvanceMap to hang at complete_rctx indefinitely.
Fix by skipping the store round-trip for empty transactions. The
on_commit callbacks are still collected and fired, preserving the
fix from b0f5e1086a6 ("crimson/osd/pg: also trigger callbacks for
empty peering transactions"). The store flush that commit added was
incidental to the callback fix and is unnecessary here: the peering
pipeline's exclusive process stage already guarantees that prior
peering transactions on the same PG have completed.
Assisted-by: Claude <claude.ai> Signed-off-by: Ronen Friedman <rfriedma@redhat.com>
Karthik U S [Wed, 10 Jun 2026 05:48:25 +0000 (11:18 +0530)]
mgr/mirroring,tools/cephfs_mirror: Handle checkpoint state transition
When a new checkpoint is being added or when the daemon gets restarted,
it will check whether the newly created checkpoint or any other old
checkpoints have already been mirrored onto the remote peer. If so, it
will transition to the correct state by checking for the highest snap id
present on the remote, and setting all the checkpoints which have snap id
lesser than or equal to that of the remote to COMPLETE. This is done by
sending an acquire notification from the mirroring module to the mirror
daemon, which is handled in the add_directory path, by adding the directory
to be checked for the state transition in the tick thread.
This path gets triggered:
a) when the daemon gets restarted
b) when the peer mapping changes
c) by sending the acquire notification from checkpoint add/now CLIs.
d) when mirroring module restarts
Fixes: https://tracker.ceph.com/issues/73454 Signed-off-by: Karthik U S <karthik.u.s1@ibm.com>
Karthik U S [Sat, 16 May 2026 00:50:03 +0000 (06:20 +0530)]
tools/cephfs_mirror: update checkpoint status during snapshot sync
When the sync completes or fails for a checkpointed snapshot, transition
the status of that checkpoint from CREATED/FAILED to COMPLETE/FAILED in
the do_sync_snaps() along with the timestamp of the event.
Fixes: https://tracker.ceph.com/issues/73454 Signed-off-by: Karthik U S <karthik.u.s1@ibm.com>
projects / ceph.git / log