Gil Bregman [Mon, 23 Feb 2026 10:56:54 +0000 (12:56 +0200)]
nvmeof: Change the NVMEOF image version to 1.7 Fixes: https://tracker.ceph.com/issues/75097 Signed-off-by: Gil Bregman <gbregman@il.ibm.com>
(cherry picked from commit 02587347b0a4e7ae1d7f5d738bd33808e2d56bc9)
Conflicts:
src/python-common/ceph/cephadm/images.py
The indentation was changed between the main branch and tentacle which
caused a conflict
Afreen Misbah [Wed, 27 May 2026 00:07:38 +0000 (05:37 +0530)]
mgr/dashboard: fix nested shell quoting in cephadm e2e start-cluster
with_libvirt wraps commands in sg libvirt -c "$1", adding an extra
shell layer. Nested double quotes inside the outer double-quoted
string caused the argument to be split — with_libvirt received a
truncated $1, producing "Unterminated quoted string" on the remote
shell.
Drop the unnecessary inner double quotes around cephadm shell
arguments since cephadm shell accepts the command as separate args.
Use single quotes for the grep pattern inside the double-quoted
string so it survives the sg subshell.
Nizamudeen A [Thu, 2 Apr 2026 10:55:19 +0000 (16:25 +0530)]
mgr/dashboard: run kcli commands in libvritd group
Also https://github.com/ceph/ceph-build/pull/2562/ Signed-off-by: Nizamudeen A <nia@redhat.com>
(cherry picked from commit 8293aa38bd9c9abca6353ebbdb64401bf3263017)
ceph-volume: skip internal raid mirror LVs in inventory
ceph-volume inventory started including all LVM mapper devices after c06bee965f1. On hosts with raid mirrored system volumes, that pulls in
hidden legs like var_rmeta_0 which have no /dev/vg/lv node and makes
cephadm's ceph-volume inventory call fail.
Skip those internal LVs in get_devices() and avoid rewriting the device
path to a missing lv_path in Device._parse().
ceph-volume: fix inventory without /dev/vg/lv (slashed paths)
Ths makes ceph-volume use UdevData.preferred_block_path() in
get_devices() so it keeps /dev/vg/lv (slashed path form) when
it exists, else /dev/mapper/<name> (dashed path form).
This is needed for thin-pool LVs and environments where udev
does not create slashed paths.
Kefu Chai [Sat, 13 Jun 2026 01:50:09 +0000 (09:50 +0800)]
python-common/cryptotools: stop using the removed X509Req API
pyOpenSSL deprecated OpenSSL.crypto.X509Req in 24.2.0 (2024-07-20) and
removed it in 26.3.0 (2026-06-12). as we don't pin pyopenssl, CI picked
up the new release, and create_self_signed_cert() started failing with:
AttributeError: module 'OpenSSL.crypto' has no attribute 'X509Req'
this took down run-tox-mgr, run-tox-mgr-dashboard-py3 and the mypy check.
we only used X509Req to build a subject name and then copied it into the
X509 cert. so drop it, and set the subject on the cert directly. the
resulting cert stays the same: subject from dname, issuer set to the same
subject, self-signed.
Kefu Chai [Tue, 23 Jun 2026 07:43:28 +0000 (15:43 +0800)]
mgr/dashboard: skip the table when an nvmeof cli result has no columns
The dashboard leaves prettytable unpinned. prettytable commit 2574492 ("Apply
some Pylint rules (PLR)", #436) rewrote _stringify_row()'s row_height as
`max(_get_size(c)[1] for c in row)`, which raises ValueError("max() iterable
argument is empty") on a row with no cells. The change is undocumented and
shipped in 3.18.0; get_string() trips on it when a table has a row but no
columns.
AnnotatedDataTextOutputFormatter builds such a table for an empty result, or
one whose only field is status or error_message, so NvmeofCLICommand.call()
returns -EINVAL and the command fails. This broke run-tox-mgr-dashboard-py3
once the tox virtualenv picked up prettytable 3.18.0.
Return an empty string when there are no columns instead of formatting a
degenerate table.
Conflicts:
src/pybind/mgr/dashboard/tests/test_nvmeof_cli.py
Resolution: tentacle has only the empty-result test case; updated its
expected stdout to ''
mgr/DaemonServer: Aggregate and globally sort OSDs for ok-to-upgrade
The 'ok-to-upgrade' command output sorting did not scale accurately
when target CRUSH buckets contained multiple child buckets (e.g., a
chassis containing multiple hosts). OSDs were previously sorted
individually per child bucket and appended sequentially. This created
fragmented, per-host sort segments rather than a globally sorted list
for the parent bucket.
Changes:
1. Fix the issue above by aggregating all child OSDs into a single vector prior
to executing a single, global sort operation based on PG counts. Additionally,
optimize memory efficiency and future-proof the logic by reserving continuous
vector blocks to avoid dynamic heap reallocations.
2. Add integration tests with chassis and rack based CRUSH hierarchies which
verifies the ok-to-upgrade functionality. In addition, the tests crucially
verify the order of OSDs returned is according to the ascending order of
acting PG count. Additionally, make minor fix-ups to lines that determine the
length of a list in JSON response by removing the redundant "| bc".
Alex Ainscow [Wed, 8 Apr 2026 10:49:58 +0000 (11:49 +0100)]
osd: Allow multiple objects with same version in missing list.
Most of the time, a single version in a PG can only correspond to a single object.
However, following a PG merge it is possible, even likely, that two objects will
have the same version. The PG Log works around this by discarding the log.
However, during backfill, it is possible for the missing list to be build with
these duplicate versions.
A recently added assert detected that this scenario was corrupting the reverse
missing list (rmissing). This behaviour has always existed, but was previously
unnoticed. It could cause some bugs and potentially loop-asserts on OSDs,
although mostly would not be noticed.
Here we fix this properly, by converting rmissing to a multimap. This is wrapped
in some insert functions, which assert that the rmissing list does not end up
with duplicate entries. The code is optimised for the case where there are no
duplicate versions.
Additionally, some of the old asserts have been rolled into the insert functions.
Fixes: https://tracker.ceph.com/issues/75778 Signed-off-by: Alex Ainscow <aainscow@uk.ibm.com>
(cherry picked from commit f3940400952b444a31f59b633fa3fa35437c87a9)
mgr: guard close_section calls in get_perf_schema_python
When a daemon exists in the daemon state map but has no perf counters
(e.g. an OSD that was running but is now down or destroyed), the loop
in get_perf_schema_python never executes, leaving prev_key_name empty
and no formatter sections opened. The unconditional close_section()
calls after the loop then trigger an assertion failure in PyFormatter
(cursor != root).
Add the same `if (!prev_key_name.empty())` guard that already protects
the close_section() calls inside the loop, so we only close sections
that were actually opened.
Nitzan Mordechai [Sun, 10 May 2026 06:51:08 +0000 (06:51 +0000)]
qa: ignore evicted client warnings for singletone bluestore
After adding mds client into singletone bluestore and increas debug
levels and bluestore_allocator=stupid client is slow enough that MDS
evicts it after ~302 seconds
mgr/DaemonServer: clarify ok-to-upgrade error message for CRUSH buckets
Refine the error string in DaemonServer.cc returned by the
ok-to-upgrade command when OSDs in a CRUSH bucket cannot be upgraded.
The original message is ambiguous. It fails to clearly convey that
stopping *any* individual OSD in that specific bucket will drop PGs
offline, meaning no OSDs within that bucket can be safely upgraded at
this time.
Update the phrasing to explicitly state that at least X PGs will go offline
if any OSD out of the total count in that CRUSH bucket is stopped. Also
standardize on capitalized acronyms (PG, OSD, CRUSH) and wrap the bucket
name in single quotes for better log readability.
ceph-volume: has_bluestore_label checks all bluestore label replica offsets
BlueStore replicates the block device label at fixed offsets (0 and
multiples of 1Gb up to 1000gb). has_bluestore_label() only read the
first 22 bytes, so disks with a wiped primary label but intact
replicas are missed.
with this commit, has_bluestore_label() scans each known offset with
seek/read and compares the ASCII prefix as bytes.
The thrashosds task is occasionally restarting OSDs and mon/mgr log
warnings are wrongly flagging this as a problem.
Fixes: https://tracker.ceph.com/issues/76747 Signed-off-by: Patrick Donnelly <pdonnell@ibm.com>
(cherry picked from commit a1430973f2c3e8758890447a1de3b906da77b60e)
Conflicts:
qa/suites/upgrade/reef-x/stress-split/1-start.yaml
qa/suites/upgrade/squid-x/stress-split/1-start.yaml
Resolution: different upgrade suites, just add the change manually
libcephsqlite: ensure atexit handlers are registered after openssl
When the sqlite3 executable encounters an error with .bail=on, it will
make a call to exit(). The atexit() handlers will execute in LIFO order.
We need to ensure that openssl (before OpenSSL 4.0 [1]) atexit handlers are
registered before libcephsqlite.
[1] http://github.com/openssl/openssl/commit/31659fe32673a6bd66abf3f8a7d803e81c6ffeed (OpenSSL 4.0 no longer arms `OPENSSL_cleanup()` function as an `atexit(3)`)
Fixes: https://tracker.ceph.com/issues/59335 Signed-off-by: Patrick Donnelly <pdonnell@ibm.com>
(cherry picked from commit 7949cd5f12eb7cc0dc85fd1b5c1d795fad1df922)
Redouane Kachach [Fri, 29 May 2026 09:09:44 +0000 (11:09 +0200)]
qa/tasks: capture CommandCrashedError when running nvme list cmd
The safe_while retry loop does not catch exceptions, so a
CommandCrashedError from `nvme list` bypasses it entirely. Catch
CommandCrashedError and continue the retry loop instead.
ceph-volume: retry lvs after empty result and "devices file is missing" stderr
When LVM's devices file is out of sync with the runtime device view (common
in teuthology/container namespaces with multipath), `lvs` can exit 0 with
empty stdout and only stderr warnings about missing mapper entries.
It can leave get_lvs() empty and cause Device() to fall through to lsblk on a
vg/lv path which can produce a misleading "not a block device" error.
With this fix, ceph-volume retries once with 'use_devicesfile=0' when it
detects this specific pattern.
Bill Scales [Fri, 15 May 2026 14:39:25 +0000 (15:39 +0100)]
osd: Fix bug when calculating min_peer_features
PeeringState calculates the minimum set of features for the set
of OSDs within a PG. There is a bug when the peer info has
already been cached where these peers features are not included
in the calculation. This can lead to the min feature set
including features that not all OSDs have.
Previously this just made some asserts less aggressive than they
should have been. Pull request https://github.com/ceph/ceph/pull/57740
uses min_peer_features to decide how to encode messages to other OSDs.
Midway through an upgrade this bug can cause an OSD to send
the wrong version of a message to a downlevel OSD causing
it to abort.
Fixes: https://tracker.ceph.com/issues/76600 Signed-off-by: Bill Scales <bill_scales@uk.ibm.com>
(cherry picked from commit ce5882778db9b92b14a6bb5eca22e3cecf2be9dc)
Casey Bodley [Wed, 25 Mar 2026 16:33:40 +0000 (12:33 -0400)]
qa/rgw: remove ragweed from verify subsuite
it's currently broken with newer python on rocky 10 and ubuntu 24
(tracked in https://tracker.ceph.com/issues/72500) and doesn't provide
interesting test coverage outside of rgw/upgrade
ceph-volume: OSD mapper lifecycle (LVM + raw) for activate
This adds small helpers so activate can consistently bring the OSD device
stack online (LVM lvchange, optional mapper open) and tear it down again,
with refresh in between. Same idea for the raw path. Crypto is handled
inside that flow when the OSD is encrypted.
ceph-volume: raw activate should ignore lvm backed OSD devices
the generic activate (`ceph-volume activate`) runs the
raw path before LVM. Raw.activate was walking lsblk / raw
list entries and could hit block devices that are actually
logical volumes from `ceph-volume lvm prepare` or `lvm batch`
(with ceph lvm tags on the lv).
That made raw activation poke at LVM backed OSDs instead of
leaving it to `lvm activate`.
with this commit ceph-volume now builds the set of LV paths
that carry those tags once (`lvs` via ceph_volume_lvm_prepare_lv_paths)
and skip any candidate path that matches, so only real raw
OSDs go through the 'raw activate path'.
Also, we now pass `with_tpm` through luks_open() calls for db and
wal so encrypted metadata uses the same systemd-cryptsetup path
as the block LV when ceph.with_tpm is set.
mon/OSDMonitor: remove unused crush rules after erasure code pools deleted
When erasure code pools are created, a corresponding Crush rule is concurrently added to the Crush map.
However, when these pools are subsequently deleted, the associated rule persists within the Crush map.
In the event that a pool is re-created with the same name, the rule already exists.
However, if any modifications were made to the rule prior to the pool's deletion,
the new pool will inherit these modifications.
The proposed solution involves the automatic deletion of the Crush rule when a pool is deleted,
but only if no other pools are utilizing that particular rule.
Casey Bodley [Tue, 12 May 2026 18:58:16 +0000 (14:58 -0400)]
librados/asio: clear cancellation slot in associated executor
the librados callback function `AsyncOp::aio_dispatch()` runs on
Objecter's finisher strand executor, and dispatches the completion
handler to its associated executor
asio cancellation is not thread-safe, so should be synchronized on that
associated executor. move the call to `slot.clear()` from that librados
callback into the AsyncHandler wrapper so it doesn't run until we've
switched to the correct executor
because our `op_cancellation` handler depends on an `AioCompletion`
pointer, we have to clear the cancellation slot before that
`AioCompletion` lifetime ends
tentacle: tentacle-p2p add centos9 to stress-split
Adds CentOS 9 Stream as a supported host OS for the tentacle-p2p
stress-split suite.
Rocky10 is not added here. This suite tests a bare-metal p2p upgrade
between Tentacle point releases. Rocky10 package support was added during
that release cycle, meaning there is no valid "FROM" package baseline to
install on Rocky10, making a bare-metal point-to-point upgrade path on Rocky10
impossible to test.
tentacle: upgrade/tentacle-p2p install pytest for rbd-python tests
The rbd-python workload runs test_librbd_python.sh, which invokes
'python3 -m pytest' to run the librbd Python API tests.
On a bare-metal install, python3-pytest is not pulled in by the
Ceph packages and is not present and the workunit fails.
Add it via extra_system_packages so the tests can run.
tentacle: suites/upgrade add centos to centos image upgrade
Previously, each suite had a single upgrade-sequence.yaml that targeted
only one image.
This commit splits each upgrade sequence into two variants teuthology
picks one per run
This applies to:
- reef-x/parallel
- reef-x/stress-split
- squid-x/parallel
- squid-x/stress-split
- telemetry/reef-x
- telemetry/squid-x
For the stress-split suites, the upgrade logic is split into a
first-half-sequence run concurrently with thrashosds,
and a second-half-sequence run after.
Both sequences contain the hardcoded target image, so each variant
needs its own copy. They were previously inlined in 1-start.yaml.
This commit extracts them into the upgrade-sequence$/ files so
each variant can target the right image.
mgr/DaemonServer: auto-tune stats period when message queue gets backed up
The mgr can get overwhelmed when there's a lot of cluster activity and
daemons are sending stats reports faster than we can process them.
This commit adds logic to monitor the messenger queue depth and bump
up mgr_stats_period when things get congested. This reduces the
frequency of daemon stat reports, allowing the mgr to process existing
reports without being overwhelmed by new ones. The period automatically
scales back down when the queue clears up.
Added mgr_stats_period_autotune (on by default) and a queue threshold
setting. Recovery happens automatically when the queue clears up.
Max period is capped at 60 seconds to prevent excessive stat delays.
tentacle has different release note entries than main; kept
tentacle's existing entries and added only the MGR stats period
auto-tuning note from this commit
Patrick Donnelly [Tue, 19 May 2026 23:15:29 +0000 (19:15 -0400)]
Merge PR #68803 into tentacle
* refs/pull/68803/head:
rgw/multisite: concurrency adjustment - consider the case caller provides 1
rgw/multisite: log concurrency state transitions in adj_concurrency
rgw/multisite: fix uninitialized LatencyMonitor average and use exponentially weighted moving average
rgw/multisite: expose lock latency as perf counter for data sync
Patrick Donnelly [Fri, 15 May 2026 17:27:08 +0000 (13:27 -0400)]
Merge PR #68776 into tentacle
* refs/pull/68776/head:
tentacle: test/neorados: Narrow Asio includes in `leak_watch_notify`
test/neorados: Don't leak watch handle
neorados: Avoid double cleanup in watch/notify
neorados: Actually enforce notification queue limit
neorados: Do not try to decode an empty response in notify
neorados: Go through linger_cancel on `io_context` shutdown
common/async: Fix removal from service list
osdc: remove implicit LingerOp reference between watch/unwatch
osdc: linger_register() returns intrusive_ptr<LingerOp>
neorados: NotifierHandler holds intrusive_ptr<LingerOp>
neorados: Notifier holds intrusive_ptr<LingerOp>
librados: aio_unwatch() delivers ENOTCONN to AioCompletion
osdc: Objecter::linger_by_cookie() for safe cast from uint64
librados: linger callbacks hold a reference to LingerOp
projects / ceph.git / log