Jon Bailey [Thu, 4 Jun 2026 10:27:07 +0000 (11:27 +0100)]
test: Remove invalid unit test
This test was talking about testing invalid ops, however with the inclusion of sync reads in EC (https://github.com/ceph/ceph/pull/67079), it is valid to perform class reads in EC. In addition, work was done around illegal ops here: https://github.com/ceph/ceph/pull/66258 and the existance of TEST(ClsHello, BadMethods) in test_cls_hello.cc covers illegal ops in that PR leading me to think this is unneccisairy. Because of these reasons, I think its better this test is removed as it is incorrect and also not working.
Patrick Donnelly [Tue, 28 Jul 2026 13:04:01 +0000 (09:04 -0400)]
Merge PR #69651 into umbrella
* refs/pull/69651/head:
test/rgw: init CephContext in unittest_rgw_cors
rgw: include str_list.h for ceph::for_each_substr in CORS header
rgw: address CORS rule-matching review feedback
fix: some issue based on CORS spec 6.2.4
reject null or empty method in matches_method
rgw: match CORS rules like Amazon S3
Reviewed-by: Adam C. Emerson <aemerson@redhat.com>
Patrick Donnelly [Mon, 27 Jul 2026 14:21:10 +0000 (10:21 -0400)]
Merge PR #69765 into umbrella
* refs/pull/69765/head:
rbd-mirror: Remove old non-primary demoted image snapshots on the local cluster
rbd-mirror: prune obsolete primary mirror snapshots after relocation
rbd-mirror: fix missing initialization of Peer UUID
Patrick Donnelly [Mon, 27 Jul 2026 14:01:13 +0000 (10:01 -0400)]
Merge PR #70287 into umbrella
* refs/pull/70287/head:
doc/rbd: elaborate on key-ref syntax (existing in S3Stream and new in NativeFormat)
qa/suites/rbd: add mon_host + key[-ref] coverage to migration-external
qa/suites/rbd: use client.0 entity in migration-external tests
librbd/migration/NativeFormat: support specifying mon_host and key via spec
Patrick Donnelly [Mon, 27 Jul 2026 13:51:53 +0000 (09:51 -0400)]
Merge PR #70176 into umbrella
* refs/pull/70176/head:
mgr/dashboard: fixing manual deployment option for osd creation flow
mgr/dashboard: fix cancel button
mgr/dashboard: Converting Create OSDs tearsheet type from Full to Wide
mgr/dashboard: carbonized OSD form component
qa/tests: drop parens from all health-code ignorelist entries
Monitor.cc logs a "Health check cleared: <CODE> (was: ...)" line
for any health check clearing mid-test, with the code bare (no
parens) -- while the "Health check failed"/"Health check failed
(unmute)" lines wrap the code in parens. Every \(CODE\) entry in
this file therefore only matches the raise/failed form and misses
the corresponding cleared form, the same gap just fixed for
POOL_FULL.
Drop the escaped parens from the remaining entries so each matches
both log forms, consistent with the bare entries already present
(OSD_ROOT_DOWN, MDS_INSUFFICIENT_STANDBY, POOL_FULL).
qa/tests: fix POOL_FULL ignorelist pattern in upgrade tests
\(POOL_FULL\) only matches the "Health check failed/update" cluster-log
format (code wrapped in parens). It misses the "Health check cleared"
format, where the code is not parenthesized (see src/mon/Monitor.cc),
so a pool clearing its full state mid-upgrade-test was not ignorelisted.
Drop the escaped parens so the pattern matches both forms, consistent
with other bare entries already in this file (OSD_ROOT_DOWN,
MDS_INSUFFICIENT_STANDBY).
Lumir Sliva [Mon, 22 Jun 2026 15:20:24 +0000 (17:20 +0200)]
rgw/lc: report x-amz-expiration only for the current version
The x-amz-expiration response header describes the current-version
Expiration lifecycle action for an object. s3_expiration_header() instead
selected between the Expiration and the NoncurrentVersionExpiration rule
based on whether the object key carried an instance id:
This was wrong in two ways. OLH resolution writes the current version's
instance id into the key before the header is computed, so even a plain
GET/HEAD of the current version (no versionId) was treated as a
non-current request and reported a NoncurrentVersionExpiration rule. And
x-amz-expiration is not meant to surface NoncurrentVersionExpiration at
all. A versioned bucket whose only rule was NoncurrentVersionExpiration
therefore returned a bogus expiry-date and rule-id on the current object,
for both a plain request and a versionId request.
Compute the header from the current-version Expiration rule only, and
return no header when the request targets a specific version. The header
helper now passes the client request key (s->object_key) rather than the
OLH-resolved key, so a plain GET/HEAD of the current version is no longer
misclassified as a versioned request.
Add a unit test covering a noncurrent-only policy (no header for either a
plain or a versioned request) and a combined policy (the current
Expiration rule for the current version, no header for a versionId
request).
Patrick Donnelly [Mon, 20 Jul 2026 18:28:42 +0000 (14:28 -0400)]
Merge PR #69553 into umbrella
* refs/pull/69553/head:
doc: add PendingReleaseNotes entry for rgw multisite DNS endpoint resolution
rgw/rest: add TODO for concurrent endpoint DNS resolution
rgw/multisite: fix endpoint unreachable detection in RGWRESTConn sync paths
rgw: store RGWEndpoint URL as boost::urls::url
doc/radosgw: expose rgw_rest_conn_connect_to_resolved_ips and rgw_rest_conn_ip_fail_timeout_secs
rgw: rename round-robin counters for brevity
rgw/zone: increase visibility into zone connections via admin socket
rgw: make CONN_STATUS_EXPIRE_SECS a cfg option
rgw/rest: track connection failures per-IP instead of per-endpoint
rgw/rest: remove unused headers
rgw/rest: consolidate endpoint_urls and resolved_endpoints into single vector
rgw: add operator<< for RGWEndpoint and simplify logging
rgw: track original URL within RGWEndpoint instead of separate member (refactor)
rgw: fix incomplete RGWRESTConn move constructor/assignment
rgw/rest: consolidate endpoint status tracking into ResolvedEndpoint
rgw/http: apply RGWEndpoint connect_to via libcurl CURLOPT_CONNECT_TO
rgw/rest: round-robin resolved endpoint IPs into curl CONNECT_TO mapping
rgw/rest: resolve multisite endpoints to all A/AAAA records (optional)
rgw: add rgw_resolve_endpoints_into_all_addresses config option
rgw/http: introduce RGWEndpoint to carry url + connect_to (refactor)
Reviewed-by: Adam C. Emerson <aemerson@redhat.com>
Patrick Donnelly [Mon, 20 Jul 2026 18:27:59 +0000 (14:27 -0400)]
Merge PR #69741 into umbrella
* refs/pull/69741/head:
tests/neorados: fix ceph_test_neorados_completions being not installed
common/async: async_cond::notify/cancel must post to handler's associated executor
Reviewed-by: Adam C. Emerson <aemerson@redhat.com>
Patrick Donnelly [Mon, 20 Jul 2026 18:26:49 +0000 (14:26 -0400)]
Merge PR #69723 into umbrella
* refs/pull/69723/head:
mgr/dashboard: Add "gw refresh_network" cmd
mgr/dashboard: bump nvmeof submodule to 1.8.2
mgr/dashboard: align nvmeof cli with missing parameters and functions from the old nvmeof cli
monitoring: fix NVMeoFMultipleNamespacesOfRBDImage
Patrick Donnelly [Mon, 20 Jul 2026 18:11:35 +0000 (14:11 -0400)]
Merge PR #70267 into umbrella
* refs/pull/70267/head:
21.1.0
debian/rules: Also exclude librgw and radosgw from dwz
debian/rules: exclude ceph-osd-crimson from dwz compression
script/buildcontainer-setup: fix package install on debian trixie
Leonid Chernin [Tue, 10 Feb 2026 06:50:51 +0000 (08:50 +0200)]
nvmeofgw: introduce nvme-gw show-all
Introduce a new read-only command, nvme-gw show-all, which
iterates over all configured pools and groups and runs
nvme-gw show for each, simplifying gateway inspection.
fixes https://tracker.ceph.com/issues/74854 Signed-off-by: Leonid Chernin <leonidc@il.ibm.com>
(cherry picked from commit ca090f3a35909d1a7c238f531445b5f5c9650109)
David Galloway [Fri, 17 Jul 2026 18:34:44 +0000 (14:34 -0400)]
container: default FROM_IMAGE to Rocky Linux 10
Since tentacle, the preferred base image for ceph containers has been
Rocky Linux 10, and the CI tag-naming logic in build.sh already assumes
rockylinux-10 is the default fromtag for every branch except reef and
squid. The actual build default was never flipped, though: anything
that ran build.sh without FROM_IMAGE set (e.g. the release container
job in ceph-build) still got a CentOS Stream 9 base.
Flip the Containerfile ARG and the build.sh fallback to
docker.io/rockylinux/rockylinux:10 so umbrella and later build from
Rocky 10 by default. Builds that want a different base can still pass
FROM_IMAGE explicitly, as the CI pipeline does.
Ilya Dryomov [Mon, 29 Jun 2026 11:39:50 +0000 (13:39 +0200)]
qa/suites/rbd: use client.0 entity in migration-external tests
Currently client.admin is passed for client_name and that doesn't
exercise client_name handling much as client.admin is the default.
When deploying multiple clusters the ceph task distributes keyrings
with only the key for the initial monitor and client.admin key. Other
keys (e.g. client.0) are present only on their respective clusters, so
client.0's key for cluster2 needs to be obtained on cluster1 explicitly
with "ceph auth get".
Leonid Chernin [Mon, 2 Feb 2026 06:01:14 +0000 (08:01 +0200)]
librbd/migration/NativeFormat: support specifying mon_host and key via spec
migration:
-take secret_key from spec
-take mon_host from the spec
-ignore source keyring, source ceph.conf - bypass them
-added validations of the new way schema
-get key from the KV DB if key in spec is actually key-ref
Fixes: https://tracker.ceph.com/issues/68177 Signed-off-by: Leonid Chernin <leonidc@il.ibm.com> Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
(cherry picked from commit 705d865ae77f4d3e80a9d3da3281ba6a824d7f29)
Kefu Chai [Fri, 23 Jan 2026 07:04:47 +0000 (15:04 +0800)]
debian/rules: exclude ceph-osd-crimson from dwz compression
When building with DWZ enabled, the debian packaging fails with:
```
dh_dwz: error: Aborting due to earlier error
```
Running the dwz command manually reveals the root cause:
```
$ dwz -mdebian/ceph-osd-crimson/usr/lib/debug/.dwz/x86_64-linux-gnu/ceph-osd-crimson.debug \
-M/usr/lib/debug/.dwz/x86_64-linux-gnu/ceph-osd-crimson.debug -- \
debian/ceph-osd-crimson/usr/bin/ceph-osd-crimson \
debian/ceph-osd-crimson/usr/bin/crimson-store-nbd
dwz: debian/ceph-osd-crimson/usr/bin/ceph-osd-crimson: Too many DIEs, not optimizing
dwz: Too few files for multifile optimization
```
The dwz tool has a limit on the number of DWARF DIEs (Debug Information
Entries) it can process. The ceph-osd-crimson binary, being a large C++
executable with extensive template usage, exceeds this limit, causing
dwz to exit with status 1 and fail the build.
This change excludes only ceph-osd-crimson from dwz processing using
the -X flag, allowing other binaries in the package to still benefit
from DWARF compression while avoiding the build failure.
Please note, we always disable DWZ in ceph-build's ceph-dev-pipeline.
Kefu Chai [Fri, 23 Jan 2026 07:04:47 +0000 (15:04 +0800)]
debian/rules: exclude ceph-osd-crimson from dwz compression
When building with DWZ enabled, the debian packaging fails with:
```
dh_dwz: error: Aborting due to earlier error
```
Running the dwz command manually reveals the root cause:
```
$ dwz -mdebian/ceph-osd-crimson/usr/lib/debug/.dwz/x86_64-linux-gnu/ceph-osd-crimson.debug \
-M/usr/lib/debug/.dwz/x86_64-linux-gnu/ceph-osd-crimson.debug -- \
debian/ceph-osd-crimson/usr/bin/ceph-osd-crimson \
debian/ceph-osd-crimson/usr/bin/crimson-store-nbd
dwz: debian/ceph-osd-crimson/usr/bin/ceph-osd-crimson: Too many DIEs, not optimizing
dwz: Too few files for multifile optimization
```
The dwz tool has a limit on the number of DWARF DIEs (Debug Information
Entries) it can process. The ceph-osd-crimson binary, being a large C++
executable with extensive template usage, exceeds this limit, causing
dwz to exit with status 1 and fail the build.
This change excludes only ceph-osd-crimson from dwz processing using
the -X flag, allowing other binaries in the package to still benefit
from DWARF compression while avoiding the build failure.
Please note, we always disable DWZ in ceph-build's ceph-dev-pipeline.
pybind/rbd: handle non-existent groups properly in list_snaps()
Calling `list_snaps()` on a non-existent group crashed the entire
Python process with a `free(): invalid pointer` error. This happened
because an unexpected `-ENOENT` error returned by `rbd_group_snap_list2`
and the exception was raised without updating the num_group_snaps to 0
from 10. Later the Cython `__dealloc__` wrapper attempted to free garbage
of uninitialized pointer spaces resulting in a crash.
Fix this behavior by resetting `self.num_group_snaps` to 0 inside
`GroupSnapIterator` before raising exception to prevent memory
corruption and accessing uninitialized pointers during cleanup call
Now `list_snaps()` consistently raise an `ObjectNotFound` error when
invoked on a non-existent group.
Also Extended the `TestGroups` test fixture with `self.dne_group` to
validate the expected error path for `list_snaps()`.
Fixes: https://tracker.ceph.com/issues/78034 Signed-off-by: VinayBhaskar-V <vvarada@redhat.com>
(cherry picked from commit 56bcf706cf91f0734899ae9de378c459a9b5af50)
David Galloway [Fri, 10 Jul 2026 20:12:01 +0000 (16:12 -0400)]
script/buildcontainer-setup: fix package install on debian trixie
Debian removed software-properties-common from the archive in trixie,
so the flat apt-get install list fails there. The package was only
needed to provide add-apt-repository for llvm.sh, which installs
clang-19 from apt.llvm.org. Trixie ships clang-19 natively, so install
it from the distro instead; run-make.sh's prepare() then finds clang-19
and skips llvm.sh entirely. Ubuntu and older Debian releases still have
software-properties-common and keep the previous behavior.
mgr/dashboard: Converting Create OSDs tearsheet type from Full to Wide Fixes: https://tracker.ceph.com/issues/77132 Signed-off-by: Devika Babrekar <devika.babrekar@ibm.com>
(cherry picked from commit 4eb4e25e5e6f14ed70fee5c41b2669cc0c557e19)
Fixes: https://tracker.ceph.com/issues/68265 Signed-off-by: Syed Ali Ul Hasan <syedaliulhasan19@gmail.com>
(cherry picked from commit 78dad340ab180532082c39a52efc0dfce244ec6e)
Nizamudeen A [Wed, 8 Jul 2026 09:27:33 +0000 (14:57 +0530)]
mgr/dashboard: fix unncessary traceback when bucket not exist
UI has an async validator which calls the GET bucket API to make sure
the bucket name doesn't exist, but the proxy
was not properly handling the http_status_codes which results in raising
a massive traceback in logs whenever you type things in the bucket name
field. So handling that gracefully by capturing the proper status codes
for both RequestException and DashboardException
BEFORE
```
File "/usr/share/ceph/mgr/dashboard/services/exception.py", line 47, in dashboard_exception_handler
return handler(*args, **kwargs)
File "/lib/python3.9/site-packages/cherrypy/_cpdispatch.py", line 54, in _call_
return self.callable(*self.args, **self.kwargs)
File "/usr/share/ceph/mgr/dashboard/controllers/_base_controller.py", line 263, in inner
ret = func(*args, **kwargs)
File "/usr/share/ceph/mgr/dashboard/controllers/_rest_controller.py", line 193, in wrapper
return func(*vpath, **params)
File "/usr/share/ceph/mgr/dashboard/controllers/rgw.py", line 357, in get
result = self.proxy(daemon_name, 'GET', 'bucket', {'bucket': bucket})
File "/usr/share/ceph/mgr/dashboard/controllers/rgw.py", line 213, in proxy
raise DashboardException(e, http_status_code=http_status_code, component='rgw')
dashboard.exceptions.DashboardException: RGW REST API failed request with status code 404
(b'{"Code":"NoSuchBucket","Message":"","RequestId":"tx00000f14e08c1af0d5615-006'
b'71f54a7-3bc6-default","HostId":"3bc6-default-default"}')
2024-10-28T09:08:55.990+0000 7f89e045b640 0 [dashboard INFO request] [::ffff:10.74.18.122:51853] [GET] [500] [0.007s] [admin] [200.0B] /api/rgw/bucket/bucket-das
```
AFTER
```
Jul 08 09:28:28 ceph-node-00 ceph-mgr[2243]: [dashboard ERROR dashboard.rest_client] RGW REST API failed GET req status: 404
Jul 08 09:28:28 ceph-node-00 ceph-mgr[2243]: [dashboard INFO dashboard.services.exception] Dashboard Exception: RGW REST API failed request with status code 404
(b'{"Code":"NoSuchBucket","Message":"","RequestId":"tx00000c029a81e5d154844-006'
b'a4e183c-14251-default","HostId":"14251-default-default"}')
Jul 08 09:28:28 ceph-node-00 ceph-mgr[2243]: [dashboard INFO dashboard.tools] [::ffff:192.168.100.1:60408] [GET] [404] [0.078s] [admin] [200.0B] /api/rgw/bucket/testsa
```
qa/suites/rbd/valgrind: pin to centos_9.stream instead of rpm_latest
This used to be the case before commit d4b977afdc59 ("qa/distros:
rename centos_latest.yaml to rpm_latest.yaml") and subsequent changes
to enable Rocky 10. When paired with valgrind, python_api_tests* jobs
fail persistently on Ubuntu and Rocky, see [1]. Switch back until that
is resolved (or for as long as centos_9.stream remains in the mix).
David Galloway [Fri, 10 Jul 2026 20:12:01 +0000 (16:12 -0400)]
script/buildcontainer-setup: fix package install on debian trixie
Debian removed software-properties-common from the archive in trixie,
so the flat apt-get install list fails there. The package was only
needed to provide add-apt-repository for llvm.sh, which installs
clang-19 from apt.llvm.org. Trixie ships clang-19 natively, so install
it from the distro instead; run-make.sh's prepare() then finds clang-19
and skips llvm.sh entirely. Ubuntu and older Debian releases still have
software-properties-common and keep the previous behavior.
Fixes: https://tracker.ceph.com/issues/78111 Signed-off-by: David Galloway <david.galloway@ibm.com>
(cherry picked from commit b5697987ae86f51d7f16a1409ce9f2b542f3de52)
Super User [Wed, 8 Jul 2026 12:05:16 +0000 (17:35 +0530)]
doc/rbd: clarify mirror resync snapshot behavior
Add documentation explaining that rbd mirror resync only copies
image state and data up to the last mirror-snapshot for
snapshot-based mirroring. Document the requirement to create
a new mirror-snapshot on current primary so that new changes
(including resize operation) gets reflected on secondary after
resync.
Nizamudeen A [Wed, 8 Jul 2026 05:21:50 +0000 (10:51 +0530)]
mgr/dashboard: teardown http requests for feature-toggle
angular 19.2.latest is aggressively tearing down the test env which
forces the active rxjs timer to emit one execution as it collapses which
produces a ghost http in the case of request wrapped in the timer
service. so for now flushing all the requests manually to prevent it
Karthik U S [Sat, 16 May 2026 03:15:50 +0000 (08:45 +0530)]
qa/test_mirroring: Add tests for mirroring checkpoints
Adding integration tests for validating the cephfs mirroring
checkpoints feature
Fixes: https://tracker.ceph.com/issues/73454 Signed-off-by: Karthik U S <karthik.u.s1@ibm.com>
(cherry picked from commit fe7fa7b0761720181cf8735ac7b5f17080c1d8a7)
Karthik U S [Thu, 4 Jun 2026 11:47:05 +0000 (17:17 +0530)]
doc: Add docs and pending release notes for mirroring checkpoints
Adding documentations and pending release notes for the mirroring
checkpoints feature.
Fixes: https://tracker.ceph.com/issues/73454 Signed-off-by: Karthik U S <karthik.u.s1@ibm.com>
(cherry picked from commit 12bd34d69ac7e136d14d25ea274e99727b559870)
Karthik U S [Wed, 10 Jun 2026 05:48:25 +0000 (11:18 +0530)]
mgr/mirroring,tools/cephfs_mirror: Handle checkpoint state transition
When a new checkpoint is being added or when the daemon gets restarted,
it will check whether the newly created checkpoint or any other old
checkpoints have already been mirrored onto the remote peer. If so, it
will transition to the correct state by checking for the highest snap id
present on the remote, and setting all the checkpoints which have snap id
lesser than or equal to that of the remote to COMPLETE. This is done by
sending an acquire notification from the mirroring module to the mirror
daemon, which is handled in the add_directory path, by adding the directory
to be checked for the state transition in the tick thread.
This path gets triggered:
a) when the daemon gets restarted
b) when the peer mapping changes
c) by sending the acquire notification from checkpoint add/now CLIs.
d) when mirroring module restarts
Fixes: https://tracker.ceph.com/issues/73454 Signed-off-by: Karthik U S <karthik.u.s1@ibm.com>
(cherry picked from commit c64fe9636514d9e9b1c286a0b21b84569e266067)
Karthik U S [Sat, 16 May 2026 00:50:03 +0000 (06:20 +0530)]
tools/cephfs_mirror: update checkpoint status during snapshot sync
When the sync completes or fails for a checkpointed snapshot, transition
the status of that checkpoint from CREATED/FAILED to COMPLETE/FAILED in
the do_sync_snaps() along with the timestamp of the event.
Fixes: https://tracker.ceph.com/issues/73454 Signed-off-by: Karthik U S <karthik.u.s1@ibm.com>
(cherry picked from commit 37163436cc87a2be691202bb565c7b5007df026f)
Karthik U S [Tue, 12 May 2026 23:21:53 +0000 (04:51 +0530)]
tools/cephfs_mirror: Helper functions for mirroring checkpoints
Implementation of helper functions and data structures for the
snapshot based mirroring checkpoints feature.
Fixes: https://tracker.ceph.com/issues/73454 Signed-off-by: Karthik U S <karthik.u.s1@ibm.com>
(cherry picked from commit 0a394ef438abd0248d078ea72e8ae5ce4f3d5e84)
Fixes: https://tracker.ceph.com/issues/73454 Signed-off-by: Karthik U S <karthik.u.s1@ibm.com>
(cherry picked from commit 959bc2eb07cacf70a4da09170e1c9bbed4b9423f)
Kotresh HR [Tue, 23 Jun 2026 15:13:18 +0000 (20:43 +0530)]
qa: Add mgr snapshot mirror status tests
Add teuthology coverage for `ceph fs snapshot mirror status`:
parity with asok peer_status, default idle metrics, stale omap
handling, error paths, filter scopes, daemon restart, and cache TTL.
Reuse existing peer_dir_status and metrics assertion helpers.
Adjust stale test wait for InstanceWatcher timeout and set a
short cache TTL in the cache test. Pass peer_uuid via --peer_uuid=
in the test helper for peer-only scope queries.
Kotresh HR [Sun, 21 Jun 2026 17:59:43 +0000 (23:29 +0530)]
doc/cephfs: Document fs snapshot mirror status mgr command
Describe the mirroring module command that reads persisted omap
metrics, including syntax, output layout, stale detection, caching,
and comparison with the admin socket peer status interface.
Document --peer_uuid as a named argument for peer-only and
directory/peer filtering.
Kotresh HR [Sun, 21 Jun 2026 17:52:29 +0000 (23:22 +0530)]
mgr/mirroring: make snapshot mirror metrics cache optional
Add snapshot_mirror_metrics_cache_enabled (default true). When disabled,
metrics_status reads omap directly and skips complete and partial caches.
When enabled, behavior is unchanged.
projects / ceph.git / log