Commit includes changes:
1) Renaming Topic to Notification destination
2) Renaming Tiering to Storage class
3) Renaming Users to User Management
4) fix storage class table refresh after delete
5) Also made changes to internal routing for topic and storage class
rgw/dedup: Grant dedup process full RGW permissions.
This is necessary to allow for the creation of intermediate SLAB objects on systems configured with Ceph authentication.
Fixes: https://tracker.ceph.com/issues/72894 Signed-off-by: Mark Kogan <mkogan@ibm.com>
Update PendingReleaseNotes
Co-authored-by: Yuval Lifshitz <yuvalif@yahoo.com> Signed-off-by: Mark Kogan <31659604+mkogan1@users.noreply.github.com>
Update PendingReleaseNotes
Co-authored-by: Yuval Lifshitz <yuvalif@yahoo.com> Signed-off-by: Mark Kogan <31659604+mkogan1@users.noreply.github.com>
(cherry picked from commit dae572d50080609c77d7131cfc99b1fb3f16d31b) Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
Resolves: rhbz#2393790
Rishabh Dave [Thu, 8 May 2025 15:05:39 +0000 (20:35 +0530)]
mgr/vol: keep clone source info even after cloning is finished
Instead of removing the information regarding source of a cloned
subvolume from the .meta file after the cloning has finished, keep it as
it is as the user may find it useful.
Justin Caratzas [Mon, 6 Oct 2025 23:25:44 +0000 (19:25 -0400)]
mgr/dashboard: add an option to control the dashboard crypto caller
Add a mgr config option `crypto_caller` that lets a ceph user override
the default behavior of using the remote crypto caller. Supported
values are `internal` and `remote`.
Justin Caratzas [Mon, 6 Oct 2025 23:25:44 +0000 (19:25 -0400)]
mgr/cephadm: always use the internal cryptocaller
The cephadm modules needs to use python cryptography module for ssh (via
asyncssh) and thus there's no need to use the remote crypto caller in
cephadm. Configure cephadm to always use the internal cryptocaller.
Justin Caratzas [Mon, 6 Oct 2025 23:25:44 +0000 (19:25 -0400)]
python-common/cryptotools: catch all failures to read cert
Previously, the internal crypto caller would catch (and convert) some
errors when reading the cert but not all cases. Move the logic to catch
the errors to a common location and do it once consistently.
Justin Caratzas [Mon, 6 Oct 2025 23:25:44 +0000 (19:25 -0400)]
python-common/cryptotools: unify and organize all endpoint functions
Lightly reorganize and make the "endpoint" functions in cryptotools.py more
consistent and uniform. Use small functions for input and output
handling so that the handling is done the same way throughout. Pass a
pre-constructed crypto caller via the args to then endpoint functions.
Make generating the private key it's own named function rather than
one single (and only) function with overloaded behavior controlled by
a cli switch.
Justin Caratzas [Mon, 6 Oct 2025 23:25:44 +0000 (19:25 -0400)]
pybind/mgr: fix test case in test_tls.py
Why violate the typing in a test? mypy never noticed this because tests
are not type checked but there seems to be no need to turn a str into
bytes to pass to a function that is typed only as taking str!
Justin Caratzas [Mon, 6 Oct 2025 23:25:43 +0000 (19:25 -0400)]
python-common/cryptotools: fix error path in verify tls function
The remote verify_tls function was not raising errors when it should.
Fix the function so that it always returns an object when it succeeds or
fails gracefully. Always parse that function in the crypto caller class.
Justin Caratzas [Mon, 6 Oct 2025 23:25:43 +0000 (19:25 -0400)]
python-common/cryptotools: create CrytpoCaller interface class
Create a class to act as a common shim between the cryptotools external
functions and the mgr. It provides common conversion mechanisms and
could possibly act as an abstraction in case we decide to make
the external function calls in different ways in the future.
Justin Caratzas [Mon, 6 Oct 2025 23:25:43 +0000 (19:25 -0400)]
pybind/mgr: Hack around the 'ImportError: PyO3 modules may only be initialized once per interpreter process' issue.
Fixes: https://tracker.ceph.com/issues/64213 Signed-off-by: Paulo E. Castro <pecastro@wormholenet.com>
(cherry picked from commit 717d0a6f3530ad3e07f4423002810327b2addcf1)
doc: update Grafana certificate configuration to use certmgr
With the introduction of certmgr, users must register their certificates
via `ceph orch certmgr cert set --hostname ...` instead of the old
config-key method. The updated docs clarify that Grafana certificates
are host-scoped and can only be provided by reference (or default to
cephadm-signed).
doc: update RGW HTTPS configuration to use certmgr and new fields
With the introduction of certmgr, RGW services now support three
certificate sources: cephadm-signed (default), inline, and reference.
Docs have been updated to:
- Show how to provide inline certificates using the new ssl_cert/ssl_key
fields instead of the deprecated rgw_frontend_ssl_certificate.
- Explain how to register and reference user-provided certs/keys
- Clarify that cephadm-signed certificates remain the default, with
optional wildcard SANs support.
The usage of rgw_frontend_ssl_certificate is still supported for
backward compatibility, but is now documented as deprecated.
Remove the code used to migrate Grafana self-signed certificates, as
it is no longer needed. The certmgr logic now handles generating new
certificates during the upgrade, eliminating the need for any migration
code or logic.
Remove the special-case code used for RGW service migration, as it is no
longer needed. The certmgr logic now handles populating the certstore
with the corresponding certificate and key entries by reading their values
directly from the spec. During RGW service redeployment as part of the
upgrade, certmgr will ensure the certstore is updated accordingly.
mgr/cephadm: Fix RGW spec validation for deprecated rgw cert field
Starting from Tentacle, the rgw_frontend_ssl_certificate field has been
deprecated in favor of the new ssl_cert and ssl_key fields. Update the
validation logic to run after this field is automatically transformed into
the new fields, ensuring proper validation of RGW specs.
mgr/cephadm: Include mgmt-gateway/oauth2-proxy in upgrade process
Add the new mgmt-gateway and oauth2-proxy services to the list of
services upgraded by cephadm, ensuring they are updated alongside the
rest of the cephadm-managed services.
pybind/mgr/volumes: add getter and setter APIs for snapdir_visibility
Conflicts:
fscrypt changes exist downstream 01a4d2a0356e5f66b7260dad7de70a5fa9cc3aa7 but not upstream,
so it led to a conflict, kept both the changes in the branch.
client: check client config and snaprealm flag before snapdir lookup
this commit adds a new client config client_respect_subvolume_snapshot_visibility
which acts as knob to have a per-client control over the snapshot visibility and
checks it along with the snaprealm flag while looking up a subvolume inode.
Dhairya Parmar [Wed, 6 Aug 2025 21:32:05 +0000 (03:02 +0530)]
common,mds: transmit SNAPDIR_VISIBILITY flag via SnapRealmInfoNew
at the time of building snap trace
Conflicts:
upstream ed6b71246137f9793f2d56b4d050b271a3da29fd made changes to generate_test_instances()
which is not present downstream in ceph-9.0-rhel-patches, so had to adjust accordingly.
mds: rebuild snaprealm cache if last_modified or change_attr changed
For the server side snapdir visibility changes to be transported to the
client — SnapRealm cache needs to be rebuilt otherwise the same metadata
would be sent via the send_snap_update() in C_MDS_inode_update_finish() while
setting the `ceph.dir.subvolume.snaps.visible` vxattr.
The condition used to check for the `seq` and `last_destroyed` against their
cached values but for the vxattr change, it's a rather non-feasible
heavylifting to update the `seq` which involves a set of steps to prepare the
op, commit the op, journal the changes and update snap-server/client(s) just
for a mere flag update (and updating last_destroyed anyway doesn't make sense
for this case). So, compare last_modified and change_attr with their cached
values to check if the SnapRealm cache should be rebuilt. These values are
incremented in the Server::handle_client_setvxattr while toggling the
snapshot visibility xattr and this would enforce a cache rebuild.
Conflicts:
upstream ed6b71246137f9793f2d56b4d050b271a3da29fd made changes to generate_test_instances()
but is not present downstream in ceph-9.0-rhel-patches, so had to resort to adjust accordingly.
librbd: fix segfault when removing non-existent group
Removing a non-existent group triggers a segfault in
librbd::mirror::GroupGetInfoRequest::send(). The issue is caused by a missing
return after finish(), which allows execution to fall through into
GroupGetInfoRequest::get_id() and access invalid memory.
Also, makesure to ignore ENOENT throughout the method Group::remove()
except at cls_client::dir_get_id()
Ramana Raja [Mon, 8 Sep 2025 02:50:51 +0000 (22:50 -0400)]
qa/workunits: add scenario to "test_force_promote_delete_group"
... in rbd_mirror_group_simple test suite.
After the group and its images are removed from the secondary, the test
can run in one of two scenarios. In Scenario 1, the test confirms that
the group is completely synced from the primary to the secondary. In
Scenario 2, the test disables and re-enables the primary, and then
confirms the group syncs from the primary to the secondary. Currently,
both of the scenarios fail occassionally when trying to confirm that
group is completely synced from the primary to the secondary.
Signed-off-by: Ramana Raja <rraja@redhat.com>
Resolves: rhbz#2399618
rbd-mirror: skip validation of primary demote snapshots
Problem:
When a primary demotion is in progress, the demote snapshot is in an incomplete
state. However, the group replayer incorrectly attempts to validate this
snapshot using validate_local_group_snapshots(), treating the cluster as if it
were secondary. This results in the group status being incorrectly set to
up+replaying instead of up+unknown.
Solution:
Avoid validating snapshots that are in the process of being demoted on the
primary. This ensures the group replayer does not mistakenly assign an
incorrect role or state during transition.
Adam King [Thu, 25 Sep 2025 20:13:18 +0000 (16:13 -0400)]
mgr/cephadm: split host cache entries if they exceed max mon store entry size
If the json blob we attempt to store for a host entry
exceeds the max mon store entry size, we become unable
to continue to store that hosts information in the
config key store. This means we only ever have the
information from the last time the json blob was
under the size limit each time the mgr fails over,
resulting in a number of stray host/daemon warnings
being generated and very outdated information being
reported by `ceph orch ps` and `ceph orch ls` around
the time of the failover
Igor Fedotov [Thu, 21 Aug 2025 10:42:54 +0000 (13:42 +0300)]
test/libcephfs: use more entries to reproduce snapdiff fragmentation
issue.
Snapdiff listing fragments have different boundaries in Reef and Squid+
releases hence original reproducer (made for Reef) doesn't work properly
in S+ releases. This patch fixes that at cost of longer execution.
This might be redundant/senseless when backporting to Reef.
Igor Fedotov [Tue, 12 Aug 2025 13:17:49 +0000 (16:17 +0300)]
mds: rollback the snapdiff fragment entries with the same name if needed.
This is required when more entries with the same name don't fit into the
fragment. With the existing means for fragment offset specification such a splitting to be
prohibited.
Signed-off-by: Dnyaneshwari <dtalweka@redhat.com>
mgr/dashboard: handle creation of new pool
Commit includes:
1) Provide link to create a new pool
2) Refactored validation on ACL mapping, removed required validator as default
3) fixed runtime error on console due to ACL length due to which the details section was not opening
4) Used rxjs operators to make API calls and making form ready once all data is available, fixing the form patch issues
5) Refactored some part of code to improve the performance
6) Added zone and pool information in details section for local storage class
Fixes: https://tracker.ceph.com/issues/72569 Signed-off-by: Naman Munet <naman.munet@ibm.com>
(cherry picked from commit 2d0e71c845643a26d4425ddac8ee0ff30153eff2)
Problem:
The readdir wouldn't list all the entries in the directory
when the osd is full with rstats enabled.
Cause:
The issue happens only in multi-mds cephfs cluster. If rstats
is enabled, the readdir would request 'Fa' cap on every dentry,
basically to fetch the size of the directories. Note that 'Fa' is
CEPH_CAP_GWREXTEND which maps to CEPH_CAP_FILE_WREXTEND and is
used by CEPH_STAT_RSTAT.
The request for the cap is a getattr call and it need not go to
the auth mds. If rstats is enabled, the getattr would go with
the mask CEPH_STAT_RSTAT which mandates the requirement for
auth-mds in 'handle_client_getattr', so that the request gets
forwarded to auth mds if it's not the auth. But if the osd is full,
the indode is fetched in the 'dispatch_client_request' before
calling the handler function of respective op, to check the
FULL cap access for certain metadata write operations. If the inode
doesn't exist, ESTALE is returned. This is wrong for the operations
like getattr, where the inode might not be in memory on the non-auth
mds and returning ESTALE is confusing and client wouldn't retry. This
is introduced by the commit 6db81d8479b539d which fixes subvolume
deletion when osd is full.
Fix:
Fetch the inode required for the FULL cap access check for the
relevant operations in osd full scenario. This makes sense because
all the operations would mostly be preceded with lookup and load
the inode in memory or they would handle ESTALE gracefully.
Venky Shankar [Fri, 29 Aug 2025 07:15:09 +0000 (07:15 +0000)]
qa/cephfs: use fuse mount for volumes/subvolume tests
Using the kernel client is a) not really required existing
volume/subvolume test and b) per-subvolume metrics is only
supported by the user-space client library.
projects / ceph-ci.git / log