Laura Flores [Fri, 7 Mar 2025 06:22:00 +0000 (06:22 +0000)]
mon, osd: add command to remove invalid pg-upmap-primary entries
The current rm-pg-upmap-primary command checks that the pgid exists
in the pgmap before continuing to remove it. Due to https://tracker.ceph.com/issues/66867,
some invalid pg-upmap-primary entires may exist for pools that have been removed.
Currently, these mappings are impossible to remove since the pgids no longer
exist in the pgmap.
This new command, rm-pg-upmap-primary-all, allows users the ability to remove
any and all pg-upmap-primary mappings in the osdmap at once, which includes
valid and invalid entries.
This command may also be helpful when upgrading from versions where users
are plagued by https://tracker.ceph.com/issues/61948. Users may use an upgraded
mon to remove all pg-upmap-primray entries (valid and invalid) so they continue
to upgrade to a safe version.
See manual testing for this patch here: https://tracker.ceph.com/issues/67179#note-12
Fixes: https://tracker.ceph.com/issues/67179 Fixes: https://tracker.ceph.com/issues/69760 Signed-off-by: Laura Flores <lflores@ibm.com>
(cherry picked from commit 6e9e2033bf0f4779bdfac9a3a4f29115459c8c0e)
This commit fixes documentation about many-to-many topic relationship for notifications. The current sentence states the same fact twice instead of clarifying.
Patrick Donnelly [Wed, 19 Mar 2025 11:59:23 +0000 (07:59 -0400)]
Merge PR #62095 into squid
* refs/pull/62095/head:
include/cephfs: dump optmetadata
mds: set alternate_name for new fullbit dentries
qa: test alternate_name recovery during mds failover
client: avoid multiple calls to path_walk
libcephfs: getcwd after chdir for API constraint
client: unwrap dentries for getcwd
client: refactor / optimize chdir
test/libcephfs: test getcwd with case insensitive dir
test/libcephfs: refactor ManyNestedDirs
include/filepath: add iterators for components
cmake/cephfs: fix options to enable client and dependencies
pybind/cephfs: use legacy noexcept for cdefs for cython 3.Y.Z
pybind/cephfs: increment ref before calling out to c++
client: skip unexpected command replies
mgr: indicate map message is acked instead of unhandled
osdc/Objecter: convert to ms_dispatch2 for ack
client: indicate maps are acked not processed
msg: add alternate statuses for ms_dispatch2 handling
tools/cephfs_mirror: do not process maps with fast dispatch
doc: add docs for volumes interface for charmap
qa: add tests for subvolume charmap settings
pybind/mgr/volumes: wire up charmap for subvol/subvolgroup
pybind/mgr: send MDS commands through cephfs client
pybind/cephfs: wire up mds_command2
mgr: add module method to send notifications
libcephfs: add mds_command2 for asynchronous commands
mgr: excise CephFS client from mgr C++ base
mgr: use std namespace
doc: add docs for CephFS charmap config
qa: add charmap tests
qa: add helpful exceptions for attr changes
qa: ignore libicu leaks
client: add wrappings for charmap manipuluation of dentry names
client: add dir_result_t::dentry::print
win32: add libicu Windows build
CMakeLists: add boost::locale dependency for client
install-deps: unconditionally install boost libraries
test/libcephfs: update root operation return values
client: refactor all path traversals through path_walk
test/libcephfs: test parallel creates
test/libcephfs: add test for lookup failure after readdir
client: init dentry shared_gen with invalid value
client: add _lookup debugging
client: remove redundant check
client: dump InodeStat from mds
mds: encode optmetadata in InodeStat sent to clients
mds: check client features for charmap
mds: add client feature bit for charmap
mds: wire up vxattr for changing charmap
mds: inherit charmap on mkdir
mds,include: add charmap optmetadata
mds,include: add inode_t optional metadata
client: hide alternate_name from API
client: move alternate_name once
client: optimize alternate_name passing to helper
client: relocate definition
client: print dentry with alternate_name on dump
client: move inode dump to print method
mds: add debugging for encoding lease stat
mds: make encode_lease a proper method
mds: add fscrypt metadata for inode stat size
client: use DentryRef for ref counting in MetaRequest
client: add DentryRef
client: add helper for determining if a perm check is necessary
client: cache client_permissions config
client: add debugging for conf changes
client: sort configs
client/UserPerm: add print method
client: note mount parameters in debug log
client: print stat mode in octal
common: add missing op string
include/filepath: add empty path check
python-common: add a utils function to replace distutils.util.strtobool
Reviewed-by: Anthony D Atri <anthony.datri@gmail.com>
Samuel Just [Thu, 13 Feb 2025 04:16:47 +0000 (04:16 +0000)]
dmclock/.../dmclock_server: do not clean clients with requests
PriorityQueueBase::do_clean() shouldn't remove ClientRec instances which
still have queued requests. Otherwise, very low priority clients might
end up having requests actually lost, which shouldn't be possible.
In the OSD, this resulted in PGRecovery items being lost if queued with
background_best_effort while expanding a cluster. Such items can
legitimately sit in the queue for a long period of time as they
represent background data migration which is allowed to be starved by an
aggressive client workload. Dropping the items broke an assumption in
the OSD that all items enqueued would eventually be dequeued resulting
in resources being leaked.
Samuel Just [Thu, 13 Feb 2025 03:54:28 +0000 (03:54 +0000)]
test/osd/TestMClockScheduler: create_item should pass prio < cutoff
Cutoff is set to 12, so let's pass something < 12 rather than 12.
Comments in some tests suggest that the intent is for create_item
to create things in the mclock queue rather than the high_queue.
Samuel Just [Thu, 13 Feb 2025 02:55:27 +0000 (02:55 +0000)]
test/osd/TestMClockScheduler: add test for very slow dequeue
Related: https://tracker.ceph.com/issues/61594 Signed-off-by: Samuel Just <sjust@redhat.com>
(cherry picked from commit b35589f7eb39e6bfabe7df1c55281f41925eca61)
Conflicts:
src/pybind/mgr/dashboard/frontend/package-lock.json (conflicts
with typescript package version, kept the existing one)
src/pybind/mgr/dashboard/frontend/package.json (conflicts with
typescript package version, kept the existing one)
src/pybind/mgr/dashboard/frontend/src/app/ceph/rgw/rgw-multisite-migrate/rgw-multisite-migrate.component.ts (conflicts with automated system user creation in main)
src/pybind/mgr/dashboard/frontend/src/app/shared/forms/cd-validators.ts (conflicts with oauthAddressTest validator)
Patrick Donnelly [Tue, 11 Mar 2025 14:28:01 +0000 (10:28 -0400)]
client: avoid multiple calls to path_walk
When we wrap a dname after a path walk, we cannot do another path walk.
Originally this method was setup to permit that but with (upcoming) encryption
it cannot work. The dname from the path_walk is in the encrypted namespace so
we cannot do another lookup in a second path_walk.
Patrick Donnelly [Tue, 11 Mar 2025 16:13:33 +0000 (12:13 -0400)]
test/libcephfs: test getcwd with case insensitive dir
Unfortunately, it's not easy to refactor this test into a shared method without
setting up an explicit test class which has been avoided up to this point. So
I'm going to just copy the code. Sorry.
pybind/cephfs: use legacy noexcept for cdefs for cython 3.Y.Z
For some newer versions of cython, it appears it requires explicitly specifying
noexcept but old versions of Cython 0.29.Z do not understand that attribute.
pybind/cephfs: increment ref before calling out to c++
At the time this construction seemed safe since the caller should have a
reference but it could conveivably be the only ref. We don't want the ref count
to reach 0.
Additionally, catch errors so this callback is genuinely noexcept.
Patrick Donnelly [Tue, 18 Feb 2025 17:41:52 +0000 (12:41 -0500)]
msg: add alternate statuses for ms_dispatch2 handling
Many dispatchers return false to allow other dispatchers also common messages
like MOSDMap or MFSMap. They implicitly depend on some dispatcher which is
always at the "tail" of the dispatcher queue to return "true" indicating the
msg was processed to avoid messages like:
but this cannot always happen when some libraries like the RadosClient used standalone.
So, add a variant for encapsulating other indications for how the message was
processed by dispatch2. For example, a message may be "acknowledged" but
explicitly allow other dispatchers to try processing the message.
Note: we're using a variant to avoid updating all of the ms_dispatch code to
use the sentinel classes.
Patrick Donnelly [Thu, 30 Jan 2025 22:08:07 +0000 (17:08 -0500)]
mgr: excise CephFS client from mgr C++ base
Linking to the client causes two copies of the Client library to be linked in
the ceph-mgr when modules also dynamically link to libcephfs via the "cephfs"
python library. This creates problems with duplicate boost::locale.
Instead, modules should just use the "cephfs" library to send commands to the
MDS.
Patrick Donnelly [Thu, 30 Jan 2025 22:05:25 +0000 (17:05 -0500)]
mgr: use std namespace
This C++ code was relying on the Client.h header to bring in these names from
the std:: namespace. A subsequent commit plans to remove that header so add
namespace qualifier now.
projects / ceph.git / log