osd/SnapMapper: maintain the prefix_itr between calls to SnapMapper::get_next_objects_to_trim()
Maintain the prefix_itr between calls to SnapMapper::get_next_objects_to_trim() to prevent searching depleted prefixes.
We got 8 distinct hash prefixes used for searching objects owned by a given PG.
On each call to SnapMapper::get_next_objects_to_trim() we start from the first prefix even after all objects mapped to it were depleted.
This means that we will be searching for 1 non-existing prefix after the first prefix was depleted, 2 after the first two prefixes were depleted... and so on until we will search 7 non-existing prefixes after the first 7 prefixes were depleted.
This is a performance improvement PR only!
It maintains the existing behavior and does not try to fix/change any of the TRIM logic.
I added an extra step after the last object is trimmed doing a full scan of the DB and only if no object was found it will return ENOENT.
This should make the new code no-worse than existing code which returns ENOENT after a full scan found no object.
It should not impact performance in real life snaps as it should only happen once per-snap.
added snap-mapper tests to rados-test-suite
disabled osd_debug_trim_objects when running (SnapMapperTest, prefix_itr) to prevent asserts(as this code does illegal inserts into DELETED snaps)
Code beautifing
Disabled the assert as there is a corner case when we retrieve the last valid object/s in a snap
The prefix_itr is advanced past the last valid value (as we completed a full scan)
If the OSD will call get_next_objects_to_trim() before the retrieved object/s was processed and removed from the SnapMapper DB it won't be found by the next call (as the prefix_itr is invalid).
The object will be found in the second-pass which will seems as if it was added after the trim was started (which is illegal) and will trigger an ASSERT
Signed-off-by: Gabriel BenHanokh <gbenhano@redhat.com>
Zac Dover [Wed, 1 Nov 2023 01:53:59 +0000 (11:53 +1000)]
doc/cephadm: edit troubleshooting.rst (1 of x)
Edit doc/cephadm/troubleshooting.rst. This commit and the PR of which it
is a part was raised in response to
https://github.com/ceph/ceph/pull/53976. The limits of reStructuredText
are particularly visible here in every instance of a BASH for-loop and
in every instance of a command stretched over multiple lines.
Co-authored-by: Anthony D'Atri <anthony.datri@gmail.com> Signed-off-by: Zac Dover <zac.dover@proton.me>
Rishabh Dave [Thu, 26 Oct 2023 10:48:31 +0000 (16:18 +0530)]
cmake: add --progress flag to git submodule update commands
Ceph has lots of submodules that needs to be cloned before building
binaries from the repository. Seeing the progress when these submodules
are being cloned is useful, especially when developers/users have a
network issue or a slow network.
Zac Dover [Mon, 30 Oct 2023 02:37:39 +0000 (12:37 +1000)]
doc/glossary: improve "BlueStore" entry
Initially s/backend/back end/ but then I added a little more information
about BlueStore's use of RocksDB to map object names to block locations
on disk.
Co-authored-by: Anthony D'Atri <anthony.datri@gmail.com> Signed-off-by: Zac Dover <zac.dover@proton.me>
There is no need for CreateSnapshotRequests.__del__() that calls
CreateSnapshotRequests.wait_for_pending().
MirrorSnapshotScheduleHandler.shutdown() already calls
CreateSnapshotRequests.wait_for_pending().
Ramana Raja [Thu, 26 Oct 2023 17:18:52 +0000 (13:18 -0400)]
mgr/rbd_support: fix recursive locking on CreateSnapshotRequests lock
The MirrorSnapshotScheduleHandler's run thread issues asynchronous
create snapshot requests using a CreateSnapshotRequests instance. When
the thread invokes a CreateSnapshotRequests instance's get_ioctx(),
the instance's class variable lock is acquired. With the class
variable lock held, the garbage collection of a CreateSnapshotRequests
instance may race in the thread. The thread would then call
CreateSnapshotRequests __del__() that tries to acquire the class
variable lock that the thread already holds. Fix this
recursive deadlock by converting the CreateSnapshotRequests lock from
a class variable to an instance variable. There is no need to share
the lock across CreateSnapshotRequests instances.
Also convert MirrorSnapshotScheduleHandler, PerfHandler and
TrashPurgeScheduleHandler class variables to instance variables
that don't need to be shared across the instances.
Fixes: https://tracker.ceph.com/issues/62994 Signed-off-by: Ramana Raja <rraja@redhat.com> Co-Authored-By: Ilya Dryomov <idryomov@gmail.com>
Adam Emerson [Sat, 28 Oct 2023 17:29:59 +0000 (13:29 -0400)]
build: Fix fmt version check
Currently, when attempting to build ceph on a system with fmt
installed, we try to build against it whatever the version. This
constantly breaks people's builds, since newer versions of fmt often
change the API.
This change specifies that versions must be below 10 as well as at or
above 8.1.1, so that on systems with a new format, we fall back to
using the submodule.
It also removes the `Findfmt.cmake` module, as that does not check
the installed version. Instead, we use the cmake config file installed by
the system package of fmt and does support version checking.
Aashish Sharma [Mon, 30 Oct 2023 07:47:37 +0000 (13:17 +0530)]
mgr/dashboard: update rgw multisite import form helper info
Change 'To obtain the token, generate it from your secondary Ceph cluster' to 'To obtain the token, generate it from your primary Ceph cluster' in rgw multisite import form helper
Zac Dover [Fri, 27 Oct 2023 06:58:28 +0000 (16:58 +1000)]
doc/rados: remove cache-tiering-related keys
Remove information related to cache-tiering-related keys from
doc/rados/operations/pools.rst. Cache-tiering is deprecated in Reef.
This PR is suitable for backporting to the Reef release branch, but not
to release branches prior to Reef.
test/librbd/fsx: wait for resize to propagate in krbd_resize()
With this changes resize request will not be blocked until the resize is
completed. Because of this the fsx test fails as it assumes that the
request to resize immediately implies changes on the device size.
Hence we have to add a wait in resize handler of fsx for the device to
actually get resized.
Zac Dover [Wed, 25 Oct 2023 23:48:57 +0000 (09:48 +1000)]
doc/rados: remove HitSet-related key information
Remove HitSet-related key information from
doc/rados/operations/pools.rst. HitSet-related keys are relevant only to
releases of Ceph that support cache tiering. Only Quincy and earlier
(inclusive) releases of Ceph support cache tiering. Backport this commit
from main to Reef, but not to Quincy or to release branches earlier than
Quincy.
John Mulligan [Wed, 11 Oct 2023 18:05:17 +0000 (14:05 -0400)]
cephadm: add a --dry-run option to cephadm shell
Instead of creating the shell, the --dry-run option prints the container
command that would be used. This can be used as a starting point for
creating custom container commands similar to what cephadm shell would
generate but with tweaks.
Signed-off-by: John Mulligan <jmulligan@redhat.com>
Redouane Kachach [Mon, 23 Oct 2023 15:18:57 +0000 (17:18 +0200)]
mgr/rook: remove all the hardcoded refs to the 'rook-ceph' namespace Fixes: https://tracker.ceph.com/issues/63291 Signed-off-by: Redouane Kachach <rkachach@redhat.com>
John Mulligan [Sun, 22 Oct 2023 12:14:24 +0000 (08:14 -0400)]
cephadm: add deployment test for osd
Add a deployment test case for OSD. OSD has some special properties that
we have extra assertions for.
Part of a series of commits to increase coverage of deployment
path features with regards to container engine options, env vars
and mounts. This will serve future refactoring efforts.
Signed-off-by: John Mulligan <jmulligan@redhat.com>
John Mulligan [Sat, 21 Oct 2023 20:27:13 +0000 (16:27 -0400)]
cephadm: add test assertions for nvmeof options, mount
Part of a series of commits to increase coverage of deployment
path features with regards to container engine options, env vars
and mounts. This will serve future refactoring efforts.
Signed-off-by: John Mulligan <jmulligan@redhat.com>
John Mulligan [Sat, 21 Oct 2023 20:29:38 +0000 (16:29 -0400)]
cephadm: add test assertions for iscsi options, mounts
Part of a series of commits to increase coverage of deployment
path features with regards to container engine options, env vars
and mounts. This will serve future refactoring efforts.
Signed-off-by: John Mulligan <jmulligan@redhat.com>
John Mulligan [Sat, 21 Oct 2023 20:20:10 +0000 (16:20 -0400)]
cephadm: add test assertions for keepalived options, mounts
Part of a series of commits to increase coverage of deployment
path features with regards to container engine options, env vars
and mounts. This will serve future refactoring efforts.
Signed-off-by: John Mulligan <jmulligan@redhat.com>
John Mulligan [Sat, 21 Oct 2023 20:16:32 +0000 (16:16 -0400)]
cephadm: add test assertion for haproxy options, mounts
Part of a series of commits to increase coverage of deployment
path features with regards to container engine options, env vars
and mounts. This will serve future refactoring efforts.
Signed-off-by: John Mulligan <jmulligan@redhat.com>
John Mulligan [Sat, 21 Oct 2023 20:12:40 +0000 (16:12 -0400)]
cephadm: add test assertions for nfs env vars, mounts
Part of a series of commits to increase coverage of deployment
path features with regards to container engine options, env vars
and mounts. This will serve future refactoring efforts.
Signed-off-by: John Mulligan <jmulligan@redhat.com>
John Mulligan [Sat, 21 Oct 2023 20:32:15 +0000 (16:32 -0400)]
cephadm: add assertions for monitoring options, mounts
Part of a series of commits to increase coverage of deployment
path features with regards to container engine options, env vars
and mounts. This will serve future refactoring efforts.
Signed-off-by: John Mulligan <jmulligan@redhat.com>
John Mulligan [Sat, 21 Oct 2023 20:35:32 +0000 (16:35 -0400)]
cephadm: add test assertion for snmp env file option
Part of a series of commits to increase coverage of deployment
path features with regards to container engine options, env vars
and mounts. This will serve future refactoring efforts.
Signed-off-by: John Mulligan <jmulligan@redhat.com>
John Mulligan [Sat, 21 Oct 2023 20:10:32 +0000 (16:10 -0400)]
cephadm: add a test assertion for tracing env var
Part of a series of commits to increase coverage of deployment
path features with regards to container engine options, env vars
and mounts. This will serve future refactoring efforts.
Signed-off-by: John Mulligan <jmulligan@redhat.com>
John Mulligan [Sat, 21 Oct 2023 20:08:25 +0000 (16:08 -0400)]
cephadm: add test assertions for ceph mgr entrypoint, mounts
Part of a series of commits to increase coverage of deployment
path features with regards to container engine options, env vars
and mounts. This will serve future refactoring efforts.
Signed-off-by: John Mulligan <jmulligan@redhat.com>
John Mulligan [Sat, 21 Oct 2023 20:04:48 +0000 (16:04 -0400)]
cephadm: add test assertions for unlimited pids option
Ensure that future changes continue to set/not set option as
appropriate.
Part of a series of commits to increase coverage of deployment
path features with regards to container engine options, env vars
and mounts. This will serve future refactoring efforts.
Signed-off-by: John Mulligan <jmulligan@redhat.com>
projects / ceph.git / log