Zac Dover [Tue, 4 Jun 2024 13:37:27 +0000 (23:37 +1000)]
doc/start: s/http/https/ in links
Replace "http" with "https" in doc/start/get-involved.rst.
This commit is, in a way, a repeat of
https://github.com/ceph/ceph/pull/57213/
(1c5383b91bd7dbfa9670c6485fcc5ff28b79f40d), which targeted the Reef
branch instead of the main branch. When this commit has been merged and
backported, I will close https://github.com/ceph/ceph/pull/57213/.
I am listing Casey Cain here as the co-author, but he is in fact the
true author of this change.
J. Eric Ivancich [Wed, 29 May 2024 18:19:25 +0000 (14:19 -0400)]
rgw: track initiator of reshard queue entries
The logic for managing the reshard queue (log) can vary depending on
whether the entry was added by an admin or by dynamic resharding. For
example, if it's a reshard reduction, dynamic resharding won't
overwrite the queue entry so as not to disrupt the reduction wait
period. On the other hand, and admin should be able to overwrite the
entry at will.
So we now track the initiator of each entry on the queue. This adds
another field to that at rest data structure, and it updates the logic
to make use of it.
Signed-off-by: J. Eric Ivancich <ivancich@redhat.com>
J. Eric Ivancich [Tue, 21 May 2024 18:06:47 +0000 (14:06 -0400)]
rgw: provide testing support to dynamic resharding with reduction
Adds a config option rgw_reshard_debug_interval that will allow us to
make the resharding algorithms run on a faster schedule by allowing
one day to be simulated by a set number of seconds.
Signed-off-by: J. Eric Ivancich <ivancich@redhat.com>
J. Eric Ivancich [Tue, 14 May 2024 19:09:03 +0000 (15:09 -0400)]
rgw: add shard reduction ability to dynamic resharding
Previously, dynamic resharding could only *increase* the number of
bucket index shards for a given bucket. This adds the ability to also
*reduce* the number of shards.
So in addition the existing 100,000 entries (current default value)
per shard trigger for an increase, there's a new trigger of 10,000
entries per shard for a decrease.
However, for buckets with object-counts that go up and down regularly,
we don't want to keep resharding up and down to chase the number of
objects. So for shard reduction to take place there's also a time
delay (default 5 days). Once the entry on the reshard queue (log) is
added for reduction, processing will not result in a reshard reduction
within this delay period as the queue is processed. Only when the
reshard entry is processed after this delay can it perform the shard
reduction.
However, if at any point between the time the shard reduction entry is
added to the queue and after the delay, if the entry is processed and
there are *not* few enough entries to trigger a shard reduction, the
entry on the reshard queue entry will be discarded.
So using the defaults, this effectively means the bucket must have few
enough objects for a shard reduction for 5 consecutive days before the
reshard will take place.
Signed-off-by: J. Eric Ivancich <ivancich@redhat.com>
Lucian Petrut [Fri, 24 May 2024 10:03:11 +0000 (10:03 +0000)]
rbd-wnbd: wait for the disk cleanup to complete
The WNBD disk removal workflow is asynchronous, which is why we'll
need to wait for the cleanup to complete when stopping the service.
The "disconnect_all_mappings" function is moved to
RbdMappingDispatcher::stop, allowing us to access the mapping list
more easily and reject new mappings after a stop has been requested.
J. Eric Ivancich [Fri, 17 May 2024 23:23:48 +0000 (19:23 -0400)]
cls/rgw: adding an entry to reshard queue has O_CREAT option
Adds the ability to prevent overwriting a reshard queue (log) entry
for a given bucket with a newer entry. This adds a flag to the op, so
it will either CREATE or make no changes. If an entry already exists
when this flag is set, -EEXIST will be returned.
This is a preparatory step to adding shard reduction to dynamic
resharding.
Signed-off-by: J. Eric Ivancich <ivancich@redhat.com>
Venky Shankar [Wed, 29 May 2024 09:34:58 +0000 (15:04 +0530)]
Merge PR #55758 into main
* refs/pull/55758/head:
doc: update 'journal reset' command with --yes-i-really-really-mean-it
qa: fix cephfs-journal-tool command options and make fs inactive
cephfs-journal-tool: Add warning messages during 'journal reset' and prevent execution on active fs
Patrick Donnelly [Tue, 28 May 2024 16:46:08 +0000 (12:46 -0400)]
Merge PR #57579 into main
* refs/pull/57579/head:
mds/quiesce: disable quiesce root debug parameters by default
mds/quiesce-agt: never send a synchronous ack
mds/quiesce-agt: add test for a rapid async ack
mds/quiesce: always abort fragmenting asynchronously to prevent reentrancy
mds/quiesce: overdrive an export if it hasn't frozen the tree yet
mds/quiesce: quiesce_inode should not hold on to remote auth pins
qa/cephfs: check that a completed quiesce doesn't hold remote auth pins
mds: add `--lifetime` parameter to the `lock path` asok command
mds/quiesce: accept a regular file as the quiesce root
mds: command_quiesce_path: rename `--wait` to `--await` for consistency
mds: command_quiesce_path: do not block the asok thread and return an adequate rc
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Zac Dover [Tue, 28 May 2024 16:27:53 +0000 (02:27 +1000)]
doc/dev: add note about intro of perf counters
Add a note to the "perf counter" section of doc/dev/perf_counters.rst
that explains that this feature was introduced in the Reef release of
Ceph. This note will prevent us from accidentally backporting
perf-counter-related PRs to Quincy.
RGW: Remove get_obj_state()/set_obj_state from SAL
RGWObjState is the state for the StoreObject class. It has historically
been accessible via get_obj_state()/set_obj_state(), but the double
pointer nature of this access has caused multiple bugs, and the
RGWObjState itself is an implementation detail that doesn't need to be
exposed.
Instead, add a load_obj_state() that loads the state from the store, and
use proper getters/setters for the data.
Signed-off-by: Daniel Gryniewicz <dang@redhat.com>
Rishabh Dave [Mon, 27 May 2024 19:37:35 +0000 (01:07 +0530)]
doc/developer_guide: update doc about installing teuthology
There are 2 more ways to install teuthology. Approach with boostrap
script is easier and more convenient while other approach is more
elaborate but manual, document both of them. Don't delete the currently
documented approach because it lets users install teuthology
conveniently in a custom virtual environment. So, keep all three.
Ilya Dryomov [Mon, 27 May 2024 13:56:26 +0000 (15:56 +0200)]
qa/suites/rbd: override extra_system_packages directly on install task
[1] and [2] added support for applying extra_system_packages overrides
directly on install task, but at the same time broke our long standing
workaround where we sneaked extra_system_packages directive in through
an override on ceph task. This is likely getting addressed in [3], but
it's better to not rely on this odd feature in the first place.
Zac Dover [Mon, 27 May 2024 11:09:40 +0000 (21:09 +1000)]
doc/cephfs: s/subvolumegroups/subvolume groups
Use the term "subvolume groups" instead of "subvolumegroups" where the
term appears in plain English. The string "subvolumegroups" is correct
in commands, and remains unchanged.
Also add formatting to command output, to make clearer that the output
is indeed output.
avanthakkar [Fri, 19 May 2023 11:37:28 +0000 (17:07 +0530)]
mgr/dashboard: add helpers for compression in pool form
Fixes: https://tracker.ceph.com/issues/61297 Signed-off-by: avanthakkar <avanjohn@gmail.com>
Adding helpers for compression mode, algorithm, min/max blob size and
compression ratio which is set to 0.875 as default.
The number of journal segments should not be based on the committed
journal_head. Otherwise, if a new journal segment is just opened and the
committed journal_head hasn't been updated, the result will be wrong.
This causes ceph_assert(get_segments_reclaimable() == 0) in
SegmentCleaner::get_next_reclaim_segment().
Leonid Usov [Mon, 20 May 2024 16:17:04 +0000 (19:17 +0300)]
mds/quiesce: overdrive an export if it hasn't frozen the tree yet
Just like with the fragmenting, we should abort an ongoing export
if a quiesce is attempted for the directory.
To minimize the stress for the system, we only allow the abort
if the export hasn't yet managed to freeze the tree. If that is the case,
then quiesce will have to wait for the export to finish.
Fixes: https://tracker.ceph.com/issues/66123 Signed-off-by: Leonid Usov <leonid.usov@ibm.com>
Leonid Usov [Mon, 20 May 2024 22:03:15 +0000 (01:03 +0300)]
mds/quiesce: quiesce_inode should not hold on to remote auth pins
1. avoid taking a remote authpin for the quiesce lock
2. drop remote authpins that were taken because of other locks
We should not be forcing a mustpin when taking quiesce lock.
This creates unnecessary overhead due to the distributed nature
of the quiesce: all ranks will execute quiesce_inode, including
the auth rank, which will authpin the inode.
Auth pinning on the auth rank is important to synchronize quiesce
with operations that are managed by the auth, like fragmenting
and exporting.
If we let a remote quiesce process take a foreign authpin then
it may block freezing on the auth, which will stall quiesce locally.
This wouldn't be a problem if the quiesce that is blocked on the auth
and the quiesce that's holding a remote authpin from the replica side
were unrelated, but in our case it may be the same logical quiesce
that effectively steps on its own toes. This creates an opportunity
for a deadlock.
Fixes: https://tracker.ceph.com/issues/66152 Signed-off-by: Leonid Usov <leonid.usov@ibm.com>
Kefu Chai [Thu, 23 May 2024 04:47:26 +0000 (12:47 +0800)]
ceph-volume: use importlib from stdlib on Python 3.8 and up
since packaging was apparently removed from pkg_resources, let's use
importlib.metadata when it is available and pkg_resources on older
Python versions.
Kefu Chai [Fri, 24 May 2024 09:51:55 +0000 (17:51 +0800)]
cmake: : link shec_utils against legacy-option-headers
in c24a6ffe20, we tried to link all target dependent on legacy option
headers against legacy-option-headers, but we missed some of them.
in our CI, we spotted build failure like:
```
In file included from /home/jenkins-build/build/workspace/ceph-pull-requests/src/erasure-code/ErasureCode.cc:26:
In file included from /home/jenkins-build/build/workspace/ceph-pull-requests/src/osd/osd_types.h:41:
In file included from /home/jenkins-build/build/workspace/ceph-pull-requests/src/common/ceph_context.h:41:
In file included from /home/jenkins-build/build/workspace/ceph-pull-requests/src/common/config_proxy.h:6:
In file included from /home/jenkins-build/build/workspace/ceph-pull-requests/src/common/config.h:27:
In file included from /home/jenkins-build/build/workspace/ceph-pull-requests/src/common/config_values.h:59:
/home/jenkins-build/build/workspace/ceph-pull-requests/src/common/options/legacy_config_opts.h:1:10: fatal error: 'global_legacy_options.h' file not found
^~~~~~~~~~~~~~~~~~~~~~~~~
1 error generated.
```
so in this change, we link the shec_utils to `legacy-option-headers`
as well to fulfill the build dependency.
projects / ceph.git / log