John Mulligan [Fri, 20 Oct 2023 14:52:56 +0000 (10:52 -0400)]
cephadm: move daemon args assignment to a variable
Move the call to _get_daemon_args out of the function call.
This will aid with the next refactoring steps, were the content
of _get_daemon_args will be lifted into get_container.
Signed-off-by: John Mulligan <jmulligan@redhat.com>
John Mulligan [Fri, 20 Oct 2023 14:30:16 +0000 (10:30 -0400)]
cephadm: ensure entrypoint is assigned a string
Ensure that the entrypoint variable is assigned a string. The
custom container class can have None and that can confuse future
refactoring.
The mypy command we're using does not notice this.
Signed-off-by: John Mulligan <jmulligan@redhat.com>
John Mulligan [Thu, 19 Oct 2023 20:25:17 +0000 (16:25 -0400)]
cephadm: make get_daemon_args private
The get_daemon_args function now only has one caller. To double check
that and to prepare for this function's eventual removal we prefix the
name with an underscore to take it private.
Signed-off-by: John Mulligan <jmulligan@redhat.com>
John Mulligan [Thu, 19 Oct 2023 20:22:37 +0000 (16:22 -0400)]
cephadm: use ceph specific get_daemon_args calls
Use the new Ceph class specific get_daemon_args method instead of the
generic get_daemon_args calls that check all sorts of daemon types when
we're running specialized mon containers, which we know for certain are
ceph based.
Signed-off-by: John Mulligan <jmulligan@redhat.com>
John Mulligan [Thu, 19 Oct 2023 18:48:17 +0000 (14:48 -0400)]
cephadm: consolidate if-blocks in get_container function
Instead of having a number of separate if-statement blocks try to
reduce the number of sections so that you can look at one block
and see what the function is doing for that daemon type.
Signed-off-by: John Mulligan <jmulligan@redhat.com>
John Mulligan [Thu, 19 Oct 2023 18:28:46 +0000 (14:28 -0400)]
cephadm: move all pids-limit logic to renamed function
Repurpose set_pids_limit_unlimited as _update_pids_limit, moving the
logic to decide if the daemon should have unlimited pids into the
function. This was the only caller of set_pids_limit_unlimited.
Signed-off-by: John Mulligan <jmulligan@redhat.com>
John Mulligan [Thu, 19 Oct 2023 18:22:49 +0000 (14:22 -0400)]
cephadm: remove get_deployment_container
Remove get_deployment_container replacing all calls to it with calls to
to_deployment_container. Now, callers can inject modifications to the
container object between calls or even have to_deployment_container
update container objects that were constructed in some other way.
Signed-off-by: John Mulligan <jmulligan@redhat.com>
John Mulligan [Thu, 19 Oct 2023 20:33:57 +0000 (16:33 -0400)]
cephadm: fix mistaken call to get_container_mounts_for_type
When this function was refactored two variants were created:
get_container_mounts_for_type and _get_container_mounts_for_type. The
latter doesn't append podman specific mounts so that it can be called
from other functions that do append podman specific mounts.
Unfortunately, I didn't call the correct function from
get_container_mounts and therefore it was redundantly doing the podman
mounts update. This corrects that error.
Signed-off-by: John Mulligan <jmulligan@redhat.com>
John Mulligan [Thu, 19 Oct 2023 13:42:34 +0000 (09:42 -0400)]
cephadm: move extract_uid_gid func to container_types module
While extract_uid_gid isn't a perfect fit for container_types it is a
fairly fundamental function for working with containers in cephadm and
doesn't require anything beyond types in containers_types and that
module's existing imports. Moving extract_uid_gid should allow us to
more easily move other functions in the future.
Signed-off-by: John Mulligan <jmulligan@redhat.com>
Ville Ojamo [Fri, 3 Nov 2023 05:44:00 +0000 (12:44 +0700)]
doc/cephadm/services: remove excess rendered indentation in osd.rst
Start bash command blocks at the left margin, removing
excessive padding/indentation that would render the
block too much towards the right.
At the same time ident the source consistently:
- Two spaces for command blocks and output blocks.
- Four spaces for notes, code blocks.
There seems to be no uniform style for this, sometimes
commands are indented with three spaces but it would
seem two spaces is common. In the end it all renders
the same I guess.
Signed-off-by: Ville Ojamo <14869000+bluikko@users.noreply.github.com>
osd/SnapMapper: maintain the prefix_itr between calls to SnapMapper::get_next_objects_to_trim()
Maintain the prefix_itr between calls to SnapMapper::get_next_objects_to_trim() to prevent searching depleted prefixes.
We got 8 distinct hash prefixes used for searching objects owned by a given PG.
On each call to SnapMapper::get_next_objects_to_trim() we start from the first prefix even after all objects mapped to it were depleted.
This means that we will be searching for 1 non-existing prefix after the first prefix was depleted, 2 after the first two prefixes were depleted... and so on until we will search 7 non-existing prefixes after the first 7 prefixes were depleted.
This is a performance improvement PR only!
It maintains the existing behavior and does not try to fix/change any of the TRIM logic.
I added an extra step after the last object is trimmed doing a full scan of the DB and only if no object was found it will return ENOENT.
This should make the new code no-worse than existing code which returns ENOENT after a full scan found no object.
It should not impact performance in real life snaps as it should only happen once per-snap.
added snap-mapper tests to rados-test-suite
disabled osd_debug_trim_objects when running (SnapMapperTest, prefix_itr) to prevent asserts(as this code does illegal inserts into DELETED snaps)
Code beautifing
Disabled the assert as there is a corner case when we retrieve the last valid object/s in a snap
The prefix_itr is advanced past the last valid value (as we completed a full scan)
If the OSD will call get_next_objects_to_trim() before the retrieved object/s was processed and removed from the SnapMapper DB it won't be found by the next call (as the prefix_itr is invalid).
The object will be found in the second-pass which will seems as if it was added after the trim was started (which is illegal) and will trigger an ASSERT
Signed-off-by: Gabriel BenHanokh <gbenhano@redhat.com>
Ionut Balutoiu [Wed, 1 Nov 2023 16:07:58 +0000 (18:07 +0200)]
rgw: fix cloud-sync multi-tenancy scenario
At the moment, we cannot set buckets prefixed with tenant ID in the
`source_bucket` field from cloud-sync profiles (non-trivial config):
https://docs.ceph.com/en/latest/radosgw/cloud-sync-module/#non-trivial-configuration
This is because the `do_find_profile` function only searches in the
profiles configured using `bucket.name`, and it ignores `bucket.tenant`.
This is problematic in the RGW multi-tenancy scenario:
https://docs.ceph.com/en/latest/radosgw/multitenancy/#rgw-multi-tenancy
At the moment, we can only configure bucket name in the profile
`source_bucket` field. In the multi-tenancy scenario, this would sync
all the buckets (from all the tenants).
Without this fix, we cannot configure a cloud-sync profile that syncs
all the buckets from a tenant to a particular S3 target.
For example, we cannot do this:
* `tenantA/test-bucket` -> S3 target A
* `tenantB/test-bucket` -> S3 target B
* `tenantC/test-bucket` -> S3 target C
We can only do this at the moment:
* `test-bucket` -> S3 target A
If `test-bucket` is present in both `tenantA` and `tenantB`, both
buckets will be synced to S3 target A.
The idea would be to be able to do this:
* `tenantA/*` -> S3 target A
* `tenantB/*` -> S3 target B
* `tenantC/*` -> S3 target C
If `test-bucket` is present in all tenants, each tenant bucket is
synced to its own S3 target.
Zac Dover [Wed, 1 Nov 2023 01:53:59 +0000 (11:53 +1000)]
doc/cephadm: edit troubleshooting.rst (1 of x)
Edit doc/cephadm/troubleshooting.rst. This commit and the PR of which it
is a part was raised in response to
https://github.com/ceph/ceph/pull/53976. The limits of reStructuredText
are particularly visible here in every instance of a BASH for-loop and
in every instance of a command stretched over multiple lines.
Co-authored-by: Anthony D'Atri <anthony.datri@gmail.com> Signed-off-by: Zac Dover <zac.dover@proton.me>
Rishabh Dave [Thu, 26 Oct 2023 10:48:31 +0000 (16:18 +0530)]
cmake: add --progress flag to git submodule update commands
Ceph has lots of submodules that needs to be cloned before building
binaries from the repository. Seeing the progress when these submodules
are being cloned is useful, especially when developers/users have a
network issue or a slow network.
Zac Dover [Mon, 30 Oct 2023 02:37:39 +0000 (12:37 +1000)]
doc/glossary: improve "BlueStore" entry
Initially s/backend/back end/ but then I added a little more information
about BlueStore's use of RocksDB to map object names to block locations
on disk.
Co-authored-by: Anthony D'Atri <anthony.datri@gmail.com> Signed-off-by: Zac Dover <zac.dover@proton.me>
projects / ceph.git / log