John Mulligan [Mon, 11 Dec 2023 19:45:46 +0000 (14:45 -0500)]
cephadm: convert init containers script to use a template
Convert the init containers run script to be based on a template, like
the sidecar run scripts are. The new script is loosely based on the
sidecars run script but only does actions in batches - logically
iterating over each init container configured.
Signed-off-by: John Mulligan <jmulligan@redhat.com>
John Mulligan [Tue, 5 Dec 2023 21:20:57 +0000 (16:20 -0500)]
cephadm: move init container script generation to a function
Move the generation of the init container run script to a small function
fixing a missing `set -e` along the way. This isolates the logic of
generating this run script.
Signed-off-by: John Mulligan <jmulligan@redhat.com>
John Mulligan [Sun, 19 Nov 2023 21:35:06 +0000 (16:35 -0500)]
cephadm: call functions instead of executing rm in rm_cluster
Convert a bunch of invocations of rm via a subprocess to function calls.
This should make it easier (or possible?) to test the function
in the unit test framework as well as possibly saving a few resources.
Signed-off-by: John Mulligan <jmulligan@redhat.com>
John Mulligan [Sun, 19 Nov 2023 21:06:38 +0000 (16:06 -0500)]
cephadm: remove a non-functional line in _rm_cluster
Remove a line from _rm_cluster that has no effect. The line uses
call_throws to execute an `rm -rf` command. The argument contains
asterisk chars that indicate that the file(s)/dir(s) to be created are
expected to match the given pattern. However, globs do not work in this
context in contemporary versions of cephadm.
To double check I added the following temporary unit test:
```
def test_does_it_glob(tmp_path):
from cephadmlib.call_wrappers import call_throws
d1 = (tmp_path / 'dir1')
d1.mkdir()
fns = ['f1.txt', 'f2.txt', 'f3.txt', 'f4']
for fn in fns:
with open(d1/fn, "w") as fh:
fh.write("xxx")
assert d1.exists()
for fn in fns:
assert (d1 / fn).exists()
ctx = FakeContext()
call_throws(ctx, ['rm', '-rf', f'{d1}/*.txt'])
print('files:', os.listdir(d1))
assert d1.exists()
for fn in fns:
if fn.endswith('.txt'):
assert not (d1 / fn).exists()
else:
assert (d1 / fn).exists()
```
If globs worked in this context this test would have passed. It does
not. I confirmed that the current implementation of call/call_wrapper
does not execute the command in a shell context.
I wondered if it was possible that an earlier version of cephadm did
execute this command in a shell context and some changes along the way
changed the behavior. I tracked the origin of the line back to 16ebc620349f6e7c9afa6b992c85900f56fcfca3 the first change to implement
rm-cluster. In this commit the code uses subprocess.check_output
directly. I am familiar with check_output and unless `shell=True` is
passed this function doesn't execute the args in a shell context. The
`shell=True` argument is not passed to check_output. This means that the
very first implementation of this line suffered from the same issue
- it would have no effect on any files except one named with actual
asterisk (`*`) characters.
While I'm sure there was a good intention behind this line, the fact
that it persisted in the code so long in a non-functional state and no
one noticed in both production and qa testing makes me feel that it can
be safely removed with no negative effect. Removing the line simplifies
the code and avoids needing to spend effort unit-testing or manually
checking a fix.
Signed-off-by: John Mulligan <jmulligan@redhat.com>
John Mulligan [Sun, 19 Nov 2023 12:56:46 +0000 (07:56 -0500)]
cephadm: update rm cluster to remove sidecar/init-ctr services
A previous patch updated the rm-daemon functionality to remove sidecar
and init-container systemd services. This patch does the same for the
rm-cluster codepath.
Signed-off-by: John Mulligan <jmulligan@redhat.com>
John Mulligan [Sun, 19 Nov 2023 00:09:29 +0000 (19:09 -0500)]
cephadm: update _rm_cluster to call terminate_service
The terminate_service function was added to encapsulate the act of
terminating a systemd service. The rm-deamon handler was updated to use
this function previously; this commit updates rm-cluster function(s)
to do the same. This needed a bunch of test updates do to how the tests
were mocking commands.
Signed-off-by: John Mulligan <jmulligan@redhat.com>
John Mulligan [Sat, 18 Nov 2023 19:03:34 +0000 (14:03 -0500)]
cephadm: add support to remove sidecar/init-ctr systemd units
Add support to the command_rm_daemon function to remove systemd services
and corresponding unit files for any sidecars or init-containers
associated with a primary daemon.
Signed-off-by: John Mulligan <jmulligan@redhat.com>
John Mulligan [Sat, 18 Nov 2023 18:07:57 +0000 (13:07 -0500)]
cephadm: replace some calls to rm with fs function calls
This should be a little cleaner, perhaps a tiny performance improvement,
and most importantly work better with the virtual-fs provided for the
unit tests and make this function more testable in the future.
Signed-off-by: John Mulligan <jmulligan@redhat.com>
John Mulligan [Sat, 18 Nov 2023 16:50:29 +0000 (11:50 -0500)]
cephadm: minor reorgs to command_rm_daemon
Make a few small chanes to command_rm_daemon to clean it up in
preparation for future changes. Move the construction of a
DaemonIdentity earlier and use the service name derived directly from it
if possible. I didn't remove the old approach of looking up a service
name in the listing because I don't know if doing so would break
old/adopted services and there's no unit test coverage for that.
Signed-off-by: John Mulligan <jmulligan@redhat.com>
John Mulligan [Sun, 19 Nov 2023 00:00:38 +0000 (19:00 -0500)]
cephadm: add a terminate_service systemd function
Add a function that encapsulates the common 3-step process found in
cephadm of stopping, resetting errors, and disabling a service.
This will be used to handle new services to stop as well as cleaning
up some places where the three steps are performed inline.
Signed-off-by: John Mulligan <jmulligan@redhat.com>
John Mulligan [Sat, 18 Nov 2023 18:59:19 +0000 (13:59 -0500)]
cephadm: add sidecars_from_dropin function
Add a sidecars_from_dropin function that takes a minimal PathInfo object
and returns a PathInfo populated with sidecars information based on
reading the content of the dropin file. This will be useful when we need
information on configured sidecars but are not on the deploy path.
Signed-off-by: John Mulligan <jmulligan@redhat.com>
John Mulligan [Sat, 18 Nov 2023 18:57:48 +0000 (13:57 -0500)]
cephadm: add unlink_file function
Add the unlink_file function to file_utils. This fills in a gap between
python 3.6 and features provided in pathlib.Path in later versions of
python. Adds an option to ignore all errors for good measure.
Signed-off-by: John Mulligan <jmulligan@redhat.com>
Rename get_unit_name_by_daemon_name to lookup__unit_name_by_daemon_name
to emphasize that the function does a full lookup (via the daemon list)
to determine the systemd service name for the given values and that
there's a potential performance implication to doing that.
Signed-off-by: John Mulligan <jmulligan@redhat.com>
John Mulligan [Fri, 17 Nov 2023 20:45:55 +0000 (15:45 -0500)]
cepahdm: avoid subclass related issues for data_dir
Do not have data dir depend on the daemon_name property. This property
is overriden by the subclass(es) but we want DaemonIdentities and
DaemonSubIdentities to share the same data_dir.
Signed-off-by: John Mulligan <jmulligan@redhat.com>
John Mulligan [Fri, 17 Nov 2023 16:23:47 +0000 (11:23 -0500)]
cephadm: use service_name method of daemon identity
Use the new service_name method of the DaemonIdentity type in some basic
cases where string concatenation was being used.
I also converted the name of the agent's helper method to match and made
it private to avoid future confusion.
Signed-off-by: John Mulligan <jmulligan@redhat.com>
John Mulligan [Fri, 29 Sep 2023 18:47:10 +0000 (14:47 -0400)]
cephadm: add systemd name generation to daemon identity
There are a bunch of places in the code that wants the full systemd
service name but the code generally is just doing string concatenation.
Add a method to avoid stringing it all the time. This will help as there
will be more systemd work in the future.
Signed-off-by: John Mulligan <jmulligan@redhat.com>
Zac Dover [Sun, 31 Dec 2023 06:22:33 +0000 (16:22 +1000)]
doc/radosgw: edit "remove a subuser"
Edit the English language in the section "Remove a Subuser" in
doc/radosgw/admin.rst. This commit is made in response to Matt
Benjamin's request for improvement of this section
(https://github.com/ceph/ceph/pull/55028#discussion_r1438599833).
Zac Dover [Wed, 20 Dec 2023 05:00:38 +0000 (15:00 +1000)]
doc/radosgw: edit compression.rst
Improve the grammar and simplify the sentence structure of
doc/radosgw/compression.rst. This commit is made in anticipation of a
near-future commit that will list the compression algorithms available
to users of Ceph.
Co-authored-by: Anthony D'Atri <anthony.datri@gmail.com> Signed-off-by: Zac Dover <zac.dover@proton.me>
Ronen Friedman [Thu, 14 Dec 2023 14:27:13 +0000 (08:27 -0600)]
osd/scrub: add a basic set of performance counters
Add a labeled set of performance counters, with the labels selecting
one of four copies of the counters - one per each scrub level and
pool type combination.
Inside the Scrubber, the relevant set of counters is selected
when the scrub is initialized.
NitzanMordhai [Wed, 28 Jun 2023 09:57:11 +0000 (09:57 +0000)]
ceph-dencoder: osd - Add missing types
Currently, ceph-dencoder lacks certain osd types, preventing us from accurately checking the ceph corpus for encode-decode mismatches.
This pull request aims to address this issue by adding the missing types to ceph-dencoder.
To successfully incorporate these types into ceph-dencoder, we need to introduce the necessary `dump` and `generate_test_instances`
functions that was missing in some types. These functions are essential for proper encode and decode of the added types.
This PR will enhance the functionality of ceph-dencoder by including the missing types, enabling a comprehensive analysis of encode-decode consistency.
With the addition of these types, we can ensure the robustness and correctness of the ceph corpus.
This update will significantly contribute to improving the overall reliability and accuracy of ceph-dencoder.
It allows for a more comprehensive assessment of the encode-decode behavior, leading to enhanced data integrity and stability within the ceph ecosystem.
Zac Dover [Tue, 19 Dec 2023 09:15:57 +0000 (19:15 +1000)]
doc/install: update "update submodules"
Remove misleading material that would give readers the wrong idea about
when stale submodules are present. This commit is made in response to
information given to me by Ilya Dryomov here: https://github.com/ceph/ceph/pull/54929#issuecomment-1859237986.
projects / ceph.git / log