Add revised Zonegroup policy for "multi-zonegroups". This commit
includes changes that Casey Bodley made in
https://github.com/ceph/ceph/pull/52324#discussion_r1253482258 and that
I have integrated into the docs only now.
Add a note about the telos of zonegroups. The information added in this
commit was originally sent to ceph-users by Casey Bodley on 30 Jun 2023
in response to a question from Yixin Jin. Alexander Patrakov then
suggested that Casey's response should be added to
doc/radsogw/multisite.rst.
which is what the migration is actually concerned about
(verification of the rgw_frontend_type in these specs).
In the case where the spec is more simple, we should
just leave the spec alone and move on. Unfortunately
the current code assumes the field will always be
there and hits an unhandled KeyError when trying to
migrate the more simple specs. This causes the
cephadm module to crash shortly after starting an
upgrade to a version that includes this migration
and it's very difficult to find the root cause. This
can be worked around by adding fields to the rgw
spec before upgrade so the "spec" field exists in
the spec and the migration works as intended.
This commit fixes the migration in the simple
case as well as adding testing for that case to
both the unit tests and orch/cephadm teuthology
upgrade tests
Fixes: https://tracker.ceph.com/issues/61889 Signed-off-by: Adam King <adking@redhat.com>
Remove seqdiag assets to determine whether the docs can be built if they
are absent. (Currently they cannot be built when they are present.) If
this works, then these diagrams will be replaced, probably with .png
files.
Describe in greater detail the function of Zones in serving S3 objects,
as requested by Alexander Patrakov here:
https://github.com/ceph/ceph/pull/49981#issuecomment-1615866374.
John Mulligan [Mon, 26 Jun 2023 16:08:34 +0000 (12:08 -0400)]
docs/cephadm: document new extra_{container,entrypoint}_args behavior
Document the new behaviors of extra_container_args and
extra_entrypoint_args. Document that current (previous in code terms)
behavior of splitting strings on spaces and document the recently added
support for ArgumentSpec objects and how they are used.
Signed-off-by: John Mulligan <jmulligan@redhat.com>
John Mulligan [Mon, 29 May 2023 20:52:31 +0000 (16:52 -0400)]
mgr/cephadm: rework how _process_ls_output builds a DaemonDescription
Previously, the _process_ls_output function would create an
DaemonDescription and then assign values to attributes one-by-one. This
prevents the class from doing any data type/value conversions unless we
resort to more complex approaches like descriptors or `__setattr__` for
example. Update the _process_ls_output function to build a
DaemonDescription using keyword args so that we have the opportunity
later to convert data passed to it in `__init__`.
Signed-off-by: John Mulligan <jmulligan@redhat.com>
John Mulligan [Wed, 17 May 2023 17:50:19 +0000 (13:50 -0400)]
python-common: replace types for extra_*_args with ArgSpec based types
Use the ArgSpec, ArgumentList, and GeneralArgList types throughout
the deployment module to support extra_container_args and
extra_entrypoint_args. The GeneralArgList type supports all possible
input forms while ArgumentList reflects the internal representation
of the argument lists.
Signed-off-by: John Mulligan <jmulligan@redhat.com>
Mark Kogan [Wed, 28 Jun 2023 14:06:27 +0000 (14:06 +0000)]
rgw: d3n: fix valgrind reported leak related to libaio ops
Fixes: https://tracker.ceph.com/issues/61661
The valgrind leak indication is a false positive in this case,
it is because the libaio internal thread have not timed out yet
when radosgw is terminated.
```
man aio_init
...
aio_idle_time
This field specifies the amount of time in seconds that a worker thread
should wait for further requests before terminating, after having
completed a previous request. The
default value is 1.
...
```
for the sake of teuthology reducing the timeout
waiting for 2 minutes for example like below would also prevent the leak report
```
❯ env
LD_LIBRARY_PATH=/mnt/nvme5n1p1/src-git/ceph--up--master-clang/build/lib/:$LD_LIBRARY_PATH
PYTHONPATH=$PYTHONPATH:/mnt/nvme5n1p1/src-git/ceph--up--master-clang/build/lib/cython_modules/lib.3
RAGWEED_CONF=$(realpath ./ragweed.conf) RAGWEED_STAGES=prepare,check tox
-- -v |& ccze -Aonolookups ; sleep 2m | pv -t ; pkill radosgw
```
Yingxin Cheng [Thu, 29 Jun 2023 05:25:28 +0000 (13:25 +0800)]
crimson/osd/heartbeat: relax the order of replacement reset and accept
With the new implementation in messenger, the order of replacement reset
and accept events cannot be determined because they are from different
connections.
Modify the heatbeat logic to tolerate the both cases.
John Mulligan [Wed, 17 May 2023 17:49:23 +0000 (13:49 -0400)]
python-common: add an ArgumentSpec type for handling args
Add a new ArgumentSpec type to the deployment package. The
ArgumentSpec serves a dual purpose:
* to allow more advanced configuration of extra arguments
* to keep backwards compatibility with string-based args
The previous versions of cephadm supported supplying extra container and
entrypoint arguments for fine-tuning of services and creating custom
containers. However, this mode assumed that spaces in an argument
always meant that the argument should be split into two parts:
"--foo bar" becomes `["--foo", "bar"]`. In some cases there's a good
reason to keep spaces as in "--title=My Little Cluster". When
an argument is expressed as a single string the ArgumentSpec is
designed to retain the existing behavior. When an argument is
expressed as a JSON object then you can explicitly express if
you want the argument split on spaces or not (not split is the default).
The alternative was to keep using strings but add some level of
shell-style quoting. This was discussed but deemed complex and
difficult to read in YAML. Round tripping that data is also challenging.
The JSON object approach also allows for future fields to be added
providing for possible extensibility.
Signed-off-by: John Mulligan <jmulligan@redhat.com>
Casey Bodley [Wed, 28 Jun 2023 18:57:49 +0000 (14:57 -0400)]
rgw: fetch_remote_obj() preserves RGW_ATTR_COMPRESSION of encrypted objects
if the source object was both compressed and encrypted, preserve its
original compression attribute so it can be decompressed the same way it
was originally compressed
Soumya Koduri [Sat, 24 Jun 2023 06:55:46 +0000 (12:25 +0530)]
rgw: Do not update mtime in `set_attrs`
As per https://docs.aws.amazon.com/AmazonS3/latest/userguide/UsingMetadata.html,
the only way for users to modify object metadata is to make a copy of the object and
set the metadata. Hence retain old mtime for any other attr changes
Venky Shankar [Wed, 28 Jun 2023 04:53:54 +0000 (10:23 +0530)]
Merge PR #49971 into main
* refs/pull/49971/head:
doc/cephfs: document MDS_CLIENTS_LAGGY health warning
qa: ignore warnings
qa: add test cases to check client eviction if an OSD is laggy
mds,messages: enable beacon to report clients lagginess
mds: do not evict client on laggy osds
common: add new config option to defer client eviction
osd: add method to check for laggy osds
Casey Bodley [Tue, 27 Jun 2023 20:45:06 +0000 (16:45 -0400)]
qa/s3tests: filter on 'sts_tests' and 'webidentity_tests'
qa/tasks/s3tests.py was adding `not test_of_sts` and `not
webidentity_test` by default, and the rgw/sts suites were adding
those attrs back in `extra_attrs`
when `extra_attrs` was changed to be additive, this started causing
InvocationError failures
instead of using `extra_attrs` to control these filters,
qa/tasks/s3tests.py now uses the `sts_tests` and `webidentity_tests`
flags from the sts yaml files to control whether or not the `not
test_of_sts`/`not webidentity_test` attrs are added to the pytest
command line
projects / ceph.git / log