Ilya Dryomov [Tue, 18 Feb 2025 16:51:47 +0000 (17:51 +0100)]
test/rbd_mirror: clear Namespace::s_instance at the end of a test
TestMockPoolReplayer.Namespaces and NamespacesError tests leave behind
a dangling pointer to a stack-allocated MockNamespace which leads to an
easily reproducible use-after-free and segfault when tests are shuffled.
Ilya Dryomov [Mon, 17 Feb 2025 11:41:51 +0000 (12:41 +0100)]
test/rbd_mirror: flush watch/notify callbacks in TestImageReplayer
TestImageReplayer establishes its own (i.e. outside of the SUT code)
watch on the header of the remote image to be able to synchronize the
execution of the test with certain notifications. This watch is
established before the remote image is opened and is teared down until
after the remote image is closed but while the image replayer is still
running. The flush that is part of image close sequence thus isn't
guaranteed to cover all callbacks, especially for snapshot-based
mirroring where UnlinkPeerRequest spawned from Replayer::unlink_peer()
generates a notification on the remote image for each completed unlink.
Since TestImageReplayer further immediately deletes C_WatchCtx, pretty
much any test can segfault when C_WatchCtx::handle_notify() is invoked
by TestWatchNotify infrastructure. Because it's a virtual method, the
segfault often involves a completely bogus instruction pointer:
John Mulligan [Thu, 27 Jul 2023 18:17:36 +0000 (14:17 -0400)]
python-common: fix valid_addr on python 3.11
The behavior on python 3.11 regarding IPv4 addresses in bracket has
changed:
```
$ python3.8 -c 'from urllib.parse import urlparse; urlparse("http://[192.168.0.1]")'
[john@edfu ~]$ python3.11 -c 'from urllib.parse import urlparse; urlparse("http://[192.168.0.1]")'
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "/usr/lib64/python3.11/urllib/parse.py", line 395, in urlparse
splitresult = urlsplit(url, scheme, allow_fragments)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib64/python3.11/urllib/parse.py", line 500, in urlsplit
_check_bracketed_host(bracketed_host)
File "/usr/lib64/python3.11/urllib/parse.py", line 448, in
_check_bracketed_host
raise ValueError(f"An IPv4 address cannot be in brackets")
ValueError: An IPv4 address cannot be in brackets
```
This breaks the test in test_valid_addr that asserts that function
valid_addr returns the string "IPv4 address wrapped in brackets is
invalid".
Move the step that checks for brackets and dots above the urllib
check so that the function continues returning the expected string.
Adam King [Wed, 12 Feb 2025 16:32:24 +0000 (11:32 -0500)]
mgr/cephadm: use double quotes for NFSv4 RecoveryBackend in ganesha conf
This came directly from someone on the ganesha team. We've actually had
this use single quotes for a long time (at least since mid 2020) but I
believe recent feature work on the ganesha side exposed the issue
Adam King [Thu, 30 Jan 2025 14:15:37 +0000 (09:15 -0500)]
mgr/cephadm: create OSD daemon deploy specs through make_daemon_spec
That function handles setting up the extra container/entrypoint
args for the daemon during initial deployment. Having the
CephadmDaemonDeploySpec made directly in the OSD deployment
workflow means initial deployments of OSDs won't have the
extra container/entrypoint args from the spec
orch: refactor boolean handling in drive group spec
The intent of 42721c03ee6f was to address an issue where boolean
parameters weren't handled correctly.
I noticed that a parameter (`tpm2`) was missed, which made me realize
that maintaining a list of these boolean parameters is necessary.
To simplify things, we should only accept `"true"` or `"false"` (in any case),
allowing us to avoid the need to maintain a list of boolean parameters.
This change introduces a `list_drive_group_spec_bool_arg` to store boolean
arguments related to drive group specifications, simplifying the validation
process for boolean values by directly checking if the values are 'true' or 'false'.
Adam King [Wed, 5 Feb 2025 22:00:06 +0000 (17:00 -0500)]
mgr/cephadm: fix typo with vrrp_interfaces in keepalive setup
This was intended to be vrrp_interfaces, the variable actually
used later in the code. Instead, due to a typo, it was setting
a variable that is unused other than in the log line right after
it is set. Issue was introduced in
https://github.com/ceph/ceph/commit/58ddc4e20f7cead1f2594241450f4beb5230c746
Improve the grammar and correct the formatting of the "Upgrading root ca
certificates" procedure that was added to the documentation in https://github.com/ceph/ceph/pull/61867
John Mulligan [Tue, 20 Aug 2024 19:01:05 +0000 (15:01 -0400)]
src/script: add a script to help build ceph using containers
The build-with-container script tries to encapsulate nearly all major
build tasks using docker/podman containers. If there's no build image
locally it will create one for your. It provides targets for building
(make), testing (make check), building rpm packages or deb packages and
is designed to be fairly easily extended.
View the comment at the top of the source file for usage details.
John Mulligan [Tue, 20 Aug 2024 19:00:57 +0000 (15:00 -0400)]
build: add files needed to create a build container
A build container contains all the tools and dependencies needed to
build ceph. It provides a Container file and small script that
helps bootstrap the container setup. This script installs a few extra
things we need before farming most of the work out to install-deps.sh.
John Mulligan [Sat, 14 Sep 2024 10:31:23 +0000 (06:31 -0400)]
build: small script tweak to allow different build dirs
Move the mkdir line to allow for other builds dir naming schemes outside
of what appears in the .gitignore file. A tiny bit of added flexibility
at little cost.
John Mulligan [Mon, 14 Nov 2022 15:57:25 +0000 (10:57 -0500)]
src/script: add helper function has_build_dir
This function returns successfully if $BUILD_DIR exists and is valid.
This is a useful building block for automation around the build and
can be used to avoid re-running commands that fail is the build dir
exists already.
John Mulligan [Tue, 1 Nov 2022 18:58:16 +0000 (14:58 -0400)]
script: add gcc-toolset-11 support to discover_compiler
In order to configure, build, and run tests in a CentOS 8 (or similar)
container we need a functioning gcc-toolset compiler. This relies on
the environment script being sourced as well. cmake does not appear
to be able to discover this compiler own its own.
John Mulligan [Tue, 1 Nov 2022 18:51:57 +0000 (14:51 -0400)]
script: add discover_compiler function to lib-build.sh
The discover_compiler function is an abstraction over the current
compiler detection code in run-make.sh. It is intended to be flexible
enough to work on {centos,rhel} systems, but currently is just an
updated version of the logic from run-make.sh. The intent is that this
function will grow and become useful for other scripts used for
building (possibly do_cmake.sh for example).
John Mulligan [Tue, 1 Nov 2022 13:57:16 +0000 (09:57 -0400)]
script: move get_processors to lib-build.sh
This function can be more useful because the NPROC value can
be provided regardless of how many cores nproc actually detects
and may be handy in a restricted environment like a container.
The new version quotes some values and uses $((...)) as per shellcheck
warning that "expr is antiquated".
John Mulligan [Mon, 31 Oct 2022 19:06:25 +0000 (15:06 -0400)]
script: add a common ci_debug function to print ci debug lines
Reduces some of the boilerplate around emitting the "CI_DEBUG:"
prefixed debug lines for the CI. Additionally, enables using
the FORCE_CI_DEBUG var to enable ci debug lines even when not
in a jenkins environment.
John Mulligan [Mon, 31 Oct 2022 17:50:56 +0000 (13:50 -0400)]
script: add lib-build.sh for common high level funcs and no main
The intention of this file is collect some of the most basic or common
shell functions used across the various build scripts.
I would also like to ensure that functions added here are validated
using `shellcheck`. Currently, there's no automation for this, just
the honor system, but eventually we can start automating validating
this and other scripts with shellcheck.
John Mulligan [Wed, 5 Oct 2022 14:19:32 +0000 (10:19 -0400)]
script: use install-deps.sh to install extra packages wanted by run-make.sh
The run-make.sh script's prepare method pulls in additional dependencies
that are needed by the CI build and tests. To avoid issues such as these
packages not being available until after install-deps.sh being run in a
container environment, we allow install-deps.sh and its new
INSTALL_EXTRA_PKGS input variable to handle all of the dependency
installation.
John Mulligan [Mon, 3 Oct 2022 19:08:30 +0000 (15:08 -0400)]
install-deps.sh: copy ubuntu/apt retry logic from run-make.sh
Copy the logic from run-make.sh into install-deps.sh so that we can later
remove it from run-make.sh. It helps prevent breakage when apt-get is
interrupted.
John Mulligan [Mon, 3 Oct 2022 18:43:19 +0000 (14:43 -0400)]
install-deps.sh: support INSTALL_EXTRA_PKGS
Instead of requiring other scripts to install packages independently,
teach install-deps.sh to install additional packages from the variable
INSTALL_EXTRA_PKGS. Now, other scripts should just set
INSTALL_EXTRA_PKGS and call install-deps.sh.
In particular, this fixes an issue installing packages in a clean (ex.
container) system that doesn't yet have repositories set up. Since this
task is performed by install-deps.sh already we avoid a chicken-and-egg
issue (or doing redundant work of setting up repos) in other scripts.
John Mulligan [Thu, 29 Sep 2022 14:34:12 +0000 (10:34 -0400)]
install-deps.sh: move functions above all "main" script body
Previously, the main part (top level body) of the script started and
then some function definitions occurred and then the main part of the
script resumed after that. I, and others, find this confusing so this
change moves the function definitions to occur before the main body of
the install-deps.sh script.
John Mulligan [Thu, 6 Oct 2022 17:43:41 +0000 (13:43 -0400)]
script: have run-make.sh honor BUILD_DIR like do_cmake.sh does
The BUILD_DIR environment variable is honored by do_cmake.sh in order to
create multiple build output directories. Before this change run-make.sh
did not support BUILD_DIR the same way as do_cmake.sh. This change makes
it possible to use BUILD_DIR with run-make.sh.
Yuval Lifshitz [Tue, 18 Feb 2025 19:09:17 +0000 (19:09 +0000)]
reef: fix issue with bucket notification test
the get_ip() function now uses the get_ip_http() implementation.
(without that test are sometimes failing with multiple RGWs).
however, the name was modified to get_ip(), and this was not updated
in all tests.
workunit/dencoder: fix corpus test for backword and forward compability
- changed the check for non-deterministic, return code 1 is also legit
- unneeded check for is_dir, if it exist
- limit the number of threads to prevent error
Fixes: https://tracker.ceph.com/issues/67263 Signed-off-by: NitzanMordhai <nmordech@redhat.com>
(cherry picked from commit 30921272ddee5e7c8aaf4bdb8d69645ce92ba379)
We are currently conducting regular ceph-dencoder tests for backward compatibility.
However, we are omitting tests for forward compatibility.
This suite will introduce tests against the ceph-objects-corpus to address forward
compatibility issues that may arise.
the script will install N-2 version and run against the latest version corpus objects
that we have, then install N-1 to N version and check them as well.
Zac Dover [Mon, 10 Feb 2025 08:12:34 +0000 (18:12 +1000)]
doc/cephadm: improve "Activate Existing OSDs".
Make three minor changes to doc/cephadm/services/osd.rst. These three
changes were suggested by Eugen Block, who reviewed this procedure after
developing it.
projects / ceph.git / log