The inter-connectedness of RadosStore and RGWRados resulted in a
segfault during RGWRados::init_complete due to the rados pointer not
being set in RadosStore yet.
Split the calls to RGWRados::initialize and RGWRados::init_complete, so
that we can set up RadosStore between them, allowing the services
created in RGWRados::init_complete to access the RadosStore.
Fixes: https://tracker.ceph.com/issues/55512 Signed-off-by: Daniel Gryniewicz <dang@redhat.com>
Rishabh Dave [Sat, 25 Sep 2021 10:33:25 +0000 (16:03 +0530)]
qa/vstart_runner: don't use os.kill for passing SIGKILL
vstart_runner.py is written assuming that it can run commands with
superuser privileges whenever possible and vstart_runner.py is meant to
be executed without sudo.
So, it's better kill a process using "sudo kill -9 <PID>", instead of
using os.kill() because os.kill() can't kill process launched with
superuser privileges.
Rishabh Dave [Sat, 25 Sep 2021 09:29:27 +0000 (14:59 +0530)]
qa/vstart_runner: get rid of "exec sudo" from "ceph -w" cmd args
Passing "exec sudo" to "ceph -w" caused "Ceph API test" CI job to fail.
Error was not related to this tracker issue but the code added for it
is reversed now in this commit. The tracker issue -
https://tracker.ceph.com/issues/49644.
Rishabh Dave [Sat, 25 Sep 2021 05:47:04 +0000 (11:17 +0530)]
qa/vstart_runner: keep cmd args as passed by user as well
We convert all cmd args to str and pass bash functions along to override
certain arguments in those command arguments. Let's save cmd args
without those bash functions since they can be useful later (for
example, printing cmd args in logs, which is the case in this patch.)
Rishabh Dave [Mon, 2 Aug 2021 17:54:52 +0000 (23:24 +0530)]
qa/vstart_runner: copy few more note points from dev guide
The intention behind copying these note points is to document the
behaviour of vstart_runner.py inside vstart_runner.py as well so that
developer don't miss it out while working on it.
Overridding commands is much better than deleting these commands from
command argument string using Python since, unlike deleting, overridding
doesn't require parsing. A note has been added for this to
vstart_runner.py's module docstring and to Ceph Developer's Guide
document.
Since functions don't work with sh shell, to make overriding work
vstart_runner.py will use bash shell here onwards.
Rishabh Dave [Tue, 13 Oct 2020 07:12:56 +0000 (12:42 +0530)]
qa/vstart_runner: accept multiple commands as str
Convert all command arguments to str from list, update checks and
adjustments performed on command arguments accordingly and update
documentation to include warnings about some critical parts of
vstart_runner.py and update tasks.cephfs.mount.MountCephFS.run_shell().
Fixes: https://tracker.ceph.com/issues/47849 Signed-off-by: Rishabh Dave <ridave@redhat.com>
Kefu Chai [Mon, 2 May 2022 03:42:17 +0000 (11:42 +0800)]
cmake/modules/BuildFIO: specify the full path to alloc libraries
before this change, we assume that "ALLOCATOR" is the library name
which can be found by linker with "ld -l <namespec>". but ideally,
cmake can find a library in a more sophiscated way used by
its "find_library()" implementation. so, in this change, instead of
relying on the default paths looked up by "ld", use the path
found by cmake.
Kefu Chai [Mon, 2 May 2022 02:46:05 +0000 (10:46 +0800)]
cmake: restructure allocator related code
before this change, the ALLOCATOR cmake option are handled at two
difference places: one to handle ALLOCATOR option, another to add
compilation options based on ALLOCATOR. after this change:
* the ALLOCATOR option is handled in a single place
* dedup the branches handling different ALLOCATORS into a single
condition: (NOT ALLOCATOR STREQUAL "libc")
* add_compile_options() calls are consolidated into a single one
Omri Zeneva [Sun, 27 Mar 2022 17:10:33 +0000 (20:10 +0300)]
rgw: support bucket name in pre request context
because bucket object is created only after authentication,
if bucket object is null upon accessing Request.Bucket.Name, we return req_state->init_state.url_bucket
doc/cephadm/services: Add missing ceph command to orch apply
In cephadm service management documentation several of the
ceph orch commands are missing the ceph part, mostly in
ceph orch apply commands but not all of them.
Add ceph in the front of the command to make them consistent
with all other commands.
Signed-off-by: Ville Ojamo <14869000+bluikko@users.noreply.github.com>
Adding LoggerSinkSet, an abstract API that enables unit-testing
of OSD components that emit cluster-log messages, and trapping the
logs.
First usage example is in the Scrubber backend unit-tests.
The log "implementation" there (note the error counter that
can be used to determine tests success):
Reformatting the OSD scrub code files to match styleguide.
Specifically:
- force 80-cols lines; and
- (sadly) force 'use tabs' (replacing each 8 indentation
blanks with a tab.
clang-format version used: 13
Configuration file used is detailed in PR comment.
Anthony D'Atri [Sat, 30 Apr 2022 07:56:21 +0000 (00:56 -0700)]
SubmittingPatches: Improve SubmittingPatches example
The example commit didn't show the convention of prefixing the message with
the relative directory path where the file lives, which has led new
contributors to innocently submit changes that aren't formatted ideally.
This adds a path to the example.
Signed-off-by: Anthony D'Atri <anthony.datri@gmail.com>
Adam Kupczyk [Fri, 29 Apr 2022 21:32:43 +0000 (23:32 +0200)]
kv/RocksDBStore: Remove feature to make WholeSpaceIterator based on bounded iterator
Iterator-bounding feature is introduced to make RocksDB iterators limited, so they
would less likely traverse over tombstones.
This is used when listing keys in fixed range, for example OMAPS for specific object.
It is problematic when extending this logic to WholeSpaceIterator,
since prefix must be taken into account.
Fixes: https://tracker.ceph.com/issues/55444 Signed-off-by: Adam Kupczyk <akupczyk@redhat.com>
This commit also explicilty sets the toolset when calling
boost's bootstrap.sh, because the latter is only capable
of autodetecting gcc if you have an unversioned `g++` in
the path, which may not be the case if only gcc11-c++ is
installed (the unversioned gcc-c++ package includes the
/usr/bin/g++ symlink, but depending on which version of
openSUSE we build on, that might point to a different gcc
version, so we don't want to use that).
Fixes: https://tracker.ceph.com/issues/55237 Signed-off-by: Tim Serong <tserong@suse.com>
but when WITH_SYSTEM_ARROW is enabled, the targets we get from
find_package() do not carry this dependency. so rgw's cmake needs to
depend on both targets
Marcus Watts [Wed, 27 Apr 2022 22:50:56 +0000 (18:50 -0400)]
qa/rgw - run sse-s3 test cases only if configured or requested
This commit adds logic to automatically detect when sse-s3 is
available and if not, disables sse-s3 tests by default.
Configuration opions are provided to override the default either way.
For debugging purposes, allow radosgw-admin to run with stores other
than RadosStore. Many operations will still fail (by crashing), so care
must be taken when running this way.
Signed-off-by: Daniel Gryniewicz <dang@redhat.com>
RGW - Allow starting RGW/dbstore without connecting to Mons
DBStore, and some other Stores like Motr, don't need to connect to the
Mons to work. However, startup automatically connects to the mons.
There's provision to not connect, but the split isn't quite right. We
need to call global_pre_init() to get config from the file, to determine
which store to start, but we then need to decide before calling
global_init() whether the configured store needs to connect to the mons.
This requires a slight change to global_init() to set no_mon_config from
the new flags.
Signed-off-by: Daniel Gryniewicz <dang@redhat.com>
Ramana Raja [Sun, 10 Apr 2022 18:21:24 +0000 (14:21 -0400)]
doc/mgr/nfs: Add commands to check the statuses
.. of NFS and ingress services after creating/deleting a NFS cluster.
The `nfs cluster info` command is not sufficient to show that the
NFS cluster is created/deleted as expected.
J. Eric Ivancich [Tue, 26 Apr 2022 16:46:08 +0000 (12:46 -0400)]
rgw: "bucket check --fix" should delete damaged multipart uploads from bi
As one of the steps in `radosgw-admin bucket check --fix ...` it looks
for bucket index entries for incomplete multipart uploads that do not
have a corresponding ".meta" entry in the same bucket index. It then
intends to delete those entries, however the function that it calls
to perform the bucket index deletions was flawed and did not direct
the removals to the appropriate shard(s), but instead a non-existant
oid.
This commit determines the appropriate shard for each of the entries
to be removed and asynchronously issues a librados call to
omap_rm_keys.
Signed-off-by: J. Eric Ivancich <ivancich@redhat.com>
Rishabh Dave [Mon, 24 Jan 2022 18:33:05 +0000 (00:03 +0530)]
qa/cephfs: don't remove sudo from the command arguments
run_shell() in qa.tasks.cephfs.mount.CephFSMount prepends "sudo" to its
command arguments but it doesn't specify to the underlying method that
"sudo" shouldn't be deleted from the command arguments.
Fixes: https://tracker.ceph.com/issues/53601 Signed-off-by: Rishabh Dave <ridave@redhat.com>
Kotresh HR [Tue, 14 Dec 2021 10:13:41 +0000 (15:43 +0530)]
qa: Fix a few tracebacks in vstart_runner
1. CommandFailedError: Command failed with status 127: \
['None/archive/coverage', 'rados' ...]
2. TypeError: a bytes-like object is required, not '_io.BytesIO'
Nizamudeen A [Tue, 26 Apr 2022 10:19:09 +0000 (15:49 +0530)]
mgr/dashboard: prometheus rules internal server error
After we increase/decrease the count of the node-exporter, we get a 500
- Internal server error from api/prometheus/rules endpoint. On further
debugging its caused by the jsonDecodder, because I guess the expected
input for the json.loads() is not a json formatted input. So to fix
that issue I can either do an error handling on the json.loads() or I
can move the json.loads() on the already existing try block. I went for
the second approach here.
Fixes: https://tracker.ceph.com/issues/54356 Signed-off-by: Nizamudeen A <nia@redhat.com>
J. Eric Ivancich [Tue, 12 Apr 2022 18:47:45 +0000 (14:47 -0400)]
rgw: address crash and race in RGWIndexCompletionManager
An atomic int was used in a modulo operator to distribute contention
among a set of locks and to track completions. Because it was an int,
enough increments would cause it to go negative (due to
twos-complement encoding and overflow) thereby causing a
crash. Additionally, even though it was atomic, the read and increment
were separate operations, leading to a race.
This commit addresses both of these issues.
Signed-off-by: J. Eric Ivancich <ivancich@redhat.com>
projects / ceph.git / log