Rishabh Dave [Sat, 30 Apr 2022 19:05:36 +0000 (00:35 +0530)]
qa/cephfs: improve test_multifs_auth.py
Remove setup_test_env() because removing this methods removes an extra
layer of abstraction which makes tests more readable. Rename
_create_client_and_keyring_file() to just _create_client() and reverse
the order of parameters in remount_with_new_client() and set default
value of cephfs_mntpt to '/'.
Rishabh Dave [Thu, 9 Jun 2022 15:09:40 +0000 (20:39 +0530)]
qa/cephfs: refactor and upgrade test_single_path_rootsquash
test_single_path_rootsquash instead of using helper methods duplicates
the code from those methods. This commit fixes that and also upgrade
this method since caps_helper was upgraded in previous commits.
Rishabh Dave [Thu, 28 Apr 2022 06:55:20 +0000 (12:25 +0530)]
qa/cephfs: add log entries for caps_helper.py
The intention is to make logs contain some information of what's being
done by caps_helper.py regardless of which test/method is calling it.
This should make logs more understandable and also add hints for
future debugging.
With this commit, the design of run_mon_cap_tests() is now aligned with
rest of CapTester -- it's not meant to be inherited by test class
anymore and it is to be called by using CapTester instance.
This commit also changes working of this method. Now instead of
obtaining FS names from Python objects representing FSs, it obtains
those names from MON cap. This removes the need to pass the FS objects
around.
Rishabh Dave [Sat, 30 Apr 2022 09:14:19 +0000 (14:44 +0530)]
qa/cephfs: make file data unique in caps_helper.py
Let's change the file data to include the file path, the CephFS name
and the host FS and CephFS mountpoint so that the bugs where, let's say,
"cephfs2" is mounted instead of "cephfs1" (where obviously both the
CephFSs lie on the same Ceph cluster) is caught by the tests due to
uniqueness of every test file's content.
Rishabh Dave [Thu, 28 Apr 2022 16:31:25 +0000 (22:01 +0530)]
qa/cephfs: simplify caps_helper.CapTester
Simplify methods in CapsTester by adding a test set, which will be a
list of tuples. The first element in tuple will be the mount object,
the second will be the test file path and the third will be the test
file data. Thus instead of having three independent class variables
that are always used together now we have list of test sets making
management of multiple test sets simpler.
Rishabh Dave [Thu, 28 Apr 2022 06:31:32 +0000 (12:01 +0530)]
qa/cephfs: upgrade and rename class CapsHelper
Modify caps_helper.py such that calling code does't have to store
returned values just to pass those values again to a method in
caps_helper.py. This is a common pattern where write_test_files()
return 3 values and all 3 passed as it is to run_cap_tests().
The easy way to do it is to upgrade caps_helper so that it can be used
an object and not just as a class supplying helper methods. The return
values will be stored by the object internally and thus resued. In case
of testing multiple FSs inside a single test method, we'll now need
multiple instances of this class to keep those values separate.
And since CapsHelper is not just a class supplying helper methods
anymore, it's being renamed to CapTester.
When CephFS mountpoint is changed inside a single test method, the test
files created at root are neither accessible nor useful. Therefore, add
an option to create the test files in the directory passed by the user.
This also increases general flexibility of the concerned method.
crimson/os/seastore/async_cleaner: don't trim backref when reclaiming space
Since the current backref cache doesn't invalidate duplicated backrefs any more
and backrefs get trimmed in the exact order of journal seqs, there's no need to trim
backrefs when reclaiming space
librbd: bail from schedule_request_lock() if already lock owner
Race condition may be hit if there are multiple pending locks for the
same image and pending callbacks. Abort exclusive lock process if
already exclusive lock owner.
Fixes: https://tracker.ceph.com/issues/56549 Signed-off-by: Christopher Hoffman <choffman@redhat.com>
Matt Benjamin [Tue, 5 Jul 2022 22:33:09 +0000 (18:33 -0400)]
rgwlc: activate lifecycle processing on non-master zones
The basic idea of this change is the same as the proposal by
Ilsoo Byun <ilsoobyun@linecorp.com>, but some details have changed.
The main differences are to use the existing
RGWLC::set(remove)_bucket_config methods, and to use the
RGWBucketInstanceMetadataHandler infrastructue to dispatch
the corresponding calls. Thank you!
Fixes: https://tracker.ceph.com/issues/44268
Related PR: #33524
Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
Matt Benjamin [Sat, 20 Nov 2021 18:45:51 +0000 (13:45 -0500)]
rgwlc: introduce lifecycle config flags extension
rgwlc: add uint32_t flags bitmap to LCFilter
This is intended to support a concise set of extensions to S3
LifecycleConfiguration, initially, just a flag that indicates a
rule is intended for execution on RGW ArchiveZone.
rgwlc: add machinery to define and recognize LCFilter flags
Add a concept of filter flags to lifecycle filter rules, an RGW
extension. The initial purpose of flags is to permit marking
specific lifecycle rules as specific to an RGW archive zone, but
other flags could be added in future.
rgwlc: add new unittest_rgw_lc to run internal checks, add a few
valid and invalid lifecycle configuration xml parses for now.
Fixes: https://tracker.ceph.com/issues/53361 Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
Ilya Dryomov [Fri, 20 May 2022 12:05:03 +0000 (14:05 +0200)]
qa/suites/rbd: disable workunit timeout for dynamic_features_no_cache
The I/O workload in this test is xfstests (qa/run_xfstests_qemu.sh)
which isn't subjected to any timeout other than global max_job_time
limit in any other subsuite (e.g. qemu/workloads/qemu_xfstests.yaml).
But here, there is a parallel "op" workload defined as a workunit.
The workunit task has a default timeout of 3 hours which is effectively
imposed on the entire job. In the "rbd cache = false" configuration,
it's sometimes exceeded.
rbd: don't default empty pool name unless namespace is specified
Commit 96f05a7956b3 ("rbd: delay determination of default pool name")
broke "rbd perf image iostat" and "rbd perf image iotop" GLOBAL_POOL_KEY
support (the ability to blend all rbd pools together into a single
view).
It doesn't really thrash anything, just repeatedly restarts the
workload on top of a dirty cache file. rbd_pwl_cache_recovery is
more on point and gets covered by existing CODEOWNERS.
* make paxos_size() unsigned, as paxos_size() returns the size of
MonMap::mon_info, so it should be always a non-negative value,
and more importantly, it represents a size.
* change the type of MonMap::removed_ranks from std::set<int>
to std::set<unsigned>. for two reasons:
- removed_ranks only tracks the rank which is greater or equal to 0
- helps to silence the warnings listed below.
MonMap::removed_ranks is persisted using encode()/decode(), but this
change is backward compatible, as we use the raw encoder to encode
signed and unsigned integers, the difference between the encoding
schema between them only matters when MSB in the number is used,
but this is not likely happen, as we neither have a negative
rank in removed_ranks, no have a rank greater than `(unsigned)-1`,
i.e., 0xffffffff.
/home/kefu/dev/ceph/src/mon/ElectionLogic.cc: In member function ‘void ElectionLogic::end_election_period()’:
/home/kefu/dev/ceph/src/mon/ElectionLogic.cc:173:23: error: comparison of integer expressions of different signedness: ‘std::set<int>::size_type’ {aka ‘long unsigned int’} and ‘int’ [-Werror=sign-compare]
173 | acked_me.size() > (elector->paxos_size() / 2)) {
| ~~~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/home/kefu/dev/ceph/src/mon/ElectionLogic.cc: In member function ‘void ElectionLogic::propose_connectivity_handler(int, epoch_t, const ConnectionTracker*)’:
/home/kefu/dev/ceph/src/mon/ElectionLogic.cc:338:28: error: comparison of integer expressions of different signedness: ‘unsigned int’ and ‘int’ [-Werror=sign-compare]
338 | for (unsigned i = 0; i < elector->paxos_size(); ++i) {
| ~~^~~~~~~~~~~~~~~~~~~~~~~
/home/kefu/dev/ceph/src/mon/ElectionLogic.cc: In member function ‘void ElectionLogic::receive_ack(int, epoch_t)’:
/home/kefu/dev/ceph/src/mon/ElectionLogic.cc:469:25: error: comparison of integer expressions of different signedness: ‘std::set<int>::size_type’ {aka ‘long unsigned int’} and ‘int’ [-Werror=sign-compare]
469 | if (acked_me.size() == elector->paxos_size()) {
| ~~~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~
cc1plus: all warnings being treated as errors
make[3]: *** [src/mon/CMakeFiles/mon.dir/build.make:328: src/mon/CMakeFiles/mon.dir/ElectionLogic.cc.o] Error 1
make[3]: *** Waiting for unfinished jobs....
make[3]: Leaving directory '/home/kefu/dev/ceph/build'
[ 48%] Built target libglobal_objs
/home/kefu/dev/ceph/src/mon/Elector.cc: In member function ‘void Elector::notify_rank_removed(int)’:
/home/kefu/dev/ceph/src/mon/Elector.cc:734:43: error: comparison of integer expressions of different signedness: ‘unsigned int’ and ‘int’ [-Werror=sign-compare]
734 | for (unsigned i = rank_removed + 1; i <= paxos_size() ; ++i) {
| ~~^~~~~~~~~~~~~~~
myoungwon oh [Tue, 28 Jun 2022 04:42:21 +0000 (13:42 +0900)]
osd: return ENOENT if pool information is invalid during tier-flush
During tier-flush, OSD sends reference increase message to target OSD.
At this point, sending message with invalid pool information (e.g., deleted pool)
causes unexpected behavior.
Therefore, this commit return ENOENT early before sending the message
Currently, the following transaction exec sequence would lead to
loss of backref:
1. Trans `A` merge a alloc backref for extent `X`
2. Trans `B` add a release backref for extent `X` to backref cache,
during which it finds an in-cache alloc backref for extent `X` and
decide not to add the release backref to cache
3. Trans `A` commit
In the above sequece, the release backref for extent `X` is lost.
This is a regression introduced when we try to optimize the backref cache.
This commit fix the issue by caching inflight backrefs in a multiset,
alloc/release ops that happen on the same paddr are queued in the order of
their happening. When doing gc, all those backrefs are merged.
projects / ceph.git / log