Mark Kogan [Wed, 25 Jun 2025 12:21:49 +0000 (12:21 +0000)]
qa/rgw: fix perl tests missing Amazon::S3 module
and a second case where perl tests can fail without error output
1. fix errors like: `Can't locate Amazon/S3.pm in @INC (you may need to
install the Amazon::S3 module)`
by priming the perl tests with installing the Amazon::S3 module from cpan
ex:
```
2025-06-23T19:18:40.162 INFO:tasks.workunit.client.0.smithi090.stderr:Can't locate Amazon/S3.pm in @INC (you may need to install the Amazon::S3 module) (@INC contains: /usr/local/lib64/perl5/5.32 ...
```
Ronen Friedman [Wed, 25 Jun 2025 14:25:08 +0000 (09:25 -0500)]
osd/scrub: some perf counters priority was '0'
Some scrub perf counters were created without specifying
individual priorities, assuming by mistake that the
default priority is '_INTERESTING'. That was not the case,
and those perf counters were not reported.
Zac Dover [Mon, 23 Jun 2025 08:18:07 +0000 (18:18 +1000)]
doc/radosgw: remove "pubsub_event_lost"
Remove "pubsub_event_lost" from the list of "Notification Performance
Statistics" in doc/radosgw/notifications.rst. "pubsub_event_lost" is now
obsolete.
Yuval Lifshitz [Mon, 16 Jun 2025 11:05:25 +0000 (11:05 +0000)]
rgw/logging: make unique part of log file both random and incremental
new format will be: 10 char incremental count (so 32bit uint fit in it).
and 6 char alphanumeric random part.
this should fix possible race conditions in case of multisite
Yuval Lifshitz [Tue, 3 Jun 2025 10:30:46 +0000 (10:30 +0000)]
rgw/logging: return the last object name that was actually comitted
when comitting a pending object that was never created we should
not reply the object name as the name of the comitted object.
instead, we should return the name of the object that was actuaslly
comitted.
Venky Shankar [Tue, 27 May 2025 07:26:12 +0000 (07:26 +0000)]
qa/cephfs: ignore `OSD_DOWN/osds down` warning
Runs have started failing a lot with the human friendly variant
of the warning. OSD_DOWN is in the ignore list, however, the human
friendly warning (osds down) isn't.
Kefu Chai [Sat, 14 Jun 2025 13:44:05 +0000 (21:44 +0800)]
cls/rbd: use default values for non-decoded fields in test instances
Previously, test instances for cls_rbd_snap used non-default values
for the "parent" field, which is ignored during decoding. The
check-generated.sh test passed because they reused the same instance
for re-encoding, preserving undecoded fields.
An upcoming change will allocate new instances for each encode/decode
verification instead of reusing instances. This will expose
discrepancies between original test instances and re-encoded values
when fields contain non-default values but aren't decoded.
This change sets ignored fields to their default values in test
instances, ensuring consistency between encoding and decoding
operations regardless of the verification approach used.
Since the incompatibility of cls_rbd_snap's on-disk format was
introduced in 32b14ed1, which was introduced Ceph v14, we will
mark this version the first incompatible version in ceph-object-corpus
in the sense that the re-encoded cls_rbd_snap with v8 struct version
is different from the original copy if its parent field is set with
< v8 struct version.
Kefu Chai [Wed, 7 May 2025 00:42:52 +0000 (08:42 +0800)]
librbd, tools: migrate from boost::variant to std::variant
Complete migration started in commit 017f333, replacing boost::variant with
std::variant throughout the librbd codebase. This change is part of our ongoing
effort to reduce third-party dependencies by leveraging C++ standard library
alternatives where possible.
Benefits include:
- Improved code readability and maintainability
- Reduced external dependency surface
- More consistent API usage with other components
Implementation note: Unlike Boost.variant, std::variant lacks built-in
operator<< support. This commit implements the necessary operator<< for
AttributeValue, our specific std::variant instantiation, to preserve the
existing behavior.
Also, despite that `apply_visit()` calls can be replaced with `visit()`
without being qualified with `std::` because of ADL, we are taking this
opportunity to adding the `std::` prefix for better readability.
Ramana Raja [Tue, 13 May 2025 16:37:52 +0000 (12:37 -0400)]
pybind/mgr/dashboard: fetch image's mirror mode
... only if the image is not disabled for mirroring.
If the image is disabled for mirroring, fetching the image's
mirroring mode is invalid. So validate that the image is not disabled
for mirroring before fetching the mirroring mode.
Ramana Raja [Tue, 6 May 2025 00:07:18 +0000 (20:07 -0400)]
pybind/mgr/rbd_support: check whether mirroring is enabled
... before fetching the mirroring mode of the image.
In the CreateSnapshotRequests class, which asynchronously issues mirror
snapshot creation requests, prevalidation includes checking that the
image is enabled for snapshot-based mirroring and is marked as primary.
Since mirroring mode can only be queried if mirroring is already
enabled, the code first fetches the image’s mirroring info to verify
that mirroring is enabled, and only then retrieves the mirroring mode.
Ramana Raja [Tue, 6 May 2025 20:19:09 +0000 (16:19 -0400)]
librbd/api/Mirror: return EINVAL from image_resync()
... when mirroring is not enabled for the image.
Mirror::image_resync() returns ENOENT when mirroring is disabled for the
image. Instead, make it return EINVAL indicating that the call is
invalid when mirroring is not enabled for the image. This also causes
the public facing C, C++, and Python APIs that resync an image to
return EINVAL or raise an equivalent exception when mirroring is not
enabled for the image.
Ramana Raja [Mon, 5 May 2025 23:37:42 +0000 (19:37 -0400)]
librbd/mirror/PromoteRequest: return EINVAL
... instead of ENOENT when mirroring is not enabled for the image.
The PromoteRequest async state machine returns ENOENT when mirroring is
not enabled for the image. Instead, make it return EINVAL similar to
DemoteRequest's behavior, which is more appropriate. This also causes
the public facing C, C++, and Python APIs that promote an image
to return EINVAL or raise an equivalent exception when mirroring is
not enabled for the image.
Ramana Raja [Mon, 5 May 2025 17:31:34 +0000 (13:31 -0400)]
librbd/api/Mirror: return EINVAL from image_get_mode()
... when the image is disabled for mirroring.
When an image is disabled for mirroring, fetching the image's
mirroring mode is invalid. So, modify the Mirror::image_get_mode()
internal API to return EINVAL instead of success when mirroring is
disabled.
The Mirror::image_get_mode() method is called by the public C++, C, and
Python APIs that fetch the mirroring mode of an image. The behavior of
these public APIs will change. They will return an error code or raise
an exception indicating that it's an invalid operation to fetch the
image's mirroring mode when mirroring is disabled.
Ramana Raja [Mon, 5 May 2025 18:29:08 +0000 (14:29 -0400)]
tools/rbd/action/MirrorPool: remove dead branch
mirror_image_get_info() API doesn't fail with ENOENT when mirroring is
disabled since commit c9c8852. So, no need to handle ENOENT error from
mirror_image_get_info() API.
VinayBhaskar-V [Thu, 15 May 2025 14:18:30 +0000 (19:48 +0530)]
librbd: retry list_snap_orders() once instead of failing sort_snaps()
If snapshot listing races with snapshot creation, rbd group snap ls
may spuriously return an unsorted listing and internal fail_if_not_sorted=true
consumers may generate a spurious EINVAL error. This is because snap orders
are listed before snaps themselves and the "missing order for snap" check in
sort_snaps() is driven by the list of snaps.
This can be improved by grabbing snap order keys one more time, adding the newly
discovered snap orders to m_snap_orders and retrying instead of immediately failing the sort.
Fixes: https://tracker.ceph.com/issues/67984 Signed-off-by: VinayBhaskar-V <vvarada@redhat.com>
(cherry picked from commit 3c21aec85e986f5206e83fcdcf5b1e0441b35f56)
Kefu Chai [Tue, 3 Jun 2025 08:07:33 +0000 (16:07 +0800)]
librbd/cache/pwl: fix memory leak in SyncPoint persist context cleanup
Previously, SyncPoint allocated two C_Gather instances tracked by raw
pointers but failed to properly clean them up when only a single sync
point existed, causing memory leaks detected by AddressSanitizer.
This change fixes the leak by modifying AbstractWriteLog::shut_down()
to check for prior sync points in the chain. When the current sync point
is the only one present, we now activate the m_prior_log_entries_persisted
context to ensure:
- The onfinish callback executes and releases the captured strong
reference to the enclosing SyncPoint
- The parent m_sync_point_persist context completes and gets properly
released
This ensures all allocated contexts are cleaned up correctly during
shutdown, eliminating the memory leak.
The ASan report:
```
Indirect leak of 2064 byte(s) in 1 object(s) allocated from:
#0 0x56440919ae2d in operator new(unsigned long) (/home/jenkins-build/build/workspace/ceph-pull-requests/build/bin/unittest_librbd+0x2f3de2d) (BuildId: 6a04677c6ee5235f1a41815df807f97c5b96d4cd)
#1 0x56440bd67751 in __gnu_cxx::new_allocator<Context*>::allocate(unsigned long, void const*) /usr/lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/ext/new_allocator.h:127:27
#2 0x56440bd676e0 in std::allocator<Context*>::allocate(unsigned long) /usr/lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/allocator.h:185:32
#3 0x56440bd676e0 in std::allocator_traits<std::allocator<Context*>>::allocate(std::allocator<Context*>&, unsigned long) /usr/lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/alloc_traits.h:464:20
#4 0x56440bd6730b in std::_Vector_base<Context*, std::allocator<Context*>>::_M_allocate(unsigned long) /usr/lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/stl_vector.h:346:20
#5 0x7fd33e00e8d1 in std::vector<Context*, std::allocator<Context*>>::reserve(unsigned long) /usr/lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/vector.tcc:78:22
#6 0x7fd33e00c51c in librbd::cache::pwl::SyncPoint::SyncPoint(unsigned long, ceph::common::CephContext*) /home/jenkins-build/build/workspace/ceph-pull-requests/src/librbd/cache/pwl/SyncPoint.cc:20:27
#7 0x56440bd65f26 in decltype(::new((void*)(0)) librbd::cache::pwl::SyncPoint(std::declval<unsigned long&>(), std::declval<ceph::common::CephContext*&>())) std::construct_at<librbd::cache::pwl::SyncPoint, unsigned long&, ceph::common::CephContext*&>(librbd::cache::pwl::SyncPoint*, unsigned long&, ceph::common::CephContext*&) /usr/lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/stl_construct.h:97:39
#8 0x56440bd65b98 in void std::allocator_traits<std::allocator<librbd::cache::pwl::SyncPoint>>::construct<librbd::cache::pwl::SyncPoint, unsigned long&, ceph::common::CephContext*&>(std::allocator<librbd::cache::pwl::SyncPoint>&, librbd::cache::pwl::SyncPoint*, unsigned long&, ceph::common::CephContext*&) /usr/lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/alloc_traits.h:518:4
#9 0x56440bd657d3 in std::_Sp_counted_ptr_inplace<librbd::cache::pwl::SyncPoint, std::allocator<librbd::cache::pwl::SyncPoint>, (__gnu_cxx::_Lock_policy)2>::_Sp_counted_ptr_inplace<unsigned long&, ceph::common::CephContext*&>(std::allocator<librbd::cache::pwl::SyncPoint>, unsigned long&, ceph::common::CephContext*&) /usr/lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/shared_ptr_base.h:519:4
#10 0x56440bd65371 in std::__shared_count<(__gnu_cxx::_Lock_policy)2>::__shared_count<librbd::cache::pwl::SyncPoint, std::allocator<librbd::cache::pwl::SyncPoint>, unsigned long&, ceph::common::CephContext*&>(librbd::cache::pwl::SyncPoint*&, std::_Sp_alloc_shared_tag<std::allocator<librbd::cache::pwl::SyncPoint>>, unsigned long&, ceph::common::CephContext*&) /usr/lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/shared_ptr_base.h:651:6
#11 0x56440bd65163 in std::__shared_ptr<librbd::cache::pwl::SyncPoint, (__gnu_cxx::_Lock_policy)2>::__shared_ptr<std::allocator<librbd::cache::pwl::SyncPoint>, unsigned long&, ceph::common::CephContext*&>(std::_Sp_alloc_shared_tag<std::allocator<librbd::cache::pwl::SyncPoint>>, unsigned long&, ceph::common::CephContext*&) /usr/lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/shared_ptr_base.h:1342:14
#12 0x56440bd650e6 in std::shared_ptr<librbd::cache::pwl::SyncPoint>::shared_ptr<std::allocator<librbd::cache::pwl::SyncPoint>, unsigned long&, ceph::common::CephContext*&>(std::_Sp_alloc_shared_tag<std::allocator<librbd::cache::pwl::SyncPoint>>, unsigned long&, ceph::common::CephContext*&) /usr/lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/shared_ptr.h:409:4
#13 0x56440bd65057 in std::shared_ptr<librbd::cache::pwl::SyncPoint> std::allocate_shared<librbd::cache::pwl::SyncPoint, std::allocator<librbd::cache::pwl::SyncPoint>, unsigned long&, ceph::common::CephContext*&>(std::allocator<librbd::cache::pwl::SyncPoint> const&, unsigned long&, ceph::common::CephContext*&) /usr/lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/shared_ptr.h:862:14
#14 0x56440bca97e7 in std::shared_ptr<librbd::cache::pwl::SyncPoint> std::make_shared<librbd::cache::pwl::SyncPoint, unsigned long&, ceph::common::CephContext*&>(unsigned long&, ceph::common::CephContext*&) /usr/lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/shared_ptr.h:878:14
#15 0x56440bd443c8 in librbd::cache::pwl::AbstractWriteLog<librbd::MockImageCtx>::new_sync_point(librbd::cache::pwl::DeferredContexts&) /home/jenkins-build/build/workspace/ceph-pull-requests/src/librbd/cache/pwl/AbstractWriteLog.cc:1905:20
#16 0x56440bd42e4c in librbd::cache::pwl::AbstractWriteLog<librbd::MockImageCtx>::flush_new_sync_point(librbd::cache::pwl::C_FlushRequest<librbd::cache::pwl::AbstractWriteLog<librbd::MockImageCtx>>*, librbd::cache::pwl::DeferredContexts&) /home/jenkins-build/build/workspace/ceph-pull-requests/src/librbd/cache/pwl/AbstractWriteLog.cc:1951:3
#17 0x56440bd9cbf2 in librbd::cache::pwl::AbstractWriteLog<librbd::MockImageCtx>::flush_new_sync_point_if_needed(librbd::cache::pwl::C_FlushRequest<librbd::cache::pwl::AbstractWriteLog<librbd::MockImageCtx>>*, librbd::cache::pwl::DeferredContexts&) /home/jenkins-build/build/workspace/ceph-pull-requests/src/librbd/cache/pwl/AbstractWriteLog.cc:1990:5
#18 0x56440bd9c636 in librbd::cache::pwl::AbstractWriteLog<librbd::MockImageCtx>::internal_flush(bool, Context*)::'lambda'(librbd::cache::pwl::GuardedRequestFunctionContext&)::operator()(librbd::cache::pwl::GuardedRequestFunctionContext&) const /home/jenkins-build/build/workspace/ceph-pull-requests/src/librbd/cache/pwl/AbstractWriteLog.cc:2152:9
#19 0x56440bd9b9b4 in boost::detail::function::void_function_obj_invoker<librbd::cache::pwl::AbstractWriteLog<librbd::MockImageCtx>::internal_flush(bool, Context*)::'lambda'(librbd::cache::pwl::GuardedRequestFunctionContext&), void, librbd::cache::pwl::GuardedRequestFunctionContext&>::invoke(boost::detail::function::function_buffer&, librbd::cache::pwl::GuardedRequestFunctionContext&) /opt/ceph/include/boost/function/function_template.hpp:100:11
#20 0x56440bd29321 in boost::function_n<void, librbd::cache::pwl::GuardedRequestFunctionContext&>::operator()(librbd::cache::pwl::GuardedRequestFunctionContext&) const /opt/ceph/include/boost/function/function_template.hpp:789:14
#21 0x56440bd28d85 in librbd::cache::pwl::GuardedRequestFunctionContext::finish(int) /home/jenkins-build/build/workspace/ceph-pull-requests/src/librbd/cache/pwl/Request.h:335:5
#22 0x5644091e0fe0 in Context::complete(int) /home/jenkins-build/build/workspace/ceph-pull-requests/src/include/Context.h:102:5
#23 0x56440bd9b378 in librbd::cache::pwl::AbstractWriteLog<librbd::MockImageCtx>::detain_guarded_request(librbd::cache::pwl::C_BlockIORequest<librbd::cache::pwl::AbstractWriteLog<librbd::MockImageCtx>>*, librbd::cache::pwl::GuardedRequestFunctionContext*, bool) /home/jenkins-build/build/workspace/ceph-pull-requests/src/librbd/cache/pwl/AbstractWriteLog.cc:1202:20
#24 0x56440bd96c50 in librbd::cache::pwl::AbstractWriteLog<librbd::MockImageCtx>::internal_flush(bool, Context*) /home/jenkins-build/build/workspace/ceph-pull-requests/src/librbd/cache/pwl/AbstractWriteLog.cc:2154:3
#25 0x56440bd1e4b5 in librbd::cache::pwl::AbstractWriteLog<librbd::MockImageCtx>::shut_down(Context*) /home/jenkins-build/build/workspace/ceph-pull-requests/src/librbd/cache/pwl/AbstractWriteLog.cc:703:3
#26 0x56440bdb9022 in librbd::cache::pwl::TestMockCacheSSDWriteLog_compare_and_write_compare_matched_Test::TestBody() /home/jenkins-build/build/workspace/ceph-pull-requests/src/test/librbd/cache/pwl/test_mock_SSDWriteLog.cc:403:7
```
Venky Shankar [Tue, 20 May 2025 12:19:41 +0000 (12:19 +0000)]
client: do not check file size when inode does not have Fc caps
Since the client is holding Fr caps, the read request can be
directly sent to the OSD. The offset/in->size comparison check
is causing the read request to return with no data since in->size
isn't yet updated when another client does an extending write.
Venky Shankar [Fri, 30 May 2025 18:11:19 +0000 (18:11 +0000)]
client: asynchronous fsync can decrement request ref twice
After the asynchronous execution context is woken up when waiting
for Fb caps reference to be released causing the clien to crash
as per:
```
0x00007f3115b2452c in __pthread_kill_implementation () from /lib64/libc.so.6
0x00007f3115ad7686 in raise () from /lib64/libc.so.6
0x00007f3115ac1833 in abort () from /lib64/libc.so.6
0x00007f3113375d0a in ceph::__ceph_assert_fail (assertion=<optimized out>, file=<optimized out>, line=<optimized out>, func=<optimized out>) at /usr/src/debug/ceph-19.2.0-124.el9cp.x86_64/src/common/assert.cc:74
0x00007f3113375e6f in ceph::__ceph_assert_fail (ctx=...) at /usr/src/debug/ceph-19.2.0-124.el9cp.x86_64/src/common/assert.cc:79
0x00007f311237db1d in xlist<MetaRequest*>::item::~item (this=<optimized out>, this=<optimized out>) at /usr/src/debug/ceph-19.2.0-124.el9cp.x86_64/src/include/xlist.h:31
MetaRequest::~MetaRequest (this=<optimized out>, this=<optimized out>) at /usr/src/debug/ceph-19.2.0-124.el9cp.x86_64/src/client/MetaRequest.cc:65
Client::put_request (this=0x564b491726c0, request=0x7f301c0165c0) at /usr/src/debug/ceph-19.2.0-124.el9cp.x86_64/src/client/Client.cc:2140
0x00007f31123c88ad in Client::C_nonblocking_fsync_state::advance (this=0x7f307002e9f0) at /usr/src/debug/ceph-19.2.0-124.el9cp.x86_64/src/client/Client.cc:11905
0x00007f3112331ccd in Context::complete (this=0x7f3070009250, r=<optimized out>) at /usr/src/debug/ceph-19.2.0-124.el9cp.x86_64/src/include/Context.h:99
0x00007f311246a964 in Client::signal_context_list(std::__cxx11::list<Context*, std::allocator<Context*> >&) [clone .constprop.0] (ls=std::__cxx11::list = {...}, this=<optimized out>)
at /usr/src/debug/ceph-19.2.0-124.el9cp.x86_64/src/client/Client.cc:4257
0x00007f3112395f45 in Client::put_cap_ref (this=0x564b491726c0, in=0x7f306807be90, cap=<optimized out>) at /usr/src/debug/ceph-19.2.0-124.el9cp.x86_64/src/client/Client.cc:3611
0x00007f31123331f3 in Client::C_Write_Finisher::finish_io (r=0, this=0x7f30240442d0) at /usr/src/debug/ceph-19.2.0-124.el9cp.x86_64/src/client/Client.cc:11381
Client::CWF_iofinish::finish (this=<optimized out>, r=0) at /usr/src/debug/ceph-19.2.0-124.el9cp.x86_64/src/client/Client.h:1481
0x00007f3112331ccd in Context::complete (this=0x7f302401afd0, r=<optimized out>) at /usr/src/debug/ceph-19.2.0-124.el9cp.x86_64/src/include/Context.h:99
0x00007f31123c5242 in Client::C_Lock_Client_Finisher::finish (this=0x7f302403c9d0, r=0) at /usr/src/debug/ceph-19.2.0-124.el9cp.x86_64/src/client/Client.cc:11372
0x00007f3112331ccd in Context::complete (this=0x7f302403c9d0, r=<optimized out>) at /usr/src/debug/ceph-19.2.0-124.el9cp.x86_64/src/include/Context.h:99
0x00007f31134374ad in Finisher::finisher_thread_entry (this=0x564b491730b0) at /usr/src/debug/ceph-19.2.0-124.el9cp.x86_64/src/common/Finisher.cc:72
0x00007f3115b227e2 in start_thread () from /lib64/libc.so.6
0x00007f3115ba7800 in clone3 () from /lib64/libc.so.6
0x0000000000000000 in ?? ()
```
Shachar Sharon [Tue, 22 Oct 2024 12:06:54 +0000 (15:06 +0300)]
client: fix memory leak in Client::CRF_iofinish::complete
Commit 1210ddf7a ("Client: Add non-blocking helper classes") introduced
Client::C_Read_Finisher Context object for async READ operations, but
it has a read-after-free bug which may cause memory leak when calling
libcephf's non-blocking ceph_ll_nonblocking_readv_writev API with async
READ:
ceph_ll_nonblocking_readv_writev (READ)
Client::ll_preadv_pwritev
...
Client::_read_async
Context::complete
Client::CRF_iofinish::complete
Client::CRF_iofinish::finish
CRF->finish_io()
Client::C_Read_Finisher::finish_io
...
delete this; // frees CRF_iofinish->CRF
if (CRF->iofinished) // use-after-free of CRF
delete this; // may not get here
A possible memory leak depends on timing and race with other thread
allocation which alters the memory address of CRF->iofinished to
false, thus skipping the last delete operation.
The check of `if (CRF->iofinished)` is unnecessary: it is always set to
true upon calling CRF->finish_io(). Thus, there is no need to have the
override function Client::CRF_iofinish::complete() as it now has the
same logic as Context::complete(). Removed.
Ronen Friedman [Thu, 19 Jun 2025 15:27:38 +0000 (10:27 -0500)]
osd/scrub: clarify that osd_scrub_auto_repair_num_errors counts objects
'osd_scrub_auto_repair_num_errors' limits the number of damaged objects
that we will try to auto-repair during a scrub. Its documentation
referred to "number of errors", which did not fit the implementation.
Fixes: https://tracker.ceph.com/issues/71754 Fixes: Red Hat BZ2316244 Signed-off-by: Ronen Friedman <rfriedma@redhat.com>
(cherry picked from commit 680b58ffd0bf5b213ec525f8d783297fb0b14343)
Shraddha Agrawal [Thu, 29 May 2025 10:10:01 +0000 (15:40 +0530)]
qa/standalone/mon/availability.sh: add test for config option
This commit adds two tests, first, to ensure we get an error
message when the feature is disabled. It checks if the config
option, enable_availability_tracking is working properly.
Second test ensures that we actually do stop calculating the
score when the feature is disabled.
Shraddha Agrawal [Thu, 22 May 2025 10:26:41 +0000 (15:56 +0530)]
mon/MgrStatMonitor: ignore duration for which feature is off
When the availability tracking feature is disabled, we should not
be updating the score. We should start recalculating the score
when the user enables the features again. Essentially, for the
purpose of calculating the score, we need to ignore the duration
for which the feature was turned off.
The score is calculated from the uptime and downtime durations
recorded in `pool_availability` object. These durations are updated
in `calc_pool_availability` by adding the diff between last_uptime/
last_downtime and now.
To discard the duration for which the feature was turned off, we
need to offset the uptime/downtime by this duration. A simple way
to do this is to update the last_uptime and last_downtime to the
timestamp when the feature is toggled on again. To implement the
same, we record the time at which the feature is toggled from off
to on. When `calc_pool_availability` is invoked, if a reset is
required, it resets last_uptime and last_downtime before proceeding
with availability calculations.
We only care about the state when the feature is toggled from off to
on. All other toggle states for the config option will not have any
effect on the score.
Shraddha Agrawal [Thu, 22 May 2025 09:16:50 +0000 (14:46 +0530)]
MgrStatMonitor: add config observer
This commit adds a config observer to MgrStatMonitor so we
can track when a user enables/disables enable_availability_tracking
config option. The time difference between disabling and then
enabling the config option will be used to offset the uptime
and/or downtime from the availability score feature.
Shraddha Agrawal [Thu, 22 May 2025 08:20:57 +0000 (13:50 +0530)]
mon/MgrStatMonitor.cc: do not update score when disabled
This commit adds changes to ensure the availability score
tracking is not updated when the feature is disabled. We
will preserve the score calculated before the feature is
turned off and start updating it again when the feature
is enabled.
src/common/options: add config option for availability score
This commit modifies src/common/options/mon.yaml.in to add a
new config option to enable/disable tracking availability
score. This config option can be modified dynamically at
runtime as well.
To enable tracking availability score, we can run the
following command:
ceph config set mon enable_availability_tracking true
By default, tracking availability score is enabled.
To disable tracking availability score:
ceph config set mon enable_availablity_tracking false
When the feature is turned off, invoking the
`availability-status` command will display an error, prompting
the user to turn on the feature using the config option.
Naveen Naidu [Thu, 29 May 2025 08:58:32 +0000 (14:28 +0530)]
osd/PeeringState: handle race condition of DeferBackfill event for Backfilling state
Currently when PG in `Backfilling` state receives a `DeferBackfill`
event, there are cases when that event could race with
`MOSDPGBackfill::OP_BACKFILL_FINISH` becasue the PG has already
finished backfilling. In such case, the following
happens:
1. PG state set to `PG_STATE_BACKFILL_WAIT`
2. Suspend backfilling
3. Discard the event
Notice that we do not reschedule backfill in the above steps, this can
lead to a situation where the PG gets stuck in a `backfill_wait` state
forever. This bug got introduced due to the following commit:
`865839f`: osd/PeeringState: check racing with OP_BACKFILL_FINISH when defering
backfill Link: https://github.com/ceph/ceph/pull/60185
This commit, fixes that by making sure that in race conditions such as
above - we only discard the event.
Add an instruction that includes the --enable-auth flag in a "git orch
apply mgmt-gateway" command, in accordance with a request made by
afreen23 here: https://github.com/ceph/ceph/pull/60440#discussion_r1953530599
projects / ceph.git / log