Adam Kupczyk [Tue, 1 Apr 2025 14:01:23 +0000 (14:01 +0000)]
os/bluestore/bluefs: Fix race condition between truncate() and unlink()
It was possible for unlink() to interrupt ongoing truncate().
As the result, unlink() finishes properly, but truncate() is not aware
of it and does:
1) updates file that is already removed
2) releases same allocations again
Now fixed by checking if file is deleted under FILE lock.
Adam Kupczyk [Mon, 31 Mar 2025 20:49:26 +0000 (20:49 +0000)]
test/store_test: Expose race in BlueFS truncate / remove
Created test that exposes race between BlueFS::truncate and BlueFS::unlink.
Test requires injection of 1ms sleep to BlueFS::truncate.
Therefore, in this form, it is unsuitable for merge.
Kefu Chai [Sun, 30 Mar 2025 03:59:12 +0000 (11:59 +0800)]
cephfs-top: Removes unused `global` statements
Recent flake8 runs were failing with:
```
py3: flake8==7.2.0,mccabe==0.7.0,pip==25.0.1,pycodestyle==2.13.0,pyflakes==3.3.0,setuptools==75.8.0,wheel==0.45.1
py3: commands[0] /home/jenkins-build/build/workspace/ceph-pull-requests/src/tools/cephfs/top> flake8 --ignore=W503 --max-line-length=100 cephfs-top
cephfs-top:344:9: F824 `global fs_list` is unused: name is never assigned in scope
cephfs-top:466:13: F824 `global current_states` is unused: name is never assigned in scope
cephfs-top:872:9: F824 `global metrics_dict` is unused: name is never assigned in scope
cephfs-top:872:9: F824 `global current_states` is unused: name is never assigned in scope
cephfs-top:911:9: F824 `global fs_list` is unused: name is never assigned in scope
cephfs-top:981:9: F824 `global current_states` is unused: name is never assigned in scope
cephfs-top:1126:13: F824 `global current_states` is unused: name is never assigned in scope
py3: exit 1 (0.77 seconds) /home/jenkins-build/build/workspace/ceph-pull-requests/src/tools/cephfs/top> flake8 --ignore=W503 --max-line-length=100 cephfs-top pid=2309605
py3: FAIL code 1 (8.15=setup[7.38]+cmd[0.77] seconds)
evaluation failed :( (8.24 seconds)
```
Since these variables are only being referenced and not assigned within
their scopes, the `global` declarations are unnecessary and can be
safely removed. This change:
- Removes all flagged `global` statements
- Fixes the failing flake8 checks in the CI pipeline
- Maintains the original code behavior as variable references still work without the `global` keyword
The `global` keyword is only needed when assigning to global variables
within a function scope, not when simply referencing them.
Kefu Chai [Sun, 30 Mar 2025 03:48:28 +0000 (11:48 +0800)]
qa: Remove unnecessary global statements in tests
Removes unused `global` statements from Python test files to fix flake8
F824 errors.
Recent flake8 runs were failing with:
```
./tasks/radosgw_admin.py:330:5: F824 `global log` is unused: name is never assigned in scope
./workunits/dencoder/test_readable.py:99:5: F824 `global incompat_paths` is unused: name is never assigned in scope
./workunits/dencoder/test_readable.py:164:5: F824 `global backward_compat` is unused: name is never assigned in scope
./workunits/dencoder/test_readable.py:165:5: F824 `global fast_shouldnt_skip` is unused: name is never assigned in scope
```
Since these variables are only being referenced and not assigned within
their scopes, the `global` declarations are unnecessary and can be
safely removed. This change:
- Removes all flagged `global` statements
- Fixes the failing flake8 checks in the CI pipeline
- Maintains the original code behavior as variable references still work
without the `global` keyword
The `global` keyword is only needed when assigning to global variables
within a function scope, not when simply referencing them.
Kefu Chai [Wed, 26 Mar 2025 03:43:43 +0000 (11:43 +0800)]
src: migrate to fmt::formattable for format support
Replace custom has_formatter concept with fmtlib's fmt::formattable
- Deprecate usage of fmt::has_formatter
- Resolve compiler warnings with recent fmtlib versions
The custom concept of has_formatter is no longer necessary with
recent updates to the fmtlib library, as fmt::formattable was introduced
in fmtlib v11.0.0. By switching to fmt::formattable, we simplify our
code and stop using the deprecated `fmt::has_formatter` template.
Kefu Chai [Sat, 29 Mar 2025 13:10:05 +0000 (21:10 +0800)]
librbd: replace deprecated atomic_store with std::atomic<shared_ptr>
Update shared pointer atomic operations to use C++20's std::atomic<std::shared_ptr<T>>
instead of the deprecated atomic_store functions. This change addresses deprecation
warnings from GCC-15's libstdc++ where atomic shared pointer operations outside the
std::atomic class are being phased out:
```
/home/kefu/dev/ceph/src/librbd/ImageCtx.cc:1010:5: warning: 'atomic_store<neorados::IOContext>' is deprecated: use 'std::atomic<std::shared_ptr<T>>' instead [-Wdeprecated-declarations]
1010 | atomic_store(&data_io_context, ctx);
| ^
/usr/lib/gcc/x86_64-redhat-linux/15/../../../../include/c++/15/bits/shared_ptr_atomic.h:181:5: note: 'atomic_store<neorados::IOContext>' has been explicitly marked deprecated here
181 | _GLIBCXX20_DEPRECATED_SUGGEST("std::atomic<std::shared_ptr<T>>")
| ^
/usr/lib/gcc/x86_64-redhat-linux/15/../../../../include/c++/15/x86_64-redhat-linux/bits/c++config.h:2055:45: note: expanded from macro '_GLIBCXX20_DEPRECATED_SUGGEST'
2055 | # define _GLIBCXX20_DEPRECATED_SUGGEST(ALT) _GLIBCXX_DEPRECATED_SUGGEST(ALT)
| ^
/usr/lib/gcc/x86_64-redhat-linux/15/../../../../include/c++/15/x86_64-redhat-linux/bits/c++config.h:2023:19: note: expanded from macro '_GLIBCXX_DEPRECATED_SUGGEST'
2023 | __attribute__ ((__deprecated__ ("use '" ALT "' instead")))
| ^
```
The implementation now uses the standard-compliant approach that's recommended in
the compiler warnings, while maintaining backward compatibility with older compilers
by conditionally selecting the appropriate implementation.
Kefu Chai [Tue, 25 Mar 2025 04:03:30 +0000 (12:03 +0800)]
common: disable OpenSSL engine support if it is disabled
OpenSSL 3.0 documentation recommends moving from the ENGINE API to the
Providers API. Recent distributions may compile OpenSSL without engine
support by default, necessitating more flexible configuration handling.
So, in this change:
- Add a CMake option `WITH_OPENSSL_ENGINE` to explicitly control engine support
- Respect `openssl_engine_opts` when engine support is enabled
- Provide clear error messaging when engine options are set but support is disabled
See also:
- OpenSSL 3.0 documentation:
https://wiki.openssl.org/index.php/OpenSSL_3.0#Engines_and_.22METHOD.22_APIs
Zac Dover [Thu, 27 Mar 2025 12:24:13 +0000 (22:24 +1000)]
src/common: add guidance for mon_warn_pg_not_scrubbed
Add an explanation of how to set the value of
"mon_warn_pg_not_scrubbed_ratio" to the confval definition of that
variable. Although this variable contains the string "mon", it is set on
the Manager. I have added a note to direct users to set this value on
the Manager.
This issue was pointed out by Petr Tlapa on Slack in late March of 2025.
This issue is part of a small project that also encompasses
https://github.com/ceph/ceph/pull/62459, which is associated with commit aeef59a50ee31072648ba0c7436b6522137614cd
Co-authored-by: Anthony D'Atri <anthony.datri@gmail.com> Signed-off-by: Zac Dover <zac.dover@proton.me>
Previously, we relied on the __GNUC__ macro to check for std::atomic<std::shared_ptr>
support, which was inaccurate. This approach failed with Clang builds using libstdc++,
even when the feature was implemented. The warning looks like:
```
/usr/lib/gcc/x86_64-redhat-linux/15/../../../../include/c++/15/bits/shared_ptr_atomic.h:131:5: note: 'atomic_load_explicit<std::vector<entity_addrvec_t>>' has been explicitly marked deprecated here
131 | _GLIBCXX20_DEPRECATED_SUGGEST("std::atomic<std::shared_ptr<T>>")
| ^
```
This change uses a standard-compliant feature test macro (__cpp_lib_atomic_shared_ptr)
to correctly detect support for std::atomic<std::shared_ptr>. This resolves compilation
issues and improves portability across different compilers and standard library
implementations.
Ilya Dryomov [Thu, 20 Mar 2025 17:28:44 +0000 (18:28 +0100)]
rbd: mirror_uuids -> mirror_uuid in remote mirror peer listing
... in "rbd mirror image status" and "rbd mirror pool status --verbose"
formatted outputs. In unformatted outputs this value is shown in place
of a name in case a site name isn't available.
Ilya Dryomov [Thu, 20 Mar 2025 16:10:47 +0000 (17:10 +0100)]
rbd: extend use of "none" placeholder to IMAGES section
Currently if there are no mirror-enabled images, IMAGES section
in "rbd mirror pool status --verbose" output isn't terminated:
$ rbd mirror pool status data --verbose
health: OK
daemon health: OK
image health: OK
images: 0 total
DAEMONS
service 4388:
...
health: OK
IMAGES$
DAEMONS section has a "none" placeholder for when there are no
rbd-mirror daemons running. Fix some issues with the separator logic
and employ the placeholder in IMAGES section:
$ rbd mirror pool status data --verbose
health: OK
daemon health: OK
image health: OK
images: 0 total
DAEMONS
service 4388:
...
health: OK
IMAGES
none
$
$ rbd mirror pool status data --verbose
health: UNKNOWN
daemon health: UNKNOWN
image health: OK
images: 0 total
rgw: fix the version in DOCODE_START() of RGWZoneGroupPlacementTier
This commit fixes an undetected merge conflict between PRs #61745
and #60159. The dencoding problem has been introduced very recently,
it is straightforward and causes failures of the make check bot
everywhere, therefore -- if no objections -- I want to merge this
patch without the Teuthology testing.
Kefu Chai [Mon, 24 Mar 2025 02:05:25 +0000 (10:05 +0800)]
librbd: migrate from boost::variant to std::variant
migrate from boost::variant to std::variant
Complete migration started in commit 017f3339c, replacing boost::variant with
std::variant throughout the librbd codebase. This change is part of our ongoing
effort to reduce third-party dependencies by leveraging C++ standard library
alternatives where possible.
Benefits include:
- Improved code readability and maintainability
- Reduced external dependency surface
- More consistent API usage with other components
Aashish Sharma [Tue, 25 Mar 2025 11:35:05 +0000 (17:05 +0530)]
mgr/dashboard: fix image filter's query on rbd-details grafana panel
The image filter on the RBD Details grafana panel is using a query with a type "label_values(ceph_rbd_read_ops{cluster=~\"$cluster\", , pool=\"$pool\"}, image)". The extra comma needs to be removed.
Ilya Dryomov [Tue, 25 Mar 2025 08:13:27 +0000 (09:13 +0100)]
mgr/rbd_support: always parse interval and start_time in Schedules::remove()
Commit 1b62447071a9 ("mgr/rbd_support: fix schedule remove") addressed
the issue that it was concerned with in a rather suboptimal way: instead
of moving the parsing of interval and start_time upfront to be able to
bail early, it wrapped from_string() constructors with try/finally and
left the conditional behavior in place.
Kefu Chai [Tue, 25 Mar 2025 04:17:35 +0000 (12:17 +0800)]
common: remove spaces before literal operator
clang 20 complains when compiling a user-defined literal operator
with spaces before the underscore. so let's drop the space to be
compliant to
[CWG2521](https://cplusplus.github.io/CWG/issues/2521.html).
```
/home/kefu/dev/ceph/src/common/options.h:415:41: warning: identifier '_min' preceded by whitespace in a literal operator declaration is deprecated [-Wdeprecated-literal-operator]
415 | constexpr unsigned long long operator"" _min (unsigned long long min) {
| ~~~~~~~~~~~^~~~
| operator""_min
```
Patrick Donnelly [Mon, 24 Mar 2025 15:50:50 +0000 (11:50 -0400)]
mgr: add status command
The Rook operator relies on a `status` (asok) command to the mgr to verify it
is operational. However, this command was actually served by the CephFS
`Client` library that the ceph-mgr daemon statically linked in and used for
sending MDS commands. That statically linked client was removed via 048fc68c
(case insensitive directory tree feature).
So, add a legitimate ceph-mgr status command which simply outputs an empty
dictionary and returns success (0). TBD on adding useful information.
Fixes: 048fc68c517f50b9978457f478ca4638f01caa09 Fixes: https://tracker.ceph.com/issues/70571 Signed-off-by: Patrick Donnelly <pdonnell@ibm.com>
projects / ceph.git / log