Matt Benjamin [Fri, 21 Feb 2025 19:43:53 +0000 (14:43 -0500)]
rgw:cksum: implement crc64nvme and combined 32- and 64-bit CRCs
* internally compensate for at-rest byteswapped crc64 representation (e.g., before combine step)
* generalize Cksum crc api for 32bit and 64bit crcs, other cleanup
* prototype abstract cksum::Combiner workflow
* add support for forward and backward handling of composite vs full object checksums
by marking the composites and the update flag day
* clean up checksum formatting and checksum type reporting
* add unit tests for Combiner interface
* validate and track requested checksum type (i.e., composite or full), plus
unit test fixture for combinations of full matrix of cksum types
* doh. GET/HEAD checksums are in headers
* add crcany license to COPYING
* return ChecksumType as header in GET/HEAD
Found by Casey in review
* avoid fmt of char* null when no checksum header supplied
A cksum_hdr_t(nullptr, nullptr) results in this case, and is intended,
but its components obviously can't be presented to std::format unless
cast to a safe pointer type.
* fail checksum mismatch with BadDigest
When uploading an S3 object with an invalid checksum, the return code
should be BadDigest to mirror more closely the AWS S3 implementation.
See: https://docs.aws.amazon.com/AmazonS3/latest/userguide/checking-object-integrity.html
Fixes: https://tracker.ceph.com/issues/70614
Reported by Alex Wojno <awojno@bloomberg.net>
* fix comparison and display of composite checksums
A string comparision bounds error and a bitmask comparison
error are fixed.
* fix build on centos9
On gcc(?13?) we can't declare rgw::cksum::FLAG_NONE
and also rgw::cksum::Cksum::FlAG_NONE.
SAD!!
* include <variant> invariantly
* fix checksum type return from complete-multipart
This one is in the XML response
* aieee, don't leak Combiners
Use unique_ptr to polymorphic type...correctly.
Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
Matt Benjamin [Tue, 18 Feb 2025 19:11:08 +0000 (14:11 -0500)]
rgw_cksum: add 64bit and 32bit crc constructions from crcany
These constructions provide conforming implemenations of Amazon
S3 CRC32 (ISO hdlc), CRC32C (iscsi), and CRC64/NVME checksums,
in particular, linear combining of adjacent checksums, by
Mark Adler.
Source: https://github.com/madler/crcany
License: approximate BSD with "optional" advertising clause
Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
rgw: fix the version in DOCODE_START() of RGWZoneGroupPlacementTier
This commit fixes an undetected merge conflict between PRs #61745
and #60159. The dencoding problem has been introduced very recently,
it is straightforward and causes failures of the make check bot
everywhere, therefore -- if no objections -- I want to merge this
patch without the Teuthology testing.
Kefu Chai [Mon, 24 Mar 2025 02:05:25 +0000 (10:05 +0800)]
librbd: migrate from boost::variant to std::variant
migrate from boost::variant to std::variant
Complete migration started in commit 017f3339c, replacing boost::variant with
std::variant throughout the librbd codebase. This change is part of our ongoing
effort to reduce third-party dependencies by leveraging C++ standard library
alternatives where possible.
Benefits include:
- Improved code readability and maintainability
- Reduced external dependency surface
- More consistent API usage with other components
Ilya Dryomov [Tue, 25 Mar 2025 08:13:27 +0000 (09:13 +0100)]
mgr/rbd_support: always parse interval and start_time in Schedules::remove()
Commit 1b62447071a9 ("mgr/rbd_support: fix schedule remove") addressed
the issue that it was concerned with in a rather suboptimal way: instead
of moving the parsing of interval and start_time upfront to be able to
bail early, it wrapped from_string() constructors with try/finally and
left the conditional behavior in place.
Kefu Chai [Tue, 25 Mar 2025 04:17:35 +0000 (12:17 +0800)]
common: remove spaces before literal operator
clang 20 complains when compiling a user-defined literal operator
with spaces before the underscore. so let's drop the space to be
compliant to
[CWG2521](https://cplusplus.github.io/CWG/issues/2521.html).
```
/home/kefu/dev/ceph/src/common/options.h:415:41: warning: identifier '_min' preceded by whitespace in a literal operator declaration is deprecated [-Wdeprecated-literal-operator]
415 | constexpr unsigned long long operator"" _min (unsigned long long min) {
| ~~~~~~~~~~~^~~~
| operator""_min
```
Patrick Donnelly [Mon, 24 Mar 2025 15:50:50 +0000 (11:50 -0400)]
mgr: add status command
The Rook operator relies on a `status` (asok) command to the mgr to verify it
is operational. However, this command was actually served by the CephFS
`Client` library that the ceph-mgr daemon statically linked in and used for
sending MDS commands. That statically linked client was removed via 048fc68c
(case insensitive directory tree feature).
So, add a legitimate ceph-mgr status command which simply outputs an empty
dictionary and returns success (0). TBD on adding useful information.
Fixes: 048fc68c517f50b9978457f478ca4638f01caa09 Fixes: https://tracker.ceph.com/issues/70571 Signed-off-by: Patrick Donnelly <pdonnell@ibm.com>
Ilya Dryomov [Fri, 21 Mar 2025 13:43:50 +0000 (14:43 +0100)]
librbd: respect rbd_default_snapshot_quiesce_mode in group_snap_create()
Make group_snap_create() behave the same as snap_create() and
mirror_image_create_snapshot(): APIs that don't take RBD_SNAP_CREATE_
flags explicitly should respect rbd_default_snapshot_quiesce_mode
option.
Zac Dover [Mon, 24 Mar 2025 12:26:11 +0000 (22:26 +1000)]
src/common: add guidance for deep-scrubbing ratio warning
Add an explanation of how to set the value of
"mon_warn_pg_not_deep_scrubbed_ratio" to the confval definition of that
variable. Although this variable contains the string "mon", it is set on
the Manager. I have added a note to direct users to set this value on
the Manager.
This issue was pointed out by Petr Tlapa on Slack in late March of 2025.
Fixes: https://tracker.ceph.com/issues/70588 Signed-off-by: Naman Munet <naman.munet@ibm.com>
Before: As the tenant was not passed in lifecycle calls made through bucket details > tiering tab, & also account_id was not resolved to the user of the bucket, hence not able to manage the Tiering configuration lifecycle changes
After: Account_id is resolved to user_id and tenant is additionally passed to the lifecycle API so as to handle the tenanted bucket scenario
Nizamudeen A [Thu, 20 Mar 2025 08:51:50 +0000 (14:21 +0530)]
mgr/dashboard: use existing pools for cephfs vol creation
We can use the newly introduced data and metadata params to create a vol
with those pools.
UI is being intelligent by filtering out the used pools and only uses
the pools that are labeled by cephfs and also not in use. To figure out
a pool is in use or not, we are fetching the pool stats and checking its
used_bytes.
Note: Using ec pools for data pool layout is something discouraged
according to offical doc: https://docs.ceph.com/en/latest/cephfs/createfs/#creating-a-file-system
We can force it but for now I have disabled it entirely in the dashboard
unless people say its okay to do it.
One more extra thing I am doing here is to add a note on deleting a
filesystem that the underlying pools and mds daemons will be removed.
Fixes: https://tracker.ceph.com/issues/70600 Signed-off-by: Nizamudeen A <nia@redhat.com>
Rishabh Dave [Mon, 3 Mar 2025 16:36:10 +0000 (22:06 +0530)]
doc/cephfs: mention new options for "fs volume create" cmd
Command "ceph fs volume create" accepts 2 new options to allow users to
pass data and metadata pool name. Update docs to include mention of both
the options.
projects / ceph-ci.git / log