]> git-server-git.apps.pok.os.sepia.ceph.com Git - ceph.git/commit
rgw:cksum: implement crc64nvme and combined 32- and 64-bit CRCs 61878/head
authorMatt Benjamin <mbenjamin@redhat.com>
Fri, 21 Feb 2025 19:43:53 +0000 (14:43 -0500)
committerMatt Benjamin <mbenjamin@redhat.com>
Mon, 31 Mar 2025 14:59:17 +0000 (10:59 -0400)
commitee03b5054147afcf6f174efe71e976394ede7715
treeb7d4ff02511556fc52a06926e3bde117492f7d13
parenta1aed60ca9d9d324ff041ee98212f39778ae2e47
rgw:cksum: implement crc64nvme and combined 32- and 64-bit CRCs

* internally compensate for at-rest byteswapped crc64 representation (e.g., before combine step)
* generalize Cksum crc api for 32bit and 64bit crcs, other cleanup
* prototype abstract cksum::Combiner workflow
* add support for forward and backward handling of composite vs full object checksums
  by marking the composites and the update flag day
* clean up checksum formatting and checksum type reporting
* add unit tests for Combiner interface
* validate and track requested checksum type (i.e., composite or full), plus
  unit test fixture for combinations of full matrix of cksum types
* doh.  GET/HEAD checksums are in headers
* add crcany license to COPYING
* return ChecksumType as header in GET/HEAD

    Found by Casey in review

* avoid fmt of char* null when no checksum header supplied

    A cksum_hdr_t(nullptr, nullptr) results in this case, and is intended,
    but its components obviously can't be presented to std::format unless
    cast to a safe pointer type.

* fail checksum mismatch with BadDigest

    When uploading an S3 object with an invalid checksum, the return code
    should be BadDigest to mirror more closely the AWS S3 implementation.
    See: https://docs.aws.amazon.com/AmazonS3/latest/userguide/checking-object-integrity.html

Fixes: https://tracker.ceph.com/issues/70614
    Reported by Alex Wojno <awojno@bloomberg.net>

* fix comparison and display of composite checksums

A string comparision bounds error and a bitmask comparison
    error are fixed.

* fix build on centos9

On gcc(?13?) we can't declare rgw::cksum::FLAG_NONE
    and also rgw::cksum::Cksum::FlAG_NONE.

    SAD!!

* include <variant> invariantly

* fix checksum type return from complete-multipart

    This one is in the XML response

* aieee, don't leak Combiners

   Use unique_ptr to polymorphic type...correctly.

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
15 files changed:
COPYING
src/rgw/CMakeLists.txt
src/rgw/driver/rados/rgw_sal_rados.cc
src/rgw/rgw_cksum.cc [new file with mode: 0644]
src/rgw/rgw_cksum.h
src/rgw/rgw_cksum_pipe.cc
src/rgw/rgw_cksum_pipe.h
src/rgw/rgw_common.h
src/rgw/rgw_crc_digest.h
src/rgw/rgw_op.cc
src/rgw/rgw_op.h
src/rgw/rgw_rest_s3.cc
src/rgw/rgw_sal.h
src/rgw/spdk/crc64.c
src/test/rgw/test_rgw_cksum.cc