rgw: add RGWObjCategory::MultiPart for multipart upload part heads

Multipart upload part head objects are bucket-indexed with
RGWObjCategory::Main, making them indistinguishable from completed
objects in bucket stats. This is causing radosgw-admin bucket stats and
the RGW-extension X-RGW-Object-Count header on HEAD Bucket
(?read-stats=true) to report inflated object counts when in-progress
or abandoned multipart uploads exists.

So, this commit adds a new Object Category: RGWObjCategory::MultiPart
(value 5) and assigns it in MultipartObjectProcessor::complete() so
part heads are now tracked under a separate "rgw.multipart" category.

The X-RGW-Object-Count header, which reads only RGWObjCategory::Main,
now excludes in-progress parts. This is more correct as AWS S3 HEAD Bucket
does not expose any object count for incomplete uploads (the header itself
seems to be an RGW extension, not part of the S3 specification).

Quota enforcement, stats aggregation, and part cleanup are all
category-agnostic so should require no changes.

Also, updated the test_cls_rgw_stats simulator to use
the new category for multipart part entries.

Fixes: https://tracker.ceph.com/issues/77509
Signed-off-by: Oguzhan Ozmen <oozmen@bloomberg.net>

diff --git a/src/cls/rgw/cls_rgw_types.cc b/src/cls/rgw/cls_rgw_types.cc
index 7f8835bd93877b27007ac7787a50e474eb458245..5ef9633a84e0e6c31dce87d3a18c68b4adef31bd 100644 (file)
--- a/src/cls/rgw/cls_rgw_types.cc
+++ b/src/cls/rgw/cls_rgw_types.cc
@@ -140,6 +140,7 @@ std::string_view to_string(RGWObjCategory c)
     case RGWObjCategory::Shadow: return "rgw.shadow";
     case RGWObjCategory::MultiMeta: return "rgw.multimeta";
     case RGWObjCategory::CloudTiered: return "rgw.cloudtiered";
+    case RGWObjCategory::MultiPart: return "rgw.multipart";
     default: return "unknown";
   }
 }

diff --git a/src/cls/rgw/cls_rgw_types.h b/src/cls/rgw/cls_rgw_types.h
index 2e0fe2fed59dff289b533e4d5988e127765cfbf1..bbab3306cb19f3b586eea07664f83e404efdabdb 100644 (file)
--- a/src/cls/rgw/cls_rgw_types.h
+++ b/src/cls/rgw/cls_rgw_types.h
@@ -194,6 +194,8 @@ enum class RGWObjCategory : uint8_t {
   MultiMeta = 3,  // b-i entries for multipart upload metadata objs
 
   CloudTiered = 4, // b-i entries which are tiered to external cloud
+
+  MultiPart = 5,   // b-i entries for multipart upload part head objs
 };
 
 std::string_view to_string(RGWObjCategory c);

diff --git a/src/rgw/driver/rados/rgw_putobj_processor.cc b/src/rgw/driver/rados/rgw_putobj_processor.cc
index 3292e1c4e973b16ee93aa3f5e34d609a528a5cf0..9eee7a5faf323f7435dc574c1f3f27d51fa5214a 100644 (file)
--- a/src/rgw/driver/rados/rgw_putobj_processor.cc
+++ b/src/rgw/driver/rados/rgw_putobj_processor.cc
@@ -557,6 +557,7 @@ int MultipartObjectProcessor::complete(
   obj_op.meta.mtime = mtime;
   obj_op.meta.owner = owner;
   obj_op.meta.bucket_owner = bucket_info.owner;
+  obj_op.meta.category = RGWObjCategory::MultiPart;
   obj_op.meta.delete_at = delete_at;
   obj_op.meta.zones_trace = zones_trace;
   obj_op.meta.modify_tail = true;

diff --git a/src/test/cls_rgw/test_cls_rgw_stats.cc b/src/test/cls_rgw/test_cls_rgw_stats.cc
index 1c12367ec038e9e3ef702239e734c20b83786b9c..26ba529259e07f3c2e066f3096801b77ea8220ea 100644 (file)
--- a/src/test/cls_rgw/test_cls_rgw_stats.cc
+++ b/src/test/cls_rgw/test_cls_rgw_stats.cc
@@ -552,7 +552,7 @@ void simulator::complete_multipart(const operation& op)
           << " size=" << part_size << dendl;
     } else {
       rgw_bucket_dir_entry_meta meta;
-      meta.category = op.meta.category;
+      meta.category = RGWObjCategory::MultiPart;
       meta.size = meta.accounted_size = part_size;
 
       int r = index_complete(ioctx, oid, part_key, op.tag, op.type,