rgw: treat -EFBIG as advance-and-retry in unordered listing

The RGW cls bucket listing may return RGWBIAdvanceAndRetryError(-EFBIG) when it cannot return any visible entries in a call but has advanced the marker; callers are expected to retry with the updated marker.

RGWRados::cls_bucket_list_unordered() treated this condition as a fatal error and aborted the listing, which could prematurely terminate listings for buckets with many non-visible entries.

This change handles RGWBIAdvanceAndRetryError by advancing to the returned marker and continuing the loop, correctly interpreting it as "advance and retry."

Fixes: https://tracker.ceph.com/issues/63721
Signed-off-by: PARK BEOMSEOK <beomseok.park@linecorp.com>

diff --git a/src/rgw/driver/rados/rgw_rados.cc b/src/rgw/driver/rados/rgw_rados.cc
index 63f332e843adb7c8af7a99fb02def1cb712f3e38..e1075b71aadd7386265d83941a65e3d85b3590f9 100644 (file)
--- a/src/rgw/driver/rados/rgw_rados.cc
+++ b/src/rgw/driver/rados/rgw_rados.cc
@@ -10925,6 +10925,7 @@ int RGWRados::cls_bucket_list_unordered(const DoutPrefixProvider *dpp,
   uint32_t count = 0u;
   std::map<std::string, bufferlist> updates;
   rgw_obj_index_key last_added_entry;
+
   while (count <= num_entries &&
         ((shard_id >= 0 && current_shard == uint32_t(shard_id)) ||
          current_shard < num_shards)) {
@@ -10933,10 +10934,44 @@ int RGWRados::cls_bucket_list_unordered(const DoutPrefixProvider *dpp,
 
     librados::ObjectReadOperation op;
     const std::string empty_delimiter;
+
+    // Keep the previous marker to detect real progress
+    const auto prev_marker = marker;
+
     cls_rgw_bucket_list_op(op, marker, prefix, empty_delimiter,
                           num_entries,
                            list_versions, &result);
     r = rgw_rados_operate(dpp, ioctx, oid, std::move(op), nullptr, y, 0, nullptr, &index_ver.epoch);
+    if (r == RGWBIAdvanceAndRetryError) {
+      // CLS could not return any visible entries in this round,
+      // but it advanced the marker; retry with the new marker.
+
+      // Copy marker from cls type into rgw index key (avoid type-mismatch assignment)
+      marker.name = result.marker.name;
+      marker.instance = result.marker.instance;
+
+      // AdvanceAndRetry expected the marker to advance; no progress observed. Return -EIO.
+      if (!(prev_marker < marker)) {
+        ldpp_dout(dpp, 0)
+          << "ERROR: " << __func__
+          << ": RGW_UNORDERED_LIST_NO_MARKER_PROGRESS_ON_ADVANCE_AND_RETRY"
+          << " oid=" << oid
+          << " prev_marker=" << prev_marker
+          << " marker=" << marker
+          << dendl;
+        return -EIO;
+      }
+
+      ldpp_dout(dpp, 10)
+        << __func__
+        << ": RGWBIAdvanceAndRetryError"
+        << " on oid=" << oid
+        << "; prev_marker=" << prev_marker
+        << " -> marker=" << marker
+        << "; advancing marker and retrying"
+        << dendl;
+      continue;
+    }
     if (r < 0) {
       ldpp_dout(dpp, 0) << "ERROR: " << __func__ <<
        ": error in rgw_rados_operate (bucket list op), r=" << r << dendl;