From: Matthew N. Heler <matthew.heler@hotmail.com>
Date: Thu, 20 Nov 2025 12:56:04 +0000 (-0600)
Subject: rgw/lc: change from spawn-per-rule to spawn-per-object
X-Git-Tag: testing/wip-pdonnell-testing-20260205.015545~20^2~1
X-Git-Url: http://git-server-git.apps.pok.os.sepia.ceph.com/?a=commitdiff_plain;h=a47c63ba0a7530d0ce506c233bb8d7839b62adb2;p=ceph-ci.git

rgw/lc: change from spawn-per-rule to spawn-per-object

Currently, bucket_lc_process() spawns one coroutine per rule per
object. This means for a bucket with N lifecycle rules, we spawn
N coroutines for each object.

This creates a workpool utilization problem when there are many
rules. With the default rgw_lc_max_wp_worker=128 and 64 rules,
only ~2 objects can be processed concurrently (128 / 64 ≈ 2).
This severely limits parallelism and causes performance degradation
when multiple tag-based or other rules are configured.

This commit changes the concurrency model to spawn one coroutine
per object instead. Each coroutine processes all applicable rules
for that object sequentially. This allows the full workpool of 128
coroutines to process 128 objects concurrently, regardless of the
number of rules.

Additionally, this fixes a timing issue where rule.update() was
called inside the async coroutine, potentially reading stale data
from the shared LCObjsLister after it had advanced. Now all rules
are updated before spawning to capture the correct lister state.

Signed-off-by: Matthew N. Heler <matthew.heler@hotmail.com>
---

diff --git a/src/rgw/rgw_lc.cc b/src/rgw/rgw_lc.cc
index 698dc7315d6..7d1339b85c2 100644
--- a/src/rgw/rgw_lc.cc
+++ b/src/rgw/rgw_lc.cc
@@ -1740,13 +1740,20 @@ int RGWLC::bucket_lc_process(string& shard_id, LCWorker* worker,
     rgw_bucket_dir_entry* o{nullptr};
     for (auto offset = 0; ol.get_obj(this, yield, &o /* , fetch_barrier */); ++offset, ol.next()) {
       const auto obj = *o;
+
+      // Update all rules to capture current lister state before spawning
       for (auto& rule : rules) {
         rule.update();
-        workpool.spawn([&pf, dpp=this, rule, obj]
-                       (boost::asio::yield_context yield) mutable {
-          pf(dpp, yield, rule, obj);
-        });
       }
+
+      // Spawn one coroutine per object to process all rules
+      workpool.spawn([&pf, dpp=this, rules_copy=rules, obj]
+                     (boost::asio::yield_context yield) mutable {
+        for (auto& rule : rules_copy) {
+          pf(dpp, yield, rule, const_cast<rgw_bucket_dir_entry&>(obj));
+        }
+      });
+
       if ((offset % 100) == 0) {
 	if (worker_should_stop(stop_at, once)) {
 	  ldpp_dout(this, 5) << __func__ << " interval budget EXPIRED worker="