Matthew N. Heler [Thu, 20 Nov 2025 18:14:14 +0000 (12:14 -0600)]
rgw/lc: reuse prefetched object tags across lifecycle filter
Prefetch tags once per object and pass them into LCOpRule, letting
the tag filter reuse the cached set and skip tag-based rules early
when tags are absent or mismatched.
Signed-off-by: Matthew N. Heler <matthew.heler@hotmail.com>
Matthew N. Heler [Thu, 20 Nov 2025 12:56:04 +0000 (06:56 -0600)]
rgw/lc: change from spawn-per-rule to spawn-per-object
Currently, bucket_lc_process() spawns one coroutine per rule per
object. This means for a bucket with N lifecycle rules, we spawn
N coroutines for each object.
This creates a workpool utilization problem when there are many
rules. With the default rgw_lc_max_wp_worker=128 and 64 rules,
only ~2 objects can be processed concurrently (128 / 64 ≈ 2).
This severely limits parallelism and causes performance degradation
when multiple tag-based or other rules are configured.
This commit changes the concurrency model to spawn one coroutine
per object instead. Each coroutine processes all applicable rules
for that object sequentially. This allows the full workpool of 128
coroutines to process 128 objects concurrently, regardless of the
number of rules.
Additionally, this fixes a timing issue where rule.update() was
called inside the async coroutine, potentially reading stale data
from the shared LCObjsLister after it had advanced. Now all rules
are updated before spawning to capture the correct lister state.
Signed-off-by: Matthew N. Heler <matthew.heler@hotmail.com>
Matthew N. Heler [Thu, 20 Nov 2025 00:36:47 +0000 (18:36 -0600)]
rgw/lc: group prefix rules
Group lifecycle rules by prefix and process them in one bucket listing,
filtering invalid or zone-incompatible rules before scanning so we avoid
rescanning the bucket for each rule.
Signed-off-by: Matthew N. Heler <matthew.heler@hotmail.com>