From: Matt Benjamin <mbenjamin@redhat.com>
Date: Thu, 12 May 2022 16:38:59 +0000 (-0400)
Subject: rgwlc: try hard to get lc locks when saving policy
X-Git-Tag: v18.0.0~381^2~1
X-Git-Url: http://git-server-git.apps.pok.os.sepia.ceph.com/?a=commitdiff_plain;h=216b7f7f4cd43132a4b63e6865e908a6a571d1bd;p=ceph.git

rgwlc: try hard to get lc locks when saving policy

During active lifecycle processing, there can be high contention for
lifecycle shard locks.

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
---

diff --git a/src/rgw/rgw_lc.cc b/src/rgw/rgw_lc.cc
index 69c0e6c32457..ad0936612b4e 100644
--- a/src/rgw/rgw_lc.cc
+++ b/src/rgw/rgw_lc.cc
@@ -2418,14 +2418,19 @@ static int guard_lc_modify(const DoutPrefixProvider *dpp,
   utime_t time(max_lock_secs, 0);
 
   int ret;
+  uint16_t retries{0};
 
+  // due to reports of starvation trying to save lifecycle policy, try hard
   do {
     ret = lock->try_lock(dpp, time, null_yield);
     if (ret == -EBUSY || ret == -EEXIST) {
       ldpp_dout(dpp, 0) << "RGWLC::RGWPutLC() failed to acquire lock on "
-          << oid << ", sleep 5, try again" << dendl;
-      sleep(5); // XXX: return retryable error
-      continue;
+			<< oid << ", retry in 100ms, ret=" << ret << dendl;
+      std::this_thread::sleep_for(std::chrono::milliseconds(100));
+      // the typical S3 client will time out in 60s
+      if(retries++ < 500) {
+	continue;
+      }
     }
     if (ret < 0) {
       ldpp_dout(dpp, 0) << "RGWLC::RGWPutLC() failed to acquire lock on "