rgw: fix stale entries in bucket indexes

If rados_osd_op_timeout is set, the primary osd is slow, the rgw_rados_operate for deleting the rgw head obj may return -ETIMEDOUT
rgw can't determine whether or not the delete succeeded, we shouldn't be calling index_op.complete_del or cancel()
Instead, we should leave that pending entry in the index so than bucket listing can recover with check_disk_state() and cls_rgw_suggest_changens()
When raced with another delete op, deleting the rgw head obj may return ENOENT, calling index_op.complete_del() instead of index_op.cancel()

Fixes: https://tracker.ceph.com/issues/58965
Signed-off-by: Shasha Lu <lu.shasha@aishu.cn>
(cherry picked from commit de9ce1e27322cdf5c1572a0b8eff4a33547757c0)

diff --git a/src/rgw/driver/rados/rgw_rados.cc b/src/rgw/driver/rados/rgw_rados.cc
index a1370b079dc6b81c058dca98c7e2a3510d63e1a5..5acfd7e4ed121188c65d7fa72f5d3e758141f52d 100644 (file)
--- a/src/rgw/driver/rados/rgw_rados.cc
+++ b/src/rgw/driver/rados/rgw_rados.cc
@@ -5918,7 +5918,11 @@ int RGWRados::Object::Delete::delete_obj(optional_yield y, const DoutPrefixProvi
   const bool need_invalidate = (r == -ECANCELED);
 
   int64_t poolid = ioctx.get_id();
-  if (r >= 0) {
+  if (r == -ETIMEDOUT) {
+    // rgw can't determine whether or not the delete succeeded, shouldn't be calling either of complete_del() or cancel()
+    // leaving that pending entry in the index so that bucket listing can recover with check_disk_state() and cls_rgw_suggest_changes()
+    ldpp_dout(dpp, 0) << "ERROR: rgw_rados_operate returned r=" << r << dendl;
+  } else if (r >= 0 || r == -ENOENT) {
     tombstone_cache_t *obj_tombstone_cache = store->get_tombstone_cache();
     if (obj_tombstone_cache) {
       tombstone_entry entry{*state};