rgw: Fix multisite Synchronization failed when read and write delete at the same...

This case is first write objA,then write and delete objA at the same
time,write early than delete.
When del objA, use information which stat  of first write objA, so the
op should del the first write data.However when try to del objA, objA
header is second write, so osd "do_xattr_cmp_str" has found idtag change
and return -125(canceled),after rgw client receive the ret -125 , it
will still do "complete_del", then do cls_obj_complete_del to write
bilog。"complete_op" in cls_rgw module  will  write bilog with second
write mtime and second ".ver.epoch". Finally, del op append behind the
second write in bilog. And the slave rgw will merge write op and del op
as del op, and del data,but master rgw complete second write and cancel
del.
This logic is problematic,  so bilog recording the del op should use
cancel op. And squash_map should skip the cancel op.

Fixes: http://tracker.ceph.com/issues/22804
Signed-off-by: Niu Pengju <pengju.niu@xtaotech.com>

diff --git a/src/rgw/rgw_data_sync.cc b/src/rgw/rgw_data_sync.cc
index 21b5bdb01ad1c1dcd8fde41e586bf9f0ef4a74e5..cc668ec13f809453c9cf0f68e966344062321f51 100644 (file)
--- a/src/rgw/rgw_data_sync.cc
+++ b/src/rgw/rgw_data_sync.cc
@@ -2761,6 +2761,9 @@ int RGWBucketShardIncrementalSyncCR::operate()
           syncstopped = false;
           continue;
         }
+        if (e.op == CLS_RGW_OP_CANCEL) {
+          continue;
+        }
         if (e.state != CLS_RGW_STATE_COMPLETE) {
           continue;
         }

diff --git a/src/rgw/rgw_op.cc b/src/rgw/rgw_op.cc
index 3b79986b33d4c02d0bbb662968c7d99ca9c2ae95..e4afecb5321eb31a7334e10464f2935154be0c9a 100644 (file)
--- a/src/rgw/rgw_op.cc
+++ b/src/rgw/rgw_op.cc
@@ -4403,6 +4403,9 @@ void RGWDeleteObj::execute()
       }
     }
 
+    if (op_ret == -ECANCELED) {
+      op_ret = 0;
+    }
     if (op_ret == -ERR_PRECONDITION_FAILED && no_precondition_error) {
       op_ret = 0;
     }

diff --git a/src/rgw/rgw_rados.cc b/src/rgw/rgw_rados.cc
index 9594a54056d0baa71f60fb9799e4428b8e52aa4b..73296c411cf007de778cf9accfc3e9fb790c5ce8 100644 (file)
--- a/src/rgw/rgw_rados.cc
+++ b/src/rgw/rgw_rados.cc
@@ -8979,12 +8979,9 @@ int RGWRados::Object::Delete::delete_obj()
 
   store->remove_rgw_head_obj(op);
   r = ref.ioctx.operate(ref.oid, &op);
-  bool need_invalidate = false;
-  if (r == -ECANCELED) {
-    /* raced with another operation, we can regard it as removed */
-    need_invalidate = true;
-    r = 0;
-  }
+
+  /* raced with another operation, object state is indeterminate */
+  const bool need_invalidate = (r == -ECANCELED);
 
   int64_t poolid = ref.ioctx.get_id();
   if (r >= 0) {