rgw: Fix multisite Synchronization failed when read and write delete at the same...

This case is first write objA,then write and delete objA at the same
time,write early than delete.
When del objA, use information which stat  of first write objA, so the
op should del the first write data.However when try to del objA, objA
header is second write, so osd "do_xattr_cmp_str" has found idtag change
and return -125(canceled),after rgw client receive the ret -125 , it
will still do "complete_del", then do cls_obj_complete_del to write
bilog。"complete_op" in cls_rgw module  will  write bilog with second
write mtime and second ".ver.epoch". Finally, del op append behind the
second write in bilog. And the slave rgw will merge write op and del op
as del op, and del data,but master rgw complete second write and cancel
del.
This logic is problematic,  so bilog recording the del op should use
cancel op. And squash_map should skip the cancel op.

Fixes: http://tracker.ceph.com/issues/22804
Signed-off-by: Niu Pengju <pengju.niu@xtaotech.com>
(cherry picked from commit a8fd508915a0acf7768fe6ba7f7414e71813f1d7)

diff --git a/src/rgw/rgw_data_sync.cc b/src/rgw/rgw_data_sync.cc
index eabef031b067cfde366afa604ad51897ca81977f..a3af8459a35d33d93cc32f1e0d48a1d60d817053 100644 (file)
--- a/src/rgw/rgw_data_sync.cc
+++ b/src/rgw/rgw_data_sync.cc
@@ -2554,6 +2554,9 @@ int RGWBucketShardIncrementalSyncCR::operate()
           syncstopped = false;
           continue;
         }
+        if (e.op == CLS_RGW_OP_CANCEL) {
+          continue;
+        }
         if (e.state != CLS_RGW_STATE_COMPLETE) {
           continue;
         }

diff --git a/src/rgw/rgw_op.cc b/src/rgw/rgw_op.cc
index 0f8133089881c7086b257b10e60efc448aad38ed..f20cbd9b862bc877599a4d29669bd697486c0a71 100644 (file)
--- a/src/rgw/rgw_op.cc
+++ b/src/rgw/rgw_op.cc
@@ -4328,6 +4328,9 @@ void RGWDeleteObj::execute()
       }
     }
 
+    if (op_ret == -ECANCELED) {
+      op_ret = 0;
+    }
     if (op_ret == -ERR_PRECONDITION_FAILED && no_precondition_error) {
       op_ret = 0;
     }

diff --git a/src/rgw/rgw_rados.cc b/src/rgw/rgw_rados.cc
index b34077fb205b7ee1dc04087e1f963c7e5dfc918c..bb37bee27f9f3298578bfd424964616c006820a4 100644 (file)
--- a/src/rgw/rgw_rados.cc
+++ b/src/rgw/rgw_rados.cc
@@ -9000,12 +9000,9 @@ int RGWRados::Object::Delete::delete_obj()
 
   store->remove_rgw_head_obj(op);
   r = ref.ioctx.operate(ref.oid, &op);
-  bool need_invalidate = false;
-  if (r == -ECANCELED) {
-    /* raced with another operation, we can regard it as removed */
-    need_invalidate = true;
-    r = 0;
-  }
+
+  /* raced with another operation, object state is indeterminate */
+  const bool need_invalidate = (r == -ECANCELED);
 
   int64_t poolid = ref.ioctx.get_id();
   if (r >= 0) {