From: Niu Pengju <pengju.niu@xtaotech.com>
Date: Sun, 8 Apr 2018 03:08:04 +0000 (+0800)
Subject: rgw: Fix multisite Synchronization failed when read and write delete at the same... 
X-Git-Tag: v12.2.5~21^2
X-Git-Url: http://git-server-git.apps.pok.os.sepia.ceph.com/?a=commitdiff_plain;h=refs%2Fpull%2F21390%2Fhead;p=ceph.git

rgw: Fix multisite Synchronization failed when read and write delete at the same time

This case is first write objA,then write and delete objA at the same
time,write early than delete.
When del objA, use information which stat  of first write objA, so the
op should del the first write data.However when try to del objA, objA
header is second write, so osd "do_xattr_cmp_str" has found idtag change
and return -125(canceled),after rgw client receive the ret -125 , it
will still do "complete_del", then do cls_obj_complete_del to write
bilog。"complete_op" in cls_rgw module  will  write bilog with second
write mtime and second ".ver.epoch". Finally, del op append behind the
second write in bilog. And the slave rgw will merge write op and del op
as del op, and del data,but master rgw complete second write and cancel
del.
This logic is problematic,  so bilog recording the del op should use
cancel op. And squash_map should skip the cancel op.

Fixes: http://tracker.ceph.com/issues/22804
Signed-off-by: Niu Pengju <pengju.niu@xtaotech.com>
(cherry picked from commit a8fd508915a0acf7768fe6ba7f7414e71813f1d7)
---

diff --git a/src/rgw/rgw_data_sync.cc b/src/rgw/rgw_data_sync.cc
index eabef031b067..a3af8459a35d 100644
--- a/src/rgw/rgw_data_sync.cc
+++ b/src/rgw/rgw_data_sync.cc
@@ -2554,6 +2554,9 @@ int RGWBucketShardIncrementalSyncCR::operate()
           syncstopped = false;
           continue;
         }
+        if (e.op == CLS_RGW_OP_CANCEL) {
+          continue;
+        }
         if (e.state != CLS_RGW_STATE_COMPLETE) {
           continue;
         }
diff --git a/src/rgw/rgw_op.cc b/src/rgw/rgw_op.cc
index 0f8133089881..f20cbd9b862b 100644
--- a/src/rgw/rgw_op.cc
+++ b/src/rgw/rgw_op.cc
@@ -4328,6 +4328,9 @@ void RGWDeleteObj::execute()
       }
     }
 
+    if (op_ret == -ECANCELED) {
+      op_ret = 0;
+    }
     if (op_ret == -ERR_PRECONDITION_FAILED && no_precondition_error) {
       op_ret = 0;
     }
diff --git a/src/rgw/rgw_rados.cc b/src/rgw/rgw_rados.cc
index b34077fb205b..bb37bee27f9f 100644
--- a/src/rgw/rgw_rados.cc
+++ b/src/rgw/rgw_rados.cc
@@ -9000,12 +9000,9 @@ int RGWRados::Object::Delete::delete_obj()
 
   store->remove_rgw_head_obj(op);
   r = ref.ioctx.operate(ref.oid, &op);
-  bool need_invalidate = false;
-  if (r == -ECANCELED) {
-    /* raced with another operation, we can regard it as removed */
-    need_invalidate = true;
-    r = 0;
-  }
+
+  /* raced with another operation, object state is indeterminate */
+  const bool need_invalidate = (r == -ECANCELED);
 
   int64_t poolid = ref.ioctx.get_id();
   if (r >= 0) {