From: jiawd <jiawendong@xtaotech.com>
Date: Fri, 12 Nov 2021 03:48:56 +0000 (+0000)
Subject: osd: full-object read crc is mismatch, because truncate modify oi.size and forget... 
X-Git-Tag: v20.0.0~1906^2~4
X-Git-Url: http://git.apps.os.sepia.ceph.com/?a=commitdiff_plain;h=aeafd2e9a452d6a10b742c885dc44927b256c3bf;p=ceph.git

osd: full-object read crc is mismatch, because truncate modify oi.size and forget to clear data_digest

when write before truncate, need trim length, if truncate is to 0,
write is [0~128k], write change to [0~0], do nothing, oi.size is 0, x1 = set_data_digest(crc32(-1)).
write is [128k~128k], write change to [128k~0], truncate oi.size to offset 128k, x2 = set_data_digest(crc32(x1)).
write is [256k~128k], write change to [256k~0], truncate oi.size to offset 256k, x3 = set_data_digest(crc32(x2)).
...
write is [4063232~128k], write change to [4063232~0], truncate oi.size to offset 4063232, xn = set_data_digest(crs32(xn-1))
Now, we can see oi.size is 4063232, and data_digest is 0xffffffff, because thelength of in_data of crc is 0 every time.
when read verify crc will reply EIO. (EC pool).

so, when truncate in write, need clear data_digest and DIGEST flag,
when write before truncate, need to trim length, when offset over than oi.size, don't truncate oi.size to offset.

Fixes: https://tracker.ceph.com/issues/53240
Signed-off-by: jiawd <jiawendong@xtaotech.com>
---

diff --git a/src/osd/PrimaryLogPG.cc b/src/osd/PrimaryLogPG.cc
index 62b8aad27d5b9..22a4e5a0cf410 100644
--- a/src/osd/PrimaryLogPG.cc
+++ b/src/osd/PrimaryLogPG.cc
@@ -6746,6 +6746,8 @@ int PrimaryLogPG::do_osd_ops(OpContext *ctx, vector<OSDOp>& ops)
               truncate_update_size_and_usage(ctx->delta_stats,
                                              oi,
                                              op.extent.truncate_size);
+	      //truncate modify oi.size, need clear old data_digest and DIGEST flag
+	      oi.clear_data_digest();
 	    }
 	  } else {
 	    dout(10) << " truncate_seq " << op.extent.truncate_seq << " > current " << seq
@@ -6764,10 +6766,16 @@ int PrimaryLogPG::do_osd_ops(OpContext *ctx, vector<OSDOp>& ops)
 
 	if (op.extent.length == 0) {
 	  if (op.extent.offset > oi.size) {
-	    t->truncate(
-	      soid, op.extent.offset);
-            truncate_update_size_and_usage(ctx->delta_stats, oi,
-                                           op.extent.offset);
+            if (seq && (seq > op.extent.truncate_seq)) {
+              //do nothing
+              //write arrived after truncate, we should not truncate to offset
+            } else {
+	      t->truncate(
+	        soid, op.extent.offset);
+              truncate_update_size_and_usage(ctx->delta_stats, oi,
+                                             op.extent.offset);
+              oi.clear_data_digest();
+            }
 	  } else {
 	    t->nop(soid);
 	  }