From: Jason Dillaman <dillaman@redhat.com>
Date: Thu, 21 Nov 2019 19:06:17 +0000 (-0500)
Subject: librbd: disable zero-copy writes by default
X-Git-Tag: v15.1.0~601^2
X-Git-Url: http://git-server-git.apps.pok.os.sepia.ceph.com/?a=commitdiff_plain;h=refs%2Fpull%2F31794%2Fhead;p=ceph.git

librbd: disable zero-copy writes by default

Add a new configuration option to allow a client to opt-in to zero-copy
writes if they can guarantee that the provided buffer will remain
stable while the IO is in-flight.

For example, QEMU cannot guarantee stable writes since it DMAs guest OS
memory through to a librbd AIO write request if not using a bounce-buffer,
and the guest OS is free to continue to modify the affected page(s) since
it will remain flagged as dirty and will eventually re-issue another write.
However, the CRC computed by the Ceph messenger layer will detect the
change and it will result in an endless session reset loop.

Fixes: https://tracker.ceph.com/issues/42889
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
---

diff --git a/src/common/options.cc b/src/common/options.cc
index d7815e3e435..52a93842ae2 100644
--- a/src/common/options.cc
+++ b/src/common/options.cc
@@ -6952,6 +6952,11 @@ static std::vector<Option> get_rbd_options() {
     .set_default(60)
     .set_description("time in seconds for detecting a hung thread"),
 
+    Option("rbd_disable_zero_copy_writes", Option::TYPE_BOOL, Option::LEVEL_ADVANCED)
+    .set_default(true)
+    .set_description("Disable the use of zero-copy writes to ensure unstable "
+                     "writes from clients cannot cause a CRC mismatch"),
+
     Option("rbd_non_blocking_aio", Option::TYPE_BOOL, Option::LEVEL_ADVANCED)
     .set_default(true)
     .set_description("process AIO ops from a dispatch thread to prevent blocking"),
diff --git a/src/librbd/ImageCtx.cc b/src/librbd/ImageCtx.cc
index 17743db9814..070233e0954 100644
--- a/src/librbd/ImageCtx.cc
+++ b/src/librbd/ImageCtx.cc
@@ -814,6 +814,12 @@ public:
       RBD_QOS_WRITE_BPS_THROTTLE,
       config.get_val<uint64_t>("rbd_qos_write_bps_limit"),
       config.get_val<uint64_t>("rbd_qos_write_bps_burst"));
+
+    if (!disable_zero_copy &&
+        config.get_val<bool>("rbd_disable_zero_copy_writes")) {
+      ldout(cct, 5) << this << ": disabling zero-copy writes" << dendl;
+      disable_zero_copy = true;
+    }
   }
 
   ExclusiveLock<ImageCtx> *ImageCtx::create_exclusive_lock() {