librbd: disable zero-copy writes by default

Add a new configuration option to allow a client to opt-in to zero-copy
writes if they can guarantee that the provided buffer will remain
stable while the IO is in-flight.

For example, QEMU cannot guarantee stable writes since it DMAs guest OS
memory through to a librbd AIO write request if not using a bounce-buffer,
and the guest OS is free to continue to modify the affected page(s) since
it will remain flagged as dirty and will eventually re-issue another write.
However, the CRC computed by the Ceph messenger layer will detect the
change and it will result in an endless session reset loop.

Fixes: https://tracker.ceph.com/issues/42889
Signed-off-by: Jason Dillaman <dillaman@redhat.com>

diff --git a/src/common/options.cc b/src/common/options.cc
index d7815e3e4351b6b86949021f847470b61d1c1b67..52a93842ae219cd9ae7d356c947f8657c243c485 100644 (file)
--- a/src/common/options.cc
+++ b/src/common/options.cc
@@ -6952,6 +6952,11 @@ static std::vector<Option> get_rbd_options() {
     .set_default(60)
     .set_description("time in seconds for detecting a hung thread"),
 
+    Option("rbd_disable_zero_copy_writes", Option::TYPE_BOOL, Option::LEVEL_ADVANCED)
+    .set_default(true)
+    .set_description("Disable the use of zero-copy writes to ensure unstable "
+                     "writes from clients cannot cause a CRC mismatch"),
+
     Option("rbd_non_blocking_aio", Option::TYPE_BOOL, Option::LEVEL_ADVANCED)
     .set_default(true)
     .set_description("process AIO ops from a dispatch thread to prevent blocking"),

diff --git a/src/librbd/ImageCtx.cc b/src/librbd/ImageCtx.cc
index 17743db98145d66c6f3d8674e5ce6ee16cfd7f9c..070233e095486b8a8e79a0a9be15fa390314300c 100644 (file)
--- a/src/librbd/ImageCtx.cc
+++ b/src/librbd/ImageCtx.cc
@@ -814,6 +814,12 @@ public:
       RBD_QOS_WRITE_BPS_THROTTLE,
       config.get_val<uint64_t>("rbd_qos_write_bps_limit"),
       config.get_val<uint64_t>("rbd_qos_write_bps_burst"));
+
+    if (!disable_zero_copy &&
+        config.get_val<bool>("rbd_disable_zero_copy_writes")) {
+      ldout(cct, 5) << this << ": disabling zero-copy writes" << dendl;
+      disable_zero_copy = true;
+    }
   }
 
   ExclusiveLock<ImageCtx> *ImageCtx::create_exclusive_lock() {