From: Jianpeng Ma <jianpeng.ma@intel.com>
Date: Wed, 8 Sep 2021 01:51:19 +0000 (+0800)
Subject: librbd: Read request need exclusive-lock when enable  pwl-cache.
X-Git-Tag: v16.2.8~301^2
X-Git-Url: http://git.apps.os.sepia.ceph.com/?a=commitdiff_plain;h=4d8cf8c2ee270212dd3d676c4d65cb9ea6111711;p=ceph.git

librbd: Read request need exclusive-lock when enable  pwl-cache.

TestLibRBD.TestFUA descript the following workload:
a)write/read the same image w/ pwl-cache
  write_image = open(image_name);
  read_image  = open(image_name);
b)i/o workload is:
   write(write_image)
      write need EXLock and require EXLOCK

  read(read_image)
     in ExclusiveLock<I>::init(), firstly read need EXLOCK
     so will require EXLOCK. write_image release EXLOCK(will
     flush data to osd and remove cache). read_image init pwl-cache
     and read-io firstly enter pwl-cache and missed and then read
     from osd.

   write(write_image)
     write need EXLOCK and require EXLOCK. This make read_image remove
     empty cache. write_image init cache pool and write data to cache.

   read(read_image)
       In send_set_require_lock(), it set write need EXLOCK.
       So read don't require EXLOCK and dirtyly read from osd.

Because second-read  don't need EXLOCK and make write_image don't
release EXLOCK(flush dirty data to osd and  shutdown pwl-cache).
This make second-read don't read the latest data.

So we should make read also need EXLOCK when enable pwl-cache.

Fixes: https://tracker.ceph.com/issues/51438

Tested-by: Feng Hualong <hualong.feng@intel.com>
Signed-off-by: Jianpeng Ma <jianpeng.ma@intel.com>
(cherry picked from commit 621facb6e66ce92ca36d566c78bc065a9666639e)
---

diff --git a/src/librbd/exclusive_lock/PreReleaseRequest.cc b/src/librbd/exclusive_lock/PreReleaseRequest.cc
index 3adc3b54a45b2..a9cd1248a7cbc 100644
--- a/src/librbd/exclusive_lock/PreReleaseRequest.cc
+++ b/src/librbd/exclusive_lock/PreReleaseRequest.cc
@@ -100,7 +100,8 @@ void PreReleaseRequest<I>::send_set_require_lock() {
   // setting the lock as required will automatically cause the IO
   // queue to re-request the lock if any IO is queued
   if (m_image_ctx.clone_copy_on_read ||
-      m_image_ctx.test_features(RBD_FEATURE_JOURNALING)) {
+      m_image_ctx.test_features(RBD_FEATURE_JOURNALING) ||
+      m_image_ctx.test_features(RBD_FEATURE_DIRTY_CACHE)) {
     m_image_dispatch->set_require_lock(m_shutting_down,
                                        io::DIRECTION_BOTH, ctx);
   } else {
diff --git a/src/test/librbd/exclusive_lock/test_mock_PreReleaseRequest.cc b/src/test/librbd/exclusive_lock/test_mock_PreReleaseRequest.cc
index ceb45ca2e68b4..466a3ab421e54 100644
--- a/src/test/librbd/exclusive_lock/test_mock_PreReleaseRequest.cc
+++ b/src/test/librbd/exclusive_lock/test_mock_PreReleaseRequest.cc
@@ -102,9 +102,14 @@ public:
     if (!mock_image_ctx.clone_copy_on_read) {
       expect_test_features(mock_image_ctx, RBD_FEATURE_JOURNALING,
                            ((mock_image_ctx.features & RBD_FEATURE_JOURNALING) != 0));
+      if ((mock_image_ctx.features & RBD_FEATURE_JOURNALING) == 0) {
+        expect_test_features(mock_image_ctx, RBD_FEATURE_DIRTY_CACHE,
+                             ((mock_image_ctx.features & RBD_FEATURE_DIRTY_CACHE) != 0));
+      }
     }
     if (mock_image_ctx.clone_copy_on_read ||
-        (mock_image_ctx.features & RBD_FEATURE_JOURNALING) != 0) {
+        (mock_image_ctx.features & RBD_FEATURE_JOURNALING) != 0 ||
+        (mock_image_ctx.features & RBD_FEATURE_DIRTY_CACHE) != 0) {
       expect_set_require_lock(mock_image_dispatch, init_shutdown,
                               librbd::io::DIRECTION_BOTH, r);
     } else {