os/bluestore: drop deferred_submit_lock

This lock serves no purpose.  It only protects the call to the aio_submit
on the OpSequencer running batch, but we are the only caller who will do
that submission, and a new batch won't be moved into place until ours is
completed and retired (by the completion for the aio we are queueing).

More importantly, this fixes a deadlock:

- thread A submits aio, but the queue is full, and blocks
- thread B blocks taking deferred_submit_lock while holding deferred_lock
- aio completion thread blocks trying to take deferred_lock, and thus
no further aios are drained.

Fixes: http://tracker.ceph.com/issues/21171
Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 7a5ef62af194373c3a8c307b05ae6d467e49c9f0)

diff --git a/src/os/bluestore/BlueStore.cc b/src/os/bluestore/BlueStore.cc
index 44e58f8099582c0935b3ffea4d6fc6cb9df1dba4..7b93f047e99bb49c4316c8cc8ee3a89ca5458441 100644 (file)
--- a/src/os/bluestore/BlueStore.cc
+++ b/src/os/bluestore/BlueStore.cc
@@ -8756,8 +8756,6 @@ void BlueStore::_deferred_submit_unlock(OpSequencer *osr)
     ++i;
   }
 
-  // demote to deferred_submit_lock, then drop that too
-  std::lock_guard<std::mutex> l(deferred_submit_lock);
   deferred_lock.unlock();
   bdev->aio_submit(&b->ioc);
 }

diff --git a/src/os/bluestore/BlueStore.h b/src/os/bluestore/BlueStore.h
index 8c5eb4a02b4e5de2b3c7a3f00d49f2a4633194eb..1e427df7047a5c993173a710dd17ec9298b60dff 100644 (file)
--- a/src/os/bluestore/BlueStore.h
+++ b/src/os/bluestore/BlueStore.h
@@ -1841,7 +1841,7 @@ private:
   interval_set<uint64_t> bluefs_extents;  ///< block extents owned by bluefs
   interval_set<uint64_t> bluefs_extents_reclaiming; ///< currently reclaiming
 
-  std::mutex deferred_lock, deferred_submit_lock;
+  std::mutex deferred_lock;
   std::atomic<uint64_t> deferred_seq = {0};
   deferred_osr_queue_t deferred_queue; ///< osr's with deferred io pending
   int deferred_queue_size = 0;         ///< num txc's queued across all osrs