os/bluestore: fix flush_commit locking

We were updating the txc state to KV_DONE and queuing the oncommits
waiters without holding any locks.  This was mostly fine, *except* that
Collection|OpSequencer::flush_commit(Context *) was looking at the state
(under qlock) and also adding items to oncommits.

The flush_commit() method is only used in 2 places: osd bench, and the
PG reset_interval_flush outgoing message blocking machinery (which is
a bit ick). The first we could get rid of, but the second is hard to
remove (despite its ick factor).

The simple fix is to take qlock while updating the state value and
working with oncommits.

Fixes: http://tracker.ceph.com/issues/21480
Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit a4042d51bcf1444350fa21763003e57aca984f87)

Conflicts:
    src/os/bluestore/BlueStore.cc

diff --git a/src/os/bluestore/BlueStore.cc b/src/os/bluestore/BlueStore.cc
index 6fdd11b170b2403af505c2cf68717d686314893f..2f9ed47eb9cb1f98d1816a7ed0200bbaaf38089a 100644 (file)
--- a/src/os/bluestore/BlueStore.cc
+++ b/src/os/bluestore/BlueStore.cc
@@ -7911,8 +7911,6 @@ void BlueStore::_txc_state_proc(TransContext *txc)
       }
       return;
     case TransContext::STATE_KV_SUBMITTED:
-      txc->log_state_latency(logger, l_bluestore_state_kv_committing_lat);
-      txc->state = TransContext::STATE_KV_DONE;
       _txc_committed_kv(txc);
       // ** fall-thru **
 
@@ -8163,9 +8161,14 @@ void BlueStore::_txc_committed_kv(TransContext *txc)
     txc->onreadable = NULL;
   }
 
-  if (!txc->oncommits.empty()) {
-    finishers[n]->queue(txc->oncommits);
+  {
+    std::lock_guard<std::mutex> l(txc->osr->qlock);
+    txc->state = TransContext::STATE_KV_DONE;
+    if (!txc->oncommits.empty()) {
+      finishers[n]->queue(txc->oncommits);
+    }
   }
+  txc->log_state_latency(logger, l_bluestore_state_kv_committing_lat);
 }
 
 void BlueStore::_txc_finish(TransContext *txc)