From: Sage Weil <sage@redhat.com>
Date: Wed, 22 Jun 2016 19:00:48 +0000 (-0400)
Subject: os/bluestore: handle bnode removal race
X-Git-Tag: v11.0.0~70^2~1
X-Git-Url: http://git-server-git.apps.pok.os.sepia.ceph.com/?a=commitdiff_plain;h=3064c819fdf6e4592061f9ef2eaf723b035fabea;p=ceph.git

os/bluestore: handle bnode removal race

One thread might try to drop a bnode while another does a lookup and
retakes a ref:

 1- drop bnode ref (-> 0)
 2- take bnode_set lock, do lookup
 2- take bnode ref (-> 1)
 2- drop lock
 1- take bnode_set lock
 1- remove from set
 1- drop lock

Fix this race by rechecking the ref count while we are holding the lock.

Signed-off-by: Sage Weil <sage@redhat.com>
---

diff --git a/src/os/bluestore/BlueStore.cc b/src/os/bluestore/BlueStore.cc
index 784432a4d68..05eabf65d87 100644
--- a/src/os/bluestore/BlueStore.cc
+++ b/src/os/bluestore/BlueStore.cc
@@ -1069,8 +1069,11 @@ void BlueStore::Bnode::put()
 {
   if (--nref == 0) {
     dout(20) << __func__ << " removing self from set " << bnode_set << dendl;
-    bnode_set->remove(this);
-    delete this;
+    if (bnode_set->remove(this)) {
+      delete this;
+    } else {
+      dout(20) << __func__ << " lost race to remove myself from set" << dendl;
+    }
   }
 }
 
diff --git a/src/os/bluestore/BlueStore.h b/src/os/bluestore/BlueStore.h
index 9b50d5a066a..2df10447646 100644
--- a/src/os/bluestore/BlueStore.h
+++ b/src/os/bluestore/BlueStore.h
@@ -456,9 +456,13 @@ public:
       std::lock_guard<std::mutex> l(lock);
       uset.insert(*b);
     }
-    void remove(Bnode *b) {
+    bool remove(Bnode *b) {
       std::lock_guard<std::mutex> l(lock);
-      uset.erase(*b);
+      if (b->nref == 0) {
+	uset.erase(*b);
+	return true;
+      }
+      return false;
     }
   };