OSD: fix race condition for heartbeat_need_update

The heartbeat_need_update member can be accessed by both OSD tick thread and
PG relevant threads, that is why the heartbeat_update_lock mutex is introduced
to protect against its change. However, in the tick thread we may still be able
to reset heartbeat_need_update to false whithout holding heartbeat_update_lock
in hand, which shall be considered as a race conditon.

This pr solves the above problem by add a new API to clear heartbeat_need_update
atomically, which fix the above potential race condition.

Fixes: #14387
Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

diff --git a/src/osd/OSD.cc b/src/osd/OSD.cc
index 7345d535301b84d1ec3744827043d4c15378048b..c4d8fa52ca2ea391a267f04263d329c74f6b9582 100644 (file)
--- a/src/osd/OSD.cc
+++ b/src/osd/OSD.cc
@@ -3658,10 +3658,11 @@ void OSD::maybe_update_heartbeat_peers()
     }
   }
 
-  Mutex::Locker l(heartbeat_lock);
   if (!heartbeat_peers_need_update())
     return;
-  heartbeat_need_update = false;
+  heartbeat_clear_peers_need_update();
+
+  Mutex::Locker l(heartbeat_lock);
 
   dout(10) << "maybe_update_heartbeat_peers updating" << dendl;
 

diff --git a/src/osd/OSD.h b/src/osd/OSD.h
index e53fc6b640ca40339c08040a7a1e41bb7392757e..5c1066a167464292faf5e36fda807b1c68d39713 100644 (file)
--- a/src/osd/OSD.h
+++ b/src/osd/OSD.h
@@ -1541,6 +1541,10 @@ private:
     Mutex::Locker l(heartbeat_update_lock);
     heartbeat_need_update = true;
   }
+  void heartbeat_clear_peers_need_update() {
+    Mutex::Locker l(heartbeat_update_lock);
+    heartbeat_need_update = false;
+  }
   void heartbeat();
   void heartbeat_check();
   void heartbeat_entry();