mds: switch submit_mutex to fair mutex for MDLog

The implementations of the Mutex (e.g. std::mutex in C++) do not
guarantee fairness, they do not guarantee that the lock will be
acquired by threads in the order that they called the lock().

In most case this works well, but in overload case the client
requests handling thread and _submit_thread could always successfully
acquire the submit_mutex in a long time, which could make the
MDLog::trim() get stuck. That means the MDS daemons will fill journal
logs into the metadata pool, but couldn't trim the expired segments
in time.

This will switch the submit_mutex to fair mutex and it could make
sure that the all the submit_mutex waiters are in FIFO order and
could get a change to be excuted in time.

Fixes: https://tracker.ceph.com/issues/58000
Signed-off-by: Xiubo Li <xiubli@redhat.com>
(cherry picked from commit 0366b807c50345aebdc7b83103568468186ebe57)

diff --git a/src/mds/MDLog.h b/src/mds/MDLog.h
index 6111f85a97f97bea5a221ca429954cd23f5a4e46..b42fa5dd7159445dd9637096ab1b0d1af2e1bab4 100644 (file)
--- a/src/mds/MDLog.h
+++ b/src/mds/MDLog.h
@@ -14,6 +14,7 @@
 #ifndef CEPH_MDLOG_H
 #define CEPH_MDLOG_H
 
+#include "common/fair_mutex.h"
 #include "include/common_fwd.h"
 
 enum {
@@ -285,8 +286,8 @@ protected:
 
   int64_t mdsmap_up_features = 0;
   std::map<uint64_t,std::list<PendingEvent> > pending_events; // log segment -> event list
-  ceph::mutex submit_mutex = ceph::make_mutex("MDLog::submit_mutex");
-  ceph::condition_variable submit_cond;
+  ceph::fair_mutex submit_mutex{"MDLog::submit_mutex"};
+  std::condition_variable_any submit_cond;
 
 private:
   friend class C_MaybeExpiredSegment;