log: fix flush/signal race

We need to signal the cond in the same interval where we hold the lock
*and* modify the queue.  Otherwise, we can have a race like:

 queue has 1 item, max is 1.
 A: enter submit_entry, signal cond, wait on condition
 B: enter submit_entry, signal cond, wait on condition
 C: flush wakes up, flushes 1 previous item
 A: retakes lock, enqueues something, exits
 B: retakes lock, condition fails, waits
  -> C is never woken up as there are 2 items waiting

Signed-off-by: Sage Weil <sage@inktank.com>
Reviewed-by: Dan Mick <dan.mick@inktank.com>

diff --git a/src/log/Log.cc b/src/log/Log.cc
index 32d8be79c4557dd2a11cd661f4be13280782b583..51a340ffcb17c2d0d3370ebc5db6799a5fd75754 100644 (file)
--- a/src/log/Log.cc
+++ b/src/log/Log.cc
@@ -136,13 +136,13 @@ void Log::set_stderr_level(int log, int crash)
 void Log::submit_entry(Entry *e)
 {
   pthread_mutex_lock(&m_queue_mutex);
-  pthread_cond_signal(&m_cond);
 
   // wait for flush to catch up
   while (m_new.m_len > m_max_new)
     pthread_cond_wait(&m_cond, &m_queue_mutex);
 
   m_new.enqueue(e);
+  pthread_cond_signal(&m_cond);
   pthread_mutex_unlock(&m_queue_mutex);
 }