msg/Pipe: do not wait for self in Pipe::stop_and_wait()

The fast dispatch code necessitated adding a wait for the fast dispatch
to complete when taking over sockets back in commit
2d5d3097c3998add1061ce253104154d72879237.  This included mark_down()
(although I am not certain mark_down was required to fix the previous set
of races).

In any case, if the fast dispatch thread itself tries to mark down its
own connection, it will deadlock in this method waiting for itself to
return and clear reader_dispatching.  Skip this wait if we are in fact
the reader thread.  This avoids the deadlock.

Alternatively, we could change mark_down() to not use stop_and_wait(), but
I am less clear about the potential races there, so I'm opting for the
minimal (though ugly) fix.

Fixes: #9057
Signed-off-by: Sage Weil <sage@redhat.com>

diff --git a/src/msg/Pipe.cc b/src/msg/Pipe.cc
index 59179e0b7fd43bf0f7562e2de7d409e972e021cc..c016a9f5a661cfe9834d0df748a3fd1fbbcf7f6e 100644 (file)
--- a/src/msg/Pipe.cc
+++ b/src/msg/Pipe.cc
@@ -1407,8 +1407,14 @@ void Pipe::stop_and_wait()
   if (state != STATE_CLOSED)
     stop();
   
+  // HACK: we work around an annoying deadlock here.  If the fast
+  // dispatch method calls mark_down() on itself, it can block here
+  // waiting for the reader_dispatching flag to clear... which will
+  // clearly never happen.  Avoid the situation by skipping the wait
+  // if we are marking our *own* connect down.
   while (reader_running &&
-        reader_dispatching)
+        reader_dispatching &&
+        !reader_thread.am_self())
     cond.Wait(pipe_lock);
 }