librbd/ManagedLock: kickstart ExclusiveLock state machine

... that is stalled waiting for lock. Do this when trying to reacquire
lock in the ImageWatcher's rewatch mechanism. This would enable the
ExclusiveLock state machine to propagate the blocklist error to the
caller trying to perform an image operation requiring an exclusive
lock.

Previous attempt, e66db763, to fix the hang due to exclusive lock
acquisiton (stuck waiting for lock) racing with client blocklisting
did not always work. e66db763 kickstarted the ExclusiveLock state
machine when the ImageWatcher tried to schedule a exclusive lock
request and the blocklisting was detected. However, there is a short
window between a watch getting deregistered and client blocklisting
getting detected as part of rewatching. If hit when trying to schedule
a lock request, the ExclusiveLock state machine wasn't kickstarted,
blocklist error wasn't propagated, and the hang resurfaced.

A more robust approach is taken to resume the ExclusiveLock state
machine stuck waiting for lock during client blocklisting. Whenever
a client's ImageWatcher loses connection to the cluster, as it happens
during blocklising, the ImageWatcher initiates a mechanism to rewatch
the image and tries to reacquire the lock. Piggyback on this rewatch
mechanism that gets triggered during client blocklisting. And when
trying to reacquire the lock, kickstart the ExclusiveLock state
machine stalled waiting for lock (STATE_WAITING_FOR_LOCK).

Fixes: https://tracker.ceph.com/issues/63009
Signed-off-by: Ramana Raja <rraja@redhat.com>

diff --git a/src/librbd/ImageWatcher.cc b/src/librbd/ImageWatcher.cc
index a3d565520e8ce211bef1fcf639d1dd4ef0fece04..7dd3d529831611ccf19b5966843e0b47cd7260b4 100644 (file)
--- a/src/librbd/ImageWatcher.cc
+++ b/src/librbd/ImageWatcher.cc
@@ -594,10 +594,6 @@ void ImageWatcher<I>::schedule_request_lock(bool use_timer, int timer_delay) {
     } else {
       m_task_finisher->queue(TASK_CODE_REQUEST_LOCK, ctx);
     }
-  } else if (is_blocklisted()) {
-    lderr(m_image_ctx.cct) << this << " blocklisted waiting for exclusive lock"
-                           << dendl;
-    m_image_ctx.exclusive_lock->handle_peer_notification(0);
   }
 }
 

diff --git a/src/librbd/ManagedLock.cc b/src/librbd/ManagedLock.cc
index 8a05957ec67512f0a88e4127417f406002681520..166a31c611473bd1e639cc94f378df2ffe115719 100644 (file)
--- a/src/librbd/ManagedLock.cc
+++ b/src/librbd/ManagedLock.cc
@@ -207,7 +207,8 @@ void ManagedLock<I>::reacquire_lock(Context *on_reacquired) {
   {
     std::lock_guard locker{m_lock};
 
-    if (m_state == STATE_WAITING_FOR_REGISTER) {
+    if (m_state == STATE_WAITING_FOR_REGISTER ||
+        m_state == STATE_WAITING_FOR_LOCK) {
       // restart the acquire lock process now that watch is valid
       ldout(m_cct, 10) << "woke up waiting (re)acquire" << dendl;
       Action active_action = get_active_action();
@@ -217,8 +218,7 @@ void ManagedLock<I>::reacquire_lock(Context *on_reacquired) {
     } else if (!is_state_shutdown() &&
                (m_state == STATE_LOCKED ||
                 m_state == STATE_ACQUIRING ||
-                m_state == STATE_POST_ACQUIRING ||
-                m_state == STATE_WAITING_FOR_LOCK)) {
+                m_state == STATE_POST_ACQUIRING)) {
       // interlock the lock operation with other state ops
       ldout(m_cct, 10) << dendl;
       execute_action(ACTION_REACQUIRE_LOCK, on_reacquired);