osd: force read bit for watch

We can get an bad reply like this:

     - watch1 received, repop started
     - con resets
     - watch2 received
     - watch2 sees watch in oi
       - replies immediately
       - does side-effects (connects new session/con)
     - watch1 repop completes
       - replies (dropped)
       - does side-effects (connects old session/con)

Two problems: we end up with the wrong session/con connected, and the watch
reply goes out before the watch is actually stable on disk.

Fix this my marking watch ops as reads.  This will make us wait for the
previous watch addition to commit before we process the next one.  Somewhat
overkill, but sufficient.

Fixes: #10441
Signed-off-by: Sage Weil <sage@redhat.com>

diff --git a/src/osd/OSD.cc b/src/osd/OSD.cc
index c50284ad45aad85cc0b942dd70b234c448e1a531..9ed551850331b961cca7ffb6cc5767f1999f8d61 100644 (file)
--- a/src/osd/OSD.cc
+++ b/src/osd/OSD.cc
@@ -8577,6 +8577,14 @@ int OSD::init_op_flags(OpRequestRef& op)
          op->set_class_write();
        break;
       }
+
+    case CEPH_OSD_OP_WATCH:
+      // force the read bit for watch since it is depends on previous
+      // watch state (and may return early if the watch exists) or, in
+      // the case of ping, is simply a read op.
+      op->set_read();
+      break;
+
     default:
       break;
     }