osd: bind to new cluster address when wrongly marked down

If we come back up on the same address, there is a possible race.  Other
nodes will mark_down when they see us go down.  If we go up first, queue
some messages, and _then_ they see that we're down and mark_down, the
messages we queued will get lost.  Since it's stateful on the cluster
backend, we need to introduce an ordering so that closing out the _old_
session doesn't break the new session.  We do this by binding to a new
address (just a different port, actually) before marking ourselves back
up.

Fixes #592.

Signed-off-by: Sage Weil <sage@newdream.net>

diff --git a/src/osd/OSD.cc b/src/osd/OSD.cc
index 98c9d337d4dc6cc9b50f54ded3ec7abbdedab6e8..4e442555715f7298d9ff1f285e3666452d17293e 100644 (file)
--- a/src/osd/OSD.cc
+++ b/src/osd/OSD.cc
@@ -2717,6 +2717,10 @@ void OSD::handle_osd_map(MOSDMap *m)
       state = STATE_BOOTING;
       up_epoch = 0;
 
+      int r = cluster_messenger->rebind();
+      if (r != 0)
+       do_shutdown = true;  // FIXME: do_restart?
+
       reset_heartbeat_peers();
     }
   }