msgr: adjust nonce on rebind()

We can have a situation where:

 - we have a pipe to a peer
 - pipe goes to standby (on peer)
 - we rebind to a new port
 - ....
 - we rebind again to the same old port
 - we connect to peer

and get reattached to the ancient pipe from two instances back.  Avoid that
by picking a new nonce each time we rebind.

Add 1,000,000 each time so that the port is still legible in the printed
output.

Signed-off-by: Sage Weil <sage@inktank.com>

diff --git a/src/msg/Accepter.cc b/src/msg/Accepter.cc
index 4d13be8fdca35cc94706da8c31ed3b9f98e98926..d6e94d1cc515c4adff90a0d608148017e691c965 100644 (file)
--- a/src/msg/Accepter.cc
+++ b/src/msg/Accepter.cc
@@ -165,6 +165,11 @@ int Accepter::rebind(const set<int>& avoid_ports)
   new_avoid.insert(addr.get_port());
   addr.set_port(0);
 
+  // adjust the nonce; we want our entity_addr_t to be truly unique.
+  nonce += 1000000;
+  msgr->my_inst.addr.nonce = nonce;
+  ldout(msgr->cct,10) << " new nonce " << nonce << " and inst " << msgr->my_inst << dendl;
+
   ldout(msgr->cct,10) << " will try " << addr << " and avoid ports " << new_avoid << dendl;
   int r = bind(addr, new_avoid);
   if (r == 0)