From d5fab790ef38147aa97317f59ae4c4787def7c84 Mon Sep 17 00:00:00 2001 From: Ricardo Dias Date: Mon, 18 Feb 2019 16:27:40 +0000 Subject: [PATCH] doc/msgr2: session establishment failure scenarios Signed-off-by: Ricardo Dias --- doc/dev/msgr2.rst | 148 +++++++++++++++++++++++++++++++++++++++++++--- 1 file changed, 141 insertions(+), 7 deletions(-) diff --git a/doc/dev/msgr2.rst b/doc/dev/msgr2.rst index 08d04ee85b9..6b4cc5f1283 100644 --- a/doc/dev/msgr2.rst +++ b/doc/dev/msgr2.rst @@ -256,6 +256,7 @@ an established session. __le64 features supported (CEPH_FEATURE_* bitmask) __le64 features required (CEPH_FEATURE_* bitmask) __le64 flags (CEPH_MSG_CONNECT_* bitmask) + __le64 cookie - client will send first, server will reply with same. if this is a new session, the client and server can proceed to the message exchange. @@ -270,6 +271,8 @@ an established session. shouldn't break any existing functionality. implementation will likely want to mask this against what the authenticated credential allows. + - cookie is the client coookie used to identify a session, and can be used + to reconnect to an existing session. - we've dropped the 'protocol_version' field from msgr1 * TAG_IDENT_MISSING_FEATURES (server->client): complain about a TAG_IDENT @@ -288,20 +291,17 @@ an established session. __le64 flags (CEPH_MSG_CONNECT_* bitmask) __le64 cookie - - The cookie can be used by the client if it is later disconnected and wants to - reconnect and resume the session. + - The server cookie can be used by the client if it is later disconnected + and wants to reconnect and resume the session. * TAG_RECONNECT (client->server): reconnect to an established session:: __le32 num_addrs entity_addr_t * num_addrs - entity_addr_t target entity addr - __le64 cookie - __le64 id (of name.id, e.g., osd.123 -> 123) + __le64 client_cookie + __le64 server_cookie __le64 global_seq __le64 connect_seq - __le64 supported_features - __le64 required_features __le64 msg_seq (the last msg seq received) * TAG_RECONNECT_OK (server->client): acknowledge a reconnect attempt:: @@ -321,6 +321,140 @@ an established session. that direction should win the race. The client should wait for that connection to complete. +* TAG_RESET_SESSION (server only): ask client to reset session:: + + __u8 full + + - full flag indicates whether peer should do a full reset, i.e., drop + message queue. + + +Example of failure scenarios: + +* First client's client_ident message is lost, and then client reconnects. + +.. ditaa:: +---------+ +--------+ + | Client | | Server | + +---------+ +--------+ + | | + c_cookie(a) | client_ident(a) | + |-------------X | + | | + | client_ident(a) | + |-------------------->| + |<--------------------| + | server_ident(b) | s_cookie(b) + | | + | session established | + | | + + +* Server's server_ident message is lost, and then client reconnects. + +.. ditaa:: +---------+ +--------+ + | Client | | Server | + +---------+ +--------+ + | | + c_cookie(a) | client_ident(a) | + |-------------------->| + | X------------| + | server_ident(b) | s_cookie(b) + | | + | | + | client_ident(a) | + |-------------------->| + |<--------------------| + | server_ident(c) | s_cookie(c) + | | + | session established | + | | + + +* Server's server_ident message is lost, and then server reconnects. + +.. ditaa:: +---------+ +--------+ + | Client | | Server | + +---------+ +--------+ + | | + c_cookie(a) | client_ident(a) | + |-------------------->| + | X------------| + | server_ident(b) | s_cookie(b) + | | + | | + | reconnect(a, b) | + |<--------------------| + |-------------------->| + | reset_session(F) | + | | + | client_ident(a) | c_cookie(a) + |<--------------------| + |-------------------->| + s_cookie(c) | server_ident(c) | + | | + + +* Connection failure after session is established, and then client reconnects. + +.. ditaa:: +---------+ +--------+ + | Client | | Server | + +---------+ +--------+ + | | + c_cookie(a) | session established | s_cookie(b) + |<------------------->| + | X------------| + | | + | reconnect(a, b) | + |-------------------->| + |<--------------------| + | reconnect_ok | + | | + + +* Connection failure after session is established because server reseted, + and then client reconnects. + +.. ditaa:: +---------+ +--------+ + | Client | | Server | + +---------+ +--------+ + | | + c_cookie(a) | session established | s_cookie(b) + |<------------------->| + | X------------| reset + | | + | reconnect(a, b) | + |-------------------->| + |<--------------------| + | reset_session(RC*) | + | | + c_cookie(c) | client_ident(c) | + |-------------------->| + |<--------------------| + | server_ident(d) | s_cookie(d) + | | + +RC* means that the reset session full flag depends on the policy.resetcheck +of the connection. + + +* Connection failure after session is established because client reseted, + and then client reconnects. + +.. ditaa:: +---------+ +--------+ + | Client | | Server | + +---------+ +--------+ + | | + c_cookie(a) | session established | s_cookie(b) + |<------------------->| + reset | X------------| + | | + c_cookie(c) | client_ident(c) | + |-------------------->| + |<--------------------| reset if policy.resetcheck + | server_ident(d) | s_cookie(d) + | | + + Message exchange ---------------- -- 2.39.5