From 141c305c1269e83f0799dd5d625fb977777c5c25 Mon Sep 17 00:00:00 2001 From: Sage Weil Date: Fri, 25 Jan 2019 02:50:16 -0600 Subject: [PATCH] doc/dev/cephx: document current implementation of cephx Signed-off-by: Sage Weil --- doc/dev/cephx.rst | 354 +++++++++++++++++++++++++++++++++ doc/dev/cephx_protocol.rst | 3 + doc/dev/msgr2.rst | 2 + src/auth/cephx/CephxProtocol.h | 55 +---- 4 files changed, 360 insertions(+), 54 deletions(-) create mode 100644 doc/dev/cephx.rst diff --git a/doc/dev/cephx.rst b/doc/dev/cephx.rst new file mode 100644 index 00000000000..d702a7988c4 --- /dev/null +++ b/doc/dev/cephx.rst @@ -0,0 +1,354 @@ +===== +Cephx +===== + +.. _cephx: + +Intro +----- + +The protocol design looks a lot like kerberos. The authorizer "KDC" +role is served by the monitor, who has a database of shared secrets +for each entity. Clients and non-monitor daemons all start by +authenticating with the monitor to obtain tickets, mostly referreed to +in the code as authorizers. These tickets provide both +*authentication* and *authorization* in that they include a +description of the *capabilities* for the entity, a concise structured +description of what actions are allowed, that can be interpreted and +enforced by the service daemons. + +Other references +---------------- + +- A write-up from 2012 on cephx as it existed at that time by Peter + Reiher: :ref:`cephx_2012_peter` + +Terms +----- + +- *monitor(s)*: central authorization authority +- *service*: the set of all daemons of a particular type (e.g., all OSDs, all MDSs) +- *client*: an entity or principal that is accessing the service +- *entity name*: the string identifier for a principal (e.g. client.admin, osd.123) +- *ticket*: a bit of data that cryptographically asserts identify and authorization + +- *principal*: a client or daemon, identified by a unique entity_name, + that shares a secret with the monitor. +- *principal_secret*: principal secret, a shared secret (16 bytes) known by the + principal and the monitor +- *mon_secret*: monitor secret, a shared secret known by all monitors +- *service_secret*: a rotating secret known by all members of a + service class (e.g., all OSDs) + +- *auth ticket*: a ticket proving identity to the monitors +- *service ticket*: a ticket proving identify and authorization to a service + + +Terminology +----------- + +``{foo, bar}^secret`` denotes encryption by secret. + + +Context +------- + +The authentication messages described here are specific to the cephx +auth implementation. The messages are transferred by the Messenger +protocol or by MAuth messages, depending on the version of the +messenger protocol. See also :ref:`msgr2`. + +An initial (messenger) handshake negotiates an authentication method to be used +(cephx vs none or krb or whatever) and an assertion of what entity the client or +daemon is attempting to authenticate as. + +Phase I: obtaining auth ticket +------------------------------ + +The cephx exchange begins with the monitor knowing who the client +claims to be, and an initial cephx message from the monitor to the +client/principal.:: + + a->p : + CephxServerChallenge { + u64 server_challenge # random (by server) + } + +The client responds by adding its own challenge, and calculating a value derived +from both challenges and its shared key principal_secret.:: + + p->a : + CephxRequestHeader { + u16 CEPHX_GET_AUTH_SESSION_KEY + } + CephXAuthenticate { + u8 1 + u64 client_challenge # random (by client) + u64 key = {client_challenge + server_challenge}^principal_secret # (roughly) + blob old_ticket # old ticket, if we are reconnecting or renewing + } + + +The monitor looks up principal_secret in database, and verifies the key is correct. +If old_ticket is present, verify it is valid, and we can reuse the same global_id. +(Otherwise, a new global_id is assigned by the monitor.):: + + a->p : + CephxReplyHeader { + u16 CEPHX_GET_AUTH_SESSION_KEY + s32 result (0) + } + u8 encoding_version = 1 + u32 num_tickets ( = 1) + ticket_info # (N = 1) + +where:: + + ticket_info { + u32 service_id # CEPH_ENTITY_TYPE_AUTH + u8 msg_version (1) + {CephXServiceTicket service_ticket}^principal_secret + {CephxTicketBlob ticket_blob}^existing session_key # if we are renewing a ticket, + CephxTicketBlob ticket_blob # otherwise + } + + CephxServiceTicket { + CryptoKey session_key # freshly generated (even if old_ticket is present) + utime_t expiration # now + auth_mon_ticket_ttl + } + + CephxTicketBlob { + u64 secret_id # which service ticket encrypted this; -1 == mon_secret + {CephXServiceTicketInfo ticket}^mon_secret + } + + CephxServiceTicketInfo { + CryptoKey session_key # same session_key as above + AuthTicket ticket + } + + AuthTicket { + EntityName name # client's identity, as proven by its possession of principal_secret + u64 global_id # newly assigned, or from old_ticket + utime_t created, renew_after, expires + AuthCapsInfo # what client is allowed to do + u32 flags = 0 # unused + } + +So: for each ticket, principal gets a part that it decrypts with its +secret to get the session_key (CephxServiceTicket). And the +CephxTicketBlob is opaque (secured by the mon secret) but can be used +later to prove who we are and what we can do (see CephxAuthorizer below). + +The client can infer that the monitor is authentic because it can decrypt the +service_ticket with its secret (i.e., the server has its secret key). + + +Phase II: Obtaining service tickets +----------------------------------- + +Now the client needs the keys used to talk to non-monitors (osd, mds, mgr).:: + + p->a : + CephxRequestHeader { + u16 CEPHX_GET_PRINCIPAL_SESSION_KEY + } + CephxAuthorizer authorizer + CephxServiceTicketRequest { + u32 keys # bitmask of CEPH_ENTITY_TYPE_NAME (MGR, OSD, MDS, etc) + } + +where:: + + CephxAuthorizer { + u8 AUTH_MODE_AUTHORIZER (1) + u64 global_id + u32 service_id # CEPH_ENTITY_TYPE_* + CephxTicketBlob auth_ticket + {CephxAuthorize msg}^session_key + } + + CephxAuthorize msg { + u8 2 + u64 nonce # random from client + bool have_challenge = false # not used here + u64 server_challenge_plus_one = 0 # not used here + } + +The monitor validates the authorizer by decrypting the auth_ticket +with ``mon_secret`` and confirming that it says this principal is who they +say they are in the CephxAuthorizer fields. Note that the nonce random bytes +aren't used here (the field exists for Phase III below). + +Assuming all is well, the authorizer can generate service tickets +based on the CEPH_ENTITY_TYPE_* bits in the ``keys`` bitmask. + +The response looks like:: + + CephxResponseHeader { + u16 CEPHX_GET_PRINCIPAL_SESSION_KEY + s32 result (= 0) + } + u8 encoding_version = 1 + u32 num_tickets + ticket_info * N + +Where, as above,:: + + ticket_info { + u32 service_id # CEPH_ENTITY_TYPE_{OSD,MGR,MDS} + u8 msg_version (1) + {CephXServiceTicket service_ticket}^principal_secret + CephxTicketBlob ticket_blob + } + + CephxServiceTicket { + CryptoKey session_key + utime_t expiration + } + + CephxTicketBlob { + u64 secret_id # which version of the (rotating) service ticket encrypted this + {CephXServiceTicketInfo ticket}^rotating_service_secret + } + + CephxServiceTicketInfo { + CryptoKey session_key + AuthTicket ticket + } + + AuthTicket { + EntityName name + u64 global_id + utime_t created, renew_after, expires + AuthCapsInfo # what you are allowed to do + u32 flags = 0 # unused + } + +This concludes the authentication exchange with the monitor. The client or daemon +now has tickets to talk to the mon and all other daemons of interest. + + +Phase III: Opening a connection to a service +-------------------------------------------- + +When a connection is opened, an "authorizer" payload is sent:: + + p->s : + CephxAuthorizer { + u8 AUTH_MODE_AUTHORIZER (1) + u64 global_id + u32 service_id # CEPH_ENTITY_TYPE_* + CephxTicketBlob auth_ticket + {CephxAuthorize msg}^session_key + } + + CephxAuthorize msg { + u8 2 + u64 nonce # random from client + bool have_challenge = false + u64 server_challenge_plus_one = 0 + } + +Note that prior to the Luminous v12.2.6 or Mimic v13.2.2 releases, the +CephxAuthorize msg did not contain a challenge, and consisted only +of:: + + CephxAuthorize msg { + u8 1 + u64 nonce # random from client + } + +The server will inspect the auth_ticket CephxTicketBlob (by decrypting +it with its current rotating service key). If it is a pre-v12.2.6 or pre-v13.2.2 +client, the server immediately replies with:: + + s->p : + {CephxAuthorizeReply reply}^session_key + +where:: + + CephxAuthorizeReply { + u64 nonce_plus_one + } + +Otherwise, the server will respond with a challenge (to prevent replay +attacks):: + + s->p : + {CephxAuthorizeChallenge challenge}^session_key + +where:: + + CephxAuthorizeChallenge { + u64 server_challenge # random from server + } + +The client decrypts and updates its CephxAuthorize msg accordingly, resending most +of the same information as before:: + + p->s : + CephxAuthorizer { + u8 AUTH_MODE_AUTHORIZER (1) + u64 global_id + u32 service_id # CEPH_ENTITY_TYPE_* + CephxTicketBlob auth_ticket + {CephxAuthorize msg}^session_key + } + +where:: + + CephxAuthorize msg { + u8 2 + u64 nonce # (new) random from client + bool have_challenge = true + u64 server_challenge_plus_one # server_challenge + 1 + } + +The server validates the ticket as before, and then also verifies the msg nonce +has it's challenge + 1, confirming this is a live authentication attempt (not a replay). + +Finally, the server responds with a reply that proves its authenticity to the client:: + + s->p : + {CephxAuthorizeReply reply}^session_key + +where:: + + CephxAuthorizeReply { + u64 nonce_plus_one + } + +The client decrypts and confirms that the server incremented nonce properly and that this +is thus a live authentication request and not a replay. + + +Rotating service secrets +------------------------ + +Daemons make use of a rotating secret for their tickets instead of a +fixed secret in order to limit the severity of a compromised daemon. +If a daemon's secret key is compromised by an attacker, that +daemon and its key can be removed from the monitor's +database, but the attacker may also have obtained a copy of the +service secret shared by all daemons. To mitigate this, service keys rotate +periodically so that after a period of time (auth_service_ticket_ttl) +the key the attacker obtained will no longer be valid.:: + + p->a : + CephxRequestHeader { + u16 CEPHX_GET_ROTATING_KEY + } + + a->p : + CephxReplyHeader { + u16 CEPHX_GET_ROTATING_KEY + s32 result = 0 + } + {CryptoKey service_key}^principal_secret + +That is, the new rotating key is simply protected by the daemon's rotating secret. + +Note that, as an implementation detail, the services keep the current key and the +prior key on hand so that the can continue to validate requests while the key is +being rotated. diff --git a/doc/dev/cephx_protocol.rst b/doc/dev/cephx_protocol.rst index ad633c2229a..7b8c17876ef 100644 --- a/doc/dev/cephx_protocol.rst +++ b/doc/dev/cephx_protocol.rst @@ -1,6 +1,9 @@ +.. _cephx_2012_peter: + ============================================================ A Detailed Description of the Cephx Authentication Protocol ============================================================ + Peter Reiher 7/13/12 diff --git a/doc/dev/msgr2.rst b/doc/dev/msgr2.rst index 78ce017dea9..12282818552 100644 --- a/doc/dev/msgr2.rst +++ b/doc/dev/msgr2.rst @@ -1,3 +1,5 @@ +.. _msgr2: + msgr2 protocol ============== diff --git a/src/auth/cephx/CephxProtocol.h b/src/auth/cephx/CephxProtocol.h index 4629bba80ad..1698c38d2f6 100644 --- a/src/auth/cephx/CephxProtocol.h +++ b/src/auth/cephx/CephxProtocol.h @@ -18,61 +18,8 @@ /* Ceph X protocol - First, the principal has to authenticate with the authenticator. A - shared-secret mechanism is being used, and the negotitaion goes like this: + See doc/dev/cephx.rst - A = Authenticator - P = Principle - S = Service - - 1. Obtaining principal/auth session key - - (Authenticate Request) - p->a : principal, principal_addr. authenticate me! - - ...authenticator does lookup in database... - - a->p : A= {principal/auth session key, validity}^principal_secret (*) - B= {principal ticket, validity, principal/auth session key}^authsecret - - - [principal/auth session key, validity] = service ticket - [principal ticket, validity, principal/auth session key] = service ticket info - - (*) annotation: ^ signifies 'encrypted by' - - At this point, if is genuine, the principal should have the principal/auth - session key at hand. The next step would be to request an authorization to - use some other service: - - 2. Obtaining principal/service session key - - p->a : B, {principal_addr, timestamp}^principal/auth session key. authorize - me! - a->p : E= {service ticket}^svcsecret - F= {principal/service session key, validity}^principal/auth session key - - principal_addr, timestamp = authenticator - - service ticket = principal name, client network address, validity, principal/service session key - - Note that steps 1 and 2 are pretty much the same thing; contacting the - authenticator and requesting for a key. - - Following this the principal should have a principal/service session key that - could be used later on for creating a session: - - 3. Opening a session to a service - - p->s : E + {principal_addr, timestamp}^principal/service session key - s->p : {timestamp+1}^principal/service/session key - - timestamp+1 = reply authenticator - - Now, the principal is fully authenticated with the service. So, logically we - have 2 main actions here. The first one would be to obtain a session key to - the service (steps 1 and 2), and the second one would be to authenticate with - the service, using that ticket. */ /* authenticate requests */ -- 2.39.5