doc added msgr on-wire-compression

Signed-off-by: Maya Gilad <ms.maya.gilad@gmail.com>

diff --git a/doc/dev/msgr2.rst b/doc/dev/msgr2.rst
index db8f3fa2e194c6d5b44af23287e3957f2f3a6d4a..af1421a3b878f76bd0a4faf7fd77a98d7e4e824c 100644 (file)
--- a/doc/dev/msgr2.rst
+++ b/doc/dev/msgr2.rst
@@ -104,7 +104,8 @@ Each frame has a 32-byte preamble::
     __le32 segment length
     __le16 segment alignment
   } * 4
-  reserved (2 bytes)
+  __u8 flags
+  reserved (1 byte)
   __le32 preamble crc
 
 An empty frame has one empty segment.  A non-empty frame can have
@@ -114,6 +115,10 @@ empty.
 If there are less than four segments, unused (trailing) segment
 length and segment alignment fields are zeroed.
 
+### Currently supported flags
+
+  1. FRAME_EARLY_DATA_COMPRESSED (see :ref:`msgr-post-compression`)
+
 The reserved bytes are zeroed.
 
 The preamble checksum is CRC32-C.  It covers everything up to
@@ -528,6 +533,64 @@ where epilogue is::
 
 late_status has the same meaning as in msgr2.1-crc mode.
 
+Compression
+-----------
+Compression handshake is implemented using msgr2 feature-based handshaking.
+In this phase, the client will indicate the server if on-wire-compression can be used for message transmitting, 
+in addition to the list of supported compression methods. If on-wire-compression is enabled for both client and server, 
+the server will choose a compression method based on client's request and its' own preferences. 
+Once the handshake is completed, both peers have setup their compression handlers (if desired). 
+
+* TAG_COMPRESSION_REQUEST (client->server): declares compression capabilities and requirements::
+
+    bool  is_compress
+    std::vector<uint32_t> preferred_methods 
+
+  - if the client identifies that both peers support compression feature, it initiates the handshake.
+  - is_compress flag indicates whether the client's configuration is to use compression.
+  - preferred_methods is a list of compression algorithms that are supported by the client.
+
+* TAG_COMPRESSION_DONE (server->client) : determines on compression settings::
+
+    bool is_compress
+    uint32_t  method
+
+  - the server determines whether compression is possible according to its' configuration.
+  - if it is possible, it will pick its' most prioritizied compression method that is also supprorted by the client.
+  - if none exists, it will determine that session between the peers will be handled without compression.
+
+.. ditaa::
+
+           +---------+              +--------+
+           | Client  |              | Server |
+           +---------+              +--------+
+                | compression request    |
+                |----------------------->|
+                |<-----------------------|
+                |   compression done     |
+
+# msgr2.x-secure mode
+
+Combining compression with encryption introduces security implications.
+Compression will not be possible when using secure mode, unless configured specifically by an admin. 
+
+.. _msgr-post-compression:
+
+Post-compression frame format 
+-----------------------------
+Depending on the negotiated connection mode from TAG_COMPRESSION_DONE, the connection is able to acccept/send compressed frames or process all frames as decompressed.
+
+# msgr2.x-force mode
+
+All subsequent frames that will be sent via the connection will be compressed if compression requirements are met (e.g, the frames size).
+
+For compressed frames, the sending peer will enable the FRAME_EARLY_DATA_COMPRESSED flag, thus allowing the accepting peer to detect it and decompress the frame.
+
+# msgr2.x-none mode
+
+FRAME_EARLY_DATA_COMPRESSED flag will be disabled in preamble.
+
+
 Message flow handshake
 ----------------------
 

diff --git a/doc/rados/configuration/msgr2.rst b/doc/rados/configuration/msgr2.rst
index 685422720cfe23fcf662cee65dada5802307a7ed..e1beaf5dc4cc3a55ee977cdeecef4b990920cbfc 100644 (file)
--- a/doc/rados/configuration/msgr2.rst
+++ b/doc/rados/configuration/msgr2.rst
@@ -146,6 +146,41 @@ secure) requirements on communication with the monitors.
 .. confval:: ms_mon_service_mode
 .. confval:: ms_mon_client_mode
 
+
+Compression modes
+-----------------
+
+The v2 protocol supports two compression modes:
+
+* *force* mode provides:
+
+  - In multi-availability zones deployment, compressing replication messages between OSDs saves latency.
+  - In the public cloud, inter-AZ communications are expensive. Thus, minimizing message 
+    size reduces network costs to cloud provider.
+  - When using instance storage on AWS (probably other public clouds as well) the instances with NVMe
+    provide low network bandwidth relative to the device bandwidth. 
+    In this case, NW compression can improve the overall performance since this is clearly 
+    the bottleneck.
+
+* *none* mode provides:
+
+  - messages are transmitted without compression. 
+
+
+Compression mode configuration options
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+For all connections, there is an option that controls compression usage in secure mode
+
+.. confval:: ms_compress_secure
+
+There is a parallel set of options that apply specifically to OSDs, 
+allowing administrators to set different requirements on communication between OSDs.
+
+.. confval:: ms_osd_compress_mode
+.. confval:: ms_osd_compress_min_size
+.. confval:: ms_osd_compression_algorithm
+
 Transitioning from v1-only to v2-plus-v1
 ----------------------------------------