doc: Add Messenger Status Section to Operations->Monitoring

Briefly document `messenger dump` feature with examples.

Signed-off-by: Marcel Lauhoff <marcel.lauhoff@clyso.com>

diff --git a/doc/rados/operations/monitoring.rst b/doc/rados/operations/monitoring.rst
index f9af6f03d16e92cda841d81be10e8cf14b94280b..5e35ed27032360e6c6f32a959862938d72fcd044 100644 (file)
--- a/doc/rados/operations/monitoring.rst
+++ b/doc/rados/operations/monitoring.rst
@@ -655,3 +655,86 @@ Runtime`_.
 
 .. _Viewing a Configuration at Runtime: ../../configuration/ceph-conf#viewing-a-configuration-at-runtime
 .. _Storage Capacity: ../../configuration/mon-config-ref#storage-capacity
+
+Messenger Status
+=================
+
+Ceph daemons and librados clients support an admin socket command
+``messenger dump`` that surfaces a snapshot of runtime information
+about connections, sockets, bound addresses, and kernel TCP stats (via
+tcp(7) TCP_INFO).
+
+.. note:: The queried messenger needs to lock the connection data
+  structures for the time it takes to create the snapshot. This lock's
+  duration is in the order of tens of milliseconds. This might
+  interfere with normal operation. Use the ``dumpcontents`` argument
+  to limit data structures dumped.
+
+Examples
+---------
+
+When a command is issued without specifying a messenger to dump, the
+list of available messengers is returned:
+
+.. prompt:: bash $
+
+   ceph tell osd.0 messenger dump
+
+.. code-block:: javascript
+
+ {
+    "messengers": [
+        "client",
+        "cluster",
+        "hb_back_client",
+        "hb_back_server",
+        "hb_front_client",
+        "hb_front_server",
+        "ms_objecter",
+        "temp_mon_client"
+    ]
+  }
+
+The ``client`` and ``cluster`` messengers correspond to the configured
+client / cluster network (see :doc:`/rados/configuration/network-config-ref`). Messengers
+with ``hb_`` prefix are part of the heartbeat system.
+
+List all current connections on the client messenger:
+
+.. code-block:: bash
+
+           ceph tell osd.0 messenger dump client \
+              | jq -r '.messenger.connections[].async_connection |
+                          [.conn_id, .socket_fd, .worker_id,
+                           if .status.connected then "connected" else "disconnected" end,
+                           .state,
+                           "\(.peer.type).\(.peer.entity_name.id).\(.peer.id)",
+                           .protocol.v2.con_mode, .protocol.v2.crypto.rx, .protocol.v2.compression.rx] |
+                          @tsv'
+
+.. code-block:: bash
+
+  249     102     0       connected       STATE_CONNECTION_ESTABLISHED    client.admin.6407       crc    PLAIN   UNCOMPRESSED
+  242     99      1       connected       STATE_CONNECTION_ESTABLISHED    client.rgw.8000.4473    crc     PLAIN   UNCOMPRESSED
+  248     89      1       connected       STATE_CONNECTION_ESTABLISHED    mgr..-1 secure  AES-128-GCM     UNCOMPRESSED
+  32      101     2       connected       STATE_CONNECTION_ESTABLISHED    client.rgw.8000.4483    crc     PLAIN   UNCOMPRESSED
+  3       86      2       connected       STATE_CONNECTION_ESTABLISHED    mon..-1 secure  AES-128-GCM     UNCOMPRESSED
+  244     102     0       connected       STATE_CONNECTION_ESTABLISHED    client.admin.6383       crc     PLAIN   UNCOMPRESSED
+
+
+Print active connections and their TCP round trip time and retransmission counters:
+
+.. code-block:: bash
+
+           ceph tell osd.0 messenger dump client --tcp-info \
+              | jq -r '.messenger.connections[].async_connection |
+                          select(.status.connected) |
+                          select(.peer.type != "client") |
+                          [.conn_id, .socket_fd, .worker_id,
+                           "\(.peer.type).\(.peer.global_id)",
+                           .tcp_info.tcpi_rtt_us, .tcp_info.tcpi_rttvar_us, .tcp_info.tcpi_total_retrans] |
+                           @tsv'
+.. code-block:: bash
+
+       248     89      1       mgr.0   863     1677    0
+       3       86      2       mon.0   230     278     0