From 4ac2a3e5f764dae9315358d11d7b00e9a08610b1 Mon Sep 17 00:00:00 2001 From: Aashish Sharma Date: Fri, 29 Apr 2022 10:43:45 +0530 Subject: [PATCH] doc: update docs for centralized logging Signed-off-by: Aashish Sharma --- doc/cephadm/services/monitoring.rst | 22 +++++++++++++++++++++ doc/mgr/dashboard.rst | 30 +++++++++++++++++++++++++++++ 2 files changed, 52 insertions(+) diff --git a/doc/cephadm/services/monitoring.rst b/doc/cephadm/services/monitoring.rst index a17beba6d1e..5f9d8289095 100644 --- a/doc/cephadm/services/monitoring.rst +++ b/doc/cephadm/services/monitoring.rst @@ -83,6 +83,28 @@ steps below: ceph orch apply grafana +.. _cephadm-monitoring-centralized-logs: + +Centralized Logging in Ceph +~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Ceph now provides centralized logging with Loki & Promtail. Centralized Log Management (CLM) consolidates all log data and pushes it to a central repository, +with an accessible and easy-to-use interface. Centralized logging is designed to make your life easier. +Some of the advantages are: + +#. **Linear event timeline**: it is easier to troubleshoot issues analyzing a single chain of events than thousands of different logs from a hundred nodes. +#. **Real-time live log monitoring**: it is impractical to follow logs from thousands of different sources. +#. **Flexible retention policies**: with per-daemon logs, log rotation is usually set to a short interval (1-2 weeks) to save disk usage. +#. **Increased security & backup**: logs can contain sensitive information and expose usage patterns. Additionally, centralized logging allows for HA, etc. + +Centralized Logging in Ceph is implemented using two new services - ``loki`` & ``promtail``. + +Loki: It is basically a log aggregation system and is used to query logs. It can be configured as a datasource in Grafana. + +Promtail: It acts as an agent that gathers logs from the system and makes them available to Loki. + +These two services are not deployed by default in a Ceph cluster. To enable the centralized logging you can follow the steps mentioned here :ref:`centralized-logging`. + .. _cephadm-monitoring-networks-ports: Networks and Ports diff --git a/doc/mgr/dashboard.rst b/doc/mgr/dashboard.rst index 60fca266e3e..11fcf482b58 100644 --- a/doc/mgr/dashboard.rst +++ b/doc/mgr/dashboard.rst @@ -1369,6 +1369,36 @@ something like this:: ... $ ceph config reset 11 +.. _centralized-logging: + +Enable Centralized Logging in Dashboard +""""""""""""""""""""""""""""""""""""""" + +To learn more about centralized logging, see :ref:`cephadm-monitoring-centralized-logs` + +1. Create the Loki service on any particular host using "Create Services" option. + +2. Similarly create the Promtail service which will be by default deployed + on all the running hosts. + +3. To see debug-level messages as well as info-level events, run the following command via CLI:: + + $ ceph config set mgr mgr/cephadm/log_to_cluster_level debug + +4. To enable logging to files, run the following commands via CLI:: + + $ ceph config set global log_to_file true + + $ ceph config set global mon_cluster_log_to_file true + +5. Click on the Daemon Logs tab under Cluster -> Logs. + +6. You can find some pre-defined labels there on clicking the Log browser button such as filename, + job etc that can help you query the logs at one go. + +7. You can query the logs with LogQL for advanced search and perform some + calculations as well - https://grafana.com/docs/loki/latest/logql/. + Reporting issues from Dashboard """"""""""""""""""""""""""""""" -- 2.39.5