From 106d31462e89e4df40bb1ed64eec523dd74b0002 Mon Sep 17 00:00:00 2001 From: Sage Weil Date: Thu, 25 Mar 2021 18:07:53 -0400 Subject: [PATCH] mon/HealthMonitor: raise AUTH_INSECURE_GLOBAL_ID_RENEWAL[_ALLOWED] Two new alerts: - AUTH_INSECURE_GLOBAL_ID_RENEWAL_ALLOWED if we are allowing clients to reclaim global_ids in an insecure manner (for backwards compatibility until clients are upgraded) - AUTH_INSECURE_GLBOAL_ID_RENEWAL if there are currently clients connected that do not know how to securely renew their global_id, as exposed by auth_expose_insecure_global_id_reclaim=true. The client auth names and IPs are listed the alert details (up to a limit, at least). The docs recommend operators mute these alerts instead of silencing, but we still include option that allow the alerts to be disabled entirely. Signed-off-by: Sage Weil (cherry picked from commit 18b343b06e5dd904af425dc99e2c848e12f3b552) Conflicts: src/mon/HealthMonitor.cc [ commit e4bf716bfa07 ("mon: store a reference as member variable") not in octopus ] --- doc/rados/operations/health-checks.rst | 67 ++++++++++++++++++++++++++ src/common/options.cc | 20 ++++++++ src/mon/HealthMonitor.cc | 39 +++++++++++++++ 3 files changed, 126 insertions(+) diff --git a/doc/rados/operations/health-checks.rst b/doc/rados/operations/health-checks.rst index 32169dab333d5..87f4bd6c927c1 100644 --- a/doc/rados/operations/health-checks.rst +++ b/doc/rados/operations/health-checks.rst @@ -122,6 +122,73 @@ The warning threshold may be adjusted with:: ceph config set global mon_data_size_warn +AUTH_INSECURE_GLOBAL_ID_RECLAIM +_______________________________ + +One or more clients or daemons are connected to the cluster that are +not securely reclaiming their global_id (a unique number identifying +each entity in the cluster) when reconnecting to a monitor. The +client is being permitted to connect anyway because the +``auth_allow_insecure_global_id_reclaim`` option is set to true (which may +be necessary until all ceph clients have been upgraded), and the +``auth_expose_insecure_global_id_reclaim`` option set to ``true`` (which +allows monitors to detect clients with insecure reclaim early by forcing them to +reconnect right after they first authenticate). + +You can identify which client(s) are using unpatched ceph client code with:: + + ceph health detail + +Clients global_id reclaim rehavior can also seen in the +``global_id_status`` field in the dump of clients connected to an +individual monitor (``reclaim_insecure`` means the client is +unpatched and is contributing to this health alert):: + + ceph tell mon.\* sessions + +We strongly recommend that all clients in the system are upgraded to a +newer version of Ceph that correctly reclaims global_id values. Once +all clients have been updated, you can stop allowing insecure reconnections +with:: + + ceph config set mon auth_allow_insecure_global_id_reclaim false + +If it is impractical to upgrade all clients immediately, you can silence +this warning temporarily with:: + + ceph health mute AUTH_INSECURE_GLOBAL_ID_RECLAIM 1w # 1 week + +Although we do NOT recommend doing so, you can also disable this warning indefinitely +with:: + + ceph config set mon mon_warn_on_insecure_global_id_reclaim false + +AUTH_INSECURE_GLOBAL_ID_RECLAIM_ALLOWED +_______________________________________ + +Ceph is currently configured to allow clients to reconnect to monitors using +an insecure process to reclaim their previous global_id because the setting +``auth_allow_insecure_global_id_reclaim`` is set to ``true``. It may be necessary to +leave this setting enabled while existing Ceph clients are upgraded to newer +versions of Ceph that correctly and securely reclaim their global_id. + +If the ``AUTH_INSECURE_GLOBAL_ID_RECLAIM`` health alert has not also been raised and +the ``auth_expose_insecure_global_id_reclaim`` setting has not been disabled (it is +on by default), then there are currently no clients connected that need to be +upgraded, and it is safe to disallow insecure global_id reclaim with:: + + ceph config set mon auth_allow_insecure_global_id_reclaim false + +If there are still clients that need to be upgraded, then this alert can be +silenced temporarily with:: + + ceph health mute AUTH_INSECURE_GLOBAL_ID_RECLAIM_ALLOWED 1w # 1 week + +Although we do NOT recommend doing so, you can also disable this warning indefinitely +with:: + + ceph config set mon mon_warn_on_insecure_global_id_reclaim_allowed false + Manager ------- diff --git a/src/common/options.cc b/src/common/options.cc index 3d4a16edede8f..4ce2689c234e4 100644 --- a/src/common/options.cc +++ b/src/common/options.cc @@ -1699,6 +1699,22 @@ std::vector