osd: do not reply to ping if internal heartbeat is not healthy

If we find that our internal threads are stalled, do not reply to ping
requests.  If we do this long enough, peers will mark us down.  If we are
only transiently unhealthy, we will reply to the next ping and they will
be satisfied.  If we are unhealthy and marked down, and eventually recover,
we will mark ourselves back up.

Signed-off-by: Sage Weil <sage@inktank.com>
(cherry picked from commit ad6b231127a6bfcbed600a7493ca3b66c68484d2)

diff --git a/src/osd/OSD.cc b/src/osd/OSD.cc
index 2662947f3f5e86f2ae426268667a6e6e2b4e073b..1835bbb98ebad0fb4c22604e21b3a71a4573a1db 100644 (file)
--- a/src/osd/OSD.cc
+++ b/src/osd/OSD.cc
@@ -2020,6 +2020,12 @@ void OSD::handle_osd_ping(MOSDPing *m)
          break;
        }
       }
+
+      if (!g_ceph_context->get_heartbeat_map()->is_healthy()) {
+       dout(10) << "internal heartbeat not healthy, dropping ping request" << dendl;
+       break;
+      }
+
       Message *r = new MOSDPing(monc->get_fsid(),
                                curmap->get_epoch(),
                                MOSDPing::PING_REPLY,