From: xie xingguo Date: Thu, 7 Apr 2016 08:43:17 +0000 (+0800) Subject: mon: OSDMonitor: trigger an immediate propose if any newly down osd is detected durin... X-Git-Tag: v11.0.0~850^2 X-Git-Url: http://git-server-git.apps.pok.os.sepia.ceph.com/?a=commitdiff_plain;h=a80d6c500ab247013a1c068c457c1b9bfbc750b2;p=ceph.git mon: OSDMonitor: trigger an immediate propose if any newly down osd is detected during tick() Currently we rely on OSDs to watch over each other and perform failure detection and report to OSDMonitor. Before we can safely and undoubtedly mark an OSD as down, enough reports from a certain number of different reporters must have been collected. Also, the victimed OSD has to be declared failed long enough before we make any final decision in order to avoid temperary problems such as network failure, network traffic jam etc., which if handled carelessly, may cause even serious problem such as flapping. Form the above analysis, even if we have gathered enough witnesses, we have to wait long enough to sentence the guilty OSD to death. Therefore we rely on the tick() thread to do such an hourglass job. However, the problem here is currently the tick() thread is unable to trigger a propose even if it has witnessed such a memont, and this is our goal to solve such an embrassing situation. Signed-off-by: xie xingguo --- diff --git a/src/mon/OSDMonitor.cc b/src/mon/OSDMonitor.cc index 2f3e02f07728..eb077b0e4446 100644 --- a/src/mon/OSDMonitor.cc +++ b/src/mon/OSDMonitor.cc @@ -1694,15 +1694,17 @@ bool OSDMonitor::can_mark_in(int i) return true; } -void OSDMonitor::check_failures(utime_t now) +bool OSDMonitor::check_failures(utime_t now) { + bool found_failure = false; for (map::iterator p = failure_info.begin(); p != failure_info.end(); ++p) { if (can_mark_down(p->first)) { - check_failure(now, p->first, p->second); + found_failure |= check_failure(now, p->first, p->second); } } + return found_failure; } bool OSDMonitor::check_failure(utime_t now, int target_osd, failure_info_t& fi) @@ -2666,7 +2668,8 @@ void OSDMonitor::tick() utime_t now = ceph_clock_now(g_ceph_context); // mark osds down? - check_failures(now); + if (check_failures(now)) + do_propose = true; // mark down osds out? diff --git a/src/mon/OSDMonitor.h b/src/mon/OSDMonitor.h index 9d4d33d239bf..3451b6a7e071 100644 --- a/src/mon/OSDMonitor.h +++ b/src/mon/OSDMonitor.h @@ -134,7 +134,7 @@ private: SimpleLRU inc_osd_cache; SimpleLRU full_osd_cache; - void check_failures(utime_t now); + bool check_failures(utime_t now); bool check_failure(utime_t now, int target_osd, failure_info_t& fi); // map thrashing