ceph_manager: workaround for 6116

This is an annoying race, we really should delay going
clean until the backfill peer has acknoledged the clean
info, but we currently don't.  In order to prevent this
bug from messing up the nightlies, we'll delay killing
the peer for 20s to make it likely that the backfill
peer has gotten the clean info.

Workaround: #6116
Signed-off-by: Samuel Just <sam.just@inktank.com>

diff --git a/teuthology/task/ceph_manager.py b/teuthology/task/ceph_manager.py
index 18f897641c0b53bdc3ad82341916d4e5157720b2..286d0c8f856a512f5d80cc34314974b233290461 100644 (file)
--- a/teuthology/task/ceph_manager.py
+++ b/teuthology/task/ceph_manager.py
@@ -219,6 +219,10 @@ class Thrasher:
         self.ceph_manager.wait_for_clean(
             timeout=self.config.get('timeout')
             )
+        # now we wait 20s to ensure that any backfill peers have heard about
+        # the cleanness
+        time.sleep(20)
+
         self.log("Recovered, killing an osd")
         self.kill_osd(mark_down=True, mark_out=True)
         self.log("Waiting for clean again")