Disable asynchronous DNS lookups.

Especially on older hosts, we keep triggering errors::

  ServerNotFoundError: Unable to find the server at
  teuthology.front.sepia.ceph.com: [Errno 3] name does not exist

That comes from libevent's evdns via gevent.dns and httplib2. The rate
of these errors is low enough that they seem to be perhaps timeouts,
or more arbitrary. Busy looping on DNS resolution calls has never
triggered them, so far.

With ``monkey.patch_all(dns=False)``, the teuthology process will
block as a whole whenever doing DNS resolution. This will hopefully be
rare enough that it won't matter.

The only real "fix" seems to be upgrading libraries and hoping for the
best; this commit can be reverted after that is done.

diff --git a/teuthology/nuke.py b/teuthology/nuke.py
index 536ebbda7359100550e331bc2d243e6732f32616..6bb0166e9c76d04985a84fbf481ce512712eacae 100644 (file)
--- a/teuthology/nuke.py
+++ b/teuthology/nuke.py
@@ -241,7 +241,7 @@ def synch_clocks(remotes, log):
         proc.exitstatus.get()
 
 def main():
-    from gevent import monkey; monkey.patch_all()
+    from gevent import monkey; monkey.patch_all(dns=False)
     from .orchestra import monkey; monkey.patch_all()
     from teuthology.run import config_file
 

diff --git a/teuthology/run.py b/teuthology/run.py
index b95c0dc53cbb4681fea99d55bec46612e61bde1d..3ebd2af5414fc06f38159f4655a57c48c4649521 100644 (file)
--- a/teuthology/run.py
+++ b/teuthology/run.py
@@ -66,7 +66,7 @@ def parse_args():
     return args
 
 def main():
-    from gevent import monkey; monkey.patch_all()
+    from gevent import monkey; monkey.patch_all(dns=False)
     from .orchestra import monkey; monkey.patch_all()
 
     import logging