From: Samuel Just <sjust@redhat.com>
Date: Mon, 28 Mar 2016 21:02:05 +0000 (-0700)
Subject: parallel: parallel.__exit__ should always wait for greenlets to finish
X-Git-Tag: 1.1.0~636^2
X-Git-Url: http://git-server-git.apps.pok.os.sepia.ceph.com/?a=commitdiff_plain;h=78b602b7f75a9f45273976a195a8b5f1bac736cd;p=teuthology.git

parallel: parallel.__exit__ should always wait for greenlets to finish

Per the class docstring, one gevent (or the with body) throwing an
exception should not stop the other events.  Instead, wait for them to
finish and then reraise the first exception we find.

A concrete problem caused by this is that with multiple parallel
installs, one failure can cause packaging operations on the other nodes
to be unceremoniously killed and left in an inconsistent state.

Signed-off-by: Samuel Just <sjust@redhat.com>
---

diff --git a/teuthology/parallel.py b/teuthology/parallel.py
index 13e426e57..ac47165a5 100644
--- a/teuthology/parallel.py
+++ b/teuthology/parallel.py
@@ -73,8 +73,9 @@ class parallel(object):
         return self
 
     def __exit__(self, type_, value, traceback):
+        self.group.join()
+
         if value is not None:
-            self.group.kill(block=True)
             return False
 
         try:
@@ -85,7 +86,6 @@ class parallel(object):
         except Exception:
             # Emit message here because traceback gets stomped when we re-raise
             log.exception("Exception in parallel execution")
-            self.group.kill(block=True)
             raise
         return True