testnode: retry when updating the apt cache to handle http errors

When updating the apt cache we have to make an http call out to
gitbuilder, if gitbuilder fails for whatever reason we should retry
before failing the playbook.

Also, this removes a 'changed_when: false' on 'Add local apt repos'.
We check it's 'changed' status to know if we need update the cache or
not. We only added that 'changed_when' to avoid seeing this task being
changed each time (because it will always change), but we actually need
that value set correctly because we use it in the 'when' clause of the
updating apt cache stanza.

Signed-off-by: Andrew Schoen <aschoen@redhat.com>

diff --git a/roles/testnode/tasks/apt/repos.yml b/roles/testnode/tasks/apt/repos.yml
index e233954c9c7ff0ad797d8f2e934789abcac8dc7f..f8a9ce424dd28fd0aec78b85a27bab34abbaa188 100644 (file)
--- a/roles/testnode/tasks/apt/repos.yml
+++ b/roles/testnode/tasks/apt/repos.yml
@@ -52,8 +52,6 @@
     mode: 0644
   with_items: apt_repos|list + common_apt_repos|list
   register: local_apt_repos
-  # Ignore changes here because we might have just removed these repos seconds ago
-  changed_when: false
 
 - name: Update apt cache.
   apt:
@@ -61,3 +59,9 @@
   when: sources|changed or
         local_apt_repos|changed or
         apt_prefs|changed
+  # Register and retry to work around transient http issues
+  register: apt_cache_update
+  until: apt_cache_update|success
+  # try for 2 minutes before failing
+  retries: 24
+  delay: 5