mgr: wait for all mgr to be available

before managing mgr modules, we must ensure all mgr are available
otherwise we can hit failure like following:

```
stdout:Error ENOENT: all mgr daemons do not support module 'restful', pass --force to force enablement
```

It happens because all mgr are not yet available when trying to manage
with mgr modules.

Closes: #3100
Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
(cherry picked from commit f596cc1711428ee801fc04be2a94b36a5e342a00)

diff --git a/roles/ceph-mgr/tasks/mgr_modules.yml b/roles/ceph-mgr/tasks/mgr_modules.yml
index 2cb68fe1d2d0c10a35e2e6fdad8313708171c174..e9ee82087a0f2f3df7eb4e681be6293def2ba58d 100644 (file)
--- a/roles/ceph-mgr/tasks/mgr_modules.yml
+++ b/roles/ceph-mgr/tasks/mgr_modules.yml
@@ -1,4 +1,14 @@
 ---
+- name: wait for all mgr to be up
+  shell: "{{ docker_exec_cmd | default('') }} ceph --cluster {{ cluster }} mgr dump -f json | python -c 'import sys, json; print(json.load(sys.stdin)[\"available\"])'"
+  register: mgr_dump
+  retries: 30
+  delay: 5
+  delegate_to: "{{ groups[mon_group_name][0] }}"
+  until:
+    - mgr_dump.rc == 0
+    - mgr_dump.stdout | bool
+
 - name: get enabled modules from ceph-mgr
   command: "{{ docker_exec_cmd_mgr | default('') }} ceph --cluster {{ cluster }} --format json mgr module ls"
   changed_when: false