osds: wait for osds to be up before creating pools

This is a follow up on #2628.
Even with the openstack pools creation moved later in the playbook,
there is still an issue because OSDs are not all UP when trying to
create pools.

Adding a task which checks for all OSDs to be UP with a `retries/until`
condition should definitively fix this issue.

Closes: https://bugzilla.redhat.com/show_bug.cgi?id=1578086
Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>

diff --git a/roles/ceph-osd/tasks/openstack_config.yml b/roles/ceph-osd/tasks/openstack_config.yml
index 238abd8f28155359fb0141faa2c5ebbc0656b208..1884a12e19e3e53ec13fc9c45de665e3a644675c 100644 (file)
--- a/roles/ceph-osd/tasks/openstack_config.yml
+++ b/roles/ceph-osd/tasks/openstack_config.yml
@@ -1,4 +1,14 @@
 ---
+- name: wait for all osd to be up
+  shell: >
+    test "$({{ hostvars[groups[mon_group_name][0]]['docker_exec_cmd'] | default('') }} ceph --cluster {{ cluster }} -s -f json | python -c 'import sys, json; print(json.load(sys.stdin)["osdmap"]["osdmap"]["num_osds"])')" = 
+    "$({{ hostvars[groups[mon_group_name][0]]['docker_exec_cmd'] | default('') }} ceph --cluster {{ cluster }} -s -f json | python -c 'import sys, json; print(json.load(sys.stdin)["osdmap"]["osdmap"]["num_up_osds"])')"
+  register: wait_for_all_osds_up
+  retries: 30
+  delay: 10
+  delegate_to: "{{ groups[mon_group_name][0] }}"
+  until: wait_for_all_osds_up.rc == 0
+
 - name: list existing pool(s)
   command: >
     {{ hostvars[groups[mon_group_name][0]]['docker_exec_cmd'] | default('') }} ceph --cluster {{ cluster }}