iscsi: Fix crashes during rolling update

During a rolling update we will run the ceph iscsigw tasks that start
the daemons then run the configure_iscsi.yml tasks which can create
iscsi objects like targets, disks, clients, etc. The problem is that
once the daemons are started they will accept confifguration requests,
or may want to update the system themself. Those operations can then
conflict with the configure_iscsi.yml tasks that setup objects and we
can end up in crashes due to the kernel being in a unsupported state.

This could also happen during creation, but is less likely due to no
objects being setup yet, so there are no watchers or users accessing the
gws yet. The fix in this patch works for both update and initial setup.

Resolves: https://bugzilla.redhat.com/show_bug.cgi?id=1795806

Signed-off-by: Mike Christie <mchristi@redhat.com>
(cherry picked from commit 77f3b5d51b84a6338847c5f6a93f22a3a6a683d2)

diff --git a/roles/ceph-iscsi-gw/tasks/main.yml b/roles/ceph-iscsi-gw/tasks/main.yml
index 77cfe618e19ff633974aebcd46b75f78ace1eabf..297eb4d884a440f65e7b084d383f231a9d364804 100644 (file)
--- a/roles/ceph-iscsi-gw/tasks/main.yml
+++ b/roles/ceph-iscsi-gw/tasks/main.yml
@@ -20,6 +20,10 @@
   when:
     - not containerized_deployment
 
+- name: include non-container/postrequisites.yml
+  include_tasks: non-container/postrequisites.yml
+  when: not containerized_deployment | bool
+
 - name: include containerized.yml
   include_tasks: container/containerized.yml
   when:

diff --git a/roles/ceph-iscsi-gw/tasks/non-container/postrequisites.yml b/roles/ceph-iscsi-gw/tasks/non-container/postrequisites.yml
new file mode 100644 (file)
index 0000000..5242525
--- /dev/null
+++ b/roles/ceph-iscsi-gw/tasks/non-container/postrequisites.yml
@@ -0,0 +1,9 @@
+- name: start rbd-target-api and rbd-target-gw
+  service:
+    name: "{{ item }}"
+    state: started
+    enabled: yes
+    masked: no
+  with_items:
+    - rbd-target-gw
+    - rbd-target-api

diff --git a/roles/ceph-iscsi-gw/tasks/non-container/prerequisites.yml b/roles/ceph-iscsi-gw/tasks/non-container/prerequisites.yml
index f3ed7e738b7c89ebb7f7acb22dd52d2252e9dc48..56e69be0ffdf0ae2a63063d0552f84a84accc0c7 100644 (file)
--- a/roles/ceph-iscsi-gw/tasks/non-container/prerequisites.yml
+++ b/roles/ceph-iscsi-gw/tasks/non-container/prerequisites.yml
@@ -89,14 +89,13 @@
     - target.stat.exists
     - target.stat.islnk == False
 
-- name: enable the rbd-target-gw service and make sure it is running
+# Only start tcmu-runner, so configure_iscsi.yml can create disks.
+# We must start rbd-target-gw/api after configure_iscsi.yml to avoid
+# races where they are both trying to setup the same object during
+# a rolling update.
+- name: start tcmu-runner
   service:
-    name: rbd-target-gw
-    enabled: yes
+    name: tcmu-runner
     state: started
-
-- name: enable the rbd-target-api service and make sure it is running
-  service:
-    name: rbd-target-api
     enabled: yes
-    state: started
+    masked: no