From 3c640078a5d9571a8974cde18e029bdd3723dd82 Mon Sep 17 00:00:00 2001 From: Tim Serong Date: Fri, 30 Jun 2017 17:24:21 +1000 Subject: [PATCH] systemd: Add explicit Before=ceph.target The PartOf= and WantedBy= directives in the various systemd unit files and targets create the following logical hierarchy: - ceph.target - ceph-fuse.target - ceph-fuse@.service - ceph-mds.target - ceph-mds@.service - ceph-mgr.target - ceph-mgr@.service - ceph-mon.target - ceph-mon@.service - ceph-osd.target - ceph-osd@.service - ceph-radosgw.target - ceph-radosgw@.service - ceph-rbd-mirror.target - ceph-rbd-mirror@.service Additionally, the ceph-{fuse,mds,mon,osd,radosgw,rbd-mirror} targets have WantedBy=multi-user.target. This gives the following behaviour: - `systemctl {start,stop,restart}` of any target will restart all dependent services (e.g.: `systemctl restart ceph.target` will restart all services; `systemctl restart ceph-mon.target` will restart all the mons, and so forth). - `systemctl {enable,disable}` for the second level targets (ceph-mon.target etc.) will cause depenent services to come up on boot, or not (of course the individual services can be enabled or disabled as well - for a service to start on boot, both the service and its target must be enabled; disabling either will cause the service to be disabled). - `systemctl {enable,disable} ceph.target` has no effect on whether or not services come up at boot; if the second level targets and services are enabled, they'll start regardless of whether ceph.target is enabled. This is due to the second level targets all having WantedBy=multi-user.target. - The OSDs will always start regardless of ceph-osd.target (unless they are explicitly masked), thanks to udev magic. So far, so good. Except, several users have encountered services not starting with the following error: Failed to start ceph-osd@5.service: Transaction order is cyclic. See system logs for details. I've not been able to reproduce this myself in such a way as to cause OSDs to fail to start, but I *have* managed to get systemd into that same confused state, as follows: - Disable ceph.target, ceph-mon.target, ceph-osd.target, ceph-mon@$(hostname).service and all ceph-osd instances. - Re-enable all of the above. At this point, everything is fine, but if I then subseqently disable ceph.target, *then* try `systemctl restart ceph.target`, I get "Failed to restart ceph.target: Transaction order is cyclic. See system logs for details." Explicitly adding Before=ceph.target to each second level target prevents systemd from becoming confused in this situation. Signed-off-by: Tim Serong (cherry picked from commit 357dfa5954a3386c92dee3289a669a2c08d41eda) Conflicts: systemd/ceph-fuse.target (file does not exist in jewel; omitted) systemd/ceph-mgr.target (file does not exist in jewel; omitted) --- systemd/ceph-mds.target | 1 + systemd/ceph-mon.target | 1 + systemd/ceph-osd.target | 1 + systemd/ceph-radosgw.target | 1 + systemd/ceph-rbd-mirror.target | 1 + 5 files changed, 5 insertions(+) diff --git a/systemd/ceph-mds.target b/systemd/ceph-mds.target index fbf1ba14a8b9d..238f3ab90c73e 100644 --- a/systemd/ceph-mds.target +++ b/systemd/ceph-mds.target @@ -1,5 +1,6 @@ [Unit] Description=ceph target allowing to start/stop all ceph-mds@.service instances at once PartOf=ceph.target +Before=ceph.target [Install] WantedBy=multi-user.target ceph.target diff --git a/systemd/ceph-mon.target b/systemd/ceph-mon.target index 87b585f83cae7..097c83ba3be83 100644 --- a/systemd/ceph-mon.target +++ b/systemd/ceph-mon.target @@ -1,5 +1,6 @@ [Unit] Description=ceph target allowing to start/stop all ceph-mon@.service instances at once PartOf=ceph.target +Before=ceph.target [Install] WantedBy=multi-user.target ceph.target diff --git a/systemd/ceph-osd.target b/systemd/ceph-osd.target index ed55fc2882e33..7f677f54d5ece 100644 --- a/systemd/ceph-osd.target +++ b/systemd/ceph-osd.target @@ -1,5 +1,6 @@ [Unit] Description=ceph target allowing to start/stop all ceph-osd@.service instances at once PartOf=ceph.target +Before=ceph.target [Install] WantedBy=multi-user.target ceph.target diff --git a/systemd/ceph-radosgw.target b/systemd/ceph-radosgw.target index 959eb51ea7bc6..1799e29e84d74 100644 --- a/systemd/ceph-radosgw.target +++ b/systemd/ceph-radosgw.target @@ -1,5 +1,6 @@ [Unit] Description=ceph target allowing to start/stop all ceph-radosgw@.service instances at once PartOf=ceph.target +Before=ceph.target [Install] WantedBy=multi-user.target ceph.target diff --git a/systemd/ceph-rbd-mirror.target b/systemd/ceph-rbd-mirror.target index 7bd94e06e885a..43e9a4c816478 100644 --- a/systemd/ceph-rbd-mirror.target +++ b/systemd/ceph-rbd-mirror.target @@ -1,5 +1,6 @@ [Unit] Description=ceph target allowing to start/stop all ceph-rbd-mirror@.service instances at once PartOf=ceph.target +Before=ceph.target [Install] WantedBy=multi-user.target ceph.target -- 2.39.5