From 63f93d524c75b81e7c7a65550171b99d02a1e39b Mon Sep 17 00:00:00 2001 From: Florian Haas Date: Tue, 26 Jun 2018 14:26:06 +0200 Subject: [PATCH] doc: Mention PURGED_SNAPDIRS and RECOVERY_DELETES in Mimic release notes 71d1dcbb98bf1b3c7953c13a1165ba1cac7e7fd0 started enforcing the PURGED_SNAPDIRS osdmap flag before am upgraded Mon could join the quorum. 9b80b14783ef895390b4153320078661627f373d extended that check to also include RECOVERY_DELETES. The Mimic release notes do not mention these flags as prerequisites for an upgrade beyond Luminous. That creates an obvious issue for users who skipped Luminous in production, and now want to upgrade from Jewel to Mimic in, say, a weekend. Update the release notes to include those flags as prerequisites for a Luminous to Mimic upgrade, explain how users can make sure that they are set, and also give users a one-liner to fix up their PGs in a pinch, if they need to. Signed-off-by: Florian Haas --- doc/releases/mimic.rst | 34 +++++++++++++++++++++++++++++++++- 1 file changed, 33 insertions(+), 1 deletion(-) diff --git a/doc/releases/mimic.rst b/doc/releases/mimic.rst index 1fd93e8f7c4fa..bcffe5d3319f8 100644 --- a/doc/releases/mimic.rst +++ b/doc/releases/mimic.rst @@ -69,6 +69,36 @@ Notes Instructions ~~~~~~~~~~~~ +#. If your cluster was originally installed with a version prior to + Luminous, ensure that it has completed at least one full scrub of + all PGs while running Luminous. Failure to do so will cause your + monitor daemons to refuse to join the quorum on start, leaving them + non-functional. + + If you are unsure whether or not your Luminous cluster has + completed a full scrub of all PGs, you can check your cluster's + state by running:: + + # ceph osd dump | grep ^flags + + In order to be able to proceed to Mimic, your OSD map must include + the ``recovery_deletes`` and ``purged_snapdirs`` flags. + + If your OSD map does not contain both these flags, you can simply + wait for approximately 24-48 hours, which in a standard cluster + configuration should be ample time for all your placement groups to + be scrubbed at least once, and then repeat the above process to + recheck. + + However, if you have just completed an upgrade to Luminous and want + to proceed to Mimic in short order, you can force a scrub on all + placement groups with a one-line shell command, like:: + + # ceph pg dump pgs_brief | cut -d " " -f 1 | xargs -n1 ceph pg scrub + + You should take into consideration that this forced scrub may + possibly have a negative impact on your Ceph clients' performance. + #. Make sure your cluster is stable and healthy (no down or recoverying OSDs). (Optional, but recommended.) @@ -177,7 +207,9 @@ Upgrading from pre-Luminous releases (like Jewel) ------------------------------------------------- You *must* first upgrade to Luminous (12.2.z) before attempting an -upgrade to Mimic. +upgrade to Mimic. In addition, your cluster must have completed at +least one scrub of all PGs while running Luminous, setting the +``recovery_deletes`` and ``purged_snapdirs`` flags in the OSD map. Upgrade compatibility notes --------------------------- -- 2.39.5