doc: Mention PURGED_SNAPDIRS and RECOVERY_DELETES in Mimic release notes

71d1dcbb98bf1b3c7953c13a1165ba1cac7e7fd0 started enforcing the
PURGED_SNAPDIRS osdmap flag before am upgraded Mon could join the
quorum.

9b80b14783ef895390b4153320078661627f373d extended that check to also
include RECOVERY_DELETES. The Mimic release notes do not mention these
flags as prerequisites for an upgrade beyond Luminous. That creates an
obvious issue for users who skipped Luminous in production, and now
want to upgrade from Jewel to Mimic in, say, a weekend.

Update the release notes to include those flags as prerequisites for a
Luminous to Mimic upgrade, explain how users can make sure that they
are set, and also give users a one-liner to fix up their PGs in a
pinch, if they need to.

Signed-off-by: Florian Haas <florian@citynetwork.eu>

diff --git a/doc/releases/mimic.rst b/doc/releases/mimic.rst
index 1fd93e8f7c4fae725384a61f8a796b4e4cc21d35..bcffe5d3319f88d09c2603ced35decfb1901f517 100644 (file)
--- a/doc/releases/mimic.rst
+++ b/doc/releases/mimic.rst
@@ -69,6 +69,36 @@ Notes
 Instructions
 ~~~~~~~~~~~~
 
+#. If your cluster was originally installed with a version prior to
+   Luminous, ensure that it has completed at least one full scrub of
+   all PGs while running Luminous.  Failure to do so will cause your
+   monitor daemons to refuse to join the quorum on start, leaving them
+   non-functional.
+
+   If you are unsure whether or not your Luminous cluster has
+   completed a full scrub of all PGs, you can check your cluster's
+   state by running::
+
+     # ceph osd dump | grep ^flags
+
+   In order to be able to proceed to Mimic, your OSD map must include
+   the ``recovery_deletes`` and ``purged_snapdirs`` flags.
+
+   If your OSD map does not contain both these flags, you can simply
+   wait for approximately 24-48 hours, which in a standard cluster
+   configuration should be ample time for all your placement groups to
+   be scrubbed at least once, and then repeat the above process to
+   recheck.
+
+   However, if you have just completed an upgrade to Luminous and want
+   to proceed to Mimic in short order, you can force a scrub on all
+   placement groups with a one-line shell command, like::
+
+     # ceph pg dump pgs_brief | cut -d " " -f 1 | xargs -n1 ceph pg scrub
+
+   You should take into consideration that this forced scrub may
+   possibly have a negative impact on your Ceph clients' performance.
+
 #. Make sure your cluster is stable and healthy (no down or
    recoverying OSDs).  (Optional, but recommended.)
 
@@ -177,7 +207,9 @@ Upgrading from pre-Luminous releases (like Jewel)
 -------------------------------------------------
 
 You *must* first upgrade to Luminous (12.2.z) before attempting an
-upgrade to Mimic.
+upgrade to Mimic.  In addition, your cluster must have completed at
+least one scrub of all PGs while running Luminous, setting the
+``recovery_deletes`` and ``purged_snapdirs`` flags in the OSD map.
 
 Upgrade compatibility notes
 ---------------------------