From: Kefu Chai Date: Fri, 24 Apr 2026 13:41:29 +0000 (+0800) Subject: doc/rados: document require_osd_release X-Git-Url: http://git-server-git.apps.pok.os.sepia.ceph.com/?a=commitdiff_plain;h=refs%2Fpull%2F68608%2Fhead;p=ceph.git doc/rados: document require_osd_release Add a standalone concept page for the OSDMap require_osd_release field, the upgrade-gate counterpart to require_min_compat_client. Cover: - how to set it and how to check it; - the full set of pre-commit guards the monitor runs, rendered as a table with each guard's error text and bypass status; - which commands and features become available as the flag is raised, per release; - the OSD boot window that refuses OSDs more than two releases ahead of the flag; - the OSD_UPGRADE_FINISHED health warning that prompts admins to set the flag after an upgrade; - the initial value on new clusters and the two mon_debug_* knobs that override it for testing. Also cross-link the new page from the related-flags table on require-min-compat-client.rst, and from the rados operations index. Signed-off-by: Kefu Chai --- diff --git a/doc/rados/operations/health-checks.rst b/doc/rados/operations/health-checks.rst index 136bffac4b2c..1954d9df83ef 100644 --- a/doc/rados/operations/health-checks.rst +++ b/doc/rados/operations/health-checks.rst @@ -647,6 +647,21 @@ Since migration of Filestore OSDs to BlueStore can take a considerable amount of time to complete, we recommend that you begin the process well in advance of any update to Reef or to later releases. +OSD_UPGRADE_FINISHED +____________________ + +All up OSDs are running a release newer than ``require_osd_release``, +which means the cluster is ready to advance the gate but the operator +has not yet done so. + +Run the following command to clear the warning: + +.. prompt:: bash # + + ceph osd require-osd-release + +See :ref:`OSD_UPGRADE_FINISHED` for full details. + OSD_UNREACHABLE _______________ diff --git a/doc/rados/operations/index.rst b/doc/rados/operations/index.rst index 5674fd46ec89..cdf63716a182 100644 --- a/doc/rados/operations/index.rst +++ b/doc/rados/operations/index.rst @@ -45,6 +45,7 @@ CRUSH algorithm. read-balancer balancer require-min-compat-client + require-osd-release crush-map crush-map-edits stretch-mode diff --git a/doc/rados/operations/require-osd-release.rst b/doc/rados/operations/require-osd-release.rst new file mode 100644 index 000000000000..59c2fc1294a3 --- /dev/null +++ b/doc/rados/operations/require-osd-release.rst @@ -0,0 +1,138 @@ +.. _require_osd_release: + +===================== + Minimum OSD Release +===================== + +The ``require_osd_release`` field of the OSDMap is the cluster-wide +declaration that every OSD is running at least the named release. It is +set by the operator at the tail of a cluster upgrade, once all OSDs have +been upgraded to the new binary. Raising it unlocks features that would +otherwise be rejected as unsafe, and it bounds how far ahead of the +cluster a new OSD release is allowed to boot. + +Unlike :ref:`require_min_compat_client`, which can be lowered when +features in use permit, ``require_osd_release`` can only move forward. + +Setting the flag +================ + +.. prompt:: bash $ + + ceph osd require-osd-release + +For example, after finishing an upgrade to *tentacle*: + +.. prompt:: bash $ + + ceph osd require-osd-release tentacle + +Monitors only accept a small set of recent release names. The current +accepted targets are ``squid``, ``tentacle``, and ``umbrella``; any +older release name is rejected with ``not supported for this release``. +The accepted set slides forward as new Ceph releases are added. + +The target release must be in the accepted set and must not be older +than the current value; setting it to the current value is a no-op. +Additionally, the Monitors verify that every Monitor in the quorum and +every up OSD advertises the target release's feature bits. The Monitor +feature check has no bypass; the OSD feature check can be overridden +with ``--yes-i-really-mean-it``, but any OSD that lacks the feature +will still be refused when it tries to boot (see `OSD boot window`_). + +Checking the current value +========================== + +.. prompt:: bash $ + + ceph osd dump | grep require_osd_release + +What raising the flag enables +============================= + +Raising ``require_osd_release`` unlocks a set of commands and features +that the monitor refuses while the flag is below the required level. + +.. list-table:: + :header-rows: 1 + :widths: 25 70 + + * - Required release + - Commands or features unlocked + * - ``luminous`` + - ``ceph osd crush set-device-class``, ``ceph osd crush + rm-device-class`` + * - ``nautilus`` + - ``pg_autoscale_mode``, ``target_size_bytes``, ``target_size_ratio``, + new-format ``pg_num`` changes + * - ``tentacle`` + - ``allow_ec_optimizations`` pool flag + +The gates are cumulative: a cluster whose ``require_osd_release`` is at +``squid`` or higher satisfies the ``luminous`` and ``nautilus`` gates +automatically. Because the current accepted-target set does not include +releases older than ``squid``, the ``luminous`` and ``nautilus`` rows +are historical on modern clusters and are listed for completeness. + +OSD boot window +=============== + +An OSD refuses to boot if it is running a release that is more than +two ahead of ``require_osd_release``. For example, with +``require_osd_release`` set to ``reef``: + +.. list-table:: + :header-rows: 1 + :widths: 25 25 50 + + * - OSD release + - Boots? + - Notes + * - ``reef`` + - yes + - Same release. + * - ``squid`` + - yes + - One release ahead. + * - ``tentacle`` + - yes + - Two releases ahead (at the edge of the window). + * - ``umbrella`` + - **no** + - ``disallowing boot of umbrella+ OSD ... because + require_osd_release < squid`` + +This sliding window forces operators to raise ``require_osd_release`` +roughly every two releases, keeping the cluster's OSD code span +bounded. + +.. _OSD_UPGRADE_FINISHED: + +Health warning: ``OSD_UPGRADE_FINISHED`` +======================================== + +If all up OSDs advertise the ``SERVER_`` feature but +``require_osd_release`` is still below that release, the Monitors emit +a ``HEALTH_WARN`` check ``OSD_UPGRADE_FINISHED`` with the message:: + + all OSDs are running or later but require_osd_release < + +The appearance of this warning is the normal cue to run +``ceph osd require-osd-release ``; the warning clears once the +flag has been raised. + +Initial value on new clusters +============================= + +New clusters start with ``require_osd_release`` set to the newest +release the Monitor code knows about. On this release that is +``umbrella``. Two developer options adjust the initial value for +testing; neither should be used in production. + +.. confval:: mon_debug_no_require_umbrella +.. confval:: mon_debug_no_require_tentacle + +Setting ``mon_debug_no_require_umbrella`` alone downshifts the initial +value to ``tentacle``; setting both flags downshifts it further to +``squid``. The ``tentacle`` flag has no effect unless the umbrella flag +is also set.