From ee03a51978c6d84236755eeae8bf71a7067063d7 Mon Sep 17 00:00:00 2001 From: Kefu Chai Date: Fri, 8 May 2026 13:46:26 +0800 Subject: [PATCH] doc/rados: add kernel client notes to require_min_compat_client Add a "Min kernel" column to the "Features gated by the flag" table: pg-upmap requires kernel 4.13, pg-upmap-primary and CRUSH MSR are not yet implemented in the kernel client. Add a warning noting the distinct failure modes (pg-upmap-primary causes I/O hangs from silently dropped misdirected ops; CRUSH MSR rules cause I/O errors because crush_find_rule fails) and that CRUSH MSR rules are added via osd setcrushmap without require_min_compat_client gating but immediately raise the features-in-use floor to squid. Signed-off-by: Kefu Chai --- .../operations/require-min-compat-client.rst | 41 ++++++++++++++++--- 1 file changed, 35 insertions(+), 6 deletions(-) diff --git a/doc/rados/operations/require-min-compat-client.rst b/doc/rados/operations/require-min-compat-client.rst index c6cc0e38f8e8..dbfa6204f5d3 100644 --- a/doc/rados/operations/require-min-compat-client.rst +++ b/doc/rados/operations/require-min-compat-client.rst @@ -106,18 +106,30 @@ Features gated by the flag Raising the flag does not enable any feature on its own. It authorizes the operator to run commands that would otherwise be rejected with an error like ``min_compat_client luminous < reef, which is required for -pg-upmap-primary``: +pg-upmap-primary``. + +The Linux kernel Ceph client (``krbd`` for RBD block devices, ``kcephfs`` +for CephFS) ships with the kernel, not with Ceph packages, so its version +is independent of the cluster release. New OSD map features reach the +kernel client only after they are implemented and merged upstream, which +can lag the Ceph daemon release by months or years. + +The *Min kernel* column shows the minimum Linux kernel version required on +kernel client hosts before the corresponding commands are used. If no +kernel clients are present in the cluster, that column can be ignored. .. list-table:: :header-rows: 1 - :widths: 60 25 + :widths: 40 20 25 * - Command - Minimum release + - Min kernel * - | ``ceph osd primary-affinity`` | ``ceph osd primary-temp`` | ``ceph osd rm-primary-temp`` - ``firefly`` + - any * - | ``ceph osd pg-upmap`` | ``ceph osd rm-pg-upmap`` | ``ceph osd pg-upmap-items`` @@ -125,18 +137,35 @@ pg-upmap-primary``: | ``ceph osd crush weight-set`` | balancer ``upmap`` mode - ``luminous`` + - 4.13 * - | ``ceph osd pg-upmap-primary`` | ``ceph osd rm-pg-upmap-primary`` | ``ceph osd rm-pg-upmap-primary-all`` | balancer ``read`` mode | balancer ``upmap-read`` mode - ``reef`` + - not yet implemented + +.. warning:: + + ``pg-upmap-primary`` (``reef``) and CRUSH MSR rules (``squid``) are + not yet implemented in the kernel client. Kernel clients that + encounter ``pg-upmap-primary`` entries will silently route I/O to + the wrong OSD, causing **I/O hangs** on the affected PGs. Kernel + clients that encounter CRUSH MSR rules will fail to compute a PG + mapping, causing **I/O errors**. Do not enable either feature while + kernel clients are present (use ``ceph features`` to check). Note + that CRUSH MSR rules are added via ``osd setcrushmap`` and are + **not** gated by ``require_min_compat_client``, but they immediately + raise the features-in-use floor to ``squid`` + (see `Lowering the flag`_). To check whether either is active:: + + ceph osd dump | grep -E "pg_upmap_primary|msr" -Note that adding CRUSH MSR rules is *not* gated at command time; however, -once an MSR rule is in the map, the features-in-use floor rises to -``squid`` (see the table above). + See :ref:`read_balancer` for removal commands. See :ref:`upmap` and :ref:`read_balancer` for the operational procedures that depend on raising the flag. CephFS clients must satisfy both this flag and any per-filesystem :ref:`cephfs_required_client_features` on -the file systems they mount. +the file systems they mount. For CephFS-specific kernel version guidance, +see :ref:`cephfs_which_kernel_version` and :doc:`/cephfs/kernel-features`. -- 2.47.3