From d91e75e1e92b73bf967740575b3c27a86ef844d5 Mon Sep 17 00:00:00 2001 From: =?utf8?q?Niklas=20Hamb=C3=BCchen?= Date: Sat, 30 Mar 2024 17:42:48 +0100 Subject: [PATCH] doc/rados/operations: Improve crush_location docs MIME-Version: 1.0 Content-Type: text/plain; charset=utf8 Content-Transfer-Encoding: 8bit * Fix incorrect syntax * Use underscores for config options, like other ceph docs did * Fix incorrect statement that crush_location_hook adds fiels; it replaces * Explain `root=default host=HOSTNAME` is not set if `crush_location` is given * Remove duplication across sections * Point out that `root=default` is important Signed-off-by: Niklas Hambüchen --- doc/rados/operations/crush-map.rst | 49 ++++++++++++++++++------------ 1 file changed, 29 insertions(+), 20 deletions(-) diff --git a/doc/rados/operations/crush-map.rst b/doc/rados/operations/crush-map.rst index 8f833d28dacb6..1c68a2a181b82 100644 --- a/doc/rados/operations/crush-map.rst +++ b/doc/rados/operations/crush-map.rst @@ -57,53 +57,62 @@ case for most clusters), its CRUSH location can be specified as follows:: ``pod``, ``pdu``, ``rack``, ``chassis``, and ``host``. These defined types suffice for nearly all clusters, but can be customized by modifying the CRUSH map. - #. Not all keys need to be specified. For example, by default, Ceph - automatically sets an ``OSD``'s location as ``root=default - host=HOSTNAME`` (as determined by the output of ``hostname -s``). -The CRUSH location for an OSD can be modified by adding the ``crush location`` -option in ``ceph.conf``. When this option has been added, every time the OSD +The CRUSH location for an OSD can be set by adding the ``crush_location`` +option in ``ceph.conf``, example: + + crush_location = root=default row=a rack=a2 chassis=a2a host=a2a1 + +When this option has been added, every time the OSD starts it verifies that it is in the correct location in the CRUSH map and moves itself if it is not. To disable this automatic CRUSH map management, add the following to the ``ceph.conf`` configuration file in the ``[osd]`` section:: - osd crush update on start = false + osd_crush_update_on_start = false Note that this action is unnecessary in most cases. +If the ``crush_location`` is not set explicitly, +a default of ``root=default host=HOSTNAME`` is used for ``OSD``s, +where the hostname is determined by the output of the ``hostname -s`` command. + +.. note:: If you switch from this default to an explicitly set ``crush_location``, + do not forget to include ``root=default`` because existing CRUSH rules refer to it. Custom location hooks --------------------- -A custom location hook can be used to generate a more complete CRUSH location -on startup. The CRUSH location is determined by, in order of preference: +A custom location hook can be used to generate a more complete CRUSH location, +on startup. + +This is useful when some location fields are not known at the time +``ceph.conf`` is written (for example, fields ``rack`` or ``datacenter`` +when deploying a single configuration across multiple datacenters). -#. A ``crush location`` option in ``ceph.conf`` -#. A default of ``root=default host=HOSTNAME`` where the hostname is determined - by the output of the ``hostname -s`` command +If configured, executed, and parsed successfully, the hook's output replaces +any previously set CRUSH location. -A script can be written to provide additional location fields (for example, -``rack`` or ``datacenter``) and the hook can be enabled via the following -config option:: +The hook hook can be enabled in ``ceph.conf`` by providing a path to an +executable file (often a script), example:: - crush location hook = /path/to/customized-ceph-crush-location + crush_location_hook = /path/to/customized-ceph-crush-location This hook is passed several arguments (see below). The hook outputs a single -line to ``stdout`` that contains the CRUSH location description. The output -resembles the following::: +line to ``stdout`` that contains the CRUSH location description. The arguments +resemble the following::: --cluster CLUSTER --id ID --type TYPE Here the cluster name is typically ``ceph``, the ``id`` is the daemon identifier or (in the case of OSDs) the OSD number, and the daemon type is -``osd``, ``mds, ``mgr``, or ``mon``. +``osd``, ``mds``, ``mgr``, or ``mon``. For example, a simple hook that specifies a rack location via a value in the -file ``/etc/rack`` might be as follows:: +file ``/etc/rack`` (assuming it contains no spaces) might be as follows:: #!/bin/sh - echo "host=$(hostname -s) rack=$(cat /etc/rack) root=default" + echo "root=default rack=$(cat /etc/rack) host=$(hostname -s)" CRUSH structure -- 2.39.5