From a4a681846499c8d0041a6632e7e31fa83e22ed0e Mon Sep 17 00:00:00 2001 From: Zac Dover Date: Fri, 17 Mar 2023 03:41:08 +1000 Subject: [PATCH] doc/rados: line-edit upmap.rst Edit all of doc/rados/upmap.rst. Signed-off-by: Zac Dover --- doc/rados/operations/upmap.rst | 102 +++++++++++++++++---------------- 1 file changed, 53 insertions(+), 49 deletions(-) diff --git a/doc/rados/operations/upmap.rst b/doc/rados/operations/upmap.rst index 366356d16e770..8cce1cf8ef994 100644 --- a/doc/rados/operations/upmap.rst +++ b/doc/rados/operations/upmap.rst @@ -1,52 +1,56 @@ .. _upmap: -Using the pg-upmap -================== +Using pg-upmap +============== -Starting in Luminous v12.2.z there is a new *pg-upmap* exception table +In Luminous v12.2.z and later releases, there is a *pg-upmap* exception table in the OSDMap that allows the cluster to explicitly map specific PGs to -specific OSDs. This allows the cluster to fine-tune the data -distribution to, in most cases, perfectly distributed PGs across OSDs. +specific OSDs. This allows the cluster to fine-tune the data distribution to, +in most cases, uniformly distribute PGs across OSDs. -The key caveat to this new mechanism is that it requires that all -clients understand the new *pg-upmap* structure in the OSDMap. +However, there is an important caveat when it comes to this new feature: it +requires all clients to understand the new *pg-upmap* structure in the OSDMap. Enabling -------- -New clusters will by default enable the `balancer module`. The cluster must only -have Luminous (and newer) clients. You can turn the balancer off with: +In order to use ``pg-upmap``, the cluster cannot have any pre-Luminous clients. +By default, new clusters enable the *balancer module*, which makes use of +``pg-upmap``. If you want to use a different balancer or you want to make your +own custom ``pg-upmap`` entries, you might want to turn off the balancer in +order to avoid conflict: .. prompt:: bash $ ceph balancer off -To allow use of the feature on existing clusters, you must tell the -cluster that it only needs to support Luminous (and newer) clients with: +To allow use of the new feature on an existing cluster, you must restrict the +cluster to supporting only Luminous (and newer) clients. To do so, run the +following command: .. prompt:: bash $ ceph osd set-require-min-compat-client luminous -This command will fail if any pre-Luminous clients or daemons are -connected to the monitors. You can see what client versions are in -use with: +This command will fail if any pre-Luminous clients or daemons are connected to +the monitors. To see which client versions are in use, run the following +command: .. prompt:: bash $ ceph features Balancer module ------------------ - -The `balancer` module for ceph-mgr will automatically balance -the number of PGs per OSD. See :ref:`balancer` +--------------- +The `balancer` module for ``ceph-mgr`` will automatically balance the number of +PGs per OSD. See :ref:`balancer` Offline optimization -------------------- -Upmap entries are updated with an offline optimizer built into ``osdmaptool``. +Upmap entries are updated with an offline optimizer that is built into +``osdmaptool``. #. Grab the latest copy of your osdmap: @@ -64,27 +68,28 @@ Upmap entries are updated with an offline optimizer built into ``osdmaptool``. [--upmap-active] It is highly recommended that optimization be done for each pool - individually, or for sets of similarly-utilized pools. You can - specify the ``--upmap-pool`` option multiple times. "Similar pools" - means pools that are mapped to the same devices and store the same - kind of data (e.g., RBD image pools, yes; RGW index pool and RGW - data pool, no). - - The ``max-optimizations`` value is the maximum number of upmap entries to - identify in the run. The default is `10` like the ceph-mgr balancer module, - but you should use a larger number if you are doing offline optimization. - If it cannot find any additional changes to make it will stop early - (i.e., when the pool distribution is perfect). - - The ``max-deviation`` value defaults to `5`. If an OSD PG count - varies from the computed target number by less than or equal - to this amount it will be considered perfect. - - The ``--upmap-active`` option simulates the behavior of the active - balancer in upmap mode. It keeps cycling until the OSDs are balanced - and reports how many rounds and how long each round is taking. The - elapsed time for rounds indicates the CPU load ceph-mgr will be - consuming when it tries to compute the next optimization plan. + individually, or for sets of similarly utilized pools. You can specify the + ``--upmap-pool`` option multiple times. "Similarly utilized pools" means + pools that are mapped to the same devices and that store the same kind of + data (for example, RBD image pools are considered to be similarly utilized; + an RGW index pool and an RGW data pool are not considered to be similarly + utilized). + + The ``max-optimizations`` value determines the maximum number of upmap + entries to identify. The default is `10` (as is the case with the + ``ceph-mgr`` balancer module), but you should use a larger number if you are + doing offline optimization. If it cannot find any additional changes to + make (that is, if the pool distribution is perfect), it will stop early. + + The ``max-deviation`` value defaults to `5`. If an OSD's PG count varies + from the computed target number by no more than this amount it will be + considered perfect. + + The ``--upmap-active`` option simulates the behavior of the active balancer + in upmap mode. It keeps cycling until the OSDs are balanced and reports how + many rounds have occurred and how long each round takes. The elapsed time + for rounds indicates the CPU load that ``ceph-mgr`` consumes when it computes + the next optimization plan. #. Apply the changes: @@ -92,14 +97,13 @@ Upmap entries are updated with an offline optimizer built into ``osdmaptool``. source out.txt - The proposed changes are written to the output file ``out.txt`` in - the example above. These are normal ceph CLI commands that can be - run to apply the changes to the cluster. - + In the above example, the proposed changes are written to the output file + ``out.txt``. The commands in this procedure are normal Ceph CLI commands + that can be run in order to apply the changes to the cluster. -The above steps can be repeated as many times as necessary to achieve -a perfect distribution of PGs for each set of pools. +The above steps can be repeated as many times as necessary to achieve a perfect +distribution of PGs for each set of pools. -You can see some (gory) details about what the tool is doing by -passing ``--debug-osd 10`` and even more with ``--debug-crush 10`` -to ``osdmaptool``. +To see some (gory) details about what the tool is doing, you can pass +``--debug-osd 10`` to ``osdmaptool``. To see even more details, pass +``--debug-crush 10`` to ``osdmaptool``. -- 2.39.5