doc/rados: Update mClock doc on steps to override OSD IOPS capacity config

Describe the steps involved to
 - Specify a global value for osd_mclock_max_capacity_iops_{ssd,hdd}, and
 - Override existing individually scoped values for OSDs determined during
   start-up for osd_mclock_max_capacity_iops_{ssd,hdd}.

The above is to help with the following:
 - Steps to override existing setting with a global value.
 - reduce the number of entries in the mon store and instead use a single
   global specification for all OSDs in the cluster in case the underlying
   hardware is the same for all OSDs.

Signed-off-by: Sridhar Seshasayee <sseshasa@redhat.com>
Fixes: https://tracker.ceph.com/issues/70774

diff --git a/doc/rados/configuration/mclock-config-ref.rst b/doc/rados/configuration/mclock-config-ref.rst
index c1287a21f1b532d0da667396668a9cfeb043e96d..9081028325ca311bf08cac6f21e83fdbbb8ecea6 100644 (file)
--- a/doc/rados/configuration/mclock-config-ref.rst
+++ b/doc/rados/configuration/mclock-config-ref.rst
@@ -714,29 +714,98 @@ general for HDDs, the bluestore throttle values are expected to be higher when
 compared to SSDs.
 
 
-Specifying  Max OSD Capacity
-----------------------------
+Set or Override Max IOPS Capacity of an OSD
+-------------------------------------------
 
-The steps in this section may be performed only if you want to override the
-max osd capacity automatically set during OSD initialization. The option
-``osd_mclock_max_capacity_iops_[hdd, ssd]`` for an OSD can be set by running the
-following command:
+The steps in this section may be performed to set or override the max IOPS
+capacity of an OSD. The ``osd_mclock_max_capacity_iops_[hdd, ssd]`` option for
+an OSD can be overridden by running a command of the following form:
 
-  .. prompt:: bash #
+.. prompt:: bash #
 
-     ceph config set osd.N osd_mclock_max_capacity_iops_[hdd,ssd] <value>
+  ceph config set osd.N osd_mclock_max_capacity_iops_[hdd,ssd] <value>
 
 For example, the following command sets the max capacity for a specific OSD
 (say "osd.0") whose underlying device type is HDD to 350 IOPS:
 
-  .. prompt:: bash #
+.. prompt:: bash #
 
-    ceph config set osd.0 osd_mclock_max_capacity_iops_hdd 350
+  ceph config set osd.0 osd_mclock_max_capacity_iops_hdd 350
 
 Alternatively, you may specify the max capacity for OSDs within the Ceph
 configuration file under the respective [osd.N] section. See
 :ref:`ceph-conf-settings` for more details.
 
+Global Override of Max IOPS Capacity for multiple OSDs
+------------------------------------------------------
+
+The max IOPS capacity of multiple OSDs may be overridden by a global config
+specification. This section shows the steps to globally override the
+individually scoped values in the mon store.
+
+    .. note:: The examples use :confval:`osd_mclock_max_capacity_iops_hdd` and
+              the steps are also applicable for SSD based OSDs in which case
+              the option to use is :confval:`osd_mclock_max_capacity_iops_ssd`.
+
+Below are steps to override the IOPS capacities of individual OSDs. Note that
+the individual value is taken by the OSD after it runs the usual startup
+benchmark.
+
+#. Run the following command to verify the individual values set for the OSDs in
+   the central config database:
+
+    .. prompt:: bash #
+
+      ceph config dump | grep osd_mclock_max_capacity_iops
+
+    ::
+
+      WHO     MASK  LEVEL  OPTION                            VALUE       RO
+      osd.0         basic  osd_mclock_max_capacity_iops_hdd  379.197568    
+      osd.1         basic  osd_mclock_max_capacity_iops_hdd  400.903575    
+      osd.2         basic  osd_mclock_max_capacity_iops_hdd  398.303428    
+      osd.3         basic  osd_mclock_max_capacity_iops_hdd  419.035854    
+
+#. If there are no individual values reported, skip to the next step. Otherwise,
+   remove all the individual values reported in the previous step with a command
+   of the following form (where 'x' is the OSD id):
+
+    .. prompt:: bash #
+
+      ceph config rm osd.x osd_mclock_max_capacity_iops_hdd
+
+#. Confirm that the `ceph config dump` command from step 1 does not show any
+   individual values.
+
+#. Set the global value of `osd_mclock_max_capacity_iops_hdd` with a command of
+   the following form:
+
+    .. prompt:: bash #
+
+      ceph config set global osd_mclock_max_capacity_iops_hdd 111
+
+#. Confirm that the global option is set by running:
+
+    .. prompt:: bash #
+
+      ceph config dump | grep osd_mclock_max_capacity_iops
+
+    ::
+
+      global        basic  osd_mclock_max_capacity_iops_hdd  111.000000    
+
+
+#. Confirm that the global setting is now in effect for any OSD that no longer
+   has a specific per-OSD central config setting:
+
+    .. prompt:: bash #
+
+      ceph config show osd.0 | grep osd_mclock_max_capacity_iops_hdd
+
+    ::
+
+      osd_mclock_max_capacity_iops_hdd                 111.000000                               mon
+
 
 .. index:: mclock; config settings