From 41327dcf0f733777f0d022d1bab7bf72b9807b3c Mon Sep 17 00:00:00 2001 From: Matan Breizman Date: Mon, 25 Nov 2024 12:05:19 +0000 Subject: [PATCH] doc/dev/crimson: update SeaStore as default backend Signed-off-by: Matan Breizman --- doc/dev/crimson/crimson.rst | 173 +++++++++++++++++------------------- 1 file changed, 82 insertions(+), 91 deletions(-) diff --git a/doc/dev/crimson/crimson.rst b/doc/dev/crimson/crimson.rst index f6d59a057ff85..f9582ec6c84b4 100644 --- a/doc/dev/crimson/crimson.rst +++ b/doc/dev/crimson/crimson.rst @@ -43,82 +43,6 @@ use a Crimson build: You'll likely need to supply the ``--allow-mismatched-release`` flag to use a non-release branch. -Configure Crimson with Bluestore -================================ - -As Bluestore is not a Crimson native `object store backend`_, -deploying Crimson with Bluestore as the back end requires setting -one of the two following configuration options: - -.. note:: - - #. These two options, along with ``crimson_alien_op_num_threads``, - can't be changed after deployment. - #. `vstart.sh`_ sets these options using the ``--crimson-smp`` flag. - - -1) ``crimson_seastar_num_threads`` - - In order to allow easier cluster deployments, this option can be used - instead of setting the CPU mask manually for each OSD. - - It's recommended to let the **number of OSDs on each host** multiplied by - ``crimson_seastar_num_threads`` to be less than the node's number of CPU - cores (``nproc``). - - For example, for deploying two nodes with eight CPU cores and two OSDs each: - - .. code-block:: yaml - - conf: - # Global to all OSDs - osd: - crimson seastar num threads: 3 - - .. note:: - - #. For optimal performance ``crimson_seastar_cpu_cores`` should be set instead. - -2) ``crimson_seastar_cpu_cores`` and ``crimson_alien_thread_cpu_cores``. - - Explicitly set the CPU core allocation for each ``crimson-osd`` - and for the BlueStore back end. It's recommended for each set to be mutually exclusive. - - For example, for deploying two nodes with eight CPU cores and two OSDs each: - - .. code-block:: yaml - - conf: - # Both nodes - osd: - crimson alien thread cpu cores: 6-7 - - # First node - osd.0: - crimson seastar cpu cores: 0-2 - osd.1: - crimson seastar cpu cores: 3-5 - - # Second node - osd.2: - crimson seastar cpu cores: 0-2 - osd.3: - crimson seastar cpu cores: 3-5 - - For a single node with eight node and three OSDs: - - .. code-block:: yaml - - conf: - osd: - crimson alien thread cpu cores: 6-7 - osd.0: - crimson seastar cpu cores: 0-1 - osd.1: - crimson seastar cpu cores: 2-3 - osd.2: - crimson seastar cpu cores: 4-5 - Running Crimson =============== @@ -182,7 +106,7 @@ The following options can be used with ``vstart.sh``. (as determined by `nproc`) will be assigned to the object store. ``--bluestore`` - Use the alienized BlueStore as the object store backend. This is the default (see below section on the `object store backend`_ for more details) + Use alienized BlueStore as the object store backend. ``--cyanstore`` Use CyanStore as the object store backend. @@ -191,7 +115,7 @@ The following options can be used with ``vstart.sh``. Use the alienized MemStore as the object store backend. ``--seastore`` - Use SeaStore as the back end object store. + Use SeaStore as the back end object store. This is the default (see below section on the `object store backend`_ for more details) ``--seastore-devs`` Specify the block device used by SeaStore. @@ -207,20 +131,11 @@ The following options can be used with ``vstart.sh``. Valid types include ``HDD``, ``SSD``(default), ``ZNS``, and ``RANDOM_BLOCK_SSD`` Note secondary devices should not be faster than the main device. -To start a cluster with a single Crimson node, run:: - - $ MGR=1 MON=1 OSD=1 MDS=0 RGW=0 ../src/vstart.sh \ - --without-dashboard --bluestore --crimson \ - --redirect-output -Another SeaStore example:: +To start a simple cluster with a single core Crimson OSD, run:: - $ MGR=1 MON=1 OSD=1 MDS=0 RGW=0 ../src/vstart.sh -n -x \ - --without-dashboard --seastore \ - --crimson --redirect-output \ - --seastore-devs /dev/sda \ - --seastore-secondary-devs /dev/sdb \ - --seastore-secondary-devs-type HDD + $ MGR=1 MON=1 OSD=1 MDS=0 RGW=0 ../src/vstart.sh -n \ + --without-dashboard --seastore --crimson Stop this ``vstart`` cluster by running:: @@ -239,7 +154,7 @@ They are: .. describe:: seastore - Seastore is still under active development. + Seastore is the default Crimson backend and is still under active development. The alienized object store backends are backed by a thread pool, which is a proxy of the alienstore adaptor running in Seastar. The proxy issues @@ -254,6 +169,82 @@ managed by the Seastar framework. They are: The object store used by the classic ``ceph-osd`` +Configure Crimson with Bluestore +================================ + +As Bluestore is not a Crimson native `object store backend`_, +deploying Crimson with Bluestore as the back end requires setting +one of the two following configuration options: + +.. note:: + + #. These two options, along with ``crimson_alien_op_num_threads``, + can't be changed after deployment. + #. `vstart.sh`_ sets these options using the ``--crimson-smp`` flag. + + +1) ``crimson_seastar_num_threads`` + + In order to allow easier cluster deployments, this option can be used + instead of setting the CPU mask manually for each OSD. + + It's recommended to set the **number of OSDs on each host** multiplied by + ``crimson_seastar_num_threads`` to be less than the node's number of CPU + cores (``nproc``). + + For example, for deploying two nodes with eight CPU cores and two OSDs each: + + .. code-block:: yaml + + conf: + # Global to all OSDs + osd: + crimson seastar num threads: 3 + + .. note:: + + #. For optimal performance ``crimson_seastar_cpu_cores`` should be set instead. + +2) ``crimson_seastar_cpu_cores`` and ``crimson_alien_thread_cpu_cores``. + + Explicitly set the CPU core allocation for each ``crimson-osd`` + and for the BlueStore back end. It's recommended for each set to be mutually exclusive. + + For example, for deploying two nodes with eight CPU cores and two OSDs each: + + .. code-block:: yaml + + conf: + # Both nodes + osd: + crimson alien thread cpu cores: 6-7 + + # First node + osd.0: + crimson seastar cpu cores: 0-2 + osd.1: + crimson seastar cpu cores: 3-5 + + # Second node + osd.2: + crimson seastar cpu cores: 0-2 + osd.3: + crimson seastar cpu cores: 3-5 + + For a single node with eight node and three OSDs: + + .. code-block:: yaml + + conf: + osd: + crimson alien thread cpu cores: 6-7 + osd.0: + crimson seastar cpu cores: 0-1 + osd.1: + crimson seastar cpu cores: 2-3 + osd.2: + crimson seastar cpu cores: 4-5 + daemonize --------- -- 2.39.5