From e45227d61541f9db115191ebc7e72c50140e7a19 Mon Sep 17 00:00:00 2001 From: Jan Fajerski Date: Mon, 29 Jun 2020 17:42:26 +0200 Subject: [PATCH] doc: update ceph-volume lvm batch docs Signed-off-by: Jan Fajerski (cherry picked from commit 7695d1ec539f54b7b7cea8d19925f0c320223f03) --- doc/ceph-volume/lvm/batch.rst | 212 ++++++++++------------------------ 1 file changed, 60 insertions(+), 152 deletions(-) diff --git a/doc/ceph-volume/lvm/batch.rst b/doc/ceph-volume/lvm/batch.rst index 78f7abf6fa420..7be4ec90559fa 100644 --- a/doc/ceph-volume/lvm/batch.rst +++ b/doc/ceph-volume/lvm/batch.rst @@ -2,37 +2,35 @@ ``batch`` =========== -This subcommand allows for multiple OSDs to be created at the same time given -an input of devices. Depending on the device type (spinning drive, or solid -state), the internal engine will decide the best approach to create the OSDs. +The subcommand allows to create multiple OSDs at the same time given +an input of devices. The ``batch`` subcommand is closely related to +drive-groups. One individual drive group specification translates to a single +``batch`` invocation. -This decision abstracts away the many nuances when creating an OSD: how large -should a ``block.db`` be? How can one mix a solid state device with spinning -devices in an efficient way? - -The process is similar to :ref:`ceph-volume-lvm-create`, and will do the -preparation and activation at once, following the same workflow for each OSD. -However, If the ``--prepare`` flag is passed then only the prepare step is taken -and the OSDs are not activated. +The subcommand is based to :ref:`ceph-volume-lvm-create`, and will use the very +same code path. All ``batch`` does is to calculate the appropriate sizes of all +volumes and skip over already created volumes. All the features that ``ceph-volume lvm create`` supports, like ``dmcrypt``, avoiding ``systemd`` units from starting, defining bluestore or filestore, -are supported. Any fine-grained option that may affect a single OSD is not -supported, for example: specifying where journals should be placed. - +are supported. +.. _ceph-volume-lvm-batch_auto: Automatic sorting of disks -------------------------- -If ``batch`` receives only a single list of data devices and other options are -passed , ``ceph-volume`` will auto-sort disks by its rotational -property and use non-rotating disks for ``block.db`` or ``journal`` depending -on the objectstore used. If all devices are to be used for standalone OSDs, -no matter if rotating or solid state, pass ``--no-auto``. +If ``batch`` receives only a single list of devices and the ``--no-auto`` option + is not passed, ``ceph-volume`` will auto-sort disks by its rotational + property and use non-rotating disks for ``block.db`` or ``journal`` depending + on the objectstore used. +This behavior is now DEPRECATED and will be removed in future releases. Instead + an ``auto`` option will be introduced to retain this behavior. +It is recommended to make use of the explicit device lists for ``block.db``, + ``block.wal`` and ``journal``. For example assuming :term:`bluestore` is used and ``--no-auto`` is not passed, -the deprecated behavior would deploy the following, depending on the devices -passed: + the deprecated behavior would deploy the following, depending on the devices + passed: #. Devices are all spinning HDDs: 1 OSD is created per device #. Devices are all SSDs: 2 OSDs are created per device @@ -42,12 +40,6 @@ passed: .. note:: Although operations in ``ceph-volume lvm create`` allow usage of ``block.wal`` it isn't supported with the ``auto`` behavior. -This default auto-sorting behavior is now DEPRECATED and will be changed in future releases. -Instead devices are not automatically sorted unless the ``--auto`` option is passed - -It is recommended to make use of the explicit device lists for ``block.db``, - ``block.wal`` and ``journal``. - .. _ceph-volume-lvm-batch_bluestore: Reporting @@ -70,156 +62,72 @@ The ``pretty`` report format (the default) would look like this:: $ ceph-volume lvm batch --report /dev/sdb /dev/sdc /dev/sdd --db-devices /dev/nvme0n1 - --> passed data devices: 3 physical, 0 LVM - --> relative data size: 1.0 - --> passed block_db devices: 1 physical, 0 LVM Total OSDs: 3 - Type Path LV Size % of device - ---------------------------------------------------------------------------------------------------- - data /dev/sdb 300.00 GB 100.00% - block_db /dev/nvme0n1 66.67 GB 33.33% - ---------------------------------------------------------------------------------------------------- - data /dev/sdc 300.00 GB 100.00% - block_db /dev/nvme0n1 66.67 GB 33.33% - ---------------------------------------------------------------------------------------------------- - data /dev/sdd 300.00 GB 100.00% - block_db /dev/nvme0n1 66.67 GB 33.33% - - -.. note:: Although operations in ``ceph-volume lvm create`` allow usage of - ``block.wal`` it isn't supported with the ``batch`` sub-command **JSON reporting** -Reporting can produce a structured output with ``--format json`` or -``--format json-pretty``:: - - $ ceph-volume lvm batch --report --format json-pretty /dev/sdb /dev/sdc /dev/sdd --db-devices /dev/nvme0n1 - --> passed data devices: 3 physical, 0 LVM - --> relative data size: 1.0 - --> passed block_db devices: 1 physical, 0 LVM - [ - { - "block_db": "/dev/nvme0n1", - "block_db_size": "66.67 GB", - "data": "/dev/sdb", - "data_size": "300.00 GB", - "encryption": "None" - }, - { - "block_db": "/dev/nvme0n1", - "block_db_size": "66.67 GB", - "data": "/dev/sdc", - "data_size": "300.00 GB", - "encryption": "None" - }, - { - "block_db": "/dev/nvme0n1", - "block_db_size": "66.67 GB", - "data": "/dev/sdd", - "data_size": "300.00 GB", - "encryption": "None" - } - ] +Reporting can produce a structured output with ``--format json``:: + + $ ceph-volume lvm batch --report /dev/sdb /dev/sdc /dev/sdd --db-devices /dev/nvme0n1 Sizing ====== When no sizing arguments are passed, `ceph-volume` will derive the sizing from the passed device lists (or the sorted lists when using the automatic sorting). -`ceph-volume batch` will attempt to fully utilize a device's available capacity. +`ceph-volume batch` will attempt to fully utilize a devices available capacity. Relying on automatic sizing is recommended. -#. Devices are all the same type (for example all spinning HDD or all SSDs): - 1 OSD is created per device, collocating the journal in the same HDD. -#. Devices are a mix of HDDs and SSDs: data is placed on the spinning device, - while the journal is created on the SSD using the sizing options from - ceph.conf and falling back to the default journal size of 5GB. +If one requires a different sizing policy for wal, db or journal devices, +`ceph-volume` offers implicit and explicit sizing rules. +Implicit sizing +--------------- +Scenarios in which either devices are under-comitted or not all data devices are +currently ready for use (due to a broken disk for example), one can still rely +on `ceph-volume` automatic sizing. +Users can provide hints to `ceph-volume` as to how many data devices should have +their external volumes on a set of fast devices. These options are: -* ``--block-db-slots`` -* ``--block-wal-slots`` -* ``--journal-slots`` +* `--block-db-slots` +* `--block-wal-slots` +* `--journal-slots` -For example, consider an OSD host that is supposed to contain 5 data devices and -one device for wal/db volumes. However, one data device is currently broken and +For example consider an OSD host that is supposed to contain 5 data devices and +one device for wal/db volumes. However one data device is currently broken and is being replaced. Instead of calculating the explicit sizes for the wal/db -volume, one can simply call:: +volume one can simply call:: -.. _ceph-volume-lvm-batch_report: + $ ceph-volume lvm batch --report /dev/sdb /dev/sdc /dev/sdd /dev/sde --db-devices /dev/nvme0n1 --block-db-slots 5 -Reporting -========= -When a call is received to create OSDs, the tool will prompt the user to -continue if the pre-computed output is acceptable. This output is useful to -understand the outcome of the received devices. Once confirmation is accepted, -the process continues. +Explicit sizing +--------------- +It is also possible to provide explicit sizes to `ceph-volume` via the arguments -* ``--block-db-size`` -* ``--block-wal-size`` -* ``--journal-size`` +* `--block-db-size` +* `--block-wal-size` +* `--journal-size` -**pretty reporting** -For two spinning devices, this is how the ``pretty`` report (the default) would -look:: +`ceph-volume` will try to satisfy the requested sizes given the passed disks. If +this is not possible, no OSDs will be deployed. - $ ceph-volume lvm batch --report /dev/sdb /dev/sdc - Total OSDs: 2 +Idempotency and disk replacements +================================= +`ceph-volume lvm batch` intends to be idempotent, i.e. calling the same command +repeatedly should result in the same outcome. For example calling:: - Type Path LV Size % of device - -------------------------------------------------------------------------------- - [data] /dev/sdb 10.74 GB 100% - -------------------------------------------------------------------------------- - [data] /dev/sdc 10.74 GB 100% + $ ceph-volume lvm batch --report /dev/sdb /dev/sdc /dev/sdd --db-devices /dev/nvme0n1 +will result in three deployed OSDs (if all disks were available). Calling this +command again, you will still end up with three OSDs and ceph-volume will exit +with return code 0. +Suppose /dev/sdc goes bad and needs to be replaced. After destroying the OSD and +replacing the hardware, you can again call the same command and `ceph-volume` +will detect that only two out of the three wanted OSDs are setup and re-create +the missing OSD. -**JSON reporting** -Reporting can produce a richer output with ``JSON``, which gives a few more -hints on sizing. This feature might be better for other tooling to consume -information that will need to be transformed. - -For two spinning devices, this is how the ``JSON`` report would look:: - - $ ceph-volume lvm batch --report --format=json /dev/sdb /dev/sdc - { - "osds": [ - { - "block.db": {}, - "data": { - "human_readable_size": "10.74 GB", - "parts": 1, - "path": "/dev/sdb", - "percentage": 100, - "size": 11534336000.0 - } - }, - { - "block.db": {}, - "data": { - "human_readable_size": "10.74 GB", - "parts": 1, - "path": "/dev/sdc", - "percentage": 100, - "size": 11534336000.0 - } - } - ], - "vgs": [ - { - "devices": [ - "/dev/sdb" - ], - "parts": 1 - }, - { - "devices": [ - "/dev/sdc" - ], - "parts": 1 - } - ] - } +This idempotency notion is tightly coupled to and extensively used by :ref:`drivegroups`. -- 2.39.5