List Devices
============
-``ceph-volume`` scans each host in the cluster from time to time in order
-to determine which devices are present and whether they are eligible to be
-used as OSDs.
+``ceph-volume`` scans each host in the cluster periodically in order
+to determine the devices that are present and responsive. It is also
+determined whether each is eligible to be used for new OSDs in a block,
+DB, or WAL role.
To print a list of devices discovered by ``cephadm``, run this command:
srv-03 /dev/sdc hdd 15R0A0P7FRD6 300G Unknown N/A N/A No
srv-03 /dev/sdd hdd 15R0A0O7FRD6 300G Unknown N/A N/A No
-Using the ``--wide`` option provides all details relating to the device,
-including any reasons that the device might not be eligible for use as an OSD.
-
-In the above example you can see fields named "Health", "Ident", and "Fault".
+In the above examples you can see fields named ``Health``, ``Ident``, and ``Fault``.
This information is provided by integration with `libstoragemgmt`_. By default,
this integration is disabled (because `libstoragemgmt`_ may not be 100%
compatible with your hardware). To make ``cephadm`` include these fields,
ceph config set mgr mgr/cephadm/device_enhanced_scan true
+Note that the columns reported by ``ceph orch device ls`` may vary from release to
+release.
+
+The ``--wide`` option shows device details,
+including any reasons that the device might not be eligible for use as an OSD.
+Example (Reef)::
+
+ HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS
+ davidsthubbins /dev/sdc hdd SEAGATE_ST20000NM002D_ZVTBJNGC17010W339UW25 18.1T No 22m ago Has a FileSystem, Insufficient space (<10 extents) on vgs, LVM detected
+ nigeltufnel /dev/sdd hdd SEAGATE_ST20000NM002D_ZVTBJNGC17010C3442787 18.1T No 22m ago Has a FileSystem, Insufficient space (<10 extents) on vgs, LVM detected
+
.. warning::
- Although the libstoragemgmt library performs standard SCSI inquiry calls,
- there is no guarantee that your firmware fully implements these standards.
+ Although the ``libstoragemgmt`` library issues standard SCSI (SES) inquiry calls,
+ there is no guarantee that your hardware and firmware properly implement these standards.
This can lead to erratic behaviour and even bus resets on some older
hardware. It is therefore recommended that, before enabling this feature,
you test your hardware's compatibility with libstoragemgmt first to avoid
There are multiple optional settings you can use to change the way OSDs are deployed.
You can add these options to the base level of an OSD spec for it to take effect.
-This example would deploy all OSDs with encryption enabled.
+This example deploys encrypted OSDs on all unused drives. Note that if Linux
+MD mirroring is used for the boot, ``/var/log``, or other volumes this spec *may*
+grab replacement or added drives before you can employ them for non-OSD purposes.
+The ``unmanaged`` attribute may be set to pause automatic deployment until you
+are ready.
.. code-block:: yaml
data_devices:
rotational: 1
db_devices:
- model: MC-55-44-XZ
- limit: 2 # db_slots is actually to be favoured here, but it's not implemented yet
+ model: MC-55-44-XZ # Select only this model for WAL+DB offload
+ limit: 2 # Select at most two for this purpose
+ db_slots: 5 # Chop the DB device into this many slices and
+ # use one for each of this many HDD OSDs
---
service_type: osd
service_id: osd_spec_ssd
placement:
host_pattern: '*'
- spec:
+ spec: # This scenario is uncommon
data_devices:
- model: MC-55-44-XZ
- db_devices:
- vendor: VendorC
+ model: MC-55-44-XZ # Select drives of this model for OSD data
+ db_devices: # Select drives of this brand for WAL+DB. Since the
+ vendor: VendorC # data devices are SAS/SATA SSDs this would make sense for NVMe SSDs
+ db_slots: 2 # Back two slower SAS/SATA SSD data devices with each NVMe slice
This would create the desired layout by using all HDDs as data_devices with two SSD assigned as dedicated db/wal devices.
The remaining SSDs(10) will be data_devices that have the 'VendorC' NVMEs assigned as dedicated db/wal devices.