From 76a38a6b104edea987303a94b4691513229a7771 Mon Sep 17 00:00:00 2001 From: Lucian Petrut Date: Mon, 15 Mar 2021 07:38:00 +0000 Subject: [PATCH] doc: reorganize Windows docs Most of the Windows documentation is currently included in the README.windows.rst file. To make it more accessible, we're moving most of it to the "doc/" folder, adding the following pages: * Installing Ceph on Windows * RBD on Windows * Windows troubleshooting We'll keep the build and manual install instructions in README.windows.rst. Note that ceph-dokan already has a separate doc page. Signed-off-by: Lucian Petrut --- README.windows.rst | 306 ++---------------- doc/cephfs/ceph-dokan.rst | 43 +-- doc/install/index.rst | 5 + .../windows-basic-config.rst | 4 +- doc/install/windows-install.rst | 102 ++++++ doc/install/windows-troubleshooting.rst | 96 ++++++ doc/rbd/rbd-integrations.rst | 1 + doc/rbd/rbd-windows.rst | 158 +++++++++ 8 files changed, 396 insertions(+), 319 deletions(-) rename doc/{cephfs => install}/windows-basic-config.rst (97%) create mode 100644 doc/install/windows-install.rst create mode 100644 doc/install/windows-troubleshooting.rst create mode 100644 doc/rbd/rbd-windows.rst diff --git a/README.windows.rst b/README.windows.rst index 4e950a777fe..fd64faae4d6 100644 --- a/README.windows.rst +++ b/README.windows.rst @@ -1,9 +1,9 @@ About ----- -Ceph Windows support is currently a work in progress. For now, the main focus -is the client side, allowing Windows hosts to consume rados, rbd and cephfs -resources. +The Ceph client tools and libraries can be natively used on Windows. This avoids +the need of having additional layers such as iSCSI gateways or SMB shares, +drastically improving the performance. .. _building: Building @@ -75,51 +75,18 @@ Make sure to explicitly pass the "OS" parameter when directly calling ``win32_deps_build.sh``. Also, be aware of the fact that it will use the distro specific package manager, which will require privileged rights. -Current status --------------- - -Ceph filesystems can be mounted using the ``ceph-dokan`` command, which -requires the Dokany package to be installed. Note that dokany is a well -maintained fork of the Dokan project, allowing filesystems to be implemented -in userspace, pretty much like Fuse. - -RBD images can be mounted using the ``rbd`` or ``rbd-wnbd`` commands. The -``WNBD`` driver is required for mapping RBD images on Windows. - -A significant number of tests from the ``tests`` directory have been ported, -providing adequate coverage. - -Supported platforms -=================== - -Windows Server 2019 and Windows Server 2016 are supported. Previous Windows -Server versions, including Windows client versions such as Windows 10, might -work but haven't been tested. - -Windows Server 2016 does not provide unix sockets, in which case the Ceph admin -socket feature will be unavailable. - -Compatibility -============= - -RBD images can be exposed to the OS and host Windows partitions or they can be -attached to Hyper-V VMs in the same way as iSCSI disks. - -At the moment, the Microsoft Failover Cluster can't use WNBD disks as -Cluster Shared Volumes (CSVs) underlying storage. The main reason is that -``WNBD`` and ``rbd-wnbd`` don't support the *SCSI Persistent Reservations* -feature yet. - -OpenStack integration has been proposed as well and will most probably be -included in the next OpenStack release, allowing RBD images managed by OpenStack -Cinder to be attached to Hyper-V VMs managed by OpenStack Nova. - .. _installing: Installing ---------- -The following project allows building an MSI installer that bundles ``ceph`` and -the ``WNBD`` driver: https://github.com/cloudbase/ceph-windows-installer +MSI installer +============= + +Using the MSI installer is the recommended way of installing Ceph on Windows. +Please check the `installation guide`_ for more details. + +Manual installation +=================== In order to manually install ``ceph``, start by unzipping the binaries that you may have obtained by following the building_ step. @@ -143,68 +110,8 @@ In order to map RBD images, the ``WNBD`` driver must be installed. Please check out this page for more details about ``WNBD`` and the install process: https://github.com/cloudbase/wnbd -Configuring ------------ - -ceph.conf -========= - -The default location for the ``ceph.conf`` file on Windows is -``%ProgramData%\ceph\ceph.conf``, which usually expands to -``C:\ProgramData\ceph\ceph.conf``. - -Below you may find a sample. Please fill in the monitor addresses -accordingly. - -.. code:: ini - - [global] - log to stderr = true - - run dir = C:/ProgramData/ceph/out - crash dir = C:/ProgramData/ceph/out - [client] - keyring = C:/ProgramData/ceph/keyring - ; log file = C:/ProgramData/ceph/out/$name.$pid.log - admin socket = C:/ProgramData/ceph/out/$name.$pid.asok - - ; client_permissions = true - ; client_mount_uid = 1000 - ; client_mount_gid = 1000 - [global] - mon host = - -Assuming that you're going to use this config sample, don't forget to -also copy your keyring file to the specified location and make sure -that the configured directories exist (e.g. ``C:\ProgramData\ceph\out``). - -Please use slashes ``/`` instead of backslashes ``\`` as path separators -within ``ceph.conf`` for the time being. - -.. _windows_service: -Windows service -=============== -On Windows, rbd-wnbd daemons are managed by a centralized service. This allows -decoupling the daemons from the Windows session from which they originate. At -the same time, the service is responsible of recreating persistent mappings, -usually when the host boots. - -Note that only one such service may run per host. - -By default, all image mappings are persistent. Non-persistent mappings can be -requested using the ``-onon-persistent`` ``rbd`` flag. - -Persistent mappings are recreated when the service starts, unless explicitly -unmapped. The service disconnects the mappings when being stopped. This also -allows adjusting the Windows service start order so that rbd images can be -mapped before starting services that may depend on it, such as VMMS. - -In order to be able to reconnect the images, ``rbd-wnbd`` stores mapping -information in the Windows registry at the following location: -``SYSTEM\CurrentControlSet\Services\rbd-wnbd``. - -The following command can be used to configure the service. Please update -the ``rbd-wnbd.exe`` path accordingly. +The following command can be used to configure the ``ceph-rbd`` service. +Please update the ``rbd-wnbd.exe`` path accordingly. .. code:: PowerShell @@ -213,182 +120,15 @@ the ``rbd-wnbd.exe`` path accordingly. -BinaryPathName "c:\ceph\rbd-wnbd.exe service" ` -StartupType Automatic -Usage ------ - -Cephfs -====== - -Please check the `ceph-dokan documentation`_ for more details. - -RBD -=== - -The ``rbd`` command can be used to create, remove, import, export, map or -unmap images exactly like it would on Linux. - -Mapping images -.............. - -In order to map RBD images, please install ``WNBD``, as mentioned by the -installing_ guide. - -The behavior and CLI is similar to the Linux counterpart, with a few -notable differences: - -* device paths cannot be requested. The disk number and path will be picked by - Windows. If a device path is provided by the used when mapping an image, it - will be used as an identifier, which can also be used when unmapping the - image. -* the ``show`` command was added, which describes a specific mapping. - This can be used for retrieving the disk path. -* the ``service`` command was added, allowing rbd-wnbd to run as a Windows service. - All mappings are currently perisistent, being recreated when the service - stops, unless explicitly unmapped. The service disconnects the mappings - when being stopped. -* the ``list`` command also includes a ``status`` column. - -The purpose of the ``service`` mode is to ensure that mappings survive reboots -and that the Windows service start order can be adjusted so that rbd images can -be mapped before starting services that may depend on it, such as VMMS. - -Please follow the windows_service_ guide in order to configure the service. - -The mapped images can either be consumed by the host directly or exposed to -Hyper-V VMs. - -Hyper-V VM disks -~~~~~~~~~~~~~~~~ - -The following sample imports an RBD image and boots a Hyper-V VM using it. - -.. code:: PowerShell - - # Feel free to use any other image. This one is convenient to use for - # testing purposes because it's very small (~15MB) and the login prompt - # prints the pre-configured password. - wget http://download.cirros-cloud.net/0.5.1/cirros-0.5.1-x86_64-disk.img ` - -OutFile cirros-0.5.1-x86_64-disk.img - - # We'll need to make sure that the imported images are raw (so no qcow2 or vhdx). - # You may get qemu-img from https://cloudbase.it/qemu-img-windows/ - # You can add the extracted location to $env:Path or update the path accordingly. - qemu-img convert -O raw cirros-0.5.1-x86_64-disk.img cirros-0.5.1-x86_64-disk.raw - - rbd import cirros-0.5.1-x86_64-disk.raw - # Let's give it a hefty 100MB size. - rbd resize cirros-0.5.1-x86_64-disk.raw --size=100MB - - rbd device map cirros-0.5.1-x86_64-disk.raw - - # Let's have a look at the mappings. - rbd device list - Get-Disk - - $mappingJson = rbd-wnbd show cirros-0.5.1-x86_64-disk.raw --format=json - $mappingJson = $mappingJson | ConvertFrom-Json - - $diskNumber = $mappingJson.disk_number - - New-VM -VMName BootFromRBD -MemoryStartupBytes 512MB - # The disk must be turned offline before it can be passed to Hyper-V VMs - Set-Disk -Number $diskNumber -IsOffline $true - Add-VMHardDiskDrive -VMName BootFromRBD -DiskNumber $diskNumber - Start-VM -VMName BootFromRBD - -Windows partitions -~~~~~~~~~~~~~~~~~~ - -The following sample creates an empty RBD image, attaches it to the host and -initializes a partition. - -.. code:: PowerShell - - rbd create blank_image --size=1G - rbd device map blank_image -onon-persistent - - $mappingJson = rbd-wnbd show blank_image --format=json - $mappingJson = $mappingJson | ConvertFrom-Json - - $diskNumber = $mappingJson.disk_number - - # The disk must be online before creating or accessing partitions. - Set-Disk -Number $diskNumber -IsOffline $false - - # Initialize the disk, partition it and create a fileystem. - Get-Disk -Number $diskNumber | ` - Initialize-Disk -PassThru | ` - New-Partition -AssignDriveLetter -UseMaximumSize | ` - Format-Volume -Force -Confirm:$false - -Troubleshooting -............... - -Wnbd -~~~~ - -For ``WNBD`` troubleshooting, please check this page: https://github.com/cloudbase/wnbd#troubleshooting - -Privileges -~~~~~~~~~~ - -Most ``rbd-wnbd`` and ``rbd device`` commands require privileged rights. Make -sure to use an elevated PowerShell or CMD command prompt. - -Crash dumps -~~~~~~~~~~~ - -Userspace crash dumps can be placed at a configurable location and enabled for all -applications or just predefined ones, as outlined here: -https://docs.microsoft.com/en-us/windows/win32/wer/collecting-user-mode-dumps. - -Whenever a Windows application crashes, an event will be submitted to the ``Application`` -Windows Event Log, having Event ID 1000. The entry will also include the process id, -the faulting module name and path as well as the exception code. - -Please note that in order to analyze crash dumps, the debug symbols are required. -We're currently buidling Ceph using ``MinGW``, so by default ``DWARF`` symbols will -be embedded in the binaries. ``windbg`` does not support such symbols but ``gdb`` -can be used. - -``gdb`` can debug running Windows processes but it cannot open Windows minidumps. -The following ``gdb`` fork may be used until this functionality is merged upstream: -https://github.com/ssbssa/gdb/releases. As an alternative, ``DWARF`` symbols -can be converted using ``cv2pdb`` but be aware that this tool has limitted C++ -support. - -ceph tool -~~~~~~~~~ - -The ``ceph`` Python tool can't be used on Windows natively yet. With minor -changes it may run, but the main issue is that Python doesn't currently allow -using ``AF_UNIX`` on Windows: https://bugs.python.org/issue33408 - -As an alternative, the ``ceph`` tool can be used through Windows Subsystem -for Linux (WSL). For example, running Windows RBD daemons may be contacted by -using: - -.. code:: bash - - ceph daemon /mnt/c/ProgramData/ceph/out/ceph-client.admin.61436.1209215304.asok help - -IO counters -~~~~~~~~~~~ - -Along with the standard RBD perf counters, the ``libwnbd`` IO counters may be -retrieved using: - -.. code:: PowerShell - - rbd-wnbd stats $imageName - -At the same time, WNBD driver counters can be fetched using: - -.. code:: PowerShell - - wnbd-client stats $mappingId +Further reading +--------------- -Note that the ``wnbd-client`` mapping identifier will be the full RBD image spec -(the ``device`` column of the ``rbd device list`` output). +* `installation guide`_ +* `RBD Windows documentation`_ +* `Ceph Dokan documentation`_ +* `Windows troubleshooting`_ -.. _ceph-dokan documentation: https://docs.ceph.com/en/latest/cephfs/ceph-dokan/ +.. _Ceph Dokan documentation: https://docs.ceph.com/en/latest/cephfs/ceph-dokan/ +.. _RBD Windows documentation: https://docs.ceph.com/en/latest/rbd/rbd-windows/ +.. _Windows troubleshooting: https://docs.ceph.com/en/latest/install/windows-troubleshooting +.. _installation guide: https://docs.ceph.com/en/latest/install/windows-install diff --git a/doc/cephfs/ceph-dokan.rst b/doc/cephfs/ceph-dokan.rst index 27eb75e20d2..6ddb57082d4 100644 --- a/doc/cephfs/ceph-dokan.rst +++ b/doc/cephfs/ceph-dokan.rst @@ -6,38 +6,7 @@ Mount CephFS on Windows It leverages Dokany, a Windows driver that allows implementing filesystems in userspace, pretty much like FUSE. -Prerequisites -============= - -Dokany ------- - -``ceph-dokan`` requires Dokany to be installed. You may fetch the installer as -well as the source code from the Dokany Github repository: -https://github.com/dokan-dev/dokany/releases - -The minimum supported Dokany version is 1.3.1. At the time of the writing, -Dokany 2.0 is in Beta stage and is unsupported. - -Supported platforms -------------------- - -Windows Server 2019 and Windows Server 2016 are supported. Previous Windows -Server versions, including Windows client versions such as Windows 10, might -work but haven't been tested. - -Configuration -============= - -``ceph-dokan`` requires minimal configuration. Please check the -`Windows configuration sample`_ to get started. - -You'll also need a keyring file. The `General Prerequisites`_ page provides a -simple example, showing how a new CephX user can be created and how its secret -key can be retrieved. - -For more details on CephX user management, see the `Client Authentication`_ -and :ref:`User Management `. +Please check the `installation guide`_ to get started. Usage ===== @@ -87,6 +56,10 @@ Unlike ``rbd-wnbd``, ``ceph-dokan`` doesn't currently provide a ``service`` command. In order for the cephfs mount to survive host reboots, consider using ``NSSM``. -.. _Windows configuration sample: ../windows-basic-config -.. _General Prerequisites: ../mount-prerequisites -.. _Client Authentication: ../client-auth +Troubleshooting +=============== + +Please consult the `Windows troubleshooting`_ page. + +.. _Windows troubleshooting: ../../install/windows-troubleshooting +.. _installation guide: ../../install/windows-install diff --git a/doc/install/index.rst b/doc/install/index.rst index 044c27d9e87..8f76477d123 100644 --- a/doc/install/index.rst +++ b/doc/install/index.rst @@ -64,5 +64,10 @@ Ceph can also be :ref:`installed manually `. index_manual +Windows +~~~~~~~ +For Windows installations, please consult this document: +`Windows installation guide`_. +.. _Windows installation guide: ./windows-install diff --git a/doc/cephfs/windows-basic-config.rst b/doc/install/windows-basic-config.rst similarity index 97% rename from doc/cephfs/windows-basic-config.rst rename to doc/install/windows-basic-config.rst index 082c10c1a56..0fe8a1b1b14 100644 --- a/doc/cephfs/windows-basic-config.rst +++ b/doc/install/windows-basic-config.rst @@ -15,7 +15,9 @@ The default location for the ``ceph.conf`` file on Windows is ``C:\ProgramData\ceph\ceph.conf``. Below you may find a sample. Please fill in the monitor addresses -accordingly:: +accordingly. + +.. code:: ini [global] log to stderr = true diff --git a/doc/install/windows-install.rst b/doc/install/windows-install.rst new file mode 100644 index 00000000000..ff10c346d82 --- /dev/null +++ b/doc/install/windows-install.rst @@ -0,0 +1,102 @@ +:orphan: + +========================== +Installing Ceph on Windows +========================== + +The Ceph client tools and libraries can be natively used on Windows. This avoids +the need of having additional layers such as iSCSI gateways or SMB shares, +drastically improving the performance. + +Prerequisites +============= + +Supported platforms +------------------- + +Windows Server 2019 and Windows Server 2016 are supported. Previous Windows +Server versions, including Windows client versions such as Windows 10, might +work but haven't been tested. + +Windows Server 2016 does not provide Unix sockets, in which case some commands +might be unavailable. + +Secure boot +----------- + +The ``WNBD`` driver hasn't been signed by Microsoft, which means that Secure Boot +must be disabled. + +Dokany +------ + +In order to mount Ceph filesystems, ``ceph-dokan`` requires Dokany to be +installed. You may fetch the installer as well as the source code from the +Dokany Github repository: https://github.com/dokan-dev/dokany/releases + +The minimum supported Dokany version is 1.3.1. At the time of the writing, +Dokany 2.0 is in Beta stage and is unsupported. + +Unlike ``WNBD``, Dokany isn't included in the Ceph MSI installer. + +MSI installer +============= + +Using the MSI installer is the recommended way of installing Ceph on Windows. +It performs the following steps: + +* copy the required files to the configured install location + (defaulting to ``C:\Program Files\Ceph``) +* install the required Visual C++ runtime +* install the WNBD driver and create a new WNBD SCSI adapter device +* register the WNBD ETW manifest +* register and start the ``ceph-rbd`` Windows service +* update the environment ``PATH`` variable + +.. + TODO: Update the link once we have a download page. + +The MSI can be downloaded from here: +https://cloudbase.it/downloads/ceph_v16_0_0_beta.msi + +As mentioned earlier, the Ceph installer does not include Dokany, which has +to be installed separately. + +A server reboot is required after uninstalling the driver, otherwise subsequent +install attempts may fail. + +The following project allows building the MSI installer: +https://github.com/cloudbase/ceph-windows-installer. It can either use prebuilt +Ceph and WNBD binaries or compile them from scratch. + +Manual installation +=================== + +The following document describes the build process and manual installation: +https://github.com/ceph/ceph/blob/master/README.windows.rst + +Configuration +============= + +Please check the `Windows configuration sample`_ to get started. + +You'll also need a keyring file. The `General CephFS Prerequisites`_ page provides a +simple example, showing how a new CephX user can be created and how its secret +key can be retrieved. + +For more details on CephX user management, see the `Client Authentication`_ +and :ref:`User Management `. + +Further reading +=============== + +* `RBD Windows documentation`_ +* `CephFS Windows documentation`_ +* `Windows troubleshooting`_ + +.. _CephFS Windows documentation: ../../cephfs/ceph-dokan +.. _Windows configuration sample: ../windows-basic-config +.. _RBD Windows documentation: ../../rbd/rbd-windows/ +.. _Windows troubleshooting: ../windows-troubleshooting +.. _General CephFS Prerequisites: ../../cephfs/mount-prerequisites +.. _Client Authentication: ../../cephfs/client-auth diff --git a/doc/install/windows-troubleshooting.rst b/doc/install/windows-troubleshooting.rst new file mode 100644 index 00000000000..355fd88032c --- /dev/null +++ b/doc/install/windows-troubleshooting.rst @@ -0,0 +1,96 @@ +:orphan: + +=============================== +Troubleshooting Ceph on Windows +=============================== + +MSI installer +~~~~~~~~~~~~~ + +The MSI source code can be consulted here: +https://github.com/cloudbase/ceph-windows-installer + +The following command can be used to generate MSI logs:: + + msiexec.exe /i $msi_full_path /l*v! $log_file + +WNBD driver installation failures will be logged here: ``C:\Windows\inf\setupapi.dev.log``. +A server reboot is required after uninstalling the driver, otherwise subsequent +install attempts may fail. + +Wnbd +~~~~ + +For ``WNBD`` troubleshooting, please check this page: https://github.com/cloudbase/wnbd#troubleshooting + +Privileges +~~~~~~~~~~ + +Most ``rbd-wnbd`` and ``rbd device`` commands require privileged rights. Make +sure to use an elevated PowerShell or CMD command prompt. + +Crash dumps +~~~~~~~~~~~ + +Userspace crash dumps can be placed at a configurable location and enabled for all +applications or just predefined ones, as outlined here: +https://docs.microsoft.com/en-us/windows/win32/wer/collecting-user-mode-dumps. + +Whenever a Windows application crashes, an event will be submitted to the ``Application`` +Windows Event Log, having Event ID 1000. The entry will also include the process id, +the faulting module name and path as well as the exception code. + +Please note that in order to analyze crash dumps, the debug symbols are required. +We're currently buidling Ceph using ``MinGW``, so by default ``DWARF`` symbols will +be embedded in the binaries. ``windbg`` does not support such symbols but ``gdb`` +can be used. + +``gdb`` can debug running Windows processes but it cannot open Windows minidumps. +The following ``gdb`` fork may be used until this functionality is merged upstream: +https://github.com/ssbssa/gdb/releases. As an alternative, ``DWARF`` symbols +can be converted using ``cv2pdb`` but be aware that this tool has limitted C++ +support. + +ceph tool +~~~~~~~~~ + +The ``ceph`` Python tool can't be used on Windows natively yet. With minor +changes it may run, but the main issue is that Python doesn't currently allow +using ``AF_UNIX`` on Windows: https://bugs.python.org/issue33408 + +As an alternative, the ``ceph`` tool can be used through Windows Subsystem +for Linux (WSL). For example, running Windows RBD daemons may be contacted by +using: + +.. code:: bash + + ceph daemon /mnt/c/ProgramData/ceph/out/ceph-client.admin.61436.1209215304.asok help + +IO counters +~~~~~~~~~~~ + +Along with the standard RBD perf counters, the ``libwnbd`` IO counters may be +retrieved using: + +.. code:: PowerShell + + rbd-wnbd stats $imageName + +At the same time, WNBD driver counters can be fetched using: + +.. code:: PowerShell + + wnbd-client stats $mappingId + +Note that the ``wnbd-client`` mapping identifier will be the full RBD image spec +(the ``device`` column of the ``rbd device list`` output). + +Missing libraries +~~~~~~~~~~~~~~~~~ + +The Ceph tools can silently exit with a -1073741515 return code if one of the +required DLLs is missing or unsupported. + +The `Dependency Walker`_ tool can be used to determine the missing library. + +.. _Dependency Walker: https://www.dependencywalker.com/ diff --git a/doc/rbd/rbd-integrations.rst b/doc/rbd/rbd-integrations.rst index 2203364f06c..50d788d5397 100644 --- a/doc/rbd/rbd-integrations.rst +++ b/doc/rbd/rbd-integrations.rst @@ -12,3 +12,4 @@ OpenStack CloudStack LIO iSCSI Gateway + Windows diff --git a/doc/rbd/rbd-windows.rst b/doc/rbd/rbd-windows.rst new file mode 100644 index 00000000000..271e85592a5 --- /dev/null +++ b/doc/rbd/rbd-windows.rst @@ -0,0 +1,158 @@ +============== +RBD on Windows +============== + +The ``rbd`` command can be used to create, remove, import, export, map or +unmap images exactly like it would on Linux. Make sure to check the +`RBD basic commands`_ guide. + +``librbd.dll`` is also available for applications that can natively use Ceph. + +Please check the `installation guide`_ to get started. + +Windows service +=============== +On Windows, ``rbd-wnbd`` daemons are managed by a centralized service. This allows +decoupling the daemons from the Windows session from which they originate. At +the same time, the service is responsible of recreating persistent mappings, +usually when the host boots. + +Note that only one such service may run per host. + +By default, all image mappings are persistent. Non-persistent mappings can be +requested using the ``-onon-persistent`` ``rbd`` flag. + +Persistent mappings are recreated when the service starts, unless explicitly +unmapped. The service disconnects the mappings when being stopped. This also +allows adjusting the Windows service start order so that RBD images can be +mapped before starting services that may depend on it, such as VMMS. + +In order to be able to reconnect the images, ``rbd-wnbd`` stores mapping +information in the Windows registry at the following location: +``SYSTEM\CurrentControlSet\Services\rbd-wnbd``. + +The following command can be used to configure the service. Please update +the ``rbd-wnbd.exe`` path accordingly:: + + New-Service -Name "ceph-rbd" ` + -Description "Ceph RBD Mapping Service" ` + -BinaryPathName "c:\ceph\rbd-wnbd.exe service" ` + -StartupType Automatic + +Note that the Ceph MSI installer takes care of creating the ``ceph-rbd`` +Windows service. + +Usage +===== + +Integration +----------- + +RBD images can be exposed to the OS and host Windows partitions or they can be +attached to Hyper-V VMs in the same way as iSCSI disks. + +Starting with Openstack Wallaby, the Nova Hyper-V driver can attach RBD Cinder +volumes to Hyper-V VMs. + +Mapping images +-------------- + +The workflow and CLI is similar to the Linux counterpart, with a few +notable differences: + +* device paths cannot be requested. The disk number and path will be picked by + Windows. If a device path is provided by the used when mapping an image, it + will be used as an identifier, which can also be used when unmapping the + image. +* the ``show`` command was added, which describes a specific mapping. + This can be used for retrieving the disk path. +* the ``service`` command was added, allowing ``rbd-wnbd`` to run as a Windows service. + All mappings are by default perisistent, being recreated when the service + stops, unless explicitly unmapped. The service disconnects the mappings + when being stopped. +* the ``list`` command also includes a ``status`` column. + +The purpose of the ``service`` mode is to ensure that mappings survive reboots +and that the Windows service start order can be adjusted so that RBD images can +be mapped before starting services that may depend on it, such as VMMS. + +The mapped images can either be consumed by the host directly or exposed to +Hyper-V VMs. + +Hyper-V VM disks +---------------- + +The following sample imports an RBD image and boots a Hyper-V VM using it:: + + # Feel free to use any other image. This one is convenient to use for + # testing purposes because it's very small (~15MB) and the login prompt + # prints the pre-configured password. + wget http://download.cirros-cloud.net/0.5.1/cirros-0.5.1-x86_64-disk.img ` + -OutFile cirros-0.5.1-x86_64-disk.img + + # We'll need to make sure that the imported images are raw (so no qcow2 or vhdx). + # You may get qemu-img from https://cloudbase.it/qemu-img-windows/ + # You can add the extracted location to $env:Path or update the path accordingly. + qemu-img convert -O raw cirros-0.5.1-x86_64-disk.img cirros-0.5.1-x86_64-disk.raw + + rbd import cirros-0.5.1-x86_64-disk.raw + # Let's give it a hefty 100MB size. + rbd resize cirros-0.5.1-x86_64-disk.raw --size=100MB + + rbd device map cirros-0.5.1-x86_64-disk.raw + + # Let's have a look at the mappings. + rbd device list + Get-Disk + + $mappingJson = rbd-wnbd show cirros-0.5.1-x86_64-disk.raw --format=json + $mappingJson = $mappingJson | ConvertFrom-Json + + $diskNumber = $mappingJson.disk_number + + New-VM -VMName BootFromRBD -MemoryStartupBytes 512MB + # The disk must be turned offline before it can be passed to Hyper-V VMs + Set-Disk -Number $diskNumber -IsOffline $true + Add-VMHardDiskDrive -VMName BootFromRBD -DiskNumber $diskNumber + Start-VM -VMName BootFromRBD + +Windows partitions +------------------ + +The following sample creates an empty RBD image, attaches it to the host and +initializes a partition:: + + rbd create blank_image --size=1G + rbd device map blank_image -onon-persistent + + $mappingJson = rbd-wnbd show blank_image --format=json + $mappingJson = $mappingJson | ConvertFrom-Json + + $diskNumber = $mappingJson.disk_number + + # The disk must be online before creating or accessing partitions. + Set-Disk -Number $diskNumber -IsOffline $false + + # Initialize the disk, partition it and create a fileystem. + Get-Disk -Number $diskNumber | ` + Initialize-Disk -PassThru | ` + New-Partition -AssignDriveLetter -UseMaximumSize | ` + Format-Volume -Force -Confirm:$false + +Limitations +----------- + +At the moment, the Microsoft Failover Cluster can't use WNBD disks as +Cluster Shared Volumes (CSVs) underlying storage. The main reason is that +``WNBD`` and ``rbd-wnbd`` don't support the *SCSI Persistent Reservations* +feature yet. + +Troubleshooting +=============== + +Please consult the `Windows troubleshooting`_ page. + +.. _Windows troubleshooting: ../../install/windows-troubleshooting +.. _installation guide: ../../install/windows-install +.. _RBD basic commands: ../rados-rbd-cmds +.. _WNBD driver: https://github.com/cloudbase/wnbd -- 2.39.5