From 29222872c490e8d6fe3e3315c7c81bb2ad7b20a4 Mon Sep 17 00:00:00 2001 From: John Wilkins Date: Mon, 11 Feb 2013 19:44:55 -0800 Subject: [PATCH] doc: Added procedures for using libvirt VMs with Ceph. fixes: #3432 Signed-off-by: John Wilkins --- doc/rbd/libvirt.rst | 225 +++++++++++++++++++++++++++++++++++++++++--- 1 file changed, 212 insertions(+), 13 deletions(-) diff --git a/doc/rbd/libvirt.rst b/doc/rbd/libvirt.rst index e0228f0838872..31a6b170fd2a8 100644 --- a/doc/rbd/libvirt.rst +++ b/doc/rbd/libvirt.rst @@ -14,15 +14,44 @@ to many different hypervisors, including: - VirtualBox - etc. -Ceph RADOS block devices support QEMU/KVM, which means you can use RADOS -block devices with software that interfaces with ``libvirt``. For example, -OpenStack's integration to Ceph uses ``libvirt`` to interact with QEMU/KVM, -and QEMU/KVM interacts with RADOS block devices via ``librbd``. +Ceph block devices support QEMU/KVM. You can use Ceph block devices with +software that interfaces with ``libvirt``. The following stack diagram +illustrates how ``libvirt`` and QEMU use Ceph block devices via ``librbd``. + + +.. ditaa:: +---------------------------------------------------+ + | libvirt | + +------------------------+--------------------------+ + | + | configures + v + +---------------------------------------------------+ + | QEMU | + +---------------------------------------------------+ + | librbd | + +------------------------+-+------------------------+ + | OSDs | | Monitors | + +------------------------+ +------------------------+ + + +The most common ``libvirt`` use case involves providing Ceph block devices to +cloud solutions like OpenStack or CloudStack. The cloud solution uses +``libvirt`` to interact with QEMU/KVM, and QEMU/KVM interacts with Ceph block +devices via ``librbd``. See `Block Devices and OpenStack`_ and `Block Devices +and CloudStack`_ for details. + +You can also use Ceph block devices with ``libvirt``, ``virsh`` and the +``libvirt`` API. See `libvirt Virtualization API`_ for details. + +Prerequisites +============= + +- `Install`_ and `configure`_ a Ceph cluster +- `Install and configure`_ QEMU/KVM -See `libvirt Virtualization API`_ for details. Installing ``libvirt`` on Ubuntu 12.04 Precise ----------------------------------------------- +============================================== ``libvirt`` packages are incorporated into the Ubuntu 12.04 precise distribution. To install ``libvirt`` on precise, execute the following:: @@ -31,12 +60,12 @@ distribution. To install ``libvirt`` on precise, execute the following:: Installing ``libvirt`` on Earlier Versions of Ubuntu ----------------------------------------------------- +==================================================== -For Ubuntu distributions 11.10 oneiric and earlier, you must build -``libvirt`` from source. Clone the ``libvirt`` repository, and use -`AutoGen`_ to generate the build. Then execute ``make`` and -``make install`` to complete the installation. For example:: +For Ubuntu distributions 11.10 oneiric and earlier, you must build ``libvirt`` +from source. Clone the ``libvirt`` repository, and use `AutoGen`_ to generate +the build. Then execute ``make`` and ``make install`` to complete the +installation. For example:: git clone git://libvirt.org/libvirt.git cd libvirt @@ -44,8 +73,178 @@ For Ubuntu distributions 11.10 oneiric and earlier, you must build make sudo make install -See `libvirt Installation`_ for details. +See `libvirt Installation`_ for details. For a reference of ``virsh`` commands, +refer to `Virsh Command Reference`_. + + +Using Ceph with Virtual Machines +================================ + +To create VMs that use Ceph block devices, use the following procedures. + + +Configuring Ceph +---------------- + +To configure Ceph for use with ``libvirt``, perform the following steps: + +#. `Create a pool`_ (or use the default). :: + + ceph osd pool create libvirt-pool 128 128 + +#. `Create a user`_ (or use ``client.admin`` for version 0.9.7 and earlier). :: + + ceph auth get-or-create client.libvirt mon 'allow r' osd 'allow class-read object_prefix rbd_children, allow rwx pool=libvirt-pool' + +#. Use QEMU to `create an image`_ in your RBD pool. :: + + qemu-img create -f rbd rbd:libvirt-pool/new-libvirt-image 2G + + + +Preparing the VM Manager +------------------------ + +You may use ``libvirt`` without a VM manager, but you may find it simpler to +create your first domain with ``virt-manager``. + +#. Install a virtual machine manager. See `KVM/VirtManager`_ for details. :: + + sudo apt-get install virt-manager + +#. Download an OS image (if necessary). + +#. Launch the virtual machine manager. :: + + sudo virt-manager + + + +Creating a VM +------------- + +To create a VM with ``virt-manager``, perform the following steps: + +#. Press the **Create New Virtual Machine** button. + +#. Name the new virtual machine. :: + + libvirt-virtual-machine + +#. Import the image. :: + + /path/to/image/debian.img + +#. Configure and start the VM. + +#. Login to the VM (root/root) + +#. Stop the VM. + + +Configuring the VM +------------------ + +To configure the VM for use with Ceph, perform the following steps: + +#. Navigate to the VM configuration file directory. :: + + cd /etc/libvirt/qemu + +#. Open the configuration file. :: + + sudo virsh edit libvirt-virtual-machine.xml + + Under ```` there should be a ```` entry. :: + + + /usr/bin/kvm + + + + +
+ + + + Replace ``/path/to/image/debian.img`` with the path to the OS image. + + **NOTE:** Use ``virsh edit`` instead of a text editor. + +#. Add the Ceph RBD image you created as a ```` entry. :: + + + + + + + + + Replace ``{monitor-host}`` with the name of your host. You may add multiple + ```` entries for your Ceph monitors. The ``dev`` attribute is the + device name that will appear under the ``/dev`` directory of your VM. The + ``bus`` attribute may be ``ide`` or ``virtio``. + +#. Save the file. + +#. If you are using `Ceph Authentication`_, you must generate a secret. :: + + cat > secret.xml < + + client.libvirt secret + + + EOF + +#. Define the secret. :: + + sudo virsh secret-define --file secret.xml + + +#. Get the ``client.libvirt`` key and save the key string to a file. :: + + sudo ceph auth list + vim client.libvirt.key + +#. Set the UUID of the secret. :: + + sudo virsh secret-set-value --secret {uuid of secret} --base64 $(cat client.libvirt.key) && rm client.libvirt.key secret.xml + + You must also set the secret manually by adding the following ```` + entry to the ```` element you entered earlier (replacing the + ``uuid`` value with the one output from the command line example above). :: + + sudo virsh edit libvirt-virtual-machine.xml + + Then, add ```` element to the domain configuration file:: + + ... + + + + +