From 69055a85da3a17258a0ec41884ec61f1fa166dec Mon Sep 17 00:00:00 2001 From: Yuan Zhou Date: Sun, 28 Apr 2019 11:12:01 +0800 Subject: [PATCH] doc: adding docs for rbd shared read-only parent image cache Signed-off-by: Yuan Zhou --- doc/rbd/index.rst | 1 + doc/rbd/rbd-persistent-cache.rst | 79 ++++++++++++++++++++++++++++++ src/doc/howtotestsharedrocache.txt | 26 ---------- 3 files changed, 80 insertions(+), 26 deletions(-) create mode 100644 doc/rbd/rbd-persistent-cache.rst delete mode 100644 src/doc/howtotestsharedrocache.txt diff --git a/doc/rbd/index.rst b/doc/rbd/index.rst index 7fc91494853cb..a0ec19693d58f 100644 --- a/doc/rbd/index.rst +++ b/doc/rbd/index.rst @@ -45,6 +45,7 @@ to operate the :ref:`Ceph RADOS Gateway `, the Kernel Modules Snapshots Mirroring + Persistent Cache LIO iSCSI Gateway QEMU libvirt diff --git a/doc/rbd/rbd-persistent-cache.rst b/doc/rbd/rbd-persistent-cache.rst new file mode 100644 index 0000000000000..a54e7efc28732 --- /dev/null +++ b/doc/rbd/rbd-persistent-cache.rst @@ -0,0 +1,79 @@ +======================= + RBD Persistent Cache +======================= + +.. index:: Ceph Block Device; Persistent Cache + +Shared Read-only Parent Image Cache +=================================== + +Cloned RBD images(`rbd-snapshots`_) from one parent usually only modify a small +portion of the image. E.g., in a VDI workload, the VMs are using the same cloned +images with only hostname and IP address changed most likely. On the booting +stage, all of these VMs would read the parent content from the RADOS cluster. +If we have a local cache of the parent image, this will help to speed up the +read process on one host as well as to save the south-north network traffic. +RBD shared read-only parent image cache requires expeclictly enabling in +ceph.conf. The ``ceph-immmutable-object-cache`` daemon is responsible for +caching the parent content on local disk, and future reads on those contents +will be serviced from the local cache. + +.. note:: RBD shared read-only parent image cache requires the Ceph Nautiaulas release or later. + +.. ditaa:: +--------------------------------------------------------+ + | QEMU | + +--------------------------------------------------------+ + | librbd(cloned images) | + +-------------------+-+----------------------------------+ + | librados | | ceph-immutable-object-cache | + +-------------------+ +----------------------------------+ + | OSDs/Mons | | Local cached parent image | + +-------------------+ +----------------------------------+ + + +Enable RBD Shared Read-only Parent Image Cache +---------------------------------------------- + +The ``ceph.conf`` file `ceph-conf`_ settings for RBD shared read-only parent +image cache should be set in the ``[client]`` section of your configuration +file. +The settings include: + +``rbd shared cache enabled`` + +:Description: Enable caching for shared read-only cache. +:Type: Boolean +:Required: No +:Default: ``false`` + +Immutable Object Cache Daemon +============================= + +The ``ceph-immutable-object-cache`` daemon is responsible for caching parent +image content on local caching directory. + +The ``ceph-immutable-object-cache`` daemon is available within the optional +``ceph-immutable-object-cache`` distribution package. + +.. important:: ``ceph-immutable-object-cache`` daemon requires the ability to + connect RADOS clusters. + +``ceph-immutable-object-cache`` daemon should use a unique Ceph user ID. +To `create a Ceph user`_, with ``ceph`` specify the ``auth get-or-create`` +command, user name, monitor caps, and OSD caps:: + + ceph auth get-or-create client.ceph-immutable-object-cache.{unique id} mon 'profile immutable-object-cache' osd 'profile rbd' + +The ``ceph-immutable-object-cache`` daemon can be managed by ``systemd`` by specifying the user +ID as the daemon instance:: + + systemctl enable ceph-immutable-object-cache@immutable-object-cache.{unique id} + +The ``ceph-immutable-object-cache`` can also be run in foreground by ``ceph-immutable-object-cache`` command:: + + ceph-immutable-object-cache -f --log-file={log_path} + +.. _rbd-snapshots: ../rbd-snapshot +.. _ceph-conf: ../../rados/configuration/ceph-conf/#running-multiple-clusters +.. _create a Ceph user: ../../rados/operations/user-management#add-a-user + diff --git a/src/doc/howtotestsharedrocache.txt b/src/doc/howtotestsharedrocache.txt deleted file mode 100644 index 7355a0cbb9ddd..0000000000000 --- a/src/doc/howtotestsharedrocache.txt +++ /dev/null @@ -1,26 +0,0 @@ -0. setup a parent image and a clone image - -``` -rbd create testimage --size 100M -rbd snap create testimage@snap1 -rbd snap protect testimage@snap1 -rbd clone testimage@snap1 child_image -``` - -1. start the ceph-immutable-obj-cache daemon - -``` -ceph-immutable-obj-cache -``` -The obj cache will be promoted to /tmp/ceph_immutable_obj_cache/, the total cache size is set to 1G. -By default this feature is disabled, so you may change these params by setting new values in ceph.conf, like: -``` -rbd shared cache enabled = true -immutable object cache path = /mnt/cache # SSD based cache dir -immutable object cache max size = 16G -``` -For now the daemon simply cleans all cache on start. So if you have lots of old cache, it may take while to start - -2. start randread or read jobs against child image -Note this is a shared ro cache. So all writes will go to RADOS directly and only read can benefit from the cache - -- 2.39.5