doc/rbd/rbd-exclusive-locks: warn about automatic lock transitions

A lot of people aren't aware of automatic lock transitions and
wrongfully assume that exclusive lock means that the image remains
locked for as long as the client is running.  Redo the explanation
and add a warning.

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
(cherry picked from commit 2af7252b332c17e3ad8a363f13e2404d1276e2bd)

diff --git a/doc/rbd/rbd-exclusive-locks.rst b/doc/rbd/rbd-exclusive-locks.rst
index 5e4b4ac87f2079755fd0f2b94a6d8574f1b77764..f9b99dfb44305c7235f4595a13cdb1e2bbf61ec8 100644 (file)
--- a/doc/rbd/rbd-exclusive-locks.rst
+++ b/doc/rbd/rbd-exclusive-locks.rst
@@ -9,49 +9,68 @@
 Exclusive locks are mechanisms designed to prevent multiple processes from
 accessing the same Rados Block Device (RBD) in an uncoordinated fashion.
 Exclusive locks are used heavily in virtualization (where they prevent VMs from
-clobbering each other's writes) and in RBD mirroring (where they are a
-prerequisite for journaling).
+clobbering each other's writes) and in `RBD mirroring`_ (where they are a
+prerequisite for journaling in journal-based mirroring and fast generation of
+incremental diffs in snapshot-based mirroring).
 
-By default, exclusive locks are enabled on newly created images. This default
+The ``exclusive-lock`` feature is enabled on newly created images. This default
 can be overridden via the ``rbd_default_features`` configuration option or the
-``--image-feature`` option for ``rbd create``.
+``--image-feature`` and ``--image-shared`` options for ``rbd create`` command.
 
 .. note::
    Many image features, including ``object-map`` and ``fast-diff``, depend upon
    exclusive locking. Disabling the ``exclusive-lock`` feature will negatively
    affect the performance of some operations.
 
-Exclusive locking is mostly transparent to the user.
+To maintain multi-client access, the ``exclusive-lock`` feature implements
+automatic cooperative lock transitions between clients. It ensures that only
+a single client can write to an RBD image at any given time and thus protects
+internal image structures such as the object map, the journal or the `PWL
+cache`_ from concurrent modification.
 
-#. Whenever any ``librbd`` client process or kernel RBD client
-   starts using an RBD image on which exclusive locking has been
-   enabled, it obtains an exclusive lock on the image before the first
-   write.
+Exclusive locking is mostly transparent to the user:
 
-#. Whenever any such client process terminates gracefully, the process
-   relinquishes the lock automatically.
+* Whenever a client (a ``librbd`` process or, in case of a ``krbd`` client,
+  a client node's kernel) needs to handle a write to an RBD image on which
+  exclusive locking has been enabled, it first acquires an exclusive lock on
+  the image. If the lock is already held by some other client, that client is
+  requested to release it.
 
-#. This graceful termination enables another, subsequent, process to acquire
-   the lock and to write to the image.
+* Whenever a client that holds an exclusive lock on an RBD image gets
+  a request to release the lock, it stops handling writes, flushes its caches
+  and releases the lock.
+
+* Whenever a client that holds an exclusive lock on an RBD image terminates
+  gracefully, the lock is also released gracefully.
+
+* A graceful release of an exclusive lock on an RBD image (whether by request
+  or due to client termination) enables another, subsequent, client to acquire
+  the lock and start handling writes.
+
+.. warning::
+   By default, the ``exclusive-lock`` feature does not prevent two or more
+   concurrently running clients from opening the same RBD image and writing to
+   it in turns (whether on the same node or not). In effect, their writes just
+   get linearized as the lock is automatically transitioned back and forth in
+   a cooperative fashion.
 
 .. note::
-   It is possible for two or more concurrently running processes to open the
-   image and to read from it. The client acquires the exclusive lock only when
-   attempting to write to the image. To disable transparent lock transitions
-   between multiple clients, the client must acquire the lock by using the
-   ``RBD_LOCK_MODE_EXCLUSIVE`` flag.
+   To disable automatic lock transitions between clients, the
+   ``RBD_LOCK_MODE_EXCLUSIVE`` flag may be specified when acquiring the
+   exclusive lock. This is exposed by the ``--exclusive`` option for ``rbd
+   device map`` command.
 
 
 Blocklisting
 ============
 
-Sometimes a client process (or, in case of a krbd client, a client node's
-kernel) that previously held an exclusive lock on an image does not terminate
-gracefully, but dies abruptly. This may be because the client process received
-a ``KILL`` or ``ABRT`` signal, or because the client node underwent a hard
-reboot or suffered a power failure. In cases like this, the exclusive lock is
-never gracefully released. This means that any new process that starts and
-attempts to use the device must break the previously held exclusive lock.
+Sometimes a client that previously held an exclusive lock on an RBD image does
+not terminate gracefully, but dies abruptly. This may be because the client
+process received a ``KILL`` or ``ABRT`` signal, or because the client node
+underwent a hard reboot or suffered a power failure. In cases like this, the
+lock is never gracefully released. This means that any new client that comes up
+and attempts to write to the image must break the previously held exclusive
+lock.
 
 However, a process (or kernel thread) may hang or merely lose network
 connectivity to the Ceph cluster for some amount of time. In that case,
@@ -80,4 +99,6 @@ Blocklisting is thus a form of storage-level resource `fencing`_.
    rbd`` capability profile, which should be set generally on all Ceph
    :ref:`client identities <user-management>` using RBD.
 
+.. _RBD mirroring: ../rbd-mirroring
+.. _PWL cache: ../rbd-persistent-write-log-cache
 .. _fencing: https://en.wikipedia.org/wiki/Fencing_(computing)