From: Zac Dover Date: Wed, 17 May 2023 12:25:38 +0000 (+1000) Subject: doc/cephfs: line-edit "Mirroring Module" X-Git-Tag: v17.2.7~405^2 X-Git-Url: http://git.apps.os.sepia.ceph.com/?a=commitdiff_plain;h=refs%2Fpull%2F51543%2Fhead;p=ceph.git doc/cephfs: line-edit "Mirroring Module" Line-edit the "Mirroring Module" section of doc/cephfs/cephfs-mirroring.rst. Add prompts and formatting where such things contribute to the realization of adequate sentences. This commit is a follow-up to https://github.com/ceph/ceph/pull/51505. Co-authored-by: Anthony D'Atri Signed-off-by: Zac Dover (cherry picked from commit dd8855d9a934bcdd6a026f1308ba7410b1e143e3) --- diff --git a/doc/cephfs/cephfs-mirroring.rst b/doc/cephfs/cephfs-mirroring.rst index bda66b7eef388..0f7677ba24b98 100644 --- a/doc/cephfs/cephfs-mirroring.rst +++ b/doc/cephfs/cephfs-mirroring.rst @@ -14,6 +14,8 @@ Requirements The primary (local) and secondary (remote) Ceph clusters version should be Pacific or later. +.. _cephfs_mirroring_creating_users: + Creating Users -------------- @@ -42,93 +44,155 @@ Mirror daemon should be spawned using `systemctl(1)` unit files:: $ cephfs-mirror --id mirror --cluster site-a -f -.. note:: User used here is `mirror` created in the `Creating Users` section. +.. note:: The user specified here is `mirror`, the creation of which is + described in the :ref:`Creating Users` + section. + +Multiple ``cephfs-mirror`` daemons may be deployed for concurrent +synchronization and high availability. Mirror daemons share the synchronization +load using a simple ``M/N`` policy, where ``M`` is the number of directories +and ``N`` is the number of ``cephfs-mirror`` daemons. + +When ``cephadm`` is used to manage a Ceph cluster, ``cephfs-mirror`` daemons can be +deployed by running the following command: + +.. prompt:: bash $ + + ceph orch apply cephfs-mirror + +To deploy multiple mirror daemons, run a command of the following form: + +.. prompt:: bash $ + + ceph orch apply cephfs-mirror --placement= + +For example, to deploy 3 `cephfs-mirror` daemons on different hosts, run a command of the following form: + +.. prompt:: bash $ + + $ ceph orch apply cephfs-mirror --placement="3 host1,host2,host3" Interface --------- -`Mirroring` module (manager plugin) provides interfaces for managing directory snapshot -mirroring. Manager interfaces are (mostly) wrappers around monitor commands for managing -file system mirroring and is the recommended control interface. +The `Mirroring` module (manager plugin) provides interfaces for managing +directory snapshot mirroring. These are (mostly) wrappers around monitor +commands for managing file system mirroring and is the recommended control +interface. Mirroring Module ---------------- -The mirroring module is responsible for assigning directories to mirror daemons for -synchronization. Multiple mirror daemons can be spawned to achieve concurrency in -directory snapshot synchronization. When mirror daemons are spawned (or terminated) -, the mirroring module discovers the modified set of mirror daemons and rebalances -the directory assignment amongst the new set thus providing high-availability. +The mirroring module is responsible for assigning directories to mirror daemons +for synchronization. Multiple mirror daemons can be spawned to achieve +concurrency in directory snapshot synchronization. When mirror daemons are +spawned (or terminated), the mirroring module discovers the modified set of +mirror daemons and rebalances directory assignments across the new set, thus +providing high-availability. + +.. note:: Deploying a single mirror daemon is recommended. Running multiple + daemons is untested. + +The mirroring module is disabled by default. To enable the mirroring module, +run the following command: -.. note:: Multiple mirror daemons is currently untested. Only a single mirror daemon - is recommended. +.. prompt:: bash $ -Mirroring module is disabled by default. To enable mirroring use:: + ceph mgr module enable mirroring - $ ceph mgr module enable mirroring +The mirroring module provides a family of commands that can be used to control +the mirroring of directory snapshots. To add or remove directories, mirroring +must be enabled for a given file system. To enable mirroring for a given file +system, run a command of the following form: -Mirroring module provides a family of commands to control mirroring of directory -snapshots. To add or remove directories, mirroring needs to be enabled for a given -file system. To enable mirroring use:: +.. prompt:: bash $ - $ ceph fs snapshot mirror enable + ceph fs snapshot mirror enable -.. note:: Mirroring module commands use `fs snapshot mirror` prefix as compared to - the monitor commands which `fs mirror` prefix. Make sure to use module - commands. +.. note:: "Mirroring module" commands are prefixed with ``fs snapshot mirror``. + This distinguishes them from "monitor commands", which are prefixed with ``fs + mirror``. Be sure (in this context) to use module commands. -To disable mirroring, use:: +To disable mirroring for a given file system, run a command of the following form: - $ ceph fs snapshot mirror disable +.. prompt:: bash $ -Once mirroring is enabled, add a peer to which directory snapshots are to be mirrored. -Peers are specified by `@` format which is also termed as `remote_cluster_spec` -(further in this document) and are assigned a unique-id (UUID) when added. See `Creating Users` -section on how to create Ceph users for mirroring. + ceph fs snapshot mirror disable -To add a peer use:: +After mirroring is enabled, add a peer to which directory snapshots are to be +mirrored. Peers are specified by the ``@`` format, which is +referred to elsewhere in this document as the ``remote_cluster_spec``. Peers +are assigned a unique-id (UUID) when added. See the :ref:`Creating +Users` section for instructions that describe +how to create Ceph users for mirroring. - $ ceph fs snapshot mirror peer_add [] [] [] +To add a peer, run a command of the following form: -`` is of format `client.@`. -`` is optional, and defaults to `` (on the remote cluster). +.. prompt:: bash $ -This requires the remote cluster ceph configuration and user keyring to be available in -the primary cluster. E.g.: Say, user `client_mirror` is created on the remote cluster which -has `rwps` permissions for the remote file system `remote_fs` (Refer `Creating Users`) and the -remote cluster is named `remote_ceph` (i.e., the remote cluster configuration file is named -`remote_ceph.conf` on the primary cluster), use the following command to add the remote filesystem -as a peer to the primary filesystem `primary_fs`:: + ceph fs snapshot mirror peer_add [] [] [] - $ ceph fs snapshot mirror peer_add primary_fs client.mirror_remote@remote_ceph remote_fs +```` is of the format ``client.@``. -To avoid maintaining remote cluster configuration file and remote ceph user keyring in the primary -cluster, users can bootstrap a peer (which stores the relevant remote cluster details in the monitor -config store on the parimary cluster). See `Bootstrap Peers` section. +```` is optional, and defaults to `` (on the remote +cluster). -Note that, `peer_add` additionally supports passing the remote cluster monitor address and the user key. -However, bootstrapping a peer is the recommended way to add a peer. +For this command to succeed, the remote cluster's Ceph configuration and user +keyring must be available in the primary cluster. For example, if a user named +``client_mirror`` is created on the remote cluster which has ``rwps`` +permissions for the remote file system named ``remote_fs`` (see `Creating +Users`) and the remote cluster is named ``remote_ceph`` (that is, the remote +cluster configuration file is named ``remote_ceph.conf`` on the primary +cluster), run the following command to add the remote filesystem as a peer to +the primary filesystem ``primary_fs``: + +.. prompt:: bash $ + + ceph fs snapshot mirror peer_add primary_fs client.mirror_remote@remote_ceph remote_fs + +To avoid having to maintain the remote cluster configuration file and remote +ceph user keyring in the primary cluster, users can bootstrap a peer (which +stores the relevant remote cluster details in the monitor config store on the +primary cluster). See the :ref:`Bootstrap +Peers` section. + +The ``peer_add`` command supports passing the remote cluster monitor address +and the user key. However, bootstrapping a peer is the recommended way to add a +peer. .. note:: Only a single peer is supported right now. -To remove a peer use:: +To remove a peer, run a command of the following form: + +.. prompt:: bash $ + + ceph fs snapshot mirror peer_remove + +To list file system mirror peers, run a command of the following form: - $ ceph fs snapshot mirror peer_remove +.. prompt:: bash $ -To list file system mirror peers use:: + ceph fs snapshot mirror peer_list - $ ceph fs snapshot mirror peer_list +To configure a directory for mirroring, run a command of the following form: -To configure a directory for mirroring, use:: +.. prompt:: bash $ - $ ceph fs snapshot mirror add + ceph fs snapshot mirror add -To stop a mirroring directory snapshots use:: +To stop mirroring directory snapshots, run a command of the following form: - $ ceph fs snapshot mirror remove +.. prompt:: bash $ -Only absolute directory paths are allowed. Also, paths are normalized by the mirroring -module, therefore, `/a/b/../b` is equivalent to `/a/b`. + ceph fs snapshot mirror remove + +Only absolute directory paths are allowed. + +Paths are normalized by the mirroring module. This means that ``/a/b/../b`` is +equivalent to ``/a/b``. Paths always start from the CephFS file-system root and +not from the host system mount point. + +For example:: $ mkdir -p /d0/d1/d2 $ ceph fs snapshot mirror add cephfs /d0/d1/d2 @@ -136,16 +200,19 @@ module, therefore, `/a/b/../b` is equivalent to `/a/b`. $ ceph fs snapshot mirror add cephfs /d0/d1/../d1/d2 Error EEXIST: directory /d0/d1/d2 is already tracked -Once a directory is added for mirroring, its subdirectory or ancestor directories are -disallowed to be added for mirroring:: +After a directory is added for mirroring, the additional mirroring of +subdirectories or ancestor directories is disallowed:: $ ceph fs snapshot mirror add cephfs /d0/d1 Error EINVAL: /d0/d1 is a ancestor of tracked path /d0/d1/d2 $ ceph fs snapshot mirror add cephfs /d0/d1/d2/d3 Error EINVAL: /d0/d1/d2/d3 is a subtree of tracked path /d0/d1/d2 -Commands to check directory mapping (to mirror daemons) and directory distribution are -detailed in `Mirroring Status` section. +The :ref:`Mirroring Status` section contains +information about the commands for checking the directory mapping (to mirror +daemons) and for checking the directory distribution. + +.. _cephfs_mirroring_bootstrap_peers: Bootstrap Peers --------------- @@ -173,6 +240,9 @@ e.g.:: $ ceph fs snapshot mirror peer_bootstrap import cephfs eyJmc2lkIjogIjBkZjE3MjE3LWRmY2QtNDAzMC05MDc5LTM2Nzk4NTVkNDJlZiIsICJmaWxlc3lzdGVtIjogImJhY2t1cF9mcyIsICJ1c2VyIjogImNsaWVudC5taXJyb3JfcGVlcl9ib290c3RyYXAiLCAic2l0ZV9uYW1lIjogInNpdGUtcmVtb3RlIiwgImtleSI6ICJBUUFhcDBCZ0xtRmpOeEFBVnNyZXozai9YYUV0T2UrbUJEZlJDZz09IiwgIm1vbl9ob3N0IjogIlt2MjoxOTIuMTY4LjAuNTo0MDkxOCx2MToxOTIuMTY4LjAuNTo0MDkxOV0ifQ== + +.. _cephfs_mirroring_mirroring_status: + Mirroring Status ----------------