From: Patrick Donnelly Date: Thu, 24 Dec 2020 00:21:20 +0000 (-0800) Subject: doc: update multiple mds scrub docs X-Git-Tag: v16.1.0~35^2 X-Git-Url: http://git-server-git.apps.pok.os.sepia.ceph.com/?a=commitdiff_plain;h=03e7f9d827dbaed7bf4d4613544246eb77a02e80;p=ceph.git doc: update multiple mds scrub docs Fixes: https://tracker.ceph.com/issues/44192 Signed-off-by: Patrick Donnelly --- diff --git a/PendingReleaseNotes b/PendingReleaseNotes index 4d4513e99b38..3de345e096e8 100644 --- a/PendingReleaseNotes +++ b/PendingReleaseNotes @@ -201,7 +201,7 @@ ceph daemon scrub_path '~mdsdir' force recursive repair * CephFS: Scrub is supported in multiple active mds setup. MDS rank 0 handles - scrub commands, and forward scrub to other mds if nessesary. + scrub commands, and forward scrub to other mds if necessary. * The following librados API calls have changed: diff --git a/doc/cephfs/scrub.rst b/doc/cephfs/scrub.rst index f3af4f21a00e..7f55df950307 100644 --- a/doc/cephfs/scrub.rst +++ b/doc/cephfs/scrub.rst @@ -17,48 +17,51 @@ via a set of scrub commands. Scrub can be classified into two parts: This document details commands to initiate and control forward scrub (referred as scrub thereafter). +.. warning:: + + CephFS forward scrubs are started and manipulated on rank 0. All scrub + commands must be directed at rank 0. + Initiate File System Scrub ========================== -To start a scrub operation for a directory tree use the following command +To start a scrub operation for a directory tree use the following command:: -:: + ceph tell mds.:0 scrub start [scrubopts] [tag] - ceph tell mds.a scrub start / recursive +where ``scrubopts`` is a comma delimited list of ``recursive``, ``force``, or +``repair`` and ``tag`` is an optional custom string tag (the default is a generated +UUID). An example command is:: + + ceph tell mds.cephfs:0 scrub start / recursive { "return_code": 0, "scrub_tag": "6f0d204c-6cfd-4300-9e02-73f382fd23c1", "mode": "asynchronous" } -Recursive scrub is asynchronous (as hinted by `mode` in the output above). Scrub tag is -a random string that can used to monitor the progress of the scrub operation (explained -further in this document). - -Custom tag can also be specified when initiating the scrub operation. Custom tags get -persisted in the metadata object for every inode in the file system tree that is being -scrubbed. +Recursive scrub is asynchronous (as hinted by `mode` in the output above). +Asynchronous scrubs must be polled using ``scrub status`` to determine the +status. -:: +The scrub tag is used to differentiate scrubs and also to mark each inode's +first data object in the default data pool (where the backtrace information is +stored) with a ``scrub_tag`` extended attribute with the value of the tag. You +can verify an inode was scrubbed by looking at the extended attribute using the +RADOS utilities. - ceph tell mds.a scrub start /a/b/c recursive tag0 - { - "return_code": 0, - "scrub_tag": "tag0", - "mode": "asynchronous" - } +Scrubs work for multiple active MDS (multiple ranks). The scrub is managed by +rank 0 and distributed across MDS as appropriate. Monitor (ongoing) File System Scrubs ==================================== -Status of ongoing scrubs can be monitored using in `scrub status` command. This commands -lists out ongoing scrubs (identified by the tag) along with the path and options used to -initiate the scrub. +Status of ongoing scrubs can be monitored and polled using in `scrub status` +command. This commands lists out ongoing scrubs (identified by the tag) along +with the path and options used to initiate the scrub:: -:: - - ceph tell mds.a scrub status + ceph tell mds.cephfs:0 scrub status { "status": "scrub active (85 inodes in the stack)", "scrubs": { @@ -72,9 +75,7 @@ initiate the scrub. `status` shows the number of inodes that are scheduled to be scrubbed at any point in time, hence, can change on subsequent `scrub status` invocations. Also, a high level summary of scrub operation (which includes the operation state and paths on which scrub is triggered) -gets displayed in `ceph status`. - -:: +gets displayed in `ceph status`:: ceph status [...] @@ -85,26 +86,26 @@ gets displayed in `ceph status`. [...] +A scrub is complete when it no longer shows up in this list (although that may +change in future releases). Any damage will be reported via cluster health warnings. + Control (ongoing) File System Scrubs ==================================== - Pause: Pausing ongoing scrub operations results in no new or pending inodes being scrubbed after in-flight RADOS ops (for the inodes that are currently being scrubbed) - finish. - -:: + finish:: - ceph tell mds.a scrub pause + ceph tell mds.cephfs:0 scrub pause { "return_code": 0 } -`scrub status` after pausing reflects the paused state. At this point, initiating new scrub -operations (via `scrub start`) would just queue the inode for scrub. + The ``scrub status`` after pausing reflects the paused state. At this point, + initiating new scrub operations (via ``scrub start``) would just queue the + inode for scrub:: -:: - - ceph tell mds.a scrub status + ceph tell mds.cephfs:0 scrub status { "status": "PAUSED (66 inodes in the stack)", "scrubs": { @@ -115,22 +116,18 @@ operations (via `scrub start`) would just queue the inode for scrub. } } -- Resume: Resuming kick starts a paused scrub operation. - -:: +- Resume: Resuming kick starts a paused scrub operation:: - ceph tell mds.a. scrub resume + ceph tell mds.cephfs:0 scrub resume { "return_code": 0 } - Abort: Aborting ongoing scrub operations removes pending inodes from the scrub queue (thereby aborting the scrub) after in-flight RADOS ops (for the inodes that - are currently being scrubbed) finish. - -:: + are currently being scrubbed) finish:: - ceph tell mds.a. scrub abort + ceph tell mds.cephfs:0 scrub abort { "return_code": 0 }