- mds stop -> mds deactivate
- mds set_max_mds -> fs set max_mds
- mds set -> fs set
- - mds cluster_down -> fs set cluster_down true
- - mds cluster_up -> fs set cluster_down false
+ - mds cluster_down -> fs set joinable false
+ - mds cluster_up -> fs set joinable true
- mds add_data_pool -> fs add_data_pool
- mds remove_data_pool -> fs rm_data_pool
- mds rm_data_pool -> fs rm_data_pool
is now redundant.
* Taking a CephFS cluster down is now done by setting the down flag which
- deactivates all MDS.
+ deactivates all MDS. For example: `ceph fs set cephfs down true`.
- * Preventing standbys from joining as new actives (formerly the cluster_down
- flag) on a file system is now accomplished by setting the joinable flag.
- This is useful mostly for testing so that a file system may be quickly
- brought down and deleted.
+ * Preventing standbys from joining as new actives (formerly the now
+ deprecated cluster_down flag) on a file system is now accomplished by
+ setting the joinable flag. This is useful mostly for testing so that a
+ file system may be quickly brought down and deleted.
* New CephFS file system attributes session_timeout and session_autoclose
are configurable via `ceph fs set`. The MDS config options
Taking the cluster down
-----------------------
-Taking a CephFS cluster down is done by setting the cluster_down flag:
+
+Taking a CephFS cluster down is done by setting the down flag:
::
- mds set <fs name> cluster_down true
+ mds set <fs_name> down true
To bring the cluster back online:
::
- mds set <fs name> cluster_down false
+ mds set <fs_name> down false
+
+This will also restore the previous value of max_mds. MDS daemons are brought
+down in a way such that journals are flushed to the metadata pool and all
+client I/O is stopped.
+
+
+Taking the cluster down rapidly for deletion or disaster recovery
+-----------------------------------------------------------------
+
+To allow rapidly deleting a file system (for testing) or to quickly bring MDS
+daemons down, the operator may also set a flag to prevent standbys from
+activating on the file system. This is done using the ``joinable`` flag:
+
+::
+
+ fs set <fs_name> joinable false
+
+Then the operator can fail all of the ranks which causes the MDS daemons to
+respawn as standbys. The file system will be left in a degraded state.
+
+::
-This will restore the previous value of max_mds.
+ # For all ranks, 0-N:
+ mds fail <fs_name>:<n>
+
+Once all ranks are inactive, the file system may also be deleted or left in
+this state for other purposes (perhaps disaster recovery).
Daemons
-------
-These commands act on specific mds daemons or ranks.
+Most commands manipulating MDSs take a ``<role>`` argument which can take one
+of three forms:
+
+::
+
+ <fs_name>:<rank>
+ <fs_id>:<rank>
+ <rank>
+
+Comamnds to manipulate MDS daemons:
::
will cause the daemon to restart. If it was active and a standby was
available, then the "failed" daemon will return as a standby.
-::
-
- mds deactivate <role>
-
-Deactivate an MDS, causing it to flush its entire journal to
-backing RADOS objects and close all open client sessions. Deactivating an MDS
-is primarily intended for bringing down a rank after reducing the number of
-active MDS (max_mds). Once the rank is deactivated, the MDS daemon will rejoin the
-cluster as a standby.
-``<role>`` can take one of three forms:
::
- <fs_name>:<rank>
- <fs_id>:<rank>
- <rank>
-
-Use ``mds deactivate`` in conjunction with adjustments to ``max_mds`` to
-shrink an MDS cluster. See :doc:`/cephfs/multimds`
-
-::
+ tell mds.<daemon name> command ...
- tell mds.<daemon name>
+Send a command to the MDS daemon(s). Use ``mds.*`` to send a command to all
+daemons. Use ``ceph tell mds.* help`` to learn available commands.
::
mds remove_data_pool # replaced by "fs rm_data_pool"
mds set # replaced by "fs set"
mds set_max_mds # replaced by "fs set max_mds"
- mds stop # replaced by "mds deactivate"
+ mds stop # obsolete
Increasing the MDS active cluster size
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-Each CephFS filesystem has a *max_mds* setting, which controls
-how many ranks will be created. The actual number of ranks
-in the filesystem will only be increased if a spare daemon is
-available to take on the new rank. For example, if there is only one MDS daemon running, and max_mds is set to two, no second rank will be created.
-
-Before ``max_mds`` can be increased, the ``allow_multimds`` flag must be set.
-The following command sets this flag for a filesystem instance.
-
-::
-
- # ceph fs set <fs_name> allow_multimds true --yes-i-really-mean-it
+Each CephFS filesystem has a *max_mds* setting, which controls how many ranks
+will be created. The actual number of ranks in the filesystem will only be
+increased if a spare daemon is available to take on the new rank. For example,
+if there is only one MDS daemon running, and max_mds is set to two, no second
+rank will be created. (Note that such a configuration is not Highly Available
+(HA) because no standby is available to take over for a failed rank. The
+cluster will complain via health warnings when configured this way.)
Set ``max_mds`` to the desired number of ranks. In the following examples
the "fsmap" line of "ceph status" is shown to illustrate the expected
an active daemon fail.
Consequently, the practical maximum of ``max_mds`` for highly available systems
-is one less than the total number of MDS servers in your system.
+is at most one less than the total number of MDS servers in your system.
To remain available in the event of multiple server failures, increase the
number of standby daemons in the system to match the number of server failures
Decreasing the number of ranks
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-All ranks, including the rank(s) to be removed must first be active. This
-means that you must have at least max_mds MDS daemons available.
-
-First, set max_mds to a lower number, for example we might go back to
-having just a single active MDS:
+Reducing the number of ranks is as simple as reducing ``max_mds``:
::
# fsmap e9: 2/2/2 up {0=a=up:active,1=c=up:active}, 1 up:standby
ceph fs set <fs_name> max_mds 1
- # fsmap e10: 2/2/1 up {0=a=up:active,1=c=up:active}, 1 up:standby
-
-Note that we still have two active MDSs: the ranks still exist even though
-we have decreased max_mds, because max_mds only restricts creation
-of new ranks.
-
-Next, use the ``ceph mds deactivate <role>`` command to remove the
-unneeded rank:
-
-::
-
- ceph mds deactivate cephfs_a:1
- telling mds.1:1 172.21.9.34:6806/837679928 to deactivate
+ # fsmap e10: 2/2/1 up {0=a=up:active,1=c=up:stopping}, 1 up:standby
+ # fsmap e10: 2/2/1 up {0=a=up:active,1=c=up:stopping}, 1 up:standby
+ ...
+ # fsmap e10: 1/1/1 up {0=a=up:active}, 2 up:standby
- # fsmap e11: 2/2/1 up {0=a=up:active,1=c=up:stopping}, 1 up:standby
- # fsmap e12: 1/1/1 up {0=a=up:active}, 1 up:standby
- # fsmap e13: 1/1/1 up {0=a=up:active}, 2 up:standby
+The cluster will automatically deactivate extra ranks incrementally until
+``max_mds`` is reached.
See :doc:`/cephfs/administration` for more details which forms ``<role>`` can
take.
-The deactivated rank will first enter the stopping state for a period
-of time while it hands off its share of the metadata to the remaining
-active daemons. This phase can take from seconds to minutes. If the
-MDS appears to be stuck in the stopping state then that should be investigated
-as a possible bug.
+Note: deactivated ranks will first enter the stopping state for a period of
+time while it hands off its share of the metadata to the remaining active
+daemons. This phase can take from seconds to minutes. If the MDS appears to
+be stuck in the stopping state then that should be investigated as a possible
+bug.
-If an MDS daemon crashes or is killed while in the 'stopping' state, a
-standby will take over and the rank will go back to 'active'. You can
-try to deactivate it again once it has come back up.
+If an MDS daemon crashes or is killed while in the ``up:stopping`` state, a
+standby will take over and the cluster monitors will against try to deactivate
+the daemon.
-When a daemon finishes stopping, it will respawn itself and go
-back to being a standby.
+When a daemon finishes stopping, it will respawn itself and go back to being a
+standby.
Manually pinning directory trees to a particular rank
ceph fs set <fs_name> max_mds 1
-2. Deactivate all non-zero ranks, from the highest rank to the lowest, while waiting for each MDS to finish stopping:
+2. Wait for cluster to deactivate non-zero ranks where only rank 0 is active and the rest are standbys.
::
- ceph mds deactivate <fs_name>:<n>
ceph status # wait for MDS to finish stopping
3. Take all standbys offline, e.g. using systemctl:
::
systemctl stop ceph-mds.target
- ceph status # confirm only one MDS is online and is active
-4. Upgrade the single active MDS, e.g. using systemctl:
+4. Confirm only one MDS is online and is rank 0 for your FS:
::
+ ceph status
+
+5. Upgrade the single active MDS, e.g. using systemctl:
+
+::
+
+ # use package manager to update cluster
systemctl restart ceph-mds.target
-5. Upgrade/start the standby daemons.
+6. Upgrade/start the standby daemons.
+
+::
+
+ # use package manager to update cluster
+ systemctl restart ceph-mds.target
-6. Restore the previous max_mds for your cluster:
+7. Restore the previous max_mds for your cluster:
::
projects / ceph.git / commitdiff