From: Patrick Donnelly Date: Thu, 5 Dec 2019 18:36:12 +0000 (-0800) Subject: mds: update default MDS cache size memory limit X-Git-Tag: v15.1.0~477^2 X-Git-Url: http://git-server-git.apps.pok.os.sepia.ceph.com/?a=commitdiff_plain;h=0efa73f0296276aadc9d20e58120c05a3f47ae03;p=ceph.git mds: update default MDS cache size memory limit 1GB is too low as a default and usually results in cache size warnings at that size; the MDS will struggle to maintain such a small cache size for most workloads. Fixes: https://tracker.ceph.com/issues/43182 Signed-off-by: Patrick Donnelly --- diff --git a/PendingReleaseNotes b/PendingReleaseNotes index 0a0bde3c9957..1579ea026a8d 100644 --- a/PendingReleaseNotes +++ b/PendingReleaseNotes @@ -252,6 +252,8 @@ cases -- reducing the number of entries requested from each bucket index shard. +* MDS default cache memory limit is now 4GB. + * The behaviour of the ``-o`` argument to the rados tool has been reverted to its orignal behaviour of indicating an output file. This reverts it to a more consisten behaviour when compared to other tools. Specifying obect size is now diff --git a/doc/cephfs/add-remove-mds.rst b/doc/cephfs/add-remove-mds.rst index c695fbbb5c2d..275e7ab932d7 100644 --- a/doc/cephfs/add-remove-mds.rst +++ b/doc/cephfs/add-remove-mds.rst @@ -15,9 +15,9 @@ Provisioning Hardware for an MDS ================================ The present version of the MDS is single-threaded and CPU-bound for most -activities, including responding to client requests. Even so, an MDS under the -most aggressive client loads still uses about 2 to 3 CPU cores. This is due to -the other miscellaneous upkeep threads working in tandem. +activities, including responding to client requests. An MDS under the most +aggressive client loads uses about 2 to 3 CPU cores. This is due to the other +miscellaneous upkeep threads working in tandem. Even so, it is recommended that an MDS server be well provisioned with an advanced CPU with sufficient cores. Development is on-going to make better use @@ -27,14 +27,17 @@ that the MDS server will improve performance by taking advantage of more cores. The other dimension to MDS performance is the available RAM for caching. The MDS necessarily manages a distributed and cooperative metadata cache among all clients and other active MDSs. Therefore it is essential to provide the MDS -with sufficient RAM to enable faster metadata access and mutation. +with sufficient RAM to enable faster metadata access and mutation. The default +MDS cache size (see also :doc:`/cephfs/cache-size-limits`) is 4GB. It is +recommended to provision at least 8GB of RAM for the MDS to support this cache +size. Generally, an MDS serving a large cluster of clients (1000 or more) will use at -least 64GB of cache (see also :doc:`/cephfs/cache-size-limits`). An MDS with a larger -cache is not well explored in the largest known community clusters; there may -be diminishing returns where management of such a large cache negatively -impacts performance in surprising ways. It would be best to do analysis with -expected workloads to determine if provisioning more RAM is worthwhile. +least 64GB of cache. An MDS with a larger cache is not well explored in the +largest known community clusters; there may be diminishing returns where +management of such a large cache negatively impacts performance in surprising +ways. It would be best to do analysis with expected workloads to determine if +provisioning more RAM is worthwhile. In a bare-metal cluster, the best practice is to over-provision hardware for the MDS server. Even if a single MDS daemon is unable to fully utilize the diff --git a/doc/cephfs/mds-config-ref.rst b/doc/cephfs/mds-config-ref.rst index 248368c17358..83caf795b897 100644 --- a/doc/cephfs/mds-config-ref.rst +++ b/doc/cephfs/mds-config-ref.rst @@ -6,7 +6,7 @@ :Description: The memory limit the MDS should enforce for its cache. :Type: 64-bit Integer Unsigned -:Default: ``1G`` +:Default: ``4G`` ``mds cache reservation`` diff --git a/src/common/options.cc b/src/common/options.cc index 51551314ee52..59865d938006 100644 --- a/src/common/options.cc +++ b/src/common/options.cc @@ -7558,7 +7558,7 @@ std::vector