mds: update default MDS cache size memory limit

1GB is too low as a default and usually results in cache size warnings
at that size; the MDS will struggle to maintain such a small cache size
for most workloads.

Fixes: https://tracker.ceph.com/issues/43182
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

diff --git a/PendingReleaseNotes b/PendingReleaseNotes
index 0a0bde3c9957b5af515c7558294df49118244bb2..1579ea026a8dcdb18d466fb57e799d68a6e4bdfc 100644 (file)
--- a/PendingReleaseNotes
+++ b/PendingReleaseNotes
@@ -252,6 +252,8 @@
   cases -- reducing the number of entries requested from each bucket
   index shard.
 
+* MDS default cache memory limit is now 4GB.
+
 * The behaviour of the ``-o`` argument to the rados tool has been reverted to
   its orignal behaviour of indicating an output file. This reverts it to a more
   consisten behaviour when compared to other tools. Specifying obect size is now

diff --git a/doc/cephfs/add-remove-mds.rst b/doc/cephfs/add-remove-mds.rst
index c695fbbb5c2d1632a01a491ff3b8c6c1b8f2b529..275e7ab932d7b4de5db1c5285e54648ad607f35e 100644 (file)
--- a/doc/cephfs/add-remove-mds.rst
+++ b/doc/cephfs/add-remove-mds.rst
@@ -15,9 +15,9 @@ Provisioning Hardware for an MDS
 ================================
 
 The present version of the MDS is single-threaded and CPU-bound for most
-activities, including responding to client requests. Even so, an MDS under the
-most aggressive client loads still uses about 2 to 3 CPU cores. This is due to
-the other miscellaneous upkeep threads working in tandem.
+activities, including responding to client requests. An MDS under the most
+aggressive client loads uses about 2 to 3 CPU cores. This is due to the other
+miscellaneous upkeep threads working in tandem.
 
 Even so, it is recommended that an MDS server be well provisioned with an
 advanced CPU with sufficient cores. Development is on-going to make better use
@@ -27,14 +27,17 @@ that the MDS server will improve performance by taking advantage of more cores.
 The other dimension to MDS performance is the available RAM for caching. The
 MDS necessarily manages a distributed and cooperative metadata cache among all
 clients and other active MDSs. Therefore it is essential to provide the MDS
-with sufficient RAM to enable faster metadata access and mutation.
+with sufficient RAM to enable faster metadata access and mutation. The default
+MDS cache size (see also :doc:`/cephfs/cache-size-limits`) is 4GB. It is
+recommended to provision at least 8GB of RAM for the MDS to support this cache
+size.
 
 Generally, an MDS serving a large cluster of clients (1000 or more) will use at
-least 64GB of cache (see also :doc:`/cephfs/cache-size-limits`). An MDS with a larger
-cache is not well explored in the largest known community clusters; there may
-be diminishing returns where management of such a large cache negatively
-impacts performance in surprising ways. It would be best to do analysis with
-expected workloads to determine if provisioning more RAM is worthwhile.
+least 64GB of cache. An MDS with a larger cache is not well explored in the
+largest known community clusters; there may be diminishing returns where
+management of such a large cache negatively impacts performance in surprising
+ways. It would be best to do analysis with expected workloads to determine if
+provisioning more RAM is worthwhile.
 
 In a bare-metal cluster, the best practice is to over-provision hardware for
 the MDS server. Even if a single MDS daemon is unable to fully utilize the

diff --git a/doc/cephfs/mds-config-ref.rst b/doc/cephfs/mds-config-ref.rst
index 248368c173582687fa79b02a4c88605e7b0f535c..83caf795b8977016361022def5dcec60c941a70c 100644 (file)
--- a/doc/cephfs/mds-config-ref.rst
+++ b/doc/cephfs/mds-config-ref.rst
@@ -6,7 +6,7 @@
 
 :Description: The memory limit the MDS should enforce for its cache.
 :Type:  64-bit Integer Unsigned
-:Default: ``1G``
+:Default: ``4G``
 
 ``mds cache reservation``
 

diff --git a/src/common/options.cc b/src/common/options.cc
index 51551314ee525f2c40768514beb7a1255f95ce07..59865d9380062063138b85cc5c842847b0399bb2 100644 (file)
--- a/src/common/options.cc
+++ b/src/common/options.cc
@@ -7558,7 +7558,7 @@ std::vector<Option> get_mds_options() {
     .set_flag(Option::FLAG_RUNTIME),
 
     Option("mds_cache_memory_limit", Option::TYPE_SIZE, Option::LEVEL_BASIC)
-    .set_default(1*(1LL<<30))
+    .set_default(4_G)
     .set_description("target maximum memory usage of MDS cache")
     .set_flag(Option::FLAG_RUNTIME)
     .set_long_description("This sets a target maximum memory usage of the MDS cache and is the primary tunable to limit the MDS memory usage. The MDS will try to stay under a reservation of this limit (by default 95%; 1 - mds_cache_reservation) by trimming unused metadata in its cache and recalling cached items in the client caches. It is possible for the MDS to exceed this limit due to slow recall from clients. The mds_health_cache_threshold (150%) sets a cache full threshold for when the MDS signals a cluster health warning."),