From: Kefu Chai <kchai@redhat.com>
Date: Fri, 15 Nov 2019 09:55:53 +0000 (+0800)
Subject: script/run-cbt.sh: set fs.aio-max-nr for seastar
X-Git-Tag: v15.1.0~889^2
X-Git-Url: http://git-server-git.apps.pok.os.sepia.ceph.com/?a=commitdiff_plain;h=refs%2Fpull%2F31667%2Fhead;p=ceph.git

script/run-cbt.sh: set fs.aio-max-nr for seastar

seastar requires 11027 for each reactor, so we need at least

fs.aio-max-nr = 11027 * n_osd

but to make it simple and to take the needs of other applications using
aio on the machine, we increase the number of io context to 32768,
and always set this setting no matter if we are starting a vstart
cluster with classic osd or crimson osd

Signed-off-by: Kefu Chai <kchai@redhat.com>
---

diff --git a/src/script/run-cbt.sh b/src/script/run-cbt.sh
index 9caddcc4c956..9205fee24f9b 100755
--- a/src/script/run-cbt.sh
+++ b/src/script/run-cbt.sh
@@ -87,6 +87,14 @@ fi
 source_dir=$(readlink -f $source_dir)
 if ! $use_existing; then
     cd $build_dir
+    # seastar uses 128*8 aio in reactor for io and 10003 aio for events pooling
+    # for each core, if it fails to enough aio context, the seastar application
+    # bails out. and take other process into consideration, let's make it
+    # 32768 per core
+    max_io=$(expr 32768 \* $(nproc))
+    if test $(/sbin/sysctl --values fs.aio-max-nr) -lt $max_io; then
+        sudo /sbin/sysctl -q -w fs.aio-max-nr=$max_io
+    fi
     if $classical; then
         MDS=0 MGR=1 OSD=3 MON=1 $source_dir/src/vstart.sh -n -X \
            --without-dashboard --memstore \