check: try to insulate the test framework from oom killer
authorDarrick J. Wong <darrick.wong@oracle.com>
Tue, 4 Jun 2019 21:16:49 +0000 (14:16 -0700)
committerEryu Guan <guaneryu@gmail.com>
Fri, 7 Jun 2019 12:29:10 +0000 (20:29 +0800)
Some of the tests in xfstests (e.g. generic/224 with 512M of memory)
consume a lot of memory, and when this happens the OOM killer will
run around stomping on processes.  Sometimes it kills the ./check
process before it kills the actual test, which means that the test
run doesn't complete.

Therefore, make the ./check process OOM-proof while bumping up the
attractiveness of the test itself, in the hopes that even if the
test OOMs we'll still be able to continue on our way.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
check

diff --git a/check b/check
index 8677141b1d0f1a3e959e7b0fb933a1d21b490bed..08419811ad618d375bdf463412110fb0d672ee06 100755 (executable)
--- a/check
+++ b/check
@@ -499,6 +499,17 @@ _expunge_test()
        return 0
 }
 
+# Make the check script unattractive to the OOM killer...
+OOM_SCORE_ADJ="/proc/self/oom_score_adj"
+test -w ${OOM_SCORE_ADJ} && echo -1000 > ${OOM_SCORE_ADJ}
+
+# ...and make the tests themselves somewhat more attractive to it, so that if
+# the system runs out of memory it'll be the test that gets killed and not the
+# test framework.
+_run_seq() {
+       bash -c "test -w ${OOM_SCORE_ADJ} && echo 250 > ${OOM_SCORE_ADJ}; exec ./$seq"
+}
+
 _detect_kmemleak
 _prepare_test_list
 
@@ -740,11 +751,11 @@ for section in $HOST_OPTIONS_SECTIONS; do
                fi
                _try_wipe_scratch_devs > /dev/null 2>&1
                if [ "$DUMP_OUTPUT" = true ]; then
-                       ./$seq 2>&1 | tee $tmp.out
+                       _run_seq 2>&1 | tee $tmp.out
                        # Because $? would get tee's return code
                        sts=${PIPESTATUS[0]}
                else
-                       ./$seq >$tmp.out 2>&1
+                       _run_seq >$tmp.out 2>&1
                        sts=$?
                fi