From: luo rixin <luorixin@huawei.com>
Date: Wed, 7 Feb 2024 03:21:50 +0000 (+0800)
Subject: cmake/AddCephTest: Specify resoureces to crimson unittest
X-Git-Tag: v20.0.0~2561^2~2
X-Git-Url: http://git-server-git.apps.pok.os.sepia.ceph.com/?a=commitdiff_plain;h=a1f9b59c1ad4106126f816a3ef4fdfae5df555bb;p=ceph.git

cmake/AddCephTest: Specify resoureces to crimson unittest

When running crimson unittest, the seastar framework always
use and only use cpu0, and with many parallel crimson unittest
jobs, all the jobs are running on cpu0, the other cpu cores
can't used, make the make check run very slow, even timeout
happens. Use set_property RESOURCE_GROUPS to specify cpu resources
to crimson unittest, and accelerate make check running.

Fixes: https://tracker.ceph.com/issues/64117

Co-authored-by: Kefu Chai <tchaikov@gmail.com>
Signed-off-by: luo rixin <luorixin@huawei.com>
---

diff --git a/cmake/modules/AddCephTest.cmake b/cmake/modules/AddCephTest.cmake
index 591552f834ea..ccd3f8dee0b5 100644
--- a/cmake/modules/AddCephTest.cmake
+++ b/cmake/modules/AddCephTest.cmake
@@ -21,6 +21,25 @@ function(add_ceph_test test_name test_path)
     CEPH_BUILD_VIRTUALENV=${CEPH_BUILD_VIRTUALENV})
   set_property(TEST ${test_name}
     PROPERTY TIMEOUT ${CEPH_TEST_TIMEOUT})
+  # Crimson seastar unittest always run with --smp N to start N threads. By default, crimson seastar unittest
+  # will take cpu cores[0, N), starting one thread per core. When running many crimson seastar unittests
+  # parallely, the front N cpu cores are shared, and the left cpu cores are idle. Lots of cpu cores are wasted.
+  # Using CTest resource allocation feature(https://cmake.org/cmake/help/latest/manual/ctest.1.html#resource-allocation),
+  # ctest can specify cpu cores resources to crimson seastar unittests.
+  # 3 steps to enable CTest resource allocation feature:
+  #  Step 1: Generate a resource specification file to describe available resource, $(nproc) CPUs with id 0 to $(nproc) - 1
+  #  Step 2: Set RESOURCE_GROUPS property to a test with value "${smp_count},cpus:1"
+  #  Step 3: Read a series of environment variables CTEST_RESOURCE_GROUP_* and set seastar smp_opts while running a test
+  list(FIND ARGV "--smp" smp_pos)
+  if(smp_pos GREATER -1)
+    if(smp_pos EQUAL ARGC)
+      message(FATAL_ERROR "${test_name} --smp requires an argument")
+    endif()
+    math(EXPR i "${smp_pos} + 1")
+    list(GET ARGV ${i} smp_count)
+    set_property(TEST ${test_name}
+      PROPERTY RESOURCE_GROUPS "${smp_count},cpus:1")
+  endif()
 endfunction()
 
 option(WITH_GTEST_PARALLEL "Enable running gtest based tests in parallel" OFF)