From: chunmei <chunmei.liu@intel.com>
Date: Thu, 2 Nov 2017 03:37:53 +0000 (-0700)
Subject: ms_dpdk_coremask and ms_async_op_threads has some relationship,
X-Git-Tag: v13.0.1~304^2
X-Git-Url: http://git-server-git.apps.pok.os.sepia.ceph.com/?a=commitdiff_plain;h=refs%2Fpull%2F18678%2Fhead;p=ceph.git

ms_dpdk_coremask and ms_async_op_threads has some relationship,
dpdk library will create threasds as many as bit 1 number in
ms_dpdk_coremask and the thredas num should be equal ms_async_op_threads+1,
by ceph default confiuration,ms_async_op_threads =3,ms_dpdk_coremask =1,
so dpdk library can't create enough threads for dpdk workers. the sytem
will stuck at NetworkStack::start().so set ms_dpdk_coremask default 0xF.
and add assert to comapre the two configurations.

Signed-off-by: chunmei <chunmei.liu@intel.com>
---

diff --git a/src/common/options.cc b/src/common/options.cc
index 89005ae35890..f3ff00ec8ab5 100644
--- a/src/common/options.cc
+++ b/src/common/options.cc
@@ -821,9 +821,10 @@ std::vector<Option> get_global_options() {
     .set_description(""),
 
     Option("ms_dpdk_coremask", Option::TYPE_STR, Option::LEVEL_ADVANCED)
-    .set_default("1")
+    .set_default("0xF")         //begin with 0x for the string
     .set_description("")
-    .set_safe(),
+    .set_safe()
+    .add_see_also("ms_async_op_threads"),
 
     Option("ms_dpdk_memory_channel", Option::TYPE_STR, Option::LEVEL_ADVANCED)
     .set_default("4")
diff --git a/src/msg/async/dpdk/dpdk_rte.cc b/src/msg/async/dpdk/dpdk_rte.cc
index 6510df103b20..abaa3d2e212c 100644
--- a/src/msg/async/dpdk/dpdk_rte.cc
+++ b/src/msg/async/dpdk/dpdk_rte.cc
@@ -53,6 +53,12 @@ namespace dpdk {
     }
 
     bool done = false;
+    const char *hexstring = c->_conf->get_val<std::string>("ms_dpdk_coremaskbit").c_str();
+    int num = (int)strtol(hexstring, NULL, 0);
+    unsigned int coremaskbit = bitcount(num);
+
+    ceph_assert(coremaskbit > c->_conf->ms_async_op_threads);
+
     t = std::thread([&]() {
       // TODO: Inherit these from the app parameters - "opts"
       std::vector<std::vector<char>> args {