common: drop stack singleton object of temp messenger for foreground ceph daemons

During the initialization, OSD needs to create a temporary messenger to read config db from the ceph-mon.
This temporary messenger will need to create async messenger threads according to the local/default value of "ms_async_op_threads" .
If this option is not specified in ceph.conf, it will by default create 3 threads, then use these threads to read config db from ceph-mon.
Those threads are associated to a stack singletion object.
Now here is the difference between OSD running in foreground and background:
a. In background mode, this singleton object will be dropped before forking the child process in function notify_pre_fork,
then the new ms_async_op_threads fetched from mon config db will be used to create later messenger threads, this is what we want.
b. In foreground mode, this singleton object will not be dropped and will be reused by all later messengers,
thus the number of threads doesn't change.

Fixes: https://tracker.ceph.com/issues/71401
Signed-off-by: dongdong tao <dongdong.tao@canonical.com>
(cherry picked from commit 30d66ff075ca72c0c3759bfccee09302b221b25f)

diff --git a/src/common/ceph_context.cc b/src/common/ceph_context.cc
index 7f92211455fa5d22ff999e21c8122105810f1884..17b658780ee32d8a6a12d91a57511262b122825f 100644 (file)
--- a/src/common/ceph_context.cc
+++ b/src/common/ceph_context.cc
@@ -1036,6 +1036,19 @@ CryptoHandler *CephContext::get_crypto_handler(int type)
   }
 }
 
+void CephContext::drop_temp_messenger_obj()
+{
+  auto i = associated_objs.begin();
+  while (i != associated_objs.end()) {
+    if (i->first.first.find("AsyncMessenger::NetworkStack") != std::string::npos) {
+      i = associated_objs.erase(i);
+      break;
+    } else {
+      ++i;
+    }
+  }
+}
+
 void CephContext::notify_pre_fork()
 {
   {

diff --git a/src/common/ceph_context.h b/src/common/ceph_context.h
index 9ae7b63c98168d1cfd21f23b627c7311cbd0df99..6a3d510ac30d4c848bff5174f32ba35f64300659 100644 (file)
--- a/src/common/ceph_context.h
+++ b/src/common/ceph_context.h
@@ -271,6 +271,7 @@ public:
     _fork_watchers.push_back(w);
   }
 
+  void drop_temp_messenger_obj();
   void notify_pre_fork();
   void notify_post_fork();
 

diff --git a/src/global/global_init.cc b/src/global/global_init.cc
index 79defaec3761c019790e2137206976a88e958a98..3f92b02170e2b0d01cb72a98c90cb9ef3d9c08c5 100644 (file)
--- a/src/global/global_init.cc
+++ b/src/global/global_init.cc
@@ -469,7 +469,7 @@ int global_init_prefork(CephContext *cct)
       chown_path(conf->pid_file, cct->get_set_uid(), cct->get_set_gid(),
                 cct->get_set_uid_string(), cct->get_set_gid_string());
     }
-
+    cct->drop_temp_messenger_obj();
     return -1;
   }