osd: shutdown our reserver_finisher earlier in the process

This finisher thread has a lot of callbacks which can hold PGRefs. Make
sure we drain them out before checking that all the PGs have finished
and have no outstanding references.

Moving this should be safe; we've already stopped the op thread et al
and the only things still running are the OSDService's objecter_finisher,
recovery_request_timer, and snap_sleep_timer (which has definitely been emptied
by the time we get here as it's synchronously cleared out on PG shutdown).

Signed-off-by: Greg Farnum <gfarnum@redhat.com>
(cherry picked from commit 66ea9c1f66ae31035e62bd4335b08948b1e8e5e2)

diff --git a/src/osd/OSD.cc b/src/osd/OSD.cc
index 25f7307959c7526a7d28cbe177cc4de5b5c521d1..7ac7396686b20af698c129bdcb20ead47edbc1d0 100644 (file)
--- a/src/osd/OSD.cc
+++ b/src/osd/OSD.cc
@@ -510,10 +510,14 @@ void OSDService::start_shutdown()
   }
 }
 
-void OSDService::shutdown()
+void OSDService::shutdown_reserver()
 {
   reserver_finisher.wait_for_empty();
   reserver_finisher.stop();
+}
+
+void OSDService::shutdown()
+{
   {
     Mutex::Locker l(watch_lock);
     watch_timer.shutdown();
@@ -2849,6 +2853,8 @@ int OSD::shutdown()
     assert(pg_stat_queue.empty());
   }
 
+  service.shutdown_reserver();
+
   // Remove PGs
 #ifdef PG_DEBUG_REFS
   service.dump_live_pgids();

diff --git a/src/osd/OSD.h b/src/osd/OSD.h
index 8fa969d3a3e5a2e4d69bf1cd3ae41b35195c4365..3e800846abae7ce64900eb8ce05a4548b7f36791 100644 (file)
--- a/src/osd/OSD.h
+++ b/src/osd/OSD.h
@@ -1039,6 +1039,7 @@ public:
   void init();
   void final_init();  
   void start_shutdown();
+  void shutdown_reserver();
   void shutdown();
 
 private: