rgw: RGWSI_Notify drains the finisher before deleting RGWWatchers

the `C_ReinitWatch` context stores a pointer to `RGWWatcher *watcher`

`RGWSI_Notify` schedules `C_ReinitWatch` callbacks to run on this
finisher thread, which can race with the destruction of those
`RGWWatchers` in `RGWSI_Notify::finalize_watch()`

to avoid this race, delay the `RGWWatcher` deletions until after we've
drained/stopped the Finisher thread in `RGWSI_Notify::shutdown()`

Fixes: https://tracker.ceph.com/issues/73361
Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit 732a9375f9e03d1ca83b99fb50631c84de76921f)

diff --git a/src/rgw/services/svc_notify.cc b/src/rgw/services/svc_notify.cc
index e037b56b848962293bbfe8d2851f75270d0bcd80..dc7d729ada56c63cf3b63d5c206d5f902624ac13 100644 (file)
--- a/src/rgw/services/svc_notify.cc
+++ b/src/rgw/services/svc_notify.cc
@@ -237,8 +237,6 @@ void RGWSI_Notify::finalize_watch(boost::asio::yield_context yield)
         });
   }
   throttle.wait();
-
-  watchers.clear();
 }
 
 int RGWSI_Notify::do_start(optional_yield y, const DoutPrefixProvider *dpp)
@@ -303,9 +301,13 @@ void RGWSI_Notify::shutdown()
       });
   context.run();
 
+  // wait for any racing C_ReinitWatch calls on the finisher thread
+  // before destroying the RGWWatchers
   finisher.wait_for_empty();
   finisher.stop();
 
+  watchers.clear();
+
   finalized = true;
 }