osd: wait for handle_osd_map transaction ondisk without doing a full sync

Doing a full sync (forcing a btrfs transaction etc) was just wrong here.
All we (might) care about is whether our Objectstore::Transaction is
stable (in journal or fs) or not.

We are still waiting for those operations to flush to the fs (to be
readable).  That may not be necessary either, but shouldn't have a big
performance impact.

Signed-off-by: Sage Weil <sage.weil@dreamhost.com>

diff --git a/src/osd/OSD.cc b/src/osd/OSD.cc
index a8845f1a3d5193c678bbfefd7811abe0e7cd2892..f84e83509e359e627ecf8f58308af1664bc16812 100644 (file)
--- a/src/osd/OSD.cc
+++ b/src/osd/OSD.cc
@@ -2963,6 +2963,12 @@ void OSD::handle_osd_map(MOSDMap *m)
     superblock.clean_thru = osdmap->get_epoch();
   }
 
+  // completion
+  Mutex ulock("OSD::handle_osd_map() ulock");
+  Cond ucond;
+  bool udone;
+  fin->add(new C_SafeCond(&ulock, &ucond, &udone));
+
   // superblock and commit
   write_superblock(t);
   int r = store->apply_transaction(t, fin);
@@ -2976,8 +2982,22 @@ void OSD::handle_osd_map(MOSDMap *m)
 
   map_lock.put_write();
 
+  /*
+   * wait for this to be stable.
+   *
+   * NOTE: This is almost certainly overkill.  The important things
+   * that need to be stable to make the peering/recovery stuff correct
+   * include:
+   *
+   *   - last_epoch_started, since that bounds how far back in time we
+   *     check old peers
+   *   - ???
+   */
   osd_lock.Unlock();
-  store->sync_and_flush();
+  ulock.Lock();
+  while (!udone)
+    ucond.Wait(ulock);
+  ulock.Unlock();
   osd_lock.Lock();
 
   // everything through current epoch now on disk; keep anything after