osd/PeeringState: separate history's pruub from pg's

(pruub = prior_readable_until_ub [upper bound])

During peering, a primary may conclude that it does not need to wait for
the prior interval(s)' read lease because it will query all such osds.
However, it is dangerous to reflect that local inference about future
peering effects in the info.history, which is freely shared with other
OSDs.  For example, if the primary cleared the history pruub, shared it,
and then failed, the next primary may conclude that it does not need to
wait for the lease to expire.

Instead, track the pruub in the conventional way.  Only at the end of
peering do we clear it (if there are no prior_interval_down_osds) before
ending our final activate infos out, when we have already talked to the
peer osds and they know the prior interval has finished.

Fixes: https://tracker.ceph.com/issues/53326
Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit 96d6bf22a5dbf14da1de0ee6128585b9dd9d60d8)

diff --git a/src/osd/PeeringState.cc b/src/osd/PeeringState.cc
index 87ee246fce83dca3b7bf0955bb828203240fc5c4..6e38ef712882ab780119ef02dc73b9a93cf28d6f 100644 (file)
--- a/src/osd/PeeringState.cc
+++ b/src/osd/PeeringState.cc
@@ -2422,6 +2422,11 @@ void PeeringState::activate(
     info.purged_snaps.swap(purged);
 
     // start up replicas
+    if (prior_readable_down_osds.empty()) {
+      dout(10) << __func__ << " no prior_readable_down_osds to wait on, clearing ub"
+              << dendl;
+      clear_prior_readable_until_ub();
+    }
     info.history.refresh_prior_readable_until_ub(pl->get_mnow(),
                                                 prior_readable_until_ub);
 
@@ -6371,10 +6376,10 @@ PeeringState::GetInfo::GetInfo(my_context ctx)
 
   prior_set = ps->build_prior();
   ps->prior_readable_down_osds = prior_set.down;
+
   if (ps->prior_readable_down_osds.empty()) {
-    psdout(10) << " no prior_set down osds, clearing prior_readable_until_ub"
+    psdout(10) << " no prior_set down osds, will clear prior_readable_until_ub before activating"
               << dendl;
-    ps->clear_prior_readable_until_ub();
   }
 
   ps->reset_min_peer_features();

diff --git a/src/osd/PeeringState.h b/src/osd/PeeringState.h
index 68f842104b03ab5a562d5f288eb90e141144189c..f6e06a15eb3f95be1eda745f31f7d636d3722383 100644 (file)
--- a/src/osd/PeeringState.h
+++ b/src/osd/PeeringState.h
@@ -1992,6 +1992,7 @@ public:
   void clear_prior_readable_until_ub() {
     prior_readable_until_ub = ceph::signedspan::zero();
     prior_readable_down_osds.clear();
+    info.history.prior_readable_until_ub = ceph::signedspan::zero();
   }
 
   void renew_lease(ceph::signedspan now) {