Fix issue where it is possible for stats to be recovered incorrectly during merge...

To hit the problem, after you take a snapshot, you:
* Perform a write
* Perform a partial write that only involves the primary
* Perform a partial write that only involves a non-primary
* Primary goes down
* Primary comes up
* Primary goes through peering and chooses a non-primary shard as its peering partner

The result of these operations is the stats reporting a size difference equal to the partial write that only involves the primary, as the non-primary is not aware of the clone operation by design and so that is missing update is copied to the osd. This commit prevents it by invalidating the stats in the case where this happens. There will be a future commit to further narrow the set of cases where stats invalidations can happen.

Signed-off-by: Jon Bailey <jonathan.bailey1@ibm.com>

diff --git a/src/osd/PeeringState.cc b/src/osd/PeeringState.cc
index b6a7ceeec726204c1bf7339701884e04408d01d7..8631043e2510a41c20577bcb8091eb11cf30080c 100644 (file)
--- a/src/osd/PeeringState.cc
+++ b/src/osd/PeeringState.cc
@@ -3499,6 +3499,10 @@ void PeeringState::proc_master_log(
   // log to be authoritative (i.e., their entries are by definitely
   // non-divergent).
   merge_log(t, oinfo, std::move(olog), from);
+  if (info.last_backfill.is_max() &&
+      pool.info.is_nonprimary_shard(from.shard)) {
+    invalidate_stats = true;
+  }
   info.stats.stats_invalid |= invalidate_stats;
   peer_info[from] = oinfo;
   psdout(10) << " peer osd." << from << " now " << oinfo