ceph_test_librados_tier: tolerage EAGAIN from pg scrub command

We may get EAGAIN if the osd happens to be down, for example due to
thrashing.  Try a few times and then give up.

Note that the other place we try to scrub we don't even check the return
value as we are poking ever pg in the pool.  And the scrub commands get
lost due to any peering event, etc.

Signed-off-by: Sage Weil <sage@inktank.com>

diff --git a/src/test/librados/tier.cc b/src/test/librados/tier.cc
index ea3b4bf844136e4fc4a9f107c2c5f158310c6212..28e8e7aaa4e91df6578a6973e4e13aa2c8517bd6 100644 (file)
--- a/src/test/librados/tier.cc
+++ b/src/test/librados/tier.cc
@@ -2490,15 +2490,20 @@ TEST_F(LibRadosTwoPoolsECPP, PromoteSnap) {
   // clones in the cache tier)
   // This test requires cache tier and base tier to have the same pg_num/pgp_num
   {
-    IoCtx cache_ioctx;
-    ASSERT_EQ(0, cluster.ioctx_create(cache_pool_name.c_str(), cache_ioctx));
-    ostringstream ss;
-    ss << "{\"prefix\": \"pg scrub\", \"pgid\": \""
-       << cache_ioctx.get_id() << "."
-       << ioctx.get_object_pg_hash_position("foo")
-       << "\"}";
-    ASSERT_EQ(0, cluster.mon_command(ss.str(), inbl, NULL, NULL));
-
+    for (int tries = 0; tries < 5; ++tries) {
+      IoCtx cache_ioctx;
+      ASSERT_EQ(0, cluster.ioctx_create(cache_pool_name.c_str(), cache_ioctx));
+      ostringstream ss;
+      ss << "{\"prefix\": \"pg scrub\", \"pgid\": \""
+        << cache_ioctx.get_id() << "."
+        << ioctx.get_object_pg_hash_position("foo")
+        << "\"}";
+      int r = cluster.mon_command(ss.str(), inbl, NULL, NULL);
+      if (r == -EAGAIN)
+       continue;
+      ASSERT_EQ(0, r);
+      break;
+    }
     // give it a few seconds to go.  this is sloppy but is usually enough time
     cout << "waiting for scrub..." << std::endl;
     sleep(15);