neocls log trimming (marker based): fix infinite loop on ENODATA

The use_awaitable_t overload of trim() has the try-catch for
ENODATA (no_message_available) inside the for(;;) loop. When
cls_log_trim returns ENODATA (i.e., nothing left to trim), the exception is
caught and silently swallowed execution falls through the catch block
back to for(;;), retrying the trim forever.

This should be a rare condition as 3 conditions should be met in a
single-cluster (once configured as multisite):
 - a realm/period exists
 - zone endpoints are configured
 - data_log* objects exist

Since in a properly setup multisite cluster, data churn is continious so
hard to notice. In the case client reported, the multisite cluster was
reverted back to single site so data_logs have no data all the time;
hence, the issue is pronounced.

This fix adds co_return inside the catch block so ENODATA exits the loop.

Fixes: https://tracker.ceph.com/issues/76563
Signed-off-by: Oguzhan Ozmen <oozmen@bloomberg.net>
(cherry picked from commit e1a6ed169dd72868eba5321b52b180fdb057b314)

diff --git a/src/neorados/cls/log.h b/src/neorados/cls/log.h
index c974aab1484a61eedfad343969179e4146bc7afe..d5c823dac57d9b4e38da7181f8f5bf6b7d085e46 100644 (file)
--- a/src/neorados/cls/log.h
+++ b/src/neorados/cls/log.h
@@ -462,8 +462,8 @@ trim(RADOS& r, Object oid, IOContext ioc,
       if (e.code() != no_message_available) {
        throw;
       }
+      co_return;
     }
-  co_return;
 }
 
 /// \brief Trim entries from the log

diff --git a/src/test/cls_log/test_neocls_log.cc b/src/test/cls_log/test_neocls_log.cc
index 87f06643cc891842a66a136f5746dfa11d93908c..2e2e5d0175eb98b15b740547a0c4ca3fc5a641be 100644 (file)
--- a/src/test/cls_log/test_neocls_log.cc
+++ b/src/test/cls_log/test_neocls_log.cc
@@ -473,9 +473,7 @@ CORO_TEST_F(neocls_log, trim_by_marker, NeoRadosTest)
 }
 
 // Test the neorados::trim() loop function (use_awaitable overloads) to verify
-// it terminates. Before any fix in neorados/cls/log.h, the use_awaitable_t
-// overloads had the try-catch for ENODATA inside the for(;;) loop, causing
-// the loop to never exit in certain cluster configs and this CPU hogging.
+// it terminates.
 CORO_TEST_F(neocls_log, trim_loop_all_entries_by_marker, NeoRadosTest)
 {
   co_await create_obj(oid);
@@ -485,8 +483,7 @@ CORO_TEST_F(neocls_log, trim_loop_all_entries_by_marker, NeoRadosTest)
   co_await generate_log(rados(), oid, pool(), 10, start_time, true,
                        asio::use_awaitable);
 
-  // call trim() loop function. Without a fix, this never returns
-  // because of where the try-catch sits relative to the for(;;)
+  // call trim() loop function.
   co_await neorados::cls::log::trim(
     rados(), oid, pool(),
     std::string_view{neorados::cls::log::begin_marker},