From 8a395f258621c95460cd6776acfd60adcde8b441 Mon Sep 17 00:00:00 2001 From: xie xingguo Date: Sat, 21 Apr 2018 10:33:49 +0800 Subject: [PATCH] osd/PGLog: assert out on performing overflowed log trimming Performing overflowed log-trim can be a sign of big trouble, e.g., the **complete_to** iterator will now point to an invalid position of the original pg-log list when the trimming is done, and hence randomly trigger **Segmentation fault**s as below: ``` 2018-03-07 17:38:46.109018 7f274a4ed700 -1 *** Caught signal (Segmentation fault) ** 1: (()+0xa51f31) [0x7f278290bf31] 2: (()+0xf370) [0x7f277fb4f370] 3: (PrimaryLogPG::recover_got(hobject_t, eversion_t)+0x266) [0x7f2782512786] 4: (PrimaryLogPG::on_local_recover(hobject_t const&, ObjectRecoveryInfo const&, std::shared_ptr, bool, ObjectStore::Tran saction*)+0x2a4) [0x7f278251f3b4] 5: (ReplicatedBackend::handle_push(pg_shard_t, PushOp const&, PushReplyOp*, ObjectStore::Transaction*)+0x2e2) [0x7f2782690f82] 6: (ReplicatedBackend::_do_push(boost::intrusive_ptr)+0x194) [0x7f2782691224] 7: (ReplicatedBackend::_handle_message(boost::intrusive_ptr)+0x2f1) [0x7f278269fd41] 8: (PGBackend::handle_message(boost::intrusive_ptr)+0x50) [0x7f27825c2470] ``` The root cause of why PGs are starting to trim more log entries than we expect is still lost to me, but setting the trap here should generally do no harm and hopefully expose the above problem a little bit more offen. We'll see. Signed-off-by: xie xingguo --- src/osd/PGLog.cc | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-) diff --git a/src/osd/PGLog.cc b/src/osd/PGLog.cc index 22476bcec6ad9..4445a9cf1a479 100644 --- a/src/osd/PGLog.cc +++ b/src/osd/PGLog.cc @@ -52,9 +52,10 @@ void PGLog::IndexedLog::trim( { if (complete_to != log.end() && complete_to->version <= s) { - generic_dout(0) << " bad trim to " << s << " when complete_to is " - << complete_to->version - << " on " << *this << dendl; + generic_derr << " bad trim to " << s << " when complete_to is " + << complete_to->version + << " on " << *this << dendl; + assert(0 == "out of order trim"); } assert(s <= can_rollback_to); -- 2.39.5