From: xie xingguo Date: Sat, 21 Apr 2018 02:33:49 +0000 (+0800) Subject: osd/PGLog: assert out on performing overflowed log trimming X-Git-Tag: v13.1.0~144^2 X-Git-Url: http://git.apps.os.sepia.ceph.com/?a=commitdiff_plain;h=8a395f258621c95460cd6776acfd60adcde8b441;p=ceph.git osd/PGLog: assert out on performing overflowed log trimming Performing overflowed log-trim can be a sign of big trouble, e.g., the **complete_to** iterator will now point to an invalid position of the original pg-log list when the trimming is done, and hence randomly trigger **Segmentation fault**s as below: ``` 2018-03-07 17:38:46.109018 7f274a4ed700 -1 *** Caught signal (Segmentation fault) ** 1: (()+0xa51f31) [0x7f278290bf31] 2: (()+0xf370) [0x7f277fb4f370] 3: (PrimaryLogPG::recover_got(hobject_t, eversion_t)+0x266) [0x7f2782512786] 4: (PrimaryLogPG::on_local_recover(hobject_t const&, ObjectRecoveryInfo const&, std::shared_ptr, bool, ObjectStore::Tran saction*)+0x2a4) [0x7f278251f3b4] 5: (ReplicatedBackend::handle_push(pg_shard_t, PushOp const&, PushReplyOp*, ObjectStore::Transaction*)+0x2e2) [0x7f2782690f82] 6: (ReplicatedBackend::_do_push(boost::intrusive_ptr)+0x194) [0x7f2782691224] 7: (ReplicatedBackend::_handle_message(boost::intrusive_ptr)+0x2f1) [0x7f278269fd41] 8: (PGBackend::handle_message(boost::intrusive_ptr)+0x50) [0x7f27825c2470] ``` The root cause of why PGs are starting to trim more log entries than we expect is still lost to me, but setting the trap here should generally do no harm and hopefully expose the above problem a little bit more offen. We'll see. Signed-off-by: xie xingguo --- diff --git a/src/osd/PGLog.cc b/src/osd/PGLog.cc index 22476bcec6ad9..4445a9cf1a479 100644 --- a/src/osd/PGLog.cc +++ b/src/osd/PGLog.cc @@ -52,9 +52,10 @@ void PGLog::IndexedLog::trim( { if (complete_to != log.end() && complete_to->version <= s) { - generic_dout(0) << " bad trim to " << s << " when complete_to is " - << complete_to->version - << " on " << *this << dendl; + generic_derr << " bad trim to " << s << " when complete_to is " + << complete_to->version + << " on " << *this << dendl; + assert(0 == "out of order trim"); } assert(s <= can_rollback_to);