Kefu Chai [Thu, 24 May 2018 09:21:42 +0000 (17:21 +0800)]
dout: declare dpp using `decltype(auto)` instead of `auto`
this makes `pdpp` an alias of `dpp`, guess this ensure GCC that the
returned `sub` is a constant.
In file included from /home/kefu/dev/ceph/src/kv/LevelDBStore.h:25,
from /home/kefu/dev/ceph/src/kv/KeyValueDB.cc:6:
/home/kefu/dev/ceph/src/osd/osd_types.h: In lambda function:
/home/kefu/dev/ceph/src/common/dout.h:101:75: error: the value of ‘pdpp’
is not usable in a constant expression
dout_impl(pdpp->get_cct(),
ceph::dout::need_dynamic(pdpp->get_subsys()), v) \
^
/home/kefu/dev/ceph/src/common/dout.h:81:58: note: in definition of
macro ‘dout_impl’
return (cctX->_conf->subsys.template should_gather<sub, v>()); \
^~~
/home/kefu/dev/ceph/src/osd/osd_types.h:2992:3: note: in expansion of
macro ‘ldpp_dout’
ldpp_dout(dpp, 10) << "build_prior all_probe " << all_probe << dendl;
^~~~~~~~~
/home/kefu/dev/ceph/src/common/dout.h:100:12: note: ‘pdpp’ was not
declared ‘constexpr’
if (auto pdpp = (dpp); pdpp) /* workaround -Wnonnull-compare for
'this' */ \
^~~~
/home/kefu/dev/ceph/src/common/dout.h:100:12: note: in definition of
macro ‘ldpp_dout’
if (auto pdpp = (dpp); pdpp) /* workaround -Wnonnull-compare for
'this' */ \
^~~~
Sage Weil [Tue, 22 May 2018 21:55:03 +0000 (16:55 -0500)]
mon/MgrMonitor: change 'unresponsive' message to info level
We generate a MGR_DOWN health warning at the appropriate points; having
this at WRN level just triggers failed teuthology runs but doesn't much
value for the user.
Clear out teuthology whitelisting for this message.
Fixes: http://tracker.ceph.com/issues/24222 Signed-off-by: Sage Weil <sage@redhat.com>
common: OpTracker doesn't visit TrackedOp when nref == 0.
The patch fixes a race condition that happens between
`unregister_inflight_op` and `visit_ops_in_flight` of
`OpTracker`. When a callable passed to the former one
turns the plain reference it gets into `TrackedOpRef`,
an almost-to-terminate `TrackedOp` (with `nref == 0`)
can be resurrected (`nref++`). This will be reflected
in extra call to `unregister_inflight_op` for same op
leading to e.g. use-after-free. For more details see:
https://tracker.ceph.com/issues/24037#note-5.
The fix deals with the problem by ensuring there will
be no call to the visitor for ops with zeroized `nref`.
Jianpeng Ma [Mon, 21 May 2018 14:46:12 +0000 (22:46 +0800)]
os/bluefs: only flush dirty devices when do _fsync.
Now _fsync call flush_bdev make data safely. But flush_bdev flush all
devices which don't care whether has data for this sync.
So add new api flush_bdev(std::array<bool, MAX_BDEV>& dirty_bdevs)
which only flush dirty devices for this sync op.
Signed-off-by: Jianpeng Ma <jianpeng.ma@intel.com>
Sage Weil [Mon, 21 May 2018 13:24:54 +0000 (08:24 -0500)]
os/bluestore: move txc on_commits assignment into ctor
This avoids adjusting the oncommits without a lock after the txc is
queued on the sequencer.
This is a bit defensive since the ObjectStore caller doesn't call
flush_commit() at the same time as queue_transaction(), but the could
change in the future.
Sage Weil [Mon, 21 May 2018 15:06:37 +0000 (10:06 -0500)]
os/bluestore: simplify and fix SharedBlob::put()
There is a narrow race possible:
A: lookup foo
A: put on foo
A: foo --nref == 0
B: lookup foo
B: put foo
B: foo --nref == 0
B: try_remove() succeeds, removes
A: try_remove() tries to remove foo again, probably crashes
We could fix this by flagging the object in some way to indicate it was
removed (maybe clearing parent?), but then we need to be careful about
dereferencing foo to get parent from put().
Fix this by moving to a simpler model: make lookup fail if nref == 0.
This eliminates the races around put() entirely because once nref reaches
0 it never goes up again.
Fixes: http://tracker.ceph.com/issues/24211 Signed-off-by: Sage Weil <sage@redhat.com>
Sage Weil [Mon, 21 May 2018 13:41:00 +0000 (08:41 -0500)]
Merge PR #22091 into master
* refs/pull/22091/head:
crush: update choose_args on bucket removal
crush: update choose_args on bucket removal, resize, or position mismatch
crush: create weight-set on demand when doing a choose-args reweight
test/cli/crushtool: use straw2 buckets for choose-args test
crush: weight_set_size -> weight_set_positions
Sage Weil [Fri, 18 May 2018 14:18:11 +0000 (09:18 -0500)]
os/bluestore: fix flush_commit locking
We were updating the txc state to KV_DONE and queuing the oncommits
waiters without holding any locks. This was mostly fine, *except* that
Collection|OpSequencer::flush_commit(Context *) was looking at the state
(under qlock) and also adding items to oncommits.
The flush_commit() method is only used in 2 places: osd bench, and the
PG reset_interval_flush outgoing message blocking machinery (which is
a bit ick). The first we could get rid of, but the second is hard to
remove (despite its ick factor).
The simple fix is to take qlock while updating the state value and
working with oncommits.
Fixes: http://tracker.ceph.com/issues/21480 Signed-off-by: Sage Weil <sage@redhat.com>
Sage Weil [Mon, 21 May 2018 12:07:59 +0000 (07:07 -0500)]
Merge PR #21540 into wip-sage-testing-20180521.120735
* refs/pull/21540/head:
tests/crypto: print compile warning when NSS is unavailable.
tests/crypto: add tests for the no-bl encrypt/decrypt, part 2.
tests/crypto: add tests for the no-bl encrypt/decrypt.
auth: use OpenSSL for CryptoAESKeyHandler's no-bl encrypt/decrypt.
auth: extend CryptoKey with no-bl encrypt/decrypt.
auth: CryptoAESKeyHandler switches from NSS to OpenSSL.
auth: the outbuf of AES should be multiple of block size
auth: cache the PK11Context for CryptoAESKeyHandler
Kefu Chai [Sun, 20 May 2018 08:52:53 +0000 (16:52 +0800)]
qa/workunits/rados/test_envlibrados_for_rocksdb: use cmake not make
* so we just rely on a single build system instead of two of them, the
other place we use cmake is cmake/modules/BuildRockDB.cmake.
* disable gflags when building rocksdb, it's optional and does not help
in the sense of testing librados support.
* disable prompts when installing on debian, to silence warnings like:
debconf: unable to initialize frontend: Dialog
* drop --force-yes option, as it is deprecated, and is replaced with
--allow-downgrades, --allow-remove-essential,
--allow-change-held-packages, but none of them apply in our case.
Sage Weil [Fri, 18 May 2018 18:11:57 +0000 (13:11 -0500)]
crush: update choose_args on bucket removal
The specific bug I see is that a bucket no longer exists but its
choose_args still does. However, I'm also taking the opportunity to
verify that the choose_args agrees with the bucket sizes and position
counts everywhere else, too. Check for
- ids or weight_sets for buckets that don't exist or aren't straw2
- weight_set_positions that don't match the choose_args
- don't fix this, just warn. i'm not sure how it would happen. :/
- weight_set sizes that don't match the bucket size
Fixes: http://tracker.ceph.com/issues/24167 Signed-off-by: Sage Weil <sage@redhat.com>
Yan, Zheng [Fri, 11 May 2018 06:55:12 +0000 (14:55 +0800)]
mds: reply session reject for open request from blacklisted client
Kernel client and old version libcephfs do not check if themselves
are blacklisted. They can be stuck at opening session after getting
blacklisted. The session reject message can avoid this.
Yan, Zheng [Fri, 18 May 2018 06:26:32 +0000 (14:26 +0800)]
client: fix issue of revoking non-auth caps
when non-auth mds revokes caps, Fcb caps can still be issued by auth
auth mds. It's wrong to flush buffer or invalidate cache when non-auth
mds revokes other caps. This bug can cause client to not respond the
revoke.
Sage Weil [Fri, 18 May 2018 18:11:57 +0000 (13:11 -0500)]
crush: update choose_args on bucket removal, resize, or position mismatch
The specific bug I see is that a bucket no longer exists but its
choose_args still does. However, I'm also taking the opportunity to
verify that the choose_args agrees with the bucket sizes and position
counts everywhere else, too. Check for
- ids or weight_sets for buckets that don't exist or aren't straw2
- weight_set_positions that don't match the choose_args
- weight_set sizes that don't match the bucket size
Fixes: http://tracker.ceph.com/issues/24167 Signed-off-by: Sage Weil <sage@redhat.com>
Sage Weil [Fri, 18 May 2018 22:10:14 +0000 (17:10 -0500)]
crush: create weight-set on demand when doing a choose-args reweight
If we try to reweight an item in a choose-args set for a given item
but the bucket doesn't have a weight_set yet, create it on the fly, and
populate it with the canonical bucket weights.
projects / ceph.git / log