Sage Weil [Mon, 21 May 2018 15:06:37 +0000 (10:06 -0500)]
os/bluestore: simplify and fix SharedBlob::put()
There is a narrow race possible:
A: lookup foo
A: put on foo
A: foo --nref == 0
B: lookup foo
B: put foo
B: foo --nref == 0
B: try_remove() succeeds, removes
A: try_remove() tries to remove foo again, probably crashes
We could fix this by flagging the object in some way to indicate it was
removed (maybe clearing parent?), but then we need to be careful about
dereferencing foo to get parent from put().
Fix this by moving to a simpler model: make lookup fail if nref == 0.
This eliminates the races around put() entirely because once nref reaches
0 it never goes up again.
Fixes: http://tracker.ceph.com/issues/24211 Signed-off-by: Sage Weil <sage@redhat.com>
Sage Weil [Mon, 21 May 2018 13:41:00 +0000 (08:41 -0500)]
Merge PR #22091 into master
* refs/pull/22091/head:
crush: update choose_args on bucket removal
crush: update choose_args on bucket removal, resize, or position mismatch
crush: create weight-set on demand when doing a choose-args reweight
test/cli/crushtool: use straw2 buckets for choose-args test
crush: weight_set_size -> weight_set_positions
Sage Weil [Mon, 21 May 2018 12:07:59 +0000 (07:07 -0500)]
Merge PR #21540 into wip-sage-testing-20180521.120735
* refs/pull/21540/head:
tests/crypto: print compile warning when NSS is unavailable.
tests/crypto: add tests for the no-bl encrypt/decrypt, part 2.
tests/crypto: add tests for the no-bl encrypt/decrypt.
auth: use OpenSSL for CryptoAESKeyHandler's no-bl encrypt/decrypt.
auth: extend CryptoKey with no-bl encrypt/decrypt.
auth: CryptoAESKeyHandler switches from NSS to OpenSSL.
auth: the outbuf of AES should be multiple of block size
auth: cache the PK11Context for CryptoAESKeyHandler
Kefu Chai [Sun, 20 May 2018 08:52:53 +0000 (16:52 +0800)]
qa/workunits/rados/test_envlibrados_for_rocksdb: use cmake not make
* so we just rely on a single build system instead of two of them, the
other place we use cmake is cmake/modules/BuildRockDB.cmake.
* disable gflags when building rocksdb, it's optional and does not help
in the sense of testing librados support.
* disable prompts when installing on debian, to silence warnings like:
debconf: unable to initialize frontend: Dialog
* drop --force-yes option, as it is deprecated, and is replaced with
--allow-downgrades, --allow-remove-essential,
--allow-change-held-packages, but none of them apply in our case.
Sage Weil [Fri, 18 May 2018 18:11:57 +0000 (13:11 -0500)]
crush: update choose_args on bucket removal
The specific bug I see is that a bucket no longer exists but its
choose_args still does. However, I'm also taking the opportunity to
verify that the choose_args agrees with the bucket sizes and position
counts everywhere else, too. Check for
- ids or weight_sets for buckets that don't exist or aren't straw2
- weight_set_positions that don't match the choose_args
- don't fix this, just warn. i'm not sure how it would happen. :/
- weight_set sizes that don't match the bucket size
Fixes: http://tracker.ceph.com/issues/24167 Signed-off-by: Sage Weil <sage@redhat.com>
Yan, Zheng [Fri, 11 May 2018 06:55:12 +0000 (14:55 +0800)]
mds: reply session reject for open request from blacklisted client
Kernel client and old version libcephfs do not check if themselves
are blacklisted. They can be stuck at opening session after getting
blacklisted. The session reject message can avoid this.
Yan, Zheng [Fri, 18 May 2018 06:26:32 +0000 (14:26 +0800)]
client: fix issue of revoking non-auth caps
when non-auth mds revokes caps, Fcb caps can still be issued by auth
auth mds. It's wrong to flush buffer or invalidate cache when non-auth
mds revokes other caps. This bug can cause client to not respond the
revoke.
Sage Weil [Fri, 18 May 2018 18:11:57 +0000 (13:11 -0500)]
crush: update choose_args on bucket removal, resize, or position mismatch
The specific bug I see is that a bucket no longer exists but its
choose_args still does. However, I'm also taking the opportunity to
verify that the choose_args agrees with the bucket sizes and position
counts everywhere else, too. Check for
- ids or weight_sets for buckets that don't exist or aren't straw2
- weight_set_positions that don't match the choose_args
- weight_set sizes that don't match the bucket size
Fixes: http://tracker.ceph.com/issues/24167 Signed-off-by: Sage Weil <sage@redhat.com>
Sage Weil [Fri, 18 May 2018 22:10:14 +0000 (17:10 -0500)]
crush: create weight-set on demand when doing a choose-args reweight
If we try to reweight an item in a choose-args set for a given item
but the bucket doesn't have a weight_set yet, create it on the fly, and
populate it with the canonical bucket weights.
auth: cache the PK11Context for CryptoAESKeyHandler
in the flame graph, 0.50% of total time is used by
CephxSessionHandler::check_message_signature(), in which 0.27% is used
by PK11_CreateContextBySymKey(). so we should cache the PK11Context.
Sage Weil [Fri, 18 May 2018 14:24:48 +0000 (09:24 -0500)]
osd/PrimaryLogPG: fix try_flush_mark_clean write contention case
Release ctx *after* we dereference it to queue up the other flush waiters.
This was a dumb mistake in the recent fix from 41609ba1b24e5384494c7959494e19302b5a2429
Fixes: http://tracker.ceph.com/issues/24174 Signed-off-by: Sage Weil <sage@redhat.com>
Updated Ceph manager dashboard documentation: added SSL to the
feature overview, added new section about configuring SSL certificates.
Rearranged installation instructions for setting host name and port.
Tiago Melo [Thu, 17 May 2018 13:33:27 +0000 (14:33 +0100)]
mgr/dashboard: Disable package-lock.json creation
Since we are not keeping track of this file, it is better to not generate it.
Otherwise we might face some package version conflicts,
which break the dashboard build.
This test is trying to reproduce a case where we don't have a session
for the given OSD, drop our big lock to open one, and when we retake the
osdmap has changed. Previously that was an easy situation to reproduce
by creating an ioctx for a pool that didn't exist (yet) and then creating
it. The missing pool would send us into the path that rechecks for an
updated osdmap and trigger the race condition.
However, as of 7037cf8f77588aba1e3f251e04f696da0eccf462 we no longer create
ioctxs for pools that don't exist. And there isn't an easy way that I can
think of to trigger the missing osd session that races with an osdmap
update. So, use the debug option that injects the delay to drop us into
the same path. This is a bit artificial in that we do actually have an
osdmap session, but the block handles that case (since the old no pool
case worked that way), so we cover the same code.
This fix ignores the fact that we still have all this "pool doesn't exist"
handling code in Objecter that deals with teh fact that you could create
an ioctx for a missing pool...that is no longer possible. Cleanup for
another day!
Fixes: http://tracker.ceph.com/issues/24150 Signed-off-by: Sage Weil <sage@redhat.com>
projects / ceph.git / log