Nathan Cutler [Fri, 23 Jun 2017 06:27:42 +0000 (08:27 +0200)]
tests: move swift.py task to qa/tasks
In preparation for moving this task from ceph/teuthology.git into ceph/ceph.git
The move is necessary because jewel-specific changes are needed, yet teuthology
does not maintain a separate branch for jewel. Also, swift.py is a
Ceph-specific task so it makes more sense to have it in Ceph.
Kefu Chai [Thu, 22 Jun 2017 00:06:43 +0000 (08:06 +0800)]
qa/workunits/rados/test-upgrade-*: whitelist tests the right way
--gtest_filter=POSTIVE_PATTERNS[-NEGATIVE_PATTERNS], so we cannot add
multiple exclusive patterns using -pattern:-pattern, instead, we should
use: -pattern:pattern
Signed-off-by: Kefu Chai <kchai@redhat.com>
Conflicts:
qa/workunits/rados/test-upgrade-v11.0.0.sh: this change is not
cherry-picked from master, because the clone-range op was removed
from master. and only supported in pre-luminous releases.
Kefu Chai [Tue, 20 Jun 2017 11:49:14 +0000 (19:49 +0800)]
qa/workunits/rados/test-upgrade-*: whitelist tests for master
The jewel-x upgrade test now runs this script against a mixed cluster on
a machine with code from master installed. That means we have to
skip any new tests that will fail on a mixed cluster. CloneRange was
removed in 0d7b0b7.
Signed-off-by: Kefu Chai <kchai@redhat.com>
Conflicts:
qa/workunits/rados/test-upgrade-v11.0.0.sh: this change is not
cherry-picked from master, because the clone-range op was removed from
master. and only supported in pre-luminous releases.
Brad Hubbard [Mon, 22 May 2017 03:21:25 +0000 (13:21 +1000)]
osd: Move scrub sleep timer to osdservice
PR 14886 erroneously creates a scrub sleep timer for every pg resulting
in a proliferation of threads. Move the timer to the osd service so
there can be only one.
Conflicts:
qa/tasks/cephfs/test_data_scan.py: difference in the
self._mount.run_shell() call in NonDefaultLayout::write (which is
being dropped by this commit) - in jewel it has "sudo", and in
master it doesn't
John Spray [Wed, 15 Mar 2017 17:51:44 +0000 (17:51 +0000)]
client: _getattr on quota_root before using in statfs
...so that after someone adjusts the quota settings
on an inode that another client is using as its mount root,
the change is visible immediately on the other client.
Sage Weil [Tue, 30 May 2017 13:58:09 +0000 (09:58 -0400)]
qa/workunits/rados/test-upgrade-*: whitelist tests for master
The jewel-x upgrade test now runs this script against a mixed cluster on
a machine with code from master installed. That means we have to skip
any new tests that will fail on a mixed cluster.
Karol Mroz [Thu, 17 Mar 2016 09:32:14 +0000 (10:32 +0100)]
rgw: rest and http client code to use param vectors
Replaces param/header lists with vectors. In these cases, we're only ever
adding to the back of the list, so a vector should be more efficient.
Also moves param_pair_t/param_vec_t higher up the include chain for
cleaner function signatures.
huanwen ren [Tue, 27 Dec 2016 10:54:45 +0000 (10:54 +0000)]
mon/OSDMonitor: fixup sortbitwise flag warning
"ceph -s" does not report warning when using
command "ceph osd unset sortbitwise" to drop
sortbitwise flag.
we should use "osdmap.get_up_osd_features() &
CEPH_FEATURE_OSD_BITWISE_HOBJ_SORT"
instead of "(osdmap.get_features(CEPH_ENTITY_TYPE_OSD, NULL) &
CEPH_FEATURE_OSD_BITWISE_HOBJ_SORT)",
because osdmap.get_features only get local "features"
msg/simple/Pipe: manual backport of fix in PR#14795
Manual backport of errno fixup from PR#14795
(6f1037e22c2a304795895498cdc955e0ef80f8e3), as noted by
https://github.com/ceph/ceph/pull/13450#discussion_r114696885.
Signed-off-by: Robin H. Johnson <robin.johnson@dreamhost.com>
David Zafman [Fri, 3 Mar 2017 23:04:02 +0000 (15:04 -0800)]
osd: Simplify DBObjectMap by no longer creating complete tables
Bump the version for new maps to 3
Make clone less efficient but simpler
Add rename operation (use instead of clone/unlink)
For now keep code that understands version 2 maps
David Zafman [Wed, 15 Feb 2017 23:02:33 +0000 (15:02 -0800)]
osd: Add automatic repair for DBObjectMap bug
Add repair command to ceph-osdomap-tool too
Under some situations the previous rm_keys() code would
generated a corrupt complete table. There is no way to
figure out what the table should look like now. By removing
the entries we fix the corruption and aren't much worse off
because the corruption caused some deleted keys to re-appear.
This doesn't breaking the parent/child relationship during
repair because some of the keys may still be contained
in the parent.
Samuel Just [Fri, 10 Feb 2017 23:50:57 +0000 (15:50 -0800)]
DBObjectMap: strengthen in_complete_region post condition
Previously, in_complete_region didn't guarantee anything about
where it left complete_iter pointing. It will be handy for
complete_iter to be pointing at the lowest interval which ends
after to_test. Make it so.
Samuel Just [Fri, 10 Feb 2017 23:48:57 +0000 (15:48 -0800)]
DBObjectMap: fix next_parent()
The previous implementation assumed that
lower_bound(parent_iter->key()) always leaves the iterator
on_parent(). There isn't any guarantee, however, that that
key isn't present on the child as well.
rgw: add suport for creating S3 type subuser of admin rest api
Fixes: http://tracker.ceph.com/issues/16682
The original code cannot support create s3 type subuser of admin rest api as when i execute the following command:
test/librados/tmap_migrate: g_ceph_context->put() upon return
Signed-off-by: Kefu Chai <kchai@redhat.com>
Conflict: test/librados/tmap_migrate.cc
this change is not cherry-picked from master, because tmap_migrate was
removed in master. so we are applying the same change in cb1cda96713b2ec0f6418c4cbe3d964c2020729c to this test.
Keystone tokens can be revoked. This causes them to fail
validation. However, in ceph, we cache them. As long as
they're in the cache we trust them. To find revoked tokens
there's a call OSI-PKI/revoked but that's only useful for
pki tokens. Installations using fernet/uuid may not even
have the proper credentials to support the call, in which
case the call blows up in various ways filling up logs
with complaints.
This code makes the revocation thread optional; by disabling it,
the complaints go away. A further fix is in the works
to use other more modern calls available in modern keystone
installations to properly deal with non-PKI/PKIZ tokens.
(NB: jewel has this logic in src/rgw/rgw_swift.cc not in src/rgw/rgw_keystone.h)
To disable the revocation thread, use at least one of these:
rgw_keystone_token_cache_size = 0
using this will cause tokens to be validated on every call.
You may instead want to set
rgw_keystone_revocation_interval = 0
using just this will disable the revocation thread,
but leaves the cache in use. That avoids the extra
validation overhead, but means token revocation won't
work very well.
Fixes: http://tracker.ceph.com/issues/9493 Fixes: http://tracker.ceph.com/issues/19499 Signed-off-by: Marcus Watts <mwatts@redhat.com>
(cherry picked from commit 003291a8cbca455c0e8731f66759395a0bb1f555)
Matt Benjamin [Sat, 5 Nov 2016 17:13:47 +0000 (13:13 -0400)]
rgw: add bucket size limit check to radosgw-admin
The change adds a new list of all buckets x all users, with
fields for bucket name, tenant name, current num_objects,
current num_shards, current objects per shard, and the
corresponding fill_status--the latter consisting of 'OK',
'WARN <n>%', or 'OVER <n>%.'
The warning check is relative to two new tunables. The threshold
max objects per shard is set as rgw_bucket_safe_max_objects_per_shard,
which defaults to 100K. The value rgw_bucket_warning_threshold is
a percent of the current safe max at which to warn (defaults to
90% of full).
From review:
* fix indentation (rgw_admin)
* if user a user_id is provided, check only buckets for that user
* update shard warn pct to be pct-of-fill (not 100 - pct-of-fill)
* print only buckets near or over per-shard limit, if --warnings-only
* s/bucket limitcheck/bucket limit check */
* sanity shard limit should be 90, not 10 (because that changed)
* fixes for memleaks and other points found by cbodley
Fixes: http://tracker.ceph.com/issues/17925 Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
(cherry picked from commit 7bc144ce36fedc16a3dedc54598b0d75fb8c68bc)
projects / ceph.git / log