Nathan Cutler [Mon, 23 Jan 2017 16:49:54 +0000 (17:49 +0100)]
Merge pull request #13041 from SUSE/wip-18448-hammer
hammer: osd: filestore: FALLOC_FL_PUNCH_HOLE must be used with FALLOC_FL_KEEP_SIZE
Reviewed-by: Sage Weil <sage@redhat.com>
John Spray [Mon, 23 Jan 2017 11:26:27 +0000 (12:26 +0100)]
Merge pull request #13022 from SUSE/wip-18602-hammer
hammer: cephfs test failures (ceph.com/qa is broken, should be download.ceph.com/qa)
Reviewed-by: John Spray <john.spray@redhat.com>
John Spray [Mon, 23 Jan 2017 11:25:43 +0000 (12:25 +0100)]
Merge pull request #13039 from SUSE/wip-fs-thrash-xfs
tests: run fs/thrash on xfs instead of btrfs
Reviewed-by: John Spray <john.spray@redhat.com>
John Spray [Tue, 17 Jan 2017 16:12:46 +0000 (17:12 +0100)]
qa: update remaining ceph.com to download.ceph.com
Fixes: http://tracker.ceph.com/issues/18574
Signed-off-by: John Spray <john.spray@redhat.com>
(cherry picked from commit
549d993d3fd8ffffa280ed4a64aca41d1c6f2da1)
Nathan Cutler [Sat, 21 Jan 2017 13:54:52 +0000 (14:54 +0100)]
os/filestore: FALLOC_FL_PUNCH_HOLE must be used with FALLOC_FL_KEEP_SIZE
Signed-off-by: xinxin shu <xinxin.shu@intel.com>
(cherry picked from commit
7bd95b595fddb8a4e618a2c7df1ba04eccf0829d)
Conflicts:
src/os/FileStore.cc (in master, this file is in a different location)
Nathan Cutler [Sat, 21 Jan 2017 13:24:25 +0000 (14:24 +0100)]
tests: run fs/thrash on xfs instead of btrfs
Signed-off-by: Nathan Cutler <ncutler@suse.com>
Nathan Cutler [Sat, 21 Jan 2017 09:00:04 +0000 (10:00 +0100)]
Merge pull request #12824 from SUSE/wip-18449-hammer
hammer: [teuthology] update rbd/singleton/all/formatted-output.yaml to support ceph-ci
Nathan Cutler [Fri, 20 Jan 2017 22:10:10 +0000 (23:10 +0100)]
Merge pull request #12906 from SUSE/wip-15943-hammer
hammer: osd/PG: fix cached_removed_snaps bug in PGPool::update after map gap
Reviewed-by: Samuel Just <sjust@redhat.com>
Nathan Cutler [Fri, 20 Jan 2017 22:02:05 +0000 (23:02 +0100)]
Merge pull request #12805 from SUSE/wip-18432-hammer
hammer: ceph-create-keys loops forever
Reviewed-by: Nathan Cutler <ncutler@suse.com>
Jason Dillaman [Tue, 17 Jan 2017 13:39:14 +0000 (08:39 -0500)]
Merge pull request #12957 from SUSE/wip-18544-hammer
hammer: [teuthology] update Ubuntu image url after ceph.com refactor
Reviewed-by: Jason Dillaman <dillaman@redhat.com>
Jason Dillaman [Tue, 17 Jan 2017 03:12:51 +0000 (22:12 -0500)]
qa/tasks/qemu: update default image url after ceph.com redesign
Fixes: http://tracker.ceph.com/issues/18542
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit
6d17befb3bbc3d83c9d23d763ad95e1e7b2e4be0)
Samuel Just [Mon, 12 Dec 2016 18:35:38 +0000 (10:35 -0800)]
PG: fix cached_removed_snaps bug in PGPool::update after map gap
5798fb3bf6d726d14a9c5cb99dc5902eba5b878a actually made 15943 worse
by always creating an out-of-date cached_removed_snaps value after
a map gap rather than only in the case where the the first map after
the gap did not remove any snapshots.
Introduced:
5798fb3bf6d726d14a9c5cb99dc5902eba5b878a
Fixes: http://tracker.ceph.com/issues/15943
Signed-off-by: Samuel Just <sjust@redhat.com>
(cherry picked from commit
5642e7e1b3bb6ffceddacd2f4030eb13a17fcccc)
Nathan Cutler [Sat, 7 Jan 2017 12:48:51 +0000 (13:48 +0100)]
tests: subst repo and branch in qemu test urls
References: http://tracker.ceph.com/issues/18440
Signed-off-by: Nathan Cutler <ncutler@suse.com>
(cherry picked from commit
2484a5546160560a4050d35400a6edce37ae48ad)
Conflicts: (hammer doesn't have these files)
qa/suites/rbd/maintenance/qemu/xfstests.yaml
qa/suites/rbd/qemu/workloads/#qemu_xfstests.yaml#
Nathan Cutler [Sat, 7 Jan 2017 12:45:10 +0000 (13:45 +0100)]
tests: subst branch and repo in qa/tasks/qemu.py
References: http://tracker.ceph.com/issues/18440
Signed-off-by: Nathan Cutler <ncutler@suse.com>
(cherry picked from commit
74689df754561e11a3377998840efcea9e780755)
Nathan Cutler [Fri, 6 Jan 2017 20:06:11 +0000 (21:06 +0100)]
tests: subst repo name in qa/tasks/cram.py
Inspired by
bcbe45d948f1c4da02e27b3be5f29a8b52745e28
Fixes: http://tracker.ceph.com/issues/18440
Signed-off-by: Nathan Cutler <ncutler@suse.com>
(cherry picked from commit
56e37e41f4dddd289dd3c1886b192cd328ed311b)
Venky Shankar [Fri, 9 Dec 2016 09:41:49 +0000 (15:11 +0530)]
cram: support fetching from sha1 branch, tag, commit hash
Signed-off-by: Venky Shankar <vshankar@redhat.com>
(cherry picked from commit
d2f0d745987a2e2eee4e1822146aad8da5d42708)
Conflicts:
suites/krbd/unmap/tasks/unmap.yaml (hammer doesn't have this)
suites/rbd/singleton/all/formatted-output.yaml (moved to qa/,
trivial resolution)
Orit Wasserman [Sun, 8 Jan 2017 07:26:03 +0000 (09:26 +0200)]
Merge pull request #12227 from ceph/wip-orit-resharding-hammer
Wip orit resharding hammer
Reviewed-by: Orit Wasserman <owasserm@redhat.com>
Alfredo Deza [Wed, 2 Nov 2016 16:28:49 +0000 (12:28 -0400)]
ceph-create-keys: wait 10 minutes to get or create the bootstrap key, not forever
Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit
bea802bd13df883d06b7b67a83ac6540e44ab6c1)
Alfredo Deza [Wed, 2 Nov 2016 16:25:32 +0000 (12:25 -0400)]
ceph-create-keys: wait 10 minutes to get or create a key, not forever
Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit
8f84681a4d97a7ea6bc04e759eccfc40204a2fdb)
Conflicts:
src/ceph-create-keys ( hammer does not have
52e978e4b3660baa9f50b1bb8247909b672142e7 )
Alfredo Deza [Wed, 2 Nov 2016 16:19:10 +0000 (12:19 -0400)]
ceph-create-keys: wait for quorum for ten minutes, not forever
Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit
32cedd2c18e48d5f9a1547e5fd69c24c7eac1638)
Samuel Just [Thu, 5 Jan 2017 17:15:13 +0000 (09:15 -0800)]
Merge pull request #11615 from mslovy/wip-17610-hammer
hammer: osd: fix fiemap issue in xfs when #extents > 1364
Reviewed-by: Samuel Just <sjust@redhat.com>
Samuel Just [Thu, 5 Jan 2017 17:14:10 +0000 (09:14 -0800)]
Merge pull request #11936 from SUSE/wip-16432-hammer
hammer: librados: bad flags can crash the osd
Reviewed-by: Samuel Just <sjust@redhat.com>
Samuel Just [Thu, 5 Jan 2017 17:12:55 +0000 (09:12 -0800)]
Merge pull request #12121 from dzafman/wip-16225-min
hammer: common: SIGABRT in TrackedOp::dump() via dump_ops_in_flight()
Reviewed-by: Samuel Just <sjust@redhat.com>
Samuel Just [Thu, 5 Jan 2017 17:11:57 +0000 (09:11 -0800)]
Merge pull request #12266 from SUSE/wip-18109-hammer
hammer: msgr: simple/Pipe: error decoding addr
Reviewed-by: Samuel Just <sjust@redhat.com>
Samuel Just [Thu, 5 Jan 2017 17:11:34 +0000 (09:11 -0800)]
Merge pull request #12417 from SUSE/wip-18132-hammer
hammer: osd: ReplicatedBackend::build_push_op: add a second config to limit omap entries/chunk independently of object data
Reviewed-by: Samuel Just <sjust@redhat.com>
Samuel Just [Thu, 5 Jan 2017 17:11:13 +0000 (09:11 -0800)]
Merge pull request #12759 from SUSE/wip-18399-hammer
hammer: tests: objecter_requests workunit fails on wip branches
Reviewed-by: Samuel Just <sjust@redhat.com>
Samuel Just [Thu, 5 Jan 2017 17:10:54 +0000 (09:10 -0800)]
Merge pull request #12762 from SUSE/wip-18405-hammer
hammer: Cannot reserve CentOS 7.2 smithi machines
Reviewed-by: Samuel Just <sjust@redhat.com>
Samuel Just [Thu, 5 Jan 2017 17:10:28 +0000 (09:10 -0800)]
Merge pull request #12685 from SUSE/wip-18273-hammer
hammer: tests: Workunits needlessly wget from git.ceph.com
Reviewed-by: Samuel Just <sjust@redhat.com>
Samuel Just [Thu, 5 Jan 2017 17:09:59 +0000 (09:09 -0800)]
Merge pull request #12687 from SUSE/wip-18281-hammer
hammer: mon: osd flag health message is misleading
Reviewed-by: Kefu Chai <kchai@redhat.com>
Nathan Cutler [Thu, 5 Jan 2017 14:20:28 +0000 (15:20 +0100)]
Merge pull request #12423 from SUSE/wip-18222-hammer
hammer: tests: qemu/tests/qemu-iotests/077 fails in dumpling, hammer, and jewel
Reviewed-by: Jason Dillaman <dillaman@redhat.com>
Nathan Cutler [Wed, 4 Jan 2017 19:15:21 +0000 (20:15 +0100)]
Merge pull request #12744 from SUSE/wip-18385-hammer
hammer: tests: Cannot clone ceph/s3-tests.git (missing branch)
Reviewed-by: Orit Wasserman <owasserm@redhat.com>
Nathan Cutler [Wed, 4 Jan 2017 10:43:29 +0000 (11:43 +0100)]
Merge pull request #12418 from SUSE/wip-18218-hammer
hammer: rgw sends omap_getvals with (u64)-1 limit
Reviewed-by: Orit Wasserman <owasserm@redhat.com>
Nathan Cutler [Wed, 4 Jan 2017 10:37:29 +0000 (11:37 +0100)]
Merge pull request #12619 from Werkov/wip-18317-hammer
hammer: rgw: TempURL in radosgw behaves now like its Swift's counterpart.
Reviewed-by: Orit Wasserman <owasserm@redhat.com>
Radoslaw Zarzynski [Fri, 4 Mar 2016 15:30:50 +0000 (16:30 +0100)]
rgw: TempURL in radosgw behaves now like its Swift's counterpart.
Signed-off-by: Radoslaw Zarzynski <rzarzynski@mirantis.com>
(cherry picked from commit
a1ea7180969dcbaf88dfcc1bfafb17e82639df64)
Conflicts:
src/rgw/rgw_swift.cc
- replace C++11 constructs,
- s->info.method is actually unconditional member of
allowed_methods, which was opaqued by C++11 ctors
- drop temp_url_make_content_disp() which has no use with
hammer's req_state
Nathan Cutler [Tue, 3 Jan 2017 14:31:27 +0000 (15:31 +0100)]
Merge pull request #12312 from YankunLi/fix_parameter_check
hammer: rbd: fix parameter check
Reviewed-by: Jason Dillaman <dillaman@redhat.com>
Nathan Cutler [Tue, 3 Jan 2017 14:31:03 +0000 (15:31 +0100)]
Merge pull request #12446 from dillaman/wip-18111
hammer: librbd: diffs to clone's first snapshot should include parent diffs
Reviewed-by: Jason Dillaman <dillaman@redhat.com>
Nathan Cutler [Tue, 3 Jan 2017 14:23:07 +0000 (15:23 +0100)]
Merge pull request #12398 from linuxbox2/hammer
hammer: rgw: do not abort when accept a CORS request with short origin
Reviewed-by: Orit Wasserman <owasserm@redhat.com>
Jason Dillaman [Tue, 3 Jan 2017 14:17:35 +0000 (09:17 -0500)]
Merge pull request #12758 from SUSE/wip-18397-hammer
hammer: tests: OSDs commit suicide in rbd suite when testing on btrfs
Reviewed-by: Jason Dillaman <dillaman@redhat.com>
Nathan Cutler [Tue, 3 Jan 2017 13:53:27 +0000 (14:53 +0100)]
qa/distros: add centos yaml; use that instead
Modeled after https://github.com/ceph/ceph/pull/12632
Fixes: http://tracker.ceph.com/issues/18405
Signed-off-by: Nathan Cutler <ncutler@suse.com>
Sage Weil [Thu, 15 Dec 2016 18:35:02 +0000 (13:35 -0500)]
qa/tasks/admin_socket: subst in repo name
It is either ceph.git or ceph-ci.git.
Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit
bcbe45d948f1c4da02e27b3be5f29a8b52745e28)
Nathan Cutler [Tue, 3 Jan 2017 10:26:26 +0000 (11:26 +0100)]
rbd: only test on xfs
btrfs is not recommended for production
See
5864d56592ef3ffe765aa1ef0865610dbc7e8725 for precendent.
Signed-off-by: Nathan Cutler <ncutler@suse.com>
Nathan Cutler [Mon, 2 Jan 2017 21:49:13 +0000 (22:49 +0100)]
tests: rbd/test_lock_fence.sh: fix rbdrw.py relative path
This commit fixes a regression introduced by
cf294777ea92f0911813a7132068584d4f73a65a
Fixes: http://tracker.ceph.com/issues/18388
Signed-off-by: Nathan Cutler <ncutler@suse.com>
(cherry picked from commit
91231de16dbe4d0e493ec617165a2b38078d122b)
Orit Wasserman [Mon, 4 Jan 2016 09:03:08 +0000 (10:03 +0100)]
use ceph-master branch for s3tests
Signed-off-by: Orit Wasserman <owasserm@redhat.com>
(cherry picked from commit
21160124028ab123b5c547499d911b0a1397214f)
Conflicts: (trivial resolution)
qa/suites/rgw/multifs/tasks/rgw_s3tests.yaml
qa/suites/rgw/verify/tasks/rgw_s3tests.yaml
qa/suites/rgw/verify/tasks/rgw_s3tests_multiregion.yaml
Nathan Cutler [Mon, 2 Jan 2017 19:34:42 +0000 (20:34 +0100)]
Merge pull request #12743 from SUSE/wip-18383
hammer: tests: populate mnt_point in qa/tasks/ceph.py
Reviewed-by: Loic Dachary <ldachary@redhat.com>
Nathan Cutler [Mon, 2 Jan 2017 13:22:43 +0000 (14:22 +0100)]
qa/tasks/ceph.py: populate mnt_point in hammer
In hammer, there is no "mnt_point" variable, yet
53225d5 was cherry-picked to
hammer - see
87e0299448908775600c62130617bad503027768 and https://github.com/ceph/ceph/pull/12455
This caused lots of "NameError: global name 'mnt_point' is not defined" errors
in hammer integration testing.
This is not a cherry-pick, because the mnt_point variable was introduced by
a54ff597faa379c8fd369352fb924651be269f95 ("tasks/ceph: convert cluster creation
to work with multiple clusters") which was not backported to hammer.
Fixes: http://tracker.ceph.com/issues/18383
Signed-off-by: Nathan Cutler <ncutler@suse.com>
weiqiaomiao [Mon, 31 Oct 2016 05:59:55 +0000 (13:59 +0800)]
rgw: fix osd crashes when execute "radosgw-admin bi list --max-entries=1" command
Fixes: http://tracker.ceph.com/issues/17745
Signed-off-by: weiqiaomiao <wei.qiaomiao@zte.com.cn>
(cherry picked from commit
51a4405a1ef59671cbd51c530a84333359d3f3dd)
Orit Wasserman [Tue, 29 Nov 2016 12:36:37 +0000 (13:36 +0100)]
rgw: use hammer rgw_obj_key api
Signed-off-by: Orit Wasserman <owasserm@redhat.com>
Orit Wasserman [Tue, 29 Nov 2016 12:15:05 +0000 (13:15 +0100)]
Revert "rgw: rgw_obj encoding fixes"
This reverts commit
c1b64c8f9c23473b758300958a94a172f1d39b64.
Signed-off-by: Orit Wasserman <owasserm@redhat.com>
Sage Weil [Thu, 22 Dec 2016 18:05:22 +0000 (13:05 -0500)]
qa/tasks/workunit: clear clone dir before retrying checkout
If we checkout ceph-ci.git, and don't find a branch,
we'll try again from ceph.git. But the checkout will
already exist and the clone will fail, so we'll still
fail to find the branch.
The same can happen if a previous workunit task already
checked out the repo.
Fix by removing the repo before checkout (the first and
second times). Note that this may break if there are
multiple workunit tasks running in parallel on the same
role. That is already racy, so if it's happening, we'll
want to switch to using a truly unique clonedir for each
instantiation.
Fixes: http://tracker.ceph.com/issues/18336
Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit
2a7013cd5a033c5be43350505d75f088e831e201)
Sage Weil [Fri, 16 Dec 2016 20:06:16 +0000 (15:06 -0500)]
qa/tasks/workunit: retry on ceph.git if checkout fails
Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit
72d73b8c8836ae35c518fa09f44805a74038f02a)
Sage Weil [Wed, 7 Dec 2016 14:50:26 +0000 (09:50 -0500)]
mon/OSDMonitor: only show interesting flags in health warning
Also add PAUSEREC flag into the list.
Fixes: http://tracker.ceph.com/issues/18175
Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit
26220f0608f5ed4a7d97fb8d10d0d12a0fcf5583)
Conflicts:
src/mon/OSDMonitor.cc (hammer lacks CEPH_OSDMAP_FULL and order of flags
is slightly different)
Sage Weil [Mon, 19 Dec 2016 19:08:11 +0000 (14:08 -0500)]
qa/workunits: include extension for nose tests
When you have a relative path you have to include the extension.
Weird.
Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit
5666fd61d6dbd40be1d79354227cabd562e829ea)
Conflicts:
qa/workunits/fs/test_python.sh (not in hammer)
qa/workunits/rbd/test_librbd_python.sh (no valgrind in hammer)
Sage Weil [Thu, 15 Dec 2016 20:10:28 +0000 (15:10 -0500)]
qa/workunits: use relative path instead of wget from git
Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit
cf294777ea92f0911813a7132068584d4f73a65a)
Conflicts: (all trivial)
qa/workunits/fs/test_python.sh
qa/workunits/rados/test_python.sh
qa/workunits/rbd/rbd_mirror.sh
qa/workunits/rbd/rbd_mirror_stress.sh
qa/workunits/rbd/test_librbd_python.sh
qa/workunits/rbd/test_lock_fence.sh
Sage Weil [Thu, 15 Dec 2016 18:26:14 +0000 (13:26 -0500)]
qa/tasks/workunit.py: add CEPH_BASE env var
Root of git checkout
Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit
27b8eac24922f8b4bd065e6e7f0bc8e2ba37b5d5)
Sage Weil [Thu, 15 Dec 2016 18:25:23 +0000 (13:25 -0500)]
qa/tasks/workunit: leave workunits inside git checkout
Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit
4602884ab8f5a256d13091f7239d938990482d95)
Sage Weil [Wed, 14 Dec 2016 17:40:08 +0000 (11:40 -0600)]
Merge pull request #12455 from liewegas/qa-suite-hammer
hammer: merge ceph-qa-suite
Sage Weil [Wed, 14 Dec 2016 17:30:00 +0000 (11:30 -0600)]
merge ceph-qa-suite
Sage Weil [Wed, 14 Dec 2016 17:29:55 +0000 (11:29 -0600)]
move ceph-qa-suite dirs into qa/
Sage Weil [Wed, 14 Dec 2016 17:27:47 +0000 (12:27 -0500)]
Revert "tasks/workunit.py: depth 1 clone"
This reverts commit
12a707cb0f7b162e15c31915eb677bbd49b5e2cf.
Sage Weil [Wed, 14 Dec 2016 17:19:44 +0000 (12:19 -0500)]
tasks/workunit.py: depth 1 clone
Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit
4faf77a649cb3f8ddf497ca81937b3dbf63a18dc)
Sage Weil [Wed, 14 Dec 2016 17:18:29 +0000 (12:18 -0500)]
tasks/workunit: remove kludge to use git.ceph.com
This was hard-coded to ceph.git (almost) and breaks when
you specify --ceph-repo. Remove it entirely. We'll see if
github.com is better at handling our load than it used to
be!
Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit
159c455a0326eef2c017b3e3cf510f918b5ec76c)
Kefu Chai [Fri, 9 Dec 2016 18:36:52 +0000 (02:36 +0800)]
tasks/ceph: restore context of osd mount path before mkfs
all newly created files and directories under the mount dir inherit the
SELinux type of their parent directory. so we need to set it before
mkfs.
Fixes: http://tracker.ceph.com/issues/16800
Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit
53225d5272a1d35d4183fcfa55a139f55f77e122)
Sage Weil [Mon, 12 Dec 2016 22:42:55 +0000 (16:42 -0600)]
Merge pull request #1278 from ceph/wip-16225-min
hammer: SIGABRT in TrackedOp::dump() via dump_ops_in_flight()
Jason Dillaman [Mon, 12 Dec 2016 16:53:00 +0000 (11:53 -0500)]
librbd: diffs to clone's first snapshot should include parent diffs
If the clone has a backing object created after the snapshot that
overwrites an extent of the parent, the parent diffs within that
extent are not included in the result.
Hammer-specific implementation due to librbd refactoring during the
Infernalis release.
Fixes: http://tracker.ceph.com/issues/18111
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
Jason Dillaman [Wed, 7 Dec 2016 14:59:39 +0000 (09:59 -0500)]
qa/workunits/rbd: removed qemu-iotest case 077
The test case is not stable due to racing console output. This
results in spurious failures.
Fixes: http://tracker.ceph.com/issues/10773
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit
2c70df978d605a45ff81971b86f5afbefbdaabb6)
Yehuda Sadeh [Mon, 28 Nov 2016 22:20:03 +0000 (14:20 -0800)]
rgw: omap_get_all() fixes
Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
(cherry picked from commit
99e866feb3f8aa0ff768d6253979b6b4fc0fa452)
Sage Weil [Mon, 21 Nov 2016 23:37:39 +0000 (18:37 -0500)]
rgw/rgw_rados: do not omap_getvals with (u64)-1 max
Fixes: http://tracker.ceph.com/issues/17985
This will make the OSD fall over if the object is big
and the OSD doesn't impose it's own fail-safe limit.
Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit
7cd123ec0dd5951628e4f8ddd4e044a0fbb82f2f)
Wanlong Gao [Thu, 23 Jun 2016 12:04:25 +0000 (20:04 +0800)]
osd: limit omap data in push op
We already have the config osd_recovery_max_chunk to limit the total
size of omap entries and omap data. But we need an individual config
to limit the number of omap entries independently. We call this config
osd_recovery_max_omap_entries_per_chunk here with the default number
of 64000.
Signed-off-by: Wanlong Gao <wanlong.gao@easystack.cn>
(cherry picked from commit
56064c5cb645254574dbad00e6c16b783115bb93)
LiuYang [Thu, 8 Dec 2016 06:21:43 +0000 (14:21 +0800)]
rgw: do not abort when accept a CORS request with short origin
Fixed: #18187
when accept a CROS request, the request http origin shorter than the bucket's corsrule
(eg. origin: http://s.com corsrule: <AllowedOrigin>*.verylongdomain.com</AllowedOrigin>).
the rgw_cors.cc::is_string_in_set() will have a wrong index, the radosrgw server will
abort.
$ curl http://test.localhost:8000/app.data -H "Origin:http://s.com"
0> 2016-12-05 03:22:29.548138
7f6add05d700 -1 *** Caught signal (Aborted) **
in thread
7f6add05d700 thread_name:civetweb-worker
ceph version
11.0.2-2168-gd2f8fb4 (
d2f8fb4a6ba75af7e6da0f5a7f1b49ec998b1631)
1: (()+0x50720a) [0x7f6b147c420a]
2: (()+0xf370) [0x7f6b09a33370]
3: (gsignal()+0x37) [0x7f6b081ca1d7]
4: (abort()+0x148) [0x7f6b081cb8c8]
5: (__gnu_cxx::__verbose_terminate_handler()+0x165) [0x7f6b08ace9d5]
6: (()+0x5e946) [0x7f6b08acc946]
7: (()+0x5e973) [0x7f6b08acc973]
8: (()+0x5eb93) [0x7f6b08accb93]
9: (std::__throw_out_of_range(char const*)+0x77) 0x7f6b08b21a17]
10: (()+0xbd97a) [0x7f6b08b2b97a]
11: (()+0x449c1e) [0x7f6b14706c1e]
12: (RGWCORSRule::is_origin_present(char const*)+0x48) [0x7f6b147073b8]
13: (RGWCORSConfiguration::host_name_rule(char const*)+0x37) [0x7f6b147074e7]
14: (RGWOp::generate_cors_headers(std::string&, std::string&, std::string&, std::string&, unsigned int*)+0xa3) [0x7f6b14593e63]
15: (dump_access_control(req_state*, RGWOp*)+0x61) [0x7f6b14653f91]
Signed-off-by: LiuYang <yippeetry@gmail.com>
(cherry picked from commit
67d4d9e64bc224e047cf333e673bb22cd6290789)
YankunLi [Mon, 5 Dec 2016 06:39:20 +0000 (14:39 +0800)]
rbd: fix parameter check
using import-diff command, the imagename must be specified.
Signed-off-by: Yankun Li <lioveni99@gmail.com>
Sage Weil [Fri, 2 Dec 2016 20:09:32 +0000 (15:09 -0500)]
Merge pull request #1276 from ceph/no-btrfs-on-hammer
rados: remove btrfs tests; use xfs instead
Kefu Chai [Wed, 23 Nov 2016 06:48:11 +0000 (14:48 +0800)]
FileStore::_do_fiemap: do not reference fiemap after it is freed
`last` points to fiemap::fm_extends[n], and if fiemap gets freed, we can
not reference any of its fieldis. so we could remember the check result before
freeing it.
Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit
c3748fa7737b6c43de2dae5d957fcff01322515c)
Conflicts:
src/os/FileStore.cc
put the parameter is_last in the right place
Sage Weil [Wed, 30 Nov 2016 02:50:52 +0000 (21:50 -0500)]
msg/simple/Pipe: handle addr decode error
Fixes: http://tracker.ceph.com/issues/18072
Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit
2f5cfbc0de5464ab80581c416413a86c3d38dfb7)
Nathan Cutler [Tue, 29 Nov 2016 21:16:34 +0000 (22:16 +0100)]
Merge pull request #11457 from tchaikov/wip-17558-hammer
hammer: mon: send updated monmap to its subscribers
Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
Nathan Cutler [Tue, 29 Nov 2016 21:12:46 +0000 (22:12 +0100)]
Merge pull request #11628 from ceph/wip-hammer-11567
hammer: rgw: Don't loop forever when reading data from 0 sized segment.
Reviewed-by: Orit Wasserman <owasserm@redhat.com>
Jianpeng Ma [Wed, 11 Feb 2015 03:18:41 +0000 (11:18 +0800)]
osd: Add func has_flag in MOSDOp.
Signed-off-by: Jianpeng Ma <jianpeng.ma@intel.com>
(manually cherry picked just one line from
d9a2ca5c5ae50dc34e184d577c7b24c56b6cc6ad)
Sage Weil [Tue, 24 May 2016 18:02:32 +0000 (14:02 -0400)]
osd: reject PARALLELEXEC ops with EINVAL
Errors are better than crashing.
Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit
407495197dd878b62f1356f6d939e33ab50d78c6)
Conflicts:
src/osd/ReplicatedPG.cc (trivial resolution)
Sage Weil [Tue, 24 May 2016 18:02:14 +0000 (14:02 -0400)]
ceph_test_rados_api_misc: test rados op with bad flas
Pass the bad PARALLELEXEC flag to remove(), which takes a flags arg.
Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit
1aa807f4f29b4b016cc737ffa443e8f30c3b7693)
Conflicts:
src/test/librados/misc.cc (dropped ioctx.remove() test because there is no
remove() in hammer librados)
Nathan Cutler [Thu, 24 Nov 2016 08:15:50 +0000 (09:15 +0100)]
Merge pull request #10437 from odivlad/fix-doc
hammer: docs: Bucket object versions
Reviewed-by: Orit Wasserman <owasserm@redhat.com>
Marcus Watts [Fri, 14 Oct 2016 01:12:36 +0000 (21:12 -0400)]
Don't loop forever when reading data from 0 sized segment.
The 0 sized segment can arise depending on how
the client uploads the object in the first place.
The cpu loop then happens during a swift `GET'.
Signed-off-by: Marcus Watts <mwatts@redhat.com>
(cherry picked from commit
46c5f9773246522e66bb2cca49345d0b62a16c42)
David Zafman [Mon, 14 Mar 2016 20:28:39 +0000 (13:28 -0700)]
common/TrackedOp: Move tracking_enabled check into register_inflight_op()
Make tracking_enabled (and the lock private.)*
Signed-off-by: David Zafman <dzafman@redhat.com>
(Partial cherry picked from commit
4e8e6172e8f6cc2adc63a429c32f5edd03982f5e)
* Backport: Exclude locking since tracking_enabled can't change after launch
David Zafman [Fri, 11 Mar 2016 05:24:25 +0000 (21:24 -0800)]
common/TrackedOp: Handle dump racing with constructor
Use is_tracked to prevent TrackedOp::dump() from trying to call
virtual function while still in OpRequest constructor.
Fixes: #8885
Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit
ad13e05499669a79bde9a219ba1089f929e0388e)
Conflicts:
src/common/TrackedOp.cc
src/common/TrackedOp.h
Hammer can't use atomic<bool> so use atomic_t instead
David Zafman [Sat, 5 Mar 2016 02:43:43 +0000 (18:43 -0800)]
common/TrackedOp: Missed locking when examining "events"
Another crash caused by a dump_ops_in_flight similar to #8885
Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit
be098c83143226e502a0587c9215cff941dbace1)
Conflicts:
src/common/TrackedOp.h (trivial)
David Zafman [Tue, 12 Jul 2016 00:58:22 +0000 (17:58 -0700)]
CLEANUP: Move locking into dump_ops_in_flight()/dump_historic_ops()
Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit
17c645f70708134f50374cadb5505b0babe2265a)
Conflicts:
src/mds/MDSRank.cc (variation of code exists in mds/MDS.cc)
src/mon/Monitor.cc (no equivalent code here)
David Zafman [Mon, 14 Mar 2016 22:47:02 +0000 (15:47 -0700)]
mds, osd: Fix missing locking for "dump_blocked_ops"
Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit
be5bed840f21da998be3b8374a8e33f208258e1b)
Conflicts:
src/mds/MDSRank.cc (missing dump_blocked_ops in mds)
David Zafman [Fri, 11 Mar 2016 05:29:13 +0000 (21:29 -0800)]
osd: cleanup: Specify both template types for create_request()
Code consistency
Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit
cecdc49e6d2f38d5eb9f85218484f7a02bd2e1b7)
Jianpeng Ma [Fri, 17 Jul 2015 02:44:06 +0000 (10:44 +0800)]
osd: add dump_blocked_ops asok command.
From ceph health we can see there are blocked ops and from 'ceph health
detai' we can see which osd has blocked ops. Later we only in log of osd
to see the detail message of blocked ops.
Now via asok dump blocked ops.
Signed-off-by: Jianpeng Ma <jianpeng.ma@intel.com>
(cherry picked from commit
cf4e8c655c4a0e6c367c88338cd6e51a3474fa2f)
Conflicts:
src/common/TrackedOp.h (trivial)
Jianpeng Ma [Wed, 8 Jul 2015 05:12:49 +0000 (13:12 +0800)]
common/TrackedOp: Should lock ops_history_lock when access shutdown.
Signed-off-by: Jianpeng Ma <jianpeng.ma@intel.com>
(cherry picked from commit
7fe72a2e7d116093ccff6683666e6289f5f12421)
Nathan Cutler [Wed, 23 Nov 2016 21:50:58 +0000 (22:50 +0100)]
Merge pull request #9873 from odivlad/fix-init-el7-hammer
hammer: build/ops: remove SYSTEMD_RUN from initscript
Reviewed-by: Orit Wasserman <owasserm@redhat.com>
Reviewed-by: Nathan Cutler <ncutler@suse.com>
Nathan Cutler [Wed, 23 Nov 2016 21:49:29 +0000 (22:49 +0100)]
Merge pull request #10238 from aiicore/hammer
hammer: mon: fix memory leak in prepare_beacon
Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: John Spray <john.spray@redhat.com>
Nathan Cutler [Wed, 23 Nov 2016 21:44:19 +0000 (22:44 +0100)]
Merge pull request #10569 from jschmid1/wip-16918-hammer
hammer: build/ops: make stop.sh more portable
Reviewed-by: Nathan Cutler <ncutler@suse.com>
Nathan Cutler [Wed, 23 Nov 2016 21:40:28 +0000 (22:40 +0100)]
Merge pull request #10724 from Vicente-Cheng/wip-16584-hammer
hammer: crush: reset bucket->h.items[i] when removing tree item
Reviewed-by: Kefu Chai <kchai@redhat.com>
Nathan Cutler [Wed, 23 Nov 2016 21:20:29 +0000 (22:20 +0100)]
Merge pull request #10904 from tchaikov/wip-14470-hammer
hammer: mon: return size_t from MonitorDBStore::Transaction::size()
Reviewed-by: Joao Eduardo Luis <joao@suse.de>
Reviewed-by: Kefu Chai <kchai@redhat.com>
Nathan Cutler [Wed, 23 Nov 2016 21:18:42 +0000 (22:18 +0100)]
Merge pull request #10905 from SUSE/wip-11101-hammer
hammer: build/ops: improve ceph.in error message
Reviewed-by: Sage Weil <sage@redhat.com>
Nathan Cutler [Wed, 23 Nov 2016 21:16:48 +0000 (22:16 +0100)]
Merge pull request #10987 from Vicente-Cheng/wip-16592-hammer
hammer: librbd: ceph 10.2.2 rbd status on image format 2 returns "(2) No such file or directory"
Reviewed-by: Jason Dillaman <dillaman@redhat.com>
Nathan Cutler [Wed, 23 Nov 2016 21:13:05 +0000 (22:13 +0100)]
Merge pull request #11125 from tchaikov/wip-17179-hammer
hammer: tools: add a tool to rebuild mon store from OSD
Reviewed-by: Nathan Cutler <ncutler@suse.com>
Nathan Cutler [Wed, 23 Nov 2016 21:10:54 +0000 (22:10 +0100)]
Merge pull request #11273 from aiicore/wip-17403-hammer
hammer: mon: OSDMonitor: Missing nearfull flag set
Reviewed-by: Sage Weil <sage@redhat.com>
Nathan Cutler [Wed, 23 Nov 2016 21:06:32 +0000 (22:06 +0100)]
Merge pull request #11618 from asheplyakov/hammer-16546
hammer: rbd: fix possible rbd data corruption
Reviewed-by: Jason Dillaman <dillaman@redhat.com>
Nathan Cutler [Wed, 23 Nov 2016 20:56:22 +0000 (21:56 +0100)]
Merge pull request #11676 from asheplyakov/hammer-16151
hammer: osd: update PGPool to detect map gaps and reset cached_removed_snaps
Reviewed-by: Samuel Just <sjust@redhat.com>
Nathan Cutler [Wed, 23 Nov 2016 20:54:39 +0000 (21:54 +0100)]
Merge pull request #11899 from SUSE/wip-17840-hammer
hammer: rgw: the value of total_time is wrong in the result of 'radosgw-admin log show' opt
Reviewed-by: Orit Wasserman <owasserm@redhat.com>