]> git.apps.os.sepia.ceph.com Git - ceph.git/log
ceph.git
8 years agoMerge pull request #16191 from smithfarm/wip-20028-kraken
Nathan Cutler [Wed, 19 Jul 2017 13:43:16 +0000 (15:43 +0200)]
Merge pull request #16191 from smithfarm/wip-20028-kraken

kraken: cephfs: Deadlock on two ceph-fuse clients accessing the same file

Reviewed-by: Yan, Zheng <zyan@redhat.com>
8 years agoMerge pull request #16183 from smithfarm/wip-20405-kraken
Nathan Cutler [Wed, 19 Jul 2017 13:32:52 +0000 (15:32 +0200)]
Merge pull request #16183 from smithfarm/wip-20405-kraken

kraken: rgw: Lifecycle thread will still handle the bucket even if it has been removed

Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: Daniel Gryniewicz <dang@redhat.com>
8 years agoMerge pull request #16181 from smithfarm/wip-20363-kraken
Nathan Cutler [Wed, 19 Jul 2017 13:31:50 +0000 (15:31 +0200)]
Merge pull request #16181 from smithfarm/wip-20363-kraken

kraken: rgw: VersionIdMarker and NextVersionIdMarker are not returned when listing object versions

Reviewed-by: Casey Bodley <cbodley@redhat.com>
8 years agoMerge pull request #16179 from smithfarm/wip-20269-kraken
Nathan Cutler [Wed, 19 Jul 2017 13:31:14 +0000 (15:31 +0200)]
Merge pull request #16179 from smithfarm/wip-20269-kraken

kraken: rgw: wrong object size after copy of uncompressed multipart objects

Reviewed-by: Casey Bodley <cbodley@redhat.com>
8 years agoMerge pull request #16174 from smithfarm/wip-20261-kraken
Nathan Cutler [Wed, 19 Jul 2017 13:30:33 +0000 (15:30 +0200)]
Merge pull request #16174 from smithfarm/wip-20261-kraken

kraken: rgw: 'radosgw-admin usage show' listing 0 bytes_sent/received

Reviewed-by: Casey Bodley <cbodley@redhat.com>
8 years agoMerge pull request #16173 from smithfarm/wip-20156-kraken
Nathan Cutler [Wed, 19 Jul 2017 13:29:57 +0000 (15:29 +0200)]
Merge pull request #16173 from smithfarm/wip-20156-kraken

kraken: fix: rgw crashed caused by shard id out of range when listing data log

Reviewed-by: Casey Bodley <cbodley@redhat.com>
8 years agoMerge pull request #16165 from smithfarm/wip-19839-kraken
Nathan Cutler [Wed, 19 Jul 2017 13:29:22 +0000 (15:29 +0200)]
Merge pull request #16165 from smithfarm/wip-19839-kraken

kraken: rgw: reduce log level of 'storing entry at' in cls_log

Reviewed-by: Casey Bodley <cbodley@redhat.com>
8 years agoMerge pull request #16163 from smithfarm/wip-19766-kraken
Nathan Cutler [Wed, 19 Jul 2017 13:28:30 +0000 (15:28 +0200)]
Merge pull request #16163 from smithfarm/wip-19766-kraken

kraken: rgw: when uploading the objects continuesly in the versioned bucket, some objects will not sync

Reviewed-by: Casey Bodley <cbodley@redhat.com>
8 years agoMerge pull request #16162 from smithfarm/wip-19725-kraken
Nathan Cutler [Wed, 19 Jul 2017 13:27:38 +0000 (15:27 +0200)]
Merge pull request #16162 from smithfarm/wip-19725-kraken

kraken: rgw: S3 v4 authentication issue with X-Amz-Expires

Reviewed-by: Casey Bodley <cbodley@redhat.com>
8 years agoMerge pull request #16161 from smithfarm/wip-19614-kraken
Nathan Cutler [Wed, 19 Jul 2017 13:25:48 +0000 (15:25 +0200)]
Merge pull request #16161 from smithfarm/wip-19614-kraken

kraken: multisite: rest api fails to decode large period on 'period commit'

Reviewed-by: Casey Bodley <cbodley@redhat.com>
8 years agoMerge pull request #16139 from smithfarm/wip-20147-kraken
Nathan Cutler [Wed, 19 Jul 2017 13:24:59 +0000 (15:24 +0200)]
Merge pull request #16139 from smithfarm/wip-20147-kraken

kraken: rgw: 'gc list --include-all' command infinite loop the first 1000 items

Reviewed-by: Casey Bodley <cbodley@redhat.com>
8 years agoMerge pull request #16096 from smithfarm/wip-19336-kraken
Nathan Cutler [Wed, 19 Jul 2017 12:29:31 +0000 (14:29 +0200)]
Merge pull request #16096 from smithfarm/wip-19336-kraken

kraken: rbd: refuse to use an ec pool that doesn't support overwrites

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
8 years agoMerge pull request #16097 from smithfarm/wip-19609-kraken
Nathan Cutler [Wed, 19 Jul 2017 12:28:08 +0000 (14:28 +0200)]
Merge pull request #16097 from smithfarm/wip-19609-kraken

kraken: tests: [librados_test_stub] cls_cxx_map_get_XYZ methods don't return correct value

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
8 years agoMerge pull request #16184 from smithfarm/wip-20154-kraken
Nathan Cutler [Wed, 19 Jul 2017 12:27:11 +0000 (14:27 +0200)]
Merge pull request #16184 from smithfarm/wip-20154-kraken

kraken: rbd: Potential IO hang if image is flattened while read request is in-flight

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
8 years agoMerge pull request #16187 from smithfarm/wip-20266-kraken
Nathan Cutler [Wed, 19 Jul 2017 12:24:05 +0000 (14:24 +0200)]
Merge pull request #16187 from smithfarm/wip-20266-kraken

kraken: rbd: [api] is_exclusive_lock_owner shouldn't return -EBUSY

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
8 years agoMerge pull request #16195 from dillaman/wip-20351-kraken
Nathan Cutler [Wed, 19 Jul 2017 12:22:57 +0000 (14:22 +0200)]
Merge pull request #16195 from dillaman/wip-20351-kraken

kraken: tests: test/librbd: decouple ceph_test_librbd_api from libceph-common

Reviewed-by: Nathan Cutler <ncutler@suse.com>
8 years agoMerge pull request #15486 from dillaman/wip-20022-kraken
Nathan Cutler [Wed, 19 Jul 2017 10:56:47 +0000 (12:56 +0200)]
Merge pull request #15486 from dillaman/wip-20022-kraken

kraken: rbd-mirror: ensure missing images are re-synced when detected

Reviewed-by: Mykola Golub <mgolub@mirantis.com>
8 years agoMerge pull request #14540 from smithfarm/wip-18910-kraken
Nathan Cutler [Wed, 19 Jul 2017 10:50:22 +0000 (12:50 +0200)]
Merge pull request #14540 from smithfarm/wip-18910-kraken

kraken: rbd-nbd: check /sys/block/nbdX/size to ensure kernel mapped correctly

Reviewed-by: Mykola Golub <mgolub@mirantis.com>
8 years agoMerge pull request #16140 from smithfarm/wip-20271-kraken
Nathan Cutler [Wed, 19 Jul 2017 06:05:11 +0000 (08:05 +0200)]
Merge pull request #16140 from smithfarm/wip-20271-kraken

kraken: tests: LibRadosMiscConnectFailure.ConnectFailure hang

Reviewed-by: Kefu Chai <kchai@redhat.com>
8 years agoMerge pull request #16112 from smithfarm/wip-20499-kraken
Nathan Cutler [Tue, 18 Jul 2017 21:14:59 +0000 (23:14 +0200)]
Merge pull request #16112 from smithfarm/wip-20499-kraken

kraken: tests: ObjectStore/StoreTest.OnodeSizeTracking/2 fails on bluestore

Reviewed-by: Sage Weil <sage@redhat.com>
8 years agoMerge pull request #14960 from yehudasa/wip-19704
Yehuda Sadeh [Tue, 18 Jul 2017 20:58:14 +0000 (13:58 -0700)]
Merge pull request #14960 from yehudasa/wip-19704

kraken: civetweb: move to post 1.8 version

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>
8 years agoMerge pull request #16143 from smithfarm/wip-20365-kraken
Nathan Cutler [Tue, 18 Jul 2017 17:50:46 +0000 (19:50 +0200)]
Merge pull request #16143 from smithfarm/wip-20365-kraken

kraken: mon: osd crush set crushmap need sanity check

Reviewed-by: Loic Dachary <ldachary@redhat.com>
8 years agoMerge pull request #16138 from smithfarm/wip-20034-kraken
Nathan Cutler [Tue, 18 Jul 2017 17:48:51 +0000 (19:48 +0200)]
Merge pull request #16138 from smithfarm/wip-20034-kraken

kraken: ceph-disk: Racing between partition creation & device node creation

Reviewed-by: Loic Dachary <ldachary@redhat.com>
8 years agoMerge pull request #16135 from smithfarm/wip-20010-kraken
Nathan Cutler [Tue, 18 Jul 2017 17:47:34 +0000 (19:47 +0200)]
Merge pull request #16135 from smithfarm/wip-20010-kraken

kraken: ceph-disk: separate ceph-osd --check-needs-* logs

Reviewed-by: Loic Dachary <ldachary@redhat.com>
8 years agoMerge pull request #16215 from liewegas/wip-17743
Sage Weil [Sun, 9 Jul 2017 02:39:59 +0000 (21:39 -0500)]
Merge pull request #16215 from liewegas/wip-17743

kraken: CMakeLists.txt: disable memstore make check test

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
8 years agoCMakeLists.txt: disable memstore make check test 16215/head
Sage Weil [Fri, 7 Jul 2017 15:38:25 +0000 (11:38 -0400)]
CMakeLists.txt: disable memstore make check test

This fails frequently on kraken, but is probably a wontfix.

See http://tracker.ceph.com/issues/17743

Signed-off-by: Sage Weil <sage@redhat.com>
8 years agorbd-mirror: ensure missing images are re-synced when detected 15486/head
Jason Dillaman [Thu, 4 May 2017 01:36:21 +0000 (21:36 -0400)]
rbd-mirror: ensure missing images are re-synced when detected

Fixes: http://tracker.ceph.com/issues/19811
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 74bd4f230a0cb7b709f2cb5c6db3dc79f0d8dede)

Conflicts:
src/test/rbd_mirror/image_replayer/test_mock_BootstrapRequest.cc: trivial resolution
src/tools/rbd_mirror/image_replayer/BootstrapRequest.h: trivial resolution

8 years agoMerge pull request #14067 from asheplyakov/19322-bp-kraken
Nathan Cutler [Fri, 7 Jul 2017 07:49:51 +0000 (09:49 +0200)]
Merge pull request #14067 from asheplyakov/19322-bp-kraken

kraken: rgw: fix break inside of yield in RGWFetchAllMetaCR

Reviewed-by: Orit Wasserman <owasserm@redhat.com>
8 years agoMerge pull request #14509 from smithfarm/wip-18499-kraken
Nathan Cutler [Fri, 7 Jul 2017 07:48:57 +0000 (09:48 +0200)]
Merge pull request #14509 from smithfarm/wip-18499-kraken

kraken: rgw: Realm set does not create a new period

Reviewed-by: Orit Wasserman <owasserm@redhat.com>
8 years agoMerge pull request #14511 from smithfarm/wip-18772-kraken
Nathan Cutler [Fri, 7 Jul 2017 07:48:01 +0000 (09:48 +0200)]
Merge pull request #14511 from smithfarm/wip-18772-kraken

kraken: rgw: crash when updating period with placement group

Reviewed-by: Orit Wasserman <owasserm@redhat.com>
8 years agoMerge pull request #14513 from smithfarm/wip-18843-kraken
Nathan Cutler [Fri, 7 Jul 2017 07:45:15 +0000 (09:45 +0200)]
Merge pull request #14513 from smithfarm/wip-18843-kraken

kraken: rgw: usage stats and quota are not operational for multi-tenant users

Reviewed-by: Orit Wasserman <owasserm@redhat.com>
8 years agoMerge pull request #14515 from smithfarm/wip-18904-kraken
Nathan Cutler [Fri, 7 Jul 2017 07:44:21 +0000 (09:44 +0200)]
Merge pull request #14515 from smithfarm/wip-18904-kraken

kraken: rgw: first write also tries to read object

Reviewed-by: Orit Wasserman <owasserm@redhat.com>
8 years agoMerge pull request #14516 from smithfarm/wip-18909-kraken
Nathan Cutler [Fri, 7 Jul 2017 07:42:36 +0000 (09:42 +0200)]
Merge pull request #14516 from smithfarm/wip-18909-kraken

kraken: rgw: the swift container acl does not support field .ref

Reviewed-by: Orit Wasserman <owasserm@redhat.com>
Reviewed-by: Radoslaw Zarzynski <rzarzynski@mirantis.com>
8 years agoMerge pull request #14517 from smithfarm/wip-19047-kraken
Nathan Cutler [Fri, 7 Jul 2017 07:25:43 +0000 (09:25 +0200)]
Merge pull request #14517 from smithfarm/wip-19047-kraken

kraken: rgw: fix use of marker in List::list_objects()

Reviewed-by: Orit Wasserman <owasserm@redhat.com>
8 years agoMerge pull request #14524 from smithfarm/wip-19477-kraken
Nathan Cutler [Fri, 7 Jul 2017 07:24:13 +0000 (09:24 +0200)]
Merge pull request #14524 from smithfarm/wip-19477-kraken

kraken: rgw: S3 v4 authentication issue with X-Amz-Expires

Reviewed-by: Orit Wasserman <owasserm@redhat.com>
Reviewed-by: Radoslaw Zarzynski <rzarzynski@mirantis.com>
8 years agoMerge pull request #14525 from smithfarm/wip-19479-kraken
Nathan Cutler [Fri, 7 Jul 2017 07:22:26 +0000 (09:22 +0200)]
Merge pull request #14525 from smithfarm/wip-19479-kraken

kraken: rgw: zonegroupmap set does not work

Reviewed-by: Orit Wasserman <owasserm@redhat.com>
8 years agoMerge pull request #14528 from smithfarm/wip-19534-kraken
Nathan Cutler [Fri, 7 Jul 2017 07:21:22 +0000 (09:21 +0200)]
Merge pull request #14528 from smithfarm/wip-19534-kraken

kraken: rgw: Error parsing xml when get bucket lifecycle

Reviewed-by: Daniel Gryniewicz <dang@redhat.com>
Reviewed-by: Orit Wasserman <owasserm@redhat.com>
8 years agoMerge pull request #14606 from asheplyakov/19608-kraken
Nathan Cutler [Fri, 7 Jul 2017 07:19:08 +0000 (09:19 +0200)]
Merge pull request #14606 from asheplyakov/19608-kraken

kraken: rgw: multisite: fetch_remote_obj() gets wrong version when copying from remote

Reviewed-by: Orit Wasserman <owasserm@redhat.com>
8 years agoMerge pull request #15384 from asheplyakov/20015-bp-kraken
Nathan Cutler [Fri, 7 Jul 2017 07:18:10 +0000 (09:18 +0200)]
Merge pull request #15384 from asheplyakov/20015-bp-kraken

kraken: cls/rgw: list_plain_entries() stops before bi_log entries

Reviewed-by: Orit Wasserman <owasserm@redhat.com>
8 years agoMerge pull request #14522 from smithfarm/wip-19472-kraken
Nathan Cutler [Fri, 7 Jul 2017 07:03:16 +0000 (09:03 +0200)]
Merge pull request #14522 from smithfarm/wip-19472-kraken

kraken: rgw: add the remove-x-delete feature to cancel swift object expiration

Reviewed-by: Radoslaw Zarzynski <rzarzynski@mirantis.com>
8 years agoMerge pull request #14523 from smithfarm/wip-19475-kraken
Nathan Cutler [Fri, 7 Jul 2017 07:00:43 +0000 (09:00 +0200)]
Merge pull request #14523 from smithfarm/wip-19475-kraken

kraken: rgw: multisite: EPERM when trying to read SLO objects as system/admin user

Reviewed-by: Radoslaw Zarzynski <rzarzynski@mirantis.com>
8 years agoMerge pull request #14519 from smithfarm/wip-19175-kraken
Nathan Cutler [Fri, 7 Jul 2017 06:58:44 +0000 (08:58 +0200)]
Merge pull request #14519 from smithfarm/wip-19175-kraken

kraken: swift API: cannot disable object versioning with empty X-Versions-Location

Reviewed-by: Radoslaw Zarzynski <rzarzynski@mirantis.com>
8 years agoMerge pull request #14526 from smithfarm/wip-19524-kraken
Nathan Cutler [Fri, 7 Jul 2017 06:56:47 +0000 (08:56 +0200)]
Merge pull request #14526 from smithfarm/wip-19524-kraken

kraken: rgw: 'radosgw-admin zone create' command with specified zone-id creates a zone with different id

Reviewed-by: Radoslaw Zarzynski <rzarzynski@mirantis.com>
8 years agoMerge pull request #14529 from smithfarm/wip-19573-kraken
Nathan Cutler [Fri, 7 Jul 2017 06:55:30 +0000 (08:55 +0200)]
Merge pull request #14529 from smithfarm/wip-19573-kraken

kraken: rgw: Response header of swift API returned by radosgw does not contain x-openstack-request-id. But Swift returns it.

Reviewed-by: Radoslaw Zarzynski <rzarzynski@mirantis.com>
8 years agotest/librbd: decouple ceph_test_librbd_api from libceph-common 16195/head
Kefu Chai [Sat, 10 Jun 2017 15:59:19 +0000 (23:59 +0800)]
test/librbd: decouple ceph_test_librbd_api from libceph-common

Fixes: http://tracker.ceph.com/issues/20175
Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit b7287fdc4d70c5ecedda78ae367b98e5d8f61c5b)

Conflicts:
src/test/librbd/CMakeLists.txt: trivial resolution

8 years agotest/librbd: replace libcommon classes using standard library
Kefu Chai [Sat, 10 Jun 2017 15:31:53 +0000 (23:31 +0800)]
test/librbd: replace libcommon classes using standard library

so ceph_test_librbd_api has less dependencies on libcommon

Fixes: http://tracker.ceph.com/issues/20175
Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit bc8fa0193c8b2fb4b749ce9976bf6efbea833e96)

Conflicts:
src/test/librbd/test_librbd.cc: trivial resolution

8 years agotest/librados: extract functions using libcommon in test.cc into test_common.cc
Kefu Chai [Sat, 10 Jun 2017 15:58:47 +0000 (23:58 +0800)]
test/librados: extract functions using libcommon in test.cc into test_common.cc

Fixes: http://tracker.ceph.com/issues/20175
Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit 82a848c2053ea69ebc6d3ec1003e18921e2d08d2)

Conflicts:
src/test/librados/CMakeLists.txt: trivial resolution
src/test/librados/test.cc: trivial resolution

8 years agomds: issue new caps when sending reply to client 16191/head
Yan, Zheng [Sat, 22 Apr 2017 04:27:12 +0000 (12:27 +0800)]
mds: issue new caps when sending reply to client

After Locker::issue_new_caps() adds new Capability data struct,
do not issue caps immediately. Let CInode::encode_inodestate()
do the job instead. This can avoid various races that early reply
is not allowed, caps that haven't been sent to client gets revoked.

Fixes: http://tracker.ceph.com/issues/19635
Signed-off-by: "Yan, Zheng" <zyan@redhat.com>
(cherry picked from commit 799703a4acb49db0b6cc99a23e4326767e694c3a)

8 years agolibrbd: filter expected error codes from is_exclusive_lock_owner 16187/head
Jason Dillaman [Mon, 5 Jun 2017 12:17:05 +0000 (08:17 -0400)]
librbd: filter expected error codes from is_exclusive_lock_owner

Fixes: http://tracker.ceph.com/issues/20182
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit d4daaf54e6bc42cd4fb2111ea20b2042941b0c31)

8 years agorgw: only log metadata on metadata master zone 16161/head
Casey Bodley [Sat, 10 Jun 2017 22:12:52 +0000 (18:12 -0400)]
rgw: only log metadata on metadata master zone

Fixes: http://tracker.ceph.com/issues/20244
Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit b8272f3607074a2f7cbfd08f7bbc82f22cf120ba)

8 years agoradosgw-admin: warn that 'realm rename' does not update other clusters
Casey Bodley [Fri, 21 Apr 2017 19:04:48 +0000 (15:04 -0400)]
radosgw-admin: warn that 'realm rename' does not update other clusters

Fixes: http://tracker.ceph.com/issues/19746
Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit 8a459c768ffecd689a53a79dfe33eb8f1bbc318f)

8 years agorgw: fix for zonegroup redirect url
Casey Bodley [Tue, 4 Apr 2017 14:42:44 +0000 (10:42 -0400)]
rgw: fix for zonegroup redirect url

local dest_url variable was shadowing the one in the enclosing scope, so
the changes were not applied and no Location header was written on redirect

Fixes: http://tracker.ceph.com/issues/19488
Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit 542e188a40f0495720b48308372366951ae41e62)

8 years agorgw: use zonegroup's master zone endpoints for bucket redirect
Casey Bodley [Tue, 4 Apr 2017 14:41:51 +0000 (10:41 -0400)]
rgw: use zonegroup's master zone endpoints for bucket redirect

if no zonegroup endpoints are set, fall back to master zone endpoints

Fixes: http://tracker.ceph.com/issues/19488
Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit 92f63c6392bdc4633a2e57cb3867051bb1a3fd55)

8 years agorgw: allow larger payload for period commit
Casey Bodley [Wed, 5 Apr 2017 18:20:20 +0000 (14:20 -0400)]
rgw: allow larger payload for period commit

testing with 3 zonegroups and 3 zones each, the period json grew larger
than 4k and caused decode failures on period commit

updated to use the new config variable rgw_max_put_param_size

Fixes: http://tracker.ceph.com/issues/19505
Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit 7f2871fe59d933b03f37fde40f1781b2320d0d50)

8 years agorgw: make a configurable size for requests with xml params
Abhishek Lekshmanan [Fri, 3 Mar 2017 15:50:33 +0000 (16:50 +0100)]
rgw: make a configurable size for requests with xml params

We currently read all user input for a few apis accepting xml, avoid
doing this and error out early, most s3 apis limit to about 1000 xml
entries, make this configurable via a new config param,
`rgw_max_put_param_size` defaulting to 1MB. Also modified
`rgw_rest_read_all_input` with an additional param to not allow for
chunked uploads, though we error out in a better way by responding with
content length required rather than S3's 501 if chunked is set.

This also adds the same behavior in RGWPutCORS reusing
`rgw_rest_read_all_input`, and using a data_deleter to manage the char*

Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>
rgw: rgw_rest_s3: make PUT CORS accept only a configured max input

Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>
(cherry picked from commit 35375b75160dc93beeb04c7fe010dc0d21952534)

8 years agolibrbd: potential read IO hang when image is flattened 16184/head
Mykola Golub [Tue, 23 May 2017 10:07:45 +0000 (12:07 +0200)]
librbd: potential read IO hang when image is flattened

Fixes: http://tracker.ceph.com/issues/19832
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 10d58618e7c632ef01b9537492239e0a042dc17e)

8 years agorgw: lifecycle thread shouldn't process the bucket which has been deleted. 16183/head
Zhang Shaowen [Wed, 14 Jun 2017 02:29:53 +0000 (10:29 +0800)]
rgw: lifecycle thread shouldn't process the bucket which has been deleted.

Fixes: http://tracker.ceph.com/issues/20285
Signed-off-by: Zhang Shaowen <zhangshaowen@cmss.chinamobile.com>
(cherry picked from commit a2b042fe9f7e6503273fa4c2a4c56e399d7c338f)

8 years agorgw: VersionIdMarker and NextVersionIdMarker should be returned when listing 16181/head
Zhang Shaowen [Tue, 9 May 2017 08:39:39 +0000 (16:39 +0800)]
rgw: VersionIdMarker and NextVersionIdMarker should be returned when listing
object versions if necessary.

Fixes: http://tracker.ceph.com/issues/19886
Signed-off-by: Zhang Shaowen <zhangshaowen@cmss.chinamobile.com>
(cherry picked from commit f805c3e08948e379b7d2c4f2faf9e7f550e4cb23)

8 years agorgw: set object accounted size correctly 16179/head
fang.yuxiang [Thu, 4 May 2017 07:58:37 +0000 (15:58 +0800)]
rgw: set object accounted size correctly

sometimes, object accounted size is set wrong,
because we don't konw the object size if don't resort to the compression info or manifest.
e.g, when i use s3cmd do copy object(bucket_A/obj_A -> bucket_B/obj_B, assume the size of obj_A is 4M).
then i use s3cmd do list bucket, I got obj_B size is 512K, it is the head size apparently.

Fixes: http://tracker.ceph.com/issues/20071
Signed-off-by: fang yuxiang <fang.yuxiang@eisoo.com>
(cherry picked from commit 539985a99eebdc72c8d2446acc1108664a162f68)

8 years agoMerge pull request #16113 from smithfarm/wip-19807-kraken
Nathan Cutler [Thu, 6 Jul 2017 15:46:06 +0000 (17:46 +0200)]
Merge pull request #16113 from smithfarm/wip-19807-kraken

kraken: tests: remove hard-coded image name from TestLibRBD.Mirror

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
Reviewed-by: Mykola Golub <mgolub@mirantis.com>
8 years agorgw: Added code to correctly account for bytes sent/ received during a 'PUT' operation. 16174/head
Pritha Srivastava [Mon, 20 Mar 2017 06:11:48 +0000 (11:41 +0530)]
rgw: Added code to correctly account for bytes sent/ received during a 'PUT' operation.

Currently, the bytes sent/ received are both set to zero after
an object is uploaded to a bucket. Added code to correct the logic.

Signed-off-by: Pritha Srivastava <prsrivas@redhat.com>
(cherry picked from commit 85735c972f5db1a110f55e01a4f6249202152553)

8 years agorgw: segment fault when shard id out of range 16173/head
redickwang [Fri, 19 May 2017 07:08:12 +0000 (15:08 +0800)]
rgw: segment fault when shard id out of range

Fixes: http://tracker.ceph.com/issues/19732
Signed-off-by: redickwang <redickwang@tencent.com>
(cherry picked from commit ff4c40fc2e3c092d17a218ae6132de4e6034c8ee)

8 years agocls/log/cls_log.cc: reduce logging noise 16165/head
Willem Jan Withagen [Sat, 29 Apr 2017 11:36:07 +0000 (13:36 +0200)]
cls/log/cls_log.cc: reduce logging noise

 - The other reference in the source as already at 20.
      ./src/cls/timeindex/cls_timeindex.cc:85:
CLS_LOG(20, "storing entry at %s", index.c_str());

   And we need not always know where in the log items are stored.
   So it looks like a leftover debug feature.

Fixes: http://tracker.ceph.com/issues/19835
Signed-off-by: Willem Jan Withagen <wjw@digiware.nl>
(cherry picked from commit d76010900bf9012f2e66335787710531772766b7)

8 years agorgw: fix versioned bucket data sync fail when upload is busy 16163/head
lvshuhua [Wed, 7 Dec 2016 07:47:47 +0000 (15:47 +0800)]
rgw: fix versioned bucket data sync fail when upload is busy

Fixes: http://tracker.ceph.com/issues/18208
Signed-off-by: lvshuhua <lvshuhua@cmss.chinamobile.com>
(cherry picked from commit ce7d00ac1642d84c1d9111156a544c37801c5adf)

8 years agorgw: don't return skew time in pre-signed url 16162/head
liuchang0812 [Fri, 10 Feb 2017 10:02:03 +0000 (18:02 +0800)]
rgw: don't return skew time in pre-signed url

Fixes: http://tracker.ceph.com/issues/18828
Signed-off-by: liuchang0812 <liuchang0812@gmail.com>
(cherry picked from commit dd8b348f4aad0124e8a4457117bf3f5f76af7bdb)

8 years agoMerge pull request #16132 from smithfarm/wip-20522-kraken
Nathan Cutler [Thu, 6 Jul 2017 07:28:26 +0000 (09:28 +0200)]
Merge pull request #16132 from smithfarm/wip-20522-kraken

kraken: FAILED assert(object_contexts.empty()) (live on master only from Jan-Feb 2017, all other instances are different)

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
8 years agoMerge pull request #15962 from asheplyakov/20443-bp-kraken
Nathan Cutler [Thu, 6 Jul 2017 07:27:37 +0000 (09:27 +0200)]
Merge pull request #15962 from asheplyakov/20443-bp-kraken

kraken: osd: unlock sdata_op_ordering_lock with sdata_lock hold to avoid missing wakeup signal

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
8 years agoMerge pull request #15066 from asheplyakov/19916-bp-kraken
Nathan Cutler [Thu, 6 Jul 2017 07:26:40 +0000 (09:26 +0200)]
Merge pull request #15066 from asheplyakov/19916-bp-kraken

kraken: osd: osd/PrimaryLogPG: do not call on_shutdown() if (pg.deleting)

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
8 years agoMerge pull request #14942 from shinobu-x/wip-18293-kraken
Nathan Cutler [Thu, 6 Jul 2017 07:24:26 +0000 (09:24 +0200)]
Merge pull request #14942 from shinobu-x/wip-18293-kraken

kraken: osd: leaked MOSDMap

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>
8 years agoMerge pull request #14736 from smithfarm/wip-19326-kraken
Nathan Cutler [Thu, 6 Jul 2017 07:20:56 +0000 (09:20 +0200)]
Merge pull request #14736 from smithfarm/wip-19326-kraken

kraken: bluestore bdev: flush no-op optimization is racy

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
8 years agoMerge pull request #14732 from smithfarm/wip-19560-kraken
Nathan Cutler [Thu, 6 Jul 2017 07:19:50 +0000 (09:19 +0200)]
Merge pull request #14732 from smithfarm/wip-19560-kraken

kraken: objecter: full_try behavior not consistent with osd

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
8 years agoMerge pull request #13542 from shinobu-x/wip-18929-kraken
Nathan Cutler [Thu, 6 Jul 2017 07:18:24 +0000 (09:18 +0200)]
Merge pull request #13542 from shinobu-x/wip-18929-kraken

 kraken: osd: bogus assert when checking acting set on recovery completion in rados/upgrade

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
8 years agoMerge pull request #14939 from rzarzynski/wip-rgw-19754-kraken
Nathan Cutler [Thu, 6 Jul 2017 07:11:09 +0000 (09:11 +0200)]
Merge pull request #14939 from rzarzynski/wip-rgw-19754-kraken

kraken: rgw: fix RadosGW hang during multi-chunk upload of AWSv4

Reviewed-by: Matt Benjamin <mbenjamin@redhat.com>
8 years agoMerge pull request #14530 from smithfarm/wip-19574-kraken
Nathan Cutler [Thu, 6 Jul 2017 06:54:16 +0000 (08:54 +0200)]
Merge pull request #14530 from smithfarm/wip-19574-kraken

kraken: rgw: unsafe access in RGWListBucket_ObjStore_SWIFT::send_response()

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>
8 years agoMerge pull request #13868 from smithfarm/wip-19160-kraken
Nathan Cutler [Thu, 6 Jul 2017 06:40:16 +0000 (08:40 +0200)]
Merge pull request #13868 from smithfarm/wip-19160-kraken

kraken: rgw: multisite: RGWMetaSyncShardControlCR gives up on EIO

Reviewed-by: Orit Wasserman <owasserm@redhat.com>
8 years agomon: osd crush set crushmap need sanity check 16143/head
Loic Dachary [Sat, 18 Mar 2017 09:04:20 +0000 (10:04 +0100)]
mon: osd crush set crushmap need sanity check

The sanity check verifying the new crushmap does not remove crush rules
that are in use is not exclusive to ceph setcrushmap.

Fixes: http://tracker.ceph.com/issues/19302
Signed-off-by: Loic Dachary <loic@dachary.org>
(cherry picked from commit ed760457bf154c10adf75c6df046eecab7eb8e4b)

8 years agoceph_test_rados_api_misc: fix LibRadosMiscConnectFailure.ConnectFailure retry 16140/head
Sage Weil [Tue, 6 Jun 2017 19:13:50 +0000 (15:13 -0400)]
ceph_test_rados_api_misc: fix LibRadosMiscConnectFailure.ConnectFailure retry

Fixes: http://tracker.ceph.com/issues/19901
Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 087dff80cac707ee9bcb5bcfc98cb3ec047bd49f)

8 years agorgw: fix 'gc list --include-all' command infinite loop the first 1000 items 16139/head
lu.shasha [Thu, 5 Jan 2017 03:50:42 +0000 (11:50 +0800)]
rgw: fix 'gc list --include-all' command infinite loop the first 1000 items

When the items to gc over 1000, 'gc list --include-all' command will infinite loop the first 1000 items.
Add next_marker to move to the next 1000 items.

Fixes: http://tracker.ceph.com/issues/19978
Signed-off-by: fang yuxiang <fang.yuxiang@eisoo.com>
Signed-off-by: Shasha Lu <lu.shasha@eisoo.com>
(cherry picked from commit fc29f52ebca63104a05515484088ff136dfb0b15)

8 years agoceph-disk: Adding retry loop in get_partition_dev() 16138/head
Erwan Velu [Fri, 31 Mar 2017 12:54:33 +0000 (14:54 +0200)]
ceph-disk: Adding retry loop in get_partition_dev()

There is very rare cases where get_partition_dev() is called before the actual partition is available in /sys/block/<device>.

It appear that waiting a very short is usually enough to get the partition beein populated.

Analysis:
update_partition() is supposed to be enough to avoid any racing between events sent by parted/sgdisk/partprobe and
the actual creation on the /sys/block/<device>/* entrypoint.
On our CI that race occurs pretty often but trying to reproduce it locally never been possible.

This patch is almost a workaround rather than a fix to the real problem.
It offer retrying after a very short to be make a chance the device to appear.
This approach have been succesful on the CI.

Note his patch is not changing the timing when the device is perfectly created on time and just differ by a 1/5th up to 2 seconds when the bug occurs.

A typical output from the build running on a CI with that code.
command_check_call: Running command: /usr/bin/udevadm settle --timeout=600
get_dm_uuid: get_dm_uuid /dev/sda uuid path is /sys/dev/block/8:0/dm/uuid
get_partition_dev: Try 1/10 : partition 2 for /dev/sda does not in /sys/block/sda
get_partition_dev: Found partition 2 for /dev/sda after 1 tries
        get_dm_uuid: get_dm_uuid /dev/sda uuid path is /sys/dev/block/8:0/dm/uuid
get_dm_uuid: get_dm_uuid /dev/sda2 uuid path is /sys/dev/block/8:2/dm/uuid

fixes: #19428

Signed-off-by: Erwan Velu <erwan@redhat.com>
(cherry picked from commit 93e7b95ed8b4c78daebf7866bb1f0826d7199075)

8 years agoceph-disk: Reporting /sys directory in get_partition_dev()
Erwan Velu [Wed, 22 Mar 2017 09:11:44 +0000 (10:11 +0100)]
ceph-disk: Reporting /sys directory in get_partition_dev()

When get_partition_dev() fails, it reports the following message :
    ceph_disk.main.Error: Error: partition 2 for /dev/sdb does not appear to exist
The code search for a directory inside the /sys/block/get_dev_name(os.path.realpath(dev)).

The issue here is the error message doesn't report that path when failing while it might be involved in.

This patch is about reporting where the code was looking at when trying to estimate if the partition was available.

Signed-off-by: Erwan Velu <erwan@redhat.com>
(cherry picked from commit 413c9fcfbe8e6ab33d73b8428090ccacc33c5d15)

8 years agoceph-disk: separate ceph-osd --check-needs-* logs 16135/head
Loic Dachary [Tue, 9 May 2017 10:32:51 +0000 (12:32 +0200)]
ceph-disk: separate ceph-osd --check-needs-* logs

It is using the OSD id zero but have nothing to do with OSD zero and
this is confusing to the user. The log themselves do not need to be kept
around and are stored in the run directory so that they can be disposed
of after reboot.

Fixes: http://tracker.ceph.com/issues/19888
Signed-off-by: Loic Dachary <loic@dachary.org>
(cherry picked from commit c7b3c46bd63b78475868e405bf20d9c142f0336a)

8 years agotest: added missing IoCtx copy/assignment methods in librados_test_stub 16096/head
Jason Dillaman [Mon, 28 Nov 2016 18:36:01 +0000 (13:36 -0500)]
test: added missing IoCtx copy/assignment methods in librados_test_stub

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit c35d307c62961072b4c00f94e37486e3371c1a21)

8 years agoReplicatedBackend: don't queue Context outside of ObjectStore with obc 16132/head
Samuel Just [Tue, 14 Feb 2017 20:47:37 +0000 (12:47 -0800)]
ReplicatedBackend: don't queue Context outside of ObjectStore with obc

We only flush the ObjectStore callbacks, not everything else.  Thus,
there isn't a guarrantee that the obc held by pull_complete_info will
be cleaned up before the Flush callback is triggered.  Instead, just
defer clearing the pull state until the callback (it'll be cleaned up
during the interval change) and remove the ObjectContext from
pull_complete_info.

Introduced: 68defc2b0561414711d4dd0a76bc5d0f46f8a3f8
Fixes: http://tracker.ceph.com/issues/18927
Signed-off-by: Samuel Just <sjust@redhat.com>
(cherry picked from commit 51eee55c475a3d931844831e040ed3d66ee59af4)

8 years agoosd/: don't leak context for Blessed*Context or RecoveryQueueAsync
Samuel Just [Fri, 3 Feb 2017 21:12:47 +0000 (13:12 -0800)]
osd/: don't leak context for Blessed*Context or RecoveryQueueAsync

This has always been a bug, but until
68defc2b0561414711d4dd0a76bc5d0f46f8a3f8, nothing deleted those contexts
without calling complete().

Fixes: http://tracker.ceph.com/issues/18809
Bug shadowed until: 68defc2b0561414711d4dd0a76bc5d0f46f8a3f8
Signed-off-by: Samuel Just <sjust@redhat.com>
(cherry picked from commit 91b74235027c8a4872dcab6b37767b12c3267061)

8 years agoMerge pull request #15792 from liewegas/wip-bluestore-leaks-kraken
Nathan Cutler [Wed, 5 Jul 2017 20:04:41 +0000 (22:04 +0200)]
Merge pull request #15792 from liewegas/wip-bluestore-leaks-kraken

kraken: os/bluestore: deep decode onode value

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
8 years agoMerge pull request #15729 from asheplyakov/20315-bp-kraken
Nathan Cutler [Wed, 5 Jul 2017 20:01:58 +0000 (22:01 +0200)]
Merge pull request #15729 from asheplyakov/20315-bp-kraken

kraken: mon: fail to form large quorum; msg/async busy loop

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
8 years agoMerge pull request #15084 from asheplyakov/19928-bp-kraken
Nathan Cutler [Wed, 5 Jul 2017 20:00:17 +0000 (22:00 +0200)]
Merge pull request #15084 from asheplyakov/19928-bp-kraken

kraken: mon crash on shutdown, lease_ack_timeout event

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
8 years agoMerge pull request #14735 from smithfarm/wip-19685-kraken
Nathan Cutler [Wed, 5 Jul 2017 19:59:03 +0000 (21:59 +0200)]
Merge pull request #14735 from smithfarm/wip-19685-kraken

kraken: osd: Give requested scrubs a higher priority

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
8 years agoMerge pull request #14733 from smithfarm/wip-19561-kraken
Nathan Cutler [Wed, 5 Jul 2017 19:57:53 +0000 (21:57 +0200)]
Merge pull request #14733 from smithfarm/wip-19561-kraken

kraken: api_misc: [  FAILED  ] LibRadosMiscConnectFailure.ConnectFailure

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
8 years agoMerge pull request #13883 from shinobu-x/wip-19119-kraken
Nathan Cutler [Wed, 5 Jul 2017 19:51:26 +0000 (21:51 +0200)]
Merge pull request #13883 from shinobu-x/wip-19119-kraken

kraken: pre-jewel "osd rm" incrementals are misinterpreted

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
8 years agoMerge pull request #14646 from gregsfortytwo/wip-17916-kraken
Nathan Cutler [Wed, 5 Jul 2017 17:39:54 +0000 (19:39 +0200)]
Merge pull request #14646 from gregsfortytwo/wip-17916-kraken

kraken: osd: pglog: with config, don't assert in the presence of stale diverg…

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
8 years agoMerge pull request #15237 from asheplyakov/20035-bp-kraken
Nathan Cutler [Wed, 5 Jul 2017 16:41:59 +0000 (18:41 +0200)]
Merge pull request #15237 from asheplyakov/20035-bp-kraken

kraken: mon/PGMap: factor mon_osd_full_ratio into MAX AVAIL calc

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
8 years agoMerge pull request #14331 from shinobu-x/wip-19396-kraken
Nathan Cutler [Wed, 5 Jul 2017 16:19:05 +0000 (18:19 +0200)]
Merge pull request #14331 from shinobu-x/wip-19396-kraken

kraken: Objecter::epoch_barrier isn't respected in _op_submit()

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
8 years agoMerge pull request #16083 from smithfarm/wip-18677-kraken
Nathan Cutler [Wed, 5 Jul 2017 15:54:30 +0000 (17:54 +0200)]
Merge pull request #16083 from smithfarm/wip-18677-kraken

kraken: osd: metadata reports filestore when using bluestore

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
8 years agoMerge pull request #16098 from smithfarm/wip-19621-kraken
Nathan Cutler [Wed, 5 Jul 2017 15:51:36 +0000 (17:51 +0200)]
Merge pull request #16098 from smithfarm/wip-19621-kraken

kraken: rbd-nbd: add signal handler

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
8 years agoMerge pull request #15612 from smithfarm/wip-19611-kraken
Nathan Cutler [Wed, 5 Jul 2017 15:50:21 +0000 (17:50 +0200)]
Merge pull request #15612 from smithfarm/wip-19611-kraken

kraken: rbd: Issues with C API image metadata retrieval functions

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
Reviewed-by: Mykola Golub <mgolub@mirantis.com>
8 years agoMerge pull request #14833 from smithfarm/wip-19794-kraken
Nathan Cutler [Wed, 5 Jul 2017 15:48:24 +0000 (17:48 +0200)]
Merge pull request #14833 from smithfarm/wip-19794-kraken

kraken: tests: test_notify.py: assert(not image.is_exclusive_lock_owner()) on line 147

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
8 years agoMerge pull request #14543 from smithfarm/wip-19173-kraken
Nathan Cutler [Wed, 5 Jul 2017 15:26:58 +0000 (17:26 +0200)]
Merge pull request #14543 from smithfarm/wip-19173-kraken

kraken: rbd: rbd_clone_copy_on_read ineffective with exclusive-lock

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
8 years agoMerge pull request #14974 from dillaman/wip-19872-kraken
Nathan Cutler [Wed, 5 Jul 2017 07:37:01 +0000 (09:37 +0200)]
Merge pull request #14974 from dillaman/wip-19872-kraken

kraken: rbd-mirror: failover and failback of unmodified image results in split-brain

Reviewed-by: Mykola Golub <mgolub@mirantis.com>
8 years agoMerge pull request #14934 from dillaman/wip-19833-kraken
Nathan Cutler [Wed, 5 Jul 2017 07:36:14 +0000 (09:36 +0200)]
Merge pull request #14934 from dillaman/wip-19833-kraken

kraken: cls_rbd: default initialize snapshot namespace for legacy clients

Reviewed-by: Mykola Golub <mgolub@mirantis.com>