]> git.apps.os.sepia.ceph.com Git - ceph.git/log
ceph.git
6 years agoosd: add creating to pg_string_state 24601/head
Dan van der Ster [Tue, 25 Sep 2018 08:39:37 +0000 (10:39 +0200)]
osd: add creating to pg_string_state

Fixes: http://tracker.ceph.com/issues/36174
Signed-off-by: Dan van der Ster <daniel.vanderster@cern.ch>
(cherry picked from commit d38f6a11701ec788e4d384aa5b0ae65b8e57da64)

Conflicts:
src/osd/osd_types.cc : Resolved in pg_string_state

6 years agoMerge pull request #24388 from pdvian/wip-36129-mimic
Yuri Weinstein [Mon, 15 Oct 2018 20:52:18 +0000 (16:52 -0400)]
Merge pull request #24388 from pdvian/wip-36129-mimic

mimic: rgw: abort_bucket_multiparts() ignores individual NoSuchUpload errors

Reviewed-by: Casey Bodley <cbodley@redhat.com>
6 years agoMerge pull request #24400 from smithfarm/wip-36201-mimic
Yuri Weinstein [Mon, 15 Oct 2018 20:51:53 +0000 (16:51 -0400)]
Merge pull request #24400 from smithfarm/wip-36201-mimic

mimic: multisite: intermittent test_bucket_index_log_trim failures

Reviewed-by: Casey Bodley <cbodley@redhat.com>
6 years agoMerge pull request #24413 from dillaman/wip-36225-mimic
Yuri Weinstein [Mon, 15 Oct 2018 20:51:11 +0000 (16:51 -0400)]
Merge pull request #24413 from dillaman/wip-36225-mimic

mimic: librbd: object map improperly flagged as invalidated

Reviewed-by: Mykola Golub <mgolub@mirantis.com>
6 years agoMerge pull request #24414 from smithfarm/wip-36208-mimic
Yuri Weinstein [Mon, 15 Oct 2018 20:50:38 +0000 (16:50 -0400)]
Merge pull request #24414 from smithfarm/wip-36208-mimic

mimic: multisite: invalid read in RGWCloneMetaLogCoroutine

Reviewed-by: Casey Bodley <cbodley@redhat.com>
6 years agoMerge pull request #24418 from pdvian/wip-36140-mimic
Yuri Weinstein [Mon, 15 Oct 2018 20:50:13 +0000 (16:50 -0400)]
Merge pull request #24418 from pdvian/wip-36140-mimic

mimic: rgw: raise debug level on redundant data sync error messages

Reviewed-by: Casey Bodley <cbodley@redhat.com>
6 years agoMerge pull request #24436 from croit/mimic-36041
Yuri Weinstein [Mon, 15 Oct 2018 20:49:40 +0000 (16:49 -0400)]
Merge pull request #24436 from croit/mimic-36041

mimic: rgw/beast: drop privileges after binding ports

Reviewed-by: Casey Bodley <cbodley@redhat.com>
6 years agoMerge pull request #24481 from pdvian/wip-36150-mimic
Yuri Weinstein [Mon, 15 Oct 2018 20:49:10 +0000 (16:49 -0400)]
Merge pull request #24481 from pdvian/wip-36150-mimic

mimic: crush/CrushWrapper: fix crush tree json dumper

Reviewed-by: Kefu Chai <kchai@redhat.com>
6 years agoMerge pull request #24516 from pdvian/wip-36211-mimic
Yuri Weinstein [Mon, 15 Oct 2018 20:48:38 +0000 (16:48 -0400)]
Merge pull request #24516 from pdvian/wip-36211-mimic

mimic: rgw: RGWAsyncGetBucketInstanceInfo does not access coroutine memory

Reviewed-by: Casey Bodley <cbodley@redhat.com>
6 years agoMerge pull request #24518 from pdvian/wip-36213-mimic
Yuri Weinstein [Mon, 15 Oct 2018 20:48:13 +0000 (16:48 -0400)]
Merge pull request #24518 from pdvian/wip-36213-mimic

mimic:rgw: fix leak of curl handle on shutdown

Reviewed-by: Casey Bodley <cbodley@redhat.com>
6 years agoMerge pull request #24526 from theanalyst/wip-mimic-36381
Yuri Weinstein [Mon, 15 Oct 2018 20:47:55 +0000 (16:47 -0400)]
Merge pull request #24526 from theanalyst/wip-mimic-36381

mimic: rgw: resharding produces invalid values of bucket stats

Reviewed-by: Casey Bodley <cbodley@redhat.com>
6 years agoMerge pull request #24536 from pdvian/wip-36216-mimic
Yuri Weinstein [Mon, 15 Oct 2018 20:47:13 +0000 (16:47 -0400)]
Merge pull request #24536 from pdvian/wip-36216-mimic

mimic: multisite: data full sync does not limit concurrent bucket sync

Reviewed-by: Casey Bodley <cbodley@redhat.com>
6 years agoMerge pull request #24537 from pdvian/wip-36228-mimic
Yuri Weinstein [Mon, 15 Oct 2018 20:46:36 +0000 (16:46 -0400)]
Merge pull request #24537 from pdvian/wip-36228-mimic

mimic: qa: add test that builds example librados programs

Reviewed-by: Neha Ojha <nojha@redhat.com>
6 years agoqa: add test that builds example librados programs 24537/head
Nathan Cutler [Thu, 19 Jul 2018 15:59:04 +0000 (17:59 +0200)]
qa: add test that builds example librados programs

Fixes: http://tracker.ceph.com/issues/15100
Signed-off-by: Nathan Cutler <ncutler@suse.com>
(cherry picked from commit c46c890d0241972cee10260f071f65b4beedf92c)

6 years agorgw: data sync checks whether the lease failed to renew 24536/head
Casey Bodley [Mon, 13 Aug 2018 19:47:29 +0000 (15:47 -0400)]
rgw: data sync checks whether the lease failed to renew

Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit 3754871e65830e772f9b902fa162853716eeb78d)

 Conflicts:
src/rgw/rgw_data_sync.cc : Resolved in full_sync

6 years agorgw: data sync holds lease over transition from full to incremental
Casey Bodley [Sat, 11 Aug 2018 15:39:35 +0000 (11:39 -0400)]
rgw: data sync holds lease over transition from full to incremental

Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit 3e9ac0f1b3f701902a3c35b61e45d938805ddb26)

6 years agorgw: enforce spawn_window for data full sync
Casey Bodley [Fri, 10 Aug 2018 20:34:42 +0000 (16:34 -0400)]
rgw: enforce spawn_window for data full sync

limit the number of concurrent bucket sync crs spawned by data full sync

Fixes: http://tracker.ceph.com/issues/26897
Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit 895c28a5b43df7fb46cb443b10153abbb48274f6)

6 years agorgw: copy actual stats from the source shards during reshard 24526/head
Abhishek Lekshmanan [Fri, 5 Oct 2018 09:19:18 +0000 (11:19 +0200)]
rgw: copy actual stats from the source shards during reshard

Currently we don't copy the actual_stats field during reshard, which makes
resharded buckets show a size_utilized as 0, which further has the problem that
a subsequent object removal would subtract the object size from the 0 size
utilized showing up large uint64_t values. Copy the size_actual from the source
object in both cls and in reshard_process. This will fix the new buckets,
existing buckets will still have to go through a bucket check --fix for their
stats to be corrected.

Fixes: http://tracker.ceph.com/issues/36290
Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>
(cherry picked from commit beb90638ae3d5329653b61bae0d6714796c41d04)

6 years agorgw: fix leak of curl handle on shutdown 24518/head
Casey Bodley [Wed, 5 Sep 2018 20:56:41 +0000 (16:56 -0400)]
rgw: fix leak of curl handle on shutdown

addresses a valgrind leak reported from curl_multi_add_handle().
requests that are still linked to the curl_multi handle need to be
unlinked and finished with ECANCELED, rather than just finished

Fixes: http://tracker.ceph.com/issues/35715
Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit dd478483962188c1805f522deb7f7c7a33c5fadb)

6 years agoMerge pull request #24176 from pdvian/wip-35836-mimic 24567/head
Neha Ojha [Wed, 10 Oct 2018 02:05:46 +0000 (19:05 -0700)]
Merge pull request #24176 from pdvian/wip-35836-mimic

mimic: mon: mgr options not parse propertly

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: John Spray <john.spray@redhat.com>
6 years agoMerge pull request #24281 from pdvian/wip-35930-mimic
Neha Ojha [Wed, 10 Oct 2018 02:00:09 +0000 (19:00 -0700)]
Merge pull request #24281 from pdvian/wip-35930-mimic

mimic: mon/OSDMonitor: invalidate max_failed_since on cancel_report

Reviewed-by: Xie Xingguo <xie.xingguo@zte.com.cn>
6 years agorgw: RGWAsyncGetBucketInstanceInfo does not access coroutine memory 24516/head
Casey Bodley [Fri, 7 Sep 2018 20:13:27 +0000 (16:13 -0400)]
rgw: RGWAsyncGetBucketInstanceInfo does not access coroutine memory

Fixes: http://tracker.ceph.com/issues/35812
Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit 6fb61c5ed8eb7cbe6d1271a8b61e465023d2e621)

6 years agocrush/CrushWrapper: fix crush tree json dumper 24481/head
songshuangyang [Fri, 14 Sep 2018 06:53:17 +0000 (14:53 +0800)]
crush/CrushWrapper: fix crush tree json dumper

    The output json string is invalid for 'osd crush tree --format=json'
    command. It contains a array of 'nodes' and a array of 'stray', but
    not in a json object, and the stray array was not implemented.
    Applications which depends on the output of the above MonCommand will
    occur json parse error.

Signed-off-by: Oshyn Song <dualyangsong@gmail.com>
(cherry picked from commit 35c0d1f45cd676f201d4031cb8f447f7ea6aee0e)

6 years agoMerge pull request #24113 from smithfarm/wip-35717-mimic
Yuri Weinstein [Sat, 6 Oct 2018 16:15:48 +0000 (09:15 -0700)]
Merge pull request #24113 from smithfarm/wip-35717-mimic

mimic: msg: challenging authorizer messages appear at debug_ms=0

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
6 years agoMerge pull request #24241 from smithfarm/wip-36121-mimic
Yuri Weinstein [Sat, 6 Oct 2018 16:14:49 +0000 (09:14 -0700)]
Merge pull request #24241 from smithfarm/wip-36121-mimic

mimic: kernel_untar_build.sh: bison: command not found

Reviewed-by: Neha Ojha <nojha@redhat.com>
6 years agoMerge pull request #24298 from pdvian/wip-35959-mimic
Yuri Weinstein [Sat, 6 Oct 2018 16:14:22 +0000 (09:14 -0700)]
Merge pull request #24298 from pdvian/wip-35959-mimic

mimic: osd/ECBackend: don't get result code of subchunk-read overwritten

Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>
Reviewed-by: Neha Ojha <nojha@redhat.com>
6 years agoMerge pull request #24309 from neha-ojha/wip-36239-mimic
Yuri Weinstein [Sat, 6 Oct 2018 16:13:49 +0000 (09:13 -0700)]
Merge pull request #24309 from neha-ojha/wip-36239-mimic

mimic: osd/PrimaryLogPG: fix potential pg-log overtrimming

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>
6 years agoMerge pull request #24344 from pdvian/wip-35963-mimic
Yuri Weinstein [Sat, 6 Oct 2018 16:13:11 +0000 (09:13 -0700)]
Merge pull request #24344 from pdvian/wip-35963-mimic

mimic: osd/PG: avoid choose_acting picking want with > pool size items

Reviewed-by: Neha Ojha <nojha@redhat.com>
6 years agoMerge pull request #24345 from pdvian/wip-35964-mimic
Yuri Weinstein [Sat, 6 Oct 2018 16:12:48 +0000 (09:12 -0700)]
Merge pull request #24345 from pdvian/wip-35964-mimic

mimic: RADOS: probably missing clone location for async_recovery_targets

Reviewed-by: Neha Ojha <nojha@redhat.com>
6 years agoMerge pull request #24233 from liewegas/wip-35998-mimic
Yuri Weinstein [Fri, 5 Oct 2018 21:41:00 +0000 (14:41 -0700)]
Merge pull request #24233 from liewegas/wip-35998-mimic

mimic: mgr/DaemonServer: fix Session leak

Reviewed-by: Neha Ojha <nojha@redhat.com>
6 years agoMerge pull request #24339 from ifed01/wip-ifed-fix-repairer-mimic
Yuri Weinstein [Fri, 5 Oct 2018 21:19:25 +0000 (14:19 -0700)]
Merge pull request #24339 from ifed01/wip-ifed-fix-repairer-mimic

mimic: os/bluestore: fix bloom filter num entry miscalculation in repairer

Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>
6 years agoMerge pull request #24352 from ifed01/wip-ifed-bluestore-tool-fixes-mimic
Yuri Weinstein [Fri, 5 Oct 2018 21:18:49 +0000 (14:18 -0700)]
Merge pull request #24352 from ifed01/wip-ifed-bluestore-tool-fixes-mimic

mimic: ceph-bluestore-tool: fix set label functionality for specific keys

Reviewed-by: Sage Weil <sage@redhat.com>
6 years agoMerge pull request #24386 from pdvian/wip-36127-mimic
Yuri Weinstein [Fri, 5 Oct 2018 21:18:12 +0000 (14:18 -0700)]
Merge pull request #24386 from pdvian/wip-36127-mimic

mimic: msg/async: clean up local buffers on dispatch

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Ricardo Dias <rdias@suse.com>
6 years agorgw/beast: drop privileges after binding ports 24436/head
Paul Emmerich [Tue, 25 Sep 2018 14:25:55 +0000 (14:25 +0000)]
rgw/beast: drop privileges after binding ports

Fixes: http://tracker.ceph.com/issues/36041
Signed-off-by: Paul Emmerich <paul.emmerich@croit.io>
(cherry picked from commit 6489566e867cee81a753b246913b61815819d9b3)

 Conflicts:
src/rgw/rgw_asio_frontend.cc

6 years agolibrbd: keep IO blocked until after snapshot object map created 24413/head
Jason Dillaman [Mon, 24 Sep 2018 19:07:15 +0000 (15:07 -0400)]
librbd: keep IO blocked until after snapshot object map created

The IO was being unblocked before object map was created, allowing
a potential copyup request to fail to update a still-to-be-created
object map.

Fixes: http://tracker.ceph.com/issues/24516
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 1e874403bf861cb8b74261308d8b73434cf90341)

Conflicts:
src/librbd/object_map/SnapshotCreateRequest.cc: trivial resolution
src/librbd/operation/SnapshotCreateRequest.cc: trivial resolution

6 years agolibrbd: do not invalidate object map if update races with copyup
Jason Dillaman [Mon, 24 Sep 2018 18:45:09 +0000 (14:45 -0400)]
librbd: do not invalidate object map if update races with copyup

The copyup state machine needs to iterate over all object maps to update
the existence for the object. If an snapshot is being removed concurrently,
it's possible to invalidate the object map for the image.

Fixes: http://tracker.ceph.com/issues/24516
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 5a1cb469879157297ab456261f9335d8b855684f)

Conflicts:
src/librbd/ObjectMap.cc: trivial resolution
src/librbd/ObjectMap.h: trivial resolution
src/librbd/io/CopyupRequest.cc: trivial resolution
src/test/librbd/deep_copy/test_mock_ObjectCopyRequest.cc: trivial resolution
src/test/librbd/test_mock_ObjectMap.cc: trivial resolution

6 years agolibrbd: do not invalidate object map when attempting to delete non-existent snapshot
Jason Dillaman [Fri, 14 Sep 2018 15:46:13 +0000 (11:46 -0400)]
librbd: do not invalidate object map when attempting to delete non-existent snapshot

If duplicate snapshot remove requests are received by the lock owner from a peer
client, the first request will remove the object map. If the second request
arrives while the first is in-progress, it will again attempt to remove the
object map but fail to load it since it's already been deleted. This incorrectly
results in the next object map being flagged as invalid.

Fixes: http://tracker.ceph.com/issues/24516
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 0a31c55ea83d85da88c7586c9a8fa8d6ec6618a7)

Conflicts:
src/librbd/object_map/SnapshotRemoveRequest.cc: trivial resolution

6 years agoMerge pull request #24420 from karunjosy/wip-36308-mimic
Nathan Cutler [Thu, 4 Oct 2018 09:39:52 +0000 (11:39 +0200)]
Merge pull request #24420 from karunjosy/wip-36308-mimic

mimic: doc: Typo error on cephfs/fuse/

Reviewed-by: Nathan Cutler <ncutler@suse.com>
6 years agodoc: Fix typo error on cephfs/fuse/ 24420/head
Karun Josy [Tue, 2 Oct 2018 11:53:25 +0000 (17:23 +0530)]
doc: Fix typo error on cephfs/fuse/

Fixes: http://tracker.ceph.com/issues/36180
Signed-off-by: Karun Josy <kjosy@redhat.com>
(cherry picked from commit d312cdbd36d9c0e76409feac1f3e289b247dc2dc)

6 years agorgw: raise debug level on redundant data sync error messages 24418/head
Casey Bodley [Fri, 7 Sep 2018 13:51:56 +0000 (09:51 -0400)]
rgw: raise debug level on redundant data sync error messages

each of these errors have already been logged at a lower level with a
more detailed error message. by logging them as ERRORs at level 0 here,
the messages could be easily confused as separate failures

Fixes: http://tracker.ceph.com/issues/35830
Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit a6e392f2b8cb885f6fede5dfa9b584659740e161)

Conflicts:
src/rgw/rgw_data_sync.cc : Resolved in operate

6 years agorgw: dont access rgw_http_req_data::client of canceled request 24414/head
Casey Bodley [Fri, 7 Sep 2018 21:01:25 +0000 (17:01 -0400)]
rgw: dont access rgw_http_req_data::client of canceled request

if a request has been canceled with RGWHTTPClient::cancel(), the client
may have been destroyed. check req_data->registered before reading
from req_data->client->receive_pause_skip

Fixes: http://tracker.ceph.com/issues/35851
Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit cdc876a3c5e3e8874c633b9355dfea51c1ee3bdf)

6 years agolibrbd: converted object map snapshot remove state machine to new style
Jason Dillaman [Fri, 14 Sep 2018 15:21:28 +0000 (11:21 -0400)]
librbd: converted object map snapshot remove state machine to new style

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 58770188ab57a53b786cf616ccfbf6acfcdc115a)

Conflicts:
src/librbd/object_map/SnapshotRemoveRequest.cc: trivial resolution

6 years agolibrbd: test_flags helper should require snap id parameter
Jason Dillaman [Fri, 14 Sep 2018 13:59:35 +0000 (09:59 -0400)]
librbd: test_flags helper should require snap id parameter

The HEAD and snapshots have potentially different flag states
since object maps get invalidated per revision.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 862082792d9c2ff23823e46937b7de9a42830cfd)

Conflicts:
src/librbd/ObjectMap.cc: trivial resolution
src/test/librbd/test_Migration.cc: DNE

6 years agoMerge pull request #24002 from pdvian/wip-35705-mimic
Yuri Weinstein [Wed, 3 Oct 2018 15:21:28 +0000 (08:21 -0700)]
Merge pull request #24002 from pdvian/wip-35705-mimic

mimic:rbd: fix error import when the input is a pipe

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
6 years agoMerge pull request #24122 from pdvian/wip-35712-mimic
Yuri Weinstein [Wed, 3 Oct 2018 15:21:04 +0000 (08:21 -0700)]
Merge pull request #24122 from pdvian/wip-35712-mimic

mimic: librbd: journaling unable request can not be sent to remote lock owner

Reviewed-by: Mykola Golub <mgolub@mirantis.com>
Reviewed-by: Jason Dillaman <dillaman@redhat.com>
6 years agoMerge pull request #24137 from pdvian/wip-35714-mimic
Yuri Weinstein [Wed, 3 Oct 2018 15:20:30 +0000 (08:20 -0700)]
Merge pull request #24137 from pdvian/wip-35714-mimic

mimic: librbd: ensure exclusive lock acquired when removing sync point snaps…

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
6 years agoMerge pull request #24240 from smithfarm/wip-35077-mimic
Yuri Weinstein [Wed, 3 Oct 2018 15:19:59 +0000 (08:19 -0700)]
Merge pull request #24240 from smithfarm/wip-35077-mimic

mimic: qa: install dependencies for rbd_workunit_kernel_untar_build

Reviewed-by: Ilya Dryomov <idryomov@redhat.com>
6 years agoMerge pull request #24244 from dillaman/wip-36117-mimic
Yuri Weinstein [Wed, 3 Oct 2018 15:19:22 +0000 (08:19 -0700)]
Merge pull request #24244 from dillaman/wip-36117-mimic

mimic: test/librbd: not valid to have different parents between image snapshots

Reviewed-by: Mykola Golub <mgolub@mirantis.com>
6 years agoMerge pull request #24321 from smithfarm/wip-36120-mimic
Yuri Weinstein [Wed, 3 Oct 2018 15:17:43 +0000 (08:17 -0700)]
Merge pull request #24321 from smithfarm/wip-36120-mimic

mimic: [rbd-mirror] failed assertion when updating mirror status

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
6 years agoMerge pull request #24356 from dillaman/wip-35957-mimic
Yuri Weinstein [Wed, 3 Oct 2018 15:16:36 +0000 (08:16 -0700)]
Merge pull request #24356 from dillaman/wip-35957-mimic

mimic: librbd: fixed assert when flattening clone with zero overlap

Reviewed-by: Mykola Golub <mgolub@mirantis.com>
6 years agoMerge pull request #24381 from alfredodeza/mimic-rm36247
Andrew Schoen [Wed, 3 Oct 2018 15:00:22 +0000 (10:00 -0500)]
Merge pull request #24381 from alfredodeza/mimic-rm36247

mimic ceph-volume: skip processing devices that don't exist when scanning system disks

Reviewed-by: Andrew Schoen <aschoen@redhat.com>
6 years agorgw: remove BucketChangeObserver from data sync thread 24400/head
Casey Bodley [Thu, 20 Sep 2018 15:37:06 +0000 (11:37 -0400)]
rgw: remove BucketChangeObserver from data sync thread

Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit f05db89637d280505321708683182f0f2c886208)

Conflicts:
src/rgw/rgw_data_sync.h
- argument lists are different

6 years agorgw: add BucketChangeObserver to RGWDataChangesLog
Casey Bodley [Thu, 20 Sep 2018 15:34:42 +0000 (11:34 -0400)]
rgw: add BucketChangeObserver to RGWDataChangesLog

this means that BucketTrimManager will track active buckets based on
local changes, rather than changes in remote datalogs or error repos

Fixes: http://tracker.ceph.com/issues/36034
Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit f3c258c49ff6899433e742b10554c83413d64a8a)

6 years agorgw: abort_bucket_multiparts() ignores individual NoSuchUpload errors 24388/head
Casey Bodley [Fri, 14 Sep 2018 18:56:23 +0000 (14:56 -0400)]
rgw: abort_bucket_multiparts() ignores individual NoSuchUpload errors

if the bucket index lists multipart meta objects that don't actually
exist in rados, this error prevents the bucket from being deleted

Fixes: http://tracker.ceph.com/issues/35986
Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit 764d6a8599eb5ea5a6382fea57e4b28f97e26d93)

6 years agomsg/async: clean up local buffers on dispatch 24386/head
Greg Farnum [Fri, 14 Sep 2018 17:58:49 +0000 (10:58 -0700)]
msg/async: clean up local buffers on dispatch

The AsyncConnection keeps local (member variable) bufferlists of incoming
messages before they're placed into the Message's front/data/middle buffers.
Previously these were reset only when a new Message is being received, which
means in steady state we store a full Message for every Connection even if
it's inactive!

Instead we obviously want to drop our local references to Message state
once it's been dispatched, so that it can go away.

Fixes: http://tracker.ceph.com/issues/35987
Signed-off-by: Greg Farnum <gfarnum@redhat.com>
(cherry picked from commit 47ed036753223c44c7bf66c64d4a4adfe7267c0a)

6 years agoceph-volume util.disk when there are no devices mapped, skip to the next one 24381/head
Alfredo Deza [Tue, 2 Oct 2018 15:18:44 +0000 (11:18 -0400)]
ceph-volume util.disk when there are no devices mapped, skip to the next one

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 9b0f472abadde26fce2a603fca5c466ebb770d4a)

6 years agoceph-volume tests.util verify devices that don't exist don't break get_devices
Alfredo Deza [Tue, 2 Oct 2018 15:18:22 +0000 (11:18 -0400)]
ceph-volume tests.util verify devices that don't exist don't break get_devices

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 0176c1965e6ed532b38dfee03af0e9e3a85149d6)

6 years agoMerge pull request #24357 from alfredodeza/mimic-rm36249
Andrew Schoen [Tue, 2 Oct 2018 11:31:50 +0000 (06:31 -0500)]
Merge pull request #24357 from alfredodeza/mimic-rm36249

mimic ceph-volume:  activate option --auto-detect-objectstore respects --no-systemd

Reviewed-by: Andrew Schoen <aschoen@redhat.com>
6 years agoceph-volume lvm.activate pass no_systemd args when detecting objectstore 24357/head
Alfredo Deza [Mon, 1 Oct 2018 15:54:19 +0000 (11:54 -0400)]
ceph-volume lvm.activate pass no_systemd args when detecting objectstore

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 29de6820a9a7108c8de41036add9332f9b910359)

6 years agoceph-volume tests.lvm verify activate works with --auto-detect-objectstore
Alfredo Deza [Mon, 1 Oct 2018 15:53:37 +0000 (11:53 -0400)]
ceph-volume tests.lvm verify activate works with --auto-detect-objectstore

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 60dbeb6184797ccffeaeea2e85ef2d7fb4a1c54c)

6 years agolibrbd: fixed assert when flattening clone with zero overlap 24356/head
Jason Dillaman [Wed, 12 Sep 2018 01:52:05 +0000 (21:52 -0400)]
librbd: fixed assert when flattening clone with zero overlap

Fixes: http://tracker.ceph.com/issues/35702
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 8e9035443322f8cbdb8c7aff5da0d1c7599a2592)

Conflicts:
src/librbd/Operations.cc: trivial resolution
src/test/librbd/test_librbd.cc: trivial resolution

6 years agoMerge pull request #24334 from alfredodeza/mimic-rm36098
Andrew Schoen [Mon, 1 Oct 2018 14:54:23 +0000 (09:54 -0500)]
Merge pull request #24334 from alfredodeza/mimic-rm36098

mimic ceph-volume:  lsblk can fail to find PARTLABEL, must fallback to blkid

Reviewed-by: Andrew Schoen <aschoen@redhat.com>
6 years agotools/ceph-bluestore-tool: update size label on bluefs expand 24351/head 24352/head
Igor Fedotov [Thu, 24 May 2018 16:05:48 +0000 (19:05 +0300)]
tools/ceph-bluestore-tool: update size label on bluefs expand

Signed-off-by: Igor Fedotov <ifedotov@suse.com>
(cherry picked from commit 71c3b58da4e7ced3422bce2b1da0e3fa9331530b)

6 years agotools/ceph-bluestore-tool: add support for proper setting size, btime,
Igor Fedotov [Thu, 24 May 2018 13:32:01 +0000 (16:32 +0300)]
tools/ceph-bluestore-tool: add support for proper setting size, btime,
description, osd_uuid fields

Signed-off-by: Igor Fedotov <ifedotov@suse.com>
(cherry picked from commit ffac450da5d6e09cf14b8363b35f21819b48f38b)

6 years agotool/ceph-bluestore-tool: avoid mon/config access when calling global_init
Igor Fedotov [Fri, 18 May 2018 14:47:29 +0000 (17:47 +0300)]
tool/ceph-bluestore-tool: avoid mon/config access when calling global_init
This caused a permanent stuck on tool's startup.

Signed-off-by: Igor Fedotov <ifedotov@suse.com>
(cherry picked from commit fa65e7bf65bea514b66f56053d5cbf2c16045f4c)

6 years agoosd/PrimaryLogPG: is_degraded_on_async_recovery_target - restrict to async_recovery_t... 24345/head
xie xingguo [Mon, 3 Sep 2018 04:05:14 +0000 (12:05 +0800)]
osd/PrimaryLogPG: is_degraded_on_async_recovery_target - restrict to async_recovery_targets only

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
(cherry picked from commit 6923a4371b6757c7a2a4dfb6320e7190ce61c645)

6 years agoosd/PrimaryLogPG: update missing_loc more carefully
xie xingguo [Tue, 14 Aug 2018 07:30:37 +0000 (15:30 +0800)]
osd/PrimaryLogPG: update missing_loc more carefully

There could be clones and hence we should update missing_loc
based on log entry itself.

Also I think the **have** field of missing_loc is not quite accurate here
but should be fine.

Fixes: http://tracker.ceph.com/issues/35546
Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
(cherry picked from commit b09dedf2e9f37ae117a909796727adea71e32422)

6 years agoosd/PG: avoid choose_acting picking want with > pool size items 24344/head
Sage Weil [Tue, 11 Sep 2018 16:12:22 +0000 (11:12 -0500)]
osd/PG: avoid choose_acting picking want with > pool size items

If the pool size recently changed, we might see an up that is larger than
the pool size.  Or, we might already have selected a primary that is not
part of up, and then add all up osds and end up with a want that is too
big.

Fixes: http://tracker.ceph.com/issues/35924
Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit c3e2990c39952a1fc4593076c5b2f0dea6cdcdd3)

6 years agoceph-volume simple.scan better detection for ceph-disk data devices 24334/head
Alfredo Deza [Thu, 27 Sep 2018 20:17:29 +0000 (16:17 -0400)]
ceph-volume simple.scan better detection for ceph-disk data devices

Uses the new ``CephDiskDevice`` to look into PARTLABEL from both lsblk
(the default) falling back to blkid, which in some cases has the right
value when lsblk fails.

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit c502e5f64aa4f0d832c14680e1731e58d8a3770b)

6 years agoceph-volume tests.util verify CephDiskDevice behavior
Alfredo Deza [Thu, 27 Sep 2018 20:15:26 +0000 (16:15 -0400)]
ceph-volume tests.util verify CephDiskDevice behavior

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 8f072eb3f940a6714089b37a6fbacd549420a454)

6 years agoceph-volume tests.conftest add blkid patching to device_info fixture
Alfredo Deza [Thu, 27 Sep 2018 20:15:00 +0000 (16:15 -0400)]
ceph-volume tests.conftest add blkid patching to device_info fixture

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 5bfc8c26e639dba955f83f53893ce311e6e047f1)

6 years agoceph-volume util.device add a ceph-disk device object to detect types
Alfredo Deza [Thu, 27 Sep 2018 20:14:28 +0000 (16:14 -0400)]
ceph-volume util.device add a ceph-disk device object to detect types

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 200e6bfd934b34aa8175330aef32c51505ec3392)

6 years agoceph-volume tests.util verify new blkid API calls
Alfredo Deza [Tue, 25 Sep 2018 19:27:47 +0000 (15:27 -0400)]
ceph-volume tests.util verify new blkid API calls

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit ead3c7e7d19635a741a9da6fc330ec12a68b3d77)

6 years agoceph-volume util.disk expand blkid's API calls
Alfredo Deza [Tue, 25 Sep 2018 19:25:13 +0000 (15:25 -0400)]
ceph-volume util.disk expand blkid's API calls

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit dbde6a5b611fdd4bcd942df5dbab94ab5076fdff)

6 years agoMerge pull request #24337 from alfredodeza/mimic-rm36251
Andrew Schoen [Fri, 28 Sep 2018 19:19:40 +0000 (14:19 -0500)]
Merge pull request #24337 from alfredodeza/mimic-rm36251

mimic ceph-volume add new ceph-handlers role from ceph-ansible

Reviewed-by: Andrew Schoen <aschoen@redhat.com>
6 years agoceph-volume tests.functional add new ceph-handlers role from ceph-ansible 24337/head
Alfredo Deza [Fri, 28 Sep 2018 16:40:09 +0000 (12:40 -0400)]
ceph-volume tests.functional add new ceph-handlers role from ceph-ansible

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 956dc5e65c6e35e27ab6524d4fb9fdd05d616cf6)

6 years agoos/bluestore: fix bloom filter num entry miscalculation in the repairer 24339/head
Igor Fedotov [Thu, 13 Sep 2018 13:22:39 +0000 (16:22 +0300)]
os/bluestore: fix bloom filter num entry miscalculation in the repairer

This could  cause an assertion due to an access to uninitialized bloom
filter. This happened  when detecting errors involving physical extents close to the end of a large partition.
E.g. it was observed in http://tracker.ceph.com/issues/25001 while trying to repair an OSD.

Fixes: https://tracker.ceph.com/issues/35971
Signed-off-by: Igor Fedotov <ifedotov@suse.com>
(cherry picked from commit 50919813a15d9c29c182062f88885890c55eb76e)

6 years agoMerge pull request #24306 from alfredodeza/mimic-batch-report-fix
Andrew Schoen [Fri, 28 Sep 2018 14:52:33 +0000 (09:52 -0500)]
Merge pull request #24306 from alfredodeza/mimic-batch-report-fix

mimic ceph-volume: fix journal and filestore data size in `lvm batch --report`

Reviewed-by: Andrew Schoen <aschoen@redhat.com>
6 years agorbd-mirror: instantiate the status formatter before changing state 24321/head
Jason Dillaman [Wed, 19 Sep 2018 18:24:31 +0000 (14:24 -0400)]
rbd-mirror: instantiate the status formatter before changing state

This will avoid a possible race between pre-queued status updates
firing between the time the state has been changed and the formatter
has been instantiated.

Fixes: http://tracker.ceph.com/issues/36084
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit bbdc545ec1602f928c89c85d61c975abdf67be5b)

6 years agoosd/PrimaryLogPG: fix potential pg-log overtrimming 24309/head
xie xingguo [Mon, 30 Jul 2018 10:56:56 +0000 (18:56 +0800)]
osd/PrimaryLogPG: fix potential pg-log overtrimming

In https://github.com/ceph/ceph/pull/21580 I set a trap to catch some wired
and random segmentfaults and in a recent QA run I was able to observe it was
successfully triggered by one of the test case, see:

```
http://qa-proxy.ceph.com/teuthology/xxg-2018-07-30_05:25:06-rados-wip-hb-peers-distro-basic-smithi/2837916/teuthology.log
```

The root cause is that there might be holes on log versions, thus the
approx_size() method should (almost) always overestimate the actual number of log entries.
As a result, we might be at the risk of overtrimming log entries.

https://github.com/ceph/ceph/pull/18338 reveals a probably easier way
to fix the above problem but unfortunately it also can cause big performance regression
and hence comes this pr..

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
(cherry picked from commit 3654d56985c67d15506fa37b56ef5b0c04e01a65)

Conflicts:
src/osd/PrimaryLogPG.cc: trivial resolution

6 years agoceph-volume: only run ceph-common once during tests 24306/head
Andrew Schoen [Tue, 25 Sep 2018 17:09:15 +0000 (12:09 -0500)]
ceph-volume: only run ceph-common once during tests

Using import_role we can have just one play and avoid rerunning
ceph-common over and over again in daemon specific plays.

Signed-off-by: Andrew Schoen <aschoen@redhat.com>
(cherry picked from commit a81386123e66e2d61391371cec4485b3148ff5a7)

6 years agoceph-volume: fail fast during tests
Andrew Schoen [Mon, 24 Sep 2018 21:41:37 +0000 (16:41 -0500)]
ceph-volume: fail fast during tests

This also rsyncs the ceph-volume code to the testing vms before
a ceph.conf is generated because ceph-volume is needed now for this
to figure out the number of osds when you're using 'lvm batch'.

Signed-off-by: Andrew Schoen <aschoen@redhat.com>
(cherry picked from commit eed244473d11a153cd7a37e0e9bfaf804a20db89)

6 years agoceph-volume: when using batch --report return journal/data size as ints
Andrew Schoen [Mon, 24 Sep 2018 21:37:35 +0000 (16:37 -0500)]
ceph-volume: when using batch --report return journal/data size as ints

Signed-off-by: Andrew Schoen <aschoen@redhat.com>
(cherry picked from commit a5e8bc97e70195cee54c4b66d10684d6cd3f5e38)

6 years agoMerge pull request #24279 from alfredodeza/mimic-rm27210
Andrew Schoen [Thu, 27 Sep 2018 16:10:29 +0000 (11:10 -0500)]
Merge pull request #24279 from alfredodeza/mimic-rm27210

mimic ceph-volume custom cluster names fail on filestore trigger

Reviewed-by: Andrew Schoen <aschoen@redhat.com>
6 years agoMerge pull request #24210 from pdvian/wip-35855-mimic
Nathan Cutler [Thu, 27 Sep 2018 14:30:21 +0000 (16:30 +0200)]
Merge pull request #24210 from pdvian/wip-35855-mimic

mimic : doc: remove deprecated 'scrubq' from ceph(8)

Reviewed-by: Nathan Cutler <ncutler@suse.com>
6 years agoosd/ECBackend: don't get result code of subchunk-read overwritten 24298/head
songweibin [Sat, 1 Sep 2018 08:42:55 +0000 (16:42 +0800)]
osd/ECBackend: don't get result code of subchunk-read overwritten

Fixes: http://tracker.ceph.com/issues/21769
Signed-off-by: songweibin <song.weibin@zte.com.cn>
(cherry picked from commit 66bfaaca20e1a4e61e523aa260d60ce368f66df2)

Conflicts:
src/osd/ECBackend.cc : Resolved in handle_sub_read

6 years agoMerge pull request #24254 from yuriw/wip-yuriw-p2p-mimic
Yuri Weinstein [Wed, 26 Sep 2018 15:38:13 +0000 (08:38 -0700)]
Merge pull request #24254 from yuriw/wip-yuriw-p2p-mimic

qa/tests: added v13.2.2 to the mix

6 years agoMerge pull request #24258 from pdvian/wip-35925-mimic
Lenz Grimmer [Wed, 26 Sep 2018 07:19:36 +0000 (09:19 +0200)]
Merge pull request #24258 from pdvian/wip-35925-mimic

mimic: mgr/dashboard: Progress bar does not stop in TableKeyValueComponent

Reviewed-by: Volker Theile <vtheile@suse.com>
6 years agomon/OSDMonitor: invalidate max_failed_since on cancel_report 24281/head
xie xingguo [Thu, 6 Sep 2018 02:44:26 +0000 (10:44 +0800)]
mon/OSDMonitor: invalidate max_failed_since on cancel_report

max_failed_since might reference the very failure-report which is to be
cancelled. We can simply invalidate it here to make **get_failed_since()**
recalculate if necessary.

Fixes: http://tracker.ceph.com/issues/35860
Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
(cherry picked from commit 91a2d408a9349db08e916e2145040df8b1f17e14)

6 years agoceph-volume tests/functional update dmcrypt test playbooks to use --cluster 24279/head
Alfredo Deza [Mon, 24 Sep 2018 18:44:31 +0000 (14:44 -0400)]
ceph-volume tests/functional update dmcrypt test playbooks to use --cluster

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 96e29adf72804cb4fca62c04416b8fe2d23587bb)

6 years agoceph-volume tests/functional/batch update tests to use 'test' cluster name
Alfredo Deza [Mon, 24 Sep 2018 16:52:50 +0000 (12:52 -0400)]
ceph-volume tests/functional/batch update tests to use 'test' cluster name

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 5f72d438da94323c0b04f261b1a60b13eaf03966)

6 years agoceph-volume tests/functional/lvm update tests to use 'test' cluster name
Alfredo Deza [Mon, 24 Sep 2018 16:52:33 +0000 (12:52 -0400)]
ceph-volume tests/functional/lvm update tests to use 'test' cluster name

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 7b0d2dc66968f81192f152fa166fbc2cb4dcb438)

6 years agoceph-volume tests/functional update playbooks to use --cluster
Alfredo Deza [Mon, 24 Sep 2018 16:51:47 +0000 (12:51 -0400)]
ceph-volume tests/functional update playbooks to use --cluster

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit ba10b6820fd7f352ed6ba78246a6976c54783cbd)

6 years agoceph-volume main move out the load_ceph_conf_path to the configuration module
Alfredo Deza [Mon, 24 Sep 2018 16:27:04 +0000 (12:27 -0400)]
ceph-volume main move out the load_ceph_conf_path to the configuration module

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 532908099eb63bd805db8aecae1ef12301076410)

6 years agoceph-volume tests.devices patch configuration.load to avoid raising non-existing...
Alfredo Deza [Mon, 24 Sep 2018 14:34:55 +0000 (10:34 -0400)]
ceph-volume tests.devices patch configuration.load to avoid raising non-existing ceph.conf

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit fe80e5399e9dd1f6b9f94404d732a76f961d9eb4)

6 years agoceph-volume tests reset the cluster name after every unit test
Alfredo Deza [Mon, 24 Sep 2018 14:29:12 +0000 (10:29 -0400)]
ceph-volume tests reset the cluster name after every unit test

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 877ee1a27e0651355812e27fbc6ac0a87ea56a68)

6 years agoceph-volume tests.functional add custom cluster name to centos7-filestore-create
Alfredo Deza [Wed, 19 Sep 2018 17:23:19 +0000 (13:23 -0400)]
ceph-volume tests.functional add custom cluster name to centos7-filestore-create

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit e61848dc97b638c6b8e60f0896d325f436142864)

6 years agoceph-volume lvm.activate load the config always by reading the cluster name from...
Alfredo Deza [Wed, 19 Sep 2018 16:04:41 +0000 (12:04 -0400)]
ceph-volume lvm.activate load the config always by reading the cluster name from the LV

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit a88cbfc73a258b51b20e060e269234962444ce63)

6 years agoceph-volume configuration be able to set conf.ceph
Alfredo Deza [Wed, 19 Sep 2018 16:04:17 +0000 (12:04 -0400)]
ceph-volume configuration be able to set conf.ceph

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 4f8226f359d94a86a39492ed653d9a9e5d8dd8f9)

6 years agomgr/dashboard: Progress bar does not stop in TableKeyValueComponent 24258/head
Volker Theile [Mon, 10 Sep 2018 15:27:58 +0000 (17:27 +0200)]
mgr/dashboard: Progress bar does not stop in TableKeyValueComponent

Fixes: https://tracker.ceph.com/issues/35907
Signed-off-by: Volker Theile <vtheile@suse.com>
(cherry picked from commit 7243d9b67dc8c15283e30f889eb4d3fd00a5701a)

Conflicts:
src/pybind/mgr/dashboard/frontend/src/app/shared/datatable/table-key-value/table-key-value.component.ts :
: Resolved in TableKeyValueComponent