]> git.apps.os.sepia.ceph.com Git - ceph.git/log
ceph.git
6 years agoMerge pull request #24600 from pdvian/wip-36296-mimic
Yuri Weinstein [Mon, 22 Oct 2018 15:34:02 +0000 (11:34 -0400)]
Merge pull request #24600 from pdvian/wip-36296-mimic

mimic : osdc/Objecter: possible race condition with connection reset

Reviewed-by: Neha Ojha <nojha@redhat.com>
6 years agoMerge pull request #24601 from pdvian/wip-36298-mimic
Yuri Weinstein [Mon, 22 Oct 2018 15:32:42 +0000 (11:32 -0400)]
Merge pull request #24601 from pdvian/wip-36298-mimic

mimic: osd: add creating to pg_string_state

Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>
6 years agoMerge pull request #24379 from smithfarm/wip-36136-mimic
Yuri Weinstein [Fri, 19 Oct 2018 20:51:33 +0000 (16:51 -0400)]
Merge pull request #24379 from smithfarm/wip-36136-mimic

mimic: mds: rctime may go back

Reviewed-by: Yan, Zheng <zyan@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
6 years agoMerge pull request #24384 from smithfarm/wip-32104-mimic
Yuri Weinstein [Fri, 19 Oct 2018 20:50:54 +0000 (16:50 -0400)]
Merge pull request #24384 from smithfarm/wip-32104-mimic

mimic: mds: allows client to create .. and . dirents

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by:  Venky Shankar <vshankar@redhat.com>

6 years agoMerge pull request #24385 from smithfarm/wip-36134-mimic
Yuri Weinstein [Fri, 19 Oct 2018 20:50:20 +0000 (16:50 -0400)]
Merge pull request #24385 from smithfarm/wip-36134-mimic

mimic: client: update ctime when modifying file content

Reviewed-by: Yan, Zheng <zyan@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
6 years agoMerge pull request #24408 from smithfarm/wip-36153-mimic
Yuri Weinstein [Fri, 19 Oct 2018 20:49:25 +0000 (16:49 -0400)]
Merge pull request #24408 from smithfarm/wip-36153-mimic

mimic: qa: fsstress workunit does not execute in parallel on same host without clobbering files

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
6 years agoMerge pull request #24411 from smithfarm/wip-36195-mimic
Yuri Weinstein [Fri, 19 Oct 2018 20:48:52 +0000 (16:48 -0400)]
Merge pull request #24411 from smithfarm/wip-36195-mimic

mimic: mds: internal op missing events time 'throttled', 'all_read', 'dispatched'

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
6 years agoMerge pull request #24438 from pdvian/wip-36156-mimic
Yuri Weinstein [Fri, 19 Oct 2018 20:48:28 +0000 (16:48 -0400)]
Merge pull request #24438 from pdvian/wip-36156-mimic

mimic: qa: extend timeout for SessionMap flush

Reviewed-by: John Spray <john.spray@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
6 years agoMerge pull request #24439 from pdvian/wip-36197-mimic
Yuri Weinstein [Fri, 19 Oct 2018 20:47:52 +0000 (16:47 -0400)]
Merge pull request #24439 from pdvian/wip-36197-mimic

mimic: ceph-fuse: add to selinux profile

Reviewed-by: Boris Ranto <branto@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
6 years agoMerge pull request #24463 from pdvian/wip-36199-mimic
Yuri Weinstein [Fri, 19 Oct 2018 20:47:09 +0000 (16:47 -0400)]
Merge pull request #24463 from pdvian/wip-36199-mimic

mimic:mds: fix mds damaged due to unexpected journal length

Reviewed-by: Yan, Zheng <zyan@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
6 years agoMerge pull request #24464 from pdvian/wip-36205-mimic
Yuri Weinstein [Fri, 19 Oct 2018 20:46:30 +0000 (16:46 -0400)]
Merge pull request #24464 from pdvian/wip-36205-mimic

mimic: libcephfs: expose CEPH_SETATTR_MTIME_NOW and CEPH_SETATTR_ATIME_NOW

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
6 years agoMerge pull request #24470 from smithfarm/wip-36203-mimic
Yuri Weinstein [Fri, 19 Oct 2018 20:45:53 +0000 (16:45 -0400)]
Merge pull request #24470 from smithfarm/wip-36203-mimic

mimic: ceph fs add_data_pool applies pool application metadata incorrectly

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: John Spray <john.spray@redhat.com>
6 years agoMerge pull request #23087 from pdvian/wip-24928-mimic
Yuri Weinstein [Fri, 19 Oct 2018 20:44:39 +0000 (16:44 -0400)]
Merge pull request #23087 from pdvian/wip-24928-mimic

mimic: qa: test_recovery_pool tries asok on wrong node

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
6 years agoMerge pull request #23506 from ukernel/mimic-26858
Yuri Weinstein [Fri, 19 Oct 2018 20:44:14 +0000 (16:44 -0400)]
Merge pull request #23506 from ukernel/mimic-26858

mimic: mds: reset heartbeat map at potential time-consuming places

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
6 years agoMerge pull request #23952 from ukernel/mimic-mds-export-size
Yuri Weinstein [Fri, 19 Oct 2018 20:43:47 +0000 (16:43 -0400)]
Merge pull request #23952 from ukernel/mimic-mds-export-size

mimic: mds: optimize the way how max export size is enforced

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
6 years agoMerge pull request #24161 from pdvian/wip-35719-mimic
Yuri Weinstein [Fri, 19 Oct 2018 20:43:24 +0000 (16:43 -0400)]
Merge pull request #24161 from pdvian/wip-35719-mimic

mimic: mds: print is_laggy message once

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
6 years agoMerge pull request #24187 from pdvian/wip-35841-mimic
Yuri Weinstein [Fri, 19 Oct 2018 20:43:02 +0000 (16:43 -0400)]
Merge pull request #24187 from pdvian/wip-35841-mimic

mimic: client: fix use-after-free in Client::link()

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
6 years agoMerge pull request #24269 from smithfarm/wip-36102-mimic
Yuri Weinstein [Fri, 19 Oct 2018 20:42:29 +0000 (16:42 -0400)]
Merge pull request #24269 from smithfarm/wip-36102-mimic

mimic: qa: remove knfs site from future releases

Reviewed-by: Yuri Weinstein <yweins@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
6 years agoMerge pull request #24272 from batrick/i35858
Yuri Weinstein [Fri, 19 Oct 2018 20:41:34 +0000 (16:41 -0400)]
Merge pull request #24272 from batrick/i35858

mimic: mon: test if gid exists in pending for prepare_beacon

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
6 years agoMerge pull request #24295 from pdvian/wip-35934-mimic
Yuri Weinstein [Fri, 19 Oct 2018 20:40:56 +0000 (16:40 -0400)]
Merge pull request #24295 from pdvian/wip-35934-mimic

mimic: client:two ceph-fuse client, one can not list out files created by an…

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
6 years agoMerge pull request #24377 from smithfarm/wip-35940-mimic
Yuri Weinstein [Fri, 19 Oct 2018 20:40:23 +0000 (16:40 -0400)]
Merge pull request #24377 from smithfarm/wip-35940-mimic

mimic: client: statfs inode count odd

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
6 years agoMerge pull request #24319 from smithfarm/wip-35979-mimic
Yuri Weinstein [Wed, 17 Oct 2018 20:11:38 +0000 (16:11 -0400)]
Merge pull request #24319 from smithfarm/wip-35979-mimic

mimic: multisite: data sync error repo processing does not back off on empty

Reviewed-by: Casey Bodley <cbodley@redhat.com>
6 years agoMerge pull request #23197 from pdvian/wip-25024-mimic
Yuri Weinstein [Wed, 17 Oct 2018 20:09:22 +0000 (16:09 -0400)]
Merge pull request #23197 from pdvian/wip-25024-mimic

mimic: qa/suites/rgw/verify/tasks/cls_rgw: test cls_rgw

Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: Nathan Cutler <ncutler@suse.com>
6 years agoMerge pull request #24071 from pdvian/wip-35708-mimic
Yuri Weinstein [Wed, 17 Oct 2018 20:06:48 +0000 (16:06 -0400)]
Merge pull request #24071 from pdvian/wip-35708-mimic

mimic: rgw: raise default rgw_curl_low_speed_time to 300 seconds

Reviewed-by: Casey Bodley <cbodley@redhat.com>
6 years agoMerge pull request #24101 from pdvian/wip-35710-mimic
Yuri Weinstein [Wed, 17 Oct 2018 20:05:48 +0000 (16:05 -0400)]
Merge pull request #24101 from pdvian/wip-35710-mimic

mimic: rgw: fix deadlock on RGWIndexCompletionManager::stop

Reviewed-by: Casey Bodley <cbodley@redhat.com>
6 years agoMerge pull request #24189 from joke-lee/mimic
Yuri Weinstein [Wed, 17 Oct 2018 20:05:17 +0000 (16:05 -0400)]
Merge pull request #24189 from joke-lee/mimic

mimic:rgw: return x-amz-version-id: null when delete obj in versioning

Reviewed-by: Casey Bodley <cbodley@redhat.com>
6 years agoMerge pull request #24235 from pdvian/wip-35857-mimic
Yuri Weinstein [Wed, 17 Oct 2018 20:04:52 +0000 (16:04 -0400)]
Merge pull request #24235 from pdvian/wip-35857-mimic

mimic: multisite: segfault on shutdown/realm reload

Reviewed-by: Casey Bodley <cbodley@redhat.com>
6 years agoMerge pull request #24363 from pdvian/wip-36125-mimic
Yuri Weinstein [Wed, 17 Oct 2018 20:04:10 +0000 (16:04 -0400)]
Merge pull request #24363 from pdvian/wip-36125-mimic

mimic:rgw: fix chunked-encoding for chunks >1MiB

Reviewed-by: Casey Bodley <cbodley@redhat.com>
6 years agoMerge pull request #24417 from pdvian/wip-36138-mimic
Yuri Weinstein [Wed, 17 Oct 2018 20:01:17 +0000 (16:01 -0400)]
Merge pull request #24417 from pdvian/wip-36138-mimic

mimic: rgw: async sync_object and remove_object does not access coroutine me…

Reviewed-by: Casey Bodley <cbodley@redhat.com>
6 years agoMerge pull request #24571 from joke-lee/mimic-list-RGWPostObj-when-enable-versioning
Yuri Weinstein [Wed, 17 Oct 2018 20:00:41 +0000 (16:00 -0400)]
Merge pull request #24571 from joke-lee/mimic-list-RGWPostObj-when-enable-versioning

mimic:rgw: list bucket can not show the object uploaded by RGWPostObj when enable bucket versioning

Reviewed-by: Casey Bodley <cbodley@redhat.com>
6 years agoMerge pull request #23026 from smithfarm/wip-24889-mimic
Yuri Weinstein [Wed, 17 Oct 2018 19:58:16 +0000 (15:58 -0400)]
Merge pull request #23026 from smithfarm/wip-24889-mimic

mimic: osd: crash in OpTracker::unregister_inflight_op via OSD::get_health_metrics

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
6 years agoMerge pull request #24116 from dzafman/wip-35992
Yuri Weinstein [Wed, 17 Oct 2018 19:57:22 +0000 (15:57 -0400)]
Merge pull request #24116 from dzafman/wip-35992

mimic: tools: ceph-objectstore-tool: Allow target level as first positional …

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
6 years agoMerge pull request #24401 from dillaman/wip-36144-mimic
Yuri Weinstein [Wed, 17 Oct 2018 19:56:50 +0000 (15:56 -0400)]
Merge pull request #24401 from dillaman/wip-36144-mimic

mimic: librbd: blacklisted client might not notice it lost the lock

Reviewed-by: Mykola Golub <mgolub@mirantis.com>
6 years agoMerge pull request #24573 from pdvian/wip-36292-mimic
Yuri Weinstein [Wed, 17 Oct 2018 19:56:08 +0000 (15:56 -0400)]
Merge pull request #24573 from pdvian/wip-36292-mimic

mimic: osd: Remove old bft= which has been superceded by backfill=

Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: David Zafman <dzafman@redhat.com>
6 years agoosd: add creating to pg_string_state 24601/head
Dan van der Ster [Tue, 25 Sep 2018 08:39:37 +0000 (10:39 +0200)]
osd: add creating to pg_string_state

Fixes: http://tracker.ceph.com/issues/36174
Signed-off-by: Dan van der Ster <daniel.vanderster@cern.ch>
(cherry picked from commit d38f6a11701ec788e4d384aa5b0ae65b8e57da64)

Conflicts:
src/osd/osd_types.cc : Resolved in pg_string_state

6 years agoosdc/Objecter: possible race condition with connection reset 24600/head
Jason Dillaman [Tue, 25 Sep 2018 18:18:00 +0000 (14:18 -0400)]
osdc/Objecter: possible race condition with connection reset

If the connection quickly fails before the private session reference
can be associated with the connection, the connection will remain
closed and any OSD ops against the session will remain stuck.

Fixes: http://tracker.ceph.com/issues/36183
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 74ca33cb49d2c258324447b1ca366ed4e604202a)

6 years agoMerge pull request #24388 from pdvian/wip-36129-mimic
Yuri Weinstein [Mon, 15 Oct 2018 20:52:18 +0000 (16:52 -0400)]
Merge pull request #24388 from pdvian/wip-36129-mimic

mimic: rgw: abort_bucket_multiparts() ignores individual NoSuchUpload errors

Reviewed-by: Casey Bodley <cbodley@redhat.com>
6 years agoMerge pull request #24400 from smithfarm/wip-36201-mimic
Yuri Weinstein [Mon, 15 Oct 2018 20:51:53 +0000 (16:51 -0400)]
Merge pull request #24400 from smithfarm/wip-36201-mimic

mimic: multisite: intermittent test_bucket_index_log_trim failures

Reviewed-by: Casey Bodley <cbodley@redhat.com>
6 years agoMerge pull request #24413 from dillaman/wip-36225-mimic
Yuri Weinstein [Mon, 15 Oct 2018 20:51:11 +0000 (16:51 -0400)]
Merge pull request #24413 from dillaman/wip-36225-mimic

mimic: librbd: object map improperly flagged as invalidated

Reviewed-by: Mykola Golub <mgolub@mirantis.com>
6 years agoMerge pull request #24414 from smithfarm/wip-36208-mimic
Yuri Weinstein [Mon, 15 Oct 2018 20:50:38 +0000 (16:50 -0400)]
Merge pull request #24414 from smithfarm/wip-36208-mimic

mimic: multisite: invalid read in RGWCloneMetaLogCoroutine

Reviewed-by: Casey Bodley <cbodley@redhat.com>
6 years agoMerge pull request #24418 from pdvian/wip-36140-mimic
Yuri Weinstein [Mon, 15 Oct 2018 20:50:13 +0000 (16:50 -0400)]
Merge pull request #24418 from pdvian/wip-36140-mimic

mimic: rgw: raise debug level on redundant data sync error messages

Reviewed-by: Casey Bodley <cbodley@redhat.com>
6 years agoMerge pull request #24436 from croit/mimic-36041
Yuri Weinstein [Mon, 15 Oct 2018 20:49:40 +0000 (16:49 -0400)]
Merge pull request #24436 from croit/mimic-36041

mimic: rgw/beast: drop privileges after binding ports

Reviewed-by: Casey Bodley <cbodley@redhat.com>
6 years agoMerge pull request #24481 from pdvian/wip-36150-mimic
Yuri Weinstein [Mon, 15 Oct 2018 20:49:10 +0000 (16:49 -0400)]
Merge pull request #24481 from pdvian/wip-36150-mimic

mimic: crush/CrushWrapper: fix crush tree json dumper

Reviewed-by: Kefu Chai <kchai@redhat.com>
6 years agoMerge pull request #24516 from pdvian/wip-36211-mimic
Yuri Weinstein [Mon, 15 Oct 2018 20:48:38 +0000 (16:48 -0400)]
Merge pull request #24516 from pdvian/wip-36211-mimic

mimic: rgw: RGWAsyncGetBucketInstanceInfo does not access coroutine memory

Reviewed-by: Casey Bodley <cbodley@redhat.com>
6 years agoMerge pull request #24518 from pdvian/wip-36213-mimic
Yuri Weinstein [Mon, 15 Oct 2018 20:48:13 +0000 (16:48 -0400)]
Merge pull request #24518 from pdvian/wip-36213-mimic

mimic:rgw: fix leak of curl handle on shutdown

Reviewed-by: Casey Bodley <cbodley@redhat.com>
6 years agoMerge pull request #24526 from theanalyst/wip-mimic-36381
Yuri Weinstein [Mon, 15 Oct 2018 20:47:55 +0000 (16:47 -0400)]
Merge pull request #24526 from theanalyst/wip-mimic-36381

mimic: rgw: resharding produces invalid values of bucket stats

Reviewed-by: Casey Bodley <cbodley@redhat.com>
6 years agoMerge pull request #24536 from pdvian/wip-36216-mimic
Yuri Weinstein [Mon, 15 Oct 2018 20:47:13 +0000 (16:47 -0400)]
Merge pull request #24536 from pdvian/wip-36216-mimic

mimic: multisite: data full sync does not limit concurrent bucket sync

Reviewed-by: Casey Bodley <cbodley@redhat.com>
6 years agoMerge pull request #24537 from pdvian/wip-36228-mimic
Yuri Weinstein [Mon, 15 Oct 2018 20:46:36 +0000 (16:46 -0400)]
Merge pull request #24537 from pdvian/wip-36228-mimic

mimic: qa: add test that builds example librados programs

Reviewed-by: Neha Ojha <nojha@redhat.com>
6 years agorgw: list bucket can not show the object uploaded by RGWPostObj 24571/head
yuliyang [Sun, 30 Sep 2018 09:26:58 +0000 (17:26 +0800)]
rgw: list bucket can not show the object uploaded by RGWPostObj
when enable bucket versioning

fix https://tracker.ceph.com/issues/36265

Signed-off-by: yuliyang <yuliyang@cmss.chinamobile.com>
(cherry picked from commit 38407f69da898eacf77027d04e2b9544f310ebff)

6 years agoosd: Remove old bft= which has been superceded by backfill= 24573/head
David Zafman [Mon, 24 Sep 2018 21:21:59 +0000 (14:21 -0700)]
osd: Remove old bft= which has been superceded by backfill=

Fixes: http://tracker.ceph.com/issues/36170
Caused by: 1b4f86cb0

Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit 6847d3fd6f7a6cd671cb7b4fa14240f0e6df3c09)

6 years agoqa: add test that builds example librados programs 24537/head
Nathan Cutler [Thu, 19 Jul 2018 15:59:04 +0000 (17:59 +0200)]
qa: add test that builds example librados programs

Fixes: http://tracker.ceph.com/issues/15100
Signed-off-by: Nathan Cutler <ncutler@suse.com>
(cherry picked from commit c46c890d0241972cee10260f071f65b4beedf92c)

6 years agorgw: data sync checks whether the lease failed to renew 24536/head
Casey Bodley [Mon, 13 Aug 2018 19:47:29 +0000 (15:47 -0400)]
rgw: data sync checks whether the lease failed to renew

Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit 3754871e65830e772f9b902fa162853716eeb78d)

 Conflicts:
src/rgw/rgw_data_sync.cc : Resolved in full_sync

6 years agorgw: data sync holds lease over transition from full to incremental
Casey Bodley [Sat, 11 Aug 2018 15:39:35 +0000 (11:39 -0400)]
rgw: data sync holds lease over transition from full to incremental

Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit 3e9ac0f1b3f701902a3c35b61e45d938805ddb26)

6 years agorgw: enforce spawn_window for data full sync
Casey Bodley [Fri, 10 Aug 2018 20:34:42 +0000 (16:34 -0400)]
rgw: enforce spawn_window for data full sync

limit the number of concurrent bucket sync crs spawned by data full sync

Fixes: http://tracker.ceph.com/issues/26897
Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit 895c28a5b43df7fb46cb443b10153abbb48274f6)

6 years agorgw: copy actual stats from the source shards during reshard 24526/head
Abhishek Lekshmanan [Fri, 5 Oct 2018 09:19:18 +0000 (11:19 +0200)]
rgw: copy actual stats from the source shards during reshard

Currently we don't copy the actual_stats field during reshard, which makes
resharded buckets show a size_utilized as 0, which further has the problem that
a subsequent object removal would subtract the object size from the 0 size
utilized showing up large uint64_t values. Copy the size_actual from the source
object in both cls and in reshard_process. This will fix the new buckets,
existing buckets will still have to go through a bucket check --fix for their
stats to be corrected.

Fixes: http://tracker.ceph.com/issues/36290
Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>
(cherry picked from commit beb90638ae3d5329653b61bae0d6714796c41d04)

6 years agorgw: fix leak of curl handle on shutdown 24518/head
Casey Bodley [Wed, 5 Sep 2018 20:56:41 +0000 (16:56 -0400)]
rgw: fix leak of curl handle on shutdown

addresses a valgrind leak reported from curl_multi_add_handle().
requests that are still linked to the curl_multi handle need to be
unlinked and finished with ECANCELED, rather than just finished

Fixes: http://tracker.ceph.com/issues/35715
Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit dd478483962188c1805f522deb7f7c7a33c5fadb)

6 years agoMerge pull request #24176 from pdvian/wip-35836-mimic 24567/head
Neha Ojha [Wed, 10 Oct 2018 02:05:46 +0000 (19:05 -0700)]
Merge pull request #24176 from pdvian/wip-35836-mimic

mimic: mon: mgr options not parse propertly

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: John Spray <john.spray@redhat.com>
6 years agoMerge pull request #24281 from pdvian/wip-35930-mimic
Neha Ojha [Wed, 10 Oct 2018 02:00:09 +0000 (19:00 -0700)]
Merge pull request #24281 from pdvian/wip-35930-mimic

mimic: mon/OSDMonitor: invalidate max_failed_since on cancel_report

Reviewed-by: Xie Xingguo <xie.xingguo@zte.com.cn>
6 years agorgw: RGWAsyncGetBucketInstanceInfo does not access coroutine memory 24516/head
Casey Bodley [Fri, 7 Sep 2018 20:13:27 +0000 (16:13 -0400)]
rgw: RGWAsyncGetBucketInstanceInfo does not access coroutine memory

Fixes: http://tracker.ceph.com/issues/35812
Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit 6fb61c5ed8eb7cbe6d1271a8b61e465023d2e621)

6 years agocrush/CrushWrapper: fix crush tree json dumper 24481/head
songshuangyang [Fri, 14 Sep 2018 06:53:17 +0000 (14:53 +0800)]
crush/CrushWrapper: fix crush tree json dumper

    The output json string is invalid for 'osd crush tree --format=json'
    command. It contains a array of 'nodes' and a array of 'stray', but
    not in a json object, and the stray array was not implemented.
    Applications which depends on the output of the above MonCommand will
    occur json parse error.

Signed-off-by: Oshyn Song <dualyangsong@gmail.com>
(cherry picked from commit 35c0d1f45cd676f201d4031cb8f447f7ea6aee0e)

6 years agomon: fix fs add_data_pool setting wrong pool metadata 24470/head
John Spray [Mon, 17 Sep 2018 07:19:33 +0000 (08:19 +0100)]
mon: fix fs add_data_pool setting wrong pool metadata

Fixes: http://tracker.ceph.com/issues/36028
Signed-off-by: John Spray <john.spray@redhat.com>
(cherry picked from commit f162fa536536fb3171519b1fb0e3f2b09d822464)

6 years agolibcephfs: expose CEPH_SETATTR_MTIME_NOW and CEPH_SETATTR_ATIME_NOW 24464/head
Zhu Shangzhong [Wed, 19 Sep 2018 06:18:29 +0000 (14:18 +0800)]
libcephfs: expose CEPH_SETATTR_MTIME_NOW and CEPH_SETATTR_ATIME_NOW

Fixes: http://tracker.ceph.com/issues/35961
Signed-off-by: Zhu Shangzhong <zhu.shangzhong@zte.com.cn>
(cherry picked from commit f39cde345d764d865d504ffd9b60abd5aee90b18)

6 years agomds: fix mds damaged due to unexpected journal length 24463/head
Zhi Zhang [Thu, 20 Sep 2018 02:55:46 +0000 (10:55 +0800)]
mds: fix mds damaged due to unexpected journal length

We used first next_safe_pos in pending_safe to update journal's safe_pos, but
it might not be flushed at this time, so journal's header may get incorrect
write_pos in some case, which could cause mds damaged.

This fix is trying to use current min next_safe_pos in pending_safe to update
journal's safe_pos.

Signed-off-by: Zhi Zhang <zhangz.david@outlook.com>
(cherry picked from commit d17790f299c282dff9c28569b10c46a54b4cfd51)

Conflicts:
src/osdc/Journaler.cc : Resolved in _finish_flush

6 years agoMerge pull request #24113 from smithfarm/wip-35717-mimic
Yuri Weinstein [Sat, 6 Oct 2018 16:15:48 +0000 (09:15 -0700)]
Merge pull request #24113 from smithfarm/wip-35717-mimic

mimic: msg: challenging authorizer messages appear at debug_ms=0

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
6 years agoMerge pull request #24241 from smithfarm/wip-36121-mimic
Yuri Weinstein [Sat, 6 Oct 2018 16:14:49 +0000 (09:14 -0700)]
Merge pull request #24241 from smithfarm/wip-36121-mimic

mimic: kernel_untar_build.sh: bison: command not found

Reviewed-by: Neha Ojha <nojha@redhat.com>
6 years agoMerge pull request #24298 from pdvian/wip-35959-mimic
Yuri Weinstein [Sat, 6 Oct 2018 16:14:22 +0000 (09:14 -0700)]
Merge pull request #24298 from pdvian/wip-35959-mimic

mimic: osd/ECBackend: don't get result code of subchunk-read overwritten

Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>
Reviewed-by: Neha Ojha <nojha@redhat.com>
6 years agoMerge pull request #24309 from neha-ojha/wip-36239-mimic
Yuri Weinstein [Sat, 6 Oct 2018 16:13:49 +0000 (09:13 -0700)]
Merge pull request #24309 from neha-ojha/wip-36239-mimic

mimic: osd/PrimaryLogPG: fix potential pg-log overtrimming

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>
6 years agoMerge pull request #24344 from pdvian/wip-35963-mimic
Yuri Weinstein [Sat, 6 Oct 2018 16:13:11 +0000 (09:13 -0700)]
Merge pull request #24344 from pdvian/wip-35963-mimic

mimic: osd/PG: avoid choose_acting picking want with > pool size items

Reviewed-by: Neha Ojha <nojha@redhat.com>
6 years agoMerge pull request #24345 from pdvian/wip-35964-mimic
Yuri Weinstein [Sat, 6 Oct 2018 16:12:48 +0000 (09:12 -0700)]
Merge pull request #24345 from pdvian/wip-35964-mimic

mimic: RADOS: probably missing clone location for async_recovery_targets

Reviewed-by: Neha Ojha <nojha@redhat.com>
6 years agoMerge pull request #24233 from liewegas/wip-35998-mimic
Yuri Weinstein [Fri, 5 Oct 2018 21:41:00 +0000 (14:41 -0700)]
Merge pull request #24233 from liewegas/wip-35998-mimic

mimic: mgr/DaemonServer: fix Session leak

Reviewed-by: Neha Ojha <nojha@redhat.com>
6 years agoMerge pull request #24339 from ifed01/wip-ifed-fix-repairer-mimic
Yuri Weinstein [Fri, 5 Oct 2018 21:19:25 +0000 (14:19 -0700)]
Merge pull request #24339 from ifed01/wip-ifed-fix-repairer-mimic

mimic: os/bluestore: fix bloom filter num entry miscalculation in repairer

Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>
6 years agoMerge pull request #24352 from ifed01/wip-ifed-bluestore-tool-fixes-mimic
Yuri Weinstein [Fri, 5 Oct 2018 21:18:49 +0000 (14:18 -0700)]
Merge pull request #24352 from ifed01/wip-ifed-bluestore-tool-fixes-mimic

mimic: ceph-bluestore-tool: fix set label functionality for specific keys

Reviewed-by: Sage Weil <sage@redhat.com>
6 years agoMerge pull request #24386 from pdvian/wip-36127-mimic
Yuri Weinstein [Fri, 5 Oct 2018 21:18:12 +0000 (14:18 -0700)]
Merge pull request #24386 from pdvian/wip-36127-mimic

mimic: msg/async: clean up local buffers on dispatch

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Ricardo Dias <rdias@suse.com>
6 years agoqa: add timeout to cleaning up workunit sandbox 24408/head
Patrick Donnelly [Sun, 30 Sep 2018 00:37:12 +0000 (17:37 -0700)]
qa: add timeout to cleaning up workunit sandbox

If there is a bug preventing rm from completing, the workunit will get stuck.

Fixes: http://tracker.ceph.com/issues/36184
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit 3a10d74f3aa4901dd9edffc0061992073ae67085)

6 years agoqa: cleanup workunit dir for each unit
Patrick Donnelly [Mon, 24 Sep 2018 18:29:10 +0000 (11:29 -0700)]
qa: cleanup workunit dir for each unit

This was wrongly dropped and moved to the finalizer.

Introduced-by: de824f74dd8ac909e47335ccd53d7a085e388e41
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit 70844f3f55004024a747854013a1efb409705d81)

6 years agoqa: add timeout to kclient umount
Patrick Donnelly [Sun, 30 Sep 2018 00:34:37 +0000 (17:34 -0700)]
qa: add timeout to kclient umount

Otherwise QA sits forever waiting for the kclient to umount when there is a
problem.

Fixes: http://tracker.ceph.com/issues/36184
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit 7a64eb9dfb908a1a8e5d2b0dcaa7ca9df52a9ab1)

6 years agoqa: use default timeout in fs workunits
Patrick Donnelly [Mon, 1 Oct 2018 01:10:05 +0000 (18:10 -0700)]
qa: use default timeout in fs workunits

Six hours is unnecessarily long.

Fixes: http://tracker.ceph.com/issues/36184
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit bdd2ddcfd862b65dfd73bc1ea09b0ad07040d445)

6 years agoqa: do not cleanup sandbox on error
Patrick Donnelly [Wed, 26 Sep 2018 14:38:58 +0000 (07:38 -0700)]
qa: do not cleanup sandbox on error

Otherwise the command will hang if the mount is broken.

Fixes: http://tracker.ceph.com/issues/36184
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit d4b8f94cf8d95ebb277b550fc6ebc3468052a39c)

6 years agoceph-fuse: add to selinux profile 24439/head
Patrick Donnelly [Thu, 20 Sep 2018 17:27:43 +0000 (10:27 -0700)]
ceph-fuse: add to selinux profile

Fixes: http://tracker.ceph.com/issues/36103
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit 2bfaac762d283a0e000074f79f704033afd6dead)

6 years agoqa: extend timeout for SessionMap flush 24438/head
Patrick Donnelly [Mon, 14 May 2018 22:19:51 +0000 (15:19 -0700)]
qa: extend timeout for SessionMap flush

Fixes: https://tracker.ceph.com/issues/24129
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit 644a535ee66eaa9fd5e03eda2b2ba09b8a8a1bd9)

6 years agorgw/beast: drop privileges after binding ports 24436/head
Paul Emmerich [Tue, 25 Sep 2018 14:25:55 +0000 (14:25 +0000)]
rgw/beast: drop privileges after binding ports

Fixes: http://tracker.ceph.com/issues/36041
Signed-off-by: Paul Emmerich <paul.emmerich@croit.io>
(cherry picked from commit 6489566e867cee81a753b246913b61815819d9b3)

 Conflicts:
src/rgw/rgw_asio_frontend.cc

6 years agolibrbd: keep IO blocked until after snapshot object map created 24413/head
Jason Dillaman [Mon, 24 Sep 2018 19:07:15 +0000 (15:07 -0400)]
librbd: keep IO blocked until after snapshot object map created

The IO was being unblocked before object map was created, allowing
a potential copyup request to fail to update a still-to-be-created
object map.

Fixes: http://tracker.ceph.com/issues/24516
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 1e874403bf861cb8b74261308d8b73434cf90341)

Conflicts:
src/librbd/object_map/SnapshotCreateRequest.cc: trivial resolution
src/librbd/operation/SnapshotCreateRequest.cc: trivial resolution

6 years agolibrbd: do not invalidate object map if update races with copyup
Jason Dillaman [Mon, 24 Sep 2018 18:45:09 +0000 (14:45 -0400)]
librbd: do not invalidate object map if update races with copyup

The copyup state machine needs to iterate over all object maps to update
the existence for the object. If an snapshot is being removed concurrently,
it's possible to invalidate the object map for the image.

Fixes: http://tracker.ceph.com/issues/24516
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 5a1cb469879157297ab456261f9335d8b855684f)

Conflicts:
src/librbd/ObjectMap.cc: trivial resolution
src/librbd/ObjectMap.h: trivial resolution
src/librbd/io/CopyupRequest.cc: trivial resolution
src/test/librbd/deep_copy/test_mock_ObjectCopyRequest.cc: trivial resolution
src/test/librbd/test_mock_ObjectMap.cc: trivial resolution

6 years agolibrbd: do not invalidate object map when attempting to delete non-existent snapshot
Jason Dillaman [Fri, 14 Sep 2018 15:46:13 +0000 (11:46 -0400)]
librbd: do not invalidate object map when attempting to delete non-existent snapshot

If duplicate snapshot remove requests are received by the lock owner from a peer
client, the first request will remove the object map. If the second request
arrives while the first is in-progress, it will again attempt to remove the
object map but fail to load it since it's already been deleted. This incorrectly
results in the next object map being flagged as invalid.

Fixes: http://tracker.ceph.com/issues/24516
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 0a31c55ea83d85da88c7586c9a8fa8d6ec6618a7)

Conflicts:
src/librbd/object_map/SnapshotRemoveRequest.cc: trivial resolution

6 years agolibrbd: use the correct error code when the exclusive lock isn't locked 24401/head
Jason Dillaman [Thu, 6 Sep 2018 21:08:12 +0000 (17:08 -0400)]
librbd: use the correct error code when the exclusive lock isn't locked

If the client is currently blacklisted, use -EBLACKLISTED, otherwise
use -EROFS.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit e8eee15518facf562adf1aaba02d3a9523cdd2c3)

Conflicts:
src/librbd/ExclusiveLock.cc: trivial resolution
src/librbd/Operations.cc: trivial resolution
src/librbd/deep_copy/ObjectCopyRequest.cc: trivial resolution
src/librbd/deep_copy/SetHeadRequest.cc: trivial resolution
src/librbd/deep_copy/SnapshotCopyRequest.cc: trivial resolution
src/librbd/deep_copy/SnapshotCreateRequest.cc: trivial resolution

6 years agolibrbd: helper to retrieve the correct error code for read-only op
Jason Dillaman [Thu, 6 Sep 2018 21:15:50 +0000 (17:15 -0400)]
librbd: helper to retrieve the correct error code for read-only op

When the exclusive lock is unlocked, the error code should be
-EBLACKLISTED when the client is blacklisted, otherwise -EROFS.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit a84fbb2565fb603ea809487d920461d14442d188)

6 years agolibrbd: reacquire lock should properly handle failed watcher
Jason Dillaman [Thu, 6 Sep 2018 17:38:17 +0000 (13:38 -0400)]
librbd: reacquire lock should properly handle failed watcher

If the watch has been lost, assume the lock has been lost but attempt
to reacquire it if and when the watch is re-established.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 2057d99f451e3007d4fd05a88faa968319d0ba90)

Conflicts:
src/librbd/ManagedLock.cc: trivial resolution

6 years agolibrbd: assume lock is unlocked if blacklisted or object deleted
Jason Dillaman [Thu, 30 Aug 2018 19:12:27 +0000 (15:12 -0400)]
librbd: assume lock is unlocked if blacklisted or object deleted

This will ensure that it's possible to potentially re-acquire the
lock should the blacklist expire before the image is closed.

Fixes: http://tracker.ceph.com/issues/34534
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 60064f68f5dd2bbf5fbab95564fa522335091f4a)

6 years agolibrbd: watcher should internally track blacklisted state
Jason Dillaman [Thu, 6 Sep 2018 13:44:59 +0000 (09:44 -0400)]
librbd: watcher should internally track blacklisted state

Since it will periodically attempt to re-acquire the watch,
it will know when the RADOS client has been blacklisted and
when the blacklist has been removed.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 9ea94f284061849e452dd61c8f89ecca18642b0d)

Conflicts:
src/librbd/Watcher.cc: trivial resolution

6 years agolibrbd: attempt to recover lost image watcher upon all failures
Jason Dillaman [Thu, 30 Aug 2018 20:51:10 +0000 (16:51 -0400)]
librbd: attempt to recover lost image watcher upon all failures

For example, if an image is blacklisted and the blacklist eventually
expires, the image should recover its watch.

Fixes: http://tracker.ceph.com/issues/34534
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 23b7447f6be87a14f84664f29431d2fdd2af4512)

Conflicts:
src/librbd/watcher/RewatchRequest.cc: trivial resolution
src/test/librbd/test_mock_Watcher.cc: trivial resolution

6 years agoMerge pull request #24420 from karunjosy/wip-36308-mimic
Nathan Cutler [Thu, 4 Oct 2018 09:39:52 +0000 (11:39 +0200)]
Merge pull request #24420 from karunjosy/wip-36308-mimic

mimic: doc: Typo error on cephfs/fuse/

Reviewed-by: Nathan Cutler <ncutler@suse.com>
6 years agodoc: Fix typo error on cephfs/fuse/ 24420/head
Karun Josy [Tue, 2 Oct 2018 11:53:25 +0000 (17:23 +0530)]
doc: Fix typo error on cephfs/fuse/

Fixes: http://tracker.ceph.com/issues/36180
Signed-off-by: Karun Josy <kjosy@redhat.com>
(cherry picked from commit d312cdbd36d9c0e76409feac1f3e289b247dc2dc)

6 years agorgw: raise debug level on redundant data sync error messages 24418/head
Casey Bodley [Fri, 7 Sep 2018 13:51:56 +0000 (09:51 -0400)]
rgw: raise debug level on redundant data sync error messages

each of these errors have already been logged at a lower level with a
more detailed error message. by logging them as ERRORs at level 0 here,
the messages could be easily confused as separate failures

Fixes: http://tracker.ceph.com/issues/35830
Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit a6e392f2b8cb885f6fede5dfa9b584659740e161)

Conflicts:
src/rgw/rgw_data_sync.cc : Resolved in operate

6 years agorgw: async sync_object and remove_object does not access coroutine memory 24417/head
Tianshan Qu [Mon, 10 Sep 2018 10:00:45 +0000 (18:00 +0800)]
rgw: async sync_object and remove_object does not access coroutine memory

Fixes: http://tracker.ceph.com/issues/35905
Signed-off-by: Tianshan Qu <tianshan@xsky.com>
(cherry picked from commit 2d38306e9333772a21ffdc9d92838e3b6b5c3148)

6 years agorgw: dont access rgw_http_req_data::client of canceled request 24414/head
Casey Bodley [Fri, 7 Sep 2018 21:01:25 +0000 (17:01 -0400)]
rgw: dont access rgw_http_req_data::client of canceled request

if a request has been canceled with RGWHTTPClient::cancel(), the client
may have been destroyed. check req_data->registered before reading
from req_data->client->receive_pause_skip

Fixes: http://tracker.ceph.com/issues/35851
Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit cdc876a3c5e3e8874c633b9355dfea51c1ee3bdf)

6 years agomds/MDCache: fix mds internal op missing events time 24411/head
Yanhu Cao [Wed, 19 Sep 2018 02:32:48 +0000 (10:32 +0800)]
mds/MDCache: fix mds internal op missing events time

Fixes: http://tracker.ceph.com/issues/36114
Signed-off-by: Yanhu Cao <gmayyyha@gmail.com>
(cherry picked from commit bd6ae6f4e29ac79e5e07373f52099338e6ab5416)

6 years agoqa: use sudo to cleanup workspace
Patrick Donnelly [Mon, 24 Sep 2018 18:02:49 +0000 (11:02 -0700)]
qa: use sudo to cleanup workspace

Files in scratch_tmp may not be owned by ubuntu.

Fixes: http://tracker.ceph.com/issues/36165
Introduced-by: de824f74dd8ac909e47335ccd53d7a085e388e41
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit 1eaf78a75498d0f739b40bf310d036c851465fad)

6 years agoqa: cleanup parallel execution of fsstress
Patrick Donnelly [Tue, 18 Sep 2018 21:57:05 +0000 (14:57 -0700)]
qa: cleanup parallel execution of fsstress

Two instances of fsstress clobber each other. Just build it in the local sandbox.

Fixes: http://tracker.ceph.com/issues/24177
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit de824f74dd8ac909e47335ccd53d7a085e388e41)

6 years agolibrbd: converted object map snapshot remove state machine to new style
Jason Dillaman [Fri, 14 Sep 2018 15:21:28 +0000 (11:21 -0400)]
librbd: converted object map snapshot remove state machine to new style

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 58770188ab57a53b786cf616ccfbf6acfcdc115a)

Conflicts:
src/librbd/object_map/SnapshotRemoveRequest.cc: trivial resolution

6 years agolibrbd: test_flags helper should require snap id parameter
Jason Dillaman [Fri, 14 Sep 2018 13:59:35 +0000 (09:59 -0400)]
librbd: test_flags helper should require snap id parameter

The HEAD and snapshots have potentially different flag states
since object maps get invalidated per revision.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 862082792d9c2ff23823e46937b7de9a42830cfd)

Conflicts:
src/librbd/ObjectMap.cc: trivial resolution
src/test/librbd/test_Migration.cc: DNE