]> git.apps.os.sepia.ceph.com Git - ceph.git/log
ceph.git
7 years agorbd-mirror: make 'flush' asok command asynchronous 22039/head
Mykola Golub [Wed, 16 May 2018 07:40:29 +0000 (10:40 +0300)]
rbd-mirror: make 'flush' asok command asynchronous

Signed-off-by: Mykola Golub <mgolub@suse.com>
(cherry picked from commit 6fce4cabe7874efef780c7adce8479db8571afe7)

7 years agorbd-mirror: don't reregister asok hook in process entry thread
Mykola Golub [Wed, 16 May 2018 07:34:50 +0000 (10:34 +0300)]
rbd-mirror: don't reregister asok hook in process entry thread

It might get stuck if the asok is busy executing a command.
Instead kick status update, which will call reregister on finish.

Fixes: http://tracker.ceph.com/issues/24141
Signed-off-by: Mykola Golub <mgolub@suse.com>
(cherry picked from commit 43e50bd582ba2d1b66f55d7f8d2b7f030da99ee6)

7 years agoMerge pull request #21836 from tchaikov/mimic-23962
Sage Weil [Wed, 16 May 2018 15:52:19 +0000 (10:52 -0500)]
Merge pull request #21836 from tchaikov/mimic-23962

mimic: pybind/ceph_daemon: expand the order of magnitude of

Reviewed-by: Sage Weil <sage@redhat.com>
7 years agoMerge pull request #21862 from cbodley/wip-gcc-8-mimic
Sage Weil [Wed, 16 May 2018 15:51:47 +0000 (10:51 -0500)]
Merge pull request #21862 from cbodley/wip-gcc-8-mimic

mimic: cmake,common,filestore: silence gcc-8 warnings/errors

7 years agoMerge pull request #21958 from tchaikov/mimic-24000
Sage Weil [Wed, 16 May 2018 15:51:09 +0000 (10:51 -0500)]
Merge pull request #21958 from tchaikov/mimic-24000

mimic: osdc: self-managed snapshot helper should catch decode exception

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
Reviewed-by: Greg Farnum <gfarnum@redhat.com>
7 years agoMerge pull request #21959 from tchaikov/mimic-23899
Sage Weil [Wed, 16 May 2018 15:50:30 +0000 (10:50 -0500)]
Merge pull request #21959 from tchaikov/mimic-23899

mimic: osd: clean up smart probe

Reviewed-by: Sage Weil <sage@redhat.com>
7 years agoMerge pull request #21974 from tchaikov/mimic-22302
Sage Weil [Wed, 16 May 2018 15:50:03 +0000 (10:50 -0500)]
Merge pull request #21974 from tchaikov/mimic-22302

mimic: selinux: Allow ceph to execute ldconfig

Reviewed-by: Ken Dreyer <kdreyer@redhat.com>
Reviewed-by: Boris Ranto <branto@redhat.com>
7 years agoMerge pull request #21977 from votdev/info_no_rgw_mimic
Lenz Grimmer [Wed, 16 May 2018 09:43:42 +0000 (11:43 +0200)]
Merge pull request #21977 from votdev/info_no_rgw_mimic

mimic: mgr/dashboard: Display notification if RGW is not configured

Reviewed-by: Ricardo Dias <rdias@suse.com>
Reviewed-by: Tatjana Dehler <tdehler@suse.com>
7 years agoMerge PR #21988 into mimic
Patrick Donnelly [Tue, 15 May 2018 13:46:33 +0000 (06:46 -0700)]
Merge PR #21988 into mimic

* refs/pull/21988/head:
mds: handle imported session race
mds: check for session import race

Reviewed-by: Zheng Yan <zyan@redhat.com>
7 years agoMerge pull request #21978 from votdev/improve_error_panel_mimic
Lenz Grimmer [Tue, 15 May 2018 11:30:02 +0000 (13:30 +0200)]
Merge pull request #21978 from votdev/improve_error_panel_mimic

mimic: mgr/dashboard: Improve error panel

Reviewed-by: Ricardo Marques <rimarques@suse.com>
Reviewed-by: Stephan Müller <smueller@suse.com>
7 years agomds: handle imported session race 21988/head
Patrick Donnelly [Mon, 14 May 2018 04:27:51 +0000 (21:27 -0700)]
mds: handle imported session race

Thanks to Yan Zheng for identifying and reviewing the fix.

Fixes: http://tracker.ceph.com/issues/24087
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit b9c3751e9e375c679bada1fab228b52420cf7a4b)

7 years agomds: check for session import race
Patrick Donnelly [Mon, 14 May 2018 02:58:17 +0000 (19:58 -0700)]
mds: check for session import race

Credit to Yan Zheng for identifying the race condition [1].

[1] https://bugzilla.redhat.com/show_bug.cgi?id=1572555#c7

Test-for: http://tracker.ceph.com/issues/24072

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit fbc25b44865f6c13c9a9c4710049f9e37169747b)

7 years agoMerge pull request #21970 from wido/mgr-brag
Kefu Chai [Mon, 14 May 2018 15:31:14 +0000 (23:31 +0800)]
Merge pull request #21970 from wido/mgr-brag

mgr/telemetry: Add Ceph Telemetry module to send reports back to project

Reviewed-by: Kefu Chai <kchai@redhat.com>
7 years agoMerge pull request #21956 from tchaikov/mimic-pr-21942
Sage Weil [Mon, 14 May 2018 13:46:37 +0000 (08:46 -0500)]
Merge pull request #21956 from tchaikov/mimic-pr-21942

mimic: qa/workunits/ceph-disk: --no-mon-config

7 years agomgr/dashboard: Improve error panel 21978/head
Volker Theile [Mon, 14 May 2018 12:57:43 +0000 (14:57 +0200)]
mgr/dashboard: Improve error panel

Signed-off-by: Volker Theile <vtheile@suse.com>
(cherry picked from commit a8dae80dce973c441db3d4b914febd4b48725f1e)

7 years agomgr/dashboard: Display useful info if RGW is not configured 21977/head
Volker Theile [Mon, 14 May 2018 12:28:51 +0000 (14:28 +0200)]
mgr/dashboard: Display useful info if RGW is not configured

Signed-off-by: Volker Theile <vtheile@suse.com>
(cherry picked from commit 83441af93cae11e62286a99e65e470a5788e8429)

7 years agomgr/dashboard: Add ModuleStatusGuardService route guard
Volker Theile [Mon, 14 May 2018 12:28:35 +0000 (14:28 +0200)]
mgr/dashboard: Add ModuleStatusGuardService route guard

Signed-off-by: Volker Theile <vtheile@suse.com>
(cherry picked from commit f49e192dd56fb471961cebe71583c16b823e0bed)

7 years agomgr/dashboard: Add info panel component
Volker Theile [Mon, 14 May 2018 12:27:50 +0000 (14:27 +0200)]
mgr/dashboard: Add info panel component

Signed-off-by: Volker Theile <vtheile@suse.com>
(cherry picked from commit d1b179d757aac258d3ce0abc162ed8f6550a535a)

7 years agoMerge pull request #21976 from ukernel/mimic-revert-23837
Yan, Zheng [Mon, 14 May 2018 12:00:37 +0000 (20:00 +0800)]
Merge pull request #21976 from ukernel/mimic-revert-23837

Revert "ceph-fuse: Delete inode's bufferhead was in Tx state would le…

7 years agoRevert "ceph-fuse: Delete inode's bufferhead was in Tx state would lead a assert... 21976/head
Yan, Zheng [Mon, 14 May 2018 11:59:01 +0000 (19:59 +0800)]
Revert "ceph-fuse: Delete inode's bufferhead was in Tx state would lead a assert fail"

This reverts commit ede7817a36e7b10f01f906000ce3ed621c0fc358.
Signed-off-by: "Yan, Zheng" <zyan@redhat.com>
7 years agoselinux: Allow ceph to block suspend 21974/head
Boris Ranto [Wed, 11 Apr 2018 13:25:59 +0000 (15:25 +0200)]
selinux: Allow ceph to block suspend

Ceph is now trying to block suspend in certain cases, we need to update
the policy accordingly.

Signed-off-by: Boris Ranto <branto@redhat.com>
(cherry picked from commit e3be372d854f9014ca711f14525ca9e510d34f1a)

7 years agoselinux: Allow ceph to execute ldconfig
Boris Ranto [Thu, 25 Jan 2018 15:31:30 +0000 (16:31 +0100)]
selinux: Allow ceph to execute ldconfig

The ceph-volume testing showed that the ceph daemons can run ldconfig in
a corner case when they are forbidden access to some files. This patch
allows ceph to execute ldconfig in Enforcing mode.

Fixes: https://tracker.ceph.com/issues/22302
Signed-off-by: Boris Ranto <branto@redhat.com>
(cherry picked from commit fa5071b6d7182f54cd7b1ffe171a4b006f5255cb)

7 years agomgr/telemetry: Add Ceph Telemetry module to send reports back to project 21970/head
Wido den Hollander [Fri, 27 Apr 2018 07:14:05 +0000 (09:14 +0200)]
mgr/telemetry: Add Ceph Telemetry module to send reports back to project

This Manager Module will send statistics and version information from
a Ceph cluster back to telemetry.ceph.com if the user has opted-in on sending
this information.

Additionally a user can tell that the information is allowed to be made
public which then allows other users to see this information.

Signed-off-by: Wido den Hollander <wido@42on.com>
7 years agobrag: Remove ceph-brag
Wido den Hollander [Fri, 27 Apr 2018 07:13:14 +0000 (09:13 +0200)]
brag: Remove ceph-brag

To be replaced by Telemetry module in Ceph Mgr

Signed-off-by: Wido den Hollander <wido@42on.com>
7 years agoosd: clean up smart probe code a bit 21959/head
Sage Weil [Fri, 11 May 2018 20:21:46 +0000 (15:21 -0500)]
osd: clean up smart probe code a bit

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit eea18b45daba33bfdab25ec3879d1c9a5297dc1e)

7 years agoosd: fix osd smart data collection segment fault issue.
Gu Zhongyan [Fri, 27 Apr 2018 10:20:02 +0000 (18:20 +0800)]
osd: fix osd smart data collection segment fault issue.
output.read_fd() could return zero length. it needs to
be handled gracefully.

always call invoke join() if spawn() is successfully returned.
otherwise there will be assert failure in ~SubProcess().

Fixes: http://tracker.ceph.com/issues/23899
Signed-off-by: Gu Zhongyan <guzhongyan@360.cn>
(cherry picked from commit 00c5a30fa9b056ec97ff0b56e195b9a21dd1c136)

7 years agomon: pool-ops against non-existent pools should return error 21958/head
Jason Dillaman [Wed, 9 May 2018 15:45:51 +0000 (11:45 -0400)]
mon: pool-ops against non-existent pools should return error

Fixes: http://tracker.ceph.com/issues/24000
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 43dbdb9d2997171868a590671fec3afe7089c3f3)

7 years agoosdc: self-managed snapshot helper should catch decode exception
Jason Dillaman [Thu, 3 May 2018 19:23:44 +0000 (15:23 -0400)]
osdc: self-managed snapshot helper should catch decode exception

Fixes: http://tracker.ceph.com/issues/24000
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 43f7b17129ea8ab0a98f615ffc4d78ccad98309f)

7 years agoqa/workunits/ceph-disk: --no-mon-config 21956/head
Kefu Chai [Fri, 11 May 2018 13:06:51 +0000 (21:06 +0800)]
qa/workunits/ceph-disk: --no-mon-config

port a095ec8b to the qa/workunits

Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit ee33dc8e53fb41b8ee23783a969ac5288d3211eb)

7 years agoMerge PR #21954 into mimic
Patrick Donnelly [Fri, 11 May 2018 22:15:23 +0000 (15:15 -0700)]
Merge PR #21954 into mimic

* refs/pull/21954/head:
ceph-fuse: Delete inode's bufferhead was in Tx state would lead a assert fail

7 years agoceph-fuse: Delete inode's bufferhead was in Tx state would lead a assert fail 21954/head
YunfeiGuan [Tue, 8 May 2018 11:35:32 +0000 (19:35 +0800)]
ceph-fuse: Delete inode's bufferhead was in Tx state would lead a assert fail

Prematurely delted the bh which was in Tx state may lead the object can't
be closed before its writer of this bh callback. Thus if inode's ref call
put_inode decrease ref to zero and release inode's oset.An assert fail occur
beacuse the oset can't be emptied.

If gather.has_subs() is true, we can't call flush_set_callback() because
this would cause a double drop of FILE_CACHE | FILE_BUFFER.

Fixes:http://tracker.ceph.com/issues/23837
Signed-off-by: Guan yunfei <yunfei.guan@xtaotech.com>
(cherry picked from commit 07e3bceea78dc8ecd76abb1cafca5c9d1fde521e)

7 years agoMerge PR #21953 into mimic
Patrick Donnelly [Fri, 11 May 2018 22:12:11 +0000 (15:12 -0700)]
Merge PR #21953 into mimic

* refs/pull/21953/head:
mds: properly reconnect client caps after loading inodes

7 years agomds: properly reconnect client caps after loading inodes 21953/head
Yan, Zheng [Wed, 2 May 2018 02:23:33 +0000 (10:23 +0800)]
mds: properly reconnect client caps after loading inodes

Commit e43c02d6 "mds: filter out blacklisted clients when importing
caps" makes MDCache::process_imported_caps() ignore clients that are
not in MDCache::rejoin_imported_session_map. The map does not contain
clients from which mds has received reconnect messages. This causes
some client caps (corresponding inodes were not in cache when mds was
in reconnect state) to get dropped.

Signed-off-by: "Yan, Zheng" <zyan@redhat.com>
(cherry picked from commit 48f60e7f274de9d76499816a528eff859bb161e3)

7 years agoMerge PR #21952 into mimic
Patrick Donnelly [Fri, 11 May 2018 22:05:46 +0000 (15:05 -0700)]
Merge PR #21952 into mimic

* refs/pull/21952/head:
mds: tighten conditions of calling rejoin_gather_finish()
mds: avoid calling rejoin_gather_finish() two times successively

7 years agomds: tighten conditions of calling rejoin_gather_finish() 21952/head
Yan, Zheng [Tue, 8 May 2018 03:32:01 +0000 (11:32 +0800)]
mds: tighten conditions of calling rejoin_gather_finish()

Handle two cases:
1. mds receives all cache rejoin messages, then receives mdsmap that
   says mds cluster enters rejoining state.
2. when opening undef inodes/dirfrags, other mds restarts.

Signed-off-by: "Yan, Zheng" <zyan@redhat.com>
(cherry picked from commit 0a38a499b86c0ee13aa0e783a8359bcce0876088)

7 years agomds: avoid calling rejoin_gather_finish() two times successively
Yan, Zheng [Tue, 8 May 2018 02:42:05 +0000 (10:42 +0800)]
mds: avoid calling rejoin_gather_finish() two times successively

If MDCache::rejoin_gather is empty and MDCache::rejoins_pending is true
when MDCache::process_imported_caps() calls maybe_send_pending_rejoins()
Both MDCache::rejoin_send_rejoins() and MDCache::process_imported_caps()
may call rejoin_gather_finish().

Signed-off-by: "Yan, Zheng" <zyan@redhat.com>
Fixes: http://tracker.ceph.com/issues/24047
(cherry picked from commit 0451dae777a2a9b1e70303d7bbc4398849f45f3e)

7 years agoMerge PR #21951 into mimic
Patrick Donnelly [Fri, 11 May 2018 22:01:52 +0000 (15:01 -0700)]
Merge PR #21951 into mimic

* refs/pull/21951/head:
mds: set could_consume to false when no purge queue item actually executed

7 years agomds: set could_consume to false when no purge queue item actually executed 21951/head
Xuehan Xu [Thu, 10 May 2018 04:22:24 +0000 (12:22 +0800)]
mds: set could_consume to false when no purge queue item actually executed

Fixes: http://tracker.ceph.com/issues/24073
Signed-off-by: Xuehan Xu <xuxuehan@360.cn>
(cherry picked from commit 46b4e6afa631058fe066bfd58c76d644d5c2181d)

7 years agoMerge pull request #21916 from smithfarm/wip-24062-mimic
Nathan Cutler [Fri, 11 May 2018 09:02:08 +0000 (11:02 +0200)]
Merge pull request #21916 from smithfarm/wip-24062-mimic

mimic: rgw: policy: modify s3:ListBucketMultiPartUploads to s3:ListBucketMul

Reviewed-by: Adam C. Emerson <aemerson@redhat.com>
7 years agoceph_release: we are an rc now
Sage Weil [Thu, 10 May 2018 17:16:44 +0000 (12:16 -0500)]
ceph_release: we are an rc now

Signed-off-by: Sage Weil <sage@redhat.com>
7 years agoMerge pull request #21930 from dillaman/wip-rbd-backports-mimic
Jason Dillaman [Thu, 10 May 2018 15:27:43 +0000 (11:27 -0400)]
Merge pull request #21930 from dillaman/wip-rbd-backports-mimic

rbd: backport of mimic bug fixes

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
7 years agoRevert "stop.sh: use --no-mon-config when trying to unmap rbd devices" 21930/head
Jason Dillaman [Wed, 9 May 2018 19:01:18 +0000 (15:01 -0400)]
Revert "stop.sh: use --no-mon-config when trying to unmap rbd devices"

This reverts commit 6c722cd5f3f829ccf14280e7e36387bafd819c7a.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 3c6b01574766ca128818461d6630866cc0c0405b)

7 years agorbd: don't load config overrides from monitor initially
Jason Dillaman [Wed, 9 May 2018 16:37:22 +0000 (12:37 -0400)]
rbd: don't load config overrides from monitor initially

The overrides will be loaded when we connect to the cluster via librados
and the current approach prevents running 'rbd help' without a running
cluster.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 46eafdabde57736d2a8b63f55cab94461885a957)

7 years agorbd-mirror: ensure remote demotion is replayed locally
Jason Dillaman [Fri, 4 May 2018 15:14:34 +0000 (11:14 -0400)]
rbd-mirror: ensure remote demotion is replayed locally

The bootstrap process cannot immediately quit if it notices the remote
image is not primary. Instead, it needs to continue if the local image is
still chained to the remote.

Fixes: http://tracker.ceph.com/issues/24009
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit d35bc94319c6fa267e973e533f4591568289ee96)

7 years agorbd-mirror: fix potential race between image sync and shut down
Jason Dillaman [Fri, 4 May 2018 13:24:43 +0000 (09:24 -0400)]
rbd-mirror: fix potential race between image sync and shut down

Fixes: http://tracker.ceph.com/issues/24008
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 4f2dbcefc5e3547926b8d34135a808266bd059b8)

7 years agorbd-mirror: ensure instance replayer is stoped before destructing
Jason Dillaman [Fri, 4 May 2018 13:23:27 +0000 (09:23 -0400)]
rbd-mirror: ensure instance replayer is stoped before destructing

Fixes: http://tracker.ceph.com/issues/24008
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 9ba94f303072374b5c7c9265c3a493fddce2f8ca)

7 years agoMerge pull request #21918 from smithfarm/wip-24003-mimic
Nathan Cutler [Thu, 10 May 2018 05:15:34 +0000 (07:15 +0200)]
Merge pull request #21918 from smithfarm/wip-24003-mimic

script: build-integration-branch: avoid Unicode error

Reviewed-by: Kefu Chai <kchai@redhat.com>
7 years agoscript: build-integration-branch: avoid Unicode error 21918/head
Nathan Cutler [Thu, 3 May 2018 20:42:04 +0000 (22:42 +0200)]
script: build-integration-branch: avoid Unicode error

When run on PRs that have non-ASCII characters in their titles, the script
fails like this when run in a non UTF-8 environment:

UnicodeEncodeError: 'latin-1' codec can't encode character u'\u2026' in
position 651: ordinal not in range(256)

Since failing like that is not useful, avoid the error by forcing ASCII
encoding and filtering out any non-ASCII characters.

Fixes: http://tracker.ceph.com/issues/24003
Signed-off-by: Nathan Cutler <ncutler@suse.com>
(cherry picked from commit 6dd31fdbec43e37bece24c0de886d2c4083901c8)

7 years agorgw: policy: modify s3:ListBucketMultiPartUploads to s3:ListBucketMultipartUploads 21916/head
xiangxiang [Tue, 20 Mar 2018 12:38:07 +0000 (20:38 +0800)]
rgw: policy: modify s3:ListBucketMultiPartUploads to s3:ListBucketMultipartUploads

according to AWS S3, the permission keyword should be s3:ListBucketMultipartUploads
rather than s3:ListBucketMultiPartUploads.

all operation permission list in AWS S3 as follows:
  https://docs.aws.amazon.com/AmazonS3/latest/dev/using-with-s3-actions.html?shortFooter=true

Signed-off-by: xiangxiang <xiangxiang@xsky.com>
(cherry picked from commit 41ef62d9047ca7469023181f203e79c63ec30d64)

7 years agodoc: fix iscsi target name when configuring target
Venky Shankar [Wed, 9 May 2018 15:07:19 +0000 (20:37 +0530)]
doc: fix iscsi target name when configuring target

Signed-off-by: Venky Shankar <vshankar@redhat.com>
(cherry picked from commit 3b2d739e4c12120d27fdaf6b0af0e0de5bb972f3)

7 years agoMerge pull request #21855 from votdev/fix_rgw_backend_mimic
Tatjana Dehler [Wed, 9 May 2018 14:59:32 +0000 (16:59 +0200)]
Merge pull request #21855 from votdev/fix_rgw_backend_mimic

mimic: mgr/dashboard: Refactor RGW backend

Reviewed-by: Sebastian Wagner <swagner@suse.com>
Reviewed-by: Tatjana Dehler <tdehler@suse.com>
7 years agoMerge pull request #21835 from tchaikov/mimic-23921
Kefu Chai [Wed, 9 May 2018 14:08:52 +0000 (22:08 +0800)]
Merge pull request #21835 from tchaikov/mimic-23921

mimic: crush, osd: handle multiple parents properly when applying pg upmaps

Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>
7 years agoMerge pull request #21834 from tchaikov/mimic-23966
Kefu Chai [Wed, 9 May 2018 13:03:50 +0000 (21:03 +0800)]
Merge pull request #21834 from tchaikov/mimic-23966

mimic: osdc: guard op->on_notify_finish with lock

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
7 years agoMerge pull request #21830 from tchaikov/mimic-qa-cephfs-snap
Kefu Chai [Wed, 9 May 2018 12:12:13 +0000 (20:12 +0800)]
Merge pull request #21830 from tchaikov/mimic-qa-cephfs-snap

mimic: qa: fix test on "ceph fs set cephfs allow_new_snaps"

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
7 years agoMerge pull request #21859 from dillaman/wip-23966-mimic
Kefu Chai [Wed, 9 May 2018 12:10:00 +0000 (20:10 +0800)]
Merge pull request #21859 from dillaman/wip-23966-mimic

librados: fix potential race condition if notify immediately fails

Reviewed-by: Kefu Chai <kchai@redhat.com>
7 years agomgr/dashboard: Refactor RGW backend 21855/head
Volker Theile [Wed, 9 May 2018 08:29:11 +0000 (10:29 +0200)]
mgr/dashboard: Refactor RGW backend

- Do some polishing in the docs.
- Refactor RgwClient::is_service_online() method. The system is considered as online if the response structure is valid. The response content itself is not validated in this case.
- Relocate NoCredentialsException and derive it from RequestException.

Signed-off-by: Volker Theile <vtheile@suse.com>
(cherry picked from commit ad997b54c2728b9a72745b69fbe80c94db6eefc8)

7 years agoMerge pull request #21875 from rjfd/wip-dashboard-fix-awsauth-mimic
Volker Theile [Wed, 9 May 2018 08:16:01 +0000 (10:16 +0200)]
Merge pull request #21875 from rjfd/wip-dashboard-fix-awsauth-mimic

mimic: mgr/dashboard: awsauth: fix python3 string decode problem

Reviewed-by: Sebastian Wagner <swagner@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>
7 years agoMerge PR #21896 into mimic
Patrick Donnelly [Tue, 8 May 2018 22:44:30 +0000 (15:44 -0700)]
Merge PR #21896 into mimic

* refs/pull/21896/head:
qa: ignore version in auth metadata comp
ceph_volume_client: allow volumes without namespace isolation

7 years agoqa: ignore version in auth metadata comp 21896/head
Patrick Donnelly [Mon, 7 May 2018 16:49:00 +0000 (09:49 -0700)]
qa: ignore version in auth metadata comp

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit 3838674f655f50a0e76020d940b747e50158ba78)

7 years agoceph_volume_client: allow volumes without namespace isolation
Ramana Raja [Thu, 3 May 2018 22:09:09 +0000 (03:39 +0530)]
ceph_volume_client: allow volumes without namespace isolation

Fixes: https://tracker.ceph.com/issues/23695
Signed-off-by: Ramana Raja <rraja@redhat.com>
(cherry picked from commit 3a7fdb8b052fee2ff1bff24c8de745278314c78c)

7 years agoMerge PR #21888 into mimic
Patrick Donnelly [Tue, 8 May 2018 16:07:40 +0000 (09:07 -0700)]
Merge PR #21888 into mimic

* refs/pull/21888/head:
MDSMonitor: promote standby after fs creation
MDSMonitor: always prints standbys even if no fs

7 years agoMDSMonitor: promote standby after fs creation 21888/head
Patrick Donnelly [Fri, 4 May 2018 17:11:10 +0000 (10:11 -0700)]
MDSMonitor: promote standby after fs creation

This avoids unnecessary health warnings. However, the original issue in i23885
still exists because the standbys are not available at fs creation time. If you
create a new file system after these standbys are available, then you will
observe that the promotion works to silence the warnings.

Fixes: http://tracker.ceph.com/issues/23885
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit 93bc8c53ef51bf4fbaafad0c2d1ad79dbff8eb8b)

7 years agoMDSMonitor: always prints standbys even if no fs
Patrick Donnelly [Fri, 4 May 2018 17:50:38 +0000 (10:50 -0700)]
MDSMonitor: always prints standbys even if no fs

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit ad75128c8e1011e3f8d17405a11fad97506d74a0)

7 years agolibrados: fix potential race condition if notify immediately fails 21859/head
Jason Dillaman [Mon, 7 May 2018 17:23:43 +0000 (10:23 -0700)]
librados: fix potential race condition if notify immediately fails

Fixes: http://tracker.ceph.com/issues/23966
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
7 years agomgr/dashboard: awsauth: fix python3 string decode problem 21875/head
Ricardo Dias [Fri, 27 Apr 2018 11:38:41 +0000 (12:38 +0100)]
mgr/dashboard: awsauth: fix python3 string decode problem

Signed-off-by: Ricardo Dias <rdias@suse.com>
(cherry picked from commit 7f332a396299825eb078c8935449b816ce613d6d)

7 years agoMerge PR #21867 into mimic
Patrick Donnelly [Mon, 7 May 2018 23:00:07 +0000 (16:00 -0700)]
Merge PR #21867 into mimic

* refs/pull/21867/head:
qa/tasks/cephfs: add test for renewing stale session
client: invalidate caps and leases when session becomes stale
client: fix race in concurrent readdir

7 years agoqa/tasks/cephfs: add test for renewing stale session 21867/head
Yan, Zheng [Tue, 1 May 2018 04:26:51 +0000 (12:26 +0800)]
qa/tasks/cephfs: add test for renewing stale session

Signed-off-by: "Yan, Zheng" <zyan@redhat.com>
(cherry picked from commit 5688476513a78cf9ab2cf3b1f65e6244f05ea73d)

7 years agoclient: invalidate caps and leases when session becomes stale
Yan, Zheng [Sat, 28 Apr 2018 04:36:43 +0000 (12:36 +0800)]
client: invalidate caps and leases when session becomes stale

Signed-off-by: "Yan, Zheng" <zyan@redhat.com>
Fixes: https://tracker.ceph.com/issues/23894
(cherry picked from commit 8b2e7d834ccf2a4ff6c7aa3d4aeee07ebe36fb59)

7 years agoclient: fix race in concurrent readdir
Yan, Zheng [Fri, 27 Apr 2018 01:13:51 +0000 (09:13 +0800)]
client: fix race in concurrent readdir

For a large directory, program needs to issue multiple readdir
syscalls to get all dentries. When there are multiple programs
read the directory concurrently. Following sequence of events
can happen.

 - program calls readdir with pos = 2. ceph sends readdir request
   to mds. The reply contains N1 entries. ceph adds these N1 entries
   to readdir cache.
 - program calls readdir with pos = N1+2. The readdir is satisfied
   by the readdir cache, N2 entries are returned. (Other program
   calls readdir in the middle, which fills the cache)
 - program calls readdir with pos = N1+N2+2. ceph sends readdir
   request to mds. The reply contains N3 entries and it reaches
   directory end. ceph adds these N3 entries to the readdir cache
   and marks directory complete.

The second readdir call does not update dirp->cache_index. ceph adds
the last N3 entries to wrong places.

Signed-off-by: "Yan, Zheng" <zyan@redhat.com>
Fixes: http://tracker.ceph.com/issues/23894
(cherry picked from commit 01e23c178d068a3983c58cf115d57f6e1cc06255)

7 years agoMerge PR #21866 into mimic
Patrick Donnelly [Mon, 7 May 2018 22:47:50 +0000 (15:47 -0700)]
Merge PR #21866 into mimic

* refs/pull/21866/head:
mds: include nfiles/nsubdirs of directory inode in MClientCaps

7 years agomds: include nfiles/nsubdirs of directory inode in MClientCaps 21866/head
Yan, Zheng [Thu, 26 Apr 2018 07:12:48 +0000 (15:12 +0800)]
mds: include nfiles/nsubdirs of directory inode in MClientCaps

Directory inode's dirstat gets updated by request reply, but not by
cap message. This causes problem for following case.

1. MDS modifies a directory
2. MDS issues CEPH_CAP_ANY_SHARED to client
3. The client satifies stat(2) by its cached metadata.

Signed-off-by: "Yan, Zheng" <zyan@redhat.com>
Fixes: http://tracker.ceph.com/issues/23855
(cherry picked from commit ee2c628f6783954e9b25fab8ac9b572a58666a91)

7 years agoos/filestore: silence -Wformat-truncation= warning 21862/head
Kefu Chai [Sun, 6 May 2018 07:25:00 +0000 (15:25 +0800)]
os/filestore: silence -Wformat-truncation= warning

Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit 3f3f378d3869b30da5d2dd5ae4dab3c0ced9e312)

7 years agoos/filestore: silence error from libstdc++ in gcc-8
Kefu Chai [Sun, 6 May 2018 06:52:06 +0000 (14:52 +0800)]
os/filestore: silence error from libstdc++ in gcc-8

this change silences following error

stl_tree.h:452:21: error: static assertion failed: comparison object
must be invocable with two arguments of key type
static_assert(__is_invocable<_Compare&, const _Key&, const _Key&>{},
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit 68caa9da7ec504021b2cbd1e91367c958cb4d742)

7 years agomessages,mon,osd: silence gcc-8 warnings related to memset()
Kefu Chai [Sun, 6 May 2018 06:39:38 +0000 (14:39 +0800)]
messages,mon,osd: silence gcc-8 warnings related to memset()

this silences warnings like:

warning: ‘void* memset(void*, int, size_t)’ writing to an object of
non-trivial type ‘struct uuid_d’; use assignment instead [-Wcl\
ass-memaccess]

uuid_d only contains boost::uuids::uuid, which is "nil" initialized in
uuid_d's ctor. so we don't need to bother with memset() it with 0.
the same applies to entity_inst_t.

Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit 557cb0614586880512c9213676f8f6dcd226eb38)

7 years agocommon,filestore: silence GCC-8 warnings
Kefu Chai [Sun, 6 May 2018 06:39:18 +0000 (14:39 +0800)]
common,filestore: silence GCC-8 warnings

should catch polymorphic exceptions by reference. this silences warnings
like:

error: catching polymorphic type ‘class std::RetryException’ by value
[-Werror=catch-value=]
     } catch (std::RetryException) {

Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit 62382040ee9b655a68b15ccb43cebb39c9dc3a45)

7 years agocmake: specify missing settings for dpdk
Kefu Chai [Sun, 6 May 2018 06:32:45 +0000 (14:32 +0800)]
cmake: specify missing settings for dpdk

* after upgrading to gcc-8, seems dpdk requires more settings to compile.
  in which, CONFIG_RTE_MAX_VFIO_GROUPS=64 is copied from config/common_base.
* librte_bus_pci.a depends on librte_pci.a, so reorder the libraries
  in DPDK_LIBRARIES.

Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit 19230378c8309e8c24ee81ae255de4b3d80b24f2)

7 years agoMerge PR #21857 into mimic
Patrick Donnelly [Mon, 7 May 2018 14:40:24 +0000 (07:40 -0700)]
Merge PR #21857 into mimic

* refs/pull/21857/head:
qa: move snap-hierarchy out of snaps workunits
qa: try snap format upgrade with multimds cluster
qa: add mds deactivation procedure for upgrades

7 years agoqa: move snap-hierarchy out of snaps workunits 21857/head
Patrick Donnelly [Thu, 3 May 2018 22:07:21 +0000 (15:07 -0700)]
qa: move snap-hierarchy out of snaps workunits

The snapshot hierarchy it leaves behind can't be cleaned up by `rm -rf` which
breaks workunit cleanup. So, don't run this as part of normal snaps test.

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit 4d37b0ee8d9594977aeae02c536b5f5fcc187e33)

7 years agoqa: try snap format upgrade with multimds cluster
Patrick Donnelly [Thu, 3 May 2018 20:15:35 +0000 (13:15 -0700)]
qa: try snap format upgrade with multimds cluster

Fixes: http://tracker.ceph.com/issues/24002
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit 0b466cb2e6ccdeb56199cb889f24cc590aedff04)

7 years agoqa: add mds deactivation procedure for upgrades
Patrick Donnelly [Thu, 3 May 2018 20:12:54 +0000 (13:12 -0700)]
qa: add mds deactivation procedure for upgrades

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit 6a788bf203dc07d32f299ce488b054addaae4f75)

7 years agopybind/ceph_daemon: expand the order of magnitude of 21836/head
YunfeiGuan [Wed, 2 May 2018 08:10:28 +0000 (16:10 +0800)]
pybind/ceph_daemon: expand the order of magnitude of
daemonperf statistics to ZB

Fixes: http://tracker.ceph.com/issues/23962
Signed-off-by: Guan yunfei <yunfei.guan@xtaotech.com>
(cherry picked from commit a9d6f8a6230b99b145fceda845abff8cf7038abe)

7 years agocrush, osd: handle multiple parents properly when applying pg upmaps 21835/head
xiexingguo [Fri, 4 May 2018 13:20:29 +0000 (21:20 +0800)]
crush, osd: handle multiple parents properly when applying pg upmaps

Fixes: http://tracker.ceph.com/issues/23921
Signed-off-by: xiexingguo <xie.xingguo@gmail.com>
(cherry picked from commit 482ec97c98107be20c9ca1f34636323703ecfa5c)

7 years agoMerge pull request #21832 from tchaikov/mimic-23627
Kefu Chai [Sun, 6 May 2018 03:23:47 +0000 (11:23 +0800)]
Merge pull request #21832 from tchaikov/mimic-23627

librados: block MgrClient::start_command until mgrmap

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
7 years agoosdc: guard op->on_notify_finish with lock 21834/head
Kefu Chai [Sun, 6 May 2018 03:11:16 +0000 (11:11 +0800)]
osdc: guard op->on_notify_finish with lock

Fixes: http://tracker.ceph.com/issues/23966
Signed-off-by: Kefu Chai <kchai@redhat.com>
7 years agolibrados: timeout on mgr_command() 21832/head
Kefu Chai [Fri, 4 May 2018 08:11:53 +0000 (16:11 +0800)]
librados: timeout on mgr_command()

because the mgrclient will be waiting for the mgrmap if the mgrmap
is not available, when the client is about to send a mgr command.
and monitor will drop the subscription requests if the client does not
have enough cap for reading mon, so unlike mon commands, the client
won't get an EACCES return code in this case.

in this change, a timeout machinary is introduced. and the client
will wait for "rados-mon-op-timeout" before it gives up. if this
setting is 0, it will wait forever.

Fixes: https://tracker.ceph.com/issues/23627
Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit eaa1179965c840c0f935c570e146395583f198b6)

7 years agolibrados: fix locking on get_required_monitor_features
John Spray [Mon, 30 Apr 2018 15:35:18 +0000 (11:35 -0400)]
librados: fix locking on get_required_monitor_features

This wasn't taking the MonClient lock: should use
with_monmap to protect access to MonClient::monmap.

Signed-off-by: John Spray <john.spray@redhat.com>
(cherry picked from commit ef517d0ea0dd4754fe725ae8d6f01209b58130be)

7 years agolibrados: config mgrclient for pre-luminous cluster
John Spray [Mon, 30 Apr 2018 15:31:27 +0000 (11:31 -0400)]
librados: config mgrclient for pre-luminous cluster

Signed-off-by: John Spray <john.spray@redhat.com>
(cherry picked from commit 8fd500d736bc3c868736537d04940c627587ec63)

7 years agomgr/MgrClient: add mgr_optional mode
John Spray [Mon, 30 Apr 2018 13:32:43 +0000 (09:32 -0400)]
mgr/MgrClient: add mgr_optional mode

This is for use when talking to pre-luminous
clusters, where we should not block waiting
for MgrMap because it might never come.

Fixes: https://tracker.ceph.com/issues/23627
Signed-off-by: John Spray <john.spray@redhat.com>
(cherry picked from commit bae47183c2621577ba564d29c82a0517ab8a7f54)

7 years agoMerge pull request #21786 from dillaman/wip-23966
Kefu Chai [Sat, 5 May 2018 10:52:44 +0000 (18:52 +0800)]
Merge pull request #21786 from dillaman/wip-23966

osdc: invoke notify finish context on linger commit failure

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
7 years agoqa: fix test on "ceph fs set cephfs allow_new_snaps" 21830/head
Kefu Chai [Sat, 5 May 2018 01:38:29 +0000 (09:38 +0800)]
qa: fix test on "ceph fs set cephfs allow_new_snaps"

snapshot is now enabled for new fs in 81d0554e, so this test should be
updated accordingly.

Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit 5910c2902e50cf3c087009ed54ccf878b9ddff8c)

7 years agoosd/OSD: drop extra/wrong *unregister_pg*
xiexingguo [Fri, 4 May 2018 13:43:51 +0000 (21:43 +0800)]
osd/OSD: drop extra/wrong *unregister_pg*

Signed-off-by: xiexingguo <xie.xingguo@gmail.com>
(cherry picked from commit 6ee6fa79b5fab532dea22399afdcceeb6624019d)

7 years agoMerge pull request #21753 from liewegas/wip-upgrade-enospc
Sage Weil [Fri, 4 May 2018 20:25:10 +0000 (15:25 -0500)]
Merge pull request #21753 from liewegas/wip-upgrade-enospc

qa/suites/upgrade/luminous-x/stress-split: avoid enospc

7 years agoMerge pull request #21824 from tchaikov/mimic-cmake-jni
Alfredo Deza [Fri, 4 May 2018 17:50:45 +0000 (13:50 -0400)]
Merge pull request #21824 from tchaikov/mimic-cmake-jni

mimic: cmake: use javac -h for creating JNI native headers

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Alfredo Deza <adeza@redhat.com>
7 years agoMerge branch 'mimic' of github.com:ceph/ceph into mimic
Alfredo Deza [Fri, 4 May 2018 17:46:20 +0000 (13:46 -0400)]
Merge branch 'mimic' of github.com:ceph/ceph into mimic

7 years agocmake: bump up JDK version to 1.8 21824/head
Kefu Chai [Fri, 4 May 2018 14:55:43 +0000 (22:55 +0800)]
cmake: bump up JDK version to 1.8

as JDK 7 is EOL. see
http://www.oracle.com/technetwork/java/eol-135779.html and
https://www.java.com/en/download/faq/java_7.xml

Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit df27ed3a3806aaba41ca7bb1019e586d0b01a049)

7 years agocmake: use javac -h for creating JNI native headers
Kefu Chai [Fri, 4 May 2018 14:32:35 +0000 (22:32 +0800)]
cmake: use javac -h for creating JNI native headers

JDK 1.10 does not offer javah anymore, so we need to use "javac -h" or
add_jar(... GENERATE_NATIVE_HEADERS) instead.

Fixes: http://tracker.ceph.com/issues/24012
Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit edc504b9d446cc1739f3e8e087914cfe66d6cac3)

7 years agomgr: Include daemon details in SLOW_OPS output
Brad Hubbard [Tue, 1 May 2018 01:15:28 +0000 (11:15 +1000)]
mgr: Include daemon details in SLOW_OPS output

Currently there is no way to see which daemons were involved in a slow
op after the op has cleared. This change allows us to record which
daemons were implicated in the logs.

Partially fixes: http://tracker.ceph.com/issues/23205

Signed-off-by: Brad Hubbard <bhubbard@redhat.com>
(cherry picked from commit b5263176dea6d204c2caa52043cefc94dd5e475f)

7 years agoqa/suites/upgarde/luminous-x/stress-split: 8 osds (not 6) 21753/head
Sage Weil [Tue, 1 May 2018 12:00:10 +0000 (07:00 -0500)]
qa/suites/upgarde/luminous-x/stress-split: 8 osds (not 6)

Signed-off-by: Sage Weil <sage@redhat.com>
7 years agoosdc: invoke notify finish context on linger commit failure 21786/head
Jason Dillaman [Wed, 2 May 2018 16:52:42 +0000 (12:52 -0400)]
osdc: invoke notify finish context on linger commit failure

Fixes: http://tracker.ceph.com/issues/23966
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
7 years agoMerge pull request #21557 from jan--f/mgr-prometheus-fix-metadata-labels
John Spray [Fri, 4 May 2018 09:56:24 +0000 (10:56 +0100)]
Merge pull request #21557 from jan--f/mgr-prometheus-fix-metadata-labels

mgr: prometheus fix metadata labels

Reviewed-by: John Spray <john.spray@redhat.com>