]> git.apps.os.sepia.ceph.com Git - ceph.git/log
ceph.git
9 years agorbd-mirror: reset local image id when creating new image 9673/head
Jason Dillaman [Mon, 13 Jun 2016 06:09:00 +0000 (02:09 -0400)]
rbd-mirror: reset local image id when creating new image

Fixes: http://tracker.ceph.com/issues/16253
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 1948047acbc129de2982d53e130da1f577af224b)

9 years agorbd-mirror: handle an attempt to delete a non-mirrored image
Jason Dillaman [Mon, 13 Jun 2016 06:07:54 +0000 (02:07 -0400)]
rbd-mirror: handle an attempt to delete a non-mirrored image

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 45dda67b3ef3d5dc5abb327fc1f242140f004d90)

9 years agocls_rbd: disallow setting an empty mirror image entry to disabling
Jason Dillaman [Mon, 13 Jun 2016 06:03:29 +0000 (02:03 -0400)]
cls_rbd: disallow setting an empty mirror image entry to disabling

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 7b2f99a97ff64451f3ae72081186dff41c507271)

9 years agoMerge pull request #9656 from gregsfortytwo/wip-jewel-16024
Gregory Farnum [Mon, 13 Jun 2016 08:26:53 +0000 (01:26 -0700)]
Merge pull request #9656 from gregsfortytwo/wip-jewel-16024

Jewel client: fix bugs accidentally disabling readahead

9 years agoMerge pull request #9655 from gregsfortytwo/wip-jewel-15508
Gregory Farnum [Mon, 13 Jun 2016 08:22:22 +0000 (01:22 -0700)]
Merge pull request #9655 from gregsfortytwo/wip-jewel-15508

Jewel mds: order directories by hash and fix simultaneous readdir races

9 years agoclient: fix simultaneous readdirs race 9655/head
Yan, Zheng [Mon, 9 May 2016 14:12:48 +0000 (22:12 +0800)]
client: fix simultaneous readdirs race

Current readdir code uses list to track the order of the dentries
in readdir replies.  When handling a readdir reply, it pushes the
resulting dentries to the back of directory's dentry_list. After
readdir finishes, the dentry_list reflects how MDS sorts dentries.

This method is racy when there are simultaneous readdirs. The fix
is use vector instead of list to trace how dentries are sorted in
its parent directory. As long as shared_gen doesn't change, each
dentry is at fixed position of the vector. So cocurrent readdirs
do not affect each other.

Fixes: http://tracker.ceph.com/issues/15508
Signed-off-by: Yan, Zheng <zyan@redhat.com>
(cherry picked from commit 9d297c5e98f814b282dadc379ab70dfa678db73e)

Signed-off-by: Greg Farnum <gfarnum@redhat.com
9 years agoclient: move dir_{release,ordered}_count into class Inode
Yan, Zheng [Mon, 25 Apr 2016 15:24:46 +0000 (23:24 +0800)]
client: move dir_{release,ordered}_count into class Inode

We close Inode::dir when it's empty. Once closing the dir, we lose
track of {release,ordered}_count. This causes direcotry to be wrongly
marked as complete. (dir is trimmed to empty in the middle of readdir)

Signed-off-by: Yan, Zheng <zyan@redhat.com>
(cherry picked from commit 235fcf68f5d96a4d9d6cc260d12da912fa7ea4a8)

Signed-off-by: Greg Farnum <gfarnum@redhat.com
9 years agoceph_test_libcephfs: check order of entries in readdir result
Yan, Zheng [Mon, 25 Apr 2016 09:59:35 +0000 (17:59 +0800)]
ceph_test_libcephfs: check order of entries in readdir result

Signed-off-by: Yan, Zheng <zyan@redhat.com>
(cherry picked from commit 9b17d145c15a469caa29853312b8dcb0b19f55f0)

Signed-off-by: Greg Farnum <gfarnum@redhat.com
9 years agomds: don't reset readdir offset if client supports hash order dentry
Yan, Zheng [Mon, 25 Apr 2016 07:52:32 +0000 (15:52 +0800)]
mds: don't reset readdir offset if client supports hash order dentry

Now the ordering of dentries is stable across directory fragmentation.
There is no need to reset readdir offset if directory get fragmented
in the middle of readdir.

Signed-off-by: Yan, Zheng <zyan@redhat.com>
(cherry picked from commit 98a01af2cd8e01f14ab7be32c31a43571ef8bb87)

Signed-off-by: Greg Farnum <gfarnum@redhat.com
9 years agoclient: using hash value to compose dentry offset
Yan, Zheng [Mon, 25 Apr 2016 07:31:27 +0000 (15:31 +0800)]
client: using hash value to compose dentry offset

If MDS sorts dentries in dirfrag in hash order, we use hash value to
compose dentry offset. dentry offset is:

  (0xff << 52) | ((24 bits hash) << 28) |
  (the nth entry hash hash collision)

This offset is stable across directory fragmentation.

Signed-off-by: Yan, Zheng <zyan@redhat.com>
(cherry picked from commit 680766ec131b95271e320f54dfe6d69ea8d4fbb3)

Signed-off-by: Greg Farnum <gfarnum@redhat.com
9 years agoclient: record 'offset' for each entry of dir_result_t::buffer
Yan, Zheng [Mon, 25 Apr 2016 02:50:17 +0000 (10:50 +0800)]
client: record 'offset' for each entry of dir_result_t::buffer

This is preparation for using hash value as dentry 'offset'

Signed-off-by: Yan, Zheng <zyan@redhat.com>
(cherry picked from commit bd6546e5e8bfb4ef8927807492d63d5285534618)

Signed-off-by: Greg Farnum <gfarnum@redhat.com
9 years agoclient: fix cached readdir after seekdir
Yan, Zheng [Mon, 9 May 2016 03:51:36 +0000 (11:51 +0800)]
client: fix cached readdir after seekdir

Client::seekdir doesn't reset dirp->at_cache_name for a forward seek
within same frag. So the dentry with name == at_cache_name may not be
the one prior to the readdir postion.

Signed-off-by: Yan, Zheng <zyan@redhat.com>
(cherry picked from commit 0e32115bae2f1ac2c59b57852976b0de5587abac)

Signed-off-by: Greg Farnum <gfarnum@redhat.com
9 years agomds: define end/complete in readdir reply as single u16 flags
Yan, Zheng [Thu, 21 Apr 2016 09:31:10 +0000 (17:31 +0800)]
mds: define end/complete in readdir reply as single u16 flags

so that we can introduce new flags for readdir reply.

Signed-off-by: Yan, Zheng <zyan@redhat.com>
(cherry picked from commit 92cfbdf526d1030891da366299b635e589dd5f8e)

Signed-off-by: Greg Farnum <gfarnum@redhat.com
9 years agomds: sort dentries in CDir in hash order
Yan, Zheng [Sat, 23 Apr 2016 08:03:48 +0000 (16:03 +0800)]
mds: sort dentries in CDir in hash order

This gives us stable ordering of dentries. (Previously ordering of
dentries changes after directory gets fragmented)

Signed-off-by: Yan, Zheng <zyan@redhat.com>
(cherry picked from commit f483224a0bc70b9eb5f62cd6cbb8b97df5b65953)

Signed-off-by: Greg Farnum <gfarnum@redhat.com
9 years agoclient: save readdir result into dir_result_t directly
Yan, Zheng [Fri, 22 Apr 2016 01:43:54 +0000 (09:43 +0800)]
client: save readdir result into dir_result_t directly

Current code saves the readdir result into MedaRequest, then updates
dir_result_t according to MetaRequest. I can't see any reason why
we need to do this.

Signed-off-by: Yan, Zheng <zyan@redhat.com>
(cherry picked from commit db5d60d1b8d01afd43da09a063f00aa6ab312903)

Signed-off-by: Greg Farnum <gfarnum@redhat.com
9 years agoclient: don't allocate dir_result_t::buffer dynamically
Yan, Zheng [Fri, 22 Apr 2016 01:31:00 +0000 (09:31 +0800)]
client: don't allocate dir_result_t::buffer dynamically

Signed-off-by: Yan, Zheng <zyan@redhat.com>
(cherry picked from commit c41ceb9f1425432295a52965de8d4a23b15fe5e5)

Signed-off-by: Greg Farnum <gfarnum@redhat.com
9 years agoclient: simplify 'offset in frag'
Yan, Zheng [Thu, 21 Apr 2016 12:53:55 +0000 (20:53 +0800)]
client: simplify 'offset in frag'

don't distinguish leftmost frag from other frags. always use 2 as
first entry's offset.

Signed-off-by: Yan, Zheng <zyan@redhat.com>
(cherry picked from commit 6572c2a24ea2abec4c7eecef92a292c2cc72c762)

Signed-off-by: Greg Farnum <gfarnum@redhat.com
9 years agoclient: move readahead ref get/put to obj constr 9656/head
Patrick Donnelly [Fri, 10 Jun 2016 21:33:04 +0000 (17:33 -0400)]
client: move readahead ref get/put to obj constr

The readahead context was incrementing the file handle reference count but only
decreasing in finish which is not called if readahead is a no-op.

This fixes a bug caught in testing where readahead was causing inode to become
disconnected:

    2016-06-10 19:46:48.953018 7f2a4351be80  1 client.4125 dump_inode: DISCONNECTED inode 10000000502 #10000000502 ref 110000000502.head(faked_ino=2307 ref=1 ll_ref=0 cap_refs={1024=0,2048=0,4096=0,8192=0} open={1=0,2=0} mode=100666 size=4194304/0 mtime=2016-06-10 19:29:45.107417 caps=-(0=pAsLsXsFscr) objectset[10000000502 ts 2/4012653 objects 0 dirty_or_tx 0] 0x7f2a24300d00)
    2016-06-10 19:46:48.953032 7f2a4351be80  2 client.4125 cache still has 0+155 items, waiting (for caps to release?)

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit b5a0773259ce9f2ef44e8e6925ab11dc1a5e55e6)

Signed-off-by: Greg Farnum <gfarnum@redhat.com
9 years agoclient: use fh min readahead to enable readahead
Patrick Donnelly [Thu, 26 May 2016 20:32:58 +0000 (16:32 -0400)]
client: use fh min readahead to enable readahead

The client was using the configured maximum readahead of 0 which was changed in
95ee69989129750fddce6a3b5644238c4b88ed74. This prevented readahead from ever
running (without setting a different default from 0).

Fixes: http://tracker.ceph.com/issues/16024
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit 01c179bcce2dc8a269d78eb1be8198b12bd161f7)

Signed-off-by: Greg Farnum <gfarnum@redhat.com
9 years agoclient: update comment
Patrick Donnelly [Fri, 27 May 2016 15:43:31 +0000 (11:43 -0400)]
client: update comment

Default was changed to unlimited in 11e4680e7394e6e02612441c964e05207b97737b.

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit bee02a073f4c551798bc490618a0e8d862893c25)

Signed-off-by: Greg Farnum <gfarnum@redhat.com
9 years agoclient: use layout.get_period() helper
Patrick Donnelly [Thu, 26 May 2016 19:08:52 +0000 (15:08 -0400)]
client: use layout.get_period() helper

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit b331d12a43a4177369a003c3ab9d525da6a89950)

Signed-off-by: Greg Farnum <gfarnum@redhat.com
9 years agoMerge pull request #9553 from Abhishekvrshny/wip-16038-jewel
Jason Dillaman [Sun, 12 Jun 2016 12:16:35 +0000 (08:16 -0400)]
Merge pull request #9553 from Abhishekvrshny/wip-16038-jewel

jewel: pybind: rbd API should default features parameter to None

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
9 years agoMerge pull request #9552 from Abhishekvrshny/wip-15981-jewel
Jason Dillaman [Sun, 12 Jun 2016 12:13:16 +0000 (08:13 -0400)]
Merge pull request #9552 from Abhishekvrshny/wip-15981-jewel

jewel: qa: dynamic_features.sh races with image deletion

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
9 years agoMerge pull request #9629 from dillaman/wip-16231
Jason Dillaman [Sun, 12 Jun 2016 12:08:39 +0000 (08:08 -0400)]
Merge pull request #9629 from dillaman/wip-16231

jewel: rbd-mirror: fix deletion propagation edge cases

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
9 years agoMerge pull request #9630 from dillaman/wip-16233
Jason Dillaman [Sun, 12 Jun 2016 12:08:23 +0000 (08:08 -0400)]
Merge pull request #9630 from dillaman/wip-16233

jewel: rbd-mirror: do not propagate deletions when pool unavailable

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
9 years agoMerge pull request #9628 from dillaman/wip-16224
Jason Dillaman [Sun, 12 Jun 2016 12:07:42 +0000 (08:07 -0400)]
Merge pull request #9628 from dillaman/wip-16224

jewel: rbd-mirror: fix journal shut down ordering

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
9 years agoMerge pull request #9627 from dillaman/wip-16217
Jason Dillaman [Sun, 12 Jun 2016 12:07:10 +0000 (08:07 -0400)]
Merge pull request #9627 from dillaman/wip-16217

jewel: rbd-mirror: refresh image after creating sync point

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
9 years agoMerge pull request #9579 from Abhishekvrshny/wip-16006-jewel
Sage Weil [Sat, 11 Jun 2016 01:48:26 +0000 (21:48 -0400)]
Merge pull request #9579 from Abhishekvrshny/wip-16006-jewel

jewel: BackoffThrottle spins unnecessarily with very small backoff while the throttle is full

Reviewed-by: Sage Weil <sage@redhat.com>
9 years agoMerge pull request #9638 from ceph/wip-16211-jewel
Sage Weil [Fri, 10 Jun 2016 21:34:19 +0000 (17:34 -0400)]
Merge pull request #9638 from ceph/wip-16211-jewel

Jewel: remove reliance on FLAG_OMAP for reads

Reviewed-by: Sage Weil <sage@redhat.com>
9 years agorbd-mirror: do not propagate deletions when pool unavailable 9630/head
Jason Dillaman [Fri, 10 Jun 2016 12:15:19 +0000 (08:15 -0400)]
rbd-mirror: do not propagate deletions when pool unavailable

Fixes: http://tracker.ceph.com/issues/16229
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit d321634414fa4b90c66474557ec990dfc935ae29)

9 years agorbd-mirror: image deleter properly handles multiple snapshots 9629/head
Jason Dillaman [Fri, 10 Jun 2016 04:12:06 +0000 (00:12 -0400)]
rbd-mirror: image deleter properly handles multiple snapshots

Fixes: http://tracker.ceph.com/issues/16226
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit d72a8cb67fbdc7361f8f9e265c659ddb96dc0f74)

9 years agorbd-mirror: propagate deletions even if image replayer is stopped
Jason Dillaman [Fri, 10 Jun 2016 02:32:33 +0000 (22:32 -0400)]
rbd-mirror: propagate deletions even if image replayer is stopped

If an image deletion which causes the replayer to stop (due to an error)
before the deletion is detected, the deletion should still occur.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit a0c526a23783fe543737f6445a2d9438cc73b137)

9 years agoqa/workunits/rbd: improve deletion propagation test case
Jason Dillaman [Fri, 10 Jun 2016 01:14:27 +0000 (21:14 -0400)]
qa/workunits/rbd: improve deletion propagation test case

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit de15b198cd7b5e94016605c8016a5cf38c07c6f6)

9 years agolibrbd: quickly check for duplicate snap name upon create 9628/head
Jason Dillaman [Wed, 8 Jun 2016 14:12:04 +0000 (10:12 -0400)]
librbd: quickly check for duplicate snap name upon create

During journal replay, quickly check for a duplicate snap name
before allocating a snapshot sequence from the OSDs and
reverting due to the duplicate name.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 8374d45c7644666103f88c410a2b1b781f3a0cb8)

9 years agorbd-mirror: stop external replayer before closing remote journaler
Jason Dillaman [Tue, 7 Jun 2016 18:45:16 +0000 (14:45 -0400)]
rbd-mirror: stop external replayer before closing remote journaler

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 88bd3002082184c045bb82547929b30da1526cfb)

9 years agolibrbd: stop journal recorder before starting external replay
Jason Dillaman [Tue, 7 Jun 2016 16:42:52 +0000 (12:42 -0400)]
librbd: stop journal recorder before starting external replay

Fixes: http://tracker.ceph.com/issues/16165
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 5aa6eb2782bdb5cde6a050a5f7a4e683800414c0)

9 years agorbd-mirror: refresh image after creating sync point snapshot 9627/head
Jason Dillaman [Wed, 8 Jun 2016 16:49:04 +0000 (12:49 -0400)]
rbd-mirror: refresh image after creating sync point snapshot

Fixes: http://tracker.ceph.com/issues/16196
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit c9a3e3be7ddc31992167d55de12bbe6849447b43)

9 years agorbd-mirror: consistently use dout/derr for debug logging
Jason Dillaman [Wed, 8 Jun 2016 16:03:33 +0000 (12:03 -0400)]
rbd-mirror: consistently use dout/derr for debug logging

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 03f5e4a917b3c18f8ccdaa5ce676ed9731dd2a8f)

9 years agoMerge pull request #9612 from dillaman/wip-16208
Jason Dillaman [Fri, 10 Jun 2016 16:32:21 +0000 (12:32 -0400)]
Merge pull request #9612 from dillaman/wip-16208

jewel: rbd-mirror: support bootstrap canceling

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
9 years agoMerge pull request #9610 from dillaman/wip-16209
Jason Dillaman [Fri, 10 Jun 2016 16:32:02 +0000 (12:32 -0400)]
Merge pull request #9610 from dillaman/wip-16209

jewel: librbd: prevent error messages when journal externally disabled

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
9 years agoMerge pull request #9611 from dillaman/wip-16210
Jason Dillaman [Fri, 10 Jun 2016 16:19:47 +0000 (12:19 -0400)]
Merge pull request #9611 from dillaman/wip-16210

jewel: librbd: journal IO error results in failed assertion in AioCompletion

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
9 years agoMerge pull request #9580 from Abhishekvrshny/wip-15996-jewel
Jason Dillaman [Fri, 10 Jun 2016 16:16:49 +0000 (12:16 -0400)]
Merge pull request #9580 from Abhishekvrshny/wip-15996-jewel

jewel: librbd: reuse ImageCtx::finisher and SafeTimer for lots of images case

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
9 years agoMerge pull request #9609 from dillaman/wip-16216
Jason Dillaman [Fri, 10 Jun 2016 16:15:51 +0000 (12:15 -0400)]
Merge pull request #9609 from dillaman/wip-16216

jewel: librbd: refresh image if needed in mirror functions

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
9 years agoMerge pull request #9608 from dillaman/wip-16189
Jason Dillaman [Fri, 10 Jun 2016 16:15:32 +0000 (12:15 -0400)]
Merge pull request #9608 from dillaman/wip-16189

jewel: cls_rbd: mirror image status summary should read full directory

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
9 years agoMerge pull request #9556 from Abhishekvrshny/wip-16162-jewel
Jason Dillaman [Fri, 10 Jun 2016 16:15:08 +0000 (12:15 -0400)]
Merge pull request #9556 from Abhishekvrshny/wip-16162-jewel

jewel: ceph_test_librbd_fsx crashes during journal replay shut down

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
9 years agoMerge pull request #9554 from Abhishekvrshny/wip-16084-jewel
Jason Dillaman [Fri, 10 Jun 2016 16:14:49 +0000 (12:14 -0400)]
Merge pull request #9554 from Abhishekvrshny/wip-16084-jewel

jewel: librbd: validate image metadata configuration overrides

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
9 years agoMerge pull request #9555 from Abhishekvrshny/wip-16147-jewel
Jason Dillaman [Fri, 10 Jun 2016 16:13:06 +0000 (12:13 -0400)]
Merge pull request #9555 from Abhishekvrshny/wip-16147-jewel

jewel: Object Map is showing as invalid, even when Object Map is disabled for that Image.

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
9 years agoMerge pull request #9550 from Abhishekvrshny/wip-15969-jewel
Jason Dillaman [Fri, 10 Jun 2016 16:12:47 +0000 (12:12 -0400)]
Merge pull request #9550 from Abhishekvrshny/wip-15969-jewel

jewel: Initial python APIs to support mirroring

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
9 years agoMerge pull request #9548 from Abhishekvrshny/wip-15963-jewel
Jason Dillaman [Fri, 10 Jun 2016 16:12:37 +0000 (12:12 -0400)]
Merge pull request #9548 from Abhishekvrshny/wip-15963-jewel

jewel: Disabling journaling feature results in "Transport endpoint is not connected" error

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
9 years agoMerge pull request #9424 from SUSE/wip-16107-jewel
Loic Dachary [Thu, 9 Jun 2016 22:55:47 +0000 (00:55 +0200)]
Merge pull request #9424 from SUSE/wip-16107-jewel

jewel: ceph gid mismatch on upgrade from hammer with pre-existing ceph user (SUSE)

Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Loic Dachary <ldachary@redhat.com>
9 years agoMerge pull request #8802 from SUSE/wip-15606-jewel
Loic Dachary [Thu, 9 Jun 2016 22:54:52 +0000 (00:54 +0200)]
Merge pull request #8802 from SUSE/wip-15606-jewel

jewel: ceph-{mds,mon,osd,radosgw} systemd unit files need wants=time-sync.target

Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Loic Dachary <ldachary@redhat.com>
9 years agoMerge pull request #9320 from SUSE/wip-16017-jewel
Samuel Just [Thu, 9 Jun 2016 22:44:13 +0000 (15:44 -0700)]
Merge pull request #9320 from SUSE/wip-16017-jewel

jewel: ceph: cli: exception when pool name has non-ascii characters

Reviewed-by: Samuel Just <sjust@redhat.com>
9 years agoMerge pull request #9270 from Abhishekvrshny/wip-15852-jewel
Samuel Just [Thu, 9 Jun 2016 22:43:48 +0000 (15:43 -0700)]
Merge pull request #9270 from Abhishekvrshny/wip-15852-jewel

jewel: mon/Monitor: memory leak on Monitor::handle_ping()

Reviewed-by: Samuel Just <sjust@redhat.com>
9 years agoMerge pull request #9237 from SUSE/wip-15961-jewel
Samuel Just [Thu, 9 Jun 2016 22:43:04 +0000 (15:43 -0700)]
Merge pull request #9237 from SUSE/wip-15961-jewel

jewel: ceph-osd valgrind invalid reads/writes

Reviewed-by: Samuel Just <sjust@redhat.com>
9 years agoMerge pull request #9103 from Abhishekvrshny/wip-15797-jewel
Samuel Just [Thu, 9 Jun 2016 22:42:32 +0000 (15:42 -0700)]
Merge pull request #9103 from Abhishekvrshny/wip-15797-jewel

jewel: deadlock in OSD::_committed_osd_maps

Reviewed-by: Samuel Just <sjust@redhat.com>
9 years agoMerge pull request #9101 from Abhishekvrshny/wip-15799-jewel
Samuel Just [Thu, 9 Jun 2016 22:41:52 +0000 (15:41 -0700)]
Merge pull request #9101 from Abhishekvrshny/wip-15799-jewel

jewel: osd boot race with noup being set

Reviewed-by: Samuel Just <sjust@redhat.com>
9 years agoMerge pull request #9104 from Abhishekvrshny/wip-15795-jewel
Samuel Just [Thu, 9 Jun 2016 22:40:43 +0000 (15:40 -0700)]
Merge pull request #9104 from Abhishekvrshny/wip-15795-jewel

jewel: LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify2Timeout/1 segv

Reviewed-by: Samuel Just <sjust@redhat.com>
9 years agoMerge pull request #9427 from SUSE/wip-16104-jewel
Samuel Just [Thu, 9 Jun 2016 22:37:43 +0000 (15:37 -0700)]
Merge pull request #9427 from SUSE/wip-16104-jewel

jewel: ceph-disk: workaround gperftool hang

Reviewed-by: Loic Dachary <ldachary@redhat.com>
9 years agoMerge pull request #9614 from ceph/wip-16113-jewel
Sage Weil [Thu, 9 Jun 2016 18:55:31 +0000 (14:55 -0400)]
Merge pull request #9614 from ceph/wip-16113-jewel

osd: hobject_t::get_max() vs is_max() discrepancy

Reviewed-by: Sage Weil <sage@redhat.com>
9 years agorbd-mirror: don't return split-brain error if we still in MIRROR_PEER_STATE_SYNCING 9612/head
Mykola Golub [Wed, 4 May 2016 12:30:05 +0000 (15:30 +0300)]
rbd-mirror: don't return split-brain error if we still in MIRROR_PEER_STATE_SYNCING

Signed-off-by: Mykola Golub <mgolub@mirantis.com>
(cherry picked from commit 3ac620c5d66d8516e68264fcdb5e97b0e803d0ec)

9 years agorbd-mirror: refactor split-brain detection code
Mykola Golub [Tue, 3 May 2016 09:48:06 +0000 (12:48 +0300)]
rbd-mirror: refactor split-brain detection code

to make it more readable and extendable.

Signed-off-by: Mykola Golub <mgolub@mirantis.com>
(cherry picked from commit 039cd2d2e3bb6a9c5042c125d2c67ec9adc02c47)

9 years agorbd-mirror: support bootstrap canceling
Mykola Golub [Tue, 31 May 2016 12:00:30 +0000 (15:00 +0300)]
rbd-mirror: support bootstrap canceling

Signed-off-by: Mykola Golub <mgolub@mirantis.com>
(cherry picked from commit 21f895f48498e518c3675a9e851559f4d071f6dd)

9 years agorbd-mirror: return ECANCELED when ImageReplayer start is canceled
Mykola Golub [Mon, 30 May 2016 13:24:24 +0000 (16:24 +0300)]
rbd-mirror: return ECANCELED when ImageReplayer start is canceled

Signed-off-by: Mykola Golub <mgolub@mirantis.com>
(cherry picked from commit f581dbcb2eca108ce9440629b8f00f3229c07b95)

9 years agolibrbd: flush journal commit positions before starting op 9611/head
Jason Dillaman [Tue, 7 Jun 2016 05:01:09 +0000 (01:01 -0400)]
librbd: flush journal commit positions before starting op

Ensure all IO has been properly flushed and committed to the
journal before starting an op that could affect the IO path.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 636129653c7cac9665d0c423f6aafad5fd0c480a)

9 years agolibrbd: leaked AioCompletion if IO issued after image shut down
Jason Dillaman [Tue, 7 Jun 2016 04:11:34 +0000 (00:11 -0400)]
librbd: leaked AioCompletion if IO issued after image shut down

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit d3d4e068396e01c58832e0ea1224ca34bafb808e)

9 years agolibrbd: resize and snap create can hang on journal replay failure
Jason Dillaman [Mon, 6 Jun 2016 23:00:47 +0000 (19:00 -0400)]
librbd: resize and snap create can hang on journal replay failure

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 1032f19b3c9d9c7916669d04af909131574b3cab)

9 years agolibrbd: missing journal state for flushing/restarting replay
Jason Dillaman [Mon, 6 Jun 2016 18:54:23 +0000 (14:54 -0400)]
librbd: missing journal state for flushing/restarting replay

Fixes: http://tracker.ceph.com/issues/16077
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit a7f5bc181c9cd677d39f95ad78134e0f85c22acb)

9 years agolibrbd: track IO in the order issued
Jason Dillaman [Fri, 13 May 2016 01:22:56 +0000 (21:22 -0400)]
librbd: track IO in the order issued

Fixes: http://tracker.ceph.com/issues/15034
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit eb8dda5296eb5c19aa27e96d6e92eaa0c352cb62)

9 years agolibrbd: AioCompletion should retrieve CephContext from ImageCtx
Jason Dillaman [Thu, 12 May 2016 19:15:04 +0000 (15:15 -0400)]
librbd: AioCompletion should retrieve CephContext from ImageCtx

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 133b1ffb21c982003c7aa9dafd585062f9b2eb51)

9 years agolibrbd: only mark async completion as done after callback
Hector Martin [Wed, 25 May 2016 11:41:40 +0000 (20:41 +0900)]
librbd: only mark async completion as done after callback

This change in behavior was introduced in fde9f78.

Signed-off-by: Hector Martin <marcan@marcan.st>
(cherry picked from commit 77f7c1cf829e5417eb356ebf3bf1fbc8ff7a25a5)

9 years agolibrbd: Journal no longer requires AioCompletion for IO events
Jason Dillaman [Mon, 9 May 2016 22:17:49 +0000 (18:17 -0400)]
librbd: Journal no longer requires AioCompletion for IO events

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 82587a31f1d8ba1765f164391796753cf97a1878)

9 years agolibrbd: bubble journal error up via existing AioObjectRequests
Jason Dillaman [Mon, 9 May 2016 21:59:09 +0000 (17:59 -0400)]
librbd: bubble journal error up via existing AioObjectRequests

The journal cannot directly fail the AioCompletion because it
might have one-or-more outstanding requests associated with
it.

Fixes: http://tracker.ceph.com/issues/15791
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 79c934742fb15c803780519b36070408599a74c5)

9 years agolibrbd: potential duplicate snap removal can result in crash 9610/head
Jason Dillaman [Tue, 7 Jun 2016 02:34:30 +0000 (22:34 -0400)]
librbd: potential duplicate snap removal can result in crash

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 276fed6b7078158c2cd04fba14a11531c27898e0)

9 years agoObjectCacher: do not merge uncommitted journal bhs
Jason Dillaman [Thu, 2 Jun 2016 04:43:53 +0000 (00:43 -0400)]
ObjectCacher: do not merge uncommitted journal bhs

Avoid the possibility of an overwrite callback for regions
that were never associated with a journal write.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit c6060ade16205337de3941a10d05e234b738fa07)

9 years agoObjectCacher: don't loop the extents of OSDRead in map_read
Zhiqiang Wang [Wed, 24 Feb 2016 13:54:34 +0000 (21:54 +0800)]
ObjectCacher: don't loop the extents of OSDRead in map_read

Looping the extents of OSDRead is done in _readx. The looping in
map_read is not needed.

Signed-off-by: Zhiqiang Wang <zhiqiang@xsky.com>
(cherry picked from commit fae912c3856cab6d03183411eecab90c6e6732d2)

9 years agoObjectCacher: fix the bytes_written in writex
Zhiqiang Wang [Wed, 24 Feb 2016 13:49:56 +0000 (21:49 +0800)]
ObjectCacher: fix the bytes_written in writex

The BufferHead returned after map_write may include other dirty data to
the left and/or right. The actual length of data been written is in the
ObjectExtent.

Signed-off-by: Zhiqiang Wang <zhiqiang@xsky.com>
(cherry picked from commit ca6f1681e4baf61682e258c5761536d67bf6bdb8)

9 years agoosdc: don't loop the extents of OSDWrite in map_write
Zhiqiang Wang [Wed, 24 Feb 2016 13:45:45 +0000 (21:45 +0800)]
osdc: don't loop the extents of OSDWrite in map_write

We are looping all the extents of the OSDWrite in writex. The looping in
map_write is not needed. And if there are same object in these
extents,they will be mapped multiple times.

Signed-off-by: Zhiqiang Wang <zhiqiang@xsky.com>
(cherry picked from commit 322f1059670898762a890a01af6d3b0dd3ccf826)

9 years agolibrbd: notify image feature update before releasing lock
Jason Dillaman [Thu, 2 Jun 2016 04:42:30 +0000 (00:42 -0400)]
librbd: notify image feature update before releasing lock

If a client is waiting for the lock, ensure it sees the latest
version of the image features.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit f601ea835bb78554ac767ccb11789df960765bf9)

9 years agolibrbd: refresh image after acquiring exclusive lock
Jason Dillaman [Thu, 2 Jun 2016 03:19:20 +0000 (23:19 -0400)]
librbd: refresh image after acquiring exclusive lock

It's possible that the object map or journaling features have
been disabled while the lock was not owned.

Fixes: http://tracker.ceph.com/issues/16114
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 03c54f52d15d6283c630bac6f75427e6829f7d0a)

9 years agolibrbd: refresh image if needed in mirror functions 9609/head
Jon Bernard [Thu, 2 Jun 2016 01:01:30 +0000 (21:01 -0400)]
librbd: refresh image if needed in mirror functions

Fixes: http://tracker.ceph.com/issues/16096
Signed-off-by: Jon Bernard <jbernard@tuxion.com>
(cherry picked from commit a355c8f6d9e792517a8d0e096a99a64474369cfc)

9 years agocls_rbd: mirror image status summary should read full directory 9608/head
Jason Dillaman [Tue, 7 Jun 2016 13:36:02 +0000 (09:36 -0400)]
cls_rbd: mirror image status summary should read full directory

Previously only retrieved the status for the first 64 images in
the rbd_mirroring directory.

Fixes: http://tracker.ceph.com/issues/16178
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 61f86e7ec14233ea0dee5b3370595ce050ee28c3)

9 years agoRevert "osd/ReplicatedPG: for copy_get get omap, firstly check ob whether has omap." 9638/head
Samuel Just [Thu, 9 Jun 2016 17:36:44 +0000 (10:36 -0700)]
Revert "osd/ReplicatedPG: for copy_get get omap, firstly check ob whether has omap."

This reverts commit 771c869651132013054599f3decfc1017092a25f.

9 years agoRevert "osd/ReplicatedPG: For omap read ops, it should check object wether has omap"
Samuel Just [Thu, 9 Jun 2016 17:36:39 +0000 (10:36 -0700)]
Revert "osd/ReplicatedPG: For omap read ops, it should check object wether has omap"

This reverts commit 98caba1d0b4dd334748e3bcaf1bfb3f79f75eb65.

9 years agoRevert "osd/ReplicatedPG: When do omapclear, it should check object whether is omap."
Samuel Just [Thu, 9 Jun 2016 17:36:31 +0000 (10:36 -0700)]
Revert "osd/ReplicatedPG: When do omapclear, it should check object whether is omap."

This reverts commit aeb8141c29cdd337961430bbf3e93a79e3e1bcd3.

Conflicts:
src/osd/ReplicatedPG.cc

9 years agoMerge pull request #8938 from SUSE/wip-15739-jewel
Nathan Cutler [Wed, 8 Jun 2016 10:00:31 +0000 (12:00 +0200)]
Merge pull request #8938 from SUSE/wip-15739-jewel

jewel: ceph.spec.in: drop support for RHEL<7 and SUSE<1210 in jewel and above

Reviewed-by: Boris Ranto <branto@redhat.com>
Reviewed-by: Ken Dreyer <kdreyer@redhat.com>
9 years agoosdc/Objecter: upper bound watch_check result 9104/head
Sage Weil [Fri, 6 May 2016 13:09:43 +0000 (09:09 -0400)]
osdc/Objecter: upper bound watch_check result

This way we always return a safe upper bound on the amount of time
since we did a check.  Among other things, this prevents us from
returning a value of 0, which is confusing.

Fixes: http://tracker.ceph.com/issues/15760
Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 2e2ce365866a7569b55aff338a82c132aa927089)

9 years agoOSD: fix deadlock in OSD::_committed_osd_maps 9103/head
Xinze Chi [Wed, 4 May 2016 01:49:40 +0000 (09:49 +0800)]
OSD: fix deadlock in OSD::_committed_osd_maps

Fixes: http://tracker.ceph.com/issues/15701
Signed-off-by: Xinze Chi <xinze@xsky.com>
(cherry picked from commit 4fbf3891aab16347d02782f7c1c0400f9e0896af)

9 years agoosd: handle boot racing with NOUP set 9101/head
Sage Weil [Tue, 3 May 2016 03:28:18 +0000 (23:28 -0400)]
osd: handle boot racing with NOUP set

This is a follow-on to 7139a232d26beef441ffbc13bc087baab3505ea8,
which handled the NOUP set + clear case when the OSD found out
about the flag being cleared.  However, it's possible that the
flag will get cleared but the OSD won't get a map update (because
it hasn't subscribed and is not doing any work).

This means that it is *more* likely than before that we will
restart the boot process even though the OSD did successfully
mark us up.  However, as before, it is unavoidable because there
is no notification of whether our boot request succeeds or not.

And it is still mostly harmless (an extra mark down + up cycle).

Fixes: http://tracker.ceph.com/issues/15678
Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 11e4242fbdb2f2f6f654d4cb3a7c95d5b38a88c2)

9 years agoTaskFinisher: use singleton SafeTimer and Finisher 9580/head
Haomai Wang [Sun, 28 Feb 2016 15:35:17 +0000 (23:35 +0800)]
TaskFinisher: use singleton SafeTimer and Finisher

Fix #13938
Signed-off-by: Haomai Wang <haomai@xsky.com>
(cherry picked from commit dc6d6ce8b29de51b69242e569517f69acfe226f4)

9 years agoBackoffThrottle: wait() if past max 9579/head
Samuel Just [Wed, 18 May 2016 19:09:10 +0000 (12:09 -0700)]
BackoffThrottle: wait() if past max

Otherwise, we risk spinning on wait_for() with a small
delay.

Fixes: http://tracker.ceph.com/issues/15953
Signed-off-by: Samuel Just <sjust@redhat.com>
(cherry picked from commit 6f835726a7df5b29a96849fc1362186fd2116d9b)

9 years agoBackoffThrottle: use wait_for instead of wait_until
Samuel Just [Wed, 18 May 2016 18:49:19 +0000 (11:49 -0700)]
BackoffThrottle: use wait_for instead of wait_until

On some platforms, wait_until won't surrender the lock
with a negative argument.

Signed-off-by: Samuel Just <sjust@redhat.com>
(cherry picked from commit 846808ff3f3837b1772f293ae71dba5eda609cc4)

9 years agotest/rbd: fsx needs to shut down the journal 9556/head
Jason Dillaman [Thu, 2 Jun 2016 14:41:18 +0000 (10:41 -0400)]
test/rbd: fsx needs to shut down the journal

Fixes: http://tracker.ceph.com/issues/16123
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit d10ffaafbd2a5831154757b848ed8504dad99069)

9 years agoremove invalid objectmap flag when objectmap is disabled 9555/head
xinxin shu [Thu, 2 Jun 2016 06:13:09 +0000 (14:13 +0800)]
remove invalid objectmap flag when objectmap is disabled
Fixes: http://tracker.ceph.com/issues/16076
Signed-off-by: xinxin shu <shuxinxin@chinac.com>
(cherry picked from commit b2d475686ee7617bb2023d753941e3d6952f0878)

9 years agorbd: check value of config override when setting image-meta 9554/head
zhuangzeqiang [Sat, 7 Oct 2017 08:52:07 +0000 (16:52 +0800)]
rbd: check value of config override when setting image-meta

Fixes: http://tracker.ceph.com/issues/15522
Signed-off-by: zhuangzeqiang <zhuang.zeqiang@h3c.com>
(cherry picked from commit 4538f8152d0e73bdefc09874113c87467bdda622)

9 years agopybind/rbd: create/clone/copy: default to None for features param 9553/head
Mykola Golub [Mon, 23 May 2016 06:18:34 +0000 (09:18 +0300)]
pybind/rbd: create/clone/copy: default to None for features param

If None is specified don't set the features so that the defaults is
used.

Fixes: http://tracker.ceph.com/issues/15982
Signed-off-by: Mykola Golub <mgolub@mirantis.com>
(cherry picked from commit b8b3a9757f7d517210a91f2893b0961b6b35afab)

9 years agolibrbd: clone: default to parent features
Mykola Golub [Mon, 23 May 2016 06:16:38 +0000 (09:16 +0300)]
librbd: clone: default to parent features

Fixes: http://tracker.ceph.com/issues/15982
Signed-off-by: Mykola Golub <mgolub@mirantis.com>
(cherry picked from commit d305eaad8d57a6f64e6d80eebcd0839a65c9a668)

9 years agoqa: dynamic_features.sh: return error only if it failed on alive QEMU 9552/head
Mykola Golub [Thu, 12 May 2016 06:15:53 +0000 (09:15 +0300)]
qa: dynamic_features.sh: return error only if it failed on alive QEMU

Fixes: #15500
Signed-off-by: Mykola Golub <mgolub@mirantis.com>
(cherry picked from commit 8a71a79d24f2e1f999a5fc0a4154a10401d5a261)

9 years agotest: initial python APIs to support mirroring 9550/head
Mykola Golub [Wed, 11 May 2016 11:33:00 +0000 (14:33 +0300)]
test: initial python APIs to support mirroring

Signed-off-by: Mykola Golub <mgolub@mirantis.com>
(cherry picked from commit bfad0ca33b9d65dcec65bc1a18596d35fb3098f3)

9 years agorbd: initial python APIs to support mirroring
Mykola Golub [Wed, 11 May 2016 11:29:17 +0000 (14:29 +0300)]
rbd: initial python APIs to support mirroring

Fixes: http://tracker.ceph.com/issues/15656
Signed-off-by: Mykola Golub <mgolub@mirantis.com>
(cherry picked from commit ef0ea8ee3c926a31b54e410c18e887415f6ea3cf)

9 years agorbd: close journal before removing 9548/head
Yuan Zhou [Mon, 16 May 2016 08:18:59 +0000 (16:18 +0800)]
rbd: close journal before removing

Otherwise there is some unwanted error message.

Fixes: http://tracker.ceph.com/issues/15863
Signed-off-by: Yuan Zhou <yuan.zhou@intel.com>
(cherry picked from commit ecdad4acca635d4b9f0b407889dde9985ab3a506)