]> git.apps.os.sepia.ceph.com Git - ceph.git/log
ceph.git
8 years agoReplicatedBackend: don't queue Context outside of ObjectStore with obc 16132/head
Samuel Just [Tue, 14 Feb 2017 20:47:37 +0000 (12:47 -0800)]
ReplicatedBackend: don't queue Context outside of ObjectStore with obc

We only flush the ObjectStore callbacks, not everything else.  Thus,
there isn't a guarrantee that the obc held by pull_complete_info will
be cleaned up before the Flush callback is triggered.  Instead, just
defer clearing the pull state until the callback (it'll be cleaned up
during the interval change) and remove the ObjectContext from
pull_complete_info.

Introduced: 68defc2b0561414711d4dd0a76bc5d0f46f8a3f8
Fixes: http://tracker.ceph.com/issues/18927
Signed-off-by: Samuel Just <sjust@redhat.com>
(cherry picked from commit 51eee55c475a3d931844831e040ed3d66ee59af4)

8 years agoosd/: don't leak context for Blessed*Context or RecoveryQueueAsync
Samuel Just [Fri, 3 Feb 2017 21:12:47 +0000 (13:12 -0800)]
osd/: don't leak context for Blessed*Context or RecoveryQueueAsync

This has always been a bug, but until
68defc2b0561414711d4dd0a76bc5d0f46f8a3f8, nothing deleted those contexts
without calling complete().

Fixes: http://tracker.ceph.com/issues/18809
Bug shadowed until: 68defc2b0561414711d4dd0a76bc5d0f46f8a3f8
Signed-off-by: Samuel Just <sjust@redhat.com>
(cherry picked from commit 91b74235027c8a4872dcab6b37767b12c3267061)

8 years agoMerge pull request #15792 from liewegas/wip-bluestore-leaks-kraken
Nathan Cutler [Wed, 5 Jul 2017 20:04:41 +0000 (22:04 +0200)]
Merge pull request #15792 from liewegas/wip-bluestore-leaks-kraken

kraken: os/bluestore: deep decode onode value

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
8 years agoMerge pull request #15729 from asheplyakov/20315-bp-kraken
Nathan Cutler [Wed, 5 Jul 2017 20:01:58 +0000 (22:01 +0200)]
Merge pull request #15729 from asheplyakov/20315-bp-kraken

kraken: mon: fail to form large quorum; msg/async busy loop

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
8 years agoMerge pull request #15084 from asheplyakov/19928-bp-kraken
Nathan Cutler [Wed, 5 Jul 2017 20:00:17 +0000 (22:00 +0200)]
Merge pull request #15084 from asheplyakov/19928-bp-kraken

kraken: mon crash on shutdown, lease_ack_timeout event

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
8 years agoMerge pull request #14735 from smithfarm/wip-19685-kraken
Nathan Cutler [Wed, 5 Jul 2017 19:59:03 +0000 (21:59 +0200)]
Merge pull request #14735 from smithfarm/wip-19685-kraken

kraken: osd: Give requested scrubs a higher priority

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
8 years agoMerge pull request #14733 from smithfarm/wip-19561-kraken
Nathan Cutler [Wed, 5 Jul 2017 19:57:53 +0000 (21:57 +0200)]
Merge pull request #14733 from smithfarm/wip-19561-kraken

kraken: api_misc: [  FAILED  ] LibRadosMiscConnectFailure.ConnectFailure

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
8 years agoMerge pull request #13883 from shinobu-x/wip-19119-kraken
Nathan Cutler [Wed, 5 Jul 2017 19:51:26 +0000 (21:51 +0200)]
Merge pull request #13883 from shinobu-x/wip-19119-kraken

kraken: pre-jewel "osd rm" incrementals are misinterpreted

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
8 years agoMerge pull request #14646 from gregsfortytwo/wip-17916-kraken
Nathan Cutler [Wed, 5 Jul 2017 17:39:54 +0000 (19:39 +0200)]
Merge pull request #14646 from gregsfortytwo/wip-17916-kraken

kraken: osd: pglog: with config, don't assert in the presence of stale diverg…

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
8 years agoMerge pull request #15237 from asheplyakov/20035-bp-kraken
Nathan Cutler [Wed, 5 Jul 2017 16:41:59 +0000 (18:41 +0200)]
Merge pull request #15237 from asheplyakov/20035-bp-kraken

kraken: mon/PGMap: factor mon_osd_full_ratio into MAX AVAIL calc

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
8 years agoMerge pull request #14331 from shinobu-x/wip-19396-kraken
Nathan Cutler [Wed, 5 Jul 2017 16:19:05 +0000 (18:19 +0200)]
Merge pull request #14331 from shinobu-x/wip-19396-kraken

kraken: Objecter::epoch_barrier isn't respected in _op_submit()

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
8 years agoMerge pull request #16083 from smithfarm/wip-18677-kraken
Nathan Cutler [Wed, 5 Jul 2017 15:54:30 +0000 (17:54 +0200)]
Merge pull request #16083 from smithfarm/wip-18677-kraken

kraken: osd: metadata reports filestore when using bluestore

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
8 years agoMerge pull request #16098 from smithfarm/wip-19621-kraken
Nathan Cutler [Wed, 5 Jul 2017 15:51:36 +0000 (17:51 +0200)]
Merge pull request #16098 from smithfarm/wip-19621-kraken

kraken: rbd-nbd: add signal handler

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
8 years agoMerge pull request #15612 from smithfarm/wip-19611-kraken
Nathan Cutler [Wed, 5 Jul 2017 15:50:21 +0000 (17:50 +0200)]
Merge pull request #15612 from smithfarm/wip-19611-kraken

kraken: rbd: Issues with C API image metadata retrieval functions

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
Reviewed-by: Mykola Golub <mgolub@mirantis.com>
8 years agoMerge pull request #14833 from smithfarm/wip-19794-kraken
Nathan Cutler [Wed, 5 Jul 2017 15:48:24 +0000 (17:48 +0200)]
Merge pull request #14833 from smithfarm/wip-19794-kraken

kraken: tests: test_notify.py: assert(not image.is_exclusive_lock_owner()) on line 147

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
8 years agoMerge pull request #14543 from smithfarm/wip-19173-kraken
Nathan Cutler [Wed, 5 Jul 2017 15:26:58 +0000 (17:26 +0200)]
Merge pull request #14543 from smithfarm/wip-19173-kraken

kraken: rbd: rbd_clone_copy_on_read ineffective with exclusive-lock

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
8 years agoMerge pull request #14974 from dillaman/wip-19872-kraken
Nathan Cutler [Wed, 5 Jul 2017 07:37:01 +0000 (09:37 +0200)]
Merge pull request #14974 from dillaman/wip-19872-kraken

kraken: rbd-mirror: failover and failback of unmodified image results in split-brain

Reviewed-by: Mykola Golub <mgolub@mirantis.com>
8 years agoMerge pull request #14934 from dillaman/wip-19833-kraken
Nathan Cutler [Wed, 5 Jul 2017 07:36:14 +0000 (09:36 +0200)]
Merge pull request #14934 from dillaman/wip-19833-kraken

kraken: cls_rbd: default initialize snapshot namespace for legacy clients

Reviewed-by: Mykola Golub <mgolub@mirantis.com>
8 years agoMerge pull request #14539 from smithfarm/wip-18771-kraken
Nathan Cutler [Wed, 5 Jul 2017 07:30:07 +0000 (09:30 +0200)]
Merge pull request #14539 from smithfarm/wip-18771-kraken

kraken: rbd: Improve compatibility between librbd + krbd for the data pool

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
8 years agoMerge pull request #14065 from asheplyakov/19212-bp-kraken
Nathan Cutler [Tue, 4 Jul 2017 11:32:46 +0000 (13:32 +0200)]
Merge pull request #14065 from asheplyakov/19212-bp-kraken

kraken: rgw: "cluster [WRN] bad locator @X on object @X...." in cluster log

Reviewed-by: Orit Wasserman <owasserm@redhat.com>
8 years agoMerge pull request #16092 from smithfarm/wip-20150-kraken
Nathan Cutler [Tue, 4 Jul 2017 11:18:48 +0000 (13:18 +0200)]
Merge pull request #16092 from smithfarm/wip-20150-kraken

kraken: ceph-disk: fails if OSD udev rule triggers prior to mount of /var

Reviewed-by: Loic Dachary <ldachary@redhat.com>
8 years agoMerge pull request #14604 from asheplyakov/19647-kraken
Nathan Cutler [Tue, 4 Jul 2017 09:37:29 +0000 (11:37 +0200)]
Merge pull request #14604 from asheplyakov/19647-kraken

kraken: ceph-disk: enable directory backed OSD at boot time

Reviewed-by: Loic Dachary <ldachary@redhat.com>
8 years agorbd-nbd: remove debug message from do_unmap 16098/head
Pan Liu [Thu, 30 Mar 2017 16:23:12 +0000 (00:23 +0800)]
rbd-nbd: remove debug message from do_unmap

Global context is not initialized when do_unmap is called.

Signed-off-by: Pan Liu <liupan1111@gmail.com>
(cherry picked from commit cd748f9b260a3c388b45091ff0bfc041257acd0c)

8 years agorbd-nbd: s/cpp_error/cpp_strerror/ to fix FTBFS
Kefu Chai [Wed, 29 Mar 2017 11:07:34 +0000 (19:07 +0800)]
rbd-nbd: s/cpp_error/cpp_strerror/ to fix FTBFS

the build failure was introduced by ff4dcf0

Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit c12ecd984df0f9ac0d7de81b9d689f6182a3646a)

8 years agorbd-nbd: polish the output info before and after ioctl NBD_DISCONNECT.
Pan Liu [Tue, 28 Mar 2017 08:48:21 +0000 (16:48 +0800)]
rbd-nbd: polish the output info before and after ioctl NBD_DISCONNECT.

Signed-off-by: Pan Liu <liupan1111@gmail.com>
(cherry picked from commit ff4dcf029028e8a3636ac71a6c5ac5380bf274e4)

8 years agorbd-nbd: support signal handle for SIGHUP, SIGINT, and SIGTERM.
Pan Liu [Tue, 28 Mar 2017 08:33:25 +0000 (16:33 +0800)]
rbd-nbd: support signal handle for SIGHUP, SIGINT, and SIGTERM.

Fixes: http://tracker.ceph.com/issues/19349
Signed-off-by: Pan Liu <liupan1111@gmail.com>
(cherry picked from commit 3ba01aa6ce052d1afa42132feffc2353d73caae6)

8 years agoMerge pull request #14616 from smithfarm/wip-18723-kraken
Nathan Cutler [Tue, 4 Jul 2017 08:26:43 +0000 (10:26 +0200)]
Merge pull request #14616 from smithfarm/wip-18723-kraken

kraken: osd: calc_clone_subsets misuses try_read_lock vs missing

Reviewed-by: Kefu Chai <kchai@redhat.com>
8 years agoceph-disk: do not setup_statedir on trigger 16092/head
Loic Dachary [Thu, 1 Jun 2017 09:37:20 +0000 (11:37 +0200)]
ceph-disk: do not setup_statedir on trigger

trigger may run when statedir is unavailable and does not use it.

Fixes: http://tracker.ceph.com/issues/19941
Signed-off-by: Loic Dachary <loic@dachary.org>
(cherry picked from commit 16bfbdd3d9988523bba31aace516c303057daa58)

8 years agoMerge pull request #14852 from yehudasa/wip-rgw-support-ragweed-kraken
Nathan Cutler [Mon, 3 Jul 2017 22:02:05 +0000 (00:02 +0200)]
Merge pull request #14852 from yehudasa/wip-rgw-support-ragweed-kraken

kraken: rgw: add apis to support ragweed suite

Reviewed-by: Nathan Cutler <ncutler@suse.com>
8 years agoMerge pull request #13514 from rzarzynski/wip-rgw-controllable-204-kraken
Nathan Cutler [Mon, 3 Jul 2017 20:28:08 +0000 (22:28 +0200)]
Merge pull request #13514 from rzarzynski/wip-rgw-controllable-204-kraken

kraken: rgw: make sending Content-Length in 204 and 304 controllable

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>
8 years agoMerge pull request #14403 from shinobu-x/wip-17331-kraken
Nathan Cutler [Mon, 3 Jul 2017 15:05:34 +0000 (17:05 +0200)]
Merge pull request #14403 from shinobu-x/wip-17331-kraken

kraken: ceph-disk list reports mount error for OSD having mount options with SELinux context

Reviewed-by: Loic Dachary <ldachary@redhat.com>
8 years agoosd: Return correct osd_objectstore in OSD metadata 16083/head
Wido den Hollander [Mon, 23 Jan 2017 20:06:05 +0000 (21:06 +0100)]
osd: Return correct osd_objectstore in OSD metadata

Do not simply read the configuration value as it might have changed
during OSD startup by reading the type from disk.

Fixes: http://tracker.ceph.com/issues/18638
Signed-off-by: Wido den Hollander <wido@42on.com>
(cherry picked from commit 8fe6a0303b02ac1033f5bfced9f94350fe3e33de)

Conflicts:
    src/osd/OSD.cc
      - g_conf->osd_objectstore was changed to cct->_conf->osd_objectstore by
        1d5e967a05ddbcceb10efe3b57e242b3b6b7eb8c which is not in kraken

8 years agolibrbd: introduce new constants for tracking max block name prefix 14539/head
Jason Dillaman [Thu, 26 Jan 2017 19:02:11 +0000 (14:02 -0500)]
librbd: introduce new constants for tracking max block name prefix

Fixes: http://tracker.ceph.com/issues/18653
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 2c08629c99d90aa7676b59263c055c9f1f577039)

8 years agolibrbd: acquire exclusive-lock during copy on read 14543/head
Venky Shankar [Mon, 20 Feb 2017 06:34:10 +0000 (12:04 +0530)]
librbd: acquire exclusive-lock during copy on read

Fixes: http://tracker.ceph.com/issues/18888
Signed-off-by: Venky Shankar <vshankar@redhat.com>
(cherry picked from commit 7dba5311b12011a4a6e8564e68150e54c5af5ddd)

Conflicts:
    src/librbd/AioImageRequestWQ.h:
      - in master this file has morphed into src/librbd/io/ImageRequestWQ.h
      - kraken has AioImageRequest<ImageCtx> instead of ImageRequest<ImageCtx>
    src/librbd/image/RefreshRequest.cc:
      - rename image context element to "aio_work_queue" (from "io_work_queue")
        because kraken doesn't have de95d862f57b56738e04d77f2351622f83f17f4a
    src/test/librbd/image/test_mock_RefreshRequest.cc:
      - rename image context element to "aio_work_queue" (from "io_work_queue")
        because kraken doesn't have de95d862f57b56738e04d77f2351622f83f17f4a

8 years agoPrimaryLogPG::try_lock_for_read: give up if missing 14616/head
Samuel Just [Wed, 18 Jan 2017 18:24:13 +0000 (10:24 -0800)]
PrimaryLogPG::try_lock_for_read: give up if missing

The only users calc_*_subsets might try to read_lock an object which is
missing on the primary.  Returning false in those cases is perfectly
reasonable and avoids the problem.

Fixes: http://tracker.ceph.com/issues/18583
Signed-off-by: Samuel Just <sjust@redhat.com>
(cherry picked from commit 3833440adea6f8bcb0093603c3a9d16360ed57ec)

8 years agoReplicatedBackend: take read locks for clone sources during recovery
Samuel Just [Wed, 23 Nov 2016 23:41:13 +0000 (15:41 -0800)]
ReplicatedBackend: take read locks for clone sources during recovery

Otherwise, we run the risk of a clone source which hasn't actually
come into existence yet being used if we grab a clone which *just*
got added the the ssc, but has not yet actually had time to be
created (can't rely on message ordering here since recovery messages
don't necessarily order with client IO!).

Fixes: http://tracker.ceph.com/issues/17831
Signed-off-by: Samuel Just <sjust@redhat.com>
(cherry picked from commit 68defc2b0561414711d4dd0a76bc5d0f46f8a3f8)

8 years agoceph_test_librados_api_misc: fix stupid LibRadosMiscConnectFailure.ConnectFailure... 14733/head
Sage Weil [Fri, 31 Mar 2017 14:06:42 +0000 (10:06 -0400)]
ceph_test_librados_api_misc: fix stupid LibRadosMiscConnectFailure.ConnectFailure test

Sometimes the cond doesn't time out and it wakes up instead.  Just repeat
the test many times to ensure that at least once it times out (usually
it doesn't; it's pretty infrequent that it doesn't).

Fixes: http://tracker.ceph.com/issues/15368
Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 8bc197400d94ee2716d3f2fa454247379a676cf9)

8 years agoosd: Give requested scrub work a higher priority 14735/head
David Zafman [Mon, 17 Apr 2017 21:58:02 +0000 (14:58 -0700)]
osd: Give requested scrub work a higher priority

Once started we now queue scrub work at higher priority than
scheduled scrubs.

Fixes: http://tracker.ceph.com/issues/15789
Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit ebab8b1f4f67fbdec1e147c580329c1e2b5cf7cd)

Conflicts:
src/osd/OSD.h - in kraken, the PGScrub() call is enclosed within
                   op_wq.queue(make_pair(...)) instead of enqueue_back()

8 years agoMerge pull request #16069 from smithfarm/wip-20345-kraken
Nathan Cutler [Mon, 3 Jul 2017 09:39:42 +0000 (11:39 +0200)]
Merge pull request #16069 from smithfarm/wip-20345-kraken

kraken: make check fails with Error EIO: load dlopen(build/lib/libec_FAKE.so): build/lib/libec_FAKE.so: cannot open shared object file: No such file or directory

Reviewed-by: Kefu Chai <kchai@redhat.com>
8 years agoceph.spec.in, debian/control: Add bc to build dependencies 16069/head
Kyr Shatskyy [Wed, 8 Feb 2017 15:02:52 +0000 (16:02 +0100)]
ceph.spec.in, debian/control: Add bc to build dependencies

The bc is missing for ceph-helpers.sh

Fixes: http://tracker.ceph.com/issues/18876
Signed-off-by: Kyr Shatskyy <kyrylo.shatskyy@gmail.com>
(cherry picked from commit 3ff4be6d6896346b7b9ec7f158fcde8866faeb38)

8 years agoqa/workunits/ceph-helpers: do not error out if is_clean
Kefu Chai [Wed, 12 Apr 2017 09:31:52 +0000 (17:31 +0800)]
qa/workunits/ceph-helpers: do not error out if is_clean

it would be a race otherwise, because we cannot be sure that the cluster
pgs are not all clean or not when run_osd() returns, but we can be sure
that they are expected to active+clean after a while. that's what
wait_for_clean() does.

Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit 6cb4503a40ae4ebee5690fda024cae8d1a506bce)

8 years agoqa/workunits/ceph-helpers: display rejected string
Kefu Chai [Wed, 12 Apr 2017 04:33:53 +0000 (12:33 +0800)]
qa/workunits/ceph-helpers: display rejected string

Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit 0196e154ed2e164cf55b0d7ed9f9cdd1f4f50100)

8 years agorgw: new rest api to retrieve object layout 14852/head
Yehuda Sadeh [Mon, 9 Jan 2017 21:04:43 +0000 (13:04 -0800)]
rgw: new rest api to retrieve object layout

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
(cherry picked from commit 2768583dc486109e49d209243675b99fdd39e92c)

8 years agorgw: rest api to read zone config params
Yehuda Sadeh [Thu, 5 Jan 2017 21:47:24 +0000 (13:47 -0800)]
rgw: rest api to read zone config params

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
(cherry picked from commit a220a2efbfd675e6abf14ae33c21005bcbf6dadf)

8 years agolibrbd: add no-op event when promoting an image 14974/head
Jason Dillaman [Thu, 4 May 2017 21:56:22 +0000 (17:56 -0400)]
librbd: add no-op event when promoting an image

The rbd-mirror process needs an event in the journal
to properly detect the transition between primary and
non-primary state between peers.

Fixes: http://tracker.ceph.com/issues/19858
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 4031555dda7597d24e9eb04b9ff29173909586f7)

Conflicts:
src/librbd/journal/DemoteRequest.cc: logic exists in Journal.cc

8 years agorbd-mirror: prevent infinite loop when computing replay status
Jason Dillaman [Thu, 4 May 2017 21:57:34 +0000 (17:57 -0400)]
rbd-mirror: prevent infinite loop when computing replay status

If the image had a non-primary predecessor epoch whose tag tid
duplicates an epoch within its own journal, an infinite loop
would result.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 3f179bf86216540d8e25aad469c604f96f6aecd8)

8 years agoMerge pull request #15869 from smithfarm/wip-swift-task-move-kraken
Zack Cerza [Mon, 26 Jun 2017 20:11:56 +0000 (14:11 -0600)]
Merge pull request #15869 from smithfarm/wip-swift-task-move-kraken

tests: move swift.py task from teuthology to ceph, phase one (kraken)

8 years agotests: swift.py: tweak imports 15869/head
Nathan Cutler [Sun, 25 Jun 2017 10:42:36 +0000 (12:42 +0200)]
tests: swift.py: tweak imports

The ".." form only works within the teuthology repo. With swift.py now in the
Ceph repo, we have to be explicit.

Error message was: "ValueError: Attempted relative import beyond toplevel
package

Signed-off-by: Nathan Cutler <ncutler@suse.com>
8 years agoMerge branch 'master' of /home/smithfarm/src/ceph/upstream/teuthology into wip-swift...
Nathan Cutler [Fri, 23 Jun 2017 06:34:53 +0000 (08:34 +0200)]
Merge branch 'master' of /home/smithfarm/src/ceph/upstream/teuthology into wip-swift-task-move-kraken

8 years agotests: move swift.py task to qa/tasks
Nathan Cutler [Fri, 23 Jun 2017 06:27:42 +0000 (08:27 +0200)]
tests: move swift.py task to qa/tasks

In preparation for moving this task from ceph/teuthology.git into ceph/ceph.git

The move is necessary because jewel-specific changes are needed, yet teuthology
does not maintain a separate branch for jewel. Also, swift.py is a
Ceph-specific task so it makes more sense to have it in Ceph.

Signed-off-by: Nathan Cutler <ncutler@suse.com>
8 years agoos/bluestore: deep decode onode value 15792/head
Sage Weil [Tue, 30 May 2017 01:55:33 +0000 (21:55 -0400)]
os/bluestore: deep decode onode value

In particular, we want the attrs (map<string,bufferptr>) to be a deep
decode so that we do not pin this buffer, and so that any changed attr
will free the previous memory.

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit f53f56350b893bfdd47fe730a4339ca5793791a0)

8 years agoMerge pull request #13181 from smithfarm/wip-18548-kraken
Nathan Cutler [Tue, 20 Jun 2017 20:59:27 +0000 (22:59 +0200)]
Merge pull request #13181 from smithfarm/wip-18548-kraken

kraken: rgw: multisite: segfault after changing value of rgw_data_log_num_shards

Reviewed-by: Orit Wasserman <owasserm@redhat.com>
8 years agoMerge pull request #13838 from smithfarm/wip-19049-kraken
Nathan Cutler [Tue, 20 Jun 2017 20:56:59 +0000 (22:56 +0200)]
Merge pull request #13838 from smithfarm/wip-19049-kraken

kraken: rgw: multisite: some yields in RGWMetaSyncShardCR::full_sync() resume in incremental_sync()

Reviewed-by: Orit Wasserman <owasserm@redhat.com>
Reviewed-by: Casey Bodley <cbodley@redhat.com>
8 years agoMerge pull request #13224 from smithfarm/wip-18780-kraken
Nathan Cutler [Tue, 20 Jun 2017 20:52:13 +0000 (22:52 +0200)]
Merge pull request #13224 from smithfarm/wip-18780-kraken

kraken: rgw: Swift API: spurious newline after http body causes weird errors

Reviewed-by: Orit Wasserman <owasserm@redhat.com>
8 years agomon: check is_shutdown() in timer callbacks 15084/head
Kefu Chai [Fri, 5 May 2017 04:02:05 +0000 (12:02 +0800)]
mon: check is_shutdown() in timer callbacks

introduce a helper class: C_MonContext, and initialize all timer events
using it, to ensure that they do check is_shutdown() before doing their
work.

Fixes: http://tracker.ceph.com/issues/19825
Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit 561cbded0c7e28231b1c7ce18663b8d7d40aad6d)

8 years agomon/Elector: call cancel_timer() in shutdown()
Kefu Chai [Thu, 4 May 2017 14:49:04 +0000 (22:49 +0800)]
mon/Elector: call cancel_timer() in shutdown()

instead of doing it manually.

Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit 12139ae529a49b6caedea89f910d034ddca094b6)

8 years agokraken: mon: add override annotation to callback classes
Alexey Sheplyakov [Wed, 17 May 2017 13:50:10 +0000 (17:50 +0400)]
kraken: mon: add override annotation to callback classes

The only purpose of this patch is to avoid merge conflicts while
cherry-picking commit 561cbded0c7e28231b1c7ce18663b8d7d40aad6d.
Alternatively one could cherry-pick 1effdfe19bf9fd6d546620b96eaf452e889b15dc,
but that one brings a lot of unrelated changes.

Signed-off-by: Alexey Sheplyakov <asheplyakov@mirantis.com>
8 years agotest/librbd/test_notify.py: don't disable feature in slave 14833/head
Mykola Golub [Mon, 24 Apr 2017 14:23:21 +0000 (16:23 +0200)]
test/librbd/test_notify.py: don't disable feature in slave

On jewel it will have stolen the exclusive lock. Instead, ensure that
object map and fast diff are already disabled on the clone before the
start of the test.

Fixes: http://tracker.ceph.com/issues/19716
Signed-off-by: Mykola Golub <mgolub@mirantis.com>
(cherry picked from commit e009e1bdd4b3997462feb9a050bd2eb201e028ba)

8 years agoMerge pull request #13174 from smithfarm/wip-18711-kraken
Nathan Cutler [Tue, 20 Jun 2017 06:46:28 +0000 (08:46 +0200)]
Merge pull request #13174 from smithfarm/wip-18711-kraken

kraken: rgw: slave zonegroup cannot enable the bucket versioning

Reviewed-by: Orit Wasserman <owasserm@redhat.com>
8 years agorgw: fix off-by-one in RGWDataChangesLog::get_info 13181/head
Casey Bodley [Wed, 11 Jan 2017 14:32:59 +0000 (09:32 -0500)]
rgw: fix off-by-one in RGWDataChangesLog::get_info

Fixes: http://tracker.ceph.com/issues/18488
Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit a0974fdcf62e60cf31bc15588e7b718da6f6ade3)

8 years agoradosgw/swift: clean up flush / newline behavior. 13224/head
Marcus Watts [Wed, 11 Jan 2017 05:06:15 +0000 (00:06 -0500)]
radosgw/swift: clean up flush / newline behavior.

The current code emits a newline after swift errors, but fails
to account for it when it calculates 'content-length'.  This results in
some clients (go github.com/ncw/swift) producing complaints about the
unsolicited newline such as this,
Unsolicited response received on idle HTTP channel starting with "\n"; err=<nil>

This logic eliminates the newline on flush.  This makes the content length
calculation correct and eliminates the stray newline.

There was already existing separator logic in the rgw plain formatter
that can emit a newline at the correct point.  It had been checking
"len" to decide if previous data had been emitted, but that's reset to 0
by flush().  So, this logic adds a new per-instance variable to separately
track state that it emitted a previous item (and should emit a newline).

Fixes: http://tracker.ceph.com/issues/18473
Signed-off-by: Marcus Watts <mwatts@redhat.com>
Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
(cherry picked from commit 5f229d6a33eae4906f22cdb90941835e47ee9f02)

8 years agorgw: fix for broken yields in RGWMetaSyncShardCR 13838/head
Casey Bodley [Tue, 29 Nov 2016 16:29:41 +0000 (11:29 -0500)]
rgw: fix for broken yields in RGWMetaSyncShardCR

Fixes: http://tracker.ceph.com/issues/18076
Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit e62d48a9bf2e309eab1a863f167af5267ebcc371)

8 years agorgw: complete versioning enablement after sending it to meta master 13174/head
Orit Wasserman [Mon, 12 Dec 2016 13:00:05 +0000 (14:00 +0100)]
rgw: complete versioning enablement after sending it to meta master

Fixes: http://tracker.ceph.com/issues/18003
Signed-off-by: Orit Wasserman <owasserm@redhat.com>
(cherry picked from commit 2d8aafb9dbe64bd9dd2b7d5ed50c6e9550cbe1ab)

8 years agomsg/async: go to open new session when existing already closed 15729/head
Haomai Wang [Tue, 13 Jun 2017 02:19:55 +0000 (10:19 +0800)]
msg/async: go to open new session when existing already closed

Fixes: http://tracker.ceph.com/issues/20230
Signed-off-by: Haomai Wang <haomai@xsky.com>
(cherry picked from commit 99f580a3959240f99061a9ad48ec591b39a9fd46)

8 years agomsg/async: fix accept_conn not remove entry in conns when lazy delete
Haomai Wang [Tue, 13 Jun 2017 02:16:47 +0000 (10:16 +0800)]
msg/async: fix accept_conn not remove entry in conns when lazy delete

Signed-off-by: Haomai Wang <haomai@xsky.com>
(cherry picked from commit bf98babb3289a7714543ff3cbd3872d80f0dc196)

8 years agolibrbd: fix rbd_metadata_list and rbd_metadata_get 15612/head
Mykola Golub [Tue, 11 Apr 2017 20:31:43 +0000 (22:31 +0200)]
librbd: fix rbd_metadata_list and rbd_metadata_get

- properly check for val_len in rbd_metadata_list
- don't expect output buffers are zero pre-filled

Fixes: http://tracker.ceph.com/issues/19588
Signed-off-by: Mykola Golub <mgolub@mirantis.com>
(cherry picked from commit 75afc74ea681402e22b6dec8b83276d145fc786b)

8 years agoMerge pull request #15421 from dzafman/wip-20125
David Zafman [Fri, 2 Jun 2017 16:54:45 +0000 (09:54 -0700)]
Merge pull request #15421 from dzafman/wip-20125

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
8 years agoosd: Object level shard errors are tracked and used if no auth available 15421/head
David Zafman [Wed, 31 May 2017 22:39:19 +0000 (15:39 -0700)]
osd: Object level shard errors are tracked and used if no auth available

Shards with object mismatch are tracked to mark them inconsistent
Fix test because storing omap_digest in object_info not behaving as before

Fixes: http://tracker.ceph.com/issues/20089
Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit 1cacbea763c7aabfeaaf4bd5e878301044184117)

8 years agomon/PGMap: factor mon_osd_full_ratio into MAX AVAIL calc 15237/head
Sage Weil [Fri, 3 Feb 2017 15:08:33 +0000 (10:08 -0500)]
mon/PGMap: factor mon_osd_full_ratio into MAX AVAIL calc

If we only fill OSDs to 95%, we should factor that into
the MAX AVAIL calculation for the pool.

Fixes: http://tracker.ceph.com/issues/18522
Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit f223ac92917f4bc18e5b9b3ad61afa155e4d088a)

8 years agoMerge pull request #14345 from ceph/wip-ceph-disk-fix-kraken
Boris Ranto [Thu, 11 May 2017 06:06:47 +0000 (08:06 +0200)]
Merge pull request #14345 from ceph/wip-ceph-disk-fix-kraken

ceph-disk: Add fix subcommand kraken back-port

Reviewed-by: Loic Dachary <ldachary@redhat.com>
Reviewed-by: Nathan Cutler <ncutler@suse.com>
8 years agoceph-disk: Add --system option for fix command 14345/head
Boris Ranto [Wed, 8 Mar 2017 08:38:39 +0000 (09:38 +0100)]
ceph-disk: Add --system option for fix command

This adds the ability to restore the labels of the underlying system
data in addition to ceph data.

Signed-off-by: Boris Ranto <branto@redhat.com>
(cherry picked from commit 8d81af42fd507c7b92c8279eb114b0a733ac1da6)

8 years agoceph-disk: Add more fix targets
Boris Ranto [Thu, 16 Feb 2017 10:34:27 +0000 (11:34 +0100)]
ceph-disk: Add more fix targets

It turns out I forgot several more directories that needs to be fixed by
this script. We need to fix /var/log/ceph, /var/run/ceph and /etc/ceph
as well.

Signed-off-by: Boris Ranto <branto@redhat.com>
(cherry picked from commit ae139307d6b2bfba47e21d29d6dbd3c8dc01b5b5)

8 years agoceph-disk: Add unit test for fix command
Boris Ranto [Thu, 9 Feb 2017 18:17:12 +0000 (19:17 +0100)]
ceph-disk: Add unit test for fix command

This will simulate the command* functions to not actually run anything
thus excercising the python code directly. It also checks that the
proper (sub-strings) are in the output.

Signed-off-by: Boris Ranto <branto@redhat.com>
(cherry picked from commit 1ec53dee9a690134936bdc3a09c9a02fecf13a9d)

8 years agoceph-disk: Add fix subcommand
Boris Ranto [Tue, 31 Jan 2017 12:19:33 +0000 (13:19 +0100)]
ceph-disk: Add fix subcommand

This subcommand will fix the SELinux labels and/or file permissions on
ceph data (/var/lib/ceph).

The command is also optimized to run the commands in parallel (per
sub-dir in /var/lib/ceph) and do restorecon and chown at the same time
to take advantage of the caching mechanisms.

Signed-off-by: Boris Ranto <branto@redhat.com>
(cherry picked from commit 6d5d30f6ed7538271579cc2ef4e2e364f01a4a6f)

8 years agoMerge pull request #14983 from liewegas/wip-denc-dump-kraken
Sage Weil [Mon, 8 May 2017 16:33:59 +0000 (11:33 -0500)]
Merge pull request #14983 from liewegas/wip-denc-dump-kraken

ceph-object-corpus: kraken objects (kraken)

8 years agotest/encoding/readable.sh: join children at finish 14983/head
Sage Weil [Mon, 27 Feb 2017 21:10:34 +0000 (16:10 -0500)]
test/encoding/readable.sh: join children at finish

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit c8a349eb4603be295a6f0e1683105cb93bfcd066)

8 years agotest/encoding/readable: fix passing of results to parent
Sage Weil [Mon, 27 Feb 2017 21:10:24 +0000 (16:10 -0500)]
test/encoding/readable: fix passing of results to parent

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 1325ba672d5b5c8a12722a395ad029c98d1ff236)

8 years agomessages/MCommand: fix type on decode
Sage Weil [Fri, 5 May 2017 20:48:25 +0000 (16:48 -0400)]
messages/MCommand: fix type on decode

Wow, this has been broken since v0.38, but apparently
the message never made it into the object corpus so
we never noticed!

In reality the bug is harmless: decode_message() will
set_header which clobbers whatever version the default
ctor fills in, so this only affects ceph-dencoder's
test.

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 3018b907c1d0375b5ae20802811591c0a46b61be)

8 years agoceph-object-corpus: kraken objects
Sage Weil [Fri, 5 May 2017 20:32:48 +0000 (16:32 -0400)]
ceph-object-corpus: kraken objects

Signed-off-by: Sage Weil <sage@redhat.com>
8 years agotest/encoding/generate-corpus-objects: fix typo
Sage Weil [Fri, 5 May 2017 20:31:02 +0000 (16:31 -0400)]
test/encoding/generate-corpus-objects: fix typo

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit b89dff472c86c435fcee767ec9d992b308c4d816)

8 years agotest/encoding/generate-corpus-objects.sh: simplify object corpus population
Sage Weil [Fri, 5 May 2017 13:19:23 +0000 (09:19 -0400)]
test/encoding/generate-corpus-objects.sh: simplify object corpus population

Script that generates corpus objects.

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit b38b663a71fdc37a2d5c3341836b6716d576cecc)

8 years agoqa/workunits/rgw/run-s3tests.sh: run s3-tests
Sage Weil [Fri, 5 May 2017 19:19:05 +0000 (15:19 -0400)]
qa/workunits/rgw/run-s3tests.sh: run s3-tests

This works out of the box with a vstart environment and

 RGW=1 ../src/vstart.sh -n -l
 PATH=bin:$PATH ../qa/workunits/rgw/run-s3tests.sh

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 40fe1d181cf8d31b80db6f8f99394bd66d2affd5)

8 years agoinclude/denc: support ENCODE_DUMP
Sage Weil [Thu, 4 May 2017 22:33:21 +0000 (18:33 -0400)]
include/denc: support ENCODE_DUMP

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 97987b4ace737b86a4dc2dce94ce9d330fbe903c)

8 years agocls_rbd: default initialize snapshot namespace for legacy clients 14934/head
Jason Dillaman [Tue, 2 May 2017 01:06:19 +0000 (21:06 -0400)]
cls_rbd: default initialize snapshot namespace for legacy clients

Creating a snapshot on >=Kraken OSDs using <=Jewel clients can result
in an improperly initialized snapshot namespace. As a result, attempting
to remove the snapshot using a >=Kraken client will result in an -EINVAL
error.

Fixes: http://tracker.ceph.com/issues/19413
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 03b0b03071f3e04754896664c69f73759ddb907a)

Conflicts:
src/cls/rbd/cls_rbd.h: trivial resolution

8 years agomgr/DaemonServer: do not crash on bad authorizer 14097/head
Sage Weil [Fri, 28 Apr 2017 22:03:59 +0000 (18:03 -0400)]
mgr/DaemonServer: do not crash on bad authorizer

Signed-off-by: Sage Weil <sage@redhat.com>
8 years agoMerge pull request #14506 from smithfarm/wip-19119-kraken
Nathan Cutler [Wed, 26 Apr 2017 17:53:49 +0000 (19:53 +0200)]
Merge pull request #14506 from smithfarm/wip-19119-kraken

kraken: doc: PendingReleaseNotes: warning about 'osd rm ...' and #13733

Reviewed-by: Sage Weil <sage@redhat.com>
8 years agoMerge pull request #14692 from smithfarm/wip-fix-hammer-jewel-x
Nathan Cutler [Wed, 26 Apr 2017 17:28:58 +0000 (19:28 +0200)]
Merge pull request #14692 from smithfarm/wip-fix-hammer-jewel-x

qa/suites/upgrade: add tiering test to hammer-jewel-x

Reviewed-by: Kefu Chai <kchai@redhat.com>
8 years agoMerge pull request #14425 from smithfarm/wip-19564-kraken
Nathan Cutler [Mon, 24 Apr 2017 20:22:14 +0000 (22:22 +0200)]
Merge pull request #14425 from smithfarm/wip-19564-kraken

kraken: build/ops: ceph-base missing dependency for psmisc in Ubuntu Xenial

Reviewed-by: Ken Dreyer <kdreyer@redhat.com>
8 years agoMerge pull request #13494 from shinobu-x/wip-18516-kraken
Nathan Cutler [Sat, 22 Apr 2017 08:55:16 +0000 (10:55 +0200)]
Merge pull request #13494 from shinobu-x/wip-18516-kraken

kraken: build/ops: systemd: Start OSDs after MONs

Reviewed-by: Nathan Cutler <ncutler@suse.com>
8 years agoMerge pull request #14588 from asheplyakov/19618-kraken
Nathan Cutler [Fri, 21 Apr 2017 15:36:42 +0000 (17:36 +0200)]
Merge pull request #14588 from asheplyakov/19618-kraken

kraken: mon/MonClient: make get_mon_log_message() atomic

Reviewed-by: Kefu Chai <kchai@redhat.com>
8 years agoMerge pull request #13543 from shinobu-x/wip-18954-kraken
Nathan Cutler [Fri, 21 Apr 2017 15:12:29 +0000 (17:12 +0200)]
Merge pull request #13543 from shinobu-x/wip-18954-kraken

kraken: ceph-disk prepare get wrong group name in bluestore

Reviewed-by: Loic Dachary <ldachary@redhat.com>
8 years agoMerge pull request #12746 from SUSE/wip-18387-kraken
Nathan Cutler [Fri, 21 Apr 2017 06:53:25 +0000 (08:53 +0200)]
Merge pull request #12746 from SUSE/wip-18387-kraken

kraken: tests: use ceph-kraken branch for s3tests

Reviewed-by: Orit Wasserman <owasserm@redhat.com>
8 years agoqa/suites/upgrade: add tiering test to hammer-jewel-x 14692/head
Kefu Chai [Mon, 6 Mar 2017 07:50:59 +0000 (15:50 +0800)]
qa/suites/upgrade: add tiering test to hammer-jewel-x

Related: http://tracker.ceph.com/issues/19185
Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit e8c04a027b5d108b30e63ab759760add57d43198)

8 years agoMerge pull request #14620 from smithfarm/wip-19659-kraken
Nathan Cutler [Thu, 20 Apr 2017 19:12:42 +0000 (21:12 +0200)]
Merge pull request #14620 from smithfarm/wip-19659-kraken

kraken: librbd: corrected resize RPC message backwards compatibility

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
8 years agoMerge pull request #14574 from smithfarm/wip-19620-kraken
Nathan Cutler [Thu, 20 Apr 2017 17:06:35 +0000 (19:06 +0200)]
Merge pull request #14574 from smithfarm/wip-19620-kraken

kraken: cephfs: MDS server crashes due to inconsistent metadata.

Reviewed-by: John Spray <john.spray@redhat.com>
8 years agoMerge pull request #14573 from smithfarm/wip-19483-kraken
Nathan Cutler [Thu, 20 Apr 2017 17:05:44 +0000 (19:05 +0200)]
Merge pull request #14573 from smithfarm/wip-19483-kraken

kraken: cephfs: No output for ceph mds rmfailed 0 --yes-i-really-mean-it command

Reviewed-by: John Spray <john.spray@redhat.com>
8 years agoMerge pull request #14572 from smithfarm/wip-19335-kraken
Nathan Cutler [Thu, 20 Apr 2017 17:04:34 +0000 (19:04 +0200)]
Merge pull request #14572 from smithfarm/wip-19335-kraken

kraken: cephfs: MDS heartbeat timeout during rejoin, when working with large amount of caps/inodes

Reviewed-by: John Spray <john.spray@redhat.com>
8 years agoMerge pull request #14571 from smithfarm/wip-19045-kraken
Nathan Cutler [Thu, 20 Apr 2017 17:03:42 +0000 (19:03 +0200)]
Merge pull request #14571 from smithfarm/wip-19045-kraken

kraken: cephfs: buffer overflow in test LibCephFS.DirLs

Reviewed-by: John Spray <john.spray@redhat.com>
8 years agoMerge pull request #14570 from smithfarm/wip-18950-kraken
Nathan Cutler [Thu, 20 Apr 2017 17:02:54 +0000 (19:02 +0200)]
Merge pull request #14570 from smithfarm/wip-18950-kraken

kraken: cephfs: mds/StrayManager: avoid reusing deleted inode in StrayManager::_purge_stray_logged

Reviewed-by: John Spray <john.spray@redhat.com>
8 years agoMerge pull request #14569 from smithfarm/wip-18899-kraken
Nathan Cutler [Thu, 20 Apr 2017 17:02:02 +0000 (19:02 +0200)]
Merge pull request #14569 from smithfarm/wip-18899-kraken

kraken: cephfs: test_open_inode fails

Reviewed-by: John Spray <john.spray@redhat.com>