]> git.apps.os.sepia.ceph.com Git - ceph.git/log
ceph.git
7 years agotools/ceph_objectstore_tool: fix 'dup' unable to duplicate meta PG 17623/head
xie xingguo [Wed, 6 Sep 2017 09:31:16 +0000 (17:31 +0800)]
tools/ceph_objectstore_tool: fix 'dup' unable to duplicate meta PG

Recently we plan to bring a Jewel cluster into Luminous.
After that is done, which turns out to be a big success,
we then try to transform all FileStore osds into BlueStore ones
offline but with no luck. The ceph_objectstore_tool keeps complaining:
--------------------------------------------------------------------
dup from filestore: /var/lib/ceph/osd/ceph-20.old
      to bluestore: /var/lib/ceph/osd/ceph-20
fsid d444b253-337d-4d15-9d63-86ae134ec9ac
65 collections
1/65 meta
cannot get bit count for collection meta: (61) No data available
--------------------------------------------------------------------

The root cause is that for FileStore Luminous will always try to rewrite
pg "bits" as a file attribute on "Load" if that is not available.
But since meta pg is never loaded (we skip it during OSD::load_pgs()),
we actually never get the chance to do so; hence making the
dup method from ceph_objectstore_tool very unhappy since it always
expects to see such a attribute from underlying store.

Fix the above problem by manually skipping loading the "bits" attribute
if underlying OS is FileStore for dup.

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
(cherry picked from commit af2c4847e42945a878e2864319185d795b0664a5)

7 years agoMerge pull request #17496 from smithfarm/wip-21236-luminous
Sage Weil [Sun, 10 Sep 2017 18:59:45 +0000 (13:59 -0500)]
Merge pull request #17496 from smithfarm/wip-21236-luminous

luminous: osd: build_initial_pg_history doesn't update up/acting/etc

7 years agoMerge pull request #17601 from liewegas/wip-21171-luminous
Sage Weil [Sun, 10 Sep 2017 18:56:51 +0000 (13:56 -0500)]
Merge pull request #17601 from liewegas/wip-21171-luminous

os/bluestore: fix deferred write deadlock, aio short return handling

7 years agoMerge pull request #17551 from dillaman/wip-21280
Jason Dillaman [Fri, 8 Sep 2017 18:04:14 +0000 (14:04 -0400)]
Merge pull request #17551 from dillaman/wip-21280

luminous: test/cls_rbd: mask newer feature bits to support upgrade tests

Reviewed-by: Mykola Golub <mgolub@mirantis.com>
7 years agoMerge pull request #17467 from smithfarm/wip-21046-luminous
Sage Weil [Fri, 8 Sep 2017 15:58:39 +0000 (10:58 -0500)]
Merge pull request #17467 from smithfarm/wip-21046-luminous

luminous: common: adding line break at end of some cli results

7 years agoMerge pull request #17472 from smithfarm/wip-21108-luminous
Sage Weil [Fri, 8 Sep 2017 15:57:33 +0000 (10:57 -0500)]
Merge pull request #17472 from smithfarm/wip-21108-luminous

luminous: mon: OSDMonitor: osd pool application get support

Reviewed-by: Sage Weil <sage@redhat.com>
7 years agoMerge pull request #17475 from smithfarm/wip-21133-luminous
Sage Weil [Fri, 8 Sep 2017 15:56:56 +0000 (10:56 -0500)]
Merge pull request #17475 from smithfarm/wip-21133-luminous

luminous: osd: PrimaryLogPG: sparse read won't trigger repair correctly

Reviewed-by: Sage Weil <sage@redhat.com>
7 years agoMerge pull request #17481 from smithfarm/wip-21182-luminous
Sage Weil [Fri, 8 Sep 2017 15:55:55 +0000 (10:55 -0500)]
Merge pull request #17481 from smithfarm/wip-21182-luminous

luminous: core: osd crush rule rename not idempotent

Reviewed-by: Sage Weil <sage@redhat.com>
7 years agoMerge pull request #17494 from smithfarm/wip-21234-luminous
Sage Weil [Fri, 8 Sep 2017 15:55:37 +0000 (10:55 -0500)]
Merge pull request #17494 from smithfarm/wip-21234-luminous

luminous: bluestore: asyn cdeferred_try_submit deadlock

Reviewed-by: Sage Weil <sage@redhat.com>
7 years agoMerge pull request #17469 from smithfarm/wip-21103-luminous
Yuri Weinstein [Fri, 8 Sep 2017 15:51:33 +0000 (08:51 -0700)]
Merge pull request #17469 from smithfarm/wip-21103-luminous

luminous: cephfs: client: missing space in some client debug log messages

Reviewed-by: Nathan Cutler <ncutler@suse.com>
7 years agoMerge pull request #17562 from theanalyst/wip-21276-luminous
Yuri Weinstein [Fri, 8 Sep 2017 15:48:05 +0000 (08:48 -0700)]
Merge pull request #17562 from theanalyst/wip-21276-luminous

luminous: os/bluestore/BlueFS.cc: 1255: FAILED assert(!log_file->fnode.extents.empty())

Reviewed-by: Sage Weil <sage@redhat.com>
7 years agoMerge pull request #17345 from liewegas/wip-pr-17311-luminous
Sage Weil [Fri, 8 Sep 2017 14:28:35 +0000 (09:28 -0500)]
Merge pull request #17345 from liewegas/wip-pr-17311-luminous

mon: use ceph_clock_now if message is self-generated

Reviewed-by: Kefu Chai <kchai@redhat.com>
7 years agoMerge pull request #17344 from liewegas/wip-pr-17330-luminous
Sage Weil [Fri, 8 Sep 2017 14:28:17 +0000 (09:28 -0500)]
Merge pull request #17344 from liewegas/wip-pr-17330-luminous

mon/OSDMonitor: make 'osd crush class rename' idempotent

7 years agoMerge pull request #17527 from liewegas/wip-pr-17510-luminous
Sage Weil [Fri, 8 Sep 2017 14:27:55 +0000 (09:27 -0500)]
Merge pull request #17527 from liewegas/wip-pr-17510-luminous

crush: fix fast rule lookup when uniform

7 years agoMerge pull request #17511 from liewegas/wip-stupid-luminous
Sage Weil [Fri, 8 Sep 2017 14:27:31 +0000 (09:27 -0500)]
Merge pull request #17511 from liewegas/wip-stupid-luminous

os/bluestore: default to stupid allocator

Reviewed-by: Brad Hubbard <bhubbard@redhat.com>
7 years agoMerge pull request #17501 from smithfarm/wip-21242-luminous
Sage Weil [Fri, 8 Sep 2017 14:26:27 +0000 (09:26 -0500)]
Merge pull request #17501 from smithfarm/wip-21242-luminous

luminous: osd: osd/PG: discard msgs from down peers

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Sage Weil <sage@redhat.com>
7 years agoMerge pull request #17500 from smithfarm/wip-21240-luminous
Sage Weil [Fri, 8 Sep 2017 14:25:37 +0000 (09:25 -0500)]
Merge pull request #17500 from smithfarm/wip-21240-luminous

luminous: mon: rate limit on health check update logging

Reviewed-by: Sage Weil <sage@redhat.com>
7 years agoMerge pull request #17498 from smithfarm/wip-21238-luminous
Sage Weil [Fri, 8 Sep 2017 14:24:50 +0000 (09:24 -0500)]
Merge pull request #17498 from smithfarm/wip-21238-luminous

luminous: osd: subscribe to new osdmap while waiting_for_healthy

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Sage Weil <sage@redhat.com>
7 years agoMerge pull request #17497 from smithfarm/wip-21237-luminous
Sage Weil [Fri, 8 Sep 2017 14:22:49 +0000 (09:22 -0500)]
Merge pull request #17497 from smithfarm/wip-21237-luminous

luminous: bluestore: osd crash when change option bluestore_csum_type from none to CRC32

Reviewed-by: Sage Weil <sage@redhat.com>
7 years agoMerge pull request #17378 from jan--f/wip-21187-luminous
Sage Weil [Fri, 8 Sep 2017 14:22:23 +0000 (09:22 -0500)]
Merge pull request #17378 from jan--f/wip-21187-luminous

luminous: core: osd/PGLog: write only changed dup entries

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
7 years agoos/bluestore/aio: handle short return from io_submit 17601/head
Sage Weil [Thu, 7 Sep 2017 20:28:59 +0000 (16:28 -0400)]
os/bluestore/aio: handle short return from io_submit

io_submit may return a value less than nr, indicating that only some of
the provided iocbs were queued.  If that happens we should loop, not
return and silently drop those aios on the floor.

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit dc17dfd9ff05b5676488c2b1bca53026b2ca6244)

7 years agoos/bluestore: more deferred debugging
Sage Weil [Wed, 30 Aug 2017 21:18:18 +0000 (17:18 -0400)]
os/bluestore: more deferred debugging

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 11ac691dcc86aeed4ccb692a79ade4be04c5ef2f)

7 years agoos/bluestore: drop deferred_submit_lock
Sage Weil [Tue, 29 Aug 2017 20:15:52 +0000 (16:15 -0400)]
os/bluestore: drop deferred_submit_lock

This lock serves no purpose.  It only protects the call to the aio_submit
on the OpSequencer running batch, but we are the only caller who will do
that submission, and a new batch won't be moved into place until ours is
completed and retired (by the completion for the aio we are queueing).

More importantly, this fixes a deadlock:

- thread A submits aio, but the queue is full, and blocks
- thread B blocks taking deferred_submit_lock while holding deferred_lock
- aio completion thread blocks trying to take deferred_lock, and thus
no further aios are drained.

Fixes: http://tracker.ceph.com/issues/21171
Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 7a5ef62af194373c3a8c307b05ae6d467e49c9f0)

7 years agoMerge pull request #17534 from liewegas/wip-pr-17459-luminous
Pan Liu [Fri, 8 Sep 2017 01:34:36 +0000 (09:34 +0800)]
Merge pull request #17534 from liewegas/wip-pr-17459-luminous

os/bluestore: add bluestore_prefer_deferred_size_hdd/ssd to tracked keys

7 years agoos/bluestore/BlueFS: prevent _compact_log_async reentry 17562/head
Sage Weil [Tue, 5 Sep 2017 19:01:02 +0000 (15:01 -0400)]
os/bluestore/BlueFS: prevent _compact_log_async reentry

_should_compact_log uses new_log != nullptr to tell whether compaction is
already in progress, but we don't set it until we are midway through the
process.  Set it at the top of the method to prevent reentry.

See 455cc6cea2e5e8bc1126b5047b6003fa583af1b5, which failed to implement
this properly.

Fixes: http://tracker.ceph.com/issues/21250
Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 6bd9db304f906e9e4a07ca5ce07ffffdbc1678f0)

7 years agotest/cls_rbd: mask newer feature bits to support upgrade tests 17551/head
Jason Dillaman [Wed, 6 Sep 2017 01:37:16 +0000 (21:37 -0400)]
test/cls_rbd: mask newer feature bits to support upgrade tests

Fixes: http://tracker.ceph.com/issues/21217
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit aeb8f29d21625f3570e2f3e6dd26594d1a6df810)

7 years agoos/bluestore: add bluestore_prefer_deferred_size_hdd/ssd to tracked keys 17534/head
xie xingguo [Tue, 5 Sep 2017 03:45:53 +0000 (11:45 +0800)]
os/bluestore: add bluestore_prefer_deferred_size_hdd/ssd to tracked keys

To make these two keys can be changed online:

./bin/ceph tell osd.* injectargs "--bluestore_prefer_deferred_size_hdd 0"
osd.0: bluestore_prefer_deferred_size_hdd = '0' (not observed, change may require restart)
osd.1: bluestore_prefer_deferred_size_hdd = '0' (not observed, change may require restart)
osd.2: bluestore_prefer_deferred_size_hdd = '0' (not observed, change may require restart)

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
(cherry picked from commit cd1225634cf4d20e485f7c30d5a0438febc7ddcc)

7 years agocrush: fix fast rule lookup when uniform 17527/head
Sage Weil [Wed, 6 Sep 2017 02:25:03 +0000 (22:25 -0400)]
crush: fix fast rule lookup when uniform

Older clients will search for the first rule with a matching ruleset,
type, and size.  The has_uniform_rules bool is only set if we have rule
ids and rulesets that line up, but we must also verify that the rest of the
mask matches or else we can get a different CRUSH mapping result because
the mask might not match and old clients will fail to find a rule and we
will find one.  We also can't just check the ruleset as the legacy clients
find the *first* (of potentially many) matching rules; hence we only do
the fast check if all rulesets == rule id.

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit f24095e0e9734531dbdbcd4bff1392c463188e8e)

7 years agoos/bluestore: mark bluestore_allocator as DEV 17511/head
Sage Weil [Wed, 6 Sep 2017 03:04:15 +0000 (23:04 -0400)]
os/bluestore: mark bluestore_allocator as DEV

We don't want users changing this without a good reason.

Signed-off-by: Sage Weil <sage@redhat.com>
7 years agoos/bluestore: put StupidAllocator btree into bluestore_alloc mempool
Sage Weil [Thu, 10 Aug 2017 15:58:40 +0000 (11:58 -0400)]
os/bluestore: put StupidAllocator btree into bluestore_alloc mempool

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 6f189d8ab3c549a5c5e01616310c6eb39119fbc1)

7 years agoos/bluestore: default 'bluefs_allocator = stupid'
Sage Weil [Thu, 10 Aug 2017 15:21:32 +0000 (11:21 -0400)]
os/bluestore: default 'bluefs_allocator = stupid'

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 2b6eea80eacd65cbb1aa3702a3a3b642304c04f5)

7 years agoos/bluestore: default to bluestore_allocator = stupid
Sage Weil [Tue, 8 Aug 2017 16:14:07 +0000 (12:14 -0400)]
os/bluestore: default to bluestore_allocator = stupid

Performance looks better, and code is simpler.

Also fix config option annotations.

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 3750cf9cafabd3296c22b8b8dff35e44d18b9dd8)

7 years agoosd/PG: discard msgs from down peers 17501/head
Kefu Chai [Thu, 24 Aug 2017 08:04:54 +0000 (16:04 +0800)]
osd/PG: discard msgs from down peers

if a repop is replied after a replica goes down in a new osdmap, and
before the pg advances to this new osdmap, the repop replies before this
repop can be discarded by that replica OSD, because the primary resets the
connection to it when handling the new osdmap marking it down, and also
resets the messenger sesssion when the replica reconnects. to avoid the
out-of-order replies, the messages from that replica should be discarded.

Fixes: http://tracker.ceph.com/issues/19605
Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit e3fce6be44506168a7a138aab93f6a4d6776397b)

7 years agomon: rate limit on health check update logging 17500/head
John Spray [Mon, 7 Aug 2017 11:52:42 +0000 (07:52 -0400)]
mon: rate limit on health check update logging

Fixes: http://tracker.ceph.com/issues/20888
Signed-off-by: John Spray <john.spray@redhat.com>
(cherry picked from commit 5b903c7b428c2c7024fdfc1dffacad0b1521b750)

7 years agomon: add helper for emitting logs by health status
John Spray [Fri, 1 Sep 2017 09:33:16 +0000 (05:33 -0400)]
mon: add helper for emitting logs by health status

We were starting to repeat our little "if(status==HEALTH_ERR"
blocks in too many places.

Signed-off-by: John Spray <john.spray@redhat.com>
(cherry picked from commit 618a27868d65aa152ac67cce8057a75c4d61def9)

7 years agoosd: subscribe to new osdmap while waiting_for_healthy 17498/head
Sage Weil [Thu, 24 Aug 2017 21:56:13 +0000 (17:56 -0400)]
osd: subscribe to new osdmap while waiting_for_healthy

If we are sitting around waiting until we are able to ping our "up" peers,
we need to be sure that our notion of "up" is still correct and we're not
just stuck on an old, stale OSDMap.

Fixes: http://tracker.ceph.com/issues/21121
Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit fbafa659dae94faba435ca449ee5e77b51108b4b)

7 years agoos/bluestore: formatting nits 17497/head
xie xingguo [Thu, 31 Aug 2017 04:42:36 +0000 (12:42 +0800)]
os/bluestore: formatting nits

So we don't output anything if csum is currently off:

_do_alloc_write blob Blob(0x7fad7cc25950 blob([0x42a80000~10000] none/0x1)

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
(cherry picked from commit 22740cc9b3e489ae82b4b3071a17793c11abce79)

7 years agoos/bluestore: don't re-initialize csum-setting for existing blobs
xie xingguo [Thu, 31 Aug 2017 03:42:37 +0000 (11:42 +0800)]
os/bluestore: don't re-initialize csum-setting for existing blobs

The global checksum setting may change, e.g., from NONE to CRC32,
which can cause improper re-initialization of the csum-settings of
existing blobs(e.g., partial write/overwrite may turn out to shrink
'csum_data').

We could develop some complicated solutions but for now let's not
bother since the above scenario is rare.

Fixes: http://tracker.ceph.com/issues/21175
Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
(cherry picked from commit 8ec4b4960f9b93ff6d85a02308ccda0bccadb599)

7 years agoosd: fix build_initial_pg_history 17496/head
w11979 [Fri, 1 Sep 2017 14:02:34 +0000 (10:02 -0400)]
osd: fix build_initial_pg_history

We need to update our info about the previous interval in order to
detect interval changes properly.

Fixes: http://tracker.ceph.com/issues/21203
Signed-off-by: w11979 <wang.wenfeng@h3c.com>
Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit afdacc6ed76dd288fdbc0f99857d9ffba23c5132)

7 years agoos/bluestore: separate finisher for deferred_try_submit 17494/head
Sage Weil [Thu, 31 Aug 2017 20:43:39 +0000 (16:43 -0400)]
os/bluestore: separate finisher for deferred_try_submit

Reusing finishers[0], which is used for completions back into the OSD,
is deadlock-prone: the OSD code might block trying to submit new IO or
while waiting for some other bluestore work to complete.

Fixes: http://tracker.ceph.com/issues/21207
Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 2b6a9d1ec0778f66b6e9d9f216df21a00caeba3f)

7 years agomon/OSDMonitor: make 'osd crush rule rename' idempotent 17481/head
xie xingguo [Tue, 29 Aug 2017 00:21:07 +0000 (08:21 +0800)]
mon/OSDMonitor: make 'osd crush rule rename' idempotent

Fixes: http://tracker.ceph.com/issues/21162
Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
(cherry picked from commit cd2191c3f192986e20beaab84b643dfaf49901f7)

7 years agoosd/PrimaryLogPG: fix sparse read won't trigger repair correctly 17475/head
xie xingguo [Thu, 24 Aug 2017 07:53:23 +0000 (15:53 +0800)]
osd/PrimaryLogPG: fix sparse read won't trigger repair correctly

Normal reads instead of hole-reads should trigger auto-repair.
Also we shall quit on successfully triggering a repair, or else
we are at risk of doing the op multiple times.

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
(cherry picked from commit 4ad25c346215dc2ddc4b1a43948b6fe01316edf2)

7 years agomon/OSDMonitor: "osd pool application ls" support 17472/head
xie xingguo [Wed, 9 Aug 2017 13:24:49 +0000 (21:24 +0800)]
mon/OSDMonitor: "osd pool application ls" support

It would be a pain if we have to call 'ceph osd dump --format=json-pretty'
to find out these each time...

Demo output:
(1) ceph osd pool application get
{
    "cephfs_data_b": {
        "cephfs": {}
    },
    "cephfs_metadata_a": {
        "cephfs": {}
    },
    "test_pool": {
        "rbd": {
            "test": "me"
        }
    }
}

(2) ceph osd pool application get test_pool
{
    "rbd": {
        "test": "me"
    }
}

(3) ceph osd pool application get test_pool rbd
{
    "test": "me"
}

(4) ceph osd pool application get test_pool rbd test
me

Fixes: http://tracker.ceph.com/issues/20976
Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
(cherry picked from commit 46586b22882c0c08c099243e189f5a165063da0c)

7 years agoclient: add blank space between function name and inode number for log 17469/head
Zhu Shangzhong [Wed, 23 Aug 2017 02:20:17 +0000 (10:20 +0800)]
client: add blank space between function name and inode number for log

Fixes: http://tracker.ceph.com/issues/21069
Signed-off-by: Zhu Shangzhong <zhu.shangzhong@zte.com.cn>
(cherry picked from commit 0cb4d379fdd2b869bd765e1fcb424a8f8a8f6a04)

7 years agocommon: add new line break when get command descriptions 17467/head
songweibin [Thu, 17 Aug 2017 03:44:43 +0000 (11:44 +0800)]
common: add new line break when get command descriptions

Signed-off-by: songweibin <song.weibin@zte.com.cn>
(cherry picked from commit 19dd4203eb930ffb7a321cb617c580d99ceca9c1)

7 years agorbd: adding newline operation when only '--format' used
songweibin [Wed, 9 Aug 2017 06:26:32 +0000 (14:26 +0800)]
rbd: adding newline operation when only '--format' used

Signed-off-by: songweibin <song.weibin@zte.com.cn>
(cherry picked from commit 992829d38cb89732f6994642c1bdaa2448e610b7)

7 years agorbd: '--pretty-format' should return error if '--format' not used
songweibin [Tue, 1 Aug 2017 11:48:04 +0000 (19:48 +0800)]
rbd: '--pretty-format' should return error if '--format' not used

Signed-off-by: songweibin <song.weibin@zte.com.cn>
(cherry picked from commit 47e7549826b55905e7f298717fedf0e8f220940a)

Conflicts:
    src/test/cli/rbd/help.t (master has several new commands compared to
                            luminous: omit these)

7 years agoMerge pull request #17383 from liewegas/wip-luminous-mon-bits-luminous
Sage Weil [Sat, 2 Sep 2017 18:47:29 +0000 (13:47 -0500)]
Merge pull request #17383 from liewegas/wip-luminous-mon-bits-luminous

mon: final luminous mon bits

Reviewed-by: Kefu Chai <kchai@redhat.com>
7 years agoPendingReleaseNotes: must upgrade to 12.2.1 before mimic 17383/head
Sage Weil [Sat, 2 Sep 2017 18:46:47 +0000 (14:46 -0400)]
PendingReleaseNotes: must upgrade to 12.2.1 before mimic

Signed-off-by: Sage Weil <sage@redhat.com>
7 years agoMerge pull request #17429 from ceph/wip-luminous-x-luminous
Josh Durgin [Fri, 1 Sep 2017 21:37:38 +0000 (14:37 -0700)]
Merge pull request #17429 from ceph/wip-luminous-x-luminous

qa: add upgrade/luminous-x suite

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
7 years agoInitial check in luminous-x suite 17429/head
Yuri Weinstein [Fri, 1 Sep 2017 21:29:41 +0000 (14:29 -0700)]
Initial check in luminous-x suite

same from master https://github.com/ceph/ceph/pull/17160

Signed-off-by: Yuri Weinstein <yweinste@redhat.com>
7 years agoMerge pull request #17408 from ceph/luminous-bz1485011
Alfredo Deza [Fri, 1 Sep 2017 11:14:15 +0000 (07:14 -0400)]
Merge pull request #17408 from ceph/luminous-bz1485011

luminous: ceph-volume Persist non-lv devices for journals

Reviewed-by: Andrew Schoen <aschoen@redhat.com>
7 years agoceph-volume tests add new ceph_* ansible dev variables required (centos) 17408/head
Alfredo Deza [Thu, 31 Aug 2017 14:27:39 +0000 (10:27 -0400)]
ceph-volume tests add new ceph_* ansible dev variables required (centos)

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit d3abfd42ef8e558a1bbb31e24d0ab3c44ee7f4c4)

7 years agoceph-volume tests add new ceph_* ansible dev variables required (xenial)
Alfredo Deza [Wed, 30 Aug 2017 18:46:54 +0000 (14:46 -0400)]
ceph-volume tests add new ceph_* ansible dev variables required (xenial)

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit ebb48c58123a1cfcfc6eccdf9f6c2b3a720f1233)

7 years agoceph-volume util create a disk utility for blkid operations
Alfredo Deza [Wed, 30 Aug 2017 18:12:33 +0000 (14:12 -0400)]
ceph-volume util create a disk utility for blkid operations

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit a19e1288e8ae044f41dff9a1770126aa540cc729)

7 years agoceph-volume lvm.prepare store the blkid uuid of a partition journal
Alfredo Deza [Wed, 30 Aug 2017 18:11:52 +0000 (14:11 -0400)]
ceph-volume lvm.prepare store the blkid uuid of a partition journal

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 5160c5398bbf107bcc826696664c503aa52d0020)

7 years agoceph-volume lvm.activate use the partuuid of a partition to link the journal
Alfredo Deza [Wed, 30 Aug 2017 18:05:22 +0000 (14:05 -0400)]
ceph-volume lvm.activate use the partuuid of a partition to link the journal

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit a1080f833b35c5de30d7b577deabf0a0e82fbbf2)

7 years agoceph-volume tests update the ansible version for functional/tox.ini
Alfredo Deza [Tue, 29 Aug 2017 17:34:21 +0000 (13:34 -0400)]
ceph-volume tests update the ansible version for functional/tox.ini

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 007655afa3def6788e07d2db10c7cd9cb4ab45b7)

7 years agoceph-volume tests add pv* related unit tests
Alfredo Deza [Tue, 29 Aug 2017 17:25:28 +0000 (13:25 -0400)]
ceph-volume tests add pv* related unit tests

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 7c22743c9aa1306242fc09ccae5dedf94e025cc4)

7 years agoceph-volume lvm.api include lv_uuid as output fields
Alfredo Deza [Tue, 29 Aug 2017 15:31:56 +0000 (11:31 -0400)]
ceph-volume lvm.api include lv_uuid as output fields

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit d85656da2531471e11cb90fcc74570c9c2b804e0)

7 years agoceph-volume lvm.activate always update the link to the journal
Alfredo Deza [Tue, 29 Aug 2017 13:38:08 +0000 (09:38 -0400)]
ceph-volume lvm.activate always update the link to the journal

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 6a1de41838cb9dbbdc4a8f59eb07e4340ccb9680)

7 years agoceph-volume lvm.activate retrieve the journal uuid if journal is a device
Alfredo Deza [Tue, 29 Aug 2017 13:26:35 +0000 (09:26 -0400)]
ceph-volume lvm.activate retrieve the journal uuid if journal is a device

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit a368fa1ad6d6265c5094eacbe7e72f642b43559d)

7 years agoceph-volume exceptions create a specifc error for multiple pvs
Alfredo Deza [Mon, 28 Aug 2017 21:33:39 +0000 (17:33 -0400)]
ceph-volume exceptions create a specifc error for multiple pvs

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 73315c49115dbb45f3bb6c9e46955b333602cec2)

7 years agoceph-volume lvm.prepare make a journal a pv, use uuids always
Alfredo Deza [Mon, 28 Aug 2017 20:52:33 +0000 (16:52 -0400)]
ceph-volume lvm.prepare make a journal a pv, use uuids always

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 913cef2d01b91e12c16e42d268043886b62c72fa)

7 years agoceph-volume lvm.api create the PVolumes class and helpers
Alfredo Deza [Mon, 28 Aug 2017 20:51:11 +0000 (16:51 -0400)]
ceph-volume lvm.api create the PVolumes class and helpers

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit f2bc125d980485c7b456e0193ec9c66ea0acbe54)

7 years agoceph-volume lvm.api create the PVolume class
Alfredo Deza [Thu, 24 Aug 2017 23:28:12 +0000 (19:28 -0400)]
ceph-volume lvm.api create the PVolume class

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 1725a5f3589ba2faddc9ce633997e5563fc66a01)

7 years agoceph-volume lvm.api add a helper to create pvs
Alfredo Deza [Thu, 24 Aug 2017 20:37:48 +0000 (16:37 -0400)]
ceph-volume lvm.api add a helper to create pvs

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 00f3fa401871f3108558fea71f6ac03660d00ba1)

7 years agoqa/suites/rados/thrash/d-luminous/at-end: check for purged_snapdirs
Sage Weil [Wed, 30 Aug 2017 21:22:33 +0000 (17:22 -0400)]
qa/suites/rados/thrash/d-luminous/at-end: check for purged_snapdirs

Signed-off-by: Sage Weil <sage@redhat.com>
7 years agomon: set purged_snapdirs OSDMap flag once snapsets have all converted
Sage Weil [Tue, 29 Aug 2017 04:01:19 +0000 (00:01 -0400)]
mon: set purged_snapdirs OSDMap flag once snapsets have all converted

This makes it easier to test whether the upgrade + conversion has
completed.  In particular, mimic+ will be able to simply test for this
flag without waiting for complete PG stats to know whether it is safe to
upgrade beyond luminous.

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit c3cbaad2a1a4df68fe39394e07c2f3ac9818b9c8)

7 years agomon: CEPH_MON_FEATURE_INCOMPAT_LUMINOUS
Sage Weil [Mon, 28 Aug 2017 19:21:44 +0000 (15:21 -0400)]
mon: CEPH_MON_FEATURE_INCOMPAT_LUMINOUS

- set this once we have a luminous quorum
- require luminous if we have this set locally
- include this on mkfs

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 5b6c451e3fdaf0785b197fdfacf1c782b932bec9)

7 years agoMerge pull request #17381 from dillaman/ritz303-wip-doc-20437
Jason Dillaman [Wed, 30 Aug 2017 20:39:09 +0000 (16:39 -0400)]
Merge pull request #17381 from dillaman/ritz303-wip-doc-20437

luminous: doc/rbd: iSCSI Gateway Documentation

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
7 years agodoc: tweaks for the RBD iSCSI configuration documentation 17381/head
Jason Dillaman [Tue, 8 Aug 2017 23:14:57 +0000 (19:14 -0400)]
doc: tweaks for the RBD iSCSI configuration documentation

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit b0abab17e3fa96ca973791a3afe077406d066035)

7 years agodoc/rbd: Replaced numbered steps with a # for dynamic numbering
Aron Gunn [Wed, 12 Jul 2017 22:11:25 +0000 (17:11 -0500)]
doc/rbd: Replaced numbered steps with a # for dynamic numbering

Fixes: http://tracker.ceph.com/issues/20437
Signed-off-by: Aron Gunn <agunn@redhat.com>
(cherry picked from commit f109031cb5c8150f042f9754f16bfa5216411eda)

7 years agodoc/rbd: Updated the content based on engineering feedback
Aron Gunn [Thu, 6 Jul 2017 22:22:38 +0000 (17:22 -0500)]
doc/rbd: Updated the content based on engineering feedback

Fixes: http://tracker.ceph.com/issues/20437
Signed-off-by: Aron Gunn <agunn@redhat.com>
(cherry picked from commit 2a59c82b3d4bbb13bc82cf62fce1d9cb5ba7e6be)

7 years agodoc/rbd: Updated the content based on engineering feedback
Aron Gunn [Thu, 6 Jul 2017 22:22:02 +0000 (17:22 -0500)]
doc/rbd: Updated the content based on engineering feedback

Fixes: http://tracker.ceph.com/issues/20437
Signed-off-by: Aron Gunn <agunn@redhat.com>
(cherry picked from commit 1d4f5b458d127065d92652c323c73704cf775754)

7 years agodoc/rbd: Updated the content based on engineering feedback
Aron Gunn [Thu, 6 Jul 2017 22:20:40 +0000 (17:20 -0500)]
doc/rbd: Updated the content based on engineering feedback

Fixes: http://tracker.ceph.com/issues/20437
Signed-off-by: Aron Gunn <agunn@redhat.com>
(cherry picked from commit 441820d95890d95ea79bae55afb3ca3b1a8a36e3)

7 years agodoc/rbd: Updated the requirements based on engineering feedback
Aron Gunn [Thu, 6 Jul 2017 22:19:01 +0000 (17:19 -0500)]
doc/rbd: Updated the requirements based on engineering feedback

Fixes: http://tracker.ceph.com/issues/20437
Signed-off-by: Aron Gunn <agunn@redhat.com>
(cherry picked from commit c57266bf82c554b93653489a4b355f53b4b86424)

7 years agodoc/rbd: Added the iSCSI gateway requirements content
Aron Gunn [Thu, 6 Jul 2017 17:35:38 +0000 (12:35 -0500)]
doc/rbd: Added the iSCSI gateway requirements content

Fixes: http://tracker.ceph.com/issues/20437
Signed-off-by: Aron Gunn <agunn@redhat.com>
(cherry picked from commit 9335742140a2b4e8b408560c600a490b2749bd9d)

7 years agodoc/rbd: Added the iSCSI gateway overview content
Aron Gunn [Thu, 6 Jul 2017 17:34:49 +0000 (12:34 -0500)]
doc/rbd: Added the iSCSI gateway overview content

Fixes: http://tracker.ceph.com/issues/20437
Signed-off-by: Aron Gunn <agunn@redhat.com>
(cherry picked from commit d022034bb3a6e1b00cf185cae0b5fa3f4b617a38)

7 years agodoc/rbd: Added the iSCSI gateway 'gwtop' monitoring content
Aron Gunn [Thu, 6 Jul 2017 17:34:10 +0000 (12:34 -0500)]
doc/rbd: Added the iSCSI gateway 'gwtop' monitoring content

Fixes: http://tracker.ceph.com/issues/20437
Signed-off-by: Aron Gunn <agunn@redhat.com>
(cherry picked from commit 7e8233429ba158c1a4cfe6e58a7ce24ffe8759c9)

7 years agodoc/rbd: Added the iSCSI targets, Ansible and CLI docs
Aron Gunn [Thu, 6 Jul 2017 17:32:53 +0000 (12:32 -0500)]
doc/rbd: Added the iSCSI targets, Ansible and CLI docs

Fixes: http://tracker.ceph.com/issues/20437
Signed-off-by: Aron Gunn <agunn@redhat.com>
(cherry picked from commit cbf654c702e7b7a8e7023c4707c4ea38d584be3e)

7 years agodoc/rbd: Added the iSCSI initiators, RHEL, Windows, and ESX docs
Aron Gunn [Thu, 6 Jul 2017 17:31:39 +0000 (12:31 -0500)]
doc/rbd: Added the iSCSI initiators, RHEL, Windows, and ESX docs

Fixes: http://tracker.ceph.com/issues/20437
Signed-off-by: Aron Gunn <agunn@redhat.com>
(cherry picked from commit f0e177fa3d8807cfa6ae17f8a0dc759a045623f9)

7 years agodoc/rbd: Updated the "toctree" adding the iSCSI gateway overview doc
Aron Gunn [Thu, 6 Jul 2017 17:28:06 +0000 (12:28 -0500)]
doc/rbd: Updated the "toctree" adding the iSCSI gateway overview doc

Fixes: http://tracker.ceph.com/issues/20437
Signed-off-by: Aron Gunn <agunn@redhat.com>
(cherry picked from commit 4ef8426bff9a32e00093f7c5d2b6b9620252293b)

7 years agodoc/rbd: Added the unmodified iSCSI initiator images for ESX and Windows 2016
Aron Gunn [Thu, 6 Jul 2017 17:23:50 +0000 (12:23 -0500)]
doc/rbd: Added the unmodified iSCSI initiator images for ESX and Windows 2016

Fixes: http://tracker.ceph.com/issues/20437
Signed-off-by: Aron Gunn <agunn@redhat.com>
(cherry picked from commit 2fd61f8de1587d2bfbbb7e15c952a29566b390a1)

7 years agoosd/PGLog: write only changed dup entries 17378/head
Josh Durgin [Thu, 24 Aug 2017 22:03:36 +0000 (18:03 -0400)]
osd/PGLog: write only changed dup entries

Writing all of the dup entries whenever one is changed causes a large
regression in performance.

Instead, keep track of ranges that need to be cleared and written
after log merging (dirty_{to,from}_dups) and the earliest dup entry we
haven't written yet during normal operation (write_from_dups). This
parallels the way we track unpersisted log entries.

Check that the correct set of dups is persisted by adding a roundtrip
through memstore to each merge_log unit test.

Fixes: http://tracker.ceph.com/issues/21026
Signed-off-by: Josh Durgin <jdurgin@redhat.com>
(cherry picked from commit 75af72f758aacc1260b8d828d0b8bb7786597fdf)

7 years agoRevert "osd: adjust osd_min_pg_log_entries"
Josh Durgin [Thu, 24 Aug 2017 22:34:48 +0000 (18:34 -0400)]
Revert "osd: adjust osd_min_pg_log_entries"

This reverts commit 1c4df03394312fe67f36448613d8b54cb1a0e2c9.

Signed-off-by: Josh Durgin <jdurgin@redhat.com>
(cherry picked from commit 4c8fd9978369448e28355d62b4d1eef3685ea63b)

7 years agoMerge pull request #17359 from tchaikov/wip-luminous-ceph-release-stable
Sage Weil [Wed, 30 Aug 2017 13:28:33 +0000 (08:28 -0500)]
Merge pull request #17359 from tchaikov/wip-luminous-ceph-release-stable

luminous: ceph_release: luminous is now 'stable' (12.2.x)

7 years agoceph_release: luminous is now 'stable' (12.2.x) 17359/head
Kefu Chai [Wed, 30 Aug 2017 04:23:42 +0000 (12:23 +0800)]
ceph_release: luminous is now 'stable' (12.2.x)

Signed-off-by: Kefu Chai <kchai@redhat.com>
7 years agoMerge tag 'v12.2.0' into luminous
Sage Weil [Tue, 29 Aug 2017 14:48:19 +0000 (10:48 -0400)]
Merge tag 'v12.2.0' into luminous

v12.2.0

7 years agomon: use ceph_clock_now if message is self-generated 17345/head
huangjun [Mon, 28 Aug 2017 14:52:37 +0000 (22:52 +0800)]
mon: use ceph_clock_now if message is self-generated

Signed-off-by: huangjun <huangjun@xsky.com>
(cherry picked from commit 56e8031dac7ee65d54e831e7e372e8771fa0f94a)

7 years agomon/OSDMonitor: make 'osd crush class rename' idempotent 17344/head
xie xingguo [Tue, 29 Aug 2017 01:33:51 +0000 (09:33 +0800)]
mon/OSDMonitor: make 'osd crush class rename' idempotent

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
(cherry picked from commit 2ee80aead88c90388871ee65d4ed31a2fa47f532)

7 years agoMerge pull request #17332 from tchaikov/wip-luminous-pr-17269
Kefu Chai [Tue, 29 Aug 2017 13:33:58 +0000 (21:33 +0800)]
Merge pull request #17332 from tchaikov/wip-luminous-pr-17269

luminous: mon: fix two stray legacy get_health() callers

Reviewed-by: Sage Weil <sage@redhat.com>
7 years ago12.2.0
Jenkins Build Slave User [Mon, 28 Aug 2017 16:30:20 +0000 (16:30 +0000)]
12.2.0

7 years agomon: fix two stray legacy get_health() callers 17332/head
Sage Weil [Fri, 25 Aug 2017 22:30:12 +0000 (18:30 -0400)]
mon: fix two stray legacy get_health() callers

These need to call the new method when appropriate.

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 69038a18070d0cd47dc857020cce115514beaa3d)

7 years agoMerge pull request #17327 from tchaikov/wip-luminous-pr-17326
Sage Weil [Tue, 29 Aug 2017 02:41:56 +0000 (21:41 -0500)]
Merge pull request #17327 from tchaikov/wip-luminous-pr-17326

luminous: doc: fix order of options in osd new

Reviewed-by: Sage Weil <sage@redhat.com>
7 years agodoc: fix order of options in osd new 17327/head
Neha Ojha [Mon, 28 Aug 2017 22:59:19 +0000 (15:59 -0700)]
doc: fix order of options in osd new

Signed-off-by: Neha Ojha <nojha@redhat.com>
(cherry picked from commit e47463028921f0414e2f64c467f74cf37a1de085)

7 years ago12.2.0 v12.2.0
Jenkins Build Slave User [Mon, 28 Aug 2017 16:30:20 +0000 (16:30 +0000)]
12.2.0

7 years agoMerge pull request #17302 from tchaikov/wip-luminous-pr-17242
Sage Weil [Mon, 28 Aug 2017 14:07:34 +0000 (09:07 -0500)]
Merge pull request #17302 from tchaikov/wip-luminous-pr-17242

luminous: mon: warn about using osd new instead of osd create

Reviewed-by: Sage Weil <sage@redhat.com>
7 years agoMerge pull request #17303 from tchaikov/wip-luminous-pr-17225
Sage Weil [Mon, 28 Aug 2017 14:06:51 +0000 (09:06 -0500)]
Merge pull request #17303 from tchaikov/wip-luminous-pr-17225

luminous: os/bluestore: print leaked extents to debug output

Reviewed-by: Sage Weil <sage@redhat.com>
7 years agoMerge pull request #17264 from liewegas/wip-21127-luminous
Sage Weil [Mon, 28 Aug 2017 14:03:30 +0000 (09:03 -0500)]
Merge pull request #17264 from liewegas/wip-21127-luminous

luminous: mon/pgmap: add objects prefix for unfound type

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Sage Weil <sage@redhat.com>