]> git.apps.os.sepia.ceph.com Git - ceph.git/log
ceph.git
7 years agoceph-volume lvm.listing initial take on listing command
Alfredo Deza [Mon, 2 Oct 2017 18:31:57 +0000 (14:31 -0400)]
ceph-volume lvm.listing initial take on listing command

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 2d80190a039798a637383513f292e6aab62cd3f8)

7 years agoceph-volume lvm.api be able to return a dictionary repr of a Volume object
Alfredo Deza [Mon, 2 Oct 2017 16:35:13 +0000 (12:35 -0400)]
ceph-volume lvm.api be able to return a dictionary repr of a Volume object

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 0cae22750f396d1c1664875edb2742c50a41bffd)

7 years agoceph-volume process allows to fully mute terminal output on commands
Alfredo Deza [Mon, 2 Oct 2017 16:12:57 +0000 (12:12 -0400)]
ceph-volume process allows to fully mute terminal output on commands

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 233f64a4fd90a19c50d57b849ffaa6a25c835f34)

7 years agoceph-volume lvm.api use a helper to get an lv from an arg
Alfredo Deza [Thu, 14 Sep 2017 11:35:47 +0000 (07:35 -0400)]
ceph-volume lvm.api use a helper to get an lv from an arg

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 3c5bbbc5822f02a038f093647c5de58bef2fe8f5)

7 years agoMerge pull request #18456 from liewegas/wip-21882-luminous
Josh Durgin [Wed, 25 Oct 2017 18:45:38 +0000 (11:45 -0700)]
Merge pull request #18456 from liewegas/wip-21882-luminous

luminous: messages/MOSDMap: do compat reencode of crush map, too

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
7 years agoqa/suites/upgrade/jewel-x: make sure min compat client is hammer for straw2 18456/head
Sage Weil [Wed, 25 Oct 2017 12:00:14 +0000 (07:00 -0500)]
qa/suites/upgrade/jewel-x: make sure min compat client is hammer for straw2

Signed-off-by: Sage Weil <sage@redhat.com>
7 years agoMerge pull request #18503 from liewegas/wip-21878-luminous
Sage Weil [Wed, 25 Oct 2017 02:36:18 +0000 (21:36 -0500)]
Merge pull request #18503 from liewegas/wip-21878-luminous

luminous: os/bluestore/BlueFS: fix race with log flush during async log compaction

Reviewed-by: Varada Kari <varada.kari@gmail.com>
7 years agoMerge pull request #18501 from liewegas/wip-21766-luminous
Sage Weil [Wed, 25 Oct 2017 02:35:29 +0000 (21:35 -0500)]
Merge pull request #18501 from liewegas/wip-21766-luminous

luminous: os/bluestore: handle compressed extents in blob unsharing checks

7 years agoMerge pull request #18459 from liewegas/wip-pr-18457-luminous
Sage Weil [Wed, 25 Oct 2017 02:34:52 +0000 (21:34 -0500)]
Merge pull request #18459 from liewegas/wip-pr-18457-luminous

luminous: qa/suites/rados/rest/mgr-restful: whitelist more health

7 years agoMerge pull request #18491 from liewegas/wip-21573-luminous
Kefu Chai [Tue, 24 Oct 2017 03:51:19 +0000 (11:51 +0800)]
Merge pull request #18491 from liewegas/wip-21573-luminous

luminous: buffer: fix ABI breakage by removing list _mempool member

Reviewed-by: Kefu Chai <kchai@redhat.com>
7 years agoos/bluestore/BlueFS: fix race with log flush during async log compaction 18503/head
Sage Weil [Fri, 20 Oct 2017 13:51:17 +0000 (08:51 -0500)]
os/bluestore/BlueFS: fix race with log flush during async log compaction

During async log compaction we rely on _flush-and_sync_log to update the
log_writer to jump_to.  However, if racing threads are also trying to flush
the log and manage to flush our new log events for us, then our flush will
turn into a no-op, and we won't update jump_to correctly at all.  This
results in a corrupted log size a bit later one.

Fix by ensuring that there are no in-progress flushes before we add our
log entries.  Also, add asserts to _flush_and_sync_log to make sure we
never bail out early if jump_to is set (which would indicate this or
another similar bug is still present).

Fixes: http://tracker.ceph.com/issues/21878
Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 4324c8bc7e66633035c15995e3f82ef91d3a5e8c)

7 years agoos/bluestore: handle compressed extents in blob unsharing checks 18501/head
Sage Weil [Wed, 11 Oct 2017 21:48:41 +0000 (16:48 -0500)]
os/bluestore: handle compressed extents in blob unsharing checks

If the blob is compressed, we aren't mapping to a range within
the allocated extents, but rather referencing the entire blob.

Fixes: http://tracker.ceph.com/issues/21766
Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit eb26cfbfbb2b9b9fe84dfe9c999e2d67fd2661c0)

7 years agoqa/suites/upgrade/jewel-x/parallel: run some jewel after completed upgrade
Sage Weil [Sat, 21 Oct 2017 03:55:29 +0000 (22:55 -0500)]
qa/suites/upgrade/jewel-x/parallel: run some jewel after completed upgrade

Signed-off-by: Sage Weil <sage@redhat.com>
7 years agoqa/suites/upgrade/jewel-x/: set up compat weight-set after cluster upgrade
Sage Weil [Sat, 21 Oct 2017 03:52:45 +0000 (22:52 -0500)]
qa/suites/upgrade/jewel-x/: set up compat weight-set after cluster upgrade

...but before we run some client workloads.  This will hopefully ensure
that older clients will behave.

Signed-off-by: Sage Weil <sage@redhat.com>
7 years agomon/OSDMonitor: add 'osd crush set-all-straw-buckets-to-straw2'
Sage Weil [Sat, 21 Oct 2017 18:03:17 +0000 (13:03 -0500)]
mon/OSDMonitor: add 'osd crush set-all-straw-buckets-to-straw2'

This is a common and recommended step that users should take once they
have clients that support straw2 (hammer!).  Let's make it easy.

This may result in a small amount of data movement, but usually it is
very little.

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 928770513cbe3a244095e36dcb41c83c0724cfe9)

7 years agocrush/CrushWrapper: add bucket_set_alg()
Sage Weil [Sat, 21 Oct 2017 17:59:16 +0000 (12:59 -0500)]
crush/CrushWrapper: add bucket_set_alg()

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit edc411ee4cb7b88a970ccfc9d29b3a3ba15d066c)

7 years agobuffer: remove list _mempool member 18491/head
Sage Weil [Thu, 19 Oct 2017 21:19:35 +0000 (16:19 -0500)]
buffer: remove list _mempool member

This broke the C++ ABI by changing the list structure size.  Also, it's
not necessary as we can infer the mempool by looking at the other list
contents.  We don't (currently) have a need to map an empty list to a
particular mempool and have that state stick.

Fixes: http://tracker.ceph.com/issues/21573
Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 9b92d87d37e3892b0096cd728b46154aed1e2d86)

7 years agobuffer: allow mempool to be passed into raw* ctors and create methods
Sage Weil [Thu, 19 Oct 2017 21:09:10 +0000 (16:09 -0500)]
buffer: allow mempool to be passed into raw* ctors and create methods

This is more convenient, and also faster than initializing it in
buffer_anon and the immediately moving it elsewhere.

Drop the optionality of the alignment argument.

No users yet.

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 481277b8c92368b8cac8fdafe1848720ec1035c2)

7 years ago/etc/sysconfig/ceph: remove jemalloc option
Sage Weil [Mon, 23 Oct 2017 17:05:39 +0000 (12:05 -0500)]
/etc/sysconfig/ceph: remove jemalloc option

This breaks when used with rocksdb, which is now the default.

See http://tracker.ceph.com/issues/20557

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 19efccef248c2d66fe3d7fdac8f56d455f2c7b27)

7 years agoetc/default/ceph: remove jemalloc option
Sage Weil [Mon, 23 Oct 2017 16:16:26 +0000 (11:16 -0500)]
etc/default/ceph: remove jemalloc option

This breaks when used with rocksdb, which is now the default.

See http://tracker.ceph.com/issues/20557

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit fba2055b4e54128db1e9a83f27bb3526d65d4db7)

7 years agoqa/suites/rados/rest/mgr-restful: whitelist more health 18459/head
Sage Weil [Sat, 21 Oct 2017 04:14:36 +0000 (23:14 -0500)]
qa/suites/rados/rest/mgr-restful: whitelist more health

The test is fiddling with OSDs.

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit ddf3e9162da542af0c5f025957b8304e7359c924)

7 years agoMerge pull request #17773 from xiexingguo/wip-luminous-object-legacy-flag
Sage Weil [Sat, 21 Oct 2017 04:10:49 +0000 (23:10 -0500)]
Merge pull request #17773 from xiexingguo/wip-luminous-object-legacy-flag

luminous: osd/PrimaryLogPG: _delete_oid - fix incorrect 'legacy' flag

Reviewed-by: Kefu Chai <kchai@redhat.com>
7 years agoMerge pull request #18362 from kmroz/wip-21543-luminous
Sage Weil [Sat, 21 Oct 2017 04:10:06 +0000 (23:10 -0500)]
Merge pull request #18362 from kmroz/wip-21543-luminous

luminous: qa/tasks: prolong revive_osd() timeout to 6 min

7 years agoMerge pull request #18388 from tchaikov/wip-pr-18346-luminous
Sage Weil [Sat, 21 Oct 2017 04:09:52 +0000 (23:09 -0500)]
Merge pull request #18388 from tchaikov/wip-pr-18346-luminous

os/filestore: print out the error if do_read_entry() fails

Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>
7 years agoMerge pull request #18398 from smithfarm/wip-21783-luminous
Sage Weil [Sat, 21 Oct 2017 04:09:17 +0000 (23:09 -0500)]
Merge pull request #18398 from smithfarm/wip-21783-luminous

luminous: cli/crushtools/build.t sometimes fails in jenkins' make check run

Reviewed-by: Kefu Chai <kchai@redhat.com>
7 years agoMerge pull request #18401 from liewegas/wip-bluestore-label-luminous
Sage Weil [Sat, 21 Oct 2017 04:08:58 +0000 (23:08 -0500)]
Merge pull request #18401 from liewegas/wip-bluestore-label-luminous

luminous: os/bluestore: ceph-bluestore-tool repair updates and prime-osd-dir support

7 years agomessages/MOSDMap: do compat reencode of crush map, too
Sage Weil [Sat, 21 Oct 2017 03:32:33 +0000 (22:32 -0500)]
messages/MOSDMap: do compat reencode of crush map, too

If we are reencoding an incremental, and it embeds a crush map, we need
to reencode that in a compatible way too.  This is especially true now
because we have the compat crush weight-sets.  Otherwise, a client may
learn the crush map through an incremental but not understand choose_args,
and not see/understand the alternate weight set.  It will send requests
to the wrong OSDs where they will just get dropped.

Fixes: http://tracker.ceph.com/issues/21882
Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 9eaafc66278d59447b29ce6cc378d0cd3ad14ad4)

7 years agoMerge pull request #17692 from smithfarm/wip-21176-luminous
Sage Weil [Thu, 19 Oct 2017 21:43:39 +0000 (16:43 -0500)]
Merge pull request #17692 from smithfarm/wip-21176-luminous

build/ops: python-numpy-devel build dependency for SUSE

Reviewed-by: Abhishek Lekshmanan <abhishek.lekshmanan@gmail.com>
Reviewed-by: Amit Kumar <amitkuma@redhat.com>
7 years agoMerge pull request #17730 from xiexingguo/wip-pr-17371
Sage Weil [Thu, 19 Oct 2017 21:42:45 +0000 (16:42 -0500)]
Merge pull request #17730 from xiexingguo/wip-pr-17371

luminous: mon, osd: per pool space-full flag support

Reviewed-by: Sage Weil <sage@redhat.com>
7 years agoMerge pull request #17779 from smithfarm/wip-rh-74-luminous
Sage Weil [Thu, 19 Oct 2017 21:42:07 +0000 (16:42 -0500)]
Merge pull request #17779 from smithfarm/wip-rh-74-luminous

tests: CentOS 7.4 is now the latest

Reviewed-by: Ken Dreyer <kdreyer@redhat.com>
7 years agoMerge pull request #17863 from smithfarm/wip-21343-luminous
Sage Weil [Thu, 19 Oct 2017 21:41:34 +0000 (16:41 -0500)]
Merge pull request #17863 from smithfarm/wip-21343-luminous

luminous: mon: DNS SRV default service name not used anymore

7 years agoMerge pull request #17864 from smithfarm/wip-21438-luminous
Sage Weil [Thu, 19 Oct 2017 21:40:45 +0000 (16:40 -0500)]
Merge pull request #17864 from smithfarm/wip-21438-luminous

luminous: core: Daemons(OSD, Mon...) exit abnormally at injectargs command

Reviewed-by: Kefu Chai <kchai@redhat.com>
7 years agoMerge pull request #17866 from smithfarm/wip-21320-luminous
Sage Weil [Thu, 19 Oct 2017 21:40:18 +0000 (16:40 -0500)]
Merge pull request #17866 from smithfarm/wip-21320-luminous

luminous: mgr: Quieten scary RuntimeError from restful module on startup

7 years agoMerge pull request #17951 from smithfarm/wip-21548-luminous
Sage Weil [Thu, 19 Oct 2017 21:39:53 +0000 (16:39 -0500)]
Merge pull request #17951 from smithfarm/wip-21548-luminous

luminous: tests: ceph_manager: bad AssertionError: failed to recover before timeout expired

7 years agoMerge pull request #18342 from xiexingguo/wip-pr-18318
Sage Weil [Thu, 19 Oct 2017 21:39:32 +0000 (16:39 -0500)]
Merge pull request #18342 from xiexingguo/wip-pr-18318

luminous: bluestore: using bluestore_compression_min_blob_size instead of bluestore_compression_max_blob_size to config comp_min_blob_size

Reviewed-by: Sage Weil <sage@redhat.com>
7 years agoos/bluestore: fsync on _write_bdev_label 18401/head
Sage Weil [Mon, 16 Oct 2017 22:46:03 +0000 (17:46 -0500)]
os/bluestore: fsync on _write_bdev_label

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 7aca8af6625156eeaa936629f83ede33d3bbe866)

7 years agoceph-bluestore-tool: implement 'set-label-key' and 'rm-label-key'
Sage Weil [Mon, 16 Oct 2017 22:45:54 +0000 (17:45 -0500)]
ceph-bluestore-tool: implement 'set-label-key' and 'rm-label-key'

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 4b468c989f9e8609e7f640cfa4385c454ae6dc17)

7 years agoos/bluestore: make _write_bdev_label static and public
Sage Weil [Mon, 16 Oct 2017 22:42:29 +0000 (17:42 -0500)]
os/bluestore: make _write_bdev_label static and public

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 0b0550c967ae0d6ae49641c6859f0b63267e2cd9)

7 years agodoc/man/8/ceph-bluestore-tool: add man page
Sage Weil [Thu, 12 Oct 2017 16:40:11 +0000 (11:40 -0500)]
doc/man/8/ceph-bluestore-tool: add man page

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 7b91e50dbd289e65f10cb98c5eea9f3171c2d1f1)

7 years agocommon/options: document 'key', 'keyfile', 'keyring'
Sage Weil [Fri, 13 Oct 2017 14:20:53 +0000 (09:20 -0500)]
common/options: document 'key', 'keyfile', 'keyring'

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit cdee8b26b9d93189dcd2875038361fb59ed21ba2)

7 years agoceph-bluestore-tool: fix show-label to use dev name as key
Sage Weil [Thu, 12 Oct 2017 18:54:06 +0000 (13:54 -0500)]
ceph-bluestore-tool: fix show-label to use dev name as key

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 4785345ace7b675a0e1c34c457aad1103aab5a1d)

7 years agoceph-bluestore-tool: drop "action ..." print
Sage Weil [Thu, 12 Oct 2017 13:48:59 +0000 (08:48 -0500)]
ceph-bluestore-tool: drop "action ..." print

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 5e486c2bd3414f4ee6519574d4d4584618144867)

7 years agoceph-bluestore-tool: only infer devs if devs.empty()
Sage Weil [Thu, 12 Oct 2017 13:48:46 +0000 (08:48 -0500)]
ceph-bluestore-tool: only infer devs if devs.empty()

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 2d217367af0df5e18394b1c6179d92db3c0b4448)

7 years agoceph-bluestore-tool: implement prime-osd-dir
Sage Weil [Wed, 11 Oct 2017 22:36:40 +0000 (17:36 -0500)]
ceph-bluestore-tool: implement prime-osd-dir

This populates an osd dir based on the metadata in a bluestore
device label.

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 72ee7e29fb77f581a537b1ff2b274c9f3a7de53f)

7 years agoos/bluestore: keep meta items for extra bdev paths
Sage Weil [Wed, 11 Oct 2017 21:33:54 +0000 (16:33 -0500)]
os/bluestore: keep meta items for extra bdev paths

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 8d07fa5ad0772b4d979dc10695d230074d1f1e69)

7 years agovstart.sh: specify block files
Sage Weil [Wed, 11 Oct 2017 22:33:00 +0000 (17:33 -0500)]
vstart.sh: specify block files

This is mostly just to test ceph-bluestore-tool prime-osd-dir.

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit d4e0c46d89c2ebe6ae5f9af1c909a09daaea9687)

7 years agoosd: store osd cephx key in "osd_key" meta slot
Sage Weil [Wed, 11 Oct 2017 21:32:16 +0000 (16:32 -0500)]
osd: store osd cephx key in "osd_key" meta slot

Note that this *only* works with --mkkey (deprecated except for
vstart.sh) and --key <key>.  If you write out a full keyring before
running --mkfs it doesn't get populated.  Users should pass --key
instead of --keyring in that case.

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 65b707358e5442eab0fd6ba4d05d9f0a7adc969f)

7 years agoceph-osd: mkkey before mkfs
Sage Weil [Wed, 11 Oct 2017 21:17:09 +0000 (16:17 -0500)]
ceph-osd: mkkey before mkfs

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 75400ead3061ba1e2820878783fe7b1ddcd9a5fb)

7 years agoos/bluestore: store meta values in main device label
Sage Weil [Wed, 11 Oct 2017 20:53:56 +0000 (15:53 -0500)]
os/bluestore: store meta values in main device label

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 33b8e8e86f6bdbee120729561c6cf433d286e2f8)

7 years agoos/bluestore: fsck: fix error prefixes
Sage Weil [Sun, 10 Sep 2017 19:37:10 +0000 (15:37 -0400)]
os/bluestore: fsck: fix error prefixes

__func__ is now _fsck instead of fsck

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 7d381af7aed14e08823eca4d0b12de14456ef6a1)

7 years agoos/bluestore: allow repair invocation
Sage Weil [Thu, 7 Sep 2017 16:29:32 +0000 (12:29 -0400)]
os/bluestore: allow repair invocation

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 659fcd279ae82c90172fd6ca07a032d75fe50c96)

7 years agocrushtool: print error message to stderr not dout(1) 18398/head
Kefu Chai [Wed, 11 Oct 2017 09:45:19 +0000 (17:45 +0800)]
crushtool: print error message to stderr not dout(1)

in hope to fix the mysterious test failure where cli/crushtool/build.t
prints nothing when error message is expected.

Fixes: http://tracker.ceph.com/issues/21758
Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit dc78d40af77ff8d5e4f3d39497e6e3c98c309306)

7 years agocrush/CrushWrapper: fix output arg for find_{takes,roots}()
Sage Weil [Sun, 23 Jul 2017 03:51:47 +0000 (23:51 -0400)]
crush/CrushWrapper: fix output arg for find_{takes,roots}()

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 60b9cfafc3128cc0cb1f89137221fcc46fcd3802)

7 years agoos/filestore: print out the error if do_read_entry() fails 18388/head
Kefu Chai [Tue, 17 Oct 2017 09:08:59 +0000 (17:08 +0800)]
os/filestore: print out the error if do_read_entry() fails

Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit 42e85fe35d2fe8f3e99bd110021fd5157cf589d7)

7 years agoMerge pull request #18363 from ceph/wip-yuriw-21822-luminous
vasukulkarni [Tue, 17 Oct 2017 22:47:31 +0000 (15:47 -0700)]
Merge pull request #18363 from ceph/wip-yuriw-21822-luminous

tests - ceph-ansible vars additions

7 years agotests - ceph-ansible vars additions 18363/head
Yuri Weinstein [Tue, 17 Oct 2017 20:14:36 +0000 (13:14 -0700)]
tests - ceph-ansible vars additions
added symlinks for distros

Fixes http://tracker.ceph.com/issues/21822
Signed-off-by: Yuri Weinstein <yweinste@redhat.com>
7 years agoMerge pull request #17975 from dillaman/wip-systemd-rbd-mirror-luminous
Sage Weil [Tue, 17 Oct 2017 21:31:25 +0000 (16:31 -0500)]
Merge pull request #17975 from dillaman/wip-systemd-rbd-mirror-luminous

luminous: systemd: rbd-mirror does not start on reboot

Reviewed-by: Sébastien Han <seb@redhat.com>
Reviewed-by: Mykola Golub <mgolub@mirantis.com>
7 years agoMerge pull request #18234 from jdurgin/wip-filestore-rocksdb-luminous
Sage Weil [Tue, 17 Oct 2017 21:30:46 +0000 (16:30 -0500)]
Merge pull request #18234 from jdurgin/wip-filestore-rocksdb-luminous

filestore: set default readahead and compaction threads for rocksdb

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Amit Kumar <amitkuma@redhat.com>
7 years agoMerge pull request #18236 from tchaikov/wip-21609-luminous
Sage Weil [Tue, 17 Oct 2017 21:29:38 +0000 (16:29 -0500)]
Merge pull request #18236 from tchaikov/wip-21609-luminous

luminous: mon,osd: fix "pg ls {forced_backfill, backfilling}"

Reviewed-by: Sage Weil <sage@redhat.com>
7 years agoMerge pull request #18257 from smithfarm/wip-21699-luminous
Sage Weil [Tue, 17 Oct 2017 21:29:08 +0000 (16:29 -0500)]
Merge pull request #18257 from smithfarm/wip-21699-luminous

luminous: mgr status module uses base 10 units

Reviewed-by: John Spray <john.spray@redhat.com>
7 years agoqa/tasks: prolong revive_osd() timeout to 6 min 18362/head
Kefu Chai [Fri, 22 Sep 2017 02:58:40 +0000 (10:58 +0800)]
qa/tasks: prolong revive_osd() timeout to 6 min

bluestore_fsck_on_mount and bluestore_fsck_on_mount_deep are enabled by
default. and bluestore is used as the default store backend. it takes
longer to perform the deep fsck with verbose log. so prolong the
revive_osd()'s timeout from 150 sec to 360 sec.

Fixes: http://tracker.ceph.com/issues/21474
Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit 42be200c564184feff1056b7771ce3c1b2cc5527)

7 years agobluestore: using bluestore_compression_min_blob_size instead of bluestore_compression... 18342/head
linbing [Mon, 16 Oct 2017 01:57:23 +0000 (09:57 +0800)]
bluestore: using bluestore_compression_min_blob_size instead of bluestore_compression_max_blob_size to config comp_min_blob_size

Signed-off-by: linbing <linbing@t2cloud.net>
(cherry picked from commit da8dc4d4c60f5fd66f6eb2ba7257c52948f24681)

7 years agoMerge pull request #18339 from tchaikov/wip-gen-state-diagram-luminous
Kefu Chai [Tue, 17 Oct 2017 07:19:01 +0000 (15:19 +0800)]
Merge pull request #18339 from tchaikov/wip-gen-state-diagram-luminous

doc: build with multiple-line state transition function signature

Reviewed-By: Nathan Cutler <ncutler@suse.com>
7 years agoMerge pull request #17835 from rzarzynski/wip-rgw-per-storage-policy-stats-luminous
Kefu Chai [Tue, 17 Oct 2017 04:44:04 +0000 (12:44 +0800)]
Merge pull request #17835 from rzarzynski/wip-rgw-per-storage-policy-stats-luminous

luminous: rgw: add support for Swift's per storage policy statistics

Reviewed-by: Matt Benjamin <mbenjami@redhat.com>
Reviewed-by: Casey Bodley <cbodley@redhat.com>
7 years agodoc parse two lines for getting the context 18339/head
Alfredo Deza [Fri, 6 Oct 2017 12:57:01 +0000 (08:57 -0400)]
doc parse two lines for getting the context

So that signatures can get parsed when they are split like:

PG::RecoveryState::RepWaitBackfillReserved::react(
  const RemoteReservationCanceled &evt)

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 9e2a52ccec4a5c833dd861ae528d52efbc0f9e5f)

7 years agodoc raise exceptions with a base class
Alfredo Deza [Fri, 6 Oct 2017 12:03:36 +0000 (08:03 -0400)]
doc raise exceptions with a base class

Although valid in very old Python, it is no longer possible to raise
"bare" exceptions without a class.

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 347b7b1f193f97a3577a971e074a23366b0c8617)

7 years agoMerge pull request #17834 from rzarzynski/wip-rgw-21148-luminous
Matt Benjamin [Sun, 15 Oct 2017 18:30:57 +0000 (14:30 -0400)]
Merge pull request #17834 from rzarzynski/wip-rgw-21148-luminous

luminous: rgw: add support for Swift's reversed account listings

7 years agoMerge pull request #17811 from smithfarm/wip-21456-luminous
Matt Benjamin [Sun, 15 Oct 2017 16:48:02 +0000 (12:48 -0400)]
Merge pull request #17811 from smithfarm/wip-21456-luminous

luminous: rgw: wrong error message is returned when putting container with a name that is too long

7 years agoMerge pull request #17832 from rzarzynski/wip-rgw-18977-luminous
Marcus Watts [Sat, 14 Oct 2017 00:42:16 +0000 (20:42 -0400)]
Merge pull request #17832 from rzarzynski/wip-rgw-18977-luminous

luminous: rgw: list_objects() honors end_marker regardless of namespace.

7 years agoMerge pull request #18275 from ceph/wip-yuriw-21776-luminous
Josh Durgin [Fri, 13 Oct 2017 22:29:45 +0000 (15:29 -0700)]
Merge pull request #18275 from ceph/wip-yuriw-21776-luminous

tests - Removed `distors` and added `supported` to run on all support…

Reviewed-by: Vasu Kulkarni <vakulkar@redhat.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>
7 years agotests - Removed `distors` and added `supported` to run on all supported OSs 18275/head
Yuri Weinstein [Thu, 12 Oct 2017 20:36:20 +0000 (13:36 -0700)]
tests - Removed `distors` and added `supported` to run on all supported OSs
Added `openstack` fragment to run on vps
Forced/hard-coded `machine_type=vps`
Added `print`s
Added only centos and ubuntu latest to support systemd (in distros dir)
Added `ceph osd set-require-min-compat-client luminous`

Fixes http://tracker.ceph.com/issues/21776

Signed-off-by: Yuri Weinstein <yweinste@redhat.com>
7 years agoMerge pull request #18050 from xiexingguo/wip-pr-17610
Sage Weil [Fri, 13 Oct 2017 19:34:56 +0000 (14:34 -0500)]
Merge pull request #18050 from xiexingguo/wip-pr-17610

luminous: os/bluestore: set bitmap freelist resolution to min_alloc_size

Reviewed-by: Sage Weil <sage@redhat.com>
7 years agoMerge pull request #18247 from liewegas/wip-seesaw-warning-luminous
Sage Weil [Fri, 13 Oct 2017 19:34:30 +0000 (14:34 -0500)]
Merge pull request #18247 from liewegas/wip-seesaw-warning-luminous

qa/suites/rados/singleton/all/mon-seesaw: whitelist MON_DOWN

7 years agoMerge pull request #18253 from ceph/wip-yuriw-21660_2-luminous_1
Sage Weil [Thu, 12 Oct 2017 19:57:38 +0000 (14:57 -0500)]
Merge pull request #18253 from ceph/wip-yuriw-21660_2-luminous_1

tests - Added yaml fragmet to cover testing...

7 years agoMerge pull request #18254 from ceph/wip-yuriw-21660_3-luminous_1
Sage Weil [Thu, 12 Oct 2017 19:57:23 +0000 (14:57 -0500)]
Merge pull request #18254 from ceph/wip-yuriw-21660_3-luminous_1

tests - Added yaml fragmet to cover testing...

7 years agoqa: s/backfill/backfilling/ 18236/head
Kefu Chai [Wed, 11 Oct 2017 03:45:33 +0000 (11:45 +0800)]
qa: s/backfill/backfilling/

it's renamed "backfilling" in 4015343f .

Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit e21114274f6a2742748a5a9b965d415241c80f3c)

7 years agomgr/status: fix ceph fs status returns error 18257/head
Yanhu Cao [Wed, 11 Oct 2017 01:31:14 +0000 (09:31 +0800)]
mgr/status: fix ceph fs status returns error

Fixes: http://tracker.ceph.com/issues/21752
Signed-off-by: Yanhu Cao <gmayyyha@gmail.com>
(cherry picked from commit 4a90daa700142ed1f58f5f2a2cc85f3049bccd64)

7 years agomgr/status: format byte quantities in base 2 multiples
John Spray [Wed, 30 Aug 2017 19:32:11 +0000 (20:32 +0100)]
mgr/status: format byte quantities in base 2 multiples

Fixes: http://tracker.ceph.com/issues/21189
Signed-off-by: John Spray <john.spray@redhat.com>
(cherry picked from commit 2a5ae210928f8c9f1e3b72863c277ae5655bb771)

7 years agotests - Added yaml fragmet to cover testing... 18254/head
Yuri Weinstein [Wed, 11 Oct 2017 20:12:45 +0000 (13:12 -0700)]
tests - Added yaml fragmet to cover testing...
... for http://tracker.ceph.com/issues/21660#note-17

for luminous-x (see similar #18200, #18226)

Signed-off-by: Yuri Weinstein <yweinste@redhat.com>
(cherry picked from commit e7fdda31803cd37604a9296a876320bfcb849d34)
Signed-off-by: Yuri Weinstein <yweinste@redhat.com>
7 years agotests - Added yaml fragmet to cover testing... 18253/head
Yuri Weinstein [Tue, 10 Oct 2017 20:48:33 +0000 (13:48 -0700)]
tests - Added yaml fragmet to cover testing...
... for http://tracker.ceph.com/issues/21660#note-17
for jewel-x (see similar https://github.com/ceph/ceph/pull/18200)

Signed-off-by: Yuri Weinstein <yweinste@redhat.com>
(cherry picked from commit b552e636ddd8ce2cfb605b4c124440d8dd1e8e47)
Signed-off-by: Yuri Weinstein <yweinste@redhat.com>
7 years agoMerge pull request #18025 from liewegas/wip-recovery-preemption-luminous
Sage Weil [Wed, 11 Oct 2017 17:20:04 +0000 (12:20 -0500)]
Merge pull request #18025 from liewegas/wip-recovery-preemption-luminous

luminous: osd: allow recovery preemption

Reviewed-by: Greg Farnum <gfarnum@redhat.com>
7 years agoMerge pull request #18154 from ktdreyer/luminous-gitignore-debian-patches
Sage Weil [Wed, 11 Oct 2017 17:19:45 +0000 (12:19 -0500)]
Merge pull request #18154 from ktdreyer/luminous-gitignore-debian-patches

luminous: .gitignore: allow debian .patch files

7 years agoMerge pull request #18189 from tchaikov/wip-retry-oserror-luminous
Sage Weil [Wed, 11 Oct 2017 17:19:08 +0000 (12:19 -0500)]
Merge pull request #18189 from tchaikov/wip-retry-oserror-luminous

luminous: ceph-disk: retry on OSError

7 years agoMerge pull request #18227 from liewegas/wip-bluestore-mempool-luminous
Sage Weil [Wed, 11 Oct 2017 17:18:27 +0000 (12:18 -0500)]
Merge pull request #18227 from liewegas/wip-bluestore-mempool-luminous

luminous: os/bluestore: several mempool accounting fixes

Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>
7 years agoqa/suites/rados/singleton/all/mon-seesaw: whitelist MON_DOWN 18247/head
Sage Weil [Wed, 11 Oct 2017 13:24:50 +0000 (08:24 -0500)]
qa/suites/rados/singleton/all/mon-seesaw: whitelist MON_DOWN

Mgr can get marked down when mon weirdness is happening.

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 0a886bc9f3e9805de04c8c60bd3d78b937737b96)

7 years agomon/PGMap: let pg_string_state() return boost::optional<>
Kefu Chai [Tue, 10 Oct 2017 12:32:08 +0000 (20:32 +0800)]
mon/PGMap: let pg_string_state() return boost::optional<>

better to be specific, so we don't run into the same problem even we
are using the MSB of uint64_t for a pg state bit in future. we could,
however use uint64_t(-1) to indicate the pg_string_state()'s failure to
parse the state string, because pg_string_state() only translate a
single state a time. but it's always better to be explicit than
implicit.

Fixes: http://tracker.ceph.com/issues/21609
Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit 260f87a9f1c4e32ff3a596ea341ad07a901f44a9)

Conflicts:
src/mon/PGMap.cc
src/osd/osd_types.cc
src/osd/osd_types.h: trivial resolution

7 years agocommon/options: enable multiple rocksdb compaction threads for filestore 18234/head
Josh Durgin [Tue, 10 Oct 2017 23:54:31 +0000 (19:54 -0400)]
common/options: enable multiple rocksdb compaction threads for filestore

One of the major benefits of rocksdb over leveldb is multithreaded
compaction. The default of 1 thread does not provide much benefit, and
is insufficient for heavy rgw workloads.

For high-write and delete omap workloads I've seen up to 8 compaction
threads be used.  There's little overhead to having a higher max than
are needed, so set the default to 8.

Signed-off-by: Josh Durgin <jdurgin@redhat.com>
(cherry picked from commit 023fa810aa6b3af305e9027e3f717e54d1bb2712)

Conflicts:
    src/common/options.cc (trivial)

7 years agocommon/options.cc: Set Filestore rocksdb compaction readahead option.
Mark Nelson [Thu, 21 Sep 2017 22:21:16 +0000 (17:21 -0500)]
common/options.cc: Set Filestore rocksdb compaction readahead option.

fixes: http://tracker.ceph.com/issues/21505

Signed-off-by: Mark Nelson <mnelson@redhat.com>
(cherry picked from commit 66567e573836c570040c8d8148c34b0cf9dc9ce2)

7 years agoos/bluestore: move several buffer{ptr,list}s into cache_other mempool 18227/head
Sage Weil [Thu, 28 Sep 2017 12:02:50 +0000 (08:02 -0400)]
os/bluestore: move several buffer{ptr,list}s into cache_other mempool

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 80c60fcde22cf1269ada45d0914543b8f7d49b3e)

7 years agoos/bluestore: put new attrs in correct mempool too
Sage Weil [Thu, 28 Sep 2017 12:14:37 +0000 (08:14 -0400)]
os/bluestore: put new attrs in correct mempool too

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit e7762ea1477e861ebc738aa0588a01c0da883d62)

7 years agoos/bluestore: put attrs in mempool
Sage Weil [Fri, 22 Sep 2017 12:09:37 +0000 (08:09 -0400)]
os/bluestore: put attrs in mempool

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit ef6524436fe095f94fce421ce26ed4c64e1424fe)

7 years agobuffer: add ptr::[try_]reassign_to_mempool
Sage Weil [Fri, 22 Sep 2017 12:08:52 +0000 (08:08 -0400)]
buffer: add ptr::[try_]reassign_to_mempool

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 937379d57a0f8097ae713adf714ba0414d36dffe)

7 years agoMerge pull request #18200 from ceph/wip-yuriw-21660-luminous
Josh Durgin [Tue, 10 Oct 2017 20:17:46 +0000 (13:17 -0700)]
Merge pull request #18200 from ceph/wip-yuriw-21660-luminous

tests - Added yaml fragmet to cover testing

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Jason Dillaman <dillaman@redhat.com>
7 years agotests - Added yaml fragmet to cover testing on standanone client.4 18200/head
Yuri Weinstein [Mon, 9 Oct 2017 19:51:59 +0000 (12:51 -0700)]
tests - Added yaml fragmet to cover testing on standanone client.4
for http://tracker.ceph.com/issues/21660#note-17

Signed-off-by: Yuri Weinstein <yweinste@redhat.com>
7 years agoceph-disk: retry on OSError 18189/head
Kefu Chai [Sat, 7 Oct 2017 14:15:11 +0000 (22:15 +0800)]
ceph-disk: retry on OSError

we are likely to
1) create partition, for instance, sdc1
2) partprobe sdc
3) udevadm settle
4) check the device by its path: /dev/sdc1

but there is chance that the uevent sent from kernel fails to reach udev
before we call "udevadm", hence "/dev/sdc1" does not exist even after
"udevadm settle" returns. so we retry in case of OSError here.

Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit 4f82dfb9e761a09484e6ba3bd027da535162783e)

7 years agoceph-disk: factor out the retry logic into a decorator
Kefu Chai [Sat, 7 Oct 2017 13:13:47 +0000 (21:13 +0800)]
ceph-disk: factor out the retry logic into a decorator

Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit 02a8620de7f200736900aafc944b2c1ac47d9910)

7 years agoMerge pull request #18167 from tchaikov/wip-luminous-sphinx
Alfredo Deza [Sun, 8 Oct 2017 16:05:25 +0000 (12:05 -0400)]
Merge pull request #18167 from tchaikov/wip-luminous-sphinx

luminous: admin: bump sphinx to 1.6

Reviewed-by: Alfredo Deza <adeza@redhat.com>
7 years agodoc: switch to upstream sphinx-ditaa 18167/head
Kefu Chai [Mon, 11 Sep 2017 13:54:33 +0000 (21:54 +0800)]
doc: switch to upstream sphinx-ditaa

it works with setuptools and is now compatible with py3

the py3 branch is created to track the upstream's master branch

Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit fda079d5ed4c8a80c677f714a680233a568d512e)

7 years agoadmin: bump sphinx to 1.6
Alfredo Deza [Tue, 15 Aug 2017 13:12:27 +0000 (09:12 -0400)]
admin: bump sphinx to 1.6

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 88a1114cb3f90e52ba718dc832617fa004d63b3c)

7 years agoMerge pull request #17914 from liewegas/wip-21498-luminous
Alfredo Deza [Fri, 6 Oct 2017 21:42:30 +0000 (17:42 -0400)]
Merge pull request #17914 from liewegas/wip-21498-luminous

ceph-disk: fix '--runtime' omission for ceph-osd service

Reviewed-by: Alfredo Deza <adeza@redhat.com>