]> git.apps.os.sepia.ceph.com Git - ceph.git/log
ceph.git
7 years agocommon/common_init: disable ms subsystem log gathering for clients 18529/head
Jason Dillaman [Fri, 20 Oct 2017 02:13:36 +0000 (22:13 -0400)]
common/common_init: disable ms subsystem log gathering for clients

The log gathering causes large performance degradation to clients
with high message throughputs. This is hopefully a short-term
workaround until per-message logging can be replaced with an
efficient data recording system for post-incident analysis
use-cases.

Fixes: http://tracker.ceph.com/issues/21860
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit a3a40413f7908b08c40dec4020034cca4a0c4798)

7 years agoMerge pull request #18491 from liewegas/wip-21573-luminous
Kefu Chai [Tue, 24 Oct 2017 03:51:19 +0000 (11:51 +0800)]
Merge pull request #18491 from liewegas/wip-21573-luminous

luminous: buffer: fix ABI breakage by removing list _mempool member

Reviewed-by: Kefu Chai <kchai@redhat.com>
7 years agobuffer: remove list _mempool member 18491/head
Sage Weil [Thu, 19 Oct 2017 21:19:35 +0000 (16:19 -0500)]
buffer: remove list _mempool member

This broke the C++ ABI by changing the list structure size.  Also, it's
not necessary as we can infer the mempool by looking at the other list
contents.  We don't (currently) have a need to map an empty list to a
particular mempool and have that state stick.

Fixes: http://tracker.ceph.com/issues/21573
Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 9b92d87d37e3892b0096cd728b46154aed1e2d86)

7 years agobuffer: allow mempool to be passed into raw* ctors and create methods
Sage Weil [Thu, 19 Oct 2017 21:09:10 +0000 (16:09 -0500)]
buffer: allow mempool to be passed into raw* ctors and create methods

This is more convenient, and also faster than initializing it in
buffer_anon and the immediately moving it elsewhere.

Drop the optionality of the alignment argument.

No users yet.

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 481277b8c92368b8cac8fdafe1848720ec1035c2)

7 years ago/etc/sysconfig/ceph: remove jemalloc option
Sage Weil [Mon, 23 Oct 2017 17:05:39 +0000 (12:05 -0500)]
/etc/sysconfig/ceph: remove jemalloc option

This breaks when used with rocksdb, which is now the default.

See http://tracker.ceph.com/issues/20557

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 19efccef248c2d66fe3d7fdac8f56d455f2c7b27)

7 years agoetc/default/ceph: remove jemalloc option
Sage Weil [Mon, 23 Oct 2017 16:16:26 +0000 (11:16 -0500)]
etc/default/ceph: remove jemalloc option

This breaks when used with rocksdb, which is now the default.

See http://tracker.ceph.com/issues/20557

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit fba2055b4e54128db1e9a83f27bb3526d65d4db7)

7 years agoMerge pull request #17773 from xiexingguo/wip-luminous-object-legacy-flag
Sage Weil [Sat, 21 Oct 2017 04:10:49 +0000 (23:10 -0500)]
Merge pull request #17773 from xiexingguo/wip-luminous-object-legacy-flag

luminous: osd/PrimaryLogPG: _delete_oid - fix incorrect 'legacy' flag

Reviewed-by: Kefu Chai <kchai@redhat.com>
7 years agoMerge pull request #18362 from kmroz/wip-21543-luminous
Sage Weil [Sat, 21 Oct 2017 04:10:06 +0000 (23:10 -0500)]
Merge pull request #18362 from kmroz/wip-21543-luminous

luminous: qa/tasks: prolong revive_osd() timeout to 6 min

7 years agoMerge pull request #18388 from tchaikov/wip-pr-18346-luminous
Sage Weil [Sat, 21 Oct 2017 04:09:52 +0000 (23:09 -0500)]
Merge pull request #18388 from tchaikov/wip-pr-18346-luminous

os/filestore: print out the error if do_read_entry() fails

Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>
7 years agoMerge pull request #18398 from smithfarm/wip-21783-luminous
Sage Weil [Sat, 21 Oct 2017 04:09:17 +0000 (23:09 -0500)]
Merge pull request #18398 from smithfarm/wip-21783-luminous

luminous: cli/crushtools/build.t sometimes fails in jenkins' make check run

Reviewed-by: Kefu Chai <kchai@redhat.com>
7 years agoMerge pull request #18401 from liewegas/wip-bluestore-label-luminous
Sage Weil [Sat, 21 Oct 2017 04:08:58 +0000 (23:08 -0500)]
Merge pull request #18401 from liewegas/wip-bluestore-label-luminous

luminous: os/bluestore: ceph-bluestore-tool repair updates and prime-osd-dir support

7 years agoMerge pull request #17692 from smithfarm/wip-21176-luminous
Sage Weil [Thu, 19 Oct 2017 21:43:39 +0000 (16:43 -0500)]
Merge pull request #17692 from smithfarm/wip-21176-luminous

build/ops: python-numpy-devel build dependency for SUSE

Reviewed-by: Abhishek Lekshmanan <abhishek.lekshmanan@gmail.com>
Reviewed-by: Amit Kumar <amitkuma@redhat.com>
7 years agoMerge pull request #17730 from xiexingguo/wip-pr-17371
Sage Weil [Thu, 19 Oct 2017 21:42:45 +0000 (16:42 -0500)]
Merge pull request #17730 from xiexingguo/wip-pr-17371

luminous: mon, osd: per pool space-full flag support

Reviewed-by: Sage Weil <sage@redhat.com>
7 years agoMerge pull request #17779 from smithfarm/wip-rh-74-luminous
Sage Weil [Thu, 19 Oct 2017 21:42:07 +0000 (16:42 -0500)]
Merge pull request #17779 from smithfarm/wip-rh-74-luminous

tests: CentOS 7.4 is now the latest

Reviewed-by: Ken Dreyer <kdreyer@redhat.com>
7 years agoMerge pull request #17863 from smithfarm/wip-21343-luminous
Sage Weil [Thu, 19 Oct 2017 21:41:34 +0000 (16:41 -0500)]
Merge pull request #17863 from smithfarm/wip-21343-luminous

luminous: mon: DNS SRV default service name not used anymore

7 years agoMerge pull request #17864 from smithfarm/wip-21438-luminous
Sage Weil [Thu, 19 Oct 2017 21:40:45 +0000 (16:40 -0500)]
Merge pull request #17864 from smithfarm/wip-21438-luminous

luminous: core: Daemons(OSD, Mon...) exit abnormally at injectargs command

Reviewed-by: Kefu Chai <kchai@redhat.com>
7 years agoMerge pull request #17866 from smithfarm/wip-21320-luminous
Sage Weil [Thu, 19 Oct 2017 21:40:18 +0000 (16:40 -0500)]
Merge pull request #17866 from smithfarm/wip-21320-luminous

luminous: mgr: Quieten scary RuntimeError from restful module on startup

7 years agoMerge pull request #17951 from smithfarm/wip-21548-luminous
Sage Weil [Thu, 19 Oct 2017 21:39:53 +0000 (16:39 -0500)]
Merge pull request #17951 from smithfarm/wip-21548-luminous

luminous: tests: ceph_manager: bad AssertionError: failed to recover before timeout expired

7 years agoMerge pull request #18342 from xiexingguo/wip-pr-18318
Sage Weil [Thu, 19 Oct 2017 21:39:32 +0000 (16:39 -0500)]
Merge pull request #18342 from xiexingguo/wip-pr-18318

luminous: bluestore: using bluestore_compression_min_blob_size instead of bluestore_compression_max_blob_size to config comp_min_blob_size

Reviewed-by: Sage Weil <sage@redhat.com>
7 years agoos/bluestore: fsync on _write_bdev_label 18401/head
Sage Weil [Mon, 16 Oct 2017 22:46:03 +0000 (17:46 -0500)]
os/bluestore: fsync on _write_bdev_label

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 7aca8af6625156eeaa936629f83ede33d3bbe866)

7 years agoceph-bluestore-tool: implement 'set-label-key' and 'rm-label-key'
Sage Weil [Mon, 16 Oct 2017 22:45:54 +0000 (17:45 -0500)]
ceph-bluestore-tool: implement 'set-label-key' and 'rm-label-key'

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 4b468c989f9e8609e7f640cfa4385c454ae6dc17)

7 years agoos/bluestore: make _write_bdev_label static and public
Sage Weil [Mon, 16 Oct 2017 22:42:29 +0000 (17:42 -0500)]
os/bluestore: make _write_bdev_label static and public

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 0b0550c967ae0d6ae49641c6859f0b63267e2cd9)

7 years agodoc/man/8/ceph-bluestore-tool: add man page
Sage Weil [Thu, 12 Oct 2017 16:40:11 +0000 (11:40 -0500)]
doc/man/8/ceph-bluestore-tool: add man page

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 7b91e50dbd289e65f10cb98c5eea9f3171c2d1f1)

7 years agocommon/options: document 'key', 'keyfile', 'keyring'
Sage Weil [Fri, 13 Oct 2017 14:20:53 +0000 (09:20 -0500)]
common/options: document 'key', 'keyfile', 'keyring'

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit cdee8b26b9d93189dcd2875038361fb59ed21ba2)

7 years agoceph-bluestore-tool: fix show-label to use dev name as key
Sage Weil [Thu, 12 Oct 2017 18:54:06 +0000 (13:54 -0500)]
ceph-bluestore-tool: fix show-label to use dev name as key

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 4785345ace7b675a0e1c34c457aad1103aab5a1d)

7 years agoceph-bluestore-tool: drop "action ..." print
Sage Weil [Thu, 12 Oct 2017 13:48:59 +0000 (08:48 -0500)]
ceph-bluestore-tool: drop "action ..." print

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 5e486c2bd3414f4ee6519574d4d4584618144867)

7 years agoceph-bluestore-tool: only infer devs if devs.empty()
Sage Weil [Thu, 12 Oct 2017 13:48:46 +0000 (08:48 -0500)]
ceph-bluestore-tool: only infer devs if devs.empty()

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 2d217367af0df5e18394b1c6179d92db3c0b4448)

7 years agoceph-bluestore-tool: implement prime-osd-dir
Sage Weil [Wed, 11 Oct 2017 22:36:40 +0000 (17:36 -0500)]
ceph-bluestore-tool: implement prime-osd-dir

This populates an osd dir based on the metadata in a bluestore
device label.

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 72ee7e29fb77f581a537b1ff2b274c9f3a7de53f)

7 years agoos/bluestore: keep meta items for extra bdev paths
Sage Weil [Wed, 11 Oct 2017 21:33:54 +0000 (16:33 -0500)]
os/bluestore: keep meta items for extra bdev paths

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 8d07fa5ad0772b4d979dc10695d230074d1f1e69)

7 years agovstart.sh: specify block files
Sage Weil [Wed, 11 Oct 2017 22:33:00 +0000 (17:33 -0500)]
vstart.sh: specify block files

This is mostly just to test ceph-bluestore-tool prime-osd-dir.

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit d4e0c46d89c2ebe6ae5f9af1c909a09daaea9687)

7 years agoosd: store osd cephx key in "osd_key" meta slot
Sage Weil [Wed, 11 Oct 2017 21:32:16 +0000 (16:32 -0500)]
osd: store osd cephx key in "osd_key" meta slot

Note that this *only* works with --mkkey (deprecated except for
vstart.sh) and --key <key>.  If you write out a full keyring before
running --mkfs it doesn't get populated.  Users should pass --key
instead of --keyring in that case.

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 65b707358e5442eab0fd6ba4d05d9f0a7adc969f)

7 years agoceph-osd: mkkey before mkfs
Sage Weil [Wed, 11 Oct 2017 21:17:09 +0000 (16:17 -0500)]
ceph-osd: mkkey before mkfs

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 75400ead3061ba1e2820878783fe7b1ddcd9a5fb)

7 years agoos/bluestore: store meta values in main device label
Sage Weil [Wed, 11 Oct 2017 20:53:56 +0000 (15:53 -0500)]
os/bluestore: store meta values in main device label

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 33b8e8e86f6bdbee120729561c6cf433d286e2f8)

7 years agoos/bluestore: fsck: fix error prefixes
Sage Weil [Sun, 10 Sep 2017 19:37:10 +0000 (15:37 -0400)]
os/bluestore: fsck: fix error prefixes

__func__ is now _fsck instead of fsck

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 7d381af7aed14e08823eca4d0b12de14456ef6a1)

7 years agoos/bluestore: allow repair invocation
Sage Weil [Thu, 7 Sep 2017 16:29:32 +0000 (12:29 -0400)]
os/bluestore: allow repair invocation

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 659fcd279ae82c90172fd6ca07a032d75fe50c96)

7 years agocrushtool: print error message to stderr not dout(1) 18398/head
Kefu Chai [Wed, 11 Oct 2017 09:45:19 +0000 (17:45 +0800)]
crushtool: print error message to stderr not dout(1)

in hope to fix the mysterious test failure where cli/crushtool/build.t
prints nothing when error message is expected.

Fixes: http://tracker.ceph.com/issues/21758
Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit dc78d40af77ff8d5e4f3d39497e6e3c98c309306)

7 years agocrush/CrushWrapper: fix output arg for find_{takes,roots}()
Sage Weil [Sun, 23 Jul 2017 03:51:47 +0000 (23:51 -0400)]
crush/CrushWrapper: fix output arg for find_{takes,roots}()

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 60b9cfafc3128cc0cb1f89137221fcc46fcd3802)

7 years agoos/filestore: print out the error if do_read_entry() fails 18388/head
Kefu Chai [Tue, 17 Oct 2017 09:08:59 +0000 (17:08 +0800)]
os/filestore: print out the error if do_read_entry() fails

Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit 42e85fe35d2fe8f3e99bd110021fd5157cf589d7)

7 years agoMerge pull request #18363 from ceph/wip-yuriw-21822-luminous
vasukulkarni [Tue, 17 Oct 2017 22:47:31 +0000 (15:47 -0700)]
Merge pull request #18363 from ceph/wip-yuriw-21822-luminous

tests - ceph-ansible vars additions

7 years agotests - ceph-ansible vars additions 18363/head
Yuri Weinstein [Tue, 17 Oct 2017 20:14:36 +0000 (13:14 -0700)]
tests - ceph-ansible vars additions
added symlinks for distros

Fixes http://tracker.ceph.com/issues/21822
Signed-off-by: Yuri Weinstein <yweinste@redhat.com>
7 years agoMerge pull request #17975 from dillaman/wip-systemd-rbd-mirror-luminous
Sage Weil [Tue, 17 Oct 2017 21:31:25 +0000 (16:31 -0500)]
Merge pull request #17975 from dillaman/wip-systemd-rbd-mirror-luminous

luminous: systemd: rbd-mirror does not start on reboot

Reviewed-by: Sébastien Han <seb@redhat.com>
Reviewed-by: Mykola Golub <mgolub@mirantis.com>
7 years agoMerge pull request #18234 from jdurgin/wip-filestore-rocksdb-luminous
Sage Weil [Tue, 17 Oct 2017 21:30:46 +0000 (16:30 -0500)]
Merge pull request #18234 from jdurgin/wip-filestore-rocksdb-luminous

filestore: set default readahead and compaction threads for rocksdb

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Amit Kumar <amitkuma@redhat.com>
7 years agoMerge pull request #18236 from tchaikov/wip-21609-luminous
Sage Weil [Tue, 17 Oct 2017 21:29:38 +0000 (16:29 -0500)]
Merge pull request #18236 from tchaikov/wip-21609-luminous

luminous: mon,osd: fix "pg ls {forced_backfill, backfilling}"

Reviewed-by: Sage Weil <sage@redhat.com>
7 years agoMerge pull request #18257 from smithfarm/wip-21699-luminous
Sage Weil [Tue, 17 Oct 2017 21:29:08 +0000 (16:29 -0500)]
Merge pull request #18257 from smithfarm/wip-21699-luminous

luminous: mgr status module uses base 10 units

Reviewed-by: John Spray <john.spray@redhat.com>
7 years agoqa/tasks: prolong revive_osd() timeout to 6 min 18362/head
Kefu Chai [Fri, 22 Sep 2017 02:58:40 +0000 (10:58 +0800)]
qa/tasks: prolong revive_osd() timeout to 6 min

bluestore_fsck_on_mount and bluestore_fsck_on_mount_deep are enabled by
default. and bluestore is used as the default store backend. it takes
longer to perform the deep fsck with verbose log. so prolong the
revive_osd()'s timeout from 150 sec to 360 sec.

Fixes: http://tracker.ceph.com/issues/21474
Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit 42be200c564184feff1056b7771ce3c1b2cc5527)

7 years agobluestore: using bluestore_compression_min_blob_size instead of bluestore_compression... 18342/head
linbing [Mon, 16 Oct 2017 01:57:23 +0000 (09:57 +0800)]
bluestore: using bluestore_compression_min_blob_size instead of bluestore_compression_max_blob_size to config comp_min_blob_size

Signed-off-by: linbing <linbing@t2cloud.net>
(cherry picked from commit da8dc4d4c60f5fd66f6eb2ba7257c52948f24681)

7 years agoMerge pull request #18339 from tchaikov/wip-gen-state-diagram-luminous
Kefu Chai [Tue, 17 Oct 2017 07:19:01 +0000 (15:19 +0800)]
Merge pull request #18339 from tchaikov/wip-gen-state-diagram-luminous

doc: build with multiple-line state transition function signature

Reviewed-By: Nathan Cutler <ncutler@suse.com>
7 years agoMerge pull request #17835 from rzarzynski/wip-rgw-per-storage-policy-stats-luminous
Kefu Chai [Tue, 17 Oct 2017 04:44:04 +0000 (12:44 +0800)]
Merge pull request #17835 from rzarzynski/wip-rgw-per-storage-policy-stats-luminous

luminous: rgw: add support for Swift's per storage policy statistics

Reviewed-by: Matt Benjamin <mbenjami@redhat.com>
Reviewed-by: Casey Bodley <cbodley@redhat.com>
7 years agodoc parse two lines for getting the context 18339/head
Alfredo Deza [Fri, 6 Oct 2017 12:57:01 +0000 (08:57 -0400)]
doc parse two lines for getting the context

So that signatures can get parsed when they are split like:

PG::RecoveryState::RepWaitBackfillReserved::react(
  const RemoteReservationCanceled &evt)

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 9e2a52ccec4a5c833dd861ae528d52efbc0f9e5f)

7 years agodoc raise exceptions with a base class
Alfredo Deza [Fri, 6 Oct 2017 12:03:36 +0000 (08:03 -0400)]
doc raise exceptions with a base class

Although valid in very old Python, it is no longer possible to raise
"bare" exceptions without a class.

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 347b7b1f193f97a3577a971e074a23366b0c8617)

7 years agoMerge pull request #17834 from rzarzynski/wip-rgw-21148-luminous
Matt Benjamin [Sun, 15 Oct 2017 18:30:57 +0000 (14:30 -0400)]
Merge pull request #17834 from rzarzynski/wip-rgw-21148-luminous

luminous: rgw: add support for Swift's reversed account listings

7 years agoMerge pull request #17811 from smithfarm/wip-21456-luminous
Matt Benjamin [Sun, 15 Oct 2017 16:48:02 +0000 (12:48 -0400)]
Merge pull request #17811 from smithfarm/wip-21456-luminous

luminous: rgw: wrong error message is returned when putting container with a name that is too long

7 years agoMerge pull request #17832 from rzarzynski/wip-rgw-18977-luminous
Marcus Watts [Sat, 14 Oct 2017 00:42:16 +0000 (20:42 -0400)]
Merge pull request #17832 from rzarzynski/wip-rgw-18977-luminous

luminous: rgw: list_objects() honors end_marker regardless of namespace.

7 years agoMerge pull request #18275 from ceph/wip-yuriw-21776-luminous
Josh Durgin [Fri, 13 Oct 2017 22:29:45 +0000 (15:29 -0700)]
Merge pull request #18275 from ceph/wip-yuriw-21776-luminous

tests - Removed `distors` and added `supported` to run on all support…

Reviewed-by: Vasu Kulkarni <vakulkar@redhat.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>
7 years agotests - Removed `distors` and added `supported` to run on all supported OSs 18275/head
Yuri Weinstein [Thu, 12 Oct 2017 20:36:20 +0000 (13:36 -0700)]
tests - Removed `distors` and added `supported` to run on all supported OSs
Added `openstack` fragment to run on vps
Forced/hard-coded `machine_type=vps`
Added `print`s
Added only centos and ubuntu latest to support systemd (in distros dir)
Added `ceph osd set-require-min-compat-client luminous`

Fixes http://tracker.ceph.com/issues/21776

Signed-off-by: Yuri Weinstein <yweinste@redhat.com>
7 years agoMerge pull request #18050 from xiexingguo/wip-pr-17610
Sage Weil [Fri, 13 Oct 2017 19:34:56 +0000 (14:34 -0500)]
Merge pull request #18050 from xiexingguo/wip-pr-17610

luminous: os/bluestore: set bitmap freelist resolution to min_alloc_size

Reviewed-by: Sage Weil <sage@redhat.com>
7 years agoMerge pull request #18247 from liewegas/wip-seesaw-warning-luminous
Sage Weil [Fri, 13 Oct 2017 19:34:30 +0000 (14:34 -0500)]
Merge pull request #18247 from liewegas/wip-seesaw-warning-luminous

qa/suites/rados/singleton/all/mon-seesaw: whitelist MON_DOWN

7 years agoMerge pull request #18253 from ceph/wip-yuriw-21660_2-luminous_1
Sage Weil [Thu, 12 Oct 2017 19:57:38 +0000 (14:57 -0500)]
Merge pull request #18253 from ceph/wip-yuriw-21660_2-luminous_1

tests - Added yaml fragmet to cover testing...

7 years agoMerge pull request #18254 from ceph/wip-yuriw-21660_3-luminous_1
Sage Weil [Thu, 12 Oct 2017 19:57:23 +0000 (14:57 -0500)]
Merge pull request #18254 from ceph/wip-yuriw-21660_3-luminous_1

tests - Added yaml fragmet to cover testing...

7 years agoqa: s/backfill/backfilling/ 18236/head
Kefu Chai [Wed, 11 Oct 2017 03:45:33 +0000 (11:45 +0800)]
qa: s/backfill/backfilling/

it's renamed "backfilling" in 4015343f .

Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit e21114274f6a2742748a5a9b965d415241c80f3c)

7 years agomgr/status: fix ceph fs status returns error 18257/head
Yanhu Cao [Wed, 11 Oct 2017 01:31:14 +0000 (09:31 +0800)]
mgr/status: fix ceph fs status returns error

Fixes: http://tracker.ceph.com/issues/21752
Signed-off-by: Yanhu Cao <gmayyyha@gmail.com>
(cherry picked from commit 4a90daa700142ed1f58f5f2a2cc85f3049bccd64)

7 years agomgr/status: format byte quantities in base 2 multiples
John Spray [Wed, 30 Aug 2017 19:32:11 +0000 (20:32 +0100)]
mgr/status: format byte quantities in base 2 multiples

Fixes: http://tracker.ceph.com/issues/21189
Signed-off-by: John Spray <john.spray@redhat.com>
(cherry picked from commit 2a5ae210928f8c9f1e3b72863c277ae5655bb771)

7 years agotests - Added yaml fragmet to cover testing... 18254/head
Yuri Weinstein [Wed, 11 Oct 2017 20:12:45 +0000 (13:12 -0700)]
tests - Added yaml fragmet to cover testing...
... for http://tracker.ceph.com/issues/21660#note-17

for luminous-x (see similar #18200, #18226)

Signed-off-by: Yuri Weinstein <yweinste@redhat.com>
(cherry picked from commit e7fdda31803cd37604a9296a876320bfcb849d34)
Signed-off-by: Yuri Weinstein <yweinste@redhat.com>
7 years agotests - Added yaml fragmet to cover testing... 18253/head
Yuri Weinstein [Tue, 10 Oct 2017 20:48:33 +0000 (13:48 -0700)]
tests - Added yaml fragmet to cover testing...
... for http://tracker.ceph.com/issues/21660#note-17
for jewel-x (see similar https://github.com/ceph/ceph/pull/18200)

Signed-off-by: Yuri Weinstein <yweinste@redhat.com>
(cherry picked from commit b552e636ddd8ce2cfb605b4c124440d8dd1e8e47)
Signed-off-by: Yuri Weinstein <yweinste@redhat.com>
7 years agoMerge pull request #18025 from liewegas/wip-recovery-preemption-luminous
Sage Weil [Wed, 11 Oct 2017 17:20:04 +0000 (12:20 -0500)]
Merge pull request #18025 from liewegas/wip-recovery-preemption-luminous

luminous: osd: allow recovery preemption

Reviewed-by: Greg Farnum <gfarnum@redhat.com>
7 years agoMerge pull request #18154 from ktdreyer/luminous-gitignore-debian-patches
Sage Weil [Wed, 11 Oct 2017 17:19:45 +0000 (12:19 -0500)]
Merge pull request #18154 from ktdreyer/luminous-gitignore-debian-patches

luminous: .gitignore: allow debian .patch files

7 years agoMerge pull request #18189 from tchaikov/wip-retry-oserror-luminous
Sage Weil [Wed, 11 Oct 2017 17:19:08 +0000 (12:19 -0500)]
Merge pull request #18189 from tchaikov/wip-retry-oserror-luminous

luminous: ceph-disk: retry on OSError

7 years agoMerge pull request #18227 from liewegas/wip-bluestore-mempool-luminous
Sage Weil [Wed, 11 Oct 2017 17:18:27 +0000 (12:18 -0500)]
Merge pull request #18227 from liewegas/wip-bluestore-mempool-luminous

luminous: os/bluestore: several mempool accounting fixes

Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>
7 years agoqa/suites/rados/singleton/all/mon-seesaw: whitelist MON_DOWN 18247/head
Sage Weil [Wed, 11 Oct 2017 13:24:50 +0000 (08:24 -0500)]
qa/suites/rados/singleton/all/mon-seesaw: whitelist MON_DOWN

Mgr can get marked down when mon weirdness is happening.

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 0a886bc9f3e9805de04c8c60bd3d78b937737b96)

7 years agomon/PGMap: let pg_string_state() return boost::optional<>
Kefu Chai [Tue, 10 Oct 2017 12:32:08 +0000 (20:32 +0800)]
mon/PGMap: let pg_string_state() return boost::optional<>

better to be specific, so we don't run into the same problem even we
are using the MSB of uint64_t for a pg state bit in future. we could,
however use uint64_t(-1) to indicate the pg_string_state()'s failure to
parse the state string, because pg_string_state() only translate a
single state a time. but it's always better to be explicit than
implicit.

Fixes: http://tracker.ceph.com/issues/21609
Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit 260f87a9f1c4e32ff3a596ea341ad07a901f44a9)

Conflicts:
src/mon/PGMap.cc
src/osd/osd_types.cc
src/osd/osd_types.h: trivial resolution

7 years agocommon/options: enable multiple rocksdb compaction threads for filestore 18234/head
Josh Durgin [Tue, 10 Oct 2017 23:54:31 +0000 (19:54 -0400)]
common/options: enable multiple rocksdb compaction threads for filestore

One of the major benefits of rocksdb over leveldb is multithreaded
compaction. The default of 1 thread does not provide much benefit, and
is insufficient for heavy rgw workloads.

For high-write and delete omap workloads I've seen up to 8 compaction
threads be used.  There's little overhead to having a higher max than
are needed, so set the default to 8.

Signed-off-by: Josh Durgin <jdurgin@redhat.com>
(cherry picked from commit 023fa810aa6b3af305e9027e3f717e54d1bb2712)

Conflicts:
    src/common/options.cc (trivial)

7 years agocommon/options.cc: Set Filestore rocksdb compaction readahead option.
Mark Nelson [Thu, 21 Sep 2017 22:21:16 +0000 (17:21 -0500)]
common/options.cc: Set Filestore rocksdb compaction readahead option.

fixes: http://tracker.ceph.com/issues/21505

Signed-off-by: Mark Nelson <mnelson@redhat.com>
(cherry picked from commit 66567e573836c570040c8d8148c34b0cf9dc9ce2)

7 years agoos/bluestore: move several buffer{ptr,list}s into cache_other mempool 18227/head
Sage Weil [Thu, 28 Sep 2017 12:02:50 +0000 (08:02 -0400)]
os/bluestore: move several buffer{ptr,list}s into cache_other mempool

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 80c60fcde22cf1269ada45d0914543b8f7d49b3e)

7 years agoos/bluestore: put new attrs in correct mempool too
Sage Weil [Thu, 28 Sep 2017 12:14:37 +0000 (08:14 -0400)]
os/bluestore: put new attrs in correct mempool too

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit e7762ea1477e861ebc738aa0588a01c0da883d62)

7 years agoos/bluestore: put attrs in mempool
Sage Weil [Fri, 22 Sep 2017 12:09:37 +0000 (08:09 -0400)]
os/bluestore: put attrs in mempool

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit ef6524436fe095f94fce421ce26ed4c64e1424fe)

7 years agobuffer: add ptr::[try_]reassign_to_mempool
Sage Weil [Fri, 22 Sep 2017 12:08:52 +0000 (08:08 -0400)]
buffer: add ptr::[try_]reassign_to_mempool

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 937379d57a0f8097ae713adf714ba0414d36dffe)

7 years agoMerge pull request #18200 from ceph/wip-yuriw-21660-luminous
Josh Durgin [Tue, 10 Oct 2017 20:17:46 +0000 (13:17 -0700)]
Merge pull request #18200 from ceph/wip-yuriw-21660-luminous

tests - Added yaml fragmet to cover testing

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Jason Dillaman <dillaman@redhat.com>
7 years agotests - Added yaml fragmet to cover testing on standanone client.4 18200/head
Yuri Weinstein [Mon, 9 Oct 2017 19:51:59 +0000 (12:51 -0700)]
tests - Added yaml fragmet to cover testing on standanone client.4
for http://tracker.ceph.com/issues/21660#note-17

Signed-off-by: Yuri Weinstein <yweinste@redhat.com>
7 years agoceph-disk: retry on OSError 18189/head
Kefu Chai [Sat, 7 Oct 2017 14:15:11 +0000 (22:15 +0800)]
ceph-disk: retry on OSError

we are likely to
1) create partition, for instance, sdc1
2) partprobe sdc
3) udevadm settle
4) check the device by its path: /dev/sdc1

but there is chance that the uevent sent from kernel fails to reach udev
before we call "udevadm", hence "/dev/sdc1" does not exist even after
"udevadm settle" returns. so we retry in case of OSError here.

Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit 4f82dfb9e761a09484e6ba3bd027da535162783e)

7 years agoceph-disk: factor out the retry logic into a decorator
Kefu Chai [Sat, 7 Oct 2017 13:13:47 +0000 (21:13 +0800)]
ceph-disk: factor out the retry logic into a decorator

Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit 02a8620de7f200736900aafc944b2c1ac47d9910)

7 years agoMerge pull request #18167 from tchaikov/wip-luminous-sphinx
Alfredo Deza [Sun, 8 Oct 2017 16:05:25 +0000 (12:05 -0400)]
Merge pull request #18167 from tchaikov/wip-luminous-sphinx

luminous: admin: bump sphinx to 1.6

Reviewed-by: Alfredo Deza <adeza@redhat.com>
7 years agodoc: switch to upstream sphinx-ditaa 18167/head
Kefu Chai [Mon, 11 Sep 2017 13:54:33 +0000 (21:54 +0800)]
doc: switch to upstream sphinx-ditaa

it works with setuptools and is now compatible with py3

the py3 branch is created to track the upstream's master branch

Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit fda079d5ed4c8a80c677f714a680233a568d512e)

7 years agoadmin: bump sphinx to 1.6
Alfredo Deza [Tue, 15 Aug 2017 13:12:27 +0000 (09:12 -0400)]
admin: bump sphinx to 1.6

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 88a1114cb3f90e52ba718dc832617fa004d63b3c)

7 years agoMerge pull request #17914 from liewegas/wip-21498-luminous
Alfredo Deza [Fri, 6 Oct 2017 21:42:30 +0000 (17:42 -0400)]
Merge pull request #17914 from liewegas/wip-21498-luminous

ceph-disk: fix '--runtime' omission for ceph-osd service

Reviewed-by: Alfredo Deza <adeza@redhat.com>
7 years agoMerge pull request #18135 from liewegas/wip-vta-luminous
Sage Weil [Fri, 6 Oct 2017 20:35:06 +0000 (15:35 -0500)]
Merge pull request #18135 from liewegas/wip-vta-luminous

luminous: cmake: disable VTA on options.cc

7 years ago.gitignore: allow debian .patch files 18154/head
Ken Dreyer [Thu, 7 Sep 2017 17:07:59 +0000 (11:07 -0600)]
.gitignore: allow debian .patch files

The Ubuntu packaging layout with git-buildpackage assumes a
"debian/patches/" directory with several .patch files in it.

When upstream's .gitignore tells Git to ignore .patch files, we have to
edit that line out downstream. When we forget to do that downstream, it
can lead to missing patches and broken downstream builds.

Allow patches in the /debian/patches directory so it's easier to
maintain an Ubuntu package based on upstream's Git repo.

Signed-off-by: Ken Dreyer <kdreyer@redhat.com>
(cherry picked from commit c734b0c0296152721b658af7b699a64b3a49d251)

7 years agoMerge pull request #18140 from liewegas/wip-21660-luminous
Sage Weil [Fri, 6 Oct 2017 12:38:53 +0000 (07:38 -0500)]
Merge pull request #18140 from liewegas/wip-21660-luminous

luminous: src/messages/MOSDMap: reencode OSDMap for older clients

7 years agoMerge pull request #18037 from ajarr/wip-21602-luminous
Sage Weil [Fri, 6 Oct 2017 03:12:22 +0000 (22:12 -0500)]
Merge pull request #18037 from ajarr/wip-21602-luminous

luminous: ceph_volume_client: add get, put, and delete object interfaces

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
7 years agoMerge pull request #18103 from liewegas/wip-21259-luminous
Sage Weil [Fri, 6 Oct 2017 02:03:42 +0000 (21:03 -0500)]
Merge pull request #18103 from liewegas/wip-21259-luminous

luminous: osd: dump bluestore debug on shutdown if debug option is set

7 years agoMerge pull request #18112 from liewegas/wip-localpool-min-size-luminous
Sage Weil [Fri, 6 Oct 2017 02:03:28 +0000 (21:03 -0500)]
Merge pull request #18112 from liewegas/wip-localpool-min-size-luminous

luminous: mgr/localpool: fix min_size, 3x default, crush rule

Reviewed-by: Alexander Marangone <amarango@redhat.com>
7 years agoMerge pull request #18126 from smithfarm/wip-sigpipe-porting-luminous
Sage Weil [Fri, 6 Oct 2017 02:02:53 +0000 (21:02 -0500)]
Merge pull request #18126 from smithfarm/wip-sigpipe-porting-luminous

luminous: msg: reimplement sigpipe blocking

Reviewed-by: Greg Farnum <gfarnum@redhat.com>
7 years agoMerge pull request #18127 from liewegas/wip-21470-luminous
Sage Weil [Fri, 6 Oct 2017 02:02:09 +0000 (21:02 -0500)]
Merge pull request #18127 from liewegas/wip-21470-luminous

luminous: os/bluestore: fix another aio stall/deadlock

7 years agoMerge pull request #18128 from liewegas/wip-bluefs-allocation-luminous
Sage Weil [Fri, 6 Oct 2017 02:01:53 +0000 (21:01 -0500)]
Merge pull request #18128 from liewegas/wip-bluefs-allocation-luminous

luminous: os/bluestore: make bluefs behave better near enospc

7 years agoMerge pull request #18131 from liewegas/wip-ifname-luminous
Sage Weil [Fri, 6 Oct 2017 02:01:27 +0000 (21:01 -0500)]
Merge pull request #18131 from liewegas/wip-ifname-luminous

luminous: common/pick_address: add {public,cluster}_network_interface option

7 years agoMerge pull request #18132 from gregsfortytwo/wip-20416-bitwise-assert-luminous
Sage Weil [Fri, 6 Oct 2017 02:01:06 +0000 (21:01 -0500)]
Merge pull request #18132 from gregsfortytwo/wip-20416-bitwise-assert-luminous

osd: make the PG's SORTBITWISE assert a more generous shutdown

Reviewed-by: Sage Weil <sage@redhat.com>
7 years agosrc/messages/MOSDMap: reencode OSDMap for older clients 18140/head
Sage Weil [Thu, 5 Oct 2017 20:26:16 +0000 (15:26 -0500)]
src/messages/MOSDMap: reencode OSDMap for older clients

We explicitly select which missing bits trigger a reencode.  We
already had jewel and earlier covered, but kraken includes all of
the previously mentioned bits but not SERVER_LUMINOUS.  This
prevents kraken clients from decoding luminous maps.

Fixes: http://tracker.ceph.com/issues/21660
Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit d2664aa34dc4c13da64d1ad187019737080069d7)

7 years agoos/bluestore: add bluestore_bluefs_min_free 18128/head
Sage Weil [Tue, 3 Oct 2017 16:18:17 +0000 (11:18 -0500)]
os/bluestore: add bluestore_bluefs_min_free

We need at least ~1GB free so we can write out new SSTs (which are 256MB
each).

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 65be614f2bfd65a1f13205a3075c87acc72c4836)

(avoided 1_G syntax that luminous doesn't support)

7 years agocmake: disable VTA on options.cc 18135/head
Kefu Chai [Thu, 31 Aug 2017 10:15:28 +0000 (18:15 +0800)]
cmake: disable VTA on options.cc

to silence following warning and to avoid compiling this file twice:

ceph/src/common/options.cc: In function ‘std::vector<Option> get_global_options()’:
ceph/src/common/options.cc:151:21: note: variable tracking
size limit exceeded with -fvar-tracking-assignments, retrying without
 std::vector<Option> get_global_options() {
                     ^~~~~~~~~~~~~~~~~~

Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit 4bb56402a6d6f642d54f329aa1e809cbe044e0c6)

7 years agoosd/PG: separate event for RemoteReservationCanceled 18025/head
Sage Weil [Wed, 4 Oct 2017 20:28:26 +0000 (15:28 -0500)]
osd/PG: separate event for RemoteReservationCanceled

Right now we transparently map a RemoteReservationRejected into a
*Canceled event because this what peers send over the wire.  Even
once new peers start sending and explicit CANCEL, old peers will
still do so, so we'll maintain this mapping for a while.

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 84d71e6a10b02591b5d5e126b346771871eb1575)

7 years agoosd/PG: separate verb 'Reject' from passive 'rejected'
Sage Weil [Wed, 4 Oct 2017 19:55:15 +0000 (14:55 -0500)]
osd/PG: separate verb 'Reject' from passive 'rejected'

This reduces pg->reject_reservation() callsites from 2 to 1 and
makes the state transitions a bit more explicit.

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit bf7f101a787652644c27aae4e752fd21f265e866)