]> git.apps.os.sepia.ceph.com Git - ceph.git/log
ceph.git
7 years agoos/bluestore: put StupidAllocator btree into bluestore_alloc mempool 16906/head
Sage Weil [Thu, 10 Aug 2017 15:58:40 +0000 (11:58 -0400)]
os/bluestore: put StupidAllocator btree into bluestore_alloc mempool

Signed-off-by: Sage Weil <sage@redhat.com>
7 years agoos/bluestore: default 'bluefs_allocator = stupid'
Sage Weil [Thu, 10 Aug 2017 15:21:32 +0000 (11:21 -0400)]
os/bluestore: default 'bluefs_allocator = stupid'

Signed-off-by: Sage Weil <sage@redhat.com>
7 years agoqa/objectstore/bluestore-bitmap: test bluestore bitmap allocator too
Sage Weil [Tue, 8 Aug 2017 16:15:16 +0000 (12:15 -0400)]
qa/objectstore/bluestore-bitmap: test bluestore bitmap allocator too

Signed-off-by: Sage Weil <sage@redhat.com>
7 years agoos/bluestore: default to bluestore_allocator = stupid
Sage Weil [Tue, 8 Aug 2017 16:14:07 +0000 (12:14 -0400)]
os/bluestore: default to bluestore_allocator = stupid

Performance looks better, and code is simpler.

Also fix config option annotations.

Signed-off-by: Sage Weil <sage@redhat.com>
7 years agoMerge PR #16876 into master
Patrick Donnelly [Mon, 7 Aug 2017 19:52:49 +0000 (12:52 -0700)]
Merge PR #16876 into master

* refs/remotes/upstream/pull/16876/head:
doc: add new fs auth command to release notes

Reviewed-by: Sage Weil <sage@redhat.com>
7 years agoMerge pull request #16871 from liewegas/wip-20920
Sage Weil [Mon, 7 Aug 2017 19:48:58 +0000 (14:48 -0500)]
Merge pull request #16871 from liewegas/wip-20920

mon: fix commands advertised during mon cluster upgrade

Reviewed-by: Greg Farnum <gfarnum@redhat.com>
7 years agodoc: add new fs auth command to release notes 16876/head
Patrick Donnelly [Mon, 7 Aug 2017 18:04:23 +0000 (11:04 -0700)]
doc: add new fs auth command to release notes

Introduced by #16761.

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
7 years agoqa/suites/upgrade/jewel-x/parallel: fix POOL_APP_NOT_ENABLED disable 16871/head
Sage Weil [Mon, 7 Aug 2017 13:49:55 +0000 (09:49 -0400)]
qa/suites/upgrade/jewel-x/parallel: fix POOL_APP_NOT_ENABLED disable

This code runs on the mgr.

Signed-off-by: Sage Weil <sage@redhat.com>
7 years agomon/MonCommands: mark 'pg force_create_pg' deprecated
Sage Weil [Sat, 5 Aug 2017 19:33:37 +0000 (15:33 -0400)]
mon/MonCommands: mark 'pg force_create_pg' deprecated

It's deprecated.

Also, this avoids a dup when we have an upgrading mon cluster
and it's also in PGMonitorCommands.

Signed-off-by: Sage Weil <sage@redhat.com>
7 years agomon: include PGMonitor commands with mixed-version mons
Sage Weil [Sat, 5 Aug 2017 19:30:15 +0000 (15:30 -0400)]
mon: include PGMonitor commands with mixed-version mons

While we have a mixed version cluster, we have to advertise our
PGMonitor commands to our peons or else commands like 'pg dump'
won't work.

Once the mon feature flag is set, we can drop that because each
mon will include the mgr commands (either those stored in paxos
or the statically compiled ones until that point).

Signed-off-by: Sage Weil <sage@redhat.com>
7 years agomon: use vector<MonCommand> throughput for commands
Sage Weil [Sat, 5 Aug 2017 19:08:26 +0000 (15:08 -0400)]
mon: use vector<MonCommand> throughput for commands

The old code was pretty messy. This is standardizes on std::vector
throughout.  We also drop the win_election command args because
when we win an election we always set the leader commands to our
commands, and we can do that inside win_command() without passing
them in from here.

Signed-off-by: Sage Weil <sage@redhat.com>
7 years agoMerge pull request #16642 from dillaman/wip-rbd-mirror-image-ids
Jason Dillaman [Mon, 7 Aug 2017 18:33:50 +0000 (14:33 -0400)]
Merge pull request #16642 from dillaman/wip-rbd-mirror-image-ids

rbd-mirror: simplify notifications for image assignment

Reviewed-by: Mykola Golub <mgolub@mirantis.com>
Reviewed-by: Venky Shankar <vshankar@redhat.com>
7 years agoMerge pull request #16799 from liewegas/wip-smithi-have-4
Sage Weil [Mon, 7 Aug 2017 17:36:42 +0000 (12:36 -0500)]
Merge pull request #16799 from liewegas/wip-smithi-have-4

qa/clusters/fixed-[23]: 4 osds per node, not 3

Vasu Kulkarni <vasu@redhat.com>

7 years agoqa/clusters/fixed-[23]: 4 osds per node, not 3 16799/head
Sage Weil [Thu, 3 Aug 2017 18:28:56 +0000 (14:28 -0400)]
qa/clusters/fixed-[23]: 4 osds per node, not 3

Smithi have 4 nvme partitions available for use.

Signed-off-by: Sage Weil <sage@redhat.com>
7 years agoMerge pull request #16873 from liewegas/wip-4-nodes
Sage Weil [Mon, 7 Aug 2017 16:27:40 +0000 (11:27 -0500)]
Merge pull request #16873 from liewegas/wip-4-nodes

qa/suites: change fixed-2.yaml users to get 4 openstack disks

Reviewed-by: Zack Cerza <zcerza@redhat.com>
7 years agoMerge pull request #16864 from smithfarm/wip-big-openstack
Sage Weil [Mon, 7 Aug 2017 16:02:59 +0000 (11:02 -0500)]
Merge pull request #16864 from smithfarm/wip-big-openstack

qa: big: add openstack.yaml

7 years agoqa/suites: change fixed-2.yaml users to get 4 openstack disks 16873/head
Sage Weil [Mon, 7 Aug 2017 15:56:20 +0000 (11:56 -0400)]
qa/suites: change fixed-2.yaml users to get 4 openstack disks

Follow-up for 4203c4f88785d8149235dd34d37f87e471084d71

Signed-off-by: Sage Weil <sage@redhat.com>
7 years agoMerge pull request #16842 from liewegas/wip-more-ec-map-discon
Sage Weil [Mon, 7 Aug 2017 15:48:56 +0000 (10:48 -0500)]
Merge pull request #16842 from liewegas/wip-more-ec-map-discon

qa/suites/rados/thrash: fix thrashing with ec vs map discon

7 years agoMerge pull request #16693 from alex-mikheev/wip_rdma_rxpool_1
Haomai Wang [Mon, 7 Aug 2017 15:45:48 +0000 (23:45 +0800)]
Merge pull request #16693 from alex-mikheev/wip_rdma_rxpool_1

msg/async/rdma: improves RX buffer management

Reviewed-by: Haomai Wang <haomai@xsky.com>
7 years agoMerge pull request #16869 from liewegas/wip-crush-ids
Sage Weil [Mon, 7 Aug 2017 15:35:47 +0000 (10:35 -0500)]
Merge pull request #16869 from liewegas/wip-crush-ids

crush/CrushWrapper: fix out of bounds access

Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>
7 years agoMerge pull request #16845 from amitkumar50/cov-librados-aio
Jos Collin [Mon, 7 Aug 2017 14:51:13 +0000 (14:51 +0000)]
Merge pull request #16845 from amitkumar50/cov-librados-aio

test/librados: Initialize member variables in aio.cc

Reviewed-by: Jos Collin <jcollin@redhat.com>
7 years agotest/librados: Initializing left members from aio.cc 16845/head
amitkuma [Sun, 6 Aug 2017 16:43:36 +0000 (22:13 +0530)]
test/librados: Initializing left members from aio.cc

Fixes the coverity Issue:

** 1322815 Uninitialized pointer field
CID 1322815 (#1 of 1): Uninitialized pointer field (UNINIT_CTOR)
2. uninit_member: Non-static class member m_sem is not initialized
in this constructor nor in any functions that it calls.

** 1322816 Uninitialized pointer field
CID 1322816 (#1 of 1): Uninitialized pointer field (UNINIT_CTOR)
2. uninit_member: Non-static class member m_sem is not initialized
in this constructor nor in any functions that it calls.

** 1322817 Uninitialized pointer field
CID 1322817 (#1 of 1): Uninitialized pointer field (UNINIT_CTOR)
2. uninit_member: Non-static class member m_sem is not initialized
in this constructor nor in any functions that it calls.

Signed-off-by: Amit Kumar amitkuma@redhat.com
7 years agorbd-mirror: clean up logging during test shut down 16642/head
Jason Dillaman [Mon, 31 Jul 2017 12:37:38 +0000 (08:37 -0400)]
rbd-mirror: clean up logging during test shut down

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
7 years agorbd-mirror: avoid silently failing to register image replayer admin sockets
Jason Dillaman [Thu, 27 Jul 2017 19:35:15 +0000 (15:35 -0400)]
rbd-mirror: avoid silently failing to register image replayer admin sockets

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
7 years agorbd-mirror: restore deletion propagation and image replayer cleanup
Jason Dillaman [Fri, 21 Jul 2017 15:18:46 +0000 (11:18 -0400)]
rbd-mirror: restore deletion propagation and image replayer cleanup

The previous intermediate commits removed handling for deletion
propagation and image replayer cleanup since this logic has been
moved from instance to image replayer. Note that eventually the
policy's release notification will be responsible for the cleanup
of image replayers.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
7 years agorbd-mirror: ensure threads are mockable for testing
Jason Dillaman [Fri, 21 Jul 2017 19:40:03 +0000 (15:40 -0400)]
rbd-mirror: ensure threads are mockable for testing

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
7 years agorbd-mirror: moved wait for deletion logic within image replayer
Jason Dillaman [Fri, 21 Jul 2017 20:12:13 +0000 (16:12 -0400)]
rbd-mirror: moved wait for deletion logic within image replayer

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
7 years agorbd-mirror: simplified acquire/release notification messages
Jason Dillaman [Thu, 20 Jul 2017 20:30:42 +0000 (16:30 -0400)]
rbd-mirror: simplified acquire/release notification messages

The other instances should already know about the possible peers, so
we can co-opt these messages for use by the policy mapper.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
7 years agorbd-mirror: co-opt image acquire/release notifications
Jason Dillaman [Thu, 20 Jul 2017 20:39:51 +0000 (16:39 -0400)]
rbd-mirror: co-opt image acquire/release notifications

This notifications will now be used by the policy when a (re)map is
required.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
7 years agorbd-mirror: new 'peer image removed' notification to instance watchers
Jason Dillaman [Thu, 20 Jul 2017 20:10:39 +0000 (16:10 -0400)]
rbd-mirror: new 'peer image removed' notification to instance watchers

Note: this commit intentionally breaks backwards compatibility with
Luminous RC instance watcher notifications.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
7 years agorbd-mirror: simplify resync handling within image replayer
Jason Dillaman [Wed, 19 Jul 2017 20:18:23 +0000 (16:18 -0400)]
rbd-mirror: simplify resync handling within image replayer

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
7 years agorbd-mirror: pre-register image id before creating image
Jason Dillaman [Wed, 19 Jul 2017 20:13:23 +0000 (16:13 -0400)]
rbd-mirror: pre-register image id before creating image

This fixes a potential race condition that could occur previously
if rbd-mirror daemon failed between creating an image and recording
the image id to the remote journal.

Fixes: http://tracker.ceph.com/issues/15764
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
7 years agorbd-mirror: clean up bootstrap recovery from missing local image
Jason Dillaman [Wed, 19 Jul 2017 18:24:36 +0000 (14:24 -0400)]
rbd-mirror: clean up bootstrap recovery from missing local image

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
7 years agorbd-mirror: image replayer should query remote mirror uuid / image id
Jason Dillaman [Wed, 19 Jul 2017 16:20:34 +0000 (12:20 -0400)]
rbd-mirror: image replayer should query remote mirror uuid / image id

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
7 years agorbd-mirror: helper state machine for querying remote peer uuid / image id
Jason Dillaman [Fri, 21 Jul 2017 00:17:23 +0000 (20:17 -0400)]
rbd-mirror: helper state machine for querying remote peer uuid / image id

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
7 years agorbd-mirror: helper state machine for retrieving image id from global id
Jason Dillaman [Wed, 19 Jul 2017 14:14:14 +0000 (10:14 -0400)]
rbd-mirror: helper state machine for retrieving image id from global id

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
7 years agoMerge pull request #16393 from PCzhangPC/fix3
Jason Dillaman [Mon, 7 Aug 2017 14:03:20 +0000 (10:03 -0400)]
Merge pull request #16393 from PCzhangPC/fix3

rbd: 'image-meta remove' for missing key does not return error

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
7 years agocrush/CrushWrapper: fix out of bounds access 16869/head
Sage Weil [Mon, 7 Aug 2017 13:36:54 +0000 (09:36 -0400)]
crush/CrushWrapper: fix out of bounds access

Fixes: http://tracker.ceph.com/issues/20926
Signed-off-by: Sage Weil <sage@redhat.com>
7 years agoMerge pull request #16851 from xiexingguo/wip-doc-update-class-cli
Sage Weil [Mon, 7 Aug 2017 13:26:35 +0000 (08:26 -0500)]
Merge pull request #16851 from xiexingguo/wip-doc-update-class-cli

doc/release-notes: update device class cli

7 years agoMerge pull request #16551 from fangyuxiangGL/data-notifier-interval
Yuri Weinstein [Mon, 7 Aug 2017 12:48:54 +0000 (05:48 -0700)]
Merge pull request #16551 from fangyuxiangGL/data-notifier-interval

rgw: add a separate configuration for data notify interval

Reviewed-by: Casey Bodley <cbodley@redhat.com>
7 years agoMerge pull request #16726 from theanalyst/rgw-crypt-debug-log
Yuri Weinstein [Mon, 7 Aug 2017 12:48:14 +0000 (05:48 -0700)]
Merge pull request #16726 from theanalyst/rgw-crypt-debug-log

rgw_crypt: log error messages during failures

Reviewed-by: Jos Collin <jcollin@redhat.com>
Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: Adam Kupczyk <akupczyk@mirantis.com>
7 years agoMerge pull request #16770 from tianshan/fix_create_bucket
Yuri Weinstein [Mon, 7 Aug 2017 12:45:52 +0000 (05:45 -0700)]
Merge pull request #16770 from tianshan/fix_create_bucket

rgw: should not restrict location_constraint same when user not provide

Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: Jos Collin <jcollin@redhat.com>
7 years agoqa: big: add openstack.yaml 16864/head
Nathan Cutler [Mon, 7 Aug 2017 10:07:36 +0000 (12:07 +0200)]
qa: big: add openstack.yaml

Signed-off-by: Nathan Cutler <ncutler@suse.com>
7 years agoMerge pull request #16862 from scienceluo/wip-luo-check-fix-branch
Jos Collin [Mon, 7 Aug 2017 09:30:48 +0000 (09:30 +0000)]
Merge pull request #16862 from scienceluo/wip-luo-check-fix-branch

common/cmdparse.cc: remove unused variable 'argnum' in dump_cmd_to_json()

Reviewed-by: Jos Collin <jcollin@redhat.com>
7 years agoCommon: remove variable 'argnum' that is modified but its new value is never used. 16862/head
Luo Kexue [Mon, 7 Aug 2017 08:21:16 +0000 (16:21 +0800)]
Common: remove variable 'argnum' that is modified but its new value is never used.

Signed-off-by: Luo Kexue <luo.kexue@zte.com.cn>
7 years agodoc/release-notes: update device class cli 16851/head
xie xingguo [Mon, 7 Aug 2017 02:04:35 +0000 (10:04 +0800)]
doc/release-notes: update device class cli

These commands are nearly stable and are unlikely to change for now...

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
7 years agoMerge pull request #16847 from amitkumar50/cov-1054870
Jos Collin [Mon, 7 Aug 2017 04:01:36 +0000 (04:01 +0000)]
Merge pull request #16847 from amitkumar50/cov-1054870

rgw: Initialize of member variable admin_specified in RGWUserAdminOpState

Reviewed-by: Jos Collin <jcollin@redhat.com>
7 years agoMerge pull request #16822 from liewegas/wip-20849
Sage Weil [Mon, 7 Aug 2017 02:43:22 +0000 (21:43 -0500)]
Merge pull request #16822 from liewegas/wip-20849

os/bluestore: _do_remove: dirty shard individually as each blob is unshared

Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>
7 years agoMerge pull request #16844 from xiexingguo/wip-bluestore-core
Sage Weil [Mon, 7 Aug 2017 01:17:57 +0000 (20:17 -0500)]
Merge pull request #16844 from xiexingguo/wip-bluestore-core

os/bluestore: default journal media to store media if bluefs is disabled

Reviewed-by: Sage Weil <sage@redhat.com>
7 years agoMerge pull request #14053 from jdurgin/wip-19191
Sage Weil [Sun, 6 Aug 2017 19:08:34 +0000 (14:08 -0500)]
Merge pull request #14053 from jdurgin/wip-19191

osd/PG: reset the missing set when restarting backfill

Reviewed-by: David Zafman <dzafman@redhat.com>
Reviewed-by: Sage Weil <sage@redhat.com>
7 years agoMerge pull request #16837 from xiexingguo/wip-still-more-class-fixes
Sage Weil [Sun, 6 Aug 2017 19:07:33 +0000 (14:07 -0500)]
Merge pull request #16837 from xiexingguo/wip-still-more-class-fixes

crush: more class fixes

Reviewed-by: Sage Weil <sage@redhat.com>
7 years agoMerge pull request #16838 from xiexingguo/wip-fix-purge
Sage Weil [Sun, 6 Aug 2017 19:07:15 +0000 (14:07 -0500)]
Merge pull request #16838 from xiexingguo/wip-fix-purge

mon/OSDMonitor: sanity check osd before performing 'osd purge'

Reviewed-by: Sage Weil <sage@redhat.com>
7 years agoMerge pull request #16832 from liewegas/wip-20915
Sage Weil [Sun, 6 Aug 2017 18:41:21 +0000 (13:41 -0500)]
Merge pull request #16832 from liewegas/wip-20915

rpm,deb: fix ceph-volume

Reviewed-by: Nathan Cutler <ncutler@suse.com>
7 years agorgw: Initilzation of admin_specified 16847/head
amitkuma [Sun, 6 Aug 2017 17:25:07 +0000 (22:55 +0530)]
rgw: Initilzation of admin_specified

Fixes the coverity issue:

** 1054870 Uninitialized scalar field
CID 1054870 (#1 of 1): Uninitialized scalar field (UNINIT_CTOR)
2. uninit_member: Non-static class member admin_specified is not
initialized in this constructor nor in any functions that it calls.

Signed-off-by: Amit Kumar amitkuma@redhat.com
7 years agoos/bluestore: _do_remove: dirty shard individually as each blob is unshared 16822/head
Sage Weil [Sun, 6 Aug 2017 14:31:49 +0000 (10:31 -0400)]
os/bluestore: _do_remove: dirty shard individually as each blob is unshared

Two problems with old code:

1- dirty_shard range is inclusive, so we might dirty the shard after b_end
2- we might unshare blobs in two shards with an unloaded shard in between,
which would mean dirtying a shard that isn't loaded.

Fix by ensuring the shard for each unshared blob is dirty individually.

Fixes: http://tracker.ceph.com/issues/20849
Signed-off-by: Sage Weil <sage@redhat.com>
7 years agoMerge pull request #16849 from liewegas/wip-jewel-x
Sage Weil [Sun, 6 Aug 2017 14:15:35 +0000 (09:15 -0500)]
Merge pull request #16849 from liewegas/wip-jewel-x

qa/suites/upgarde/jewel-x/parallel: more whitelisting

7 years agoqa/suites/upgarde/jewel-x/parallel: more whitelisting 16849/head
Sage Weil [Sun, 6 Aug 2017 14:04:14 +0000 (10:04 -0400)]
qa/suites/upgarde/jewel-x/parallel: more whitelisting

Signed-off-by: Sage Weil <sage@redhat.com>
7 years agoMerge pull request #16830 from liewegas/wip-jewel-x
Sage Weil [Sun, 6 Aug 2017 13:57:55 +0000 (08:57 -0500)]
Merge pull request #16830 from liewegas/wip-jewel-x

upgrade/jewel-x: a few fixes

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
7 years agoqa/suites/upgrade/jewel-x/parallel: more whitelisting 16830/head
Sage Weil [Sat, 5 Aug 2017 18:44:05 +0000 (14:44 -0400)]
qa/suites/upgrade/jewel-x/parallel: more whitelisting

Signed-off-by: Sage Weil <sage@redhat.com>
7 years agoqa/suites/upgrade/*-x/parallel: whitelist more stuff
Sage Weil [Sat, 5 Aug 2017 02:28:54 +0000 (22:28 -0400)]
qa/suites/upgrade/*-x/parallel: whitelist more stuff

Signed-off-by: Sage Weil <sage@redhat.com>
7 years agoqa/suites/upgrade/*/parallel: disable POOL_APP_NOT_ENABLED
Sage Weil [Sat, 5 Aug 2017 02:25:52 +0000 (22:25 -0400)]
qa/suites/upgrade/*/parallel: disable POOL_APP_NOT_ENABLED

There is some other random workload running (that creates pools)
while we upgrade and wait for healthy.  Just disable the warning
for these tests.

Signed-off-by: Sage Weil <sage@redhat.com>
7 years agomon: add mon_warn_on_pool_no_app option
Sage Weil [Sat, 5 Aug 2017 02:24:07 +0000 (22:24 -0400)]
mon: add mon_warn_on_pool_no_app option

Signed-off-by: Sage Weil <sage@redhat.com>
7 years agoqa/suites/upgrade/jewel-x/parallel: whitelist OSD_DOWN
Sage Weil [Fri, 4 Aug 2017 18:39:53 +0000 (14:39 -0400)]
qa/suites/upgrade/jewel-x/parallel: whitelist OSD_DOWN

We restart OSDs during the upgrade.

Signed-off-by: Sage Weil <sage@redhat.com>
7 years agomon/MgrMonitor: use static mgr commands if none defined yet
Sage Weil [Fri, 4 Aug 2017 18:37:41 +0000 (14:37 -0400)]
mon/MgrMonitor: use static mgr commands if none defined yet

During upgrade, we may get a command (like 'pg dump') before we
pass our first post-upgrade paxos round embedding the static mgr
commands.  Until then, use the static mgr commands (that's what
we're proposing anyway).

Signed-off-by: Sage Weil <sage@redhat.com>
7 years agomsg/async/rdma: improves RX buffer management 16693/head
Adir Lev [Wed, 3 May 2017 11:17:51 +0000 (11:17 +0000)]
msg/async/rdma: improves RX buffer management

The commit adds following changes:
- rx buffers are allocated from the memory pool (boost::pool)
- flat memory layout for buffer data/metadata to reduce cpu cache misses
- number of receive buffers can be much larger than receive queue len
- post new buffers to the srq as soon as possible.
- stat counters

Signed-off-by: Alex Mikheev <alexm@mellanox.com>
7 years agoos/bluestore: default journal media to store media if bluefs is disabled 16844/head
xie xingguo [Sun, 6 Aug 2017 10:50:17 +0000 (18:50 +0800)]
os/bluestore: default journal media to store media if bluefs is disabled

So we won't prevent bluestore-without-bluefs backed OSDs from booting:
0> 2017-08-06 18:28:07.431316 7ffa1c95fd00 -1 /home/xxg/build/ceph-dev/src/os/bluestore/BlueStore.cc: In function
'virtual bool BlueStore::is_journal_rotational()' thread 7ffa1c95fd00 time 2017-08-06 18:28:07.428503
/home/xxg/build/ceph-dev/src/os/bluestore/BlueStore.cc: 4401: FAILED assert(bluefs)

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
7 years agoMerge pull request #16780 from amitkumar50/cov-1414521
Jos Collin [Sun, 6 Aug 2017 09:05:34 +0000 (09:05 +0000)]
Merge pull request #16780 from amitkumar50/cov-1414521

libradosstriper: Initialize member variable m_writeRc in WriteCompletionData

Reviewed-by: Jos Collin <jcollin@redhat.com>
7 years agoMerge pull request #16781 from amitkumar50/cov-1414516
Jos Collin [Sun, 6 Aug 2017 06:47:59 +0000 (06:47 +0000)]
Merge pull request #16781 from amitkumar50/cov-1414516

msg: Initialize member variables in Infiniband

Reviewed-by: Jos Collin <jcollin@redhat.com>
7 years agomsg: Initilization of uninitialized class Infiniband members 16781/head
amitkuma [Thu, 3 Aug 2017 16:09:36 +0000 (21:39 +0530)]
msg: Initilization of uninitialized class Infiniband members

Fixes coverity issues:

1414516 Uninitialized pointer field
2. uninit_member: Non-static class member max_send_wr is not initialized in this constructor nor in any functions that it calls.
4. uninit_member: Non-static class member max_recv_wr is not initialized in this constructor nor in any functions that it calls.
6. uninit_member: Non-static class member max_sge is not initialized in this constructor nor in any functions that it calls.
8. uninit_member: Non-static class member ib_physical_port is not initialized in this constructor nor in any functions that it calls.
10. uninit_member: Non-static class member memory_manager is not initialized in this constructor nor in any functions that it calls.
12. uninit_member: Non-static class member srq is not initialized in this constructor nor in any functions that it calls.
14. uninit_member: Non-static class member device is not initialized in this constructor nor in any functions that it calls.
CID 1414516 (#1 of 1): Uninitialized pointer field (UNINIT_CTOR)
16. uninit_member: Non-static class member pd is not initialized in this constructor nor in any functions that it calls.

Signed-off-by: Amit Kumar amitkuma@redhat.com
7 years agoMerge pull request #16662 from ivancich/wip-dmclock-options-doc
Sage Weil [Sat, 5 Aug 2017 18:36:37 +0000 (13:36 -0500)]
Merge pull request #16662 from ivancich/wip-dmclock-options-doc

doc: op queue and mclock related options

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Sage Weil <sage@redhat.com>
7 years agoMerge pull request #16824 from liewegas/wip-more-scrub-time
Sage Weil [Sat, 5 Aug 2017 18:35:55 +0000 (13:35 -0500)]
Merge pull request #16824 from liewegas/wip-more-scrub-time

qa/tasks/ceph: wait longer for scrub

7 years agoMerge pull request #16841 from liewegas/wip-debug-ceph
Sage Weil [Sat, 5 Aug 2017 18:35:13 +0000 (13:35 -0500)]
Merge pull request #16841 from liewegas/wip-debug-ceph

qa/tasks/ceph: debug osd setup

7 years agoqa/clusters/fixed-2: 4 osds per node, not 3 16842/head
Sage Weil [Sat, 5 Aug 2017 18:34:27 +0000 (14:34 -0400)]
qa/clusters/fixed-2: 4 osds per node, not 3

We need this for the thrashing with ec k=2 m=2 pools.

Signed-off-by: Sage Weil <sage@redhat.com>
7 years agoqa/suites/rados/thrash/workloads/cache-agent-big: m=2
Sage Weil [Sat, 5 Aug 2017 18:33:13 +0000 (14:33 -0400)]
qa/suites/rados/thrash/workloads/cache-agent-big: m=2

...because we do the test_map_discontinuity thing.

Signed-off-by: Sage Weil <sage@redhat.com>
7 years agoMerge pull request #16633 from dillaman/wip-bootstrap-rbd-mirror
Sage Weil [Sat, 5 Aug 2017 18:27:32 +0000 (13:27 -0500)]
Merge pull request #16633 from dillaman/wip-bootstrap-rbd-mirror

mon: added bootstrap-rbd auth profile

Reviewed-by: Sage Weil <sage@redhat.com>
7 years agoMerge pull request #16704 from dillaman/wip-20644
Sage Weil [Sat, 5 Aug 2017 18:24:09 +0000 (13:24 -0500)]
Merge pull request #16704 from dillaman/wip-20644

rbd-mirror: assertion failure during pool replayer shut down

Reviewed-by: Venky Shankar <vshankar@redhat.com>
7 years agoqa/tasks/ceph: debug osd setup 16841/head
Sage Weil [Sat, 5 Aug 2017 17:53:26 +0000 (13:53 -0400)]
qa/tasks/ceph: debug osd setup

I've seen a couple rbd runs that seem to skip the next block :/

Signed-off-by: Sage Weil <sage@redhat.com>
7 years agoceph.spec.in: ceph_volume but not ceph_volume_client in ceph-base 16832/head
Sage Weil [Fri, 4 Aug 2017 19:47:10 +0000 (15:47 -0400)]
ceph.spec.in: ceph_volume but not ceph_volume_client in ceph-base

Fixes: http://tracker.ceph.com/issues/20915
Signed-off-by: Sage Weil <sage@redhat.com>
7 years agoMerge pull request #16784 from amitkumar50/cov-1244200
Jos Collin [Sat, 5 Aug 2017 15:41:15 +0000 (15:41 +0000)]
Merge pull request #16784 from amitkumar50/cov-1244200

test: Initialize pointer msg in MonClientHelper

Reviewed-by: Jos Collin <jcollin@redhat.com>
7 years agoMerge pull request #16839 from amitkumar50/cov-messages-3
Jos Collin [Sat, 5 Aug 2017 15:35:22 +0000 (15:35 +0000)]
Merge pull request #16839 from amitkumar50/cov-messages-3

messages: Initialize member variables

Reviewed-by: Jos Collin <jcollin@redhat.com>
7 years agomessages: Initializing variables various classes 16839/head
amitkuma [Sat, 5 Aug 2017 19:01:28 +0000 (00:31 +0530)]
messages: Initializing variables various classes

Fixes the coverity Issues:

** 1019615 Uninitialized scalar field
2. uninit_member: Non-static class member service_type is not initialized
 in this constructor nor in any functions that it calls.
CID 1019615 (#1 of 1): Uninitialized scalar field (UNINIT_CTOR)
4. uninit_member: Non-static class member service_op is not initialized
 in this constructor nor in any functions that it calls.

** 1019618 Uninitialized scalar field
2. uninit_member: Non-static class member op is not initialized in this
 constructor nor in any functions that it calls.
4. uninit_member: Non-static class member cookie is not initialized in
 this constructor nor in any functions that it calls.
CID 1019618 (#1 of 1): Uninitialized scalar field (UNINIT_CTOR)
6. uninit_member: Non-static class member last_committed is not initialized
 in this constructor nor in any functions that it calls.

** 1019620 Uninitialized scalar field
2. uninit_member: Non-static class member epoch is not initialized in this
 constructor nor in any functions that it calls.
CID 1019620 (#1 of 1): Uninitialized scalar field (UNINIT_CTOR)
4. uninit_member: Non-static class member request_ack is not initialized
in this constructor nor in any functions that it calls.

** 1019621 Uninitialized scalar field
2. uninit_member: Non-static class member epoch is not initialized in this
constructor nor in any functions that it calls.
CID 1019621 (#1 of 1): Uninitialized scalar field (UNINIT_CTOR)
4. uninit_member: Non-static class member round is not initialized in this
constructor nor in any functions that it calls

Signed-off-by: Amit Kumar amitkuma@redhat.com
7 years agoMerge pull request #16819 from amitkumar50/cov-messages-1
Jos Collin [Sat, 5 Aug 2017 15:15:45 +0000 (15:15 +0000)]
Merge pull request #16819 from amitkumar50/cov-messages-1

messages: Initialize member variables

Reviewed-by: Jos Collin <jcollin@redhat.com>
Reviewed-by: Greg Farnum <gfarnum@redhat.com>
7 years agomessages: Initializing member variables of module messages 16819/head
amitkuma [Fri, 4 Aug 2017 17:11:18 +0000 (22:41 +0530)]
messages: Initializing member variables of module messages

Fixes coverity Issue:

CID 717307 (#1 of 1): Uninitialized scalar field (UNINIT_CTOR)
2. uninit_member: Non-static class member epoch is not initialized in this constructor nor in any functions that it calls.
2. uninit_member: Non-static class member epoch is not initialized in this constructor nor in any functions that it calls.
CID 717308 (#1 of 1): Uninitialized scalar field (UNINIT_CTOR)
4. uninit_member: Non-static class member query_epoch is not initialized in this constructor nor in any functions that it calls.
CID 717310 (#1 of 1): Uninitialized scalar field (UNINIT_CTOR)
2. uninit_member: Non-static class member epoch is not initialized in this constructor nor in any functions that it calls.
CID 717311 (#1 of 1): Uninitialized scalar field (UNINIT_CTOR)
2. uninit_member: Non-static class member epoch is not initialized in this constructor nor in any functions that it calls.
CID 717312 (#1 of 1): Uninitialized scalar field (UNINIT_CTOR)
2. uninit_member: Non-static class member epoch is not initialized in this constructor nor in any functions that it calls.
uninit_member: Non-static class member op is not initialized in this constructor nor in any functions that it calls.
uninit_member: Non-static class member map_epoch is not initialized in this constructor nor in any functions that it calls.
CID 717313 (#1 of 1): Uninitialized scalar field (UNINIT_CTOR)
uninit_member: Non-static class member query_epoch is not initialized in this constructor nor in any functions that it calls.
CID 717314 (#1 of 1): Uninitialized scalar field (UNINIT_CTOR)
2. uninit_member: Non-static class member map_epoch is not initialized in this constructor nor in any functions that it calls.
CID 717315 (#1 of 1): Uninitialized scalar field (UNINIT_CTOR)
2. uninit_member: Non-static class member epoch is not initialized in this constructor nor in any functions that it calls.
uninit_member: Non-static class member map_epoch is not initialized in this constructor nor in any functions that it calls.
CID 717316 (#1 of 1): Uninitialized scalar field (UNINIT_CTOR)
uninit_member: Non-static class member op is not initialized in this constructor nor in any functions that it calls.
uninit_member: Non-static class member map_epoch is not initialized in this constructor nor in any functions that it calls.
CID 717317 (#1 of 1): Uninitialized scalar field (UNINIT_CTOR)
uninit_member: Non-static class member min_epoch is not initialized in this constructor nor in any functions that it calls.
uninit_member: Non-static class member repair is not initialized in this constructor nor in any functions that it calls.
CID 717318 (#1 of 1): Uninitialized scalar field (UNINIT_CTOR)
4. uninit_member: Non-static class member deep is not initialized in this constructor nor in any functions that it calls.
uninit_member: Non-static class member map_epoch is not initialized in this constructor nor in any functions that it calls.
uninit_member: Non-static class member acks_wanted is not initialized in this constructor nor in any functions that it calls.
uninit_member: Non-static class member old_exists is not initialized in this constructor nor in any functions that it calls.
uninit_member: Non-static class member old_size is not initialized in this constructor nor in any functions that it calls.
uninit_member: Non-static class member first is not initialized in this constructor nor in any functions that it calls.
CID 717319 (#1 of 1): Uninitialized scalar field (UNINIT_CTOR)
uninit_member: Non-static class member complete is not initialized in this constructor nor in any functions that it calls.
uninit_member: Non-static class member map_epoch is not initialized in this constructor nor in any functions that it calls.
uninit_member: Non-static class member ack_type is not initialized in this constructor nor in any functions that it calls.
CID 717320 (#1 of 1): Uninitialized scalar field (UNINIT_CTOR)
uninit_member: Non-static class member result is not initialized in this constructor nor in any functions that it calls.
CID 717321 (#1 of 1): Uninitialized scalar field (UNINIT_CTOR)
uninit_member: Non-static class member epoch is not initialized in this constructor nor in any functions that it calls
uninit_member: Non-static class member pool is not initialized in this constructor nor in any functions that it calls.
uninit_member: Non-static class member op is not initialized in this constructor nor in any functions that it calls.
uninit_member: Non-static class member auid is not initialized in this constructor nor in any functions that it calls.
CID 717322 (#1 of 1): Uninitialized scalar field (UNINIT_CTOR)
uninit_member: Non-static class member crush_rule is not initialized in this constructor nor in any functions that it calls.
uninit_member: Non-static class member replyCode is not initialized in this constructor nor in any functions that it calls.
CID 717323 (#1 of 1): Uninitialized scalar field (UNINIT_CTOR)
uninit_member: Non-static class member epoch is not initialized in this constructor nor in any functions that it calls.

Signed-off-by: Amit Kumar amitkuma@redhat.com
7 years agoMerge pull request #16785 from amitkumar50/cov-1274326
Jos Collin [Sat, 5 Aug 2017 13:56:07 +0000 (13:56 +0000)]
Merge pull request #16785 from amitkumar50/cov-1274326

test: Initialize pointer variables in TestMemIoCtxImpl

Reviewed-by: Jos Collin <jcollin@redhat.com>
7 years agorados: Initializing uninitialized pointer variables 16785/head
amitkuma [Thu, 3 Aug 2017 17:16:25 +0000 (22:46 +0530)]
rados: Initializing uninitialized pointer variables

Fixes the coverity issue:

1274326 Uninitialized pointer field
2. uninit_member: Non-static class member m_client is not initialized in
this constructor nor in any functions that it calls.
CID 1274326 (#1 of 1): Uninitialized pointer field (UNINIT_CTOR)
4. uninit_member: Non-static class member m_pool is not initialized in
this constructor nor in any functions that it calls.

Signed-off-by: Amit Kumar amitkuma@redhat.com
7 years agomon/OSDMonitor: sanity check osd before performing 'osd purge' 16838/head
xie xingguo [Sat, 5 Aug 2017 11:21:33 +0000 (19:21 +0800)]
mon/OSDMonitor: sanity check osd before performing 'osd purge'

This will pervent OSDMonitor from crashing on purging a very large
non-existent osd id as below:

osd e11 prepare_command_osd_purge purging osd.8
    -1> 2017-08-05 18:59:44.994319 7f6076968700 10 mon.a@0(leader).osd e11 prepare_command_osd_destroy osd.8 does not exist.
     0> 2017-08-05 18:59:45.002309 7f6076968700 -1 /home/xxg/build/ceph-dev/src/osd/OSDMap.h: In function 'int OSDMap::get_state(int) const'
 thread 7f6076968700 time 2017-08-05 18:59:44.994336
/home/xxg/build/ceph-dev/src/osd/OSDMap.h: 690: FAILED assert(o < max_osd)

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
7 years agocrush: automatically kill dead classes 16837/head
xie xingguo [Sat, 5 Aug 2017 10:25:21 +0000 (18:25 +0800)]
crush: automatically kill dead classes

If a class is no more referenced by any devices or crush rules,
it shall be considered as dead.

This patch makes Ceph automatically recycles those dead classes,
so user does not to explicitly call 'class rm', which is unsafe
and annoying.

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
7 years agocrush: remove 'class rm' command
xie xingguo [Sat, 5 Aug 2017 08:21:27 +0000 (16:21 +0800)]
crush: remove 'class rm' command

The current version is broken. E.g., it should only remove a class
which is never referenced by any device.

Since we now create new classes automatically, we shall automatically
recycle dead classes too. So this command is definitely unuseful.
(Actually it is weird that we keep 'class rm' without keeping the
 corresponding 'class create' command).

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
7 years agocrush: do not automatically recycle class for 'rm-device-class'
xie xingguo [Sat, 5 Aug 2017 07:34:58 +0000 (15:34 +0800)]
crush: do not automatically recycle class for 'rm-device-class'

This will prevent the current crush rule from referencing a non-existent
shadow tree and hence avoid a coredump such as below:

 0> 2017-08-05 09:54:19.943349 7f73887d6700 -1 /clove/vm/xxg/rpm/ceph/rpmbuild/BUILD/ceph-12.1.2.1/src/crush/CrushWrapper.cc: In function 'int CrushWrapper::get_rule_weight_osd_map(unsigned
 int, std::map<int, float>*)' thread 7f73887d6700 time 2017-08-05 09:54:19.941291
/clove/vm/xxg/rpm/ceph/rpmbuild/BUILD/ceph-12.1.2.1/src/crush/CrushWrapper.cc: 1631: FAILED assert(b)

 ceph version 12.1.2.1-11-gd0f812a (d0f812a3a757b319c26794f558b57770663ab324) luminous (rc)
 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x110) [0x7f7398b66ea0]
 2: (CrushWrapper::get_rule_weight_osd_map(unsigned int, std::map<int, float, std::less<int>, std::allocator<std::pair<int const, float> > >*)+0x54e) [0x7f7398daac4e]
 3: (PGMap::get_rule_avail(OSDMap const&, int) const+0x68) [0x7f73989a6428]
 4: (PGMap::get_rules_avail(OSDMap const&, std::map<int, long, std::less<int>, std::allocator<std::pair<int const, long> > >*) const+0x35c) [0x7f73989b748c]
 5: (PGMap::encode_digest(OSDMap const&, ceph::buffer::list&, unsigned long) const+0x16) [0x7f73989b7506]
 6: (DaemonServer::send_report()+0x2a4) [0x7f73989f5474]
 7: (DaemonServer::maybe_ready(int)+0x2f9) [0x7f73989f6129]
 8: (DaemonServer::ms_dispatch(Message*)+0xce) [0x7f73989ff68e]
 9: (DispatchQueue::entry()+0x792) [0x7f7398dd2a22]
 10: (DispatchQueue::DispatchThread::entry()+0xd) [0x7f7398c1429d]
 11: (()+0x7df3) [0x7f739640cdf3]
 12: (clone()+0x6d) [0x7f73954f23ed]

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
7 years agoMerge pull request #16791 from amitkumar50/cov-717285
Jos Collin [Sat, 5 Aug 2017 10:32:13 +0000 (10:32 +0000)]
Merge pull request #16791 from amitkumar50/cov-717285

messages: Initializing member variable in MMDSCacheRejoin

Reviewed-by: Jos Collin <jcollin@redhat.com>
7 years agoMerge pull request #16783 from amitkumar50/cov-1414515
Jos Collin [Sat, 5 Aug 2017 09:51:55 +0000 (09:51 +0000)]
Merge pull request #16783 from amitkumar50/cov-1414515

msg: Remove unused variable perf_counter in RDMAStack

Reviewed-by: Jos Collin <jcollin@redhat.com>
7 years agoMerge pull request #16810 from amitkumar50/cov-717293
Jos Collin [Sat, 5 Aug 2017 08:40:31 +0000 (08:40 +0000)]
Merge pull request #16810 from amitkumar50/cov-717293

messages: Initialize members in MMDSTableRequest

Reviewed-by: Jos Collin <jcollin@redhat.com>
7 years agoMerge PR #16833 into master
Patrick Donnelly [Sat, 5 Aug 2017 03:26:09 +0000 (20:26 -0700)]
Merge PR #16833 into master

* refs/remotes/upstream/pull/16833/head:
qa: whitelist expected MDS_CLIENT_OLDEST_TID warn
qa: ignore insufficient standby during failover
qa: fix read-only whitelist
mds: MDS_DAMAGED to MDS_DAMAGE
doc: remove duplicate CephFS health check doc

7 years agoMerge PR #16825 into master
Patrick Donnelly [Sat, 5 Aug 2017 03:26:07 +0000 (20:26 -0700)]
Merge PR #16825 into master

* refs/remotes/upstream/pull/16825/head:
Changing 'int const' to 'const int'

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
7 years agoMerge PR #16751 into master
Patrick Donnelly [Sat, 5 Aug 2017 03:26:05 +0000 (20:26 -0700)]
Merge PR #16751 into master

* refs/remotes/upstream/pull/16751/head:
mds: Added NULL check before dereference

Reviewed-by: Jos Collin <jcollin@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
7 years agoMerge PR #16750 into master
Patrick Donnelly [Sat, 5 Aug 2017 03:26:03 +0000 (20:26 -0700)]
Merge PR #16750 into master

* refs/remotes/upstream/pull/16750/head:
client: Added NULL check before dereference

Reviewed-by: John Spray <john.spray@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
7 years agoMerge PR #16725 into master
Patrick Donnelly [Sat, 5 Aug 2017 03:26:01 +0000 (20:26 -0700)]
Merge PR #16725 into master

* refs/remotes/upstream/pull/16725/head:
mds: Removing unused variables from StrayManager

Reviewed-by: Jos Collin <jcollin@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
7 years agoMerge PR #16671 into master
Patrick Donnelly [Sat, 5 Aug 2017 03:25:59 +0000 (20:25 -0700)]
Merge PR #16671 into master

* refs/remotes/upstream/pull/16671/head:
Client: Add function "get_default_data_pool_name" for java client

Reviewed-by: Zheng Yan <zyan@redhat.com>
7 years agoMerge PR #16570 into master
Patrick Donnelly [Sat, 5 Aug 2017 03:25:56 +0000 (20:25 -0700)]
Merge PR #16570 into master

* refs/remotes/upstream/pull/16570/head:
mds: clean up obsolete declarations/headers

Reviewed-by: Zheng Yan <zyan@redhat.com>
7 years agoqa: whitelist expected MDS_CLIENT_OLDEST_TID warn 16833/head
Patrick Donnelly [Sat, 5 Aug 2017 03:21:13 +0000 (20:21 -0700)]
qa: whitelist expected MDS_CLIENT_OLDEST_TID warn

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>