]> git.apps.os.sepia.ceph.com Git - ceph.git/log
ceph.git
7 years agocrush: fix type mismatch 17014/head
xie xingguo [Tue, 15 Aug 2017 04:18:30 +0000 (12:18 +0800)]
crush: fix type mismatch

Pool IDs are of type int64_t instead of uint64_t.

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
7 years agocrush: fix bucket_adjust_item_weight() won't update weight-set correctly
xie xingguo [Mon, 14 Aug 2017 07:57:07 +0000 (15:57 +0800)]
crush: fix bucket_adjust_item_weight() won't update weight-set correctly

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
7 years agocrush: CrushWrapper::add_bucket - do not allow caller pass in null 'idout'
xie xingguo [Mon, 14 Aug 2017 06:23:24 +0000 (14:23 +0800)]
crush: CrushWrapper::add_bucket - do not allow caller pass in null 'idout'

*** Caught signal (Segmentation fault) **
 in thread 7f495c0f6300 thread_name:crushtool
 ceph version 12.1.2-768-gab69125 (ab6912523e779174f92f0b0fc10372bd0b645415) mimic (dev)
 1: (()+0x1a3d1) [0x7f495c1343d1]
 2: (()+0xf370) [0x7f4951deb370]
 3: (CrushWrapper::add_bucket(int, int, int, int, int, int*, int*, int*)+0x84) [0x7f49538ba084]
 4: (CrushCompiler::parse_bucket(__gnu_cxx::__normal_iterator<boost::spirit::tree_node<boost::spirit::node_val_data<char const*, boost::spirit::nil_t> >*, std::vector<boost::spirit::tree_node<boost::spirit::node_val_data<char const*, boost::spirit::nil_t> >, std::allocator<boost::spirit::tree_node<boost::spirit::node_val_data<char const*, boost::spirit::nil_t> > > > > const&)+0xef0) [0x7f49538dc170]
 5: (CrushCompiler::parse_crush(__gnu_cxx::__normal_iterator<boost::spirit::tree_node<boost::spirit::node_val_data<char const*, boost::spirit::nil_t> >*, std::vector<boost::spirit::tree_node<boost::spirit::node_val_data<char const*, boost::spirit::nil_t> >, std::allocator<boost::spirit::tree_node<boost::spirit::node_val_data<char const*, boost::spirit::nil_t> > > > > const&)+0x130) [0x7f49538dcba0]
 6: (CrushCompiler::compile(std::istream&, char const*)+0xb93) [0x7f49538deaa3]
 7: (main()+0x2615) [0x7f495c126015]
 8: (__libc_start_main()+0xf5) [0x7f49507ccb35]
 9: (()+0xf4b0) [0x7f495c1294b0]
2017-08-14 13:31:25.498050 7f495c0f6300 -1 *** Caught signal (Segmentation fault) **
 in thread 7f495c0f6300 thread_name:crushtool

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
7 years agocrush: update crush_choose_arg_map size on resizing
xie xingguo [Mon, 14 Aug 2017 06:19:09 +0000 (14:19 +0800)]
crush: update crush_choose_arg_map size on resizing

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
7 years agocrush: fix bucket_remove_item() won't update weight-set simultaneously
xie xingguo [Mon, 14 Aug 2017 06:15:53 +0000 (14:15 +0800)]
crush: fix bucket_remove_item() won't update weight-set simultaneously

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
7 years agocrush: fix bucket_add_item() won't update weight-set simultaneously
xie xingguo [Mon, 14 Aug 2017 06:13:59 +0000 (14:13 +0800)]
crush: fix bucket_add_item() won't update weight-set simultaneously

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
7 years agocrush: fix bucket index to weight-set
xie xingguo [Mon, 14 Aug 2017 06:12:16 +0000 (14:12 +0800)]
crush: fix bucket index to weight-set

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
7 years agoMerge pull request #16976 from liewegas/wip-osd-empty 17017/head
Sage Weil [Sun, 13 Aug 2017 19:02:44 +0000 (14:02 -0500)]
Merge pull request #16976 from liewegas/wip-osd-empty

mgr: implement 'osd safe-to-destroy' and 'osd ok-to-stop' commands

Reviewed-by: Gregory Farnum <gfarnum@redhat.com>
7 years agoMerge pull request #16625 from Yan-waller/wip-walle-0727testlibrados
Sage Weil [Sun, 13 Aug 2017 19:02:22 +0000 (14:02 -0500)]
Merge pull request #16625 from Yan-waller/wip-walle-0727testlibrados

test/librados: reorder ASSERT_EQ() arguments

Reviewed-by: Jos Collin <jcollin@redhat.com>
7 years agoMerge pull request #16883 from liewegas/wip-20939
Sage Weil [Sun, 13 Aug 2017 18:59:43 +0000 (13:59 -0500)]
Merge pull request #16883 from liewegas/wip-20939

crush,mon: fix weight-set vs crush device classes

Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>
7 years agoqa/suites/rados/objectstore/objectstore: less debug
Sage Weil [Sun, 13 Aug 2017 18:41:43 +0000 (14:41 -0400)]
qa/suites/rados/objectstore/objectstore: less debug

Saw an ENOSPC.

Signed-off-by: Sage Weil <sage@redhat.com>
7 years agoMerge pull request #17007 from alex-mikheev/wip_rdma_notify_fix
Haomai Wang [Sun, 13 Aug 2017 12:13:59 +0000 (20:13 +0800)]
Merge pull request #17007 from alex-mikheev/wip_rdma_notify_fix

src/msg/rdma: fixes failure on assert in notify()

Reviewed-by: Haomai Wang <haomai@xsky.com>
7 years agosrc/msg/rdma: fixes failure on assert in notify() 17007/head
Alex Mikheev [Sun, 13 Aug 2017 10:55:32 +0000 (10:55 +0000)]
src/msg/rdma: fixes failure on assert in notify()

The commit fixes incorrect eventfd handling introduced in
2e75b876d1e8e9c2ac556808f958fcbfeaad7d52

Signed-off-by: Alex Mikheev <alexm@mellanox.com>
7 years agoMerge pull request #16978 from gmayyyha/mgr-ms-verify-auth
Yuri Weinstein [Sat, 12 Aug 2017 21:45:28 +0000 (14:45 -0700)]
Merge pull request #16978 from gmayyyha/mgr-ms-verify-auth

mgr/DaemonServer.cc: add 'is_valid=false' when decode caps error

Reviewed-by: Jos Collin <jcollin@redhat.com>
Reviewed-by: John Spray <john.spray@redhat.com>
7 years agoMerge pull request #16906 from liewegas/wip-bluestore-stupid
Sage Weil [Sat, 12 Aug 2017 21:10:31 +0000 (16:10 -0500)]
Merge pull request #16906 from liewegas/wip-bluestore-stupid

os/bluestore: switch default allocator to stupid; test both bitmap and stupid in qa

Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>
7 years agocrush/CrushWrapper: fixing timing of removal in remove_item_under 16883/head
Sage Weil [Sat, 12 Aug 2017 19:19:45 +0000 (15:19 -0400)]
crush/CrushWrapper: fixing timing of removal in remove_item_under

Do it after we reweight, not before

Signed-off-by: Sage Weil <sage@redhat.com>
7 years agocrush/CrushWrapper: fix iterator invalidation in cleanup_dead_classes
Sage Weil [Sat, 12 Aug 2017 18:45:42 +0000 (14:45 -0400)]
crush/CrushWrapper: fix iterator invalidation in cleanup_dead_classes

Signed-off-by: Sage Weil <sage@redhat.com>
7 years agocrush/CrushWrapper: keep weights and/or ids null if empty
Sage Weil [Wed, 9 Aug 2017 21:27:49 +0000 (17:27 -0400)]
crush/CrushWrapper: keep weights and/or ids null if empty

Signed-off-by: Sage Weil <sage@redhat.com>
7 years agocrush/builder: fix ENOENT when removing last bucket item
Sage Weil [Wed, 9 Aug 2017 21:25:12 +0000 (17:25 -0400)]
crush/builder: fix ENOENT when removing last bucket item

We were decrementing size and then breaking out ENOENT condition check.
Fix by decrementing size only after we break out of the loop and verify
we found the item.

Fix a follow-on bug by avoiding realloc when we have 0 items left.  This case
was never exercised before due to the ENOENT issue; now we return explicitly.
It's really not necessary to realloc at all, probably, since these are very
small arrays, but in any case leaving a single item allocation there in place of
a 0-length allocation is fine.  (And the 0-length allocation behvaior on realloc
is undefined.. may either return a pointer or NULL.)

Signed-off-by: Sage Weil <sage@redhat.com>
7 years agoMerge pull request #17005 from liewegas/wip-verify-slowness
Sage Weil [Sat, 12 Aug 2017 18:19:23 +0000 (13:19 -0500)]
Merge pull request #17005 from liewegas/wip-verify-slowness

qa/suites/rados/verify/validater/valgrind: whitelist PG_

7 years agoqa/suites/rados/verify/validater/valgrind: whitelist PG_ 17005/head
Sage Weil [Sat, 12 Aug 2017 18:18:59 +0000 (14:18 -0400)]
qa/suites/rados/verify/validater/valgrind: whitelist PG_

Peering might be slow due to valgrind.

Signed-off-by: Sage Weil <sage@redhat.com>
7 years agoMerge pull request #17004 from liewegas/wip-clock-skew-whitelist
Sage Weil [Sat, 12 Aug 2017 18:15:50 +0000 (13:15 -0500)]
Merge pull request #17004 from liewegas/wip-clock-skew-whitelist

qa/suites/rados/multimon/tasks/mon_lock_with_skew: whitelist PG_

7 years agoqa/suites/rados/multimon/tasks/mon_lock_with_skew: whitelist PG_ 17004/head
Sage Weil [Sat, 12 Aug 2017 18:15:15 +0000 (14:15 -0400)]
qa/suites/rados/multimon/tasks/mon_lock_with_skew: whitelist PG_

Default pool pgs not up because mons too broken for OSDs to peer.

Signed-off-by: Sage Weil <sage@redhat.com>
7 years agoMerge pull request #15921 from joscollin/wip-CID1411830-Uninitialized-pointer-field
Sage Weil [Sat, 12 Aug 2017 18:03:58 +0000 (13:03 -0500)]
Merge pull request #15921 from joscollin/wip-CID1411830-Uninitialized-pointer-field

test/osd: Non-static class members not initialized in UnsetRedirectOp

Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>
7 years agotest/osd: Non-static class member not initialized in UnsetRedirectOp 15921/head
Jos Collin [Mon, 26 Jun 2017 14:07:14 +0000 (19:37 +0530)]
test/osd: Non-static class member not initialized in UnsetRedirectOp

Fixes the coverity scan report:
CID 1411830: Uninitialized pointer field (UNINIT_CTOR)
2. uninit_member: Non-static class member completion is not initialized in this constructor nor in any functions that it calls.
4. uninit_member: Non-static class member comp is not initialized in this constructor nor in any functions that it calls.

Signed-off-by: Jos Collin <jcollin@redhat.com>
7 years agoMerge pull request #15922 from joscollin/wip-CID1411829-Uninitialized-scalar-field
Jos Collin [Sat, 12 Aug 2017 07:50:05 +0000 (07:50 +0000)]
Merge pull request #15922 from joscollin/wip-CID1411829-Uninitialized-scalar-field

test/osd: initialize Non-static class members in WeightedTestGenerator

Reviewed-by: Xie Xingguo <xie.xingguo@zte.com.cn>
7 years agoMerge pull request #16924 from liewegas/wip-20923
Sage Weil [Sat, 12 Aug 2017 02:55:34 +0000 (21:55 -0500)]
Merge pull request #16924 from liewegas/wip-20923

os/bluestore: fail early on very large objects

Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>
7 years agotest/osd/safe-to-destroy.sh: test 'osd safe-to-destroy' 16976/head
Sage Weil [Thu, 10 Aug 2017 19:03:34 +0000 (15:03 -0400)]
test/osd/safe-to-destroy.sh: test 'osd safe-to-destroy'

This is hard with workunits/cephtool/test.sh because we don't
control the whole cluster.

Signed-off-by: Sage Weil <sage@redhat.com>
7 years agomgr: implement 'osd safe-to-destroy' and 'ok-to-stop' commands
Sage Weil [Thu, 10 Aug 2017 18:06:02 +0000 (14:06 -0400)]
mgr: implement 'osd safe-to-destroy' and 'ok-to-stop' commands

An osd is safe to destroy if

- we have osd_stat for it
- osd_stat indicates no pgs stored
- all pgs are known
- no pgs map to it

An osd is ok ot stop if

- we have pg stats
- no pgs will drop below min_size

Signed-off-by: Sage Weil <sage@redhat.com>
7 years agomgr/ClusterState: record osd_stat for out osds too
Sage Weil [Thu, 10 Aug 2017 19:00:46 +0000 (15:00 -0400)]
mgr/ClusterState: record osd_stat for out osds too

I'm not quite sure why we were doing this. :/

Signed-off-by: Sage Weil <sage@redhat.com>
7 years agoosd/osd_types: include number of locally stored PGs in osd_stat_t
Sage Weil [Thu, 10 Aug 2017 18:05:10 +0000 (14:05 -0400)]
osd/osd_types: include number of locally stored PGs in osd_stat_t

Signed-off-by: Sage Weil <sage@redhat.com>
7 years agoosd/OSDMap: add parse_osd_id_list helper
Sage Weil [Thu, 10 Aug 2017 22:13:40 +0000 (18:13 -0400)]
osd/OSDMap: add parse_osd_id_list helper

Signed-off-by: Sage Weil <sage@redhat.com>
7 years agoMerge pull request #16992 from liewegas/wip-20977
Sage Weil [Sat, 12 Aug 2017 02:36:53 +0000 (21:36 -0500)]
Merge pull request #16992 from liewegas/wip-20977

os/bluestore: do not segv on kraken upgrade debug print

Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>
7 years agoMerge pull request #16993 from liewegas/wip-mimic
Sage Weil [Sat, 12 Aug 2017 02:36:32 +0000 (21:36 -0500)]
Merge pull request #16993 from liewegas/wip-mimic

common: kick off mimic

Reviewed-by: Gregory Farnum <gfarnum@redhat.com>
Reviewed-by: Abhishek Lekshmanan <abhishek.lekshmanan@gmail.com>
7 years agoMerge pull request #16994 from liewegas/wip-20983
Sage Weil [Sat, 12 Aug 2017 02:35:35 +0000 (21:35 -0500)]
Merge pull request #16994 from liewegas/wip-20983

os/bluestore: fix clone dirty_range again

Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>
7 years agoMerge pull request #16995 from liewegas/wip-20738
Sage Weil [Sat, 12 Aug 2017 02:34:48 +0000 (21:34 -0500)]
Merge pull request #16995 from liewegas/wip-20738

os/memstore: memstore_page_set=false

Reviewed-by: Adam Emerson <aemerson@redhat.com>
7 years agoMerge pull request #16996 from liewegas/wip-mgr-vs-clock-skew
Sage Weil [Sat, 12 Aug 2017 02:34:07 +0000 (21:34 -0500)]
Merge pull request #16996 from liewegas/wip-mgr-vs-clock-skew

qa/suites/rados/multimon: whitelist mgr down vs clock skew test

7 years agoMerge pull request #16898 from amitkumar50/cov-messages-6
Gregory Farnum [Fri, 11 Aug 2017 21:25:16 +0000 (14:25 -0700)]
Merge pull request #16898 from amitkumar50/cov-messages-6

messages: Initialization of member variables

Reviewed-by: Greg Farnum <gfarnum@redhat.com>
7 years agoos/bluestore: do not segv on kraken upgrade debug print 16992/head
Sage Weil [Fri, 11 Aug 2017 15:58:42 +0000 (11:58 -0400)]
os/bluestore: do not segv on kraken upgrade debug print

When loading an onode from kraken we have a compat path that calls
get_ref before the SharedBlob pointer is initialized.  This is fine except
that if debugging is enabled the operator<< on the Blob will segv on
printing *b.shared_blob (which is NULL).

Fix operator<< to print something else if it is NULL.  shared_blob does
get set up right after the call to decode() so having it be NULL at this
point is otherwise harmless.

Fixes: http://tracker.ceph.com/issues/20977
Signed-off-by: Sage Weil <sage@redhat.com>
7 years agoqa/suites/rados/multimon: whitelist mgr down vs clock skew test 16996/head
Sage Weil [Fri, 11 Aug 2017 17:42:02 +0000 (13:42 -0400)]
qa/suites/rados/multimon: whitelist mgr down vs clock skew test

Clock skew might make us fail the mgr.

Signed-off-by: Sage Weil <sage@redhat.com>
7 years agoos/memstore: memstore_page_set=false 16995/head
Sage Weil [Fri, 11 Aug 2017 17:31:51 +0000 (13:31 -0400)]
os/memstore: memstore_page_set=false

This regularly returns bad results, see http://tracker.ceph.com/issues/20738

Signed-off-by: Sage Weil <sage@redhat.com>
7 years agoos/bluestore: fix clone dirty_range again 16994/head
Sage Weil [Fri, 11 Aug 2017 16:46:09 +0000 (12:46 -0400)]
os/bluestore: fix clone dirty_range again

If we are cloning a blob for a 1 byte logical extent then dirty_range_begin
will equal _end and we won't dirty the source onode (with possibly newly
shared blobs).

Fix by using a separate flag to indicate whether we are dirtying instead
of overloading the begin/end markers for this.  Note that even if they
are equal dirty_range will still dirty the shard in question.

This is a result of 0ae5d92d42500e5ab08253a00bda47b957767ebc.

Fixes: http://tracker.ceph.com/issues/20983
Signed-off-by: Sage Weil <sage@redhat.com>
7 years agoinclude/rados: nautilus is next 16993/head
Sage Weil [Fri, 11 Aug 2017 16:14:51 +0000 (12:14 -0400)]
include/rados: nautilus is next

Signed-off-by: Sage Weil <sage@redhat.com>
7 years agoinclude/ceph_features: define SERVER_NAUTILUS
Sage Weil [Fri, 11 Aug 2017 16:12:39 +0000 (12:12 -0400)]
include/ceph_features: define SERVER_NAUTILUS

Nautilus will be the next major release after luminous.

Signed-off-by: Sage Weil <sage@redhat.com>
7 years agoinclude/ceph_features: incarnation 3 can begin!
Sage Weil [Fri, 11 Aug 2017 16:11:47 +0000 (12:11 -0400)]
include/ceph_features: incarnation 3 can begin!

With all upgrades passing through luminous, we can now start
reusing bits retired in luminous.  Our sentinel bitmask will be the
combination of SERVER_MIMIC and SERVER_JEWEL (i.e.,
CEPH_FEATUREMASK_SERVER_MIMIC).

Signed-off-by: Sage Weil <sage@redhat.com>
7 years agoinclude/ceph_features: SERVER_M -> SERVER_MIMIC
Sage Weil [Fri, 11 Aug 2017 16:09:54 +0000 (12:09 -0400)]
include/ceph_features: SERVER_M -> SERVER_MIMIC

Signed-off-by: Sage Weil <sage@redhat.com>
7 years agoceph_release: kick off mimic 13.y.z
Sage Weil [Fri, 11 Aug 2017 15:43:17 +0000 (11:43 -0400)]
ceph_release: kick off mimic 13.y.z

Signed-off-by: Sage Weil <sage@redhat.com>
7 years agoMerge pull request #16628 from theanalyst/rgw-policy-list-buckets
Matt Benjamin [Fri, 11 Aug 2017 15:53:01 +0000 (11:53 -0400)]
Merge pull request #16628 from theanalyst/rgw-policy-list-buckets

rgw: policy: support for s3 conditionals in ListBucket Op

7 years agoMerge pull request #16968 from liewegas/wip-store-test-expect-death
Sage Weil [Fri, 11 Aug 2017 13:45:02 +0000 (08:45 -0500)]
Merge pull request #16968 from liewegas/wip-store-test-expect-death

ceph_test_objectstore: drop expect regex

7 years agoceph_test_objectstore: drop expect regex 16968/head
Sage Weil [Thu, 10 Aug 2017 14:41:47 +0000 (10:41 -0400)]
ceph_test_objectstore: drop expect regex

If logging is enabled (as it now is in teuthology) this won't match the
forked output.

Signed-off-by: Sage Weil <sage@redhat.com>
7 years agoqa/workunits/mon/crush_ops.sh: test weight sets vs device classes
Sage Weil [Mon, 7 Aug 2017 22:30:39 +0000 (18:30 -0400)]
qa/workunits/mon/crush_ops.sh: test weight sets vs device classes

Signed-off-by: Sage Weil <sage@redhat.com>
7 years agomon/OSDMonitor: remove choose_args when pool is removed
Sage Weil [Mon, 7 Aug 2017 22:26:09 +0000 (18:26 -0400)]
mon/OSDMonitor: remove choose_args when pool is removed

Signed-off-by: Sage Weil <sage@redhat.com>
7 years agocrush/CrushWrapper: fill in weight-sets when we build shadow trees
Sage Weil [Mon, 7 Aug 2017 21:56:06 +0000 (17:56 -0400)]
crush/CrushWrapper: fill in weight-sets when we build shadow trees

When we build the shadow buckets for the class hierarchies, we need
to fill in the weight-sets for each shadow bucket too.

Skip the ids vector for now since it's not yet used by anything.

Fixes: http://tracker.ceph.com/issues/20939
Signed-off-by: Sage Weil <sage@redhat.com>
7 years agocrush/CrushWrapper: remove unused 'unused' arg for trim_roots_with_classes
Sage Weil [Mon, 7 Aug 2017 20:54:11 +0000 (16:54 -0400)]
crush/CrushWrapper: remove unused 'unused' arg for trim_roots_with_classes

Signed-off-by: Sage Weil <sage@redhat.com>
7 years agocrush: do add/remove before updating weight-sets
Sage Weil [Mon, 7 Aug 2017 20:50:44 +0000 (16:50 -0400)]
crush: do add/remove before updating weight-sets

Signed-off-by: Sage Weil <sage@redhat.com>
7 years agoMerge pull request #16991 from liewegas/wip-doc-bluestore-migration
Sage Weil [Fri, 11 Aug 2017 13:37:28 +0000 (08:37 -0500)]
Merge pull request #16991 from liewegas/wip-doc-bluestore-migration

doc/rados/operations/bluestore-migration: typos and whitespace

7 years agodoc/rados/operations/bluestore-migration: typos and whitespace 16991/head
Sage Weil [Tue, 8 Aug 2017 19:09:50 +0000 (15:09 -0400)]
doc/rados/operations/bluestore-migration: typos and whitespace

Signed-off-by: Sage Weil <sage@redhat.com>
7 years agoMerge pull request #16532 from liupan1111/wip-remove-reb
Sage Weil [Fri, 11 Aug 2017 13:32:12 +0000 (08:32 -0500)]
Merge pull request #16532 from liupan1111/wip-remove-reb

kv/RocksDBStore: extract common code to a new function

Reviewed-by: Sage Weil <sage@redhat.com>
7 years agoMerge pull request #16594 from wjwithagen/wip-wjw-reduce-info-messages
Sage Weil [Fri, 11 Aug 2017 13:31:53 +0000 (08:31 -0500)]
Merge pull request #16594 from wjwithagen/wip-wjw-reduce-info-messages

common: make some message informative, instead of error

Reviewed-by: Kefu Chai <kchai@redhat.com>
7 years agoMerge pull request #16964 from kungf/mon_timer
Sage Weil [Fri, 11 Aug 2017 13:30:42 +0000 (08:30 -0500)]
Merge pull request #16964 from kungf/mon_timer

mon: return directly after health_events_cleanup

Reviewed-by: Sage Weil <sage@redhat.com>
7 years agoMerge pull request #16936 from TsaiJin/wip-set-polling-thread-name
Yuri Weinstein [Fri, 11 Aug 2017 13:20:51 +0000 (06:20 -0700)]
Merge pull request #16936 from TsaiJin/wip-set-polling-thread-name

msg/async/rdma: fix the bug that rdma polling thread uses the same thread name with msg worker

Reviewed-by: Haomai Wang <haomai@xsky.com>
Reviewed-by: Pan Liu <liupan1111@gmail.com>
7 years agoMerge pull request #16961 from xiexingguo/wip-class-rename
Yuri Weinstein [Fri, 11 Aug 2017 13:18:57 +0000 (06:18 -0700)]
Merge pull request #16961 from xiexingguo/wip-class-rename

crush: "osd crush class rename" support

Reviewed-by: Sage Weil <sage@redhat.com>
7 years agomessages: Initialization of member variables 16898/head
amitkuma [Tue, 8 Aug 2017 18:41:13 +0000 (00:11 +0530)]
messages: Initialization of member variables

Fixes the coverity issues:

** 717271 Uninitialized scalar field
2. uninit_member: Non-static class member from_mds is not initialized
 in this constructor nor in any functions that it calls.
4. uninit_member: Non-static class member dir_rep is not initialized
 in this constructor nor in any functions that it calls.
CID 717271 (#1 of 1): Uninitialized scalar field (UNINIT_CTOR)
6. uninit_member: Non-static class member discover is not initialized
 in this constructor nor in any functions that it calls.

** 717272 Uninitialized scalar field
2. uninit_member: Non-static class member want_base_dir is not initialized
in this constructor nor in any functions that it calls.
CID 717272 (#1 of 1): Uninitialized scalar field (UNINIT_CTOR)
4. uninit_member: Non-static class member want_xlocked is not initialized
in this constructor nor in any functions that it calls.

** 717274 Uninitialized scalar field
2. uninit_member: Non-static class member wanted_base_dir is not initialized
 in this constructor nor in any functions that it calls.
4. uninit_member: Non-static class member wanted_xlocked is not initialized
 in this constructor nor in any functions that it calls.
6. uninit_member: Non-static class member flag_error_dn is not initialized
 in this constructor nor in any functions that it calls.
8. uninit_member: Non-static class member flag_error_dir is not initialized
 in this constructor nor in any functions that it calls.
10. uninit_member: Non-static class member unsolicited is not initialized
 in this constructor nor in any functions that it calls.
12. uninit_member: Non-static class member dir_auth_hint is not initialized
 in this constructor nor in any functions that it calls.
CID 717274 (#1 of 1): Uninitialized scalar field (UNINIT_CTOR)
14. uninit_member: Non-static class member starts_with is not initialized
in this constructor nor in any functions that it calls.

** 717275 Uninitialized scalar field
CID 717275 (#1 of 1): Uninitialized scalar field (UNINIT_CTOR)
2. uninit_member: Non-static class member from is not initialized in this
constructor nor in any functions that it calls.

Signed-off-by: Amit Kumar amitkuma@redhat.com
7 years agoMerge pull request #16973 from jan--f/mgr-plugin-doc-accessor-init
John Spray [Fri, 11 Aug 2017 11:23:48 +0000 (12:23 +0100)]
Merge pull request #16973 from jan--f/mgr-plugin-doc-accessor-init

doc/mgr/plugins: mgr accessor during init causes exception

Reviewed-by: John Spray <john.spray@redhat.com>
7 years agoMerge pull request #16983 from renhwztetecs/renhw-wip-rados-tool-cleanup
Xie Xingguo [Fri, 11 Aug 2017 05:29:30 +0000 (13:29 +0800)]
Merge pull request #16983 from renhwztetecs/renhw-wip-rados-tool-cleanup

rados/tool: fixup rados stat command hint

Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>
7 years agoMerge branch 'master' into wip-set-polling-thread-name 16936/head
Jin Cai [Fri, 11 Aug 2017 04:55:16 +0000 (12:55 +0800)]
Merge branch 'master' into wip-set-polling-thread-name

7 years agorados/tool: fixup rados stat command hint 16983/head
huanwen ren [Fri, 11 Aug 2017 02:50:59 +0000 (10:50 +0800)]
rados/tool: fixup rados stat command hint

Signed-off-by: huanwen ren <ren.huanwen@zte.com.cn>
7 years agoMerge pull request #16709 from dzafman/wip-standalone
Sage Weil [Fri, 11 Aug 2017 02:33:43 +0000 (21:33 -0500)]
Merge pull request #16709 from dzafman/wip-standalone

qa/standalone: misc fixes

7 years agoMerge pull request #16954 from gregsfortytwo/wip-20891-pool-metadata
Sage Weil [Fri, 11 Aug 2017 02:29:37 +0000 (21:29 -0500)]
Merge pull request #16954 from gregsfortytwo/wip-20891-pool-metadata

mon: fix "fs new" pool metadata update, tests

Reviewed-by: Sage Weil <sage@redhat.com>
7 years agoMerge pull request #16981 from yuyuyu101/wip_rdma_mt_fix
Haomai Wang [Fri, 11 Aug 2017 02:21:07 +0000 (10:21 +0800)]
Merge pull request #16981 from yuyuyu101/wip_rdma_mt_fix

msg/async/rdma: fixes crash for multi rados client within one process

7 years agomon: return directly after health_events_cleanup 16964/head
kungf [Thu, 10 Aug 2017 12:05:00 +0000 (20:05 +0800)]
mon: return directly after health_events_cleanup

when mon_health_to_clog was set false, all health events was cleanup,
no need to judge the change of mon_health_to_clog_interval and
mon_health_to_clog_tick_interval.

Signed-off-by: wang yang <yang.wang@easystack.cn>
7 years agoMerge pull request #16664 from amitkumar50/coverity-1414508-3
Haomai Wang [Fri, 11 Aug 2017 02:16:33 +0000 (10:16 +0800)]
Merge pull request #16664 from amitkumar50/coverity-1414508-3

rdma: Assign instead of compare

Reviewed-by: Haomai Wang <haomai@xsky.com>
7 years agoMerge pull request #16965 from TsaiJin/wip-use-rdtsc-to-get-time
Haomai Wang [Fri, 11 Aug 2017 02:14:46 +0000 (10:14 +0800)]
Merge pull request #16965 from TsaiJin/wip-use-rdtsc-to-get-time

src/msg/async/rdma: decrease cpu usage by rdtsc instruction

Reviewed-by: Haomai Wang <haomai@xsky.com>
7 years agomsg/async/rdma: add atomic and lock protection 16981/head
Haomai Wang [Fri, 11 Aug 2017 02:02:45 +0000 (10:02 +0800)]
msg/async/rdma: add atomic and lock protection

Signed-off-by: Haomai Wang <haomai@xsky.com>
7 years agomsg/async/rdma: fixes crash in fio
Alex Mikheev [Mon, 12 Jun 2017 08:32:38 +0000 (08:32 +0000)]
msg/async/rdma: fixes crash in fio

fio creates multiple CephContext in a single process.
Crash(es) happen because rdma stack has a global resources that
are still used from one ceph context while have already been destroyed
by another context.

The commit removes global instances of RDMA dispatcher and infiniband
and makes them context (rdma stack) specific.

Signed-off-by: Adir Lev <adirl@mellanox.com>
Signed-off-by: Alex Mikheev <alexm@mellanox.com>
7 years agoMerge pull request #16980 from ceph/revert-16893-wip-fix-multicct-ib
Haomai Wang [Fri, 11 Aug 2017 01:58:17 +0000 (09:58 +0800)]
Merge pull request #16980 from ceph/revert-16893-wip-fix-multicct-ib

Revert "msg/async/rdma: fix multi cephcontext confllicting"

7 years agoRevert "msg/async/rdma: fix multi cephcontext confllicting" 16980/head
Haomai Wang [Fri, 11 Aug 2017 01:57:41 +0000 (09:57 +0800)]
Revert "msg/async/rdma: fix multi cephcontext confllicting"

7 years agoMerge pull request #16977 from liewegas/wip-no-pidfile
David Zafman [Fri, 11 Aug 2017 01:41:38 +0000 (18:41 -0700)]
Merge pull request #16977 from liewegas/wip-no-pidfile

test/CMakeLists: disable test_pidfile.sh

Reviewed-by: Xie Xingguo <xie.xingguo@zte.com.cn>
7 years agomgr/DaemonServer.cc: add 'is_valid=false' when decode caps error 16978/head
Yanhu Cao [Thu, 10 Aug 2017 09:16:48 +0000 (17:16 +0800)]
mgr/DaemonServer.cc: add 'is_valid=false' when decode caps error

Signed-off-by: Yanhu Cao <gmayyyha@gmail.com>
7 years agocrush: "osd crush class rename" support 16961/head
xie xingguo [Thu, 10 Aug 2017 09:28:04 +0000 (17:28 +0800)]
crush: "osd crush class rename" support

In 076a6abd80cc90ebcb901f908f880ef030721b2a I killed the 'class rename' command
and thought it was totally useless but I was wrong.

Consider the following user case:
(1) randomly choose some OSDs(e.g., from different hosts) and try to make them for private use only,
    say, by grouping them into 'pool1'
(2) ceph osd crush set-device-class pool1 'OSDs from (1)'
(3) ceph osd crush rule create-replicated rule_for_pool1 default host pool1
(4) ceph osd pool rename pool1 pool2
(5) ceph osd crush class rename pool1 pool2

From the above user case, we need to safely change a pool name without worrying
any risk of data migration. That is why the 'osd crush class rename' command
is still needed here.

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
7 years agotest/CMakeLists: disable test_pidfile.sh 16977/head
Sage Weil [Thu, 10 Aug 2017 19:41:38 +0000 (15:41 -0400)]
test/CMakeLists: disable test_pidfile.sh

Too flaky, see http://tracker.ceph.com/issues/20975

Signed-off-by: Sage Weil <sage@redhat.com>
7 years agoqa: Fix broken test_activate_osd() due to missing space 16709/head
David Zafman [Thu, 10 Aug 2017 01:58:21 +0000 (18:58 -0700)]
qa: Fix broken test_activate_osd() due to missing space

Signed-off-by: David Zafman <dzafman@redhat.com>
7 years agoqa: Fix races with waiting for scrubs
David Zafman [Wed, 9 Aug 2017 15:43:57 +0000 (08:43 -0700)]
qa: Fix races with waiting for scrubs

The trigger_scrub sets the last_scrub_stamp backwards to
force a scheduled scrub.  In a small window this stamp could get propagated
to the mgr.  A test failure occurred because wait_for_scrub() was confused
by seeing a backward moving date.

The most critical change is having wait_for_scrub() make sure that the
date advances past the previous in value.

A test failed because the random backoff kept delayed triggered scrub, so
set osd_scrub_backoff throughout.

Signed-off-by: David Zafman <dzafman@redhat.com>
7 years agoqa: Testing of ceph-helpers.sh, teardown on fail to dump logs, save cores
David Zafman [Wed, 9 Aug 2017 14:19:36 +0000 (07:19 -0700)]
qa: Testing of ceph-helpers.sh, teardown on fail to dump logs, save cores

Signed-off-by: David Zafman <dzafman@redhat.com>
7 years agoosd: Avoid confusion over legacy snaps when head_exists corrupt
David Zafman [Tue, 8 Aug 2017 14:22:33 +0000 (07:22 -0700)]
osd: Avoid confusion over legacy snaps when head_exists corrupt

Signed-off-by: David Zafman <dzafman@redhat.com>
7 years agoosd: Fix Paxos shutdown handling for commit_finish race
David Zafman [Mon, 7 Aug 2017 19:48:27 +0000 (12:48 -0700)]
osd: Fix Paxos shutdown handling for commit_finish race

Fixes: http://tracker.ceph.com/issues/20921
Signed-off-by: David Zafman <dzafman@redhat.com>
7 years agoqa: Add support for core dumps
David Zafman [Fri, 4 Aug 2017 19:34:47 +0000 (12:34 -0700)]
qa: Add support for core dumps

Save core dumps when running tests locally
Dump logs to output whenever cores seen

Signed-off-by: David Zafman <dzafman@redhat.com>
7 years agoMerge pull request #16918 from liewegas/wip-doc-bluestore-migration
Sage Weil [Thu, 10 Aug 2017 19:15:31 +0000 (14:15 -0500)]
Merge pull request #16918 from liewegas/wip-doc-bluestore-migration

doc/rados/operations/bluestore-migration: document bluestore migration process

Reviewed-by: Vasu Kulkarni <vasu@redhat.com>
7 years agodoc/rados/operations/bluestore-migration: document bluestore migration process 16918/head
Sage Weil [Tue, 8 Aug 2017 19:09:50 +0000 (15:09 -0400)]
doc/rados/operations/bluestore-migration: document bluestore migration process

Signed-off-by: Sage Weil <sage@redhat.com>
7 years agoMerge pull request #16975 from theanalyst/doc-12-1-3-release-notes
Sage Weil [Thu, 10 Aug 2017 18:30:05 +0000 (13:30 -0500)]
Merge pull request #16975 from theanalyst/doc-12-1-3-release-notes

doc: 12.1.3 release notes

Reviewed-by: Sage Weil <sage@redhat.com>
7 years agoMerge pull request #16974 from liewegas/wip-release-notes
Sage Weil [Thu, 10 Aug 2017 18:29:25 +0000 (13:29 -0500)]
Merge pull request #16974 from liewegas/wip-release-notes

doc/release-notes: remove mention of crush weight optimization

Reviewed-by: Abhishek Lekshmanan <abhishek.lekshmanan@gmail.com>
7 years agoqa: test that "fs new" correctly set the application_metadata 16954/head
Greg Farnum [Thu, 10 Aug 2017 17:28:09 +0000 (10:28 -0700)]
qa: test that "fs new" correctly set the application_metadata

Signed-off-by: Greg Farnum <gfarnum@redhat.com>
7 years agodoc: 12.1.3 release notes 16975/head
Abhishek Lekshmanan [Thu, 10 Aug 2017 18:00:19 +0000 (20:00 +0200)]
doc: 12.1.3 release notes

Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>
7 years agodoc/release-notes: remove mention of crush weight optimization 16974/head
Sage Weil [Thu, 10 Aug 2017 17:36:51 +0000 (13:36 -0400)]
doc/release-notes: remove mention of crush weight optimization

It's not there yet.

Signed-off-by: Sage Weil <sage@redhat.com>
7 years agoMerge pull request #15330 from guihecheng/rgw_file-fsid-mount
Matt Benjamin [Thu, 10 Aug 2017 17:32:27 +0000 (13:32 -0400)]
Merge pull request #15330 from guihecheng/rgw_file-fsid-mount

rgw_file: introduce new fsid and rgw_mount

7 years agomdsmon: treat the osdmon correctly when doing plugged updates
Greg Farnum [Wed, 9 Aug 2017 21:34:44 +0000 (14:34 -0700)]
mdsmon: treat the osdmon correctly when doing plugged updates

Make sure it's writeable before invoking changes, and propose_pending()
on it when we're done.
Make the PaxosService::C_RetryMessage public so we can do this from FSCommands.

Maybe-
Fixes: http://tracker.ceph.com/issues/20959
Signed-off-by: Greg Farnum <gfarnum@redhat.com>
7 years agodoc/mgr/plugins: mgr accessor during init causes exception 16973/head
Jan Fajerski [Thu, 10 Aug 2017 17:01:34 +0000 (19:01 +0200)]
doc/mgr/plugins: mgr accessor during init causes exception

Currently calling a mgr accessor during a plugins __init__ method causes
a cyclic locking exception.

Signed-off-by: Jan Fajerski <jfajerski@suse.com>
7 years agoMerge pull request #16648 from ovh/bp-kill-str-conv
Sage Weil [Thu, 10 Aug 2017 16:19:24 +0000 (11:19 -0500)]
Merge pull request #16648 from ovh/bp-kill-str-conv

osdc, class_api: kill implicit string conversions

Reviewed-by: Gregory Farnum <gfarnum@redhat.com>
7 years agoMerge pull request #16850 from liewegas/wip-bluestore-dirty-range
Sage Weil [Thu, 10 Aug 2017 16:17:32 +0000 (11:17 -0500)]
Merge pull request #16850 from liewegas/wip-bluestore-dirty-range

os/bluestore: fix dirty_shard off-by-one

Reviewed-by: Liu-Chunmei <chunmei.liu@intel.com>
7 years agoMerge pull request #16872 from markhpc/wip-bluestore-faster_gncb
Sage Weil [Thu, 10 Aug 2017 16:15:38 +0000 (11:15 -0500)]
Merge pull request #16872 from markhpc/wip-bluestore-faster_gncb

os/bluestore:  clearer comments, not slower code.

Reviewed-by: Sage Weil <sage@redhat.com>