ceph.git
5 years ago14.1.1 26933/head v14.1.1
Jenkins Build Slave User [Mon, 11 Mar 2019 16:42:55 +0000 (16:42 +0000)]
14.1.1

5 years agoMerge PR #26858 into master
Ilya Dryomov [Mon, 11 Mar 2019 13:38:01 +0000 (14:38 +0100)]
Merge PR #26858 into master

* refs/pull/26858/head:
qa: krbd deep-flatten test
qa/suites/krbd: enable deep-flatten feature

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
5 years agoMerge pull request #26825 from idryomov/wip-copyup-truncate
Jason Dillaman [Mon, 11 Mar 2019 12:23:29 +0000 (08:23 -0400)]
Merge pull request #26825 from idryomov/wip-copyup-truncate

librbd: don't do create+truncate for discards with copyup

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
5 years agoMerge pull request #26867 from liewegas/wip-doc-msgr2
Xie Xingguo [Mon, 11 Mar 2019 00:36:48 +0000 (08:36 +0800)]
Merge pull request #26867 from liewegas/wip-doc-msgr2

doc/rados/configuration/msgr2: some documentation about msgr2

Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>
5 years agoMerge pull request #26801 from wjwithagen/wjw-fix-use-binutils-ld
Jason Dillaman [Sun, 10 Mar 2019 18:24:56 +0000 (14:24 -0400)]
Merge pull request #26801 from wjwithagen/wjw-fix-use-binutils-ld

build: LLVM ld does not like the versioning scheme

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Jason Dillaman <dillaman@redhat.com>
5 years agoMerge PR #26859 into master
Patrick Donnelly [Sun, 10 Mar 2019 17:38:16 +0000 (10:38 -0700)]
Merge PR #26859 into master

* refs/pull/26859/head:
qa: ignore slow metadata io wrn during osd thrash

Reviewed-by: Yuri Weinstein <yweins@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>
5 years agoMerge pull request #26876 from wjwithagen/wjw-fix-src/test/librbd/operation/test_mock...
Jason Dillaman [Sun, 10 Mar 2019 14:21:06 +0000 (10:21 -0400)]
Merge pull request #26876 from wjwithagen/wjw-fix-src/test/librbd/operation/test_mock_TrimRequest.cc

librbd: avoid aggregate-initializing any static_visitor

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
5 years agolibrbd: avoid aggregate-initializing any static_visitor 26876/head
Willem Jan Withagen [Sun, 10 Mar 2019 10:51:09 +0000 (11:51 +0100)]
librbd: avoid aggregate-initializing any static_visitor

This will result in an error complaining about a protected dtor.
also see:
    https://github.com/ceph/ceph/pull/26838

Signed-off-by: Willem Jan Withagen <wjw@digiware.nl>
5 years agodoc/rados/configuration/msgr2: some documentation about msgr2 26867/head
Sage Weil [Fri, 8 Mar 2019 22:52:38 +0000 (16:52 -0600)]
doc/rados/configuration/msgr2: some documentation about msgr2

This doesn't integrate very well into network-config.rst, mostly because
that document is horribly out of data and I don't know where to start.
:(

Signed-off-by: Sage Weil <sage@redhat.com>
5 years agoMerge PR #26849 into master
Sage Weil [Sat, 9 Mar 2019 23:38:27 +0000 (17:38 -0600)]
Merge PR #26849 into master

* refs/pull/26849/head:
Revert "msg/async: msgr2: expected tags validation"
msg/async/ProtocolV2: state machine verifications

Reviewed-by: Sage Weil <sage@redhat.com>
5 years agoMerge PR #26869 into master
Sage Weil [Sat, 9 Mar 2019 19:22:56 +0000 (13:22 -0600)]
Merge PR #26869 into master

* refs/pull/26869/head:
qa/standalone/erasure-code/test-erasure-eio.sh: still need to create rbd pool
qa/standalone/scrub/osd-scrub-repair: fix unfound grep
qa/standalone/ceph-helpers: fix test_wait_for_clean
qa/standalone/ceph-helpers.sh: fix test_run_mon
qa/standalone/osd/osd-rep-recov-eio: fix better

Reviewed-by: Kefu Chai <kchai@redhat.com>
5 years agoMerge PR #26713 into master
Patrick Donnelly [Sat, 9 Mar 2019 18:15:26 +0000 (10:15 -0800)]
Merge PR #26713 into master

* refs/pull/26713/head:
mds: wait for client to release shared cap when re-acquiring xlock

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
5 years agoMerge PR #26828 into master
Sage Weil [Sat, 9 Mar 2019 17:00:57 +0000 (11:00 -0600)]
Merge PR #26828 into master

* refs/pull/26828/head:
src/tools/kvstore_tool.cc: Move bluestore start-up code

Reviewed-by: Sage Weil <sage@redhat.com>
5 years agoqa/standalone/erasure-code/test-erasure-eio.sh: still need to create rbd pool 26869/head
Sage Weil [Sat, 9 Mar 2019 15:34:49 +0000 (09:34 -0600)]
qa/standalone/erasure-code/test-erasure-eio.sh: still need to create rbd pool

Signed-off-by: Sage Weil <sage@redhat.com>
5 years agoMerge PR #26660 into master
Sage Weil [Sat, 9 Mar 2019 15:29:33 +0000 (09:29 -0600)]
Merge PR #26660 into master

* refs/pull/26660/head:
include/intarith: enforce the same type for p2*() arguments

Reviewed-by: Sage Weil <sage@redhat.com>
5 years agoMerge PR #26831 into master
Sage Weil [Sat, 9 Mar 2019 15:28:18 +0000 (09:28 -0600)]
Merge PR #26831 into master

* refs/pull/26831/head:
mon/OSDMonitor: add newline
mgr/pg_autoscaler: drop undefined variable 'cr_name'

Reviewed-by: Sage Weil <sage@redhat.com>
5 years agoMerge PR #26779 into master
Sage Weil [Sat, 9 Mar 2019 15:27:56 +0000 (09:27 -0600)]
Merge PR #26779 into master

* refs/pull/26779/head:
mgr/status: fix 'ceph fs status' AttributeError

Reviewed-by: Jos Collin <jcollin@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Sebastian Wagner <swagner@suse.com>
5 years agosrc/tools/kvstore_tool.cc: Move bluestore start-up code 26828/head
Brad Hubbard [Fri, 8 Mar 2019 01:31:32 +0000 (11:31 +1000)]
src/tools/kvstore_tool.cc: Move bluestore start-up code

Move this code into a separate function to avoid ICE.

Fixes: http://tracker.ceph.com/issues/38633
Signed-off-by: Brad Hubbard <bhubbard@redhat.com>
5 years agoqa/standalone/scrub/osd-scrub-repair: fix unfound grep
Sage Weil [Sat, 9 Mar 2019 00:23:48 +0000 (18:23 -0600)]
qa/standalone/scrub/osd-scrub-repair: fix unfound grep

It's now "1/2 unfound":

             1/2 objects unfound (50.000%)

..presumably due to the rbd pool init creating the rbd_directory.

Signed-off-by: Sage Weil <sage@redhat.com>
5 years agoqa/standalone/ceph-helpers: fix test_wait_for_clean
Sage Weil [Sat, 9 Mar 2019 00:07:10 +0000 (18:07 -0600)]
qa/standalone/ceph-helpers: fix test_wait_for_clean

Signed-off-by: Sage Weil <sage@redhat.com>
5 years agoqa/standalone/ceph-helpers.sh: fix test_run_mon
Sage Weil [Fri, 8 Mar 2019 23:21:05 +0000 (17:21 -0600)]
qa/standalone/ceph-helpers.sh: fix test_run_mon

- Only create each osd once
- forget the first osdmap dump test; it's pointless

Signed-off-by: Sage Weil <sage@redhat.com>
5 years agoqa/standalone/osd/osd-rep-recov-eio: fix better
Sage Weil [Fri, 8 Mar 2019 23:06:05 +0000 (17:06 -0600)]
qa/standalone/osd/osd-rep-recov-eio: fix better

- no need for the default pool size
- no initial osds or it will collide with setup_osds later
- no need for rbd pool at all

Signed-off-by: Sage Weil <sage@redhat.com>
5 years agoqa: ignore slow metadata io wrn during osd thrash 26859/head
Patrick Donnelly [Fri, 8 Mar 2019 18:49:10 +0000 (10:49 -0800)]
qa: ignore slow metadata io wrn during osd thrash

Fixes: http://tracker.ceph.com/issues/38651
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
5 years agoMerge PR #26838 into master
Ilya Dryomov [Fri, 8 Mar 2019 17:44:21 +0000 (18:44 +0100)]
Merge PR #26838 into master

* refs/pull/26838/head:
librbd: avoid aggregate-initializing IsWriteOpVisitor

Reviewed-by: Ilya Dryomov <idryomov@gmail.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
5 years agoqa: krbd deep-flatten test 26858/head
Ilya Dryomov [Fri, 8 Mar 2019 15:33:37 +0000 (16:33 +0100)]
qa: krbd deep-flatten test

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
5 years agoMerge pull request #26826 from ceph/wip-simple-scan-all
Alfredo Deza [Fri, 8 Mar 2019 16:49:07 +0000 (11:49 -0500)]
Merge pull request #26826 from ceph/wip-simple-scan-all

ceph-volume: `simple scan` will now scan all running ceph-disk OSDs

Reviewed-by: Alfredo Deza <adeza@redhat.com>
5 years agoceph-volume: make systemctl.get_running_osds resilient to garbage output 26826/head
Andrew Schoen [Fri, 8 Mar 2019 15:53:08 +0000 (09:53 -0600)]
ceph-volume: make systemctl.get_running_osds resilient to garbage output

Makes systemd.systemctl.get_running_osds more resilient to output
that doesn't match what we're expecting to parse. This also handles
the case of a nonzero exit status by retuning an empty list.

Signed-off-by: Andrew Schoen <aschoen@redhat.com>
5 years agoceph-volume: remove obsoleted simple unit test
Andrew Schoen [Thu, 7 Mar 2019 18:38:02 +0000 (12:38 -0600)]
ceph-volume: remove obsoleted simple unit test

This test is no longer valid because `simple scan`
without any arguments will now attempt to scan all
running OSDs.

Signed-off-by: Andrew Schoen <aschoen@redhat.com>
5 years agoceph-volume: tox tests should ignore the tests/functional directory
Andrew Schoen [Thu, 7 Mar 2019 18:37:00 +0000 (12:37 -0600)]
ceph-volume: tox tests should ignore the tests/functional directory

Without this the functional testinfra tests try to run when we
are intending to only run the unit tests.

Signed-off-by: Andrew Schoen <aschoen@redhat.com>
5 years agoceph-volume: use `simple scan` and `simple activate --all` in tests
Andrew Schoen [Thu, 7 Mar 2019 17:57:03 +0000 (11:57 -0600)]
ceph-volume: use `simple scan` and `simple activate --all` in tests

In a couple of the functional tests for simple we should run `simple
scan` without any arguments and `simple activate --all` to increase
the test coverage. We want to test pointing at a diretory and scanning
all running OSDs.

Signed-off-by: Andrew Schoen <aschoen@redhat.com>
5 years agoceph-volume: update docs and help menu for `simple scan`
Andrew Schoen [Thu, 7 Mar 2019 17:10:19 +0000 (11:10 -0600)]
ceph-volume: update docs and help menu for `simple scan`

Signed-off-by: Andrew Schoen <aschoen@redhat.com>
5 years agoceph-volume: adds unit tests for systemd.systemctl.get_running_osd_ids
Andrew Schoen [Thu, 7 Mar 2019 16:01:40 +0000 (10:01 -0600)]
ceph-volume: adds unit tests for systemd.systemctl.get_running_osd_ids

Signed-off-by: Andrew Schoen <aschoen@redhat.com>
5 years agoceph-volume: update `simple scan` to scan all running OSDs
Andrew Schoen [Wed, 6 Mar 2019 22:52:33 +0000 (16:52 -0600)]
ceph-volume: update `simple scan` to scan all running OSDs

If no argument is passed to `ceph-volume simple scan` it will
inspect any running osds and scan them if they were created by
ceph-disk.

Signed-off-by: Andrew Schoen <aschoen@redhat.com>
5 years agoceph-volume: add systemd.systemctl.get_running_osd_ids
Andrew Schoen [Wed, 6 Mar 2019 22:51:06 +0000 (16:51 -0600)]
ceph-volume: add systemd.systemctl.get_running_osd_ids

This is used to retrieve a list of osd ids from the currently
running ceph-osd systemd units.

Signed-off-by: Andrew Schoen <aschoen@redhat.com>
5 years agoMerge pull request #26815 from tchaikov/wip-38584
Patrick Nawracay [Fri, 8 Mar 2019 15:24:18 +0000 (16:24 +0100)]
Merge pull request #26815 from tchaikov/wip-38584

doc/releases/nautilus: dashboard package notes

Reviewed-by: Laura Paduano <lpaduano@suse.com>
Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>
5 years agoMerge PR #26823 into master
Sage Weil [Fri, 8 Mar 2019 15:04:21 +0000 (09:04 -0600)]
Merge PR #26823 into master

* refs/pull/26823/head:
qa/suites: disable valgrind leak checks on ceph-mgr
mgr: skip shutdown and exit

Reviewed-by: Tim Serong <tserong@suse.com>
Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
5 years agolibrbd: avoid aggregate-initializing IsWriteOpVisitor 26838/head
Willem Jan Withagen [Fri, 8 Mar 2019 13:59:09 +0000 (14:59 +0100)]
librbd: avoid aggregate-initializing IsWriteOpVisitor

Clang 8 complains about missing dtor, and this circumvents this problem.
```
/home/jenkins/workspace/ceph-master/src/librbd/io/ImageDispatchSpec.cc:136:48:
error: temporary of type 'boost::static_visitor<bool>' has protected
destructor
   return boost::apply_visitor(IsWriteOpVisitor{}, m_request);
                                                ^
/home/jenkins/workspace/ceph-master/src/librbd/io/ImageDispatchSpec.cc:152:28:
note: in instantiation of member function
'librbd::io::ImageDispatchSpec<librbd::ImageCtx>::is_write_op' requested
here
template class librbd::io::ImageDispatchSpec<librbd::ImageCtx>;
                            ^
/usr/local/include/boost/variant/static_visitor.hpp:53:5: note: declared
protected here
     ~static_visitor() = default;
     ^
1 error generated.
gmake[2]: *** [src/librbd/CMakeFiles/rbd_internal.dir/build.make:895:
src/librbd/CMakeFiles/rbd_internal.dir/io/ImageDispatchSpec.cc.o] Error 1
```

The details of the problem are explained in more detail in:
https://github.com/ceph/ceph/pull/26301/commits/09870d7a083913b5a3830ba8b0be04d0b0996aca

Signed-off-by: Willem Jan Withagen <wjw@digiware.nl>
5 years agoRevert "msg/async: msgr2: expected tags validation" 26849/head
Ricardo Dias [Fri, 8 Mar 2019 12:56:49 +0000 (12:56 +0000)]
Revert "msg/async: msgr2: expected tags validation"

This reverts commit e9ac3e0530b73ebb43db291d6562712b28c1e653.

Signed-off-by: Ricardo Dias <rdias@suse.com>
5 years agomsg/async/ProtocolV2: state machine verifications
Ricardo Dias [Fri, 8 Mar 2019 12:33:03 +0000 (12:33 +0000)]
msg/async/ProtocolV2: state machine verifications

Signed-off-by: Ricardo Dias <rdias@suse.com>
5 years agoMerge PR #26796 into master
Sage Weil [Fri, 8 Mar 2019 14:11:47 +0000 (08:11 -0600)]
Merge PR #26796 into master

* refs/pull/26796/head:
common/options: osd_memory_target et al are TYPE_SIZE

Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>
5 years agoMerge pull request #26827 from adamemerson/wip-doc-s3-op-acl
Casey Bodley [Fri, 8 Mar 2019 13:54:57 +0000 (08:54 -0500)]
Merge pull request #26827 from adamemerson/wip-doc-s3-op-acl

Document mappings of S3 Operations to ACL grants

Reviewed-by: Casey Bodley <cbodley@redhat.com>
5 years agoMerge PR #26832 into master
Sage Weil [Fri, 8 Mar 2019 13:04:35 +0000 (07:04 -0600)]
Merge PR #26832 into master

* refs/pull/26832/head:
msg/async/ProtocolV2: end scope of std::hex properly

Reviewed-by: Sage Weil <sage@redhat.com>
5 years agomon/OSDMonitor: add newline 26831/head
xie xingguo [Fri, 8 Mar 2019 08:02:28 +0000 (16:02 +0800)]
mon/OSDMonitor: add newline

To be in concert with other pool attributes..

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
5 years agomgr/pg_autoscaler: drop undefined variable 'cr_name'
xie xingguo [Fri, 8 Mar 2019 07:47:56 +0000 (15:47 +0800)]
mgr/pg_autoscaler: drop undefined variable 'cr_name'

```
a/pg_autoscaler/module.py:359: error: Name 'cr_name' is not defined
```

Actually _get_pool_status should have prevented any pool
with a zero-ed capacity from going this far, so it is safe
to simply drop the obviously redundant check here.

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
5 years agoMerge PR #26794 into master
Sage Weil [Fri, 8 Mar 2019 04:12:52 +0000 (22:12 -0600)]
Merge PR #26794 into master

* refs/pull/26794/head:
mon/MgrMonitor: only try to update always_on_modules if >= NAUTILUS
qa/standalone/mon/msgr-v2-transition: add some tests for enabling msgr v2
mon/MonmapMonitor: add 'ceph mon set-addrs <name> <addrvec>' command
Revert "mon/MonClient: disable ms_bind_msgr2 if NAUTILUS feature not set"
mon/OSDMonitor: use legacy_equals to compare osd addrs
msg/msg_types: make legacy_equals() symmetrical
mon/MDSMonitor: stop using get_orig_source_inst()

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>
5 years agoMerge PR #26770 into master
Sage Weil [Fri, 8 Mar 2019 04:10:12 +0000 (22:10 -0600)]
Merge PR #26770 into master

* refs/pull/26770/head:
qa/standalone/osd/osd-force-create-pg: create more pgs
qa/standalone: make sure an osd is running before create_rbd_pool

Reviewed-by: Mykola Golub <mgolub@suse.com>
5 years agoMerge PR #26504 into master
Sage Weil [Fri, 8 Mar 2019 03:58:01 +0000 (21:58 -0600)]
Merge PR #26504 into master

* refs/pull/26504/head:
osd: fixup OpTracker destruct assert, waiting_for_osdmap take ref

Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>
Reviewed-by: Sage Weil <sage@redhat.com>
5 years agomgr/status: fix 'ceph fs status' AttributeError 26779/head
Yanhu Cao [Wed, 6 Mar 2019 06:49:04 +0000 (14:49 +0800)]
mgr/status: fix 'ceph fs status' AttributeError

Fixes: http://tracker.ceph.com/issues/38122
Signed-off-by: Yanhu Cao <gmayyyha@gmail.com>
5 years agoMerge PR #26803 into master
Sage Weil [Thu, 7 Mar 2019 22:59:24 +0000 (16:59 -0600)]
Merge PR #26803 into master

* refs/pull/26803/head:
msg/async/ProtocolV1: fix locking around authorizer_buf

Reviewed-by: Ricardo Dias <rdias@suse.com>
5 years agoMerge PR #26781 into master
Patrick Donnelly [Thu, 7 Mar 2019 22:40:16 +0000 (14:40 -0800)]
Merge PR #26781 into master

* refs/pull/26781/head:
mds: avoid calling clientreplay_done() prematurely

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
5 years agoMerge PR #26773 into master
Patrick Donnelly [Thu, 7 Mar 2019 22:38:48 +0000 (14:38 -0800)]
Merge PR #26773 into master

* refs/pull/26773/head:
mds: refactor subtree accessors

Reviewed-by: Zheng Yan <zyan@redhat.com>
5 years agomon/MgrMonitor: only try to update always_on_modules if >= NAUTILUS 26794/head
Sage Weil [Thu, 7 Mar 2019 19:42:34 +0000 (13:42 -0600)]
mon/MgrMonitor: only try to update always_on_modules if >= NAUTILUS

Pre-nautilus mgrmap doesn't encode have this field, so we'll just loop.

Signed-off-by: Sage Weil <sage@redhat.com>
5 years agoqa/standalone/mon/msgr-v2-transition: add some tests for enabling msgr v2
Sage Weil [Wed, 6 Mar 2019 22:23:08 +0000 (16:23 -0600)]
qa/standalone/mon/msgr-v2-transition: add some tests for enabling msgr v2

Signed-off-by: Sage Weil <sage@redhat.com>
5 years agomon/MonmapMonitor: add 'ceph mon set-addrs <name> <addrvec>' command
Sage Weil [Wed, 6 Mar 2019 22:21:31 +0000 (16:21 -0600)]
mon/MonmapMonitor: add 'ceph mon set-addrs <name> <addrvec>' command

This lets us explicitly adjust the IPs/ports a monitor binds to.

Signed-off-by: Sage Weil <sage@redhat.com>
5 years agoRevert "mon/MonClient: disable ms_bind_msgr2 if NAUTILUS feature not set"
Sage Weil [Wed, 6 Mar 2019 17:54:43 +0000 (11:54 -0600)]
Revert "mon/MonClient: disable ms_bind_msgr2 if NAUTILUS feature not set"

This reverts commit f3ddb1c9b88067b1a4b54d23d825ca4d62e10586.

That behavior complicated the process of moving to v1 -> v2+v1 (extra
daemon restart, for example).  Also, it wasn't sufficient: there are lots
of other reasons besides the mon version that could lead to a mismatch
between what we bind to locally and what is published in the cluster
maps.  Better to embrace that reality and address those other issues.

Signed-off-by: Sage Weil <sage@redhat.com>
5 years agomon/OSDMonitor: use legacy_equals to compare osd addrs
Sage Weil [Wed, 6 Mar 2019 17:22:18 +0000 (11:22 -0600)]
mon/OSDMonitor: use legacy_equals to compare osd addrs

The OSDs may bind to v2+v1 due to ms_bind_msgr2, but we may only use the
v1 address in the OSDMap due to

 - the osd->mon connection being v1 (mon isn't using v2 yet)
 - the osd require_osd_release < nautilus
 - ???

Conversely, the OSD may end up with a v2+v1 address in the osdmap, but
may connect to the current monitor via v1 only, due to the mon only being
bound to a v1 address.

If/when this happens, make sure we are doing a loose comparison of the
osdmap addrs vs the osd's addr they are connecting from.

Fixes: http://tracker.ceph.com/issues/38598
Signed-off-by: Sage Weil <sage@redhat.com>
5 years agomsg/msg_types: make legacy_equals() symmetrical
Sage Weil [Wed, 6 Mar 2019 17:05:07 +0000 (11:05 -0600)]
msg/msg_types: make legacy_equals() symmetrical

The prior version would return true for

 [v2,v1] vs [v2,v1]
 [v2,v1] vs [v1]

but not

 [v1] vs [v2,v1]

Now that final case is also true, which means a.legacy_equals(b) ==
b.legacy_equals(a) for any combination.  This is what you would expect
for an equality operator... even a fuzzy one.

Signed-off-by: Sage Weil <sage@redhat.com>
5 years agomon/MDSMonitor: stop using get_orig_source_inst()
Sage Weil [Wed, 6 Mar 2019 17:04:51 +0000 (11:04 -0600)]
mon/MDSMonitor: stop using get_orig_source_inst()

Use get_orig_source() + get_orig_source_addrs()

Signed-off-by: Sage Weil <sage@redhat.com>
5 years agodoc/radosgw: Document mappings of S3 Operations to ACL grants 26827/head
Adam C. Emerson [Thu, 7 Mar 2019 20:18:42 +0000 (15:18 -0500)]
doc/radosgw: Document mappings of S3 Operations to ACL grants

Since the ACL documentation is a bit sparse, let people know exactly
what operations they can expect to enable with a given ACL grant.

Fixes: https://tracker.ceph.com/issues/38523
Signed-off-by: Adam C. Emerson <aemerson@redhat.com>
5 years agoMerge PR #26802 into master
Sage Weil [Thu, 7 Mar 2019 19:49:58 +0000 (13:49 -0600)]
Merge PR #26802 into master

* refs/pull/26802/head:
qa/suites/upgrade/mimic-x/parallel: run master rados/test.sh

Reviewed-by: Yuri Weinstein <yweins@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>
5 years agotest/librbd: create shouldn't be called in DiscardTruncate{AssertExists,} 26825/head
Ilya Dryomov [Wed, 6 Mar 2019 14:43:12 +0000 (15:43 +0100)]
test/librbd: create shouldn't be called in DiscardTruncate{AssertExists,}

MockTestMemIoCtxImpl is a NiceMock which suppresses all "uninteresting
method call" warnings.  Add explicit Times(0) expectations.

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
5 years agolibrbd: don't do create+truncate for discards with copyup
Ilya Dryomov [Wed, 6 Mar 2019 14:43:12 +0000 (15:43 +0100)]
librbd: don't do create+truncate for discards with copyup

With copyup, create is unnecessary because the object will be created
as a result of the copyup.  create's place will be taken by assert_exists
from write_object().

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
5 years agotest/librbd: observe the object map interaction in DiscardRemoveTruncate
Ilya Dryomov [Wed, 6 Mar 2019 14:43:12 +0000 (15:43 +0100)]
test/librbd: observe the object map interaction in DiscardRemoveTruncate

Undo the fixup made in merge commit a7351ce449bb ("Merge pull request
#20809 from idryomov/wip-librbd-discard-create-truncate").  We do want
to observe the object map interaction here -- specific changes towards
that were made in commits 513baf92ef01 ("test/librbd: clone with parent
features in DiscardRemoveTruncate") and 9d6ee9a64986 ("test/librbd:
expect_object_may_exist(..., false) in DiscardRemoveTruncate").

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
5 years agoqa/suites: disable valgrind leak checks on ceph-mgr 26823/head
Sage Weil [Thu, 7 Mar 2019 17:23:19 +0000 (11:23 -0600)]
qa/suites: disable valgrind leak checks on ceph-mgr

We've disabled the "clean" shutdown in ceph-mgr due to
https://tracker.ceph.com/issues/38621

Until then, no valgrind leak checks!

Signed-off-by: Sage Weil <sage@redhat.com>
5 years agomgr: skip shutdown and exit
Sage Weil [Thu, 7 Mar 2019 17:22:27 +0000 (11:22 -0600)]
mgr: skip shutdown and exit

The python subinterpreter teardown is broken; see
https://tracker.ceph.com/issues/38621

Instead, just exit with the appropriate error code.  That way we don't
crash and leave a core and ugly stack trace behind.

Users won't care, although our leak checking will.

Signed-off-by: Sage Weil <sage@redhat.com>
5 years agoMerge PR #26821 into master
Patrick Donnelly [Thu, 7 Mar 2019 16:48:05 +0000 (08:48 -0800)]
Merge PR #26821 into master

* refs/pull/26821/head:
mds,mgr,mon: warn future respawn() copy-pasters to set thread name

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
5 years agoMerge PR #26764 into master
Sage Weil [Thu, 7 Mar 2019 14:52:56 +0000 (08:52 -0600)]
Merge PR #26764 into master

* refs/pull/26764/head:
mgr: 'osd df' by specified class or (crush) name
mon/OSDMonitor: add 'osd crush get-device-class' command

5 years agomds,mgr,mon: warn future respawn() copy-pasters to set thread name 26821/head
Sage Weil [Thu, 7 Mar 2019 14:50:53 +0000 (08:50 -0600)]
mds,mgr,mon: warn future respawn() copy-pasters to set thread name

Signed-off-by: Sage Weil <sage@redhat.com>
5 years agocommon/options: osd_memory_target et al are TYPE_SIZE 26796/head
Sage Weil [Wed, 6 Mar 2019 19:28:57 +0000 (13:28 -0600)]
common/options: osd_memory_target et al are TYPE_SIZE

This makes us interpret "2G" and 2 GiB, not 2 billion bytes.

Signed-off-by: Sage Weil <sage@redhat.com>
5 years agodoc/mgr/dashboard: note dashboard as a separate package 26815/head
Kefu Chai [Thu, 7 Mar 2019 14:31:17 +0000 (22:31 +0800)]
doc/mgr/dashboard: note dashboard as a separate package

Fixes: https://tracker.ceph.com/issues/38584
Signed-off-by: Kefu Chai <kchai@redhat.com>
5 years agodoc/releases/nautilus: dashboard package notes
Kefu Chai [Thu, 7 Mar 2019 12:08:44 +0000 (20:08 +0800)]
doc/releases/nautilus: dashboard package notes

Fixes: https://tracker.ceph.com/issues/38584
Signed-off-by: Kefu Chai <kchai@redhat.com>
5 years agoMerge pull request #26743 from votdev/cleanup
Sebastian Wagner [Thu, 7 Mar 2019 13:14:51 +0000 (14:14 +0100)]
Merge pull request #26743 from votdev/cleanup

mgr/dashboard: Cleanup Python code

Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>
Reviewed-by: Tatjana Dehler <tdehler@suse.com>
5 years agoMerge PR #26797 into master
Sage Weil [Thu, 7 Mar 2019 12:36:04 +0000 (06:36 -0600)]
Merge PR #26797 into master

* refs/pull/26797/head:
ceph-mon: reset process name to 'ceph-mon' on startup

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: João Eduardo Luís <joao@suse.de>
5 years agoMerge PR #26795 into master
Sage Weil [Thu, 7 Mar 2019 12:35:49 +0000 (06:35 -0600)]
Merge PR #26795 into master

* refs/pull/26795/head:
mon/MgrStatMonitor: fix typo

Reviewed-by: Neha Ojha <nojha@redhat.com>
5 years agoMerge PR #26027 into master
Sage Weil [Thu, 7 Mar 2019 12:32:30 +0000 (06:32 -0600)]
Merge PR #26027 into master

* refs/pull/26027/head:
osd,mon,mds: support get|set tcmalloc release rate

Reviewed-by: Sage Weil <sage@redhat.com>
5 years agoMerge PR #26674 into master
Sage Weil [Thu, 7 Mar 2019 12:31:03 +0000 (06:31 -0600)]
Merge PR #26674 into master

* refs/pull/26674/head:
os/bluestore: log long aio operations in KernelDevice
os/bluestore: treat bdev_debug_aio_suicide_timeout==0 as no suicide
os/bluestore: track omap iterator latencies.
os/bluestore: log slow operations/long lasting states

Reviewed-by: Sage Weil <sage@redhat.com>
5 years agoMerge PR #26718 into master
Sage Weil [Thu, 7 Mar 2019 12:30:46 +0000 (06:30 -0600)]
Merge PR #26718 into master

* refs/pull/26718/head:
rpm: put numpy and scipy deps in distro conditionals

Reviewed-by: Kefu Chai <kchai@redhat.com>
5 years agoMerge PR #26725 into master
Sage Weil [Thu, 7 Mar 2019 12:30:29 +0000 (06:30 -0600)]
Merge PR #26725 into master

* refs/pull/26725/head:
doc/releases/nautilus: ask users to opt in to telemetry
doc/mgr/telemtry: update docs
mgr/telemetry: drop config-set and config-show; add just show
mgr/telemetry: make 'telemetry show' readable by a human
mgr/telemetry: add 'telemetry on' and 'telemetry off' commands
mgr/telemetry: off by default

Reviewed-by: Wido den Hollander <wido@42on.com>
Reviewed-by: Dan Mick <dmick@redhat.com>
5 years agoMerge pull request #26716 from rjfd/wip-dashboard-ganesha-docs
Ricardo Dias [Thu, 7 Mar 2019 12:10:37 +0000 (12:10 +0000)]
Merge pull request #26716 from rjfd/wip-dashboard-ganesha-docs

mgr/dashboard: nfs-ganesha: controller API documentation

Reviewed-by: Jeff Layton <jlayton@redhat.com>
Reviewed-by: Lenz Grimmer <lgrimmer@suse.com>
5 years agomsg/async/ProtocolV2: end scope of std::hex properly 26832/head
xie xingguo [Thu, 7 Mar 2019 11:49:29 +0000 (19:49 +0800)]
msg/async/ProtocolV2: end scope of std::hex properly

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
5 years agoMerge PR #26775 into master
Ilya Dryomov [Thu, 7 Mar 2019 11:19:36 +0000 (12:19 +0100)]
Merge PR #26775 into master

* refs/pull/26775/head:
qa/suites/krbd/wac: bluestore snippet is placed incorrectly

Reviewed-by: Mike Christie <mchristi@redhat.com>
5 years agoMerge pull request #26778 from xiexingguo/wip-fix-wrong-lpr
Xie Xingguo [Thu, 7 Mar 2019 08:40:59 +0000 (16:40 +0800)]
Merge pull request #26778 from xiexingguo/wip-fix-wrong-lpr

osd/PrimaryLogPG: fix last_peering_reset checking on manifest flushing

Reviewed-by: Myoungwon Oh <ommw@sk.com>
5 years agomsg/async/ProtocolV1: fix locking around authorizer_buf 26803/head
Sage Weil [Wed, 6 Mar 2019 23:40:48 +0000 (17:40 -0600)]
msg/async/ProtocolV1: fix locking around authorizer_buf

Fix two problems:

- we are accessing authorizer_buf without the connection lock, and
  under the lock we are modifying it (in connect()).
- if we receive two connect_msg's with a different length, we won't
  have a buffer that's large enough.

Fixes: http://tracker.ceph.com/issues/38524
Signed-off-by: Sage Weil <sage@redhat.com>
5 years agoqa/suites/upgrade/mimic-x/parallel: run master rados/test.sh 26802/head
Sage Weil [Wed, 6 Mar 2019 22:50:12 +0000 (16:50 -0600)]
qa/suites/upgrade/mimic-x/parallel: run master rados/test.sh

We rename ceph_test_rados_api_tier to add _pp, so the mimic version doesn't
work.  And in any case, at this stage the client host has master installed.

Signed-off-by: Sage Weil <sage@redhat.com>
5 years agoqa/standalone/osd/osd-force-create-pg: create more pgs 26770/head
Sage Weil [Wed, 6 Mar 2019 18:15:42 +0000 (12:15 -0600)]
qa/standalone/osd/osd-force-create-pg: create more pgs

Avoid warnings about too few pgs.

Signed-off-by: Sage Weil <sage@redhat.com>
5 years agoqa/standalone: make sure an osd is running before create_rbd_pool
Sage Weil [Tue, 5 Mar 2019 15:19:38 +0000 (09:19 -0600)]
qa/standalone: make sure an osd is running before create_rbd_pool

'rbd pool init' now does IO.  Drop the pool, or change the pool size to 1.

Fixes: http://tracker.ceph.com/issues/38585
Signed-off-by: Sage Weil <sage@redhat.com>
5 years agobuild: LLVM ld does not like the versioning scheme 26801/head
Willem Jan Withagen [Wed, 6 Mar 2019 22:23:33 +0000 (23:23 +0100)]
build: LLVM ld does not like the versioning scheme

There is a few essentail differences between the way versioning
and a version-map is interpreted for binutils ld and LLVM ld.
The easy way out for the time being is to use the one that works.

Signed-off-by: Willem Jan Withagen <wjw@digiware.nl>
5 years agoceph-mon: reset process name to 'ceph-mon' on startup 26797/head
Sage Weil [Wed, 6 Mar 2019 19:49:50 +0000 (13:49 -0600)]
ceph-mon: reset process name to 'ceph-mon' on startup

After a respawn the process name ends up as 'exe' (because we do execve
on /proc/$pid/exe).  Reset to 'ceph-mon' so that killall works.

Fixes: http://tracker.ceph.com/issues/38604
Signed-off-by: Sage Weil <sage@redhat.com>
5 years agomon/MgrStatMonitor: fix typo 26795/head
Sage Weil [Wed, 6 Mar 2019 19:27:41 +0000 (13:27 -0600)]
mon/MgrStatMonitor: fix typo

Signed-off-by: Sage Weil <sage@redhat.com>
5 years agoMerge pull request #26633 from jtlayton/wip-nfs-scale
Sebastian Wagner [Wed, 6 Mar 2019 18:08:48 +0000 (19:08 +0100)]
Merge pull request #26633 from jtlayton/wip-nfs-scale

mgr/orchestrator: Allow the orchestrator to scale the NFS server count

Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>
5 years agoMerge PR #26748 into master
Sage Weil [Wed, 6 Mar 2019 15:22:42 +0000 (09:22 -0600)]
Merge PR #26748 into master

* refs/pull/26748/head:
ceph_test_rados_[a]io_pp: add test verifying read-into-bufferlist behavior

Reviewed-by: Neha Ojha <nojha@redhat.com>
5 years agorpm: put numpy and scipy deps in distro conditionals 26718/head
Nathan Cutler [Fri, 1 Mar 2019 12:54:07 +0000 (13:54 +0100)]
rpm: put numpy and scipy deps in distro conditionals

Without this patch, attempts to install the ceph-mgr-diskprediction-local RPM
fail on SUSE platforms with the following error:

can't install ceph-mgr-diskprediction-local-14.1.0.402+ga396e8bf3b-3742.1.noarch:
  nothing provides numpy needed by ceph-mgr-diskprediction-local-14.1.0.402+ga396e8bf3b-3742.1.noarch
  nothing provides scipy needed by ceph-mgr-diskprediction-local-14.1.0.402+ga396e8bf3b-3742.1.noarch

Also take into account package naming differences between Fedora and
RHEL/CentOS.

Signed-off-by: Kefu Chai <kchai@redhat.com>
Signed-off-by: Nathan Cutler <ncutler@suse.com>
5 years agomgr/orchestrator: allow scaling the NFS server count up and down 26633/head
Jeff Layton [Mon, 25 Feb 2019 14:21:08 +0000 (09:21 -0500)]
mgr/orchestrator: allow scaling the NFS server count up and down

Add a new 'ceph orchestrator nfs update' command that will take the
NFS clustername and a new count as arguments. That will get translated
to a StatelessServiceSpec and passed to update_stateless_service.

Also, add the necessary stubs to the test_orchestrator and the CLI
QA test.

Signed-off-by: Jeff Layton <jlayton@redhat.com>
5 years agomgr/rook: allow scaling nfs count
Jeff Layton [Mon, 25 Feb 2019 14:27:02 +0000 (09:27 -0500)]
mgr/rook: allow scaling nfs count

Allow rook to handle scaling the NFS server count up and down in an NFS
cluster. We just manifest these changes as change to the
spec.server.active field in the CRD.

Signed-off-by: Jeff Layton <jlayton@redhat.com>
5 years agoorchestrator: drop id_ argument from update_stateless_service
Jeff Layton [Thu, 28 Feb 2019 17:33:04 +0000 (12:33 -0500)]
orchestrator: drop id_ argument from update_stateless_service

...we have a "name" field in StatelessServiceSpec.

Signed-off-by: Jeff Layton <jlayton@redhat.com>
5 years agomgr/orchestrator: just keep a single count value in StatelessServiceSpec
Jeff Layton [Tue, 26 Feb 2019 19:53:45 +0000 (14:53 -0500)]
mgr/orchestrator: just keep a single count value in StatelessServiceSpec

We currently have a min_size/max_size values in here, but we don't have
any orchestrators that can take advantage of two values. Let's just keep
a simple count for now, until we do.

Signed-off-by: Jeff Layton <jlayton@redhat.com>
5 years agoMerge pull request #26388 from tspmelo/wip-iscsi-no-auth-ui
Ricardo Marques [Wed, 6 Mar 2019 12:11:05 +0000 (12:11 +0000)]
Merge pull request #26388 from tspmelo/wip-iscsi-no-auth-ui

mgr/dashboard: Add UI for disabling ACL authentication

Reviewed-by: Ricardo Marques <rimarques@suse.com>
5 years agoMerge pull request #26382 from ricardoasmarques/no-auth
Volker Theile [Wed, 6 Mar 2019 11:45:33 +0000 (12:45 +0100)]
Merge pull request #26382 from ricardoasmarques/no-auth

mgr/dashboard: iSCSI - Add support for disabling ACL authentication (backend)

Reviewed-by: Volker Theile <vtheile@suse.com>
5 years agoMerge pull request #26783 from cyx1231st/wip-fix-async-echo
Kefu Chai [Wed, 6 Mar 2019 10:15:09 +0000 (18:15 +0800)]
Merge pull request #26783 from cyx1231st/wip-fix-async-echo

test/crimson: add dummy_auth to test_async_echo

Reviewed-by: Kefu Chai <kchai@redhat.com>
5 years agotest/crimson: add dummy_auth to test_async_echo 26783/head
Yingxin Cheng [Wed, 6 Mar 2019 09:29:04 +0000 (17:29 +0800)]
test/crimson: add dummy_auth to test_async_echo

fix echo between async-msgrs

Signed-off-by: Yingxin Cheng <yingxincheng@gmail.com>