]> git.apps.os.sepia.ceph.com Git - ceph.git/log
ceph.git
6 years agomgr/deepsea: always use 'password' parameter for salt-api auth 26904/head
Tim Serong [Tue, 12 Mar 2019 09:07:49 +0000 (20:07 +1100)]
mgr/deepsea: always use 'password' parameter for salt-api auth

Prior to https://github.com/saltstack/salt/commit/71d5601507, the
salt-api expected the password to be sent using the 'sharedsecret'
parameter if using shared secrets, and the 'password' parameter
for other authentication types.  The above commit unifies this so
that we always only need to use the 'password' parameter.

Signed-off-by: Tim Serong <tserong@suse.com>
6 years agoMerge PR #26763 into master
Sage Weil [Mon, 11 Mar 2019 22:29:24 +0000 (17:29 -0500)]
Merge PR #26763 into master

* refs/pull/26763/head:
mgr/diskprediction_cloud: Fixed cannot import string.maketrans in the python 3.x

Reviewed-by: Sebastian Wagner <swagner@suse.com>
6 years agoMerge PR #26885 into master
Sage Weil [Mon, 11 Mar 2019 20:36:34 +0000 (15:36 -0500)]
Merge PR #26885 into master

* refs/pull/26885/head:
doc/releases/nautilus: add msgr2 refs

Reviewed-by: Neha Ojha <nojha@redhat.com>
6 years agoMerge PR #26736 into master
Sage Weil [Mon, 11 Mar 2019 20:33:14 +0000 (15:33 -0500)]
Merge PR #26736 into master

* refs/pull/26736/head:
mgr: push localized config handling into BaseMgr*Module
mgr: Fix broken get_localized_module_option function

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
6 years agoMerge PR #26856 into master
Sage Weil [Mon, 11 Mar 2019 20:01:15 +0000 (15:01 -0500)]
Merge PR #26856 into master

* refs/pull/26856/head:
os/bluestore: make osd_memory_limit default to .8x the cgroup limit
os/bluestore: observe osd_memory_target config options
common/config: set osd_memory_target based on POD_MEMORY_REQUEST
common/util: include group limit in daemon metadata
common/util: get_cgroup_memory_limit() helper
common/config: pass entity_type down into parse_env()

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Sébastien Han <seb@redhat.com>
6 years agoMerge PR #26879 into master
Sage Weil [Mon, 11 Mar 2019 20:00:56 +0000 (15:00 -0500)]
Merge PR #26879 into master

* refs/pull/26879/head:
vstart.sh: make 'db' volume 1G instead of 64M
os/bluestore: make bluefs spillover messages nicer

Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>
Reviewed-by: Igor Fedotov <ifedotov@suse.com>
6 years agoMerge pull request #26878 from dillaman/wip-pool-init-validation
Mykola Golub [Mon, 11 Mar 2019 17:41:48 +0000 (19:41 +0200)]
Merge pull request #26878 from dillaman/wip-pool-init-validation

librbd: bypass pool validation if "rbd_validate_pool" is false

Reviewed-by: Mykola Golub <mgolub@suse.com>
6 years agoMerge PR #26888 into master
Sage Weil [Mon, 11 Mar 2019 17:37:06 +0000 (12:37 -0500)]
Merge PR #26888 into master

* refs/pull/26888/head:
doc/releases/nautilus: add 'ceph-volume simple scan' to upgrade instructions

Reviewed-by: Alfredo Deza <adeza@redhat.com>
Reviewed-by: Andrew Schoen <aschoen@redhat.com>
6 years agoos/bluestore: make osd_memory_limit default to .8x the cgroup limit 26856/head
Sage Weil [Fri, 8 Mar 2019 19:16:19 +0000 (13:16 -0600)]
os/bluestore: make osd_memory_limit default to .8x the cgroup limit

We do this at runtime in _set_cache_sizes so that changes to these values
at runtime will be picked up.

Signed-off-by: Sage Weil <sage@redhat.com>
6 years agomgr: push localized config handling into BaseMgr*Module 26736/head
Sage Weil [Fri, 8 Mar 2019 14:48:39 +0000 (08:48 -0600)]
mgr: push localized config handling into BaseMgr*Module

We need both (1) the raw key name (in order to look up the correct option
name and transform the result into the correct type) and (2) the optional
prefix (in order to look up both possible keys).  This simplifies the
mgr_module implementation.

There is a weird mismatch between the way that BaseMgrModule and
BaseMgrStandbyModule implement this method.  I suspect they could be
unified to work the same, but I'll leave that for another day.

Signed-off-by: Sage Weil <sage@redhat.com>
6 years agomgr: Fix broken get_localized_module_option function
Volker Theile [Mon, 4 Mar 2019 08:55:40 +0000 (09:55 +0100)]
mgr: Fix broken get_localized_module_option function

Fixes: https://tracker.ceph.com/issues/38560
Signed-off-by: Volker Theile <vtheile@suse.com>
6 years agoMerge PR #26818 into master
Patrick Donnelly [Mon, 11 Mar 2019 16:21:43 +0000 (09:21 -0700)]
Merge PR #26818 into master

* refs/pull/26818/head:
qa/cephfs: relax min_caps_per_client check

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
6 years agodoc/releases/nautilus: add msgr2 refs 26885/head
Sage Weil [Mon, 11 Mar 2019 13:52:28 +0000 (08:52 -0500)]
doc/releases/nautilus: add msgr2 refs

Signed-off-by: Sage Weil <sage@redhat.com>
6 years agoMerge pull request #26883 from rhcs-dashboard/wip-38657-master
Lenz Grimmer [Mon, 11 Mar 2019 14:56:07 +0000 (15:56 +0100)]
Merge pull request #26883 from rhcs-dashboard/wip-38657-master

mgr/dashboard: Change 'Client Recovery' title

Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Volker Theile <vtheile@suse.com>
6 years agoMerge pull request #26721 from dillaman/wip-38538
Mykola Golub [Mon, 11 Mar 2019 14:44:42 +0000 (16:44 +0200)]
Merge pull request #26721 from dillaman/wip-38538

librbd: improve object map performance under high IOPS workloads

Reviewed-by: Mykola Golub <mgolub@suse.com>
6 years agodoc/releases/nautilus: add 'ceph-volume simple scan' to upgrade instructions 26888/head
Sage Weil [Mon, 11 Mar 2019 14:14:02 +0000 (09:14 -0500)]
doc/releases/nautilus: add 'ceph-volume simple scan' to upgrade instructions

Signed-off-by: Sage Weil <sage@redhat.com>
6 years agoMerge pull request #26799 from wjwithagen/wjw-fix-/src/test/rgw/CMakeLists.txt
Casey Bodley [Mon, 11 Mar 2019 14:40:24 +0000 (10:40 -0400)]
Merge pull request #26799 from wjwithagen/wjw-fix-/src/test/rgw/CMakeLists.txt

build/rgw: unittest_rgw_dmclock_scheduler does not need Boost_LIBRARIES

Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
6 years agoMerge PR #26877 into master
Sage Weil [Mon, 11 Mar 2019 14:27:13 +0000 (09:27 -0500)]
Merge PR #26877 into master

* refs/pull/26877/head:
mgr/MgrStandby: exit with 0 result code, as if we'd done an orderly shutdown

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>
6 years agoos/bluestore: observe osd_memory_target config options
Sage Weil [Fri, 8 Mar 2019 18:22:36 +0000 (12:22 -0600)]
os/bluestore: observe osd_memory_target config options

Signed-off-by: Sage Weil <sage@redhat.com>
6 years agocommon/config: set osd_memory_target based on POD_MEMORY_REQUEST
Sage Weil [Fri, 8 Mar 2019 16:46:40 +0000 (10:46 -0600)]
common/config: set osd_memory_target based on POD_MEMORY_REQUEST

Signed-off-by: Sage Weil <sage@redhat.com>
6 years agoMerge PR #26858 into master
Ilya Dryomov [Mon, 11 Mar 2019 13:38:01 +0000 (14:38 +0100)]
Merge PR #26858 into master

* refs/pull/26858/head:
qa: krbd deep-flatten test
qa/suites/krbd: enable deep-flatten feature

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
6 years agomgr/MgrStandby: exit with 0 result code, as if we'd done an orderly shutdown 26877/head
Sage Weil [Sun, 10 Mar 2019 15:48:21 +0000 (10:48 -0500)]
mgr/MgrStandby: exit with 0 result code, as if we'd done an orderly shutdown

This matches the prior behavior with shutdown().  It also makes
teuthology happy.

Signed-off-by: Sage Weil <sage@redhat.com>
6 years agomgr/dashboard: Change 'Client Recovery' title 26883/head
Ernesto Puerta [Mon, 11 Mar 2019 12:38:02 +0000 (13:38 +0100)]
mgr/dashboard: Change 'Client Recovery' title

Rename 'Client Recovery' info-card title to 'Recovery Throughput' as:
- 'Client' is a misleading concept there, as long as all recovery is
server-side driven.
- No indication of the kind of metric (e.g.: IOPS, throughput, etc).

Fixes: https://tracker.ceph.com/issues/38657
Signed-off-by: Ernesto Puerta <epuertat@redhat.com>
6 years agoMerge pull request #26825 from idryomov/wip-copyup-truncate
Jason Dillaman [Mon, 11 Mar 2019 12:23:29 +0000 (08:23 -0400)]
Merge pull request #26825 from idryomov/wip-copyup-truncate

librbd: don't do create+truncate for discards with copyup

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
6 years agoMerge pull request #26867 from liewegas/wip-doc-msgr2
Xie Xingguo [Mon, 11 Mar 2019 00:36:48 +0000 (08:36 +0800)]
Merge pull request #26867 from liewegas/wip-doc-msgr2

doc/rados/configuration/msgr2: some documentation about msgr2

Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>
6 years agovstart.sh: make 'db' volume 1G instead of 64M 26879/head
Sage Weil [Sun, 10 Mar 2019 21:47:15 +0000 (16:47 -0500)]
vstart.sh: make 'db' volume 1G instead of 64M

64M is barely usable.

Signed-off-by: Sage Weil <sage@redhat.com>
6 years agoos/bluestore: make bluefs spillover messages nicer
Sage Weil [Sun, 10 Mar 2019 21:39:51 +0000 (16:39 -0500)]
os/bluestore: make bluefs spillover messages nicer

Was

 osd:50 spilled over to slow device: 1.3 GiB

Now

 osd.0 spilled over 1.7 GiB metadata from 'db' device (1000 MiB used of 1024 MiB) to slow device

Signed-off-by: Sage Weil <sage@redhat.com>
6 years agolibrbd: bypass pool validation if "rbd_validate_pool" is false 26878/head
Jason Dillaman [Sun, 10 Mar 2019 19:49:15 +0000 (15:49 -0400)]
librbd: bypass pool validation if "rbd_validate_pool" is false

This config option is already used by the create image state
machine. This should address a broken test case when validating
pool vs self-managed snapshots.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
6 years agoMerge pull request #26801 from wjwithagen/wjw-fix-use-binutils-ld
Jason Dillaman [Sun, 10 Mar 2019 18:24:56 +0000 (14:24 -0400)]
Merge pull request #26801 from wjwithagen/wjw-fix-use-binutils-ld

build: LLVM ld does not like the versioning scheme

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Jason Dillaman <dillaman@redhat.com>
6 years agoMerge PR #26859 into master
Patrick Donnelly [Sun, 10 Mar 2019 17:38:16 +0000 (10:38 -0700)]
Merge PR #26859 into master

* refs/pull/26859/head:
qa: ignore slow metadata io wrn during osd thrash

Reviewed-by: Yuri Weinstein <yweins@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>
6 years agolibrbd: improve object map performance under high IOPS workloads 26721/head
Jason Dillaman [Thu, 28 Feb 2019 21:43:27 +0000 (16:43 -0500)]
librbd: improve object map performance under high IOPS workloads

Do not zero-fill the BitVector's bitset prior to decoding the data.
Additionally, only read-update-modify the portions of the footer
that are potentially affected by the updated state.

Fixes: http://tracker.ceph.com/issues/38538
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
6 years agoMerge pull request #26876 from wjwithagen/wjw-fix-src/test/librbd/operation/test_mock...
Jason Dillaman [Sun, 10 Mar 2019 14:21:06 +0000 (10:21 -0400)]
Merge pull request #26876 from wjwithagen/wjw-fix-src/test/librbd/operation/test_mock_TrimRequest.cc

librbd: avoid aggregate-initializing any static_visitor

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
6 years agolibrbd: avoid aggregate-initializing any static_visitor 26876/head
Willem Jan Withagen [Sun, 10 Mar 2019 10:51:09 +0000 (11:51 +0100)]
librbd: avoid aggregate-initializing any static_visitor

This will result in an error complaining about a protected dtor.
also see:
    https://github.com/ceph/ceph/pull/26838

Signed-off-by: Willem Jan Withagen <wjw@digiware.nl>
6 years agodoc/rados/configuration/msgr2: some documentation about msgr2 26867/head
Sage Weil [Fri, 8 Mar 2019 22:52:38 +0000 (16:52 -0600)]
doc/rados/configuration/msgr2: some documentation about msgr2

This doesn't integrate very well into network-config.rst, mostly because
that document is horribly out of data and I don't know where to start.
:(

Signed-off-by: Sage Weil <sage@redhat.com>
6 years agoMerge PR #26849 into master
Sage Weil [Sat, 9 Mar 2019 23:38:27 +0000 (17:38 -0600)]
Merge PR #26849 into master

* refs/pull/26849/head:
Revert "msg/async: msgr2: expected tags validation"
msg/async/ProtocolV2: state machine verifications

Reviewed-by: Sage Weil <sage@redhat.com>
6 years agoMerge PR #26869 into master
Sage Weil [Sat, 9 Mar 2019 19:22:56 +0000 (13:22 -0600)]
Merge PR #26869 into master

* refs/pull/26869/head:
qa/standalone/erasure-code/test-erasure-eio.sh: still need to create rbd pool
qa/standalone/scrub/osd-scrub-repair: fix unfound grep
qa/standalone/ceph-helpers: fix test_wait_for_clean
qa/standalone/ceph-helpers.sh: fix test_run_mon
qa/standalone/osd/osd-rep-recov-eio: fix better

Reviewed-by: Kefu Chai <kchai@redhat.com>
6 years agoMerge PR #26713 into master
Patrick Donnelly [Sat, 9 Mar 2019 18:15:26 +0000 (10:15 -0800)]
Merge PR #26713 into master

* refs/pull/26713/head:
mds: wait for client to release shared cap when re-acquiring xlock

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
6 years agoMerge PR #26828 into master
Sage Weil [Sat, 9 Mar 2019 17:00:57 +0000 (11:00 -0600)]
Merge PR #26828 into master

* refs/pull/26828/head:
src/tools/kvstore_tool.cc: Move bluestore start-up code

Reviewed-by: Sage Weil <sage@redhat.com>
6 years agoqa/standalone/erasure-code/test-erasure-eio.sh: still need to create rbd pool 26869/head
Sage Weil [Sat, 9 Mar 2019 15:34:49 +0000 (09:34 -0600)]
qa/standalone/erasure-code/test-erasure-eio.sh: still need to create rbd pool

Signed-off-by: Sage Weil <sage@redhat.com>
6 years agoMerge PR #26660 into master
Sage Weil [Sat, 9 Mar 2019 15:29:33 +0000 (09:29 -0600)]
Merge PR #26660 into master

* refs/pull/26660/head:
include/intarith: enforce the same type for p2*() arguments

Reviewed-by: Sage Weil <sage@redhat.com>
6 years agoMerge PR #26831 into master
Sage Weil [Sat, 9 Mar 2019 15:28:18 +0000 (09:28 -0600)]
Merge PR #26831 into master

* refs/pull/26831/head:
mon/OSDMonitor: add newline
mgr/pg_autoscaler: drop undefined variable 'cr_name'

Reviewed-by: Sage Weil <sage@redhat.com>
6 years agoMerge PR #26779 into master
Sage Weil [Sat, 9 Mar 2019 15:27:56 +0000 (09:27 -0600)]
Merge PR #26779 into master

* refs/pull/26779/head:
mgr/status: fix 'ceph fs status' AttributeError

Reviewed-by: Jos Collin <jcollin@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Sebastian Wagner <swagner@suse.com>
6 years agosrc/tools/kvstore_tool.cc: Move bluestore start-up code 26828/head
Brad Hubbard [Fri, 8 Mar 2019 01:31:32 +0000 (11:31 +1000)]
src/tools/kvstore_tool.cc: Move bluestore start-up code

Move this code into a separate function to avoid ICE.

Fixes: http://tracker.ceph.com/issues/38633
Signed-off-by: Brad Hubbard <bhubbard@redhat.com>
6 years agoqa/standalone/scrub/osd-scrub-repair: fix unfound grep
Sage Weil [Sat, 9 Mar 2019 00:23:48 +0000 (18:23 -0600)]
qa/standalone/scrub/osd-scrub-repair: fix unfound grep

It's now "1/2 unfound":

             1/2 objects unfound (50.000%)

..presumably due to the rbd pool init creating the rbd_directory.

Signed-off-by: Sage Weil <sage@redhat.com>
6 years agoqa/standalone/ceph-helpers: fix test_wait_for_clean
Sage Weil [Sat, 9 Mar 2019 00:07:10 +0000 (18:07 -0600)]
qa/standalone/ceph-helpers: fix test_wait_for_clean

Signed-off-by: Sage Weil <sage@redhat.com>
6 years agoqa/standalone/ceph-helpers.sh: fix test_run_mon
Sage Weil [Fri, 8 Mar 2019 23:21:05 +0000 (17:21 -0600)]
qa/standalone/ceph-helpers.sh: fix test_run_mon

- Only create each osd once
- forget the first osdmap dump test; it's pointless

Signed-off-by: Sage Weil <sage@redhat.com>
6 years agoqa/standalone/osd/osd-rep-recov-eio: fix better
Sage Weil [Fri, 8 Mar 2019 23:06:05 +0000 (17:06 -0600)]
qa/standalone/osd/osd-rep-recov-eio: fix better

- no need for the default pool size
- no initial osds or it will collide with setup_osds later
- no need for rbd pool at all

Signed-off-by: Sage Weil <sage@redhat.com>
6 years agocommon/util: include group limit in daemon metadata
Sage Weil [Fri, 8 Mar 2019 17:27:30 +0000 (11:27 -0600)]
common/util: include group limit in daemon metadata

Signed-off-by: Sage Weil <sage@redhat.com>
6 years agocommon/util: get_cgroup_memory_limit() helper
Sage Weil [Fri, 8 Mar 2019 17:27:20 +0000 (11:27 -0600)]
common/util: get_cgroup_memory_limit() helper

Signed-off-by: Sage Weil <sage@redhat.com>
6 years agocommon/config: pass entity_type down into parse_env()
Sage Weil [Fri, 8 Mar 2019 15:35:22 +0000 (09:35 -0600)]
common/config: pass entity_type down into parse_env()

No functional change, but we'll need this shortly.

Signed-off-by: Sage Weil <sage@redhat.com>
6 years agoqa: ignore slow metadata io wrn during osd thrash 26859/head
Patrick Donnelly [Fri, 8 Mar 2019 18:49:10 +0000 (10:49 -0800)]
qa: ignore slow metadata io wrn during osd thrash

Fixes: http://tracker.ceph.com/issues/38651
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
6 years agoMerge PR #26838 into master
Ilya Dryomov [Fri, 8 Mar 2019 17:44:21 +0000 (18:44 +0100)]
Merge PR #26838 into master

* refs/pull/26838/head:
librbd: avoid aggregate-initializing IsWriteOpVisitor

Reviewed-by: Ilya Dryomov <idryomov@gmail.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
6 years agobuild/rgw: unittest_rgw_dmclock_scheduler does not need Boost_LIBRARIES 26799/head
Willem Jan Withagen [Wed, 6 Mar 2019 21:53:21 +0000 (22:53 +0100)]
build/rgw: unittest_rgw_dmclock_scheduler does not need Boost_LIBRARIES

Otherwise linking could error like:
```
/usr/local/bin/ld: /usr/local/lib/libboost_python27.so: undefined reference to `PyUnicodeUCS4_FromEncodedObject'
/usr/local/bin/ld: /usr/local/lib/libboost_python27.so: undefined reference to `PyNumber_InPlaceDivide'
........
/usr/local/bin/ld: /usr/local/lib/libboost_python27.so: undefined reference to `PyStaticMethod_Type'
/usr/local/bin/ld: /usr/local/lib/libboost_python27.so: undefined reference to `PyTuple_Size'
c++: error: linker command failed with exit code 1 (use -v to see invocation)
gmake[2]: *** [src/test/rgw/CMakeFiles/unittest_rgw_dmclock_scheduler.dir/build.make:147: bin/unittest_rgw_dmclock_scheduler] Error 1
```

Signed-off-by: Willem Jan Withagen <wjw@digiware.nl>
6 years agoqa: krbd deep-flatten test 26858/head
Ilya Dryomov [Fri, 8 Mar 2019 15:33:37 +0000 (16:33 +0100)]
qa: krbd deep-flatten test

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
6 years agoMerge pull request #26826 from ceph/wip-simple-scan-all
Alfredo Deza [Fri, 8 Mar 2019 16:49:07 +0000 (11:49 -0500)]
Merge pull request #26826 from ceph/wip-simple-scan-all

ceph-volume: `simple scan` will now scan all running ceph-disk OSDs

Reviewed-by: Alfredo Deza <adeza@redhat.com>
6 years agoceph-volume: make systemctl.get_running_osds resilient to garbage output 26826/head
Andrew Schoen [Fri, 8 Mar 2019 15:53:08 +0000 (09:53 -0600)]
ceph-volume: make systemctl.get_running_osds resilient to garbage output

Makes systemd.systemctl.get_running_osds more resilient to output
that doesn't match what we're expecting to parse. This also handles
the case of a nonzero exit status by retuning an empty list.

Signed-off-by: Andrew Schoen <aschoen@redhat.com>
6 years agoceph-volume: remove obsoleted simple unit test
Andrew Schoen [Thu, 7 Mar 2019 18:38:02 +0000 (12:38 -0600)]
ceph-volume: remove obsoleted simple unit test

This test is no longer valid because `simple scan`
without any arguments will now attempt to scan all
running OSDs.

Signed-off-by: Andrew Schoen <aschoen@redhat.com>
6 years agoceph-volume: tox tests should ignore the tests/functional directory
Andrew Schoen [Thu, 7 Mar 2019 18:37:00 +0000 (12:37 -0600)]
ceph-volume: tox tests should ignore the tests/functional directory

Without this the functional testinfra tests try to run when we
are intending to only run the unit tests.

Signed-off-by: Andrew Schoen <aschoen@redhat.com>
6 years agoceph-volume: use `simple scan` and `simple activate --all` in tests
Andrew Schoen [Thu, 7 Mar 2019 17:57:03 +0000 (11:57 -0600)]
ceph-volume: use `simple scan` and `simple activate --all` in tests

In a couple of the functional tests for simple we should run `simple
scan` without any arguments and `simple activate --all` to increase
the test coverage. We want to test pointing at a diretory and scanning
all running OSDs.

Signed-off-by: Andrew Schoen <aschoen@redhat.com>
6 years agoceph-volume: update docs and help menu for `simple scan`
Andrew Schoen [Thu, 7 Mar 2019 17:10:19 +0000 (11:10 -0600)]
ceph-volume: update docs and help menu for `simple scan`

Signed-off-by: Andrew Schoen <aschoen@redhat.com>
6 years agoceph-volume: adds unit tests for systemd.systemctl.get_running_osd_ids
Andrew Schoen [Thu, 7 Mar 2019 16:01:40 +0000 (10:01 -0600)]
ceph-volume: adds unit tests for systemd.systemctl.get_running_osd_ids

Signed-off-by: Andrew Schoen <aschoen@redhat.com>
6 years agoceph-volume: update `simple scan` to scan all running OSDs
Andrew Schoen [Wed, 6 Mar 2019 22:52:33 +0000 (16:52 -0600)]
ceph-volume: update `simple scan` to scan all running OSDs

If no argument is passed to `ceph-volume simple scan` it will
inspect any running osds and scan them if they were created by
ceph-disk.

Signed-off-by: Andrew Schoen <aschoen@redhat.com>
6 years agoceph-volume: add systemd.systemctl.get_running_osd_ids
Andrew Schoen [Wed, 6 Mar 2019 22:51:06 +0000 (16:51 -0600)]
ceph-volume: add systemd.systemctl.get_running_osd_ids

This is used to retrieve a list of osd ids from the currently
running ceph-osd systemd units.

Signed-off-by: Andrew Schoen <aschoen@redhat.com>
6 years agoMerge pull request #26815 from tchaikov/wip-38584
Patrick Nawracay [Fri, 8 Mar 2019 15:24:18 +0000 (16:24 +0100)]
Merge pull request #26815 from tchaikov/wip-38584

doc/releases/nautilus: dashboard package notes

Reviewed-by: Laura Paduano <lpaduano@suse.com>
Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>
6 years agoMerge PR #26823 into master
Sage Weil [Fri, 8 Mar 2019 15:04:21 +0000 (09:04 -0600)]
Merge PR #26823 into master

* refs/pull/26823/head:
qa/suites: disable valgrind leak checks on ceph-mgr
mgr: skip shutdown and exit

Reviewed-by: Tim Serong <tserong@suse.com>
Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
6 years agolibrbd: avoid aggregate-initializing IsWriteOpVisitor 26838/head
Willem Jan Withagen [Fri, 8 Mar 2019 13:59:09 +0000 (14:59 +0100)]
librbd: avoid aggregate-initializing IsWriteOpVisitor

Clang 8 complains about missing dtor, and this circumvents this problem.
```
/home/jenkins/workspace/ceph-master/src/librbd/io/ImageDispatchSpec.cc:136:48:
error: temporary of type 'boost::static_visitor<bool>' has protected
destructor
   return boost::apply_visitor(IsWriteOpVisitor{}, m_request);
                                                ^
/home/jenkins/workspace/ceph-master/src/librbd/io/ImageDispatchSpec.cc:152:28:
note: in instantiation of member function
'librbd::io::ImageDispatchSpec<librbd::ImageCtx>::is_write_op' requested
here
template class librbd::io::ImageDispatchSpec<librbd::ImageCtx>;
                            ^
/usr/local/include/boost/variant/static_visitor.hpp:53:5: note: declared
protected here
     ~static_visitor() = default;
     ^
1 error generated.
gmake[2]: *** [src/librbd/CMakeFiles/rbd_internal.dir/build.make:895:
src/librbd/CMakeFiles/rbd_internal.dir/io/ImageDispatchSpec.cc.o] Error 1
```

The details of the problem are explained in more detail in:
https://github.com/ceph/ceph/pull/26301/commits/09870d7a083913b5a3830ba8b0be04d0b0996aca

Signed-off-by: Willem Jan Withagen <wjw@digiware.nl>
6 years agoRevert "msg/async: msgr2: expected tags validation" 26849/head
Ricardo Dias [Fri, 8 Mar 2019 12:56:49 +0000 (12:56 +0000)]
Revert "msg/async: msgr2: expected tags validation"

This reverts commit e9ac3e0530b73ebb43db291d6562712b28c1e653.

Signed-off-by: Ricardo Dias <rdias@suse.com>
6 years agomsg/async/ProtocolV2: state machine verifications
Ricardo Dias [Fri, 8 Mar 2019 12:33:03 +0000 (12:33 +0000)]
msg/async/ProtocolV2: state machine verifications

Signed-off-by: Ricardo Dias <rdias@suse.com>
6 years agoMerge PR #26796 into master
Sage Weil [Fri, 8 Mar 2019 14:11:47 +0000 (08:11 -0600)]
Merge PR #26796 into master

* refs/pull/26796/head:
common/options: osd_memory_target et al are TYPE_SIZE

Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>
6 years agoMerge pull request #26827 from adamemerson/wip-doc-s3-op-acl
Casey Bodley [Fri, 8 Mar 2019 13:54:57 +0000 (08:54 -0500)]
Merge pull request #26827 from adamemerson/wip-doc-s3-op-acl

Document mappings of S3 Operations to ACL grants

Reviewed-by: Casey Bodley <cbodley@redhat.com>
6 years agoMerge PR #26832 into master
Sage Weil [Fri, 8 Mar 2019 13:04:35 +0000 (07:04 -0600)]
Merge PR #26832 into master

* refs/pull/26832/head:
msg/async/ProtocolV2: end scope of std::hex properly

Reviewed-by: Sage Weil <sage@redhat.com>
6 years agomon/OSDMonitor: add newline 26831/head
xie xingguo [Fri, 8 Mar 2019 08:02:28 +0000 (16:02 +0800)]
mon/OSDMonitor: add newline

To be in concert with other pool attributes..

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
6 years agomgr/pg_autoscaler: drop undefined variable 'cr_name'
xie xingguo [Fri, 8 Mar 2019 07:47:56 +0000 (15:47 +0800)]
mgr/pg_autoscaler: drop undefined variable 'cr_name'

```
a/pg_autoscaler/module.py:359: error: Name 'cr_name' is not defined
```

Actually _get_pool_status should have prevented any pool
with a zero-ed capacity from going this far, so it is safe
to simply drop the obviously redundant check here.

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
6 years agoMerge PR #26794 into master
Sage Weil [Fri, 8 Mar 2019 04:12:52 +0000 (22:12 -0600)]
Merge PR #26794 into master

* refs/pull/26794/head:
mon/MgrMonitor: only try to update always_on_modules if >= NAUTILUS
qa/standalone/mon/msgr-v2-transition: add some tests for enabling msgr v2
mon/MonmapMonitor: add 'ceph mon set-addrs <name> <addrvec>' command
Revert "mon/MonClient: disable ms_bind_msgr2 if NAUTILUS feature not set"
mon/OSDMonitor: use legacy_equals to compare osd addrs
msg/msg_types: make legacy_equals() symmetrical
mon/MDSMonitor: stop using get_orig_source_inst()

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>
6 years agoMerge PR #26770 into master
Sage Weil [Fri, 8 Mar 2019 04:10:12 +0000 (22:10 -0600)]
Merge PR #26770 into master

* refs/pull/26770/head:
qa/standalone/osd/osd-force-create-pg: create more pgs
qa/standalone: make sure an osd is running before create_rbd_pool

Reviewed-by: Mykola Golub <mgolub@suse.com>
6 years agoMerge PR #26504 into master
Sage Weil [Fri, 8 Mar 2019 03:58:01 +0000 (21:58 -0600)]
Merge PR #26504 into master

* refs/pull/26504/head:
osd: fixup OpTracker destruct assert, waiting_for_osdmap take ref

Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>
Reviewed-by: Sage Weil <sage@redhat.com>
6 years agomgr/status: fix 'ceph fs status' AttributeError 26779/head
Yanhu Cao [Wed, 6 Mar 2019 06:49:04 +0000 (14:49 +0800)]
mgr/status: fix 'ceph fs status' AttributeError

Fixes: http://tracker.ceph.com/issues/38122
Signed-off-by: Yanhu Cao <gmayyyha@gmail.com>
6 years agoMerge PR #26803 into master
Sage Weil [Thu, 7 Mar 2019 22:59:24 +0000 (16:59 -0600)]
Merge PR #26803 into master

* refs/pull/26803/head:
msg/async/ProtocolV1: fix locking around authorizer_buf

Reviewed-by: Ricardo Dias <rdias@suse.com>
6 years agoMerge PR #26781 into master
Patrick Donnelly [Thu, 7 Mar 2019 22:40:16 +0000 (14:40 -0800)]
Merge PR #26781 into master

* refs/pull/26781/head:
mds: avoid calling clientreplay_done() prematurely

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
6 years agoMerge PR #26773 into master
Patrick Donnelly [Thu, 7 Mar 2019 22:38:48 +0000 (14:38 -0800)]
Merge PR #26773 into master

* refs/pull/26773/head:
mds: refactor subtree accessors

Reviewed-by: Zheng Yan <zyan@redhat.com>
6 years agomon/MgrMonitor: only try to update always_on_modules if >= NAUTILUS 26794/head
Sage Weil [Thu, 7 Mar 2019 19:42:34 +0000 (13:42 -0600)]
mon/MgrMonitor: only try to update always_on_modules if >= NAUTILUS

Pre-nautilus mgrmap doesn't encode have this field, so we'll just loop.

Signed-off-by: Sage Weil <sage@redhat.com>
6 years agoqa/standalone/mon/msgr-v2-transition: add some tests for enabling msgr v2
Sage Weil [Wed, 6 Mar 2019 22:23:08 +0000 (16:23 -0600)]
qa/standalone/mon/msgr-v2-transition: add some tests for enabling msgr v2

Signed-off-by: Sage Weil <sage@redhat.com>
6 years agomon/MonmapMonitor: add 'ceph mon set-addrs <name> <addrvec>' command
Sage Weil [Wed, 6 Mar 2019 22:21:31 +0000 (16:21 -0600)]
mon/MonmapMonitor: add 'ceph mon set-addrs <name> <addrvec>' command

This lets us explicitly adjust the IPs/ports a monitor binds to.

Signed-off-by: Sage Weil <sage@redhat.com>
6 years agoRevert "mon/MonClient: disable ms_bind_msgr2 if NAUTILUS feature not set"
Sage Weil [Wed, 6 Mar 2019 17:54:43 +0000 (11:54 -0600)]
Revert "mon/MonClient: disable ms_bind_msgr2 if NAUTILUS feature not set"

This reverts commit f3ddb1c9b88067b1a4b54d23d825ca4d62e10586.

That behavior complicated the process of moving to v1 -> v2+v1 (extra
daemon restart, for example).  Also, it wasn't sufficient: there are lots
of other reasons besides the mon version that could lead to a mismatch
between what we bind to locally and what is published in the cluster
maps.  Better to embrace that reality and address those other issues.

Signed-off-by: Sage Weil <sage@redhat.com>
6 years agomon/OSDMonitor: use legacy_equals to compare osd addrs
Sage Weil [Wed, 6 Mar 2019 17:22:18 +0000 (11:22 -0600)]
mon/OSDMonitor: use legacy_equals to compare osd addrs

The OSDs may bind to v2+v1 due to ms_bind_msgr2, but we may only use the
v1 address in the OSDMap due to

 - the osd->mon connection being v1 (mon isn't using v2 yet)
 - the osd require_osd_release < nautilus
 - ???

Conversely, the OSD may end up with a v2+v1 address in the osdmap, but
may connect to the current monitor via v1 only, due to the mon only being
bound to a v1 address.

If/when this happens, make sure we are doing a loose comparison of the
osdmap addrs vs the osd's addr they are connecting from.

Fixes: http://tracker.ceph.com/issues/38598
Signed-off-by: Sage Weil <sage@redhat.com>
6 years agomsg/msg_types: make legacy_equals() symmetrical
Sage Weil [Wed, 6 Mar 2019 17:05:07 +0000 (11:05 -0600)]
msg/msg_types: make legacy_equals() symmetrical

The prior version would return true for

 [v2,v1] vs [v2,v1]
 [v2,v1] vs [v1]

but not

 [v1] vs [v2,v1]

Now that final case is also true, which means a.legacy_equals(b) ==
b.legacy_equals(a) for any combination.  This is what you would expect
for an equality operator... even a fuzzy one.

Signed-off-by: Sage Weil <sage@redhat.com>
6 years agomon/MDSMonitor: stop using get_orig_source_inst()
Sage Weil [Wed, 6 Mar 2019 17:04:51 +0000 (11:04 -0600)]
mon/MDSMonitor: stop using get_orig_source_inst()

Use get_orig_source() + get_orig_source_addrs()

Signed-off-by: Sage Weil <sage@redhat.com>
6 years agodoc/radosgw: Document mappings of S3 Operations to ACL grants 26827/head
Adam C. Emerson [Thu, 7 Mar 2019 20:18:42 +0000 (15:18 -0500)]
doc/radosgw: Document mappings of S3 Operations to ACL grants

Since the ACL documentation is a bit sparse, let people know exactly
what operations they can expect to enable with a given ACL grant.

Fixes: https://tracker.ceph.com/issues/38523
Signed-off-by: Adam C. Emerson <aemerson@redhat.com>
6 years agoMerge PR #26802 into master
Sage Weil [Thu, 7 Mar 2019 19:49:58 +0000 (13:49 -0600)]
Merge PR #26802 into master

* refs/pull/26802/head:
qa/suites/upgrade/mimic-x/parallel: run master rados/test.sh

Reviewed-by: Yuri Weinstein <yweins@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>
6 years agotest/librbd: create shouldn't be called in DiscardTruncate{AssertExists,} 26825/head
Ilya Dryomov [Wed, 6 Mar 2019 14:43:12 +0000 (15:43 +0100)]
test/librbd: create shouldn't be called in DiscardTruncate{AssertExists,}

MockTestMemIoCtxImpl is a NiceMock which suppresses all "uninteresting
method call" warnings.  Add explicit Times(0) expectations.

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
6 years agolibrbd: don't do create+truncate for discards with copyup
Ilya Dryomov [Wed, 6 Mar 2019 14:43:12 +0000 (15:43 +0100)]
librbd: don't do create+truncate for discards with copyup

With copyup, create is unnecessary because the object will be created
as a result of the copyup.  create's place will be taken by assert_exists
from write_object().

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
6 years agotest/librbd: observe the object map interaction in DiscardRemoveTruncate
Ilya Dryomov [Wed, 6 Mar 2019 14:43:12 +0000 (15:43 +0100)]
test/librbd: observe the object map interaction in DiscardRemoveTruncate

Undo the fixup made in merge commit a7351ce449bb ("Merge pull request
#20809 from idryomov/wip-librbd-discard-create-truncate").  We do want
to observe the object map interaction here -- specific changes towards
that were made in commits 513baf92ef01 ("test/librbd: clone with parent
features in DiscardRemoveTruncate") and 9d6ee9a64986 ("test/librbd:
expect_object_may_exist(..., false) in DiscardRemoveTruncate").

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
6 years agoqa/suites: disable valgrind leak checks on ceph-mgr 26823/head
Sage Weil [Thu, 7 Mar 2019 17:23:19 +0000 (11:23 -0600)]
qa/suites: disable valgrind leak checks on ceph-mgr

We've disabled the "clean" shutdown in ceph-mgr due to
https://tracker.ceph.com/issues/38621

Until then, no valgrind leak checks!

Signed-off-by: Sage Weil <sage@redhat.com>
6 years agomgr: skip shutdown and exit
Sage Weil [Thu, 7 Mar 2019 17:22:27 +0000 (11:22 -0600)]
mgr: skip shutdown and exit

The python subinterpreter teardown is broken; see
https://tracker.ceph.com/issues/38621

Instead, just exit with the appropriate error code.  That way we don't
crash and leave a core and ugly stack trace behind.

Users won't care, although our leak checking will.

Signed-off-by: Sage Weil <sage@redhat.com>
6 years agoMerge PR #26821 into master
Patrick Donnelly [Thu, 7 Mar 2019 16:48:05 +0000 (08:48 -0800)]
Merge PR #26821 into master

* refs/pull/26821/head:
mds,mgr,mon: warn future respawn() copy-pasters to set thread name

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
6 years agoMerge PR #26764 into master
Sage Weil [Thu, 7 Mar 2019 14:52:56 +0000 (08:52 -0600)]
Merge PR #26764 into master

* refs/pull/26764/head:
mgr: 'osd df' by specified class or (crush) name
mon/OSDMonitor: add 'osd crush get-device-class' command

6 years agomds,mgr,mon: warn future respawn() copy-pasters to set thread name 26821/head
Sage Weil [Thu, 7 Mar 2019 14:50:53 +0000 (08:50 -0600)]
mds,mgr,mon: warn future respawn() copy-pasters to set thread name

Signed-off-by: Sage Weil <sage@redhat.com>
6 years agocommon/options: osd_memory_target et al are TYPE_SIZE 26796/head
Sage Weil [Wed, 6 Mar 2019 19:28:57 +0000 (13:28 -0600)]
common/options: osd_memory_target et al are TYPE_SIZE

This makes us interpret "2G" and 2 GiB, not 2 billion bytes.

Signed-off-by: Sage Weil <sage@redhat.com>
6 years agodoc/mgr/dashboard: note dashboard as a separate package 26815/head
Kefu Chai [Thu, 7 Mar 2019 14:31:17 +0000 (22:31 +0800)]
doc/mgr/dashboard: note dashboard as a separate package

Fixes: https://tracker.ceph.com/issues/38584
Signed-off-by: Kefu Chai <kchai@redhat.com>
6 years agodoc/releases/nautilus: dashboard package notes
Kefu Chai [Thu, 7 Mar 2019 12:08:44 +0000 (20:08 +0800)]
doc/releases/nautilus: dashboard package notes

Fixes: https://tracker.ceph.com/issues/38584
Signed-off-by: Kefu Chai <kchai@redhat.com>