]> git.apps.os.sepia.ceph.com Git - ceph.git/log
ceph.git
6 years agoosd, test: Add test case with osd support for overdue PG scrubs and deep scrubs 25112/head
David Zafman [Tue, 27 Nov 2018 00:48:52 +0000 (16:48 -0800)]
osd, test: Add test case with osd support for overdue PG scrubs and deep scrubs

Add trigger_deep_scrub osd command for testing
Publish stats when trigger_scrub/trigger_deep_scrub is used for testing
Add optional argument to trigger_scrub/trigger_deep_scrub
for amount of extra time to change last scrub stamps

Signed-off-by: David Zafman <dzafman@redhat.com>
6 years agomon: Fix scrub health warning handling and change config to a ratio
David Zafman [Tue, 15 Jan 2019 20:12:39 +0000 (12:12 -0800)]
mon: Fix scrub health warning handling and change config to a ratio

Make this mon_warn code clearer since it involves 2 values
Code used mon scrub interval instead of pg scrub interval
Rename config values to include _pg_ and ratio to make it more clear
Fix scrub warniing handling use per-pool intervals when specified

Fixes: http://tracker.ceph.com/issues/37264
Signed-off-by: David Zafman <dzafman@redhat.com>
6 years agocommon: Fix some configuration descriptions
David Zafman [Tue, 22 Jan 2019 02:20:20 +0000 (18:20 -0800)]
common: Fix some configuration descriptions

osd_scrub_backoff_ratio
osd_deep_scrub_randomize_ratio

Signed-off-by: David Zafman <dzafman@redhat.com>
6 years agoosd: Increase log level in PG::_update_calc_stats()
David Zafman [Sat, 19 Jan 2019 01:57:21 +0000 (17:57 -0800)]
osd: Increase log level in PG::_update_calc_stats()

Signed-off-by: David Zafman <dzafman@redhat.com>
6 years agotest: Correct typo trying to call flush_pg_stats
David Zafman [Tue, 27 Nov 2018 00:46:41 +0000 (16:46 -0800)]
test: Correct typo trying to call flush_pg_stats

Signed-off-by: David Zafman <dzafman@redhat.com>
6 years agoMerge PR #26017 into master
Patrick Donnelly [Wed, 23 Jan 2019 18:06:34 +0000 (10:06 -0800)]
Merge PR #26017 into master

* refs/pull/26017/head:
ceph: cleanup mds ceph status output

Reviewed-by: Sage Weil <sage@redhat.com>
6 years agoMerge pull request #26079 from trociny/wip-38006
Jason Dillaman [Wed, 23 Jan 2019 16:48:05 +0000 (11:48 -0500)]
Merge pull request #26079 from trociny/wip-38006

librbd: properly filter out trashed non-user images on purge

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
6 years agoMerge pull request #25984 from tspmelo/wip-fix-summary
Lenz Grimmer [Wed, 23 Jan 2019 15:48:58 +0000 (16:48 +0100)]
Merge pull request #25984 from tspmelo/wip-fix-summary

mgr/dashboard: Fix summary refresh call stack

Reviewed-by: Ricardo Marques <rimarques@suse.com>
Reviewed-by: Sebastian Krah <skrah@suse.com>
Reviewed-by: Stephan Müller <smueller@suse.com>
6 years agomgr/dashboard: Fix summary refresh call stack 25984/head
Tiago Melo [Fri, 11 Jan 2019 12:08:01 +0000 (11:08 -0100)]
mgr/dashboard: Fix summary refresh call stack

Currently each time we created a task it would call the summary refresh method.
Since that method was a recursive method, overtime we would get N calls to the
refresh each 5 seconds.

Signed-off-by: Tiago Melo <tmelo@suse.com>
6 years agoMerge pull request #25988 from Exotelis/branding
Lenz Grimmer [Wed, 23 Jan 2019 14:21:39 +0000 (15:21 +0100)]
Merge pull request #25988 from Exotelis/branding

mgr/dashboard: Add guideline how to brand the UI and update the color scheme

Reviewed-by: Kanika Murarka <kmurarka@redhat.com>
Reviewed-by: Tatjana Dehler <tdehler@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>
6 years agoMerge pull request #25991 from jtlayton/wip-dashboard
Lenz Grimmer [Wed, 23 Jan 2019 11:04:35 +0000 (12:04 +0100)]
Merge pull request #25991 from jtlayton/wip-dashboard

mgr/dashboard: add a script to run an API request on a rook cluster

Reviewed-by: Ricardo Dias <rdias@suse.com>
6 years agomgr/dashboard: Fix linting errors 25988/head
Sebastian Krah [Wed, 23 Jan 2019 08:20:51 +0000 (09:20 +0100)]
mgr/dashboard: Fix linting errors

Fixes linting error in two files

Signed-off-by: Sebastian Krah <skrah@suse.com>
6 years agoMerge pull request #25774 from yehudasa/wip-rgw-tiering-3
Matt Benjamin [Tue, 22 Jan 2019 17:42:21 +0000 (12:42 -0500)]
Merge pull request #25774 from yehudasa/wip-rgw-tiering-3

rgw: rados tiering

6 years agolibrbd: properly filter out trashed non-user images on purge 26079/head
Mykola Golub [Tue, 22 Jan 2019 15:54:20 +0000 (17:54 +0200)]
librbd: properly filter out trashed non-user images on purge

Fixes: https://tracker.ceph.com/issues/38006
Signed-off-by: Mykola Golub <mgolub@suse.com>
6 years agoMerge pull request #26073 from rhcs-dashboard/add-tox-ini-command
Lenz Grimmer [Tue, 22 Jan 2019 15:23:05 +0000 (16:23 +0100)]
Merge pull request #26073 from rhcs-dashboard/add-tox-ini-command

mgr/dashboard: added command to tox.ini

Reviewed-by: Sebastian Wagner <swagner@suse.com>
Reviewed-by: Stephan Müller <smueller@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>
6 years agoMerge pull request #26020 from rhcs-dashboard/issue-37854
Lenz Grimmer [Tue, 22 Jan 2019 15:12:05 +0000 (16:12 +0100)]
Merge pull request #26020 from rhcs-dashboard/issue-37854

mgr/dashboard: Fixed performance details context for host list row selection

Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Kanika Murarka <kmurarka@redhat.com>
6 years agoMerge pull request #26007 from rhcs-dashboard/upgrade-python-dev-deps
Lenz Grimmer [Tue, 22 Jan 2019 14:52:02 +0000 (15:52 +0100)]
Merge pull request #26007 from rhcs-dashboard/upgrade-python-dev-deps

mgr/dashboard: upgraded python dev dependencies

Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Laura Paduano <lpaduano@suse.com>
Reviewed-by: Patrick Nawracay <pnawracay@suse.com>
Reviewed-by: Sebastian Wagner <swagner@suse.com>
6 years agomgr/dashboard: added command to tox.ini 26073/head
alfonsomthd [Tue, 22 Jan 2019 12:26:38 +0000 (13:26 +0100)]
mgr/dashboard: added command to tox.ini

* Added 'run' command to run arbitrary commands so
  you don't have to run the whole test/lint suite.
  Use case examples:
  - Run only 1 test:
  tox -e py3-run pytest tests/test_rgw_client.py::RgwClientTest::test_ssl_verify
  - Run pycodestyle for only 1 file:
  tox -e py3-run pycodestyle tests/test_rgw_client.py

Signed-off-by: Alfonso Martínez <almartin@redhat.com>
6 years agoMerge pull request #26032 from jtlayton/wip-rook
Sebastian Wagner [Tue, 22 Jan 2019 09:28:25 +0000 (10:28 +0100)]
Merge pull request #26032 from jtlayton/wip-rook

mgr/rook: remove dead code and fix bug in url fetching code

Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>
6 years agoMerge pull request #25895 from jan--f/c-v-func-test-local
Andrew Schoen [Mon, 21 Jan 2019 17:40:47 +0000 (11:40 -0600)]
Merge pull request #25895 from jan--f/c-v-func-test-local

ceph-volume: functional tests, add libvirt customization

Reviewed-by: Andrew Schoen <aschoen@redhat.com>
6 years agoMerge pull request #26044 from tchaikov/wip-build-doc-pyyaml
Kefu Chai [Mon, 21 Jan 2019 17:00:15 +0000 (01:00 +0800)]
Merge pull request #26044 from tchaikov/wip-build-doc-pyyaml

doc: bump up sphinx and pyyaml versions

Reviewed-by: Alfredo Deza <adeza@redhat.com>
6 years agodoc: bump up sphinx and pyyaml versions 26044/head
Kefu Chai [Sun, 20 Jan 2019 06:51:11 +0000 (14:51 +0800)]
doc: bump up sphinx and pyyaml versions

* bump up pyyaml version to address CVE-2017-18342, see
  https://nvd.nist.gov/vuln/detail/CVE-2017-18342
* bump up sphinx to the latest stable

Signed-off-by: Kefu Chai <kchai@redhat.com>
6 years agoMerge pull request #26052 from tchaikov/wip-37976
Kefu Chai [Mon, 21 Jan 2019 15:04:55 +0000 (23:04 +0800)]
Merge pull request #26052 from tchaikov/wip-37976

mgr/zabbix: drop "total_objects" field

Reviewed-by: Sage Weil <sage@redhat.com>
6 years agoMerge pull request #26054 from tchaikov/wip-crimson-dispatcher-fix-ftbfs
Kefu Chai [Mon, 21 Jan 2019 14:30:26 +0000 (22:30 +0800)]
Merge pull request #26054 from tchaikov/wip-crimson-dispatcher-fix-ftbfs

crimson/osd: remove "force_new" from ms_get_authorizer()

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
6 years agoMerge pull request #26045 from hsiang41/Ticket_33736
Kefu Chai [Mon, 21 Jan 2019 14:17:10 +0000 (22:17 +0800)]
Merge pull request #26045 from hsiang41/Ticket_33736

mgr/diskprediction_cloud: fix divide by zero when total_size is 0

Reviewed-by: Kefu Chai <kchai@redhat.com>
6 years agocrimson/osd: remove "force_new" from ms_get_authorizer() 26054/head
Kefu Chai [Mon, 21 Jan 2019 11:26:10 +0000 (19:26 +0800)]
crimson/osd: remove "force_new" from ms_get_authorizer()

it was removed in e62f45d1, so update crimson-osd accordingly

Signed-off-by: Kefu Chai <kchai@redhat.com>
6 years agoMerge pull request #26046 from liewegas/wip-37976
Kefu Chai [Mon, 21 Jan 2019 10:44:41 +0000 (18:44 +0800)]
Merge pull request #26046 from liewegas/wip-37976

mgr/telemetry: fix total_objects

Reviewed-by: Kefu Chai <kchai@redhat.com>
6 years agomgr/zabbix: drop "total_objects" field 26052/head
Kefu Chai [Mon, 21 Jan 2019 10:38:24 +0000 (18:38 +0800)]
mgr/zabbix: drop "total_objects" field

This field was removed from df output a while back in
342f309

Signed-off-by: Kefu Chai <kchai@redhat.com>
6 years agoMerge pull request #25712 from familyuu/configuration
Lenz Grimmer [Mon, 21 Jan 2019 08:34:35 +0000 (09:34 +0100)]
Merge pull request #25712 from familyuu/configuration

mgr/dashboard: Add a 'clear filter' button to configuration page

Reviewed-by: Sebastian Krah <skrah@suse.com>
Reviewed-by: Tatjana Dehler <tdehler@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>
6 years agomgr/diskprediction_cloud: fix divide by zero when total_size is 0 26045/head
hsiang41 [Sun, 20 Jan 2019 13:22:24 +0000 (21:22 +0800)]
mgr/diskprediction_cloud: fix divide by zero when total_size is 0

Fix devide by zero when total_size is o.

Ref ticket: https://tracker.ceph.com/issues/37736

Signed-off-by: Rick Chen <rick.chen@prophetstor.com>
6 years agomgr/telemetry: fix total_objects 26046/head
Sage Weil [Sun, 20 Jan 2019 21:59:50 +0000 (15:59 -0600)]
mgr/telemetry: fix total_objects

This field was removed from df output a while back in
342f309645df886fb96eb401634e38376553e6d9

Fixes: http://tracker.ceph.com/issues/37976
Signed-off-by: Sage Weil <sage@redhat.com>
6 years agoMerge PR #26031 into master
Sage Weil [Sun, 20 Jan 2019 21:55:50 +0000 (15:55 -0600)]
Merge PR #26031 into master

* refs/pull/26031/head:
mgr/BaseMgrModule: fix mon commands targetting a specific monitor
mgr/ActivePyModules: fix RestoreThread calls for get('devices')

Reviewed-by: Kefu Chai <kchai@redhat.com>
6 years agoMerge pull request #25490 from Aran85/fix-ec-using-cache
Kefu Chai [Sun, 20 Jan 2019 02:33:51 +0000 (10:33 +0800)]
Merge pull request #25490 from Aran85/fix-ec-using-cache

osd: keep using cache even if op will invalid cache

Reviewed-by: Kefu Chai <kchai@redhat.com>
6 years agoMerge pull request #25920 from wjwithagen/wjw-fix-freebsd-numa
Kefu Chai [Sun, 20 Jan 2019 02:32:21 +0000 (10:32 +0800)]
Merge pull request #25920 from wjwithagen/wjw-fix-freebsd-numa

common/numa: Add shim routines for NUMA on FreeBSD.

6 years agoMerge pull request #25938 from majianpeng/msg-remove-memcopy-message-header
Kefu Chai [Sun, 20 Jan 2019 02:31:26 +0000 (10:31 +0800)]
Merge pull request #25938 from majianpeng/msg-remove-memcopy-message-header

msg/async: reduce additional ceph_msg_header copy.

Reviewed-by: Xie Xingguo <xie.xingguo@zte.com.cn>
Reviewed-by: Kefu Chai <kchai@redhat.com>
6 years agoMerge pull request #26021 from majianpeng/osd-remove-unused-code
Kefu Chai [Sun, 20 Jan 2019 02:30:34 +0000 (10:30 +0800)]
Merge pull request #26021 from majianpeng/osd-remove-unused-code

osd: remove unused fields

Reviewed-by: Kefu Chai <kchai@redhat.com>
6 years agoMerge pull request #26040 from tchaikov/wip-blkdev-stat
Kefu Chai [Sun, 20 Jan 2019 02:28:22 +0000 (10:28 +0800)]
Merge pull request #26040 from tchaikov/wip-blkdev-stat

common/blkdev: check retval of stat()

Reviewed-by: Sage Weil <sage@redhat.com>
6 years agoMerge pull request #26041 from tchaikov/wip-kvstore-tool-gcc-ice
Kefu Chai [Sun, 20 Jan 2019 02:27:52 +0000 (10:27 +0800)]
Merge pull request #26041 from tchaikov/wip-kvstore-tool-gcc-ice

tools/ceph_kvstore_tool: extract StoreTool into kvstore_tool.cc

Reviewed-by: Sage Weil <sage@redhat.com>
6 years agoceph: cleanup mds ceph status output 26017/head
Patrick Donnelly [Thu, 17 Jan 2019 18:14:17 +0000 (10:14 -0800)]
ceph: cleanup mds ceph status output

This is primarily to make the CephFS line more readable with lots of file
systems with more understandable output.

From something like:

    mds: a-2/2/2 up  {0=c=up:active,1=b=up:active}, 1 up:standby

Now with 1 FS and 1 active:

    mds: a:1 {0=p=up:active}, 15 up:standby

2 FS, 2 actives:

    mds: a:1 b:1 {a:0=o=up:active,b:0=n=up:active}, 14 up:standby

2 FS, 3 actives:

    mds: a:1 b:2 {a:0=o=up:active,b:0=n=up:active,b:1=p=up:active}, 13 up:standby

3 FS and 2 active and 1 reconnect:

    mds: a:1/1 b:1 c:1 {a:0=n=up:reconnect,b:0=p=up:active,c:0=o=up:active}, 13 up:standby

3 FS and 1 failed rank:

    mds: a:0/1 b:1 c:1 {b:0=p=up:active,c:0=o=up:active}, 14 up:standby, 1 failed

3 FS, 4 actives:

    mds: a:1 b:2 c:1 {a:0=o=up:active,b:0=n=up:active,b:1=p=up:active,c:0=l=up:active}, 12 up:standby

3 FS, 3 actives, 1 FS failed:

    mds: a:1 b:2 c:0/1 {a:0=o=up:active,b:0=n=up:active,b:1=p=up:active}, 13 up:standby, 1 failed

3 FS, 5 actives:

    mds: a:1 b:2 c:2 5 up:active, 11 up:standby

4 FS:

    mds: 4 fs {a:0=o=up:active,b:0=n=up:active,c:0=l=up:active,d:0=p=up:active}, 12 up:standby

5 FS:

    mds: 5 fs 5 up:active, 11 up:standby

5 FS, 1 degraded:

    mds: 5 fs (degraded: c:0/1) {a:0=o=up:active,b:0=n=up:active,d:0=p=up:active,e:0=m=up:active}, 12 up:standby, 1 failed

8 FS, 1 degraded:

    mds: 8 fs (degraded: c:0/1) 7 up:active, 9 up:standby, 1 failed

Fixes: http://tracker.ceph.com/issues/37954
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
6 years agoMerge PR #26012 into master
Patrick Donnelly [Sun, 20 Jan 2019 01:21:42 +0000 (17:21 -0800)]
Merge PR #26012 into master

* refs/pull/26012/head:
qa: add test that down fs does not ERR
mon/MDSMonitor: skip offline ERR for down fs

Reviewed-by: Douglas Fuller <dfuller@redhat.com>
6 years agoMerge PR #25993 into master
Patrick Donnelly [Sun, 20 Jan 2019 01:21:09 +0000 (17:21 -0800)]
Merge PR #25993 into master

* refs/pull/25993/head:
mds/FSMap: simplify encode/decode
mds: alias reference type for Filesystem
mds: add missing memory header include
include/encoding: eliminate smart_ptr temporaries

Reviewed-by: Zheng Yan <zyan@redhat.com>
6 years agoMerge PR #25976 into master
Patrick Donnelly [Sun, 20 Jan 2019 01:20:18 +0000 (17:20 -0800)]
Merge PR #25976 into master

* refs/pull/25976/head:
Objecter::calc_op_budget: Fix invalid access to extent union member

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
Reviewed-by: Zheng Yan <zyan@redhat.com>
6 years agoMerge PR #25974 into master
Patrick Donnelly [Sun, 20 Jan 2019 01:19:19 +0000 (17:19 -0800)]
Merge PR #25974 into master

* refs/pull/25974/head:
MDSMonitor: add missing OSDMonitor writeable check
MDSMonitor: codify constraint for writeable OSDMap

Reviewed-by: Sage Weil <sage@redhat.com>
6 years agoMerge PR #25973 into master
Patrick Donnelly [Sun, 20 Jan 2019 01:18:08 +0000 (17:18 -0800)]
Merge PR #25973 into master

* refs/pull/25973/head:
qa: use simpler fs fail to bring fs down
MDSMonitor: add fs fail command

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Douglas Fuller <dfuller@redhat.com>
6 years agoMerge PR #25971 into master
Patrick Donnelly [Sun, 20 Jan 2019 01:17:04 +0000 (17:17 -0800)]
Merge PR #25971 into master

* refs/pull/25971/head:
qa: test hex dump_inode
pybind/ceph_argparse: accept int with non-decimal base

Reviewed-by: Kefu Chai <kchai@redhat.com>
6 years agoMerge PR #25650 into master
Patrick Donnelly [Sun, 20 Jan 2019 01:16:17 +0000 (17:16 -0800)]
Merge PR #25650 into master

* refs/pull/25650/head:
client: add ceph_stat/ceph_lstat/ceph_fstat api

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Jeff Layton <jlayton@redhat.com>
6 years agotool/ceph_kvstore_tool: extract duration calc out 26041/head
Kefu Chai [Sat, 19 Jan 2019 16:48:40 +0000 (00:48 +0800)]
tool/ceph_kvstore_tool: extract duration calc out

simplify the code a little bit

Signed-off-by: Kefu Chai <kchai@redhat.com>
6 years agotools/ceph_kvstore_tool: extract StoreTool into kvstore_tool.cc
Kefu Chai [Sat, 19 Jan 2019 16:01:55 +0000 (00:01 +0800)]
tools/ceph_kvstore_tool: extract StoreTool into kvstore_tool.cc

to workaround n bug in GCC 7.3, which leads to ICE like

build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/14.0.1-2751-gb268822/rpm/el7/BUILD/ceph-14.0.1-2751-gb268822/src/tools/ceph_kvstore_tool.cc:625:1:
internal compiler error: Segmentation fault
 }
 ^
Please submit a full bug report,
with preprocessed source if appropriate.
See <http://bugzilla.redhat.com/bugzilla> for instructions.

Signed-off-by: Kefu Chai <kchai@redhat.com>
6 years agomgr/BaseMgrModule: fix mon commands targetting a specific monitor 26031/head
Sage Weil [Fri, 18 Jan 2019 21:14:19 +0000 (15:14 -0600)]
mgr/BaseMgrModule: fix mon commands targetting a specific monitor

We just need to use the right variant of start_mon_command().

Fixes: http://tracker.ceph.com/issues/37967
Signed-off-by: Sage Weil <sage@redhat.com>
6 years agomgr/ActivePyModules: fix RestoreThread calls for get('devices')
Sage Weil [Fri, 18 Jan 2019 15:24:34 +0000 (09:24 -0600)]
mgr/ActivePyModules: fix RestoreThread calls for get('devices')

Fixes 61aa7e2e0230bff49e757c32932d1db4bcad5f67

Fixes: http://tracker.ceph.com/issues/37736
Signed-off-by: Sage Weil <sage@redhat.com>
6 years agocommon/blkdev: check retval of stat() 26040/head
Kefu Chai [Sat, 19 Jan 2019 13:09:59 +0000 (21:09 +0800)]
common/blkdev: check retval of stat()

Signed-off-by: Kefu Chai <kchai@redhat.com>
6 years agoqa: use simpler fs fail to bring fs down 25973/head
Patrick Donnelly [Tue, 15 Jan 2019 22:10:09 +0000 (14:10 -0800)]
qa: use simpler fs fail to bring fs down

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
6 years agorgw: align head object size with pool alignment 25774/head
Yehuda Sadeh [Sat, 19 Jan 2019 05:04:50 +0000 (21:04 -0800)]
rgw: align head object size with pool alignment

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
6 years agoMerge pull request #25919 from yuriw/wip-yuriw-distros-master
David Galloway [Fri, 18 Jan 2019 18:03:17 +0000 (13:03 -0500)]
Merge pull request #25919 from yuriw/wip-yuriw-distros-master

qa/tests: added rhel 7.6

6 years agoMerge PR #26019 into master
Sage Weil [Fri, 18 Jan 2019 17:27:07 +0000 (11:27 -0600)]
Merge PR #26019 into master

* refs/pull/26019/head:
mon: implement hiding commands in ceph tool

Reviewed-by: Sage Weil <sage@redhat.com>
6 years agomgr/rook: wrap url fetch function in try/except block 26032/head
Jeff Layton [Fri, 18 Jan 2019 17:15:18 +0000 (12:15 -0500)]
mgr/rook: wrap url fetch function in try/except block

It's possible for the fetch of this object to fail if the service is
being torn down. Handle that situation gracefully, but log a message.

Signed-off-by: Jeff Layton <jlayton@redhat.com>
6 years agoMerge pull request #25996 from yehudasa/wip-37885
Casey Bodley [Fri, 18 Jan 2019 16:45:29 +0000 (11:45 -0500)]
Merge pull request #25996 from yehudasa/wip-37885

qa/rgw/tempest: either force os_type or select random distro

Reviewed-by: Casey Bodley <cbodley@redhat.com>
6 years agoMerge pull request #26015 from noahdesu/osd-find-fsid
Noah Watkins [Fri, 18 Jan 2019 16:26:47 +0000 (08:26 -0800)]
Merge pull request #26015 from noahdesu/osd-find-fsid

cli: dump osd-fsid as part of osd find <id>

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Sébastien Han <seb@redhat.com>
6 years agoMerge pull request #22797 from dzafman/wip-19753
David Zafman [Fri, 18 Jan 2019 15:42:00 +0000 (07:42 -0800)]
Merge pull request #22797 from dzafman/wip-19753

osd: Deny reservation if expected backfill size would put us over bac…

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>
6 years agoMerge PR #25132 into master
Sage Weil [Fri, 18 Jan 2019 15:38:59 +0000 (09:38 -0600)]
Merge PR #25132 into master

* refs/pull/25132/head:
os/bluestore: implement kv update on bluefs extents info change.
os/bluestore: use original algorithm to calculate BlueFS gift size for explicit allocation.
os/bluestore: get rid of bluefs space gifting performed in background
os/bluestore: do KV/FM/allocator opening in two stages.
os/bluestore: detach FreelistManager from DB instance.
test/osd/store_test: add test case for explicit bluefs expanding.
os/bluestore: implement explicit bluefs space allocation at slow device
os/bluestore: unify allocator dump on bluefs space failures
os/bluestore: stop keeping bluefs_extents at KV

Reviewed-by: Sage Weil <sage@redhat.com>
6 years agomgr/rook: remove init_rook
Jeff Layton [Fri, 18 Jan 2019 15:23:21 +0000 (10:23 -0500)]
mgr/rook: remove init_rook

John evidently added this code early on in development, when we figured
we might be taking over a cluster that is already running under k8s.

I don't see how we'd ever make that work, so just remove this function
for now, and the commented-out callsite.

Reported-by: Sebastian Wagner <sebastian.wagner@suse.com>
Signed-off-by: Jeff Layton <jlayton@redhat.com>
6 years agoMerge pull request #25972 from noahdesu/doc-release-ver-sort
Kefu Chai [Fri, 18 Jan 2019 14:52:50 +0000 (22:52 +0800)]
Merge pull request #25972 from noahdesu/doc-release-ver-sort

doc: sort releases by date and version

Reviewed-by: Nathan Cutler <ncutler@suse.com>
6 years agoMerge pull request #25891 from ifed01/wip-ifed-fix-compress-errors
Kefu Chai [Fri, 18 Jan 2019 14:41:08 +0000 (22:41 +0800)]
Merge pull request #25891 from ifed01/wip-ifed-fix-compress-errors

os/bluestore: do not assert on non-zero err codes from  compress() call

Reviewed-by: Sage Weil <sage@redhat.com>
6 years agoMerge pull request #15168 from qwren/qat-crypto
Kefu Chai [Fri, 18 Jan 2019 14:39:13 +0000 (22:39 +0800)]
Merge pull request #15168 from qwren/qat-crypto

crypto: add openssl support for RGW encryption

Reviewed-by: Kefu Chai <kchai@redhat.com>
6 years agoMerge pull request #25961 from p-na/wip-pna-fix-argparse
Kefu Chai [Fri, 18 Jan 2019 14:38:21 +0000 (22:38 +0800)]
Merge pull request #25961 from p-na/wip-pna-fix-argparse

ceph_argparse: fix --verbose

Reviewed-by: Kefu Chai <kchai@redhat.com>
6 years agoMerge pull request #25324 from votdev/fix_raw_input
Alfredo Deza [Fri, 18 Jan 2019 14:19:27 +0000 (09:19 -0500)]
Merge pull request #25324 from votdev/fix_raw_input

ceph-volume: Adapt code to support Python3

Reviewed-by: Alfredo Deza <adeza@redhat.com>
6 years agoMerge pull request #25912 from sebastian-philipp/orchestrator-drive-group
Sebastian Wagner [Fri, 18 Jan 2019 13:59:48 +0000 (14:59 +0100)]
Merge pull request #25912 from sebastian-philipp/orchestrator-drive-group

mgr/orchestrator: Extend DriveGroupSpec

6 years agoMerge pull request #26006 from jtlayton/wip-orchestrator
Sebastian Wagner [Fri, 18 Jan 2019 13:57:54 +0000 (14:57 +0100)]
Merge pull request #26006 from jtlayton/wip-orchestrator

Clean up field names in ServiceDescription and add a service field

Reviewed-by: Ricardo Dias <rdias@suse.com>
Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>
6 years agomgr/dashboard: Mark variables as default
Sebastian Krah [Thu, 17 Jan 2019 08:35:11 +0000 (09:35 +0100)]
mgr/dashboard: Mark variables as default

Mark all variables as default, that they can get overwritten.
Also import the file vendor.variables

Signed-off-by: Sebastian Krah <skrah@suse.com>
6 years agomgr/dashboard: Add vendor.overrides.scss and vendor.variables.scss
Sebastian Krah [Wed, 16 Jan 2019 15:20:34 +0000 (16:20 +0100)]
mgr/dashboard: Add vendor.overrides.scss and vendor.variables.scss

Adds the files vendor.overrides.scss and vendor.variables.scss.
In those files the css and variable defaults can be overwritten.

Signed-off-by: Sebastian Krah <skrah@suse.com>
6 years agomgr/dashboard: Adapt to ceph.com branding
Sebastian Krah [Wed, 16 Jan 2019 12:17:39 +0000 (13:17 +0100)]
mgr/dashboard: Adapt to ceph.com branding

Changes the primary, secondary and accent color and adds a radius to all
buttons to match the branding of ceph.com

Signed-off-by: Sebastian Krah <skrah@suse.com>
6 years agomgr/dashboard: Introduce new scss variable
Sebastian Krah [Tue, 15 Jan 2019 10:15:20 +0000 (11:15 +0100)]
mgr/dashboard: Introduce new scss variable

Adds a new scss variable called color-primary that the primary color only has
to be changed at one point

Signed-off-by: Sebastian Krah <skrah@suse.com>
6 years agomgr/dashboard: Add branding documentation
Sebastian Krah [Tue, 8 Jan 2019 13:56:25 +0000 (14:56 +0100)]
mgr/dashboard: Add branding documentation

Explains how to brand the ui without running in merge conflicts once a source
file gets updated

Signed-off-by: Sebastian Krah <skrah@suse.com>
6 years agoMerge PR #25981 into master
Sage Weil [Fri, 18 Jan 2019 13:16:36 +0000 (07:16 -0600)]
Merge PR #25981 into master

* refs/pull/25981/head:
crush: switch to *get_or_create_class_id* helper whenever possible

Reviewed-by: Sage Weil <sage@redhat.com>
6 years agomgr/orchestrator: Extend DriveGroupSpec 25912/head
Sebastian Wagner [Fri, 11 Jan 2019 12:33:05 +0000 (13:33 +0100)]
mgr/orchestrator: Extend DriveGroupSpec

Also: Added tox.

Co-authored-by: Juan Miguel Olmo Martínez <jolmomar@redhat.com>
Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
6 years agoMerge pull request #25602 from familyuu/pool
Lenz Grimmer [Fri, 18 Jan 2019 12:07:49 +0000 (13:07 +0100)]
Merge pull request #25602 from familyuu/pool

mgr/dashboard: Add pool cache tiering details tab

Reviewed-by: Laura Paduano <lpaduano@suse.com>
Reviewed-by: Stephan Müller <smueller@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>
6 years agoMerge pull request #25651 from votdev/improve_basemgrmodule
Tim Serong [Fri, 18 Jan 2019 09:15:57 +0000 (19:15 +1000)]
Merge pull request #25651 from votdev/improve_basemgrmodule

mgr: Allow modules to get/set other module options

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Sebastian Wagner <swagner@suse.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
6 years agomgr/dashboard: Add a 'clear filter' button to configuration page 25712/head
guodan1 [Thu, 27 Dec 2018 09:08:51 +0000 (17:08 +0800)]
mgr/dashboard: Add a 'clear filter' button to configuration page

Fixes:http://tracker.ceph.com/issues/36173

Signed-off-by: familyuu <guodan1@lenovo.com>
6 years agoMerge pull request #25304 from tchaikov/wip-crimson-osd
Kefu Chai [Fri, 18 Jan 2019 06:07:42 +0000 (14:07 +0800)]
Merge pull request #25304 from tchaikov/wip-crimson-osd

crimson/osd: crimson osd driver

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
6 years agoosd:remove unused fileds. 26021/head
Jianpeng Ma [Fri, 18 Jan 2019 05:35:39 +0000 (13:35 +0800)]
osd:remove unused fileds.

Session::received_map_epoch only used by OSD::dispatch_op_fast. But this
func removed in 3cc48278bf0ee5c9535, So remvoe related code.

Signed-off-by: Jianpeng Ma <jianpeng.ma@intel.com>
6 years agocrimson/osd: add mkfs() 25304/head
Kefu Chai [Fri, 18 Jan 2019 04:27:30 +0000 (12:27 +0800)]
crimson/osd: add mkfs()

we will need it for preparing crimson-osd without help of ceph-osd.

Signed-off-by: Kefu Chai <kchai@redhat.com>
6 years agocrimson/osd: send beacon to mon periodically
Kefu Chai [Mon, 14 Jan 2019 09:48:18 +0000 (17:48 +0800)]
crimson/osd: send beacon to mon periodically

so monitor won't mark crimson-osd down and out

Signed-off-by: Kefu Chai <kchai@redhat.com>
6 years agocrimson/osd: enable crimson-osd to boot
Kefu Chai [Fri, 11 Jan 2019 10:47:39 +0000 (18:47 +0800)]
crimson/osd: enable crimson-osd to boot

* add state.h to encapsulate the state represeting different stages
  related to booting an OSD. the boot process of an OSD can be blocked
  by
  - waiting for PG consuming updated osdmaps
  - waiting for osdmaps marking osd.{whoami} up
  - waiting for new osdmaps to bring this osd up to speed.
  - waiting for current OSD to be healthy
  we could chain these "waits" in a more seastarized way, and let
OSD::start() wait on the future returned by this chain. but that'd
requires adding some seastar::shard_future<> as member variables of
`OSD` class, which is a little bit more convoluted than the state
machine approach used in this change. we could switch over to the
`future<>` chain approach, if we found that these futures could have
more consumers than merely `OSD::start()`.
* all osdmaps are now stored in an `std::map` in `OSD`, we can
  improve it by
  - caching it using an LRU cache
  - trimming the stale ones
  - persisting the evicted maps into the meta collection in ObjectStore
* superblock is not persited to store, neither is it read from the
  store.

Signed-off-by: Kefu Chai <kchai@redhat.com>
6 years agotest/crimson: s/get_connection/connect_to/
Kefu Chai [Wed, 16 Jan 2019 07:54:38 +0000 (15:54 +0800)]
test/crimson: s/get_connection/connect_to/

in aa3d6f66, we removed Messenger::get_connection(), so update the test
accordingly.

Signed-off-by: Kefu Chai <kchai@redhat.com>
6 years agomessages/MOSDBeacon: include PaxosServiceMessage.h
Kefu Chai [Mon, 14 Jan 2019 09:50:41 +0000 (17:50 +0800)]
messages/MOSDBeacon: include PaxosServiceMessage.h

Signed-off-by: Kefu Chai <kchai@redhat.com>
6 years agocrimson: integrate mon::Client into OSD
Kefu Chai [Fri, 11 Jan 2019 10:21:25 +0000 (18:21 +0800)]
crimson: integrate mon::Client into OSD

and register them as dispatchers using ChainedDispatchers

Signed-off-by: Kefu Chai <kchai@redhat.com>
6 years agocrimson: add ChainedDispatchers
Kefu Chai [Fri, 11 Jan 2019 09:53:15 +0000 (17:53 +0800)]
crimson: add ChainedDispatchers

it will be used to glue multiple dispatchers for subscribing events
emitted by messenger

Signed-off-by: Kefu Chai <kchai@redhat.com>
6 years agocrimson: add prototype of crimson/osd
Kefu Chai [Tue, 9 Oct 2018 07:17:35 +0000 (15:17 +0800)]
crimson: add prototype of crimson/osd

Signed-off-by: Kefu Chai <kchai@redhat.com>
6 years agomgr/dashboard: Fixed performance details context for host list row selection 26020/head
Neha Gupta [Fri, 18 Jan 2019 04:36:41 +0000 (13:36 +0900)]
mgr/dashboard: Fixed performance details context for host list row selection
Fixes: http://tracker.ceph.com/issues/37854
Signed-off-by: Neha Gupta <gnehapk@gmail.com>
6 years agocrimson: add a dummy object store
Kefu Chai [Mon, 7 Jan 2019 10:50:05 +0000 (18:50 +0800)]
crimson: add a dummy object store

CyanStore only provides just-enough facility for booting an OSD.

Signed-off-by: Kefu Chai <kchai@redhat.com>
6 years agocrimson: set src for message
Kefu Chai [Thu, 10 Jan 2019 13:15:18 +0000 (21:15 +0800)]
crimson: set src for message

monitor will panic at seeing a MOSDBoot message which is not from an
OSD. see OSDMonitor::preprocess_boot()
```
ceph_assert(m->get_orig_source_inst().name.is_osd());
```

Signed-off-by: Kefu Chai <kchai@redhat.com>
6 years agocrimson: ignore v2 only addvec
Kefu Chai [Thu, 10 Jan 2019 13:14:28 +0000 (21:14 +0800)]
crimson: ignore v2 only addvec

crimson need to be v2 compatible. but let's leave this for another PR.

Signed-off-by: Kefu Chai <kchai@redhat.com>
6 years agocmake: silence -Wsign-compare in libfmt
Kefu Chai [Mon, 14 Jan 2019 03:26:25 +0000 (11:26 +0800)]
cmake: silence -Wsign-compare in libfmt

* silence -Wsign-compare in libfmt
* silence -Wattributes in seastar

to silence warnings like:

/home/jenkins-build/build/workspace/ceph-pull-requests-arm64/src/seastar/fmt/include/fmt/format.h:
In instantiation of 'constexpr bool fmt::v5::internal::find(Ptr, Ptr, T,
Ptr&) [with bool IS_CONSTEXPR = false; T = char; Ptr = const wchar_t*]':
/home/jenkins-build/build/workspace/ceph-pull-requests-arm64/src/seastar/fmt/include/fmt/format.h:2154:32:
required from 'constexpr void
fmt::v5::internal::parse_format_string(fmt::v5::basic_string_view<Char>,
Handler&&)::writer::operator()(const Char*, const Char*) [with bool
IS_CONSTEXPR = false; Char = wchar_t; Handler =
fmt::v5::format_handler<fmt::v5::arg_formatter<fmt::v5::back_insert_range<fmt::v5::internal::basic_buffer<wchar_t>
> >, wchar_t,
fmt::v5::basic_format_context<std::back_insert_iterator<fmt::v5::internal::basic_buffer<wchar_t>
>, wchar_t> >&]'
/home/jenkins-build/build/workspace/ceph-pull-requests-arm64/src/seastar/fmt/include/fmt/format.h:2164:3:
required from 'constexpr void
fmt::v5::internal::parse_format_string(fmt::v5::basic_string_view<Char>,
Handler&&) [with bool IS_CONSTEXPR = false; Char = wchar_t; Handler =
fmt::v5::format_handler<fmt::v5::arg_formatter<fmt::v5::back_insert_range<fmt::v5::internal::basic_buffer<wchar_t>
> >, wchar_t,
fmt::v5::basic_format_context<std::back_insert_iterator<fmt::v5::internal::basic_buffer<wchar_t>
>, wchar_t> >&]'
/home/jenkins-build/build/workspace/ceph-pull-requests-arm64/src/seastar/fmt/include/fmt/format.h:3346:39:
required from 'typename Context::iterator fmt::v5::vformat_to(typename
ArgFormatter::range, fmt::v5::basic_string_view<Char>,
fmt::v5::basic_format_args<Context>) [with ArgFormatter =
fmt::v5::arg_formatter<fmt::v5::back_insert_range<fmt::v5::internal::basic_buffer<wchar_t>
> >; Char = wchar_t; Context =
fmt::v5::basic_format_context<std::back_insert_iterator<fmt::v5::internal::basic_buffer<wchar_t>
>, wchar_t>; typename Context::iterator =
std::back_insert_iterator<fmt::v5::internal::basic_buffer<wchar_t> >;
typename ArgFormatter::range =
fmt::v5::back_insert_range<fmt::v5::internal::basic_buffer<wchar_t> >]'
/home/jenkins-build/build/workspace/ceph-pull-requests-arm64/src/seastar/fmt/include/fmt/format.h:3457:64:
required from here

/home/jenkins-build/build/workspace/ceph-pull-requests-arm64/src/seastar/fmt/include/fmt/format.h:2120:14:
error: comparison of integer expressions of different signedness: 'const
wchar_t' and 'char' [-Werror=sign-compare]
     if (*out == value)
         ~~~~~^~~~~~~~
cc1plus: all warnings being treated as errors

where libfmt compares a wchar_t with a literal '}', which is char.
because the former is unsigned, and the latter is of a signed type,
GCC is annoyed. but since both of them are ASCII, and when performing
comparison, the signed one is converted to unsigned, then result of
comparison is correct per-se. hence, it's safe to silence this very
warning.

and warning like:

/home/jenkins-build/build/workspace/ceph-pull-requests-arm64/src/seastar/src/core/future-util.cc:61:5:
required from 'seastar::future<> seastar::sleep_abortable(typename
Clock::duration, seastar::abort_source&) [with Clock =
std::chrono::_V2::steady_clock; typename Clock::duration =
std::chrono::duration<long int, std::ratio<1, 1000000000> >]'
/home/jenkins-build/build/workspace/ceph-pull-requests-arm64/src/seastar/src/core/future-util.cc:68:105:
required from here
/home/jenkins-build/build/workspace/ceph-pull-requests-arm64/src/seastar/src/core/future-util.cc:48:28:
error: 'seastar::sleep_abortable(typename Clock::duration,
seastar::abort_source&)::sleeper::sleeper(typename Clock::duration,
seastar::abort_source&) [with Clock = std::chrono::_V2::steady_clock;
typename Clock::duration = std::chrono::duration<long int, std::ratio<1,
1000000000> >]::<lambda()>' declared with greater visibility than the
type of its field 'seastar::sleep_abortable(typename Clock::duration,
seastar::abort_source&)::sleeper::sleeper(typename Clock::duration,
seastar::abort_source&) [with Clock = std::chrono::_V2::steady_clock;
typename Clock::duration = std::chrono::duration<long int, std::ratio<1,
1000000000> >]::<lambda()>::<this capture>' [-Werror=attributes]
                 : tmr([this] { done.set_value(); }) {
                            ^
Signed-off-by: Kefu Chai <kchai@redhat.com>
6 years agoseastar: pick up the cmake fix
Kefu Chai [Fri, 11 Jan 2019 12:55:01 +0000 (20:55 +0800)]
seastar: pick up the cmake fix

to pick up the fix for seastar/issues/576

Signed-off-by: Kefu Chai <kchai@redhat.com>
6 years agocrimson: set default keyring path
Kefu Chai [Thu, 10 Jan 2019 13:12:25 +0000 (21:12 +0800)]
crimson: set default keyring path

otherwise crimson-osd won't be able to load the keyring

Signed-off-by: Kefu Chai <kchai@redhat.com>
6 years agocrimson: add entity_addrvec_t interfaces to Messenger
Kefu Chai [Wed, 9 Jan 2019 15:51:44 +0000 (23:51 +0800)]
crimson: add entity_addrvec_t interfaces to Messenger

Signed-off-by: Kefu Chai <kchai@redhat.com>
6 years agocrimson/monc: add subcription helpers
Kefu Chai [Fri, 4 Jan 2019 12:56:25 +0000 (20:56 +0800)]
crimson/monc: add subcription helpers

Signed-off-by: Kefu Chai <kchai@redhat.com>
6 years agocrimson/monc: add start() method
Kefu Chai [Fri, 21 Dec 2018 10:30:37 +0000 (18:30 +0800)]
crimson/monc: add start() method

to boot strap mon::Client once seastar and msgr is ready. this allows us
to allocate mon::Client on stack.

Signed-off-by: Kefu Chai <kchai@redhat.com>
6 years agotest/crimson/monc: start/stop perf counter
Kefu Chai [Fri, 21 Dec 2018 10:34:22 +0000 (18:34 +0800)]
test/crimson/monc: start/stop perf counter

in CephContext::CephContext(), we assume that
ceph::common::local_perf_coll() is ready when a CephContext is to be
contructed. so we need to prepare start it before creating CephContext.

Signed-off-by: Kefu Chai <kchai@redhat.com>
6 years agomon: implement hiding commands in ceph tool 26019/head
Patrick Donnelly [Fri, 18 Jan 2019 04:08:53 +0000 (20:08 -0800)]
mon: implement hiding commands in ceph tool

Otherwise ceph.in doing get_command_descriptions sees the command does not
exist and will print an error.

Fixes: https://tracker.ceph.com/issues/37956
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>