]> git.apps.os.sepia.ceph.com Git - ceph.git/log
ceph.git
6 years agoosd/PG: remove unused functions 26155/head
Kefu Chai [Sat, 26 Jan 2019 17:18:25 +0000 (01:18 +0800)]
osd/PG: remove unused functions

Signed-off-by: Kefu Chai <kchai@redhat.com>
6 years agoMerge pull request #23904 from xiexingguo/wip-27985
David Zafman [Sat, 26 Jan 2019 00:44:22 +0000 (16:44 -0800)]
Merge pull request #23904 from xiexingguo/wip-27985

osd/PG: fix misused FORCE_RECOVERY[BACKFILL] flags

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: David Zafman <dzafman@redhat.com>
6 years agoMerge PR #25254 into master
Patrick Donnelly [Fri, 25 Jan 2019 18:52:47 +0000 (10:52 -0800)]
Merge PR #25254 into master

* refs/pull/25254/head:
mds: optimize resuming stale caps
client: avoid unnecessary wakeup when handling RENEWCAPS
client: set cap->wanted when adding new cap
client: don't wakeup cap waiters twice when mds recovered
mds: optimize revoking stale caps
mds: put notable caps at the front of session's caps list
mds: track if client has writeable range in Capability
mds: add session pointer to Capability
client: skip updating 'wanted' caps if caps are already issued
client: sync 'retain caps' logical from kernel client

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
6 years agoMerge pull request #25497 from jmolmo/wip_ansible_orchestrator
Sebastian Wagner [Fri, 25 Jan 2019 18:37:22 +0000 (19:37 +0100)]
Merge pull request #25497 from jmolmo/wip_ansible_orchestrator

mgr/ansible: Create/Remove OSDs

Reviewed-by: Jeff Layton <jlayton@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>
Reviewed-by: Sébastien Han <seb@redhat.com>
6 years agoMerge pull request #25848 from majianpeng/debug-buffer-compiler-error
Kefu Chai [Fri, 25 Jan 2019 17:03:05 +0000 (01:03 +0800)]
Merge pull request #25848 from majianpeng/debug-buffer-compiler-error

common/buffer: fix compiler bug when enable DEBUG_BUFFER.

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
6 years agoMerge pull request #26022 from theanalyst/common-mclock-prio
Kefu Chai [Fri, 25 Jan 2019 14:07:17 +0000 (22:07 +0800)]
Merge pull request #26022 from theanalyst/common-mclock-prio

common, osd: set mclock priority as 1 by default.

Reviewed-by: J. Eric Ivancich <ivancich@redhat.com>
6 years agoMerge pull request #26043 from tchaikov/wip-37975
Kefu Chai [Fri, 25 Jan 2019 14:06:27 +0000 (22:06 +0800)]
Merge pull request #26043 from tchaikov/wip-37975

osd: unmount store after service.shutdown()

Reviewed-by: Neha Ojha <nojha@redhat.com>
6 years agoMerge pull request #26053 from Aran85/pipeline-fix
Kefu Chai [Fri, 25 Jan 2019 14:05:58 +0000 (22:05 +0800)]
Merge pull request #26053 from Aran85/pipeline-fix

osd: ec saves a write access to the memory under most circumstances

Reviewed-by: Kefu Chai <kchai@redhat.com>
6 years agoMerge pull request #26119 from smithfarm/wip-fix-suse-buildrequires
Ricardo Marques [Fri, 25 Jan 2019 12:55:38 +0000 (12:55 +0000)]
Merge pull request #26119 from smithfarm/wip-fix-suse-buildrequires

rpm: fix xmlsec1 build dependency for dashboard make check

Reviewed-by: Ricardo Marques <rimarques@suse.com>
6 years agomds: optimize resuming stale caps 25254/head
Yan, Zheng [Mon, 10 Dec 2018 03:37:32 +0000 (11:37 +0800)]
mds: optimize resuming stale caps

If client doesn't want any cap, there is no need to re-issue stale
caps.

A special case is that client wants some caps, but skipped updating
'wanted'. For this case, client needs to update 'wanted' when stale
session get renewed.

Fixes: http://tracker.ceph.com/issues/38043
Signed-off-by: "Yan, Zheng" <zyan@redhat.com>
6 years agoMerge pull request #26120 from idryomov/wip-doc-osd-map-config
Ilya Dryomov [Fri, 25 Jan 2019 07:51:36 +0000 (08:51 +0100)]
Merge pull request #26120 from idryomov/wip-doc-osd-map-config

doc/rados/configuration: refresh osdmap section

Reviewed-by: Neha Ojha <nojha@redhat.com>
6 years agoMerge pull request #26122 from neha-ojha/wip-fix-crush-tunables
Neha Ojha [Fri, 25 Jan 2019 03:47:35 +0000 (19:47 -0800)]
Merge pull request #26122 from neha-ojha/wip-fix-crush-tunables

qa/suites/rados/thrash: change crush_tunables to jewel in rados_api_tests

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
6 years agoMerge pull request #26057 from yaozongyou/remove-unused-class-read_log_and_missing_error
Kefu Chai [Fri, 25 Jan 2019 02:48:51 +0000 (10:48 +0800)]
Merge pull request #26057 from yaozongyou/remove-unused-class-read_log_and_missing_error

osd: remove unused class read_log_and_missing_error

Reviewed-by: Kefu Chai <kchai@redhat.com>
6 years agoMerge pull request #26101 from neha-ojha/wip-pglog-overrides
Neha Ojha [Fri, 25 Jan 2019 01:15:21 +0000 (17:15 -0800)]
Merge pull request #26101 from neha-ojha/wip-pglog-overrides

qa/overrides/short_pg_log.yaml: reduce osd_{min,max}_pg_log_entries

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
6 years agoqa/suites/rados/thrash: change crush_tunables to jewel in rados_api_tests 26122/head
Neha Ojha [Fri, 25 Jan 2019 00:51:27 +0000 (16:51 -0800)]
qa/suites/rados/thrash: change crush_tunables to jewel in rados_api_tests

Fixes: http://tracker.ceph.com/issues/38042
Signed-off-by: Neha Ojha <nojha@redhat.com>
6 years agodoc/rados/configuration: refresh osdmap section 26120/head
Ilya Dryomov [Thu, 24 Jan 2019 20:07:09 +0000 (21:07 +0100)]
doc/rados/configuration: refresh osdmap section

"osd map cache size" and "osd map message max" were reduced in commit
855955e58e63 ("osd: reduce size of osdmap cache, messages").

"osd map cache bl size" and "osd map cache bl inc size" were removed
six years ago.

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
6 years agorpm: fix xmlsec1 build dependency for dashboard make check 26119/head
Nathan Cutler [Thu, 24 Jan 2019 18:31:20 +0000 (19:31 +0100)]
rpm: fix xmlsec1 build dependency for dashboard make check

Fixes: 46d286c7f71f4c43766d2a3d7597816346e36151
Signed-off-by: Nathan Cutler <ncutler@suse.com>
6 years agoMerge pull request #25796 from linuxbox2/luminous-fix-ldap-secret
Matt Benjamin [Thu, 24 Jan 2019 17:13:51 +0000 (12:13 -0500)]
Merge pull request #25796 from linuxbox2/luminous-fix-ldap-secret

rgw: fix ldap secret parsing

6 years agoMerge pull request #26084 from ceph/wip-rm24871
Sébastien Han [Thu, 24 Jan 2019 14:23:19 +0000 (15:23 +0100)]
Merge pull request #26084 from ceph/wip-rm24871

ceph-volume: mark a device not available if it belongs to ceph-disk

6 years agoMerge pull request #25472 from s0nea/wip-dashboard-osd-recv-speed-validation
Lenz Grimmer [Thu, 24 Jan 2019 13:33:11 +0000 (14:33 +0100)]
Merge pull request #25472 from s0nea/wip-dashboard-osd-recv-speed-validation

mgr/dashboard: Validate the OSD recovery priority form input values

Reviewed-by: Kanika Murarka <kmurarka@redhat.com>
Reviewed-by: Stephan Müller <smueller@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>
6 years agoMerge pull request #26034 from alfredodeza/wip-rm37963
Andrew Schoen [Thu, 24 Jan 2019 13:20:32 +0000 (07:20 -0600)]
Merge pull request #26034 from alfredodeza/wip-rm37963

ceph-volume fix TypeError on dmcrypt when using Python3

Reviewed-by: Andrew Schoen <aschoen@redhat.com>
6 years agoMerge pull request #26039 from xiexingguo/wip-37940
Xie Xingguo [Thu, 24 Jan 2019 00:52:56 +0000 (08:52 +0800)]
Merge pull request #26039 from xiexingguo/wip-37940

osd/OSDMap: be more aggressive when trying to balance

Reviewed-by: Sage Weil <sage@redhat.com>
6 years agoqa/overrides/short_pg_log.yaml: reduce osd_{min,max}_pg_log_entries 26101/head
Neha Ojha [Wed, 23 Jan 2019 20:46:42 +0000 (12:46 -0800)]
qa/overrides/short_pg_log.yaml: reduce osd_{min,max}_pg_log_entries

Fixes: http://tracker.ceph.com/issues/38025
Signed-off-by: Neha Ojha <nojha@redhat.com>
6 years agoMerge PR #26017 into master
Patrick Donnelly [Wed, 23 Jan 2019 18:06:34 +0000 (10:06 -0800)]
Merge PR #26017 into master

* refs/pull/26017/head:
ceph: cleanup mds ceph status output

Reviewed-by: Sage Weil <sage@redhat.com>
6 years agoMerge pull request #26079 from trociny/wip-38006
Jason Dillaman [Wed, 23 Jan 2019 16:48:05 +0000 (11:48 -0500)]
Merge pull request #26079 from trociny/wip-38006

librbd: properly filter out trashed non-user images on purge

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
6 years agoMerge pull request #25984 from tspmelo/wip-fix-summary
Lenz Grimmer [Wed, 23 Jan 2019 15:48:58 +0000 (16:48 +0100)]
Merge pull request #25984 from tspmelo/wip-fix-summary

mgr/dashboard: Fix summary refresh call stack

Reviewed-by: Ricardo Marques <rimarques@suse.com>
Reviewed-by: Sebastian Krah <skrah@suse.com>
Reviewed-by: Stephan Müller <smueller@suse.com>
6 years agomgr/dashboard: Fix summary refresh call stack 25984/head
Tiago Melo [Fri, 11 Jan 2019 12:08:01 +0000 (11:08 -0100)]
mgr/dashboard: Fix summary refresh call stack

Currently each time we created a task it would call the summary refresh method.
Since that method was a recursive method, overtime we would get N calls to the
refresh each 5 seconds.

Signed-off-by: Tiago Melo <tmelo@suse.com>
6 years agoMerge pull request #25988 from Exotelis/branding
Lenz Grimmer [Wed, 23 Jan 2019 14:21:39 +0000 (15:21 +0100)]
Merge pull request #25988 from Exotelis/branding

mgr/dashboard: Add guideline how to brand the UI and update the color scheme

Reviewed-by: Kanika Murarka <kmurarka@redhat.com>
Reviewed-by: Tatjana Dehler <tdehler@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>
6 years agomgr/ansible: Solve merge problems 25497/head
Juan Miguel Olmo Martínez [Wed, 23 Jan 2019 12:49:07 +0000 (13:49 +0100)]
mgr/ansible: Solve merge problems

Add/fix <forgotten> things after a <git rebase --onto>

Signed-off-by: Juan Miguel Olmo Martínez <jolmomar@redhat.com>
6 years agomgr/dashboard: add validators to osdRecvSpeedForm 25472/head
Tatjana Dehler [Mon, 17 Dec 2018 11:26:24 +0000 (12:26 +0100)]
mgr/dashboard: add validators to osdRecvSpeedForm

Fixes: https://tracker.ceph.com/issues/37436
Signed-off-by: Tatjana Dehler <tdehler@suse.com>
6 years agomgr/ansible: Adapt code to use DriveGroupSpec and OrchestratorClientMixin
Juan Miguel Olmo Martínez [Wed, 23 Jan 2019 08:23:33 +0000 (09:23 +0100)]
mgr/ansible: Adapt code to use DriveGroupSpec and OrchestratorClientMixin

Apart of this two major changes, and due the long time this PR has been opened, other minor modifications coming from the orchestrators world has been merged.

Not solved yet the discussion around <osd add/rm vs osd create/remove>. Once we have a a common code base we can decide what names we should use and what implementation is the more adequate.

Signed-off-by: Juan Miguel Olmo Martínez <jolmomar@redhat.com>
6 years agomgr/ansible: Add create/remove osd functionality
Juan Miguel Olmo Martínez [Tue, 11 Dec 2018 19:29:46 +0000 (20:29 +0100)]
mgr/ansible: Add create/remove osd functionality

- Create/remove osd functionality
- Addressed typo find by @tchaikov
- Improved code in functional orchestrator methods following @leseb suggestions
- Addressed @sebastian-philipp  suggestions
- Use <drive group> specification for create OSD in all the orchestrators
- fixed errors calling HandleCommandResult

Signed-off-by: Juan Miguel Olmo Martínez <jolmomar@redhat.com>
6 years agoMerge pull request #25991 from jtlayton/wip-dashboard
Lenz Grimmer [Wed, 23 Jan 2019 11:04:35 +0000 (12:04 +0100)]
Merge pull request #25991 from jtlayton/wip-dashboard

mgr/dashboard: add a script to run an API request on a rook cluster

Reviewed-by: Ricardo Dias <rdias@suse.com>
6 years agomgr/dashboard: do a little clean-up of the component
Tatjana Dehler [Tue, 11 Dec 2018 08:45:30 +0000 (09:45 +0100)]
mgr/dashboard: do a little clean-up of the component

- order component methods alphabetically
- add missing newline
- uncheck checkbox and clear validations if a new profile has been choosen

Signed-off-by: Tatjana Dehler <tdehler@suse.com>
6 years agomgr/dashboard: convert ``priorityAttrs`` from array to object
Tatjana Dehler [Tue, 11 Dec 2018 14:51:58 +0000 (15:51 +0100)]
mgr/dashboard: convert ``priorityAttrs`` from array to object

Convert ``priorityAttrs`` from array to object for better handling,
e.g. to get rid of unnecessary for-loops.

Signed-off-by: Tatjana Dehler <tdehler@suse.com>
6 years agomgr/dashboard: call API only once to get config options infos
Tatjana Dehler [Fri, 7 Dec 2018 14:26:32 +0000 (15:26 +0100)]
mgr/dashboard: call API only once to get config options infos

Refactor the component in order to call API only once to get the infos
about the relevant config options and (if set) the current values.

Signed-off-by: Tatjana Dehler <tdehler@suse.com>
6 years agomgr/dashboard: move config option types to their own class
Tatjana Dehler [Thu, 29 Nov 2018 14:51:34 +0000 (15:51 +0100)]
mgr/dashboard: move config option types to their own class

Move the config option types information into their own class to
reuse them easily in other components.

Signed-off-by: Tatjana Dehler <tdehler@suse.com>
6 years agomgr/dashboard: Fix linting errors 25988/head
Sebastian Krah [Wed, 23 Jan 2019 08:20:51 +0000 (09:20 +0100)]
mgr/dashboard: Fix linting errors

Fixes linting error in two files

Signed-off-by: Sebastian Krah <skrah@suse.com>
6 years agoceph-volume: mark a device not available if it belongs to ceph-disk 26084/head
Andrew Schoen [Tue, 22 Jan 2019 15:32:00 +0000 (09:32 -0600)]
ceph-volume: mark a device not available if it belongs to ceph-disk

The `ceph-volume inventory` command will now show if a device is being
used by ceph-disk and mark it not available if so.

Fixes: https://tracker.ceph.com/issues/24871
Signed-off-by: Andrew Schoen <aschoen@redhat.com>
6 years agoMerge pull request #25774 from yehudasa/wip-rgw-tiering-3
Matt Benjamin [Tue, 22 Jan 2019 17:42:21 +0000 (12:42 -0500)]
Merge pull request #25774 from yehudasa/wip-rgw-tiering-3

rgw: rados tiering

6 years agolibrbd: properly filter out trashed non-user images on purge 26079/head
Mykola Golub [Tue, 22 Jan 2019 15:54:20 +0000 (17:54 +0200)]
librbd: properly filter out trashed non-user images on purge

Fixes: https://tracker.ceph.com/issues/38006
Signed-off-by: Mykola Golub <mgolub@suse.com>
6 years agoMerge pull request #26073 from rhcs-dashboard/add-tox-ini-command
Lenz Grimmer [Tue, 22 Jan 2019 15:23:05 +0000 (16:23 +0100)]
Merge pull request #26073 from rhcs-dashboard/add-tox-ini-command

mgr/dashboard: added command to tox.ini

Reviewed-by: Sebastian Wagner <swagner@suse.com>
Reviewed-by: Stephan Müller <smueller@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>
6 years agoMerge pull request #26020 from rhcs-dashboard/issue-37854
Lenz Grimmer [Tue, 22 Jan 2019 15:12:05 +0000 (16:12 +0100)]
Merge pull request #26020 from rhcs-dashboard/issue-37854

mgr/dashboard: Fixed performance details context for host list row selection

Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Kanika Murarka <kmurarka@redhat.com>
6 years agoMerge pull request #26007 from rhcs-dashboard/upgrade-python-dev-deps
Lenz Grimmer [Tue, 22 Jan 2019 14:52:02 +0000 (15:52 +0100)]
Merge pull request #26007 from rhcs-dashboard/upgrade-python-dev-deps

mgr/dashboard: upgraded python dev dependencies

Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Laura Paduano <lpaduano@suse.com>
Reviewed-by: Patrick Nawracay <pnawracay@suse.com>
Reviewed-by: Sebastian Wagner <swagner@suse.com>
6 years agomgr/dashboard: added command to tox.ini 26073/head
alfonsomthd [Tue, 22 Jan 2019 12:26:38 +0000 (13:26 +0100)]
mgr/dashboard: added command to tox.ini

* Added 'run' command to run arbitrary commands so
  you don't have to run the whole test/lint suite.
  Use case examples:
  - Run only 1 test:
  tox -e py3-run pytest tests/test_rgw_client.py::RgwClientTest::test_ssl_verify
  - Run pycodestyle for only 1 file:
  tox -e py3-run pycodestyle tests/test_rgw_client.py

Signed-off-by: Alfonso Martínez <almartin@redhat.com>
6 years agoMerge pull request #26032 from jtlayton/wip-rook
Sebastian Wagner [Tue, 22 Jan 2019 09:28:25 +0000 (10:28 +0100)]
Merge pull request #26032 from jtlayton/wip-rook

mgr/rook: remove dead code and fix bug in url fetching code

Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>
6 years agoMerge pull request #25895 from jan--f/c-v-func-test-local
Andrew Schoen [Mon, 21 Jan 2019 17:40:47 +0000 (11:40 -0600)]
Merge pull request #25895 from jan--f/c-v-func-test-local

ceph-volume: functional tests, add libvirt customization

Reviewed-by: Andrew Schoen <aschoen@redhat.com>
6 years agoMerge pull request #26044 from tchaikov/wip-build-doc-pyyaml
Kefu Chai [Mon, 21 Jan 2019 17:00:15 +0000 (01:00 +0800)]
Merge pull request #26044 from tchaikov/wip-build-doc-pyyaml

doc: bump up sphinx and pyyaml versions

Reviewed-by: Alfredo Deza <adeza@redhat.com>
6 years agodoc: bump up sphinx and pyyaml versions 26044/head
Kefu Chai [Sun, 20 Jan 2019 06:51:11 +0000 (14:51 +0800)]
doc: bump up sphinx and pyyaml versions

* bump up pyyaml version to address CVE-2017-18342, see
  https://nvd.nist.gov/vuln/detail/CVE-2017-18342
* bump up sphinx to the latest stable

Signed-off-by: Kefu Chai <kchai@redhat.com>
6 years agoMerge pull request #26052 from tchaikov/wip-37976
Kefu Chai [Mon, 21 Jan 2019 15:04:55 +0000 (23:04 +0800)]
Merge pull request #26052 from tchaikov/wip-37976

mgr/zabbix: drop "total_objects" field

Reviewed-by: Sage Weil <sage@redhat.com>
6 years agoMerge pull request #26054 from tchaikov/wip-crimson-dispatcher-fix-ftbfs
Kefu Chai [Mon, 21 Jan 2019 14:30:26 +0000 (22:30 +0800)]
Merge pull request #26054 from tchaikov/wip-crimson-dispatcher-fix-ftbfs

crimson/osd: remove "force_new" from ms_get_authorizer()

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
6 years agoMerge pull request #26045 from hsiang41/Ticket_33736
Kefu Chai [Mon, 21 Jan 2019 14:17:10 +0000 (22:17 +0800)]
Merge pull request #26045 from hsiang41/Ticket_33736

mgr/diskprediction_cloud: fix divide by zero when total_size is 0

Reviewed-by: Kefu Chai <kchai@redhat.com>
6 years agoosd: remove unused class read_log_and_missing_error 26057/head
Yao Zongyou [Mon, 21 Jan 2019 13:10:12 +0000 (21:10 +0800)]
osd: remove unused class read_log_and_missing_error

Signed-off-by: Yao Zongyou <yaozongyou@vip.qq.com>
6 years agocrimson/osd: remove "force_new" from ms_get_authorizer() 26054/head
Kefu Chai [Mon, 21 Jan 2019 11:26:10 +0000 (19:26 +0800)]
crimson/osd: remove "force_new" from ms_get_authorizer()

it was removed in e62f45d1, so update crimson-osd accordingly

Signed-off-by: Kefu Chai <kchai@redhat.com>
6 years agoosd: ec saves a write access to the memory under most circumstances 26053/head
Zengran Zhang [Mon, 21 Jan 2019 11:19:16 +0000 (19:19 +0800)]
osd: ec saves a write access to the memory under most circumstances

Signed-off-by: Kefu Chai <kchai@redhat.com>
Signed-off-by: Zengran Zhang <zhangzengran@sangfor.com.cn>
6 years agoMerge pull request #26046 from liewegas/wip-37976
Kefu Chai [Mon, 21 Jan 2019 10:44:41 +0000 (18:44 +0800)]
Merge pull request #26046 from liewegas/wip-37976

mgr/telemetry: fix total_objects

Reviewed-by: Kefu Chai <kchai@redhat.com>
6 years agomgr/zabbix: drop "total_objects" field 26052/head
Kefu Chai [Mon, 21 Jan 2019 10:38:24 +0000 (18:38 +0800)]
mgr/zabbix: drop "total_objects" field

This field was removed from df output a while back in
342f309

Signed-off-by: Kefu Chai <kchai@redhat.com>
6 years agoMerge pull request #25712 from familyuu/configuration
Lenz Grimmer [Mon, 21 Jan 2019 08:34:35 +0000 (09:34 +0100)]
Merge pull request #25712 from familyuu/configuration

mgr/dashboard: Add a 'clear filter' button to configuration page

Reviewed-by: Sebastian Krah <skrah@suse.com>
Reviewed-by: Tatjana Dehler <tdehler@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>
6 years agomgr/diskprediction_cloud: fix divide by zero when total_size is 0 26045/head
hsiang41 [Sun, 20 Jan 2019 13:22:24 +0000 (21:22 +0800)]
mgr/diskprediction_cloud: fix divide by zero when total_size is 0

Fix devide by zero when total_size is o.

Ref ticket: https://tracker.ceph.com/issues/37736

Signed-off-by: Rick Chen <rick.chen@prophetstor.com>
6 years agomgr/telemetry: fix total_objects 26046/head
Sage Weil [Sun, 20 Jan 2019 21:59:50 +0000 (15:59 -0600)]
mgr/telemetry: fix total_objects

This field was removed from df output a while back in
342f309645df886fb96eb401634e38376553e6d9

Fixes: http://tracker.ceph.com/issues/37976
Signed-off-by: Sage Weil <sage@redhat.com>
6 years agoMerge PR #26031 into master
Sage Weil [Sun, 20 Jan 2019 21:55:50 +0000 (15:55 -0600)]
Merge PR #26031 into master

* refs/pull/26031/head:
mgr/BaseMgrModule: fix mon commands targetting a specific monitor
mgr/ActivePyModules: fix RestoreThread calls for get('devices')

Reviewed-by: Kefu Chai <kchai@redhat.com>
6 years agoosd: unmount store after service.shutdown() 26043/head
Kefu Chai [Sun, 20 Jan 2019 05:05:26 +0000 (13:05 +0800)]
osd: unmount store after service.shutdown()

in service.shutdown(), the store's cache is accessed, so we need to
destroy the store after service.shutdown()

Fixes: http://tracker.ceph.com/issues/37975
Signed-off-by: Kefu Chai <kchai@redhat.com>
6 years agoMerge pull request #25490 from Aran85/fix-ec-using-cache
Kefu Chai [Sun, 20 Jan 2019 02:33:51 +0000 (10:33 +0800)]
Merge pull request #25490 from Aran85/fix-ec-using-cache

osd: keep using cache even if op will invalid cache

Reviewed-by: Kefu Chai <kchai@redhat.com>
6 years agoMerge pull request #25920 from wjwithagen/wjw-fix-freebsd-numa
Kefu Chai [Sun, 20 Jan 2019 02:32:21 +0000 (10:32 +0800)]
Merge pull request #25920 from wjwithagen/wjw-fix-freebsd-numa

common/numa: Add shim routines for NUMA on FreeBSD.

6 years agoMerge pull request #25938 from majianpeng/msg-remove-memcopy-message-header
Kefu Chai [Sun, 20 Jan 2019 02:31:26 +0000 (10:31 +0800)]
Merge pull request #25938 from majianpeng/msg-remove-memcopy-message-header

msg/async: reduce additional ceph_msg_header copy.

Reviewed-by: Xie Xingguo <xie.xingguo@zte.com.cn>
Reviewed-by: Kefu Chai <kchai@redhat.com>
6 years agoMerge pull request #26021 from majianpeng/osd-remove-unused-code
Kefu Chai [Sun, 20 Jan 2019 02:30:34 +0000 (10:30 +0800)]
Merge pull request #26021 from majianpeng/osd-remove-unused-code

osd: remove unused fields

Reviewed-by: Kefu Chai <kchai@redhat.com>
6 years agoMerge pull request #26040 from tchaikov/wip-blkdev-stat
Kefu Chai [Sun, 20 Jan 2019 02:28:22 +0000 (10:28 +0800)]
Merge pull request #26040 from tchaikov/wip-blkdev-stat

common/blkdev: check retval of stat()

Reviewed-by: Sage Weil <sage@redhat.com>
6 years agoMerge pull request #26041 from tchaikov/wip-kvstore-tool-gcc-ice
Kefu Chai [Sun, 20 Jan 2019 02:27:52 +0000 (10:27 +0800)]
Merge pull request #26041 from tchaikov/wip-kvstore-tool-gcc-ice

tools/ceph_kvstore_tool: extract StoreTool into kvstore_tool.cc

Reviewed-by: Sage Weil <sage@redhat.com>
6 years agoceph: cleanup mds ceph status output 26017/head
Patrick Donnelly [Thu, 17 Jan 2019 18:14:17 +0000 (10:14 -0800)]
ceph: cleanup mds ceph status output

This is primarily to make the CephFS line more readable with lots of file
systems with more understandable output.

From something like:

    mds: a-2/2/2 up  {0=c=up:active,1=b=up:active}, 1 up:standby

Now with 1 FS and 1 active:

    mds: a:1 {0=p=up:active}, 15 up:standby

2 FS, 2 actives:

    mds: a:1 b:1 {a:0=o=up:active,b:0=n=up:active}, 14 up:standby

2 FS, 3 actives:

    mds: a:1 b:2 {a:0=o=up:active,b:0=n=up:active,b:1=p=up:active}, 13 up:standby

3 FS and 2 active and 1 reconnect:

    mds: a:1/1 b:1 c:1 {a:0=n=up:reconnect,b:0=p=up:active,c:0=o=up:active}, 13 up:standby

3 FS and 1 failed rank:

    mds: a:0/1 b:1 c:1 {b:0=p=up:active,c:0=o=up:active}, 14 up:standby, 1 failed

3 FS, 4 actives:

    mds: a:1 b:2 c:1 {a:0=o=up:active,b:0=n=up:active,b:1=p=up:active,c:0=l=up:active}, 12 up:standby

3 FS, 3 actives, 1 FS failed:

    mds: a:1 b:2 c:0/1 {a:0=o=up:active,b:0=n=up:active,b:1=p=up:active}, 13 up:standby, 1 failed

3 FS, 5 actives:

    mds: a:1 b:2 c:2 5 up:active, 11 up:standby

4 FS:

    mds: 4 fs {a:0=o=up:active,b:0=n=up:active,c:0=l=up:active,d:0=p=up:active}, 12 up:standby

5 FS:

    mds: 5 fs 5 up:active, 11 up:standby

5 FS, 1 degraded:

    mds: 5 fs (degraded: c:0/1) {a:0=o=up:active,b:0=n=up:active,d:0=p=up:active,e:0=m=up:active}, 12 up:standby, 1 failed

8 FS, 1 degraded:

    mds: 8 fs (degraded: c:0/1) 7 up:active, 9 up:standby, 1 failed

Fixes: http://tracker.ceph.com/issues/37954
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
6 years agoMerge PR #26012 into master
Patrick Donnelly [Sun, 20 Jan 2019 01:21:42 +0000 (17:21 -0800)]
Merge PR #26012 into master

* refs/pull/26012/head:
qa: add test that down fs does not ERR
mon/MDSMonitor: skip offline ERR for down fs

Reviewed-by: Douglas Fuller <dfuller@redhat.com>
6 years agoMerge PR #25993 into master
Patrick Donnelly [Sun, 20 Jan 2019 01:21:09 +0000 (17:21 -0800)]
Merge PR #25993 into master

* refs/pull/25993/head:
mds/FSMap: simplify encode/decode
mds: alias reference type for Filesystem
mds: add missing memory header include
include/encoding: eliminate smart_ptr temporaries

Reviewed-by: Zheng Yan <zyan@redhat.com>
6 years agoMerge PR #25976 into master
Patrick Donnelly [Sun, 20 Jan 2019 01:20:18 +0000 (17:20 -0800)]
Merge PR #25976 into master

* refs/pull/25976/head:
Objecter::calc_op_budget: Fix invalid access to extent union member

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
Reviewed-by: Zheng Yan <zyan@redhat.com>
6 years agoMerge PR #25974 into master
Patrick Donnelly [Sun, 20 Jan 2019 01:19:19 +0000 (17:19 -0800)]
Merge PR #25974 into master

* refs/pull/25974/head:
MDSMonitor: add missing OSDMonitor writeable check
MDSMonitor: codify constraint for writeable OSDMap

Reviewed-by: Sage Weil <sage@redhat.com>
6 years agoMerge PR #25973 into master
Patrick Donnelly [Sun, 20 Jan 2019 01:18:08 +0000 (17:18 -0800)]
Merge PR #25973 into master

* refs/pull/25973/head:
qa: use simpler fs fail to bring fs down
MDSMonitor: add fs fail command

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Douglas Fuller <dfuller@redhat.com>
6 years agoMerge PR #25971 into master
Patrick Donnelly [Sun, 20 Jan 2019 01:17:04 +0000 (17:17 -0800)]
Merge PR #25971 into master

* refs/pull/25971/head:
qa: test hex dump_inode
pybind/ceph_argparse: accept int with non-decimal base

Reviewed-by: Kefu Chai <kchai@redhat.com>
6 years agoMerge PR #25650 into master
Patrick Donnelly [Sun, 20 Jan 2019 01:16:17 +0000 (17:16 -0800)]
Merge PR #25650 into master

* refs/pull/25650/head:
client: add ceph_stat/ceph_lstat/ceph_fstat api

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Jeff Layton <jlayton@redhat.com>
6 years agotool/ceph_kvstore_tool: extract duration calc out 26041/head
Kefu Chai [Sat, 19 Jan 2019 16:48:40 +0000 (00:48 +0800)]
tool/ceph_kvstore_tool: extract duration calc out

simplify the code a little bit

Signed-off-by: Kefu Chai <kchai@redhat.com>
6 years agotools/ceph_kvstore_tool: extract StoreTool into kvstore_tool.cc
Kefu Chai [Sat, 19 Jan 2019 16:01:55 +0000 (00:01 +0800)]
tools/ceph_kvstore_tool: extract StoreTool into kvstore_tool.cc

to workaround n bug in GCC 7.3, which leads to ICE like

build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/14.0.1-2751-gb268822/rpm/el7/BUILD/ceph-14.0.1-2751-gb268822/src/tools/ceph_kvstore_tool.cc:625:1:
internal compiler error: Segmentation fault
 }
 ^
Please submit a full bug report,
with preprocessed source if appropriate.
See <http://bugzilla.redhat.com/bugzilla> for instructions.

Signed-off-by: Kefu Chai <kchai@redhat.com>
6 years agomgr/BaseMgrModule: fix mon commands targetting a specific monitor 26031/head
Sage Weil [Fri, 18 Jan 2019 21:14:19 +0000 (15:14 -0600)]
mgr/BaseMgrModule: fix mon commands targetting a specific monitor

We just need to use the right variant of start_mon_command().

Fixes: http://tracker.ceph.com/issues/37967
Signed-off-by: Sage Weil <sage@redhat.com>
6 years agomgr/ActivePyModules: fix RestoreThread calls for get('devices')
Sage Weil [Fri, 18 Jan 2019 15:24:34 +0000 (09:24 -0600)]
mgr/ActivePyModules: fix RestoreThread calls for get('devices')

Fixes 61aa7e2e0230bff49e757c32932d1db4bcad5f67

Fixes: http://tracker.ceph.com/issues/37736
Signed-off-by: Sage Weil <sage@redhat.com>
6 years agocommon/blkdev: check retval of stat() 26040/head
Kefu Chai [Sat, 19 Jan 2019 13:09:59 +0000 (21:09 +0800)]
common/blkdev: check retval of stat()

Signed-off-by: Kefu Chai <kchai@redhat.com>
6 years agoosd/OSDMap: be more aggressive when trying to balance 26039/head
xie xingguo [Fri, 18 Jan 2019 10:14:52 +0000 (18:14 +0800)]
osd/OSDMap: be more aggressive when trying to balance

Previously we'd require the absolute deviation >= 1 before
an osd can fill in the overfull or underfull set, which as
a result can get some osds stuck severely underfull forever.

This patch tries to get us out of those corner cases mentioned
above and can be scrutinized from two aspects:
- a standard deviation is introduced to evaluate the efficiency
  of balancing, therefore making the distribution of pgs always
  converge to the perfect status (standard deviation == 0).
- populate overfull or underfull osds more aggressively by
  gradually allowing the absolute deviations converging towards
  to 0 instead of 1.

It turns out the balancer module works even better now after
applying this patch. E.g.:
```
OSD=5 MON=1 MGR=1 MDS=0 ../src/vstart.sh -x -l -b -n -d
bin/ceph osd set-require-min-compat-client luminous
bin/ceph balancer mode upmap
bin/ceph balancer on
bin/ceph osd pool create rbd 117
// wait until automatic balancing is done
bin/ceph osd pool create aaa 133
```
__before__:
```
ID CLASS WEIGHT  REWEIGHT SIZE   RAW USE DATA    OMAP META  AVAIL   %USE  VAR  PGS STATUS
 0   hdd 0.00980  1.00000 10 GiB 1.1 GiB 3.9 MiB  0 B 1 GiB 9.0 GiB 10.60 1.00 151     up
 1   hdd 0.00980  1.00000 10 GiB 1.1 GiB 3.9 MiB  0 B 1 GiB 9.0 GiB 10.60 1.00 153     up
 2   hdd 0.00980  1.00000 10 GiB 1.1 GiB 3.9 MiB  0 B 1 GiB 9.0 GiB 10.60 1.00 147     up
 3   hdd 0.00980  1.00000 10 GiB 1.1 GiB 3.9 MiB  0 B 1 GiB 9.0 GiB 10.60 1.00 149     up
 4   hdd 0.00980  1.00000 10 GiB 1.1 GiB 3.9 MiB  0 B 1 GiB 9.0 GiB 10.60 1.00 150     up
                    TOTAL 50 GiB 5.3 GiB  19 MiB  0 B 5 GiB  45 GiB 10.60
```
__after__:
```
ID CLASS WEIGHT  REWEIGHT SIZE   RAW USE DATA    OMAP META  AVAIL   %USE  VAR  PGS STATUS
 0   hdd 0.00980  1.00000 10 GiB 1.1 GiB 6.2 MiB  0 B 1 GiB 9.0 GiB 10.62 1.00 150     up
 1   hdd 0.00980  1.00000 10 GiB 1.1 GiB 6.2 MiB  0 B 1 GiB 9.0 GiB 10.62 1.00 151     up
 2   hdd 0.00980  1.00000 10 GiB 1.1 GiB 6.2 MiB  0 B 1 GiB 9.0 GiB 10.62 1.00 149     up
 3   hdd 0.00980  1.00000 10 GiB 1.1 GiB 6.2 MiB  0 B 1 GiB 9.0 GiB 10.62 1.00 150     up
 4   hdd 0.00980  1.00000 10 GiB 1.1 GiB 6.2 MiB  0 B 1 GiB 9.0 GiB 10.62 1.00 150     up
                    TOTAL 50 GiB 5.3 GiB  31 MiB  0 B 5 GiB  45 GiB 10.62
```

Fixes: http://tracker.ceph.com/issues/37940
Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
6 years agoqa: use simpler fs fail to bring fs down 25973/head
Patrick Donnelly [Tue, 15 Jan 2019 22:10:09 +0000 (14:10 -0800)]
qa: use simpler fs fail to bring fs down

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
6 years agorgw: align head object size with pool alignment 25774/head
Yehuda Sadeh [Sat, 19 Jan 2019 05:04:50 +0000 (21:04 -0800)]
rgw: align head object size with pool alignment

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
6 years agoMerge pull request #25919 from yuriw/wip-yuriw-distros-master
David Galloway [Fri, 18 Jan 2019 18:03:17 +0000 (13:03 -0500)]
Merge pull request #25919 from yuriw/wip-yuriw-distros-master

qa/tests: added rhel 7.6

6 years agoceph-volume tests verify behavior of create_dmcrypt_key in python 3 26034/head
Alfredo Deza [Fri, 18 Jan 2019 17:51:04 +0000 (12:51 -0500)]
ceph-volume tests verify behavior of create_dmcrypt_key in python 3

Signed-off-by: Alfredo Deza <adeza@redhat.com>
6 years agoceph-volume util.encryption ensure int is used for urandom (py3 compat)
Alfredo Deza [Fri, 18 Jan 2019 17:50:38 +0000 (12:50 -0500)]
ceph-volume util.encryption ensure int is used for urandom (py3 compat)

Signed-off-by: Alfredo Deza <adeza@redhat.com>
6 years agoMerge PR #26019 into master
Sage Weil [Fri, 18 Jan 2019 17:27:07 +0000 (11:27 -0600)]
Merge PR #26019 into master

* refs/pull/26019/head:
mon: implement hiding commands in ceph tool

Reviewed-by: Sage Weil <sage@redhat.com>
6 years agomgr/rook: wrap url fetch function in try/except block 26032/head
Jeff Layton [Fri, 18 Jan 2019 17:15:18 +0000 (12:15 -0500)]
mgr/rook: wrap url fetch function in try/except block

It's possible for the fetch of this object to fail if the service is
being torn down. Handle that situation gracefully, but log a message.

Signed-off-by: Jeff Layton <jlayton@redhat.com>
6 years agoMerge pull request #25996 from yehudasa/wip-37885
Casey Bodley [Fri, 18 Jan 2019 16:45:29 +0000 (11:45 -0500)]
Merge pull request #25996 from yehudasa/wip-37885

qa/rgw/tempest: either force os_type or select random distro

Reviewed-by: Casey Bodley <cbodley@redhat.com>
6 years agoMerge pull request #26015 from noahdesu/osd-find-fsid
Noah Watkins [Fri, 18 Jan 2019 16:26:47 +0000 (08:26 -0800)]
Merge pull request #26015 from noahdesu/osd-find-fsid

cli: dump osd-fsid as part of osd find <id>

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Sébastien Han <seb@redhat.com>
6 years agoMerge pull request #22797 from dzafman/wip-19753
David Zafman [Fri, 18 Jan 2019 15:42:00 +0000 (07:42 -0800)]
Merge pull request #22797 from dzafman/wip-19753

osd: Deny reservation if expected backfill size would put us over bac…

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>
6 years agoMerge PR #25132 into master
Sage Weil [Fri, 18 Jan 2019 15:38:59 +0000 (09:38 -0600)]
Merge PR #25132 into master

* refs/pull/25132/head:
os/bluestore: implement kv update on bluefs extents info change.
os/bluestore: use original algorithm to calculate BlueFS gift size for explicit allocation.
os/bluestore: get rid of bluefs space gifting performed in background
os/bluestore: do KV/FM/allocator opening in two stages.
os/bluestore: detach FreelistManager from DB instance.
test/osd/store_test: add test case for explicit bluefs expanding.
os/bluestore: implement explicit bluefs space allocation at slow device
os/bluestore: unify allocator dump on bluefs space failures
os/bluestore: stop keeping bluefs_extents at KV

Reviewed-by: Sage Weil <sage@redhat.com>
6 years agomgr/rook: remove init_rook
Jeff Layton [Fri, 18 Jan 2019 15:23:21 +0000 (10:23 -0500)]
mgr/rook: remove init_rook

John evidently added this code early on in development, when we figured
we might be taking over a cluster that is already running under k8s.

I don't see how we'd ever make that work, so just remove this function
for now, and the commented-out callsite.

Reported-by: Sebastian Wagner <sebastian.wagner@suse.com>
Signed-off-by: Jeff Layton <jlayton@redhat.com>
6 years agoMerge pull request #25972 from noahdesu/doc-release-ver-sort
Kefu Chai [Fri, 18 Jan 2019 14:52:50 +0000 (22:52 +0800)]
Merge pull request #25972 from noahdesu/doc-release-ver-sort

doc: sort releases by date and version

Reviewed-by: Nathan Cutler <ncutler@suse.com>
6 years agoMerge pull request #25891 from ifed01/wip-ifed-fix-compress-errors
Kefu Chai [Fri, 18 Jan 2019 14:41:08 +0000 (22:41 +0800)]
Merge pull request #25891 from ifed01/wip-ifed-fix-compress-errors

os/bluestore: do not assert on non-zero err codes from  compress() call

Reviewed-by: Sage Weil <sage@redhat.com>
6 years agoMerge pull request #15168 from qwren/qat-crypto
Kefu Chai [Fri, 18 Jan 2019 14:39:13 +0000 (22:39 +0800)]
Merge pull request #15168 from qwren/qat-crypto

crypto: add openssl support for RGW encryption

Reviewed-by: Kefu Chai <kchai@redhat.com>
6 years agoMerge pull request #25961 from p-na/wip-pna-fix-argparse
Kefu Chai [Fri, 18 Jan 2019 14:38:21 +0000 (22:38 +0800)]
Merge pull request #25961 from p-na/wip-pna-fix-argparse

ceph_argparse: fix --verbose

Reviewed-by: Kefu Chai <kchai@redhat.com>
6 years agoMerge pull request #25324 from votdev/fix_raw_input
Alfredo Deza [Fri, 18 Jan 2019 14:19:27 +0000 (09:19 -0500)]
Merge pull request #25324 from votdev/fix_raw_input

ceph-volume: Adapt code to support Python3

Reviewed-by: Alfredo Deza <adeza@redhat.com>
6 years agoMerge pull request #25912 from sebastian-philipp/orchestrator-drive-group
Sebastian Wagner [Fri, 18 Jan 2019 13:59:48 +0000 (14:59 +0100)]
Merge pull request #25912 from sebastian-philipp/orchestrator-drive-group

mgr/orchestrator: Extend DriveGroupSpec