]> git.apps.os.sepia.ceph.com Git - ceph.git/log
ceph.git
5 years agodoc: document rank option for journal reset 31201/head
Patrick Donnelly [Mon, 28 Oct 2019 18:13:04 +0000 (11:13 -0700)]
doc: document rank option for journal reset

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
5 years agoMerge pull request #31180 from liewegas/revert-bluestore-pinned-list
Kefu Chai [Mon, 28 Oct 2019 15:18:02 +0000 (23:18 +0800)]
Merge pull request #31180 from liewegas/revert-bluestore-pinned-list

os/bluestore: revert cache pinned list

Reviewed-by: Mark Nelson <mnelson@redhat.com>
5 years agoMerge pull request #31071 from sebastian-philipp/python-common-mypy
Sebastian Wagner [Mon, 28 Oct 2019 14:07:08 +0000 (15:07 +0100)]
Merge pull request #31071 from sebastian-philipp/python-common-mypy

python-common: Add mypy testing

Reviewed-by: Jan Fajerski <jfajerski@suse.com>
5 years agoMerge pull request #30995 from wjwithagen/wjw-wip-volume-zfs-inventory
Jan Fajerski [Mon, 28 Oct 2019 13:57:37 +0000 (14:57 +0100)]
Merge pull request #30995 from wjwithagen/wjw-wip-volume-zfs-inventory

ceph-volume-zfs: add the inventory command

5 years agoMerge pull request #31014 from tchaikov/wip-38378
Kefu Chai [Mon, 28 Oct 2019 12:53:07 +0000 (20:53 +0800)]
Merge pull request #31014 from tchaikov/wip-38378

mgr/dashboard: accept expected exception when SSL handshaking

Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Sebastian Wagner <swagner@suse.com>
5 years agoMerge pull request #31092 from liewegas/wip-revert-42026-test
Kefu Chai [Mon, 28 Oct 2019 12:36:13 +0000 (20:36 +0800)]
Merge pull request #31092 from liewegas/wip-revert-42026-test

Revert "test: librados startup/shutdown racer test"

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
Reviewed-by: Jeff Layton <jlayton@redhat.com>
5 years agoMerge pull request #31153 from ceph/wip-restful-node-items
Kefu Chai [Mon, 28 Oct 2019 12:29:20 +0000 (20:29 +0800)]
Merge pull request #31153 from ceph/wip-restful-node-items

restful: Query nodes_by_id for items

Reviewed-by: Kefu Chai <kchai@redhat.com>
5 years agoMerge pull request #31176 from tchaikov/wip/mgr/diskprediction_cloud/cleanup
Kefu Chai [Mon, 28 Oct 2019 12:27:22 +0000 (20:27 +0800)]
Merge pull request #31176 from tchaikov/wip/mgr/diskprediction_cloud/cleanup

mgr/diskprediction_cloud: refactor timeout() decorator

Reviewed-by: Rick Chen <rick.chen@prophetstor.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
5 years agoMerge pull request #31175 from tchaikov/wip/mon/OSDMonitor/cleanup
Kefu Chai [Mon, 28 Oct 2019 12:24:48 +0000 (20:24 +0800)]
Merge pull request #31175 from tchaikov/wip/mon/OSDMonitor/cleanup

mon/OSDMonitor: use initializer_list<> for {si,iec}_options

Reviewed-by: Sage Weil <sage@redhat.com>
5 years agoMerge pull request #30279 from runsisi/wip-fix-strerror_r
Kefu Chai [Mon, 28 Oct 2019 12:11:14 +0000 (20:11 +0800)]
Merge pull request #30279 from runsisi/wip-fix-strerror_r

common: fix compat of strerror_r

Reviewed-by: Kefu Chai <kchai@redhat.com>
5 years agoMerge pull request #29836 from majianpeng/fix-async-msgr-perfcounter-bug
Kefu Chai [Mon, 28 Oct 2019 12:09:58 +0000 (20:09 +0800)]
Merge pull request #29836 from majianpeng/fix-async-msgr-perfcounter-bug

msg/async: Don't dec(msgr_active_connections) if conn still in accept…

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Ricardo Dias <rdias@suse.com>
5 years agoMerge pull request #31156 from cyx1231st/wip-seastar-msgr-testpeer
Kefu Chai [Mon, 28 Oct 2019 04:27:11 +0000 (12:27 +0800)]
Merge pull request #31156 from cyx1231st/wip-seastar-msgr-testpeer

test/crimson: implement a remote async TestPeer for crimson msgr tests

Reviewed-by: Kefu Chai <kchai@redhat.com>
5 years agotest/crimson: add nonstop option in async TestPeer 31156/head
Yingxin Cheng [Fri, 25 Oct 2019 02:51:08 +0000 (10:51 +0800)]
test/crimson: add nonstop option in async TestPeer

If set, TestPeer won't shutdown if all tests are done, and will wait for
another round of tests.

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
5 years agotest/crimson: add delay to markdown_peer()
Yingxin Cheng [Fri, 25 Oct 2019 02:15:04 +0000 (10:15 +0800)]
test/crimson: add delay to markdown_peer()

Let test sleep awhile for peer markdown propagated.

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
5 years agotest/crimson: async msgr TestPeer for v2 failover tests
Yingxin Cheng [Wed, 23 Oct 2019 04:42:11 +0000 (12:42 +0800)]
test/crimson: async msgr TestPeer for v2 failover tests

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
5 years agotest/crimson: allow v2 failover tests to connect to remote TestPeer
Yingxin Cheng [Wed, 23 Oct 2019 02:48:54 +0000 (10:48 +0800)]
test/crimson: allow v2 failover tests to connect to remote TestPeer

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
5 years agotest/crimson: extract msgr test commands to header
Yingxin Cheng [Tue, 22 Oct 2019 02:24:08 +0000 (10:24 +0800)]
test/crimson: extract msgr test commands to header

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
5 years agoMerge pull request #30610 from ivancich/wip-no-schedule-reshard-during-reshard
J. Eric Ivancich [Sun, 27 Oct 2019 18:47:39 +0000 (14:47 -0400)]
Merge pull request #30610 from ivancich/wip-no-schedule-reshard-during-reshard

rgw: prevent bucket reshard scheduling if bucket is resharding

Reviewed-by: Matt Benjamin <mbenjamin@redhat.com>
Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: Abhishek Lekshmanan <abhishek@suse.com>
5 years agoMerge pull request #30795 from ivancich/wip-dynamic-reshard-adjustments
J. Eric Ivancich [Sun, 27 Oct 2019 18:46:01 +0000 (14:46 -0400)]
Merge pull request #30795 from ivancich/wip-dynamic-reshard-adjustments

rgw: adjust allowable bucket index shard counts for dynamic resharding

Reviewed-by: Casey Bodley <cbodley@redhat.com>
5 years agoMerge pull request #30845 from croit/rgw-bucket-stats-add-num-shards
J. Eric Ivancich [Sun, 27 Oct 2019 18:44:47 +0000 (14:44 -0400)]
Merge pull request #30845 from croit/rgw-bucket-stats-add-num-shards

rgw: add num_shards to radosgw-admin bucket stats

Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: Abhishek Lekshmanan <abhishek@suse.com>
Reviewed-by: J. Eric Ivancich <ivancich@redhat.com>
5 years agoMerge pull request #30956 from joke-lee/rgw-storage-class-post-obj
J. Eric Ivancich [Sun, 27 Oct 2019 18:42:27 +0000 (14:42 -0400)]
Merge pull request #30956 from joke-lee/rgw-storage-class-post-obj

rgw: set correct storage class for post object upload

Reviewed-by: Abhishek Lekshmanan <abhishek@suse.com>
5 years agoMerge pull request #30970 from theanalyst/rgw/list-common-prefix
J. Eric Ivancich [Sun, 27 Oct 2019 18:39:46 +0000 (14:39 -0400)]
Merge pull request #30970 from theanalyst/rgw/list-common-prefix

rgw: url encode common prefixes for List Objects response

Reviewed-by: Casey Bodley <cbodley@redhat.com>
5 years agoMerge pull request #31179 from liewegas/cleanup-exec-bit
Kefu Chai [Sun, 27 Oct 2019 15:58:59 +0000 (23:58 +0800)]
Merge pull request #31179 from liewegas/cleanup-exec-bit

os/bluestore,comon,erasure-code: chmod -x source files

Reviewed-by: Kefu Chai <kchai@redhat.com>
5 years agoMerge pull request #31171 from liewegas/bug-42496
Kefu Chai [Sun, 27 Oct 2019 15:57:57 +0000 (23:57 +0800)]
Merge pull request #31171 from liewegas/bug-42496

qa/tasks/cbt: run stop-all.sh while shutting down

Reviewed-by: Kefu Chai <kchai@redhat.com>
5 years agoRevert "Merge pull request #30964 from markhpc/wip-bs-cache-trim-pinned" 31180/head
Sage Weil [Sun, 27 Oct 2019 14:59:10 +0000 (09:59 -0500)]
Revert "Merge pull request #30964 from markhpc/wip-bs-cache-trim-pinned"

This reverts commit 304c37f52165363cad4aaf31386d62ea5a573425, reversing
changes made to 7b4f9a083f0c8cc015f7b0cc12a3bc88db632610.

This causes some bluestore test failures due to an ENOENT right after a
new object is created.  The simplest reproducer is the
ObjectStore/StoreTest.FiemapEmpty/2 test.

Fixes: https://tracker.ceph.com/issues/42495
Signed-off-by: Sage Weil <sage@redhat.com>
5 years agoos/bluestore,comon,erasure-code: chmod -x source files 31179/head
Sage Weil [Sun, 27 Oct 2019 14:25:53 +0000 (09:25 -0500)]
os/bluestore,comon,erasure-code: chmod -x source files

Fixes

*** WARNING: ./usr/src/debug/ceph-15.0.0-6512.g62bd825.el8.x86_64/src/common/obj_bencher.cc is executable but has empty or no shebang, removing executable bit
*** WARNING: ./usr/src/debug/ceph-15.0.0-6512.g62bd825.el8.x86_64/src/erasure-code/shec/determinant.c is executable but has empty or no shebang, removing executable bit
*** WARNING: ./usr/src/debug/ceph-15.0.0-6512.g62bd825.el8.x86_64/src/os/bluestore/AvlAllocator.cc is executable but has empty or no shebang, removing executable bit
*** WARNING: ./usr/src/debug/ceph-15.0.0-6512.g62bd825.el8.x86_64/src/os/bluestore/AvlAllocator.h is executable but has empty or no shebang, removing executable bit

in rpm build.

Signed-off-by: Sage Weil <sage@redhat.com>
5 years agoMerge PR #31170 into master
Sage Weil [Sun, 27 Oct 2019 14:21:56 +0000 (09:21 -0500)]
Merge PR #31170 into master

* refs/pull/31170/head:
src/telemetry: remove, now lives in ceph-telemetry.git

Reviewed-by: Sage Weil <sage@redhat.com>
5 years agoMerge PR #31131 into master
Sage Weil [Sun, 27 Oct 2019 14:20:51 +0000 (09:20 -0500)]
Merge PR #31131 into master

* refs/pull/31131/head:
crush: remove invalid upmap items

Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>
5 years agoMerge PR #31154 into master
Sage Weil [Sun, 27 Oct 2019 14:19:56 +0000 (09:19 -0500)]
Merge PR #31154 into master

* refs/pull/31154/head:
auth/cephx: always initialize local variables

Reviewed-by: Neha Ojha <nojha@redhat.com>
5 years agoMerge PR #31167 into master
Sage Weil [Sun, 27 Oct 2019 14:19:40 +0000 (09:19 -0500)]
Merge PR #31167 into master

* refs/pull/31167/head:
os/bluestore: do not mark per_pool_omap updated unless we fixed it

Reviewed-by: Igor Fedotov <ifedotov@suse.com>
5 years agomgr/dashboard: accept socket error 0 31014/head
Kefu Chai [Sun, 27 Oct 2019 03:10:22 +0000 (11:10 +0800)]
mgr/dashboard: accept socket error 0

see
https://github.com/cherrypy/cheroot/commit/9d73226ab28a07a38cbb6816edf9ff46be4f9de3

Fixes: https://tracker.ceph.com/issues/38378
Signed-off-by: Kefu Chai <kchai@redhat.com>
5 years agomgr/dashboard: accept exceptions from builtin SSL
Kefu Chai [Sun, 27 Oct 2019 03:03:20 +0000 (11:03 +0800)]
mgr/dashboard: accept exceptions from builtin SSL

see also https://github.com/cherrypy/cheroot/pull/4

so we don't panic when client is trying to talk with us with an
unsupported protocol, the exception should be accepted, and the
client can fallback to supported protocol.

Fixes: https://tracker.ceph.com/issues/38378
Signed-off-by: Kefu Chai <kchai@redhat.com>
5 years agomgr/dashboard: extract monkey patches for cherrypy out
Kefu Chai [Sun, 27 Oct 2019 02:57:20 +0000 (10:57 +0800)]
mgr/dashboard: extract monkey patches for cherrypy out

so they are less distracting to the dashboard developers. and probably
less scaring.

Signed-off-by: Kefu Chai <kchai@redhat.com>
5 years agomgr/diskprediction_cloud: refactor timeout() decorator 31176/head
Kefu Chai [Sun, 27 Oct 2019 02:00:01 +0000 (10:00 +0800)]
mgr/diskprediction_cloud: refactor timeout() decorator

* timeout() is never passed any parameter when being called, so let's
  remove the parameters list of "seconds" and "error_message"
* use `getattr()` instead of `hasattr()` for retrieving the
  member variable of `self`
* pass `self` to wrapper function explicitly.
* return `func()` right away.
* hardwire the error message of `TimeoutError` to "Timer expired",
  because
  - as neither errno.ETIME nor errno.ETIMEOUT is portable
  - the only caller of `TimeoutError` is `timeout()`, so there is
    no need to have the flexibility to pass a different error message
* use `wraps()` as a decorator, simpler this way.

Signed-off-by: Kefu Chai <kchai@redhat.com>
5 years agomon/OSDMonitor: use initializer_list<> for {si,iec}_options 31175/head
Kefu Chai [Sun, 27 Oct 2019 01:43:39 +0000 (09:43 +0800)]
mon/OSDMonitor: use initializer_list<> for {si,iec}_options

* use initializer_list<> for {si,iec}_options, no need to uset set<>
* remove the comments, the variable names are self-documented.

Signed-off-by: Kefu Chai <kchai@redhat.com>
5 years agoMerge pull request #31010 from pdvian/wip-fix-target-max
Kefu Chai [Sun, 27 Oct 2019 01:38:23 +0000 (09:38 +0800)]
Merge pull request #31010 from pdvian/wip-fix-target-max

mon/OSDMonitor.cc: Allow pool set target_max_(objects/bytes) with SI/IEC units

Reviewed-by: Kefu Chai <kchai@redhat.com>
5 years agoMerge pull request #30434 from smithfarm/wip-41820
Kefu Chai [Sun, 27 Oct 2019 01:18:16 +0000 (09:18 +0800)]
Merge pull request #30434 from smithfarm/wip-41820

qa: enable dashboard tests to be run with "--suite rados/dashboard"

Reviewed-by: Laura Paduano <lpaduano@suse.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
5 years agoMerge pull request #31005 from tchaikov/wip/qa/tasks/ceph/cleanup
Kefu Chai [Sun, 27 Oct 2019 01:16:37 +0000 (09:16 +0800)]
Merge pull request #31005 from tchaikov/wip/qa/tasks/ceph/cleanup

qa/tasks/ceph.py: remove unused variables

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>
5 years agoqa/tasks/cbt: run stop-all.sh when finishing up 31171/head
Sage Weil [Sat, 26 Oct 2019 00:20:05 +0000 (19:20 -0500)]
qa/tasks/cbt: run stop-all.sh when finishing up

stop-all.sh will work if the right deps are there (currently we lack 'nc')

also killall -9 java to be sure.

Fixes: https://tracker.ceph.com/issues/42496
Signed-off-by: Sage Weil <sage@redhat.com>
5 years agoMerge PR #31086 into master
Sage Weil [Sat, 26 Oct 2019 16:42:31 +0000 (11:42 -0500)]
Merge PR #31086 into master

* refs/pull/31086/head:
doc/bootstrap.rst: fix github's url

Reviewed-by: Sage Weil <sage@redhat.com>
5 years agoMerge PR #31169 into master
Sage Weil [Sat, 26 Oct 2019 15:55:19 +0000 (10:55 -0500)]
Merge PR #31169 into master

* refs/pull/31169/head:
mgr/ssh: implement (synchronous) describe_service
ceph-daemon: ls: replace 'active' bool with 'state' enum
ceph-daemon: include container_id and version in 'ls' output
ceph-daemon: tolerate systemctl is-* exit codes
mgr/ssh: modernize timeout config option

Reviewed-by: Sebastian Wagner <swagner@suse.com>
5 years agodoc/bootstrap.rst: fix github's url 31086/head
Alexandre Bruyelles [Wed, 23 Oct 2019 13:31:38 +0000 (15:31 +0200)]
doc/bootstrap.rst: fix github's url

Signed-off-by: Alexandre Bruyelles <jack@jack.fr.eu.org>
5 years agoMerge pull request #31126 from wjwithagen/wjw-fix-std-bitset
Kefu Chai [Sat, 26 Oct 2019 01:30:54 +0000 (09:30 +0800)]
Merge pull request #31126 from wjwithagen/wjw-fix-std-bitset

rgw: Select the std::bitset to resolv ambiguity

Reviewed-by: Adam C. Emerson <aemerson@redhat.com>
Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
5 years agosrc/telemetry: remove, now lives in ceph-telemetry.git 31170/head
Dan Mick [Fri, 25 Oct 2019 23:44:05 +0000 (16:44 -0700)]
src/telemetry: remove, now lives in ceph-telemetry.git

Signed-off-by: Dan Mick <dan.mick@redhat.com>
5 years agomgr/ssh: implement (synchronous) describe_service 31169/head
Sage Weil [Fri, 25 Oct 2019 17:50:50 +0000 (12:50 -0500)]
mgr/ssh: implement (synchronous) describe_service

Signed-off-by: Sage Weil <sage@redhat.com>
5 years agoceph-daemon: ls: replace 'active' bool with 'state' enum
Sage Weil [Fri, 25 Oct 2019 23:43:18 +0000 (18:43 -0500)]
ceph-daemon: ls: replace 'active' bool with 'state' enum

('running', 'inactive', 'error', 'unknown')

Signed-off-by: Sage Weil <sage@redhat.com>
5 years agoceph-daemon: include container_id and version in 'ls' output
Sage Weil [Fri, 25 Oct 2019 22:50:26 +0000 (17:50 -0500)]
ceph-daemon: include container_id and version in 'ls' output

Signed-off-by: Sage Weil <sage@redhat.com>
5 years agoceph-daemon: tolerate systemctl is-* exit codes
Sage Weil [Fri, 25 Oct 2019 21:46:09 +0000 (16:46 -0500)]
ceph-daemon: tolerate systemctl is-* exit codes

Signed-off-by: Sage Weil <sage@redhat.com>
5 years agomgr/ssh: modernize timeout config option
Sage Weil [Thu, 24 Oct 2019 13:21:39 +0000 (08:21 -0500)]
mgr/ssh: modernize timeout config option

(and rename it)

Signed-off-by: Sage Weil <sage@redhat.com>
5 years agoMerge PR #31141 into master
Sage Weil [Fri, 25 Oct 2019 19:30:33 +0000 (14:30 -0500)]
Merge PR #31141 into master

* refs/pull/31141/head:
ceph-daemon: fix warning message
ceph-daemon: fix enable with conflicting ceph.target files
test_ceph_daemon.sh: use same fsid as qa/standalone/test_ceph_daemon.sh
test_ceph_daemon.sh: use other latest-master image
ceph-daemon: fix pathify (fix shell --config/--keyring)
ceph-daemon: /var/run/ceph -> /var/run/ceph/$fsid

Reviewed-by: Sebastian Wagner <swagner@suse.com>
5 years agoceph-daemon: fix warning message 31141/head
Sage Weil [Fri, 25 Oct 2019 17:46:52 +0000 (12:46 -0500)]
ceph-daemon: fix warning message

Signed-off-by: Sage Weil <sage@redhat.com>
5 years agoMerge pull request #31164 from tchaikov/wip/doc/rados/fix-typo
Josh Durgin [Fri, 25 Oct 2019 17:41:42 +0000 (10:41 -0700)]
Merge pull request #31164 from tchaikov/wip/doc/rados/fix-typo

doc/rados/deployment/ceph-deploy-mon: fix typo

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
5 years agoceph-daemon: fix enable with conflicting ceph.target files
Sage Weil [Fri, 25 Oct 2019 16:26:32 +0000 (11:26 -0500)]
ceph-daemon: fix enable with conflicting ceph.target files

+ make description better

Signed-off-by: Sage Weil <sage@redhat.com>
5 years agoos/bluestore: do not mark per_pool_omap updated unless we fixed it 31167/head
Sage Weil [Fri, 25 Oct 2019 16:02:28 +0000 (11:02 -0500)]
os/bluestore: do not mark per_pool_omap updated unless we fixed it

We only fix the per-pool-omap issues if we do a non-shallow fsck.

Fixes: https://tracker.ceph.com/issues/42490
Signed-off-by: Sage Weil <sage@redhat.com>
5 years agodoc/rados/deployment/ceph-deploy-mon: fix typo 31164/head
Kefu Chai [Fri, 25 Oct 2019 14:38:11 +0000 (22:38 +0800)]
doc/rados/deployment/ceph-deploy-mon: fix typo

s/comingling/commingling/

Signed-off-by: Kefu Chai <kchai@redhat.com>
5 years agoMerge pull request #31113 from sungjunyoung/do_cmake-warning
Kefu Chai [Fri, 25 Oct 2019 14:24:42 +0000 (22:24 +0800)]
Merge pull request #31113 from sungjunyoung/do_cmake-warning

do_cmake: Warn user about slow debug performance only for not set

Reviewed-by: Kefu Chai <kchai@redhat.com>
5 years agoMerge PR #31130 into master
Sage Weil [Fri, 25 Oct 2019 14:18:20 +0000 (09:18 -0500)]
Merge PR #31130 into master

* refs/pull/31130/head:
ceph-daemon: only set up crash dir mount if it exists

Reviewed-by: Sebastian Wagner <swagner@suse.com>
5 years agoMerge PR #31138 into master
Sage Weil [Fri, 25 Oct 2019 14:11:36 +0000 (09:11 -0500)]
Merge PR #31138 into master

* refs/pull/31138/head:
mon: fix tell to hybrid octopus/pre-octopus mons

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>
5 years agorestful: Use node_id for _gather_leaf_ids 31153/head
Boris Ranto [Fri, 25 Oct 2019 12:24:19 +0000 (14:24 +0200)]
restful: Use node_id for _gather_leaf_ids

The _gather_leaf_ids function doesn't need the node structure, it only
needs the id.

Signed-off-by: Boris Ranto <branto@redhat.com>
5 years agorestful: Query nodes_by_id for items
Boris Ranto [Thu, 24 Oct 2019 14:54:05 +0000 (16:54 +0200)]
restful: Query nodes_by_id for items

The node dict that is passed to the _gather_leaf_ids function from the
_gather_osds function does not have 'items' in it. We also can't use
buckets at this point since those only exist for leaf nodes, not all
nodes.

We need to query the nodes_by_id dict to get 'items' for a node inside
the _gather_leaf_ids function instead.

Signed-off-by: Boris Ranto <branto@redhat.com>
5 years agopython-common: Add mypy testing 31071/head
Sebastian Wagner [Wed, 16 Oct 2019 11:14:36 +0000 (13:14 +0200)]
python-common: Add mypy testing

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
5 years agoMerge pull request #31006 from zy751713126/fix_features
Jason Dillaman [Fri, 25 Oct 2019 12:34:25 +0000 (08:34 -0400)]
Merge pull request #31006 from zy751713126/fix_features

librbd: fix rbd_features_to_string output

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
5 years agoMerge pull request #30955 from runsisi/wip-fix-dup-lock
Jason Dillaman [Fri, 25 Oct 2019 12:33:51 +0000 (08:33 -0400)]
Merge pull request #30955 from runsisi/wip-fix-dup-lock

librbd: force reacquire lock if blacklist is disabled

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
5 years agoMerge pull request #30954 from mxdInspur/thickimg_create_progress
Jason Dillaman [Fri, 25 Oct 2019 12:33:12 +0000 (08:33 -0400)]
Merge pull request #30954 from mxdInspur/thickimg_create_progress

rbd:  creating thick-provision image progress percent info exceeds 100%

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
5 years agoMerge pull request #31143 from badone/wip-tracker-38466-enable-librabbitmq-devel
Kefu Chai [Fri, 25 Oct 2019 11:15:43 +0000 (19:15 +0800)]
Merge pull request #31143 from badone/wip-tracker-38466-enable-librabbitmq-devel

ceph.spec.in: enable amqp_endpoint on RHEL8 by default

Reviewed-by: Kefu Chai <kchai@redhat.com>
5 years agoMerge pull request #31110 from tchaikov/wip-cmake/enable-seastar-with-dpdk
Kefu Chai [Fri, 25 Oct 2019 10:00:03 +0000 (18:00 +0800)]
Merge pull request #31110 from tchaikov/wip-cmake/enable-seastar-with-dpdk

cmake: support `Seastar_DPDK=ON` option

Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: Yingxin Cheng <yingxin.cheng@intel.com>
5 years agoauth/cephx: always initialize local variables 31154/head
Kefu Chai [Fri, 25 Oct 2019 08:42:55 +0000 (16:42 +0800)]
auth/cephx: always initialize local variables

to silence GCC warnings like:

rc/auth/cephx/CephxProtocol.h:309:5: warning: 'type' may be used uninitialized in this function [-Wmaybe-uninitialized]
     if (i != tickets_map.end())
     ^~

Signed-off-by: Kefu Chai <kchai@redhat.com>
5 years agoceph.spec.in: Enable amqp_endpoint on RHEL8 by default 31143/head
Brad Hubbard [Fri, 25 Oct 2019 05:29:37 +0000 (15:29 +1000)]
ceph.spec.in: Enable amqp_endpoint on RHEL8 by default

RHEL/CentOS 8 now provide librabbitmq-devel so we can enable it as a
build requirement.

Fixes: https://tracker.ceph.com/issues/38466
Signed-off-by: Brad Hubbard <bhubbard@redhat.com>
5 years agotest_ceph_daemon.sh: use same fsid as qa/standalone/test_ceph_daemon.sh
Sage Weil [Fri, 25 Oct 2019 02:27:27 +0000 (21:27 -0500)]
test_ceph_daemon.sh: use same fsid as qa/standalone/test_ceph_daemon.sh

That way these scripts both clean up for each other (and are obviously
bogus clusters).

Signed-off-by: Sage Weil <sage@redhat.com>
5 years agotest_ceph_daemon.sh: use other latest-master image
Sage Weil [Fri, 25 Oct 2019 02:26:21 +0000 (21:26 -0500)]
test_ceph_daemon.sh: use other latest-master image

This one is newer?

Signed-off-by: Sage Weil <sage@redhat.com>
5 years agoceph-daemon: fix pathify (fix shell --config/--keyring)
Sage Weil [Fri, 25 Oct 2019 02:25:59 +0000 (21:25 -0500)]
ceph-daemon: fix pathify (fix shell --config/--keyring)

The relative path works on one of my machines but not my laptop; the
full CWD works everywhere.  This is presumably better anyway!

Signed-off-by: Sage Weil <sage@redhat.com>
5 years agoceph-daemon: /var/run/ceph -> /var/run/ceph/$fsid
Sage Weil [Fri, 25 Oct 2019 02:03:34 +0000 (21:03 -0500)]
ceph-daemon: /var/run/ceph -> /var/run/ceph/$fsid

This is better than having a single /var/run/ceph on the host with a
weird naming scheme.  Among other things, it means that we can access
the asok for any daemon for a given fsid from any container on the same
host with the same fsid (notably, a shell).

Signed-off-by: Sage Weil <sage@redhat.com>
5 years agomon: fix tell to hybrid octopus/pre-octopus mons 31138/head
Sage Weil [Thu, 24 Oct 2019 23:56:04 +0000 (18:56 -0500)]
mon: fix tell to hybrid octopus/pre-octopus mons

We can't decide whether to use the new tell command style based on the
monmap.min_mon_release alone because some mons may be octopus even though
that hasn't updated yet.  The same goes for if we look at the combined
features for the cluster--the underlying problem is the monmap doesn't
tell us which mons are octopus and which ones aren't, so we don't know
how to behave.

Instead, allow octopus+ mons to advertise the converted tell commands
going forward, for compatibility with pre-octopus clients (who do the old
style of tell) and for octopus+ clients talking to a min_mon_release <
octopus cluster.

Signed-off-by: Sage Weil <sage@redhat.com>
5 years agoceph-daemon: only set up crash dir mount if it exists 31130/head
Sage Weil [Thu, 24 Oct 2019 13:41:33 +0000 (08:41 -0500)]
ceph-daemon: only set up crash dir mount if it exists

Sometimes we run containers on a host that doesn't have a crash dir set
up (becuase no daemon has been deployed).  Examples include shell and
ceph-volume.

Signed-off-by: Sage Weil <sage@redhat.com>
5 years agoMerge pull request #31133 from dzafman/wip-42476
David Zafman [Fri, 25 Oct 2019 00:23:48 +0000 (17:23 -0700)]
Merge pull request #31133 from dzafman/wip-42476

ceph-objectstore-tool: call collection_bits() crashes on the meta col…

Reviewed-by: Sage Weil <sage@redhat.com>
5 years agoMerge pull request #31135 from athanatos/sjust/wip-alien-echo-fix
Kefu Chai [Thu, 24 Oct 2019 23:50:32 +0000 (07:50 +0800)]
Merge pull request #31135 from athanatos/sjust/wip-alien-echo-fix

test_alien_echo: update to use crimson:: namespace

Reviewed-by: Kefu Chai <kchai@redhat.com>
5 years agoMerge PR #31121 into master
Sage Weil [Thu, 24 Oct 2019 23:29:52 +0000 (18:29 -0500)]
Merge PR #31121 into master

* refs/pull/31121/head:
mon/MonClient: fix mon tell to older mons

Reviewed-by: Neha Ojha <nojha@redhat.com>
5 years agoMerge PR #31095 into master
Patrick Donnelly [Thu, 24 Oct 2019 22:17:48 +0000 (15:17 -0700)]
Merge PR #31095 into master

* refs/pull/31095/head:
qa: do not check pg count for new data_isolated volume

Reviewed-by: Sage Weil <sage@redhat.com>
5 years agotest/crimson/test_alien_echo: update to use crimson:: namespace 31135/head
Samuel Just [Thu, 24 Oct 2019 20:18:29 +0000 (13:18 -0700)]
test/crimson/test_alien_echo: update to use crimson:: namespace

See: bd023211d999f1e2b70f48254507524d0763bd33
Signed-off-by: Samuel Just <sjust@redhat.com>
5 years agoMerge pull request #30964 from markhpc/wip-bs-cache-trim-pinned
Mark Nelson [Thu, 24 Oct 2019 19:50:11 +0000 (14:50 -0500)]
Merge pull request #30964 from markhpc/wip-bs-cache-trim-pinned

os/bluestore: Keep separate onode cache pinned list.

5 years agoceph-objectstore-tool: call collection_bits() crashes on the meta collection 31133/head
David Zafman [Thu, 24 Oct 2019 18:31:52 +0000 (11:31 -0700)]
ceph-objectstore-tool: call collection_bits() crashes on the meta collection

Skip new check for meta collection
test:
    Turn off osd_pool_default_pg_autoscale_mode just like bash tests do
    Fix test by checking for new error message

Caused by: f88b3534549bb2ede76e9f3e8b6b641b5886a9f4

Fixes: https://tracker.ceph.com/issues/42476
Signed-off-by: David Zafman <dzafman@redhat.com>
5 years agoMerge pull request #31132 from smithfarm/wip-42474
Yuri Weinstein [Thu, 24 Oct 2019 17:43:47 +0000 (10:43 -0700)]
Merge pull request #31132 from smithfarm/wip-42474

build-integration-branch: take PRs in chronological order

5 years agodo_cmake: Disable shell command printing feature before cmake execution 31113/head
Junyoung, Sung [Thu, 24 Oct 2019 17:17:59 +0000 (02:17 +0900)]
do_cmake: Disable shell command printing feature before cmake execution

Signed-off-by: Junyoung, Sung <junyoung.sung@navercorp.com>
5 years agobuild-integration-branch: change shebang to python3 31132/head
Nathan Cutler [Thu, 24 Oct 2019 16:54:55 +0000 (18:54 +0200)]
build-integration-branch: change shebang to python3

The word "python" in the shebang means Python 2, but Python 2 is deprecated and
should no longer be used.

Signed-off-by: Nathan Cutler <ncutler@suse.com>
5 years agobuild-integration-branch: take PRs in chronological order
Nathan Cutler [Thu, 24 Oct 2019 16:29:02 +0000 (18:29 +0200)]
build-integration-branch: take PRs in chronological order

Fixes: https://tracker.ceph.com/issues/42474
Signed-off-by: Nathan Cutler <ncutler@suse.com>
5 years agocrush: remove invalid upmap items 31131/head
huangjun [Thu, 24 Oct 2019 15:44:12 +0000 (23:44 +0800)]
crush: remove invalid upmap items

We can not cancel in verify_upmap if remap an osd to different root bucket,

cluster topology:
osd.0 ~ osd.29 belongs to datacenter 1
osd30 ~ osd.59 belongs to datacenter 2
crush rule:
  take datacenter 1
  chooseleaf 2 host
  emit
  take datacenter 2
  chooseleaf 2 host
  emit
The pg's primary osd in datacenter 1.

We should cancel the pgid from upmap_items like below
1) from [26,12,54,46] to [30,12,54,46]
pg_upmap_items [26,30]

2) from [16,25,53,31] to [40,25,53,0]
pg_upmap_items [16,20,31,0]

Signed-off-by: huangjun <huangjun@xsky.com>
5 years agodo_cmake: Warn user about slow debug performance only for not set
Junyoung, Sung [Thu, 24 Oct 2019 07:57:57 +0000 (16:57 +0900)]
do_cmake: Warn user about slow debug performance only for not set

Signed-off-by: Junyoung, Sung <junyoung.sung@navercorp.com>
5 years agorgw: Select the std::bitset to resolv ambiguity 31126/head
Willem Jan Withagen [Thu, 24 Oct 2019 15:09:56 +0000 (17:09 +0200)]
rgw: Select the std::bitset to resolv ambiguity

Signed-off-by: Willem Jan Withagen <wjw@digiware.nl>
5 years agomon/MonClient: fix mon tell to older mons 31121/head
Sage Weil [Thu, 24 Oct 2019 13:00:32 +0000 (08:00 -0500)]
mon/MonClient: fix mon tell to older mons

For older mons, we need to call into send_command so that we trigger the
compat code that futzes with our mon sessions.

Signed-off-by: Sage Weil <sage@redhat.com>
5 years agoMerge PR #30848 into master
Sage Weil [Thu, 24 Oct 2019 11:20:29 +0000 (06:20 -0500)]
Merge PR #30848 into master

* refs/pull/30848/head:
mon/PGMap: drop indentation on 'df' human output

Reviewed-by: Kefu Chai <kchai@redhat.com>
5 years agocmake: support `Seastar_DPDK=ON` option 31110/head
Kefu Chai [Thu, 24 Oct 2019 06:14:40 +0000 (14:14 +0800)]
cmake: support `Seastar_DPDK=ON` option

seastar does not build any 3rd party libraries. but we do. currently, we
build DPDK/SPDK when they are enabled for the messenger backend and
bluestore backend.

in this change, the same function is reused for preparing the DPDK
libraries if `Seastar_DPDK` is enabled. probably we should outsource
this work to install-deps.sh in future, but before that, let's keep
this as a builtin "feature".

Signed-off-by: Kefu Chai <kchai@redhat.com>
5 years agocmake: enable Finddpdk to find DPDK to be built
Kefu Chai [Thu, 24 Oct 2019 09:35:26 +0000 (17:35 +0800)]
cmake: enable Finddpdk to find DPDK to be built

find_package(dpdk...) is used by seastar to find DPDK if Seastar_DPDK is
enabled. but `build_dpdk()` only exposes `dpdk::dpdk`, and it does not
set the variables like `dpdk_INCLUDE_DIRS` and `dpdk_LIBRARIES` when it
gets called. so we need to adapt Finddpdk to query them from `dpdk::dpdk`
target.

Signed-off-by: Kefu Chai <kchai@redhat.com>
5 years agoMerge pull request #31112 from smithfarm/wip-distros-sle
Nathan Cutler [Thu, 24 Oct 2019 09:28:41 +0000 (11:28 +0200)]
Merge pull request #31112 from smithfarm/wip-distros-sle

qa/distros: add SLE-12-SP3 and SLE-15-SP1

Reviewed-by: Kyr Shatskyy <kyrylo.shatskyy@suse.de>
5 years agoqa/distros: add SLE-12-SP3 and SLE-15-SP1 31112/head
Nathan Cutler [Thu, 24 Oct 2019 07:47:22 +0000 (09:47 +0200)]
qa/distros: add SLE-12-SP3 and SLE-15-SP1

Ceph luminous is known to run on SLE-12-SP3 and nautilus on SLE-15-SP1, so add
these two to qa/distros/all.

Signed-off-by: Nathan Cutler <ncutler@suse.com>
5 years agocmake: define dpdk_LIBRARIES properly
Kefu Chai [Thu, 24 Oct 2019 06:10:32 +0000 (14:10 +0800)]
cmake: define dpdk_LIBRARIES properly

dpdk_LIBRARIES should not be a list of dpdk library target, it should be
a list of paths to them. this also align with the definition of
`Finddpdk.cmake` in seastar project. so, if `Seastar_DPDK` is defined,
we should be able to offer the expected `dpdk_LIBRARIES` for seastar, as
our `Finddpdk.cmake` has higher priority than the one in seastar, and is
used when `find_package(dpdk...)` is called.

Signed-off-by: Kefu Chai <kchai@redhat.com>
5 years agocmake: refactor BuildDPDK
Kefu Chai [Thu, 24 Oct 2019 06:07:18 +0000 (14:07 +0800)]
cmake: refactor BuildDPDK

* extract `do_export_dpdk()` into a separated function
* no need to check for `dpdk-ext` target before calling `build_dpdk()`,
  as the name of this target should be hidden from the caller of this
  function.

Signed-off-by: Kefu Chai <kchai@redhat.com>
5 years agoMerge pull request #31069 from tchaikov/wip/crimson/replace-ceph-with-crimson
Kefu Chai [Thu, 24 Oct 2019 05:57:11 +0000 (13:57 +0800)]
Merge pull request #31069 from tchaikov/wip/crimson/replace-ceph-with-crimson

crimson: s/ceph/crimson/ in namespace names

Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Yingxin Cheng <yingxin.cheng@intel.com>
5 years agotest/crimson: silence deprecated warnings 31069/head
Kefu Chai [Wed, 23 Oct 2019 08:27:08 +0000 (16:27 +0800)]
test/crimson: silence deprecated warnings

* do not discard returned future<>
* do not use variadic future<>

Signed-off-by: Kefu Chai <kchai@redhat.com>
5 years agocrimson: s/ceph/crimson/ in namespace names
Kefu Chai [Wed, 23 Oct 2019 08:08:38 +0000 (16:08 +0800)]
crimson: s/ceph/crimson/ in namespace names

to help differentiate the symbols shared by classic and crimson osd,
after this change, all crimson code will live in "crimson::" namespaces.
and in a follow-up change, all classic code used by crimson will live in
"ceph::" namespaces.

Signed-off-by: Kefu Chai <kchai@redhat.com>
5 years agocrimson/osd/osd_operation.h: s/ceph_assert/assert/
Kefu Chai [Thu, 24 Oct 2019 03:57:09 +0000 (11:57 +0800)]
crimson/osd/osd_operation.h: s/ceph_assert/assert/

so we don't need to include `include/ceph_assert.h` for it. and
presumably, `ceph_assert()` is for fail the application early when it is
not able to recover from a failure, while `assert()` is for debugging.

Signed-off-by: Kefu Chai <kchai@redhat.com>