]> git.apps.os.sepia.ceph.com Git - ceph.git/log
ceph.git
5 years agoscript/run_mypy: Support mypy 0.740 31192/head
Sebastian Wagner [Mon, 28 Oct 2019 12:13:33 +0000 (13:13 +0100)]
script/run_mypy: Support mypy 0.740

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
Co-Authored-By: Ernesto Puerta <37327689+epuertat@users.noreply.github.com>
5 years agoMerge pull request #31156 from cyx1231st/wip-seastar-msgr-testpeer
Kefu Chai [Mon, 28 Oct 2019 04:27:11 +0000 (12:27 +0800)]
Merge pull request #31156 from cyx1231st/wip-seastar-msgr-testpeer

test/crimson: implement a remote async TestPeer for crimson msgr tests

Reviewed-by: Kefu Chai <kchai@redhat.com>
5 years agotest/crimson: add nonstop option in async TestPeer 31156/head
Yingxin Cheng [Fri, 25 Oct 2019 02:51:08 +0000 (10:51 +0800)]
test/crimson: add nonstop option in async TestPeer

If set, TestPeer won't shutdown if all tests are done, and will wait for
another round of tests.

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
5 years agotest/crimson: add delay to markdown_peer()
Yingxin Cheng [Fri, 25 Oct 2019 02:15:04 +0000 (10:15 +0800)]
test/crimson: add delay to markdown_peer()

Let test sleep awhile for peer markdown propagated.

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
5 years agotest/crimson: async msgr TestPeer for v2 failover tests
Yingxin Cheng [Wed, 23 Oct 2019 04:42:11 +0000 (12:42 +0800)]
test/crimson: async msgr TestPeer for v2 failover tests

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
5 years agotest/crimson: allow v2 failover tests to connect to remote TestPeer
Yingxin Cheng [Wed, 23 Oct 2019 02:48:54 +0000 (10:48 +0800)]
test/crimson: allow v2 failover tests to connect to remote TestPeer

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
5 years agotest/crimson: extract msgr test commands to header
Yingxin Cheng [Tue, 22 Oct 2019 02:24:08 +0000 (10:24 +0800)]
test/crimson: extract msgr test commands to header

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
5 years agoMerge pull request #30610 from ivancich/wip-no-schedule-reshard-during-reshard
J. Eric Ivancich [Sun, 27 Oct 2019 18:47:39 +0000 (14:47 -0400)]
Merge pull request #30610 from ivancich/wip-no-schedule-reshard-during-reshard

rgw: prevent bucket reshard scheduling if bucket is resharding

Reviewed-by: Matt Benjamin <mbenjamin@redhat.com>
Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: Abhishek Lekshmanan <abhishek@suse.com>
5 years agoMerge pull request #30795 from ivancich/wip-dynamic-reshard-adjustments
J. Eric Ivancich [Sun, 27 Oct 2019 18:46:01 +0000 (14:46 -0400)]
Merge pull request #30795 from ivancich/wip-dynamic-reshard-adjustments

rgw: adjust allowable bucket index shard counts for dynamic resharding

Reviewed-by: Casey Bodley <cbodley@redhat.com>
5 years agoMerge pull request #30845 from croit/rgw-bucket-stats-add-num-shards
J. Eric Ivancich [Sun, 27 Oct 2019 18:44:47 +0000 (14:44 -0400)]
Merge pull request #30845 from croit/rgw-bucket-stats-add-num-shards

rgw: add num_shards to radosgw-admin bucket stats

Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: Abhishek Lekshmanan <abhishek@suse.com>
Reviewed-by: J. Eric Ivancich <ivancich@redhat.com>
5 years agoMerge pull request #30956 from joke-lee/rgw-storage-class-post-obj
J. Eric Ivancich [Sun, 27 Oct 2019 18:42:27 +0000 (14:42 -0400)]
Merge pull request #30956 from joke-lee/rgw-storage-class-post-obj

rgw: set correct storage class for post object upload

Reviewed-by: Abhishek Lekshmanan <abhishek@suse.com>
5 years agoMerge pull request #30970 from theanalyst/rgw/list-common-prefix
J. Eric Ivancich [Sun, 27 Oct 2019 18:39:46 +0000 (14:39 -0400)]
Merge pull request #30970 from theanalyst/rgw/list-common-prefix

rgw: url encode common prefixes for List Objects response

Reviewed-by: Casey Bodley <cbodley@redhat.com>
5 years agoMerge pull request #31179 from liewegas/cleanup-exec-bit
Kefu Chai [Sun, 27 Oct 2019 15:58:59 +0000 (23:58 +0800)]
Merge pull request #31179 from liewegas/cleanup-exec-bit

os/bluestore,comon,erasure-code: chmod -x source files

Reviewed-by: Kefu Chai <kchai@redhat.com>
5 years agoMerge pull request #31171 from liewegas/bug-42496
Kefu Chai [Sun, 27 Oct 2019 15:57:57 +0000 (23:57 +0800)]
Merge pull request #31171 from liewegas/bug-42496

qa/tasks/cbt: run stop-all.sh while shutting down

Reviewed-by: Kefu Chai <kchai@redhat.com>
5 years agoos/bluestore,comon,erasure-code: chmod -x source files 31179/head
Sage Weil [Sun, 27 Oct 2019 14:25:53 +0000 (09:25 -0500)]
os/bluestore,comon,erasure-code: chmod -x source files

Fixes

*** WARNING: ./usr/src/debug/ceph-15.0.0-6512.g62bd825.el8.x86_64/src/common/obj_bencher.cc is executable but has empty or no shebang, removing executable bit
*** WARNING: ./usr/src/debug/ceph-15.0.0-6512.g62bd825.el8.x86_64/src/erasure-code/shec/determinant.c is executable but has empty or no shebang, removing executable bit
*** WARNING: ./usr/src/debug/ceph-15.0.0-6512.g62bd825.el8.x86_64/src/os/bluestore/AvlAllocator.cc is executable but has empty or no shebang, removing executable bit
*** WARNING: ./usr/src/debug/ceph-15.0.0-6512.g62bd825.el8.x86_64/src/os/bluestore/AvlAllocator.h is executable but has empty or no shebang, removing executable bit

in rpm build.

Signed-off-by: Sage Weil <sage@redhat.com>
5 years agoMerge PR #31170 into master
Sage Weil [Sun, 27 Oct 2019 14:21:56 +0000 (09:21 -0500)]
Merge PR #31170 into master

* refs/pull/31170/head:
src/telemetry: remove, now lives in ceph-telemetry.git

Reviewed-by: Sage Weil <sage@redhat.com>
5 years agoMerge PR #31131 into master
Sage Weil [Sun, 27 Oct 2019 14:20:51 +0000 (09:20 -0500)]
Merge PR #31131 into master

* refs/pull/31131/head:
crush: remove invalid upmap items

Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>
5 years agoMerge PR #31154 into master
Sage Weil [Sun, 27 Oct 2019 14:19:56 +0000 (09:19 -0500)]
Merge PR #31154 into master

* refs/pull/31154/head:
auth/cephx: always initialize local variables

Reviewed-by: Neha Ojha <nojha@redhat.com>
5 years agoMerge PR #31167 into master
Sage Weil [Sun, 27 Oct 2019 14:19:40 +0000 (09:19 -0500)]
Merge PR #31167 into master

* refs/pull/31167/head:
os/bluestore: do not mark per_pool_omap updated unless we fixed it

Reviewed-by: Igor Fedotov <ifedotov@suse.com>
5 years agoMerge pull request #31010 from pdvian/wip-fix-target-max
Kefu Chai [Sun, 27 Oct 2019 01:38:23 +0000 (09:38 +0800)]
Merge pull request #31010 from pdvian/wip-fix-target-max

mon/OSDMonitor.cc: Allow pool set target_max_(objects/bytes) with SI/IEC units

Reviewed-by: Kefu Chai <kchai@redhat.com>
5 years agoMerge pull request #30434 from smithfarm/wip-41820
Kefu Chai [Sun, 27 Oct 2019 01:18:16 +0000 (09:18 +0800)]
Merge pull request #30434 from smithfarm/wip-41820

qa: enable dashboard tests to be run with "--suite rados/dashboard"

Reviewed-by: Laura Paduano <lpaduano@suse.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
5 years agoMerge pull request #31005 from tchaikov/wip/qa/tasks/ceph/cleanup
Kefu Chai [Sun, 27 Oct 2019 01:16:37 +0000 (09:16 +0800)]
Merge pull request #31005 from tchaikov/wip/qa/tasks/ceph/cleanup

qa/tasks/ceph.py: remove unused variables

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>
5 years agoqa/tasks/cbt: run stop-all.sh when finishing up 31171/head
Sage Weil [Sat, 26 Oct 2019 00:20:05 +0000 (19:20 -0500)]
qa/tasks/cbt: run stop-all.sh when finishing up

stop-all.sh will work if the right deps are there (currently we lack 'nc')

also killall -9 java to be sure.

Fixes: https://tracker.ceph.com/issues/42496
Signed-off-by: Sage Weil <sage@redhat.com>
5 years agoMerge PR #31086 into master
Sage Weil [Sat, 26 Oct 2019 16:42:31 +0000 (11:42 -0500)]
Merge PR #31086 into master

* refs/pull/31086/head:
doc/bootstrap.rst: fix github's url

Reviewed-by: Sage Weil <sage@redhat.com>
5 years agoMerge PR #31169 into master
Sage Weil [Sat, 26 Oct 2019 15:55:19 +0000 (10:55 -0500)]
Merge PR #31169 into master

* refs/pull/31169/head:
mgr/ssh: implement (synchronous) describe_service
ceph-daemon: ls: replace 'active' bool with 'state' enum
ceph-daemon: include container_id and version in 'ls' output
ceph-daemon: tolerate systemctl is-* exit codes
mgr/ssh: modernize timeout config option

Reviewed-by: Sebastian Wagner <swagner@suse.com>
5 years agodoc/bootstrap.rst: fix github's url 31086/head
Alexandre Bruyelles [Wed, 23 Oct 2019 13:31:38 +0000 (15:31 +0200)]
doc/bootstrap.rst: fix github's url

Signed-off-by: Alexandre Bruyelles <jack@jack.fr.eu.org>
5 years agoMerge pull request #31126 from wjwithagen/wjw-fix-std-bitset
Kefu Chai [Sat, 26 Oct 2019 01:30:54 +0000 (09:30 +0800)]
Merge pull request #31126 from wjwithagen/wjw-fix-std-bitset

rgw: Select the std::bitset to resolv ambiguity

Reviewed-by: Adam C. Emerson <aemerson@redhat.com>
Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
5 years agosrc/telemetry: remove, now lives in ceph-telemetry.git 31170/head
Dan Mick [Fri, 25 Oct 2019 23:44:05 +0000 (16:44 -0700)]
src/telemetry: remove, now lives in ceph-telemetry.git

Signed-off-by: Dan Mick <dan.mick@redhat.com>
5 years agomgr/ssh: implement (synchronous) describe_service 31169/head
Sage Weil [Fri, 25 Oct 2019 17:50:50 +0000 (12:50 -0500)]
mgr/ssh: implement (synchronous) describe_service

Signed-off-by: Sage Weil <sage@redhat.com>
5 years agoceph-daemon: ls: replace 'active' bool with 'state' enum
Sage Weil [Fri, 25 Oct 2019 23:43:18 +0000 (18:43 -0500)]
ceph-daemon: ls: replace 'active' bool with 'state' enum

('running', 'inactive', 'error', 'unknown')

Signed-off-by: Sage Weil <sage@redhat.com>
5 years agoceph-daemon: include container_id and version in 'ls' output
Sage Weil [Fri, 25 Oct 2019 22:50:26 +0000 (17:50 -0500)]
ceph-daemon: include container_id and version in 'ls' output

Signed-off-by: Sage Weil <sage@redhat.com>
5 years agoceph-daemon: tolerate systemctl is-* exit codes
Sage Weil [Fri, 25 Oct 2019 21:46:09 +0000 (16:46 -0500)]
ceph-daemon: tolerate systemctl is-* exit codes

Signed-off-by: Sage Weil <sage@redhat.com>
5 years agomgr/ssh: modernize timeout config option
Sage Weil [Thu, 24 Oct 2019 13:21:39 +0000 (08:21 -0500)]
mgr/ssh: modernize timeout config option

(and rename it)

Signed-off-by: Sage Weil <sage@redhat.com>
5 years agoMerge PR #31141 into master
Sage Weil [Fri, 25 Oct 2019 19:30:33 +0000 (14:30 -0500)]
Merge PR #31141 into master

* refs/pull/31141/head:
ceph-daemon: fix warning message
ceph-daemon: fix enable with conflicting ceph.target files
test_ceph_daemon.sh: use same fsid as qa/standalone/test_ceph_daemon.sh
test_ceph_daemon.sh: use other latest-master image
ceph-daemon: fix pathify (fix shell --config/--keyring)
ceph-daemon: /var/run/ceph -> /var/run/ceph/$fsid

Reviewed-by: Sebastian Wagner <swagner@suse.com>
5 years agoceph-daemon: fix warning message 31141/head
Sage Weil [Fri, 25 Oct 2019 17:46:52 +0000 (12:46 -0500)]
ceph-daemon: fix warning message

Signed-off-by: Sage Weil <sage@redhat.com>
5 years agoMerge pull request #31164 from tchaikov/wip/doc/rados/fix-typo
Josh Durgin [Fri, 25 Oct 2019 17:41:42 +0000 (10:41 -0700)]
Merge pull request #31164 from tchaikov/wip/doc/rados/fix-typo

doc/rados/deployment/ceph-deploy-mon: fix typo

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
5 years agoceph-daemon: fix enable with conflicting ceph.target files
Sage Weil [Fri, 25 Oct 2019 16:26:32 +0000 (11:26 -0500)]
ceph-daemon: fix enable with conflicting ceph.target files

+ make description better

Signed-off-by: Sage Weil <sage@redhat.com>
5 years agoos/bluestore: do not mark per_pool_omap updated unless we fixed it 31167/head
Sage Weil [Fri, 25 Oct 2019 16:02:28 +0000 (11:02 -0500)]
os/bluestore: do not mark per_pool_omap updated unless we fixed it

We only fix the per-pool-omap issues if we do a non-shallow fsck.

Fixes: https://tracker.ceph.com/issues/42490
Signed-off-by: Sage Weil <sage@redhat.com>
5 years agodoc/rados/deployment/ceph-deploy-mon: fix typo 31164/head
Kefu Chai [Fri, 25 Oct 2019 14:38:11 +0000 (22:38 +0800)]
doc/rados/deployment/ceph-deploy-mon: fix typo

s/comingling/commingling/

Signed-off-by: Kefu Chai <kchai@redhat.com>
5 years agoMerge pull request #31113 from sungjunyoung/do_cmake-warning
Kefu Chai [Fri, 25 Oct 2019 14:24:42 +0000 (22:24 +0800)]
Merge pull request #31113 from sungjunyoung/do_cmake-warning

do_cmake: Warn user about slow debug performance only for not set

Reviewed-by: Kefu Chai <kchai@redhat.com>
5 years agoMerge PR #31130 into master
Sage Weil [Fri, 25 Oct 2019 14:18:20 +0000 (09:18 -0500)]
Merge PR #31130 into master

* refs/pull/31130/head:
ceph-daemon: only set up crash dir mount if it exists

Reviewed-by: Sebastian Wagner <swagner@suse.com>
5 years agoMerge PR #31138 into master
Sage Weil [Fri, 25 Oct 2019 14:11:36 +0000 (09:11 -0500)]
Merge PR #31138 into master

* refs/pull/31138/head:
mon: fix tell to hybrid octopus/pre-octopus mons

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>
5 years agoMerge pull request #31006 from zy751713126/fix_features
Jason Dillaman [Fri, 25 Oct 2019 12:34:25 +0000 (08:34 -0400)]
Merge pull request #31006 from zy751713126/fix_features

librbd: fix rbd_features_to_string output

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
5 years agoMerge pull request #30955 from runsisi/wip-fix-dup-lock
Jason Dillaman [Fri, 25 Oct 2019 12:33:51 +0000 (08:33 -0400)]
Merge pull request #30955 from runsisi/wip-fix-dup-lock

librbd: force reacquire lock if blacklist is disabled

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
5 years agoMerge pull request #30954 from mxdInspur/thickimg_create_progress
Jason Dillaman [Fri, 25 Oct 2019 12:33:12 +0000 (08:33 -0400)]
Merge pull request #30954 from mxdInspur/thickimg_create_progress

rbd:  creating thick-provision image progress percent info exceeds 100%

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
5 years agoMerge pull request #31143 from badone/wip-tracker-38466-enable-librabbitmq-devel
Kefu Chai [Fri, 25 Oct 2019 11:15:43 +0000 (19:15 +0800)]
Merge pull request #31143 from badone/wip-tracker-38466-enable-librabbitmq-devel

ceph.spec.in: enable amqp_endpoint on RHEL8 by default

Reviewed-by: Kefu Chai <kchai@redhat.com>
5 years agoMerge pull request #31110 from tchaikov/wip-cmake/enable-seastar-with-dpdk
Kefu Chai [Fri, 25 Oct 2019 10:00:03 +0000 (18:00 +0800)]
Merge pull request #31110 from tchaikov/wip-cmake/enable-seastar-with-dpdk

cmake: support `Seastar_DPDK=ON` option

Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: Yingxin Cheng <yingxin.cheng@intel.com>
5 years agoauth/cephx: always initialize local variables 31154/head
Kefu Chai [Fri, 25 Oct 2019 08:42:55 +0000 (16:42 +0800)]
auth/cephx: always initialize local variables

to silence GCC warnings like:

rc/auth/cephx/CephxProtocol.h:309:5: warning: 'type' may be used uninitialized in this function [-Wmaybe-uninitialized]
     if (i != tickets_map.end())
     ^~

Signed-off-by: Kefu Chai <kchai@redhat.com>
5 years agoceph.spec.in: Enable amqp_endpoint on RHEL8 by default 31143/head
Brad Hubbard [Fri, 25 Oct 2019 05:29:37 +0000 (15:29 +1000)]
ceph.spec.in: Enable amqp_endpoint on RHEL8 by default

RHEL/CentOS 8 now provide librabbitmq-devel so we can enable it as a
build requirement.

Fixes: https://tracker.ceph.com/issues/38466
Signed-off-by: Brad Hubbard <bhubbard@redhat.com>
5 years agotest_ceph_daemon.sh: use same fsid as qa/standalone/test_ceph_daemon.sh
Sage Weil [Fri, 25 Oct 2019 02:27:27 +0000 (21:27 -0500)]
test_ceph_daemon.sh: use same fsid as qa/standalone/test_ceph_daemon.sh

That way these scripts both clean up for each other (and are obviously
bogus clusters).

Signed-off-by: Sage Weil <sage@redhat.com>
5 years agotest_ceph_daemon.sh: use other latest-master image
Sage Weil [Fri, 25 Oct 2019 02:26:21 +0000 (21:26 -0500)]
test_ceph_daemon.sh: use other latest-master image

This one is newer?

Signed-off-by: Sage Weil <sage@redhat.com>
5 years agoceph-daemon: fix pathify (fix shell --config/--keyring)
Sage Weil [Fri, 25 Oct 2019 02:25:59 +0000 (21:25 -0500)]
ceph-daemon: fix pathify (fix shell --config/--keyring)

The relative path works on one of my machines but not my laptop; the
full CWD works everywhere.  This is presumably better anyway!

Signed-off-by: Sage Weil <sage@redhat.com>
5 years agoceph-daemon: /var/run/ceph -> /var/run/ceph/$fsid
Sage Weil [Fri, 25 Oct 2019 02:03:34 +0000 (21:03 -0500)]
ceph-daemon: /var/run/ceph -> /var/run/ceph/$fsid

This is better than having a single /var/run/ceph on the host with a
weird naming scheme.  Among other things, it means that we can access
the asok for any daemon for a given fsid from any container on the same
host with the same fsid (notably, a shell).

Signed-off-by: Sage Weil <sage@redhat.com>
5 years agomon: fix tell to hybrid octopus/pre-octopus mons 31138/head
Sage Weil [Thu, 24 Oct 2019 23:56:04 +0000 (18:56 -0500)]
mon: fix tell to hybrid octopus/pre-octopus mons

We can't decide whether to use the new tell command style based on the
monmap.min_mon_release alone because some mons may be octopus even though
that hasn't updated yet.  The same goes for if we look at the combined
features for the cluster--the underlying problem is the monmap doesn't
tell us which mons are octopus and which ones aren't, so we don't know
how to behave.

Instead, allow octopus+ mons to advertise the converted tell commands
going forward, for compatibility with pre-octopus clients (who do the old
style of tell) and for octopus+ clients talking to a min_mon_release <
octopus cluster.

Signed-off-by: Sage Weil <sage@redhat.com>
5 years agoceph-daemon: only set up crash dir mount if it exists 31130/head
Sage Weil [Thu, 24 Oct 2019 13:41:33 +0000 (08:41 -0500)]
ceph-daemon: only set up crash dir mount if it exists

Sometimes we run containers on a host that doesn't have a crash dir set
up (becuase no daemon has been deployed).  Examples include shell and
ceph-volume.

Signed-off-by: Sage Weil <sage@redhat.com>
5 years agoMerge pull request #31133 from dzafman/wip-42476
David Zafman [Fri, 25 Oct 2019 00:23:48 +0000 (17:23 -0700)]
Merge pull request #31133 from dzafman/wip-42476

ceph-objectstore-tool: call collection_bits() crashes on the meta col…

Reviewed-by: Sage Weil <sage@redhat.com>
5 years agoMerge pull request #31135 from athanatos/sjust/wip-alien-echo-fix
Kefu Chai [Thu, 24 Oct 2019 23:50:32 +0000 (07:50 +0800)]
Merge pull request #31135 from athanatos/sjust/wip-alien-echo-fix

test_alien_echo: update to use crimson:: namespace

Reviewed-by: Kefu Chai <kchai@redhat.com>
5 years agoMerge PR #31121 into master
Sage Weil [Thu, 24 Oct 2019 23:29:52 +0000 (18:29 -0500)]
Merge PR #31121 into master

* refs/pull/31121/head:
mon/MonClient: fix mon tell to older mons

Reviewed-by: Neha Ojha <nojha@redhat.com>
5 years agoMerge PR #31095 into master
Patrick Donnelly [Thu, 24 Oct 2019 22:17:48 +0000 (15:17 -0700)]
Merge PR #31095 into master

* refs/pull/31095/head:
qa: do not check pg count for new data_isolated volume

Reviewed-by: Sage Weil <sage@redhat.com>
5 years agotest/crimson/test_alien_echo: update to use crimson:: namespace 31135/head
Samuel Just [Thu, 24 Oct 2019 20:18:29 +0000 (13:18 -0700)]
test/crimson/test_alien_echo: update to use crimson:: namespace

See: bd023211d999f1e2b70f48254507524d0763bd33
Signed-off-by: Samuel Just <sjust@redhat.com>
5 years agoMerge pull request #30964 from markhpc/wip-bs-cache-trim-pinned
Mark Nelson [Thu, 24 Oct 2019 19:50:11 +0000 (14:50 -0500)]
Merge pull request #30964 from markhpc/wip-bs-cache-trim-pinned

os/bluestore: Keep separate onode cache pinned list.

5 years agoceph-objectstore-tool: call collection_bits() crashes on the meta collection 31133/head
David Zafman [Thu, 24 Oct 2019 18:31:52 +0000 (11:31 -0700)]
ceph-objectstore-tool: call collection_bits() crashes on the meta collection

Skip new check for meta collection
test:
    Turn off osd_pool_default_pg_autoscale_mode just like bash tests do
    Fix test by checking for new error message

Caused by: f88b3534549bb2ede76e9f3e8b6b641b5886a9f4

Fixes: https://tracker.ceph.com/issues/42476
Signed-off-by: David Zafman <dzafman@redhat.com>
5 years agoMerge pull request #31132 from smithfarm/wip-42474
Yuri Weinstein [Thu, 24 Oct 2019 17:43:47 +0000 (10:43 -0700)]
Merge pull request #31132 from smithfarm/wip-42474

build-integration-branch: take PRs in chronological order

5 years agodo_cmake: Disable shell command printing feature before cmake execution 31113/head
Junyoung, Sung [Thu, 24 Oct 2019 17:17:59 +0000 (02:17 +0900)]
do_cmake: Disable shell command printing feature before cmake execution

Signed-off-by: Junyoung, Sung <junyoung.sung@navercorp.com>
5 years agobuild-integration-branch: change shebang to python3 31132/head
Nathan Cutler [Thu, 24 Oct 2019 16:54:55 +0000 (18:54 +0200)]
build-integration-branch: change shebang to python3

The word "python" in the shebang means Python 2, but Python 2 is deprecated and
should no longer be used.

Signed-off-by: Nathan Cutler <ncutler@suse.com>
5 years agobuild-integration-branch: take PRs in chronological order
Nathan Cutler [Thu, 24 Oct 2019 16:29:02 +0000 (18:29 +0200)]
build-integration-branch: take PRs in chronological order

Fixes: https://tracker.ceph.com/issues/42474
Signed-off-by: Nathan Cutler <ncutler@suse.com>
5 years agocrush: remove invalid upmap items 31131/head
huangjun [Thu, 24 Oct 2019 15:44:12 +0000 (23:44 +0800)]
crush: remove invalid upmap items

We can not cancel in verify_upmap if remap an osd to different root bucket,

cluster topology:
osd.0 ~ osd.29 belongs to datacenter 1
osd30 ~ osd.59 belongs to datacenter 2
crush rule:
  take datacenter 1
  chooseleaf 2 host
  emit
  take datacenter 2
  chooseleaf 2 host
  emit
The pg's primary osd in datacenter 1.

We should cancel the pgid from upmap_items like below
1) from [26,12,54,46] to [30,12,54,46]
pg_upmap_items [26,30]

2) from [16,25,53,31] to [40,25,53,0]
pg_upmap_items [16,20,31,0]

Signed-off-by: huangjun <huangjun@xsky.com>
5 years agodo_cmake: Warn user about slow debug performance only for not set
Junyoung, Sung [Thu, 24 Oct 2019 07:57:57 +0000 (16:57 +0900)]
do_cmake: Warn user about slow debug performance only for not set

Signed-off-by: Junyoung, Sung <junyoung.sung@navercorp.com>
5 years agorgw: Select the std::bitset to resolv ambiguity 31126/head
Willem Jan Withagen [Thu, 24 Oct 2019 15:09:56 +0000 (17:09 +0200)]
rgw: Select the std::bitset to resolv ambiguity

Signed-off-by: Willem Jan Withagen <wjw@digiware.nl>
5 years agomon/MonClient: fix mon tell to older mons 31121/head
Sage Weil [Thu, 24 Oct 2019 13:00:32 +0000 (08:00 -0500)]
mon/MonClient: fix mon tell to older mons

For older mons, we need to call into send_command so that we trigger the
compat code that futzes with our mon sessions.

Signed-off-by: Sage Weil <sage@redhat.com>
5 years agoMerge PR #30848 into master
Sage Weil [Thu, 24 Oct 2019 11:20:29 +0000 (06:20 -0500)]
Merge PR #30848 into master

* refs/pull/30848/head:
mon/PGMap: drop indentation on 'df' human output

Reviewed-by: Kefu Chai <kchai@redhat.com>
5 years agocmake: support `Seastar_DPDK=ON` option 31110/head
Kefu Chai [Thu, 24 Oct 2019 06:14:40 +0000 (14:14 +0800)]
cmake: support `Seastar_DPDK=ON` option

seastar does not build any 3rd party libraries. but we do. currently, we
build DPDK/SPDK when they are enabled for the messenger backend and
bluestore backend.

in this change, the same function is reused for preparing the DPDK
libraries if `Seastar_DPDK` is enabled. probably we should outsource
this work to install-deps.sh in future, but before that, let's keep
this as a builtin "feature".

Signed-off-by: Kefu Chai <kchai@redhat.com>
5 years agocmake: enable Finddpdk to find DPDK to be built
Kefu Chai [Thu, 24 Oct 2019 09:35:26 +0000 (17:35 +0800)]
cmake: enable Finddpdk to find DPDK to be built

find_package(dpdk...) is used by seastar to find DPDK if Seastar_DPDK is
enabled. but `build_dpdk()` only exposes `dpdk::dpdk`, and it does not
set the variables like `dpdk_INCLUDE_DIRS` and `dpdk_LIBRARIES` when it
gets called. so we need to adapt Finddpdk to query them from `dpdk::dpdk`
target.

Signed-off-by: Kefu Chai <kchai@redhat.com>
5 years agoMerge pull request #31112 from smithfarm/wip-distros-sle
Nathan Cutler [Thu, 24 Oct 2019 09:28:41 +0000 (11:28 +0200)]
Merge pull request #31112 from smithfarm/wip-distros-sle

qa/distros: add SLE-12-SP3 and SLE-15-SP1

Reviewed-by: Kyr Shatskyy <kyrylo.shatskyy@suse.de>
5 years agoqa/distros: add SLE-12-SP3 and SLE-15-SP1 31112/head
Nathan Cutler [Thu, 24 Oct 2019 07:47:22 +0000 (09:47 +0200)]
qa/distros: add SLE-12-SP3 and SLE-15-SP1

Ceph luminous is known to run on SLE-12-SP3 and nautilus on SLE-15-SP1, so add
these two to qa/distros/all.

Signed-off-by: Nathan Cutler <ncutler@suse.com>
5 years agocmake: define dpdk_LIBRARIES properly
Kefu Chai [Thu, 24 Oct 2019 06:10:32 +0000 (14:10 +0800)]
cmake: define dpdk_LIBRARIES properly

dpdk_LIBRARIES should not be a list of dpdk library target, it should be
a list of paths to them. this also align with the definition of
`Finddpdk.cmake` in seastar project. so, if `Seastar_DPDK` is defined,
we should be able to offer the expected `dpdk_LIBRARIES` for seastar, as
our `Finddpdk.cmake` has higher priority than the one in seastar, and is
used when `find_package(dpdk...)` is called.

Signed-off-by: Kefu Chai <kchai@redhat.com>
5 years agocmake: refactor BuildDPDK
Kefu Chai [Thu, 24 Oct 2019 06:07:18 +0000 (14:07 +0800)]
cmake: refactor BuildDPDK

* extract `do_export_dpdk()` into a separated function
* no need to check for `dpdk-ext` target before calling `build_dpdk()`,
  as the name of this target should be hidden from the caller of this
  function.

Signed-off-by: Kefu Chai <kchai@redhat.com>
5 years agoMerge pull request #31069 from tchaikov/wip/crimson/replace-ceph-with-crimson
Kefu Chai [Thu, 24 Oct 2019 05:57:11 +0000 (13:57 +0800)]
Merge pull request #31069 from tchaikov/wip/crimson/replace-ceph-with-crimson

crimson: s/ceph/crimson/ in namespace names

Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Yingxin Cheng <yingxin.cheng@intel.com>
5 years agotest/crimson: silence deprecated warnings 31069/head
Kefu Chai [Wed, 23 Oct 2019 08:27:08 +0000 (16:27 +0800)]
test/crimson: silence deprecated warnings

* do not discard returned future<>
* do not use variadic future<>

Signed-off-by: Kefu Chai <kchai@redhat.com>
5 years agocrimson: s/ceph/crimson/ in namespace names
Kefu Chai [Wed, 23 Oct 2019 08:08:38 +0000 (16:08 +0800)]
crimson: s/ceph/crimson/ in namespace names

to help differentiate the symbols shared by classic and crimson osd,
after this change, all crimson code will live in "crimson::" namespaces.
and in a follow-up change, all classic code used by crimson will live in
"ceph::" namespaces.

Signed-off-by: Kefu Chai <kchai@redhat.com>
5 years agocrimson/osd/osd_operation.h: s/ceph_assert/assert/
Kefu Chai [Thu, 24 Oct 2019 03:57:09 +0000 (11:57 +0800)]
crimson/osd/osd_operation.h: s/ceph_assert/assert/

so we don't need to include `include/ceph_assert.h` for it. and
presumably, `ceph_assert()` is for fail the application early when it is
not able to recover from a failure, while `assert()` is for debugging.

Signed-off-by: Kefu Chai <kchai@redhat.com>
5 years agoMerge pull request #31066 from ideepika/wip-doc-adds-cmake-options
Neha Ojha [Thu, 24 Oct 2019 03:59:36 +0000 (09:29 +0530)]
Merge pull request #31066 from ideepika/wip-doc-adds-cmake-options

doc: Adds cmake build options for optionally skipping few components

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>
5 years agoMerge PR #30859 into master
Sage Weil [Thu, 24 Oct 2019 02:05:42 +0000 (21:05 -0500)]
Merge PR #30859 into master

* refs/pull/30859/head:
auth: EACCES, not EPERM
mon: shunt old tell commands from cli interface to asok
mon: allow mgr to tell mon.foo smart
mon: include quorum features in quorum_status
qa/workunits/mon/caps.sh: fix test
ceph_test_rados_api_cmd: fix MonDescribe test
Merge branch 'vstart-fs-auth' of git://github.com/batrick/ceph into wip-cleanup-mon-asok
test/pybind/test_ceph_argparse: fix tests
vstart: add volume client keys to keyring
vstart: use fs authorize to create master client key
vstart: redirect some output to stderr
vstart: output command strings to stderr
qa/workunits/cephtool/test.sh: fix 'quorum enter' caller
qa: change mon_status calls to quorum_status or tell commands
mon: fix 'heap ...' command
mon: consolidate 'sync force' commands
mon: allow asok commands to return an error code
mon: move 'quorum enter|exit' and 'mon_status' to asok
mon: fix 'smart' asok command
mon: remove old 'config set' and 'injectargs'

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
5 years agoMerge PR #31094 into master
Sage Weil [Thu, 24 Oct 2019 00:46:06 +0000 (19:46 -0500)]
Merge PR #31094 into master

* refs/pull/31094/head:
ceph-daemon: remove redundant --privileged
test_ceph_daemon: test unit, enter, shell
ceph-daemon: drop exec
ceph-daemon: fix exit code for run, shell, enter, exec
ceph-daemon: allow optional command for 'enter'
ceph-daemon: fix LANG for 'enter' command
ceph-daemon: allow shell to take optional command
qa/suites/rados/singleton-nomsgr/ceph-daemon: run test_ceph_daemon.sh
qa/standalone/test_ceph_daemon.sh: add new functional tests
test_ceph_daemon.sh: use newer image
ceph-daemon: unconditionally enable and start crash unit
ceph-daemon: fix crash unit cleanup
ceph-daemon: include 'crash' unit/item in 'ls' output
ceph-daemon: fix 'ls'
mgr/orchestrator: s/sdd/ssd/
mgr/ssh: remove stdout/stderr kludges
ceph-daemon: fix ceph-volume command to write stdout to stdout

Reviewed-by: Sebastian Wagner <swagner@suse.com>
5 years agoMerge pull request #30650 from athanatos/sjust/wip-dmclock-server-only
Gregory Farnum [Wed, 23 Oct 2019 20:35:13 +0000 (13:35 -0700)]
Merge pull request #30650 from athanatos/sjust/wip-dmclock-server-only

dmclock server side refactor

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: J. Eric Ivancich <ivancich@redhat.com>
5 years agoPendingReleaseNotes: note mClockScheduler config changes 30650/head
Samuel Just [Tue, 22 Oct 2019 21:07:53 +0000 (14:07 -0700)]
PendingReleaseNotes: note mClockScheduler config changes

Signed-off-by: Samuel Just <sjust@redhat.com>
5 years agoosd/: remove legacy schedulers
Samuel Just [Thu, 26 Sep 2019 22:04:09 +0000 (15:04 -0700)]
osd/: remove legacy schedulers

The consensus seems to be that PrioritizedQueue is strictly worse than
WeightedPriorityQueue.

mClockClientQueue and mClockClassQueue are superceded by
mClockScheduler.

Signed-off-by: Samuel Just <sjust@redhat.com>
5 years agoosd/scheduler: introduce mClockScheduler
Samuel Just [Tue, 17 Sep 2019 19:10:52 +0000 (12:10 -0700)]
osd/scheduler: introduce mClockScheduler

mClockScheduler schedules items based on op_scheduler_class
with configured mclock parameters.  Items which should be
scheduled immediately (op_scheduler_class::immediate) are placed
into a single queue as it's not clear that there's a reason to
differentiate among them.

A subsequent patch will add support for client provided mclock
params and dmclock request state.

Signed-off-by: Samuel Just <sjust@redhat.com>
5 years agoMerge PR #31057 into master
Sage Weil [Wed, 23 Oct 2019 20:20:56 +0000 (15:20 -0500)]
Merge PR #31057 into master

* refs/pull/31057/head:
common/thread: Fix race condition in make_named_thread

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Ilya Dryomov <idryomov@redhat.com>
5 years agoMerge PR #31054 into master
Sage Weil [Wed, 23 Oct 2019 20:20:43 +0000 (15:20 -0500)]
Merge PR #31054 into master

* refs/pull/31054/head:
qa/suites/upgrade/*-x-singleton: suppress TOO_FEW_PGS warning

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
5 years agoMerge PR #30882 into master
Sage Weil [Wed, 23 Oct 2019 20:18:26 +0000 (15:18 -0500)]
Merge PR #30882 into master

* refs/pull/30882/head:
ceph-daemon: deploy a logrotate.d config file for each cluster

Reviewed-by: Sebastian Wagner <swagner@suse.com>
5 years agoceph-daemon: remove redundant --privileged 31094/head
Sage Weil [Wed, 23 Oct 2019 20:08:26 +0000 (15:08 -0500)]
ceph-daemon: remove redundant --privileged

The only caller, command_shell, passes this.

Signed-off-by: Sage Weil <sage@redhat.com>
5 years agotest_ceph_daemon: test unit, enter, shell
Sage Weil [Wed, 23 Oct 2019 16:34:46 +0000 (11:34 -0500)]
test_ceph_daemon: test unit, enter, shell

Signed-off-by: Sage Weil <sage@redhat.com>
5 years agoceph-daemon: drop exec
Sage Weil [Wed, 23 Oct 2019 16:25:12 +0000 (11:25 -0500)]
ceph-daemon: drop exec

It's not identical to enter.  enter seems more intuitive to me, but that
may be because I'm not a longtime docker user.

Signed-off-by: Sage Weil <sage@redhat.com>
5 years agoceph-daemon: fix exit code for run, shell, enter, exec
Sage Weil [Wed, 23 Oct 2019 16:23:32 +0000 (11:23 -0500)]
ceph-daemon: fix exit code for run, shell, enter, exec

Signed-off-by: Sage Weil <sage@redhat.com>
5 years agoceph-daemon: allow optional command for 'enter'
Sage Weil [Wed, 23 Oct 2019 16:14:49 +0000 (11:14 -0500)]
ceph-daemon: allow optional command for 'enter'

Default to interactive shell; otherwise run a specific command, just like
the 'shell' command.

Signed-off-by: Sage Weil <sage@redhat.com>
5 years agoceph-daemon: fix LANG for 'enter' command
Sage Weil [Wed, 23 Oct 2019 16:07:59 +0000 (11:07 -0500)]
ceph-daemon: fix LANG for 'enter' command

We did the same thing for 'shell' in 02d4f8c0970489c7184bbbe6b239af4a0067e38e

Signed-off-by: Sage Weil <sage@redhat.com>
5 years agoceph-daemon: allow shell to take optional command
Sage Weil [Wed, 23 Oct 2019 16:05:00 +0000 (11:05 -0500)]
ceph-daemon: allow shell to take optional command

This is different from 'exec' in that it creates a new container instead
of entering an existing one.

Signed-off-by: Sage Weil <sage@redhat.com>
5 years agoqa/suites/rados/singleton-nomsgr/ceph-daemon: run test_ceph_daemon.sh
Sage Weil [Wed, 23 Oct 2019 15:57:55 +0000 (10:57 -0500)]
qa/suites/rados/singleton-nomsgr/ceph-daemon: run test_ceph_daemon.sh

Signed-off-by: Sage Weil <sage@redhat.com>
5 years agoqa/standalone/test_ceph_daemon.sh: add new functional tests
Sage Weil [Wed, 23 Oct 2019 15:54:33 +0000 (10:54 -0500)]
qa/standalone/test_ceph_daemon.sh: add new functional tests

- sudo as needed
- clean up afterward

There is still a bit of missing coverage, but this captures most of it.

Signed-off-by: Sage Weil <sage@redhat.com>