]> git.apps.os.sepia.ceph.com Git - ceph.git/log
ceph.git
5 years agoMerge pull request #33178 from rhcs-dashboard/42475-rgw-api-keys-allowed
Lenz Grimmer [Mon, 2 Mar 2020 13:54:31 +0000 (13:54 +0000)]
Merge pull request #33178 from rhcs-dashboard/42475-rgw-api-keys-allowed

mgr/dashboard: do not show RGW API keys if only read-only privileges

Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Laura Paduano <lpaduano@suse.com>
Reviewed-by: Tatjana Dehler <tdehler@suse.com>
5 years agoMerge pull request #33638 from tchaikov/wip-btree
Kefu Chai [Mon, 2 Mar 2020 11:16:09 +0000 (19:16 +0800)]
Merge pull request #33638 from tchaikov/wip-btree

include/cpp-btree: use the same type when allocate/deallocate

Reviewed-by: Adam Kupczyk <akupczyk@redhat.com>
5 years agoMerge pull request #33645 from liewegas/fix-mon-down-cephtool-test
Kefu Chai [Mon, 2 Mar 2020 11:11:42 +0000 (19:11 +0800)]
Merge pull request #33645 from liewegas/fix-mon-down-cephtool-test

qa/suites/rados/singleton-bluestore/cephtool: whitelist MON_DOWN

Reviewed-by: Kefu Chai <kchai@redhat.com>
5 years agoMerge pull request #33536 from yuvalif/sync_object_write_with_return_value_op
Kefu Chai [Mon, 2 Mar 2020 11:10:29 +0000 (19:10 +0800)]
Merge pull request #33536 from yuvalif/sync_object_write_with_return_value_op

librados: allow passing flags to operate sync APIs

Reviewed-By: Casey Bodley <cbodley@redhat.com>
Reviewed-by: Sage Weil <sage@redhat.com>
5 years agoMerge pull request #33583 from rzarzynski/wip-recovery-cleanup1
Kefu Chai [Mon, 2 Mar 2020 11:01:29 +0000 (19:01 +0800)]
Merge pull request #33583 from rzarzynski/wip-recovery-cleanup1

osd: tiny clean-ups around the backfill

Reviewed-by: David Zafman <dzafman@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
5 years agoMerge pull request #33593 from dzafman/wip-cot-fix
Kefu Chai [Mon, 2 Mar 2020 10:58:19 +0000 (18:58 +0800)]
Merge pull request #33593 from dzafman/wip-cot-fix

test: Fix failing ceph_objectstore_tool.py test

Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
5 years agoMerge pull request #33373 from rzarzynski/wip-bl-bumpup_api_verion
Kefu Chai [Mon, 2 Mar 2020 10:25:21 +0000 (18:25 +0800)]
Merge pull request #33373 from rzarzynski/wip-bl-bumpup_api_verion

common, include: bump the version of ceph::buffer's C++ API.

Reviewed-by: Kefu Chai <kchai@redhat.com>
5 years agoMerge pull request #33572 from p-se/wip-pse-grafana-unit-issue
Lenz Grimmer [Mon, 2 Mar 2020 10:24:30 +0000 (10:24 +0000)]
Merge pull request #33572 from p-se/wip-pse-grafana-unit-issue

mgr/dashboard: fix stray requests/error in Grafana unit test

Reviewed-by: Kiefer Chang <kiefer.chang@suse.com>
Reviewed-by: Tatjana Dehler <tdehler@suse.com>
5 years agoMerge pull request #31041 from liu-chunmei/ceph_seastar_alien_blue_store
Kefu Chai [Mon, 2 Mar 2020 10:15:24 +0000 (18:15 +0800)]
Merge pull request #31041 from liu-chunmei/ceph_seastar_alien_blue_store

crimson:: add alien blue store

Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
5 years agoMerge pull request #33595 from cyx1231st/wip-crimson-osd-fixes
Kefu Chai [Mon, 2 Mar 2020 09:36:10 +0000 (17:36 +0800)]
Merge pull request #33595 from cyx1231st/wip-crimson-osd-fixes

crimson/osd: misc fixes for OSD reboot-ability

Reviewed-by: Kefu Chai <kchai@redhat.com>
5 years agoMerge pull request #33597 from optimistyzy/228_improve_sync_read
Kefu Chai [Mon, 2 Mar 2020 09:30:34 +0000 (17:30 +0800)]
Merge pull request #33597 from optimistyzy/228_improve_sync_read

NVMEDevice: Remove the unnecessary aio_wait in sync read

Reviewed-by: Kefu Chai <kchai@redhat.com>
5 years agoinclude/cpp-btree: use the same type when allocate/deallocate 33638/head
Kefu Chai [Sat, 29 Feb 2020 06:51:28 +0000 (14:51 +0800)]
include/cpp-btree: use the same type when allocate/deallocate

btree_set<> by default uses `std::allocator<Key>`, and btree_map by
default uses `std::allocator<std::pair<Key, Value>>`.

before this change, btree uses the allocator directly for allocating n
elements where element is `Key` or `std::pair<Key, Value>` respectively,
while "n" is actually supposed to be the number of bytes used by each
node which is being allocated.
but, what we need to allocate is actually a "node_type" for holding
multiple slots, and each slot holds an element. in addition to the
slots, a node also keeps track of metadata for btree itself. in short,
what we allocate now is (in bytes):

  alignof(sizeof(node_type)) * sizeof(element)

but what we should allocate is (in bytes):

  alignof(sizeof(node_type))

in this change:

* always rebind the allocator to the correct aligned type with given
  alignment
* extract the allocator related helpers into a template class

Signed-off-by: Kefu Chai <kchai@redhat.com>
5 years agoMerge pull request #33650 from joscollin/wip-unused-nvme
Kefu Chai [Mon, 2 Mar 2020 05:26:00 +0000 (13:26 +0800)]
Merge pull request #33650 from joscollin/wip-unused-nvme

test: silence warning unused variable nvme

Reviewed-by: Kefu Chai <kchai@redhat.com>
5 years agotest: silence warning unused variable nvme 33650/head
Jos Collin [Mon, 2 Mar 2020 03:34:39 +0000 (09:04 +0530)]
test: silence warning unused variable nvme

Signed-off-by: Jos Collin <jcollin@redhat.com>
5 years agoMerge PR #33512 into master
Sage Weil [Mon, 2 Mar 2020 01:53:35 +0000 (19:53 -0600)]
Merge PR #33512 into master

* refs/pull/33512/head:
os/bluestore/BlueFS: less verbose about alloc adjustments

Reviewed-by: Igor Fedotov <ifedotov@suse.com>
5 years agoMerge PR #33575 into master
Sage Weil [Mon, 2 Mar 2020 01:53:25 +0000 (19:53 -0600)]
Merge PR #33575 into master

* refs/pull/33575/head:
mgr/telemetry: fix and document proxy usage

Reviewed-by: Lenz Grimmer <lgrimmer@suse.com>
Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Yaarit Hatuka <yaarithatuka@gmail.com>
5 years agoMerge PR #33644 into master
Sage Weil [Sun, 1 Mar 2020 23:15:29 +0000 (17:15 -0600)]
Merge PR #33644 into master

* refs/pull/33644/head:
qa/suites/rados/verity/tasks/mon_recovery: whitelist SLOW_OPS

Reviewed-by: Kefu Chai <kchai@redhat.com>
5 years agoMerge PR #33541 into master
Sage Weil [Sun, 1 Mar 2020 22:55:06 +0000 (16:55 -0600)]
Merge PR #33541 into master

* refs/pull/33541/head:
mgr/test_orchestrator: add daemon_action
qa/tasksmgr/test_orchestrator_cli: fix test_service_action
qa/tasks/test_orchestrator_cli: disable test_osd_create exception check
mgr/test_orchestrator: update_foo -> apply_foo
mgr/test_orchestrator: add force flag to remove_daemons
qa/tasks/mgr/test_orchestrator_cli: update
qa/tasks/mgr/test_orchestrator_cli: fix test_load_data
mgr/test_orchestrator: fix list_daemons

Reviewed-by: Joshua Schmid <jschmid@suse.de>
Reviewed-by: Sebastian Wagner <swagner@suse.com>
5 years agoMerge PR #33627 into master
Sage Weil [Sun, 1 Mar 2020 20:26:09 +0000 (14:26 -0600)]
Merge PR #33627 into master

* refs/pull/33627/head:
doc/cephadm/administration: clarify log gathering

Reviewed-by: Sage Weil <sage@redhat.com>
5 years agoMerge PR #33553 into master
Sage Weil [Sun, 1 Mar 2020 20:18:39 +0000 (14:18 -0600)]
Merge PR #33553 into master

* refs/pull/33553/head:
mgr/cephadm: orch ls: include specs with no daemons
mgr/cephadm: orch ls: show spec size
mgr/orch: remove unused fields in RGWSpec
mgr/orch: fix ServiceSpec fields
mgr/cephadm: simplify spec apply
pybind/mgr/mgr_module: revert PersistentStoreDict seperator
mgr/cephadm: apply services after refreshing inventory
mgr/cephadm: catch and log exceptions from apply
mgr/orch: no extra whitespace in stored json specs
mgr/cephadm: drop daemon_type arg to _apply_service
mgr/cephadm: use _apply() helper for all apply_ methods
mgr/cephadm: replace PersistentStoreDict with SpecStore
mgr/cephadm: do not remove service spec when removing a daemon
mgr/cephadm: rename completion variables&cleanup
mgr/cephadm: leverage service specs

Reviewed-by: Sage Weil <sage@redhat.com>
5 years agomgr/test_orchestrator: add daemon_action 33541/head
Sage Weil [Sun, 1 Mar 2020 20:18:16 +0000 (14:18 -0600)]
mgr/test_orchestrator: add daemon_action

Signed-off-by: Sage Weil <sage@redhat.com>
5 years agoqa/tasksmgr/test_orchestrator_cli: fix test_service_action
Sage Weil [Sun, 1 Mar 2020 17:52:36 +0000 (11:52 -0600)]
qa/tasksmgr/test_orchestrator_cli: fix test_service_action

Signed-off-by: Sage Weil <sage@redhat.com>
5 years agoqa/tasks/test_orchestrator_cli: disable test_osd_create exception check
Sage Weil [Sun, 1 Mar 2020 16:48:42 +0000 (10:48 -0600)]
qa/tasks/test_orchestrator_cli: disable test_osd_create exception check

I'm not sure what behavior we want here...

Signed-off-by: Sage Weil <sage@redhat.com>
5 years agomgr/cephadm: orch ls: include specs with no daemons 33553/head
Sage Weil [Sun, 1 Mar 2020 14:27:11 +0000 (08:27 -0600)]
mgr/cephadm: orch ls: include specs with no daemons

This way you see entries like

NAME           RUNNING  REFRESHED  IMAGE NAME                                      IMAGE ID        SPEC
mds.foo            0/1  -          <unknown>                                       <unknown>     present

before any daemons have been created.

Signed-off-by: Sage Weil <sage@redhat.com>
5 years agomgr/cephadm: orch ls: show spec size
Sage Weil [Sun, 1 Mar 2020 14:23:40 +0000 (08:23 -0600)]
mgr/cephadm: orch ls: show spec size

Signed-off-by: Sage Weil <sage@redhat.com>
5 years agomgr/orch: remove unused fields in RGWSpec
Sage Weil [Sun, 1 Mar 2020 15:05:50 +0000 (09:05 -0600)]
mgr/orch: remove unused fields in RGWSpec

Signed-off-by: Sage Weil <sage@redhat.com>
5 years agomgr/orch: fix ServiceSpec fields
Sage Weil [Sun, 1 Mar 2020 13:48:17 +0000 (07:48 -0600)]
mgr/orch: fix ServiceSpec fields

- service_type is required.  Make it the first position arg to the ctor.
- service_id is the id *only* and optional.
- service_name() is the full service name (no change)

The old 'name' was previously used as the id only, so it was poorly named,
and optional, but in this series was changed to include the type, breaking
naming for a bunch of things (e.g., daemons called mds.mds.fsname.xyz).

Signed-off-by: Sage Weil <sage@redhat.com>
5 years agomgr/cephadm: simplify spec apply
Sage Weil [Sun, 1 Mar 2020 03:09:57 +0000 (21:09 -0600)]
mgr/cephadm: simplify spec apply

- Teach _apply_service how to pick the create (and config) functions, so
  that we don't need any weird wrappers in the callers.
- Replace trigger_deploy() and _apply_services() with a simpler
  _apply_all_services()
- Drop all of the per-type _apply_foo() methods.

Signed-off-by: Sage Weil <sage@redhat.com>
5 years agopybind/mgr/mgr_module: revert PersistentStoreDict seperator
Sage Weil [Sun, 1 Mar 2020 02:47:38 +0000 (20:47 -0600)]
pybind/mgr/mgr_module: revert PersistentStoreDict seperator

This fixes the self-test.

Signed-off-by: Sage Weil <sage@redhat.com>
5 years agomgr/cephadm: apply services after refreshing inventory
Sage Weil [Sat, 29 Feb 2020 21:59:25 +0000 (15:59 -0600)]
mgr/cephadm: apply services after refreshing inventory

Ensure we have an inventory to apply against.

Signed-off-by: Sage Weil <sage@redhat.com>
5 years agomgr/cephadm: catch and log exceptions from apply
Sage Weil [Sat, 29 Feb 2020 21:55:40 +0000 (15:55 -0600)]
mgr/cephadm: catch and log exceptions from apply

Signed-off-by: Sage Weil <sage@redhat.com>
5 years agomgr/orch: no extra whitespace in stored json specs
Sage Weil [Fri, 28 Feb 2020 23:05:26 +0000 (17:05 -0600)]
mgr/orch: no extra whitespace in stored json specs

Signed-off-by: Sage Weil <sage@redhat.com>
5 years agomgr/cephadm: drop daemon_type arg to _apply_service
Sage Weil [Fri, 28 Feb 2020 23:05:05 +0000 (17:05 -0600)]
mgr/cephadm: drop daemon_type arg to _apply_service

This is part of ServiceSpec now.

Signed-off-by: Sage Weil <sage@redhat.com>
5 years agomgr/cephadm: use _apply() helper for all apply_ methods
Sage Weil [Fri, 28 Feb 2020 22:55:31 +0000 (16:55 -0600)]
mgr/cephadm: use _apply() helper for all apply_ methods

Clean up a few that were missed (due to racing PRs, presumably).

'creation' -> 'update', since an apply doesn't necessarily create (it may
destroy!).

Signed-off-by: Sage Weil <sage@redhat.com>
5 years agomgr/cephadm: replace PersistentStoreDict with SpecStore
Sage Weil [Fri, 28 Feb 2020 22:38:26 +0000 (16:38 -0600)]
mgr/cephadm: replace PersistentStoreDict with SpecStore

Explicit implementation of the dict of specs.

Signed-off-by: Sage Weil <sage@redhat.com>
5 years agomgr/cephadm: do not remove service spec when removing a daemon
Sage Weil [Fri, 28 Feb 2020 21:32:04 +0000 (15:32 -0600)]
mgr/cephadm: do not remove service spec when removing a daemon

This makes our behavior similar to kube: if you kill a pod, the operator
or controller will come along and create a new one (probably somewhere
else).

Signed-off-by: Sage Weil <sage@redhat.com>
5 years agomgr/cephadm: rename completion variables&cleanup
Joshua Schmid [Thu, 27 Feb 2020 12:30:56 +0000 (13:30 +0100)]
mgr/cephadm: rename completion variables&cleanup

Signed-off-by: Joshua Schmid <jschmid@suse.de>
5 years agomgr/cephadm: leverage service specs
Joshua Schmid [Wed, 26 Feb 2020 13:26:42 +0000 (14:26 +0100)]
mgr/cephadm: leverage service specs

Fixes: https://tracker.ceph.com/issues/44205
This does a couple of things:

* Change the way apply_$service() works:

Instead of triggering the deployment mechanism it will rather
transform the already passed ServiceSpec into a json representation
and save it in a persistent mon_store section.

`mgr/cephadm/service_spec/$service|daemon_type/service_name`

These locations will be periodically checked in the serve() thread.
This works since all the apply_$service_type functions are idempotent.

* Allow to save a config-like specification in the mon_store.

`ceph orch apply -i <service_spec_file.yaml>`

will read the specified services and save them in the mon store
section like mentioned above. The same serve() mechanism like above
also applies to deployment.

Signed-off-by: Joshua Schmid <jschmid@suse.de>
5 years agoqa/suites/rados/singleton-bluestore/cephtool: whitelist MON_DOWN 33645/head
Sage Weil [Sun, 1 Mar 2020 14:03:21 +0000 (08:03 -0600)]
qa/suites/rados/singleton-bluestore/cephtool: whitelist MON_DOWN

cephtool/test.sh now includes a test that disallows mon from the quorum
for a short period.

Signed-off-by: Sage Weil <sage@redhat.com>
5 years agoqa/suites/rados/verity/tasks/mon_recovery: whitelist SLOW_OPS 33644/head
Sage Weil [Sun, 1 Mar 2020 13:58:11 +0000 (07:58 -0600)]
qa/suites/rados/verity/tasks/mon_recovery: whitelist SLOW_OPS

The mon can see slow ops when thrashing.

Signed-off-by: Sage Weil <sage@redhat.com>
5 years agomgr/test_orchestrator: update_foo -> apply_foo
Sage Weil [Sun, 1 Mar 2020 13:23:54 +0000 (07:23 -0600)]
mgr/test_orchestrator: update_foo -> apply_foo

Signed-off-by: Sage Weil <sage@redhat.com>
5 years agoMerge PR #33639 into master
Sage Weil [Sun, 1 Mar 2020 13:22:45 +0000 (07:22 -0600)]
Merge PR #33639 into master

* refs/pull/33639/head:
pybind/mgr/mgr_module: fix standby module logging options

Reviewed-by: Kefu Chai <kchai@redhat.com>
5 years agoMerge pull request #31109 from liupengs/wip-msg-async-fix-event-center-block
Kefu Chai [Sun, 1 Mar 2020 07:47:19 +0000 (15:47 +0800)]
Merge pull request #31109 from liupengs/wip-msg-async-fix-event-center-block

msg/async/rdma: unblock event center if the peer is down when connecting

Reviewed-by: Kefu Chai <kchai@redhat.com>
5 years agomsg/async/rdma: move C_handle_connection to RDMAConnectionSocketImpl.cc 31109/head
liupengs [Sun, 1 Mar 2020 05:04:37 +0000 (13:04 +0800)]
msg/async/rdma: move C_handle_connection to RDMAConnectionSocketImpl.cc

Signed-off-by: Peng Liu <liupeng37@baidu.com>
5 years agoMerge pull request #33591 from badone/wip-install-deps-set-gpgcheck-for-reals
Kefu Chai [Sun, 1 Mar 2020 05:43:26 +0000 (13:43 +0800)]
Merge pull request #33591 from badone/wip-install-deps-set-gpgcheck-for-reals

install-deps.sh: Actually set gpgcheck to false

Reviewed-by: Dan Mick <dan.mick@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
5 years agomsg/async/rdma: fix bug event center is blocked by rdma construct connection for...
liupengs [Sun, 17 Nov 2019 15:03:07 +0000 (23:03 +0800)]
msg/async/rdma: fix bug event center is blocked by rdma construct connection for transport ib sync msg

We construct a tcp connection to transport ib sync msg, if the
remote node is shutdown (shutdown by accident), the net.connect will be blocked until timeout
is reached, which cause the event center be blocked.

This bug may cause mon probe timeout and osd not reply, and so on.

Signed-off-by: Peng Liu <liupeng37@baidu.com>
5 years agomgr/test_orchestrator: add force flag to remove_daemons
Sage Weil [Sun, 1 Mar 2020 01:29:38 +0000 (19:29 -0600)]
mgr/test_orchestrator: add force flag to remove_daemons

Signed-off-by: Sage Weil <sage@redhat.com>
5 years agoqa/tasks/mgr/test_orchestrator_cli: update
Sage Weil [Sat, 29 Feb 2020 23:37:18 +0000 (17:37 -0600)]
qa/tasks/mgr/test_orchestrator_cli: update

Most of these were broken due to CLI changes weeks ago.

Signed-off-by: Sage Weil <sage@redhat.com>
5 years agoMerge PR #33634 into master
Sage Weil [Sat, 29 Feb 2020 22:13:07 +0000 (16:13 -0600)]
Merge PR #33634 into master

* refs/pull/33634/head:
qa/workunits/cephadm/test_cephadm.sh: dump logs on exit
qa/workunits/cephadm/test_cephadm.sh: add `cleanup` function

Reviewed-by: Sage Weil <sage@redhat.com>
5 years agopybind/mgr/mgr_module: fix standby module logging options 33639/head
Sage Weil [Sat, 29 Feb 2020 14:58:59 +0000 (08:58 -0600)]
pybind/mgr/mgr_module: fix standby module logging options

We need to define the module options and their default so that
_configure_logging can succeed.

Broken by 8ec3b3d3cc7d4c2d2678f460de27ecc8cb07d8fe

Signed-off-by: Sage Weil <sage@redhat.com>
5 years agoMerge PR #33438 into master
Sage Weil [Sat, 29 Feb 2020 03:32:47 +0000 (21:32 -0600)]
Merge PR #33438 into master

* refs/pull/33438/head:
cephadm: add prometheus adopt

Reviewed-by: Sage Weil <sage@redhat.com>
5 years agoMerge PR #33433 into master
Sage Weil [Sat, 29 Feb 2020 03:32:36 +0000 (21:32 -0600)]
Merge PR #33433 into master

* refs/pull/33433/head:
cephadm: also return JSON decode error.

Reviewed-by: Sage Weil <sage@redhat.com>
5 years agoMerge pull request #33596 from badone/wip-serve-doc-python3
Kefu Chai [Sat, 29 Feb 2020 01:58:01 +0000 (09:58 +0800)]
Merge pull request #33596 from badone/wip-serve-doc-python3

admin/serve-doc: Switch to python3 only

Reviewed-by: Kefu Chai <kchai@redhat.com>
5 years agoadmin/serve-doc: Switch to python3 only 33596/head
Brad Hubbard [Fri, 28 Feb 2020 05:25:17 +0000 (15:25 +1000)]
admin/serve-doc: Switch to python3 only

Signed-off-by: Brad Hubbard <bhubbard@redhat.com>
5 years agoqa/workunits/cephadm/test_cephadm.sh: dump logs on exit 33634/head
Michael Fritch [Fri, 28 Feb 2020 21:46:59 +0000 (14:46 -0700)]
qa/workunits/cephadm/test_cephadm.sh: dump logs on exit

dumps the last few lines from each of the surviving daemon logs

Signed-off-by: Michael Fritch <mfritch@suse.com>
5 years agoqa/workunits/cephadm/test_cephadm.sh: add `cleanup` function
Michael Fritch [Fri, 28 Feb 2020 21:45:42 +0000 (14:45 -0700)]
qa/workunits/cephadm/test_cephadm.sh: add `cleanup` function

moves logic for clean-up during `trap EXIT` into a function

Signed-off-by: Michael Fritch <mfritch@suse.com>
5 years agoMerge pull request #33624 from trociny/wip-dateutil-dep
Jason Dillaman [Fri, 28 Feb 2020 20:40:01 +0000 (15:40 -0500)]
Merge pull request #33624 from trociny/wip-dateutil-dep

rpm,deb: fix python dateutil module dependency

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
Reviewed-by: Nathan Cutler <ncutler@suse.com>
5 years agoNVMEDevice: Remove the unnecessary aio_wait in sync read 33597/head
Ziye Yang [Fri, 28 Feb 2020 17:12:57 +0000 (01:12 +0800)]
NVMEDevice: Remove the unnecessary aio_wait in sync read

Using the aio_wait are unncessary, since all the async read
submission and completion happen in the same thread.

Signed-off-by: Ziye Yang <ziye.yang@intel.com>
5 years agodoc/cephadm/administration: clarify log gathering 33627/head
Nathan Cutler [Fri, 28 Feb 2020 17:16:17 +0000 (18:16 +0100)]
doc/cephadm/administration: clarify log gathering

This is an attempt to bring the current state of the documentation more
into line with the current state of the cephadm code.

However, when I try to grab logs from a daemon on a host other than the
one where the daemon is running, I get an empty log...

References: https://tracker.ceph.com/issues/44354
Signed-off-by: Nathan Cutler <ncutler@suse.com>
5 years agorpm,deb: fix python dateutil module dependency 33624/head
Mykola Golub [Fri, 28 Feb 2020 16:24:01 +0000 (16:24 +0000)]
rpm,deb: fix python dateutil module dependency

(needed for mgr/rbd_support)

Signed-off-by: Mykola Golub <mgolub@suse.com>
5 years agoMerge pull request #33063 from theanalyst/boto3-ext-append
Casey Bodley [Fri, 28 Feb 2020 14:31:14 +0000 (09:31 -0500)]
Merge pull request #33063 from theanalyst/boto3-ext-append

examples: rgw: add boto3 append & get usage api extensions

Reviewed-by: Yuval Lifshitz <yuvalif@yahoo.com>
5 years agoMerge pull request #33527 from alimaredia/wip-rgw-realm-pull-log-msg
Casey Bodley [Fri, 28 Feb 2020 14:29:18 +0000 (09:29 -0500)]
Merge pull request #33527 from alimaredia/wip-rgw-realm-pull-log-msg

rgw: increase log level for same or older period pull msg

Reviewed-by: Yuval Lifshitz <yuvalif@yahoo.com>
5 years agoMerge pull request #33600 from ceph/doc_config_set
Sebastian Wagner [Fri, 28 Feb 2020 12:58:49 +0000 (13:58 +0100)]
Merge pull request #33600 from ceph/doc_config_set

doc/cephadm: adjust syntax for config set

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>
5 years agodoc/cephadm: adjust syntax for config set 33600/head
Joshua Schmid [Fri, 28 Feb 2020 09:05:31 +0000 (10:05 +0100)]
doc/cephadm: adjust syntax for config set

Signed-off-by: Joshua Schmid <jschmid@suse.de>
5 years agoMerge PR #33226 into master
Sage Weil [Fri, 28 Feb 2020 09:17:39 +0000 (03:17 -0600)]
Merge PR #33226 into master

* refs/pull/33226/head:
unittest_auth: update for new ms_*_mode semantics
auth: treat mgr the same as mon when selecting auth mode

Reviewed-by: Sage Weil <sage@redhat.com>
5 years agoMerge PR #33542 into master
Sage Weil [Fri, 28 Feb 2020 09:15:58 +0000 (03:15 -0600)]
Merge PR #33542 into master

* refs/pull/33542/head:
mon/Monitor: fail forwarded tell commands

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: João Eduardo Luís <joao@suse.de>
5 years agoMerge PR #33558 into master
Sage Weil [Fri, 28 Feb 2020 09:15:46 +0000 (03:15 -0600)]
Merge PR #33558 into master

* refs/pull/33558/head:
common/config: fix lack of normalize_key_name() apply.

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Jianpeng Ma <jianpeng.ma@intel.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
5 years agocrimson/osd: fix load_pg() during initialization 33595/head
Yingxin Cheng [Fri, 28 Feb 2020 04:27:12 +0000 (12:27 +0800)]
crimson/osd: fix load_pg() during initialization

fix a regression introduced by 80ac50f which dropped PG::read_state()
incorrectly.

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
5 years agocrimson/osd: use cancellable timer to dispatch delayed events
Yingxin Cheng [Fri, 28 Feb 2020 04:22:54 +0000 (12:22 +0800)]
crimson/osd: use cancellable timer to dispatch delayed events

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
5 years agocrimson/osd: fix conditions to update up_epoch
Yingxin Cheng [Fri, 28 Feb 2020 04:19:03 +0000 (12:19 +0800)]
crimson/osd: fix conditions to update up_epoch

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
5 years agocrimson: complete context delete in alien world 31041/head
Chunmei Liu [Fri, 28 Feb 2020 01:50:15 +0000 (17:50 -0800)]
crimson: complete context delete in alien world

This way, we allocate and release only the OnCommit Context on the bluestore
side.  Similarly, the crimson_wrapper context containing the passed in
contexts is allocated and released only on the crimson side.

Further, update do_transaction to also call on_applied and on_applied_sync
for completeness.

Signed-off-by: Samuel Just <sjust@redhat.com>
5 years agocrimson:add alien blue store
Chunmei Liu [Wed, 22 Jan 2020 07:25:02 +0000 (23:25 -0800)]
crimson:add alien blue store

adds an alien store wrapper for ceph blue store, which will
create a thread pool, so crimson-osd can call bluestore API
which is running in Posix thread from seastar thread.

Signed-off-by: Chunmei Liu <chunmei.liu@intel.com>
5 years agocrimson:disable heap profiler for seastar
Chunmei Liu [Wed, 22 Jan 2020 07:06:41 +0000 (23:06 -0800)]
crimson:disable heap profiler for seastar

alien bluestore will link with heap profiler, but seastar not support
heap profiler with seastar defaut allocator, so disable it.

Signed-off-by: Chunmei Liu <chunmei.liu@intel.com>
5 years agobluestore: define dout_context for dout
Chunmei Liu [Wed, 22 Jan 2020 07:05:25 +0000 (23:05 -0800)]
bluestore: define dout_context for dout

this helps to silences the unused variable when compiling BlueStore.cc
for alienstore, where we will not use cct for writing logs.

Signed-off-by: Kefu Chai <kchai@redhat.com>
5 years agoos/bluestore: add BlueStore::omap_get_values()
Chunmei Liu [Wed, 22 Jan 2020 07:03:56 +0000 (23:03 -0800)]
os/bluestore: add BlueStore::omap_get_values()

used by crimson::alienstore. as crimson is not using
`ObjectStore::get_omap_iterator()` for implementing
CEPH_OSD_OP_OMAPGETVALS op. instead, it's using a paged variant of
`ObjectStore::omap_get_values()`, which only exists in
CyanStore before this change.

Signed-off-by: Chunmei Liu <chunmei.liu@intel.com>
5 years agocrimson:add more futurized store API for alien store
Chunmei Liu [Tue, 21 Jan 2020 19:48:08 +0000 (11:48 -0800)]
crimson:add more futurized store API for alien store

since alien store is futurized store, need change all store
API to be futurized.

Signed-off-by: Chunmei Liu <chunmei.liu@intel.com>
5 years agocrimson:common add TOPNSPC namespace for ceph and crimson
Chunmei Liu [Fri, 17 Jan 2020 23:09:10 +0000 (15:09 -0800)]
crimson:common add TOPNSPC namespace for ceph and crimson

some code coexist in crimson seastar environment and posix environment,
so add namespace to avoid same function conflict, for example add namespace
for CephContext, since the new namespace for classic ceph-osd,
need modify all files declare use CephContext by including "common_fwd.h"
which defined the namespace for each environment.

Signed-off-by: Chunmei Liu <chunmei.liu@intel.com>
5 years agoMerge pull request #33566 from dzafman/wip-44296
Kefu Chai [Fri, 28 Feb 2020 03:42:47 +0000 (11:42 +0800)]
Merge pull request #33566 from dzafman/wip-44296

test: Expect being off by up to 2 and make sure all PGs are active+clean

Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
5 years agoMerge pull request #33592 from tchaikov/wip-doc-search
Kefu Chai [Fri, 28 Feb 2020 03:33:05 +0000 (11:33 +0800)]
Merge pull request #33592 from tchaikov/wip-doc-search

doc,admin: fix the builtin search

Reviewed-by: Brad Hubbard <bhubbard@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>
5 years agoinstall-deps.sh: Actually set gpgcheck to false 33591/head
Brad Hubbard [Fri, 28 Feb 2020 01:47:55 +0000 (11:47 +1000)]
install-deps.sh: Actually set gpgcheck to false

The current command appears to do nothing.

Fixes: https://tracker.ceph.com/issues/44344
Signed-off-by: Brad Hubbard <bhubbard@redhat.com>
5 years agotest: Fix failing ceph_objectstore_tool.py test 33593/head
David Zafman [Fri, 28 Feb 2020 01:18:05 +0000 (17:18 -0800)]
test: Fix failing ceph_objectstore_tool.py test

The -N option to vstart.sh was removed, use -k

Old hinfo_key binary happen to be utf-8 decodable, now it
throws an exception trying to decode it. Use new
option to ceph-objectstore-tool to treat stdout as a terminal
and convert binary data to base64.

Signed-off-by: David Zafman <dzafman@redhat.com>
5 years agodoc: drop http://ayni.ceph.com/public/js/ceph.js 33592/head
Kefu Chai [Fri, 28 Feb 2020 01:50:49 +0000 (09:50 +0800)]
doc: drop http://ayni.ceph.com/public/js/ceph.js

seems we will never have this file in foreseeable future.

See-also: https://tracker.ceph.com/issues/11490
See-also: https://tracker.ceph.com/issues/18075
Signed-off-by: Kefu Chai <kchai@redhat.com>
5 years agotest: Expect being off by up to 2 and make sure all PGs are active+clean 33566/head
David Zafman [Tue, 25 Feb 2020 23:32:58 +0000 (15:32 -0800)]
test: Expect being off by up to 2 and make sure all PGs are active+clean

Fixes: https://tracker.ceph.com/issues/44296
Signed-off-by: David Zafman <dzafman@redhat.com>
5 years agodoc/_static/js/ceph.js: avoid the "//" in URI
Kefu Chai [Fri, 28 Feb 2020 01:50:17 +0000 (09:50 +0800)]
doc/_static/js/ceph.js: avoid the "//" in URI

it's just annoying.

Signed-off-by: Kefu Chai <kchai@redhat.com>
5 years agoadmin/doc-requirements: bump sphinx version
Kefu Chai [Fri, 28 Feb 2020 01:49:44 +0000 (09:49 +0800)]
admin/doc-requirements: bump sphinx version

to pick up the fix to address the search when using dirhtml builder

see https://github.com/sphinx-doc/sphinx/pull/6741

Signed-off-by: Kefu Chai <kchai@redhat.com>
5 years agoMerge PR #33554 into master
Sage Weil [Fri, 28 Feb 2020 01:49:05 +0000 (19:49 -0600)]
Merge PR #33554 into master

* refs/pull/33554/head:
mgr/cephadm: Configure Alertmanagers in Prometheus configuration
mgr/cephadm: Create configuration for alertmanager
mgr/cephadm: Provision alertmanager
mgr/orchestrator: Add alertmanager hooks

Reviewed-by: Sebastian Wagner <swagner@suse.com>
Reviewed-by: Sage Weil <sage@redhat.com>
5 years agoMerge PR #33585 into master
Sage Weil [Fri, 28 Feb 2020 01:48:24 +0000 (19:48 -0600)]
Merge PR #33585 into master

* refs/pull/33585/head:
mgr/orchestrator: fix 'orch host label rm' help text

Reviewed-by: Michael Fritch <mfritch@suse.com>
5 years agoMerge PR #33586 into master
Sage Weil [Fri, 28 Feb 2020 01:48:13 +0000 (19:48 -0600)]
Merge PR #33586 into master

* refs/pull/33586/head:
mgr/orch: fix 'orch ls' table spacing

Reviewed-by: Michael Fritch <mfritch@suse.com>
Reviewed-by: Sebastian Wagner <swagner@suse.com>
5 years agoMerge PR #33587 into master
Sage Weil [Fri, 28 Feb 2020 01:47:50 +0000 (19:47 -0600)]
Merge PR #33587 into master

* refs/pull/33587/head:
remove cephadm-adoption-corpus as submodule

Reviewed-by: Michael Fritch <mfritch@suse.com>
Reviewed-by: Sebastian Wagner <swagner@suse.com>
5 years agoMerge PR #33488 into master
Sage Weil [Fri, 28 Feb 2020 01:46:20 +0000 (19:46 -0600)]
Merge PR #33488 into master

* refs/pull/33488/head:
doc/mgr/cephadm: document how to see the logs
ceph: -W as shorthand for --watch-channel
pybind/mgr_module: make mgr log message prefix a bit less verbose
mgr/cephadm: silence remoto logger info messages
mgr/cephadm: add misc info messages for major events
mgr/cephadm: adjust logging
mgr/cephadm: log to cluster channel by default
pybind/mgr_module: infrastructure to allow logging to cluster log
mgr/mgr_module: log to mgr log even when log_to_file
mon: accept cephadm channel to 'ceph log last'
ceph: allow watching channels aside from cluster and audit
mon/LogMonitor: fix debug output for logmonitor output file parsing
mgr: allow logging to any channel

Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
5 years agoMerge PR #32983 into master
Sage Weil [Fri, 28 Feb 2020 01:44:54 +0000 (19:44 -0600)]
Merge PR #32983 into master

* refs/pull/32983/head:
mgr/cephadm: remove/replace osds

Reviewed-by: Sebastian Wagner <swagner@suse.com>
5 years agoMerge pull request #33389 from trociny/wip-43619
Jason Dillaman [Thu, 27 Feb 2020 23:21:14 +0000 (18:21 -0500)]
Merge pull request #33389 from trociny/wip-43619

rbd: schedule for running trash purge operations

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
5 years agoqa/tasks/mgr/test_orchestrator_cli: fix test_load_data
Sage Weil [Mon, 24 Feb 2020 20:02:17 +0000 (14:02 -0600)]
qa/tasks/mgr/test_orchestrator_cli: fix test_load_data

Signed-off-by: Sage Weil <sage@redhat.com>
5 years agomgr/test_orchestrator: fix list_daemons
Sage Weil [Tue, 25 Feb 2020 22:50:14 +0000 (16:50 -0600)]
mgr/test_orchestrator: fix list_daemons

Signed-off-by: Sage Weil <sage@redhat.com>
5 years agoMerge pull request #32554 from dzafman/wip-dump-perf
David Zafman [Thu, 27 Feb 2020 21:48:29 +0000 (13:48 -0800)]
Merge pull request #32554 from dzafman/wip-dump-perf

Improve internal python to c++ interface

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>
5 years agoMerge pull request #33493 from zhengchengyao/read
Jason Dillaman [Thu, 27 Feb 2020 21:18:20 +0000 (16:18 -0500)]
Merge pull request #33493 from zhengchengyao/read

librbd: improved support for balanced and localized reads

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
5 years agoMerge pull request #33563 from dillaman/wip-rbd-op-threads
Jason Dillaman [Thu, 27 Feb 2020 21:16:57 +0000 (16:16 -0500)]
Merge pull request #33563 from dillaman/wip-rbd-op-threads

librbd: fix potential race conditions

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
5 years agotest: Disable self-test of diskprediction_cloud since it isn't loaded 32554/head
David Zafman [Thu, 16 Jan 2020 16:15:41 +0000 (16:15 +0000)]
test: Disable self-test of diskprediction_cloud since it isn't loaded

See qa/packages/packages.yaml

Signed-off-by: David Zafman <dzafman@redhat.com>
5 years agomon mgr osd: Add dump_osd_times interface for python
David Zafman [Fri, 10 Jan 2020 23:42:19 +0000 (23:42 +0000)]
mon mgr osd: Add dump_osd_times interface for python

The dump_osd_times interface is here for future use of a manager module.
The osd_stats and pg_dump python interfaces don't return network ping
information.

User facing ping time information is formatted 3 decimal places
Use dump_float() (low overhead) for network ping times
   used by python dump_osd_times and dump commands like
   "ceph --formet=json pg dump osds" which for now yields output like
   >>>> "1min": 0.61599999999999999,

Signed-off-by: David Zafman <dzafman@redhat.com>
5 years agomgr/progress: Add pg_ready interface for python to get pgmap_ready state
David Zafman [Thu, 9 Jan 2020 00:02:51 +0000 (16:02 -0800)]
mgr/progress: Add pg_ready interface for python to get pgmap_ready state

Fixes: https://tracker.ceph.com/issues/43557
Signed-off-by: David Zafman <dzafman@redhat.com>