ceph.git
2 hours agoMerge pull request #31338 from ivancich/wip-clean-up-ordered-list master
J. Eric Ivancich [Thu, 20 Feb 2020 19:06:23 +0000 (14:06 -0500)]
Merge pull request #31338 from ivancich/wip-clean-up-ordered-list

rgw: clean up ordered list

Reviewed-by: Casey Bodley <cbodley@redhat.com>
4 hours agoMerge pull request #33338 from smithfarm/wip-qa-cephadm-smoke-opensuse
Nathan Cutler [Thu, 20 Feb 2020 17:03:52 +0000 (18:03 +0100)]
Merge pull request #33338 from smithfarm/wip-qa-cephadm-smoke-opensuse

qa/cephadm/smoke: run on opensuse_15.1

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Sebastian Wagner <swagner@suse.com>
5 hours agoqa/cephadm: "ls -l" cephadm after downloading it 33338/head
Nathan Cutler [Thu, 20 Feb 2020 16:26:07 +0000 (17:26 +0100)]
qa/cephadm: "ls -l" cephadm after downloading it

For better debugging when the download goes awry.

Signed-off-by: Nathan Cutler <ncutler@suse.com>
5 hours agoqa/cephadm.py: better check for cephadm download success
Nathan Cutler [Thu, 20 Feb 2020 11:12:56 +0000 (12:12 +0100)]
qa/cephadm.py: better check for cephadm download success

When the URL passed to "curl --silent ..." does not exist, the
resulting file will be populated with the string:

    404: Not Found

If that (or something similar) happens, the file size will be
suspiciously low, like < 1000 bytes. Fail the test immediately in this
case.

Signed-off-by: Nathan Cutler <ncutler@suse.com>
5 hours agoqa/tasks/cephadm.py: lop off .git more robustly
Nathan Cutler [Thu, 20 Feb 2020 10:59:53 +0000 (11:59 +0100)]
qa/tasks/cephadm.py: lop off .git more robustly

The line

    rest.replace('.git/', '/')

was added to accommodate weird folks who run teuthology-suite
with an option like this:

    --suite-repo https://github.com/ceph/ceph.git/

but they might just as well give the option like this:

    --suite-repo https://github.com/ceph/ceph.git

Signed-off-by: Nathan Cutler <ncutler@suse.com>
5 hours agoMerge pull request #31994 from pritha-srivastava/wip-rgw-assume-role-web-id-doc
Matt Benjamin [Thu, 20 Feb 2020 15:58:07 +0000 (10:58 -0500)]
Merge pull request #31994 from pritha-srivastava/wip-rgw-assume-role-web-id-doc

rgw: adding documentation for AssumeRoleWithWebIdentity.

7 hours agoMerge pull request #33166 from dillaman/wip-43936
Mykola Golub [Thu, 20 Feb 2020 13:53:56 +0000 (15:53 +0200)]
Merge pull request #33166 from dillaman/wip-43936

rbd-mirror: initial snapshot replay state machine

Reviewed-by: Mykola Golub <mgolub@suse.com>
10 hours agoMerge pull request #33425 from tchaikov/wip-crimson-watcher
Kefu Chai [Thu, 20 Feb 2020 11:11:46 +0000 (19:11 +0800)]
Merge pull request #33425 from tchaikov/wip-crimson-watcher

crimson/osd: capture watcher when calling its member function

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
10 hours agoMerge pull request #33423 from tchaikov/wip-doc-search
Kefu Chai [Thu, 20 Feb 2020 11:09:34 +0000 (19:09 +0800)]
Merge pull request #33423 from tchaikov/wip-doc-search

doc: fix errors in "search" page and use relative address for releases.json

Reviewed-by: Brad Hubbard <bhubbard@redhat.com>
10 hours agoqa/cephadm/smoke: run on opensuse_15.1
Nathan Cutler [Fri, 14 Feb 2020 17:24:54 +0000 (18:24 +0100)]
qa/cephadm/smoke: run on opensuse_15.1

Signed-off-by: Nathan Cutler <ncutler@suse.com>
10 hours agoqa/distros: add opensuse_15.2.yaml
Nathan Cutler [Fri, 14 Feb 2020 17:22:52 +0000 (18:22 +0100)]
qa/distros: add opensuse_15.2.yaml

We don't have a FOG image for this, yet, and Shaman is not building for
it (yet), either, but we might as well get this file in place now, since
we already know it will be needed at some point.

Signed-off-by: Nathan Cutler <ncutler@suse.com>
10 hours agoqa/distros: drop obsolete opensuse distros
Nathan Cutler [Fri, 14 Feb 2020 17:22:02 +0000 (18:22 +0100)]
qa/distros: drop obsolete opensuse distros

Signed-off-by: Nathan Cutler <ncutler@suse.com>
11 hours agoMerge pull request #33371 from leseb/raw-list
Sébastien Han [Thu, 20 Feb 2020 10:41:01 +0000 (11:41 +0100)]
Merge pull request #33371 from leseb/raw-list

ceph-volume: silence 'ceph-bluestore-tool' failures

11 hours agoMerge pull request #33407 from rhcs-dashboard/coverage-venv-python-version
Lenz Grimmer [Thu, 20 Feb 2020 10:15:02 +0000 (10:15 +0000)]
Merge pull request #33407 from rhcs-dashboard/coverage-venv-python-version

mgr/dashboard: coverage venv python version same as mgr

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Laura Paduano <lpaduano@suse.com>
Reviewed-by: Stephan Müller <smueller@suse.com>
13 hours agocrimson/osd: capture watcher when calling its member function 33425/head
Kefu Chai [Thu, 20 Feb 2020 08:09:19 +0000 (16:09 +0800)]
crimson/osd: capture watcher when calling its member function

otherwise the extracted node handler will be destroyed, and so does the
watcher enclosed in it.

Signed-off-by: Kefu Chai <kchai@redhat.com>
15 hours agodoc: do not use absolute address 33423/head
Kefu Chai [Thu, 20 Feb 2020 06:42:32 +0000 (14:42 +0800)]
doc: do not use absolute address

use relative address instead, to ease the pain when testing.

Signed-off-by: Kefu Chai <kchai@redhat.com>
15 hours agodoc: do not add suffix for search result links
Kefu Chai [Thu, 20 Feb 2020 06:34:01 +0000 (14:34 +0800)]
doc: do not add suffix for search result links

since sphinx 1.5, SOURCELINK_SUFFIX is always added to the source file,
and search page always tries to add it to the highlighted source result.

Signed-off-by: Kefu Chai <kchai@redhat.com>
15 hours agodoc: do not try to patch #edit-on-github if not found
Kefu Chai [Thu, 20 Feb 2020 05:05:45 +0000 (13:05 +0800)]
doc: do not try to patch #edit-on-github if not found

we add this element in _templates/page.html, which is only used for the
content pages, not in the "search" page. to avoid the js errors, just
disabled it if it's not found.

Signed-off-by: Kefu Chai <kchai@redhat.com>
22 hours agoMerge PR #33394 into master
Sage Weil [Wed, 19 Feb 2020 22:58:47 +0000 (16:58 -0600)]
Merge PR #33394 into master

* refs/pull/33394/head:
mgr/cephadm: trigger device refresh after osd creation
mgr/orch,cephadm: implement 'device zap'
mgr/cephadm: remove inventory_cache
mgr/cephadm: reimplement 'device ls' in terms of cache
mgr/cephadm: fix invalidate helpers
mgr/cephadm: scrape and cache devices, too
mgr/cephadm: rename DaemonCache methods etc to generalize

Reviewed-by: Michael Fritch <mfritch@suse.com>
22 hours agoMerge PR #33400 into master
Sage Weil [Wed, 19 Feb 2020 22:46:16 +0000 (16:46 -0600)]
Merge PR #33400 into master

* refs/pull/33400/head:
mgr/progress: 'progress clear' command should clear events in 'ceph -s'

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
23 hours agoMerge PR #33402 into master
Sage Weil [Wed, 19 Feb 2020 22:45:45 +0000 (16:45 -0600)]
Merge PR #33402 into master

* refs/pull/33402/head:
mgr/pg_autoscaler: fix division by zero

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
23 hours agoMerge PR #33352 into master
Sage Weil [Wed, 19 Feb 2020 22:45:01 +0000 (16:45 -0600)]
Merge PR #33352 into master

* refs/pull/33352/head:
ceph.in: catch KeyboardInterrupt from outer code
ceph.in: fix second exception on ^C

Reviewed-by: Kefu Chai <kchai@redhat.com>
23 hours agoMerge PR #33374 into master
Sage Weil [Wed, 19 Feb 2020 22:35:10 +0000 (16:35 -0600)]
Merge PR #33374 into master

* refs/pull/33374/head:
cephadm: remove distutils dependency
mgr/cephadm: add prepare-host command
cephadm: prepare-host during bootstrap
cephadm: add prepare-host command
cephadm: include docker.io in image name

Reviewed-by: Sebastian Wagner <swagner@suse.com>
24 hours agomgr/cephadm: trigger device refresh after osd creation 33394/head
Sage Weil [Tue, 18 Feb 2020 19:18:47 +0000 (13:18 -0600)]
mgr/cephadm: trigger device refresh after osd creation

Presumably the new OSD(s) consumed one or more devices.

Signed-off-by: Sage Weil <sage@redhat.com>
24 hours agomgr/orch,cephadm: implement 'device zap'
Sage Weil [Tue, 18 Feb 2020 19:06:51 +0000 (13:06 -0600)]
mgr/orch,cephadm: implement 'device zap'

Signed-off-by: Sage Weil <sage@redhat.com>
24 hours agomgr/cephadm: remove inventory_cache
Sage Weil [Tue, 18 Feb 2020 18:40:20 +0000 (12:40 -0600)]
mgr/cephadm: remove inventory_cache

No longer used.

Clean up _get_hosts now that there is only one user.

Signed-off-by: Sage Weil <sage@redhat.com>
24 hours agomgr/cephadm: reimplement 'device ls' in terms of cache
Sage Weil [Tue, 18 Feb 2020 18:36:04 +0000 (12:36 -0600)]
mgr/cephadm: reimplement 'device ls' in terms of cache

Signed-off-by: Sage Weil <sage@redhat.com>
24 hours agomgr/cephadm: fix invalidate helpers
Sage Weil [Tue, 18 Feb 2020 19:07:13 +0000 (13:07 -0600)]
mgr/cephadm: fix invalidate helpers

These should wake up the serve() thread.

Signed-off-by: Sage Weil <sage@redhat.com>
24 hours agomgr/cephadm: scrape and cache devices, too
Sage Weil [Tue, 18 Feb 2020 15:46:48 +0000 (09:46 -0600)]
mgr/cephadm: scrape and cache devices, too

Signed-off-by: Sage Weil <sage@redhat.com>
24 hours agomgr/cephadm: rename DaemonCache methods etc to generalize
Sage Weil [Tue, 18 Feb 2020 15:16:31 +0000 (09:16 -0600)]
mgr/cephadm: rename DaemonCache methods etc to generalize

The HostCache is going to store both daemon and device state.

Signed-off-by: Sage Weil <sage@redhat.com>
24 hours agoMerge PR #32677 into master
Sage Weil [Wed, 19 Feb 2020 20:50:40 +0000 (14:50 -0600)]
Merge PR #32677 into master

* refs/pull/32677/head:
build/ops: make/install osd_support package
mgr/osd_support: move docs to docs/mgr/osd_support.rst
mgr/osd_support: new module for osd utility calls

Reviewed-by: Neha Ojha <nojha@redhat.com>
24 hours agoMerge PR #33123 into master
Sage Weil [Wed, 19 Feb 2020 20:50:20 +0000 (14:50 -0600)]
Merge PR #33123 into master

* refs/pull/33123/head:
qa/tasks/cephadm: deploy node-exporter
mgr/orch: deploy node-exporter

Reviewed-by: Sebastian Wagner <swagner@suse.com>
26 hours agorgw: clean up interenal shard logic in ordered bucket list 31338/head
J. Eric Ivancich [Sat, 2 Nov 2019 21:54:54 +0000 (17:54 -0400)]
rgw: clean up interenal shard logic in ordered bucket list

The logic of RGWRados::cls_bucket_list_ordered is complex as it needs
to query multiple bucket index shards and manage their returned lists,
and walk through them in a manner to produce an appropriately ordered
set of dir entries to the caller. The previous implementation used
three of parallel std::vectors (e.g., the related data was spread
across the vectors), which needed to each be maintained
appropriately. This clean-up changes this to a single vector of
structs, and the struct both consolidates the related data and
encapsulates much of the related logic.

Additionally, const correctness is expanded.

Signed-off-by: J. Eric Ivancich <ivancich@redhat.com>
26 hours agoMerge PR #33359 into master
Sage Weil [Wed, 19 Feb 2020 18:58:51 +0000 (12:58 -0600)]
Merge PR #33359 into master

* refs/pull/33359/head:
mgr/dashboard/tests/test_orchestrator: disable service test
mgr/cephadm: fix race between check_hosts and host rm
mgr/volumes: fix remove_mds call
mgr/rook: hackly describe_service -> list_daemons
mgr/tests/test_orchestrator: fix ServiceDescription test
mgr/test_orchestrator: describe_service -> list_daemons
mgr/orch: resurrect ServiceDescription, 'orch ls'
qa/tasks/mgr/dashboard/test_orchestrator: disable services test

Reviewed-by: Sebastian Wagner <swagner@suse.com>
28 hours agomgr/dashboard/tests/test_orchestrator: disable service test 33359/head
Sage Weil [Tue, 18 Feb 2020 22:31:36 +0000 (16:31 -0600)]
mgr/dashboard/tests/test_orchestrator: disable service test

Signed-off-by: Sage Weil <sage@redhat.com>
28 hours agomgr/cephadm: fix race between check_hosts and host rm
Sage Weil [Tue, 18 Feb 2020 22:28:22 +0000 (16:28 -0600)]
mgr/cephadm: fix race between check_hosts and host rm

Fixes: https://tracker.ceph.com/issues/44188
Signed-off-by: Sage Weil <sage@redhat.com>
28 hours agorbd-mirror: do not attempt to replay incomplete primary snapshots 33166/head
Jason Dillaman [Wed, 19 Feb 2020 17:03:10 +0000 (12:03 -0500)]
rbd-mirror: do not attempt to replay incomplete primary snapshots

In case we race with the creation of a primary mirror snapshot,
do not attempt to replay incomplete snapshots. Once the image
state has been successfully written, the snapshot will be flagged
as complete and an update notification will be sent.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
28 hours agoceph.in: catch KeyboardInterrupt from outer code 33352/head
Sage Weil [Sat, 15 Feb 2020 15:10:52 +0000 (09:10 -0600)]
ceph.in: catch KeyboardInterrupt from outer code

This avoids getting random tracebacks when you control-C early in the
process.

Signed-off-by: Sage Weil <sage@redhat.com>
28 hours agolibrbd: delay marking primary mirror snapshot as completed
Jason Dillaman [Wed, 19 Feb 2020 17:02:09 +0000 (12:02 -0500)]
librbd: delay marking primary mirror snapshot as completed

Until the image state object(s) are written, the snapshot should
be considered incomplete and not ready for replay.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
28 hours agomgr/volumes: fix remove_mds call
Sage Weil [Tue, 18 Feb 2020 22:31:36 +0000 (16:31 -0600)]
mgr/volumes: fix remove_mds call

Fixes: https://tracker.ceph.com/issues/44176
Signed-off-by: Sage Weil <sage@redhat.com>
28 hours agomgr/rook: hackly describe_service -> list_daemons
Sage Weil [Mon, 17 Feb 2020 01:32:07 +0000 (19:32 -0600)]
mgr/rook: hackly describe_service -> list_daemons

Signed-off-by: Sage Weil <sage@redhat.com>
28 hours agomgr/tests/test_orchestrator: fix ServiceDescription test
Sage Weil [Mon, 17 Feb 2020 01:31:22 +0000 (19:31 -0600)]
mgr/tests/test_orchestrator: fix ServiceDescription test

Signed-off-by: Sage Weil <sage@redhat.com>
28 hours agomgr/test_orchestrator: describe_service -> list_daemons
Sage Weil [Mon, 17 Feb 2020 01:27:46 +0000 (19:27 -0600)]
mgr/test_orchestrator: describe_service -> list_daemons

Signed-off-by: Sage Weil <sage@redhat.com>
28 hours agomgr/orch: resurrect ServiceDescription, 'orch ls'
Sage Weil [Sun, 16 Feb 2020 16:17:27 +0000 (10:17 -0600)]
mgr/orch: resurrect ServiceDescription, 'orch ls'

This is a meta-service description that maps to multiple daemons.  Remove
the old useless properties, and add in size and running counts.

Signed-off-by: Sage Weil <sage@redhat.com>
28 hours agoqa/tasks/mgr/dashboard/test_orchestrator: disable services test
Sage Weil [Sun, 16 Feb 2020 15:39:27 +0000 (09:39 -0600)]
qa/tasks/mgr/dashboard/test_orchestrator: disable services test

We need to either fill out the list_daemons APIs in the dashboard and test
that, or redesign and reimplement the services abstractions in the
orchestration layer.  Until then, disable this test.

Signed-off-by: Sage Weil <sage@redhat.com>
29 hours agoMerge pull request #33333 from sebastian-philipp/cephadm-mon-apply-host-no-network
Sebastian Wagner [Wed, 19 Feb 2020 15:49:17 +0000 (16:49 +0100)]
Merge pull request #33333 from sebastian-philipp/cephadm-mon-apply-host-no-network

mgr/cephadm: mon: Don't show traceback for user errors

Reviewed-by: Sage Weil <sage@redhat.com>
29 hours agoMerge pull request #33310 from sebastian-philipp/cephadm-add-host-exception
Sebastian Wagner [Wed, 19 Feb 2020 15:48:47 +0000 (16:48 +0100)]
Merge pull request #33310 from sebastian-philipp/cephadm-add-host-exception

mgr/cephadm: Convert HostNotFound to OrchestratorError

Reviewed-by: Sage Weil <sage@redhat.com>
30 hours agorbd-mirror: snapshot replayer should only unlink peers for unneeded snaps
Jason Dillaman [Fri, 14 Feb 2020 15:48:39 +0000 (10:48 -0500)]
rbd-mirror: snapshot replayer should only unlink peers for unneeded snaps

Once the snapshot will no longer be needed by the peer for delta sync,
it should be unlinked. This will ensure the primary image will not
remove snapshots that will be needed for the next sync point.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
30 hours agorbd-mirror: initial snapshot-based mirroring replayer logic
Jason Dillaman [Thu, 6 Feb 2020 18:25:33 +0000 (13:25 -0500)]
rbd-mirror: initial snapshot-based mirroring replayer logic

This initial implementation will scan the remote image for new mirror
snapshots. If a new primary snapshot it detected, it will start syncing
it. If the remote was demoted, it will notify the image replayer to
shut down. If an in-progress sync was previously interrupted, it will
attempt to restart it.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
30 hours agorbd-mirror: pass pool meta cache to replayer state machine
Jason Dillaman [Thu, 6 Feb 2020 20:12:46 +0000 (15:12 -0500)]
rbd-mirror: pass pool meta cache to replayer state machine

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
30 hours agorbd-mirror: keep remote image open after bootstrap for snapshot mirroring
Jason Dillaman [Thu, 6 Feb 2020 15:52:29 +0000 (10:52 -0500)]
rbd-mirror: keep remote image open after bootstrap for snapshot mirroring

The snapshot-based mirroring requires an open remote image to watch for
snapshot creation and to handle the deep-copy from remote to local image.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
30 hours agotest/pybind/rbd: demoted images cannot be deleted
Jason Dillaman [Tue, 11 Feb 2020 21:08:30 +0000 (16:08 -0500)]
test/pybind/rbd: demoted images cannot be deleted

Now that we don't delete the last mirror snapshot, the test is failing
because you cannot delete a demoted image.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
30 hours agolibrbd: permit last mirror peer to be removed from mirror snapshot
Jason Dillaman [Sun, 9 Feb 2020 17:52:06 +0000 (12:52 -0500)]
librbd: permit last mirror peer to be removed from mirror snapshot

This applies only in the case where there are no additional mirror
snapshots in order to preserve the current state.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
30 hours agolibrbd: tweak create non-primary mirror snapshot request
Jason Dillaman [Fri, 7 Feb 2020 20:06:22 +0000 (15:06 -0500)]
librbd: tweak create non-primary mirror snapshot request

The replayer will need to pass in the demotion state as well as the
snapshot sequences for the copied snapshots.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
30 hours agolibrbd: no longer re-create full mirror snapshot namespace to get id
Jason Dillaman [Fri, 7 Feb 2020 19:54:52 +0000 (14:54 -0500)]
librbd: no longer re-create full mirror snapshot namespace to get id

Only the namespace type is required to locate the snap id by name.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
30 hours agolibrbd: only use snapshot namespace type in unique key sort
Jason Dillaman [Fri, 7 Feb 2020 18:30:00 +0000 (13:30 -0500)]
librbd: only use snapshot namespace type in unique key sort

Don't use the internal snapshot namespace comparator since for
the mirror snapshot the data can change. The important fact is
that snapshot names should be unique within the namespace type.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
30 hours agolibrbd: drop MirrorNonPrimarySnapshotNamespace
Jason Dillaman [Fri, 7 Feb 2020 02:14:41 +0000 (21:14 -0500)]
librbd: drop MirrorNonPrimarySnapshotNamespace

The required fields (and state enum) have already been added to the
MirrorSnapshotNamespace. This drops the namespace and tweaks existing
code to use it.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
30 hours agolibrbd: rename MirrorPrimarySnapshotNamespace to MirrorSnapshotNamespace
Jason Dillaman [Tue, 4 Feb 2020 12:57:19 +0000 (07:57 -0500)]
librbd: rename MirrorPrimarySnapshotNamespace to MirrorSnapshotNamespace

This is the first pass at merging primary and non-primary mirror snapshots
together into a unified namespace.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
30 hours agolibrbd: collapse API mirror primary/non-primary namespaces to single type
Jason Dillaman [Mon, 3 Feb 2020 20:09:28 +0000 (15:09 -0500)]
librbd: collapse API mirror primary/non-primary namespaces to single type

Eventually we will want to be able to chain multiple peers together.
In order to avoid special logic for syncing non-primary snapshots from
a peer, just treat all mirror snapshots the same.

This also clears some confusion for how to handle demoted snapshots
since otherwise it would involve creating a primary-snapshot on the
non-primary image before the image was fully synced.

Finally, the copied flag was renamed to completed since we should
eventually only set this to true on primary snapshots after the
image state has been created.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
30 hours agoceph-volume: silence 'ceph-bluestore-tool' failures 33371/head
Sébastien Han [Wed, 19 Feb 2020 15:03:02 +0000 (16:03 +0100)]
ceph-volume: silence 'ceph-bluestore-tool' failures

If 'ceph-bluestore-tool' fails on a device, the json output of the list
command will be messed up. Ignoring stderr of that command fixes this.

Signed-off-by: Sébastien Han <seb@redhat.com>
30 hours agoMerge PR #33387 into master
Sage Weil [Wed, 19 Feb 2020 14:54:29 +0000 (08:54 -0600)]
Merge PR #33387 into master

* refs/pull/33387/head:
spec: ceph-mgr: add missing python3-pyyaml

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Dimitri Savineau <dsavinea@redhat.com>
Reviewed-by: Sage Weil <sage@redhat.com>
31 hours agoqa/tasks/cephadm: deploy node-exporter 33123/head
Sage Weil [Fri, 14 Feb 2020 19:13:10 +0000 (13:13 -0600)]
qa/tasks/cephadm: deploy node-exporter

Signed-off-by: Sage Weil <sage@redhat.com>
31 hours agomgr/orch: deploy node-exporter
Patrick Seidensal [Wed, 5 Feb 2020 15:17:52 +0000 (16:17 +0100)]
mgr/orch: deploy node-exporter

Signed-off-by: Patrick Seidensal <pseidensal@suse.com>
32 hours agocephadm: remove distutils dependency 33374/head
Sage Weil [Tue, 18 Feb 2020 14:44:36 +0000 (08:44 -0600)]
cephadm: remove distutils dependency

Signed-off-by: Sage Weil <sage@redhat.com>
32 hours agomgr/cephadm: add prepare-host command
Sage Weil [Mon, 17 Feb 2020 16:55:45 +0000 (10:55 -0600)]
mgr/cephadm: add prepare-host command

Signed-off-by: Sage Weil <sage@redhat.com>
32 hours agocephadm: prepare-host during bootstrap
Sage Weil [Mon, 17 Feb 2020 16:46:20 +0000 (10:46 -0600)]
cephadm: prepare-host during bootstrap

Signed-off-by: Sage Weil <sage@redhat.com>
32 hours agocephadm: add prepare-host command
Sage Weil [Mon, 17 Feb 2020 16:44:44 +0000 (10:44 -0600)]
cephadm: add prepare-host command

Signed-off-by: Sage Weil <sage@redhat.com>
32 hours agocephadm: include docker.io in image name
Sage Weil [Mon, 17 Feb 2020 16:41:00 +0000 (10:41 -0600)]
cephadm: include docker.io in image name

This removes a configuration dependency on the container environment.

Signed-off-by: Sage Weil <sage@redhat.com>
32 hours agoMerge pull request #32895 from tspmelo/wip-sidebar-notification-v2
Laura Paduano [Wed, 19 Feb 2020 12:58:28 +0000 (13:58 +0100)]
Merge pull request #32895 from tspmelo/wip-sidebar-notification-v2

mgr/dashboard: Improve Notification sidebar

Reviewed-by: Lenz Grimmer <lgrimmer@suse.com>
Reviewed-by: Stephan Müller <smueller@suse.com>
Reviewed-by: Laura Paduano <lpaduano@suse.com>
33 hours agoMerge pull request #33410 from capri1989/docs-fix
Sebastian Wagner [Wed, 19 Feb 2020 12:39:43 +0000 (13:39 +0100)]
Merge pull request #33410 from capri1989/docs-fix

doc/mgr/orchestrator.rst: updated current implementation status

Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>
33 hours agoMerge PR #33249 into master
Sage Weil [Wed, 19 Feb 2020 12:33:59 +0000 (06:33 -0600)]
Merge PR #33249 into master

* refs/pull/33249/head:
mgr/cephadm: move cutoff calc inside helper
mgr/orch: clean up service_action, remove_service
mgr/cephadm: move into DaemonCache class
mgr/cephadm: fix tests
mgr/cephadm: implete hacky 'refresh=True' path
mgr/cephadm: raise health alert when scrape fails
mgr/cephadm: persist cached daemon state
mgr/orch: serialize DaemonDescription last_refresh
mgr/cephadm: move _get_daemons() impl into list_daemons
mgr/cephadm: replace remaining _get_daemons() with daemon cache
mgr/cephadm: use daemon map for service|daemon removal
mgr/cephadm: avoid _get_daemons for service|daemon actions
mgr/cephadm: replace daemon_cache with an explicit set of dicts
mgr/cephadm: move DaemonDescription construction into helper

Reviewed-by: Sebastian Wagner <swagner@suse.com>
34 hours agoMerge pull request #33408 from tchaikov/wip-crimson-editor-prop
Kefu Chai [Wed, 19 Feb 2020 11:35:42 +0000 (19:35 +0800)]
Merge pull request #33408 from tchaikov/wip-crimson-editor-prop

crimson: add editor properties header

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
34 hours agodoc/mgr/orchestrator.rst updated current implementation status 33410/head
Kai Wagner [Wed, 19 Feb 2020 11:07:50 +0000 (12:07 +0100)]
doc/mgr/orchestrator.rst updated current implementation status

Signed-off-by: Kai Wagner <kwagner@suse.com>
35 hours agoMerge pull request #33060 from votdev/issue_39252_rgw_beast
Lenz Grimmer [Wed, 19 Feb 2020 09:51:10 +0000 (09:51 +0000)]
Merge pull request #33060 from votdev/issue_39252_rgw_beast

mgr/dashboard: RGW port autodetection does not support "Beast" RGW frontend

Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
36 hours agobuild/ops: make/install osd_support package 32677/head
Joshua Schmid [Wed, 19 Feb 2020 09:33:26 +0000 (10:33 +0100)]
build/ops: make/install osd_support package

Signed-off-by: Joshua Schmid <jschmid@suse.de>
37 hours agomgr/osd_support: move docs to docs/mgr/osd_support.rst
Joshua Schmid [Thu, 30 Jan 2020 16:47:18 +0000 (17:47 +0100)]
mgr/osd_support: move docs to docs/mgr/osd_support.rst

Signed-off-by: Joshua Schmid <jschmid@suse.de>
37 hours agomgr/osd_support: new module for osd utility calls
Joshua Schmid [Tue, 14 Jan 2020 17:54:10 +0000 (18:54 +0100)]
mgr/osd_support: new module for osd utility calls

it lets you start/stop and monitor osd draining processes

Signed-off-by: Joshua Schmid <jschmid@suse.de>
38 hours agomgr/dashboard: coverage venv python version same as mgr 33407/head
Alfonso Martínez [Wed, 19 Feb 2020 07:39:55 +0000 (08:39 +0100)]
mgr/dashboard: coverage venv python version same as mgr

As https://github.com/ceph/ceph/pull/31525 is merged,
coverage dep. in run-backend-api-tests.sh has to be installed in venv
with the same python version as ceph-mgr.

Signed-off-by: Alfonso Martínez <almartin@redhat.com>
41 hours agoMerge PR #33395 into master
Sage Weil [Wed, 19 Feb 2020 04:10:50 +0000 (22:10 -0600)]
Merge PR #33395 into master

* refs/pull/33395/head:
cephadm: error out on filestore OSDs

Reviewed-by: Michael Fritch <mfritch@suse.com>
42 hours agoMerge pull request #33396 from liewegas/fix-cephadm-upgrade-test
Kefu Chai [Wed, 19 Feb 2020 03:08:16 +0000 (11:08 +0800)]
Merge pull request #33396 from liewegas/fix-cephadm-upgrade-test

qa/suites/rados/cephadm/upgrade: fix initial version

Reviewed-by: Kefu Chai <kchai@redhat.com>
42 hours agoMerge PR #32949 into master
Sage Weil [Wed, 19 Feb 2020 02:56:54 +0000 (20:56 -0600)]
Merge PR #32949 into master

* refs/pull/32949/head:
cephadm: separate out require files in config-json
cephadm: add alertmanager deployment feature

Reviewed-by: Patrick Seidensal <pseidensal@suse.com>
43 hours agospec: ceph-mgr: add missing python3-pyyaml 33387/head
Sebastian Wagner [Tue, 18 Feb 2020 14:30:25 +0000 (15:30 +0100)]
spec: ceph-mgr: add missing python3-pyyaml

Remove stray req from the -k8events package, since the base ceph-mgr now
requires it.

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
44 hours agoMerge PR #33370 into master
Sage Weil [Wed, 19 Feb 2020 01:44:01 +0000 (19:44 -0600)]
Merge PR #33370 into master

* refs/pull/33370/head:
mgr/orchestrator_cli: sort host list

Reviewed-by: Kiefer Chang <kiefer.chang@suse.com>
45 hours agomgr/pg_autoscaler: fix division by zero 33402/head
Sage Weil [Tue, 18 Feb 2020 23:57:00 +0000 (17:57 -0600)]
mgr/pg_autoscaler: fix division by zero

Fixes: https://tracker.ceph.com/issues/44186
Signed-off-by: Sage Weil <sage@redhat.com>
47 hours agomgr/progress: 'progress clear' command should clear events in 'ceph -s' 33400/head
Sage Weil [Tue, 18 Feb 2020 22:09:42 +0000 (16:09 -0600)]
mgr/progress: 'progress clear' command should clear events in 'ceph -s'

Signed-off-by: Sage Weil <sage@redhat.com>
47 hours agomgr/cephadm: move cutoff calc inside helper 33249/head
Sage Weil [Mon, 17 Feb 2020 13:54:06 +0000 (07:54 -0600)]
mgr/cephadm: move cutoff calc inside helper

Signed-off-by: Sage Weil <sage@redhat.com>
47 hours agomgr/orch: clean up service_action, remove_service
Sage Weil [Mon, 17 Feb 2020 13:50:50 +0000 (07:50 -0600)]
mgr/orch: clean up service_action, remove_service

Standardize on service_name argument that looks like 'mgr', 'mds.fsname',
'mds', or some other daemon name prefix.  Avoid the ambiguously-named
service_type+service_name combinations.

Signed-off-by: Sage Weil <sage@redhat.com>
47 hours agomgr/cephadm: move into DaemonCache class
Sage Weil [Mon, 17 Feb 2020 13:37:58 +0000 (07:37 -0600)]
mgr/cephadm: move into DaemonCache class

Signed-off-by: Sage Weil <sage@redhat.com>
47 hours agomgr/cephadm: fix tests
Sage Weil [Fri, 14 Feb 2020 18:50:32 +0000 (12:50 -0600)]
mgr/cephadm: fix tests

Signed-off-by: Sage Weil <sage@redhat.com>
47 hours agomgr/cephadm: implete hacky 'refresh=True' path
Sage Weil [Fri, 14 Feb 2020 18:50:23 +0000 (12:50 -0600)]
mgr/cephadm: implete hacky 'refresh=True' path

Signed-off-by: Sage Weil <sage@redhat.com>
47 hours agomgr/cephadm: raise health alert when scrape fails
Sage Weil [Fri, 14 Feb 2020 18:20:27 +0000 (12:20 -0600)]
mgr/cephadm: raise health alert when scrape fails

Signed-off-by: Sage Weil <sage@redhat.com>
47 hours agomgr/cephadm: persist cached daemon state
Sage Weil [Fri, 14 Feb 2020 18:07:13 +0000 (12:07 -0600)]
mgr/cephadm: persist cached daemon state

- load cached state on startup
- persist state after a scrape only
- scrape everything after startup

Note that we modify our in-memory cache when we add or remove a service
and then immediately trigger a scrape, but we do not invalidate the
persisted state, since it's simpler (and presumably a good idea) to
simply re-scrape everything after a mgr restart.

Signed-off-by: Sage Weil <sage@redhat.com>
47 hours agomgr/orch: serialize DaemonDescription last_refresh
Sage Weil [Fri, 14 Feb 2020 18:00:57 +0000 (12:00 -0600)]
mgr/orch: serialize DaemonDescription last_refresh

Signed-off-by: Sage Weil <sage@redhat.com>
47 hours agomgr/cephadm: move _get_daemons() impl into list_daemons
Sage Weil [Fri, 14 Feb 2020 17:13:03 +0000 (11:13 -0600)]
mgr/cephadm: move _get_daemons() impl into list_daemons

Signed-off-by: Sage Weil <sage@redhat.com>
47 hours agomgr/cephadm: replace remaining _get_daemons() with daemon cache
Sage Weil [Fri, 14 Feb 2020 17:08:18 +0000 (11:08 -0600)]
mgr/cephadm: replace remaining _get_daemons() with daemon cache

New function _get_daemons_by_type() returns immediately with a result
from the cache.

Remove the with_daemons decorator.

Push daemon list fetch into _add_new_daemon.

Signed-off-by: Sage Weil <sage@redhat.com>
2 days agomgr/cephadm: use daemon map for service|daemon removal
Sage Weil [Wed, 12 Feb 2020 22:44:56 +0000 (16:44 -0600)]
mgr/cephadm: use daemon map for service|daemon removal

Signed-off-by: Sage Weil <sage@redhat.com>
2 days agomgr/cephadm: avoid _get_daemons for service|daemon actions
Sage Weil [Wed, 12 Feb 2020 22:39:53 +0000 (16:39 -0600)]
mgr/cephadm: avoid _get_daemons for service|daemon actions

Signed-off-by: Sage Weil <sage@redhat.com>
2 days agomgr/cephadm: replace daemon_cache with an explicit set of dicts
Sage Weil [Fri, 14 Feb 2020 16:49:37 +0000 (10:49 -0600)]
mgr/cephadm: replace daemon_cache with an explicit set of dicts

- Cache DaemonDescription explicitly
- explicit timestamp for the host
- serve() scrapes inventory based on that timestamp

For the moment, persistence is broken, and --refresh is broken.

Signed-off-by: Sage Weil <sage@redhat.com>
2 days agomgr/cephadm: move DaemonDescription construction into helper
Sage Weil [Wed, 12 Feb 2020 22:30:41 +0000 (16:30 -0600)]
mgr/cephadm: move DaemonDescription construction into helper

Signed-off-by: Sage Weil <sage@redhat.com>
2 days agoMerge PR #33390 into master
Patrick Donnelly [Tue, 18 Feb 2020 20:04:52 +0000 (12:04 -0800)]
Merge PR #33390 into master

* refs/pull/33390/head:
mds: remove dead get_commands code

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Greg Farnum <gfarnum@redhat.com>