]> git.apps.os.sepia.ceph.com Git - ceph.git/log
ceph.git
5 years agolibrbd: permit mirroring to be enabled upon image creation
Jason Dillaman [Mon, 20 Apr 2020 21:04:02 +0000 (17:04 -0400)]
librbd: permit mirroring to be enabled upon image creation

The librbd create API's ImageOptions parameter now supports passing
the mirror image mode.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 37f6fa8fe95729af845100e28df8b8e0668c2270)

5 years agolibrbd: mirror enable state machine might need to open image
Jason Dillaman [Mon, 20 Apr 2020 22:16:40 +0000 (18:16 -0400)]
librbd: mirror enable state machine might need to open image

If attempting to create a snapshot-based mirroring primary snapshot,
the image needs to first be opened. If we weren't supplied an image,
open the image, create the snapshot, and close the image again.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 931c9f0fd9332136466a8d81c9708b86f597f4be)

5 years agolibrbd: pass bit-flags to image::CreateRequest
Jason Dillaman [Mon, 20 Apr 2020 19:23:53 +0000 (15:23 -0400)]
librbd: pass bit-flags to image::CreateRequest

The current boolean for skip enable mirror will be able to be changed to
a tri-state to force enable mirror (in addition to the current auto-enable
if in pool-mode).

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit b0c3ca63c2dff11c823d0d4456a2a07f1594fb9f)

5 years agoqa/workunits/rbd: fix peer_add helper
Mykola Golub [Tue, 28 Apr 2020 16:04:43 +0000 (17:04 +0100)]
qa/workunits/rbd: fix peer_add helper

We need to temporary disable "exit on error" mode so it does not
abort when `rbd mirror pool peer add` returns "already exists"
error code.

Signed-off-by: Mykola Golub <mgolub@suse.com>
(cherry picked from commit 1ad816ae0c2d4f0db3d26e0ae993947154bab6c8)

5 years agoqa/workunits/rbd: retry the addition of a mirror pool peer
Mykola Golub [Sat, 25 Apr 2020 07:36:25 +0000 (08:36 +0100)]
qa/workunits/rbd: retry the addition of a mirror pool peer

fb4311f5 has fixed this for setup, but "remove mirroring pool"
test needs fixing too.

Fixes: https://tracker.ceph.com/issues/44938
Signed-off-by: Mykola Golub <mgolub@suse.com>
(cherry picked from commit 7eced158a9a3c47cc408b35219b4428e97e018fb)

5 years agorbd-mirror: track in-flight start/stop/restart in instance replayer
Jason Dillaman [Fri, 17 Apr 2020 15:17:05 +0000 (11:17 -0400)]
rbd-mirror: track in-flight start/stop/restart in instance replayer

The shut down waits for in-flight ops to complete but the
start/stop/restart operations were previously not tracked. This
could cause a potential race and crash between an image replayer
operation and the instance replayer shutting down.

Fixes: https://tracker.ceph.com/issues/45072
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 31140a940ea1909c4b5d68ef4593cb582a527354)

5 years agorbd-mirror: switch to common C_TrackedOp context class
Jason Dillaman [Fri, 17 Apr 2020 14:20:44 +0000 (10:20 -0400)]
rbd-mirror: switch to common C_TrackedOp context class

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 64f8d9c30c53077f48107e83ba836897aa99e609)

5 years agocommon: add helper C_TrackerOp context class
Jason Dillaman [Fri, 17 Apr 2020 14:02:10 +0000 (10:02 -0400)]
common: add helper C_TrackerOp context class

This wraps the functionality of starting and finishing a tracked op
into the standard context interface.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 4bd9d1501f3832206ef12155cec3f008e3160822)

5 years agorbd-mirror: utilize structured binding in 'for' loop
Jason Dillaman [Tue, 14 Apr 2020 22:05:59 +0000 (18:05 -0400)]
rbd-mirror: utilize structured binding in 'for' loop

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 847f178c4bfb50ab1c1b21222fcb47759119aa43)

5 years agorbd-mirror: unlink from remote snapshot if required
Jason Dillaman [Wed, 8 Apr 2020 02:12:19 +0000 (22:12 -0400)]
rbd-mirror: unlink from remote snapshot if required

If a previous remote snapshot was synced but the unlink failed,
ensure we retry the unlink so that the remote can cleanup the unused
snapshot.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit cb7b91dc02b64cb15f8d21e830a698bd4173b35a)

5 years agorbd-mirror: prune unnecessary non-primary mirror snapshots
Jason Dillaman [Mon, 6 Apr 2020 20:46:52 +0000 (16:46 -0400)]
rbd-mirror: prune unnecessary non-primary mirror snapshots

Once a non-primary snapshot is no longer required for syncing, delete it
from the image.

Fixes: https://tracker.ceph.com/issues/44105
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 281af0de86b17244a2c64af80db0bf84de6d9819)

5 years agorbd-mirror: propagate full snap-seq mapping in non-primary snapshots
Jason Dillaman [Mon, 6 Apr 2020 20:21:35 +0000 (16:21 -0400)]
rbd-mirror: propagate full snap-seq mapping in non-primary snapshots

Previously only newly created user snapshots were included in the
non-primary snapshot snap-seq mapping table. However, we need to
retain a full history of the mapping table if we want to be able to
prune non-primary snapshots.

Failovers are a special case since we won't have a valid snap seq mapping
so it will need to be rebuilt. Luckily, both sides should be read-only
in the previous state so we can use the snapshot names to find matches.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit cb8187c0dd22ed9098ab51e47777aee5aec471a7)

5 years agorbd-mirror: ignore non-primary read-only state for remote images
Jason Dillaman [Tue, 7 Apr 2020 23:12:03 +0000 (19:12 -0400)]
rbd-mirror: ignore non-primary read-only state for remote images

snapshot-based mirroring needs to be able to potentially delete a
demotion snapshot during the unlink process. Previously, these
snapshots have been left while the read-only error was ignored.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 6bf2132cf041f6ab16c86e46b5acd4cf2ce12060)

5 years agorbd: fix missing space when listing non-primary mirror snapshots
Jason Dillaman [Tue, 7 Apr 2020 23:11:42 +0000 (19:11 -0400)]
rbd: fix missing space when listing non-primary mirror snapshots

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 0ca7817eceafd7756eee967e7d740afeedd517e7)

5 years agolibrbd: fixed race condition on demotion of snapshot-based mirrored image
Jason Dillaman [Thu, 9 Apr 2020 03:06:05 +0000 (23:06 -0400)]
librbd: fixed race condition on demotion of snapshot-based mirrored image

A pending refresh could occur after setting the non-primary feature flag but
before the creation of the demotion snapshot. This would prevent the snapshot
from being created and would leave the image in a half-primary state.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit a3acdbd069b5c5bd62e528cbf2fbc33fe6f23d92)

5 years agolibrbd: store mirror peer uuids in non-primary demoted snapshots
Jason Dillaman [Wed, 8 Apr 2020 23:41:23 +0000 (19:41 -0400)]
librbd: store mirror peer uuids in non-primary demoted snapshots

This will allow a remote rbd-mirror process to have a snapshot to use for
delta sync operations during failover.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 0102ce88701dd52208520b8c0ab2e505548ed3e1)

5 years agolibrbd: additional debug logs for mirror snapshot unlink peer
Jason Dillaman [Wed, 8 Apr 2020 21:50:59 +0000 (17:50 -0400)]
librbd: additional debug logs for mirror snapshot unlink peer

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit eed00eb17969791107ddb8df0e909790c17bd573)

5 years agotest/rbd-mirror: fix gmock warnings during snapshot-based replayer tests
Jason Dillaman [Mon, 6 Apr 2020 18:37:04 +0000 (14:37 -0400)]
test/rbd-mirror: fix gmock warnings during snapshot-based replayer tests

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 6a342bb5e01fea205088592128f62b55261408bf)

5 years agorbd-mirror: fixed race condition with snapshot sync and shutdown
Jason Dillaman [Tue, 7 Apr 2020 16:13:46 +0000 (12:13 -0400)]
rbd-mirror: fixed race condition with snapshot sync and shutdown

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 8a18a7fc71944c55e7a5fa47e55572546b4088db)

5 years agorbd-mirror: add snapshot-based replay performance metrics
Jason Dillaman [Fri, 3 Apr 2020 17:11:27 +0000 (13:11 -0400)]
rbd-mirror: add snapshot-based replay performance metrics

The mirror image status for replaying snapshot-based images now includes
bytes per second and per snapshot, in addition to an estimated number of
seconds until the image is fully synced.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 03aa081435abf12429fbe98b7ff1095a4b37d830)

5 years agorbd-mirror: switch to json_spirit formatter for snapshot image status
Jason Dillaman [Thu, 2 Apr 2020 17:43:09 +0000 (13:43 -0400)]
rbd-mirror: switch to json_spirit formatter for snapshot image status

This will make it cleaner and easier to add additional data fields
to the existing JSON replaying status.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit de445810e97b345f647ffc5f5792dfc2ad589f06)

5 years agorbd-mirror: add journal-based replay performance metrics
Jason Dillaman [Wed, 1 Apr 2020 23:24:15 +0000 (19:24 -0400)]
rbd-mirror: add journal-based replay performance metrics

The mirror image status for replaying journal-based images now includes
bytes and entries per second in addition to an estimated number of seconds
until the image is fully synced.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit c0dc96bf137dac6942e14800e173bfb9bda233eb)

5 years agorbd-mirror: switch to json_spirit formatter for journal image status
Jason Dillaman [Wed, 1 Apr 2020 19:26:39 +0000 (15:26 -0400)]
rbd-mirror: switch to json_spirit formatter for journal image status

The free-form journal replay status description is now JSON-encoded. The
"master"/"mirror" designators have been changed to "primary"/"non_primary"
to better align with RBD terminology.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit e04422f2c60a9ce891da496b71a568264e39233e)

5 years agorbd-mirror: periodically poll image replayer status
Jason Dillaman [Thu, 2 Apr 2020 18:50:37 +0000 (14:50 -0400)]
rbd-mirror: periodically poll image replayer status

When metrics are incorporated, there might not be a forced status update
if no new data is available to replicate. However, we will want the metrics
to decrease over time.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 133cce8af50653d147d7147fc4bc7269ad10b5e1)

5 years agolibrbd: pass bytes-read statistics from deep-copy through handler
Jason Dillaman [Fri, 3 Apr 2020 19:24:14 +0000 (15:24 -0400)]
librbd: pass bytes-read statistics from deep-copy through handler

These simple stats will be utilized by rbd-mirror to compute throughput
metrics for snapshot-based mirroring.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 0b6cbdeb9deec8184a43f1b7533c8316b22b9f5d)

5 years agolibrbd: pass specialized handler interface to deep-copy
Jason Dillaman [Fri, 3 Apr 2020 15:12:48 +0000 (11:12 -0400)]
librbd: pass specialized handler interface to deep-copy

The initial version mimics the existing ProgressContext callback
interface. Later commits will add additional deep-copy unique methods.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit c7d6e762d19284a05b88daba6c27cbbdbdb210e9)

5 years agoMerge pull request #34760 from callithea/wip-45281-octopus
Lenz Grimmer [Tue, 28 Apr 2020 07:23:07 +0000 (09:23 +0200)]
Merge pull request #34760 from callithea/wip-45281-octopus

Reviewed-by: Stephan Müller <smueller@suse.com>
Reviewed-by: Tiago Melo <tmelo@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>
5 years agoMerge pull request #34467 from rhcs-dashboard/wip-44992-octopus
Lenz Grimmer [Tue, 28 Apr 2020 07:18:42 +0000 (09:18 +0200)]
Merge pull request #34467 from rhcs-dashboard/wip-44992-octopus

Reviewed-by: Laura Paduano <lpaduano@suse.com>
5 years agoMerge pull request #34708 from rhcs-dashboard/wip-45233-octopus
Lenz Grimmer [Tue, 28 Apr 2020 07:12:52 +0000 (09:12 +0200)]
Merge pull request #34708 from rhcs-dashboard/wip-45233-octopus

Reviewed-by: Brad Hubbard <bhubbard@redhat.com>
Reviewed-by: Laura Paduano <lpaduano@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>
5 years agoMerge pull request #34631 from shyukri/wip-44834-octopus
Laura Paduano [Mon, 27 Apr 2020 11:01:43 +0000 (13:01 +0200)]
Merge pull request #34631 from shyukri/wip-44834-octopus

octopus: mgr/dashboard: 'Prometheus / All Alerts' page shows progress bar

Reviewed-by: Sebastian Krah <skrah@suse.com>
Reviewed-by: Tatjana Dehler <tdehler@suse.com>
Reviewed-by: Laura Paduano <lpaduano@suse.com>
5 years agoMerge pull request #34547 from tspmelo/wip-45083-octopus
Laura Paduano [Mon, 27 Apr 2020 10:58:44 +0000 (12:58 +0200)]
Merge pull request #34547 from tspmelo/wip-45083-octopus

octopus: mgr/dashboard: Fix iSCSI's username and password validation

Reviewed-by: Ricardo Marques <rimarques@suse.com>
Reviewed-by: Stephan Müller <smueller@suse.com>
Reviewed-by: Laura Paduano <lpaduano@suse.com>
5 years agoMerge pull request #34498 from votdev/wip-45020-octopus
Laura Paduano [Mon, 27 Apr 2020 10:57:20 +0000 (12:57 +0200)]
Merge pull request #34498 from votdev/wip-45020-octopus

octopus: mgr/dashboard: use FQDN for failover redirection

Reviewed-by: Lenz Grimmer <lgrimmer@suse.com>
Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Stephan Müller <smueller@suse.com>
5 years agoMerge pull request #34211 from s0nea/wip-44712-octopus
Laura Paduano [Mon, 27 Apr 2020 10:55:28 +0000 (12:55 +0200)]
Merge pull request #34211 from s0nea/wip-44712-octopus

octopus: mgr/dashboard: add crush rule test suite

Reviewed-by: Stephan Müller <smueller@suse.com>
Reviewed-by: Laura Paduano <lpaduano@suse.com>
5 years agomgr/dashboard: Fix ServiceDetails and PoolDetails unit tests 34760/head
Tiago Melo [Mon, 30 Mar 2020 14:01:27 +0000 (14:01 +0000)]
mgr/dashboard: Fix ServiceDetails and PoolDetails unit tests

While testing the upgrade to Angular 9,
these 2 unit tests were consistently failling.

Fixes: https://tracker.ceph.com/issues/42929
Signed-off-by: Tiago Melo <tmelo@suse.com>
(cherry picked from commit d7e7d2b09e463a978d2f881b6b1978fd2606651e)

5 years agoMerge pull request #34750 from yuriw/wip-yuriw-distro-octupus
Yuri Weinstein [Fri, 24 Apr 2020 20:19:49 +0000 (13:19 -0700)]
Merge pull request #34750 from yuriw/wip-yuriw-distro-octupus

qa/tests: clean up distro and point to centos 8.1

5 years agoqa/tests: clean up distro and point to centos 8.1 34750/head
Yuri Weinstein [Fri, 24 Apr 2020 20:02:17 +0000 (13:02 -0700)]
qa/tests: clean up distro and point to centos 8.1

Signed-off-by: Yuri Weinstein <yweinste@redhat.com>
5 years agoMerge pull request #34687 from sebastian-philipp/octopus-backport-34558-34216-34415...
Sebastian Wagner [Fri, 24 Apr 2020 16:02:24 +0000 (18:02 +0200)]
Merge pull request #34687 from sebastian-philipp/octopus-backport-34558-34216-34415-34555-34556-34490-34529-34346-32951-34584-34612-34565-34568-34621

octopus: cephadm: Batch backport April (2)

Reviewed-by: Joshua Schmid <jschmid@suse.de>
Reviewed-by: Michael Fritch <mfritch@suse.com>
5 years agoMerge pull request #34548 from tspmelo/wip-45084-octopus
Lenz Grimmer [Fri, 24 Apr 2020 13:56:21 +0000 (15:56 +0200)]
Merge pull request #34548 from tspmelo/wip-45084-octopus

Reviewed-by: Laura Paduano <lpaduano@suse.com>
Reviewed-by: Ricardo Marques <rimarques@suse.com>
5 years agoMerge pull request #34625 from shyukri/wip-45051-octopus
Lenz Grimmer [Fri, 24 Apr 2020 13:56:00 +0000 (15:56 +0200)]
Merge pull request #34625 from shyukri/wip-45051-octopus

Reviewed-by: Laura Paduano <lpaduano@suse.com>
Reviewed-by: Stephan Müller <smueller@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>
5 years agoMerge pull request #34630 from shyukri/wip-44837-octopus
Lenz Grimmer [Fri, 24 Apr 2020 13:55:33 +0000 (15:55 +0200)]
Merge pull request #34630 from shyukri/wip-44837-octopus

Reviewed-by: Laura Paduano <lpaduano@suse.com>
Reviewed-by: Stephan Müller <smueller@suse.com>
5 years agoMerge pull request #34686 from sebastian-philipp/octopus-backport-34443
Sebastian Wagner [Fri, 24 Apr 2020 09:36:24 +0000 (11:36 +0200)]
Merge pull request #34686 from sebastian-philipp/octopus-backport-34443

octopus: cephadm: Update images used

Reviewed-by: Michael Fritch <mfritch@suse.com>
5 years agoMerge pull request #34638 from smithfarm/wip-45036-octopus
Yuri Weinstein [Thu, 23 Apr 2020 20:53:29 +0000 (13:53 -0700)]
Merge pull request #34638 from smithfarm/wip-45036-octopus

octopus: rbd: ignore tx-only mirror peers when adding new peers

Reviewed-by: Mykola Golub <mgolub@mirantis.com>
Reviewed-by: Jason Dillaman <dillaman@redhat.com>
5 years agomgr/dashboard: fix tasks.mgr.dashboard.test_rgw.RgwBucketTest.test_all 34708/head
Alfonso Martínez [Tue, 21 Apr 2020 13:27:06 +0000 (15:27 +0200)]
mgr/dashboard: fix tasks.mgr.dashboard.test_rgw.RgwBucketTest.test_all

Increased the time for getting 2nd TOTP.

Fixes: https://tracker.ceph.com/issues/44405
Signed-off-by: Alfonso Martínez <almartin@redhat.com>
(cherry picked from commit ee2b2d19781e531770350ad2abe53ce876cebe3b)

5 years agoMerge pull request #34526 from smithfarm/wip-45034-octopus
Yuri Weinstein [Wed, 22 Apr 2020 18:17:52 +0000 (11:17 -0700)]
Merge pull request #34526 from smithfarm/wip-45034-octopus

octopus: spec: address some warnings raised by RPM 4.15.1

Reviewed-by: Kefu Chai <kchai@redhat.com>
5 years agoMerge pull request #34519 from smithfarm/wip-45059-octopus
Yuri Weinstein [Wed, 22 Apr 2020 18:17:16 +0000 (11:17 -0700)]
Merge pull request #34519 from smithfarm/wip-45059-octopus

octopus: qa/suites/rados/rest: don't pass empty dict as data arg

Reviewed-by: Brad Hubbard <bhubbard@redhat.com>
5 years agoMerge pull request #34395 from votdev/wip-44918-octopus
Yuri Weinstein [Wed, 22 Apr 2020 18:16:41 +0000 (11:16 -0700)]
Merge pull request #34395 from votdev/wip-44918-octopus

octopus: monitoring: alert for prediction of disk and pool fill up broken

Reviewed-by: Patrick Seidensal <pnawracay@suse.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
5 years agoMerge pull request #34648 from shyukri/wip-44892-octopus
Lenz Grimmer [Wed, 22 Apr 2020 15:16:09 +0000 (17:16 +0200)]
Merge pull request #34648 from shyukri/wip-44892-octopus

Reviewed-by: Kiefer Chang <kiefer.chang@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>
5 years agoMerge pull request #34418 from votdev/wip-44955-octopus
Lenz Grimmer [Wed, 22 Apr 2020 13:44:57 +0000 (15:44 +0200)]
Merge pull request #34418 from votdev/wip-44955-octopus

Reviewed-by: Laura Paduano <lpaduano@suse.com>
Reviewed-by: Tiago Melo <tmelo@suse.com>
5 years agotest_cephadm.sh: remove unmaintained test_cephadm.sh 34687/head
Michael Fritch [Fri, 17 Apr 2020 20:26:29 +0000 (14:26 -0600)]
test_cephadm.sh: remove unmaintained test_cephadm.sh

the script has become unmaintained and duplicates logic found in
qa/workunits/cephadm/test_cephadm.sh

Signed-off-by: Michael Fritch <mfritch@suse.com>
(cherry picked from commit 879e6850992685115d0bd0a38b4391722b4e646f)

5 years agoqa/workunits/cephadm/test_cephadm.sh: add CLEANUP var
Michael Fritch [Fri, 17 Apr 2020 20:17:02 +0000 (14:17 -0600)]
qa/workunits/cephadm/test_cephadm.sh: add CLEANUP var

preserve the current test state during EXIT:
`CLEANUP=false ../qa/workunits/cephadm/test_cephadm.sh`

Signed-off-by: Michael Fritch <mfritch@suse.com>
(cherry picked from commit 6b211447230d3b6d6d8d49b780750cf6aeeace6d)

5 years agomgr/orch: upgrade: better input validation
Sebastian Wagner [Wed, 15 Apr 2020 13:23:05 +0000 (15:23 +0200)]
mgr/orch: upgrade: better input validation

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
(cherry picked from commit 67a29846f80fe8903f2dbce5dca0f5a0134d55b8)

5 years agomgr/orch: cleanup python imports
Sebastian Wagner [Wed, 15 Apr 2020 12:57:54 +0000 (14:57 +0200)]
mgr/orch: cleanup python imports

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
(cherry picked from commit 62ed783dc243e076e00be93ea8ab2dd1fb262d58)

5 years agocephadm: bail out early during adopt if data dir not present
Tim Serong [Thu, 16 Apr 2020 05:42:08 +0000 (15:42 +1000)]
cephadm: bail out early during adopt if data dir not present

In case someone tries to run this again on an already adopted daemon...

Signed-off-by: Tim Serong <tserong@suse.com>
(cherry picked from commit bed711d3d1256758bdf7428339316bcca6f9dae0)

5 years agocephadm: add --force-start option to adopt
Tim Serong [Thu, 16 Apr 2020 05:32:02 +0000 (15:32 +1000)]
cephadm: add --force-start option to adopt

Current behaviour is to only start a newly adopted ceph daemon if it was
already running before the adopt.  Adding a --force-start option allows
the adopt command to start newly adopted daemons that weren't originally
running, to save the user having to manually invoke `systemctl start
ceph-$FSID@$DAEMMON.$ID`.

Signed-off-by: Tim Serong <tserong@suse.com>
(cherry picked from commit 225b25512916ec4a9577ca7f2ab39faaf122827f)

5 years agocephadm: don't throw when disabling ceph-volume simple unit
Tim Serong [Thu, 16 Apr 2020 05:09:10 +0000 (15:09 +1000)]
cephadm: don't throw when disabling ceph-volume simple unit

When adopting OSDs, if a ceph-volume simple service is already disabled
(or otherwise missing) the previous implementation would raise an error,
thus killing the adopt.

Signed-off-by: Tim Serong <tserong@suse.com>
(cherry picked from commit 6834ad983762b7cbf950c6a6e71aa44ec6657c32)

5 years agocephadm: handle adopting offline OSDSs
Tim Serong [Wed, 15 Apr 2020 09:26:19 +0000 (19:26 +1000)]
cephadm: handle adopting offline OSDSs

The current adopt behavior expects OSDs to be online, in order to read
/var/lib/ceph/osd/ceph-$ID/fsid.  To handle the case where OSDs
are offline, this change first checks to see if that file is present,
and if not, falls back to calling `ceph-volume lvm list` to see if
there's a matching OSD there, and if that doesn't work, it checks
/etc/ceph/osd/*.json to see if there's a matching old-style simple
OSD present.

For LVM OSDs, the only thing we need is the ODS's fsid; the remainer
of the adopt procedure "just works", as the various other files
in /var/lib/ceph/$FSID/osd.$ID are created by magic anyway when the
OSD is activated, so it doesn't matter if they're not present at
adoption time.

For simple (ceph-disk created) OSDs, we actually need all the files under
/var/lib/ceph/osd/ceph-$ID/ to be moved to /var/lib/ceph/$FSID/osd.$ID
so if a simple OSD is found, it's mounted first, so the existing
move_files() a bit further down around line 3200 continues to work.

Fixes: https://tracker.ceph.com/issues/45095
Signed-off-by: Tim Serong <tserong@suse.com>
(cherry picked from commit 9169119bd3c0ac976871e7f3321d9d7a53335c82)

5 years agoAdd troubleshooting guidance for ssh connection failures.
Zac Dover [Fri, 17 Apr 2020 04:12:37 +0000 (14:12 +1000)]
Add troubleshooting guidance for ssh connection failures.

For more thorough information about this commit, see:
https://tracker.ceph.com/issues/44905

Signed-off-by: Zac Dover <zac.dover@gmail.com>
Update doc/cephadm/troubleshooting.rst

Co-Authored-By: Michael Fritch <mfritch@suse.com>
ibid

Signed-off-by: Zac Dover <zac.dover@gmail.com>
Update doc/cephadm/troubleshooting.rst

Co-Authored-By: Sebastian Wagner <sebastian@spawnhost.de>
Update doc/cephadm/troubleshooting.rst

Co-Authored-By: Sebastian Wagner <sebastian@spawnhost.de>
Added mgfritch's suggestion regarding checking to see if the public key
is listed in the authorized_keys file.

Signed-off-by: Zac Dover <zac.dover@gmail.com>
(cherry picked from commit c4765afb81489b1df71d87be32c566ed55d29a28)

5 years agomgr/test_orchestrator: fix error when listing services
Kiefer Chang [Thu, 16 Apr 2020 07:34:51 +0000 (15:34 +0800)]
mgr/test_orchestrator: fix error when listing services

- Adapting attribute change of ServiceSpec.
- Update dummy data.

Fixes: https://tracker.ceph.com/issues/45108
Signed-off-by: Kiefer Chang <kiefer.chang@suse.com>
(cherry picked from commit 223c8d816f7f1d3d5ea737c540bae4553334aabf)

5 years agomgr/orch: Fixes some deserialization errors
Kiefer Chang [Thu, 16 Apr 2020 07:30:24 +0000 (15:30 +0800)]
mgr/orch: Fixes some deserialization errors

Fixes errors when calling `from_json` of these classes:
- InventoryHost: parsing labels
- ServiceDescription: `last_refresh` and `created` fields should be parsed
  to datetime type.

Signed-off-by: Kiefer Chang <kiefer.chang@suse.com>
(cherry picked from commit 45fd5ee275aa94409bbe9d9c3874c9aad82850a8)

5 years agocephadm: add RGW SSL support
Matthew Oliver [Thu, 23 Jan 2020 00:02:24 +0000 (11:02 +1100)]
cephadm: add RGW SSL support

This patch adds SSL support to RGW when using cephadm.
If an SSL certificate is provided, inside the json supplied with:

  cpeh orchestrator rgw create -i rgw.json

Then the SSL cert and/or key will be added to pushed into the mon config-key database
using the key `rgw/cert/<rgw_realm>/<rgw_zone>.[crt|key]`.
Which will then be referenced in the config:

  rgw_frontends = beast port=80 ssl_port=443 ssl_certificate=config://rgw/cert/<rgw_realm>/<rgw_zone>.crt

And if an ssl key is also supplied this becomes something like:

  rgw_frontends = beast port=80 ssl_port=443 ssl_certificate=config://rgw/cert/<rgw_realm>/<rgw_zone>.crt ssl_key=config://rgw/cert/<rgw_realm>/<rgw_zone>.key

Of course you could also just upload the cert and key yourself to
config-key location, and ssl will be enabled as well. But this patch
let's you either supply them via `-i` or as a manual upload step.

Co-Authored-By: Michael Fritch <mfritch@suse.com>
Co-Authored-By: Sebastian Wagner <sebastian@spawnhost.de>
Signed-off-by: Matthew Oliver <moliver@suse.com>
(cherry picked from commit 8204b1d524b155fac56c3937d1aec41a18ab2d52)

5 years agomgr/cephadm: add replacement flag to _create_osd
Joshua Schmid [Thu, 2 Apr 2020 09:33:35 +0000 (11:33 +0200)]
mgr/cephadm: add replacement flag to _create_osd

Signed-off-by: Joshua Schmid <jschmid@suse.de>
(cherry picked from commit e670b1ea83f66d977c919ef956cc4ae0db9cb453)

5 years agomgr/cephadm: use --osd-ids instead of --osd-id
Joshua Schmid [Wed, 1 Apr 2020 14:26:33 +0000 (16:26 +0200)]
mgr/cephadm: use --osd-ids instead of --osd-id

Signed-off-by: Joshua Schmid <jschmid@suse.de>
(cherry picked from commit fa88483f660f2bf79d1c1ca77d7104b3a42ed05d)

5 years agomgr/cephadm: add support for osd_id_claims
Joshua Schmid [Wed, 1 Apr 2020 13:56:08 +0000 (15:56 +0200)]
mgr/cephadm: add support for osd_id_claims

Signed-off-by: Joshua Schmid <jschmid@suse.de>
(cherry picked from commit 5b32c3e51b517332b7ac26b161b7b6b4fae05b85)

5 years agocephadm: warn when no valid version arguments are supplied to add-repo
Maran Hidskes [Mon, 13 Apr 2020 08:20:57 +0000 (10:20 +0200)]
cephadm: warn when no valid version arguments are supplied to add-repo

Currently add-repo will add a broken repository by default, this commit gives a pretty error to instruct the user to use one of the version arguments.

Fixes #45029

Signed-off-by: Maran Hidskes <maran@protonmail.com>
(cherry picked from commit 115b3d7b7f3b4cf4149240fc14e374697a7cd6b1)

5 years agocephadm: add-repo: Don't test for podman
Sebastian Wagner [Thu, 9 Apr 2020 07:50:37 +0000 (09:50 +0200)]
cephadm: add-repo: Don't test for podman

Fixes:

```
root@buster:/cephadm# ./cephadm add-repo
Unable to locate any of ['podman', 'docker']
```

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
(cherry picked from commit 0d22708cc2f858491604b09f87e7af742884cb3c)

5 years agomgr/cephadm: corrected documentation for starting an upgrade
ahaase-c4u [Tue, 14 Apr 2020 18:44:22 +0000 (20:44 +0200)]
mgr/cephadm: corrected documentation for starting an upgrade

Signed-off-by: Andreas Haase <andreas.haase@community4you.de>
(cherry picked from commit bf3d06d180135901903251e6e73b66f4fbac289b)

5 years agomgr/cephadm: Fixed warn_on_stray_hosts setting
Andreas Haase [Tue, 14 Apr 2020 15:09:04 +0000 (17:09 +0200)]
mgr/cephadm: Fixed warn_on_stray_hosts setting

Fixes: https://tracker.ceph.com/issues/45065
Signed-off-by: Andreas Haase <andreas.haase@community4you.de>
(cherry picked from commit b07c2da63f9fb9e70a647a989675bbdf309c14f3)

5 years agomgr/orch: Fix DaemonDescription().daemon_id() for RGWs
Sebastian Wagner [Mon, 6 Apr 2020 10:25:56 +0000 (12:25 +0200)]
mgr/orch: Fix DaemonDescription().daemon_id() for RGWs

Turns out, users put dots into their RGW service names.

Fixes: https://tracker.ceph.com/issues/44934
Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
(cherry picked from commit 39fdb5c67c4d3834c5157e8d6da20933a6f7bfb4)

5 years agomgr/cephadm: enable osd previews
Joshua Schmid [Wed, 1 Apr 2020 08:12:28 +0000 (10:12 +0200)]
mgr/cephadm: enable osd previews

Signed-off-by: Joshua Schmid <jschmid@suse.de>
(cherry picked from commit 462a68ab87446af8299d4f71c5ff22f77dfeea64)

5 years agocephadm: use CephContainer during command `enter`
Michael Fritch [Tue, 14 Apr 2020 21:42:44 +0000 (15:42 -0600)]
cephadm: use CephContainer during command `enter`

instead of the `get_container` helper func

Signed-off-by: Michael Fritch <mfritch@suse.com>
(cherry picked from commit 3c762d0ed92d38a2c47bf273bf17600f101ed1bb)

5 years agocephadm: Update images used 34686/head
Sebastian Wagner [Wed, 22 Apr 2020 13:01:30 +0000 (15:01 +0200)]
cephadm: Update images used

Print warning when using cephadm from master

See also "use quay octopus tip until 15.2 tag is available"
a9b15c7e1a0c14376cd66f166370694294398494.

See also  "update default container images"
1f05f7578794380f969a7e93db07345626b3e4df.

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
(cherry picked from commit de448fafe0fe7d6889f3f75ac37f870cde00cf1c)
Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
5 years agoMerge pull request #34495 from votdev/wip-44953-octopus
Lenz Grimmer [Wed, 22 Apr 2020 12:14:51 +0000 (14:14 +0200)]
Merge pull request #34495 from votdev/wip-44953-octopus

Reviewed-by: Laura Paduano <lpaduano@suse.com>
Reviewed-by: Stephan Müller <smueller@suse.com>
5 years agoMerge pull request #34654 from ceph/wip-yuriw-octopus-client-upgrade-octopus-pacific
Josh Durgin [Mon, 20 Apr 2020 23:05:32 +0000 (16:05 -0700)]
Merge pull request #34654 from ceph/wip-yuriw-octopus-client-upgrade-octopus-pacific

qa/tests: initial checkin for clients upgrades tests

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
5 years agoqa/tests: initial checkin for clients upgrades tests 34654/head
Yuri Weinstein [Mon, 20 Apr 2020 22:53:51 +0000 (15:53 -0700)]
qa/tests: initial checkin for clients upgrades tests

for future release 'pacific'

(Note to self: moved from wrongly pushed to master branch)

Signed-off-by: Yuri Weinstein <yweinste@redhat.com>
5 years agomgr/dashboard: shorten `Container ID` and `Container image ID` in Services page 34648/head
Volker Theile [Tue, 31 Mar 2020 09:13:21 +0000 (11:13 +0200)]
mgr/dashboard: shorten `Container ID` and `Container image ID` in Services page

Fixes: https://tracker.ceph.com/issues/44539
Signed-off-by: Volker Theile <vtheile@suse.com>
(cherry picked from commit ed51f8039abad07fe8c44c233c8e30dc4810fe8b)

5 years agoqa/workunits/rbd: retry the addition of a mirror pool peer 34638/head
Jason Dillaman [Wed, 15 Apr 2020 20:27:07 +0000 (16:27 -0400)]
qa/workunits/rbd: retry the addition of a mirror pool peer

We might race with the remote rbd-mirror daemon creating a
tx-only peer when adding a new peer. Therefore, delete the
tx-only peer and attempt to re-create it.

Fixes: https://tracker.ceph.com/issues/44938
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit fb4311f597a98b6870d7895e6403fb32356bfbe9)

5 years agolibrbd: mirror peer add error message should include site name
Jason Dillaman [Wed, 15 Apr 2020 20:26:09 +0000 (16:26 -0400)]
librbd: mirror peer add error message should include site name

It previously included the pointer to string holding the generated
uuid (neither of which would mean much to an end user).

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit ef68f686be6d5cb9ca221e65deeff311bc397aa4)

5 years agorbd: improved error message when adding a duplicate mirror pool peer
Jason Dillaman [Wed, 15 Apr 2020 19:21:55 +0000 (15:21 -0400)]
rbd: improved error message when adding a duplicate mirror pool peer

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 74cded6b0de90660db46d0dea0cd35ca0bba563e)

5 years agorbd: ignore tx-only mirror peers when adding new peers
Jason Dillaman [Mon, 6 Apr 2020 14:55:28 +0000 (10:55 -0400)]
rbd: ignore tx-only mirror peers when adding new peers

There is a restriction for supporting only a single RX peer but
we should support multiple TX-only peers.

Fixes: https://tracker.ceph.com/issues/44938
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 366ecdb26d90b9dfa483458ce5c28712e47a6341)

5 years agomgr/dashboard: 'Prometheus / All Alerts' page shows progress bar 34631/head
Volker Theile [Mon, 30 Mar 2020 08:17:55 +0000 (10:17 +0200)]
mgr/dashboard: 'Prometheus / All Alerts' page shows progress bar

Fixes: https://tracker.ceph.com/issues/44805
Signed-off-by: Volker Theile <vtheile@suse.com>
(cherry picked from commit 1efe1a76da4671bd2df301a9e7080eda2bb01cd2)

5 years agomgr/dashboard: Improve iSCSI CHAP messages 34630/head
Ricardo Marques [Thu, 5 Mar 2020 11:00:28 +0000 (11:00 +0000)]
mgr/dashboard: Improve iSCSI CHAP messages

iSCSI username and password can also contain numbers

Fixes: https://tracker.ceph.com/issues/44618
Signed-off-by: Ricardo Marques <rimarques@suse.com>
(cherry picked from commit 6ec153f3329c8c6b7c6b6b874083b1bd5d6d3392)

5 years agomgr/dashboard: lint error on plugins/debug.py 34625/head
Volker Theile [Wed, 8 Apr 2020 07:55:05 +0000 (09:55 +0200)]
mgr/dashboard: lint error on plugins/debug.py

Make pylint for Python 3.8 and older versions happy.

Fixes: https://tracker.ceph.com/issues/44589
Signed-off-by: Volker Theile <vtheile@suse.com>
(cherry picked from commit 45076ed13a809c8d0837e72700647f1cecc3298d)

5 years agoMerge pull request #34543 from ceph/wip-yuriw-optopus-p2p-octopus
Yuri Weinstein [Thu, 16 Apr 2020 20:49:15 +0000 (13:49 -0700)]
Merge pull request #34543 from ceph/wip-yuriw-optopus-p2p-octopus

qa/tests: octopus - initial checkin for octopus-p2p tests

5 years agoqa/tests: octopus - initial checkin for octopus-p2p tests 34543/head
yuriw [Mon, 13 Apr 2020 19:15:38 +0000 (12:15 -0700)]
qa/tests: octopus - initial checkin for octopus-p2p tests

Signed-off-by: Yuri Weinstein <yweinste@gmail.com>
5 years agoMerge pull request #34309 from Exotelis/octopus-translations
Yuri Weinstein [Thu, 16 Apr 2020 15:42:28 +0000 (08:42 -0700)]
Merge pull request #34309 from Exotelis/octopus-translations

octopus: mgr/dashboard: Update translations on octopus

Reviewed-by: Tatjana Dehler <tdehler@suse.com>
Reviewed-by: Stephan Müller <smueller@suse.com>
5 years agoMerge pull request #34474 from rhcs-dashboard/wip-44996-octopus
Yuri Weinstein [Thu, 16 Apr 2020 15:40:15 +0000 (08:40 -0700)]
Merge pull request #34474 from rhcs-dashboard/wip-44996-octopus

octopus: rpm: add python3-saml as install dependency

Reviewed-by: Ricardo Marques <rimarques@suse.com>
Reviewed-by: Nathan Cutler <ncutler@suse.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
5 years agoMerge pull request #34350 from sebastian-philipp/octopus-backport-34180
Sebastian Wagner [Thu, 16 Apr 2020 14:52:05 +0000 (16:52 +0200)]
Merge pull request #34350 from sebastian-philipp/octopus-backport-34180

octopus: cephadm: Fix check_ip_port to work with IPv6

Reviewed-by: Michael Fritch <mfritch@suse.com>
5 years agoMerge pull request #34554 from sebastian-philipp/octopus-backport-31885-33954-34357...
Sebastian Wagner [Thu, 16 Apr 2020 14:50:40 +0000 (16:50 +0200)]
Merge pull request #34554 from sebastian-philipp/octopus-backport-31885-33954-34357-34053-34439-34383-34220-34485-34249-34454-34223-34382-34084-34525-34344

octopus: cephadm: Batch backport April (1)

Reviewed-by: Kiefer Chang <kiefer.chang@suse.com>
Reviewed-by: Michael Fritch <mfritch@suse.com>
5 years agoMerge pull request #34299 from liewegas/bp-34150-octopus
Sebastian Wagner [Wed, 15 Apr 2020 15:55:29 +0000 (17:55 +0200)]
Merge pull request #34299 from liewegas/bp-34150-octopus

octopus: cephadm: rm-cluster clean up /etc/ceph

Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>
5 years agoMerge pull request #34387 from smithfarm/wip-44897-octopus
Yuri Weinstein [Wed, 15 Apr 2020 14:41:16 +0000 (07:41 -0700)]
Merge pull request #34387 from smithfarm/wip-44897-octopus

octopus: pybind/rbd: fix no lockers are obtained, ImageNotFound exception will be output

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
Reviewed-by: Mykola Golub <mgolub@mirantis.com>
5 years agomgr/rook: Fix wrong data output when listing devices 34554/head
Juan Miguel Olmo Martínez [Wed, 1 Apr 2020 10:01:35 +0000 (12:01 +0200)]
mgr/rook: Fix wrong data output when listing devices

Information used to build the <device ls> output is not the right one.
Rook provides in the devices configmap the item <cephVolumeData> which contains the information returned by <ceph-volume>.
This is the right source information to build the devices list.

Signed-off-by: Juan Miguel Olmo Martínez <jolmomar@redhat.com>
(cherry picked from commit e3a5fa49005a7d3ca374beca1dd8cd3e416819a9)

5 years agoUpdate index.rst
Tim [Sat, 11 Apr 2020 12:25:51 +0000 (08:25 -0400)]
Update index.rst

clean up "only only"

Signed-off-by: Tim <elatllat@gmail.com>
(cherry picked from commit d7c7ccb4390a91c0c791b924141963305809b2ab)

5 years agomgr/cephadm: add test for "Offline" host status
Sebastian Wagner [Thu, 2 Apr 2020 12:41:51 +0000 (14:41 +0200)]
mgr/cephadm: add test for "Offline" host status

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
(cherry picked from commit 7368d77b31234c59b576303e1949ae85025fafd9)

5 years agomgr/cephadm: Daemons running of unreachable servers are offline
Sebastian Wagner [Thu, 26 Mar 2020 13:30:37 +0000 (14:30 +0100)]
mgr/cephadm: Daemons running of unreachable servers are offline

Fixes: https://tracker.ceph.com/issues/44602
Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
(cherry picked from commit adca3e08e8e9f4a683f7c6cef8e96705cc340188)

5 years agomgr/cephadm: Set SSH connection timeout to 30s
Sebastian Wagner [Thu, 26 Mar 2020 12:51:19 +0000 (13:51 +0100)]
mgr/cephadm: Set SSH connection timeout to 30s

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
(cherry picked from commit 152b4b4b298bb7f2f79159a49a4f75dfa9b739c7)

5 years agomgr/cephadm: No daemon and device refresh if host if out
Sebastian Wagner [Fri, 20 Mar 2020 14:15:20 +0000 (15:15 +0100)]
mgr/cephadm: No daemon and device refresh if host if out

To make things faster

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
(cherry picked from commit 201a9c17eeb1051ce562205d316f4baeebb58b5f)

5 years agomgr/cephadm: in-memory set of offline hosts
Sebastian Wagner [Fri, 20 Mar 2020 14:13:26 +0000 (15:13 +0100)]
mgr/cephadm: in-memory set of offline hosts

Fixes: https://tracker.ceph.com/issues/43839
Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
(cherry picked from commit 05cf5796f36bc2d826a29aff4a124124df87cbc6)

5 years agomgr/cephadm: remove unused method `_check_hosts`
Sebastian Wagner [Fri, 20 Mar 2020 14:17:03 +0000 (15:17 +0100)]
mgr/cephadm: remove unused method `_check_hosts`

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
(cherry picked from commit e093c3754382f64133178b78d30d113e0f660bb9)

5 years agomgr/cephadm: type annotation for HostCache.mgr
Sebastian Wagner [Fri, 20 Mar 2020 14:11:08 +0000 (15:11 +0100)]
mgr/cephadm: type annotation for HostCache.mgr

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
(cherry picked from commit 52e05f8d20a0042c5ba7bc2d30a9dbaf259f8809)

5 years agomgr/cephadm: add CACHEINODE and EXPORT_DEFAULTS sections
Michael Fritch [Thu, 9 Apr 2020 20:49:14 +0000 (14:49 -0600)]
mgr/cephadm: add CACHEINODE and EXPORT_DEFAULTS sections

Signed-off-by: Michael Fritch <mfritch@suse.com>
(cherry picked from commit 25f4dedd3e75c81b19911fd33b171e613ab1c559)