Jason Dillaman [Mon, 10 Feb 2020 15:16:59 +0000 (10:16 -0500)]
test/pybind: fix expectations for mirror snapshots
When snapshot-based mirroring is enabled, the primary image
should always have at least one mirror-based snapshot to help
distinguish it from an incomplete non-primary image.
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
Jason Dillaman [Thu, 30 Jan 2020 19:49:46 +0000 (14:49 -0500)]
librbd: relax snapshot mirror promotion state determination
If no mirroring-related snapshots exist on the image, it cannot
be determined if the image is primary/non-primary. In the case of
rbd-mirror, this will be the case after a new mirrored image is
created locally but before it can be synced.
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
Jason Dillaman [Thu, 30 Jan 2020 22:45:40 +0000 (17:45 -0500)]
librbd: avoid refreshing within the mirror state machines
The refresh has been moved up to the API layer. Also avoid querying
the MirrorImage struct when creating mirror primary snapshots since
(1) when enabling the record won't exist yet and (2) in the demote/
promote cases it would have just been read by the previous state
machine.
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
Jason Dillaman [Fri, 31 Jan 2020 02:21:30 +0000 (21:21 -0500)]
librbd: optionally ignore missing peers when creating promoted mirror snapshot
The initial enable steps as well as demote/promote should ignore the lack of
peers. Otherwise, you would never be able to enable snapshot mirroring
without first creating TX peers.
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
Jason Dillaman [Thu, 30 Jan 2020 19:13:18 +0000 (14:13 -0500)]
test/rbd-mirror: image replayer test should test both mirroring mode
Attempt to re-use most of the existing image replayer unit tests under
both supported modes. Any existing tests that are purely journal will
only be tested under journal-mode.
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
When cloning a mirrored image, instead of looking at the parent
image's journal state to tell if its still being synced, we can
instead verify that it doesn't have an rbd-mirror image sync
snapshots.
If it doesn't, the image isn't being actively synced, but it
might not have started either. However, the clone state machine
will open the local parent image and attempt to find the specified
snapshot. If it exists, the image sync is complete.
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
Jason Dillaman [Wed, 29 Jan 2020 01:37:44 +0000 (20:37 -0500)]
rbd-mirror: image deleter should support snapshot-based mirroring
Read the current mirror info from the local image instead of directly
reading from the journal. The image needs to be opened before potentially
resetting the journal since journaling might not be enabled.
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
Jason Dillaman [Mon, 27 Jan 2020 21:45:54 +0000 (16:45 -0500)]
librbd: helper state machines for updating/removing mirror images
When the mirror image state is updated or removed, we need to send
notifications to peer listeners like rbd-mirror. These two new
state machines allow re-use of that logic to avoid the duplication.
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
Jason Dillaman [Mon, 27 Jan 2020 18:16:56 +0000 (13:16 -0500)]
rbd-mirror: remove unnecessary get mirror info call from bootstrap
Both the prepare local and remote state machines will now retrieve their
respective mirror info data so it does not need to be repeated at the
higher-level bootstrap state machine.
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
Jason Dillaman [Mon, 27 Jan 2020 17:43:51 +0000 (12:43 -0500)]
rbd-mirror: store remote promotion state within state builder
This simplifies the factory method for the prepare state since it
no longer needs to be an parameter. It will also permit the removal
of the extra get mirror info call in bootstrap.
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
Jason Dillaman [Tue, 28 Jan 2020 20:24:09 +0000 (15:24 -0500)]
rbd-mirror: cache local and remote pool meta properties for all pools
When replicating cloned images across pools, the replayer will
need access to the meta from the parent image pool. By caching
the data, it avoids the need to have the image replayer query
the parent pools.
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
Jason Dillaman [Mon, 27 Jan 2020 15:41:40 +0000 (10:41 -0500)]
rbd-mirror: utilize get mirror info within prepare local image request
This avoids lots of duplicated logic between the two state machines and
provides a path to support both journal and snapshot-based mirroring
with the same class.
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
Jason Dillaman [Wed, 29 Jan 2020 23:46:07 +0000 (18:46 -0500)]
qa: move existing rbd-mirror functional test to journal-specific test
A new functional test for snapshot-based mirroring will be created and
the other stress-tests should eventually be applied to both snapshot-
and journal-based mirroring.
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
Patrick Donnelly [Thu, 30 Jan 2020 15:06:21 +0000 (07:06 -0800)]
Merge PR #32397 into master
* refs/pull/32397/head:
mds: Move StrayManager initializations to its header
mds: Remove extra spaces in StrayManager header.
mds: Reorganize structure members in StrayManager header
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Lenz Grimmer [Thu, 30 Jan 2020 13:43:46 +0000 (13:43 +0000)]
mgr/dashboard: Change project name to "Ceph Dashboard" (#32959)
mgr/dashboard: Change project name to "Ceph Dashboard"
Reviewed-by: Laura Paduano <lpaduano@suse.com> Reviewed-by: Patrick Seidensal <pnawracay@suse.com> Reviewed-by: Stephan Müller <smueller@suse.com> Reviewed-by: Tatjana Dehler <tdehler@suse.com> Reviewed-by: Volker Theile <vtheile@suse.com>
Sage Weil [Thu, 30 Jan 2020 13:01:47 +0000 (07:01 -0600)]
Merge PR #32972 into master
* refs/pull/32972/head:
python-common/ceph/deployment/translate: use 'prepare' instead of 'batch' for trivial case
qa/tasks/cephadm: pass short dev name to osd prepare
mgr/cephadm: fix detection of just-created OSDs
mgr/cephadm: properly indent raise conditions
mgr/cephadm: add warning to other orchestrators
mgr/cephadm: separate acceptance criterias for Devices
mgr/cephadm: fix typos
mgr/cephadm: move utils in test/utils.py
mgr/ssh: increase disk size to 20G
drivegroups: add support for drivegroups + tests
mgr/orch_cli: allow multiple drivegroups
drivegroups: translate disk spec to ceph-volume call
Reviewed-by: Jan Fajerski <jfajerski@suse.com> Reviewed-by: Joshua Schmid <jschmid@suse.de>
Gregory Farnum [Thu, 30 Jan 2020 10:27:06 +0000 (11:27 +0100)]
doc: remove the CephFS-Hadoop instructions
These have not aged gracefully, and in particular include instructions
for setting pool size 1 to let Hadoop control the replication — but I've
heard reports of users setting up multiple size-1 pools and then wondering
where their data went when an OSD dies.
Aleksei Zakharov [Fri, 20 Dec 2019 15:05:05 +0000 (18:05 +0300)]
mgr/prometheus: pg count by pool
If we have all other stats by pool, it's better to have total
count by pool too. We always can sum() all of total, but it's
hard to count by-pool total.
Signed-off-by: Aleksei Zakharov <zakharov.a.g@yandex.ru>
Kefu Chai [Wed, 29 Jan 2020 12:46:45 +0000 (20:46 +0800)]
mgr/cephadm: init attrs created by settattr()
to address the test falure caused by mypy:
```
mypy run-test: commands[0] | mypy --config-file=../../mypy.ini ansible/module.py cephadm/module.py mgr_module.py mgr_util.py orchestrator.py orchestrator_cli/module.py rook/module.py
test_orchestrator/module.py
cephadm/module.py: note: In member "_check_for_strays" of class "CephadmOrchestrator":
cephadm/module.py:596: error: "CephadmOrchestrator" has no attribute "warn_on_stray_hosts"
cephadm/module.py:596: error: "CephadmOrchestrator" has no attribute "warn_on_stray_services"
cephadm/module.py:599: error: "CephadmOrchestrator" has no attribute "warn_on_stray_services"
Found 3 errors in 1 file (checked 8 source files)
```
see also https://github.com/python/mypy/issues/5719
Sébastien Han [Fri, 24 Jan 2020 15:29:54 +0000 (16:29 +0100)]
ceph-volume: add db and wal support to raw mode
Using the raw mode, the OSD preparation can now be called with extra
arguments to specify a block device for either rocksdb db or wal.
The code expects the device to be available and no validation will be
performed. Also, the entire block device will be consumed.
The goal is run this on Kubernetes with Rook and OSD are running on PVC.
Users will request PVC from a certain size, the request will be
acknowledged and a block will be created, then attached to the machine.
Later on consumed by Rook's preparation job.
Lenz Grimmer [Wed, 29 Jan 2020 10:04:28 +0000 (11:04 +0100)]
mgr/dashboard: Change project name to "Ceph Dashboard"
The Dashboard's "About" page still referred to the
application as "Ceph Manager Dashboard".
Changed the `projectName` constant to
"Ceph Dashboard" to resolve this.
Sage Weil [Tue, 28 Jan 2020 19:33:49 +0000 (13:33 -0600)]
osd: dispatch_context and queue split finish on early bail-out
If we bail out of advance_pg early because there is an upcoming merge, we
still need to dispatch_context() on rctx before we drop the PG lock. And
the rctx that we submit needs to include the on_applied finisher comit
to call _finish_splits.
This is noticeable (at least) when there is a split and merge that are
both known. When we process the split, the new child is added to new_pgs.
When we get to the merge epoch, we stop early and take the bail-out
path.
Fix by adding a dispatch_context call for this path. And further make sure
that both dispatch_context callers in this function queue up the
new_pgs event.
Fixes: https://tracker.ceph.com/issues/43825 Signed-off-by: Sage Weil <sage@redhat.com>
projects / ceph.git / log