Jason Dillaman [Wed, 22 Apr 2020 17:51:58 +0000 (13:51 -0400)]
rbd-mirror: skip snapshot image sync if mirror snapshot is marked clean
This is currently only utilized for the case where a newly created image
has mirroring enabled at time of creation, but it could be expanded in the
future if we track writes.
Fixes: https://tracker.ceph.com/issues/44596 Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit fbede28d1a7713248765cf91136bb19b3b3fdac2)
Jason Dillaman [Wed, 22 Apr 2020 15:57:45 +0000 (11:57 -0400)]
librbd: EnableRequest now accepts a boolean to indicate a clean image
If the image is clean, it's treated as if it was newly created and
therefore clean since snapshot id 0. The CreateRequest and
CloneRequest state machines pass true for this bool if mirroring
is being enabled during creation.
Jason Dillaman [Mon, 20 Apr 2020 22:16:40 +0000 (18:16 -0400)]
librbd: mirror enable state machine might need to open image
If attempting to create a snapshot-based mirroring primary snapshot,
the image needs to first be opened. If we weren't supplied an image,
open the image, create the snapshot, and close the image again.
Jason Dillaman [Mon, 20 Apr 2020 19:23:53 +0000 (15:23 -0400)]
librbd: pass bit-flags to image::CreateRequest
The current boolean for skip enable mirror will be able to be changed to
a tri-state to force enable mirror (in addition to the current auto-enable
if in pool-mode).
Or Friedmann [Sun, 8 Mar 2020 13:34:48 +0000 (15:34 +0200)]
rgw: Disable prefetch of entire head object when GET request with range header
Disable prefetch of entire head object when GET request with range header.
The current behavior for the RGW is getting the whole object although the client asked only for a small bytes offset.
For example: If the client asked for bytes=0-1, The RGW will anyway fetch 0-4194304
Fixes: https://tracker.ceph.com/issues/44508 Signed-off-by: Or Friedmann <ofriedma@redhat.com>
(cherry picked from commit 2be5af0006169cb54547034aa98b7eacb8751d59)
Introduced in 4d42b4c5a0ed ("common/TextTable: default to 2 spaces
separating columns") and 41f003518a07 ("common/TextTable: only pad
between columns").
Jason Dillaman [Fri, 17 Apr 2020 15:17:05 +0000 (11:17 -0400)]
rbd-mirror: track in-flight start/stop/restart in instance replayer
The shut down waits for in-flight ops to complete but the
start/stop/restart operations were previously not tracked. This
could cause a potential race and crash between an image replayer
operation and the instance replayer shutting down.
Fixes: https://tracker.ceph.com/issues/45072 Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 31140a940ea1909c4b5d68ef4593cb582a527354)
Jason Dillaman [Mon, 6 Apr 2020 20:21:35 +0000 (16:21 -0400)]
rbd-mirror: propagate full snap-seq mapping in non-primary snapshots
Previously only newly created user snapshots were included in the
non-primary snapshot snap-seq mapping table. However, we need to
retain a full history of the mapping table if we want to be able to
prune non-primary snapshots.
Failovers are a special case since we won't have a valid snap seq mapping
so it will need to be rebuilt. Luckily, both sides should be read-only
in the previous state so we can use the snapshot names to find matches.
Jason Dillaman [Tue, 7 Apr 2020 23:12:03 +0000 (19:12 -0400)]
rbd-mirror: ignore non-primary read-only state for remote images
snapshot-based mirroring needs to be able to potentially delete a
demotion snapshot during the unlink process. Previously, these
snapshots have been left while the read-only error was ignored.
Jason Dillaman [Thu, 9 Apr 2020 03:06:05 +0000 (23:06 -0400)]
librbd: fixed race condition on demotion of snapshot-based mirrored image
A pending refresh could occur after setting the non-primary feature flag but
before the creation of the demotion snapshot. This would prevent the snapshot
from being created and would leave the image in a half-primary state.
The mirror image status for replaying snapshot-based images now includes
bytes per second and per snapshot, in addition to an estimated number of
seconds until the image is fully synced.
The mirror image status for replaying journal-based images now includes
bytes and entries per second in addition to an estimated number of seconds
until the image is fully synced.
Jason Dillaman [Wed, 1 Apr 2020 19:26:39 +0000 (15:26 -0400)]
rbd-mirror: switch to json_spirit formatter for journal image status
The free-form journal replay status description is now JSON-encoded. The
"master"/"mirror" designators have been changed to "primary"/"non_primary"
to better align with RBD terminology.
Jason Dillaman [Thu, 2 Apr 2020 18:50:37 +0000 (14:50 -0400)]
rbd-mirror: periodically poll image replayer status
When metrics are incorporated, there might not be a forced status update
if no new data is available to replicate. However, we will want the metrics
to decrease over time.
Igor Fedotov [Mon, 3 Feb 2020 15:50:50 +0000 (18:50 +0300)]
os/bluestore: do not use 'unused' bitmap if makes no sense.
The processing logic which relies on 'unused' bitmap makes sense for
bluestore setup where min alloc size is different from device block
size. Now omitting if that's not true.
This is directly fixed in octopus since it's needed
to get the dashboard backend API tests running
based on the previous commits of this PR.
There is a fix in master but this one
uses "from io import StringIO" while
we still need to be python2 compatible in octopus.
So this import is done with "six" (expects str in python2
instead of "io" (expects unicode
in python2).
This import line is not cherry-picked from master
due to the above mentioned reasons.
rgw: reshard: skip stale bucket id entries from reshard queue
If we encounter a reshard queue entry that has an older ID compared to the
bucket's current ID, it'd mean that some other process or a manual reshard has
already processed this entry, skip processing the entry this time. An
alternative is to verify the num_shards that we have in queue >= the current
shards, but this would mean that we may reshard a recently manual resharded
bucket again which might not be intended
Tim Serong [Thu, 16 Apr 2020 05:32:02 +0000 (15:32 +1000)]
cephadm: add --force-start option to adopt
Current behaviour is to only start a newly adopted ceph daemon if it was
already running before the adopt. Adding a --force-start option allows
the adopt command to start newly adopted daemons that weren't originally
running, to save the user having to manually invoke `systemctl start
ceph-$FSID@$DAEMMON.$ID`.
Tim Serong [Thu, 16 Apr 2020 05:09:10 +0000 (15:09 +1000)]
cephadm: don't throw when disabling ceph-volume simple unit
When adopting OSDs, if a ceph-volume simple service is already disabled
(or otherwise missing) the previous implementation would raise an error,
thus killing the adopt.
Tim Serong [Wed, 15 Apr 2020 09:26:19 +0000 (19:26 +1000)]
cephadm: handle adopting offline OSDSs
The current adopt behavior expects OSDs to be online, in order to read
/var/lib/ceph/osd/ceph-$ID/fsid. To handle the case where OSDs
are offline, this change first checks to see if that file is present,
and if not, falls back to calling `ceph-volume lvm list` to see if
there's a matching OSD there, and if that doesn't work, it checks
/etc/ceph/osd/*.json to see if there's a matching old-style simple
OSD present.
For LVM OSDs, the only thing we need is the ODS's fsid; the remainer
of the adopt procedure "just works", as the various other files
in /var/lib/ceph/$FSID/osd.$ID are created by magic anyway when the
OSD is activated, so it doesn't matter if they're not present at
adoption time.
For simple (ceph-disk created) OSDs, we actually need all the files under
/var/lib/ceph/osd/ceph-$ID/ to be moved to /var/lib/ceph/$FSID/osd.$ID
so if a simple OSD is found, it's mounted first, so the existing
move_files() a bit further down around line 3200 continues to work.
Co-Authored-By: Sebastian Wagner <sebastian@spawnhost.de>
Update doc/cephadm/troubleshooting.rst
Co-Authored-By: Sebastian Wagner <sebastian@spawnhost.de>
Added mgfritch's suggestion regarding checking to see if the public key
is listed in the authorized_keys file.
Fixes errors when calling `from_json` of these classes:
- InventoryHost: parsing labels
- ServiceDescription: `last_refresh` and `created` fields should be parsed
to datetime type.
Matthew Oliver [Thu, 23 Jan 2020 00:02:24 +0000 (11:02 +1100)]
cephadm: add RGW SSL support
This patch adds SSL support to RGW when using cephadm.
If an SSL certificate is provided, inside the json supplied with:
cpeh orchestrator rgw create -i rgw.json
Then the SSL cert and/or key will be added to pushed into the mon config-key database
using the key `rgw/cert/<rgw_realm>/<rgw_zone>.[crt|key]`.
Which will then be referenced in the config:
Of course you could also just upload the cert and key yourself to
config-key location, and ssl will be enabled as well. But this patch
let's you either supply them via `-i` or as a manual upload step.
Co-Authored-By: Michael Fritch <mfritch@suse.com> Co-Authored-By: Sebastian Wagner <sebastian@spawnhost.de> Signed-off-by: Matthew Oliver <moliver@suse.com>
(cherry picked from commit 8204b1d524b155fac56c3937d1aec41a18ab2d52)
projects / ceph.git / log