Jason Dillaman [Wed, 25 May 2016 18:00:34 +0000 (14:00 -0400)]
rbd-mirror: stop stale replayers before starting new replayers
If the connection details are tweaked for a remote peer, stop
the existing replayer before potentially starting a new replayer
against the same remote.
Jason Dillaman [Tue, 24 May 2016 02:21:33 +0000 (22:21 -0400)]
journal: eliminate watch delay for object refetches
The randomized write sizes of the modified rbd-mirror stress
test results in a lot of journal object with few entries.
Immediately fetch objects when performing a refetch check prior
to closing an empty object.
Jason Dillaman [Mon, 23 May 2016 18:57:03 +0000 (14:57 -0400)]
journal: keep active tag to assist with pruning watched objects
It's possible that there might be additional entries to prune in
objects that haven't been prefetched yet. Keep the active tag
to allow these entries to be pruned after they have been loaded.
Jason Dillaman [Mon, 23 May 2016 15:01:05 +0000 (11:01 -0400)]
journal: cleanup watch refetch flag handling
Clear the refetch required flag while scheduling the watch
and remove the stale object after the watch completes if still
empty. Previously, it was possible for the flag to become
out-of-sync with whether or not it was actually refreshed
and pruned.
Fixes: http://tracker.ceph.com/issues/15993 Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit ff2cc27ae592646b495bf1b614d35bd50c091a3d)
Ricardo Dias [Fri, 13 May 2016 15:44:53 +0000 (16:44 +0100)]
rbd-mirror: Unregister clients from non-primary images journal
A non-primary image may have registered clients on its journal
(for instance a primary image that was later demoted). We must
unregister the clients when disabling image mirroring with the
force option.
Mykola Golub [Wed, 25 May 2016 18:54:16 +0000 (21:54 +0300)]
test: workaround failure in journal.sh
With the changes to ensure that the commit position of a new
client is initialized to the minimum position of other clients,
the 'journal inspect/export' commands return zero records because
the master client has committed all of its entries.
Workaround this by restoring the initial commit position after
writing to the image.
Robin H. Johnson [Fri, 20 May 2016 23:00:33 +0000 (16:00 -0700)]
rgw: fix manager selection when APIs customized
When modifying rgw_enable_apis per RGW instance, such as for staticsites, you
can end up with RESTManager instance being null in some cases, which returns a
HTTP 405 MethodNotAllowed to all requests.
Example configuration to trigger the bug:
rgw_enable_apis = s3website
Backport: jewel
X-Note: Patch from Yehuda in private IRC discussion, 2016/05/20. Fixes: http://tracker.ceph.com/issues/15973 Fixes: http://tracker.ceph.com/issues/15974 Signed-off-by: Robin H. Johnson <robin.johnson@dreamhost.com>
(cherry picked from commit 7c7a465b55f7100eab0f140bf54f9420abd1c776)
Kefu Chai [Fri, 13 May 2016 03:26:31 +0000 (11:26 +0800)]
osd/OpRequest: reset connection upon unregister
this helps to free the resources referenced by the connection, among
other things, in the case of MOSDOp, the OSD::Session and OSDMap. this
helps to free the resource earlier and trim the osdmaps in time.
Kefu Chai [Thu, 12 May 2016 12:28:11 +0000 (20:28 +0800)]
osd: reset session->osdmap if session is not waiting for a map anymore
we should release the osdmap reference once we are done with it,
otherwise we might need to wait very long to update that reference with
a newer osdmap ref. this appears to be an OSDMap leak: it is held by an
quiet OSD::Session forever.
the osdmap is not reset in OSD::session_notify_pg_create(), because its
only caller is wake_pg_waiters(), which will call
dispatch_session_waiting() later. and dispatch_session_waiting() will
check the session->osdmap, and will also reset the osdmap if
session->waiting_for_pg.empty().
Jason Dillaman [Thu, 19 May 2016 00:53:26 +0000 (20:53 -0400)]
rbd-mirror: disable librbd caching for replicated images
Each image has its own cache and each cache uses its own thread. With
a large replicated cluster, this could result in thousands of extra
threads and gigabytes of extra memory.
Fixes: http://tracker.ceph.com/issues/15930 Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit ea35f148257282fe3f3ae02fe7a26cf245cda952)
Jason Dillaman [Thu, 19 May 2016 19:50:04 +0000 (15:50 -0400)]
librbd: delay commit of overwritten journal event
With the cache enabled and write-after-write IOs to the same
object extents, it was possible for the overwritten journal event
to be committed before the overwriter journal event was written
to disk. If a client crash occurs before the event is written,
the image will be inconsistent on replay.
Fixes: http://tracker.ceph.com/issues/15938 Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit e8bf64cc85ffe3d2dda23eab1834f7a5f104f6fe)
Jason Dillaman [Thu, 19 May 2016 17:48:22 +0000 (13:48 -0400)]
qa/workunits/rbd: record rbd CLI debug messages during mirror stress
The debug messages from 'rbd bench-write' and 'rbd snap create',
in addition to the existing debug messages from rbd-mirror, make
it possible to determine the source of any image inconsistency.
Yehuda Sadeh [Mon, 16 May 2016 21:35:12 +0000 (14:35 -0700)]
rgw: keep track of written_objs correctly
Fixes: http://tracker.ceph.com/issues/15886
Only add a rados object to the written_objs list if the write
was successful. Otherwise if the write will be canceled for some
reason, we'd remove an object that we didn't write to. This was
a problem in a case where there's multiple writes that went to
the same part. The second writer should fail the write, since
we do an exclusive write. However, we added the object's name
to the written_objs list anyway, which was a real problem when
the old processor was disposed (as it was clearing the objects).
Jason Dillaman [Tue, 17 May 2016 01:17:09 +0000 (21:17 -0400)]
qa/workunits/rbd: rbd-mirror daemon stress test
This test repeatedly runs rbd bench-write, kills the process
randomly to create an unclean journal shutdown, and verifies
that the image content replicates correctly.
Jason Dillaman [Sun, 15 May 2016 13:52:41 +0000 (09:52 -0400)]
journal: skip partially complete tag entries during playback
If a journal client does not fully write out its buffered entries
before quiting, replay should skip over all remaining out-of-
sequence entries for the tag.
Fixes: http://tracker.ceph.com/issues/15864 Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 9454f7e4c62437b1c288f371009feba1fd374584)
Jason Dillaman [Sat, 14 May 2016 22:58:41 +0000 (18:58 -0400)]
journal: close, advance, and open object set ordering
Flush in-flight appends to open objects before advancing the
active object set. Additionally, don't start recording to the
new objects until after advancing the active set.
Jason Dillaman [Fri, 13 May 2016 20:17:37 +0000 (16:17 -0400)]
journal: ignore flush on closed/overflowed object
The journal would be in-progress on transitioning to a new
object recorder in a newer object set. Once the records
re-attach to the new object player they will automatically
flush.
Jason Dillaman [Fri, 13 May 2016 18:49:07 +0000 (14:49 -0400)]
journal: re-fetch active object before advancing set during replay
During a live replay, it's possible that an append and and overflow
into the next object could race with the live playback of the same
object. Re-fetch an "empty" object at least once before advancing
to next set to ensure all records have been read.
Fixes: http://tracker.ceph.com/issues/15665 Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 6056f8c45c99bd37cb18933a37cc238c7e9a7c7d)
Karol Mroz [Mon, 2 May 2016 12:01:27 +0000 (14:01 +0200)]
global: don't link lttng into libglobal
Rely on dynamic initialization instead. Linking lttng in this way had
the unfortunate side effect of causing radosgw to segfault (when
daemonized) during sigterm processing (ie. during lttng_ust_exit()).
This was originally removed in 638738f, but accidentally re-added via 5f61d36.
* enable it using dh_systemd_enable
* start the target using dh_systemd_start
* move the dh_installinit, dh_systemd_enable, dh_systemd_start calls
down, so they can identify the service files if they care about them.
Kefu Chai [Mon, 9 May 2016 06:15:36 +0000 (14:15 +0800)]
osd: remove all stale osdmaps in handle_osd_map()
in a large cluster, there are better chances that the OSD fails to trim
the cached osdmap in a timely manner. and sometimes, it is just unable
to keep up with the incoming osdmap if skip_maps, so the osdmap cache
can keep building up to over 250GB in size. in this change
* publish_superblock() before trimming the osdmaps, so other osdmap
consumers of OSDService.superblock won't access the osdmaps being
removed.
* trim all stale osdmaps in batch of conf->osd_target_transaction_size
if skip_maps is true. in my test, it happens when the osd only
receives the osdmap from monitor occasionally because the osd happens
to be chosen when monitor wants to share a new osdmap with a random
osd.
* always use dedicated transaction(s) for trimming osdmaps. so even in
the normal case where we are able to trim all stale osdmaps in a
single batch, a separated transaction is used. we can piggy back
the commits for removing maps, but we keep it this way for simplicity.
* use std::min() instead MIN() for type safety
projects / ceph.git / log