Sage Weil [Sat, 14 Apr 2012 00:11:54 +0000 (17:11 -0700)]
filestore: two-phase guard
For certain operations (collection_add) we need a two-phase guard, and an
"in-progress" state.
- before exposing an object in a new location, we need to mark it so that
old operations affecting the target name don't touch the new object.
- can't just set the guard before starting or else we can't distinguish
between a collection_add that was in-progress and one that happend a
long time ago.
We may need the same for collection_rename().
Signed-off-by: Sage Weil <sage.weil@dreamhost.com>
Sage Weil [Sat, 14 Apr 2012 00:17:36 +0000 (17:17 -0700)]
filestore: simple failure injections via --filestore-kill-at <n>
This will make filestore suicide (_exit(1)) on the n'th potential failure
call site. We can potentially fail:
- before a transaction
- between each op
- at the end
Additionally, we instrument the guards:
- before/after/inside _set_replay_guard
- between significant steps of callers of _set_replay_guard
All instrumentation points are inside _do_transactions(), so if everything
is done in a single sequencer (or from a single thread) the failure
point is deterministic.
That said, use an atomic so we will still reliably fail (at some point)
when there are multiple filestore threads in action.
Signed-off-by: Sage Weil <sage.weil@dreamhost.com>
Sage Weil [Fri, 13 Apr 2012 18:30:49 +0000 (11:30 -0700)]
filestore: replay collection_move using add+remove
This approximates the buggy collection_move. It is still buggy. It is
only there to replay old journals.
Rip out buggy (and now unused) collection_move code.
For the record, the problem there is that a crash between setting the guard
and unlinking the old name will not remove the old name on replay because
the guard for the link stage is indistinguishable from that for the unlink
stage.
Signed-off-by: Sage Weil <sage.weil@dreamhost.com>
Sage Weil [Fri, 13 Apr 2012 16:56:04 +0000 (09:56 -0700)]
filestore: implement collection_move() as add + remove
This ensures we get add and remove steps with different spos values, which
makes the guard work. The collection_move implementation breaks on replay
because those values match, so the just-set guard prevents unlink from
happening.
Signed-off-by: Sage Weil <sage.weil@dreamhost.com>
Sage Weil [Tue, 10 Apr 2012 22:24:49 +0000 (15:24 -0700)]
filestore: fix collection_move guard
We had a sequence like:
1- write A block 1
2- write A block 2
3- write A block 3
4- write A block 4
5- move A -> B
- link B
- unlink A
- set guard on B <crash>
- replay 3, 4, 5
with the result being B with only half of its content. The problem is that
we destroyed the old link _and_ didn't guard the new content. Instead,
set the guard before the link, and replay the unlink step here
unconditionally.
Fixes: #2178 Signed-off-by: Sage Weil <sage.weil@dreamhost.com>
Sage Weil [Fri, 6 Apr 2012 16:33:43 +0000 (09:33 -0700)]
encoding: use iterator to copy_in encoded length
This gives us a pointer to the position into the list where the final
length value will be copied. Previously we used bl.copy_in(), which takes
a byte offset and needs iterator over the bufferlist to seek to the
correct position, resulting in O(n^2) encoding time for large structures.
Fixes: #2161 Reported-by: Jim Schutt <jaschut@sandia.gov> Diagnosed-by: Ake van der Meer <petrabbit@xs4all.nl> Signed-off-by: Sage Weil <sage.weil@dreamhost.com>
Sage Weil [Tue, 3 Apr 2012 21:21:53 +0000 (14:21 -0700)]
rgw: throttle at num_threads * 2
If we throttle at num_threads, then nothing gets into the workqueue until
a worker thread is idle, which means you pay the latency of setting it up
and queueing it. This way we keep some requests ready to go.
Greg Farnum [Wed, 28 Mar 2012 22:06:32 +0000 (15:06 -0700)]
msgr: clean up Pipe::do_sendmsg.
Document it as with the tcp stuff, remove an if(0)'d debugging block,
and remove the useless "sd" parameter since it's always the same as
the Pipe's sd member.
Greg Farnum [Tue, 27 Mar 2012 19:57:14 +0000 (12:57 -0700)]
msgr: make a bunch of stuff private.
Why were all these data members public? They're accessed by Pipes
and the Accepter and stuff, so maybe that's why...but that's all
internal interface stuff.
Convert ms_addr and _my_name to be references to their fields in
the entity_inst_t my_inst.
This way we can use const references for accessing all of them,
instead of the bizarre distinction we had before for get_myinst().
Greg Farnum [Mon, 19 Mar 2012 20:12:14 +0000 (13:12 -0700)]
msgr: change the signature of get_myaddr()
Return a const reference to the actual address, instead of copying it.
All current users are happy with this, and I can't see a good reason
to copy it instead.
Greg Farnum [Thu, 8 Mar 2012 00:43:04 +0000 (16:43 -0800)]
msgr: get_connection() is required to establish a connection if none exists.
Making an allowance for lossy server connections is silly. Just don't
ask for the Connection in that case. (There aren't any users who
rely on the previous behavior.)
Document that requirement in Messenger.h!
Greg Farnum [Sat, 31 Mar 2012 00:07:19 +0000 (17:07 -0700)]
ceph_mon: fix fsid parsing.
fsid is a field in the CephContext _conf structure and is parsed by
the standard options parsing library before it gets to the ceph_mon
custom parsing.
Instead do the standard parsing, and check that member directly
to decide if we want to (over)write the monmap's fsid.
Sage Weil [Fri, 30 Mar 2012 23:14:05 +0000 (16:14 -0700)]
osd: update_stats() on reads too
Update pg stats on any op completion (read or write), not just writes. Do
the calls with log_op_stats() for consistency's sake. Skip if the request
was an error.
Fixes: #2209 Signed-off-by: Sage Weil <sage.weil@dreamhost.com> Reviewed-by: Samuel Just <samuel.just@dreamhost.com>
Tommi Virtanen [Wed, 28 Mar 2012 20:55:01 +0000 (13:55 -0700)]
doc: Convert the mailing list mention to not be a section heading.
If toctree is inside a section, the subtree is inside the section too.
We don't want all of dev/* to be under "Mailing list".
I have not found a decent workaround for this. The toplevel toctree
avoids this purely by the fact that it is the topmost toctree. Right
now that means you should 1) avoid having more than a few paragraphs of
text before the toctree for that subtree (put most of the content after
the toctree; clumsy if the toctree is long), or 2) put the toptree
immediately after the document title, make it :hidden:, and let the
reader use links in the text or the ToC in the sidebar to navigate.
See start/index for an example of this.
Signed-off-by: Tommi Virtanen <tommi.virtanen@dreamhost.com>
Sage Weil [Fri, 30 Mar 2012 16:51:45 +0000 (09:51 -0700)]
filestore: set guard on collection_move
During recovery we submit transactions like:
- delete a/foo
- move tmp/foo to a/foo
This prevents the EEXIST check in collection_move from doing any good,
since the destination never exists. We need to do that remove at least
sometimes, because we may be overwriting an existing/older version of the
object.
So,
- set the guard after we do the move, so that
- the delete won't be repated, and
- the EEXIST check will work
Also check the guard for good measure (although that doesn't do anything
specifically useful in this scenario).
Fixes: #2164 Signed-off-by: Sage Weil <sage@newdream.net> Reviewed-by: Josh Durgin <josh.durgin@dreamhost.com> Reviewed-by: Samuel Just <samuel.just@dreamhost.com>
Sage Weil [Thu, 29 Mar 2012 05:32:30 +0000 (22:32 -0700)]
osd: discard heartbeat_peer in note_down_osd
Discard the heartbeat_peer as soon as we find out, along with queued
failures, or else the heartbeat_check may come along (without map_lock)
and requeue a failure. And then later, when we try to report it, we'll
osdmap->get_inst() on a now-down OSD and fail miserably.
Reported-by: Wido den Hollander <wido@widodh.nl> Signed-off-by: Sage Weil <sage.weil@dreamhost.com>
test: test_workload_gen: Add callback for collection destruction.
When we remove a collection, we must cleanup after the coll_entry_t we
once had on the available collections set. For some reason, we weren't
doing this.
This commit adds a new callback, which inherits from the 'OnReadable'
callback on the WorkloadGenerator class, that will be responsible for
deleting the coll_entry_t once we know the collection transaction
destroying the collection has finished.
test: test_workload_gen: Change CLI option and add '--help' usage.
With this commit, we support the following options (and old ones are no
longer available):
--test-num-colls VAL Set the number of collections
--test-num-objs-per-coll VAL Set the number of objects per
collection
--test-destroy-coll-per-N-trans VAL Set how many transactions to run
before destroying a collection.
And --help will show the program's usage description.
test: test_workload_gen: Default arguments, and minor changes.
Besides adding support for default arguments, passed onto global_init(),
this commit fixes a conflict in Makefile.am, and a missing lib
dependency. Also, we didn't used to pay attention to the return values
from store->mkfs() and store->mount(), and now do.
test: test_workload_gen: CodeStyle compliance and cleanup.
This commit aims at the compliance with Ceph's CodeStyle, as well
as cleaning up some lingering unused code.
Also, now we allow changing the default OSD data and journal
locations, as well as the OSD journal size, by providing the
options '--osd-data <PATH>', '--osd-journal <PATH>' and
'--osd-journal-size <VAL>' on the CLI arguments. If not provided,
these will default to 'workload_gen_dir', 'workload_gen_journal'
and '400', respectively.
In it's current state, the workload generator will queue a lot of
transactions onto the FileStore, and will wait if needed in case
there are too many in-flight transactions.
The workload generator will perform the transactions over a
pre-determined number of collections and objects, which may very
well be defined at runtime by using the options '-C <VAL>' and
'-O <VAL>' for collections and objects per collection, respectively.
If these are not provided, the program will default to 30 collections
and 6000 objects per collection.
Sage Weil [Tue, 27 Mar 2012 22:12:07 +0000 (15:12 -0700)]
osd: fix handling of recovery sources when osds go down
If a source osd goes down, we need to
- reset any pulls (already did that before)
- remove peer from missing_loc so that we know what is now unfound
- restart recovery/discover_all_missing in case new stuff is now unfound
This fixes a bug like so:
- we peer
- we find an object we need to recover on a stray osd
- that osd goes down
- recover_primary() thinks unfound=0 but it really is 1
... recover_primary 3270c60b/mds0_sessionmap/head 4'1 (missing) (missing head)
... pull 3270c60b/mds0_sessionmap/head v 4'1 but it is unfound
projects / ceph.git / log