Sam Lang [Fri, 14 Dec 2012 03:23:27 +0000 (17:23 -1000)]
client: Add config option to inject sleep for tick
Testing the tick delay with a fork/suspend is causing
corruption in the lockdep code. This approach uses
a config option to sleep the tick thread for a number
of seconds, avoiding the entire fork/suspend mess.
Josh Durgin [Tue, 11 Dec 2012 06:34:05 +0000 (22:34 -0800)]
rbd.py: check for new librbd methods before use
This way attempting to use format 2 images works when you upgrade the
python bindings before librbd, and attempting to use functions
that librbd does not have results in more understandable errors.
Sage Weil [Fri, 14 Dec 2012 00:26:43 +0000 (16:26 -0800)]
osd: up != acting okay on mkpg
This can happen when:
- mon sends create pg
- it gets created
- osd remaps the pg to a different osd
but osd does not update pg status to the mon
- mkpg resent to the new osd
or something along those lines. It seems unusual, but in the end who
really cares why the mon doesn't know about the pg creation yet.
Note that this check was added in the initial commit where acting/up was
added; there is no specific condition of concern we are protecting against.
Instead, ignore the message. We'll get a query soon anwyay.
This 'fixes' #3614.
Signed-off-by: Sage Weil <sage@inktank.com> Reviewed-by: Samuel Just <sam.just@inktank.com>
mon: OSDMonitor: don't allow creation of pools with > 65535 pgs
There are some limitations to the number of possible pg's per pool, and
by allowing the 'osd pool create' command to succeed, we were making room
to some anomalous behavior.
Dan Mick [Thu, 13 Dec 2012 22:06:17 +0000 (14:06 -0800)]
rbd: handle images disappearing while in ls -l
rbd.list() returns a list of names, but nothing stops them from
going away before rbd.open(); check for ENOENT and ignore if that
happens; warn on other errors
Signed-off-by: Dan Mick <dan.mick@inktank.com> Reviewed-by: Josh Durgin <josh.durgin@inktank.com>
Sam Lang [Thu, 13 Dec 2012 00:28:12 +0000 (14:28 -1000)]
filestore: Don't keep checking for syncfs if found
Valgrind outputs a warning for unrecognized system calls,
and does so for the syscall(__SYS_syncfs,...) and
syscall(__NR_syncfs, ...) calls. This patch avoids making
those calls (and the warning, when run in valgrind) if the
syncfs libc call is available.
INFO:teuthology.task.ceph.osd.1.err:--10568-- WARNING: unhandled syscall: 306
INFO:teuthology.task.ceph.osd.1.err:--10568-- You may be able to write your own handler.
INFO:teuthology.task.ceph.osd.1.err:--10568-- Read the file README_MISSING_SYSCALL_OR_IOCTL.
INFO:teuthology.task.ceph.osd.1.err:--10568-- Nevertheless we consider this a bug. Please report
INFO:teuthology.task.ceph.osd.1.err:--10568-- it at http://valgrind.org/support/bug_reports.html.
Yehuda Sadeh [Fri, 30 Nov 2012 00:48:46 +0000 (16:48 -0800)]
rgw: option to provide alternative s3 put obj success code
Fixes: #3529
Added a new option: rgw_s3_success_create_obj_status.
Expected values are 0, 200, 201, 204. A value of 0
will skip the special handling altogether. Any value
other than the specified will default to 200.
Sage Weil [Wed, 12 Dec 2012 01:15:56 +0000 (17:15 -0800)]
os/JournalingObjectStore: un-break op quiescing during journal replay
Commit d9dce4e9273adb4279519d65a0d8bfdfecb5c516 broke journal replay
because the commit thread may try to do a commit, and the ops are not
being applied via the normal work queue. Add back in a simpler form of the
old op quiescing (simpler because there is a single thread doing the
replay).
Signed-off-by: Sage Weil <sage@inktank.com> Reviewed-by: Samuel Just <sam.just@inktank.com>
Josh Durgin [Tue, 11 Dec 2012 20:26:21 +0000 (12:26 -0800)]
st_rados_watch: tolerate extra notifies
With retries, it's possible for notifies to be received more than once
when they are resent to different OSDs, since the OSDs only track them
in memory.
Yehuda Sadeh [Tue, 11 Dec 2012 21:41:50 +0000 (13:41 -0800)]
mds: shutdown cleanly if can't authenticate
Fixes: #3590
This was triggered when tried to run mds with cephx enabled
against a mon without cephx support. We didn't handle the
returned error at all, so this one fixes it. It also makes
sure that we don't continue initialization until rotating
keys are in place (as the osd does).
Josh Durgin [Tue, 11 Dec 2012 17:54:44 +0000 (09:54 -0800)]
objecter: don't use new tid when retrying notifies
Watches update the on-disk state in the OSD, and aren't idempotent,
so refreshing them must be treated as a separate transaction by the OSD.
Notifies are just in-memory state, and resending them will result in
acceptable behavior:
- if it's the same osd, the resent op will be recognized as a duplicate
- if it's a different osd, a new notify will be triggered since the new osd
can't tell whether the original notify was received by any watchers
Using a new tid for each resend can cause some unecessary extra work,
as the first case turns into the second.
Rename operation can call predirty_journal_parents() several times.
So a directory fragment's rstat can also be modified several times.
But only the first modification is journaled because EMetaBlob::add_dir()
does not update existing dirlump.
For example: when hanlding 'mv a/b/c a/c', Server::_rename_prepare may
first decrease directory a and b's nested files count by one, then
increases directory a's nested files count by one.
Sage Weil [Tue, 11 Dec 2012 00:41:19 +0000 (16:41 -0800)]
config: do not always print config file missing errors
Do not generate errors each time we fail to open a config file; only
generate one at the end if a search path was specified and none were
usable, right before we (already) exit. This avoids spamming stderr
about each path we tried in the search list before we found a good one.
Samuel Just [Mon, 10 Dec 2012 21:38:24 +0000 (13:38 -0800)]
config_opts.h: adjust recovery defaults
osd max backfills: 5 was too low for a default, 10
seems to work better in testing. The message
priority system should minimize disruption of
push and pull operations anyway.
osd recovery max chunk: 1MB was too small for a
default. 8MB is reasonable for a single push
and will allow us to recover an rbd block in
one push rather then 4 reducing client io
latency during log-based recovery.
osd recovery op priority: 10 rather than 30 will
further reduce the client io latency impact of
push and pull operations.
Sage Weil [Sun, 9 Dec 2012 05:44:54 +0000 (21:44 -0800)]
mon: fix leak of pool op reply data
We pass a pointer because it is an optional argument, but we shouldn't
put the bufferlist on the heap or else we have to manage it's life
cycle, and that's fragile (and previously broken).
Sage Weil [Fri, 7 Dec 2012 00:18:07 +0000 (16:18 -0800)]
filestore: simplify op quescing
The delicate balancing with op_apply_start() and that fact that it can
block was making it very hard to determine how long commit_start() should
wait, since requests in the workqueue threads could op_apply_start() in
any order. For example,
threadA: gets osr1 from wq
threadA: gets osr2 from wq
threadA: dequeue seq 11 from osr1, op_apply_start
threadC: commit_start on 11
threadA: op_apply_finish on seq 11
threadC: commit_started, commit_finish
threadB: dequeue seq 10 from osr2
<failed assert, badness>
Instead, rip out all this code, and use the ThreadPool pause() method to
quiesce operations. Keep some of the (now unnecessary) fields around
for sanity checks (blocked, open_ops, max_applying_seq, etc.).
Samuel Just [Tue, 4 Dec 2012 19:36:58 +0000 (11:36 -0800)]
PG: remove last_epoch_started asserts in proc_primary_info
These asserts are valid for a uniform cluster, but they won't hold
for a replica running a version without the info.last_epoch_started
patch.
Signed-off-by: Samuel Just <sam.just@inktank.com> Reviewed-by: Greg Farnum <greg@inktank.com>
(cherry picked from commit 0756052cff542ab02d653b40c37a645b395f31b3)
Sage Weil [Wed, 5 Dec 2012 13:56:37 +0000 (05:56 -0800)]
mds: move from EXCL to SYNC if nobody wants to write
We were moving to the MIX even if nobody wanted to write; that is not
useful, since if we only want to read SYNC will let us cache those reads.
SYNC is also a more friendly place (all things equal) to be.
mon: Monitor: rework 'paxos' to a list instead of a vector
After adding the gv patches, during Monitor::recovered_leader() we started
waking up contexts following the order of the 'paxos' vector. However,
given that the mdsmon has a forgotten dependency on the osdmon paxos
machine, we were incurring in a situation in which we proposed a value
through the osdmon before creating a new pending value (but by being
active, the mdsmon would go through with it nonetheless).
This is easily fixed by making sure that the mdsmon callbacks are only
awaken *after* the osdmon has been taken care of.
Fixes: #3495 Signed-off-by: Joao Eduardo Luis <joao.luis@inktank.com>
Samuel Just [Thu, 6 Dec 2012 01:05:38 +0000 (17:05 -0800)]
OSD: store current pg epoch in info and load at that epoch
Prior to split, this did not matter. With split, however, it's
crucial that a pg go through advance_pg() for the map causing
the split. During operation, a PG lags the OSD superblock
epoch. If the OSD dies after the OSD epoch passes the split
but before the pg epoch passes the split, the PG will be
reloaded at the OSD epoch and won't see the split operation.
The PG collection might after that point contain incorrect
objects which should have been split into a child.
projects / ceph.git / log