Yan, Zheng [Fri, 18 Jan 2013 14:54:02 +0000 (22:54 +0800)]
mds: move variables special to rename into MDRequest::more
My previous patches add two pointers (ambiguous_auth_inode and
auth_pin_freeze) to class Mutation. They are both used by cross
authority rename, both point to the renamed inode. Later patches
need add more rename special state to MDRequest, So just move them
into MDRequest::more
Yan, Zheng [Mon, 21 Jan 2013 02:04:03 +0000 (10:04 +0800)]
mds: don't journal opened non-auth inode
If we journal opened non-auth inode, during journal replay, the corresponding
entry will add non-auth objects to the cache. But the MDS does not journal all
subsequent modifications (rmdir,rename) to these non-auth objects, so the code
that manages cache and subtree may get confused. Besides non-auth objects will
be trimmed at the resolve stage.
Yan, Zheng [Wed, 16 Jan 2013 11:58:49 +0000 (19:58 +0800)]
mds: don't replace existing slave request
The MDS may receive a client request, but find there is an existing
slave request. It means other MDS is handling the same request, so
we should not replace the slave request with a new client request,
just forward the request.
The client request may include embeded cap releases, we need process
them even the request is forwarded.
Yan, Zheng [Wed, 16 Jan 2013 11:38:38 +0000 (19:38 +0800)]
mds: always use {push,pop}_projected_linkage to change linkage
Current code skips using {push,pop}_projected_linkage to modify replica
dentry's linkage. This confuses EMetaBlob::add_dir_context() and makes
it record out-of-date path when TO_ROOT mode is used. This patch changes
the code to always use {push,pop}_projected_linkage to modify dentry's
linkage. It makes sure MDCache::create_subtree_map() record correct and
up-to-date subtree map.
Yan, Zheng [Sat, 19 Jan 2013 01:49:04 +0000 (09:49 +0800)]
mds: send resolve messages after all MDS reach resolve stage
Current code sends resolve messages when resolving MDS set changes.
There is no need to send resolve messages when some MDS leave the
resolve stage. Sending message while some MDS are replaying is also
not very useful.
Yan, Zheng [Fri, 18 Jan 2013 11:41:48 +0000 (19:41 +0800)]
mds: split reslove into two sub-stages
The resolve stage serves to disambiguate the fate of uncommitted slave
updates and resolve subtrees authority. The MDS sends resolve message
that claims subtrees authority immediately when reslove stage is entered,
When receiving a resolve message, the MDS also processes it immediately.
This may cause problem if there are uncommitted slave rename and some of
them need rollback later. It's because slave rename rollback may modify
subtree map.
The fix is split reslove into two sub-stages, the first sub-stage serves
to disambiguate slave updates, do slave commit or rollback. After the
the first sub-stage finishes, the MDS sends resolve messages that claim
subtrees authority to other MDS and processes received resolve messages.
Yan, Zheng [Sat, 19 Jan 2013 05:00:29 +0000 (13:00 +0800)]
mds: fix slave rename rollback
The main issue of old slave rename rollback code is that it assumes
all affected objects are in the cache. The assumption is not true
when MDS does rollback in the resolve stage. This patch removes the
assumption and makes Server::do_rename_rollback() check individual
object and roll back change.
Yan, Zheng [Sat, 19 Jan 2013 04:57:31 +0000 (12:57 +0800)]
mds: preserve non-auth/unlinked objects until slave commit
The MDS should not trim objects in non-auth subtree immediately after
replaying a slave rename. Because the slave rename may require rollback
later and these objects are needed for rollback.
Yan, Zheng [Fri, 18 Jan 2013 06:08:45 +0000 (14:08 +0800)]
mds: force journal straydn for rename if necessary
rename may overwrite an empty directory inode and move it into stray
directory. MDS who has auth subtree beneath the overwrited directory
need journal the stray dentry when handling rename slave request.
Yan, Zheng [Fri, 18 Jan 2013 02:47:21 +0000 (10:47 +0800)]
mds: fix "had dentry linked to wrong inode" warning
The reason of "had dentry linked to wrong inode" warning is that
Server::_rename_prepare() adds the destdir to the EMetaBlob before
adding the straydir. So during MDS recovers, the destdir is first
replayed. The old inode is directly replaced by the source inode.
We can void the warning by adding the straydir first.
Yan, Zheng [Sat, 19 Jan 2013 00:30:23 +0000 (08:30 +0800)]
mds: don't set xlocks on dentries done when early reply rename
_rename_finish() does not send dentry link/unlink message to replicas.
We should prevent dentries that are modified by the rename operation
from getting new replicas while the rename operation is committing.
So don't set xlocks on dentries "done".
Yan, Zheng [Sun, 6 Jan 2013 01:15:55 +0000 (09:15 +0800)]
mds: properly set error_dentry for discover reply
If MDCache::handle_discover() receives an 'discover path' request but
can not find the base inode. It should properly set the 'error_dentry'
to make sure MDCache::handle_discover_reply() checks correct object's
wait queue.
Yan, Zheng [Fri, 4 Jan 2013 02:36:50 +0000 (10:36 +0800)]
mds: introduce XSYN to SYNC lock state transition
If lock is in XSYN state, Locker::simple_sync() firstly try changing
lock state to EXCL. If it fail to change lock state to EXCL, it just
returns. So Locker::simple_sync() does not guarantee the lock state
eventually changes to SYNC. This issue can cause replica that requests
read lock hang. The fix is introduce an intermediate state for XSYN
to SYNC transition.
Yan, Zheng [Thu, 17 Jan 2013 07:29:21 +0000 (15:29 +0800)]
mds: allow journaling multiple root inodes in EMetaBlob
In some cases (rename, rmdir, subtree map), we may need journal multiple
root inodes (/, mdsdir) in one EMetaBlob. This patch modifies EMetaBlob
format to support journaling multiple root inodes.
Yan, Zheng [Sat, 5 Jan 2013 02:07:11 +0000 (10:07 +0800)]
mds: lock remote inode's primary dentry during rename
commit 1203cd2110 (mds: allow open_remote_ino() to open xlocked dentry)
makes Server::handle_client_rename() xlocks remote inodes' primary
dentry so witness MDS can open xlocked dentry. But I added remote inodes'
projected primary dentries to the xlock list. This is wrong because
projected dentries are invisible for path traverse.
Yan, Zheng [Thu, 17 Jan 2013 06:50:44 +0000 (14:50 +0800)]
mds: check deleted directory in Server::rdlock_path_xlock_dentry
Commit b03eab22e4 (mds: forbid creating file in deleted directory)
is not complete, mknod, mkdir and symlink are missed. Move the ckeck
into Server::rdlock_path_xlock_dentry() fixes the issue.
Yan, Zheng [Fri, 11 Jan 2013 07:46:59 +0000 (15:46 +0800)]
mds: fix end check in Server::handle_client_readdir()
commit 1174dd3188 (don't retry readdir request after issuing caps)
introduced an bug that wrongly marks 'end' in the the readdir reply.
The code that touches existing dentries re-uses an iterator, and the
iterator is used for checking if readdir is end.
Dan Mick [Tue, 30 Oct 2012 21:02:53 +0000 (14:02 -0700)]
rbd-fuse: add simple RBD FUSE client
Currently written in C on FUSE hi-level interfaces, so error reporting
could be better. No serious work done for performance. But it's
usable as it stands.
Specify -c <conf> and a mountpoint, and images show up as files in
that mountpoint. You can create new images; they'll be created
with attributes stored in xattrs:
Images may be truncated or extended by rewriting. Currently
once an image is opened, it's not closed, so it can't be deleted
or changed outside of the fuse path.
Dan Mick [Sat, 26 Jan 2013 05:22:45 +0000 (21:22 -0800)]
s3/php: update to 1.5? version of API
Something like v1.5 of the Amazon PHP library requires the AmazonS3
constructor to be given an array of parameters rather than using
the globals. More research needs to happen, and particularly
about the v2 API, but this might solve someone's problem with
v1.5 while we do that research.
Ross Turk [Fri, 25 Jan 2013 20:48:31 +0000 (12:48 -0800)]
doc: wider sidebar, larger font, cleaned tip CSS
The sidebar is now about a hundred pixels wider and the fonts
are larger throughout. This works a lot better when you get
deep into the doc structure - it used to wrap horribly.
I also fixed how literals look inside .tip and .important.
For some reason, the lookup() retry loop (for when happened to
race with a removal and grab an invalid WeakPtr) locked
the lock again. This causes the #3836 crash since the lock
is already locked. It's rare since it requires a lookup between
invalidation of the WeakPtr and removal of the WeakPtr entry.
Fixes: #3836
Backport: bobtail Signed-off-by: Samuel Just <sam.just@inktank.com>
Sage Weil [Fri, 25 Jan 2013 17:29:37 +0000 (09:29 -0800)]
osd: share incoming maps via Connection*, not addrs
Kill a set of parallel methods that are using the old addr/inst-based
msgr APIs, and instead use Connection handles. This is much safer and gets
us closer to killing the old msgr API.
Sage Weil [Fri, 25 Jan 2013 17:27:00 +0000 (09:27 -0800)]
osd: pass new maps to dead osds via existing Connection
Previously we were sending these maps to dead osds via their old addrs
using a new outgoing connection and setting the flags so that the msgr
would clean up. That mechanism is possibly buggy and fragile, and we can
avoid it entirely if we just reuse the existing heartbeat Connection.
Sage Weil [Fri, 25 Jan 2013 17:25:28 +0000 (09:25 -0800)]
osd: requeue osdmaps on heartbeat connections for cluster connection
If we receive an OSDMap on the cluster connection, requeue it for the
cluster messenger, and process it there where we normally do. This avoids
any concerns about locking and ordering rules.
The allowance is not only added for btrfs as of commit e639254a0c5f8e3528fa8f2b2b451296653556bc, which makes us happy
for both non-btrfs (lower latency) and btrfs (better small io
throughput, no big stall during commit).
Sage Weil [Thu, 24 Jan 2013 06:16:49 +0000 (22:16 -0800)]
os/FileStore: only adjust up op queue for btrfs
We only need to adjust up the op queue limits during commit for btrfs,
because the snapshot initiation (async create) is currently
high-latency and the op queue is quiesced during that period.
This lets us revert 44dca5c, which disabled the extra allowance because
it is generally bad for non-btrfs writeahead mode.
Samuel Just [Thu, 24 Jan 2013 19:07:37 +0000 (11:07 -0800)]
OSD: use TPHandle in peering_wq
Implement _process overload with TPHandle argument and use
that to ping the hb map between pgs and between map epochs
when advancing a pg. The thread will still timeout if
genuinely stuck at any point.
Fixes: 3905
Backport: bobtail Signed-off-by: Samuel Just <sam.just@inktank.com>
Samuel Just [Wed, 23 Jan 2013 20:15:10 +0000 (12:15 -0800)]
ReplicatedPG: start scanning omap at omap_recovered_to
Previously, we started scanning omap after omap_recovered_to.
This is a problem since the break in the loop implies that
omap_recovered_to is the first key not recovered.
Backport: bobtail Signed-off-by: Samuel Just <sam.just@inktank.com>
Sage Weil [Wed, 23 Jan 2013 02:08:22 +0000 (18:08 -0800)]
os/FileStore: add stall injection into filestore op queue
Allow admin to artificially induce a stall in the op queue. Forces the
thread(s) to sleep for N seconds. We pause for 1 second increments and
recheck the value so that a previously stalled thread can be unwedged by
reinjecting a lower value (or 0). To stall indefinitely, just injust
very large number.
Sage Weil [Wed, 23 Jan 2013 02:01:07 +0000 (18:01 -0800)]
osd: hold lock while calling start_boot on startup
This probably doesn't strictly matter because start_boot doesn't need the
lock (currently) and few other threads should be running, but it is
better to be consistent.
Sage Weil [Wed, 23 Jan 2013 01:56:32 +0000 (17:56 -0800)]
osd: do not reply to ping if internal heartbeat is not healthy
If we find that our internal threads are stalled, do not reply to ping
requests. If we do this long enough, peers will mark us down. If we are
only transiently unhealthy, we will reply to the next ping and they will
be satisfied. If we are unhealthy and marked down, and eventually recover,
we will mark ourselves back up.
David Zafman [Wed, 9 Jan 2013 03:24:13 +0000 (19:24 -0800)]
osd: Add digest of omap for deep-scrub
Add ScrubMap encode/decode v4 message with omap digest
Compute digest of header and key/value. Use bufferlist
to reflect structure and compute as we go, clearing
bufferlist to reduce memory usage.
Signed-off-by: David Zafman <david.zafman@inktank.com> Reviewed-by: Samuel Just <sam.just@inktank.com>
projects / ceph.git / log