Sage Weil [Fri, 21 Oct 2011 16:56:19 +0000 (09:56 -0700)]
osd: simplify finalizing scrub on replica
We can simply call osr.flush() (with pg lock held) to ensure that prior
writes are visible and scrubbable. This avoids the funky handoff to
op_applied() (which didn't seem to work for me just now, although I didn't
fully debug.
Sage Weil [Fri, 21 Oct 2011 16:14:15 +0000 (09:14 -0700)]
osd: PriorSet: acting/up membership implies still alive
If the osd is in the acting or up sets, we can assume they are still alive,
even though we don't know that for sure, because if they are not, we will
rebuild PriorSet.
Note that we have a dependency here on up_thru that we could/should rebuild
PriorSet based on, IF we think it might change the value of the CRASHED
flag and IF we care enough. Right now we don't. Marking CRASHED when we
don't need to is conservative, and not dangerous.
Josh Durgin [Fri, 21 Oct 2011 00:13:21 +0000 (17:13 -0700)]
OSDMonitor: reweight towards average utilization
The existing reweight-by-utilization calculation did not take into
account the current weight of an OSD, and depended in part on the
threshold given by the user. Also send the user both the old and new
weights.
Sage Weil [Thu, 20 Oct 2011 22:46:11 +0000 (15:46 -0700)]
osd: PgPriorSet: kill whoami; make PG arg strictly optional
It is only used for the debug output prefix. Make it so we can leave it
out entirely (e.g. for unit tests).
We don't want to, say, pass in the string prefix itself, or else we are
stuck with generating that string even on low debug levels where it won't
be used.
Sage Weil [Thu, 20 Oct 2011 21:11:20 +0000 (14:11 -0700)]
osd: fix requeue_ops
The ls argument passed to requeue_ops() is a reference, and one of the
methods we call (say, _handle_op) might want to requeue the message on the
same list we were passed, leading to an infinite loop.
Set ls contents aside to avoid that.
Signed-off-by: Sage Weil <sage.weil@dreamhost.com>
Sage Weil [Thu, 20 Oct 2011 16:19:45 +0000 (09:19 -0700)]
perfcounters: use simple names
We don't need to uniquely identify ourselves in the global namespace with
the PerfCounter name.. only in the current process. Collectd will handle
the per-daemon naming part.
Signed-off-by: Sage Weil <sage.weil@dreamhost.com>
Sage Weil [Tue, 18 Oct 2011 21:12:06 +0000 (14:12 -0700)]
test_filestore_idempotent: simple tool to generate a worklaod of non-idempotent operations
Generate a workload of operations that are non-idempotent. These are:
transaction {
clone A -> A.($n-1)
write $n to A
}
$n++
loop!
If we apply any transaction to the file system more than once, we will
find that the A.$n object does not contain $n, but instead contains
some larger value.
First run in 'write' mode to generate a workload and fake a crash.
Then run in 'verify' mode to see if the result was bad.
Sage Weil [Tue, 18 Oct 2011 20:38:21 +0000 (13:38 -0700)]
mds: handle xattrs on inode creation
Allow mknod, mkdir, symlink, create to provide xattrs for the new
inode. This will be used by the kclient to set ACLs on new inodes
based on the parent directory.
Sage Weil [Tue, 18 Oct 2011 23:04:22 +0000 (16:04 -0700)]
radosgw-admin: fix conflict with KeyType in libnss
rgw/rgw_admin.cc:459:6: error: using typedef-name 'KeyType' after 'enum'
/usr/include/nss3/keythi.h:69:3: error: 'KeyType' has a previous declaration here
Sage Weil [Tue, 18 Oct 2011 18:40:06 +0000 (11:40 -0700)]
osd: PgPriorSet: restructure lost checks for prior set
When we add down osds to the cur set, we block peering because there
are OSDs that may have data we need and they are not currently up.
When that happens, marking those OSDs as lost may allow peering to
proceed.
Keep an explicit map blocked_by for exactly that set of OSDs (a subset
of cur), and compare lost_by values in prior_set_affected() to that.
Any single OSD from a given interval surviving is sufficient to ensure
that an ACKed write during that interval was committed to disk.
Currently, at least.
In any case, update the prior set calculation to reflect that. Also
make the survival conditional a bit smarter, to include both last_clean
interval (from the OSDs previous interval of up-ness) as well as the
current interval [up_from..up_thru).
Sage Weil [Tue, 18 Oct 2011 00:51:53 +0000 (17:51 -0700)]
osd: do not short-cut up_thru update for new PGs
Commit e731885d2550ee985bf875ab5bb5faf28f1693eb made it possible for
a new PG to go active without forcing the OSDs up_thru to update.
This was motivated by the desire for PG creation by radosgw to go
faster. Radosgw no longer creates a pool per bucket, so this is not
useful there, and it is unclear what other application (that is not
abusing rados pools) would need it.
Since it complicates the prior set calculation for dubious reasons,
let's revert it.
Sage Weil [Tue, 18 Oct 2011 00:44:32 +0000 (17:44 -0700)]
osd: PgPriorSet: revert start_since_joining check
Commit 5b78f5db8c200edcc949033e1badae70fecd2e08 added a check to
prevent some sort of badness when osds were marked lost, but I can't
figure out what it was. Remove the check for now until we can
reproduce/observe the badness in practice, and then write a test and
better motivated/docomented fix.
Sage Weil [Tue, 18 Oct 2011 00:02:23 +0000 (17:02 -0700)]
osd: PgPriorSet: do not include UP osds in prior.cur
The up osds are not (directly) relevant since they are not necessarily
members of the PG. We only care about acting OSDs, which may have
committed writes to the PG during this past interval.
The issue: we redo the prior set calculation if the up_thru for these
OSDs changes in the current map, but the prior_set result does not
depend on the current map's up_thru values in any way; it only depends
on the up_thru in the last epoch of each past interval, and that is
fixed in the past.
Sage Weil [Mon, 17 Oct 2011 15:50:54 +0000 (08:50 -0700)]
ceph.spec: work around build.opensuse.org
The redhat-rpm-config isn't installed on build.opensuse.org, which means
the processor is set to i386 instead of something less ancient. This
breaks compilation on 32-bit x86.
Sage Weil [Mon, 17 Oct 2011 03:37:28 +0000 (20:37 -0700)]
osd: fix add_next_event Missing::item::have
The missing set should be accurate up to the current point in the log. The
log_tail has no bearing on that, nor does last_update, since we're
processing new events in forward order, and updating missing as we go.
Sage Weil [Thu, 13 Oct 2011 19:47:57 +0000 (12:47 -0700)]
osd: all_unfound_are_queried_or_lost
The check to make isn't whether all locations are lost, but whether all
locations are either lost or have been queried and don't have the object(s)
we want.
Sage Weil [Wed, 12 Oct 2011 17:38:23 +0000 (10:38 -0700)]
cephtool: ability to send commands directly to osds
This makes commands beginning with 'tell <target>' magic in that they go
to the given target instead of to the monitor. This is slightly odd, but
I think it gives the most natural interface for the user, with the tool
Doing The Right Thing for you. E.g.,
ceph tell <someone> something (direct to some daemon)
ceph do something (goes to monitor to do X)
Sage Weil [Wed, 12 Oct 2011 00:51:07 +0000 (17:51 -0700)]
msg: add MCommand, MCommandReply message types
These are similar to MMonCommand[Ack], but aren't PaxosServiceMessage
children, don't include the command in the reply (useless), have a more
generic name.
Signed-off-by: Sage Weil <sage.weil@dreamhost.com>
Sage Weil [Fri, 14 Oct 2011 23:49:28 +0000 (16:49 -0700)]
filestore: assert on any unexpected error
Right now, the only errors we expect out of the underlying filesystem are
-ENOENT, -ENODATA, or (as a workaround for extN xattr suckage) -ENOSPC
for certain setxattr operations.
Samuel Just [Thu, 13 Oct 2011 21:35:13 +0000 (14:35 -0700)]
PG: Fix log.empty confusion
Previously, log.empty meant that the log.head was everion_t(). However,
it was in a few places used to mean that log.head == log.tail. Now,
log.empty means log.head == log.tail and log.null() indicates that
log.head is eversion_t().
Signed-off-by: Samuel Just <samuel.just@dreamhost.com>
Samuel Just [Thu, 13 Oct 2011 21:35:13 +0000 (14:35 -0700)]
PG: Fix log.empty confusion
Previously, log.empty meant that the log.head was everion_t(). However,
it was in a few places used to mean that log.head == log.tail. Now,
log.empty means log.head == log.tail and log.null() indicates that
log.head is eversion_t().
Signed-off-by: Samuel Just <samuel.just@dreamhost.com>
Interval tree is an optimized data structure for representing and
querying intervals. Elementary intervals are represented as nodes of an
avl tree and the corresponding data is stored on these nodes based on a
concept of span. This representation allows log(n) (where n is the
number of data) storage. The balanced avl tree allows a log(n) query.
The implementation is a template class that is instantiated based on
parameters : - Interval type - Data type
Signed-off-by: Jojy George Varghese <jvarghese@scalecomputing.com>
Sage Weil [Thu, 13 Oct 2011 16:53:41 +0000 (09:53 -0700)]
osd: bound generate_past_intervals() by oldest map
The oldest osdmap we maintain is a lower bound on last_epoch_clean for the
entire system (assuming the monitor is doing it's job right). We can stop
generating past intervals when we hit it.
Signed-off-by: Sage Weil <sage.weil@dreamhost.com>
Greg Farnum [Wed, 12 Oct 2011 23:37:55 +0000 (16:37 -0700)]
cls_rgw: rewrite rgw_bucket_complete_op to use update.
Unfortunately we can't do multiple writes via the interface -- the
second one will clobber the first one. So use the update functionality
and go through that pain instead.
projects / ceph.git / log