Loic Dachary [Sun, 31 Aug 2014 17:14:55 +0000 (19:14 +0200)]
mailmap: add .peoplemap
In .mailmap, an individual may be represented by multiple mails which
may reflect an organization affiliation change that needs to be
preserved to find out which organization sponsored which commit.
The .peoplemap file aggregates these mails to show a single line for
each person
Sage Weil [Sun, 31 Aug 2014 04:56:31 +0000 (21:56 -0700)]
common/LogEntry: fix warning on i386
common/LogEntry.cc: In member function 'void LogEntry::log_to_syslog(std::string, std::string)':
warning: common/LogEntry.cc:161:20: format '%lu' expects argument of type 'long unsigned int', but argument 4 has type 'uint64_t {aka long long unsigned int}' [-Wformat=]
msg.c_str());
^
Loic Dachary [Wed, 27 Aug 2014 15:05:02 +0000 (17:05 +0200)]
erasure-code: ECRecPred must be true if all chunks can be recovered
ECRecPred assumes recovering any number of chunk is possible as long as
at least K chunks are available. It builds the want() set accordingly in
the constructor and arbitrarily set the first K chunks. But it would be
the same if it set the last K chunks.
While this is correct for jerasure and isa plugins, it is not true in
general. The predicate should assume that all chunks are going to be
recovered and return true if they can all be recovered. Otherwise, the
following can happen:
* a PG has chunks 0,1,2,3 for K=2, M=2
* ECRecPred is initialized a set want to 0,1 because K=2
* ECRecPred claims the plugin can recover when provided 0,1
* the plugin is then required to recover 0,1 using 2,3 and fails
This can happen for the LRC plugin with k=4,m=2,l=3 which is
Loic Dachary [Sun, 8 Jun 2014 15:25:41 +0000 (17:25 +0200)]
erasure-code: test LRC pool, profile and crush ruleset
Using the command line to create an LRC pool validates that the plugin
can be loaded and the ruleset created immediately after the crush table
is populated with the default rulesets.
Add a test that takes out the first OSD holding data for an object and
checks the recovery is effective by retrieving the object content and
checking it is as expected.
Loic Dachary [Fri, 15 Aug 2014 16:33:38 +0000 (18:33 +0200)]
erasure-code: high level LRC configuration
Add a configuration mode to the LRC plugin, suitable for most use cases
and simpler to explain and understand. It uses k,m parameters similar to
the jerasure and isa plugins and adds l for locality. For instance:
It is less flexible because k+m must be a multiple of l and other
similar relationships and also because it cannot express recursive
configurations with locality in the datacenter and locality in a rack
within the datacenter.
Loic Dachary [Sun, 8 Jun 2014 12:19:18 +0000 (14:19 +0200)]
erasure-code: locally repairable code plugin
Recursively apply erasure code techniques so that recovering from the
loss of some chunks only require a subset of the available chunks, most
of the time.
Loic Dachary [Sun, 24 Aug 2014 18:52:24 +0000 (20:52 +0200)]
erasure-code: delegate chunk remapping to the plugin
Remapping the chunks after they have been encoded by the plugin is fine
if remapping is transparent to the plugin. However, the goal is to
delegate the remapping of the chunks to the plugin, for LRC in
particular. The chunks are therefore reordered according to the
directions found in the mapping array before they are passed to
decode_chunks.
Loic Dachary [Sun, 24 Aug 2014 18:57:16 +0000 (20:57 +0200)]
erasure-code: override the default only if the plugin match
When setting a new erasure coded profile, the key/value of the default
profile are re-used so that ruleset-failure-domain=osd can be overridden
like so:
ceph osd erasure-code-profile set ruleset-failure-domain=osd
This is however not appropriate if the plugin is different from the
defaut plugin (for instance LRC instead of jerasure) because the
parameters are not the same.
If the plugin is changed, the default profile is not reused.
John Spray [Fri, 29 Aug 2014 17:34:39 +0000 (18:34 +0100)]
tools: use cout instead of cerr in journal tool
Aside from being a bit odd to begin with, using stderr
was causing tests to fail because the output was polluted
by log output which is also on stderr.
Fixes: 9281 Signed-off-by: John Spray <john.spray@redhat.com>
Sage Weil [Fri, 29 Aug 2014 15:29:35 +0000 (08:29 -0700)]
mds/RecoveryQueue: do not start prioritized items synchronously
When we prioritize an item move it into a second priority list/set, but
do not start immediately, so that we still obey the max. When we go to
start an item, pull items first off the priority list, then off the regular
list.
John Spray [Thu, 28 Aug 2014 23:53:44 +0000 (00:53 +0100)]
client: fix dispatcher ordering (broken fuse)
Objecter never saw any OSD maps because of 1e1ee480 and
the dispatchers being in the wrong order -- ignoring map
in Client was hiding it from Objecter.
Fixes: #9266 Signed-off-by: John Spray <john.spray@redhat.com>
David Zafman [Wed, 20 Aug 2014 08:33:45 +0000 (01:33 -0700)]
ceph_objectstore_tool: Bug fixes and test improvements
ceph_objectgstore_tool:
Fix bugs in the way collection_list_partial() was being called
which caused objects to be seen over and over again.
Unit test:
Fix get_objs() to walk pg tree for pg with sub-directories
Create more objects to test object listing code
Limit number of larger objects
Limit number of objects which get attributes and omaps
David Zafman [Wed, 30 Jul 2014 19:39:49 +0000 (12:39 -0700)]
Complete replacement of ceph_filestore_tool and ceph_filestore_dump
with unified ceph_objectstore_tool
Move list-lost-objects and fix-lost-objects features from
ceph_filestore_tool to ceph_objectstore_tool as list-lost, fix-lost
Change --type to --op for info, log, export...operations
Add --type for the ObjectStore type (defaults to filestore)
Change --filestore-path to --data-path
Update installation, Makefile.am, and .gitignore
Fix and rename test case to match
Add some additional invalid option checks
Signed-off-by: David Zafman <david.zafman@inktank.com>
David Zafman [Wed, 21 May 2014 19:45:33 +0000 (12:45 -0700)]
test: ceph_filestore_dump.sh test improvements
Add some usage error tests
Don't use the same var in second for loop
Add xattr/omap to rep pool and xattr to ec pool
Add list, get-bytes and set-bytes testing
Add list-attrs and get-attr
Signed-off-by: David Zafman <david.zafman@inktank.com>
David Zafman [Wed, 14 May 2014 19:42:21 +0000 (12:42 -0700)]
common,ceph_filestore_dump: Add ability for utilities to suppress library dout output
Suppress dout output with CODE_ENVIRONMENT_UTILITY_NODOUT
ceph_filestore_dump turns on dout output if --debug specified
When used it can still be enable with --log-to-stderr --err-to-stderr
Signed-off-by: David Zafman <david.zafman@inktank.com>
Sage Weil [Thu, 14 Aug 2014 21:52:40 +0000 (14:52 -0700)]
mds/RecoveryQueue: add method to prioritize a file recovery; fix logging
Add a prioritize() method to make file recovery start immediately for the
given inode. Note that this doesn't respect the max recovery limit: if
someone stats it, they are blocking, and we start the recovery immediately.
Also fix up the dout logging a bit so that everything is prefixed
consistently.
Sage Weil [Thu, 14 Aug 2014 21:39:29 +0000 (14:39 -0700)]
mds: change mds_max_file_recover from 5 -> 32
These are reasonably cheap operations (stat) and we should be too worried
about queueing up a bunch of them.
Ideally this sort of thing would magically tune to the throughput we can
get from the cluster, but until then, let's choose a default that works for
more users.
Sage Weil [Thu, 28 Aug 2014 17:59:18 +0000 (10:59 -0700)]
test/mon/*: prime mon with initial command before injection
The osdmonitor_prepare_command is very fragile. Send an initial command
to the mon beforehand. This seems to prevent the initial command from
getting combined into an early mon proposal with some other stuff.
Alternatively, we could remove these tests and this mechanism entirely as
it is likely to great in the future when the next set of mon changes are
made, but they have shown themselves to be useful it catching other
regressions, so we'll patch them up for a bit longer.
projects / ceph.git / log