John Spray [Wed, 17 Dec 2014 14:06:53 +0000 (14:06 +0000)]
tools/cephfs: add recover_dentries to journaltool
This is intended as a comparatively safe recovery
operation, where we compare the versions
of journalled dentries with backing store dentries,
and write into the backing store only when the
existing contents are older than the journal
or invalid.
Fixes: #9883 Signed-off-by: John Spray <john.spray@redhat.com>
Sage Weil [Mon, 19 Jan 2015 00:49:20 +0000 (16:49 -0800)]
mon: handle case where mon_globalid_prealloc > max_global_id
This triggers with the new larger mon_globalid_prealloc value. It didn't
trigger on the existing cluster I tested on because it already had a very
large max.
Sage Weil [Sun, 18 Jan 2015 18:39:25 +0000 (10:39 -0800)]
mon: change mon_globalid_prealloc to 10000 (from 100)
100 ids (session 100 authentications) can be consumed quite quickly if
the monitor is being queried by the CLI via scripts or on a large cluster,
especially if the propose interval is long (many seconds). These live in
a 64-bit value and are only "lost" if we have a mon election before they
are consumed, so there's no real risk here.
Backport: giant, firefly Reviewed-by: Joao Eduardo Luis <joao@redhat.com> Signed-off-by: Sage Weil <sage@redhat.com>
Yehuda Sadeh [Fri, 16 Jan 2015 01:30:24 +0000 (17:30 -0800)]
rgw: bilog marker related fixes
Fix the way we parse the marker. Instead of specifying whether it's a
sharded or not sharded bucket, we pass a shard_id. If string itself
points to a singe shard, we'll use the passed shard_id, otherwise we'll
parse the string and determine the shard id by that. In this way when
referencing a single shard we can get the marker with either shard id
specified or not. This works with the non-shard case too.
Adjust the bilog listing function, set it to work with the new
interface. It was broken before, and there are multiple fixes to it.
mon: mkfs compatset may be different from runtime compatset
When we create a monitor we set a given number of compat features on
disk to clearly state the features a given monitor supports -- mostly to
break backward compatibility when such compatibility cannot be
guaranteed.
However, we may wish to toggle some features during runtime; e.g., wait
for all the monitors in the quorum to support a given feature before
flipping a switch and state that all monitors now require feature X.
We are already flipping those switches during runtime, but we weren't
allowing the monitor to set a subset of those features during mkfs.
While the initial approach worked fine with clusters being upgraded and
fresh clusters, it could become weird in a mixed-version environment.
Backport: emperor,firefly,giant
Signed-off-by: Joao Eduardo Luis <joao@redhat.com>
Jianpeng Ma [Fri, 16 Jan 2015 08:14:17 +0000 (16:14 +0800)]
test: Using different filename for different test case.
Some test case use tmp file to test.But they used same file and create
in the same directory. If we do in parallel, it will cause error.
So different test case use own their tmp file.
Signed-off-by: Jianpeng Ma <jianpeng.ma@intel.com>
John Spray [Fri, 2 Jan 2015 17:48:25 +0000 (17:48 +0000)]
tools: create cephfs-table-tool
It was unnatural to shoehorn resetting tables
into the journaltool. This new tool initially
can simply dump or reset the session/snap/ino
tables, and would also be a place for any
more complex operations in future.
John Spray [Fri, 16 Jan 2015 00:00:56 +0000 (00:00 +0000)]
mds: abstract SessionMapStore from SessionMap
This is similar to what I did for InodeStore a while back:
introduce a logical separation between the persisted attributers
(and their encoding) and the live/runtime behavioural code. This
results in a handy SessionMapStore class that can be used for
encode/decode from tools.
Also give it a reset_state method so that it matches the
prototype of the MDSTable subclasses for the benefit of
cephfs-table-tool.
Loic Dachary [Tue, 6 Jan 2015 20:55:25 +0000 (21:55 +0100)]
erasure-code: tests use different pool/profile names
Use different erasure coded pool names and profiles to avoid deletion /
creation races. The more expensive alternative is to run a different
cluster for each test.
Add a new section to the PG troubleshooting section that covers the most
common problems reported when an erasure coded pool fails to properly
map PGs to enough OSDs.
Loic Dachary [Thu, 18 Dec 2014 00:25:54 +0000 (01:25 +0100)]
erasure-code: set max_size to chunk_count() instead of 20
The ruleset created for an erasure coded pool has max_size set to a
fixed value of 20, which may be incorrect when more than 20 chunks are
needed and lead to obscure errors. Set it to the number of chunks,
i.e. k+m most of the time.
In a cluster with few OSDs (9 for instance), setting max_size to 20
causes performance problems when injecting a new crushmap. The monitor
will call CrushTester::test which tries 1024 mappins for all sizes
ranging from min_size to max_size. Each attempt to map more OSDs than
available will exhaust all retries (50 by default) and it takes a
significant amount of time. In a cluster with 9 OSDs, testing one such
ruleset can take up to 5 seconds.
Since the test blocks the monitor leader, a few erasure coded rulesets
will block the monitor long enough to exceed the timeouts and trigger an
election.
Loic Dachary [Wed, 17 Dec 2014 15:06:55 +0000 (16:06 +0100)]
crush: set_choose_tries = 100 for erasure code rulesets
It is common for people to try to map 9 OSDs out of a 9 OSDs total ceph
cluster. The default tries (50) will frequently lead to bad mappings for
this use case. Changing it to 100 makes no significant CPU performance
difference, as tested manually by running crushtool on one million
mappings.
We no longer convert stores on upgrade. Users coming from bobtail or
before sould go through an interim version such as cuttlefish, dumpling,
firefly or giant.
Signed-off-by: Joao Eduardo Luis <joao@redhat.com>
ceph_mon: no longer attempt store conversion on start
People upgrading from bobtail or previous clusters should first go
through an interim version (quite a few to pick from: cuttlefish,
dumpling, firefly, giant).
Signed-off-by: Joao Eduardo Luis <joao@redhat.com>
Loic Dachary [Thu, 15 Jan 2015 12:23:47 +0000 (13:23 +0100)]
tests: adapt to new json-pretty format
The json-pretty format was modified for readability and now includes
additional newlines / spaces. Either switch to json to avoid dealing
with space changes or modify the expected output to include them.
Loic Dachary [Thu, 15 Jan 2015 11:28:12 +0000 (12:28 +0100)]
common: restore format fallback semantic
When Formatter::create replaced new_formatter, the handling of an
invalid format was also incorrectly changed. When an invalid format (for
instance "plain") was specified, new_formatter returned a NULL pointer
which was sometime handled by creating a json-pretty formatter and
sometimes differently.
A new Formatter::create prototype with a fallback argument is added and
is used if it is not the empty string and that the format is not
known. This prototype is used where new_formatter returning NULL was
replaced by a json-pretty formatter.
Josh Durgin [Wed, 14 Jan 2015 23:01:38 +0000 (15:01 -0800)]
qa: ignore duplicates in rados ls
These can happen with split or with state changes due to reordering
results within the hash range requested. It's easy enough to filter
them out at this stage.
Matt Benjamin [Sat, 10 Jan 2015 20:17:59 +0000 (15:17 -0500)]
Fix make check blockers.
Replace ceph-helpers.sh check for ms_nocrc with the new formula
for this. Fixes make check for default build.
Additionally, fix linkage of several unittests when building with
--enable-xio.
xio: add missing noinst headers
The common/address_helper.h file was not mentioned, also
msg/xio/XioSubmit.h.
Fix for Message.cc compilation error when Xio disabled.
Mention simple_dispatcher.h and xio_dispatcher.h in noinst_HEADERS.
xio: require boost-regex.
Make address_helper conditional on Xio.
This carries over to simple_client/simple_server,
for convenience.
Matt Benjamin [Tue, 30 Dec 2014 21:51:00 +0000 (16:51 -0500)]
Dequeue XioMsg on send-fail
If a message send hard fails, don't omit to remove it from the
send_q--this results in an assert when the queue is in safe-mode,
but with safe mode disabled, send_q would be corrupted.
Dont fall through and erase the iterator twice. Continue the loop,
as in the incoming release case.
projects / ceph.git / log