The code moved from be_select_auth_object to be_compare_scrubmaps 74bd8708dfbfd3c8e7ba3f41d8534609dcbc1237 but the j iterator is use
differently although it has the same type. Use map.begin() as a
fallback instead.
Sage Weil [Mon, 22 Dec 2014 15:32:36 +0000 (07:32 -0800)]
osd: scrub: only assume shard digest == oi digest for replicated pools
For an EC object, the digest we get from scrub is for the *shard*, and that
is not the same as the *object* digest in the object_info_t. Skip these
checks; we already have the per-shard digest that is verified in the EC
backend.
Fixes: #10409 Signed-off-by: Sage Weil <sage@redhat.com>
Loic Dachary [Sun, 21 Dec 2014 13:22:31 +0000 (14:22 +0100)]
tests: avoid bash == --shell confusion in docker-test-helper.sh
Do not force interactive mode when the bash script is specified by the
user because it should be usable to run a small script instead of
starting an interactive shell session.
Loic Dachary [Sun, 21 Dec 2014 08:39:01 +0000 (09:39 +0100)]
tests: replace --enable-docker with --enable-root-make-check
The --enable-docker logic was that each test that needs privileges to
run uses a container to do so. The problem with this approach and
make -j8 check is that such tests compete for a single container.
The --enable-root-make-check activates tests that require privileges and
assume it is ok to use sudo to acquire privileges. The decision to run
in a container is then taken by the caller who will run something like:
docker-tests.sh ./configure --enable-root-make-check
docker-tests.sh make -j8 check
Loic Dachary [Sun, 21 Dec 2014 08:33:46 +0000 (09:33 +0100)]
tests: docker-test.sh commands are relative to git root
Instead of being in the src directory. Forcing the working directory to
src is convenient to run unittests individually without the need to
change directory, but it is confusing to the user.
Loic Dachary [Sat, 20 Dec 2014 22:28:28 +0000 (23:28 +0100)]
tests: ceph.spec.in or debian/control rebuild docker images
If any of the files in test/$os_type is newer than the creation date of
a docker image, the image is removed and rebuilt from scratch. For
instance, when a package is added to debian/control, the debian based
images are rebuilt.
Loic Dachary [Sat, 20 Dec 2014 17:20:58 +0000 (18:20 +0100)]
tests: docker images must use install-deps.sh
The dockerfiles must run install-deps.sh instead of duplicating the
package list that is found in ceph.spec.in and debian/control.
A directory is created for each os_type and provided as a context for
docker build to use. The former $os_type.dockerfile is moved into
$os_type/Dockerfile.in (the .in as a reminder that it will be variable
substituted).
Loic Dachary [Sat, 20 Dec 2014 18:30:59 +0000 (19:30 +0100)]
tests: run-make-check.sh install jq
jq is useful to parse json from the command line. It is however not
packaged for all supported distributions (precise has it in the backport
repository which is usually not active) and cannot be conveniently added
to debian/control because it targets all distributions and has no
conditionals.
Loic Dachary [Sat, 20 Dec 2014 17:57:59 +0000 (18:57 +0100)]
install-deps.sh: do not require sudo when root
If the user is root, do not use sudo. The sudo package is not installed
by default on centos by default and when building from script it may be
that root is running install-deps.sh on a freshly install distribution.
Sage Weil [Fri, 19 Dec 2014 19:48:27 +0000 (11:48 -0800)]
librados: add rados_watch_flush() call
Add a call so that callers can make sure all queued callbacks have
completed before shutting down the ioctx. This avoids a segv triggered
by the LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify2Timeout/1
test due to the ioctx being destroyed when the in-progress callback
does a notify_ack.
Sage Weil [Wed, 17 Dec 2014 00:39:35 +0000 (16:39 -0800)]
osd: scrub: wait for digest updates to apply before next scrub chunk
Wait for any digest updates to apply before we scrub the next chunk. This
bounds the number of repops we initiate by the size of the scrub chunk, and
it generally nicer to the cluster.
Sage Weil [Wed, 17 Dec 2014 00:11:50 +0000 (16:11 -0800)]
osd: change omap data encoding in object_copy_data_t
Pass the omap data in the struct as an opaque (encoded) bufferlist. This
avoids a decode into an STL map, simplifying the crc calculation. The
win isn't huge (yet) because we still turn it into a map to pass it down
to the ObjectStore method, but that too will change in time.
Sage Weil [Wed, 10 Dec 2014 17:32:50 +0000 (09:32 -0800)]
osd: scrub: set a min age before we update whole-object digest
If an object is being actively updated, the whole-object digest will
quickly be invalidated. On deep scrub, only record that digest if the
object is a few hours old. Otherwise, we are wasting an IO.
Sage Weil [Wed, 10 Dec 2014 00:36:15 +0000 (16:36 -0800)]
osd: scrub: disqualify shards with digests that disagree with oi digest
The auth may or may not know that the digest is truly bad. Note that we
can only relate scrub digests to those in oi if the scrub used a seed of
-1; older OSDs use 0.
Sage Weil [Wed, 10 Dec 2014 00:00:56 +0000 (16:00 -0800)]
osd: use -1 for deep scrub digest seed on new OSDs
0 is a weak initial value for a CRC since it doesn't change with a sequence
of 0 bytes (which are relatively common). -1 is better. Use -1 when
everyone in the acting set supports it.
Sage Weil [Mon, 3 Nov 2014 01:11:45 +0000 (17:11 -0800)]
osd/ReplicatedPG: set and invalidate data/omap digests on osd ops
When we create an empty object, we have a known digest of -1. In a few
cases, we also know the digest from the op (e.g., writefull). The rest
of the time we invalidate any existing digest.
Sage Weil [Fri, 19 Dec 2014 16:37:00 +0000 (08:37 -0800)]
osdc/Objecter: do notify completion callback in fast-dispatch context
The notify completion has exactly one user, the librados caller which
does nothing but take a local (inner) lock and signal a Cond. Do this
in the fast-dispatch context for simplicity.
Notably, this makes the notify completion (and timeout) trigger a
notify2() return (with ETIMEDOUT) even when the finisher queue that
normally delivers notify is busy.. for example with a notify that is
being very slow. In our case, the unit test is doing a sleep(3) to
test timeouts but also prevented the ETIMEDOUT notification from
being delivered to the caller. This patch resolves that.
Sage Weil [Fri, 19 Dec 2014 00:49:06 +0000 (16:49 -0800)]
osd: only verfy OSDMap crc if it is known
Only verify we encode a full map with the correct CRC if we actually
have the value in the Incremental. Otherwise, any map from an old
mon will fail. And we'll try to request a full map with a message the
old mon doesn't understand.
projects / ceph.git / log