Loic Dachary [Thu, 7 May 2015 21:12:33 +0000 (23:12 +0200)]
tests: fail make check if nproc is too low
When running tests in parallel with make -jX, the ulimit -u (number of
processor / thread per user) needs to be at least X * 1024. If not it
will fail in mysterious ways. Since there is no convenient way to figure
out the value of X ( see
http://blog.jgc.org/2015/03/gnu-make-insanity-finding-value-of-j.html
for a non trivial an entertaining solution) add a very conservative
check that assumes the user will run make -jX where X is nproc / 2.
It will be annoying for users who want to run make check, not use -j,
and have a low ulimit -u. But the error suggest a way to override this
with
make CHECK_ULIMIT=false check
This is a minor irritation compared to the puzzling behavior of make
check when ulimit is exceeded.
John Spray [Thu, 7 May 2015 17:42:01 +0000 (18:42 +0100)]
client: fix error handling in check_pool_perm
Previously, on an error such as a pool not existing,
the caller doing the check would error out, but
anyone waiting on waiting_for_pool_perm would
block indefinitely (symptom was that reads on a
file with a bogus layout would block forever).
Fix by triggering the wait list on errors and
clear the CHECKING state so that the other callers
also perform the check and find the error.
Additionally, don't return the RADOS error code
up to filesystem users, because it can be
misleading. For example, nonexistent pool is
ENOENT, but we shouldn't give ENOENT on IO
to a file which does exist, we should give EIO.
Loic Dachary [Thu, 7 May 2015 08:30:23 +0000 (10:30 +0200)]
doc: clarify the release cycle and LTS meaning
* Explain the development cycle and stable release cycle
* Replace the 18 month duration with the LTS+2 overlap logic
* Explain that the estimated retirement is an educated guess
that depends on the stable release frequency
* Replace Support with Stable in LTS to avoid confusion with
professional services
* Do not promise bugs will be handled, only that they will be triaged
Jason Dillaman [Tue, 21 Apr 2015 14:15:58 +0000 (10:15 -0400)]
librbd: allow updates to snapshot object maps
The modified copyup behavior will need to update snapshot object
maps when performing a copyup since the object will now exist within
all snapshots of the image.
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
Jason Dillaman [Mon, 20 Apr 2015 16:12:05 +0000 (12:12 -0400)]
librados_test_stub: support AIO snapshot context
AIO operations can be executed with a different snapshot
context from the IoCtx. librbd takes advantage of this
capability to perform a deep object copyup.
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
Jason Dillaman [Thu, 16 Apr 2015 20:27:56 +0000 (16:27 -0400)]
librbd: copyup should use empty snapshot context
This allows child object to exist through all existing
snapshots as well as HEAD. If the operation was a CoW,
issue the write ops as a separate RADOS op with the
correct snapshot context.
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
Jason Dillaman [Thu, 16 Apr 2015 18:15:10 +0000 (14:15 -0400)]
librbd: move copyup class method call to CopyupRequest
Move AbstractWrite's invocation of copyup to the CopyupRequest
class. The AioRequest write path will now always create a
CopyupRequest, which will now append the actual write ops to the
copyup.
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
Jason Dillaman [Wed, 15 Apr 2015 18:25:08 +0000 (14:25 -0400)]
librbd: add new deep-flatten RBD feature
Copy-on-read/write will now use an empty snapshot context to
deep-copy the object through all existing snapshots. With the
deep-flatten feature enabled, the flatten operation will now
disconnect a child image from its parent even if it has existing
snapshots.
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
Jason Dillaman [Thu, 7 May 2015 01:28:25 +0000 (21:28 -0400)]
librbd: fast diff should treat all _EXISTS objects as dirty
With the fast-diff feature enabled, if the object map marks the
object as _EXISTS, it can only occur due to a write -- otherwise
it would be _EXISTS_CLEAN. Therefore, ensure it is properly
flagged as an updated object for diff_iterate.
Fixes: #11553 Signed-off-by: Jason Dillaman <dillaman@redhat.com>
Sage Weil [Wed, 6 May 2015 21:01:34 +0000 (14:01 -0700)]
librbd: fix warning
librbd/librbd.cc: In member function 'int librbd::Image::metadata_get(const string&, std::string*)':
librbd/librbd.cc:845:76: warning: suggest braces around empty body in an 'else' statement [-Wempty-body]
Jason Dillaman [Wed, 6 May 2015 18:48:50 +0000 (14:48 -0400)]
librbd: ignore lack of support for metadata on older OSDs
If an Infernalis librbd attempts to open an image stored on a
pre-Infernalis OSD, the new config metadata operations won't
be supported. This error can be safely ignored.
Fixes: #11549 Signed-off-by: Jason Dillaman <dillaman@redhat.com>
Tim Serong [Fri, 1 May 2015 15:59:53 +0000 (01:59 +1000)]
json_spirit: use utf8 intenally when parsing \uHHHH
When the python CLI is given non-ASCII characters, it converts them to
\uHHHH escapes in JSON. json_spirit parses these internally into 16 bit
characters, which could only work if json_spirit were built to use
std::wstring, which it isn't; it's using std::string, so the high byte
ends up being zero'd, leaving the low byte which is effectively garbage.
This hack^H^H^H^H change makes json_spirit convert to utf8 internally
instead, which can be stored just fine inside a std::string.
Note that this implementation still assumes \uHHHH escapes are four hex
digits, so it'll only cope with characters in the Basic Multilingual
Plane. Still, that's rather a lot more characters than it could cope
with before ;)
(For characters outside the BMP, Python seems to generate escapes in the
form \uHHHHHHHH, i.e. 8 hex digits, which the current implementation
doesn't expect to see)
Fixes: #7387 Signed-off-by: Tim Serong <tserong@suse.com>
Zhiqiang Wang [Wed, 6 May 2015 08:14:29 +0000 (16:14 +0800)]
librbd: fix the image format detection
If the detection of the old format fails with reasons other than
-ENOENT, we should return with this error. Otherwise, if we continue the
new format detection and fail with -ENOENT, the caller will get the
missleading failure information.
Signed-off-by: Zhiqiang Wang <zhiqiang.wang@intel.com>
Loic Dachary [Fri, 1 May 2015 14:46:48 +0000 (16:46 +0200)]
ceph-disk: use ceph-detect-init instead of hard coded values
The ceph-detect-init implements the same detection logic as ceph-deploy.
It is better than the current hardcoded values that assume Ubuntu is
upstart and all the rest is sysvinit.
Loic Dachary [Sat, 2 May 2015 15:17:24 +0000 (17:17 +0200)]
ceph-detect-init: integration with automake / make check
* Add the files to the distribution tarbal via EXTRA_DIST
* Build the module via the all-local target
* Add run-tox.sh to the tests run via make check
* Add manual page
It is meant to be used by both ceph-disk and ceph-deploy to avoid
duplicating the logic. Some operating systems implement more than one
init system and ceph-detect-init will not return the default one, it
will return the init system suitable for ceph deployment.
ceph-detect-init is implemented as a standalone python module suitable
to be published at https://pypi.python.org/pypi/ceph-detect-init, with
unit tests and integration tests (based on docker) for the following
platforms:
rados cli: add preventing using --block-size with bench seq and rand
Add handling of situation when '-b|--block-size' option is using with
'bench seq' or 'bench rand' which is erroneous according to the help message:
-b op_size
set the size of write ops for put or benchmarking
'trans_size' description from header: "size of the write/read to perform"
But really 'object_size' is used in write/read operations. 'trans_size' is used
only in ObjBencher::status_printer for calc current and average bandwidth.
As result - bad statistics in case 'trans_size' and and 'object_size' are different.
Ken Dreyer [Tue, 5 May 2015 17:25:46 +0000 (11:25 -0600)]
SubmittingPatches: clarify backport procedure
Developers should not add "Backport: " fields to Git commits, because
this data is immutable after the commits are merged. It makes more sense
to handle this information in Redmine instead.
Replace install-data-local: with install-data-local:: (two :) so that
other Makefile.am can have the same target and all get concatenated
instead of being overridden.
Loic Dachary [Sat, 2 May 2015 14:11:51 +0000 (16:11 +0200)]
build: make-debs.sh NPROC overrides make -j
Building a package is disk intensive and running make -j8 on a machine
with a spinner can actually be slower than make. NPROC=1 make-deb.sh
overrides the default value of make -jX.
Loic Dachary [Sat, 2 May 2015 13:59:12 +0000 (15:59 +0200)]
install-deps.sh: pip wheel for python dependencies
For all tox.ini in the source tree which also have requirement.txt
files, create a wheelhouse with the dependencies. This allows tox to
setup test environment with no access to the network with something
like (in tox.ini):
Haomai Wang [Tue, 5 May 2015 05:58:39 +0000 (13:58 +0800)]
AsyncConnection: Avoid lockdep detect failed
Previously we used a atomic field to avoid deadlock, but it still let lockdep
detect failed. So we remove the atomic field and unlock connection firstly.
Haomai Wang [Tue, 5 May 2015 05:53:23 +0000 (13:53 +0800)]
AsyncConnection: Don't dispatch event when connection is stopped
When marking down connection, previously we will call stop which will dispatch event
regardless of the staus of the connection. If this connection is already down and
its events all has cleaned, we will hit NULL event.
projects / ceph.git / log