]> git.apps.os.sepia.ceph.com Git - ceph.git/log
ceph.git
9 years agolibrbd: don't cancel request lock early 5319/head
Jason Dillaman [Fri, 15 May 2015 15:18:29 +0000 (11:18 -0400)]
librbd: don't cancel request lock early

It's possible that a stale notice is received and will
be discarded after the request lock has been canceled.
This will stale the client.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit d9dd5c5890029107df40ed438f23fb9865d7de29)

9 years agotests: new test for transitioning exclusive lock
Jason Dillaman [Thu, 14 May 2015 20:13:38 +0000 (16:13 -0400)]
tests: new test for transitioning exclusive lock

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit f97ce46ea4c9c7cdc6e37e3759871d5b41b25769)

9 years agotests: verify that librbd will periodically resend lock request
Jason Dillaman [Thu, 7 May 2015 17:31:50 +0000 (13:31 -0400)]
tests: verify that librbd will periodically resend lock request

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit d2a1c226ab3f7e202f62896f0c80c4cf3607cdaf)

9 years agocommon: Mutex shouldn't register w/ lockdep if disabled
Jason Dillaman [Fri, 15 May 2015 14:49:36 +0000 (10:49 -0400)]
common: Mutex shouldn't register w/ lockdep if disabled

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 879b8a73e2452332b26b8f3428ff5e3e0af8ddad)

9 years agolibrbd: improve debugging output for ImageWatcher
Jason Dillaman [Fri, 15 May 2015 14:47:04 +0000 (10:47 -0400)]
librbd: improve debugging output for ImageWatcher

Include the instance pointer so that different images
can be differentiated in the logs.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit b951a7398e3e749cb64a3f6a604212a5627e82a0)

9 years agolibrados_test_stub: watcher id should be the instance id (gid)
Jason Dillaman [Fri, 15 May 2015 14:45:04 +0000 (10:45 -0400)]
librados_test_stub: watcher id should be the instance id (gid)

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 3e1e561cc29043eb64dff8afb3a2c1ed77788e93)

9 years agolibrbd: retry lock requests periodically until acquired
Jason Dillaman [Thu, 7 May 2015 16:51:49 +0000 (12:51 -0400)]
librbd: retry lock requests periodically until acquired

If the exclusive lock owner acks the lock release request but crashes
before it actually releases the lock, the requestor will wait forever.
Therefore, after a certain timeout, retry the request again until it
succeeds.

Fixes: #11537
Backport: hammer
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 37c74e6e5274208d1b6efaf315afec03ea7eaa82)

9 years agolibrbd: don't hold owner_lock for write during flush
Jason Dillaman [Thu, 7 May 2015 16:35:36 +0000 (12:35 -0400)]
librbd: don't hold owner_lock for write during flush

The various IO callback codepaths will attempt to take
the lock, which will result in deadlock since the flush
cannot complete.

Backport: hammer
Fixes: #11537
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 2b6d0633d5c89de3a557cdd72621dfc19f0540de)

9 years agolockdep: do not automatically collect all backtraces 5296/head
Jason Dillaman [Mon, 22 Jun 2015 19:30:02 +0000 (15:30 -0400)]
lockdep: do not automatically collect all backtraces

It is expensive to collect backtraces every time a lock is
checked in order to provide cycle backtraces.  The backtraces
can be forced on for specific locks or globally via the new
config option "lockdep_force_backtrace".

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 7354d25f56c5e004f288febdee2b6961c069163b)

9 years agolibrbd: flush operations need to acquire owner lock
Jason Dillaman [Tue, 9 Jun 2015 17:20:54 +0000 (13:20 -0400)]
librbd: flush operations need to acquire owner lock

Cache writeback operations will expect the owner lock to be held.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit adfa2e0070ccca7b6556d3bfc5fac7ce4d43a4d0)

9 years agolibrbd: avoid infinite loop if copyup fails
Jason Dillaman [Thu, 21 May 2015 04:13:31 +0000 (00:13 -0400)]
librbd: avoid infinite loop if copyup fails

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 43e0e3cd63f0067217ed0811d73f6c546f3027be)

9 years agolibrbd: flush pending ops while not holding lock
Jason Dillaman [Tue, 12 May 2015 14:19:48 +0000 (10:19 -0400)]
librbd: flush pending ops while not holding lock

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 3d5cef38c37e5dda6b23751ad560851f1304d86d)

9 years agotests: fix possible deadlock in librbd ImageWatcher tests
Jason Dillaman [Tue, 12 May 2015 14:07:21 +0000 (10:07 -0400)]
tests: fix possible deadlock in librbd ImageWatcher tests

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 742a85d667b26b3490d96270b5c500b08f2a5283)

9 years agotests: enable lockdep for librbd unit tests
Jason Dillaman [Mon, 11 May 2015 17:59:49 +0000 (13:59 -0400)]
tests: enable lockdep for librbd unit tests

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 45cb9cb5980e90235b8244d0d61ece40bde4784a)

9 years agolibrbd: owner_lock should be held during flush request
Jason Dillaman [Thu, 7 May 2015 18:17:37 +0000 (14:17 -0400)]
librbd: owner_lock should be held during flush request

Flush might result in the cache writing out dirty objects, which
would require that the owner_lock be held.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit c9142fe35372cf69b7a56f334622a775a6b7c43f)

9 years agoosdc: ObjectCacher flusher might needs additional locks
Jason Dillaman [Thu, 7 May 2015 18:06:16 +0000 (14:06 -0400)]
osdc: ObjectCacher flusher might needs additional locks

librbd requires the ObjectCacher flusher thread to acquire
an additional lock in order to maintain lock ordering
constraints.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit a38f9e5104a6e08e130dc4f15ad19a06d9e63719)

9 years agolibrbd: fix recursive locking issues
Jason Dillaman [Thu, 30 Apr 2015 20:11:03 +0000 (16:11 -0400)]
librbd: fix recursive locking issues

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 1b57cc1da7a51e6f8ffea689b94ef843732c20a4)

9 years agolibrbd: simplify state machine handling of exclusive lock
Jason Dillaman [Thu, 30 Apr 2015 20:04:28 +0000 (16:04 -0400)]
librbd: simplify state machine handling of exclusive lock

It is expected that all IO is flushed and all async ops are cancelled
prior to releasing the exclusive lock.  Therefore, replace handling of
lost exclusive locks in state machines with an assertion.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit d6b733dbdd0aeb5d1e136dcbf30c58c80952651e)

9 years agolibrbd: ObjectMap::aio_update can acquire snap_lock out-of-order
Jason Dillaman [Thu, 30 Apr 2015 19:32:38 +0000 (15:32 -0400)]
librbd: ObjectMap::aio_update can acquire snap_lock out-of-order

Detected during an fsx run where a refresh and CoR were occurring
concurrently.  The refresh held the snap_lock and was waiting on
the object_map_lock, while the CoR held object_map_lock and was
waiting for snap_lock.

Fixes: #11577
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 8cbd92b1fe835b1eb3a898976f9507f51cc115b2)

9 years agolibrbd: move copyup class method call to CopyupRequest
Jason Dillaman [Thu, 16 Apr 2015 18:15:10 +0000 (14:15 -0400)]
librbd: move copyup class method call to CopyupRequest

Move AbstractWrite's invocation of copyup to the CopyupRequest
class.  The AioRequest write path will now always create a
CopyupRequest, which will now append the actual write ops to the
copyup.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 7be3df67809925164237cc185f9f29e145f45768)

9 years agolibrbd: simplify AioRequest constructor parameters
Jason Dillaman [Tue, 31 Mar 2015 16:28:11 +0000 (12:28 -0400)]
librbd: simplify AioRequest constructor parameters

Moved all parent overlap computation to within AioRequest so that
callers don't need to independently compute the overlap.  Also
removed the need to pass the snap_id for write operations since
it can only be CEPH_NOSNAP.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 46515971edab8888284b1a8307ffca88a9c75f5c)

9 years agolibrbd/AioRequest.h: fix UNINIT_CTOR
Danny Al-Gaaf [Sat, 14 Mar 2015 00:16:31 +0000 (01:16 +0100)]
librbd/AioRequest.h: fix UNINIT_CTOR

Fix for:

CID 1274319: Uninitialized scalar field (UNINIT_CTOR)
 uninit_member: Non-static class member m_object_state is not
 initialized in this constructor nor in any functions that it calls.

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>
(cherry picked from commit 48f18ea0e1c4c6de1921ea2359448deb761461e7)

9 years agolibrbd: add object state accessor to ObjectMap
Jason Dillaman [Fri, 17 Jul 2015 19:04:10 +0000 (15:04 -0400)]
librbd: add object state accessor to ObjectMap

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
9 years agolibrbd: AsyncObjectThrottle should always hold owner_lock
Jason Dillaman [Thu, 30 Apr 2015 19:41:59 +0000 (15:41 -0400)]
librbd: AsyncObjectThrottle should always hold owner_lock

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit c352bcdc0f63eea55677fe3c3b5f0c61347c0db3)

9 years agolibrbd: execute flush completion outside of cache_lock
Jason Dillaman [Thu, 30 Apr 2015 19:34:43 +0000 (15:34 -0400)]
librbd: execute flush completion outside of cache_lock

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 5f157f20980de7e9a05fb933fb57efdc759da78b)

9 years agolibrbd: add AsyncRequest task enqueue helper method
Jason Dillaman [Thu, 30 Apr 2015 19:17:54 +0000 (15:17 -0400)]
librbd: add AsyncRequest task enqueue helper method

In order to support the invariant that all state machine
callbacks occur without holding locks, transitions that
don't always involve a librados call should queue their
callback.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 218bc2d0f8d90c9c64408cb22e26680e88138844)

9 years agolibrbd: disable lockdep on AioCompletion
Jason Dillaman [Thu, 7 May 2015 19:32:27 +0000 (15:32 -0400)]
librbd: disable lockdep on AioCompletion

It is only used by clients and it causes a large slowdown
in performance due to the rate at which the lock is constructed/
destructed for each IO request.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 66e74641baeed9aadb7e474d6e6b142b7725722d)

9 years agolibrbd: AioCompletion shouldn't hold its lock during callback
Jason Dillaman [Thu, 30 Apr 2015 17:42:19 +0000 (13:42 -0400)]
librbd: AioCompletion shouldn't hold its lock during callback

The callback routine most likely will attempt to retrieve the result
code, which will result in a recursive lock attempt.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 3ad19ae349ebb27ff6027e40883735240fa97a3c)

9 years agolibrbd: give locks unique names to prevent false lockdep failures
Jason Dillaman [Thu, 30 Apr 2015 17:51:03 +0000 (13:51 -0400)]
librbd: give locks unique names to prevent false lockdep failures

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit c474ee42b87975c04eeb2b40b976deb5a5e2d518)

9 years agolibrbd: complete cache read in a new thread context
Jason Dillaman [Thu, 30 Apr 2015 17:40:16 +0000 (13:40 -0400)]
librbd: complete cache read in a new thread context

The ObjectCacher complete the read callback while still holding
the cache lock.  This introduces lock ordering issues which are
resolved by queuing the completion to execute in a clean (unlocked)
context.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 0024677dc8adfd610418ca47599dd95d3a5a6612)

9 years agolibrbd: require callers to ObjectMap::aio_update to acquire lock
Jason Dillaman [Thu, 19 Mar 2015 19:35:59 +0000 (15:35 -0400)]
librbd: require callers to ObjectMap::aio_update to acquire lock

This is needed to allow an atomic compare and update operation
from the rebuild object map utility.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 2db758cb4cb6d88cbaf9842b1e5d0872a185d8df)

9 years agolog: fix helgrind warnings regarding possible data race
Jason Dillaman [Thu, 30 Apr 2015 17:38:29 +0000 (13:38 -0400)]
log: fix helgrind warnings regarding possible data race

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit c1e14451775049bbd95bcb19a0b62ab5e2c0a7bb)

9 years agolibrados_test_stub: fix helgrind warnings
Jason Dillaman [Thu, 30 Apr 2015 17:37:56 +0000 (13:37 -0400)]
librados_test_stub: fix helgrind warnings

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit b65ae4bc26f2dcaaa9518d5cce0e8b83ea310de8)

9 years agolibrados_test_stub: add support for flushing watches
Jason Dillaman [Thu, 30 Apr 2015 17:36:26 +0000 (13:36 -0400)]
librados_test_stub: add support for flushing watches

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 6e400b9049ede5870e40e4dd2cb41874550eac25)

9 years agocommon: lockdep now support unregistering once destructed
Jason Dillaman [Thu, 30 Apr 2015 17:29:12 +0000 (13:29 -0400)]
common: lockdep now support unregistering once destructed

librbd use of an image hierarchy resulted in lock names being
re-used and incorrectly analyzed.  librbd now uses unique lock
names per instance, but to prevent an unbounded growth of
tracked locks, we now remove lock tracking once a lock is
destructed.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 7c7df2ce9f837628535d21df61ae4f13d809c4fa)

9 years agocommon: add valgrind.h convenience wrapper
Jason Dillaman [Thu, 30 Apr 2015 17:26:41 +0000 (13:26 -0400)]
common: add valgrind.h convenience wrapper

Conditionally support helgrind annotations if valgrind support is
enabled during the build.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 650ad32d74a24d248dd8e19bff1bbd31f0bb224b)

9 years agolibrbd: add work queue for op completions
Jason Dillaman [Wed, 8 Apr 2015 21:24:08 +0000 (17:24 -0400)]
librbd: add work queue for op completions

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 21f990efe6c6d710376d5b1a49fc8cd53aa020e6)

9 years agoWorkQueue: ContextWQ can now accept a return code
Jason Dillaman [Wed, 8 Apr 2015 20:46:34 +0000 (16:46 -0400)]
WorkQueue: ContextWQ can now accept a return code

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit e5ffae578e83c1e4fca3f328c937e7c9be8cc03c)

9 years agoMerge pull request #5265 from SUSE/wip-12368-hammer
Loic Dachary [Tue, 28 Jul 2015 20:30:23 +0000 (22:30 +0200)]
Merge pull request #5265 from SUSE/wip-12368-hammer

linking ceph to tcmalloc causes segfault on SUSE SLE11-SP3

Reviewed-by: Loic Dachary <ldachary@redhat.com>
9 years agoMerge pull request #5280 from ceph/wip-12384-hammer
Loic Dachary [Tue, 28 Jul 2015 20:27:40 +0000 (22:27 +0200)]
Merge pull request #5280 from ceph/wip-12384-hammer

librbd: add valgrind memory checks for unit tests

Reviewed-by: Loic Dachary <ldachary@redhat.com>
9 years agoMerge pull request #5279 from ceph/wip-12237-hammer
Loic Dachary [Tue, 28 Jul 2015 20:26:25 +0000 (22:26 +0200)]
Merge pull request #5279 from ceph/wip-12237-hammer

A client opening an image mid-resize can result in the object map being invalidated

Reviewed-by: Loic Dachary <ldachary@redhat.com>
9 years agoMerge pull request #5283 from SUSE/wip-12397-hammer
Loic Dachary [Tue, 28 Jul 2015 20:10:03 +0000 (22:10 +0200)]
Merge pull request #5283 from SUSE/wip-12397-hammer

ceph.spec.in: 95-ceph-osd.rules, mount.ceph, and mount.fuse.ceph not installed properly on SUSE

Reviewed-by: Loic Dachary <ldachary@redhat.com>
9 years agoMerge pull request #5206 from SUSE/wip-11998-hammer
Loic Dachary [Tue, 28 Jul 2015 19:54:33 +0000 (21:54 +0200)]
Merge pull request #5206 from SUSE/wip-11998-hammer

/usr/bin/ceph from ceph-common is broken without installing ceph

Reviewed-by: Loic Dachary <ldachary@redhat.com>
9 years agoMerge pull request #5055 from SUSE/wip-12044-hammer
Loic Dachary [Tue, 28 Jul 2015 19:47:29 +0000 (21:47 +0200)]
Merge pull request #5055 from SUSE/wip-12044-hammer

rgw/logrotate.conf calls service with wrong init script name

Reviewed-by: Loic Dachary <ldachary@redhat.com>
9 years agoMerge pull request #5040 from SUSE/wip-11964-hammer
Loic Dachary [Tue, 28 Jul 2015 19:46:11 +0000 (21:46 +0200)]
Merge pull request #5040 from SUSE/wip-11964-hammer

systemd: Increase max files open limit for OSD daemon

Reviewed-by: Loic Dachary <ldachary@redhat.com>
9 years agoMerge pull request #5038 from SUSE/wip-11876-hammer
Loic Dachary [Tue, 28 Jul 2015 19:45:44 +0000 (21:45 +0200)]
Merge pull request #5038 from SUSE/wip-11876-hammer

ceph-post-file fails on rhel7

Reviewed-by: Loic Dachary <ldachary@redhat.com>
9 years agoMerge pull request #5030 from SUSE/wip-12092-hammer
Loic Dachary [Tue, 28 Jul 2015 19:45:25 +0000 (21:45 +0200)]
Merge pull request #5030 from SUSE/wip-12092-hammer

packaging: add SuSEfirewall2 service files

Reviewed-by: Loic Dachary <ldachary@redhat.com>
9 years agoMerge pull request #5028 from SUSE/wip-12090-hammer
Loic Dachary [Tue, 28 Jul 2015 19:38:14 +0000 (21:38 +0200)]
Merge pull request #5028 from SUSE/wip-12090-hammer

rcceph script is buggy

Reviewed-by: Loic Dachary <ldachary@redhat.com>
9 years agoMerge pull request #5026 from SUSE/wip-12087-hammer
Loic Dachary [Tue, 28 Jul 2015 19:37:44 +0000 (21:37 +0200)]
Merge pull request #5026 from SUSE/wip-12087-hammer

max files open limit for OSD daemon is too low

Reviewed-by: Loic Dachary <ldachary@redhat.com>
10 years agolibrados_test_stub: read op should return number of bytes read 5279/head
Jason Dillaman [Sun, 5 Jul 2015 14:47:38 +0000 (10:47 -0400)]
librados_test_stub: read op should return number of bytes read

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit f8a7b507983e31399831e802e99429b95386ed41)

10 years agotests: fixed TestObjectMap.InvalidateFlagInMemoryOnly
Jason Dillaman [Sun, 5 Jul 2015 14:35:28 +0000 (10:35 -0400)]
tests: fixed TestObjectMap.InvalidateFlagInMemoryOnly

librados and librados_test_stub return different result codes
for a read full object operation.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 2ace2b77f8ed83e753fe4a48bcc997f5d1dd465f)

10 years agolibrbd: don't attempt to invalidate an object map in R/O mode
Jason Dillaman [Sun, 5 Jul 2015 15:09:09 +0000 (11:09 -0400)]
librbd: don't attempt to invalidate an object map in R/O mode

The ImageWatcher is not initialized when in R/O mode, which
resulted in a NULL pointer dereference.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 64d740f8fa10ba872e324ec2580a4d8c3f99a9ce)

10 years agotests: add new unit tests for object map invalidation
Jason Dillaman [Tue, 23 Jun 2015 15:17:12 +0000 (11:17 -0400)]
tests: add new unit tests for object map invalidation

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 0215e9753c09460f6fc84ded9397e36a209f2e32)

10 years agolibrbd: move object map codes to common location
Jason Dillaman [Wed, 25 Mar 2015 13:41:13 +0000 (09:41 -0400)]
librbd: move object map codes to common location

These codes will need to be accessible from cls_rbd and librbd.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 4ac584c34d576b489ed4c4862703b8fb427b3bc2)

10 years agolibrbd: only update image flags when holding exclusive lock
Jason Dillaman [Tue, 23 Jun 2015 15:14:51 +0000 (11:14 -0400)]
librbd: only update image flags when holding exclusive lock

It was possible for a client to open an image while another client
was shrinking an image.  This would result in the former invalidating
the object map on-disk if it openned the image between updating the
image header and resizing the object map.

Fixes: #11791
Backport: hammer
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit eb81a6a7e391327ac993fd406443b206a7f7bffc)

10 years agolibrbd: new ImageWatcher::is_lock_supported method
Jason Dillaman [Fri, 17 Jul 2015 16:43:46 +0000 (12:43 -0400)]
librbd: new ImageWatcher::is_lock_supported method

The new version does not attempt to acquire the snap_lock, to avoid
cases where a recursive lock would result.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
10 years agoceph.spec.in: install 95-ceph-osd.rules, mount.ceph, and mount.fuse.ceph properly... 5283/head
Nathan Cutler [Thu, 9 Jul 2015 19:38:46 +0000 (21:38 +0200)]
ceph.spec.in: install 95-ceph-osd.rules, mount.ceph, and mount.fuse.ceph properly on SUSE

http://tracker.ceph.com/issues/12261 Fixes: #12261

Signed-off-by: Nathan Cutler <ncutler@suse.com>
(cherry picked from commit 5ce38b9536efabf99a236c7a9d15c149fa4c16a6)

10 years agodebian: move ceph_argparse into ceph-common 5206/head
Ken Dreyer [Tue, 14 Apr 2015 13:58:17 +0000 (07:58 -0600)]
debian: move ceph_argparse into ceph-common

Prior to this commit, if a user installed the "ceph-common" Debian
package without installing "ceph", then /usr/bin/ceph would crash
because it was missing the ceph_argparse library.

Ship the ceph_argparse library in "ceph-common" instead of "ceph". (This
was the intention of the original commit that moved argparse to "ceph",
2a23eac54957e596d99985bb9e187a668251a9ec)

http://tracker.ceph.com/issues/11388 Refs: #11388

Reported-by: Jens Rosenboom <j.rosenboom@x-ion.de>
Signed-off-by: Ken Dreyer <kdreyer@redhat.com>
(cherry picked from commit 110608e5bdd9e2f03020ad41f0c2d756684d4417)

Conflicts:
debian/ceph.install
            There is no ceph_daemon.py in hammer
        debian/control
            Depends/Replaces/Breaks version adapted (from 9.0.0 to 0.94.2)
            also adapted ceph-dbg Replaces/Breaks

10 years agotest: potential memory leak in FlushAioPP 5280/head
Zhiqiang Wang [Fri, 20 Mar 2015 08:15:42 +0000 (16:15 +0800)]
test: potential memory leak in FlushAioPP

Should call the release function instead of deleting it to free
librbd::RBD::AioCompletion and librbd::AioCompletion. Otherwise there is
a potential memory leak.

Signed-off-by: Zhiqiang Wang <zhiqiang.wang@intel.com>
(cherry picked from commit ada7ec860cb7901c560c12a5af36dc7c23051b76)

10 years agopybind: fix valgrind warning on rbd_get_parent_info call
Jason Dillaman [Tue, 28 Apr 2015 19:25:49 +0000 (15:25 -0400)]
pybind: fix valgrind warning on rbd_get_parent_info call

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 2586e3ba1e20603a87c833513e09dae9281beb4d)

10 years agoosdc: invalid read of freed memory
Jason Dillaman [Tue, 28 Apr 2015 15:12:00 +0000 (11:12 -0400)]
osdc: invalid read of freed memory

The bytes not in cache stat was potentially reading the bh length
from a deleted bufferhead.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 5ccc4422d6172376bd6f1be8d3a99c0a54eab807)

10 years agokrbd: fix incorrect types in the krbd API
Jason Dillaman [Tue, 28 Apr 2015 14:56:15 +0000 (10:56 -0400)]
krbd: fix incorrect types in the krbd API

The C API functions were referencing the C++ CephContext
instead of the C rados_config_t.  Additionally, the ceph
namespace was missing on the Formatter class.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 740fd275a60630e60b3bcf41637a2ca486885d9c)

10 years agofsx: cleanup crypto library at exit
Jason Dillaman [Tue, 28 Apr 2015 14:54:47 +0000 (10:54 -0400)]
fsx: cleanup crypto library at exit

Also made small tweaks so that it can be compiled under
a C++ compiler.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit c44f8e7fbc19924a9453d8c032c624ebb6c0296f)

10 years agotests: add run-rbd-valgrind-unit-tests.sh
Jason Dillaman [Fri, 24 Apr 2015 18:29:59 +0000 (14:29 -0400)]
tests: add run-rbd-valgrind-unit-tests.sh

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 5534faaa469b8a6a4c9687aad1a6723f3e859353)

10 years agovalgrind: update valgrind suppressions for lttng-ust
Jason Dillaman [Fri, 24 Apr 2015 04:23:03 +0000 (00:23 -0400)]
valgrind: update valgrind suppressions for lttng-ust

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 8d87bdf597aad3d6be47aedd216a673bd9093a24)

10 years agolibrbd: TaskFinisher should finish all queued tasks
Jason Dillaman [Fri, 24 Apr 2015 04:21:15 +0000 (00:21 -0400)]
librbd: TaskFinisher should finish all queued tasks

The destructor wasn't waiting for all Finisher tasks
to complete before stopping the thread.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 8e20240e4155e2f0398e79f4c0095d2d6ba1d4cb)

10 years agotests: fix valgrind errors with librbd unit test
Jason Dillaman [Fri, 24 Apr 2015 03:10:23 +0000 (23:10 -0400)]
tests: fix valgrind errors with librbd unit test

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit ed5472a10eb515e2a177a640c3f6ed929db9ee4f)

10 years agotests: librbd should release global data before exit
Jason Dillaman [Fri, 24 Apr 2015 03:09:45 +0000 (23:09 -0400)]
tests: librbd should release global data before exit

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 6ab1bb5614a5d257a82cf8ea280eef5c90cf765b)

10 years agolibrados_test_stub: cleanup singleton memory allocation
Jason Dillaman [Fri, 24 Apr 2015 03:08:51 +0000 (23:08 -0400)]
librados_test_stub: cleanup singleton memory allocation

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 54c88255b74741d882b88f791497862635357634)

10 years agoMerge pull request #5046 from ceph/wip-12109-hammer
Loic Dachary [Fri, 17 Jul 2015 17:48:05 +0000 (19:48 +0200)]
Merge pull request #5046 from ceph/wip-12109-hammer

librbd: new QA client upgrade tests

Reviewed-by: Loic Dachary <ldachary@redhat.com>
10 years agoMerge pull request #5159 from theanalyst/wip-11701-hammer
Samuel Just [Thu, 16 Jul 2015 21:58:49 +0000 (14:58 -0700)]
Merge pull request #5159 from theanalyst/wip-11701-hammer

make the all osd/filestore thread pool suicide timeouts separately configurable

Reviewed-by: Samuel Just <sjust@redhat.com>
10 years agoConditional-compile against minimal tcmalloc. 5265/head
Thorsten Behrens [Sun, 15 Mar 2015 23:13:38 +0000 (00:13 +0100)]
Conditional-compile against minimal tcmalloc.

Certain older systems (SLE11 in this case) do not provide the full
tcmalloc functionality, due to e.g. incomplete libunwind
pieces. Use --with-tcmalloc-minimal to enable the cut-down
version.

Here's how the various mem allocator switches interact now:

--with-jemalloc: overrides --with-tcmalloc & --with-tcmalloc-minimal
--with-tcmalloc-minimal: overrides --with-tcmalloc
--with-tcmalloc: the default. use --without-tcmalloc to disable

Signed-off-by: Thorsten Behrens <tbehrens@suse.com>
(cherry picked from commit c6f1c07113ca19547fdac10cd9b817a60142aee2)

10 years agoMerge pull request #5252 from ceph/wip-12021-hammer
Sage Weil [Thu, 16 Jul 2015 15:04:52 +0000 (11:04 -0400)]
Merge pull request #5252 from ceph/wip-12021-hammer

OSDMonitor: allow addition of cache pool with non-empty snaps with co…

Reviewed-by: Sage Weil <sage@redhat.com>
10 years agoMerge pull request #4891 from theanalyst/wip-11740-hammer
Loic Dachary [Wed, 15 Jul 2015 23:15:51 +0000 (01:15 +0200)]
Merge pull request #4891 from theanalyst/wip-11740-hammer

crush: take crashes due to invalid arg

Reviewed-by: Loic Dachary <ldachary@redhat.com>
10 years agoOSD: add command_wq suicide timeout 5159/head
Samuel Just [Wed, 6 May 2015 17:49:00 +0000 (10:49 -0700)]
OSD: add command_wq suicide timeout

Signed-off-by: Samuel Just <sjust@redhat.com>
(cherry picked from commit df4e5de819c30003cfbe50a071c49039cf534419)

Conflicts:
src/common/config_opts.h
Trivial merge conflict

10 years agoOSD: add remove_wq suicide timeout
Samuel Just [Wed, 6 May 2015 17:54:31 +0000 (10:54 -0700)]
OSD: add remove_wq suicide timeout

Signed-off-by: Samuel Just <sjust@redhat.com>
(cherry picked from commit f2fbfa32a16666be46359f0eab7b04ca80a753f5)

10 years agoOSD: add scrub_wq suicide timeout
Samuel Just [Wed, 6 May 2015 17:52:40 +0000 (10:52 -0700)]
OSD: add scrub_wq suicide timeout

Signed-off-by: Samuel Just <sjust@redhat.com>
(cherry picked from commit 547a7041edc833f3cc8e04d388574809e30a8af6)

10 years agoOSD: add snap_trim_wq suicide timeout
Samuel Just [Wed, 6 May 2015 17:51:28 +0000 (10:51 -0700)]
OSD: add snap_trim_wq suicide timeout

Signed-off-by: Samuel Just <sjust@redhat.com>
(cherry picked from commit e1073a4a577211672148a4112bd633831552d66f)

10 years agoOSD: add recovery_wq suicide timeout
Samuel Just [Wed, 6 May 2015 17:50:19 +0000 (10:50 -0700)]
OSD: add recovery_wq suicide timeout

Signed-off-by: Samuel Just <sjust@redhat.com>
(cherry picked from commit 85311b656852af75bfbbc6699f92fc6aa233c316)

Conflicts: src/common/config_opts.h

There was a merge conflict due to introduction of `osd_recovery_sleep`
which was introduced in #3829

10 years agoOSD: add op_wq suicide timeout
Samuel Just [Wed, 6 May 2015 18:02:19 +0000 (11:02 -0700)]
OSD: add op_wq suicide timeout

Signed-off-by: Samuel Just <sjust@redhat.com>
10 years agoMerge pull request #4899 from theanalyst/wip-11911-hammer
Loic Dachary [Tue, 14 Jul 2015 18:43:02 +0000 (20:43 +0200)]
Merge pull request #4899 from theanalyst/wip-11911-hammer

start_flush: filter out removed snaps before determining snapc's

Reviewed-by: Samuel Just <sjust@redhat.com>
10 years agoMerge pull request #4868 from SUSE/wip-11879-hammer
Loic Dachary [Tue, 14 Jul 2015 18:42:45 +0000 (20:42 +0200)]
Merge pull request #4868 from SUSE/wip-11879-hammer

Clock skew causes missing summary and confuses Calamari

Reviewed-by: Samuel Just <sjust@redhat.com>
10 years agoMerge pull request #4883 from SUSE/wip-11638-hammer
Loic Dachary [Tue, 14 Jul 2015 14:42:12 +0000 (16:42 +0200)]
Merge pull request #4883 from SUSE/wip-11638-hammer

ceph.spec.in: ceph-common subpackage def needs tweaking for SUSE/openSUSE

Reviewed-by: Ken Dreyer <kdreyer@redhat.com>
10 years agoMerge pull request #5208 from tchaikov/wip-11975-hammer
Kefu Chai [Sat, 11 Jul 2015 18:19:57 +0000 (02:19 +0800)]
Merge pull request #5208 from tchaikov/wip-11975-hammer

tests: TEST_crush_reject_empty must not run a mon

Reviewed-by: Loic Dachary <ldachary@redhat.com>
10 years agocrush/CrushTester: return EINVAL if crushtool returns non-zero 5208/head
Kefu Chai [Sat, 11 Jul 2015 15:04:33 +0000 (23:04 +0800)]
crush/CrushTester: return EINVAL if crushtool returns non-zero

this backports a tiny part of ec02441, otherwise
CrushTester will return 1, and "ceph" cli will take it
as EPERM, which is miss leading, and fails
osd-crush.sh:TEST_crush_reject_empty.

Signed-off-by: Kefu Chai <kchai@redhat.com>
10 years agotests: TEST_crush_reject_empty must not run a mon
Loic Dachary [Fri, 10 Jul 2015 14:23:47 +0000 (16:23 +0200)]
tests: TEST_crush_reject_empty must not run a mon

* Back in Hammer, the osd-crush.sh individual tests did not run the
  monitor, it was taken care of by the run() function. An attempt to run
  another mon fails with:

  error: IO lock testdir/osd-crush/a/store.db/LOCK: Resource temporarily
  unavailable

  This problem was introduced by cc1cc033930e8690a57674e842a003f6bbc7a242
  from https://github.com/ceph/ceph/pull/4936
* replace test/mon/mon-test-helpers.sh with test/ceph-helpers.sh as
  we need run_osd() in this newly added test
* update the run-dir of commands: ceph-helpers.sh use the different
  convention for the run-dir of daemons.

http://tracker.ceph.com/issues/11975 Refs: #11975

Signed-off-by: Loic Dachary <ldachary@redhat.com>
10 years agoceph-helpers: implement test_expect_failure
Loic Dachary [Tue, 21 Apr 2015 14:11:33 +0000 (16:11 +0200)]
ceph-helpers: implement test_expect_failure

To display the output in case the command did not fail with the expected
output.

Signed-off-by: Loic Dachary <ldachary@redhat.com>
(cherry picked from commit 5871781b10ff0b26c731b70d1898c474006cbee3)

10 years agotests: display the output of failed make check runs
Loic Dachary [Wed, 10 Jun 2015 21:16:01 +0000 (23:16 +0200)]
tests: display the output of failed make check runs

After a make check fails, it shows a summary but not the output of the
failed tests although they contain information to diagnose the problem.

Set the VERBOSE=true automake variable which is documented to collect
and display the failed script output at the end of a run (the content of
the test-suite.log file (valid from automake-1.11 up).

http://www.gnu.org/software/automake/manual/automake.html#index-VERBOSE

Also remove the run-make-check.sh that did the same in a way that is not
compatible with automake-1.11.

Signed-off-by: Loic Dachary <ldachary@redhat.com>
(cherry picked from commit 3a55cb029bb7db9542d2b14f2deda90feb0ae0f6)

10 years agoMerge pull request #4889 from theanalyst/wip-11484-hammer
Loic Dachary [Fri, 10 Jul 2015 08:17:24 +0000 (10:17 +0200)]
Merge pull request #4889 from theanalyst/wip-11484-hammer

OPT_INT option interprets 3221225472 as -1073741824, and crashes in Throttle::Throttle()

Reviewed-by: Kefu Chai <kchai@redhat.com>
10 years agoMerge pull request #4776 from tchaikov/wip-11279-hammer
Loic Dachary [Fri, 10 Jul 2015 08:16:25 +0000 (10:16 +0200)]
Merge pull request #4776 from tchaikov/wip-11279-hammer

ceph: cli interactive mode does not understand quotes

Reviewed-by: Kefu Chai <kchai@redhat.com>
10 years agoMerge pull request #4657 from ceph/wip-hammer-11535-admin-socket
Loic Dachary [Fri, 10 Jul 2015 08:14:42 +0000 (10:14 +0200)]
Merge pull request #4657 from ceph/wip-hammer-11535-admin-socket

common/admin_socket: close socket descriptor in destructor

Reviewed-by: Loic Dachary <ldachary@redhat.com>
10 years agoMerge pull request #4687 from SUSE/wip-7387-hammer
Kefu Chai [Fri, 10 Jul 2015 08:07:48 +0000 (16:07 +0800)]
Merge pull request #4687 from SUSE/wip-7387-hammer

utf8 and old gcc breakage on RHEL6.5

Reviewed-by: Kefu Chai <kchai@redhat.com>
10 years agoMerge pull request #5122 from theanalyst/wip-11982-hammer
Kefu Chai [Fri, 10 Jul 2015 08:00:00 +0000 (16:00 +0800)]
Merge pull request #5122 from theanalyst/wip-11982-hammer

ceph fails to compile with boost 1.58

Reviewed-by: Kefu Chai <kchai@redhat.com>
10 years agoMerge pull request #4936 from ceph/wip-11975-hammer
Kefu Chai [Fri, 10 Jul 2015 07:59:35 +0000 (15:59 +0800)]
Merge pull request #4936 from ceph/wip-11975-hammer

mon crashes when "ceph osd tree 85 --format json"

Reviewed-by: Kefu Chai <kchai@redhat.com>
10 years agoMerge pull request #4892 from theanalyst/wip-11760-hammer
Loic Dachary [Fri, 10 Jul 2015 07:48:44 +0000 (09:48 +0200)]
Merge pull request #4892 from theanalyst/wip-11760-hammer

ceph-disk: get_partition_type fails on /dev/cciss...

Reviewed-by: Loic Dachary <ldachary@redhat.com>
10 years agoMerge pull request #4877 from SUSE/wip-11902-hammer
Loic Dachary [Fri, 10 Jul 2015 07:43:57 +0000 (09:43 +0200)]
Merge pull request #4877 from SUSE/wip-11902-hammer

admin/build-doc: script fails silently under certain circumstances

Reviewed-by: Loic Dachary <ldachary@redhat.com>
10 years agomon: add "--check" to CrushTester::test_with_crushtool() 4936/head
Kefu Chai [Tue, 26 May 2015 10:11:59 +0000 (18:11 +0800)]
mon: add "--check" to CrushTester::test_with_crushtool()

so we don't need to call CrushTester::check_name_maps() in OSDMonitor.cc
anymore.

Fixes: #11680
Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit c6e634875316cf17368d497e6dc4f6f4b5dd65d2)

10 years agocrushtool: rename "--check-names" to "--check"
Kefu Chai [Tue, 26 May 2015 09:51:50 +0000 (17:51 +0800)]
crushtool: rename "--check-names" to "--check"

* because "--check" also checks for the max_id

Note: edited since we do not have the fix introduced in 46103b2 in
      hammer.

Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit 9381d53acdce85fcbff828926b911e050ba36e51)

10 years agomon: check the new crush map against osdmap.max_osd
Kefu Chai [Tue, 26 May 2015 08:58:23 +0000 (16:58 +0800)]
mon: check the new crush map against osdmap.max_osd

Fixes: #11680
Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit 22e6bd6e01d5df3f3e897562597e22ca1737f8c8)

10 years agocrushtool: enable check against max_id
Kefu Chai [Tue, 26 May 2015 07:35:10 +0000 (15:35 +0800)]
crushtool: enable check against max_id

add an argument "max_id" for "--check-names" to check if any item
has an id greater or equal to given "max_id" in crush map.

Note: edited since we do not have the fix introduced in 46103b2 in
      hammer.

Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit d0658dd3cdf072b2a7c2a1986f8785a697c591ee)