]> git.apps.os.sepia.ceph.com Git - ceph.git/log
ceph.git
8 years agolibrbd: possible deadlock if cluster connection closed after image 11037/head
Jason Dillaman [Fri, 9 Sep 2016 14:42:50 +0000 (10:42 -0400)]
librbd: possible deadlock if cluster connection closed after image

Fixes: http://tracker.ceph.com/issues/17254
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
9 years agoMerge pull request #10922 from ceph/wip-jlayton-statx
jtlayton [Tue, 6 Sep 2016 11:04:51 +0000 (07:04 -0400)]
Merge pull request #10922 from ceph/wip-jlayton-statx

First pile of statx patches

9 years agoMerge pull request #10981 from SUSE/wip-fix-install-deps
Nathan Cutler [Mon, 5 Sep 2016 16:17:52 +0000 (18:17 +0200)]
Merge pull request #10981 from SUSE/wip-fix-install-deps

build/ops: fix /etc/os-release parsing in install-deps.sh

Reviewed-by: Jan Fajerski <jfajerski@suse.com>
Reviewed-by: Loic Dachary <ldachary@redhat.com>
9 years agobuild/ops: fix /etc/os-release parsing in install-deps.sh 10981/head
Nathan Cutler [Mon, 5 Sep 2016 13:40:33 +0000 (15:40 +0200)]
build/ops: fix /etc/os-release parsing in install-deps.sh

85a370e35fc42031a7f7e24dea9e50a649c0f309 introduced the DISTRO variable whose
value was obtained by parsing /etc/os-release like this:

DISTRO=$(grep  "^ID=" /etc/os-release | sed "s/ID=//")

This unfortunately picks up the double-quotes, so on a CentOS system DISTRO
will be equal to '"centos"'.

Signed-off-by: Nathan Cutler <ncutler@suse.com>
9 years agoMerge pull request #10969 from xiexingguo/xxg-wip-fix-cache-logger
Igor Fedotov [Mon, 5 Sep 2016 11:51:59 +0000 (14:51 +0300)]
Merge pull request #10969 from xiexingguo/xxg-wip-fix-cache-logger

os/bluestore: replace store with logger in Cache

Reviewed-by: Igor Fedotov <ifedotov@mirantis.com>
9 years agoMerge pull request #9592 from trociny/wip-15632
Jason Dillaman [Mon, 5 Sep 2016 01:54:36 +0000 (21:54 -0400)]
Merge pull request #9592 from trociny/wip-15632

librbd: API methods to directly acquire and release the exclusive lock

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
9 years agoMerge pull request #9758 from SUSE/wip-btrfs-not-ideal
Loic Dachary [Sun, 4 Sep 2016 17:58:19 +0000 (19:58 +0200)]
Merge pull request #9758 from SUSE/wip-btrfs-not-ideal

doc: remove btrfs contradiction

Reviewed-by: Loic Dachary <ldachary@redhat.com>
9 years agoMerge pull request #10469 from wjwithagen/wip-wjw-compileflag
Kefu Chai [Sun, 4 Sep 2016 03:10:34 +0000 (11:10 +0800)]
Merge pull request #10469 from wjwithagen/wip-wjw-compileflag

cmake: supress more warnings

Reviewed-by: Kefu Chai <kchai@redhat.com>
9 years agoos/bluestore: check iterator before using during fsck 10969/head
xie xingguo [Sat, 3 Sep 2016 03:38:00 +0000 (11:38 +0800)]
os/bluestore: check iterator before using during fsck

Also drop redundant checking.

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
9 years agoos/bluestore: replace store with logger in Cache
xie xingguo [Sat, 3 Sep 2016 01:06:43 +0000 (09:06 +0800)]
os/bluestore: replace store with logger in Cache

As what we really want is to log cache-related statistics.

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
9 years agoMerge pull request #10945 from dillaman/wip-17188
Mykola Golub [Fri, 2 Sep 2016 19:00:41 +0000 (22:00 +0300)]
Merge pull request #10945 from dillaman/wip-17188

librbd: deadlock when replaying journal during image open

Reviewed-by: Mykola Golub <mgolub@mirantis.com>
9 years agoSQUASH: client: request all shared caps if ctime was requested 10922/head
Jeff Layton [Fri, 2 Sep 2016 18:27:08 +0000 (14:27 -0400)]
SQUASH: client: request all shared caps if ctime was requested

Much like the change_attr, ctime changes can potentially be cached on
clients. Request all shared caps if the want mask specifies the ctime.

Signed-off-by: Jeff Layton <jlayton@redhat.com>
9 years agoMerge pull request #10746 from ceph/wip-async-default
Sage Weil [Fri, 2 Sep 2016 18:02:57 +0000 (13:02 -0500)]
Merge pull request #10746 from ceph/wip-async-default

msg: make async backend default

9 years agoMerge pull request #10938 from liewegas/wip-bluestore-csum-error
Sage Weil [Fri, 2 Sep 2016 16:23:40 +0000 (11:23 -0500)]
Merge pull request #10938 from liewegas/wip-bluestore-csum-error

os/bluestore: fix/improve csum error message

Reviewed-by: Igor Fedotov <ifedotov@mirantis.com>
9 years agoMerge pull request #10961 from xiexingguo/xxg-wip-bluestore-09-02
Sage Weil [Fri, 2 Sep 2016 16:23:27 +0000 (11:23 -0500)]
Merge pull request #10961 from xiexingguo/xxg-wip-bluestore-09-02

os/bluestore: add cache-related stats

Reviewed-by: Igor Fedotov <ifedotov@mirantis.com>
Reviewed-by: Sage Weil <sage@redhat.com>
9 years agoMerge pull request #10953 from liewegas/wip-bluestore-fixes
Igor Fedotov [Fri, 2 Sep 2016 15:26:07 +0000 (18:26 +0300)]
Merge pull request #10953 from liewegas/wip-bluestore-fixes

os/bluestore: misc fixes

Reviewed- by: Igor Fedotov ifedotov@mirantis.com

9 years agoSQUASH: mds/client: add routines to copy to/from ceph_mds_request_head_legacy
Jeff Layton [Fri, 2 Sep 2016 12:36:59 +0000 (08:36 -0400)]
SQUASH: mds/client: add routines to copy to/from ceph_mds_request_head_legacy

Signed-off-by: Jeff Layton <jlayton@redhat.com>
9 years agoos/bluestore: trivial cleanups 10961/head
xie xingguo [Fri, 2 Sep 2016 10:10:33 +0000 (18:10 +0800)]
os/bluestore: trivial cleanups

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
9 years agoos/bluestore: don't overwrite type if it does already exist and at desired type
xie xingguo [Fri, 2 Sep 2016 09:48:39 +0000 (17:48 +0800)]
os/bluestore: don't overwrite type if it does already exist and at desired type

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
9 years agosrc/CMakeLists.txt: -Woverloaded-virtual is already included in -Wall on Clang 10469/head
Willem Jan Withagen [Thu, 28 Jul 2016 09:25:03 +0000 (11:25 +0200)]
src/CMakeLists.txt: -Woverloaded-virtual is already included in -Wall on Clang

Signed-off-by: Willem Jan Withagen <wjw@digiware.nl>
9 years agoMerge pull request #10956 from ceph/wip-cmake
Kefu Chai [Fri, 2 Sep 2016 09:22:37 +0000 (17:22 +0800)]
Merge pull request #10956 from ceph/wip-cmake

cmake: pass -DINTEL* to gf-complete cflags

Reviewed-by: Loic Dachary <ldachary@redhat.com>
9 years agoos/bluestore: conditional checking label of bluefs devices
xie xingguo [Fri, 2 Sep 2016 08:45:05 +0000 (16:45 +0800)]
os/bluestore: conditional checking label of bluefs devices

As NVMe devices may not support to do so.

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
9 years agocmake: pass -DINTEL* to gf-complete cflags 10956/head
Kefu Chai [Fri, 2 Sep 2016 06:38:21 +0000 (14:38 +0800)]
cmake: pass -DINTEL* to gf-complete cflags

* in addition to the -march=foobar, we also need to pass -DFOOBAR to
  gf-complete sources' cflags, so that the SIMD instructions can be
  generated as expected.
* also extract the compiler support for instruction extensions detections
  into SIMDExt.cmake.
* and only check the compiler support if CMAKE_SYSTEM_PROCESSOR matches
  with the instruction-set/arch to be checked.

Signed-off-by: tone.zhang <tone.zhang@linaro.org>
Signed-off-by: Kefu Chai <kchai@redhat.com>
9 years agoMerge pull request #10934 from tchaikov/wip-kill-warnings
Brad Hubbard [Fri, 2 Sep 2016 07:32:46 +0000 (17:32 +1000)]
Merge pull request #10934 from tchaikov/wip-kill-warnings

Wip kill warnings

Reviewed-by: Brad Hubbard <bhubbard@redhat.com>
9 years agoos/bluestore: end scope of std::hex properly
xie xingguo [Fri, 2 Sep 2016 03:37:05 +0000 (11:37 +0800)]
os/bluestore: end scope of std::hex properly

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
9 years agoos/bluestore: add onode/buffer cache statistics
xie xingguo [Fri, 2 Sep 2016 03:33:33 +0000 (11:33 +0800)]
os/bluestore: add onode/buffer cache statistics

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
9 years agoSQUASH: client: querying for stx_version field requires getting all shared caps
Jeff Layton [Thu, 1 Sep 2016 22:49:46 +0000 (18:49 -0400)]
SQUASH: client: querying for stx_version field requires getting all shared caps

Otherwise, someone could potentially query for just CEPH_STATX_VERSION,
and see it as unchanged, even when there are changes buffered up on
other clients.

By doing this, I don't think we'll incur any perf hit in the common use
case which is ganesha querying for all attributes. We are adding Xs
here, but unless there is a lot of xattr activity I don't think that
will generally cause a lot of exclusive cap recalls.

Also, we don't actually need AUTH caps to fetch CEPH_STATX_RDEV, remove
that from statx_to_mask. PIN is sufficient there.

Signed-off-by: Jeff Layton <jlayton@redhat.com>
9 years agoSQUASH: mds/client: don't get clever with ceph_mds_request_head_legacy encode/decode
Jeff Layton [Thu, 1 Sep 2016 18:18:55 +0000 (14:18 -0400)]
SQUASH: mds/client: don't get clever with ceph_mds_request_head_legacy encode/decode

As requested by Greg...

Declare a legacy object on the stack, and do an extra copy to/from it.

Signed-off-by: Jeff Layton <jlayton@redhat.com>
9 years agoSQUASH: client: always take a change_attr update from the server if it's newer
Jeff Layton [Thu, 1 Sep 2016 15:16:06 +0000 (11:16 -0400)]
SQUASH: client: always take a change_attr update from the server if it's newer

I think this is more correct in the client. If we see a newer
change_attr, then we always want to take note of that fact, even if
we have the right caps.

Signed-off-by: Jeff Layton <jlayton@redhat.com>
9 years agoSQUASH: Revert "mds: ensure that change_attr reflects metadata changes on clients...
Jeff Layton [Thu, 1 Sep 2016 15:16:05 +0000 (11:16 -0400)]
SQUASH: Revert "mds: ensure that change_attr reflects metadata changes on clients that hold CAP_FILE_EXCL"

This reverts commit 26ab2fa10041ce34b25aa1535715131a2709a27f.

I don't think this is necessary. We don't necessarily need to increment
the change attribute on every change, as long as we ensure that it gets
bumped iff there were changes since the last time you queried for it.

IOW: it's ok to have a single change_attr change encompass a large set
of changes, as long as you ensure that that it is larger by at least 1
after all of those changes.

In order to look at (and potentially cache) other attributes under that
change_attribute (e.g. owner or mode), you need to recall any outstanding
exclusive caps for those attrs. That causes their change_attrs to be
synched to the largest, which is enough to ensure that it changed in
some way.

Signed-off-by: Jeff Layton <jlayton@redhat.com>
9 years agoSQUASH: client: request AUTH caps on the directory during create if we'll need to...
Jeff Layton [Thu, 1 Sep 2016 18:35:59 +0000 (14:35 -0400)]
SQUASH: client: request AUTH caps on the directory during create if we'll need to check perms locally

Signed-off-by: Jeff Layton <jlayton@redhat.com>
9 years agoMerge pull request #10517 from wjwithagen/wip-wjw-freebsd-cmake-execludes
Kefu Chai [Thu, 1 Sep 2016 17:38:44 +0000 (01:38 +0800)]
Merge pull request #10517 from wjwithagen/wip-wjw-freebsd-cmake-execludes

cmake: FreeBSD specific excludes in CMakeLists.txt files

Reviewed-by: Kefu Chai <kchai@redhat.com>
9 years agoceph_test_objectstore: fix TryMoveRename test 10953/head
Sage Weil [Thu, 1 Sep 2016 16:37:27 +0000 (12:37 -0400)]
ceph_test_objectstore: fix TryMoveRename test

Put the objects in the right pool.

Signed-off-by: Sage Weil <sage@redhat.com>
9 years agoceph_test_objectstore: clean up Rename test a bit
Sage Weil [Thu, 1 Sep 2016 16:04:27 +0000 (12:04 -0400)]
ceph_test_objectstore: clean up Rename test a bit

Drop useless remove calls.

Leave in the no-op setattr call since it triggered a
bluestore bug :)

Signed-off-by: Sage Weil <sage@redhat.com>
9 years agoceph_test_objectstore: assert umount succeeds
Sage Weil [Wed, 31 Aug 2016 20:49:13 +0000 (16:49 -0400)]
ceph_test_objectstore: assert umount succeeds

Signed-off-by: Sage Weil <sage@redhat.com>
9 years agoos/bluestore: make object existence check robust
Sage Weil [Thu, 1 Sep 2016 15:43:55 +0000 (11:43 -0400)]
os/bluestore: make object existence check robust

We may have the OnodeRef defined but the object may have
exists==false.  Move the check out of the lookup block
so that we get the right behavior.

Signed-off-by: Sage Weil <sage@redhat.com>
9 years agoos/bluestore: use _setattr for SETATTR
Sage Weil [Thu, 1 Sep 2016 15:20:17 +0000 (11:20 -0400)]
os/bluestore: use _setattr for SETATTR

This is a bit faster.

Signed-off-by: Sage Weil <sage@redhat.com>
9 years agoMerge pull request #10843 from liewegas/wip-osdmap-reencode
Sage Weil [Thu, 1 Sep 2016 14:56:36 +0000 (09:56 -0500)]
Merge pull request #10843 from liewegas/wip-osdmap-reencode

messages/MOSDMap: mark as enlighten OSDMap encoder

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>
9 years agoMerge pull request #10950 from xiexingguo/xxg-wip-blue-09-01
Sage Weil [Thu, 1 Sep 2016 14:22:56 +0000 (09:22 -0500)]
Merge pull request #10950 from xiexingguo/xxg-wip-blue-09-01

os/bluestore: gifting bluefs more carefully

Reviewed-by: Sage Weil <sage@redhat.com>
9 years agoos/bluestore: don't over-gift if bluefs free is smaller than bluestore_bluefs_min 10950/head
xie xingguo [Thu, 1 Sep 2016 10:42:58 +0000 (18:42 +0800)]
os/bluestore: don't over-gift if bluefs free is smaller than bluestore_bluefs_min

- If free space of bluefs is smaller than bluestore_bluefs_min, then raise it
  exactly up to bluestore_bluefs_min.

- Pre-check if bluestore_bluefs_min will exceed bluestore_bluefs_max_ratio * bluefs_total,
  so if free space of bluefs is smaller than bluestore_bluefs_min_ratio * bluefs_total,
  we'll still be able to gift.

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
9 years agoos/bluestore: end scope of std::hex properly
xie xingguo [Thu, 1 Sep 2016 03:33:25 +0000 (11:33 +0800)]
os/bluestore: end scope of std::hex properly

So it does not extend to the following output stream and
causes unexpected behaviour.

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
9 years agolibrbd: deadlock when replaying journal during image open 10945/head
Jason Dillaman [Thu, 1 Sep 2016 01:33:54 +0000 (21:33 -0400)]
librbd: deadlock when replaying journal during image open

Fixes: http://tracker.ceph.com/issues/17188
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
9 years agolibrbd: improve image state machine debug log messages
Jason Dillaman [Thu, 1 Sep 2016 00:56:54 +0000 (20:56 -0400)]
librbd: improve image state machine debug log messages

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
9 years agoMerge pull request #10404 from ilc/build_fix
Sage Weil [Wed, 31 Aug 2016 21:19:40 +0000 (16:19 -0500)]
Merge pull request #10404 from ilc/build_fix

make-dist/ceph.spec.in: Fix srpm build breakage.

Reviewed-by: Ken Dreyer <kdreyer@redhat.com>
9 years agoMerge pull request #10846 from stiopaa1/mds_mdstable_addConstToMemFun
Gregory Farnum [Wed, 31 Aug 2016 21:05:21 +0000 (14:05 -0700)]
Merge pull request #10846 from stiopaa1/mds_mdstable_addConstToMemFun

mds/MDSTable: add const to member functions

Reviewed-by: Greg Farnum <gfarnum@redhat.com>
9 years agoMerge pull request #10781 from stiopaa1/osdc_objectcacher_moveClassesToCC
Gregory Farnum [Wed, 31 Aug 2016 21:04:03 +0000 (14:04 -0700)]
Merge pull request #10781 from stiopaa1/osdc_objectcacher_moveClassesToCC

osdc/ObjectCacher: move C_ReadFinish, C_RetryRead

Reviewed-by: Greg Farnum <gfarnum@redhat.com>
9 years agoMerge pull request #10744 from stiopaa1/osdc_journaler_moveCDelayFlushClass
Gregory Farnum [Wed, 31 Aug 2016 21:03:50 +0000 (14:03 -0700)]
Merge pull request #10744 from stiopaa1/osdc_journaler_moveCDelayFlushClass

osdc/Journaler: move C_DelayFlush class to .cc

Reviewed-by: Greg Farnum <gfarnum@redhat.com>
9 years agoMerge pull request #10668 from stiopaa1/mon_mdsmonitor_moveCUpdatedClass
Gregory Farnum [Wed, 31 Aug 2016 21:03:43 +0000 (14:03 -0700)]
Merge pull request #10668 from stiopaa1/mon_mdsmonitor_moveCUpdatedClass

mon/MDSMonitor: move C_Updated class to .cc file

Reviewed-by: Greg Farnum <gfarnum@redhat.com>
9 years agoMerge pull request #10628 from jcsp/wip-16909
Gregory Farnum [Wed, 31 Aug 2016 21:02:57 +0000 (14:02 -0700)]
Merge pull request #10628 from jcsp/wip-16909

mon: tear down standby replays on MDS rank stop

Reviewed-by: Greg Farnum <gfarnum@redhat.com>
9 years agoMerge pull request #10937 from ceph/dang-build
Ali Maredia [Wed, 31 Aug 2016 19:53:39 +0000 (15:53 -0400)]
Merge pull request #10937 from ceph/dang-build

Allow compressor build without YASM

Reviewed-by: Ali Maredia <amaredia@redhat.com>
9 years agoos/bluestore: improve failed csum debug output 10938/head
Sage Weil [Tue, 30 Aug 2016 18:13:24 +0000 (14:13 -0400)]
os/bluestore: improve failed csum debug output

Signed-off-by: Sage Weil <sage@redhat.com>
9 years agoscript/bdev_grep: update to use hex
Sage Weil [Wed, 31 Aug 2016 15:19:29 +0000 (11:19 -0400)]
script/bdev_grep: update to use hex

Signed-off-by: Sage Weil <sage@redhat.com>
9 years agoAllow compressor build without YASM 10937/head
Daniel Gryniewicz [Wed, 31 Aug 2016 16:00:42 +0000 (12:00 -0400)]
Allow compressor build without YASM

The isa-l compressors need YASM to build.  Conditionalize on that.

Signed-off-by: Daniel Gryniewicz <dang@redhat.com>
9 years agoMerge pull request #10931 from xiexingguo/xxg-wip-fix-isa-l-assert
Igor Fedotov [Wed, 31 Aug 2016 14:29:42 +0000 (17:29 +0300)]
Merge pull request #10931 from xiexingguo/xxg-wip-fix-isa-l-assert

os/bluestore: get rid off "isa-l" type in ZLibCompressor ctor

Reviewed-by: Igor Fedotov <ifedotov@mirantis.com>
Reviewed-by: Sage Weil <sage@redhat.com>
9 years agoMerge pull request #10923 from xiexingguo/xxg-wip-bluefs-08-30
Sage Weil [Wed, 31 Aug 2016 14:21:03 +0000 (09:21 -0500)]
Merge pull request #10923 from xiexingguo/xxg-wip-bluefs-08-30

os/bluestore/bluefs: use map to track dirty files

Reviewed-by: Sage Weil <sage@redhat.com>
9 years agoos/bluestore: get rid off "isa-l" type in ZLibCompressor ctor 10931/head
xie xingguo [Wed, 31 Aug 2016 07:26:24 +0000 (15:26 +0800)]
os/bluestore: get rid off "isa-l" type in ZLibCompressor ctor

This is introduced by https://github.com/ceph/ceph/pull/10158 and causes
the following failure:

/home/xxg/ceph-dev/src/os/bluestore/bluestore_types.h: 284: FAILED assert(0 == "invalid compression algorithm")

 ceph version v11.0.0-1941-g47c8a21 (47c8a214356983f0539f24fe1c5724b3bbae0406)
 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x85) [0x7f5668c1ad45]
 2: (BlueStore::_do_alloc_write(BlueStore::TransContext*, BlueStore::WriteContext*)+0x1a65) [0x7f5668b19d55]
 3: (BlueStore::_do_write(BlueStore::TransContext*, boost::intrusive_ptr<BlueStore::Collection>&, boost::intrusive_ptr<BlueStore::Onode>, unsigned long, unsigned long, ceph::buffer::list&, unsigned int)+0x4ac) [0x7f5668b3b3fc]
 4: (BlueStore::_write(BlueStore::TransContext*, boost::intrusive_ptr<BlueStore::Collection>&, boost::intrusive_ptr<BlueStore::Onode>&, unsigned long, unsigned long, ceph::buffer::list&, unsigned int)+0x10f) [0x7f5668b3bf2f]
 5: (BlueStore::_txc_add_transaction(BlueStore::TransContext*, ObjectStore::Transaction*)+0x1067) [0x7f5668b405f7]
 6: (BlueStore::queue_transactions(ObjectStore::Sequencer*, std::vector<ObjectStore::Transaction, std::allocator<ObjectStore::Transaction> >&, std::shared_ptr<TrackedOp>, ThreadPool::TPHandle*)+0x382) [0x7f5668b42432]
 7: (SyntheticWorkloadState::write()+0x3a5) [0x7f5668a2ace5]
 8: (doSyntheticTest(boost::scoped_ptr<ObjectStore>&, unsigned long, unsigned long, unsigned long)+0xb9d) [0x7f56689dfa7d]
 9: (do_matrix_choose(char const* (*) [10], int, int, int, boost::scoped_ptr<ObjectStore>&)+0x29d) [0x7f56689dfffd]
 10: (do_matrix_choose(char const* (*) [10], int, int, int, boost::scoped_ptr<ObjectStore>&)+0x9e) [0x7f56689dfdfe]
 11: (do_matrix_choose(char const* (*) [10], int, int, int, boost::scoped_ptr<ObjectStore>&)+0x9e) [0x7f56689dfdfe]
 12: (do_matrix_choose(char const* (*) [10], int, int, int, boost::scoped_ptr<ObjectStore>&)+0x9e) [0x7f56689dfdfe]
 13: (do_matrix_choose(char const* (*) [10], int, int, int, boost::scoped_ptr<ObjectStore>&)+0x9e) [0x7f56689dfdfe]
 14: (do_matrix_choose(char const* (*) [10], int, int, int, boost::scoped_ptr<ObjectStore>&)+0x9e) [0x7f56689dfdfe]
 15: (do_matrix(char const* (*) [10], boost::scoped_ptr<ObjectStore>&)+0x317) [0x7f56689e0347]
 16: (()+0x297f3c) [0x7f56689e0f3c]
 17: (StoreTest_SyntheticMatrixCompressionAlgorithm_Test::TestBody()+0x118) [0x7f56689e1078]
 18: (void testing::internal::HandleExceptionsInMethodIfSupported<testing::Test, void>(testing::Test*, void (testing::Test::*)(), char const*)+0x43) [0x7f5668d673e3]
 19: (testing::Test::Run()+0xb7) [0x7f5668d5a127]
 20: (testing::TestInfo::Run()+0x9e) [0x7f5668d5a1ce]
 21: (testing::TestCase::Run()+0xa5) [0x7f5668d5a2d5]
 22: (testing::internal::UnitTestImpl::RunAllTests()+0x278) [0x7f5668d5a5b8]
 23: (testing::UnitTest::Run()+0x68) [0x7f5668d5a888]
 24: (main()+0x2f4) [0x7f566895be84]
 25: (__libc_start_main()+0xf5) [0x7f5663e5ab15]
 26: (()+0x265ba5) [0x7f56689aeba5]
 NOTE: a copy of the executable, or `objdump -rdS <executable>` is needed to interpret this.

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
9 years agotest/perf_local: fix -Wmaybe-uninitialized warnings 10934/head
Kefu Chai [Wed, 31 Aug 2016 10:46:53 +0000 (18:46 +0800)]
test/perf_local: fix -Wmaybe-uninitialized warnings

Signed-off-by: Kefu Chai <kchai@redhat.com>
9 years agotest,pybind: silence -Wsign-compare warnings
Kefu Chai [Wed, 31 Aug 2016 10:44:36 +0000 (18:44 +0800)]
test,pybind: silence -Wsign-compare warnings

Signed-off-by: Kefu Chai <kchai@redhat.com>
9 years agoos/CMakeLists.txt: undubble kv.cc usage. 10517/head
Willem Jan Withagen [Wed, 31 Aug 2016 09:23:06 +0000 (11:23 +0200)]
os/CMakeLists.txt: undubble kv.cc usage.

 - kv.cc is both in bluestore and in kstore.
   But atm. both are the same
 - Bluestore is not build on FreeBSD (no AIO)

So use the kstore one to be the one to build against.

Signed-off-by: Willem Jan Withagen <wjw@digiware.nl>
9 years agoos/bluestore/bluefs: drop redundant eof check for read_random() 10923/head
xie xingguo [Wed, 31 Aug 2016 02:47:50 +0000 (10:47 +0800)]
os/bluestore/bluefs: drop redundant eof check for read_random()

Because we have done the sanity check and fixed the length
at the entrance, so we don't have to check it again,
especially in the while loop, which is less efficient.

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
9 years agoos/bluestore/bluefs: update mtime of inode in unified path
xie xingguo [Wed, 31 Aug 2016 01:55:04 +0000 (09:55 +0800)]
os/bluestore/bluefs: update mtime of inode in unified path

To reduce code redundancy.

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
9 years agoos/bluestore/bluefs: use map to track dirty files
xie xingguo [Tue, 30 Aug 2016 07:37:46 +0000 (15:37 +0800)]
os/bluestore/bluefs: use map to track dirty files

So dirty files are arranged by dirty_seq and then
those of same dirty_seq are grouped into list and
as a result we can clean up dirty files much faster
during _flush_and_sync_log().

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
9 years agoMerge pull request #10158 from Ved-vampir/zlib_isal_extension
Dan Mick [Tue, 30 Aug 2016 22:22:08 +0000 (15:22 -0700)]
Merge pull request #10158 from Ved-vampir/zlib_isal_extension

Isa-l extention for zlib compression plugin

9 years agoMerge pull request #10858 from atheism/rgw_getattr-return
Matt Benjamin [Tue, 30 Aug 2016 19:38:19 +0000 (15:38 -0400)]
Merge pull request #10858 from atheism/rgw_getattr-return

rgw: remove duplicated calls to getattr

verified: f23

9 years agoMerge pull request #10927 from liewegas/wip-0.94.9
Alfredo Deza [Tue, 30 Aug 2016 19:11:05 +0000 (15:11 -0400)]
Merge pull request #10927 from liewegas/wip-0.94.9

doc/release-notes: v0.94.9

Reviewed-by: Alfredo Deza <adeza@redhat.com>
9 years agodoc/release-notes: v0.94.9 10927/head
Sage Weil [Tue, 30 Aug 2016 19:08:49 +0000 (15:08 -0400)]
doc/release-notes: v0.94.9

Signed-off-by: Sage Weil <sage@redhat.com>
9 years agoMerge pull request #10745 from linuxbox2/rgw-nfs-31
Casey Bodley [Tue, 30 Aug 2016 18:59:45 +0000 (14:59 -0400)]
Merge pull request #10745 from linuxbox2/rgw-nfs-31

rgw nfs v3 completions

Reviewed-by: Casey Bodley <cbodley@redhat.com>
9 years agoMerge pull request #10926 from ifed01/wip-bluestore-fix-statfs-test
Sage Weil [Tue, 30 Aug 2016 17:30:25 +0000 (12:30 -0500)]
Merge pull request #10926 from ifed01/wip-bluestore-fix-statfs-test

os/bluestore: yet another statfs test fix

Reviewed-by: Sage Weil <sage@redhat.com>
9 years agoMerge pull request #10875 from athanatos/wip-16908
Samuel Just [Tue, 30 Aug 2016 16:56:59 +0000 (09:56 -0700)]
Merge pull request #10875 from athanatos/wip-16908

Revert "osd/ReplicatedPG: for sync-read it don't cacl l_osd_op_r_prep…

Reviewed-by: Greg Farnum <gfarnum@redhat.com>
9 years agoRevert "osd/ReplicatedPG: for sync-read it don't cacl l_osd_op_r_prepare_lat." 10875/head
Samuel Just [Thu, 25 Aug 2016 16:30:04 +0000 (09:30 -0700)]
Revert "osd/ReplicatedPG: for sync-read it don't cacl l_osd_op_r_prepare_lat."

Accessing the OpContext* after execute_ctx is invalid, on error, it
will have been deleted.

This reverts commit 52be772788d9d96accaa7af9eaf9f29a3792df49.

Fixes: http://tracker.ceph.com/issues/16908
Signed-off-by: Samuel Just <sjust@redhat.com>
9 years agoMerge pull request #10881 from ceph/wip-kill-warnings
Loic Dachary [Tue, 30 Aug 2016 16:18:11 +0000 (18:18 +0200)]
Merge pull request #10881 from ceph/wip-kill-warnings

Wip kill warnings

Reviewed-by: Loic Dachary <ldachary@redhat.com>
9 years agoos/bluestore: yet another statfs test fix 10926/head
Igor Fedotov [Tue, 30 Aug 2016 15:26:24 +0000 (15:26 +0000)]
os/bluestore: yet another statfs test fix

Signed-off-by: Igor Fedotov <ifedotov@mirantis.com>
9 years agoMerge pull request #10917 from ceph/wip-cmake
Ali Maredia [Tue, 30 Aug 2016 14:39:32 +0000 (10:39 -0400)]
Merge pull request #10917 from ceph/wip-cmake

cmake changes related to LTTng-UST

Reviewed-by: Ali Maredia <amaredia@redhat.com>
9 years agoMerge pull request #10768 from liewegas/wip-bluestore-blobbuffers
Igor Fedotov [Tue, 30 Aug 2016 12:05:08 +0000 (15:05 +0300)]
Merge pull request #10768 from liewegas/wip-bluestore-blobbuffers

os/bluestore: avoid blob_t reencode when unchanged

Reviewed-by: Igor Fedotov <ifedotov@mirantis.com>
9 years agoMerge pull request #10910 from liewegas/wip-bluestore-statfs
Igor Fedotov [Tue, 30 Aug 2016 12:04:37 +0000 (15:04 +0300)]
Merge pull request #10910 from liewegas/wip-bluestore-statfs

os/bluestore: fix statfs tests

Reviewed-by: Igor Fedotov <ifedotov@mirantis.com>
9 years agoMerge pull request #10017 from jan--f/os-release-based-install-deps
Nathan Cutler [Tue, 30 Aug 2016 09:41:29 +0000 (11:41 +0200)]
Merge pull request #10017 from jan--f/os-release-based-install-deps

build/ops: install-deps.sh based on /etc/os-release

Reviewed-by: Ken Dreyer <kdreyer@redhat.com>
Reviewed-by: Nathan Cutler <ncutler@suse.com>
9 years agocmake: let librados_api_obj depend on librados-tp 10917/head
Kefu Chai [Tue, 30 Aug 2016 05:33:59 +0000 (13:33 +0800)]
cmake: let librados_api_obj depend on librados-tp

so we don't need to messing up with the target names.

Signed-off-by: Kefu Chai <kchai@redhat.com>
9 years agocmake: replace Findlttng-ust.cmake with FindLTTngUST.cmake
Kefu Chai [Tue, 30 Aug 2016 05:33:24 +0000 (13:33 +0800)]
cmake: replace Findlttng-ust.cmake with FindLTTngUST.cmake

find_package(lttng-ust REQUIRED) fails to find the lttng library without
this change. because find_path(LTTNG_LIBRARY_DIR ...) does not search
in the default library paths. and the second mode of
FIND_PACKAGE_HANDLE_STANDARD_ARGS() does not stop the cmake with a fatal
message, even some of the required vars are missing. so use the
implemetantion from cmake upstream.

Signed-off-by: Kefu Chai <kchai@redhat.com>
9 years agocmake: remove unused librados_api target
Kefu Chai [Tue, 30 Aug 2016 05:32:52 +0000 (13:32 +0800)]
cmake: remove unused librados_api target

Signed-off-by: Kefu Chai <kchai@redhat.com>
9 years agoMerge pull request #9930 from yuyuyu101/wip-16472
Yuri Weinstein [Mon, 29 Aug 2016 23:44:19 +0000 (16:44 -0700)]
Merge pull request #9930 from yuyuyu101/wip-16472

msg/simple: wait dispatch_queue until all pipes closed

Reviewed-by: Gregory Farnum <gfarnum@redhat.com>
9 years agomessages/MOSDMap: mark as enlighten OSDMap encoder 10843/head
Sage Weil [Mon, 29 Aug 2016 19:15:06 +0000 (15:15 -0400)]
messages/MOSDMap: mark as enlighten OSDMap encoder

Back in 70094407c184baaf963577cc0d98edde12e8e9da we require that anyone
encoding an OSDMap pass in a special feature bit indicating that they are
'enlightened' and understand the rules around encoding OSDMaps (basically,
only mons get to do it).  We forgot to update MOSDMap, which may have to
reencode an OSDMap without some features to talk to a really old client.
We haven't noticed until now because the old set of features we had to do
this for are really old.

We are about to introduce new features (for addr2 encoding) that change
that, and this fix is needed to prevent an assert when doing a reencode.

Signed-off-by: Sage Weil <sage@redhat.com>
9 years agoMerge pull request #9121 from dillaman/wip-15688
Mykola Golub [Mon, 29 Aug 2016 18:31:00 +0000 (21:31 +0300)]
Merge pull request #9121 from dillaman/wip-15688

librbd: initial hooks for client-side, image-extent cache in IO path

Reviewed-by: Mykola Golub <mgolub@mirantis.com>
9 years agoceph_test_objectstore: disable statfs checks on available space 10910/head
Sage Weil [Fri, 26 Aug 2016 19:15:22 +0000 (15:15 -0400)]
ceph_test_objectstore: disable statfs checks on available space

These are too sensitive to other activity, like bluefs
files and bluefs vs bluestore freespace balancing.

Signed-off-by: Sage Weil <sage@redhat.com>
9 years agoceph_test_objectstore: fix ObjectStore/StoreTest.BluestoreStatFSTest/2 on ssd
Sage Weil [Fri, 26 Aug 2016 19:07:57 +0000 (15:07 -0400)]
ceph_test_objectstore: fix ObjectStore/StoreTest.BluestoreStatFSTest/2 on ssd

Signed-off-by: Sage Weil <sage@redhat.com>
9 years agoMerge pull request #10897 from batrick/caps-graphic
John Spray [Mon, 29 Aug 2016 16:15:36 +0000 (17:15 +0100)]
Merge pull request #10897 from batrick/caps-graphic

doc: add graphic for cap bit field

Reviewed-by: Jeff Layton <jlayton@redhat.com>
Reviewed-by: Greg Farnum <gfarnum@redhat.com>
9 years agoinstall-deps.sh: initial distro detection based on /etc/os-release 10017/head
Jan Fajerski [Wed, 29 Jun 2016 07:40:07 +0000 (09:40 +0200)]
install-deps.sh: initial distro detection based on /etc/os-release

This avoids initial distribution detection (before lsb_release is available)
based on the presence of a specific package manager. This caused an
issue with openSUSE Tumbleweed before since both zypper and apt-get are
available.

Signed-off-by: Jan Fajerski <jfajerski@suse.com>
9 years agoclient: extend the Btime test to cover fstatx as well
Jeff Layton [Mon, 29 Aug 2016 11:16:41 +0000 (07:16 -0400)]
client: extend the Btime test to cover fstatx as well

Signed-off-by: Jeff Layton <jlayton@redhat.com>
9 years agoclient: add a ceph_fstatx
Jeff Layton [Mon, 29 Aug 2016 11:16:41 +0000 (07:16 -0400)]
client: add a ceph_fstatx

Signed-off-by: Jeff Layton <jlayton@redhat.com>
9 years agoclient: add the ability to set the btime
Jeff Layton [Mon, 29 Aug 2016 11:16:41 +0000 (07:16 -0400)]
client: add the ability to set the btime

This adds a new set of libcephfs calls: ceph_ll_setattrx and
ceph_setattrx. This allows clients to set the btime in addition to other
values that are typically settable via ceph_setattr calls.

Currently, the setattrx mask uses the same CEPH_SETATTR values that the
ceph_setattr interface uses. I'm not sure this is what we will want
though. Would it be better to rephrase that via STATX_* constants?

Signed-off-by: Jeff Layton <jlayton@redhat.com>
9 years agoMDS: allow the MDS to accept requests to set the btime
Jeff Layton [Mon, 29 Aug 2016 11:16:41 +0000 (07:16 -0400)]
MDS: allow the MDS to accept requests to set the btime

Unfortunately, the only option here is to rev the MClientRequest
version as the ceph_mds_request_head is not currently versioned. Add a
new ceph_mds_request_head, which contains a new ceph_mds_request_args
structure.

The new ceph_mds_request_head is now versioned via a __le16
at the beginning of it, and then the args structure is expanded to hold
the btime. When we get a legacy ceph_mds_request_head, we just set the
new fields to zero. When encoding a reply to a legacy client, we simply
don't encode the version in the head, or the btime in the setattr union
member.

Reluctantly-Signed-off-by: Jeff Layton <jlayton@redhat.com>
9 years agocephfs: rename ceph_mds_request_head and _args with a _legacy postfix
Jeff Layton [Mon, 29 Aug 2016 11:16:40 +0000 (07:16 -0400)]
cephfs: rename ceph_mds_request_head and _args with a _legacy postfix

We're going to need to introduce new versions of these structures in
order to expand the setattr union member. Rename the existing ones so
that it's clear that they are for legacy clients and servers.

Signed-off-by: Jeff Layton <jlayton@redhat.com>
9 years agotest: add test for change attribute of directories
Jeff Layton [Mon, 29 Aug 2016 11:16:40 +0000 (07:16 -0400)]
test: add test for change attribute of directories

Signed-off-by: Jeff Layton <jlayton@redhat.com>
9 years agomds: add change_attr to frag_info_t
Jeff Layton [Mon, 29 Aug 2016 11:16:40 +0000 (07:16 -0400)]
mds: add change_attr to frag_info_t

To give us change attribute support for directories. Whenever we gather
the dirfrags, we just select the largest change_attr out of the set.

Signed-off-by: Jeff Layton <jlayton@redhat.com>
9 years agomds: make frag_info_t add_dirty() function take a pointer to touched_mtime
Jeff Layton [Mon, 29 Aug 2016 11:16:40 +0000 (07:16 -0400)]
mds: make frag_info_t add_dirty() function take a pointer to touched_mtime

...rather than messing around with references. While we're at it, we
can also make the argument optional, which allows us to drop an unused
stack variable from CDir::split.

Signed-off-by: Jeff Layton <jlayton@redhat.com>
9 years agotests: add a ChangeAttr test
Jeff Layton [Mon, 29 Aug 2016 11:16:40 +0000 (07:16 -0400)]
tests: add a ChangeAttr test

Signed-off-by: Jeff Layton <jlayton@redhat.com>
9 years agolibcephfs: return the change_attr in the statx.stx_version field
Jeff Layton [Mon, 29 Aug 2016 11:16:40 +0000 (07:16 -0400)]
libcephfs: return the change_attr in the statx.stx_version field

Signed-off-by: Jeff Layton <jlayton@redhat.com>
9 years agomds/client: bump the change_attr at the appropriate time for files
Jeff Layton [Mon, 29 Aug 2016 11:16:40 +0000 (07:16 -0400)]
mds/client: bump the change_attr at the appropriate time for files

The semantics for a change_attr are that it should be incremented
whenever there is a change to the ctime in the inode. Add those
increments for the simple case of regular files. Directories however can
be fragmented so we'll need to do something more elaborate there.

Signed-off-by: Jeff Layton <jlayton@redhat.com>
9 years agomds: ensure that change_attr reflects metadata changes on clients that hold CAP_FILE_EXCL
Jeff Layton [Mon, 29 Aug 2016 14:33:10 +0000 (10:33 -0400)]
mds: ensure that change_attr reflects metadata changes on clients that hold CAP_FILE_EXCL

Suppose we have two clients. client1 holds FILE_EXCL cap and client2
holds AUTH_EXCL.  Both have the change_attr at the same value (call it
1).  client1 does 2 writes and its change_attr goes to 3. The client1
then queries for the change_attr and gets back 3 from the cache. The MDS
then recalls FILE_EXCL from client1 and now the MDS and client1 have the
same change_attr (3).

client2 then does a chmod on the file, and its change_attr goes to 2.
client1 then does a statx with STX_VERSION|STX_MODE. The MDS recalls the
AUTH_EXCL cap from client2, the change_attr in the MClientCaps is less
than the one in the MDS inode, so it gets discarded. client1 then sees
a new mode but the change_attr value has not changed, which violates the
rules.

Fix this with an extra increment of the MDS copy of the change_attr when
the caps being returned are dirty, and they don't contain exclusive write
caps.

Signed-off-by: Jeff Layton <jlayton@redhat.com>
9 years agosrc/test/common/CMakeLists.txt: BLKID exclude
Willem Jan Withagen [Sun, 31 Jul 2016 21:53:03 +0000 (23:53 +0200)]
src/test/common/CMakeLists.txt: BLKID exclude

Signed-off-by: Willem Jan Withagen <wjw@digiware.nl>
9 years agosrc/test/CMakeLists.txt: RADOSGW|RBD exclude
Willem Jan Withagen [Sun, 31 Jul 2016 21:51:46 +0000 (23:51 +0200)]
src/test/CMakeLists.txt: RADOSGW|RBD exclude

Signed-off-by: Willem Jan Withagen <wjw@digiware.nl>