]> git.apps.os.sepia.ceph.com Git - ceph.git/log
ceph.git
8 years agoos/bluestore: assert shared blob cache cleared on split 10963/head
Sage Weil [Wed, 7 Sep 2016 14:32:50 +0000 (10:32 -0400)]
os/bluestore: assert shared blob cache cleared on split

Signed-off-by: Sage Weil <sage@redhat.com>
8 years agoos/bluestore: dump some stats after fsck
Sage Weil [Tue, 6 Sep 2016 22:34:34 +0000 (18:34 -0400)]
os/bluestore: dump some stats after fsck

Signed-off-by: Sage Weil <sage@redhat.com>
8 years agoos/bluestore: instrument onode reshard events
Sage Weil [Tue, 6 Sep 2016 21:45:04 +0000 (17:45 -0400)]
os/bluestore: instrument onode reshard events

Signed-off-by: Sage Weil <sage@redhat.com>
8 years agoos/bluestore: instrument transaction count
Sage Weil [Tue, 6 Sep 2016 21:44:46 +0000 (17:44 -0400)]
os/bluestore: instrument transaction count

Signed-off-by: Sage Weil <sage@redhat.com>
8 years agoos/bluestore: instrument big/small writes
Sage Weil [Tue, 6 Sep 2016 21:32:15 +0000 (17:32 -0400)]
os/bluestore: instrument big/small writes

Signed-off-by: Sage Weil <sage@redhat.com>
8 years agoos/bluestore: make blob_t unused helpers use logical length
Sage Weil [Tue, 6 Sep 2016 20:50:46 +0000 (16:50 -0400)]
os/bluestore: make blob_t unused helpers use logical length

These were taking min_alloc_size, but this can change
across mounts; better to use the logical blob length
instead (that's what we want anyway!).

Signed-off-by: Sage Weil <sage@redhat.com>
8 years agoos/bluestore: use block_size for allocator unit
Sage Weil [Tue, 6 Sep 2016 19:28:14 +0000 (15:28 -0400)]
os/bluestore: use block_size for allocator unit

We need to handle objects written during previous mounts
that may have had a smaller min_alloc_size.  Use
block_size, which is a safe lower bound.

Signed-off-by: Sage Weil <sage@redhat.com>
8 years agoos/bluestore: fix fsck used_block bitmap
Sage Weil [Tue, 6 Sep 2016 19:16:52 +0000 (15:16 -0400)]
os/bluestore: fix fsck used_block bitmap

This has to be block_size bits because min_alloc_size
can vary over mounts.

Signed-off-by: Sage Weil <sage@redhat.com>
8 years agoos/bluestore: optimize compress_extent_map
Sage Weil [Fri, 2 Sep 2016 18:08:48 +0000 (14:08 -0400)]
os/bluestore: optimize compress_extent_map

Only examine the range we just wrote to (and to the left
and right).

Signed-off-by: Sage Weil <sage@redhat.com>
8 years agoos/bluestore: make nid and blobid allocation less racy
Sage Weil [Thu, 1 Sep 2016 18:31:52 +0000 (14:31 -0400)]
os/bluestore: make nid and blobid allocation less racy

We could bump the _max value for a TransContext in it's
prepare state, have it wait for a long time on IO, and
let another txc allocate and commit something with
an id higher than the previous max.

Fix this first by pushing the max ids into the
TransContext where we can deal with them at commit time,
and then making _kv_sync_thread bump the committed
max in a safe way.

Note that this will need to change if/when we do
these commits in parallel.

Signed-off-by: Sage Weil <sage@redhat.com>
8 years agoos/bluestore: shard extent map
Sage Weil [Fri, 2 Sep 2016 16:50:39 +0000 (12:50 -0400)]
os/bluestore: shard extent map

Rewrote much of the persistence of onode metadata.  The
highlights:

 - extents and blobs stored together (the blob with the
   first referencing extent).
 - extents sharded across multiple k/v keys
 - if a blob if referenced from multiple blobs, it's
   stored in the onode key (called a "spanning blob").
 - when we clone a blob we copy the metadata, but mark
   it shared and put (just) the ref_map on the underlying
   blocks in a shared_blob key.  at this point we also
   assign a globally unique id (sbid = shared blob id)
   so the key has a unique name.
 - we instantiate a SharedBlob in memory regardless of
   whether we need to load the ref_map (which is only
   needed for deallocations!).  the BufferSpace is
   attached to this SharedBlob so we get unified caching
   across clones.

Signed-off-by: Sage Weil <sage@redhat.com>
8 years agoceph_test_objectstore: test shards for longer
Sage Weil [Thu, 1 Sep 2016 21:55:37 +0000 (17:55 -0400)]
ceph_test_objectstore: test shards for longer

Signed-off-by: Sage Weil <sage@redhat.com>
8 years agoceph_test_objectstore: occasional umount/fsck/mount
Sage Weil [Thu, 1 Sep 2016 21:44:22 +0000 (17:44 -0400)]
ceph_test_objectstore: occasional umount/fsck/mount

Signed-off-by: Sage Weil <sage@redhat.com>
8 years agoceph_test_objectstore: add SyntheticMatrixSharding
Sage Weil [Fri, 2 Sep 2016 16:41:54 +0000 (12:41 -0400)]
ceph_test_objectstore: add SyntheticMatrixSharding

Signed-off-by: Sage Weil <sage@redhat.com>
8 years agoMerge pull request #10870 from ceph/wip_build_cls_rbd
Casey Bodley [Tue, 6 Sep 2016 21:47:32 +0000 (17:47 -0400)]
Merge pull request #10870 from ceph/wip_build_cls_rbd

cmake: add dependency from ceph_smalliobenchrbd to cls libraries

Reviewed-by: Jason Dillaman <jdillama@redhat.com>
Reviewed-by: Casey Bodley <cbodley@redhat.com>
8 years agoMerge pull request #10912 from stiopaa1/journal_journaltimer_moveCRemoveSet
Jason Dillaman [Tue, 6 Sep 2016 20:20:54 +0000 (16:20 -0400)]
Merge pull request #10912 from stiopaa1/journal_journaltimer_moveCRemoveSet

journal: move JournalTrimmer::C_RemoveSet struct

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
8 years agoAdd dependencies from ceph_smalliobenchrbd on cls_rbd, cls_journal, 10870/head
J. Eric Ivancich [Tue, 6 Sep 2016 17:27:43 +0000 (13:27 -0400)]
Add dependencies from ceph_smalliobenchrbd on cls_rbd, cls_journal,
and cls_lock.

Signed-off-by: J. Eric Ivancich <ivancich@redhat.com>
8 years agoMerge pull request #10984 from ceph/wip-kfsx-exclusive-lock
Jason Dillaman [Tue, 6 Sep 2016 15:39:32 +0000 (11:39 -0400)]
Merge pull request #10984 from ceph/wip-kfsx-exclusive-lock

test/librbd/fsx: enable exclusive-lock feature in krbd mode

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
8 years agoMerge pull request #10964 from liewegas/wip-bluefs-compaction
Sage Weil [Tue, 6 Sep 2016 15:10:43 +0000 (10:10 -0500)]
Merge pull request #10964 from liewegas/wip-bluefs-compaction

os/bluestore/BlueFS: fix async compaction logging bug

8 years agoMerge pull request #10966 from liewegas/wip-bluefs-runway
Sage Weil [Tue, 6 Sep 2016 14:12:53 +0000 (09:12 -0500)]
Merge pull request #10966 from liewegas/wip-bluefs-runway

os/bluestore/BlueFS: factor unflushed log into runway calculation

Reviewed-by: Varada Kari <varada.kari@sandisk.com>
Reviewed-by: Somnath Roy <somnath.roy@sandisk.com>
8 years agoMerge pull request #10970 from xiexingguo/xxg-wip-honour-alloweio-flag
Sage Weil [Tue, 6 Sep 2016 14:11:28 +0000 (09:11 -0500)]
Merge pull request #10970 from xiexingguo/xxg-wip-honour-alloweio-flag

os/bluestore: honour allow-eio flag; use global compressor if possible

Reviewed-by: Sage Weil <sage@redhat.com>
8 years agoMerge pull request #10974 from dillaman/wip-17210
Mykola Golub [Tue, 6 Sep 2016 12:38:43 +0000 (15:38 +0300)]
Merge pull request #10974 from dillaman/wip-17210

librbd: potential double-unwatch of watch handle upon error

Reviewed-by: Mykola Golub <mgolub@mirantis.com>
8 years agoMerge pull request #10774 from renhwztetecs/renhw-wip-cephfs-python-umount
John Spray [Tue, 6 Sep 2016 11:51:26 +0000 (12:51 +0100)]
Merge pull request #10774 from renhwztetecs/renhw-wip-cephfs-python-umount

libcephfs: add unmount function in cephfs.pyx

Reviewed-by: John Spray <john.spray@redhat.com>
Reviewed-by: Ramana Raja <rraja@redhat.com>
8 years agoMerge pull request #10914 from batrick/mds-no-maxmds
John Spray [Tue, 6 Sep 2016 11:50:20 +0000 (12:50 +0100)]
Merge pull request #10914 from batrick/mds-no-maxmds

mds: remove max_mds config option

Reviewed-by: John Spray <john.spray@redhat.com>
8 years agoMerge pull request #10922 from ceph/wip-jlayton-statx
jtlayton [Tue, 6 Sep 2016 11:04:51 +0000 (07:04 -0400)]
Merge pull request #10922 from ceph/wip-jlayton-statx

First pile of statx patches

8 years agotest/cephfs: add mount&&unmount test 10774/head
huanwen ren [Tue, 30 Aug 2016 07:06:26 +0000 (15:06 +0800)]
test/cephfs: add mount&&unmount test

Signed-off-by: huanwen ren <ren.huanwen@zte.com.cn>
8 years agoMerge pull request #10981 from SUSE/wip-fix-install-deps
Nathan Cutler [Mon, 5 Sep 2016 16:17:52 +0000 (18:17 +0200)]
Merge pull request #10981 from SUSE/wip-fix-install-deps

build/ops: fix /etc/os-release parsing in install-deps.sh

Reviewed-by: Jan Fajerski <jfajerski@suse.com>
Reviewed-by: Loic Dachary <ldachary@redhat.com>
8 years agobuild/ops: fix /etc/os-release parsing in install-deps.sh 10981/head
Nathan Cutler [Mon, 5 Sep 2016 13:40:33 +0000 (15:40 +0200)]
build/ops: fix /etc/os-release parsing in install-deps.sh

85a370e35fc42031a7f7e24dea9e50a649c0f309 introduced the DISTRO variable whose
value was obtained by parsing /etc/os-release like this:

DISTRO=$(grep  "^ID=" /etc/os-release | sed "s/ID=//")

This unfortunately picks up the double-quotes, so on a CentOS system DISTRO
will be equal to '"centos"'.

Signed-off-by: Nathan Cutler <ncutler@suse.com>
8 years agoos/bluestore: use global compressor if possible 10970/head
xie xingguo [Sat, 3 Sep 2016 09:51:19 +0000 (17:51 +0800)]
os/bluestore: use global compressor if possible

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
8 years agoMerge pull request #10969 from xiexingguo/xxg-wip-fix-cache-logger
Igor Fedotov [Mon, 5 Sep 2016 11:51:59 +0000 (14:51 +0300)]
Merge pull request #10969 from xiexingguo/xxg-wip-fix-cache-logger

os/bluestore: replace store with logger in Cache

Reviewed-by: Igor Fedotov <ifedotov@mirantis.com>
8 years agoMerge pull request #9592 from trociny/wip-15632
Jason Dillaman [Mon, 5 Sep 2016 01:54:36 +0000 (21:54 -0400)]
Merge pull request #9592 from trociny/wip-15632

librbd: API methods to directly acquire and release the exclusive lock

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
8 years agolibrbd: potential double-unwatch of watch handle upon error 10974/head
Jason Dillaman [Sun, 4 Sep 2016 14:48:48 +0000 (10:48 -0400)]
librbd: potential double-unwatch of watch handle upon error

Fixes: http://tracker.ceph.com/issues/17210
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
8 years agoMerge pull request #9758 from SUSE/wip-btrfs-not-ideal
Loic Dachary [Sun, 4 Sep 2016 17:58:19 +0000 (19:58 +0200)]
Merge pull request #9758 from SUSE/wip-btrfs-not-ideal

doc: remove btrfs contradiction

Reviewed-by: Loic Dachary <ldachary@redhat.com>
8 years agoMerge pull request #10469 from wjwithagen/wip-wjw-compileflag
Kefu Chai [Sun, 4 Sep 2016 03:10:34 +0000 (11:10 +0800)]
Merge pull request #10469 from wjwithagen/wip-wjw-compileflag

cmake: supress more warnings

Reviewed-by: Kefu Chai <kchai@redhat.com>
8 years agoos/bluestore: honour allow-eio flag during read
xie xingguo [Sat, 3 Sep 2016 08:49:29 +0000 (16:49 +0800)]
os/bluestore: honour allow-eio flag during read

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
8 years agoos/bluestore: check iterator before using during fsck 10969/head
xie xingguo [Sat, 3 Sep 2016 03:38:00 +0000 (11:38 +0800)]
os/bluestore: check iterator before using during fsck

Also drop redundant checking.

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
8 years agoos/bluestore: replace store with logger in Cache
xie xingguo [Sat, 3 Sep 2016 01:06:43 +0000 (09:06 +0800)]
os/bluestore: replace store with logger in Cache

As what we really want is to log cache-related statistics.

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
8 years agoos/bluestore/BlueFS: factor unflushed log into runway calculation 10966/head
Sage Weil [Fri, 2 Sep 2016 19:31:20 +0000 (15:31 -0400)]
os/bluestore/BlueFS: factor unflushed log into runway calculation

Include the size of the in-memory buffer we haven't yet flushed
into the runway calculation when deciding whether to allocate
more log space.

Reported-by: Somnath Roy <somnath.roy@sandisk.com>
Signed-off-by: Sage Weil <sage@redhat.com>
8 years agoMerge pull request #10945 from dillaman/wip-17188
Mykola Golub [Fri, 2 Sep 2016 19:00:41 +0000 (22:00 +0300)]
Merge pull request #10945 from dillaman/wip-17188

librbd: deadlock when replaying journal during image open

Reviewed-by: Mykola Golub <mgolub@mirantis.com>
8 years agoSQUASH: client: request all shared caps if ctime was requested 10922/head
Jeff Layton [Fri, 2 Sep 2016 18:27:08 +0000 (14:27 -0400)]
SQUASH: client: request all shared caps if ctime was requested

Much like the change_attr, ctime changes can potentially be cached on
clients. Request all shared caps if the want mask specifies the ctime.

Signed-off-by: Jeff Layton <jlayton@redhat.com>
8 years agoMerge pull request #10746 from ceph/wip-async-default
Sage Weil [Fri, 2 Sep 2016 18:02:57 +0000 (13:02 -0500)]
Merge pull request #10746 from ceph/wip-async-default

msg: make async backend default

8 years agoscripts/bdev_grep: parse bluefs style extents too 10964/head
Sage Weil [Fri, 2 Sep 2016 16:07:10 +0000 (12:07 -0400)]
scripts/bdev_grep: parse bluefs style extents too

(they use + instead of ~)

Signed-off-by: Sage Weil <sage@redhat.com>
8 years agoos/bluestore/BlueFS: fix async compaction txn
Sage Weil [Fri, 2 Sep 2016 16:06:49 +0000 (12:06 -0400)]
os/bluestore/BlueFS: fix async compaction txn

Our transaction writes are labeled with a seq and uuid
to avoid replaying over garbage.

Two bugs, one real, one potential.

1) The second async compaction transactoin didn't have
its seq and uuid set, so replay always stopped.

2) We were writing two separate transactions, one with
all the new metadata, and the next one with a jump to
the new log offset.  If the first write completed but
it was torn and the second transaction didn't hit disk,
we might see an old transaction with seq == 2 and the
same uuid and replay that instead.

Fix both of these by making the async log txn one single
transaction that jumps directly to the new log offset.

Signed-off-by: Sage Weil <sage@redhat.com>
8 years agoMerge pull request #10938 from liewegas/wip-bluestore-csum-error
Sage Weil [Fri, 2 Sep 2016 16:23:40 +0000 (11:23 -0500)]
Merge pull request #10938 from liewegas/wip-bluestore-csum-error

os/bluestore: fix/improve csum error message

Reviewed-by: Igor Fedotov <ifedotov@mirantis.com>
8 years agoMerge pull request #10961 from xiexingguo/xxg-wip-bluestore-09-02
Sage Weil [Fri, 2 Sep 2016 16:23:27 +0000 (11:23 -0500)]
Merge pull request #10961 from xiexingguo/xxg-wip-bluestore-09-02

os/bluestore: add cache-related stats

Reviewed-by: Igor Fedotov <ifedotov@mirantis.com>
Reviewed-by: Sage Weil <sage@redhat.com>
8 years agoMerge pull request #10953 from liewegas/wip-bluestore-fixes
Igor Fedotov [Fri, 2 Sep 2016 15:26:07 +0000 (18:26 +0300)]
Merge pull request #10953 from liewegas/wip-bluestore-fixes

os/bluestore: misc fixes

Reviewed- by: Igor Fedotov ifedotov@mirantis.com

8 years agotest/librbd/fsx: enable exclusive-lock feature in krbd mode 10984/head
Ilya Dryomov [Thu, 18 Aug 2016 10:14:22 +0000 (12:14 +0200)]
test/librbd/fsx: enable exclusive-lock feature in krbd mode

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
8 years agoSQUASH: mds/client: add routines to copy to/from ceph_mds_request_head_legacy
Jeff Layton [Fri, 2 Sep 2016 12:36:59 +0000 (08:36 -0400)]
SQUASH: mds/client: add routines to copy to/from ceph_mds_request_head_legacy

Signed-off-by: Jeff Layton <jlayton@redhat.com>
8 years agoos/bluestore: trivial cleanups 10961/head
xie xingguo [Fri, 2 Sep 2016 10:10:33 +0000 (18:10 +0800)]
os/bluestore: trivial cleanups

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
8 years agoos/bluestore: don't overwrite type if it does already exist and at desired type
xie xingguo [Fri, 2 Sep 2016 09:48:39 +0000 (17:48 +0800)]
os/bluestore: don't overwrite type if it does already exist and at desired type

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
8 years agosrc/CMakeLists.txt: -Woverloaded-virtual is already included in -Wall on Clang 10469/head
Willem Jan Withagen [Thu, 28 Jul 2016 09:25:03 +0000 (11:25 +0200)]
src/CMakeLists.txt: -Woverloaded-virtual is already included in -Wall on Clang

Signed-off-by: Willem Jan Withagen <wjw@digiware.nl>
8 years agoMerge pull request #10956 from ceph/wip-cmake
Kefu Chai [Fri, 2 Sep 2016 09:22:37 +0000 (17:22 +0800)]
Merge pull request #10956 from ceph/wip-cmake

cmake: pass -DINTEL* to gf-complete cflags

Reviewed-by: Loic Dachary <ldachary@redhat.com>
8 years agoos/bluestore: conditional checking label of bluefs devices
xie xingguo [Fri, 2 Sep 2016 08:45:05 +0000 (16:45 +0800)]
os/bluestore: conditional checking label of bluefs devices

As NVMe devices may not support to do so.

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
8 years agocmake: pass -DINTEL* to gf-complete cflags 10956/head
Kefu Chai [Fri, 2 Sep 2016 06:38:21 +0000 (14:38 +0800)]
cmake: pass -DINTEL* to gf-complete cflags

* in addition to the -march=foobar, we also need to pass -DFOOBAR to
  gf-complete sources' cflags, so that the SIMD instructions can be
  generated as expected.
* also extract the compiler support for instruction extensions detections
  into SIMDExt.cmake.
* and only check the compiler support if CMAKE_SYSTEM_PROCESSOR matches
  with the instruction-set/arch to be checked.

Signed-off-by: tone.zhang <tone.zhang@linaro.org>
Signed-off-by: Kefu Chai <kchai@redhat.com>
8 years agoMerge pull request #10934 from tchaikov/wip-kill-warnings
Brad Hubbard [Fri, 2 Sep 2016 07:32:46 +0000 (17:32 +1000)]
Merge pull request #10934 from tchaikov/wip-kill-warnings

Wip kill warnings

Reviewed-by: Brad Hubbard <bhubbard@redhat.com>
8 years agoos/bluestore: end scope of std::hex properly
xie xingguo [Fri, 2 Sep 2016 03:37:05 +0000 (11:37 +0800)]
os/bluestore: end scope of std::hex properly

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
8 years agoos/bluestore: add onode/buffer cache statistics
xie xingguo [Fri, 2 Sep 2016 03:33:33 +0000 (11:33 +0800)]
os/bluestore: add onode/buffer cache statistics

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
8 years agoSQUASH: client: querying for stx_version field requires getting all shared caps
Jeff Layton [Thu, 1 Sep 2016 22:49:46 +0000 (18:49 -0400)]
SQUASH: client: querying for stx_version field requires getting all shared caps

Otherwise, someone could potentially query for just CEPH_STATX_VERSION,
and see it as unchanged, even when there are changes buffered up on
other clients.

By doing this, I don't think we'll incur any perf hit in the common use
case which is ganesha querying for all attributes. We are adding Xs
here, but unless there is a lot of xattr activity I don't think that
will generally cause a lot of exclusive cap recalls.

Also, we don't actually need AUTH caps to fetch CEPH_STATX_RDEV, remove
that from statx_to_mask. PIN is sufficient there.

Signed-off-by: Jeff Layton <jlayton@redhat.com>
8 years agoSQUASH: mds/client: don't get clever with ceph_mds_request_head_legacy encode/decode
Jeff Layton [Thu, 1 Sep 2016 18:18:55 +0000 (14:18 -0400)]
SQUASH: mds/client: don't get clever with ceph_mds_request_head_legacy encode/decode

As requested by Greg...

Declare a legacy object on the stack, and do an extra copy to/from it.

Signed-off-by: Jeff Layton <jlayton@redhat.com>
8 years agoSQUASH: client: always take a change_attr update from the server if it's newer
Jeff Layton [Thu, 1 Sep 2016 15:16:06 +0000 (11:16 -0400)]
SQUASH: client: always take a change_attr update from the server if it's newer

I think this is more correct in the client. If we see a newer
change_attr, then we always want to take note of that fact, even if
we have the right caps.

Signed-off-by: Jeff Layton <jlayton@redhat.com>
8 years agoSQUASH: Revert "mds: ensure that change_attr reflects metadata changes on clients...
Jeff Layton [Thu, 1 Sep 2016 15:16:05 +0000 (11:16 -0400)]
SQUASH: Revert "mds: ensure that change_attr reflects metadata changes on clients that hold CAP_FILE_EXCL"

This reverts commit 26ab2fa10041ce34b25aa1535715131a2709a27f.

I don't think this is necessary. We don't necessarily need to increment
the change attribute on every change, as long as we ensure that it gets
bumped iff there were changes since the last time you queried for it.

IOW: it's ok to have a single change_attr change encompass a large set
of changes, as long as you ensure that that it is larger by at least 1
after all of those changes.

In order to look at (and potentially cache) other attributes under that
change_attribute (e.g. owner or mode), you need to recall any outstanding
exclusive caps for those attrs. That causes their change_attrs to be
synched to the largest, which is enough to ensure that it changed in
some way.

Signed-off-by: Jeff Layton <jlayton@redhat.com>
8 years agoSQUASH: client: request AUTH caps on the directory during create if we'll need to...
Jeff Layton [Thu, 1 Sep 2016 18:35:59 +0000 (14:35 -0400)]
SQUASH: client: request AUTH caps on the directory during create if we'll need to check perms locally

Signed-off-by: Jeff Layton <jlayton@redhat.com>
8 years agoMerge pull request #10517 from wjwithagen/wip-wjw-freebsd-cmake-execludes
Kefu Chai [Thu, 1 Sep 2016 17:38:44 +0000 (01:38 +0800)]
Merge pull request #10517 from wjwithagen/wip-wjw-freebsd-cmake-execludes

cmake: FreeBSD specific excludes in CMakeLists.txt files

Reviewed-by: Kefu Chai <kchai@redhat.com>
8 years agoceph_test_objectstore: fix TryMoveRename test 10953/head
Sage Weil [Thu, 1 Sep 2016 16:37:27 +0000 (12:37 -0400)]
ceph_test_objectstore: fix TryMoveRename test

Put the objects in the right pool.

Signed-off-by: Sage Weil <sage@redhat.com>
8 years agoceph_test_objectstore: clean up Rename test a bit
Sage Weil [Thu, 1 Sep 2016 16:04:27 +0000 (12:04 -0400)]
ceph_test_objectstore: clean up Rename test a bit

Drop useless remove calls.

Leave in the no-op setattr call since it triggered a
bluestore bug :)

Signed-off-by: Sage Weil <sage@redhat.com>
8 years agoceph_test_objectstore: assert umount succeeds
Sage Weil [Wed, 31 Aug 2016 20:49:13 +0000 (16:49 -0400)]
ceph_test_objectstore: assert umount succeeds

Signed-off-by: Sage Weil <sage@redhat.com>
8 years agoos/bluestore: make object existence check robust
Sage Weil [Thu, 1 Sep 2016 15:43:55 +0000 (11:43 -0400)]
os/bluestore: make object existence check robust

We may have the OnodeRef defined but the object may have
exists==false.  Move the check out of the lookup block
so that we get the right behavior.

Signed-off-by: Sage Weil <sage@redhat.com>
8 years agoos/bluestore: use _setattr for SETATTR
Sage Weil [Thu, 1 Sep 2016 15:20:17 +0000 (11:20 -0400)]
os/bluestore: use _setattr for SETATTR

This is a bit faster.

Signed-off-by: Sage Weil <sage@redhat.com>
8 years agoMerge pull request #10843 from liewegas/wip-osdmap-reencode
Sage Weil [Thu, 1 Sep 2016 14:56:36 +0000 (09:56 -0500)]
Merge pull request #10843 from liewegas/wip-osdmap-reencode

messages/MOSDMap: mark as enlighten OSDMap encoder

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>
8 years agoMerge pull request #10950 from xiexingguo/xxg-wip-blue-09-01
Sage Weil [Thu, 1 Sep 2016 14:22:56 +0000 (09:22 -0500)]
Merge pull request #10950 from xiexingguo/xxg-wip-blue-09-01

os/bluestore: gifting bluefs more carefully

Reviewed-by: Sage Weil <sage@redhat.com>
8 years agoos/bluestore: don't over-gift if bluefs free is smaller than bluestore_bluefs_min 10950/head
xie xingguo [Thu, 1 Sep 2016 10:42:58 +0000 (18:42 +0800)]
os/bluestore: don't over-gift if bluefs free is smaller than bluestore_bluefs_min

- If free space of bluefs is smaller than bluestore_bluefs_min, then raise it
  exactly up to bluestore_bluefs_min.

- Pre-check if bluestore_bluefs_min will exceed bluestore_bluefs_max_ratio * bluefs_total,
  so if free space of bluefs is smaller than bluestore_bluefs_min_ratio * bluefs_total,
  we'll still be able to gift.

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
8 years agoos/bluestore: end scope of std::hex properly
xie xingguo [Thu, 1 Sep 2016 03:33:25 +0000 (11:33 +0800)]
os/bluestore: end scope of std::hex properly

So it does not extend to the following output stream and
causes unexpected behaviour.

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
8 years agovstart.sh: add option to turn on multimds 10914/head
Patrick Donnelly [Wed, 24 Aug 2016 23:01:30 +0000 (19:01 -0400)]
vstart.sh: add option to turn on multimds

This makes the necessary `ceph fs` calls to get it running.

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
8 years agomds: remove max_mds config option
Patrick Donnelly [Mon, 29 Aug 2016 20:25:10 +0000 (16:25 -0400)]
mds: remove max_mds config option

It is now required that all changes to max_mds use the run-time `ceph fs
set max_mds` command. The rationale for this change is that it is
confusing to have a configuration for max_mds which is only observed at
file system creation.

Fixes: http://tracker.ceph.com/issues/17105
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
8 years agolibrbd: deadlock when replaying journal during image open 10945/head
Jason Dillaman [Thu, 1 Sep 2016 01:33:54 +0000 (21:33 -0400)]
librbd: deadlock when replaying journal during image open

Fixes: http://tracker.ceph.com/issues/17188
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
8 years agolibrbd: improve image state machine debug log messages
Jason Dillaman [Thu, 1 Sep 2016 00:56:54 +0000 (20:56 -0400)]
librbd: improve image state machine debug log messages

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
8 years agoMerge pull request #10404 from ilc/build_fix
Sage Weil [Wed, 31 Aug 2016 21:19:40 +0000 (16:19 -0500)]
Merge pull request #10404 from ilc/build_fix

make-dist/ceph.spec.in: Fix srpm build breakage.

Reviewed-by: Ken Dreyer <kdreyer@redhat.com>
8 years agoMerge pull request #10846 from stiopaa1/mds_mdstable_addConstToMemFun
Gregory Farnum [Wed, 31 Aug 2016 21:05:21 +0000 (14:05 -0700)]
Merge pull request #10846 from stiopaa1/mds_mdstable_addConstToMemFun

mds/MDSTable: add const to member functions

Reviewed-by: Greg Farnum <gfarnum@redhat.com>
8 years agoMerge pull request #10781 from stiopaa1/osdc_objectcacher_moveClassesToCC
Gregory Farnum [Wed, 31 Aug 2016 21:04:03 +0000 (14:04 -0700)]
Merge pull request #10781 from stiopaa1/osdc_objectcacher_moveClassesToCC

osdc/ObjectCacher: move C_ReadFinish, C_RetryRead

Reviewed-by: Greg Farnum <gfarnum@redhat.com>
8 years agoMerge pull request #10744 from stiopaa1/osdc_journaler_moveCDelayFlushClass
Gregory Farnum [Wed, 31 Aug 2016 21:03:50 +0000 (14:03 -0700)]
Merge pull request #10744 from stiopaa1/osdc_journaler_moveCDelayFlushClass

osdc/Journaler: move C_DelayFlush class to .cc

Reviewed-by: Greg Farnum <gfarnum@redhat.com>
8 years agoMerge pull request #10668 from stiopaa1/mon_mdsmonitor_moveCUpdatedClass
Gregory Farnum [Wed, 31 Aug 2016 21:03:43 +0000 (14:03 -0700)]
Merge pull request #10668 from stiopaa1/mon_mdsmonitor_moveCUpdatedClass

mon/MDSMonitor: move C_Updated class to .cc file

Reviewed-by: Greg Farnum <gfarnum@redhat.com>
8 years agoMerge pull request #10628 from jcsp/wip-16909
Gregory Farnum [Wed, 31 Aug 2016 21:02:57 +0000 (14:02 -0700)]
Merge pull request #10628 from jcsp/wip-16909

mon: tear down standby replays on MDS rank stop

Reviewed-by: Greg Farnum <gfarnum@redhat.com>
8 years agoMerge pull request #10937 from ceph/dang-build
Ali Maredia [Wed, 31 Aug 2016 19:53:39 +0000 (15:53 -0400)]
Merge pull request #10937 from ceph/dang-build

Allow compressor build without YASM

Reviewed-by: Ali Maredia <amaredia@redhat.com>
8 years agoos/bluestore: improve failed csum debug output 10938/head
Sage Weil [Tue, 30 Aug 2016 18:13:24 +0000 (14:13 -0400)]
os/bluestore: improve failed csum debug output

Signed-off-by: Sage Weil <sage@redhat.com>
8 years agoscript/bdev_grep: update to use hex
Sage Weil [Wed, 31 Aug 2016 15:19:29 +0000 (11:19 -0400)]
script/bdev_grep: update to use hex

Signed-off-by: Sage Weil <sage@redhat.com>
9 years agoAllow compressor build without YASM 10937/head
Daniel Gryniewicz [Wed, 31 Aug 2016 16:00:42 +0000 (12:00 -0400)]
Allow compressor build without YASM

The isa-l compressors need YASM to build.  Conditionalize on that.

Signed-off-by: Daniel Gryniewicz <dang@redhat.com>
9 years agoMerge pull request #10931 from xiexingguo/xxg-wip-fix-isa-l-assert
Igor Fedotov [Wed, 31 Aug 2016 14:29:42 +0000 (17:29 +0300)]
Merge pull request #10931 from xiexingguo/xxg-wip-fix-isa-l-assert

os/bluestore: get rid off "isa-l" type in ZLibCompressor ctor

Reviewed-by: Igor Fedotov <ifedotov@mirantis.com>
Reviewed-by: Sage Weil <sage@redhat.com>
9 years agoMerge pull request #10923 from xiexingguo/xxg-wip-bluefs-08-30
Sage Weil [Wed, 31 Aug 2016 14:21:03 +0000 (09:21 -0500)]
Merge pull request #10923 from xiexingguo/xxg-wip-bluefs-08-30

os/bluestore/bluefs: use map to track dirty files

Reviewed-by: Sage Weil <sage@redhat.com>
9 years agoos/bluestore: get rid off "isa-l" type in ZLibCompressor ctor 10931/head
xie xingguo [Wed, 31 Aug 2016 07:26:24 +0000 (15:26 +0800)]
os/bluestore: get rid off "isa-l" type in ZLibCompressor ctor

This is introduced by https://github.com/ceph/ceph/pull/10158 and causes
the following failure:

/home/xxg/ceph-dev/src/os/bluestore/bluestore_types.h: 284: FAILED assert(0 == "invalid compression algorithm")

 ceph version v11.0.0-1941-g47c8a21 (47c8a214356983f0539f24fe1c5724b3bbae0406)
 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x85) [0x7f5668c1ad45]
 2: (BlueStore::_do_alloc_write(BlueStore::TransContext*, BlueStore::WriteContext*)+0x1a65) [0x7f5668b19d55]
 3: (BlueStore::_do_write(BlueStore::TransContext*, boost::intrusive_ptr<BlueStore::Collection>&, boost::intrusive_ptr<BlueStore::Onode>, unsigned long, unsigned long, ceph::buffer::list&, unsigned int)+0x4ac) [0x7f5668b3b3fc]
 4: (BlueStore::_write(BlueStore::TransContext*, boost::intrusive_ptr<BlueStore::Collection>&, boost::intrusive_ptr<BlueStore::Onode>&, unsigned long, unsigned long, ceph::buffer::list&, unsigned int)+0x10f) [0x7f5668b3bf2f]
 5: (BlueStore::_txc_add_transaction(BlueStore::TransContext*, ObjectStore::Transaction*)+0x1067) [0x7f5668b405f7]
 6: (BlueStore::queue_transactions(ObjectStore::Sequencer*, std::vector<ObjectStore::Transaction, std::allocator<ObjectStore::Transaction> >&, std::shared_ptr<TrackedOp>, ThreadPool::TPHandle*)+0x382) [0x7f5668b42432]
 7: (SyntheticWorkloadState::write()+0x3a5) [0x7f5668a2ace5]
 8: (doSyntheticTest(boost::scoped_ptr<ObjectStore>&, unsigned long, unsigned long, unsigned long)+0xb9d) [0x7f56689dfa7d]
 9: (do_matrix_choose(char const* (*) [10], int, int, int, boost::scoped_ptr<ObjectStore>&)+0x29d) [0x7f56689dfffd]
 10: (do_matrix_choose(char const* (*) [10], int, int, int, boost::scoped_ptr<ObjectStore>&)+0x9e) [0x7f56689dfdfe]
 11: (do_matrix_choose(char const* (*) [10], int, int, int, boost::scoped_ptr<ObjectStore>&)+0x9e) [0x7f56689dfdfe]
 12: (do_matrix_choose(char const* (*) [10], int, int, int, boost::scoped_ptr<ObjectStore>&)+0x9e) [0x7f56689dfdfe]
 13: (do_matrix_choose(char const* (*) [10], int, int, int, boost::scoped_ptr<ObjectStore>&)+0x9e) [0x7f56689dfdfe]
 14: (do_matrix_choose(char const* (*) [10], int, int, int, boost::scoped_ptr<ObjectStore>&)+0x9e) [0x7f56689dfdfe]
 15: (do_matrix(char const* (*) [10], boost::scoped_ptr<ObjectStore>&)+0x317) [0x7f56689e0347]
 16: (()+0x297f3c) [0x7f56689e0f3c]
 17: (StoreTest_SyntheticMatrixCompressionAlgorithm_Test::TestBody()+0x118) [0x7f56689e1078]
 18: (void testing::internal::HandleExceptionsInMethodIfSupported<testing::Test, void>(testing::Test*, void (testing::Test::*)(), char const*)+0x43) [0x7f5668d673e3]
 19: (testing::Test::Run()+0xb7) [0x7f5668d5a127]
 20: (testing::TestInfo::Run()+0x9e) [0x7f5668d5a1ce]
 21: (testing::TestCase::Run()+0xa5) [0x7f5668d5a2d5]
 22: (testing::internal::UnitTestImpl::RunAllTests()+0x278) [0x7f5668d5a5b8]
 23: (testing::UnitTest::Run()+0x68) [0x7f5668d5a888]
 24: (main()+0x2f4) [0x7f566895be84]
 25: (__libc_start_main()+0xf5) [0x7f5663e5ab15]
 26: (()+0x265ba5) [0x7f56689aeba5]
 NOTE: a copy of the executable, or `objdump -rdS <executable>` is needed to interpret this.

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
9 years agotest/perf_local: fix -Wmaybe-uninitialized warnings 10934/head
Kefu Chai [Wed, 31 Aug 2016 10:46:53 +0000 (18:46 +0800)]
test/perf_local: fix -Wmaybe-uninitialized warnings

Signed-off-by: Kefu Chai <kchai@redhat.com>
9 years agotest,pybind: silence -Wsign-compare warnings
Kefu Chai [Wed, 31 Aug 2016 10:44:36 +0000 (18:44 +0800)]
test,pybind: silence -Wsign-compare warnings

Signed-off-by: Kefu Chai <kchai@redhat.com>
9 years agoos/CMakeLists.txt: undubble kv.cc usage. 10517/head
Willem Jan Withagen [Wed, 31 Aug 2016 09:23:06 +0000 (11:23 +0200)]
os/CMakeLists.txt: undubble kv.cc usage.

 - kv.cc is both in bluestore and in kstore.
   But atm. both are the same
 - Bluestore is not build on FreeBSD (no AIO)

So use the kstore one to be the one to build against.

Signed-off-by: Willem Jan Withagen <wjw@digiware.nl>
9 years agoos/bluestore/bluefs: drop redundant eof check for read_random() 10923/head
xie xingguo [Wed, 31 Aug 2016 02:47:50 +0000 (10:47 +0800)]
os/bluestore/bluefs: drop redundant eof check for read_random()

Because we have done the sanity check and fixed the length
at the entrance, so we don't have to check it again,
especially in the while loop, which is less efficient.

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
9 years agoos/bluestore/bluefs: update mtime of inode in unified path
xie xingguo [Wed, 31 Aug 2016 01:55:04 +0000 (09:55 +0800)]
os/bluestore/bluefs: update mtime of inode in unified path

To reduce code redundancy.

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
9 years agoos/bluestore/bluefs: use map to track dirty files
xie xingguo [Tue, 30 Aug 2016 07:37:46 +0000 (15:37 +0800)]
os/bluestore/bluefs: use map to track dirty files

So dirty files are arranged by dirty_seq and then
those of same dirty_seq are grouped into list and
as a result we can clean up dirty files much faster
during _flush_and_sync_log().

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
9 years agolibcephfs: add unmount function in cephfs.pyx
huanwen ren [Thu, 18 Aug 2016 11:59:34 +0000 (19:59 +0800)]
libcephfs: add unmount function in cephfs.pyx

we need unmount function when change the mount point

Signed-off-by: huanwen ren <ren.huanwen@zte.com.cn>
9 years agoMerge pull request #10158 from Ved-vampir/zlib_isal_extension
Dan Mick [Tue, 30 Aug 2016 22:22:08 +0000 (15:22 -0700)]
Merge pull request #10158 from Ved-vampir/zlib_isal_extension

Isa-l extention for zlib compression plugin

9 years agoMerge pull request #10858 from atheism/rgw_getattr-return
Matt Benjamin [Tue, 30 Aug 2016 19:38:19 +0000 (15:38 -0400)]
Merge pull request #10858 from atheism/rgw_getattr-return

rgw: remove duplicated calls to getattr

verified: f23

9 years agoMerge pull request #10927 from liewegas/wip-0.94.9
Alfredo Deza [Tue, 30 Aug 2016 19:11:05 +0000 (15:11 -0400)]
Merge pull request #10927 from liewegas/wip-0.94.9

doc/release-notes: v0.94.9

Reviewed-by: Alfredo Deza <adeza@redhat.com>
9 years agodoc/release-notes: v0.94.9 10927/head
Sage Weil [Tue, 30 Aug 2016 19:08:49 +0000 (15:08 -0400)]
doc/release-notes: v0.94.9

Signed-off-by: Sage Weil <sage@redhat.com>