]> git.apps.os.sepia.ceph.com Git - ceph.git/log
ceph.git
9 years agoMerge pull request #7547 from yuyuyu101/wip-14679
Haomai Wang [Mon, 8 Feb 2016 06:15:37 +0000 (14:15 +0800)]
Merge pull request #7547 from yuyuyu101/wip-14679

AsyncConnection: avoid debug log in cleanup_handler

9 years agoMerge pull request #7554 from tchaikov/wip-ceph-disk-test
Kefu Chai [Sun, 7 Feb 2016 17:52:16 +0000 (01:52 +0800)]
Merge pull request #7554 from tchaikov/wip-ceph-disk-test

tests: ceph-disk.sh should compare with the resolved path

Reviewed-by: Loic Dachary <ldachary@redhat.com>
9 years agoMerge pull request #7551 from JevonQ/update-libcephfs-version
Sage Weil [Sun, 7 Feb 2016 14:01:04 +0000 (09:01 -0500)]
Merge pull request #7551 from JevonQ/update-libcephfs-version

libcephfs: update LIBCEPHFS_VERSION to indicate the interface was changed

Reviewed-by: Sage Weil <sage@redhat.com>
9 years agotests: ceph-disk.sh should compare with the resolved path 7554/head
Kefu Chai [Sun, 7 Feb 2016 12:36:58 +0000 (04:36 -0800)]
tests: ceph-disk.sh should compare with the resolved path

Signed-off-by: Kefu Chai <kchai@redhat.com>
9 years agolibcephfs: Update LIBCEPHFS_VERSION to indicate the interface was changed 7551/head
Jevon Qiao [Sun, 7 Feb 2016 06:00:53 +0000 (14:00 +0800)]
libcephfs: Update LIBCEPHFS_VERSION to indicate the interface was changed

Signed-off-by: Jevon Qiao <scaleqiao@gmail.com>
9 years agoMerge pull request #7549 from tchaikov/wip-fix-ceph-disk-tests
Kefu Chai [Sat, 6 Feb 2016 15:43:31 +0000 (23:43 +0800)]
Merge pull request #7549 from tchaikov/wip-fix-ceph-disk-tests

ceph-disk/test: fix test_prepare.py::TestPrepare tests

Reviewed-by: Loic Dachary <ldachary@redhat.com>
9 years agoceph-disk/test: fix test_prepare.py::TestPrepare tests 7549/head
Kefu Chai [Sat, 6 Feb 2016 14:25:38 +0000 (06:25 -0800)]
ceph-disk/test: fix test_prepare.py::TestPrepare tests

actually we are verifying if the variable is an instance of
specified class. for example, the `prepare.data` should be
a `PrepareFilestoreData` if `--bluestore` is not specified.

Signed-off-by: Kefu Chai <kchai@redhat.com>
9 years agoAsyncConnection: avoid debug log in cleanup_handler 7547/head
Haomai Wang [Sat, 6 Feb 2016 06:52:43 +0000 (14:52 +0800)]
AsyncConnection: avoid debug log in cleanup_handler

local connection will be stop and call cleanup_handler after messenger
is down
introduced in
commit(https://github.com/ceph/ceph/commit/9da2fffd31562ed5d0b795d7862b3ebec66aba40)

Fix #14679

Signed-off-by: Haomai Wang <haomai@xsky.com>
9 years agoMerge pull request #7540 from dillaman/wip-librados-test-stub-unwatch
Josh Durgin [Sat, 6 Feb 2016 01:41:03 +0000 (17:41 -0800)]
Merge pull request #7540 from dillaman/wip-librados-test-stub-unwatch

librados_test_stub: protect against notify/unwatch race

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
9 years agoMerge pull request #7529 from dillaman/wip-14412
Josh Durgin [Sat, 6 Feb 2016 01:39:03 +0000 (17:39 -0800)]
Merge pull request #7529 from dillaman/wip-14412

librbd: journal framework for tracking exclusive lock transitions

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
9 years agoMerge pull request #7377 from liewegas/wip-datadir-search
Sage Weil [Sat, 6 Feb 2016 01:03:50 +0000 (20:03 -0500)]
Merge pull request #7377 from liewegas/wip-datadir-search

config: add $data_dir/config to config search path

9 years agoMerge pull request #7466 from adamemerson/wip-bsd-time-second-try
Sage Weil [Sat, 6 Feb 2016 01:03:35 +0000 (20:03 -0500)]
Merge pull request #7466 from adamemerson/wip-bsd-time-second-try

common: time: have skewing-now call non-skewing now

9 years agoMerge pull request #7463 from dachary/wip-13422-pid-file
Sage Weil [Sat, 6 Feb 2016 01:03:00 +0000 (20:03 -0500)]
Merge pull request #7463 from dachary/wip-13422-pid-file

global: do not start two daemons with a single pid-file (part 2)

Reviewed-by: Kefu Chai <kchai@redhat.com>
9 years agoMerge pull request #7460 from xiexingguo/xxg-wip-block
Sage Weil [Sat, 6 Feb 2016 01:02:34 +0000 (20:02 -0500)]
Merge pull request #7460 from xiexingguo/xxg-wip-block

osd: blockdevice: avoid implicit cast and add guard

Reviewed-by: Sage Weil <sage@redhat.com>
9 years agoMerge pull request #7530 from xiexingguo/xxg-wip-nid
Sage Weil [Sat, 6 Feb 2016 01:02:07 +0000 (20:02 -0500)]
Merge pull request #7530 from xiexingguo/xxg-wip-nid

osd: bluestore: assign nid for newly created onode

Reviewed-by: Sage Weil <sage@redhat.com>
9 years agoMerge pull request #7521 from liewegas/wip-memstore
Sage Weil [Sat, 6 Feb 2016 01:01:47 +0000 (20:01 -0500)]
Merge pull request #7521 from liewegas/wip-memstore

osd: memstore: set Collection::cid on create

9 years agoMerge pull request #7486 from ceph/wip-bluestore-oref
Sage Weil [Sat, 6 Feb 2016 01:01:32 +0000 (20:01 -0500)]
Merge pull request #7486 from ceph/wip-bluestore-oref

osd: bluestore: fix two wal bugs; reuse OnodeRefs when applying txc

Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>
9 years agoMerge pull request #7531 from jjhuo/master
Sage Weil [Sat, 6 Feb 2016 01:01:05 +0000 (20:01 -0500)]
Merge pull request #7531 from jjhuo/master

osd: bluestore: remove unused intrusive member hook in enode

Reviewed-by: Sage Weil <sage@redhat.com>
9 years agoMerge pull request #7535 from liewegas/wip-bluefs-mkfs
Sage Weil [Sat, 6 Feb 2016 01:00:31 +0000 (20:00 -0500)]
Merge pull request #7535 from liewegas/wip-bluefs-mkfs

osd: bluestore/BlueFS: initialize super block_size earlier in mkfs

Reviewed-by: Dan Mick <dmick@redhat.com>
9 years agoMerge pull request #7536 from liewegas/wip-bluestore-block
Sage Weil [Sat, 6 Feb 2016 01:00:07 +0000 (20:00 -0500)]
Merge pull request #7536 from liewegas/wip-bluestore-block

osd: bluestore: fix block device file creation

9 years agoMerge pull request #7541 from linuxbox2/cmake-feb5
Ali Maredia [Sat, 6 Feb 2016 00:53:08 +0000 (19:53 -0500)]
Merge pull request #7541 from linuxbox2/cmake-feb5

Cmake feb5

9 years agocmake: add libboost_system to EXTRALIBS 7541/head
Matt Benjamin [Fri, 5 Feb 2016 21:43:43 +0000 (16:43 -0500)]
cmake: add libboost_system to EXTRALIBS

This concisely fixes several unittest builds, and reflects the
fact that this library dependency has moved into several areas
of the codebase (libcephfs, librbd, librgw).

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
9 years agocmake: don't try to install src/ceph-disk directory
Matt Benjamin [Fri, 5 Feb 2016 20:54:19 +0000 (15:54 -0500)]
cmake: don't try to install src/ceph-disk directory

The proper cmake boilerplate to call ceph-disk/ceph-disk's
python installer is not present, FYI.

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
9 years agocmake: don't add_test cephtool_test_osd twice
Matt Benjamin [Fri, 5 Feb 2016 20:51:00 +0000 (15:51 -0500)]
cmake: don't add_test cephtool_test_osd twice

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
9 years agolibrados_test_stub: protect against notify/unwatch race 7540/head
Jason Dillaman [Fri, 5 Feb 2016 21:14:39 +0000 (16:14 -0500)]
librados_test_stub: protect against notify/unwatch race

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
9 years agolibrbd: partial implementation of journal client / tag allocation 7529/head
Jason Dillaman [Fri, 5 Feb 2016 04:42:25 +0000 (23:42 -0500)]
librbd: partial implementation of journal client / tag allocation

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
9 years agolibrbd: initial version of journal client and tag metadata
Jason Dillaman [Fri, 5 Feb 2016 03:54:10 +0000 (22:54 -0500)]
librbd: initial version of journal client and tag metadata

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
9 years agojournal: added tag support methods
Jason Dillaman [Thu, 4 Feb 2016 21:27:32 +0000 (16:27 -0500)]
journal: added tag support methods

librbd, for example, will allocate a new tag after acquiring
the exclusive lock.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
9 years agocls_journal: new get_client / client_update methods
Jason Dillaman [Fri, 5 Feb 2016 01:32:54 +0000 (20:32 -0500)]
cls_journal: new get_client / client_update methods

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
9 years agocls_journal: client registration should hold opaque data structure
Jason Dillaman [Thu, 4 Feb 2016 18:46:50 +0000 (13:46 -0500)]
cls_journal: client registration should hold opaque data structure

The opaque structure will support journal client applications to
store and retrieve complex state.

Fixes: #13298
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
9 years agocls_journal: new tag management methods and handling
Jason Dillaman [Wed, 3 Feb 2016 22:33:24 +0000 (17:33 -0500)]
cls_journal: new tag management methods and handling

In the case of librbd, a new tag will be allocated when the
exclusive lock is acquired.  All tags for the same dataset
(e.g. librbd image) will belong to the same class.  Tags are
automatically pruned on tag create / client unregister
if no other clients' commit position would require the tags.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
9 years agojournal: switched entry tags to use id instead of string
Jason Dillaman [Mon, 1 Feb 2016 23:16:38 +0000 (18:16 -0500)]
journal: switched entry tags to use id instead of string

Later commits will add the ability to allocate tags and
associate them with registered clients.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
9 years agoMerge pull request #7474 from jack-changtao/wip-cmake2
Kefu Chai [Fri, 5 Feb 2016 17:14:58 +0000 (01:14 +0800)]
Merge pull request #7474 from jack-changtao/wip-cmake2

CMake: For CMake version <= 2.8.11, use LINK_PRIVATE and LINK_PUBLIC

Reviewed-by: Kefu Chai <kchai@redhat.com>
9 years agoMerge pull request #7108 from jdurgin/wip-rbd-mirroring
Jason Dillaman [Fri, 5 Feb 2016 17:02:32 +0000 (12:02 -0500)]
Merge pull request #7108 from jdurgin/wip-rbd-mirroring

rbd-mirror: daemon skeleton

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
9 years agotime: Remove constexpr from non-literal type in test 7466/head
Adam C. Emerson [Fri, 5 Feb 2016 16:22:55 +0000 (11:22 -0500)]
time: Remove constexpr from non-literal type in test

This fixes a compile error under clang and fixes another FreeBSD porting
issue.

Signed-off-by: Adam C. Emerson <aemerson@redhat.com>
9 years agotime: Have skewing-now call non-skewing now
Adam C. Emerson [Mon, 1 Feb 2016 15:40:54 +0000 (10:40 -0500)]
time: Have skewing-now call non-skewing now

For the real-time clocks, Ceph's testing infrastructure likes to be able to
inject a skew. To avoid pulling CephContext into ceph_time.h these are moved to
ceph_time.cc. The original way this was done called clock_gettime in both
places.

This is an unnecessary duplication and apparently error-prone. So only call
clock_gettime from one place.

Signed-off-by: Adam C. Emerson <aemerson@redhat.com>
9 years agoMerge pull request #7538 from tchaikov/wip-fix-exe-test
Loic Dachary [Fri, 5 Feb 2016 15:52:28 +0000 (22:52 +0700)]
Merge pull request #7538 from tchaikov/wip-fix-exe-test

tests: mon: "chmod +x mon-created-time.sh"

Reviewed-by: Loic Dachary <ldachary@redhat.com>
9 years agotests: mon: "chmod +x mon-created-time.sh" 7538/head
Kefu Chai [Fri, 5 Feb 2016 15:45:33 +0000 (23:45 +0800)]
tests: mon: "chmod +x mon-created-time.sh"

otherwise it always fails to run.

Signed-off-by: Kefu Chai <kchai@redhat.com>
9 years agoos/bluestore/KernelDevice: use _exit(1) for failure injection 7486/head
Sage Weil [Fri, 5 Feb 2016 14:51:59 +0000 (09:51 -0500)]
os/bluestore/KernelDevice: use _exit(1) for failure injection

This way we don't generate a core file, which spams the log and trips up
teuthology.

Signed-off-by: Sage Weil <sage@redhat.com>
9 years agoos/bluestore: fix block device file creation 7536/head
Sage Weil [Fri, 5 Feb 2016 14:20:40 +0000 (09:20 -0500)]
os/bluestore: fix block device file creation

Just make a separate flag to indicate whether we create a block
file.  This lets us drop the weird touch in vstart.sh, and default
to creating a token 'block' file on --mkfs.

Signed-off-by: Sage Weil <sage@redhat.com>
9 years agoMerge pull request #7519 from xiexingguo/xxg-wip-finisher 7461/head
Sage Weil [Fri, 5 Feb 2016 14:17:43 +0000 (09:17 -0500)]
Merge pull request #7519 from xiexingguo/xxg-wip-finisher

librados: shutdown finisher in a more graceful way

Reviewed-by: Sage Weil <sage@redhat.com>
9 years agoMerge pull request #7506 from liewegas/wip-bluestore-block
Sage Weil [Fri, 5 Feb 2016 14:17:15 +0000 (09:17 -0500)]
Merge pull request #7506 from liewegas/wip-bluestore-block

osd: bluestore: set default wal size; improve weird implicit block file creation logic

Reviewed-by: Loic Dachary <ldachary@redhat.com>
9 years agoMerge pull request #7503 from tchaikov/wip-cmake-spdk
Sage Weil [Fri, 5 Feb 2016 14:16:59 +0000 (09:16 -0500)]
Merge pull request #7503 from tchaikov/wip-cmake-spdk

build: spdk submodule; cmake

Reviewed-by: Kefu Chai <kchai@redhat.com>
9 years agoMerge pull request #7218 from dachary/wip-13942-ceph-disk
Sage Weil [Fri, 5 Feb 2016 14:15:26 +0000 (09:15 -0500)]
Merge pull request #7218 from dachary/wip-13942-ceph-disk

ceph-disk: support bluestore

Reviewed-by: Sage Weil <sage@redhat.com>
9 years agoos/bluestore/BlueFS: initialize super block_size earlier in mkfs 7535/head
Sage Weil [Fri, 5 Feb 2016 13:41:03 +0000 (08:41 -0500)]
os/bluestore/BlueFS: initialize super block_size earlier in mkfs

Reported-by: Dan Mick <dmick@redhat.com>
Signed-off-by: Sage Weil <sage@redhat.com>
9 years agoos/kstore: kill dead code 7530/head
xie xingguo [Fri, 5 Feb 2016 09:41:07 +0000 (17:41 +0800)]
os/kstore: kill dead code

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
9 years agoos/kstore: fix race condition
xie xingguo [Fri, 5 Feb 2016 09:37:49 +0000 (17:37 +0800)]
os/kstore: fix race condition

The get_omap_iterator shall check the existence of onode too,
which may race with the rename method.

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
9 years agoos/bluestore: fix race condition
xie xingguo [Fri, 5 Feb 2016 09:34:51 +0000 (17:34 +0800)]
os/bluestore: fix race condition

The get_omap_iterator shall check the existence of onode too, which may
race with the rename method.

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
9 years agoos/bluestore: remove unused intrusive member hook in enode 7531/head
Jianjian Huo [Fri, 5 Feb 2016 05:44:38 +0000 (00:44 -0500)]
os/bluestore: remove unused intrusive member hook in enode

Enode only uses intrusive unordered_set base hook, member hook is not neccesary.
This should save some memory space, when we have tons of enodes.

Signed-off-by: Jianjian Huo <samuel.huo@gmail.com>
9 years agoglobal: do not start two daemons with a single pid-file (part 2) 7463/head
Loic Dachary [Mon, 1 Feb 2016 12:32:13 +0000 (19:32 +0700)]
global: do not start two daemons with a single pid-file (part 2)

Fixes the following bugs:

* the fd is open(O_WRONLY) and cannot be read from, safe_read
  always fails and never removes the pid file.

* pidfile_open(g_conf) is close(STDOUT_FILENO) and there is a risk that
  pidfile_open gets STDOUT_FILENO only to have it closed and redirected
  to /dev/null.

* Before writing the file, ftruncate it so that overriding a file
  containing the pid 1234 with the pid 89 does not end up being
  a file with 8934.

* Before reading the file, lseek back to offset 0 otherwise it
  will read nothing.

* tests_pidfile was missing an argument when failing
  TEST_without_pidfile and killed all process with ceph in their name,
  leading to chaos and no useful error message.

* lstat(fd) cannot possibly return a result different from the one
  obtained right after the file was open, stat(path) must be used
  instead.

In addition to fixing the bugs above, refactor the pidfile.cc
implementation to:

* be systematic about error reporting (using cerr for when removing
  the pidfile because derr is not available at this point and derr
  when creating the pidfile).

* replace pidfile_open / pidfile_write with just pidfile_write since
  there never is a case when they are not used together.

More test cases are added to test_pidfile to verify the bugs above are
fixed.

http://tracker.ceph.com/issues/13422 Fixes: #13422

Signed-off-by: Loic Dachary <loic@dachary.org>
9 years agoos/kstore: assign nid for newly created onode
xie xingguo [Fri, 5 Feb 2016 04:07:33 +0000 (12:07 +0800)]
os/kstore: assign nid for newly created onode

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
9 years agoos/bluestore: assign nid for newly created onode
xie xingguo [Fri, 5 Feb 2016 04:06:10 +0000 (12:06 +0800)]
os/bluestore: assign nid for newly created onode

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
9 years agoMerge pull request #7522 from ktdreyer/wip-admin-build-doc-deps
Kefu Chai [Fri, 5 Feb 2016 03:31:41 +0000 (11:31 +0800)]
Merge pull request #7522 from ktdreyer/wip-admin-build-doc-deps

admin/build-doc: depend on zlib1g-dev and graphviz

Reviewed-by: Kefu Chai <kchai@redhat.com>
9 years agounittest_config: cope with $data_dir not being expanded on non-daemons 7377/head
Sage Weil [Sat, 30 Jan 2016 18:56:21 +0000 (13:56 -0500)]
unittest_config: cope with $data_dir not being expanded on non-daemons

Signed-off-by: Sage Weil <sage@redhat.com>
9 years agocommon: add $data_dir/config to config search path
Sage Weil [Tue, 26 Jan 2016 21:06:35 +0000 (16:06 -0500)]
common: add $data_dir/config to config search path

Signed-off-by: Sage Weil <sage@redhat.com>
9 years agocommon/config: expand $data_dir in config path
Sage Weil [Tue, 26 Jan 2016 21:06:20 +0000 (16:06 -0500)]
common/config: expand $data_dir in config path

Signed-off-by: Sage Weil <sage@redhat.com>
9 years agoglobal: add data_dir_option for all daemons
Sage Weil [Tue, 26 Jan 2016 14:18:37 +0000 (09:18 -0500)]
global: add data_dir_option for all daemons

This let's us use a generic $data_dir substitution that will map
to rgw_data, osd_data, etc.

Signed-off-by: Sage Weil <sage@redhat.com>
9 years agoMerge pull request #7478 from jack-changtao/wip-cache-tier
Sage Weil [Thu, 4 Feb 2016 21:17:17 +0000 (16:17 -0500)]
Merge pull request #7478 from jack-changtao/wip-cache-tier

osd: enforce cache_min_evict_age

Reviewed-by: Sage Weil <sage@redhat.com>
9 years agoMerge pull request #7512 from athanatos/wip-admin-malloc
Sage Weil [Thu, 4 Feb 2016 21:15:54 +0000 (16:15 -0500)]
Merge pull request #7512 from athanatos/wip-admin-malloc

common: admin socket commands for tcmalloc heap get/set operations

Reviewed-by: Sage Weil <sage@redhat.com>
9 years agoMerge pull request #7510 from athanatos/wip-14278
Sage Weil [Thu, 4 Feb 2016 21:06:35 +0000 (16:06 -0500)]
Merge pull request #7510 from athanatos/wip-14278

osd: correctly remove pg shards which are no longer acting

Reviewed-by: Sage Weil <sage@redhat.com>
9 years agoMerge pull request #7511 from athanatos/wip-14423
Sage Weil [Thu, 4 Feb 2016 21:04:34 +0000 (16:04 -0500)]
Merge pull request #7511 from athanatos/wip-14423

Wip 14423

Reviewed-by: Sage Weil <sage@redhat.com>
9 years agoMerge pull request #7401 from batrick/systemd-sandbox
Sage Weil [Thu, 4 Feb 2016 20:25:05 +0000 (15:25 -0500)]
Merge pull request #7401 from batrick/systemd-sandbox

systemd: add systemd sandboxing to services

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Boris Ranto <branto@redhat.com>
9 years ago.gitignore: add rbd-mirror 7108/head
Josh Durgin [Thu, 4 Feb 2016 19:42:52 +0000 (11:42 -0800)]
.gitignore: add rbd-mirror

Signed-off-by: Josh Durgin <jdurgin@redhat.com>
9 years agoos/bluestore/BlueStore: fix enode uniqueness
Sage Weil [Thu, 4 Feb 2016 18:04:53 +0000 (13:04 -0500)]
os/bluestore/BlueStore: fix enode uniqueness

We were failing to set o->enode, which meant that there were
multiple instances of the same enode alive at once.  Avoid this
category of bug by changing _txc_release to take the onode ref
and assign it there, and removing almost all of the local EnodeRef
instances.

Signed-off-by: Sage Weil <sage@redhat.com>
9 years agoadmin/build-doc: depend on zlib1g-dev and graphviz 7522/head
Ken Dreyer [Thu, 4 Feb 2016 16:41:40 +0000 (09:41 -0700)]
admin/build-doc: depend on zlib1g-dev and graphviz

The docs currently require zlib1g-dev (on Ubuntu) and graphviz (on RHEL
and Ubuntu) in order to build.

Signed-off-by: Ken Dreyer <kdreyer@redhat.com>
9 years agoMerge pull request #7328 from ceph/wip-mtime-bugfix
Yehuda Sadeh [Thu, 4 Feb 2016 16:36:02 +0000 (08:36 -0800)]
Merge pull request #7328 from ceph/wip-mtime-bugfix

Bug fix for mtime anomalies in RadosGW and other places

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>
9 years agoos/bluestore/BlueStore: default Onode::true to false
Sage Weil [Thu, 4 Feb 2016 15:30:13 +0000 (10:30 -0500)]
os/bluestore/BlueStore: default Onode::true to false

This patch is mostly a wash, except that _rename now avoids doing the
_do_remove() work in the case where the object didn't exist before.

Reported-by: ie xingguo <xie.xingguo@zte.com.cn>
Signed-off-by: Sage Weil <sage@redhat.com>
9 years agoMerge pull request #7453 from ceph/wip-14566
Sage Weil [Thu, 4 Feb 2016 15:10:01 +0000 (10:10 -0500)]
Merge pull request #7453 from ceph/wip-14566

msg/async: fix potential race condition

9 years agoos/memstore/MemStore: set Collection::cid on create 7521/head
Sage Weil [Thu, 4 Feb 2016 14:24:22 +0000 (09:24 -0500)]
os/memstore/MemStore: set Collection::cid on create

This was broken by the collection handles merge in
2e52a8b17c348bb3356eb76a8a0f6ef6efbe5bd3 because the c->cid
value was never initialized and now we started to rely on it.

Signed-off-by: Sage Weil <sage@redhat.com>
9 years agoCacheTier: set cache_min_evict_age is effective 7478/head
changtao [Mon, 1 Feb 2016 17:23:37 +0000 (01:23 +0800)]
CacheTier: set cache_min_evict_age is effective

Signed-off-by: Tao Chang <changtao@hihuron.com>
9 years agoMerge pull request #7505 from xiexingguo/xxg-wip-kstorelru
Sage Weil [Thu, 4 Feb 2016 13:43:42 +0000 (08:43 -0500)]
Merge pull request #7505 from xiexingguo/xxg-wip-kstorelru

os/kstore: insert new onode to the front position of onode LRU

Reviewed-by: Sage Weil <sage@redhat.com>
9 years agoMerge pull request #7492 from jjhuo/master
Sage Weil [Thu, 4 Feb 2016 13:43:19 +0000 (08:43 -0500)]
Merge pull request #7492 from jjhuo/master

os/bluestore: insert new onode to the front position of onode LRU

Reviewed-by: Sage Weil <sage@redhat.com>
9 years agoMerge branch 'xxg-wip-14474' of git://github.com/xiexingguo/ceph
Sage Weil [Thu, 4 Feb 2016 13:36:06 +0000 (08:36 -0500)]
Merge branch 'xxg-wip-14474' of git://github.com/xiexingguo/ceph

Reviewed-by: Sage Weil <sage@redhat.com>
9 years agoMerge pull request #7380 from yuyuyu101/async-nip
Sage Weil [Thu, 4 Feb 2016 13:39:15 +0000 (08:39 -0500)]
Merge pull request #7380 from yuyuyu101/async-nip

msg/async: reduce extra tcp packet for message ack

9 years agoMerge pull request #7398 from theanalyst/fix/12744
Sage Weil [Thu, 4 Feb 2016 13:38:32 +0000 (08:38 -0500)]
Merge pull request #7398 from theanalyst/fix/12744

mon: PG Monitor should report waiting for backfill

Reviewed-by: Kefu Chai <kchai@redhat.com>
9 years agoMerge pull request #7447 from liewegas/wip-parallel-rados-test
Sage Weil [Thu, 4 Feb 2016 13:38:09 +0000 (08:38 -0500)]
Merge pull request #7447 from liewegas/wip-parallel-rados-test

test: run rados api tests in parallel

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
9 years agoMerge pull request #7455 from yuyuyu101/bufferlist-move
Sage Weil [Thu, 4 Feb 2016 13:37:55 +0000 (08:37 -0500)]
Merge pull request #7455 from yuyuyu101/bufferlist-move

buffer: use move construct to append/push_back/push_front

Reviewed-by: Sage Weil <sage@redhat.com>
9 years agoMerge pull request #7459 from dwj192/mon-created
Sage Weil [Thu, 4 Feb 2016 13:37:22 +0000 (08:37 -0500)]
Merge pull request #7459 from dwj192/mon-created

mon: fix monmap creation stamp

Reviewed-by: Sage Weil <sage@redhat.com>
9 years agoMerge pull request #7477 from xiexingguo/wip-fix-obj
Sage Weil [Thu, 4 Feb 2016 13:36:40 +0000 (08:36 -0500)]
Merge pull request #7477 from xiexingguo/wip-fix-obj

osdc/objecter: fix race condition and result code overflow

Reviewed-by: Sage Weil <sage@redhat.com>
9 years agoMerge pull request #7482 from branch-predictor/bp-faster-health
Sage Weil [Thu, 4 Feb 2016 13:33:56 +0000 (08:33 -0500)]
Merge pull request #7482 from branch-predictor/bp-faster-health

mon: reduce CPU and memory manager pressure of pg health check

Reviewed-by: Sage Weil <sage@redhat.com>
9 years agoMerge pull request #7518 from ceph/wip-14637-man-radosgw-admin-orphans
Orit Wasserman [Thu, 4 Feb 2016 10:05:54 +0000 (11:05 +0100)]
Merge pull request #7518 from ceph/wip-14637-man-radosgw-admin-orphans

doc: add orphans commands to radosgw-admin(8)

9 years agoceph-disk: use the type file for bluestore 7218/head
Loic Dachary [Sun, 24 Jan 2016 10:07:58 +0000 (17:07 +0700)]
ceph-disk: use the type file for bluestore

The type file in the OSD bluestore data exists and contains the
bluestore string. ceph-disk activate should use it instead of
the "osd objectstore" configuration value. It is better in case the
configuration file changes between prepare and activate.

The fsid file cannot be used by bluestore to signify that ceph-osd
--mkfs has completed successfully because it is pre-populated by
ceph-disk. Introduce the mkfs_done file, dedicated to this, instead of
overloading an existing file.

Signed-off-by: Sage Weil <sage@redhat.com>
Signed-off-by: Loic Dachary <loic@dachary.org>
9 years agotests: ceph-disk tests pid files must exist
Loic Dachary [Mon, 1 Feb 2016 11:26:05 +0000 (18:26 +0700)]
tests: ceph-disk tests pid files must exist

http://tracker.ceph.com/issues/13422 made it so ceph-osd won't start
unless the pidfile can be created successfully. The default location
being the current directory, ceph-osd must explicitly be told to write
in a directory where it has write permissions.

Signed-off-by: Loic Dachary <loic@dachary.org>
9 years agotests: ceph-disk qa workunit for bluestore
Loic Dachary [Thu, 28 Jan 2016 06:53:28 +0000 (13:53 +0700)]
tests: ceph-disk qa workunit for bluestore

Signed-off-by: Loic Dachary <loic@dachary.org>
9 years agoceph-disk: bluestore deactivate / destroy
Loic Dachary [Thu, 28 Jan 2016 04:59:10 +0000 (11:59 +0700)]
ceph-disk: bluestore deactivate / destroy

It is straightforward because it entirely relies on information
collected by ceph-disk list which has full support for bluestore.
It loops on all possible auxiliary devices (as found in Spaces.NAMES)
and does the associated deactivate / destruction which is merely about
handling dmcrypt map / unmap.

Signed-off-by: Loic Dachary <loic@dachary.org>
9 years agoceph-disk: bluestore list
Loic Dachary [Thu, 28 Jan 2016 04:12:47 +0000 (11:12 +0700)]
ceph-disk: bluestore list

The objectstore journal and the bluestore block auxiliary device are
handled in the same way. Each occurrence of journal in the code is
replaced with a variable.

A few helpers are added to the Ptype class to factorize the most common
lookups but the code logic is unmodified with one exception: the
more_osd_info previously added a journal_uuid entry regarless. If there
was no journal_uuid file, it would be None. It is changed to only add
the {block,journal}_uuid entry if the corresponding file exist.

Signed-off-by: Loic Dachary <loic@dachary.org>
9 years agoceph-disk: bluestore trigger
Loic Dachary [Thu, 28 Jan 2016 04:53:49 +0000 (11:53 +0700)]
ceph-disk: bluestore trigger

Copy paste the journal code and s/journal/block/

More work will be needed to support multiple auxiliary
devices (block.wal etc). But the goal is to minimize the change because
this commit is part of a series of commits focusing on refactoring
prepare, not the entire ceph-disk codebase.

Signed-off-by: Loic Dachary <loic@dachary.org>
9 years agoceph-disk: bluestore activate
Loic Dachary [Thu, 28 Jan 2016 04:48:55 +0000 (11:48 +0700)]
ceph-disk: bluestore activate

Only support the block file for now. The refactoring consist of
replacing main_activate_journal with main_activate_space and a name
argument (block, journal). More work will be needed to support multiple
auxiliary devices (block.wal etc). But the goal is to minimize the
change because this commit is part of a series of commits focusing on
refactoring prepare, not the entire ceph-disk codebase.

Signed-off-by: Loic Dachary <loic@dachary.org>
9 years agoceph-disk: bluestore prepare
Loic Dachary [Thu, 28 Jan 2016 04:43:22 +0000 (11:43 +0700)]
ceph-disk: bluestore prepare

Only support the block file for now. It is handled the same as the
journal, only with a different name (block) and it's own set of ptypes
depending on multipath or dmcrypt.

Signed-off-by: Loic Dachary <loic@dachary.org>
9 years agoceph-disk: refactor prepare
Loic Dachary [Tue, 19 Jan 2016 09:49:40 +0000 (16:49 +0700)]
ceph-disk: refactor prepare

The logic / code path is only modified to the extent necessary for the
refactor.

The Prepare class roughly replaces the prepare_main function but also
handles the prepare subcommand argument parsing. It creates the data and
journal objects and delegate the actual work to them via the prepare()
method.

The Prepare class assumes that preparing an OSD consists on the
following phases:

  * optionally prepare auxiliary devices, such as the journal
  * prepare a data directory or device
  * populate the data directory with fsid etc. and optionally
    symbolic links to the auxiliary devices

The PrepareDefault class is derived from Prepare and implements the
current model where there only is one auxiliary device, the journal.

The PrepareJournal class implements the *journal* functions
and is based on a generic class, PrepareSpace which handles the
allocation of an auxiliary device. The only journal specific feature is
left to the PrepareJournal class: querying the OSD to figure out if
a journal is wanted or not.

The OSD data directory is prepared via the PrepareData class. It creates
a file system if necessary (i.e. if a device) and populate the data
directory. Further preparation is then delegated to the auxiliary
devices (i.e. adding a symlink to the device for a journal).

There was some code paths related dmcrypt / multipath devices in
the prepare functions, although it is orthogonal. A class tree for
Devices was created to isolate that.

Although that was the primary reason for adding a new class tree, two
other aspects have also been moved there: ptypes and partition creation.
The ptypes are organized into a data structure with a few helpers in
the hope it will be easier to maintain. All references to the *_UUID
variables have been updated.

The creation of a partition is delegated to sgdisk and a wrapper helps
reduce the code redundancy.

The ptype of a given partition depends on the type of the device (is it
dmcrypt'ed or a multipath device ?). It is best implemented by
derivation so the prepare function does not need to be concerned about
how the ptype of a partition is determined.

Many functions could be refactored into a Device class and its
derivatives, but that was not done to minimize the size of the refactor.

  Device knows how to create a partition and figure out the ptype tobe
  DevicePartition a regular device partition
  DevicePartitionMultipath a partition of a multipath device
  DevicePartitionCrypt base class for luks/plain dmcrypt, can map/unmap
  DevicePartitionCryptPlain knows how to setup dmcrypt plain
  DevicePartitionCryptLuks knows how to setup dmcrypt plain

The CryptHelpers class is introduced to factorize the code snippets that
were duplicated in various places but that do not really belong
because they are convenience wrappers to figure out:

   * if dmrypt should be used
   * the keysize
   * the dmcrypt type (plain or luks)

Signed-off-by: Loic Dachary <loic@dachary.org>
9 years agolibrados/RadosClient: stop client's finisher in a more graceful way 7519/head
xie xingguo [Thu, 4 Feb 2016 08:22:56 +0000 (16:22 +0800)]
librados/RadosClient: stop client's finisher in a more graceful way

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
9 years agoos/filestore/JournalingObjectStore: stop journal in a more graceful way
xie xingguo [Thu, 4 Feb 2016 08:19:27 +0000 (16:19 +0800)]
os/filestore/JournalingObjectStore: stop journal in a more graceful way

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
9 years agopackaging: add an rbd-mirror package
Josh Durgin [Thu, 4 Feb 2016 07:44:10 +0000 (23:44 -0800)]
packaging: add an rbd-mirror package

Depend on ceph-common for /etc/ceph. Startup scripts will be added later.

Signed-off-by: Josh Durgin <jdurgin@redhat.com>
9 years agoman: stub manpage for rbd-mirror
Josh Durgin [Thu, 4 Feb 2016 07:41:58 +0000 (23:41 -0800)]
man: stub manpage for rbd-mirror

Signed-off-by: Josh Durgin <jdurgin@redhat.com>
9 years agorbd-mirror: skeleton of a mirroring daemon
Josh Durgin [Fri, 11 Dec 2015 23:20:19 +0000 (15:20 -0800)]
rbd-mirror: skeleton of a mirroring daemon

This isn't functional yet, since ImageReplayer doesn't do any replay
yet.  Just the parts for monitoring other clusters for changes
(ClusterWatcher and PoolWatcher) are tested, with simple
unit and functional tests.

Signed-off-by: Josh Durgin <jdurgin@redhat.com>
9 years agotests: generic ceph-disk qa check_osd_status
Loic Dachary [Thu, 28 Jan 2016 06:48:00 +0000 (13:48 +0700)]
tests: generic ceph-disk qa check_osd_status

The helper function no longer has journal hardcoded.

Signed-off-by: Loic Dachary <loic@dachary.org>
9 years agotests: pytest must not truncate data structures
Loic Dachary [Thu, 28 Jan 2016 04:07:47 +0000 (11:07 +0700)]
tests: pytest must not truncate data structures

Signed-off-by: Loic Dachary <loic@dachary.org>
9 years agotests: ceph-disk no longer test separate journals
Loic Dachary [Sun, 24 Jan 2016 10:09:00 +0000 (17:09 +0700)]
tests: ceph-disk no longer test separate journals

Remove unused argument.

Signed-off-by: Loic Dachary <loic@dachary.org>
9 years agotests: workaround ceph-disk global side effects
Loic Dachary [Tue, 19 Jan 2016 11:33:05 +0000 (18:33 +0700)]
tests: workaround ceph-disk global side effects

Because some variables are global in ceph-disk, tests that modify them
interact with each other in non-predictable ways. This will go away
eventually but requires a significant refactor. Workaround by running
one py.test per test file.

Signed-off-by: Loic Dachary <loic@dachary.org>
9 years agoceph-disk: move check_journal_reqs (no change)
Loic Dachary [Tue, 19 Jan 2016 09:43:42 +0000 (16:43 +0700)]
ceph-disk: move check_journal_reqs (no change)

To help with the prepare refactor.

Signed-off-by: Loic Dachary <loic@dachary.org>