]> git.apps.os.sepia.ceph.com Git - ceph.git/log
ceph.git
9 years agorbd: deprecate image format 1 7841/head
Jason Dillaman [Mon, 29 Feb 2016 02:27:50 +0000 (21:27 -0500)]
rbd: deprecate image format 1

The rbd cli will warn about the deprecation when attempting to create
image format 1 images.  librbd will log an error message when opening
a format 1 RBD image.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
9 years agoMerge pull request #7747 from vsjulu/master
Kefu Chai [Sat, 27 Feb 2016 16:01:55 +0000 (00:01 +0800)]
Merge pull request #7747 from vsjulu/master

xio: fix incorrect ip being assigned in case of multiple RDMA ports

Reviewed-by: Kefu Chai <kchai@redhat.com>
9 years agoMerge pull request #7630 from rzarzynski/wip-rgw-admin-token-cache
Yehuda Sadeh [Sat, 27 Feb 2016 14:22:09 +0000 (06:22 -0800)]
Merge pull request #7630 from rzarzynski/wip-rgw-admin-token-cache

rgw: add support for caching of Keystone admin token.

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>
9 years agoMerge pull request #7825 from ceph/wip-rgw-openssl
Yehuda Sadeh [Sat, 27 Feb 2016 14:17:15 +0000 (06:17 -0800)]
Merge pull request #7825 from ceph/wip-rgw-openssl

rgw: link civetweb with openssl (Sage, Marcus Watts)

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>
9 years agocmake fix: build civetweb with the right include path. 7825/head
Marcus Watts [Sat, 27 Feb 2016 07:34:14 +0000 (02:34 -0500)]
cmake fix: build civetweb with the right include path.

In an earlier commit, civetweb.c got moved into a separate cmake 'object'
library civetweb_common_objs so that it could be built just once for use
with several different targets.  At the time, there was a separate global
"include_directory" to give the right include path for including "civetweb.h".
A later commit in master created an rgw_a library, and restricted civetweb.h's
include path for only building rgw_a objects.  So, as the product of
these these two commits, the target_include_directories command for the
civetweb include path now needs to be applied to civetweb_common_objs
and not rgw_a.

Signed-off-by: Marcus Watts <mwatts@redhat.com>
9 years agoMerge pull request #7650 from avnerbh/fix1_rand
Sage Weil [Fri, 26 Feb 2016 23:58:28 +0000 (18:58 -0500)]
Merge pull request #7650 from avnerbh/fix1_rand

msg: add thread safety for "random" Messenger + fix wrong usage of random functions

Reviewed-by: Haomai Wang <haomai@xsky.com>
9 years agoMerge pull request #7651 from xiexingguo/xxg-wip-fixtypecheck
Sage Weil [Fri, 26 Feb 2016 23:57:19 +0000 (18:57 -0500)]
Merge pull request #7651 from xiexingguo/xxg-wip-fixtypecheck

osd: bluestore: improve fs-type verification and tidy up

Reviewed-by: Sage Weil <sage@redhat.com>
9 years agoMerge pull request #7654 from rldleblanc/wip-wpq-optimization
Sage Weil [Fri, 26 Feb 2016 23:36:52 +0000 (18:36 -0500)]
Merge pull request #7654 from rldleblanc/wip-wpq-optimization

osd: WeightedPriorityQueue: move to intrusive containers

9 years agoMerge pull request #7821 from rzarzynski/wip-rgw-14903
Yehuda Sadeh [Fri, 26 Feb 2016 21:37:44 +0000 (13:37 -0800)]
Merge pull request #7821 from rzarzynski/wip-rgw-14903

rgw: fix wrong handling of limit=0 during listing of Swift account.

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>
9 years agorgw: change naming of utility functions in rgw_keystone.cc. 7630/head
Radoslaw Zarzynski [Fri, 26 Feb 2016 21:17:09 +0000 (22:17 +0100)]
rgw: change naming of utility functions in rgw_keystone.cc.

Slightly improve code formatting and const-correctness as well.

Signed-off-by: Radoslaw Zarzynski <rzarzynski@mirantis.com>
9 years agoUse fixed version of civetweb to avoid warning. 6757/head
Marcus Watts [Tue, 23 Feb 2016 23:35:21 +0000 (18:35 -0500)]
Use fixed version of civetweb to avoid warning.

The warning about SSLv23_client_method is because when using dynamic
linking, this should always be non-zero.  That's inside of civetweb,
so this commit is to pull the submodule up to the fix.

Signed-off-by: Marcus Watts <mwatts@redhat.com>
9 years agoFix ssl link error.
Marcus Watts [Sat, 20 Feb 2016 08:29:27 +0000 (03:29 -0500)]
Fix ssl link error.

When linking against -lssl, should also include -lcrypto.

// fixup merge with previous

Signed-off-by: Marcus Watts <mwatts@redhat.com>
9 years agoUse ld.so to link in ssl crypto (cmake changes).
Marcus Watts [Thu, 11 Feb 2016 00:14:06 +0000 (19:14 -0500)]
Use ld.so to link in ssl crypto (cmake changes).

Using dlopen() to load in ssl crypto makes me cringe.
Also it requires either installing the devel package,
or appending library versioning to the library name
civetweb tries to load, which is ugly.  This is only
a license problem for people whose system distribution
does not include openssl.

Signed-off-by: Marcus Watts <mwatts@redhat.com>
9 years agorgw: fix wrong handling of limit=0 during listing of Swift account. 7821/head
Radoslaw Zarzynski [Fri, 26 Feb 2016 15:42:01 +0000 (16:42 +0100)]
rgw: fix wrong handling of limit=0 during listing of Swift account.

Fixes: #14903
Signed-off-by: Radoslaw Zarzynski <rzarzynski@mirantis.com>
9 years agoMerge pull request #7777 from theanalyst/rgw/insecure-keystone
Yehuda Sadeh [Fri, 26 Feb 2016 19:18:05 +0000 (11:18 -0800)]
Merge pull request #7777 from theanalyst/rgw/insecure-keystone

rgw: allow authentication keystone with self signed certs

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>
9 years agoMerge pull request #7809 from cbodley/wip-rgw-includes
Yehuda Sadeh [Fri, 26 Feb 2016 18:50:04 +0000 (10:50 -0800)]
Merge pull request #7809 from cbodley/wip-rgw-includes

rgw: use pimpl pattern for RGWPeriodHistory

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>
9 years agoMerge pull request #7822 from rzarzynski/wip-rgw-keystone-proper-parse-ordering
Yehuda Sadeh [Fri, 26 Feb 2016 18:44:01 +0000 (10:44 -0800)]
Merge pull request #7822 from rzarzynski/wip-rgw-keystone-proper-parse-ordering

rgw: try to parse Keystone token in order appropriate to configuration.

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>
9 years agoMerge pull request #7803 from oritwas/wip-14826
Yehuda Sadeh [Fri, 26 Feb 2016 18:38:12 +0000 (10:38 -0800)]
Merge pull request #7803 from oritwas/wip-14826

rgw: reset return code in when iterating over the bucket the objects

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>
9 years agoXIO: Changes to fix incorrect ip being assigned in case of multiple RDMA ports.remove... 7747/head
vsjulu [Mon, 22 Feb 2016 19:40:41 +0000 (11:40 -0800)]
XIO: Changes to fix incorrect ip being assigned in case of multiple RDMA ports.remove rdma local and return error instead to avoid issues with using rdma_local incase of multiple ports.

Signed-off-by: Subramanyam Varanasi <s.varanasi@ssi.samsung.com>
9 years agorgw: try to parse Keystone token in order appropriate to configuration. 7822/head
Radoslaw Zarzynski [Fri, 26 Feb 2016 14:25:20 +0000 (15:25 +0100)]
rgw: try to parse Keystone token in order appropriate to configuration.

Signed-off-by: Radoslaw Zarzynski <rzarzynski@mirantis.com>
9 years agorgw: add insecure option to the http client 7777/head
Abhishek Lekshmanan [Thu, 25 Feb 2016 10:07:51 +0000 (11:07 +0100)]
rgw: add insecure option to the http client

This allows the http client to turn off ssl certificate peer checking,
which is turned on by default. This is useful in cases like when
Keystone is SSL terminated with a self signed certificate.
The option `rgw_keystone_verify_ssl` (default true) can be toggled if
self signed certs are used, so that swift and s3 apis using keystone
authentication can work.

Fixes: #14583
Reported-by: Karol Mroz <mroz.karol@gmail.com>
Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>
9 years agorgw: use pimpl pattern for RGWPeriodHistory 7809/head
Casey Bodley [Thu, 25 Feb 2016 18:07:54 +0000 (13:07 -0500)]
rgw: use pimpl pattern for RGWPeriodHistory

this removes the dependency on RGWPeriod from rgw_rados.h, which breaks
a circular dependency between rgw_rados.h -> rgw_metadata.h ->
rgw_period_history.h -> rgw_rados.h

Reported-by: Willem Jan Withagen <wjw@digiware.nl>
Signed-off-by: Casey Bodley <cbodley@redhat.com>
9 years agodoc: final v9.2.1 release notes
Sage Weil [Fri, 26 Feb 2016 15:24:22 +0000 (10:24 -0500)]
doc: final v9.2.1 release notes

Signed-off-by: Sage Weil <sage@redhat.com>
9 years agoMerge pull request #7694 from Abhishekvrshny/wip-infernalis-release-notes
Sage Weil [Fri, 26 Feb 2016 15:24:20 +0000 (10:24 -0500)]
Merge pull request #7694 from Abhishekvrshny/wip-infernalis-release-notes

release-notes: draft v9.2.1 release notes

9 years agoMerge pull request #7813 from xiexingguo/xxg-wip-doc
Sage Weil [Fri, 26 Feb 2016 14:54:20 +0000 (09:54 -0500)]
Merge pull request #7813 from xiexingguo/xxg-wip-doc

doc: small fixes

9 years agodoc: batch small fixes, including typo, syntax etc. 7813/head
xiexingguo [Fri, 26 Feb 2016 15:25:02 +0000 (23:25 +0800)]
doc: batch small fixes, including typo, syntax etc.

Signed-off-by: xiexingguo <xie.xingguo@zte.com.cn>
9 years agodoc: fix typo
xiexingguo [Fri, 26 Feb 2016 14:39:57 +0000 (22:39 +0800)]
doc: fix typo

Signed-off-by: xiexingguo <xie.xingguo@zte.com.cn>
9 years agoMerge pull request #7805 from ErwanAliasr1/evelu-check-btrfs
Loic Dachary [Fri, 26 Feb 2016 04:34:42 +0000 (11:34 +0700)]
Merge pull request #7805 from ErwanAliasr1/evelu-check-btrfs

Fixing BTFS issue at 'make check'

Reviewed-by: Loic Dachary <ldachary@redhat.com>
9 years agoMerge pull request #7802 from branch-predictor/bp-increase-async-threads
Haomai Wang [Fri, 26 Feb 2016 02:45:45 +0000 (10:45 +0800)]
Merge pull request #7802 from branch-predictor/bp-increase-async-threads

config: increase default async op threads

Reviewed-by: Haomai Wang <haomai@xsky.com>
9 years agoMerge pull request #7808 from cbodley/wip-cmake-jerasure
Ali Maredia [Thu, 25 Feb 2016 20:09:09 +0000 (15:09 -0500)]
Merge pull request #7808 from cbodley/wip-cmake-jerasure

cmake: add ErasureCode.cc to jerasure plugins

9 years agocmake: add ErasureCode.cc to jerasure plugins 7808/head
Casey Bodley [Thu, 25 Feb 2016 19:57:56 +0000 (14:57 -0500)]
cmake: add ErasureCode.cc to jerasure plugins

Signed-off-by: Casey Bodley <cbodley@redhat.com>
9 years agorgw: link against system openssl (instead of dlopen at runtime)
Sage Weil [Thu, 29 Oct 2015 05:06:36 +0000 (01:06 -0400)]
rgw: link against system openssl (instead of dlopen at runtime)

Signed-off-by: Sage Weil <sage@redhat.com>
9 years agoMerge pull request #7748 from ceph/wip-cmake-2-22-16
Casey Bodley [Thu, 25 Feb 2016 19:06:16 +0000 (14:06 -0500)]
Merge pull request #7748 from ceph/wip-cmake-2-22-16

cmake: fix paths to various EC source files

9 years agocmake: installing _tp libraries into lib 7748/head
Ali Maredia [Thu, 25 Feb 2016 19:02:11 +0000 (14:02 -0500)]
cmake: installing _tp libraries into lib

Signed-off-by: Ali Maredia <amaredia@redhat.com>
9 years agocmake: Removed _tp libraries from linking
Ali Maredia [Thu, 25 Feb 2016 18:48:26 +0000 (13:48 -0500)]
cmake: Removed _tp libraries from linking

Signed-off-by: Ali Maredia <amaredia@redhat.com>
9 years agorgw: add support for caching of Keystone admin token.
Radoslaw Zarzynski [Fri, 12 Feb 2016 18:49:30 +0000 (19:49 +0100)]
rgw: add support for caching of Keystone admin token.

Signed-off-by: Radoslaw Zarzynski <rzarzynski@mirantis.com>
9 years agotest/ceph_objectstore_tool: Deleting btrfs subvolumes before exiting 7805/head
Erwan Velu [Thu, 25 Feb 2016 17:06:17 +0000 (18:06 +0100)]
test/ceph_objectstore_tool: Deleting btrfs subvolumes before exiting

When running the ceph_objectstore_tool test suite, we shall delete the btrfs
subvolumes that have been created before trying deleting the content unless that will generate a permission denied
issue.

Signed-off-by: Erwan Velu <erwan@redhat.com>
9 years agoceph-helpers: Delete remaining subvolumes on destroy_osd()
Erwan Velu [Thu, 25 Feb 2016 15:01:24 +0000 (16:01 +0100)]
ceph-helpers: Delete remaining subvolumes on destroy_osd()

Before deleting $dir/$id, we have to delete the associated subvolumes unless the
rm will fail.

Signed-off-by: Erwan Velu <erwan@redhat.com>
9 years agoMerge pull request #7793 from tchaikov/wip-fix-dencoder-MirrorImage
Jason Dillaman [Thu, 25 Feb 2016 15:33:19 +0000 (10:33 -0500)]
Merge pull request #7793 from tchaikov/wip-fix-dencoder-MirrorImage

cls_rbd: fix the test for ceph-dencoder

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
9 years agorgw: reset return code in when iterating over the bucket the objects 7803/head
Orit Wasserman [Thu, 25 Feb 2016 15:02:15 +0000 (16:02 +0100)]
rgw: reset return code in when iterating over the bucket the objects

Fixes: #14826
Signed-off-by: Orit Wasserman <owasserm@redhat.com>
9 years agocls_rbd: fix the test for ceph-dencoder 7793/head
Kefu Chai [Thu, 25 Feb 2016 07:11:44 +0000 (15:11 +0800)]
cls_rbd: fix the test for ceph-dencoder

initialize MirrorImage::state to a certain value, otherwise the
generated test instances would be undetermined, and hence fail
the dencoder tests.

Signed-off-by: Kefu Chai <kchai@redhat.com>
9 years agoMerge pull request #7792 from tchaikov/wip-13505
Loic Dachary [Thu, 25 Feb 2016 15:04:26 +0000 (22:04 +0700)]
Merge pull request #7792 from tchaikov/wip-13505

fix FTBFS introduced by d0af316

Reviewed-by: Loic Dachary <ldachary@redhat.com>
9 years agoceph-helpers.sh: Deleting forgoten btrfs subvolumes
Erwan Velu [Thu, 25 Feb 2016 13:08:10 +0000 (14:08 +0100)]
ceph-helpers.sh: Deleting forgoten btrfs subvolumes

While running a make check on a btrfs system, many subvolumes are let at the end
of the build. It's pretty common to have several hundreds of those.

btrfs is pretty sensible to the path when requesting a subvolume removal.
The current code was misleading the path and didn't deleted the remaining
volumes.

This patch list the current subvolumes, filter thoses created by the
test process and ajust the path because brtfs reports
  erwan/chroot/ceph/src/testdir/test-7202/dev/osd1/snap_439
while regarding the current working directory we want to delete :
  testdir/test-7202/dev/osd1/snap_439

Signed-off-by: Erwan Velu <erwan@redhat.com>
9 years agoconfig: increase default async op threads 7802/head
Piotr Dałek [Thu, 25 Feb 2016 14:12:52 +0000 (15:12 +0100)]
config: increase default async op threads

Default value of two async threads is enough for light workloads,
but is insufficient for heavy ones, reducing maximum throughput on
slower CPUs that have plenty of logical cores. Three async threads
give noticeable perf jump without any noticeable downsides.

Signed-off-by: Piotr Dałek <piotr.dalek@ts.fujitsu.com>
9 years agoMerge pull request #7731 from SUSE/wip-shebang
Loic Dachary [Thu, 25 Feb 2016 14:03:21 +0000 (21:03 +0700)]
Merge pull request #7731 from SUSE/wip-shebang

ceph_detect_init/__init__.py: remove shebang

Reviewed-by: Loic Dachary <ldachary@redhat.com>
9 years agoosd: always cleanup the scrub results 7792/head
Kefu Chai [Thu, 25 Feb 2016 13:06:04 +0000 (21:06 +0800)]
osd: always cleanup the scrub results

the destructor of Scrub::Store asserts that `this->results` should
always be empty when it is free'd. before this change, if we are
repairing, the results is not flushed or cleaned up, this crashes
osd daemon when a new PG internval comes. also we should not keep
the results around even we don't want to populate them to the omap
if we are repairing, it's just a waste.

Signed-off-by: Kefu Chai <kchai@redhat.com>
9 years agopybind: fix the FTBFS introduced by d0af316
Kefu Chai [Thu, 25 Feb 2016 06:28:32 +0000 (14:28 +0800)]
pybind: fix the FTBFS introduced by d0af316

Signed-off-by: Kefu Chai <kchai@redhat.com>
9 years agoMerge pull request #7734 from dachary/wip-14834-pidfile-test
Kefu Chai [Thu, 25 Feb 2016 08:19:08 +0000 (16:19 +0800)]
Merge pull request #7734 from dachary/wip-14834-pidfile-test

tests: test_pidfile.sh lingering processes

Reviewed-by: Kefu Chai <kchai@redhat.com>
9 years agolibrados: add missing tracpoint for scrub APIs
Kefu Chai [Thu, 25 Feb 2016 06:26:39 +0000 (14:26 +0800)]
librados: add missing tracpoint for scrub APIs

Signed-off-by: Kefu Chai <kchai@redhat.com>
9 years agotests: test_pidfile.sh lingering processes 7734/head
Loic Dachary [Mon, 22 Feb 2016 04:38:46 +0000 (11:38 +0700)]
tests: test_pidfile.sh lingering processes

For the mon, using ps aux to find the process will only show part of the
command line and the $RUNID may be missing. For the OSD, the $RUNID is
passed using an environment variable and will not show at all.

There is no need to test both ceph-osd and ceph-mon since they both end
up in the same code path, pidfile_write. Add a log line that only shows
when the --pid-file is empty and grep for it in the simplest possible
way.

The TEST_pidfile test failed to kill a process because there was
no pidfile with its pid. This is fixed by backuping the pidfile before
it is modified.

http://tracker.ceph.com/issues/14834 Fixes: #14834

Signed-off-by: Loic Dachary <loic@dachary.org>
9 years agoMerge pull request #6898 from tchaikov/wip-13505
Kefu Chai [Thu, 25 Feb 2016 04:48:14 +0000 (12:48 +0800)]
Merge pull request #6898 from tchaikov/wip-13505

add scrub persist/query API

Reviewed-by: Samuel Just <sjust@redhat.com>
9 years agorados: add "list-inconsistent-snapset" cmd 6898/head
Kefu Chai [Sat, 30 Jan 2016 13:22:19 +0000 (21:22 +0800)]
rados: add "list-inconsistent-snapset" cmd

to list inconsistent snapsets of a given PG, this command exposes
get_inconsistent_snapsets() rados API to user.

Fixes: #13505
Signed-off-by: Kefu Chai <kchai@redhat.com>
9 years agorados: add "list-inconsistent-obj" cmd
Kefu Chai [Sat, 30 Jan 2016 08:35:27 +0000 (16:35 +0800)]
rados: add "list-inconsistent-obj" cmd

to list inconsistent objects of a given PG, this command exposes
get_inconsistent_objects() rados API to user.

Signed-off-by: Kefu Chai <kchai@redhat.com>
9 years agolibrados: add get_inconsistent_snapsets() API
Kefu Chai [Sat, 30 Jan 2016 07:51:33 +0000 (15:51 +0800)]
librados: add get_inconsistent_snapsets() API

Fixes: #13505
Signed-off-by: Kefu Chai <kchai@redhat.com>
9 years agolibrados: add get_inconsistent_objects() API
Kefu Chai [Sat, 30 Jan 2016 07:05:31 +0000 (15:05 +0800)]
librados: add get_inconsistent_objects() API

Fixes: #13505
Signed-off-by: Kefu Chai <kchai@redhat.com>
9 years agoosd: add CEPH_OSD_OP_SCRUBLS pg op
Kefu Chai [Sat, 30 Jan 2016 06:51:17 +0000 (14:51 +0800)]
osd: add CEPH_OSD_OP_SCRUBLS pg op

it is a new pg op which returns the encoded objects stored when
scrubbing.

Fixes: #13505
Signed-off-by: Kefu Chai <kchai@redhat.com>
9 years agoosd: persist inconsistent snapsets using omap
Kefu Chai [Sat, 30 Jan 2016 07:22:49 +0000 (15:22 +0800)]
osd: persist inconsistent snapsets using omap

the inconsistent snapsets are identified in ReplicatedPG::_scrub()
after we compared the authorized objects with their replica/shards.
these inconsistent information are stored in the omap of objects
with prefix "SCRUB_SS_".

Signed-off-by: Kefu Chai <kchai@redhat.com>
9 years agolibrados: add `inconsistent_snapset_t` type
Kefu Chai [Sat, 30 Jan 2016 07:09:43 +0000 (15:09 +0800)]
librados: add `inconsistent_snapset_t` type

for presenting the inconsistent snapsets found in scrub

Signed-off-by: Kefu Chai <kchai@redhat.com>
9 years agoosd/: clear scrub store safetly
Samuel Just [Tue, 23 Feb 2016 19:50:46 +0000 (11:50 -0800)]
osd/: clear scrub store safetly

Signed-off-by: Samuel Just <sjust@redhat.com>
9 years agoosd: persist inconsistent objs using omap
Kefu Chai [Sat, 30 Jan 2016 06:38:48 +0000 (14:38 +0800)]
osd: persist inconsistent objs using omap

persist inconsistent objects found when comparing the ScrubMaps
collected from replica/shards. the discrepancies between the auth
copy and the replica are identified as inconsistencies. and hence
encoded into the omap of an object of the temp coll of the PG in
question.
scrub_types.{h,cpp} are introduced to hide the details of how we
persist the scrub types from the librados client.

Signed-off-by: Kefu Chai <kchai@redhat.com>
9 years agoosd: more constness to spg_t
Kefu Chai [Sat, 30 Jan 2016 08:48:08 +0000 (16:48 +0800)]
osd: more constness to spg_t

Signed-off-by: Kefu Chai <kchai@redhat.com>
9 years agolibrados: add `inconsistent_obj_t` types
Kefu Chai [Sat, 30 Jan 2016 05:42:24 +0000 (13:42 +0800)]
librados: add `inconsistent_obj_t` types

which present the inconsistent objects found in scrub

Fixes: #13505
Signed-off-by: Kefu Chai <kchai@redhat.com>
9 years agorados: add "list-inconsistent-pg" command
Kefu Chai [Sat, 30 Jan 2016 05:29:05 +0000 (13:29 +0800)]
rados: add "list-inconsistent-pg" command

to list inconsistent PGs of a given pool. this command exposes
the underlying get_inconsistent_pgs() API to user.

Signed-off-by: Kefu Chai <kchai@redhat.com>
9 years agopybind: add Rados.get_inconsistent_pgs method
Kefu Chai [Wed, 24 Feb 2016 13:42:34 +0000 (21:42 +0800)]
pybind: add Rados.get_inconsistent_pgs method

Signed-off-by: Kefu Chai <kchai@redhat.com>
9 years agolibrados: add get_inconsistent_pgs() to librados
Kefu Chai [Fri, 11 Dec 2015 10:13:20 +0000 (18:13 +0800)]
librados: add get_inconsistent_pgs() to librados

to list the inconsistent PGs of given pool, it's a wrapper
around the "ceph pg ls" command.

Fixes: #13505
Signed-off-by: Kefu Chai <kchai@redhat.com>
9 years agotools/rados: support more --format options
Kefu Chai [Wed, 2 Dec 2015 08:02:14 +0000 (16:02 +0800)]
tools/rados: support more --format options

Signed-off-by: Kefu Chai <kchai@redhat.com>
9 years agoMerge pull request #7597 from rzarzynski/wip-rgw-keystone-proper-fail
Yehuda Sadeh [Wed, 24 Feb 2016 22:40:02 +0000 (14:40 -0800)]
Merge pull request #7597 from rzarzynski/wip-rgw-keystone-proper-fail

rgw: improve error handling in S3/Keystone integration

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>
9 years agoMerge pull request #7051 from wonzhq/scrub-cmp-map
Sage Weil [Wed, 24 Feb 2016 19:35:38 +0000 (11:35 -0800)]
Merge pull request #7051 from wonzhq/scrub-cmp-map

osd: avoid FORCE updating digest been overwritten by MAYBE when comparing scrub map

Reviewed-by: Kefu Chai <kchai@redhat.com>
9 years agoMerge pull request #7617 from liewegas/wip-14745
Sage Weil [Wed, 24 Feb 2016 18:45:59 +0000 (10:45 -0800)]
Merge pull request #7617 from liewegas/wip-14745

osd: fix forced prmootion for CALL ops

Reviewed-by: Sage Weil <sage@redhat.com>
9 years agoMerge pull request #7286 from ktdreyer/wip-10587-init-ceph-disk
Sage Weil [Wed, 24 Feb 2016 18:42:37 +0000 (10:42 -0800)]
Merge pull request #7286 from ktdreyer/wip-10587-init-ceph-disk

init-ceph.in: skip ceph-disk if it is not present

Reviewed-by: Kefu Chai <kchai@redhat.com>
9 years agoMerge pull request #7599 from ifed01/wip-14707
Sage Weil [Wed, 24 Feb 2016 18:42:04 +0000 (10:42 -0800)]
Merge pull request #7599 from ifed01/wip-14707

log: fix stack overflow when flushing large log lines

Reviewed-by: Sage Weil <sage@redhat.com>
9 years agoMerge pull request #7643 from jazeltq/add_count_github
Sage Weil [Wed, 24 Feb 2016 18:41:17 +0000 (10:41 -0800)]
Merge pull request #7643 from jazeltq/add_count_github

osd: filejournal: report journal entry count

Reviewed-by: Sage Weil <sage@redhat.com>
9 years agoMerge pull request #7658 from majianpeng/bluestore
Sage Weil [Wed, 24 Feb 2016 18:40:26 +0000 (10:40 -0800)]
Merge pull request #7658 from majianpeng/bluestore

osd: bluestore: misc fixes

Reviewed-by: Sage Weil <sage@redhat.com>
9 years agoMerge pull request #7677 from xiexingguo/xxg-wip-14786
Sage Weil [Wed, 24 Feb 2016 18:39:27 +0000 (10:39 -0800)]
Merge pull request #7677 from xiexingguo/xxg-wip-14786

osd: fix fusestore hanging during stop/quit

Reviewed-by: Sage Weil <sage@redhat.com>
9 years agoMerge pull request #7680 from ceph/wip-da-SCA-20160203
Sage Weil [Wed, 24 Feb 2016 18:38:06 +0000 (10:38 -0800)]
Merge pull request #7680 from ceph/wip-da-SCA-20160203

common: various fixes from SCA runs

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Jason Dillaman <jdillama@redhat.com>
Reviewed-by: Greg Farnum <gfarnum@redhat.com>
9 years agoMerge pull request #7681 from jjhuo/kstore-updates
Sage Weil [Wed, 24 Feb 2016 18:37:13 +0000 (10:37 -0800)]
Merge pull request #7681 from jjhuo/kstore-updates

osd: kstore: sync up kstore with recent bluestore updates

Reviewed-by: Sage Weil <sage@redhat.com>
9 years agoMerge pull request #7759 from trociny/wip-rbd-mirror-image-replayer-improvements
Jason Dillaman [Wed, 24 Feb 2016 16:40:43 +0000 (11:40 -0500)]
Merge pull request #7759 from trociny/wip-rbd-mirror-image-replayer-improvements

rbd-mirror: ImageReplayer improvements

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
9 years agoMerge pull request #7736 from trociny/fix-librbd-asok-empty-name
Jason Dillaman [Wed, 24 Feb 2016 16:07:22 +0000 (11:07 -0500)]
Merge pull request #7736 from trociny/fix-librbd-asok-empty-name

librbd: retrieve image name when opening by id

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
9 years agoMerge pull request #7620 from jdurgin/wip-14419
Jason Dillaman [Wed, 24 Feb 2016 15:35:28 +0000 (10:35 -0500)]
Merge pull request #7620 from jdurgin/wip-14419

cls_rbd: mirroring directory

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
9 years agocls_rbd: add methods for keeping track of mirrored images
Josh Durgin [Fri, 12 Feb 2016 04:38:07 +0000 (20:38 -0800)]
cls_rbd: add methods for keeping track of mirrored images

These will track whether local images should be mirrored, and map them
to a unique global id. There's a state field for safely disabling
mirroring while operating on multiple objects.

Fixes: #14419
Signed-off-by: Josh Durgin <jdurgin@redhat.com>
9 years agolibrbd: rename rbd_pool_settings object to rbd_mirroring
Josh Durgin [Thu, 4 Feb 2016 20:15:00 +0000 (12:15 -0800)]
librbd: rename rbd_pool_settings object to rbd_mirroring

We'll use this object only for mirroring-related purposes, not generic
settings on a pool.

Refs: #14419
Signed-off-by: Josh Durgin <jdurgin@redhat.com>
9 years agoMerge pull request #7762 from ErwanAliasr1/evelu-check
Loic Dachary [Wed, 24 Feb 2016 14:42:02 +0000 (21:42 +0700)]
Merge pull request #7762 from ErwanAliasr1/evelu-check

Improving 'make check' for ceph-disk

Reviewed-by: Loic Dachary <ldachary@redhat.com>
9 years agolibrbd: init asok_hook on open so name is always known 7736/head
Mykola Golub [Tue, 23 Feb 2016 20:20:51 +0000 (22:20 +0200)]
librbd: init asok_hook on open so name is always known

Signed-off-by: Mykola Golub <mgolub@mirantis.com>
9 years agolibrbd: get image name on open if it is opened by id
Mykola Golub [Tue, 23 Feb 2016 09:28:00 +0000 (11:28 +0200)]
librbd: get image name on open if it is opened by id

Signed-off-by: Mykola Golub <mgolub@mirantis.com>
9 years agoMerge pull request #7743 from JiYou/open-test-for-bug-2339
Kefu Chai [Wed, 24 Feb 2016 13:57:05 +0000 (21:57 +0800)]
Merge pull request #7743 from JiYou/open-test-for-bug-2339

test: enable test for bug #2339 which has been resolved.

Reviewed-by: Kefu Chai <kchai@redhat.com>
9 years agoMerge pull request #7773 from roidayan/xio_remove_redundant_magic_methods
Haomai Wang [Wed, 24 Feb 2016 13:49:21 +0000 (21:49 +0800)]
Merge pull request #7773 from roidayan/xio_remove_redundant_magic_methods

xio: remove redundant magic methods

Reviewed-by: Haomai Wang<haomai@xsky.com>
9 years agoMerge pull request #7771 from xiexingguo/xxg-wip-fixawatch
Jason Dillaman [Wed, 24 Feb 2016 13:35:06 +0000 (08:35 -0500)]
Merge pull request #7771 from xiexingguo/xxg-wip-fixawatch

librados: do not clear handle for aio_watch()

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
9 years agoxio: remove redundant magic methods 7773/head
Roi Dayan [Mon, 15 Feb 2016 07:42:11 +0000 (09:42 +0200)]
xio: remove redundant magic methods

The same methods are inherited from Messenger.

Signed-off-by: Roi Dayan <roid@mellanox.com>
9 years agolibrados: remove unused local variables 7771/head
xie xingguo [Wed, 24 Feb 2016 08:59:33 +0000 (16:59 +0800)]
librados: remove unused local variables

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
9 years agolibrados: do not clear handle for aio_watch()
xie xingguo [Wed, 24 Feb 2016 08:56:23 +0000 (16:56 +0800)]
librados: do not clear handle for aio_watch()

which is needed for aio_unwatch() to work.

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
9 years agotests/ceph-disk: Let teardown clearing data 7762/head
Erwan Velu [Tue, 23 Feb 2016 16:54:21 +0000 (17:54 +0100)]
tests/ceph-disk: Let teardown clearing data

When running the ceph-disk check suite, 'test_mark_init' & 'test_activate_dir' had a
permission denied of the freshly created stuff like in :

tests/ceph-disk.sh:237: test_mark_init:  /bin/rm -fr /ceph/src/ceph-disk/test-ceph-disk/dir
/bin/rm: cannot remove « /ceph/src/ceph-disk/test-ceph-disk/dir/snap_1 »: Permission denied
/bin/rm: cannot remove « /ceph/src/ceph-disk/test-ceph-disk/dir/current »: Permission denied
/bin/rm: cannot remove « /ceph/src/ceph-disk/test-ceph-disk/dir/snap_2 »: Permission denied

Those two tests are starting a mon with run_mon.
Trying to delete the files while the daemon are still alive is impossible.
This patch removes the explicit 'rm' and let the teardown doing the cleaning
stuff by stopping daemons & cleaning content.

By using this patch, make check is now sucessful on ceph-disk.

Signed-off-by: Erwan Velu <erwan@redhat.com>
9 years agotests/ceph-disk: Using dummy device mappers
Erwan Velu [Tue, 23 Feb 2016 15:14:48 +0000 (16:14 +0100)]
tests/ceph-disk: Using dummy device mappers

When doing a make test, if your local setup was running a "dm-0" or "dm-1",
the make check was failing.

The detection of the local "dm-x" were leading to a wrong comparaison with
the expected simulated values.

The fix is using a dummy name for that "dm" to prevent any collision with the local setup.

Prior that patch, a typical output of that error looked like :

>               assert expect == main.list_devices()
E               assert [{'partitions...: '/dev/Xda'}] == [{'partitions'...: '/dev/Xda'}]
E                 At index 0 diff: {'partitions': [{'dmcrypt': {'holders': ['dm-0'], 'type': 'plain'}, 'ptype': '4fbd7e29-9d25-41b8-afd0-5ec00ceff05d', 'path': '/dev/Xda1', 'is_partition': True, 'mount': None, 'uuid': '56244cf5-83ef-4984-888a-2d8b8e0e04b2', 'type': 'data', 'state': 'unprepared', 'fs_type': None}], 'path': '/dev/Xda'} != {'partitions': [{'ptype': '4fbd7e29-9d25-41b8-afd0-5ec00ceff05d', 'whoami': None, 'path': '/dev/Xda1', 'is_partition': True, 'mount': '/var/cache/ccache', 'uuid': '56244cf5-83ef-4984-888a-2d8b8e0e04b2', 'ceph_fsid': None, 'fs_type': 'btrfs', 'dmcrypt': {'holders': ['dm-0'], 'type': 'plain'}, 'type': 'data', 'state': 'active'}], 'path': '/dev/Xda'}
E                 Full diff:
E                 - [{'partitions': [{'dmcrypt': {'holders': ['dm-0'], 'type': 'plain'},
E                 + [{'partitions': [{'ceph_fsid': None,
E                 +                   'dmcrypt': {'holders': ['dm-0'], 'type': 'plain'},
E                 -                   'fs_type': None,
E                 ?                              ^^^^
E                 +                   'fs_type': 'btrfs',
E                 ?                              ^^^^^^^
E                 'is_partition': True,
E                 -                   'mount': None,
E                 +                   'mount': '/var/cache/ccache',
E                 'path': '/dev/Xda1',
E                 'ptype': '4fbd7e29-9d25-41b8-afd0-5ec00ceff05d',
E                 -                   'state': 'unprepared',
E                 ?                             ^^^^ -----
E                 +                   'state': 'active',
E                 ?                             ^^^^^
E                 'type': 'data',
E                 -                   'uuid': '56244cf5-83ef-4984-888a-2d8b8e0e04b2'}],
E                 ?                                                                 --
E                 +                   'uuid': '56244cf5-83ef-4984-888a-2d8b8e0e04b2',
E                 +                   'whoami': None}],
E                 'path': '/dev/Xda'}]

tests/test_main.py:342: AssertionError

Signed-off-by: Erwan Velu <erwan@redhat.com>
9 years agotests/ceph-disk: Creating missing working dir
Erwan Velu [Tue, 23 Feb 2016 10:36:51 +0000 (11:36 +0100)]
tests/ceph-disk: Creating missing working dir

When running run-tox.sh in a very simple env,
the test will fail if '/var/lib/ceph/tmp' doesn't exist.

This patch adds a check to create this directory if required as mkdtemp doesn't do it for you.

Prior this patch, the following behavior was seen :

tests/test_main.py:342:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
ceph_disk/main.py:3753: in list_devices
    fstype=fs_type, options='')
ceph_disk/main.py:1217: in mount
    dir=STATEDIR + '/tmp',
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

suffix = '', prefix = 'mnt.', dir = '/var/lib/ceph/tmp'

    def mkdtemp(suffix="", prefix=template, dir=None):
        """User-callable function to create and return a unique temporary
        directory.  The return value is the pathname of the directory.

        Arguments are as for mkstemp, except that the 'text' argument is
        not accepted.

        The directory is readable, writable, and searchable only by the
        creating user.

        Caller is responsible for deleting the directory when done with it.
        """

        if dir is None:
            dir = gettempdir()

        names = _get_candidate_names()

        for seq in xrange(TMP_MAX):
            name = names.next()
            file = _os.path.join(dir, prefix + name + suffix)
            try:
>               _os.mkdir(file, 0700)
E               OSError: [Errno 2] No such file or directory: '/var/lib/ceph/tmp/mnt.KoAV85'

/usr/lib64/python2.7/tempfile.py:333: OSError

Signed-off-by: Erwan Velu <erwan@redhat.com>
9 years agoMerge pull request #7649 from yuyuyu101/wip-async-watch
Jason Dillaman [Wed, 24 Feb 2016 04:41:03 +0000 (23:41 -0500)]
Merge pull request #7649 from yuyuyu101/wip-async-watch

librados: implement async watch/unwatch

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
9 years agotest: enable test for bug #2339 which has been resolved. 7743/head
You Ji [Mon, 22 Feb 2016 15:52:38 +0000 (08:52 -0700)]
test: enable test for bug #2339 which has been resolved.

Signed-off-by: You Ji <jiyou09@gmail.com>
9 years agoMerge pull request #7761 from dillaman/wip-14847
Josh Durgin [Wed, 24 Feb 2016 02:07:12 +0000 (18:07 -0800)]
Merge pull request #7761 from dillaman/wip-14847

librbd: fix state machine race conditions during shut down

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
9 years agoRadosClient: call watch_flush before finisher stop 7649/head
Haomai Wang [Tue, 23 Feb 2016 04:17:50 +0000 (12:17 +0800)]
RadosClient: call watch_flush before finisher stop

Signed-off-by: Haomai Wang <haomai@xsky.com>
9 years agotest/common/test_weighted_priority_queue Fix the unit tests since the 7654/head
Robert LeBlanc [Mon, 15 Feb 2016 17:52:05 +0000 (17:52 +0000)]
test/common/test_weighted_priority_queue Fix the unit tests since the
changes to WeightedPriorityQueue there is no strict round robin
dequeueing of classes. Removed that test from the unittest.

Signed-off-by: Robert LeBlanc <robert.leblanc@endurance.com>
9 years agocommon/WeightedPriorityQueue Rewrote the queue to use intrusive contianers. Microbenc...
Robert LeBlanc [Mon, 15 Feb 2016 17:48:49 +0000 (17:48 +0000)]
common/WeightedPriorityQueue Rewrote the queue to use intrusive contianers. Microbenchmarks
show 60-70% of execution time compared to before.

Signed-off-by: Robert LeBlanc <robert.leblanc@endurance.com>
9 years agocls_rbd: async version of dir_get_name
Mykola Golub [Tue, 23 Feb 2016 09:26:37 +0000 (11:26 +0200)]
cls_rbd: async version of dir_get_name

Signed-off-by: Mykola Golub <mgolub@mirantis.com>