test/Makefile-client: ship LibradosTestStub.h in tarball

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #5868 from jmunhoz/wip-x-amz-request-id-doc

doc: rgw: update x-amz-request-id status

Reviewed-by: Abhishek Lekshmanan <abhishek.lekshmanan@ril.com>

Merge pull request #5863 from ceph/wip-13015

rgw: preserve all attrs if intra-zone copy

doc: rgw: update x-amz-request-id status

Signed-off-by: Javier M. Mellid <jmunhoz@igalia.com>

Merge pull request #5343 from hjwsm1989/wip-12442

  mon/MonClient: fix error in 'ceph ping mon.id'

Reviewed-by: Joao Eduardo Luis <joao@suse.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #5816 from dillaman/wip-librbd-gmock

Initial support for gmock-based unit tests in librbd

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #5557 from guangyy/wip-12666

rgw: expose the number of unhealthy workers through admin socket

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #5786 from ceph/wip-12911

rbd export-diff should export data chunks in-order

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #5858 from jecluis/wip-test-mon-scrub

test: mon/mon-scrub.sh: have test running on make check

Reviewed-by: Loic Dachary <ldachary@redhat.com>

rgw: preserve all attrs if intra-zone copy

Fixes: #13015
Intra zone copy requires that all objects' attributes are preserved.
This was broken at commit: e41d97c8e38bb60d7e09e9801c0179efe7af1734

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

Merge pull request #5442 from ceph/wip-12551

ReplicatedPG,Objecter: copy_get should include truncate_seq and size

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #5490 from dwj192/master

rgw:add --reset-regions for regionmap update

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>

test/Makefile.am: run mon/mon-scrub.sh as part of checks

Signed-off-by: Joao Eduardo Luis <joao@suse.de>

test: mon/mon-scrub.sh: port clashed with other tests

This will allow the test to be run during make check.

Signed-off-by: Joao Eduardo Luis <joao@suse.de>

Merge pull request #5828 from lebauce/fix-swift-prefix

rgw: add delimiter to prefix only when path is specified

Reviewed-by: Radoslaw Zarzynski <rzarzynski@mirantis.com>

ReplicatedPG,Objecter: copy_get should include truncate_seq and size

Otherwise, we break CephFS over cache tiers.

Fixes: #12551
Signed-off-by: Samuel Just <sjust@redhat.com>

Merge pull request #5837 from tchaikov/wip-12968

mon: disable gmt_hitset if not supported

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Loic Dachary <ldachary@redhat.com>

Merge pull request #5842 from dzafman/wip-cot-failure

test: Fix failure test to find message anywhere in stderr

Reviewed-by: Loic Dachary <ldachary@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #5845 from wonzhq/async-rx

AsyncMessenger: add instance name in debug log when processing msg

Reviewed-by: Haomai Wang <haomaiwang@gmail.com>

Merge pull request #5779 from dzafman/wip-misc-fix-guangyy

osd: check the length of the map before accessing the first element

Reviewed-by: Loic Dachary <ldachary@redhat.com>

Merge pull request #5843 from dillaman/wip-12885

librbd: diff_iterate needs to handle holes in parent images

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #5592 from xinxinsh/wip-fix-rbd-metadata

fix bugs of rbd metadata operation

Reviewed-by: Haomai Wang <haomaiwang@gmail.com>
Reviewed-by: Jason Dillaman <dillaman@redhat.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>

test: Fix failure test to find message anywhere in stderr

Consolidate test_failure() and test_failure_tty()

Signed-off-by: David Zafman <dzafman@redhat.com>

rados: Fix usage for "notify" command

Signed-off-by: David Zafman <dzafman@redhat.com>

Merge pull request #5834 from ceph/wip-leveldb-hang

mon: MonitorDBStore: make get_next_key() work properly

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #4955 from kylinstorage/librados-2read-segmentation-fault-v1

bug fix: librados segmentation fault, when two read ops share one Aio…

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #4925 from XinzeChi/wip-hit-set-size

osd: fix hit_set_map size for tier pool

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #5723 from H3C/wip-msg-bugfix1

msg: we should set the socket options before connect or listen in order to have it take effect.

Reviewed-by: Haomai Wang <haomaiwang@gmail.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #5829 from dachary/wip-12731-jerasure-warnings

erasure-code: fix gf-complete warnings

Reviewed-by: Kefu Chai <kchai@redhat.com>

AsyncMessenger: add instance name in debug log when processing msg

To better debug.

Signed-off-by: Zhiqiang Wang <zhiqiang.wang@intel.com>

rgw:add --reset-regions for regionmap update

Fixes: #12964
Signed-off-by: Weijun Duan <duanweijun@h3c.com>

Merge pull request #5781 from dzafman/wip-eccleanup

osd, test: Minor clean-up from fast-read and error handling ec changes

Reviewed-by: Kefu Chai <kchai@redhat.com>

librbd: diff_iterate needs to handle holes in parent images

If a clone's parent image snapshot includes a discarded extent,
this was previously causing an assert failure.  Instead, ignore
any discard holes in the parent image.

Fixes: #12885
Backport: hammer
Signed-off-by: Jason Dillaman <dillaman@redhat.com>

tests: new test case for librbd diff_iterate over discard extents

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #5838 from ceph/wip-12861

qa/workunits/rados/test_alloc_hint.sh: sudo to ls files

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #5396 from XinzeChi/wip-scrub-promote

bug fix: osd: requeue_scrub when kick_object_context_blocked

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: David Zafman <dzafman@redhat.com>

Merge pull request #5772 from yuyuyu101/wip-12912

AsyncConnection: Close connection when unregistered connection met WAIT

Merge pull request #5733 from cxwshawn/rgw-fix

rgw: init_rados failed leads to repeated delete

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>

Merge pull request #5809 from rzarzynski/wip-rgw_putobj_debug_improvement

rgw: improve debugs in RGWPutObj and RGWPutObj_ObjStore_SWIFT.

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>

Merge pull request #5567 from guce/rgw_check_key_parameter_conflict

rgw: add key parameter conflict check for radosgw-admin command line.

Merge pull request #5819 from rzarzynski/wip-12950

rgw: don't append empty ETag HTTP header.

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>

Merge remote-tracking branch 'gh/infernalis'

qa/workunits/rados/test_alloc_hint.sh: sudo to ls files

The osd data dir is owned by ceph and not readable by other
non-root users.

Fixes: #12861
Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #5257 from t-miyamae/wip-mshec-fixlogic4

erasure code: shec performance optimization by decoding cache

Reviewed-by: Loic Dachary <ldachary@redhat.com>

rgw: init_rados failed leads to repeated delete

Fixes: #12978
Signed-off-by: Xiaowei Chen <chen.xiaowei@h3c.com>

Merge pull request #5463 from kylinstorage/wip-evict_ops

Allow evict operations to be throttled

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #5745 from ceph/wip-12904

osd: sparse reads returning invalid extent map

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #5580 from xinxinsh/wip-merge-two-setattr

merge two continuous OP_SETATTR ops into one OP_SETATTRS

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

mon: disable gmt_hitset if not supported

the gmt_hitset is enabled by default in the ctor of pg_pool_t, this
is intentional. because we want to remove this setting and make
gmt_hitset=true as a default in future. but this forces us to
disable it explicitly when preparing a new pool if any OSD does
not support gmt hitset.

Fixes: #12968
Signed-off-by: Kefu Chai <kchai@redhat.com>

test: mon: mon-scrub.sh: test 'mon scrub'

In its current state is used to catch regressions on 'ceph mon scrub'
hanging due to an infinite loop.

Signed-off-by: Joao Eduardo Luis <joao@suse.de>

Merge pull request #5785 from liewegas/wip-12861

qa/workunits/rados/test_alloc_hint.sh: sudo to list files

mon: MonitorDBStore: make get_next_key() work properly

We introduced a significant bug with 2cc7aee, when we fixed issue #11786.
Although that patch would fix the problem described in #11786, we
managed to not increment the iterator upon returning the current key.
This would have the iterator iterating over the same key, forever and
ever.

Signed-off-by: Joao Eduardo Luis <joao@suse.de>

Merge pull request #5784 from dachary/wip-docker-helper

fix docker-test.sh for CentOS 7

Reviewed-by: Kefu Chai <kchai@redhat.com>

tests: drop docker-tests.sh root and /dev support

Now that ceph-disk.sh activation tests switched to the ceph-disk suite
as found in https://github.com/ceph/ceph-qa-suite/tree/master/suites/ceph-disk,
it does not need for root and /dev support from docker-tests.sh. There
currently is no other use case and since it's generally not a good idea
to run anything as root in a container anyway.

Signed-off-by: Loic Dachary <ldachary@redhat.com>

tests: CentOS 7 needs systemd-container

It is now needed by the Ceph dependencies and won't work unless the fake
version of it is replaced by the actual one.

Signed-off-by: Loic Dachary <ldachary@redhat.com>

rgw: add delimiter to prefix only when path is specified

http://tracker.ceph.com/issues/12960
Fixes: #12960
Signed-off-by: Sylvain Baubeau <sbaubeau@redhat.com>

Merge pull request #5728 from trociny/wip-12706-infernalis

ceph-disk: use /sys/dev/block/maj:min/partition to see if partition

Reviewed-by: Loic Dachary <ldachary@redhat.com>

refine tests for metadata ops

Signed-off-by: xinxin shu <xinxin.shu@intel.com>

improve error handle of rbd metadata operation & format output

Signed-off-by: xinxin shu <xinxin.shu@intel.com>

fix metadata loading error if we open an image

Signed-off-by: xinxin shu <xinxin.shu@intel.com>

msg: we should set the socket options before connect or listen in order to have it take effect.

Signed-off-by: Ruifeng Yang <149233652@qq.com>

erasure-code: fix gf-complete warnings

Update to the latest gf-complete v2 branch which has the compilation
warning fixed. In addition the default compilation flag upstream has
been set to display such warnings.

git log --oneline --graph d384952c68a64d93ac7af6341d5519ea5d2958b9..9caeefbf2860e56a75502f4d3342deed5b5ba265
* 9caeefb add -Wsign-compare and address the warnings
* 5979f98 gf_w64.c: fix integer overflow
* 93a9845 gf_w64.c: fix integer overflow
* 513c87b gf_w64.c: fix integer overflow
* 4d6fa89 gf_w64.c: fix integer overflow

http://tracker.ceph.com/issues/12731 Fixes: #12731

Signed-off-by: Loic Dachary <ldachary@redhat.com>

Merge remote-tracking branch 'gh/infernalis'

Merge pull request #4895 from XinzeChi/wip-outdata-set

bug fix: osd: avoid multi set osd_op.outdata in tier pool

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #5707 from athanatos/wip-12809

PG::handle_advance_map: on_pool_change after handling the map change

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #5751 from yuyuyu101/wip-12908

AsyncConnection: Don't use unsafe feature as message encode feature

Reviewed-by: Sage Weil <sage@redhat.com>

Merge remote-tracking branch 'gh/infernalis'

qa/workunits/post-file.sh: sudo

Only root can read the key in /usr/share/...

Signed-off-by: Sage Weil <sage@redhat.com>

osd: dump full map bl at 20 when crc doesn't match

This will help us debug cases where the encoding doesn't match due
to a bug.

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #5827 from tchaikov/wip-doc-daemon

doc: fix the typo in command example

Reviewed-by: Loic Dachary <ldachary@redhat.com>

doc: fix the typo in command example

always indent using tab, the rendered html looks good, but it helps with
editor to highlight the codeblock properly.

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #5817 from liewegas/wip-ceph-daemon

doc: 'ceph --admin-daemon ...' -> 'ceph daemon ...'

Reviewed-by: Loic Dachary <ldachary@redhat.com>

Merge pull request #5193 from SUSE/wip-babeltrace-suse

ceph.spec.in: fix lttng/babeltrace conditionals

Reviewed-by: Ken Dreyer <kdreyer@redhat.com>

Merge pull request #5796 from SUSE/wip-group-cond

ceph.spec.in: put distro conditionals around Group:

Reviewed-by: Ken Dreyer <kdreyer@redhat.com>

doc: 'ceph --admin-daemon ...' -> 'ceph daemon ...'

Signed-off-by: Sage Weil <sage@redhat.com>

tests: base gmock class support for librbd

Created mock classes to represent a few central librbd classes
and a basic gmock test fixture for future gmock-based unit tests.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

librbd: support templating of ImageCtx for async state machines

This will facilitate the creation of unit test mocks to verify
non-librados actions. The templates (by default) will be flagged
as extern to avoid the translation unit bloat.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #5624 from vumrao/wip-vumrao-12714

rgw : setting max number of buckets for users via ceph.conf option

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>

ceph.spec.in: put distro conditional around Group:

Neither RHEL and Fedora require a Group: line, so retain
only for SLE/openSUSE (and set the right value for these distros)

Signed-off-by: Nathan Cutler <ncutler@suse.com>

Merge pull request #5798 from SUSE/wip-fix-license

ceph.spec.in: fix License line

Reviewed-by: Ken Dreyer <kdreyer@redhat.com>

librados_test_stub: add mock class for IoCtx operations

Unit tests can now use gmock to simulate responses from the OSDs
via a mocked librados library.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #5801 from dreamhost/wip-12939-header

Fix casing of Content-Type header

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>

Merge pull request #5811 from vumrao/wip-vumrao-12944

doc: Add pgcalc tool link in placement-groups document

Reviewed-by: Loic Dachary <ldachary@redhat.com>

ceph.spec.in: fix lttng/babeltrace conditionals

lttng and babeltrace are build dependencies for rbd-replay-prep. Make
sure the right package names are used. Enable for SLE12, as well as
for openSUSE 13.1 and higher.

Move the BuildRequires out of the ceph-test subpackage and into the
distro-conditional dependencies section.

Make ordering of BuildRequires a little more alphabetical.

http://tracker.ceph.com/issues/12360 Fixes: #12360

Signed-off-by: Nathan Cutler <ncutler@suse.com>
sqme

Merge pull request #5791 from SUSE/wip-drop-lsb-release

ceph.spec.in: drop lsb-release dependency from ceph-common

Reviewed-by: Ken Dreyer <kdreyer@redhat.com>

rgw: don't append empty ETag HTTP header.

Fixes: #12950
Signed-off-by: Radoslaw Zarzynski <rzarzynski@mirantis.com>

Merge pull request #5808 from ceph/wip-simplify-logrotate

logrotate.conf: Simplify log files reopening after log rotation

Reviewed-by: Milan Broz <mbroz@redhat.com>
Reviewed-by: Sage Weil <sage@redhat.com>

doc: Add pgcalc tool link in placement-groups document

Fixes #12944

Signed-off-by: Vikhyat Umrao <vumrao@redhat.com>

Merge pull request #5771 from ceph/wip-fix-i386-build

memstore: fix the build on i386

Reviewed-by: Loic Dachary <ldachary@redhat.com>

rgw: improve debugs in RGWPutObj and RGWPutObj_ObjStore_SWIFT.

Signed-off-by: Radoslaw Zarzynski <rzarzynski@mirantis.com>

Merge pull request #5794 from dachary/wip-12933-erasure-code-dir

tests: ceph-erasure-code-corpus must test SIMD variants

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #5804 from tchaikov/wip-fix-build-boost-1.59

mon: fix the build with boost 1.59

Reviewed-by: Loic Dachary <ldachary@redhat.com>

erasure code: shec performance optimization with decoding cache

continued from https://github.com/ceph/ceph/pull/4132

Signed-off-by: Takanori Nakao <nakao.takanori@jp.fujitsu.com>

logrotate.conf: Simplify log files reopening after log rotation

SIGHUP (1) signal makes ceph services reopen its log files so we do not
need to call any init scripts magic, here.

Additionally, the old approach caused problems with rhel-based systems
since it found the service command and tried to reload ceph.service
instead of ceph.target.

This also caused problems with SELinux as processes with context
logrotate_t are not allowed to issue systemctl reload on a process.

This patch is also in sync with other logrotate postrotate scripts which
do exactly the same thing (send SIGHUP to the daemon).

Signed-off-by: Boris Ranto <branto@redhat.com>

Merge pull request #5802 from liewegas/wip-coverity

a few coverity fixes

Reviewed-by: Loic Dachary <ldachary@redhat.com>

mon: fix the build with boost 1.59

cmd_vartype is a
boost::variant<std::string, bool, int64_t, double,
               std::vector<std::string>>, so we can not get a
int8_t from it.

Fixes: #12922
Signed-off-by: Kefu Chai <kchai@redhat.com>

librados_test_stub: add reference counting to pools

This will ensure that in-flight operations can't access freed
pool memory.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

gtest: enable use of TR1 tuples

Since the TR1 tuple is already in use within the Ceph project,
this flag needs to be enabled to avoid gmock link errors within
gtest.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

rbd: export diff needs should write chunks in-order

Use new OrderedThrottle class to ensure multiple AIO reads are
written to the export diff in-order.

Fixes: #12911
Signed-off-by: Jason Dillaman <dillaman@redhat.com>

librbd: migrate diff iterate to new OrderedThrottle implementation

Signed-off-by: Jason Dillaman <dillaman@redhat.com>