logrotate: ignore postrotate error for radosgw

Fix the annoying logrotate error for radosgw.

Fixes: #13033
Signed-off-by: Dan van der Ster <daniel.vanderster@cern.ch>

logrotate: ignore exit status 1 from killall

If any of ceph-osd, ceph-mon, ceph-mds were not running then
killall -q will exit status 1, leading to anacron sending a mail

  error: error running shared postrotate script for '/var/log/ceph/*.log '

Fix by overriding the exit status with || true.

Fixes: #13033
Signed-off-by: Dan van der Ster <daniel.vanderster@cern.ch>

Merge pull request #5800 from ceph/wip-user

ceph-disk: do async startup on upstart; fix dmcrypt create/activate bug

Reviewed-by: Loic Dachary <ldachary@redhat.com>

Merge pull request #5031 from trociny/wip-10098

ceph-objectstore-tool: add mark-complete operation

Reviewed-by: David Zafman <dzafman@redhat.com>

Merge pull request #5866 from jmunhoz/wip-requestid-on-error

rgw: include RequestId as part of the Error response

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>

Merge pull request #5859 from jecluis/wip-test-mon-ping

test: mon/mon-ping.sh: make sure 'ceph mon ping' works as expected

Reviewed-by: Loic Dachary <ldachary@redhat.com>

Merge pull request #5873 from dillaman/wip-rbd-missing-return

rbd: missing return statement within do_metadata_remove

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

rbd: missing return statement within do_metadata_remove

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #5780 from dzafman/wip-12387

osd: When generating past intervals due to an import end at pg epoch

Reviewed-by: Sage Weil <sage@redhat.com>

test/Makefile-client: ship LibradosTestStub.h in tarball

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #5868 from jmunhoz/wip-x-amz-request-id-doc

doc: rgw: update x-amz-request-id status

Reviewed-by: Abhishek Lekshmanan <abhishek.lekshmanan@ril.com>

Merge pull request #5863 from ceph/wip-13015

rgw: preserve all attrs if intra-zone copy

doc: rgw: update x-amz-request-id status

Signed-off-by: Javier M. Mellid <jmunhoz@igalia.com>

rgw: include RequestId as part of the Error response

The RGW error responses must contain a RequestId to be compliant with
the Amazon S3 errors. This RequestId is the ID of the request associated
with the error.

Fixes: #13020
Signed-off-by: Javier M. Mellid <jmunhoz@igalia.com>

test: mon/mon-ping.sh: make sure 'ceph mon ping' works as expected

Signed-off-by: Joao Eduardo Luis <joao@suse.de>

Merge pull request #5343 from hjwsm1989/wip-12442

  mon/MonClient: fix error in 'ceph ping mon.id'

Reviewed-by: Joao Eduardo Luis <joao@suse.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

ceph-objectstore-tool: add mark-complete operation

It is supposed to be used as a last resort to fix a cluster that has
PGs in 'incomplete' state, using the following procedure:

1) stop the osd that is primary for the incomplete PG;
2) run:
  ceph-objectstore-tool --data-path ... --journal-path ... --pgid $PGID --op mark-complete
3) start the osd.

Fixes: #10098
Signed-off-by: Mykola Golub <mgolub@mirantis.com>

Merge pull request #5816 from dillaman/wip-librbd-gmock

Initial support for gmock-based unit tests in librbd

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #5557 from guangyy/wip-12666

rgw: expose the number of unhealthy workers through admin socket

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #5786 from ceph/wip-12911

rbd export-diff should export data chunks in-order

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #5858 from jecluis/wip-test-mon-scrub

test: mon/mon-scrub.sh: have test running on make check

Reviewed-by: Loic Dachary <ldachary@redhat.com>

rgw: preserve all attrs if intra-zone copy

Fixes: #13015
Intra zone copy requires that all objects' attributes are preserved.
This was broken at commit: e41d97c8e38bb60d7e09e9801c0179efe7af1734

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

Merge pull request #5442 from ceph/wip-12551

ReplicatedPG,Objecter: copy_get should include truncate_seq and size

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #5490 from dwj192/master

rgw:add --reset-regions for regionmap update

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>

test/Makefile.am: run mon/mon-scrub.sh as part of checks

Signed-off-by: Joao Eduardo Luis <joao@suse.de>

test: mon/mon-scrub.sh: port clashed with other tests

This will allow the test to be run during make check.

Signed-off-by: Joao Eduardo Luis <joao@suse.de>

Merge pull request #5828 from lebauce/fix-swift-prefix

rgw: add delimiter to prefix only when path is specified

Reviewed-by: Radoslaw Zarzynski <rzarzynski@mirantis.com>

ReplicatedPG,Objecter: copy_get should include truncate_seq and size

Otherwise, we break CephFS over cache tiers.

Fixes: #12551
Signed-off-by: Samuel Just <sjust@redhat.com>

Merge pull request #5837 from tchaikov/wip-12968

mon: disable gmt_hitset if not supported

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Loic Dachary <ldachary@redhat.com>

Merge pull request #5842 from dzafman/wip-cot-failure

test: Fix failure test to find message anywhere in stderr

Reviewed-by: Loic Dachary <ldachary@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #5845 from wonzhq/async-rx

AsyncMessenger: add instance name in debug log when processing msg

Reviewed-by: Haomai Wang <haomaiwang@gmail.com>

Merge pull request #5779 from dzafman/wip-misc-fix-guangyy

osd: check the length of the map before accessing the first element

Reviewed-by: Loic Dachary <ldachary@redhat.com>

Merge pull request #5843 from dillaman/wip-12885

librbd: diff_iterate needs to handle holes in parent images

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #5592 from xinxinsh/wip-fix-rbd-metadata

fix bugs of rbd metadata operation

Reviewed-by: Haomai Wang <haomaiwang@gmail.com>
Reviewed-by: Jason Dillaman <dillaman@redhat.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>

test: Fix failure test to find message anywhere in stderr

Consolidate test_failure() and test_failure_tty()

Signed-off-by: David Zafman <dzafman@redhat.com>

rados: Fix usage for "notify" command

Signed-off-by: David Zafman <dzafman@redhat.com>

Merge pull request #5834 from ceph/wip-leveldb-hang

mon: MonitorDBStore: make get_next_key() work properly

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #4955 from kylinstorage/librados-2read-segmentation-fault-v1

bug fix: librados segmentation fault, when two read ops share one Aio…

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #4925 from XinzeChi/wip-hit-set-size

osd: fix hit_set_map size for tier pool

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #5723 from H3C/wip-msg-bugfix1

msg: we should set the socket options before connect or listen in order to have it take effect.

Reviewed-by: Haomai Wang <haomaiwang@gmail.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #5829 from dachary/wip-12731-jerasure-warnings

erasure-code: fix gf-complete warnings

Reviewed-by: Kefu Chai <kchai@redhat.com>

AsyncMessenger: add instance name in debug log when processing msg

To better debug.

Signed-off-by: Zhiqiang Wang <zhiqiang.wang@intel.com>

rgw:add --reset-regions for regionmap update

Fixes: #12964
Signed-off-by: Weijun Duan <duanweijun@h3c.com>

Merge pull request #5781 from dzafman/wip-eccleanup

osd, test: Minor clean-up from fast-read and error handling ec changes

Reviewed-by: Kefu Chai <kchai@redhat.com>

librbd: diff_iterate needs to handle holes in parent images

If a clone's parent image snapshot includes a discarded extent,
this was previously causing an assert failure.  Instead, ignore
any discard holes in the parent image.

Fixes: #12885
Backport: hammer
Signed-off-by: Jason Dillaman <dillaman@redhat.com>

tests: new test case for librbd diff_iterate over discard extents

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #5838 from ceph/wip-12861

qa/workunits/rados/test_alloc_hint.sh: sudo to ls files

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #5396 from XinzeChi/wip-scrub-promote

bug fix: osd: requeue_scrub when kick_object_context_blocked

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: David Zafman <dzafman@redhat.com>

Merge pull request #5772 from yuyuyu101/wip-12912

AsyncConnection: Close connection when unregistered connection met WAIT

Merge pull request #5733 from cxwshawn/rgw-fix

rgw: init_rados failed leads to repeated delete

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>

Merge pull request #5809 from rzarzynski/wip-rgw_putobj_debug_improvement

rgw: improve debugs in RGWPutObj and RGWPutObj_ObjStore_SWIFT.

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>

Merge pull request #5567 from guce/rgw_check_key_parameter_conflict

rgw: add key parameter conflict check for radosgw-admin command line.

Merge pull request #5819 from rzarzynski/wip-12950

rgw: don't append empty ETag HTTP header.

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>

Merge remote-tracking branch 'gh/infernalis'

qa/workunits/rados/test_alloc_hint.sh: sudo to ls files

The osd data dir is owned by ceph and not readable by other
non-root users.

Fixes: #12861
Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #5257 from t-miyamae/wip-mshec-fixlogic4

erasure code: shec performance optimization by decoding cache

Reviewed-by: Loic Dachary <ldachary@redhat.com>

rgw: init_rados failed leads to repeated delete

Fixes: #12978
Signed-off-by: Xiaowei Chen <chen.xiaowei@h3c.com>

Merge pull request #5463 from kylinstorage/wip-evict_ops

Allow evict operations to be throttled

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #5745 from ceph/wip-12904

osd: sparse reads returning invalid extent map

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #5580 from xinxinsh/wip-merge-two-setattr

merge two continuous OP_SETATTR ops into one OP_SETATTRS

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

mon: disable gmt_hitset if not supported

the gmt_hitset is enabled by default in the ctor of pg_pool_t, this
is intentional. because we want to remove this setting and make
gmt_hitset=true as a default in future. but this forces us to
disable it explicitly when preparing a new pool if any OSD does
not support gmt hitset.

Fixes: #12968
Signed-off-by: Kefu Chai <kchai@redhat.com>

test: mon: mon-scrub.sh: test 'mon scrub'

In its current state is used to catch regressions on 'ceph mon scrub'
hanging due to an infinite loop.

Signed-off-by: Joao Eduardo Luis <joao@suse.de>

Merge pull request #5785 from liewegas/wip-12861

qa/workunits/rados/test_alloc_hint.sh: sudo to list files

mon: MonitorDBStore: make get_next_key() work properly

We introduced a significant bug with 2cc7aee, when we fixed issue #11786.
Although that patch would fix the problem described in #11786, we
managed to not increment the iterator upon returning the current key.
This would have the iterator iterating over the same key, forever and
ever.

Signed-off-by: Joao Eduardo Luis <joao@suse.de>

Merge pull request #5784 from dachary/wip-docker-helper

fix docker-test.sh for CentOS 7

Reviewed-by: Kefu Chai <kchai@redhat.com>

tests: drop docker-tests.sh root and /dev support

Now that ceph-disk.sh activation tests switched to the ceph-disk suite
as found in https://github.com/ceph/ceph-qa-suite/tree/master/suites/ceph-disk,
it does not need for root and /dev support from docker-tests.sh. There
currently is no other use case and since it's generally not a good idea
to run anything as root in a container anyway.

Signed-off-by: Loic Dachary <ldachary@redhat.com>

tests: CentOS 7 needs systemd-container

It is now needed by the Ceph dependencies and won't work unless the fake
version of it is replaced by the actual one.

Signed-off-by: Loic Dachary <ldachary@redhat.com>

rgw: add delimiter to prefix only when path is specified

http://tracker.ceph.com/issues/12960
Fixes: #12960
Signed-off-by: Sylvain Baubeau <sbaubeau@redhat.com>

Merge pull request #5728 from trociny/wip-12706-infernalis

ceph-disk: use /sys/dev/block/maj:min/partition to see if partition

Reviewed-by: Loic Dachary <ldachary@redhat.com>

refine tests for metadata ops

Signed-off-by: xinxin shu <xinxin.shu@intel.com>

improve error handle of rbd metadata operation & format output

Signed-off-by: xinxin shu <xinxin.shu@intel.com>

fix metadata loading error if we open an image

Signed-off-by: xinxin shu <xinxin.shu@intel.com>

msg: we should set the socket options before connect or listen in order to have it take effect.

Signed-off-by: Ruifeng Yang <149233652@qq.com>

erasure-code: fix gf-complete warnings

Update to the latest gf-complete v2 branch which has the compilation
warning fixed. In addition the default compilation flag upstream has
been set to display such warnings.

git log --oneline --graph d384952c68a64d93ac7af6341d5519ea5d2958b9..9caeefbf2860e56a75502f4d3342deed5b5ba265
* 9caeefb add -Wsign-compare and address the warnings
* 5979f98 gf_w64.c: fix integer overflow
* 93a9845 gf_w64.c: fix integer overflow
* 513c87b gf_w64.c: fix integer overflow
* 4d6fa89 gf_w64.c: fix integer overflow

http://tracker.ceph.com/issues/12731 Fixes: #12731

Signed-off-by: Loic Dachary <ldachary@redhat.com>

Merge remote-tracking branch 'gh/infernalis'

Merge pull request #4895 from XinzeChi/wip-outdata-set

bug fix: osd: avoid multi set osd_op.outdata in tier pool

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #5707 from athanatos/wip-12809

PG::handle_advance_map: on_pool_change after handling the map change

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #5751 from yuyuyu101/wip-12908

AsyncConnection: Don't use unsafe feature as message encode feature

Reviewed-by: Sage Weil <sage@redhat.com>

Merge remote-tracking branch 'gh/infernalis'

qa/workunits/post-file.sh: sudo

Only root can read the key in /usr/share/...

Signed-off-by: Sage Weil <sage@redhat.com>

osd: dump full map bl at 20 when crc doesn't match

This will help us debug cases where the encoding doesn't match due
to a bug.

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #5827 from tchaikov/wip-doc-daemon

doc: fix the typo in command example

Reviewed-by: Loic Dachary <ldachary@redhat.com>

doc: fix the typo in command example

always indent using tab, the rendered html looks good, but it helps with
editor to highlight the codeblock properly.

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #5817 from liewegas/wip-ceph-daemon

doc: 'ceph --admin-daemon ...' -> 'ceph daemon ...'

Reviewed-by: Loic Dachary <ldachary@redhat.com>

Merge pull request #5193 from SUSE/wip-babeltrace-suse

ceph.spec.in: fix lttng/babeltrace conditionals

Reviewed-by: Ken Dreyer <kdreyer@redhat.com>

Merge pull request #5796 from SUSE/wip-group-cond

ceph.spec.in: put distro conditionals around Group:

Reviewed-by: Ken Dreyer <kdreyer@redhat.com>

doc: 'ceph --admin-daemon ...' -> 'ceph daemon ...'

Signed-off-by: Sage Weil <sage@redhat.com>

tests: base gmock class support for librbd

Created mock classes to represent a few central librbd classes
and a basic gmock test fixture for future gmock-based unit tests.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

librbd: support templating of ImageCtx for async state machines

This will facilitate the creation of unit test mocks to verify
non-librados actions. The templates (by default) will be flagged
as extern to avoid the translation unit bloat.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #5624 from vumrao/wip-vumrao-12714

rgw : setting max number of buckets for users via ceph.conf option

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>

ceph.spec.in: put distro conditional around Group:

Neither RHEL and Fedora require a Group: line, so retain
only for SLE/openSUSE (and set the right value for these distros)

Signed-off-by: Nathan Cutler <ncutler@suse.com>

Merge pull request #5798 from SUSE/wip-fix-license

ceph.spec.in: fix License line

Reviewed-by: Ken Dreyer <kdreyer@redhat.com>

librados_test_stub: add mock class for IoCtx operations

Unit tests can now use gmock to simulate responses from the OSDs
via a mocked librados library.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #5801 from dreamhost/wip-12939-header

Fix casing of Content-Type header

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>

Merge pull request #5811 from vumrao/wip-vumrao-12944

doc: Add pgcalc tool link in placement-groups document

Reviewed-by: Loic Dachary <ldachary@redhat.com>

ceph.spec.in: fix lttng/babeltrace conditionals

lttng and babeltrace are build dependencies for rbd-replay-prep. Make
sure the right package names are used. Enable for SLE12, as well as
for openSUSE 13.1 and higher.

Move the BuildRequires out of the ceph-test subpackage and into the
distro-conditional dependencies section.

Make ordering of BuildRequires a little more alphabetical.

http://tracker.ceph.com/issues/12360 Fixes: #12360

Signed-off-by: Nathan Cutler <ncutler@suse.com>
sqme

Merge pull request #5791 from SUSE/wip-drop-lsb-release

ceph.spec.in: drop lsb-release dependency from ceph-common

Reviewed-by: Ken Dreyer <kdreyer@redhat.com>

rgw: don't append empty ETag HTTP header.

Fixes: #12950
Signed-off-by: Radoslaw Zarzynski <rzarzynski@mirantis.com>

Merge pull request #5808 from ceph/wip-simplify-logrotate

logrotate.conf: Simplify log files reopening after log rotation

Reviewed-by: Milan Broz <mbroz@redhat.com>
Reviewed-by: Sage Weil <sage@redhat.com>

doc: Add pgcalc tool link in placement-groups document

Fixes #12944

Signed-off-by: Vikhyat Umrao <vumrao@redhat.com>