src/ceph_osd.cc Add missing newline to usage message

Signed-off-by: Willem Jan Withagen <wjw@digiware.nl>

Merge pull request #6920 from oritwas/wip-mstart-cmake

ajdust mstart and mstop script to run with cmake build

Merge pull request #7523 from tchaikov/wip-fix-cmake

cmake: fix the build of tests

Merge pull request #7607 from FlorentCoppint/master

mount.fuse.ceph: better parsing of arguments passed to mount.fuse.ceph by mount command

Reviewed-by: Sage Weil <sage@redhat.com>

Better parsing of arguments passed to mount.fuse.ceph by mount command.

Signed-off-by: Florent Bautista <florent@coppint.com>

Merge pull request #5961 from majianpeng/small-fix

osd: clean up CMPXATTR checks

Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #7533 from Cdiscount/master

debian: include cpio in build-requiers

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #7537 from ifed01/wip-no-promote-for-delete-fix

osd: fix unnecessary object promotion when deleting from cache pool

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #7567 from branch-predictor/bp-slimdown-conns

msg: significantly reduce minimal memory usage of connections

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #7579 from javacruft/wip-systemd-escaping

systemd: correctly escape block device paths

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #7581 from jcsp/wip-asok-lockdep

librados: mix lock cycle (un)registering asok commands

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #7604 from ceph/wip-selinux-update-radosgw-log

selinux: allow log files to be located in /var/log/radosgw

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #7606 from tchaikov/wip-fix-ceph-disk

tests: ceph-disk.sh: use "readlink -f" instead for fullpath

Reviewed-by: Loic Dachary <ldachary@redhat.com>

tests: ceph-disk.sh: use "readlink -f" instead for fullpath

if $PATH has ".." in it, and the program happen to be located in
"..", `which program` will print `../program` instead of its fullpath,
so we should always use `readlink -f` for the fullpath.

Signed-off-by: Kefu Chai <kchai@redhat.com>

selinux: Allow log files to be located in /var/log/radosgw

We do suggest users to put their logs in /var/log/radosgw in the
documentation at times. We should also label that directory with
ceph_var_log_t so that ceph daemons can also write there.

The commit also updates the man page for this policy. This man page is
automatically generated by

* sepolicy manpage -p . -d ceph_t

and have not been reloaded in a while. Hence, it contains few more
changes than the new radosgw directory.

Signed-off-by: Boris Ranto <branto@redhat.com>

Merge pull request #7601 from jjhuo/master

os/bluestore: fix a typo in SPDK path parsing

Reviewed-by: Haomai Wang <haomai@xsky.com>

os/bluestore: fix a typo in SPDK path parsing

SPDK block path with bluestore is something like spdk:55cd2e404bd73932.
The size of the prefix to compare should be 5.
sizeof(SPDK_PREFIX)-1 returns 5, while sizeof(SPDK_PREFIX-1) returns 8.

Signed-off-by: Jianjian Huo <samuel.huo@gmail.com>

Merge pull request #7588 from dmick/master

common/page.cc: _page_mask has too many bits

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #7413 from YankunLi/dev

rgw: remove duplicated code in RGWRados::get_bucket_info()

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>

Merge branch 'wip-11642-0203'

rgw: make function parameter 'errordoc_key' passed by reference

Signed-off-by: Bo Cai <cai.bo@h3c.com>

Merge branch 'master' of github.com:ceph/ceph

Merge pull request #7544 from ceph/wip-rh-covscan

rgw/rgw_orphan: check the return value of save_state

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>

Merge pull request #7586 from bsedyang/master

rgw: user quota may not adjust on bucket removal

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>

doc/release-notes: v10.0.3

Finalized release notes.

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #7592 from dachary/wip-release-notes

release-notes: draft v10.0.3 release notes

Merge pull request #7594 from tchaikov/fix-ceph-disk-test

tests: ceph-disk.sh: should use "readlink -f" instead

Reviewed-by: Loic Dachary <ldachary@redhat.com>

tests: ceph-disk.sh: should use "readlink -f" instead

turns out trusty does not have `realpath` unless the package named
`realpath` is installed.

Reported-by: Joao Eduardo Luis <joao@suse.de>
Signed-off-by: Kefu Chai <kchai@redhat.com>

release-notes: draft v10.0.3 release notes

Signed-off-by: Loic Dachary <loic@dachary.org>

scripts: ceph-release-notes fix merge messages handling

Strip the title line if it is the same as the pr title instead of
ignoring the message entirely.

Signed-off-by: Loic Dachary <loic@dachary.org>

cmake: remove Boost libraries from EXTRALIBS

* and remove ${Boost_SYSTEM_LIBRARY} from test_rados_api_tier

Signed-off-by: Kefu Chai <kchai@redhat.com>

log: remove "boost/asio.hpp" from Log.h

graylog uses boost/asio.hpp which introduces the link-time dependency on
libboost_system to the compilation units which includes Log.h and
LogClient.h. so it appears that perfglue/heap_profiler.cc is referencing
libboost_system.so, and fails the cmake build of all tests which links
against tcmalloc.

in this change, we:

* remove unnecessary #includes from Graylog.h
* forward declare Graylog class, so that "Graylog.h" is not included in
  any header files to avoid the link-time dependency pollution

Signed-off-by: Kefu Chai <kchai@redhat.com>

cmake: disable warnings introduced by 457f023

Signed-off-by: Kefu Chai <kchai@redhat.com>

Revert "cmake: add libboost_system to EXTRALIBS"

This reverts commit 21438a6773a3153556de07096919ab46c4540eee.

Signed-off-by: Kefu Chai <kchai@redhat.com>

common/page.cc: _page_mask has too many bits

It's not used anywhere, but in case it is one day...

Signed-off-by: Dan Mick <dan.mick@redhat.com>

rgw: user quota may not adjust on bucket removal

Description:
If the user/admin removes a bucket using --force/--purge-objects options with s3cmd/radosgw-admin respectively, the user stats will continue to reflect the deleted objects for quota purposes, and there seems to be no way to reset them. User stats need to be sync'ed prior to bucket removal.

Solution:
Sync user stats before removing a bucket.

Fixes: #14507
Signed-off-by: Edward Yang eyang@us.fujitsu.com

Merge pull request #7496 from xiexingguo/xxg-wip-fixnvme

osd: BlueStore/NVMEDevice: fix compiling and fd leak

Reviewed-by: Haomai Wang <haomai@xsky.com>

Merge pull request #7349 from yuyuyu101/add-pwritev

osd: FileStore: use pwritev instead of lseek+writev

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #7504 from renhwsky/renhw-wip-mon-shutdown

mon: unregister command on shutdown

Reviewed-by: Joao Eduardo Luis <joao@suse.de>

Merge pull request #7550 from tchaikov/wip-osd-cleanup

osd: a fix for HeartbeatDispatcher and cleanups

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #7553 from XinzeChi/wip-osdmap-affinity

osd: OSDMap: reset osd_primary_affinity shared_ptr when deepish_copy_from

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #7560 from tchaikov/wip-fix-icl

crush/CrushTester: workaround a bug in boost::icl

Merge pull request #7568 from liewegas/wip-bluestore-mutex

osd: bluestore/blockdevice: use std::mutex et al

debian: fix build-depends

Signed-off-by: Rémi BUISSON <remi.buisson@cdiscount.com>

Merge pull request #7578 from ceph/fix-stripper

Stripper: Fix incorrect push_front -> append_zero change

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

osdc/Objecter: drop lock to unregister asok hooks

unregister blocks until any running hook has
completed, and that running hook might
be one that takes the objecter lock, so you
can't hold it while unregistering.

Signed-off-by: John Spray <john.spray@redhat.com>

common: asok should drop lock to call hook

Otherwise, hooks cannot take any locks which
were held during registration of hooks, because
it creates a cycle.

Signed-off-by: John Spray <john.spray@redhat.com>

Correctly escape block device paths for systemd

Ensure that the leading / is stripped from block device
paths before escaping for using in systemd unit names.

Fixes: #14706
Signed-off-by: James Page <james.page@ubuntu.com>

Stripper: Fix incorrect push_front -> append_zero change

Signed-off-by: Haomai Wang <haomai@xsky.com>

Merge pull request #7444 from dreamhost/wip-rgw-static-website-robbat2

rgw: approximate AmazonS3 HostId error field.

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>

Merge remote-tracking branch 'gh/jewel'

v10.0.3

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #7572 from ceph/wip-cmake-2-8-16

cmake: Added new unittests to make check

Merge pull request #7574 from SUSE/wip-spec-license

ceph.spec.in: add license declaration

Reviewed-by: Sage Weil <sage@redhat.com>

ceph.spec.in: add license declaration

Signed-off-by: Nathan Cutler <ncutler@suse.com>

cmake: Added new unittests to make check

Added unittest_shunique_lock & unittest_mutex_debug
to make check, and made them link ALLOC_LIBS

Signed-off-by: Ali Maredia <amaredia@redhat.com>

Merge pull request #7569 from SUSE/wip-copyright2

ceph.spec.in: add copyright notice

Reviewed-by: Sage Weil <sage@redhat.com>

ceph.spec.in: add copyright notice

http://tracker.ceph.com/issues/14694 Fixes: #14694

Signed-off-by: Nathan Cutler <ncutler@suse.com>

os/bluestore/BlockDevice: std::atomic_int

Signed-off-by: Sage Weil <sage@redhat.com>

os/bluestore/BlockDevice: use std::mutex and std::condition_variable

Pull aio wait code into an IOContext method too.

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #7513 from athanatos/wip-ceph-test-rados-2

ceph_test_rados: use less CPU

Merge pull request #5931 from ceph/wip-cxx11concurrency

librados: move to c++11 concurrency types

Reviewed-by: Matt Benjamin <mbenjami@redhat.com>
Reviewed-by: Casey Bodley <cbodley@redhat.com>

msg/simple: significantly reduce minimal memory usage of Pipes

Pipe class instance weights over 17 KB in size, and most of it is used by
msgvec structure, which weights 16 KB. Reduce it to 4KB by reducing
number of iovecs in this array down to 256 (= 1024/4). This won't cause
noticeable perf decrease, as even under extreme loads, we exceed 256 used
iovecs in only less than 1% of all write calls. For high-density nodes and
large clusters, savings will exceed hundreds of megabytes per node.

Signed-off-by: Piotr Dałek <piotr.dalek@ts.fujitsu.com>

msg/async: significantly reduce minimal memory usage of AsyncConnections

AsyncConnection class instance weights over 18 KB in size, and most of it
is used by msgvec structure, which weights 16 KB. Reduce it to 4KB by reducing
number of iovecs in this array down to 256 (= 1024/4). This won't cause
noticeable perf decrease, as even under extreme loads, we exceed 256 used
iovecs in only less than 1% of all write calls. For high-density nodes and
large clusters, savings will exceed hundreds of megabytes per node.

Signed-off-by: Piotr Dałek <piotr.dalek@ts.fujitsu.com>

Merge branch 'print-blocked-op' of git://github.com/majianpeng/ceph

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #6608 from esnyder/wip-6634

osd: remove unused SnapContext member snapc from MOSDSubOp message

Reviewed-by: Xiaoxi Chen <xiaoxi.chen@intel.com>
Reviewed-by: Sage Weil <sage@redhat.com>

Merge branch 'master' of git://github.com/joconcepts/ceph

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #7181 from songbaisen/song5

crush: reply quickly from get_immediate_parent

Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>

Merge pull request #7437 from Ved-vampir/wip_zlib_plugin

common: add zlib compression plugin

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #7527 from liewegas/wip-no-rgw-prestart

systemd/ceph-radosgw-prestart.sh: remove

Reviewed-by: Nathan Cutler <ncutler@suse.com>

OSD/ReplicatedPG: Fixes unneccessary object promotion when deleting from the cache that lacks hit_set configured.

Signed-off-by: Igor Fedotov <ifedotov@mirantis.com>

Merge pull request #7547 from yuyuyu101/wip-14679

AsyncConnection: avoid debug log in cleanup_handler

crush/CrushTester: workaround a bug in boost::icl

the boost::icl in boost v1.55 has a bug when working with libc++,

> Assertion failed: (this->_map.find(inter_val) == this->_map.end()),
> function gap_insert, file
> /usr/local/include/boost/icl/interval_base_map.hpp, line 555.

see https://svn.boost.org/trac/boost/ticket/9501 and
    https://svn.boost.org/trac/boost/ticket/9987

the bug was fixed in boost v1.56. as a workaround we use the the
`set` from `boost::container` instead of the `std::set` from libc++.
we should drop this workaround once we bump the supported boost
version from 1.42 to 1.56.

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #7554 from tchaikov/wip-ceph-disk-test

tests: ceph-disk.sh should compare with the resolved path

Reviewed-by: Loic Dachary <ldachary@redhat.com>

OSDMap: reset osd_primary_affinity shared_ptr when deepish_copy_from

Base on current logic, OSDMonitor may call create_pending and
encode_pending twice for the some epoch.

In encode_pending:

tmp.deepish_copy_from(osdmap);
tmp.apply_incremental(pending_inc);

This Op would change the tmp osd_primary_affinity, but the osd_primary_affinity
is declared as ceph::shared_ptr, so this would change the osdmap too. When this
round encode_pending is proposed fail. We may call encode_pending again, but the
osdmap is changed last round, so the pending_inc would be wrong.

Fixes: #14686
Signed-off-by: Xinze Chi <xinze@xsky.com>

Merge pull request #7551 from JevonQ/update-libcephfs-version

libcephfs: update LIBCEPHFS_VERSION to indicate the interface was changed

Reviewed-by: Sage Weil <sage@redhat.com>

tests: ceph-disk.sh should compare with the resolved path

Signed-off-by: Kefu Chai <kchai@redhat.com>

libcephfs: Update LIBCEPHFS_VERSION to indicate the interface was changed

Signed-off-by: Jevon Qiao <scaleqiao@gmail.com>

AsyncMessenger: remove unused variables

* remove AsyncMessenger::listen_sd
* remove `ee` in KqueueDriver::del_event()

Signed-off-by: Kefu Chai <kchai@redhat.com>

crypto: remove unused Digest::sec_type

Signed-off-by: Kefu Chai <kchai@redhat.com>

journal: remove unused ObjectPlayer::m_fetch_interval

Signed-off-by: Kefu Chai <kchai@redhat.com>

librbd: remove unused C_DiffObject::m_request_num

Signed-off-by: Kefu Chai <kchai@redhat.com>

librbd: replace atomic_t with std::atomic<int>

it fails to compile with clang, and turns out it should be
`ceph::atomic_t`, but I take this chance to replace it
with std::atomic<> to phase out atomic_t.

Signed-off-by: Kefu Chai <kchai@redhat.com>

ECUtil: remove unused stripe_info_t::stripe_size

Signed-off-by: Kefu Chai <kchai@redhat.com>

osd: init HeartbeatDispatcher using OSD->cct

its base class 'Dispatcher' is uninitialized when used here to access
'Dispatcher::cct', so we'd better use OSD::cct to initialize it instead.

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #7549 from tchaikov/wip-fix-ceph-disk-tests

ceph-disk/test: fix test_prepare.py::TestPrepare tests

Reviewed-by: Loic Dachary <ldachary@redhat.com>

ceph-disk/test: fix test_prepare.py::TestPrepare tests

actually we are verifying if the variable is an instance of
specified class. for example, the `prepare.data` should be
a `PrepareFilestoreData` if `--bluestore` is not specified.

Signed-off-by: Kefu Chai <kchai@redhat.com>

AsyncConnection: avoid debug log in cleanup_handler

local connection will be stop and call cleanup_handler after messenger
is down
introduced in
commit(https://github.com/ceph/ceph/commit/9da2fffd31562ed5d0b795d7862b3ebec66aba40)

Fix #14679

Signed-off-by: Haomai Wang <haomai@xsky.com>

Merge pull request #7540 from dillaman/wip-librados-test-stub-unwatch

librados_test_stub: protect against notify/unwatch race

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #7529 from dillaman/wip-14412

librbd: journal framework for tracking exclusive lock transitions

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

rgw/rgw_orphan: check the return value of save_state

It was discovered by covscan that we do not check the return value of
save_state in this one case.

Signed-off-by: Boris Ranto <branto@redhat.com>

Merge pull request #7377 from liewegas/wip-datadir-search

config: add $data_dir/config to config search path

Merge pull request #7466 from adamemerson/wip-bsd-time-second-try

common: time: have skewing-now call non-skewing now

Merge pull request #7463 from dachary/wip-13422-pid-file

global: do not start two daemons with a single pid-file (part 2)

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #7460 from xiexingguo/xxg-wip-block

osd: blockdevice: avoid implicit cast and add guard

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #7530 from xiexingguo/xxg-wip-nid

osd: bluestore: assign nid for newly created onode

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #7521 from liewegas/wip-memstore

osd: memstore: set Collection::cid on create

Merge pull request #7486 from ceph/wip-bluestore-oref

osd: bluestore: fix two wal bugs; reuse OnodeRefs when applying txc

Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>

Merge pull request #7531 from jjhuo/master

osd: bluestore: remove unused intrusive member hook in enode

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #7535 from liewegas/wip-bluefs-mkfs

osd: bluestore/BlueFS: initialize super block_size earlier in mkfs

Reviewed-by: Dan Mick <dmick@redhat.com>