CMakeLists.txt/FreeBSD: Boost_python needs explicit linking with libpython*.so

 - Otherwise losts of unresolved errors will result.

Signed-off-by: Willem Jan Withagen <wjw@digiware.nl>

Merge pull request #12734 from liupan1111/wip-fix-nvme-driver-compile-issue

spdk: upgrade spdk to v16.12

Reviewed-by: Haomai Wang <haomai@xsky.com>

Merge pull request #12751 from tchaikov/wip-18395

ceph_test_rados_api_watch_notify: move global variables into test class

Reviewed-by: Brad Hubbard <bhubbard@redhat.com>

Merge pull request #12730 from Liuchang0812/fix-dev-doc-pglog

doc/dev: update log_based_pg.rst, fix some display problem

Reviewed-by: Kefu Chai <kchai@redhat.com>

ceph_test_rados_api_watch_notify: move global variables into test class

so the class instances using mempool are destructed before the mempools
are destroyed. this avoids the "invalid read" issue reported by
valgrind.

Fixes: http://tracker.ceph.com/issues/18395
Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #12713 from badone/wip-ceph_test_rados_api_watch_notify-valgrind-fixes

test_rados_watch_notify: Fix trivial memory leaks

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #12680 from badone/wip-ceph_test_rados_api_misc-valgrind-fixes

ceph_test_rados_api_misc: Fix trivial memory leak

Reviewed-by: Kefu Chai <kchai@redhat.com>

doc/dev: update log_based_pg.rst, fix some display problem

Signed-off-by: liuchang0812 <liuchang0812@gmail.com>

os/bluestore: add more useful log information in NVMEDevice.cc

Signed-off-by: Pan Liu <pan.liu@istuary.com>

Merge pull request #12747 from SUSE/wip-18388

tests: rbd/test_lock_fence.sh: fix rbdrw.py relative path

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

tests: rbd/test_lock_fence.sh: fix rbdrw.py relative path

This commit fixes a regression introduced by
cf294777ea92f0911813a7132068584d4f73a65a

Fixes: http://tracker.ceph.com/issues/18388
Signed-off-by: Nathan Cutler <ncutler@suse.com>

os/bluestore: fixed compilation issue when use spdk v16.12

Signed-off-by: Pan Liu <pan.liu@istuary.com>

spdk: fixed the compilation error for testcases when use spdk v16.12

Signed-off-by: Pan Liu <pan.liu@istuary.com>

dpdk: make compilation successful when use dpdk v16.11

Signed-off-by: Pan Liu <pan.liu@istuary.com>

Merge pull request #12706 from Liuchang0812/wip-qa-rados-1228

test: update test_rados_tool.sh, use POOL and OBJ var

Reviewed-by: Kefu Chai <kchai@redhat.com>

spdk: upgrade spdk version to 16.12

Signed-off-by: Pan Liu <pan.liu@istuary.com>

dpdk: upgrade dpdk version to 16.11

Signed-off-by: Pan Liu <pan.liu@istuary.com>

Merge pull request #12716 from kylinstorage/wip-ctypos_h

misc: fix code typos in header files

Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>

Merge pull request #12721 from chendave/wip-authx-options

auth: Let's not use the deprecated cephx option

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #12676 from chendave/master

common/ceph_context: Show clear message if all features are enabled

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #12633 from liewegas/wip-15784

msg/simple: call clear_pipe in wait() shutdown path

Reviewed-by: Haomai Wang <haomai@xsky.com>

Merge pull request #12692 from XinzeChi/wip-bluestore-statfs

os/bluestore: _txc_release_alloc when do wal cleaning

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #12691 from runsisi/wip-dump-osdop

osd/OpRequest: dump both name and addr for the client op

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #12363 from wjwithagen/wip-wjw-freebsd-strerror

common/module.c: do not use strerror_r the GNU way.

Reviewed-by: Sage Weil <sage@redhat.com>

cephx: Let's not use the deprecated option

Enable cephx by setting the option of "auth supported" has been
deprecated for a long time, it's better to use the recommended
options instead.

Signed-off-by: Dave Chen <wei.d.chen@intel.com>

Merge pull request #12718 from kylinstorage/wip-orig

organizationmap: add Xianxia Xiao to Kylin Cloud team

organizationmap: add Xianxia Xiao to Kylin Cloud team

Signed-off-by: Yunchuan Wen <yunchuan.wen@kylin-cloud.com>

src/tools/rbd_mirror/Replayer.h : fix typos

Signed-off-by: Xianxia Xiao <xianxia.xiao@kylin-cloud.com>

src/messages/MMDSSlaveRequest.h : fix typos

Signed-off-by: Xianxia Xiao <xianxia.xiao@kylin-cloud.com>

src/tools/rbd_recover_tool/epoch_h : fix typos

Signed-off-by: Xianxia Xiao <xianxia.xiao@kylin-cloud.com>

src/librados/IoCtxImpl.h : fix typos

Signed-off-by: Xianxia Xiao <xianxia.xiao@kylin-cloud.com>

src/journal/JournalTrimmer.h : fix typos

Signed-off-by: Xianxia Xiao <xianxia.xiao@kylin-cloud.com>

src/common/hobject.h : fix typos

Signed-off-by: Xianxia Xiao <xianxia.xiao@kylin-cloud.com>

Merge pull request #12652 from badone/wip-coverity-TracepointProvider

tracing: Fix handle leak in TracepointProvider

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

tracing: Fix handle leak in TracepointProvider

Silences Coverity 1397733

Signed-off-by: Brad Hubbard <bhubbard@redhat.com>

test_rados_watch_notify: Fix trivial memory leaks

Signed-off-by: Brad Hubbard <bhubbard@redhat.com>

Merge pull request #12711 from yuyuyu101/wip-fix-rdma

msg/async/rdma: fix ceph_clock_now calls

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #12451 from liewegas/wip-osd-on-applied

osd: never send rados ack (only commit)

Reviewed-by: Samuel Just <sjust@redhat.com>

Merge pull request #12480 from dx9/wip-boost-python

cmake: fix hard coded boost python lib

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #12406 from songbaisen/s5

tool: add some ceph relate processes to ps-ceph.pl

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #12621 from fiskn/master

doc: Update disk thread section to reflect that scrubbing is no longe…

test: update test_rados_tool.sh, use POOL and OBJ var

* replace hard-code pool name with $POOL
* replace hard-code object name with $OBJ
* introduce a new variable called $POOL_EC
* clean up pool
* simplify test case

Signed-off-by: liuchang0812 <liuchang0812@gmail.com>

msg/async/rdma: fix ceph_clock_now calls

Signed-off-by: Haomai Wang <haomai@xsky.com>

Merge pull request #12403 from liewegas/wip-mon-mgr-cleanups

mon: clean up some osdmon/pgmon interactions

Reviewed-by: Kefu Chai <kchai@redhat.com>

qa/workunits: remove crash_replay_interval tests

Signed-off-by: Sage Weil <sage@redhat.com>

doc: remove crash_replay_interval mentions

Signed-off-by: Sage Weil <sage@redhat.com>

mon/MDSMonitor: no need to set crash_replay_interval on data pools

Signed-off-by: Sage Weil <sage@redhat.com>

osd/PG: remove waiting_for_ack

No longer used.

Signed-off-by: Sage Weil <sage@redhat.com>

osd: simplify dup/resent logic

No need to special ack path.

Signed-off-by: Sage Weil <sage@redhat.com>

osd: remove replay_queue

Signed-off-by: Sage Weil <sage@redhat.com>

osd/PG: do not go into replay state

Signed-off-by: Sage Weil <sage@redhat.com>

osd/PG: do not queue ops for replay

We no longer send ACKs, so this can't happen anyway.

Signed-off-by: Sage Weil <sage@redhat.com>

osd: we can discard "replay" ops

Signed-off-by: Sage Weil <sage@redhat.com>

osd/ReplicatedPG: do not send ACK separate from COMMIT

This never comes before COMMIT, making it useless to clients.  And it's
just unnecessary overhead.

Signed-off-by: Sage Weil <sage@redhat.com>

osd: remove 'rlat' read latency metrics

These are measuring how long of a delay we see before ops are readable.
They are calculated in the on_applied path, which we'd like to remove,
and weren't reliably calculated anyway.

Signed-off-by: Sage Weil <sage@redhat.com>

osd/ReplicatedPG: kill hit_set_flushing

This was added before we started taking the ondisk lock in order to allow
reading a just-written hitset.  Since this is a rare path anyway, just
take the ondisk lock.

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #12522 from hjwsm1989/ecwrite

osd/ECBackend: Send write message to peers first, then do local write

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #12524 from yuyuyu101/wip-bluestore-flush

os/bluestore: prevent lock for almost "flush" calls

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #12528 from kylinstorage/wip-JournalingObjectStore-cleanup

os/filestore/JournalingObjectStore cleanup

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #12670 from liewegas/wip-nits

mon: fix a few nits

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #12684 from xiexingguo/xxg-wip-fix-onodesizetracking

os/bluestore: fix OnodeSizeTracking testing

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #12707 from tchaikov/wip-dpdk-v16.07

builds with dpdk v16.07

Reviewed-by: Haomai Wang <haomai@xsky.com>

msg/async: bump dpdk to v16.07

as some distros include it already. so sync it up to avoid building
failures on them with dpdk-dev package(s) shipped by distro.

Signed-off-by: Kefu Chai <kchai@redhat.com>

msg/async: be compatible with dpdk v16.07

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #12698 from yuyuyu101/wip-dpdk-fix

msg/async/dpdk: fix compile errors

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #12699 from wjwithagen/patch-15

qa/test_rados_tool.sh: POSIX dd only accepts 'k' as multiplier

Reviewed-by: Kefu Chai <kchai@redhat.com>

osd/OpRequest: dump both name and addr for the client op

Signed-off-by: runsisi <runsisi@zte.com.cn>

mon/OSDMonitor: remove workaround for dumpling-era bug

We will definitely have published (and stored) a new osdmap since then.

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #12703 from kylinstorage/wip-remove-unused-variable

os/bluestore/BlueStore.cc: remove unused variable

Reviewed-by: Igor Fedotov <ifedotov@mirantis.com>

Merge pull request #12594 from wjwithagen/wip-wjw-freebsd-rebind-timing

common/config_opts.h: FreeBSD timing changed due to no SO_REUSEADDR

Reviewed-by: Sage Weil <sage@redhat.com>

os/bluestore/BlueStore.cc: remove unused variable

Signed-off-by: Li Wang <li.wang@kylin-cloud.com>

qa/test_rados_tool.sh: POSIX dd only accepts 'k' as multiplier

FreeBSD does not want KB as 1k multiplier.
see also: http://www.unix.com/man-page/posix/1posix/dd/

Signed-off-by: Willem Jan Withagen <wjw@digiware.nl>

test_userspace_event: fix missing std

Signed-off-by: Haomai Wang <haomai@xsky.com>

msg/dpdk/TCP: remove unneeded CephContext argument

Signed-off-by: Haomai Wang <haomai@xsky.com>

os/bluestore: _txc_release_alloc when do wal cleaning

Signed-off-by: Xinze Chi <xinze@xsky.com>

Merge pull request #12688 from Adirl/clean_line_end

msg/async/rdma: clean line endings

Reviewed-by: Haomai Wang <haomai@xsky.com>

os/bluestore: fix OnodeSizeTracking testing

[  FAILED  ] 1 test, listed below:
[  FAILED  ] ObjectStore/StoreTest.OnodeSizeTracking/2, where GetParam() = "bluestore"

 1 FAILED TEST

The above test failure happens as the bluestore mount() process
will try to load all collections and put them into the coll_map,
which will be also tracked as mempool::bluestore_meta_other.
So total_bytes from mempool won't be equal to zero.

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

common/ceph_context: Show clear message if all features are enabled

It's saying the following dangerous and experimental featuresare enabled
while the value is "*", it's not quite clear what's "*" stands for.

The patch updates the message so that it's more clear when "*" is used
in place.

Signed-off-by: Dave Chen <wei.d.chen@intel.com>

Merge pull request #12336 from kylinstorage/wip-cleanup-use-string-ref

osd: cleanup: use string & to avoid unnecessary copy

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #12569 from liewegas/wip-bluestore-raw-unused

os/bluestore/bluestore_types: drop std::bitset for blob unused

Reviewed-by: Igor Fedotov <ifedotov@mirantis.com>
Reviewed-by: Varada Kari <varada.kari@sandisk.com>

Merge pull request #12612 from fangyuxiangGL/master

osd/ReplicatedBackend: do not set omap header if it is empty

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Haomai Wang <haomai@xsky.com>

Merge pull request #12668 from hjwsm1989/pg-unfound

osd/PG: add new have_unfound() function in MissingLoc

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #12674 from Liuchang0812/tools-rados-offset-support

tools/rados: add a parameter "--offset" to rados put command

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #12675 from Liuchang0812/fix-test-case-in-rados

test: replace hard-code binary names with varibles

Reviewed-by: Kefu Chai <kchai@redhat.com>

mon/OSDMonitor: ceph osd pool get/set crush_rule by name

This means users don't have to manually translate a rule
they just created to a ruleset ID in order to map a pool
to it.

Signed-off-by: Sage Weil <sage@redhat.com>

mon/PGMap: remove CATEGORY column from 'df detail'

Signed-off-by: Sage Weil <sage@redhat.com>

mon/PGMap: fix alignment of 'cache io' in ceph -s output

Currently it looks like

  client io 3508 B/s wr, 0 op/s rd, 5 op/s wr
  cache io 0 op/s promote

but should be

  client io 3508 B/s wr, 0 op/s rd, 5 op/s wr
   cache io 0 op/s promote

Signed-off-by: Sage Weil <sage@redhat.com>

mon: 'fs set-default' not 'fs set_default'

The convention is to use - for command names and
_ for variable names.

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #12655 from ceph/wip-fix-fio

test/fio_ceph_objectstore: fix fio plugin build failure caused by rec…

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #12682 from renhwztetecs/renhw-wip-mon-osdmonitor-sortbitwise

mon/OSDMonitor: fixup sortbitwise flag warning

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #12654 from yuyuyu101/wip-fix-async-subop

messages/MOSDSubOp: Make encode_payload can be reentrant

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #12662 from Liuchang0812/doc-fix

doc: fix document about rados mon

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #12198 from tchaikov/wip-pybind-freebsd

pybind: conditional compile the linux specific constant

Reviewed-by: Willem Jan Withagen <wjw@digiware.nl>

mon/OSDMonitor: fixup sortbitwise flag warning

"ceph -s" does not report warning when using
command "ceph osd unset sortbitwise" to drop
sortbitwise flag.
we should use "osdmap.get_up_osd_features() &
CEPH_FEATURE_OSD_BITWISE_HOBJ_SORT"
instead of "(osdmap.get_features(CEPH_ENTITY_TYPE_OSD, NULL) &
CEPH_FEATURE_OSD_BITWISE_HOBJ_SORT)",
because osdmap.get_features only get local "features"

Signed-off-by: huanwen ren <ren.huanwen@zte.com.cn>

ceph_test_rados_api_misc: Fix trivial memory leak

Signed-off-by: Brad Hubbard <bhubbard@redhat.com>

ReplicatedBackend: do not set omap header if it is empty

In replicatedpg backfill, objects have no omap header or keys previously, but an empty omap header will be inserted when they are pushed to replica, which leads leveldb performance degrade seriously when cluster has huge number of objects.
so my opinion is that we does't insert omap header if is size is zero when handle PushOp in recovery.

Signed-off-by: fang yuxiang <fang.yuxiang@eisoo.com>

tools/rados: add parameter offset for put cmd and test case for it

add a parameter "--offset" for put command
add a test case for rados put command

Signed-off-by: liuchang0812 <liuchang0812@gmail.com>

doc: fix document about rados mon

Signed-off-by: liuchang0812 <liuchang0812@gmail.com>

Merge pull request #12663 from badone/wip-ceph_test_rados_api_c_write_operations-valgrind-fixes

c_write_operations.cc: Fix trivial memory leak

Reviewed-by: Kefu Chai <kchai@redhat.com>

test: replace hard-code binary names with varibles

This test case couldn't run under customize env CEPH_BIN.

Signed-off-by: liuchang0812 <liuchang0812@gmail.com>