RDMA: Fix compilation warning

Infiniband.cc: In member function ‘int Infiniband::recv_msg(CephContext*, int, IBSYNMsg&)’:
Infiniband.cc:883:86: warning: format ‘%x’ expects argument of type ‘unsigned int*’, but argument 3 has type ‘uint16_t* {aka short unsigned int*}’ [-Wformat=]
     sscanf(msg, "%x:%x:%x:%x:%s", &(im.lid), &(im.qpn), &(im.psn),
     &(im.peer_qpn),gid);

Signed-off-by: Amir Vadai <amir@vadai.me>

RDMA: Move defenitions from RDMAStack.h into .cc

Signed-off-by: Amir Vadai <amir@vadai.me>

RDMA: Move defenitions from Infiniband.h into .cc

Signed-off-by: Amir Vadai <amir@vadai.me>

Merge pull request #12706 from Liuchang0812/wip-qa-rados-1228

test: update test_rados_tool.sh, use POOL and OBJ var

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #12716 from kylinstorage/wip-ctypos_h

misc: fix code typos in header files

Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>

Merge pull request #12721 from chendave/wip-authx-options

auth: Let's not use the deprecated cephx option

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #12676 from chendave/master

common/ceph_context: Show clear message if all features are enabled

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #12633 from liewegas/wip-15784

msg/simple: call clear_pipe in wait() shutdown path

Reviewed-by: Haomai Wang <haomai@xsky.com>

Merge pull request #12692 from XinzeChi/wip-bluestore-statfs

os/bluestore: _txc_release_alloc when do wal cleaning

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #12691 from runsisi/wip-dump-osdop

osd/OpRequest: dump both name and addr for the client op

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #12363 from wjwithagen/wip-wjw-freebsd-strerror

common/module.c: do not use strerror_r the GNU way.

Reviewed-by: Sage Weil <sage@redhat.com>

cephx: Let's not use the deprecated option

Enable cephx by setting the option of "auth supported" has been
deprecated for a long time, it's better to use the recommended
options instead.

Signed-off-by: Dave Chen <wei.d.chen@intel.com>

Merge pull request #12718 from kylinstorage/wip-orig

organizationmap: add Xianxia Xiao to Kylin Cloud team

organizationmap: add Xianxia Xiao to Kylin Cloud team

Signed-off-by: Yunchuan Wen <yunchuan.wen@kylin-cloud.com>

src/tools/rbd_mirror/Replayer.h : fix typos

Signed-off-by: Xianxia Xiao <xianxia.xiao@kylin-cloud.com>

src/messages/MMDSSlaveRequest.h : fix typos

Signed-off-by: Xianxia Xiao <xianxia.xiao@kylin-cloud.com>

src/tools/rbd_recover_tool/epoch_h : fix typos

Signed-off-by: Xianxia Xiao <xianxia.xiao@kylin-cloud.com>

src/librados/IoCtxImpl.h : fix typos

Signed-off-by: Xianxia Xiao <xianxia.xiao@kylin-cloud.com>

src/journal/JournalTrimmer.h : fix typos

Signed-off-by: Xianxia Xiao <xianxia.xiao@kylin-cloud.com>

src/common/hobject.h : fix typos

Signed-off-by: Xianxia Xiao <xianxia.xiao@kylin-cloud.com>

Merge pull request #12652 from badone/wip-coverity-TracepointProvider

tracing: Fix handle leak in TracepointProvider

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

tracing: Fix handle leak in TracepointProvider

Silences Coverity 1397733

Signed-off-by: Brad Hubbard <bhubbard@redhat.com>

Merge pull request #12711 from yuyuyu101/wip-fix-rdma

msg/async/rdma: fix ceph_clock_now calls

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #12451 from liewegas/wip-osd-on-applied

osd: never send rados ack (only commit)

Reviewed-by: Samuel Just <sjust@redhat.com>

Merge pull request #12480 from dx9/wip-boost-python

cmake: fix hard coded boost python lib

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #12406 from songbaisen/s5

tool: add some ceph relate processes to ps-ceph.pl

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #12621 from fiskn/master

doc: Update disk thread section to reflect that scrubbing is no longe…

test: update test_rados_tool.sh, use POOL and OBJ var

* replace hard-code pool name with $POOL
* replace hard-code object name with $OBJ
* introduce a new variable called $POOL_EC
* clean up pool
* simplify test case

Signed-off-by: liuchang0812 <liuchang0812@gmail.com>

msg/async/rdma: fix ceph_clock_now calls

Signed-off-by: Haomai Wang <haomai@xsky.com>

Merge pull request #12403 from liewegas/wip-mon-mgr-cleanups

mon: clean up some osdmon/pgmon interactions

Reviewed-by: Kefu Chai <kchai@redhat.com>

qa/workunits: remove crash_replay_interval tests

Signed-off-by: Sage Weil <sage@redhat.com>

doc: remove crash_replay_interval mentions

Signed-off-by: Sage Weil <sage@redhat.com>

mon/MDSMonitor: no need to set crash_replay_interval on data pools

Signed-off-by: Sage Weil <sage@redhat.com>

osd/PG: remove waiting_for_ack

No longer used.

Signed-off-by: Sage Weil <sage@redhat.com>

osd: simplify dup/resent logic

No need to special ack path.

Signed-off-by: Sage Weil <sage@redhat.com>

osd: remove replay_queue

Signed-off-by: Sage Weil <sage@redhat.com>

osd/PG: do not go into replay state

Signed-off-by: Sage Weil <sage@redhat.com>

osd/PG: do not queue ops for replay

We no longer send ACKs, so this can't happen anyway.

Signed-off-by: Sage Weil <sage@redhat.com>

osd: we can discard "replay" ops

Signed-off-by: Sage Weil <sage@redhat.com>

osd/ReplicatedPG: do not send ACK separate from COMMIT

This never comes before COMMIT, making it useless to clients.  And it's
just unnecessary overhead.

Signed-off-by: Sage Weil <sage@redhat.com>

osd: remove 'rlat' read latency metrics

These are measuring how long of a delay we see before ops are readable.
They are calculated in the on_applied path, which we'd like to remove,
and weren't reliably calculated anyway.

Signed-off-by: Sage Weil <sage@redhat.com>

osd/ReplicatedPG: kill hit_set_flushing

This was added before we started taking the ondisk lock in order to allow
reading a just-written hitset.  Since this is a rare path anyway, just
take the ondisk lock.

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #12522 from hjwsm1989/ecwrite

osd/ECBackend: Send write message to peers first, then do local write

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #12524 from yuyuyu101/wip-bluestore-flush

os/bluestore: prevent lock for almost "flush" calls

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #12528 from kylinstorage/wip-JournalingObjectStore-cleanup

os/filestore/JournalingObjectStore cleanup

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #12670 from liewegas/wip-nits

mon: fix a few nits

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #12684 from xiexingguo/xxg-wip-fix-onodesizetracking

os/bluestore: fix OnodeSizeTracking testing

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #12707 from tchaikov/wip-dpdk-v16.07

builds with dpdk v16.07

Reviewed-by: Haomai Wang <haomai@xsky.com>

msg/async: bump dpdk to v16.07

as some distros include it already. so sync it up to avoid building
failures on them with dpdk-dev package(s) shipped by distro.

Signed-off-by: Kefu Chai <kchai@redhat.com>

msg/async: be compatible with dpdk v16.07

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #12698 from yuyuyu101/wip-dpdk-fix

msg/async/dpdk: fix compile errors

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #12699 from wjwithagen/patch-15

qa/test_rados_tool.sh: POSIX dd only accepts 'k' as multiplier

Reviewed-by: Kefu Chai <kchai@redhat.com>

osd/OpRequest: dump both name and addr for the client op

Signed-off-by: runsisi <runsisi@zte.com.cn>

mon/OSDMonitor: remove workaround for dumpling-era bug

We will definitely have published (and stored) a new osdmap since then.

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #12703 from kylinstorage/wip-remove-unused-variable

os/bluestore/BlueStore.cc: remove unused variable

Reviewed-by: Igor Fedotov <ifedotov@mirantis.com>

Merge pull request #12594 from wjwithagen/wip-wjw-freebsd-rebind-timing

common/config_opts.h: FreeBSD timing changed due to no SO_REUSEADDR

Reviewed-by: Sage Weil <sage@redhat.com>

os/bluestore/BlueStore.cc: remove unused variable

Signed-off-by: Li Wang <li.wang@kylin-cloud.com>

qa/test_rados_tool.sh: POSIX dd only accepts 'k' as multiplier

FreeBSD does not want KB as 1k multiplier.
see also: http://www.unix.com/man-page/posix/1posix/dd/

Signed-off-by: Willem Jan Withagen <wjw@digiware.nl>

test_userspace_event: fix missing std

Signed-off-by: Haomai Wang <haomai@xsky.com>

msg/dpdk/TCP: remove unneeded CephContext argument

Signed-off-by: Haomai Wang <haomai@xsky.com>

os/bluestore: _txc_release_alloc when do wal cleaning

Signed-off-by: Xinze Chi <xinze@xsky.com>

Merge pull request #12688 from Adirl/clean_line_end

msg/async/rdma: clean line endings

Reviewed-by: Haomai Wang <haomai@xsky.com>

os/bluestore: fix OnodeSizeTracking testing

[  FAILED  ] 1 test, listed below:
[  FAILED  ] ObjectStore/StoreTest.OnodeSizeTracking/2, where GetParam() = "bluestore"

 1 FAILED TEST

The above test failure happens as the bluestore mount() process
will try to load all collections and put them into the coll_map,
which will be also tracked as mempool::bluestore_meta_other.
So total_bytes from mempool won't be equal to zero.

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

common/ceph_context: Show clear message if all features are enabled

It's saying the following dangerous and experimental featuresare enabled
while the value is "*", it's not quite clear what's "*" stands for.

The patch updates the message so that it's more clear when "*" is used
in place.

Signed-off-by: Dave Chen <wei.d.chen@intel.com>

Merge pull request #12336 from kylinstorage/wip-cleanup-use-string-ref

osd: cleanup: use string & to avoid unnecessary copy

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #12569 from liewegas/wip-bluestore-raw-unused

os/bluestore/bluestore_types: drop std::bitset for blob unused

Reviewed-by: Igor Fedotov <ifedotov@mirantis.com>
Reviewed-by: Varada Kari <varada.kari@sandisk.com>

Merge pull request #12612 from fangyuxiangGL/master

osd/ReplicatedBackend: do not set omap header if it is empty

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Haomai Wang <haomai@xsky.com>

Merge pull request #12668 from hjwsm1989/pg-unfound

osd/PG: add new have_unfound() function in MissingLoc

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #12674 from Liuchang0812/tools-rados-offset-support

tools/rados: add a parameter "--offset" to rados put command

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #12675 from Liuchang0812/fix-test-case-in-rados

test: replace hard-code binary names with varibles

Reviewed-by: Kefu Chai <kchai@redhat.com>

mon/OSDMonitor: ceph osd pool get/set crush_rule by name

This means users don't have to manually translate a rule
they just created to a ruleset ID in order to map a pool
to it.

Signed-off-by: Sage Weil <sage@redhat.com>

mon/PGMap: remove CATEGORY column from 'df detail'

Signed-off-by: Sage Weil <sage@redhat.com>

mon/PGMap: fix alignment of 'cache io' in ceph -s output

Currently it looks like

  client io 3508 B/s wr, 0 op/s rd, 5 op/s wr
  cache io 0 op/s promote

but should be

  client io 3508 B/s wr, 0 op/s rd, 5 op/s wr
   cache io 0 op/s promote

Signed-off-by: Sage Weil <sage@redhat.com>

mon: 'fs set-default' not 'fs set_default'

The convention is to use - for command names and
_ for variable names.

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #12655 from ceph/wip-fix-fio

test/fio_ceph_objectstore: fix fio plugin build failure caused by rec…

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #12682 from renhwztetecs/renhw-wip-mon-osdmonitor-sortbitwise

mon/OSDMonitor: fixup sortbitwise flag warning

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #12654 from yuyuyu101/wip-fix-async-subop

messages/MOSDSubOp: Make encode_payload can be reentrant

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #12662 from Liuchang0812/doc-fix

doc: fix document about rados mon

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #12198 from tchaikov/wip-pybind-freebsd

pybind: conditional compile the linux specific constant

Reviewed-by: Willem Jan Withagen <wjw@digiware.nl>

mon/OSDMonitor: fixup sortbitwise flag warning

"ceph -s" does not report warning when using
command "ceph osd unset sortbitwise" to drop
sortbitwise flag.
we should use "osdmap.get_up_osd_features() &
CEPH_FEATURE_OSD_BITWISE_HOBJ_SORT"
instead of "(osdmap.get_features(CEPH_ENTITY_TYPE_OSD, NULL) &
CEPH_FEATURE_OSD_BITWISE_HOBJ_SORT)",
because osdmap.get_features only get local "features"

Signed-off-by: huanwen ren <ren.huanwen@zte.com.cn>

ReplicatedBackend: do not set omap header if it is empty

In replicatedpg backfill, objects have no omap header or keys previously, but an empty omap header will be inserted when they are pushed to replica, which leads leveldb performance degrade seriously when cluster has huge number of objects.
so my opinion is that we does't insert omap header if is size is zero when handle PushOp in recovery.

Signed-off-by: fang yuxiang <fang.yuxiang@eisoo.com>

tools/rados: add parameter offset for put cmd and test case for it

add a parameter "--offset" for put command
add a test case for rados put command

Signed-off-by: liuchang0812 <liuchang0812@gmail.com>

doc: fix document about rados mon

Signed-off-by: liuchang0812 <liuchang0812@gmail.com>

Merge pull request #12663 from badone/wip-ceph_test_rados_api_c_write_operations-valgrind-fixes

c_write_operations.cc: Fix trivial memory leak

Reviewed-by: Kefu Chai <kchai@redhat.com>

test: replace hard-code binary names with varibles

This test case couldn't run under customize env CEPH_BIN.

Signed-off-by: liuchang0812 <liuchang0812@gmail.com>

Merge pull request #12672 from liupan1111/wip-fix-nvme-driver-compile-issue

bluestore: fixed compilation error when enable spdk

Reviewed-by: Haomai Wang <haomai@xsky.com>

bluestore: fixed compilation error when enable spdk

Signed-off-by: Pan Liu <pan.liu@istuary.com>

msg/async/rdma: clean line endings

Change-Id: I59359ae585b66018fe54d5e841a6506c077c606f
Signed-off-by: Adir Lev <adirl@mellanox.com>

Merge pull request #12659 from kylinstorage/wip-remove-obsolete-configuration-option

common/config_opts.h: remove obsolete configuration option

Reviewed-by: Sage Weil <sage@redhat.com>

  osd/PG: add new have_unfound() function in MissingLoc
          No need to lookup all objects in needs_recovery_map if only want to
          know whether if there are unfound objects.

Signed-off-by: huangjun <hjwsm1989@gmail.com>

Merge pull request #12648 from Adirl/fix_RoCEv2_deafult_value

msg/async/rdma: fix RoCE v2 deafult value

Reviewed-by: Haomai Wang <haomai@xsky.com>

c_write_operations.cc: Fix trivial memory leak

Signed-off-by: Brad Hubbard <bhubbard@redhat.com>

Merge pull request #12657 from badone/wip-c_read_operations-valgrind-fixes

test/librados/c_read_operations.cc: Fix valgrind errors

Reviewed-by: liuchang0812 <liuchang0812@gmail.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #12660 from liupan1111/wip-fix-dpdk-compile

async: Fixed compilation error when enable -DWITH_DPDK

Reviewed-by: Haomai Wang <haomai@xsky.com>

async: Fixed compilation error when enable -DWITH_DPDK

Signed-off-by: Pan Liu <pan.liu@istuary.com>

test/librados/c_read_operations.cc: Fix valgrind errors

Fixes: http://tracker.ceph.com/issues/18354
Signed-off-by: Brad Hubbard <bhubbard@redhat.com>

common/config_opts.h: remove obsolete configuration option

Signed-off-by: Li Wang <li.wang@kylin-cloud.com>

Merge pull request #12474 from cbodley/wip-rgw-test-versioned

test/rgw: add test for versioned object sync
Reviewed-by: Orit wasserman <owasserm@redhat.com>

Merge pull request #12597 from badone/wip-osdmap-nocopy

osd/OSDMap: Uncomment code to enable private default constructors

Reviewed-by: Sage Weil <sage@redhat.com>

test/fio_ceph_objectstore: fix fio plugin build failure caused by recent g_ceph_context removal

Signed-off-by: Igor Fedotov <ifedotov@mirantis.com>