Merge pull request #13742 from liupan1111/wip-cleanup-journal

os/filestore: use existing variable for same func.

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #12177 from kylinstorage/wip-remove-unneeded-loop

os/filestore/FileStore.cc: remove unneeded loop

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #13741 from rzarzynski/wip-bs-fastcrc32-in-rocks

os/bluestore: enable SSE-assisted CRC32 calculations in RocksDB

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #13768 from tchaikov/wip-clang-fixes

librados, osd: clang fixes

Reviewed-by: Brad Hubbard <bhubbard@redhat.com>

Merge pull request #13794 from liewegas/wip-clog-newlines

common: remove \n on clog messages

Reviewed-by: Brad Hubbard <bhubbard@redhat.com>

Merge pull request #13796 from liewegas/wip-debian-base-dbg

debian/control: add ceph-base-dbg

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #13799 from optimistyzy/36_1

bluestore, NVMEDEVICE: Specify the max io completion in conf

Reviewed-by: Haomai Wang <haomai@xsky.com>
Reviewed-by: Pan Liu <liupan1111@gmail.com>

Merge pull request #13806 from vshankar/rbd-internal-api-move

test/librbd: move tests using non-public api to internal

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
Reviewed-by: Mykola Golub <mgolub@mirantis.com>

Merge pull request #13755 from liewegas/wip-19131

osd/osd_internal_types: wake snaptrimmer on put_read lock, too

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #13411 from yehudasa/wip-vstart-rgw-fix

vstart: don't configure rgw_dns_name

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #13425 from dzafman/wip-15912

osd: various changes for preventing internal ENOSPC condition

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #13783 from cbodley/wip-18980

rgw: if user.email is empty, dont try to delete

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>

Merge pull request #13379 from jecluis/wip-mon-luminous-features

mon: enable luminous monmap feature on full quorum

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #13582 from guzhongyan/cleanup_execute_ctx

osd: correct the func name in execute_ctx() log messages

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #13699 from yuyuyu101/wipasync-name

msg/async: set thread name for msgr worker

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #13726 from dachary/wip-crushtool-json

crush: add --dump to crushtool

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #13730 from idryomov/wip-osd-rm-incremental

osd/OSDMap: don't set weight to IN when OSD is destroyed

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #13715 from wangzhengyong/master

os/bluestore: remove unused local variable "pos"

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #13728 from ifed01/wip-bluestore-fix-smallwrite

os/bluestore: fix a bug in small write handling on sharded extents

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #13736 from rzarzynski/wip-bs-bitmap-exact-type-in-leaf

os/bluestore: leverage the type knowledge in BitMapAreaLeaf.

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #13743 from rzarzynski/wip-bs-bitmap-zone-direct-vector

os/bluestore: remove unneeded indirection in BitMapZone.

Reviewed-by: Igor Fedotov <ifedotov@mirantis.com>
Reviewed-by: Sage Weil <sage@redhat.com>

test / librbd: move non-public api usage tests to internal

Signed-off-by: Venky Shankar <vshankar@redhat.com>

test / librbd: create_image_full_pp() / clone_image_pp() helpers

Signed-off-by: Venky Shankar <vshankar@redhat.com>

bluestore, NVMEDEVICE: Specify the max io completion in conf

I do not think that 4 is meaninful. If a value needs to be determined,
it should be configured in the configuration file.

Signed-off-by: optimistyzy <optimistyzy@gmail.com>

os/filestore: use existing variable to assign newsize.

Signed-off-by: Pan Liu <liupan1111@gmail.com>

Merge pull request #13228 from trociny/wip-qa-rbd_mirror-opts

qa/workunits/rbd: tweak rbd-mirror config to spead up testing

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

common: remove \n on clog messages

Confirmed these aren't needed, and are inconsistently included.
Remove them where they do appear.

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #13772 from Liuchang0812/wip-add-override-in-mgr-headers

mgr: add override in headers

Reviewed-by: Kefu Chai <kchai@redhat.com>

os/bluestore: enable SSE-assisted CRC32 calculations in RocksDB.

By default RocksDB extensively employs CRC32. It has two paths
for the checksum calculation:
 * rocksdb::crc32c::Slow_CRC32,
 * rocksdb::crc32c::Fast_CRC32.

The fast path depends on a run-time discovery of CPU capabilities
AND a compile-time define __SSE4_2__. Although my systems really
offer SSE4.2 support, the macro was undefined resulting in poor
RocksDB performance visible especially during WAL transactions.

The patch (awkwardly) adds the -msse4.2 to CXXFLAGS for RocksDB.

Signed-off-by: Radoslaw Zarzynski <rzarzynski@mirantis.com>

debian/control: add ceph-base-dbg

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #13765 from guihecheng/rgwfs-time-attrs

rgw_file: posix style atime,ctime,mtime

Merge pull request #13712 from linuxbox2/wip-dtor-fhcache

rgw_file: RGWFileHandle dtor must also cond-unlink from FHCache

Merge pull request #13645 from yehudasa/wip-rgw-support-ragweed

rgw: add apis to support ragweed
Reviewed-by: Orit Wasserman <owasserm@redhat.com>

Merge pull request #13413 from zhaochao/fix-mailmap-affiation

mailmap: fix Zhao Chao affiliation.

Reviewed-by: Abhishek Lekshmanan <abhishek@suse.com>

rgw: delete_system_obj() fails on empty object name

Signed-off-by: Casey Bodley <cbodley@redhat.com>

rgw: if user.email is empty, dont try to delete

Fixes: http://tracker.ceph.com/issues/18980
Signed-off-by: Casey Bodley <cbodley@redhat.com>

qa/suites/rados/thrash: add no-thrash item to matrix

Run all of our cases with no thrashing.  This would have uncovered
http://tracker.ceph.com/issues/19131 much sooner!

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #13767 from tchaikov/wip-vstart-no-multiple-mgr-x

vstart.sh: do not add host for mgr.* section if not $overwrite_conf

Reviewed-by: Sage Weil <sage@redhat.com>

rgw_file: RGWFileHandle dtor must also cond-unlink from FHCache

Formerly masked in part by the reclaim() action, direct-delete now
substitutes for reclaim() iff its LRU lane is over its high-water
mark, and in particular, like reclaim() the destructor is certain
to see handles still interned on the FHcache when nfs-ganesha is
recycling objects from its own LRU.

Fixes: http://tracker.ceph.com/issues/19112
Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>

mgr: add override in headers

Signed-off-by: liuchang0812 <liuchang0812@gmail.com>

osd/PGLog: always use ldpp_dout() to print with dpp

lderr(dpp->get_cct()) could crash if dpp is null.

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #13584 from theanalyst/doc/release-luminous

doc: releases update the luminous, hammer, jewel release dates

Reviewed-by: Loic Dachary <ldachary@redhat.com>
Reviewed-by: Nathan Cutler <ncutler@suse.com>

os/bluestore: "noid" is not always necessary in clone op

Signed-off-by: wangzhengyong@cmss.chinamobile.com

librados: rados_inconsistent_pg_list() check "buf" param for null before copying to it

Signed-off-by: Kefu Chai <kchai@redhat.com>

vstart.sh: do not add host for mgr.* section if not $overwrite_conf

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #13678 from liewegas/wip-readable

test/encoding: fix readable.sh bugs; fix ceph-object-corpus

Reviewed-by: Kefu Chai <kchai@redhat.com>

rgw_file: posix style atime,ctime,mtime

As an ganesha FSAL backend, rgw_file should properly maintain
the atime,ctime,mtime properly against operations such as:
(read,write) for file
(create,unlink,mkdir,rmdir,rename) for dir
(setattr) for file and dir

Signed-off-by: Gui Hecheng <guihecheng@cmss.chinamobile.com>

Merge pull request #13762 from Liuchang0812/wip-remove-unnecessary-code-in-mon

mon: remove unnecessary function declaration

Reviewed-by: Kefu Chai <kchai@redhat.com>

mon: remove unnecessary function declaration

Signed-off-by: liuchang0812 <liuchang0812@gmail.com>

Merge pull request #12566 from Liuchang0812/wip-rados

tools/rados: remove useless function declaration

Reviewed-by: Kefu Chai <kchai@redhat.com>

os/filestore/FileStore.cc: remove unneeded loop

Signed-off-by: Li Wang <li.wang@kylin-cloud.com>

Merge pull request #12421 from cernceph/wip-dvanders-split

os/filestore/HashIndex: be loud about splits

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #12627 from liupan1111/wip-fix-remove-when-full

rbd: When Ceph cluster becomes full, should allow user to remove rbd …

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #13289 from songbaisen/s13

rados:  we need to get the latest osdmap when pool does not exists

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>

os/bluestore: remove unneeded indirection in BitMapZone.

BitMapZone aggregates BmapEntries through a pointer to independently-
allocated std::vector which in turn has its own pointer to the real
data. This indirection is unnecessary as std::vector is pretty cheap
in the terms of memory overhead. For instance, on x86-64 & GCC 5.4.0
sizeof(std::vector) == 24, which translates to 3 raw pointers.

Stripping the indirection could have positive impact on processor's
caches and allow to minimize the number of memory accesses.

Signed-off-by: Radoslaw Zarzynski <rzarzynski@mirantis.com>

Merge pull request #13599 from kylinstorage/cleanup_bitmapallocator

os/bluestore/BitMapAllocator: rm unused variable

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #13744 from smithfarm/wip-19129

build/ops: add psmisc dependency to ceph-base (deb and rpm)

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #13750 from Saumay-Agrawal/master

mon: Update OSDMon.cc comments

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #13751 from Saumay-Agrawal/patch-1

doc: Update sample.ceph.conf

Reviewed-by: Sage Weil <sage@redhat.com>

osd/osd_internal_types: wake snaptrimmer on put_read lock, too

The snaptrimmer can block taking a write lock, which might happen due to
a conficting EC read.  When the EC read completes, we need to wake up the
snaptrimmer.

Fixes: http://tracker.ceph.com/issues/19131
Signed-off-by: Sage Weil <sage@redhat.com>

Update sample.ceph.conf

I have fixed the cleanup issue #19090: Wrong hard-coded URLs
Please add these changes to your documentation.

Update OSDMon.cc

I have fixed the issue cleanup #19090: Wrong hard-coded URLs
I have replaced the wrong URLs with correct URLs redirecting to the Tunables headline under Crush Map documentation.
Please add it to the documentation.

Merge pull request #13748 from tchaikov/wip-disable-dwz

rpm: disable dwz to speed up valgrind

Reviewed-by: Ken Dreyer <kdreyer@redhat.com>
Reviewed-by: Sage Weil <sage@redhat.com>

mon: test 'mon feature' cli

Signed-off-by: Joao Eduardo Luis <joao@suse.de>

qa/workunits/ceph-helpers: add wait_for_quorum()

Takes optional timeout and desired quorum size

Signed-off-by: Joao Eduardo Luis <joao@suse.de>

mon: enable persistent monmap features on full quorum

We will now only enable persistent features automatically
when ALL the monitors in the monmap are in the quorum.
 #noMonitorLeftBehind

Signed-off-by: Joao Eduardo Luis <joao@suse.de>

mon: better 'mon features' cli

Allows listing supported and currently set monmap features, as well as
setting and unsetting them.

Signed-off-by: Joao Eduardo Luis <joao@suse.de>

Merge pull request #13571 from trociny/wip-18784

rbd-mirror A/A: leader should track up/down rbd-mirror instances

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #13746 from tahia-khan/wip-doc-quick-ceph-deploy

doc: deletes duplicated word and clarifies an example

Reviewed-by: Abhishek Lekshmanan <abhishek@suse.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #13689 from Liuchang0812/wip-19107-cleanup-confutil

common: use ref to avoid unnecessary memory copy

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #13739 from Liuchang0812/fix-unittest-hostname

test: unittest_hostname compile error on freebsd

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Willem Jan Withagen <wjw@digiware.nl>

Merge pull request #13693 from Liuchang0812/wip-add-override-in-mon-headers

mon: add override in headers

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #13701 from dillaman/wip-19010

librbd: simplify image open/close semantics

Reviewed-by: Mykola Golub <mgolub@mirantis.com>

common: use ref to avoid unnecessary memory copy

Fixes: http://tracker.ceph.com/issues/19107
Signed-off-by: liuchang0812 <liuchang0812@gmail.com>

doc: deletes duplicated word and clarifies an example

Signed-off-by: Tahia Khan <tahia.khan@gmail.com>

Merge pull request #13679 from dillaman/wip-librbd-image-cache-clip

librbd: image-extent cache needs to clip out-of-bounds read buffers

Reviewed-by: Mykola Golub <mgolub@mirantis.com>

mon: add override in headers

Signed-off-by: liuchang0812 <liuchang0812@gmail.com>

build/ops: add psmisc dependency to ceph-base (deb and rpm)

Fixes: http://tracker.ceph.com/issues/19129
Signed-off-by: Nathan Cutler <ncutler@suse.com>

os/bluestore: switch to alloc_dbg_assert in BmapEntityListIter::next().

Signed-off-by: Radoslaw Zarzynski <rzarzynski@mirantis.com>

os/bluestore: inline for the sake of BitMapAreaLeaf::child_check_n_lock.

Signed-off-by: Radoslaw Zarzynski <rzarzynski@mirantis.com>

mailmap: fix Zhao Chao affiliation.

Signed-off-by: Zhao Chao <zhaochao1984@gmail.com>

Merge pull request #13427 from ly798/update-info

mailmap: Liu Yang affiliation

Reviewed-by: Abhishek Lekshmanan <abhishek@suse.com>
Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>

Merge pull request #13738 from yuyuyu101/wip-fix-ib

msg/async/rdma: fix potential racing connection usage

Reviewed-by: Adir lev <adirl@mellanox.com>

Merge pull request #13655 from Adirl/ulimit

msg/async/rdma: check ulimit

Reviewed-by: Haomai Wang <haomai@xsky.com>

msg/async/rdma: check ulimit

On RDMA, memlock should be unlimited. If it is not then
warn the user

issue: 973654

Change-Id: If1e07722aaca9df88714828ca3f7398a2504e262
Signed-off-by: Sarit Zubakov <saritz@mellanox.com>
Signed-off-by: Adir Lev <adirl@mellanox.com>

Merge pull request #13510 from yuyuyu101/wip-rdma-buffer-size

common/config_opts: improve rdma buffer size to 128k

test: unittest_hostname compile error on freebsd

Signed-off-by: liuchang0812 <liuchang0812@gmail.com>

Merge pull request #13713 from drunkard/master

doc: misc minor fixes

Reviewed-by: Kefu Chai <kchai@redhat.com>

rpm: disable dwz to speed up valgrind

Fixes: http://tracker.ceph.com/issues/19099
Signed-off-by: Kefu Chai <kchai@redhat.com>

msg/async/rdma: fix incorrect method name

Signed-off-by: Haomai Wang <haomai@xsky.com>

msg/async/rdma: keep qp_conns lock to prevent connection alive

Signed-off-by: Haomai Wang <haomai@xsky.com>

msg/async/rdma: remove unused variables

Signed-off-by: Haomai Wang <haomai@xsky.com>

Merge pull request #13525 from yuyuyu101/wip-infiniband-fork

msg/async/rdma: make Infiniband can be forkable

msg/async/rdma: restart Infiniband resources to handle fork properly

Signed-off-by: Haomai Wang <haomai@xsky.com>

Merge pull request #13666 from Adirl/chunk_size

msg/async/rdma: rename chunk_size to buffer_size

Reviewed-by: Haomai Wang <haomai@xsky.com>

doc: add missed commas

Signed-off-by: Drunkard Zhang <gongfan193@gmail.com>

doc: add mark for quoted content

Signed-off-by: Drunkard Zhang <gongfan193@gmail.com>

doc: command typo fix, s/radosgw_admin/radosgw-admin

Signed-off-by: Drunkard Zhang <gongfan193@gmail.com>

doc: fix aiming of command ping

Signed-off-by: Drunkard Zhang <gongfan193@gmail.com>

doc: hyperlink typo fix in dev/index

The doc compiler complains about this.

Signed-off-by: Drunkard Zhang <gongfan193@gmail.com>