librbd: relax "is parent mirrored" check when enabling mirroring for pool

If the parent is in the same pool and has the journaling feature enabled
we can assume the mirroring will eventually be enabled for it.

Fixes: http://tracker.ceph.com/issues/19130
Signed-off-by: Mykola Golub <mgolub@mirantis.com>

Merge pull request #13323 from yehudasa/wip-18079-2

librados: use cursor for nobjects listing

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #13262 from batrick/multimds-thrasher

Add multimds:thrash sub-suite and fix bugs in thrasher for multimds

Reviewed-by: John Spray <john.spray@redhat.com>

Merge pull request #13560 from wjwithagen/wip-wjw-ceph-disk-tests

ceph-disk/tests: Certain partition types do not work on FreeBSD

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #13810 from yuyuyu101/wip-rdma-inflight

msg/async/rdma: destroy QueuePair if needed

Reviewed-by: Adir lev <adirl@mellanox.com>

Merge pull request #13742 from liupan1111/wip-cleanup-journal

os/filestore: use existing variable for same func.

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #12177 from kylinstorage/wip-remove-unneeded-loop

os/filestore/FileStore.cc: remove unneeded loop

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #13741 from rzarzynski/wip-bs-fastcrc32-in-rocks

os/bluestore: enable SSE-assisted CRC32 calculations in RocksDB

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #13768 from tchaikov/wip-clang-fixes

librados, osd: clang fixes

Reviewed-by: Brad Hubbard <bhubbard@redhat.com>

Merge pull request #13794 from liewegas/wip-clog-newlines

common: remove \n on clog messages

Reviewed-by: Brad Hubbard <bhubbard@redhat.com>

Merge pull request #13796 from liewegas/wip-debian-base-dbg

debian/control: add ceph-base-dbg

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #13799 from optimistyzy/36_1

bluestore, NVMEDEVICE: Specify the max io completion in conf

Reviewed-by: Haomai Wang <haomai@xsky.com>
Reviewed-by: Pan Liu <liupan1111@gmail.com>

Merge pull request #13806 from vshankar/rbd-internal-api-move

test/librbd: move tests using non-public api to internal

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
Reviewed-by: Mykola Golub <mgolub@mirantis.com>

Merge pull request #13755 from liewegas/wip-19131

osd/osd_internal_types: wake snaptrimmer on put_read lock, too

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #13411 from yehudasa/wip-vstart-rgw-fix

vstart: don't configure rgw_dns_name

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #13425 from dzafman/wip-15912

osd: various changes for preventing internal ENOSPC condition

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #13783 from cbodley/wip-18980

rgw: if user.email is empty, dont try to delete

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>

Merge pull request #13379 from jecluis/wip-mon-luminous-features

mon: enable luminous monmap feature on full quorum

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #13582 from guzhongyan/cleanup_execute_ctx

osd: correct the func name in execute_ctx() log messages

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #13699 from yuyuyu101/wipasync-name

msg/async: set thread name for msgr worker

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #13726 from dachary/wip-crushtool-json

crush: add --dump to crushtool

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #13730 from idryomov/wip-osd-rm-incremental

osd/OSDMap: don't set weight to IN when OSD is destroyed

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #13715 from wangzhengyong/master

os/bluestore: remove unused local variable "pos"

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #13728 from ifed01/wip-bluestore-fix-smallwrite

os/bluestore: fix a bug in small write handling on sharded extents

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #13736 from rzarzynski/wip-bs-bitmap-exact-type-in-leaf

os/bluestore: leverage the type knowledge in BitMapAreaLeaf.

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #13743 from rzarzynski/wip-bs-bitmap-zone-direct-vector

os/bluestore: remove unneeded indirection in BitMapZone.

Reviewed-by: Igor Fedotov <ifedotov@mirantis.com>
Reviewed-by: Sage Weil <sage@redhat.com>

msg/async/rdma: destroy QueuePair if needed

Signed-off-by: Haomai Wang <haomai@xsky.com>

test / librbd: move non-public api usage tests to internal

Signed-off-by: Venky Shankar <vshankar@redhat.com>

test / librbd: create_image_full_pp() / clone_image_pp() helpers

Signed-off-by: Venky Shankar <vshankar@redhat.com>

bluestore, NVMEDEVICE: Specify the max io completion in conf

I do not think that 4 is meaninful. If a value needs to be determined,
it should be configured in the configuration file.

Signed-off-by: optimistyzy <optimistyzy@gmail.com>

os/filestore: use existing variable to assign newsize.

Signed-off-by: Pan Liu <liupan1111@gmail.com>

Merge pull request #13228 from trociny/wip-qa-rbd_mirror-opts

qa/workunits/rbd: tweak rbd-mirror config to spead up testing

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

common: remove \n on clog messages

Confirmed these aren't needed, and are inconsistently included.
Remove them where they do appear.

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #13772 from Liuchang0812/wip-add-override-in-mgr-headers

mgr: add override in headers

Reviewed-by: Kefu Chai <kchai@redhat.com>

os/bluestore: enable SSE-assisted CRC32 calculations in RocksDB.

By default RocksDB extensively employs CRC32. It has two paths
for the checksum calculation:
 * rocksdb::crc32c::Slow_CRC32,
 * rocksdb::crc32c::Fast_CRC32.

The fast path depends on a run-time discovery of CPU capabilities
AND a compile-time define __SSE4_2__. Although my systems really
offer SSE4.2 support, the macro was undefined resulting in poor
RocksDB performance visible especially during WAL transactions.

The patch (awkwardly) adds the -msse4.2 to CXXFLAGS for RocksDB.

Signed-off-by: Radoslaw Zarzynski <rzarzynski@mirantis.com>

debian/control: add ceph-base-dbg

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #13765 from guihecheng/rgwfs-time-attrs

rgw_file: posix style atime,ctime,mtime

Merge pull request #13712 from linuxbox2/wip-dtor-fhcache

rgw_file: RGWFileHandle dtor must also cond-unlink from FHCache

Merge pull request #13645 from yehudasa/wip-rgw-support-ragweed

rgw: add apis to support ragweed
Reviewed-by: Orit Wasserman <owasserm@redhat.com>

Merge pull request #13413 from zhaochao/fix-mailmap-affiation

mailmap: fix Zhao Chao affiliation.

Reviewed-by: Abhishek Lekshmanan <abhishek@suse.com>

rgw: delete_system_obj() fails on empty object name

Signed-off-by: Casey Bodley <cbodley@redhat.com>

rgw: if user.email is empty, dont try to delete

Fixes: http://tracker.ceph.com/issues/18980
Signed-off-by: Casey Bodley <cbodley@redhat.com>

qa/suites/rados/thrash: add no-thrash item to matrix

Run all of our cases with no thrashing.  This would have uncovered
http://tracker.ceph.com/issues/19131 much sooner!

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #13767 from tchaikov/wip-vstart-no-multiple-mgr-x

vstart.sh: do not add host for mgr.* section if not $overwrite_conf

Reviewed-by: Sage Weil <sage@redhat.com>

rgw_file: RGWFileHandle dtor must also cond-unlink from FHCache

Formerly masked in part by the reclaim() action, direct-delete now
substitutes for reclaim() iff its LRU lane is over its high-water
mark, and in particular, like reclaim() the destructor is certain
to see handles still interned on the FHcache when nfs-ganesha is
recycling objects from its own LRU.

Fixes: http://tracker.ceph.com/issues/19112
Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>

mgr: add override in headers

Signed-off-by: liuchang0812 <liuchang0812@gmail.com>

osd/PGLog: always use ldpp_dout() to print with dpp

lderr(dpp->get_cct()) could crash if dpp is null.

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #13584 from theanalyst/doc/release-luminous

doc: releases update the luminous, hammer, jewel release dates

Reviewed-by: Loic Dachary <ldachary@redhat.com>
Reviewed-by: Nathan Cutler <ncutler@suse.com>

os/bluestore: "noid" is not always necessary in clone op

Signed-off-by: wangzhengyong@cmss.chinamobile.com

librados: rados_inconsistent_pg_list() check "buf" param for null before copying to it

Signed-off-by: Kefu Chai <kchai@redhat.com>

vstart.sh: do not add host for mgr.* section if not $overwrite_conf

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #13678 from liewegas/wip-readable

test/encoding: fix readable.sh bugs; fix ceph-object-corpus

Reviewed-by: Kefu Chai <kchai@redhat.com>

rgw_file: posix style atime,ctime,mtime

As an ganesha FSAL backend, rgw_file should properly maintain
the atime,ctime,mtime properly against operations such as:
(read,write) for file
(create,unlink,mkdir,rmdir,rename) for dir
(setattr) for file and dir

Signed-off-by: Gui Hecheng <guihecheng@cmss.chinamobile.com>

Merge pull request #13762 from Liuchang0812/wip-remove-unnecessary-code-in-mon

mon: remove unnecessary function declaration

Reviewed-by: Kefu Chai <kchai@redhat.com>

mon: remove unnecessary function declaration

Signed-off-by: liuchang0812 <liuchang0812@gmail.com>

Merge pull request #12566 from Liuchang0812/wip-rados

tools/rados: remove useless function declaration

Reviewed-by: Kefu Chai <kchai@redhat.com>

os/filestore/FileStore.cc: remove unneeded loop

Signed-off-by: Li Wang <li.wang@kylin-cloud.com>

Merge pull request #12421 from cernceph/wip-dvanders-split

os/filestore/HashIndex: be loud about splits

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #12627 from liupan1111/wip-fix-remove-when-full

rbd: When Ceph cluster becomes full, should allow user to remove rbd …

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #13289 from songbaisen/s13

rados:  we need to get the latest osdmap when pool does not exists

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>

os/bluestore: remove unneeded indirection in BitMapZone.

BitMapZone aggregates BmapEntries through a pointer to independently-
allocated std::vector which in turn has its own pointer to the real
data. This indirection is unnecessary as std::vector is pretty cheap
in the terms of memory overhead. For instance, on x86-64 & GCC 5.4.0
sizeof(std::vector) == 24, which translates to 3 raw pointers.

Stripping the indirection could have positive impact on processor's
caches and allow to minimize the number of memory accesses.

Signed-off-by: Radoslaw Zarzynski <rzarzynski@mirantis.com>

Merge pull request #13599 from kylinstorage/cleanup_bitmapallocator

os/bluestore/BitMapAllocator: rm unused variable

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #13744 from smithfarm/wip-19129

build/ops: add psmisc dependency to ceph-base (deb and rpm)

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #13750 from Saumay-Agrawal/master

mon: Update OSDMon.cc comments

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #13751 from Saumay-Agrawal/patch-1

doc: Update sample.ceph.conf

Reviewed-by: Sage Weil <sage@redhat.com>

osd/osd_internal_types: wake snaptrimmer on put_read lock, too

The snaptrimmer can block taking a write lock, which might happen due to
a conficting EC read.  When the EC read completes, we need to wake up the
snaptrimmer.

Fixes: http://tracker.ceph.com/issues/19131
Signed-off-by: Sage Weil <sage@redhat.com>

Update sample.ceph.conf

I have fixed the cleanup issue #19090: Wrong hard-coded URLs
Please add these changes to your documentation.

Update OSDMon.cc

I have fixed the issue cleanup #19090: Wrong hard-coded URLs
I have replaced the wrong URLs with correct URLs redirecting to the Tunables headline under Crush Map documentation.
Please add it to the documentation.

Merge pull request #13748 from tchaikov/wip-disable-dwz

rpm: disable dwz to speed up valgrind

Reviewed-by: Ken Dreyer <kdreyer@redhat.com>
Reviewed-by: Sage Weil <sage@redhat.com>

mon: test 'mon feature' cli

Signed-off-by: Joao Eduardo Luis <joao@suse.de>

qa/workunits/ceph-helpers: add wait_for_quorum()

Takes optional timeout and desired quorum size

Signed-off-by: Joao Eduardo Luis <joao@suse.de>

mon: enable persistent monmap features on full quorum

We will now only enable persistent features automatically
when ALL the monitors in the monmap are in the quorum.
 #noMonitorLeftBehind

Signed-off-by: Joao Eduardo Luis <joao@suse.de>

mon: better 'mon features' cli

Allows listing supported and currently set monmap features, as well as
setting and unsetting them.

Signed-off-by: Joao Eduardo Luis <joao@suse.de>

Merge pull request #13571 from trociny/wip-18784

rbd-mirror A/A: leader should track up/down rbd-mirror instances

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #13746 from tahia-khan/wip-doc-quick-ceph-deploy

doc: deletes duplicated word and clarifies an example

Reviewed-by: Abhishek Lekshmanan <abhishek@suse.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #13689 from Liuchang0812/wip-19107-cleanup-confutil

common: use ref to avoid unnecessary memory copy

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #13739 from Liuchang0812/fix-unittest-hostname

test: unittest_hostname compile error on freebsd

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Willem Jan Withagen <wjw@digiware.nl>

Merge pull request #13693 from Liuchang0812/wip-add-override-in-mon-headers

mon: add override in headers

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #13701 from dillaman/wip-19010

librbd: simplify image open/close semantics

Reviewed-by: Mykola Golub <mgolub@mirantis.com>

common: use ref to avoid unnecessary memory copy

Fixes: http://tracker.ceph.com/issues/19107
Signed-off-by: liuchang0812 <liuchang0812@gmail.com>

doc: deletes duplicated word and clarifies an example

Signed-off-by: Tahia Khan <tahia.khan@gmail.com>

Merge pull request #13679 from dillaman/wip-librbd-image-cache-clip

librbd: image-extent cache needs to clip out-of-bounds read buffers

Reviewed-by: Mykola Golub <mgolub@mirantis.com>

mon: add override in headers

Signed-off-by: liuchang0812 <liuchang0812@gmail.com>

build/ops: add psmisc dependency to ceph-base (deb and rpm)

Fixes: http://tracker.ceph.com/issues/19129
Signed-off-by: Nathan Cutler <ncutler@suse.com>

os/bluestore: switch to alloc_dbg_assert in BmapEntityListIter::next().

Signed-off-by: Radoslaw Zarzynski <rzarzynski@mirantis.com>

os/bluestore: inline for the sake of BitMapAreaLeaf::child_check_n_lock.

Signed-off-by: Radoslaw Zarzynski <rzarzynski@mirantis.com>

mailmap: fix Zhao Chao affiliation.

Signed-off-by: Zhao Chao <zhaochao1984@gmail.com>

Merge pull request #13427 from ly798/update-info

mailmap: Liu Yang affiliation

Reviewed-by: Abhishek Lekshmanan <abhishek@suse.com>
Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>

Merge pull request #13738 from yuyuyu101/wip-fix-ib

msg/async/rdma: fix potential racing connection usage

Reviewed-by: Adir lev <adirl@mellanox.com>

Merge pull request #13655 from Adirl/ulimit

msg/async/rdma: check ulimit

Reviewed-by: Haomai Wang <haomai@xsky.com>

msg/async/rdma: check ulimit

On RDMA, memlock should be unlimited. If it is not then
warn the user

issue: 973654

Change-Id: If1e07722aaca9df88714828ca3f7398a2504e262
Signed-off-by: Sarit Zubakov <saritz@mellanox.com>
Signed-off-by: Adir Lev <adirl@mellanox.com>

Merge pull request #13510 from yuyuyu101/wip-rdma-buffer-size

common/config_opts: improve rdma buffer size to 128k

test: unittest_hostname compile error on freebsd

Signed-off-by: liuchang0812 <liuchang0812@gmail.com>

Merge pull request #13713 from drunkard/master

doc: misc minor fixes

Reviewed-by: Kefu Chai <kchai@redhat.com>

rpm: disable dwz to speed up valgrind

Fixes: http://tracker.ceph.com/issues/19099
Signed-off-by: Kefu Chai <kchai@redhat.com>

msg/async/rdma: fix incorrect method name

Signed-off-by: Haomai Wang <haomai@xsky.com>

msg/async/rdma: keep qp_conns lock to prevent connection alive

Signed-off-by: Haomai Wang <haomai@xsky.com>

msg/async/rdma: remove unused variables

Signed-off-by: Haomai Wang <haomai@xsky.com>

Merge pull request #13525 from yuyuyu101/wip-infiniband-fork

msg/async/rdma: make Infiniband can be forkable

msg/async/rdma: restart Infiniband resources to handle fork properly

Signed-off-by: Haomai Wang <haomai@xsky.com>