Merge pull request #12681 from xiexingguo/xxg-wip-bluestore-1227

os/bluestore: generate same onode extent-shard keys in a more efficient way

Reviewed-by: Igor Fedotov <ifedotov@mirantis.com>

Merge pull request #12696 from xiexingguo/xxg-wip-improve-loops

os/bluestore: miscellaneous fixes to BitAllocator

Reviewed-by: Ramesh Chander <Ramesh.Chander@sandisk.com>

Merge pull request #12719 from xiexingguo/xxg-wip-bluestore-1229

os/bluestore/BitmapFreelistManager: readability improvements

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #12942 from Liuchang0812/cleanup-bluestore

os/bluestore: avoid unnecessary memory copy, use variable reference in BlockDevice::Open

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Igor Fedotov <ifedotov@mirantis.com>

Merge pull request #12946 from ifed01/wip-bluestore-pextent-vector-mempooled

os/bluestore: Add bluestore pextent vector to mempool

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #12964 from jcsp/wip-18574

qa: update remaining ceph.com to download.ceph.com

Merge pull request #12962 from dillaman/wip-rbd-boost-crash

rbd: remove direct linking to static boost libraries

Reviewed-by: Kefu Chai <kchai@redhat.com>

qa: update remaining ceph.com to download.ceph.com

Fixes: http://tracker.ceph.com/issues/18574
Signed-off-by: John Spray <john.spray@redhat.com>

rbd: remove direct linking to static boost libraries

The boost libraries are included in libceph-common. If linked
directly, boost::regex will abort during shutdown since it uses
a global static variable and it will be destructed twice.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #12885 from rjfd/wip-18325

librbd: allow to open an image without opening the parent image

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #12922 from trociny/wip-managed_lock_refactoring

librbd: managed lock refactoring

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #12953 from dillaman/wip-18542

qa/tasks/qemu: update default image url after ceph.com redesign

Reviewed-by: Mykola Golub <mgolub@mirantis.com>

Merge pull request #12954 from dillaman/wip-rbd-coverity

rbd: correct coverity warnings

Reviewed-by: Mykola Golub <mgolub@mirantis.com>

Merge pull request #12467 from bassam/pr-external-rocksdb

cmake: support for external rocksdb

Reviewed-by: Kefu Chai <kchai@redhat.com>

rbd: correct coverity warnings

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #9028 from tchaikov/wip-bt-dump

common/BackTrace: add operator<<

Reviewed-by: Brad Hubbard <bhubbard@redhat.com>

qa/tasks/qemu: update default image url after ceph.com redesign

Fixes: http://tracker.ceph.com/issues/18542
Signed-off-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #12807 from runsisi/wip-fix-pg-ls

mon/PGMonitor: fix description for ceph pg ls

Reviewed-by: Kefu Chai <kchai@redhat.com>

mon/PGMonitor: ceph pg ls-by-pool only supports string type of pool arg

Signed-off-by: runsisi <runsisi@zte.com.cn>

Merge pull request #12928 from tmuthamizhan/wip-ansible-fs-suite

qa/suites/ceph-ansible: removing fs workunit

Reviewed-by: Yuri Weinstein <yweins@redhat.com>

librbd: helper methods to query and break lock

Signed-off-by: Mykola Golub <mgolub@mirantis.com>

librbd: managed_lock: make AcquireRequest use GetLockRequest and BreakRequest

Initially this was implemented for exclusive_lock (03533b9,
23f60fe) but was lost when merging managed_lock refactoring.

Signed-off-by: Mykola Golub <mgolub@mirantis.com>

Merge pull request #12915 from jcsp/wip-18361

client: populate metadata during mount

Reviewed-by: Yan, Zheng <zyan@redhat.com>

Merge pull request #12107 from mslovy/wip-monmsg-minwait

mon: no delay for single message MSG_ALIVE and MSG_PGTEMP

Reviewed-by: Gregory Farnum <gfarnum@redhat.com>
Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #11955 from batrick/log-2-to-1-write

log: use one write system call per message

Reviewed-by: Gregory Farnum <gfarnum@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #9924 from Yan-waller/yj-wiph-commonentitynamecc

common/config: fix return type of string::find and use string::npos

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #12925 from liewegas/wip-crypto-shutdown-fixes

common: misc fixes detected by crypto shutdown assert

Reviewed-by: Brad Hubbard <bhubbard@redhat.com>

Merge pull request #12944 from liewegas/wip-centos-latest

qa/distros: make centos_latest 7.3

Reviewed-by: Yuri Weinstein <yweins@redhat.com>

Merge pull request #12841 from cbodley/wip-18407

rgw multisite: fix ref counting of completions

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>

Merge pull request #12605 from cbodley/wip-18300

rgw: RGWMetaSyncShardCR drops stack refs on destruction

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>

librbd: allow to open an image without opening parent image

Fixes: http://tracker.ceph.com/issues/18325
Signed-off-by: Ricardo Dias <rdias@suse.com>

cmake: support for external rocksdb

add support for building with an external rocksdb.

Signed-off-by: Bassam Tabbara <bassam.tabbara@quantum.com>

Merge pull request #12798 from wjwithagen/wip-wjw-krbd-optional

rbd: enable rbd on FreeBSD (without KRBD)

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Jason Dillaman <dillaman@redhat.com>

os/bluestore: add pextent vector to mempool

Signed-off-by: Igor Fedotov <ifedotov@mirantis.com>

test/objectstore: add bufferptr & blouestore_blob_t::extents to "sizeof" UT

Signed-off-by: Igor Fedotvo <ifedotov@mirantis.com>

Merge pull request #12930 from chendave/wip_cephx_refactor

msg: Revert the change from assert(0)-> ceph_abort() where is not applicable

Reviewed-by: Sage Weil <sage@redhat.com>

qa/distros: make centos_latest 7.3

This will fix smithi runs and break vps runs until we get a 7.3 cloud-init
image ready.

Signed-off-by: Sage Weil <sage@redhat.com>

os/bluestore: avoid unnecessary memory copy, use variable reference in BlockDevice::Open

Signed-off-by: liuchang0812 <liuchang0812@gmail.com>

Merge pull request #12939 from johnlinp/fix-nspr-include

cmake: search for nspr include files for both suffixes: nspr4 and nspr

Reviewed-by: Kefu Chai <kchai@redhat.com>

cmake: search for nspr include files for both suffixes: nspr4 and nspr

Fixes: http://tracker.ceph.com/issues/18535
Signed-off-by: John Lin <johnlinp@gmail.com>

Merge pull request #12936 from badone/wip-pg-states-doc-fix

doc: min_size advice is not helpful

Reviewed-by: Kefu Chai <kchai@redhat.com>

os/bluestore: generate same onode extent-shard keys in a more efficient way

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

doc: min_size advice is not helpful

When we are in the incomplete state adjusting min_size will not help with recovery.

Signed-off-by: Brad Hubbard <bhubbard@redhat.com>

Merge pull request #12247 from YankunLi/fix_bf_append_parameter

bufferlist: cleanup semantical wrong for bufferlist::append

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #12345 from rook/wip-embedded-add-rgw-mds

cmake: add RGW and MDS to libcephd

Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

common/BackTrace: add operator<<

replace BackTrace::print() with the operator<< where the former is used.

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #12784 from LiumxNL/fix-wrongly-delete-routed-op

mon: fix wrongly delete routed pgstats op

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #12651 from liewegas/wip-mempool-string

os/bluestore: put strings in mempool

Reviewed-by: Igor Fedotov <ifedotov@mirantis.com>

Merge pull request #12661 from xiexingguo/xxg-wip-cleanup-bitalloc

os/bluestore: cleanup BitAllocator

Reviewed-by: Ramesh Chander <Ramesh.Chander@sandisk.com>

Merge pull request #12700 from ifed01/wip-bluestore-collection-everywhere

os/bluestore: various onode changes to reduce its in-memory footprint.

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #12876 from xiexingguo/xxg-wip-be-for-cache

os/bluestore: use Best-Effort policy when evicting onode from cache

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #12877 from xiexingguo/xxg-wip-compress-opt

os/bluestore: conditionally load crr option

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #12900 from XinzeChi/wip-bluestore-valid

os/bluestore: nullptr in OmapIteratorImpl::valid

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #12844 from athanatos/wip-17831

osd/ReplicatedBackend: take read locks for clone sources during recovery

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Revert the change from assert(0)-> ceph_abort() where is not applicable

Signed-off-by: Dave Chen <wei.d.chen@intel.com>

removing fs workunit

removing fs workunit for now until tracker issue 18528 is resolved.

Signed-off-by: Tamil Muthamizhan <tmuthami@redhat.com>

Merge pull request #12924 from dillaman/wip-librbd-test-header

test: correct language mode in file headers

Reviewed-by: Mykola Golub <mgolub@mirantis.com>

Merge pull request #12892 from ceph/wip-cd-fs-fix

qa/tasks/ceph-deploy: use the new create option during instantiation

Reviewed-by: Alfredo Deza <adeza@redhat.com>

rgw: RGWCloneMetaLogCoroutine uses RGWMetadataLogInfoCompletion

Fixes: http://tracker.ceph.com/issues/18412
Signed-off-by: Casey Bodley <cbodley@redhat.com>

rgw: expose completion for RGWMetadataLog::get_info_async()

Signed-off-by: Casey Bodley <cbodley@redhat.com>

rgw: RGWMetaSyncShardCR drops stack refs on destruction

if the coroutine is canceled before collect_children() can clean up
all of its child stacks, those stack refs will leak. store these
stacks as boost::intrusive_ptr so the ref is dropped automatically on
destruction

Fixes: http://tracker.ceph.com/issues/18300
Signed-off-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #12884 from cbodley/wip-18488

rgw: fix off-by-one in RGWDataChangesLog::get_info

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>

os/bluestore: put Onode::key in mempool

This is a pain in the butt because std::string and
std::basic_string<...,custom allocator> are incompatible.

Signed-off-by: Sage Weil <sage@redhat.com>

os/bluestore: avoid substr

This creates an intermediate string, which is a waste.

Signed-off-by: Sage Weil <sage@redhat.com>

kv: add char* versions of get, set, rmkey

Signed-off-by: Sage Weil <sage@redhat.com>

os/bluestore,kstore: combine kv helpers in header

Signed-off-by: Sage Weil <sage@redhat.com>

os/bluestore: use mempool string for onode_t attr map

Signed-off-by: Sage Weil <sage@redhat.com>

include/denc: parameterize string by allocator

Signed-off-by: Sage Weil <sage@redhat.com>

mempool: Add string to mempool data types.

Signed-off-by: Allen Samuels <allen.samuels@sandisk.com>

global/signal_handler: dump backtrace even from within logging code

We can dump the backtrace regardless of where we hit the assertion.

Signed-off-by: Sage Weil <sage@redhat.com>

global/signal_handler: behave if g_ceph_context or _log are null

This can happen if we trigger an assertion while g_ceph_context is being
torn down.

Signed-off-by: Sage Weil <sage@redhat.com>

unittest_crypto: fix leaks

Signed-off-by: Sage Weil <sage@redhat.com>

mon/MonClient: use std::unique_ptr<>; fix leak

We were leaking several heap items from
LibRadosMiscConnectFailure.ConnectFailure when connect() was called
multiple times (after failure).  Use std::unique_ptr to avoid maintaining
fragile cleanup paths.

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #12233 from kylinstorage/optimize_request_auth_log

osd/pg: bound the portion of the log we request in GetLog::GetLog()

Reviewed-by: Samuel Just <sjust@redhat.com>

Merge pull request #12919 from tchaikov/wip-qa-cleanup

tests: remove temporary file

Reviewed-by: Abhishek Lekshmanan <alekshmanan@suse.com>

Merge pull request #12615 from linuxbox2/wip-sanitizers

build/cmake:  provide asan, tsan, ubsan builds

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Daniel Gryniewicz <dang@redhat.com>

Merge pull request #12920 from Liuchang0812/update-document-for-logging-level

doc: explain about logging levels

Reviewed-by: Sage Weil <sage@redhat.com>

test: correct language mode in file headers

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #12887 from jronak/documentation

Doc:Fixes Python Swift client commands

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #11879 from trociny/wip-15371

rbd-mirror: delayed replication support

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

doc: explain about logging levels

Signed-off-by: liuchang0812 <liuchang0812@gmail.com>

Merge pull request #12870 from xiaoxichen/mds_readdir

mds/server: skip unwanted dn in handle_client_readdir

Reviewed-by: Yan, Zheng <zyan@redhat.com>
Reviewed-by: John Spray <john.spray@redhat.com>
Reviewed-by: Greg Farnum <gfarnum@redhat.com>

Merge pull request #12831 from jcsp/wip-libcephfs-refcount

libcephfs: fix cct refcount constructing from rados

Reviewed-by: Greg Farnum <gfarnum@redhat.com>

Doc: Fixes Python Swift client commands

Fixes: http://tracker.ceph.com/issues/17746
Signed-off-by: Ronak Jain <ronakjain@outlook.in>

Merge pull request #12859 from ukernel/wip-18460

client: fix Client::handle_cap_flushsnap_ack() crash

Reviewed-by: Greg Farnum <gfarnum@redhat.com>

Merge pull request #12852 from ukernel/wip-18461

mds: finish clientreplay requests before requesting active state

Reviewed-by: Greg Farnum <gfarnum@redhat.com>

Merge pull request #12833 from ukernel/wip-18396

tasks/cephfs: fix kernel force umount

Reviewed-by: Greg Farnum <gfarnum@redhat.com>

Merge pull request #12808 from ukernel/wip-18306

mds: fix null pointer dereference in Locker::handle_client_caps

Reviewed-by: Greg Farnum <gfarnum@redhat.com>

Merge pull request #12749 from ukernel/wip-18179

mds: propagate error encountered during opening inode by number

Reviewed-by: John Spray <john.spray@redhat.com>

Merge pull request #12501 from ganeshmaharaj/fix-osd-tracing

tracing: Fix error in including all files in osd_tp

Reviewed-by: Kefu Chai <kchai@redhat.com>

tests: remove temporary file

Signed-off-by: Kefu Chai <kchai@redhat.com>

os/bluestore/BitAllocator: kill find_any_free_bits()

Which has no consumers.

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

os/bluestore: replace m_max_alloc_size with m_max_blocks

So we can get rid of the annoying transfer each time we
try to add an extent into an ExtentList.
Also simplify the add_extents() method a little.

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

Merge pull request #12848 from dzafman/wip-18469

ceph-objectstore-tool: Handle object names that are also valid json

Reviewed-by: Dan MIck <dmick@redhat.com>

Merge pull request #12873 from Liuchang0812/wip-static-check

test: clean up unused variable

Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>

client: populate metadata during mount

This way we avoid having to over-write the "root"
metadata during mount, and any user-set overrides (such
as bad values injected by tests) will survive.

Because Client instances may also open sessions without
mounting to send commands, add a call into populate_metadata
from mds_command as well.

Fixes: http://tracker.ceph.com/issues/18361
Signed-off-by: John Spray <john.spray@redhat.com>

Merge pull request #12913 from zmc/wip-smoke-openstack

Add openstack requirements to smoke suite

Reviewed-by: Yuri Weinstein <yweins@redhat.com>

Merge pull request #12914 from tmuthamizhan/wip-ansible-fs-suite

qa: specify client for fs workunit

Reviewed-by: Yuri Weinstein <yweins@redhat.com>

qa: specify client for fs workunit
s/all/client.0

Signed-off-by: Tamil Muthamizhan <tmuthami@redhat.com>

Add openstack requirements to smoke suite

Signed-off-by: Zack Cerza <zack@redhat.com>