librbd: clean up pre-release lock handling

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

test: separate testing of exclusive lock from managed lock

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

librbd: removed ManagedLock dependency from ancillary classes

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #12972 from linuxbox2/wip-librgw-shut

rgw: librgw shut

Merge pull request #12907 from theanalyst/wip-18064

rgw: sync status compares the current master period

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #12357 from dongbula/fix-versioned-bucket-datasync-fail

rgw: fix versioned bucket data sync fail when upload is busy

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #12945 from jimmyway/wip-add-cmd-snap-set-limit-prompt

rbd: add error prompt when input command 'snap set limit' is incomplete

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #12886 from rjfd/wip-shared-managed-lock

librbd: support for shared locking in ManagedLock

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

librgw: remove call to crypto::shutdown

The call is cognate to removed code in radosgw, and triggers an
assert that the crypto module has already been shutdown.  With this
and preceding changes, I see clean shutdown on abnormal termination
of nfs-ganesha during startup.

Fixes: http://tracker.ceph.com/issues/18585
Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>

rgw:  s/static std::map/static pointer/ for ext_mime_map

In some librgw shutdown cases, it appears that the shared
object runtime has partially destructed the static map when
our shutdown code is running.

Fixes: http://tracker.ceph.com/issues/18585
Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>

rgw_file:  signal RGWLibFrontend shutdown correctly

The shutdown member of RGWLibProcess should be set.

Fixes: http://tracker.ceph.com/issues/18585
Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>

Merge pull request #12927 from rjfd/wip-fix-rpath-suse

cmake: fix rpath on shared libraries and binaries targets

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #12750 from zhangsw/feature-rgw-lc-validatei

rgw: add lifecycle validation according to S3.

Merge pull request #12827 from Liuchang0812/wip-18459

doc: correct S3 lifecycle support explain

Merge pull request #12681 from xiexingguo/xxg-wip-bluestore-1227

os/bluestore: generate same onode extent-shard keys in a more efficient way

Reviewed-by: Igor Fedotov <ifedotov@mirantis.com>

Merge pull request #12696 from xiexingguo/xxg-wip-improve-loops

os/bluestore: miscellaneous fixes to BitAllocator

Reviewed-by: Ramesh Chander <Ramesh.Chander@sandisk.com>

Merge pull request #12719 from xiexingguo/xxg-wip-bluestore-1229

os/bluestore/BitmapFreelistManager: readability improvements

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #12942 from Liuchang0812/cleanup-bluestore

os/bluestore: avoid unnecessary memory copy, use variable reference in BlockDevice::Open

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Igor Fedotov <ifedotov@mirantis.com>

Merge pull request #12946 from ifed01/wip-bluestore-pextent-vector-mempooled

os/bluestore: Add bluestore pextent vector to mempool

Reviewed-by: Sage Weil <sage@redhat.com>

cmake: fix rpath on shared libraries and executables

The problem was due to the attempt of CMake to rewrite the RPATH of
the executables, or shared libraries, that didn't have any RPATH linked
during the building phase.

Currently INSTALL_RPATH is already set for all binaries. This patch
sets an empty INSTALL_RPATH for all binaries that don't depend on
any internal (also being built) target.

Signed-off-by: Ricardo Dias <rdias@suse.com>

doc: correct S3 lifecycle support explain

Fixes: http://tracker.ceph.com/issues/18459
Signed-off-by: liuchang0812 <liuchang0812@gmail.com>

librbd: Support for shared locking in ManagedLock

Signed-off-by: Ricardo Dias <rdias@suse.com>

Merge pull request #12964 from jcsp/wip-18574

qa: update remaining ceph.com to download.ceph.com

Merge pull request #12962 from dillaman/wip-rbd-boost-crash

rbd: remove direct linking to static boost libraries

Reviewed-by: Kefu Chai <kchai@redhat.com>

qa: update remaining ceph.com to download.ceph.com

Fixes: http://tracker.ceph.com/issues/18574
Signed-off-by: John Spray <john.spray@redhat.com>

rbd: remove direct linking to static boost libraries

The boost libraries are included in libceph-common. If linked
directly, boost::regex will abort during shutdown since it uses
a global static variable and it will be destructed twice.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #12885 from rjfd/wip-18325

librbd: allow to open an image without opening the parent image

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #12922 from trociny/wip-managed_lock_refactoring

librbd: managed lock refactoring

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #12953 from dillaman/wip-18542

qa/tasks/qemu: update default image url after ceph.com redesign

Reviewed-by: Mykola Golub <mgolub@mirantis.com>

Merge pull request #12954 from dillaman/wip-rbd-coverity

rbd: correct coverity warnings

Reviewed-by: Mykola Golub <mgolub@mirantis.com>

Merge pull request #12467 from bassam/pr-external-rocksdb

cmake: support for external rocksdb

Reviewed-by: Kefu Chai <kchai@redhat.com>

rbd: correct coverity warnings

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #9028 from tchaikov/wip-bt-dump

common/BackTrace: add operator<<

Reviewed-by: Brad Hubbard <bhubbard@redhat.com>

qa/tasks/qemu: update default image url after ceph.com redesign

Fixes: http://tracker.ceph.com/issues/18542
Signed-off-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #12807 from runsisi/wip-fix-pg-ls

mon/PGMonitor: fix description for ceph pg ls

Reviewed-by: Kefu Chai <kchai@redhat.com>

rgw: modify error code in lifecycle to be compatible with S3.

Signed-off-by: Zhang Shaowen <zhangshaowen@cmss.chinamobile.com>

mon/PGMonitor: ceph pg ls-by-pool only supports string type of pool arg

Signed-off-by: runsisi <runsisi@zte.com.cn>

Merge pull request #12928 from tmuthamizhan/wip-ansible-fs-suite

qa/suites/ceph-ansible: removing fs workunit

Reviewed-by: Yuri Weinstein <yweins@redhat.com>

librbd: helper methods to query and break lock

Signed-off-by: Mykola Golub <mgolub@mirantis.com>

librbd: managed_lock: make AcquireRequest use GetLockRequest and BreakRequest

Initially this was implemented for exclusive_lock (03533b9,
23f60fe) but was lost when merging managed_lock refactoring.

Signed-off-by: Mykola Golub <mgolub@mirantis.com>

Merge pull request #12915 from jcsp/wip-18361

client: populate metadata during mount

Reviewed-by: Yan, Zheng <zyan@redhat.com>

Merge pull request #12107 from mslovy/wip-monmsg-minwait

mon: no delay for single message MSG_ALIVE and MSG_PGTEMP

Reviewed-by: Gregory Farnum <gfarnum@redhat.com>
Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #11955 from batrick/log-2-to-1-write

log: use one write system call per message

Reviewed-by: Gregory Farnum <gfarnum@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #9924 from Yan-waller/yj-wiph-commonentitynamecc

common/config: fix return type of string::find and use string::npos

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #12925 from liewegas/wip-crypto-shutdown-fixes

common: misc fixes detected by crypto shutdown assert

Reviewed-by: Brad Hubbard <bhubbard@redhat.com>

Merge pull request #12944 from liewegas/wip-centos-latest

qa/distros: make centos_latest 7.3

Reviewed-by: Yuri Weinstein <yweins@redhat.com>

Merge pull request #12841 from cbodley/wip-18407

rgw multisite: fix ref counting of completions

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>

Merge pull request #12605 from cbodley/wip-18300

rgw: RGWMetaSyncShardCR drops stack refs on destruction

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>

librbd: allow to open an image without opening parent image

Fixes: http://tracker.ceph.com/issues/18325
Signed-off-by: Ricardo Dias <rdias@suse.com>

cmake: support for external rocksdb

add support for building with an external rocksdb.

Signed-off-by: Bassam Tabbara <bassam.tabbara@quantum.com>

Merge pull request #12798 from wjwithagen/wip-wjw-krbd-optional

rbd: enable rbd on FreeBSD (without KRBD)

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Jason Dillaman <dillaman@redhat.com>

os/bluestore: add pextent vector to mempool

Signed-off-by: Igor Fedotov <ifedotov@mirantis.com>

test/objectstore: add bufferptr & blouestore_blob_t::extents to "sizeof" UT

Signed-off-by: Igor Fedotvo <ifedotov@mirantis.com>

Merge pull request #12930 from chendave/wip_cephx_refactor

msg: Revert the change from assert(0)-> ceph_abort() where is not applicable

Reviewed-by: Sage Weil <sage@redhat.com>

qa/distros: make centos_latest 7.3

This will fix smithi runs and break vps runs until we get a 7.3 cloud-init
image ready.

Signed-off-by: Sage Weil <sage@redhat.com>

rbd: add error prompt when input command 'snap set limit' is incomplete

Signed-off-by: Tang Jin <tang.jin@istuary.com>

os/bluestore: avoid unnecessary memory copy, use variable reference in BlockDevice::Open

Signed-off-by: liuchang0812 <liuchang0812@gmail.com>

Merge pull request #12939 from johnlinp/fix-nspr-include

cmake: search for nspr include files for both suffixes: nspr4 and nspr

Reviewed-by: Kefu Chai <kchai@redhat.com>

cmake: search for nspr include files for both suffixes: nspr4 and nspr

Fixes: http://tracker.ceph.com/issues/18535
Signed-off-by: John Lin <johnlinp@gmail.com>

Merge pull request #12936 from badone/wip-pg-states-doc-fix

doc: min_size advice is not helpful

Reviewed-by: Kefu Chai <kchai@redhat.com>

os/bluestore: generate same onode extent-shard keys in a more efficient way

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

doc: min_size advice is not helpful

When we are in the incomplete state adjusting min_size will not help with recovery.

Signed-off-by: Brad Hubbard <bhubbard@redhat.com>

Merge pull request #12247 from YankunLi/fix_bf_append_parameter

bufferlist: cleanup semantical wrong for bufferlist::append

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #12345 from rook/wip-embedded-add-rgw-mds

cmake: add RGW and MDS to libcephd

Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

common/BackTrace: add operator<<

replace BackTrace::print() with the operator<< where the former is used.

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #12784 from LiumxNL/fix-wrongly-delete-routed-op

mon: fix wrongly delete routed pgstats op

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #12651 from liewegas/wip-mempool-string

os/bluestore: put strings in mempool

Reviewed-by: Igor Fedotov <ifedotov@mirantis.com>

Merge pull request #12661 from xiexingguo/xxg-wip-cleanup-bitalloc

os/bluestore: cleanup BitAllocator

Reviewed-by: Ramesh Chander <Ramesh.Chander@sandisk.com>

Merge pull request #12700 from ifed01/wip-bluestore-collection-everywhere

os/bluestore: various onode changes to reduce its in-memory footprint.

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #12876 from xiexingguo/xxg-wip-be-for-cache

os/bluestore: use Best-Effort policy when evicting onode from cache

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #12877 from xiexingguo/xxg-wip-compress-opt

os/bluestore: conditionally load crr option

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #12900 from XinzeChi/wip-bluestore-valid

os/bluestore: nullptr in OmapIteratorImpl::valid

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #12844 from athanatos/wip-17831

osd/ReplicatedBackend: take read locks for clone sources during recovery

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Revert the change from assert(0)-> ceph_abort() where is not applicable

Signed-off-by: Dave Chen <wei.d.chen@intel.com>

removing fs workunit

removing fs workunit for now until tracker issue 18528 is resolved.

Signed-off-by: Tamil Muthamizhan <tmuthami@redhat.com>

Merge pull request #12924 from dillaman/wip-librbd-test-header

test: correct language mode in file headers

Reviewed-by: Mykola Golub <mgolub@mirantis.com>

Merge pull request #12892 from ceph/wip-cd-fs-fix

qa/tasks/ceph-deploy: use the new create option during instantiation

Reviewed-by: Alfredo Deza <adeza@redhat.com>

rgw: RGWCloneMetaLogCoroutine uses RGWMetadataLogInfoCompletion

Fixes: http://tracker.ceph.com/issues/18412
Signed-off-by: Casey Bodley <cbodley@redhat.com>

rgw: expose completion for RGWMetadataLog::get_info_async()

Signed-off-by: Casey Bodley <cbodley@redhat.com>

rgw: RGWMetaSyncShardCR drops stack refs on destruction

if the coroutine is canceled before collect_children() can clean up
all of its child stacks, those stack refs will leak. store these
stacks as boost::intrusive_ptr so the ref is dropped automatically on
destruction

Fixes: http://tracker.ceph.com/issues/18300
Signed-off-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #12884 from cbodley/wip-18488

rgw: fix off-by-one in RGWDataChangesLog::get_info

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>

os/bluestore: put Onode::key in mempool

This is a pain in the butt because std::string and
std::basic_string<...,custom allocator> are incompatible.

Signed-off-by: Sage Weil <sage@redhat.com>

os/bluestore: avoid substr

This creates an intermediate string, which is a waste.

Signed-off-by: Sage Weil <sage@redhat.com>

kv: add char* versions of get, set, rmkey

Signed-off-by: Sage Weil <sage@redhat.com>

os/bluestore,kstore: combine kv helpers in header

Signed-off-by: Sage Weil <sage@redhat.com>

os/bluestore: use mempool string for onode_t attr map

Signed-off-by: Sage Weil <sage@redhat.com>

include/denc: parameterize string by allocator

Signed-off-by: Sage Weil <sage@redhat.com>

mempool: Add string to mempool data types.

Signed-off-by: Allen Samuels <allen.samuels@sandisk.com>

global/signal_handler: dump backtrace even from within logging code

We can dump the backtrace regardless of where we hit the assertion.

Signed-off-by: Sage Weil <sage@redhat.com>

global/signal_handler: behave if g_ceph_context or _log are null

This can happen if we trigger an assertion while g_ceph_context is being
torn down.

Signed-off-by: Sage Weil <sage@redhat.com>

unittest_crypto: fix leaks

Signed-off-by: Sage Weil <sage@redhat.com>

mon/MonClient: use std::unique_ptr<>; fix leak

We were leaking several heap items from
LibRadosMiscConnectFailure.ConnectFailure when connect() was called
multiple times (after failure).  Use std::unique_ptr to avoid maintaining
fragile cleanup paths.

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #12233 from kylinstorage/optimize_request_auth_log

osd/pg: bound the portion of the log we request in GetLog::GetLog()

Reviewed-by: Samuel Just <sjust@redhat.com>

Merge pull request #12919 from tchaikov/wip-qa-cleanup

tests: remove temporary file

Reviewed-by: Abhishek Lekshmanan <alekshmanan@suse.com>

Merge pull request #12615 from linuxbox2/wip-sanitizers

build/cmake:  provide asan, tsan, ubsan builds

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Daniel Gryniewicz <dang@redhat.com>

rgw_admin: read master log shards from master's current period

Also make the sync output look similar to the output of data sync
Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>

rgw: allow getting master log shards info on specified period

This is needed for rgw admin's sync status or else we end up always
publishing that we're behind since we are always checking against
master's first period to sync from

Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>

Merge pull request #12920 from Liuchang0812/update-document-for-logging-level

doc: explain about logging levels

Reviewed-by: Sage Weil <sage@redhat.com>

test: correct language mode in file headers

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #12887 from jronak/documentation

Doc:Fixes Python Swift client commands

Reviewed-by: Casey Bodley <cbodley@redhat.com>