os/bluestore: kill BufferSpace.empty()

Because:
1. All other public methods are called under the protection of cache->lock,
which makes the usage of this method is weird.

2. The only caller is _dump_extent_map(), and we can do it without this
in a simpler way.

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

Merge pull request #12867 from ceph/wip-ceph-deploy-workaround

qa/tasks/ceph-deploy: create-keys explicitly

Reviewed-by: Alfredo Deza <adeza@redhat.com>

Merge pull request #12626 from adamemerson/wip-denconstruction

include/denc: improvements

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #12862 from optimistyzy/110_1

os/bluestore/NVMEdevice: fix the unrelease segs issue

Reviewed-by: Haomai Wang <haomai@xsky.com>

Merge pull request #12091 from wjwithagen/wip-wjw-nosetests

nosetests: use /usr/bin/env to find nosetests

Reviewed-by: Loic Dachary <ldachary@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

denc: Support boost::optional

Signed-off-by: Adam C. Emerson <aemerson@redhat.com>

denc: Support tuples

We could support zero-length tuples, but contiguous appender doesn't
like zero-length writes, so I've ruled them out for now.

Signed-off-by: Adam C. Emerson <aemerson@redhat.com>

denc: Add support for std::array

Signed-off-by: Adam C. Emerson <aemerson@redhat.com>

denc: Support boost::container::flat_set

Signed-off-by: Adam C. Emerson <aemerson@redhat.com>

denc: Scrap the container boilerplate!

Using template templates, kill off the reduplication in
container encoding.

Signed-off-by: Adam C. Emerson <aemerson@redhat.com>

nosetests: start using /usr/bin/env to find noestests

 - Option nocapture is not really required so no problem
   with the fact that FreeBSD env does not work for params
   for nosetests

Signed-off-by: Willem Jan Withagen <wjw@digiware.nl>

Merge pull request #12821 from SUSE/wip-drop-rbd_cli_tests

tests: drop rbd_cli_tests.pl and RbdLib.pm

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #12850 from xiexingguo/xxg-wip-bluestore-20170110

os/bluestore: shrink buffer_map key into uint32_t

Reviewed-by: Igor Fedotov <ifedotov@mirantis.com>

Merge pull request #12815 from rjfd/wip-journal-unittest-build-error

tests: fix template specialization of PromoteRequest class

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #12853 from xiexingguo/xxg-wip-kill-orphan

os/bluestore: kill orphan declaration of do_write_check_depth()

Reviewed-by: Igor Fedotov <ifedotov@mirantis.com>

Merge pull request #12856 from xiexingguo/xxg-wip-clean-kd

os/bluestore/KernelDevice: kill zeros

Reviewed-by: Igor Fedotov <ifedotov@mirantis.com>

Merge pull request #12858 from yangdongsheng/rbd_cleanup

rbd: cleanup: fix the typo in namespace comment

NVMEdevice: fix the unrelease segs issue

When the read or flush command fails, we need to
release segs. Or we can abort it like what we do
for write.

Signed-off-by: optimistyzy <optimistyzy@gmail.com>

os/bluestore: shrink buffer_map key into uint32_t

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

rbd: cleanup: fix the typo in namespace comment

Signed-off-by: Dongsheng Yang <dongsheng.yang@easystack.cn>

Merge pull request #12855 from badone/wip-md_config_t-_apply_changes-unchangeable

config: Improve warning for unobserved value

Reviewed-by: Kefu Chai <kchai@redhat.com>

os/bluestore/KernelDevice: reuse r

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

os/bluestore/KernelDevice: drop redundant assignment statements

The memset() call has already done it.

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

os/bluestore/NVMeDevice: kill zeros

See https://github.com/ceph/ceph/commit/2cd63d2209aaa0221a3f0a69b3f49ab1e01d9e20
The aio_zero() interface is deprecated.

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

os/bluestore/KernelDevice: kill zeros

This is not needed anymore and can save us 1MB memory.

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

config: Improve warning for unobserved value

When a variable is not being observed we currently mark it
"unchangable". This can be misleading so try something hopefully a
little more informative.

Fixes: http://tracker.ceph.com/issues/18424
Signed-off-by: Brad Hubbard <bhubbard@redhat.com>

Merge pull request #12854 from liupan1111/wip-fix-async-dpdk-issue

async: fixed coredump when enable dpdk.

Reviewed-by: Haomai Wang <haomai@xsky.com>

async: fixed coredump when enable dpdk.

Signed-off-by: Pan Liu <pan.liu@istuary.com>

os/bluestore: kill orphan declaration of do_write_check_depth()

The gc logic is gone and this is not relevant anymore.

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

Merge pull request #12849 from xiexingguo/xxg-wip-fix-compile-crash

librados: fix compile errors from simplified aio completions

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #9032 from tchaikov/wip-15786

doc: document repair/scrub features

Reviewed-by: David Zafman <dzafman@redhat.com>

librados: fix building crash

See https://github.com/ceph/ceph/pull/12607

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

Remove debug overrides

the high level of debug for mon/osd is causing remoto to hang during get key

Signed-off-by: Vasu Kulkarni <vasu@redhat.com>

Add ceph-create-keys to explicitly create admin/bootstrap keys

Signed-off-by: Vasu Kulkarni <vasu@redhat.com>

Merge pull request #12726 from liewegas/wip-18372

osd: _exit() intead of exit() for failure injection

Reviewed-by: Samuel Just <sjust@redhat.com>

Merge pull request #12607 from liewegas/wip-librados-killack

librados,osdc: kill ack vs commit distinction

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: John Spray <john.spray@redhat.com>

Merge pull request #12695 from hjwsm1989/mark-unfound-lost

osd/PrimaryLogPG: optimal pick_newest_available

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #12724 from kylinstorage/wip-fix-coverity-warning-for-uninitialized-members

osd: fix coverity warning for uninitialized members

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #12725 from liewegas/wip-bluestore-bluefs-reclaim

os/bluestore/BlueFS: fix reclaim_blocks

Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>
Reviewed-by: Varada Kari <varada.kari@sandisk.com>

Merge pull request #12727 from liewegas/wip-18369

osd/PG: publish PG stats when backfill-related states change

Reviewed-by: Samuel Just <sjust@redhat.com>

Merge pull request #12731 from oritwas/wip-rgw-realm-set

rgw realm set fixes

Reviewed-by: Casey Bodley <cbodley@redhat.com>

rbd: fix template specialization of PromoteRequest class

Signed-off-by: Ricardo Dias <rdias@suse.com>

Merge pull request #12274 from ukernel/wip-multimds-misc

mds: misc multimds fixes

Reviewed-by: Greg Farnum <gfarnum@redhat.com>

Merge pull request #12588 from jcsp/wip-18311

mds: check for errors decoding backtraces

Reviewed-by: Greg Farnum <gfarnum@redhat.com>

Merge pull request #12580 from Liuchang0812/fix-clang-warn-client

client: fix clang warn of "argument is an uninitialized value"

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #12557 from stiopaa1/libcephfs_memoryLeakCephContext

libcephfs.cc: fix memory leak

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #12161 from jcsp/wip-remove-cap

client: simplify remove_cap interface

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #12515 from kylinstorage/wip-safe-trace

client: fix potential buffer overflow

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #12560 from wjwithagen/wip-wjw-rocksdb-NO-FAIL

src/CMakeLists.txt: disable -Werror on rocksdb

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #12799 from tchaikov/wip-sepia-bt

script/sepia_bt.sh: download packages from shaman not gitbuilder

Reviewed-by: Brad Hubbard <bhubbard@redhat.com>

tests: drop rbd_cli_tests.pl and RbdLib.pm

The rbd_cli_tests Perl script is not maintained and currently serves no
purpose. The RbdLib.pm module was only used by rbd_functional_tests.pl (which
was dropped by 276ffb46319a78a047f3c1f401d8125ac9e17c1e) and rbd_cli_tests.pl
so drop it as well.

Fixes: http://tracker.ceph.com/issues/14825
Signed-off-by: Nathan Cutler <ncutler@suse.com>

Merge pull request #12816 from SUSE/wip-18440

tests: subst repo and branch in git.ceph.com URL in qa/tasks/cram.py and qa/tasks/qemu.py

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #12818 from SUSE/wip-drop-rbd_functional_tests

tests: drop unused rbd_functional_tests.pl script

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #12639 from dillaman/wip-18327

librbd: new API method to force break a peer's exclusive lock

Reviewed-by: Mykola Golub <mgolub@mirantis.com>

radosgw-admin: check for name mistmatch in realm set

Signed-off-by: Orit Wasserman <owasserm@redhat.com>

radosgw-admin: relam set can use input redirection

Signed-off-by: Orit Wasserman <owasserm@redhat.com>

radosgw-admin: realm set should create a new realm

Fixes: http://tracker.ceph.com/issues/18333
Signed-off-by: Orit Wasserman <owasserm@redhat.com>

Merge pull request #12812 from optimistyzy/16_0

NVMEDevice: fix bug in data_buf_next_sge

Reviewed-by: Haomai Wang <haomai@xsky.com>

Merge pull request #12767 from linuxbox2/rgw-invalid-aio-read4

rgw: remove invalid read size4

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>

NVMEDevice: Eliminate virt to physical address translation in data_buf_next_sge

After SPDK is updated, SPDK requires virtual address
instead of physical address in huge page memory. Thus,
we do not need to pass the physical address since SPDK
library will do the address translation work.If we
still use the original code, it seems a bug.

In this patch, we will also use spdk_zmalloc instead of
dpdk's rte_malloc_socket function.

Signed-off-by: optimistyzy <optimistyzy@gmail.com>

rgw_rados: sanitize dout print in GWRados::get_obj_iterate_cb(...)

We cannot assume pbl may be deferenced.  Per review, move the dout
print into the r < 0 condition--since it's now an error, make it's
trace level 0.

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>

rgw_rados: add guard assert in add_io()

Use the iterator-returning insert operation in std::map, check
assert the insert case.  As a side effect, this makes use of the
inserted object record more clear.

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>

tests: drop unused rbd_functional_tests.pl script

This script is no longer used.

Signed-off-by: Nathan Cutler <ncutler@suse.com>

tests: subst repo and branch in qemu test urls

References: http://tracker.ceph.com/issues/18440
Signed-off-by: Nathan Cutler <ncutler@suse.com>

tests: subst branch and repo in qa/tasks/qemu.py

References: http://tracker.ceph.com/issues/18440
Signed-off-by: Nathan Cutler <ncutler@suse.com>

tests: subst repo name in krbd/unmap/tasks/unmap.yaml

Signed-off-by: Nathan Cutler <ncutler@suse.com>

tests: subst repo name in qa/tasks/cram.py

Inspired by bcbe45d948f1c4da02e27b3be5f29a8b52745e28

Fixes: http://tracker.ceph.com/issues/18440
Signed-off-by: Nathan Cutler <ncutler@suse.com>

Merge pull request #12549 from trociny/wip-16555

librbd: permit removal of image being bootstrapped by rbd-mirror

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #12638 from yangdongsheng/rbd_remove_incompatible

librbd: don't continue to remove an image w/ incompatible features

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #12787 from jamincollins/arch-linux

ceph-detect-init: adding Arch Linux support

Reviewed-by: Dan Mick <dmick@redhat.com>

Merge pull request #11776 from ceph/wip-rgw-openssl-7

Wip rgw openssl 7

librados: blacklist_add should wait for latest OSD map

This ensures that future operations against the OSDs force
a OSD map update to notice the blacklisted client.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

librbd: prevent assertion failure when journal IO is blacklisted

Fixes: http://tracker.ceph.com/issues/18429
Signed-off-by: Jason Dillaman <dillaman@redhat.com>

librbd: ignore blacklist error when releasing exclusive lock

This ensures the journal and object map are properly closed so that the
image can be properly closed w/o failing any assertions.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

librbd: fail immediately if the exclusive lock cannot be acquired

Fixes: http://tracker.ceph.com/issues/16988
Signed-off-by: Jason Dillaman <dillaman@redhat.com>

librbd: add new lock_get_owners / lock_break_lock API methods

If the client application supports failover, let the application
force break the current lock and blacklist the owner. This is
required in case the current lock owner is alive from the point-of-view
of librbd but failover was required due to a higher level reason.

Fixes: http://tracker.ceph.com/issues/18327
Signed-off-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #12096 from wjwithagen/wip-wjw-readme-20161121

README.FreeBSD: update current status

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #12628 from SUSE/wip-18309-alt

ceph_fuse: pid_file default to empty

Reviewed-by: John Spray <john.spray@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

librbd: separate break lock logic into standalone state machine

The current lockers are now queried before the lock is attempted to
prevent any possible race conditions when one or more clients attempt
to break the lock of a dead client.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

librbd: separate locker query into standalone state machine

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #12811 from wjwithagen/wip-wjw-rbd-udev-blkid

cmake: cleanup the use of udev and blkid in target_link_lib()

Reviewed-by: Kefu Chai <kefu@redhat.com>

Merge pull request #12478 from renhwztetecs/renhw-wip-client-lookup_parent

client: don't request lookup parent if ino is root

Reviewed-by: John Spray <john.spray@redhat.com>

Merge pull request #12276 from kylinstorage/wip-cleanup-makeshared

cleanup: use std::make_shared to replace new

Reviewed-by: John Spray <john.spray@redhat.com>

tests: add no_client_pidfile override to fs/recovery tests

Signed-off-by: Nathan Cutler <ncutler@suse.com>

tests: override yaml to set client pid file to empty string

Due to http://tracker.ceph.com/issues/18309 the pid file for fuse clients
should always be set to the empty string. (Teuthology's default ceph.conf
sets it to /var/run/ceph/$cluster-$name.pid)

This commit adds a reusable yaml facet for this purpose.

Signed-off-by: Nathan Cutler <ncutler@suse.com>

Cmake: cleanup the use of udev and blkid in target_link_lib()

Signed-off-by: Willem Jan Withagen <wjw@digiware.nl>

doc: update troubleshooting-pg with scrub/repair feature

update rados/troubleshooting/troubleshooting-pg.rst with the
scrub/repair feature

Fixes: http://tracker.ceph.com/issues/15786
Signed-off-by: David Zafman <dzafman@redhat.com>
Signed-off-by: Kefu Chai <kchai@redhat.com>

doc: update rados.8 with list-inconsistent-* commands

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #12780 from optimistyzy/201714_1

NVMEDevice: optimize sector_size usage

Reviewed-by: Haomai Wang <haomai@xsky.com>
Reviewed-by: Pan Liu <pan.liu@istuary.com>

Merge pull request #12776 from optimistyzy/201714

cmake: remove Findpciaccess.cmake

Reviewed-by: Kefu Chai <kefu@redhat.com>
Reviewed-by: Pan Liu <pan.liu@istuary.com>

Merge pull request #12806 from yuriw/wip-fix-distro-master

qa: Using centos 7.2 for `latest` version

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Using centos 7.2 as latest version

Signed-off-by: Yuri Weinstein <yweinste@redhat.com>

Merge pull request #12804 from yuriw/wip-fix-distro-master

qa: Cleaned up distros to use `latest` versions

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

qa: Cleaned up distros to use `latest` versions

Signed-off-by: Yuri Weinstein <yweinste@redhat.com>

fuse: remove dead code after usage() call

usage() calls generic_client_usage(), which calls exit(-1), which never
returns.

Signed-off-by: Nathan Cutler <ncutler@suse.com>

ceph_fuse: pid_file default to empty

Fixes: http://tracker.ceph.com/issues/18309
Signed-off-by: Nathan Cutler <ncutler@suse.com>

Merge pull request #12788 from athanatos/wip-oi-mismatch

PrimaryLogPG: don't update digests for objects with mismatched names

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #12413 from ovh/wip-support-hammer-in-rbd_recover_tool

tools: support hammer in rbd_recover_tool

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #12050 from dillaman/wip-16180

librados: asynchronous selfmanaged_snap_create/selfmanaged_snap_remove APIs

Reviewed-by: Mykola Golub <mgolub@mirantis.com>

Merge pull request #12763 from wjwithagen/wip-wjw-freebsd-boostpython

CMakeLists.txt: boost_python.so requires libpython.*.so on FreeBSD

Reviewed-by: Kefu Chai <kchai@redhat.com>