qa/tasks: add test_open_ino_errors

Validate that errors encountered during opening inos are properly
propagated

Signed-off-by: Yan, Zheng <zyan@redhat.com>

mds: propagate error encountered during opening inode by number

Fixes: http://tracker.ceph.com/issues/18179
Signed-off-by: Yan, Zheng <zyan@redhat.com>

Merge pull request #12849 from xiexingguo/xxg-wip-fix-compile-crash

librados: fix compile errors from simplified aio completions

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #9032 from tchaikov/wip-15786

doc: document repair/scrub features

Reviewed-by: David Zafman <dzafman@redhat.com>

librados: fix building crash

See https://github.com/ceph/ceph/pull/12607

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

Merge pull request #12726 from liewegas/wip-18372

osd: _exit() intead of exit() for failure injection

Reviewed-by: Samuel Just <sjust@redhat.com>

Merge pull request #12607 from liewegas/wip-librados-killack

librados,osdc: kill ack vs commit distinction

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: John Spray <john.spray@redhat.com>

Merge pull request #12695 from hjwsm1989/mark-unfound-lost

osd/PrimaryLogPG: optimal pick_newest_available

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #12724 from kylinstorage/wip-fix-coverity-warning-for-uninitialized-members

osd: fix coverity warning for uninitialized members

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #12725 from liewegas/wip-bluestore-bluefs-reclaim

os/bluestore/BlueFS: fix reclaim_blocks

Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>
Reviewed-by: Varada Kari <varada.kari@sandisk.com>

Merge pull request #12727 from liewegas/wip-18369

osd/PG: publish PG stats when backfill-related states change

Reviewed-by: Samuel Just <sjust@redhat.com>

Merge pull request #12731 from oritwas/wip-rgw-realm-set

rgw realm set fixes

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #12274 from ukernel/wip-multimds-misc

mds: misc multimds fixes

Reviewed-by: Greg Farnum <gfarnum@redhat.com>

Merge pull request #12588 from jcsp/wip-18311

mds: check for errors decoding backtraces

Reviewed-by: Greg Farnum <gfarnum@redhat.com>

Merge pull request #12580 from Liuchang0812/fix-clang-warn-client

client: fix clang warn of "argument is an uninitialized value"

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #12557 from stiopaa1/libcephfs_memoryLeakCephContext

libcephfs.cc: fix memory leak

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #12161 from jcsp/wip-remove-cap

client: simplify remove_cap interface

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #12515 from kylinstorage/wip-safe-trace

client: fix potential buffer overflow

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #12560 from wjwithagen/wip-wjw-rocksdb-NO-FAIL

src/CMakeLists.txt: disable -Werror on rocksdb

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #12799 from tchaikov/wip-sepia-bt

script/sepia_bt.sh: download packages from shaman not gitbuilder

Reviewed-by: Brad Hubbard <bhubbard@redhat.com>

Merge pull request #12816 from SUSE/wip-18440

tests: subst repo and branch in git.ceph.com URL in qa/tasks/cram.py and qa/tasks/qemu.py

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #12818 from SUSE/wip-drop-rbd_functional_tests

tests: drop unused rbd_functional_tests.pl script

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #12639 from dillaman/wip-18327

librbd: new API method to force break a peer's exclusive lock

Reviewed-by: Mykola Golub <mgolub@mirantis.com>

radosgw-admin: check for name mistmatch in realm set

Signed-off-by: Orit Wasserman <owasserm@redhat.com>

radosgw-admin: relam set can use input redirection

Signed-off-by: Orit Wasserman <owasserm@redhat.com>

radosgw-admin: realm set should create a new realm

Fixes: http://tracker.ceph.com/issues/18333
Signed-off-by: Orit Wasserman <owasserm@redhat.com>

Merge pull request #12812 from optimistyzy/16_0

NVMEDevice: fix bug in data_buf_next_sge

Reviewed-by: Haomai Wang <haomai@xsky.com>

Merge pull request #12767 from linuxbox2/rgw-invalid-aio-read4

rgw: remove invalid read size4

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>

NVMEDevice: Eliminate virt to physical address translation in data_buf_next_sge

After SPDK is updated, SPDK requires virtual address
instead of physical address in huge page memory. Thus,
we do not need to pass the physical address since SPDK
library will do the address translation work.If we
still use the original code, it seems a bug.

In this patch, we will also use spdk_zmalloc instead of
dpdk's rte_malloc_socket function.

Signed-off-by: optimistyzy <optimistyzy@gmail.com>

rgw_rados: sanitize dout print in GWRados::get_obj_iterate_cb(...)

We cannot assume pbl may be deferenced.  Per review, move the dout
print into the r < 0 condition--since it's now an error, make it's
trace level 0.

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>

rgw_rados: add guard assert in add_io()

Use the iterator-returning insert operation in std::map, check
assert the insert case.  As a side effect, this makes use of the
inserted object record more clear.

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>

tests: drop unused rbd_functional_tests.pl script

This script is no longer used.

Signed-off-by: Nathan Cutler <ncutler@suse.com>

tests: subst repo and branch in qemu test urls

References: http://tracker.ceph.com/issues/18440
Signed-off-by: Nathan Cutler <ncutler@suse.com>

tests: subst branch and repo in qa/tasks/qemu.py

References: http://tracker.ceph.com/issues/18440
Signed-off-by: Nathan Cutler <ncutler@suse.com>

tests: subst repo name in krbd/unmap/tasks/unmap.yaml

Signed-off-by: Nathan Cutler <ncutler@suse.com>

tests: subst repo name in qa/tasks/cram.py

Inspired by bcbe45d948f1c4da02e27b3be5f29a8b52745e28

Fixes: http://tracker.ceph.com/issues/18440
Signed-off-by: Nathan Cutler <ncutler@suse.com>

Merge pull request #12549 from trociny/wip-16555

librbd: permit removal of image being bootstrapped by rbd-mirror

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #12638 from yangdongsheng/rbd_remove_incompatible

librbd: don't continue to remove an image w/ incompatible features

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #12787 from jamincollins/arch-linux

ceph-detect-init: adding Arch Linux support

Reviewed-by: Dan Mick <dmick@redhat.com>

Merge pull request #11776 from ceph/wip-rgw-openssl-7

Wip rgw openssl 7

librados: blacklist_add should wait for latest OSD map

This ensures that future operations against the OSDs force
a OSD map update to notice the blacklisted client.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

librbd: prevent assertion failure when journal IO is blacklisted

Fixes: http://tracker.ceph.com/issues/18429
Signed-off-by: Jason Dillaman <dillaman@redhat.com>

librbd: ignore blacklist error when releasing exclusive lock

This ensures the journal and object map are properly closed so that the
image can be properly closed w/o failing any assertions.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

librbd: fail immediately if the exclusive lock cannot be acquired

Fixes: http://tracker.ceph.com/issues/16988
Signed-off-by: Jason Dillaman <dillaman@redhat.com>

librbd: add new lock_get_owners / lock_break_lock API methods

If the client application supports failover, let the application
force break the current lock and blacklist the owner. This is
required in case the current lock owner is alive from the point-of-view
of librbd but failover was required due to a higher level reason.

Fixes: http://tracker.ceph.com/issues/18327
Signed-off-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #12096 from wjwithagen/wip-wjw-readme-20161121

README.FreeBSD: update current status

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #12628 from SUSE/wip-18309-alt

ceph_fuse: pid_file default to empty

Reviewed-by: John Spray <john.spray@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

librbd: separate break lock logic into standalone state machine

The current lockers are now queried before the lock is attempted to
prevent any possible race conditions when one or more clients attempt
to break the lock of a dead client.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

librbd: separate locker query into standalone state machine

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #12811 from wjwithagen/wip-wjw-rbd-udev-blkid

cmake: cleanup the use of udev and blkid in target_link_lib()

Reviewed-by: Kefu Chai <kefu@redhat.com>

Merge pull request #12478 from renhwztetecs/renhw-wip-client-lookup_parent

client: don't request lookup parent if ino is root

Reviewed-by: John Spray <john.spray@redhat.com>

Merge pull request #12276 from kylinstorage/wip-cleanup-makeshared

cleanup: use std::make_shared to replace new

Reviewed-by: John Spray <john.spray@redhat.com>

tests: add no_client_pidfile override to fs/recovery tests

Signed-off-by: Nathan Cutler <ncutler@suse.com>

tests: override yaml to set client pid file to empty string

Due to http://tracker.ceph.com/issues/18309 the pid file for fuse clients
should always be set to the empty string. (Teuthology's default ceph.conf
sets it to /var/run/ceph/$cluster-$name.pid)

This commit adds a reusable yaml facet for this purpose.

Signed-off-by: Nathan Cutler <ncutler@suse.com>

Cmake: cleanup the use of udev and blkid in target_link_lib()

Signed-off-by: Willem Jan Withagen <wjw@digiware.nl>

doc: update troubleshooting-pg with scrub/repair feature

update rados/troubleshooting/troubleshooting-pg.rst with the
scrub/repair feature

Fixes: http://tracker.ceph.com/issues/15786
Signed-off-by: David Zafman <dzafman@redhat.com>
Signed-off-by: Kefu Chai <kchai@redhat.com>

doc: update rados.8 with list-inconsistent-* commands

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #12780 from optimistyzy/201714_1

NVMEDevice: optimize sector_size usage

Reviewed-by: Haomai Wang <haomai@xsky.com>
Reviewed-by: Pan Liu <pan.liu@istuary.com>

Merge pull request #12776 from optimistyzy/201714

cmake: remove Findpciaccess.cmake

Reviewed-by: Kefu Chai <kefu@redhat.com>
Reviewed-by: Pan Liu <pan.liu@istuary.com>

Merge pull request #12806 from yuriw/wip-fix-distro-master

qa: Using centos 7.2 for `latest` version

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Using centos 7.2 as latest version

Signed-off-by: Yuri Weinstein <yweinste@redhat.com>

Merge pull request #12804 from yuriw/wip-fix-distro-master

qa: Cleaned up distros to use `latest` versions

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

qa: Cleaned up distros to use `latest` versions

Signed-off-by: Yuri Weinstein <yweinste@redhat.com>

fuse: remove dead code after usage() call

usage() calls generic_client_usage(), which calls exit(-1), which never
returns.

Signed-off-by: Nathan Cutler <ncutler@suse.com>

ceph_fuse: pid_file default to empty

Fixes: http://tracker.ceph.com/issues/18309
Signed-off-by: Nathan Cutler <ncutler@suse.com>

Merge pull request #12788 from athanatos/wip-oi-mismatch

PrimaryLogPG: don't update digests for objects with mismatched names

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #12413 from ovh/wip-support-hammer-in-rbd_recover_tool

tools: support hammer in rbd_recover_tool

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #12050 from dillaman/wip-16180

librados: asynchronous selfmanaged_snap_create/selfmanaged_snap_remove APIs

Reviewed-by: Mykola Golub <mgolub@mirantis.com>

Merge pull request #12763 from wjwithagen/wip-wjw-freebsd-boostpython

CMakeLists.txt: boost_python.so requires libpython.*.so on FreeBSD

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #12562 from majianpeng/rockdb-option

rocksdb: add option: writable_file_max_buffer_size = 0

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #12760 from tchaikov/wip-18371

ceph-disk: convert none str to str before printing it

Reviewed-by: Loic Dachary <ldachary@redhat.com>
Reviewed-by: Alfredo Deza <adeza@redhat.com>

Merge pull request #12728 from Liuchang0812/fix-17861

osd: add asock command to dump the scrub queue

Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #12775 from xiexingguo/xxg-wip-shutup-warning

common: get rid of "warning: ignoring return value of ‘strerror_r’"

Reviewed-by: Kefu Chai <kchai@redhat.com>

qa/tasks: add test_corrupt_backtrace

Validate that we get EIO and a damage table entry
when seeing a decode error on a backtrace.

Signed-off-by: John Spray <john.spray@redhat.com>

mds: check for errors decoding backtraces

Fixes: http://tracker.ceph.com/issues/18311
Signed-off-by: John Spray <john.spray@redhat.com>

Merge pull request #12769 from bachp/cmake-keyutils

cmake: search for Keyutils in default paths

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #12785 from wjwithagen/wip-wjw-cmake-parse_secret_objs

src/CMakeLists.txt: Move parse_secret_objs setting within definition block

Reviewed-by: Kefu Chai <kchai@redhat.com>

script/sepia_bt.sh: download packages from shaman not gitbuilder

also download ceph-test packages if we are debugging ceph_test_*.

Signed-off-by: Kefu Chai <kchai@redhat.com>

tools: Support hammer in rbd_recover_tool

Signed-off-by: Bartłomiej Święcki <bartlomiej.swiecki@corp.ovh.com>

NVMEDevice: optimize sector_size usage

spdk_nvme_ns_get_sector_size returns 32 bit size,
so we just use 32 bit size for sector_size.

Signed-off-by: optimistyzy <optimistyzy@gmail.com>

Merge pull request #12772 from hjwsm1989/mempool

  include/mempool: fix typo in comments

Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>

Merge pull request #12779 from badone/wip-check_new_interval-comment

osd/osd_types: Move comment to more relevant position

Reviewed-by: Samuel Just <sjust@redhat.com>

Merge pull request #12610 from tmuthamizhan/wip-ansible-suite

qa/suite: Added a smoke suite for ceph-ansible

Reviewed-by: Yuri Weinstein <yweins@redhat.com>

CMakeLists.txt/FreeBSD: Boost_python needs explicit linking with libpython*.so

 - Otherwise losts of unresolved errors will result.

Signed-off-by: Willem Jan Withagen <wjw@digiware.nl>

Merge pull request #12181 from SUSE/wip-18037

osd: improve error message when FileStore op fails due to EPERM

Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Joao Eduardo Luis <joao@suse.de>

Adding Arch Linux support

Signed-off-by: Jamin W. Collins <jamin.collins@gmail.com>

ceph-disk: convert none str to str before printing it

Error('somethings goes wrong', e) is thrown if exception `e` is caught
in ceph-disk, where e is not a string. so we can not just concat it in
Error's __str__(). so cast it to str before doing so.

introduced by d0e29c7

Fixes: http://tracker.ceph.com/issues/18371
Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #12697 from kylinstorage/wip-rbd-bench

rbd: additional validation for 'bench' optional parameters

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #12319 from gaurav36/bug-18125-rbd-mirror-enable-disable

librbd: warning message for mirroring pool option

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #12191 from wjwithagen/wip-wjw-freebsd-radosgw-admin

rgw_file: radosgw-admin can be built under FreeBSD

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #12737 from amirv/refactor

Move code from .h into .cc

Reviewed-by: Haomai Wang <haomai@xsky.com>

src/CMakeLists.txt: Move parse_secret_objs setting within definition block

 - parse_secret_objs is only used when secret.c is actually used
   and that is with CEPHFS or RBD

This removes a Cmake warning on FreeBSD

Signed-off-by: Willem Jan Withagen <wjw@digiware.nl>

cmake: search for Keyutils in default paths and cleanup

This makes it more portable

Remove /usr/include as it is searched by find_path anyway.
Remove unnecessary include that weren't used.

Signed-off-by: Pascal Bach <pascal.bach@nextrem.ch>

Merge pull request #12705 from badone/wip-ceph_test_rados_api_tier-valgrind-fixes

test/librados/tier.cc: Fix valgrind errors

Reviewed-by: Sage Weil <sage@redhat.com>

rbd: warning message for mirroring pool option

When user enable/disable mirroring option on a pool, than
re-enabling/disabling mirroring option should give warning
message if it is already enable/disable.
It should also give warning when changing the mode from pool
to image or vice versa.

Fixes: http://tracker.ceph.com/issues/18125
Reported-by: Jason Dillaman <dillaman@redhat.com>
Signed-off-by: Gaurav Kumar Garg <garg.gaurav52@gmail.com>

Merge pull request #12544 from hjwsm1989/oc

osdc/ObjectCacher: use state instead of get_state()

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #8942 from majianpeng/Finisher-optimize

common/Finisher: Using queue(list<context*>) instead queue(context*).

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #9760 from stiopaa1/msg_messengercc_move

msg/Messenger.cc: add std::move

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #12732 from Liuchang0812/wip-issue-17449

mgr: Make stats period configurable

Reviewed-by: Kefu Chai <kchai@redhat.com>

osd: add asock command to dump the scrub queue

* add a public method to dumps scrubs
* register new command "dump_scrubs" to asock_admin

Fixes: http://tracker.ceph.com/issues/17861
Signed-off-by: liuchang0812 <liuchang0812@gmail.com>