config: Improve warning for unobserved value

When a variable is not being observed we currently mark it
"unchangable". This can be misleading so try something hopefully a
little more informative.

Fixes: http://tracker.ceph.com/issues/18424
Signed-off-by: Brad Hubbard <bhubbard@redhat.com>

Merge pull request #12816 from SUSE/wip-18440

tests: subst repo and branch in git.ceph.com URL in qa/tasks/cram.py and qa/tasks/qemu.py

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #12818 from SUSE/wip-drop-rbd_functional_tests

tests: drop unused rbd_functional_tests.pl script

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #12639 from dillaman/wip-18327

librbd: new API method to force break a peer's exclusive lock

Reviewed-by: Mykola Golub <mgolub@mirantis.com>

Merge pull request #12812 from optimistyzy/16_0

NVMEDevice: fix bug in data_buf_next_sge

Reviewed-by: Haomai Wang <haomai@xsky.com>

Merge pull request #12767 from linuxbox2/rgw-invalid-aio-read4

rgw: remove invalid read size4

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>

NVMEDevice: Eliminate virt to physical address translation in data_buf_next_sge

After SPDK is updated, SPDK requires virtual address
instead of physical address in huge page memory. Thus,
we do not need to pass the physical address since SPDK
library will do the address translation work.If we
still use the original code, it seems a bug.

In this patch, we will also use spdk_zmalloc instead of
dpdk's rte_malloc_socket function.

Signed-off-by: optimistyzy <optimistyzy@gmail.com>

rgw_rados: sanitize dout print in GWRados::get_obj_iterate_cb(...)

We cannot assume pbl may be deferenced.  Per review, move the dout
print into the r < 0 condition--since it's now an error, make it's
trace level 0.

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>

rgw_rados: add guard assert in add_io()

Use the iterator-returning insert operation in std::map, check
assert the insert case.  As a side effect, this makes use of the
inserted object record more clear.

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>

tests: drop unused rbd_functional_tests.pl script

This script is no longer used.

Signed-off-by: Nathan Cutler <ncutler@suse.com>

tests: subst repo and branch in qemu test urls

References: http://tracker.ceph.com/issues/18440
Signed-off-by: Nathan Cutler <ncutler@suse.com>

tests: subst branch and repo in qa/tasks/qemu.py

References: http://tracker.ceph.com/issues/18440
Signed-off-by: Nathan Cutler <ncutler@suse.com>

tests: subst repo name in krbd/unmap/tasks/unmap.yaml

Signed-off-by: Nathan Cutler <ncutler@suse.com>

tests: subst repo name in qa/tasks/cram.py

Inspired by bcbe45d948f1c4da02e27b3be5f29a8b52745e28

Fixes: http://tracker.ceph.com/issues/18440
Signed-off-by: Nathan Cutler <ncutler@suse.com>

Merge pull request #12549 from trociny/wip-16555

librbd: permit removal of image being bootstrapped by rbd-mirror

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #12638 from yangdongsheng/rbd_remove_incompatible

librbd: don't continue to remove an image w/ incompatible features

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #12787 from jamincollins/arch-linux

ceph-detect-init: adding Arch Linux support

Reviewed-by: Dan Mick <dmick@redhat.com>

Merge pull request #11776 from ceph/wip-rgw-openssl-7

Wip rgw openssl 7

librados: blacklist_add should wait for latest OSD map

This ensures that future operations against the OSDs force
a OSD map update to notice the blacklisted client.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

librbd: prevent assertion failure when journal IO is blacklisted

Fixes: http://tracker.ceph.com/issues/18429
Signed-off-by: Jason Dillaman <dillaman@redhat.com>

librbd: ignore blacklist error when releasing exclusive lock

This ensures the journal and object map are properly closed so that the
image can be properly closed w/o failing any assertions.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

librbd: fail immediately if the exclusive lock cannot be acquired

Fixes: http://tracker.ceph.com/issues/16988
Signed-off-by: Jason Dillaman <dillaman@redhat.com>

librbd: add new lock_get_owners / lock_break_lock API methods

If the client application supports failover, let the application
force break the current lock and blacklist the owner. This is
required in case the current lock owner is alive from the point-of-view
of librbd but failover was required due to a higher level reason.

Fixes: http://tracker.ceph.com/issues/18327
Signed-off-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #12096 from wjwithagen/wip-wjw-readme-20161121

README.FreeBSD: update current status

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #12628 from SUSE/wip-18309-alt

ceph_fuse: pid_file default to empty

Reviewed-by: John Spray <john.spray@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

librbd: separate break lock logic into standalone state machine

The current lockers are now queried before the lock is attempted to
prevent any possible race conditions when one or more clients attempt
to break the lock of a dead client.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

librbd: separate locker query into standalone state machine

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #12811 from wjwithagen/wip-wjw-rbd-udev-blkid

cmake: cleanup the use of udev and blkid in target_link_lib()

Reviewed-by: Kefu Chai <kefu@redhat.com>

Merge pull request #12478 from renhwztetecs/renhw-wip-client-lookup_parent

client: don't request lookup parent if ino is root

Reviewed-by: John Spray <john.spray@redhat.com>

Merge pull request #12276 from kylinstorage/wip-cleanup-makeshared

cleanup: use std::make_shared to replace new

Reviewed-by: John Spray <john.spray@redhat.com>

tests: add no_client_pidfile override to fs/recovery tests

Signed-off-by: Nathan Cutler <ncutler@suse.com>

tests: override yaml to set client pid file to empty string

Due to http://tracker.ceph.com/issues/18309 the pid file for fuse clients
should always be set to the empty string. (Teuthology's default ceph.conf
sets it to /var/run/ceph/$cluster-$name.pid)

This commit adds a reusable yaml facet for this purpose.

Signed-off-by: Nathan Cutler <ncutler@suse.com>

Cmake: cleanup the use of udev and blkid in target_link_lib()

Signed-off-by: Willem Jan Withagen <wjw@digiware.nl>

Merge pull request #12780 from optimistyzy/201714_1

NVMEDevice: optimize sector_size usage

Reviewed-by: Haomai Wang <haomai@xsky.com>
Reviewed-by: Pan Liu <pan.liu@istuary.com>

Merge pull request #12776 from optimistyzy/201714

cmake: remove Findpciaccess.cmake

Reviewed-by: Kefu Chai <kefu@redhat.com>
Reviewed-by: Pan Liu <pan.liu@istuary.com>

Merge pull request #12806 from yuriw/wip-fix-distro-master

qa: Using centos 7.2 for `latest` version

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Using centos 7.2 as latest version

Signed-off-by: Yuri Weinstein <yweinste@redhat.com>

Merge pull request #12804 from yuriw/wip-fix-distro-master

qa: Cleaned up distros to use `latest` versions

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

qa: Cleaned up distros to use `latest` versions

Signed-off-by: Yuri Weinstein <yweinste@redhat.com>

fuse: remove dead code after usage() call

usage() calls generic_client_usage(), which calls exit(-1), which never
returns.

Signed-off-by: Nathan Cutler <ncutler@suse.com>

ceph_fuse: pid_file default to empty

Fixes: http://tracker.ceph.com/issues/18309
Signed-off-by: Nathan Cutler <ncutler@suse.com>

Merge pull request #12788 from athanatos/wip-oi-mismatch

PrimaryLogPG: don't update digests for objects with mismatched names

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #12413 from ovh/wip-support-hammer-in-rbd_recover_tool

tools: support hammer in rbd_recover_tool

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #12050 from dillaman/wip-16180

librados: asynchronous selfmanaged_snap_create/selfmanaged_snap_remove APIs

Reviewed-by: Mykola Golub <mgolub@mirantis.com>

Merge pull request #12763 from wjwithagen/wip-wjw-freebsd-boostpython

CMakeLists.txt: boost_python.so requires libpython.*.so on FreeBSD

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #12562 from majianpeng/rockdb-option

rocksdb: add option: writable_file_max_buffer_size = 0

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #12760 from tchaikov/wip-18371

ceph-disk: convert none str to str before printing it

Reviewed-by: Loic Dachary <ldachary@redhat.com>
Reviewed-by: Alfredo Deza <adeza@redhat.com>

Merge pull request #12728 from Liuchang0812/fix-17861

osd: add asock command to dump the scrub queue

Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #12775 from xiexingguo/xxg-wip-shutup-warning

common: get rid of "warning: ignoring return value of ‘strerror_r’"

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #12769 from bachp/cmake-keyutils

cmake: search for Keyutils in default paths

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #12785 from wjwithagen/wip-wjw-cmake-parse_secret_objs

src/CMakeLists.txt: Move parse_secret_objs setting within definition block

Reviewed-by: Kefu Chai <kchai@redhat.com>

tools: Support hammer in rbd_recover_tool

Signed-off-by: Bartłomiej Święcki <bartlomiej.swiecki@corp.ovh.com>

NVMEDevice: optimize sector_size usage

spdk_nvme_ns_get_sector_size returns 32 bit size,
so we just use 32 bit size for sector_size.

Signed-off-by: optimistyzy <optimistyzy@gmail.com>

Merge pull request #12772 from hjwsm1989/mempool

  include/mempool: fix typo in comments

Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>

Merge pull request #12779 from badone/wip-check_new_interval-comment

osd/osd_types: Move comment to more relevant position

Reviewed-by: Samuel Just <sjust@redhat.com>

Merge pull request #12610 from tmuthamizhan/wip-ansible-suite

qa/suite: Added a smoke suite for ceph-ansible

Reviewed-by: Yuri Weinstein <yweins@redhat.com>

CMakeLists.txt/FreeBSD: Boost_python needs explicit linking with libpython*.so

 - Otherwise losts of unresolved errors will result.

Signed-off-by: Willem Jan Withagen <wjw@digiware.nl>

Merge pull request #12181 from SUSE/wip-18037

osd: improve error message when FileStore op fails due to EPERM

Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Joao Eduardo Luis <joao@suse.de>

Adding Arch Linux support

Signed-off-by: Jamin W. Collins <jamin.collins@gmail.com>

ceph-disk: convert none str to str before printing it

Error('somethings goes wrong', e) is thrown if exception `e` is caught
in ceph-disk, where e is not a string. so we can not just concat it in
Error's __str__(). so cast it to str before doing so.

introduced by d0e29c7

Fixes: http://tracker.ceph.com/issues/18371
Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #12697 from kylinstorage/wip-rbd-bench

rbd: additional validation for 'bench' optional parameters

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #12319 from gaurav36/bug-18125-rbd-mirror-enable-disable

librbd: warning message for mirroring pool option

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #12191 from wjwithagen/wip-wjw-freebsd-radosgw-admin

rgw_file: radosgw-admin can be built under FreeBSD

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #12737 from amirv/refactor

Move code from .h into .cc

Reviewed-by: Haomai Wang <haomai@xsky.com>

src/CMakeLists.txt: Move parse_secret_objs setting within definition block

 - parse_secret_objs is only used when secret.c is actually used
   and that is with CEPHFS or RBD

This removes a Cmake warning on FreeBSD

Signed-off-by: Willem Jan Withagen <wjw@digiware.nl>

cmake: search for Keyutils in default paths and cleanup

This makes it more portable

Remove /usr/include as it is searched by find_path anyway.
Remove unnecessary include that weren't used.

Signed-off-by: Pascal Bach <pascal.bach@nextrem.ch>

Merge pull request #12705 from badone/wip-ceph_test_rados_api_tier-valgrind-fixes

test/librados/tier.cc: Fix valgrind errors

Reviewed-by: Sage Weil <sage@redhat.com>

rbd: warning message for mirroring pool option

When user enable/disable mirroring option on a pool, than
re-enabling/disabling mirroring option should give warning
message if it is already enable/disable.
It should also give warning when changing the mode from pool
to image or vice versa.

Fixes: http://tracker.ceph.com/issues/18125
Reported-by: Jason Dillaman <dillaman@redhat.com>
Signed-off-by: Gaurav Kumar Garg <garg.gaurav52@gmail.com>

Merge pull request #12544 from hjwsm1989/oc

osdc/ObjectCacher: use state instead of get_state()

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #8942 from majianpeng/Finisher-optimize

common/Finisher: Using queue(list<context*>) instead queue(context*).

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #9760 from stiopaa1/msg_messengercc_move

msg/Messenger.cc: add std::move

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #12732 from Liuchang0812/wip-issue-17449

mgr: Make stats period configurable

Reviewed-by: Kefu Chai <kchai@redhat.com>

osd: add asock command to dump the scrub queue

* add a public method to dumps scrubs
* register new command "dump_scrubs" to asock_admin

Fixes: http://tracker.ceph.com/issues/17861
Signed-off-by: liuchang0812 <liuchang0812@gmail.com>

Merge pull request #12777 from wjin/clean_posixworker

msg/async: rm nonused thread variable in posixworker

Reviewed-by: Kefu Chai <kchai@redhat.com>

common: get rid of "warning: ignoring return value of ‘strerror_r’"

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

Merge pull request #12778 from badone/wip-check_new_interval-logging

osd/osd_type: Fix logging output

Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>

Merge pull request #12671 from badone/wip-ceph_test_rados_api_cmd-valgrind-fixes

test/librados/cmd.cc: Fix trivial memory leaks

Reviewed-by: Kefu Chai <kchai@redhat.com>

osd/osd_types: Move comment to more relevant position

The PriorSet constructor does not reference up_thru at all so the
comment makes more sense in pg_interval_t::check_new_interval.

Signed-off-by: Brad Hubbard <bhubbard@redhat.com>

osd/osd_type: Fix logging output

Logging should reflect the actual function name.

Signed-off-by: Brad Hubbard <bhubbard@redhat.com>

Merge pull request #12656 from badone/wip-c_read_operations-leak

test/librados/c_read_operations.cc: Fix trivial memory leak

Reviewed-by: Kefu Chai <kchai@redhat.com>

cmake,modules: remove Findpciaccess.cmake

Now SPDK uses the DPDK's library for PCI device
enumeration, we can remove this dependency

Signed-off-by: optimistyzy <optimistyzy@gmail.com>

Merge pull request #12442 from badone/wip-coverity-librados-TestCase

test/librados: Silence Coverity memory leak warnings

Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

msg/async: rm nonused thread variable in posixworker

Signed-off-by: Wei Jin <wjin.cn@gmail.com>

Merge pull request #12690 from badone/wip-ceph_test_rados_api_snapshots-valgrind-fixes

test/librados/snapshots.cc: Fix memory leak

Reviewed-by: Kefu Chai <kchai@redhat.com>

  include/mempool: fix typo in comments

Signed-off-by: huangjun <hjwsm1989@gmail.com>

Merge pull request #12770 from yuriw/wip-fix-distro-master

qa: fixed distros links

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Added a smoke suite for ceph-ansible

Smoke suite with basic tests for ceph ansible deployment

Signed-off-by: Tamil Muthamizhan <tmuthami@redhat.com>

qa: fixed distros links

Signed-off-by: Yuri Weinstein <yweinste@redhat.com>

Merge pull request #11787 from SUSE/wip-17802

doc: clarify "ceph quorum" syntax

Reviewed-by: Joao Eduardo Luis <joao@suse.de>
Reviewed-by: Kefu Chai <kchai@redhat.com>

osd: improve error message when FileStore op fails due to EPERM

References: http://tracker.ceph.com/issues/18037
Signed-off-by: Nathan Cutler <ncutler@suse.com>

PrimaryLogPG: don't update digests for objects with mismatched names

I've only seen this on one cluster, but let's not issue repops during
scrub on objects where the object_info_t::soid value is not correct.
The cluster in question has been through many different non-release
kernels and osd versions, so the objects presumably came about due to an
old xfs or filestore bug.  They recently became fatal since we made
filestore crash on ENOENT for setattrs.  In the past, the cluster just
silently tolerated them.

http://tracker.ceph.com/issues/18409 is a larger feature to detect these
better and repair them automatically.

Related: http://tracker.ceph.com/issues/18409
Signed-off-by: Samuel Just <sjust@redhat.com>

Merge pull request #12385 from dlan17/master

cmake: explictly call find_package(PythonInterp) first to fix build err

Reviewed-by: Ali Maredia <amaredia@redhat.com>

Merge pull request #12658 from oritwas/wip-rgw-17239

rgw: clear master_zonegroup when reseting RGWPeriodMap

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #12723 from gaurav36/rbd-remove-unused-condition

rbd: remove unused condition within group action handler

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #12509 from SUSE/wip-18262

doc: add verbiage to rbdmap manpage

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #12667 from trociny/wip-librados_test_stub-tmap_udate

test: librados_test_stub: tmap_update: return -ENOENT when removing nonexisent key

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #12712 from liewegas/wip-18366

mon/OSDMonitor: set last_force_op_resend on overlay pool too

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #12717 from hrchu/patch-1

doc: Update keystone.rst

Reviewed-by: Sage Weil <sage@redhat.com>

doc: Update keystone.rst

Make it clear that project/domain options accept name instead of id.

Signed-off-by: Chu, Hua-Rong <hrchu@cht.com.tw>

Merge pull request #12734 from liupan1111/wip-fix-nvme-driver-compile-issue

spdk: upgrade spdk to v16.12

Reviewed-by: Haomai Wang <haomai@xsky.com>