build/ops: add psmisc dependency to ceph-base (deb and rpm)

Fixes: http://tracker.ceph.com/issues/19129
Signed-off-by: Nathan Cutler <ncutler@suse.com>

Merge pull request #13716 from Adirl/move_counter

msg/async/rdma: move active_queue_pairs perf counter dec to polling

Reviewed-by: Haomai Wang <haomai@xsky.com>

msg/async/rdma: move active_queue_pairs perf counter dec to polling

removing dead qp's is actually done at polling. if polling is busy then
dead qp will not be removed and active_queue_pair counter is not correct.

issue: 992513

Change-Id: I825e813ce0632fd01f6d29adc87e0e33a2bc13d9
Signed-off-by: DanielBar-On <danielbo@mellanox.com>

Merge pull request #13703 from linuxbox2/wip-is-linked

rgw_file:  use fh_hook::is_linked() to check residence
verified by hand f23

rgw_file:  use fh_hook::is_linked() to check residence

Previously we assumed that !deleted handles were resident--there
is an observed case where a !deleted handle is !linked.  Since
we currently use safe_link mode, an is_linked() check is
available, and exhaustive.

Fixes: http://tracker.ceph.com/issues/19111
Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>

Merge pull request #13671 from liewegas/wip-filestore-omap

os/filestore: debug which omap keys are set

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #13546 from cbodley/wip-19019

rgw: RGWMetaSyncShardControlCR retries with backoff on all error codes
tested by-had, using fault injection

Merge pull request #13415 from ifed01/wip-fix-storetest

test/store_test: add deferred test case setup to support explicit min…

Reviewed-by: Varada Kari <varada.kari@sandisk.com>
Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #13476 from songbaisen/s18

osd: small clear up and optimize on _recover_now and should_share_map function

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #13686 from wjwithagen/wip-wjw-fsx-linux-2

test: Need to exclude the fsx executable also on FreeBSD

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #13523 from wjwithagen/wip-wjw-clang-mempool

include/mempool.h: fix Clangs complaint about types

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #13659 from majianpeng/compiler-warning

common/EventTrace: fix compiler warning.

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #13667 from jcsp/wip-mgr-misc

mgr: use unique_ptr for MgrStandby::active_mgr

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #13669 from jcsp/wip-mgr-get-version

mgr: add a get_version to the python interface

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #13692 from Liuchang0812/wip-add-override-in-auth-headers

common/auth: add override in headers

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #13591 from batrick/mds-static-counters

mds: automate MDS object count tracking

Reviewed-by: John Spray <john.spray@redhat.com>

Merge pull request #13581 from jcsp/wip-mdsmonitor-refactor

mon: refactor MDSMonitor command handling

Reviewed-by: Yan, Zheng <zyan@redhat.com>

Merge pull request #13532 from ukernel/wip-18995

ceph_fuse: fix daemonization when pid file is non-empty

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #13464 from jcsp/wip-sessionmap-ioerror

mds: fix IO error handling in SessionMap

Reviewed-by: Yan, Zheng <zyan@redhat.com>

Merge pull request #13282 from jcsp/wip-fuse-mount-teardown

tasks/cephfs: tear down on mount() failure

Reviewed-by: Yan, Zheng <zyan@redhat.com>

Merge pull request #13676 from yehudasa/wip-19096

rgw: don't init rgw_obj from rgw_obj_key when it's incorrect to do so
Reviewed-by: Orit Wasserman <owasserm@redhat.com>

Merge pull request #13600 from dachary/wip-17821-revert

ceph-disk: Revert "Revert "change get_dmcrypt_key test to support different cluster name""

Reviewed-by: Kefu Chai <kchai@redhat.com>

common/auth: add override in headers

Signed-off-by: liuchang0812 <liuchang0812@gmail.com>

src/test/librbd/CMakeLists.txt: Need to exclude the fsx executable also on FreeBSD

Signed-off-by: Willem Jan Withagen <wjw@digiware.nl>

Merge pull request #13682 from tahia-khan/wip-doc-architecture-rados-hyperlink

doc: fixes a broken hyperlink to RADOS paper in architecture

Reviewed-by: Kefu Chai <kchai@redhat.com>

doc: fixes a broken hyperlink to RADOS paper in architecture

Signed-off-by: Tahia Khan <tahia.khan@gmail.com>

Merge pull request #13138 from liewegas/wip-vstart

vstart: clean up usage a bit

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #13663 from tchaikov/wip-ceph-git-for-workunit

qa/tasks/workunit: use ceph.git as an alternative of ceph-ci.git for cloning workunit

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #13650 from trociny/wip-entity_addr-parse

msg: end parameter in entity_addr_t::parse is optional

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #13630 from wjwithagen/wip-wjw-fsx-linux

test/librbd/CMakeLists.txt: ceph_test_librbd_fsx requires linux includes/libs

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
Reviewed-by: Brad Hubbard <bhubbard@redhat.com>

Merge pull request #13196 from vshankar/rbd-ut-fix

test: fix rbd unit test cases w/ striping feature

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #12645 from guihecheng/wip-rbd-writesame-api

librbd: add writesame API

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

rgw: RGWMetaSyncShardControlCR retries with backoff on all error codes

RGWBackoffControlCR only treats EBUSY and EAGAIN as 'temporary' error
codes, with all other errors being fatal when exit_on_error is set

to RGWMetaSyncShardControlCR, a 'fatal' error means that no further sync
is possible on that shard until the gateway restarts

this changes RGWMetaSyncShardControlCR to set exit_on_error to false, so
that it will continue to retry with backoff no matter what error code it
gets

Fixes: http://tracker.ceph.com/issues/19019
Signed-off-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #13672 from zmc/wip-openstack-volumes

qa/suites/{ceph-ansible,rest}: OpenStack volumes

vstart.sh: set host for mgr.* conf sections

This makes init-ceph work.

Signed-off-by: Sage Weil <sage@redhat.com>

rgw: don't init rgw_obj from rgw_obj_key when it's incorrect to do so

Fixes: http://tracker.ceph.com/issues/19096
rgw_obj_key currently deals with the bucket index key, and not
representing a (name, instance, ns) tupple. Need to initialize
it in two steps.

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

Merge pull request #13675 from liewegas/wip-words

doc/release-notes: avoid 'production-ready' in describing kraken

Reviewed-by: Ian Colle <icolle@redhat.com>

Merge pull request #13547 from liewegas/wip-kill-copy-get-classic

osd: remove copy-get-classic

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #13574 from liewegas/wip-mon-mapping

mon/OSDMonitor: make mapping job behave if mon_osd_prime_pg_temp = false

Reviewed-by: Kefu Chai <kchai@redhat.com>

common/EventTrace: fix compiler warning.

/mnt/ceph/src/osd/OSD.cc: In member function ‘void
OSD::dequeue_op(PGRef, OpRequestRef, ThreadPool::TPHandle&)’:
/mnt/ceph/src/osd/OSD.cc:9112:42: error: invalid conversion from ‘const
Message*’ to ‘Message*’ [-fpermissive]
   OID_EVENT_TRACE_WITH_MSG(op->get_req(), "DEQUEUE_OP_BEGIN", false);
                                          ^
In file included from /mnt/ceph/src/osd/OSD.h:57:0,
                 from /mnt/ceph/src/osd/OSD.cc:38:
/mnt/ceph/src/common/EventTrace.h:72:15: note:   initializing argument 1
of ‘static void EventTrace::trace_oid_event(Message*, const char*, const
char*, const char*, int, bool)’
   static void trace_oid_event(Message *m, const char *event, const char
*file,
               ^
/mnt/ceph/src/osd/OSD.cc:9158:42: error: invalid conversion from ‘const
Message*’ to ‘Message*’ [-fpermissive]
   OID_EVENT_TRACE_WITH_MSG(op->get_req(), "DEQUEUE_OP_END", true);
                                          ^
In file included from /mnt/ceph/src/osd/OSD.h:57:0,
                 from /mnt/ceph/src/osd/OSD.cc:38:
/mnt/ceph/src/common/EventTrace.h:72:15: note:   initializing argument 1
of ‘static void EventTrace::trace_oid_event(Message*, const char*, const
char*, const char*, int, bool)’
   static void trace_oid_event(Message *m, const char *event, const char
*file,
               ^
src/osd/CMakeFiles/osd.dir/build.make:62: recipe for target
'src/osd/CMakeFiles/osd.dir/OSD.cc.o' failed

Signed-off-by: Jianpeng Ma <jianpeng.ma@intel.com>

doc/release-notes: choose terms carefully

Production-ready is in the eye of the beholder.

Signed-off-by: Sage Weil <sage@redhat.com>

qa/suites/rest: Openstack volumes

Signed-off-by: Zack Cerza <zack@redhat.com>

qa/suites/ceph-ansible: Openstack volumes

Signed-off-by: Zack Cerza <zack@redhat.com>

os/filestore: debug which omap keys are set

This may help us find http://tracker.ceph.com/issues/19067

Signed-off-by: Sage Weil <sage@redhat.com>

test/CMakeLists.txt: include ceph-mgr

Signed-off-by: Sage Weil <sage@redhat.com>

vstart.sh: fix init-ceph stop on vstart

Do this before the old ceph.conf is deleted!

Signed-off-by: Sage Weil <sage@redhat.com>

vstart.sh: fix init-ceph path in cmake environment

Signed-off-by: Sage Weil <sage@redhat.com>

ceph_common.sh: add mgr support to init-ceph

This is mainly for the benefit of vstart.

Signed-off-by: Sage Weil <sage@redhat.com>

stop.sh: add 'mgr' arg

Signed-off-by: Sage Weil <sage@redhat.com>

vstart.sh: drop -r arg for radosgw, and use RGW=n instead

Simpler!  The start_* vars are all stupid.

Signed-off-by: Sage Weil <sage@redhat.com>

vstart.sh: respect CEPH_NUM_RGW

Signed-off-by: Sage Weil <sage@redhat.com>

vstart.sh: start 1 ceph-mgr by default

Signed-off-by: Sage Weil <sage@redhat.com>

vstart.sh: default to -n ; add -N | --not-new for restart case

I never don't do -n; I'm guessing others are in the same boat.

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #13665 from xiexingguo/wip-fix-ec-op-out

osd/ECBackend: drop duplicated pending_commit field from << operator

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #13661 from dachary/wip-crush-doc

doc: crush typo in algorithm description

Reviewed-by: Kefu Chai <kchai@redhat.com>

mgr: add a get_version to the python interface

We had the metadata for all the remote services,
but no quick way for the python module to
ask "what version is the process I'm running in?"

Signed-off-by: John Spray <john.spray@redhat.com>

test/store_test: add deferred test case setup to support explicit min_alloc_size specification.

Fixes: #18857
Signed-off-by: Igor Fedotov <ifedotov@mirantis.com>

mgr: use unique_ptr for MgrStandby::active_mgr

It should always have been one of these.

Signed-off-by: John Spray <john.spray@redhat.com>

Merge pull request #13509 from yuyuyu101/wip-tx-zerocopy

msg/async/rdma: cleanup

Reviewed-by: Adir lev <adirl@mellanox.com>

Merge pull request #13531 from LiumxNL/fix-osd-utilization

mon/OSDMonitor: fix dividing by zero in OSDUtilizationDumper

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #13526 from ganeshmaharaj/tp_library_fixes

debian: Add missing tp files in deb packaging

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #13553 from yonghengdexin735/wip-zzz-dbev

os/bluestore/BlueStore.cc:remove unuse code in _open_bdev()

Reviewed-by: Igor Fedotov <ifedotov@mirantis.com>

Merge pull request #13554 from songbaisen/s19

mon:  crush straw_calc_version value is 0 or 1 not 0 to 2.

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #13653 from rzarzynski/wip-bs-bitmap-fast-exhausted-check

os/bluestore: the exhausted check in BitMapZone can be lock-less.

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #13656 from tchaikov/wip-19015

mon/MonClient: discard stray messages from non-acitve conns

Reviewed-by: Sage Weil <sage@redhat.com>

osd/ECBackend: drop duplicated pending_commit field from << operator

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

qa/tasks/workunit: use ceph.git as an alternative of ceph-ci.git for workunit repo

if we run upgrade test, where, for example, "jewel" is not in
ceph-ci.git repo, we should check ceph.git to clone the workunits.

Signed-off-by: Kefu Chai <kchai@redhat.com>

msg/async/rdma: accelerate tx/rx buffer ownership lookup

Signed-off-by: Haomai Wang <haomai@xsky.com>

msg/async/rdma: cleanup

Signed-off-by: Haomai Wang <haomai@xsky.com>

doc: crush typo in algorithm description

Signed-off-by: Loic Dachary <ldachary@redhat.com>

msg/async/rdma: cleanup Infiniband, remove unused functions

Signed-off-by: Haomai Wang <haomai@xsky.com>

Merge pull request #13603 from Adirl/fix_compile

msg/async/rdma: Fix broken compilation

Reviewed-by: Haomai Wang <haomai@xsky.com>

Merge pull request #13652 from liupan1111/wip-cleanup-rbd-nbd

rbd-nbd: fix a typo "moudle"

Reviewed-by: Mykola Golub <mgolub@mirantis.com>

auth/cephx: print challenges in hex

this matches the CephxServiceHandler::handle_request(), hence would be
easier for debugging.

Signed-off-by: Kefu Chai <kchai@redhat.com>

mon/MonClient: discard stray messages from non-acitve conns

Fixes: http://tracker.ceph.com/issues/19015
Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #13227 from ukernel/wip-multimds-misc

mds: misc multimds fixes

Reviewed-by: John Spray <john.spray@redhat.com>

Merge pull request #13648 from tchaikov/wip-doc-osd-agent-options

doc: document `osd-agent-{max,low}-ops` options

Reviewed-by: Zhiqiang Wang <zhiqiang@xsky.com>
Reviewed-by: Brad Hubbard <bhubbard@redhat.com>

rbd-nbd: fix a typo "moudle"

Signed-off-by: Pan Liu <liupan1111@gmail.com>

doc: document `osd-agent-{max,low}-ops` options

Signed-off-by: Kefu Chai <kchai@redhat.com>

bluestore: the exhausted check in BitMapZone can be lock-less.

Before the patch BitMapZone::is_exhausted() required from its
callers to acquire appropriate lock. However, fulfilling this
condition is not really necessary to use the method correctly
while it can significantly hurt performance.

The change allows BitMapAreaLeaf::child_check_n_lock() to not
acquire the lock while examining zones for being exhausted.

Signed-off-by: Radoslaw Zarzynski <rzarzynski@mirantis.com>

Merge pull request #13649 from liewegas/wip-ceph-scrub-debug

qa/tasks/ceph.py: debug which pgs aren't scrubbing

Reviewed-by: Brad Hubbard <bhubbard@redhat.com>

msg: end parameter in entity_addr_t::parse is optional

Signed-off-by: Mykola Golub <mgolub@mirantis.com>

qa/tasks/ceph.py: debug which pgs aren't scrubbing

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #13478 from xiaoxichen/fix_osdc_perfcounter

osdc: fix osdc_osd_seesion perf counter.

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #13626 from wjwithagen/wip-wjw-freebsd-enodata-fixed

do_freebsd.sh: Remove ENODATA requirement

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #13640 from zmc/wip-fs-openstack

qa/suites/fs: Add openstack volume configuration

Merge pull request #13619 from kylinstorage/cleanup_bitallocator

BitAllocator: delete useless codes

Reviewed-by: Brad Hubbard <bhubbard@redhat.com>

qa/suites/fs: Add openstack volume configuration

Signed-off-by: Zack Cerza <zack@redhat.com>

Merge pull request #13594 from athanatos/wip-snap-trim-sleep

osd: add snap trim reservation and re-implement osd_snap_trim_sleep

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #13570 from athanatos/wip-18937

osd: don't use ORDERSNAP for flush; always request/send ondisk ack

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Sage Weil <sage@redhat.com>

test/librbd/CMakeLists.txt: ceph_test_librbd_fsx requires linux includes/libs

Signed-off-by: Willem Jan Withagen <wjw@digiware.nl>

Merge pull request #13637 from zmc/wip-openstack-volumes

qa/suites/{knfs,hadoop,samba}: OpenStack volume configuration

Merge pull request #13634 from zmc/wip-kcephfs-openstack

qa/suites/kcephfs: Openstack volume configuration

qa/suites/samba: Openstack volume configuration

Signed-off-by: Zack Cerza <zack@redhat.com>

qa/suites/hadoop: Openstack volume configuration

Signed-off-by: Zack Cerza <zack@redhat.com>

qa/suites/knfs: Add openstack volume configuration

Signed-off-by: Zack Cerza <zack@redhat.com>

qa/suites/kcephfs: Openstack volume configuration

Signed-off-by: Zack Cerza <zack@redhat.com>

osd,osdc: eliminate FLAG_ONDISK and helpers

The objecter actually always needs to get a response in order to
be able to not continually resend ops (even if the caller didn't
provide a callback).  Thus, it makes no sense for an MOSDOp to
ever not have FLAG_ONDISK set.  Therefore, we'll just remove the
helper and assume it's always there (it's safe to send a response
the client didn't ask for, the error paths already do that).  On
the Objecter side, we'll just unconditionally fill in ONDISK for
the benefit of pre-luminous OSDs.

Fixes: http://tracker.ceph.com/issues/18961
Signed-off-by: Samuel Just <sjust@redhat.com>

PrimaryLogPG::start_flush: don't use ORDERSNAP, eliminate the second delete

I think that whole thing was a misguided attempt to avoid deleting head
if it exists in the base tier (in reality it doesn't matter since head
would have to be logically dirty and anything we actually care about
would be preserved by sending a new enough seq to cause a clone).

Introduced in 4843fd510b33a71999cdf9c2cfa2b4c318fa80fd, but the real
logical error happened in f3df50188b54e60e28a276762c370477538bbb07.

I suggest never backporting this patch.  If you want to try, keep in
mind that the last version didn't turn up as busted for 2 years.

Fixes: f3df50188b54e60e28a276762c370477538bbb07
Signed-off-by: Samuel Just <sjust@redhat.com>

Merge pull request #13631 from ceph/wip-krbd-openstack

qa/suites/krbd: Add openstack volume configuration