os/filestore: use existing variable to assign newsize.

Signed-off-by: Pan Liu <liupan1111@gmail.com>

Merge pull request #13427 from ly798/update-info

mailmap: Liu Yang affiliation

Reviewed-by: Abhishek Lekshmanan <abhishek@suse.com>
Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>

Merge pull request #13738 from yuyuyu101/wip-fix-ib

msg/async/rdma: fix potential racing connection usage

Reviewed-by: Adir lev <adirl@mellanox.com>

Merge pull request #13655 from Adirl/ulimit

msg/async/rdma: check ulimit

Reviewed-by: Haomai Wang <haomai@xsky.com>

msg/async/rdma: check ulimit

On RDMA, memlock should be unlimited. If it is not then
warn the user

issue: 973654

Change-Id: If1e07722aaca9df88714828ca3f7398a2504e262
Signed-off-by: Sarit Zubakov <saritz@mellanox.com>
Signed-off-by: Adir Lev <adirl@mellanox.com>

Merge pull request #13510 from yuyuyu101/wip-rdma-buffer-size

common/config_opts: improve rdma buffer size to 128k

Merge pull request #13713 from drunkard/master

doc: misc minor fixes

Reviewed-by: Kefu Chai <kchai@redhat.com>

msg/async/rdma: fix incorrect method name

Signed-off-by: Haomai Wang <haomai@xsky.com>

msg/async/rdma: keep qp_conns lock to prevent connection alive

Signed-off-by: Haomai Wang <haomai@xsky.com>

msg/async/rdma: remove unused variables

Signed-off-by: Haomai Wang <haomai@xsky.com>

Merge pull request #13525 from yuyuyu101/wip-infiniband-fork

msg/async/rdma: make Infiniband can be forkable

msg/async/rdma: restart Infiniband resources to handle fork properly

Signed-off-by: Haomai Wang <haomai@xsky.com>

Merge pull request #13666 from Adirl/chunk_size

msg/async/rdma: rename chunk_size to buffer_size

Reviewed-by: Haomai Wang <haomai@xsky.com>

doc: add missed commas

Signed-off-by: Drunkard Zhang <gongfan193@gmail.com>

doc: add mark for quoted content

Signed-off-by: Drunkard Zhang <gongfan193@gmail.com>

doc: command typo fix, s/radosgw_admin/radosgw-admin

Signed-off-by: Drunkard Zhang <gongfan193@gmail.com>

doc: fix aiming of command ping

Signed-off-by: Drunkard Zhang <gongfan193@gmail.com>

doc: hyperlink typo fix in dev/index

The doc compiler complains about this.

Signed-off-by: Drunkard Zhang <gongfan193@gmail.com>

Merge pull request #13680 from yuyuyu101/wip-tx-zerocopy

msg/async/rdma: refactor tx handle flow to get rid of locks

Reviewed-by: Adir lev <adirl@mellanox.com>

Merge pull request #13706 from zmc/wip-openstack-volumes

qa/suites/ceph-deploy: Drop OpenStack volume count

Merge pull request #13657 from badone/wip-rgw-setuid-no-coredump

rgw: set dumpable flag after setuid post ff0e521

Reviewed-by: Matt Benjamin <mbenjami@redhat.com>

Merge pull request #12998 from Liuchang0812/wip-11081

common: fix that $host always expands to localhost instead of actual hostname

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #13651 from rzarzynski/wip-bs-bitmap-nocct-in-entry

os/bluestore: remove CephContext* from BmapEntry.

Reviewed-by: Igor Fedotov <ifedotov@mirantis.com>
Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #13662 from majianpeng/aio-memset

os/aio: remove the redundant memset(struct iocb)

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #13668 from songbaisen/m3

osd: the osd should not share map with others when it is in stopping state

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #13702 from liewegas/wip-trackedop-desc

common/TrackedOp: make TrackedOp::reset_desc() safe

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

qa/suites/ceph-deploy: Drop OpenStack volume count

Looks like we only need two per node, since there is only one OSD per
node, and ceph-deploy wants two disks per OSD to account for the
journal.

Signed-off-by: Zack Cerza <zack@redhat.com>

Merge pull request #13729 from dillaman/wip-librbd-compile-warnings

librbd: eliminate compiler warnings

Reviewed-by: Mykola Golub <mgolub@mirantis.com>

Merge pull request #11995 from SUSE/wip_timeouts_ceph-create-keys

ceph-create-keys: Add connection timeouts.

Reviewed-by: Loic Dachary <ldachary@redhat.com>

msg/async: must destruct ConnectedSocket in the his own thread

Signed-off-by: Haomai Wang <haomai@xsky.com>

msg/async/rdma: refactor RDMAStack to accelerate tx handle

previously Dispatcher thread will poll both rx and tx events, then dispatch
these events to RDMAWorker and RDMAConnectedSocketImpl.

Actually tx event handling is a lightweight task and we make these handling
inline now. rx event dispatching is still working.

Another change is adding tx cq to make event polling separated.

removing lots of codes yet.

Signed-off-by: Haomai Wang <haomai@xsky.com>

librbd: eliminate compiler warnings

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #13343 from liewegas/wip-faster-dispatch

osd: faster dispatch

Reviewed-by: Greg Farnum <gfarnum@redhat.com>

Merge pull request #13557 from prallabh/healthcheck

rgw: Correct the return codes for the health check feature

Reviewed-by: Radoslaw Zarzynski <rzarzynski@mirantis.com>

Merge pull request #13685 from tchaikov/wip-19097

mgr/MgrClient: use unique_ptr for MgrClient::session

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #13716 from Adirl/move_counter

msg/async/rdma: move active_queue_pairs perf counter dec to polling

Reviewed-by: Haomai Wang <haomai@xsky.com>

msg/async/rdma: move active_queue_pairs perf counter dec to polling

removing dead qp's is actually done at polling. if polling is busy then
dead qp will not be removed and active_queue_pair counter is not correct.

issue: 992513

Change-Id: I825e813ce0632fd01f6d29adc87e0e33a2bc13d9
Signed-off-by: DanielBar-On <danielbo@mellanox.com>

rgw: Correct the return codes for the health check feature
Fixes: http://tracker.ceph.com/issues/19025
Signed-off-by: Pavan Rallabhandi <PRallabhandi@walmartlabs.com>

mon/MonClient: do not assert on auth in build_authorizer()

* there is chance that some connections is still trying to authorize
  itself after the MonClient is shut down. do not assert in this case,
  but it is a sign of bug, or bad shutdown sequence, so print a message to
  dout().
* do not use active_con->get_auth() as an alternative to `this->auth` if
  it is not available. because we promote the authorized conn in
  pending_cons as the active_con, and std::swap(active_conn->auth, this->auth)
  with the monc_lock. so there is no point to return active_con->get_auth(),
  as it's always null.

Signed-off-by: Kefu Chai <kchai@redhat.com>

doc: command typo fix

Signed-off-by: Drunkard Zhang <gongfan193@gmail.com>

doc: remove dead doc from index

Signed-off-by: Drunkard Zhang <gongfan193@gmail.com>

doc: hyperlink fix

Signed-off-by: Drunkard Zhang <gongfan193@gmail.com>

Merge pull request #13703 from linuxbox2/wip-is-linked

rgw_file:  use fh_hook::is_linked() to check residence
verified by hand f23

rgw_file:  use fh_hook::is_linked() to check residence

Previously we assumed that !deleted handles were resident--there
is an observed case where a !deleted handle is !linked.  Since
we currently use safe_link mode, an is_linked() check is
available, and exhaustive.

Fixes: http://tracker.ceph.com/issues/19111
Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>

Merge pull request #13671 from liewegas/wip-filestore-omap

os/filestore: debug which omap keys are set

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #13546 from cbodley/wip-19019

rgw: RGWMetaSyncShardControlCR retries with backoff on all error codes
tested by-had, using fault injection

Merge pull request #13415 from ifed01/wip-fix-storetest

test/store_test: add deferred test case setup to support explicit min…

Reviewed-by: Varada Kari <varada.kari@sandisk.com>
Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #13476 from songbaisen/s18

osd: small clear up and optimize on _recover_now and should_share_map function

Reviewed-by: Kefu Chai <kchai@redhat.com>

common/TrackedOp: make TrackedOp::reset_desc() safe

It is possible for a reset_desc() call to clear the desc char* while
get_desc() is executing such that it will return a nullptr to the caller.
This can lead to bad results, like a crash in std::string() (which does
not like to take null).

Fix this by not clearing desc.  Instead, set a separate flag to indicate
that desc should be (safely) rebuilt on the next get_desc() call.

Fixes: http://tracker.ceph.com/issues/19110
Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #13686 from wjwithagen/wip-wjw-fsx-linux-2

test: Need to exclude the fsx executable also on FreeBSD

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #13523 from wjwithagen/wip-wjw-clang-mempool

include/mempool.h: fix Clangs complaint about types

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #13659 from majianpeng/compiler-warning

common/EventTrace: fix compiler warning.

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #13667 from jcsp/wip-mgr-misc

mgr: use unique_ptr for MgrStandby::active_mgr

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #13669 from jcsp/wip-mgr-get-version

mgr: add a get_version to the python interface

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #13692 from Liuchang0812/wip-add-override-in-auth-headers

common/auth: add override in headers

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #13591 from batrick/mds-static-counters

mds: automate MDS object count tracking

Reviewed-by: John Spray <john.spray@redhat.com>

Merge pull request #13581 from jcsp/wip-mdsmonitor-refactor

mon: refactor MDSMonitor command handling

Reviewed-by: Yan, Zheng <zyan@redhat.com>

Merge pull request #13532 from ukernel/wip-18995

ceph_fuse: fix daemonization when pid file is non-empty

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #13464 from jcsp/wip-sessionmap-ioerror

mds: fix IO error handling in SessionMap

Reviewed-by: Yan, Zheng <zyan@redhat.com>

Merge pull request #13282 from jcsp/wip-fuse-mount-teardown

tasks/cephfs: tear down on mount() failure

Reviewed-by: Yan, Zheng <zyan@redhat.com>

Merge pull request #13676 from yehudasa/wip-19096

rgw: don't init rgw_obj from rgw_obj_key when it's incorrect to do so
Reviewed-by: Orit Wasserman <owasserm@redhat.com>

Merge pull request #13600 from dachary/wip-17821-revert

ceph-disk: Revert "Revert "change get_dmcrypt_key test to support different cluster name""

Reviewed-by: Kefu Chai <kchai@redhat.com>

common/auth: add override in headers

Signed-off-by: liuchang0812 <liuchang0812@gmail.com>

src/test/librbd/CMakeLists.txt: Need to exclude the fsx executable also on FreeBSD

Signed-off-by: Willem Jan Withagen <wjw@digiware.nl>

Merge pull request #13682 from tahia-khan/wip-doc-architecture-rados-hyperlink

doc: fixes a broken hyperlink to RADOS paper in architecture

Reviewed-by: Kefu Chai <kchai@redhat.com>

mgr/MgrClient: use unique_ptr for MgrClient::session

* use unique_ptr<MgrSessionState> for MgrClient::session, so we don't leak it.
* also reset it in MgrClient::shutdown(), so the connection held by session can
  be released properly, and hence the MonClient will not be asked for
  an AuthAuthorizer after being shut down.

Fixes: http://tracker.ceph.com/issues/19097
Signed-off-by: Kefu Chai <kchai@redhat.com>

doc: fixes a broken hyperlink to RADOS paper in architecture

Signed-off-by: Tahia Khan <tahia.khan@gmail.com>

Merge pull request #13138 from liewegas/wip-vstart

vstart: clean up usage a bit

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #13663 from tchaikov/wip-ceph-git-for-workunit

qa/tasks/workunit: use ceph.git as an alternative of ceph-ci.git for cloning workunit

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #13650 from trociny/wip-entity_addr-parse

msg: end parameter in entity_addr_t::parse is optional

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #13630 from wjwithagen/wip-wjw-fsx-linux

test/librbd/CMakeLists.txt: ceph_test_librbd_fsx requires linux includes/libs

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
Reviewed-by: Brad Hubbard <bhubbard@redhat.com>

Merge pull request #13196 from vshankar/rbd-ut-fix

test: fix rbd unit test cases w/ striping feature

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #12645 from guihecheng/wip-rbd-writesame-api

librbd: add writesame API

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

rgw: RGWMetaSyncShardControlCR retries with backoff on all error codes

RGWBackoffControlCR only treats EBUSY and EAGAIN as 'temporary' error
codes, with all other errors being fatal when exit_on_error is set

to RGWMetaSyncShardControlCR, a 'fatal' error means that no further sync
is possible on that shard until the gateway restarts

this changes RGWMetaSyncShardControlCR to set exit_on_error to false, so
that it will continue to retry with backoff no matter what error code it
gets

Fixes: http://tracker.ceph.com/issues/19019
Signed-off-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #13672 from zmc/wip-openstack-volumes

qa/suites/{ceph-ansible,rest}: OpenStack volumes

vstart.sh: set host for mgr.* conf sections

This makes init-ceph work.

Signed-off-by: Sage Weil <sage@redhat.com>

rgw: don't init rgw_obj from rgw_obj_key when it's incorrect to do so

Fixes: http://tracker.ceph.com/issues/19096
rgw_obj_key currently deals with the bucket index key, and not
representing a (name, instance, ns) tupple. Need to initialize
it in two steps.

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

Merge pull request #13675 from liewegas/wip-words

doc/release-notes: avoid 'production-ready' in describing kraken

Reviewed-by: Ian Colle <icolle@redhat.com>

os/aio: remove the redundant memset(struct iocb)

In fact, io_prep_read/write/pwritev/preadv do the memset(struct iocb).
So no need do it.

Signed-off-by: Jianpeng Ma <jianpeng.ma@intel.com>

Merge pull request #13547 from liewegas/wip-kill-copy-get-classic

osd: remove copy-get-classic

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #13574 from liewegas/wip-mon-mapping

mon/OSDMonitor: make mapping job behave if mon_osd_prime_pg_temp = false

Reviewed-by: Kefu Chai <kchai@redhat.com>

common/EventTrace: fix compiler warning.

/mnt/ceph/src/osd/OSD.cc: In member function ‘void
OSD::dequeue_op(PGRef, OpRequestRef, ThreadPool::TPHandle&)’:
/mnt/ceph/src/osd/OSD.cc:9112:42: error: invalid conversion from ‘const
Message*’ to ‘Message*’ [-fpermissive]
   OID_EVENT_TRACE_WITH_MSG(op->get_req(), "DEQUEUE_OP_BEGIN", false);
                                          ^
In file included from /mnt/ceph/src/osd/OSD.h:57:0,
                 from /mnt/ceph/src/osd/OSD.cc:38:
/mnt/ceph/src/common/EventTrace.h:72:15: note:   initializing argument 1
of ‘static void EventTrace::trace_oid_event(Message*, const char*, const
char*, const char*, int, bool)’
   static void trace_oid_event(Message *m, const char *event, const char
*file,
               ^
/mnt/ceph/src/osd/OSD.cc:9158:42: error: invalid conversion from ‘const
Message*’ to ‘Message*’ [-fpermissive]
   OID_EVENT_TRACE_WITH_MSG(op->get_req(), "DEQUEUE_OP_END", true);
                                          ^
In file included from /mnt/ceph/src/osd/OSD.h:57:0,
                 from /mnt/ceph/src/osd/OSD.cc:38:
/mnt/ceph/src/common/EventTrace.h:72:15: note:   initializing argument 1
of ‘static void EventTrace::trace_oid_event(Message*, const char*, const
char*, const char*, int, bool)’
   static void trace_oid_event(Message *m, const char *event, const char
*file,
               ^
src/osd/CMakeFiles/osd.dir/build.make:62: recipe for target
'src/osd/CMakeFiles/osd.dir/OSD.cc.o' failed

Signed-off-by: Jianpeng Ma <jianpeng.ma@intel.com>

doc/release-notes: choose terms carefully

Production-ready is in the eye of the beholder.

Signed-off-by: Sage Weil <sage@redhat.com>

qa/suites/rest: Openstack volumes

Signed-off-by: Zack Cerza <zack@redhat.com>

qa/suites/ceph-ansible: Openstack volumes

Signed-off-by: Zack Cerza <zack@redhat.com>

os/filestore: debug which omap keys are set

This may help us find http://tracker.ceph.com/issues/19067

Signed-off-by: Sage Weil <sage@redhat.com>

test/CMakeLists.txt: include ceph-mgr

Signed-off-by: Sage Weil <sage@redhat.com>

vstart.sh: fix init-ceph stop on vstart

Do this before the old ceph.conf is deleted!

Signed-off-by: Sage Weil <sage@redhat.com>

vstart.sh: fix init-ceph path in cmake environment

Signed-off-by: Sage Weil <sage@redhat.com>

ceph_common.sh: add mgr support to init-ceph

This is mainly for the benefit of vstart.

Signed-off-by: Sage Weil <sage@redhat.com>

stop.sh: add 'mgr' arg

Signed-off-by: Sage Weil <sage@redhat.com>

vstart.sh: drop -r arg for radosgw, and use RGW=n instead

Simpler!  The start_* vars are all stupid.

Signed-off-by: Sage Weil <sage@redhat.com>

vstart.sh: respect CEPH_NUM_RGW

Signed-off-by: Sage Weil <sage@redhat.com>

vstart.sh: start 1 ceph-mgr by default

Signed-off-by: Sage Weil <sage@redhat.com>

vstart.sh: default to -n ; add -N | --not-new for restart case

I never don't do -n; I'm guessing others are in the same boat.

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #13665 from xiexingguo/wip-fix-ec-op-out

osd/ECBackend: drop duplicated pending_commit field from << operator

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #13661 from dachary/wip-crush-doc

doc: crush typo in algorithm description

Reviewed-by: Kefu Chai <kchai@redhat.com>

osd: the osd should not share map with others when it is in stopping state.

     the osd may not share its map with others when the osd is in stopping state.
     because it may from unhealthy to stopping.

Signed-off-by: song baisen <song.baisen@zte.com.cn>

mgr: add a get_version to the python interface

We had the metadata for all the remote services,
but no quick way for the python module to
ask "what version is the process I'm running in?"

Signed-off-by: John Spray <john.spray@redhat.com>

test/store_test: add deferred test case setup to support explicit min_alloc_size specification.

Fixes: #18857
Signed-off-by: Igor Fedotov <ifedotov@mirantis.com>