rbd-nbd: only try to load nbd module once when open nbd device, during do_map operation.

Signed-off-by: Pan Liu <liupan1111@gmail.com>

rbd-nbd: warn when kernel parameters are ignored

Fixes: http://tracker.ceph.com/issues/19108
Signed-off-by: Pan Liu <liupan1111@gmail.com>

Merge pull request #13138 from liewegas/wip-vstart

vstart: clean up usage a bit

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #13663 from tchaikov/wip-ceph-git-for-workunit

qa/tasks/workunit: use ceph.git as an alternative of ceph-ci.git for cloning workunit

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #13650 from trociny/wip-entity_addr-parse

msg: end parameter in entity_addr_t::parse is optional

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #13630 from wjwithagen/wip-wjw-fsx-linux

test/librbd/CMakeLists.txt: ceph_test_librbd_fsx requires linux includes/libs

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
Reviewed-by: Brad Hubbard <bhubbard@redhat.com>

Merge pull request #13196 from vshankar/rbd-ut-fix

test: fix rbd unit test cases w/ striping feature

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #12645 from guihecheng/wip-rbd-writesame-api

librbd: add writesame API

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #13672 from zmc/wip-openstack-volumes

qa/suites/{ceph-ansible,rest}: OpenStack volumes

vstart.sh: set host for mgr.* conf sections

This makes init-ceph work.

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #13675 from liewegas/wip-words

doc/release-notes: avoid 'production-ready' in describing kraken

Reviewed-by: Ian Colle <icolle@redhat.com>

Merge pull request #13547 from liewegas/wip-kill-copy-get-classic

osd: remove copy-get-classic

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #13574 from liewegas/wip-mon-mapping

mon/OSDMonitor: make mapping job behave if mon_osd_prime_pg_temp = false

Reviewed-by: Kefu Chai <kchai@redhat.com>

doc/release-notes: choose terms carefully

Production-ready is in the eye of the beholder.

Signed-off-by: Sage Weil <sage@redhat.com>

qa/suites/rest: Openstack volumes

Signed-off-by: Zack Cerza <zack@redhat.com>

qa/suites/ceph-ansible: Openstack volumes

Signed-off-by: Zack Cerza <zack@redhat.com>

test/CMakeLists.txt: include ceph-mgr

Signed-off-by: Sage Weil <sage@redhat.com>

vstart.sh: fix init-ceph stop on vstart

Do this before the old ceph.conf is deleted!

Signed-off-by: Sage Weil <sage@redhat.com>

vstart.sh: fix init-ceph path in cmake environment

Signed-off-by: Sage Weil <sage@redhat.com>

ceph_common.sh: add mgr support to init-ceph

This is mainly for the benefit of vstart.

Signed-off-by: Sage Weil <sage@redhat.com>

stop.sh: add 'mgr' arg

Signed-off-by: Sage Weil <sage@redhat.com>

vstart.sh: drop -r arg for radosgw, and use RGW=n instead

Simpler!  The start_* vars are all stupid.

Signed-off-by: Sage Weil <sage@redhat.com>

vstart.sh: respect CEPH_NUM_RGW

Signed-off-by: Sage Weil <sage@redhat.com>

vstart.sh: start 1 ceph-mgr by default

Signed-off-by: Sage Weil <sage@redhat.com>

vstart.sh: default to -n ; add -N | --not-new for restart case

I never don't do -n; I'm guessing others are in the same boat.

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #13665 from xiexingguo/wip-fix-ec-op-out

osd/ECBackend: drop duplicated pending_commit field from << operator

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #13661 from dachary/wip-crush-doc

doc: crush typo in algorithm description

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #13509 from yuyuyu101/wip-tx-zerocopy

msg/async/rdma: cleanup

Reviewed-by: Adir lev <adirl@mellanox.com>

Merge pull request #13531 from LiumxNL/fix-osd-utilization

mon/OSDMonitor: fix dividing by zero in OSDUtilizationDumper

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #13526 from ganeshmaharaj/tp_library_fixes

debian: Add missing tp files in deb packaging

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #13553 from yonghengdexin735/wip-zzz-dbev

os/bluestore/BlueStore.cc:remove unuse code in _open_bdev()

Reviewed-by: Igor Fedotov <ifedotov@mirantis.com>

Merge pull request #13554 from songbaisen/s19

mon:  crush straw_calc_version value is 0 or 1 not 0 to 2.

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #13653 from rzarzynski/wip-bs-bitmap-fast-exhausted-check

os/bluestore: the exhausted check in BitMapZone can be lock-less.

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #13656 from tchaikov/wip-19015

mon/MonClient: discard stray messages from non-acitve conns

Reviewed-by: Sage Weil <sage@redhat.com>

osd/ECBackend: drop duplicated pending_commit field from << operator

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

qa/tasks/workunit: use ceph.git as an alternative of ceph-ci.git for workunit repo

if we run upgrade test, where, for example, "jewel" is not in
ceph-ci.git repo, we should check ceph.git to clone the workunits.

Signed-off-by: Kefu Chai <kchai@redhat.com>

msg/async/rdma: accelerate tx/rx buffer ownership lookup

Signed-off-by: Haomai Wang <haomai@xsky.com>

msg/async/rdma: cleanup

Signed-off-by: Haomai Wang <haomai@xsky.com>

doc: crush typo in algorithm description

Signed-off-by: Loic Dachary <ldachary@redhat.com>

msg/async/rdma: cleanup Infiniband, remove unused functions

Signed-off-by: Haomai Wang <haomai@xsky.com>

Merge pull request #13603 from Adirl/fix_compile

msg/async/rdma: Fix broken compilation

Reviewed-by: Haomai Wang <haomai@xsky.com>

Merge pull request #13652 from liupan1111/wip-cleanup-rbd-nbd

rbd-nbd: fix a typo "moudle"

Reviewed-by: Mykola Golub <mgolub@mirantis.com>

auth/cephx: print challenges in hex

this matches the CephxServiceHandler::handle_request(), hence would be
easier for debugging.

Signed-off-by: Kefu Chai <kchai@redhat.com>

mon/MonClient: discard stray messages from non-acitve conns

Fixes: http://tracker.ceph.com/issues/19015
Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #13227 from ukernel/wip-multimds-misc

mds: misc multimds fixes

Reviewed-by: John Spray <john.spray@redhat.com>

Merge pull request #13648 from tchaikov/wip-doc-osd-agent-options

doc: document `osd-agent-{max,low}-ops` options

Reviewed-by: Zhiqiang Wang <zhiqiang@xsky.com>
Reviewed-by: Brad Hubbard <bhubbard@redhat.com>

rbd-nbd: fix a typo "moudle"

Signed-off-by: Pan Liu <liupan1111@gmail.com>

doc: document `osd-agent-{max,low}-ops` options

Signed-off-by: Kefu Chai <kchai@redhat.com>

bluestore: the exhausted check in BitMapZone can be lock-less.

Before the patch BitMapZone::is_exhausted() required from its
callers to acquire appropriate lock. However, fulfilling this
condition is not really necessary to use the method correctly
while it can significantly hurt performance.

The change allows BitMapAreaLeaf::child_check_n_lock() to not
acquire the lock while examining zones for being exhausted.

Signed-off-by: Radoslaw Zarzynski <rzarzynski@mirantis.com>

Merge pull request #13649 from liewegas/wip-ceph-scrub-debug

qa/tasks/ceph.py: debug which pgs aren't scrubbing

Reviewed-by: Brad Hubbard <bhubbard@redhat.com>

msg: end parameter in entity_addr_t::parse is optional

Signed-off-by: Mykola Golub <mgolub@mirantis.com>

qa/tasks/ceph.py: debug which pgs aren't scrubbing

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #13478 from xiaoxichen/fix_osdc_perfcounter

osdc: fix osdc_osd_seesion perf counter.

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #13626 from wjwithagen/wip-wjw-freebsd-enodata-fixed

do_freebsd.sh: Remove ENODATA requirement

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #13640 from zmc/wip-fs-openstack

qa/suites/fs: Add openstack volume configuration

Merge pull request #13619 from kylinstorage/cleanup_bitallocator

BitAllocator: delete useless codes

Reviewed-by: Brad Hubbard <bhubbard@redhat.com>

qa/suites/fs: Add openstack volume configuration

Signed-off-by: Zack Cerza <zack@redhat.com>

Merge pull request #13594 from athanatos/wip-snap-trim-sleep

osd: add snap trim reservation and re-implement osd_snap_trim_sleep

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #13570 from athanatos/wip-18937

osd: don't use ORDERSNAP for flush; always request/send ondisk ack

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Sage Weil <sage@redhat.com>

test/librbd/CMakeLists.txt: ceph_test_librbd_fsx requires linux includes/libs

Signed-off-by: Willem Jan Withagen <wjw@digiware.nl>

Merge pull request #13637 from zmc/wip-openstack-volumes

qa/suites/{knfs,hadoop,samba}: OpenStack volume configuration

Merge pull request #13634 from zmc/wip-kcephfs-openstack

qa/suites/kcephfs: Openstack volume configuration

qa/suites/samba: Openstack volume configuration

Signed-off-by: Zack Cerza <zack@redhat.com>

qa/suites/hadoop: Openstack volume configuration

Signed-off-by: Zack Cerza <zack@redhat.com>

qa/suites/knfs: Add openstack volume configuration

Signed-off-by: Zack Cerza <zack@redhat.com>

qa/suites/kcephfs: Openstack volume configuration

Signed-off-by: Zack Cerza <zack@redhat.com>

osd,osdc: eliminate FLAG_ONDISK and helpers

The objecter actually always needs to get a response in order to
be able to not continually resend ops (even if the caller didn't
provide a callback).  Thus, it makes no sense for an MOSDOp to
ever not have FLAG_ONDISK set.  Therefore, we'll just remove the
helper and assume it's always there (it's safe to send a response
the client didn't ask for, the error paths already do that).  On
the Objecter side, we'll just unconditionally fill in ONDISK for
the benefit of pre-luminous OSDs.

Fixes: http://tracker.ceph.com/issues/18961
Signed-off-by: Samuel Just <sjust@redhat.com>

PrimaryLogPG::start_flush: don't use ORDERSNAP, eliminate the second delete

I think that whole thing was a misguided attempt to avoid deleting head
if it exists in the base tier (in reality it doesn't matter since head
would have to be logically dirty and anything we actually care about
would be preserved by sending a new enough seq to cause a clone).

Introduced in 4843fd510b33a71999cdf9c2cfa2b4c318fa80fd, but the real
logical error happened in f3df50188b54e60e28a276762c370477538bbb07.

I suggest never backporting this patch.  If you want to try, keep in
mind that the last version didn't turn up as busted for 2 years.

Fixes: f3df50188b54e60e28a276762c370477538bbb07
Signed-off-by: Samuel Just <sjust@redhat.com>

Merge pull request #13631 from ceph/wip-krbd-openstack

qa/suites/krbd: Add openstack volume configuration

Merge pull request #13534 from liewegas/wip-objecter-fixes

osdc/Objecter: fix bugs in explicit naming of op spg_t

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

qa/suites/krbd: Add openstack volume configuration

Signed-off-by: Zack Cerza <zack@redhat.com>

Merge pull request #13545 from liewegas/wip-18933

osd: constify OpRequest::get_req(); fix a few cases of operator<< vs mutated message races

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #13625 from tchaikov/wip-suite-repo-for-workunit

qa/tasks/workunit: use the suite repo for cloning workunit

Reviewed-by: Ilya Dryomov <idryomov@gmail.com>
Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #13614 from linuxbox2/wip-valid-dirname

rgw_file: ensure valid_s3_object_name for directories, too

Merge pull request #12102 from yangdongsheng/async_remove

librbd: asynchronous image removal state machine

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

do_freebsd.sh: the EMBEDDED target is not (yet) working for FreeBSD

Signed-off-by: Willem Jan Withagen <wjw@digiware.nl>

do_freebsd.sh: Remove ENODATA requirement

 - This was there because cython did not know about ENOATTR
   But since cython 0.25.2 tus is fixed.
   And we don't need to edit /usr/include/errno.h

Signed-off-by: Willem Jan Withagen <wjw@digiware.nl>

Merge pull request #13620 from tchaikov/wip-sepia-bt

script/sepia_bt.sh: get sha1,release from t.log if it's not in core

Reviewed-by: Brad Hubbard <bhubbard@redhat.com>

qa/tasks/workunit: use the suite repo for cloning workunit

as "workunits" reside in ceph/qa/workunits, it's more intuitive to
respect suite-repo option when cloning workunits.

Signed-off-by: Kefu Chai <kchai@redhat.com>

script/sepia_bt.sh: get sha1,release from t.log if it's not in core

* sometimes, the coredump comes from python, so we should get the sha1 and
  release in a different and more fragile way.
* also, the distro of Centos7 comes from python is "Centos Linux", so we
  should normalize its distro name and distro version.
* add "-v" option to be more chatty.
* normalize the $prog if $prog is */python*
* fix the pkg_path if the distro is centos7

Signed-off-by: Kefu Chai <kchai@redhat.com>

BitAllocator: delete useless codes

Signed-off-by: Jie Wang <jie.wang@kylin-cloud.com>

test/librbd: skip expect object request send when skip_partial_discard on

With skip_partial_discard on, there may be no object request send calls,
because they are skipped.

Signed-off-by: Gui Hecheng <guihecheng@cmss.chinamobile.com>

Merge pull request #13312 from trociny/wip-18783

rbd-mirror: InstanceWatcher watch/notify stub for leader/follower RPC

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #13616 from liewegas/wip-primary-temp

mon/OSDMonitor: remove trivial PGMap dependency for 'osd primary-temp' command

Reviewed-by: Greg Farnum <gfarnum@redhat.com>

Merge pull request #13611 from zmc/wip-rgw-openstack

qa/suites/rgw: Add openstack volume configuration

Reviewed-by: Yuri Weinstein <yweins@redhat.com>

mon/OSDMonitor: remove trivial PGMap dependency for 'osd primary-temp' command

Signed-off-by: Sage Weil <sage@redhat.com>

rgw_file: ensure valid_s3_object_name for directories, too

The logic in RGWLibFS::mkdir() validated bucket names, but not
object names (though RGWLibFS::create() did so).

The negative side effect of this was not creating illegal objects
(we won't), but in a) failing with -EIO and b) more importantly,
not removing up the proposed object from FHCache, which produced a
boost assert when recycled.

Fixes: http://tracker.ceph.com/issues/19066
Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>

Merge pull request #13607 from linuxbox2/wip-parentref

rgw: wip parentref

qa/suites/rgw: Add openstack volume configuration

Without this, OSDs will fail to create on instances whose root fs isn't
xfs.

Signed-off-by: Zack Cerza <zack@redhat.com>

Merge pull request #13480 from wido/srpm

make-srpm: Pass first parameter to make-dist for building SRPM

Reviewed-by: Kefu Chai <kchai@redhat.com>

librbd: adopt override for writesame related header files

Signed-off-by: Gui Hecheng <guihecheng@cmss.chinamobile.com>

librbd: make writesame get all zeros with rbd_skip_partial_discard on

Now writesame tends to do a discard on all zero buffer, but
if rbd_skip_partial_discard is turned on, discard will not do any
zeroing work on partial discard, so we read random data while expecting
all zeros.
To fix it, we introduce the ImageCtx::skip_partial_discard and
pass an extra parameter to discard. Writesame will always pass false
and other discards will pass the ImageCtx::skip_partial_discard
which is set by the rbd_skip_partial_discard option.

Signed-off-by: Gui Hecheng <guihecheng@cmss.chinamobile.com>

osd: remove copy-get-classic

This hasn't been used since well before hammer.

Signed-off-by: Sage Weil <sage@redhat.com>

rgw_file:  return of RGWFileHandle::FLAG_EXACT_MATCH

Allow callers of rgw_lookup() on objects attested in an
rgw_readdir() callback the ability to bypass exact match in
RGWLibFS::stat_leaf() case 2, but restore exact match enforcement
for general lookups.

This preserves required common_prefix namespace behavior, but
prevents clients from eerily permitting things like "cd sara0" via
partial name match on "sara01."

Fixes: http://tracker.ceph.com/issues/19059
Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>

Merge pull request #13595 from tchaikov/wip-enable-bluestore-ceph-dencoder

ceph-dencoder: enable bluestore types

Reviewed-by: Sage Weil <sage@redhat.com>

osdc/Objecter: _calc_target on all ops so that we notice splits

We need to make sure we update the mapping and get an accurate actual_pgid
value by recalcuating the mapping on every map change.  Otherwise, we may
not notice a split (and subsequent actual_pgid change) and resend the same
op with a stale spg_t.  To fix this,

- _calc_target on need_resend
- update target regardless of current con

Signed-off-by: Sage Weil <sage@redhat.com>

osdc/Objecter: refactor pool dne check to make op->session optional

Signed-off-by: Sage Weil <sage@redhat.com>

osdc/Objecter: track latest epoch in op_target_t

Signed-off-by: Sage Weil <sage@redhat.com>

osd: warn on ops directed to the wrong pg_t

Check whether the request hobj maps to the current pg_t.  If we have the
osd_debug_misdirected_ops setting enabled (as teuthology does), assert out
as well so that the error is easy to spot.  This catches bugs in the
Objecter (especially the new code that explicitly names the spg_t for the
request).

Signed-off-by: Sage Weil <sage@redhat.com>

vstart.sh: osd debug misdirected ops = true

Signed-off-by: Sage Weil <sage@redhat.com>