os/aio: remove the redundant memset(struct iocb)

In fact, io_prep_read/write/pwritev/preadv do the memset(struct iocb).
So no need do it.

Signed-off-by: Jianpeng Ma <jianpeng.ma@intel.com>

Merge pull request #13603 from Adirl/fix_compile

msg/async/rdma: Fix broken compilation

Reviewed-by: Haomai Wang <haomai@xsky.com>

Merge pull request #13652 from liupan1111/wip-cleanup-rbd-nbd

rbd-nbd: fix a typo "moudle"

Reviewed-by: Mykola Golub <mgolub@mirantis.com>

Merge pull request #13227 from ukernel/wip-multimds-misc

mds: misc multimds fixes

Reviewed-by: John Spray <john.spray@redhat.com>

Merge pull request #13648 from tchaikov/wip-doc-osd-agent-options

doc: document `osd-agent-{max,low}-ops` options

Reviewed-by: Zhiqiang Wang <zhiqiang@xsky.com>
Reviewed-by: Brad Hubbard <bhubbard@redhat.com>

rbd-nbd: fix a typo "moudle"

Signed-off-by: Pan Liu <liupan1111@gmail.com>

doc: document `osd-agent-{max,low}-ops` options

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #13649 from liewegas/wip-ceph-scrub-debug

qa/tasks/ceph.py: debug which pgs aren't scrubbing

Reviewed-by: Brad Hubbard <bhubbard@redhat.com>

qa/tasks/ceph.py: debug which pgs aren't scrubbing

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #13478 from xiaoxichen/fix_osdc_perfcounter

osdc: fix osdc_osd_seesion perf counter.

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #13626 from wjwithagen/wip-wjw-freebsd-enodata-fixed

do_freebsd.sh: Remove ENODATA requirement

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #13640 from zmc/wip-fs-openstack

qa/suites/fs: Add openstack volume configuration

Merge pull request #13619 from kylinstorage/cleanup_bitallocator

BitAllocator: delete useless codes

Reviewed-by: Brad Hubbard <bhubbard@redhat.com>

qa/suites/fs: Add openstack volume configuration

Signed-off-by: Zack Cerza <zack@redhat.com>

Merge pull request #13594 from athanatos/wip-snap-trim-sleep

osd: add snap trim reservation and re-implement osd_snap_trim_sleep

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #13570 from athanatos/wip-18937

osd: don't use ORDERSNAP for flush; always request/send ondisk ack

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #13637 from zmc/wip-openstack-volumes

qa/suites/{knfs,hadoop,samba}: OpenStack volume configuration

Merge pull request #13634 from zmc/wip-kcephfs-openstack

qa/suites/kcephfs: Openstack volume configuration

qa/suites/samba: Openstack volume configuration

Signed-off-by: Zack Cerza <zack@redhat.com>

qa/suites/hadoop: Openstack volume configuration

Signed-off-by: Zack Cerza <zack@redhat.com>

qa/suites/knfs: Add openstack volume configuration

Signed-off-by: Zack Cerza <zack@redhat.com>

qa/suites/kcephfs: Openstack volume configuration

Signed-off-by: Zack Cerza <zack@redhat.com>

osd,osdc: eliminate FLAG_ONDISK and helpers

The objecter actually always needs to get a response in order to
be able to not continually resend ops (even if the caller didn't
provide a callback).  Thus, it makes no sense for an MOSDOp to
ever not have FLAG_ONDISK set.  Therefore, we'll just remove the
helper and assume it's always there (it's safe to send a response
the client didn't ask for, the error paths already do that).  On
the Objecter side, we'll just unconditionally fill in ONDISK for
the benefit of pre-luminous OSDs.

Fixes: http://tracker.ceph.com/issues/18961
Signed-off-by: Samuel Just <sjust@redhat.com>

PrimaryLogPG::start_flush: don't use ORDERSNAP, eliminate the second delete

I think that whole thing was a misguided attempt to avoid deleting head
if it exists in the base tier (in reality it doesn't matter since head
would have to be logically dirty and anything we actually care about
would be preserved by sending a new enough seq to cause a clone).

Introduced in 4843fd510b33a71999cdf9c2cfa2b4c318fa80fd, but the real
logical error happened in f3df50188b54e60e28a276762c370477538bbb07.

I suggest never backporting this patch.  If you want to try, keep in
mind that the last version didn't turn up as busted for 2 years.

Fixes: f3df50188b54e60e28a276762c370477538bbb07
Signed-off-by: Samuel Just <sjust@redhat.com>

Merge pull request #13631 from ceph/wip-krbd-openstack

qa/suites/krbd: Add openstack volume configuration

Merge pull request #13534 from liewegas/wip-objecter-fixes

osdc/Objecter: fix bugs in explicit naming of op spg_t

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

qa/suites/krbd: Add openstack volume configuration

Signed-off-by: Zack Cerza <zack@redhat.com>

Merge pull request #13545 from liewegas/wip-18933

osd: constify OpRequest::get_req(); fix a few cases of operator<< vs mutated message races

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #13625 from tchaikov/wip-suite-repo-for-workunit

qa/tasks/workunit: use the suite repo for cloning workunit

Reviewed-by: Ilya Dryomov <idryomov@gmail.com>
Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #13614 from linuxbox2/wip-valid-dirname

rgw_file: ensure valid_s3_object_name for directories, too

Merge pull request #12102 from yangdongsheng/async_remove

librbd: asynchronous image removal state machine

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

do_freebsd.sh: the EMBEDDED target is not (yet) working for FreeBSD

Signed-off-by: Willem Jan Withagen <wjw@digiware.nl>

do_freebsd.sh: Remove ENODATA requirement

 - This was there because cython did not know about ENOATTR
   But since cython 0.25.2 tus is fixed.
   And we don't need to edit /usr/include/errno.h

Signed-off-by: Willem Jan Withagen <wjw@digiware.nl>

Merge pull request #13620 from tchaikov/wip-sepia-bt

script/sepia_bt.sh: get sha1,release from t.log if it's not in core

Reviewed-by: Brad Hubbard <bhubbard@redhat.com>

qa/tasks/workunit: use the suite repo for cloning workunit

as "workunits" reside in ceph/qa/workunits, it's more intuitive to
respect suite-repo option when cloning workunits.

Signed-off-by: Kefu Chai <kchai@redhat.com>

script/sepia_bt.sh: get sha1,release from t.log if it's not in core

* sometimes, the coredump comes from python, so we should get the sha1 and
  release in a different and more fragile way.
* also, the distro of Centos7 comes from python is "Centos Linux", so we
  should normalize its distro name and distro version.
* add "-v" option to be more chatty.
* normalize the $prog if $prog is */python*
* fix the pkg_path if the distro is centos7

Signed-off-by: Kefu Chai <kchai@redhat.com>

BitAllocator: delete useless codes

Signed-off-by: Jie Wang <jie.wang@kylin-cloud.com>

Merge pull request #13312 from trociny/wip-18783

rbd-mirror: InstanceWatcher watch/notify stub for leader/follower RPC

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #13616 from liewegas/wip-primary-temp

mon/OSDMonitor: remove trivial PGMap dependency for 'osd primary-temp' command

Reviewed-by: Greg Farnum <gfarnum@redhat.com>

Merge pull request #13611 from zmc/wip-rgw-openstack

qa/suites/rgw: Add openstack volume configuration

Reviewed-by: Yuri Weinstein <yweins@redhat.com>

mon/OSDMonitor: remove trivial PGMap dependency for 'osd primary-temp' command

Signed-off-by: Sage Weil <sage@redhat.com>

rgw_file: ensure valid_s3_object_name for directories, too

The logic in RGWLibFS::mkdir() validated bucket names, but not
object names (though RGWLibFS::create() did so).

The negative side effect of this was not creating illegal objects
(we won't), but in a) failing with -EIO and b) more importantly,
not removing up the proposed object from FHCache, which produced a
boost assert when recycled.

Fixes: http://tracker.ceph.com/issues/19066
Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>

Merge pull request #13607 from linuxbox2/wip-parentref

rgw: wip parentref

qa/suites/rgw: Add openstack volume configuration

Without this, OSDs will fail to create on instances whose root fs isn't
xfs.

Signed-off-by: Zack Cerza <zack@redhat.com>

Merge pull request #13480 from wido/srpm

make-srpm: Pass first parameter to make-dist for building SRPM

Reviewed-by: Kefu Chai <kchai@redhat.com>

rgw_file:  return of RGWFileHandle::FLAG_EXACT_MATCH

Allow callers of rgw_lookup() on objects attested in an
rgw_readdir() callback the ability to bypass exact match in
RGWLibFS::stat_leaf() case 2, but restore exact match enforcement
for general lookups.

This preserves required common_prefix namespace behavior, but
prevents clients from eerily permitting things like "cd sara0" via
partial name match on "sara01."

Fixes: http://tracker.ceph.com/issues/19059
Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>

Merge pull request #13595 from tchaikov/wip-enable-bluestore-ceph-dencoder

ceph-dencoder: enable bluestore types

Reviewed-by: Sage Weil <sage@redhat.com>

osdc/Objecter: _calc_target on all ops so that we notice splits

We need to make sure we update the mapping and get an accurate actual_pgid
value by recalcuating the mapping on every map change.  Otherwise, we may
not notice a split (and subsequent actual_pgid change) and resend the same
op with a stale spg_t.  To fix this,

- _calc_target on need_resend
- update target regardless of current con

Signed-off-by: Sage Weil <sage@redhat.com>

osdc/Objecter: refactor pool dne check to make op->session optional

Signed-off-by: Sage Weil <sage@redhat.com>

osdc/Objecter: track latest epoch in op_target_t

Signed-off-by: Sage Weil <sage@redhat.com>

osd: warn on ops directed to the wrong pg_t

Check whether the request hobj maps to the current pg_t.  If we have the
osd_debug_misdirected_ops setting enabled (as teuthology does), assert out
as well so that the error is easy to spot.  This catches bugs in the
Objecter (especially the new code that explicitly names the spg_t for the
request).

Signed-off-by: Sage Weil <sage@redhat.com>

vstart.sh: osd debug misdirected ops = true

Signed-off-by: Sage Weil <sage@redhat.com>

rgw_file: rgw_lookup: don't ref for "/" or ".."

These refs won't be returned by nfs-ganesha, and are sufficiently
magical that other consumers should be persuaded to understand
their specialness.

Fixes: http://tracker.ceph.com/issues/19060
Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>

Merge pull request #13569 from athanatos/wip-17831-18583-18809-18927

ReplicatedBackend: don't queue Context outside of ObjectStore with obc

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #13470 from wangzhengyong/master

os/bluestore/BitAllocator: fix bug of checking required blocks

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #13516 from wjwithagen/wip-wjw-test-onjectstore

test/ceph-test-objectstore: Don't always include BlueStore code

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #13521 from dachary/wip-crush-uniform

crush: allow uniform buckets with no items

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #13540 from liewegas/wip-debug-osd

osd: debug con in ms_handle_connect

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #13565 from liewegas/wip-fix-pool-race-test

ceph_test_rados_api_misc: avoid livelock from PoolCreationRace

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

rbd-mirror A/A: InstanceWatcher watch/notify stub for leader/follower RPC

Fixes: http://tracker.ceph.com/issues/18783
Signed-off-by: Mykola Golub <mgolub@mirantis.com>

librbd: add ManagedLock destroy method (useful for mock tests)

Signed-off-by: Mykola Golub <mgolub@mirantis.com>

cls/rbd: methods to track rbd-mirror instances

Signed-off-by: Mykola Golub <mgolub@mirantis.com>

msg/async/rdma: Fix broken compilation

Resposible commit is: eb0f624

issue:985021

Change-Id: I5b4afc537a9351dd18f5ea08bf4d72a1b03a8635
Signed-off-by: Sarit Zubakov <saritz@mellanox.com>

Merge pull request #13568 from dillaman/wip-18990

rbd-mirror: deleting a snapshot during sync can result in read errors

Reviewed-by: Mykola Golub <mgolub@mirantis.com>

ceph-dencoder: s/WITH_LIBAIO/HAVE_LIBAIO/

* s/WITH_LIBAIO/HAVE_LIBAIO/: as HAVE_LIBAIO is used to detect if libaio
  is installed and is exposed in the acconfig.h.
* do not test bluestore_blob_t with ceph-dencoder, as it repurposes the
  "feature" parameter for struct_v.

Signed-off-by: Willem Jan Withagen <wjw@digiware.nl>
Signed-off-by: Kefu Chai <kchai@redhat.com>

os/bluestore: fix bluestore_blob_t::generate_test_instances()

add extends before marking unused ranges, otherwise add_unused()
asserts if (offset + len < blob_len)
this method is supposed to be used by ceph-dencoder, but
bluestore_blob_t's codec is quite different. we are not testing its
encoding in ceph-dencoder at this moment.

Signed-off-by: Kefu Chai <kchai@redhat.com>

unittest: add unittest for RemoveRequest

Signed-off-by: Dongsheng Yang <dongsheng.yang@easystack.cn>

unittest: fixup the mock method of open

Signed-off-by: Dongsheng Yang <dongsheng.yang@easystack.cn>

unittest: add mock method of try_acquire_lock

Signed-off-by: Dongsheng Yang <dongsheng.yang@easystack.cn>

librbd: cleanup: remove the unused function ObjectMap::remove()

Signed-off-by: Dongsheng Yang <dongsheng.yang@easystack.cn>

rbd: add a aio_remove() in objectMap

Signed-off-by: Dongsheng Yang <dongsheng.yang@easystack.cn>

librbd: use create/destroy methods in RemoveRequest

Signed-off-by: Dongsheng Yang <dongsheng.yang@easystack.cn>

unittest: add a unittest for removing non-exist image.

Signed-off-by: Dongsheng Yang <dongsheng.yang@easystack.cn>

qa: rbd: rename the copy.sh to generic.sh

The copy.sh is not only testing the rbd copy, but also
others such as rbd ls, rbd remove. Then rename it to generic.sh

Signed-off-by: Dongsheng Yang <dongsheng.yang@easystack.cn>

qa: add a test for rbd rm NOT_EXIST

Signed-off-by: Dongsheng Yang <dongsheng.yang@easystack.cn>

librbd: use async image removal state machine

Signed-off-by: Venky Shankar <vshankar@redhat.com>

librbd: asynchronous image removal state machine

Signed-off-by: Dongsheng Yang <dongsheng.yang@easystack.cn>
Signed-off-by: Venky Shankar <vshankar@redhat.com>

Merge pull request #13589 from dachary/wip-crush-doc

doc: crush: API documentation fixes

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #13590 from linuxbox2/wip-badbuck

rgw_file: avoid stranding invalid-name bucket handles in fhcache

Merge pull request #13324 from dongbula/make-sync-thread-name-clear

rgw: make sync thread name clear

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #13361 from ddiss/rbdmap_unmap_from_file

rbdmap: consider /etc/ceph/rbdmap when unmapping images

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #13387 from Jing-Scott/clean-up-rgw-list-redundant-code

rgw: clean up the redundant assignment in last_entry_in_listing

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #13460 from theanalyst/fix/rgw-rest-user-fixme

wip: rgw: rest_admin/user avoid double checking input args

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #13461 from theanalyst/doc/rgw-system-user

doc: rgw: make a note abt system users vs normal users

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #13576 from jronak/doc

rgw:Fixes typo in rgw_admin.cc

Reviewed-by: Abhishek Lekshmanan <abhishek@suse.com>
Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #13578 from Liuchang0812/wip-fix-typo-in-rgw

rgw: fix typo in comment

Reviewed-by: Casey Bodley <cbodley@redhat.com>

rgw_file: avoid stranding invalid-name bucket handles in fhcache

To avoid a string copy in the common mkdir path, handles for
proposed buckets currently are staged in the handle table, before
being rejected.  They need to be destaged, not just marked deleted
(because deleted objects are now assumed not to be linked, as of
beaeff059375b44188160dbde8a81dd4f4f8c6eb).

This triggered an unhandled Boost assert when deleting staged
handles, as current safe_link mode requires first removing from
the FHCache.

Fixes: http://tracker.ceph.com/issues/19036
Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>

Merge pull request #12897 from renhwztetecs/renhw-wip-client-cleanup

client: priority to verify the correctness of the "flag"

Reviewed-by: John Spray <john.spray@redhat.com>

Merge pull request #13359 from jcsp/wip-logrotate-sshexception

qa: handle SSHException in logrotate

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #13429 from ukernel/wip-18941

test/libcephfs: avoid buffer overflow when testing ceph_getdents()

Reviewed-by: John Spray <john.spray@redhat.com>

Merge pull request #13455 from ukernel/wip-18953

mds: ignore fs full check for CEPH_MDS_OP_SETFILELOCK

Reviewed-by: John Spray <john.spray@redhat.com>

Merge pull request #13465 from jcsp/wip-mds-voidfn

mds: replace C_VoidFn in MDSDaemon with lambdas

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #13471 from batrick/fs-new-eexist

mon: detect existing fs and duplicate name earlier

Reviewed-by: John Spray <john.spray@redhat.com>

Merge pull request #13572 from theanalyst/doc/changelog/v0.94.10

doc: add changelog for v0.94.10

Reviewed-by: Sage Weil <sweil@redhat.com>
Reviewed-by: Loic Dachary <ldachary@redhat.com>
Reviewed-by: Nathan Cutler <ncutler@suse.com>

Merge pull request #13152 from smithfarm/wip-hammer-release-notes

doc: hammer 0.94.10 release notes

Reviewed-by: Abhishek Lekshmanan <abhishek@suse.com>

Merge pull request #13573 from dachary/wip-17821-revert

ceph-disk: dmcrypt activate must use the same cluster as prepare

Reviewed-by: Kefu Chai <kchai@redhat.com>

rgw: Fixes typo in rgw_admin.cc

Issue: http://tracker.ceph.com/issues/19026
Signed-off-by: Ronak Jain <ronakjain@outlook.in>

rgw: fix typo in comment

Signed-off-by: liuchang0812 <liuchang0812@gmail.com>

ceph-disk: dmcrypt activate must use the same cluster as prepare

When dmcrypt is used, the fsid cannot be retrieved from the data
partition because it is encrypted. Store the fsid in the lockbox to
enable dmcrypt activation using the same logic as regular activation.

The fsid is used to retrive the cluster name that was used during
prepare, reason why activation does not and must not have a --cluster
argument.

Fixes: http://tracker.ceph.com/issues/17821
Signed-off-by: Loic Dachary <ldachary@redhat.com>

rbd-mirror: retry object copy after -ENOENT error

Fixes: http://tracker.ceph.com/issues/18990
Signed-off-by: Jason Dillaman <dillaman@redhat.com>