Merge pull request #7773 from roidayan/xio_remove_redundant_magic_methods

xio: remove redundant magic methods

Reviewed-by: Haomai Wang<haomai@xsky.com>

Merge pull request #7771 from xiexingguo/xxg-wip-fixawatch

librados: do not clear handle for aio_watch()

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

xio: remove redundant magic methods

The same methods are inherited from Messenger.

Signed-off-by: Roi Dayan <roid@mellanox.com>

librados: remove unused local variables

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

librados: do not clear handle for aio_watch()

which is needed for aio_unwatch() to work.

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

Merge pull request #7649 from yuyuyu101/wip-async-watch

librados: implement async watch/unwatch

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #7761 from dillaman/wip-14847

librbd: fix state machine race conditions during shut down

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

RadosClient: call watch_flush before finisher stop

Signed-off-by: Haomai Wang <haomai@xsky.com>

Merge pull request #7632 from zaitcev/cleanup-dead-code-3

rgw: Drop unused usage_exit from rgw_admin.cc

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>

Merge pull request #7285 from magicrobotmonkey/patch-1

Increase verbosity level on RGWObjManifest line

Reviewed-by: Yehuda Saeh <yehuda@redhat.com>

Merge pull request #7618 from zaitcev/wip-5073-subuser-bis

rgw: Fix subuser harder with tenants

Reviewed-y: Yehuda Sadeh <yehuda@redhat.com>

Merge pull request #7763 from dachary/wip-releases

doc: fix 0.94.4 and 0.94.5 ordering

Merge pull request #7364 from javenwu/master

common: default cluster name to config file prefix

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #7548 from yuyuyu101/librbd-avoid-memcpy

librbd: reduce mem copies to user-buffer during read

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #7667 from jdurgin/wip-14780

Revert "librbd: use task finisher per CephContext"

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #7625 from branch-predictor/bp-async-more

msg/async: smarter MSG_MORE

Reviewed-by: Haomai Wang <haomai@xsky.com>

doc: fix 0.94.4 and 0.94.5 ordering

Signed-off-by: Loic Dachary <loic@dachary.org>

Merge pull request #7760 from theanalyst/fix/rgw-fcgi-build

rgw: fcgi should include acconfig

librbd: deterministically unblock writes on snap set

Previously this was handled in the state machine's destructor,
but that can lead to nondeterministic behavior due to interactions
between threads.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

librbd: avoid close race-condition within ImageState

The lock was previously unlocked/locked to avoid lock cycles. This
creates an issue when closing the image because the another thread
might have already deleted the ImageState before the lock could be
re-locked.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

rgw: fcgi should include acconfig

As it references FASTCGI_INCLUDE_DIR which is referenced from it

Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>

doc/release-notes: v0.94.6 final notes

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #7689 from dachary/wip-release-notes

release-notes: draft v0.94.6 release notes

msg/async: cut the middle-man

Get rid of complete_bl and let messenger write directly to outcoming_bl of
a connection. Also, if message bufferlist is small enough, append its
contents to outcoming_bl directly, so we'll use less iovecs and in best
case, pack entire message (together with header and footer added in
write_message()) in single bufferptr.

Signed-off-by: Piotr Dałek <piotr.dalek@ts.fujitsu.com>

Merge pull request #7725 from ceph/wip-fix-async-valgrind

AsyncMessenger: fix valgrind leak

Merge pull request #7758 from dachary/wip-ceph-disk-prepare-help

ceph-disk: fix prepare --help

Reviewed-by: Kefu Chai <kchai@redhat.com>

ceph-disk: fix prepare --help

Signed-off-by: Loic Dachary <loic@dachary.org>

Merge pull request #7756 from dachary/wip-ceph-disk-args

ceph-disk: deactivate / destroy PATH arg are optional

Reviewed-by: Kefu Chai <kchai@redhat.com>

ceph-disk: deactivate / destroy PATH arg are optional

And the mandatory positional arguments need not be == 1, it is enough to
leave the nargs parameter out.

Signed-off-by: Loic Dachary <loic@dachary.org>

Merge pull request #7687 from yangdongsheng/ceph-disk_nargs

ceph-disk: make some arguments as required if necessary

Reviewed-by: Loic Dachary <ldachary@redhat.com>

Merge pull request #7640 from batrick/vstart-btrfs-missing

vstart.sh: silence a harmless msg where btrfs is not found

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #7737 from trociny/wip-librados_test_stub-debug

test: more debug logging for TestWatchNotify

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #7714 from SUSE/wip-qa-typo

doc: standardize @param (not @parma, @parmam, @params)

Reviewed-by: Abhishek Lekshmanan <abhishek@suse.com>

Merge pull request #7738 from trociny/test-librbd-poolname

test: create pools for rbd tests with different prefix

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #7727 from jmunhoz/ceph-fix-14829

vstart.sh: avoid race condition starting rgw via vstart.sh

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #7614 from trociny/wip-rbd-mirroring-image-reply

rbd-mirror: implement ImageReplayer

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #7744 from dachary/wip-14838-ceph-disk-dmcrypt

ceph-disk: s/dmcrpyt/dmcrypt/

Reviewed-by: Nathan Cutler <ncutler@suse.com>

ceph-disk: s/dmcrpyt/dmcrypt/

http://tracker.ceph.com/issues/14838 Fixes: #14838

Signed-off-by: Frode Sandholtbraaten <fsa@basefarm.no>
Signed-off-by: Loic Dachary <loic@dachary.org>

Merge pull request #7688 from tchaikov/wip-14788

buffer: hide iterator_impl symbols

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

test: create pools for rbd tests with different prefix

This makes easier to find out what test fails to cleanup.

Signed-off-by: Mykola Golub <mgolub@mirantis.com>

Merge pull request #7723 from tchaikov/wip-rgw-test

test/radosgw-admin: update the expected usage outputs

Reviewed-by: Loic Dachary <ldachary@redhat.com>

test_rados_watch_notify: add async_watch_flush tests

Signed-off-by: Haomai Wang <haomai@xsky.com>

Merge pull request #7647 from yuyuyu101/wip-spdk

NVMEDevice: refactor probe/attach codes and support zero command

Reviewed-by: Kefu Chai <kchai@redhat.com>

test: more debug for TestWatchNotify

Signed-off-by: Mykola Golub <mgolub@mirantis.com>

NVMEDevice: accept spdk api name changes

Signed-off-by: Haomai Wang <haomai@xsky.com>

spdk: update spdk submodule to accept new interface changes

Signed-off-by: Haomai Wang <haomai@xsky.com>

NVMEDevice: add default constructor for Task and clean up

Signed-off-by: Haomai Wang <haomai@xsky.com>

NVMEDevice: only activate zero command when backend supported

Signed-off-by: Haomai Wang <haomai@xsky.com>

BlockDevice: detect symbol file basename

Signed-off-by: Haomai Wang <haomai@xsky.com>

BlueStore: fix spdk link file create failed

Signed-off-by: Haomai Wang <haomai@xsky.com>

NVMEDevice: fix atomic and lock changes

Signed-off-by: Haomai Wang <haomai@xsky.com>

NVMEDevice: use nvme_probe to simply init

Signed-off-by: Haomai Wang <haomai@xsky.com>

spdk: update spdk submodule

Signed-off-by: Haomai Wang <haomai@xsky.com>

NVMEDevice: use nvme zero command instead of writing zero

Signed-off-by: Haomai Wang <haomai@xsky.com>

NVMEDevice: cleanup task iterator process

Signed-off-by: Haomai Wang <haomai@xsky.com>

vstart.sh: avoid race condition starting rgw via vstart.sh

In slow environments running vstart.sh hits a race condition setting up
the rgw server. Creating the S3 and Swift users before booting rgw
avoids the issue.

Fixes: #14829
Signed-off-by: Javier M. Mellid <jmunhoz@igalia.com>

test/radosgw-admin: update the expected usage outputs

to match with the latest radosgw-admin changes.

Signed-off-by: Kefu Chai <kchai@redhat.com>

test: rbd-mirror: add tool to test ImageReplayer

Signed-off-by: Mykola Golub <mgolub@mirantis.com>

test: rbd-mirror: add ImageReplayer tests

Signed-off-by: Mykola Golub <mgolub@mirantis.com>

rbd-mirror: implement ImageReplayer

Signed-off-by: Mykola Golub <mgolub@mirantis.com>

librados: add async_watch_flush api

Signed-off-by: Haomai Wang <haomai@xsky.com>

RadosClient: add async watch_flush version

Signed-off-by: Haomai Wang <haomai@xsky.com>

Objecter: make linger_callback_flush accept context instead of blocking

Signed-off-by: Haomai Wang <haomai@xsky.com>

librados: add C api impl and tracing codes

Signed-off-by: Haomai Wang <haomai@xsky.com>

librados: remove unused variables

Signed-off-by: Haomai Wang <haomai@xsky.com>

test: add aio_unwatch unittest

Signed-off-by: Haomai Wang <haomai@xsky.com>

librados: support aio_unwatch api

Signed-off-by: Haomai Wang <haomai@xsky.com>

librados: add unittest for rados aio watch

Signed-off-by: Haomai Wang <haomai@xsky.com>

librados: add async watch api

Signed-off-by: Haomai Wang <haomai@xsky.com>

journal: wrappers to get journal metadata

Signed-off-by: Mykola Golub <mgolub@mirantis.com>

librbd: allow to use journal replay externally

Signed-off-by: Mykola Golub <mgolub@mirantis.com>

cls::journal: don't print client data

It makes `rbd journal status` output look ugly.

It is a temporary solution and is going to be improved in
the future (it could be useful to see data but decoded).

Signed-off-by: Mykola Golub <mgolub@mirantis.com>

test/bufferlist:Add test case for  bufferlist::is_provided_buffer.

Signed-off-by: Jianpeng Ma <jianpeng.ma@intel.com>

Striper: Add function 'assemble_result(CephContext *cct, char *buffer, size_t len)' in StripedReadResult.

For sparse read, it can avoid copying zero around.

Signed-off-by: Jianpeng Ma <jianpeng.ma@intel.com>

Striper: Add total_intended_len in StripedReadResult

Using this field record the total len which we wanted.

Signed-off-by: Jianpeng Ma <jianpeng.ma@intel.com>

osdc/objectCacher: Remove the unsed field.

Signed-off-by: Jianpeng Ma <jianpeng.ma@intel.com>

bufferlist: Add new func is_provided_buffer(const char *dst)

Using this function to judge whether copy data from bufferlist to
dst. It mainly used librados C interface to avoid copy data from librados
to caller.

Signed-off-by: Jianpeng Ma <jianpeng.ma@intel.com>

Merge pull request #7627 from dillaman/wip-librbd-read-replay

librbd: block read requests until journal replayed

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #7718 from dillaman/wip-librbd-mirror-peer

librbd: update of mirror pool mode and mirror peer handling

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #7698 from dillaman/wip-librbd-replay-flush

librbd: differentiate journal replay flush vs shut down

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #7720 from ceph/wip-rgw-aws4

rgw support for aws authentication v4 (Javier M. Mellid)

Merge remote-tracking branch 'jmunhoz/wip-aws4' into wip-aws4

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
Conflicts:
src/rgw/rgw_auth_s3.h
src/rgw/rgw_client_io.cc
src/rgw/rgw_client_io.h
src/rgw/rgw_common.h
src/rgw/rgw_main.cc
src/rgw/rgw_op.cc
src/rgw/rgw_op.h
src/rgw/rgw_rest.cc
src/rgw/rgw_rest_metadata.cc
src/rgw/rgw_rest_s3.cc

Merge pull request #7719 from ceph/wip-rgw-keystone-v3

rgw keystone v3 (Mark Barnes, Radoslaw Zarzynski)

support for keystone v3

Reviewed-by: Matt Benjamin <mbenjamin@redhat.com>
Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>

librbd: update mirror peer data structure

Replace cluster uuid with mirror peer uuid and support an
optional pool id within a peer (when using internal APIs
for testing).

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

librbd: support image and pool-level mirroring modes

If pool-level mirroring is enabled, all journaled images within
the pool are eligible for mirroring.  If image-level mirroring
is enabled, mirroring needs to be explicitly enabled on each
image.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

rgw: Keystone token parsing doesn't need to know API version.

This patch targets an issue with S3Extension in Keystone:
requested tokens always conform to Keystone API v2 - regardless
of the version used to make the request.

Previous implementation of KeystoneToken::parse() as well
as KeystoneToken::decode_json() had to know the API version
explicitly. Thus, they might be affected in the future by
changes in S3-compatibility middleware of Keystone.

Signed-off-by: Radoslaw Zarzynski <rzarzynski@mirantis.com>

rgw: S3 always must use token format of Keystone v2.

Signed-off-by: Radoslaw Zarzynski <rzarzynski@mirantis.com>

rgw: Keystone token parsing should fail on misformed JSONs.

Signed-off-by: Radoslaw Zarzynski <rzarzynski@mirantis.com>

rgw: improve debugs in RGWPostHTTPData.

Signed-off-by: Radoslaw Zarzynski <rzarzynski@mirantis.com>

rgw: ONLY formatting improvements for Keystone-related code.

Signed-off-by: Radoslaw Zarzynski <rzarzynski@mirantis.com>

rgw: handle Keystone API version with dedicated enum.

Signed-off-by: Radoslaw Zarzynski <rzarzynski@mirantis.com>

rgw: fix wrong format of admin token request in Keystone v3.

Signed-off-by: Radoslaw Zarzynski <rzarzynski@mirantis.com>

rgw: rework serialization in token retrieval for Keystone's admin.

Signed-off-by: Radoslaw Zarzynski <rzarzynski@mirantis.com>

rgw: fix Keystone v2 regression in role's JSON parsing

Signed-off-by: Radoslaw Zarzynski <rzarzynski@mirantis.com>

rgw: accommodate Keystone v3 in S3-related auth mechanisms.

This patch is based on changes proposed by Mark Barnes
<mark.barnes@ocado.com> (lperiquito) in commit
af71f6a518529ea6cccb25bd46da2b6d1458c1da.

Signed-off-by: Mark Barnes <mark.barnes@ocado.com>
Signed-off-by: Radoslaw Zarzynski <rzarzynski@mirantis.com>

rgw: accommodate Keystone v3 in RGWSwift class.

This patch is based on changes proposed by Mark Barnes
<mark.barnes@ocado.com> (lperiquito) in commit
af71f6a518529ea6cccb25bd46da2b6d1458c1da.

Signed-off-by: Mark Barnes <mark.barnes@ocado.com>
Signed-off-by: Radoslaw Zarzynski <rzarzynski@mirantis.com>

rgw: basic data structures for Keystone v3.

The change has been extracted from changeset proposed
in commit af71f6a518529ea6cccb25bd46da2b6d1458c1da by
Mark Barnes <mark.barnes@ocado.com> (lperiquito).

Signed-off-by: Mark Barnes <mark.barnes@ocado.com>
Signed-off-by: Radoslaw Zarzynski <rzarzynski@mirantis.com>

rgw: unify RGWValidateKeystoneToken with RGWPostHTTPData.

The change has been extracted from changeset proposed
in commit af71f6a518529ea6cccb25bd46da2b6d1458c1da by
Mark Barnes <mark.barnes@ocado.com> (lperiquito).

Signed-off-by: Mark Barnes <mark.barnes@ocado.com>
Signed-off-by: Radoslaw Zarzynski <rzarzynski@mirantis.com>

rgw: make RGWPostHTTPData able to extract X-Subject-Token.

The change has been extracted from changeset proposed
in commit af71f6a518529ea6cccb25bd46da2b6d1458c1da by
Mark Barnes <mark.barnes@ocado.com> (lperiquito).

Signed-off-by: Mark Barnes <mark.barnes@ocado.com>
Signed-off-by: Radoslaw Zarzynski <rzarzynski@mirantis.com>

PendingReleaseNotes: document the dropped buffer symbols

the symbols of buffer::list::iterator_impl<> were wrongly exposed
in previous infernalis release, and the clients linked against
librados are very likely using them. so we need to document this
change.

Signed-off-by: Kefu Chai <kchai@redhat.com>