Merge pull request #13470 from wangzhengyong/master

os/bluestore/BitAllocator: fix bug of checking required blocks

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #13516 from wjwithagen/wip-wjw-test-onjectstore

test/ceph-test-objectstore: Don't always include BlueStore code

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #13521 from dachary/wip-crush-uniform

crush: allow uniform buckets with no items

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #13540 from liewegas/wip-debug-osd

osd: debug con in ms_handle_connect

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #13565 from liewegas/wip-fix-pool-race-test

ceph_test_rados_api_misc: avoid livelock from PoolCreationRace

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #13568 from dillaman/wip-18990

rbd-mirror: deleting a snapshot during sync can result in read errors

Reviewed-by: Mykola Golub <mgolub@mirantis.com>

Merge pull request #13589 from dachary/wip-crush-doc

doc: crush: API documentation fixes

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #13590 from linuxbox2/wip-badbuck

rgw_file: avoid stranding invalid-name bucket handles in fhcache

Merge pull request #13324 from dongbula/make-sync-thread-name-clear

rgw: make sync thread name clear

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #13361 from ddiss/rbdmap_unmap_from_file

rbdmap: consider /etc/ceph/rbdmap when unmapping images

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #13387 from Jing-Scott/clean-up-rgw-list-redundant-code

rgw: clean up the redundant assignment in last_entry_in_listing

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #13460 from theanalyst/fix/rgw-rest-user-fixme

wip: rgw: rest_admin/user avoid double checking input args

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #13461 from theanalyst/doc/rgw-system-user

doc: rgw: make a note abt system users vs normal users

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #13576 from jronak/doc

rgw:Fixes typo in rgw_admin.cc

Reviewed-by: Abhishek Lekshmanan <abhishek@suse.com>
Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #13578 from Liuchang0812/wip-fix-typo-in-rgw

rgw: fix typo in comment

Reviewed-by: Casey Bodley <cbodley@redhat.com>

rgw_file: avoid stranding invalid-name bucket handles in fhcache

To avoid a string copy in the common mkdir path, handles for
proposed buckets currently are staged in the handle table, before
being rejected.  They need to be destaged, not just marked deleted
(because deleted objects are now assumed not to be linked, as of
beaeff059375b44188160dbde8a81dd4f4f8c6eb).

This triggered an unhandled Boost assert when deleting staged
handles, as current safe_link mode requires first removing from
the FHCache.

Fixes: http://tracker.ceph.com/issues/19036
Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>

Merge pull request #12897 from renhwztetecs/renhw-wip-client-cleanup

client: priority to verify the correctness of the "flag"

Reviewed-by: John Spray <john.spray@redhat.com>

Merge pull request #13359 from jcsp/wip-logrotate-sshexception

qa: handle SSHException in logrotate

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #13429 from ukernel/wip-18941

test/libcephfs: avoid buffer overflow when testing ceph_getdents()

Reviewed-by: John Spray <john.spray@redhat.com>

Merge pull request #13455 from ukernel/wip-18953

mds: ignore fs full check for CEPH_MDS_OP_SETFILELOCK

Reviewed-by: John Spray <john.spray@redhat.com>

Merge pull request #13465 from jcsp/wip-mds-voidfn

mds: replace C_VoidFn in MDSDaemon with lambdas

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #13471 from batrick/fs-new-eexist

mon: detect existing fs and duplicate name earlier

Reviewed-by: John Spray <john.spray@redhat.com>

Merge pull request #13572 from theanalyst/doc/changelog/v0.94.10

doc: add changelog for v0.94.10

Reviewed-by: Sage Weil <sweil@redhat.com>
Reviewed-by: Loic Dachary <ldachary@redhat.com>
Reviewed-by: Nathan Cutler <ncutler@suse.com>

Merge pull request #13152 from smithfarm/wip-hammer-release-notes

doc: hammer 0.94.10 release notes

Reviewed-by: Abhishek Lekshmanan <abhishek@suse.com>

Merge pull request #13573 from dachary/wip-17821-revert

ceph-disk: dmcrypt activate must use the same cluster as prepare

Reviewed-by: Kefu Chai <kchai@redhat.com>

rgw: Fixes typo in rgw_admin.cc

Issue: http://tracker.ceph.com/issues/19026
Signed-off-by: Ronak Jain <ronakjain@outlook.in>

rgw: fix typo in comment

Signed-off-by: liuchang0812 <liuchang0812@gmail.com>

ceph-disk: dmcrypt activate must use the same cluster as prepare

When dmcrypt is used, the fsid cannot be retrieved from the data
partition because it is encrypted. Store the fsid in the lockbox to
enable dmcrypt activation using the same logic as regular activation.

The fsid is used to retrive the cluster name that was used during
prepare, reason why activation does not and must not have a --cluster
argument.

Fixes: http://tracker.ceph.com/issues/17821
Signed-off-by: Loic Dachary <ldachary@redhat.com>

rbd-mirror: retry object copy after -ENOENT error

Fixes: http://tracker.ceph.com/issues/18990
Signed-off-by: Jason Dillaman <dillaman@redhat.com>

rbd-mirror: object copy should always reference valid snapshots

If a remote snapshot is deleted while an image sync is in-progress,
associate the read request against the most recent, valid remote
snapshot for a given snapshot object clone.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

rbd-mirror: replace complex object op tuple with struct

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #13515 from smithfarm/wip-17613

rpm: build ceph-resource-agents by default

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Ken Dreyer <kdreyer@redhat.com>

doc: add changelog for v0.94.10

Generated as
```
$ git log tags/v0.94.9..tags/v0.94.10 --after="Tue Aug 30 11:33:36 2016 +0000"
```

in order to avoid the qa suite merge commits
Signed-off-by: Abhishek Lekshmanan <alekshmanan@suse.com>

Revert "ceph-disk: change get_dmcrypt_key test to support different cluster name"

Merged by mistake, teuthology ceph-disk fails

This reverts commit 1c747473600a9a7f528ad71a889c35d85ac115b1.

Signed-off-by: Loic Dachary <ldachary@redhat.com>

Revert "ceph-disk: Adding cluster name support for dmcrypt"

Merged by mistake, teuthology ceph-disk fails

This reverts commit d98d4dbc2741295723f0a316b5bf81f07b140041.

Signed-off-by: Loic Dachary <ldachary@redhat.com>

Merge pull request #12623 from LiumxNL/fix-boot-epoch

osd: correct epoch setting of osd boot msg

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #13418 from dreamhost/wip-18928

msg/async: support IPv6 QoS.

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Haomai Wang <haomai@xsky.com>

Merge pull request #13530 from tangwenjun3/wip-tang-nvme

os/bluestore rm unused variable in aio_read()

Reviewed-by: Kefu Chai <kchai@redhat.com>

ceph_test_rados_api_misc: avoid livelock from PoolCreationRace

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #13536 from Liuchang0812/wip-add-override-in-rbd-headers

librbd: add override keyword in header files

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #13558 from ceph/wip-19016-remove-mon_osd_min_down_reports

doc: remove mentions about mon_osd_min_down_reports

Reviewed-by: Ken Dreyer <kdreyer@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #12295 from wjwithagen/wip-wjw-TEMP_FAILURE

rgw/rgw_main.cc: fix parenteses and function result

Reviewed-by: Matt Benjamin <mbenjamin@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #13291 from prallabh/obj_stat

rgw: Let the object stat command be shown in the usage

Reviewed-by: Casey Bodley <cbodley@redhat.com>

doc: remove mentions about mon_osd_min_down_reports

Fixes: http://tracker.ceph.com/issues/19016
Signed-off-by: Bara Ancincova (bara@redhat.com)

Merge pull request #13550 from tchaikov/wip-18968

mon/MonClient: persist global_id across re-connecting

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

librbd: add override in header files

Fixes: http://tracker.ceph.com/issues/19012
Signed-off-by: liuchang0812 <liuchang0812@gmail.com>

rgw: Let the object stat command be shown in the usage

Fixes: http://tracker.ceph.com/issues/19013
Signed-off-by: Pavan Rallabhandi <PRallabhandi@walmartlabs.com>

mon/MonClient: persist global_id across re-connecting

MonClient should re-use previously assigned global-id after
re-connecting. and should not have its global-id changed once it is
authenticated unless monitor thinks otherwise.

Fixes: http://tracker.ceph.com/issues/18968
Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #13346 from batrick/mds-doc-fix

doc: update to new ceph fs commands

Reviewed-by: John Spray <john.spray@redhat.com>

Merge pull request #13529 from linuxbox2/wip-dir-orphan

rgw: wip dir orphan

rgw_file: fix marker computation

Fixes: http://tracker.ceph.com/issues/19018
Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>

Merge pull request #13447 from dillaman/wip-13025

librbd: scatter/gather support for the C API

Reviewed-by: Mykola Golub <mgolub@mirantis.com>

Merge pull request #13537 from liewegas/wip-19005

messages/MOSDOp: fix pg_t decoding for version <7 decoding

Reviewed-by: Ilya Dryomov <idryomov@gmail.com>

Merge pull request #13385 from zhangsw/feature-lifecycle-noncurrent

rgw: add support for noncurrentversion expiration in s3 lifecycle.

Merge pull request #13518 from tchaikov/wip-fix-pgp-num

test: Thrasher: do not update pools_to_fix_pgp_num if nothing happens

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #12223 from cbodley/wip-rgw-meta-sync-yield

rgw: fix for broken yields in RGWMetaSyncShardCR

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>

Merge pull request #13517 from liewegas/wip-kraken-x

qa/suites/upgrade/kraken-x

Reviewed-by: Yuri Weinstein <yweins@redhat.com>

Merge pull request #13538 from Adirl/ENOMEM

msg/async/rdma: Print error only on ENOMEM

Reviewed-by: Haomai Wang <haomai@xsky.com>

Merge pull request #13539 from Adirl/destructor

msg/async/rdma: update destructor message

Reviewed-by: Haomai Wang <haomai@xsky.com>

osd: debug con in ms_handle_connect

Signed-off-by: Sage Weil <sage@redhat.com>

msg/async/rdma: update destructor message

issue:939258

Change-Id: Ib00bb5661a0e80fb258babb63d23458daa8b7025
Signed-off-by: Sarit Zubakov <saritz@mellanox.com>

msg/async/rdma: Print error only on ENOMEM

issue: 931114

Change-Id: I1e1710245c2c056bd2b8cccae9d96676a8f49678
Signed-off-by: Sarit Zubakov <saritz@mellanox.com>

messages/MOSDOp: fix pg_t decoding for version <7 decoding

This was broken by f6e219a4df8553336839bb914733a78090561ec8

Fixes: http://tracker.ceph.com/issues/19005
Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #13065 from ceph/wip-fuse-client-config-edits

doc: minor changes in fuse client config reference

Reviewed-by: John Spray <john.spray@redhat.com>

librbd: new rbd_aio_readv/rbd_aio_writev API methods

The RBD C API now has scatter/gather IO support via the API. The C++
API uses bufferlists which can also support scatter/gather IO.

Fixes: http://tracker.ceph.com/issues/13025
Signed-off-by: Jason Dillaman <dillaman@redhat.com>

librbd: ReadResult helper class for reassembling partial reads

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

librbd: move IO-related classes to new io namespace

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #13528 from yuyuyu101/wip-fix-rdma-fd

msg/async/rdma: don't need to delete event when tcp connection isn't …

Reviewed-by: Adir lev <adirl@mellanox.com>

os/bluestore rm unused variable in aio_read()

Signed-off-by: tangwenjun <tang.wenjun3@zte.com.cn>

Merge pull request #13468 from vumrao/wip-vumrao-18932

auth: 'ceph auth import -i' overwrites caps, if caps are not specified

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #13267 from yuyuyu101/wip-fix-lock

msg/simple: fix missing unlock when already bind

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #10125 from stiopaa1/msg_asyncmess_changeReturnTypeIntToVoid

msg/AsyncMessenger: remove unused method

Reviewed-by: Haomai Wang <haomai@xsky.com>

Merge pull request #13339 from wjwithagen/wip-wjw-include-ceph_features

common: include/ceph_features.h uses uint64_t, which is in sys/types.h

Reviewed-by: Sage Weil <sage@redhat.com>

rgw_file: rgw_readdir can't list multi-segment dirs

This issue has one root cause in librgw, namely that the marker
argument to these requests was incorrectly formatted (though the
marker cache was working as intended).

Secondarily, for nfs-ganesha users, there is a compounding issue
that the RGW fsal was required by "temporary" convention to
populate the entire dirent cache for a directory on a single
readdir() invocation--the cache_inode/mdcache implementations
invariantly pass (before future 2.5 changesets, currently in
progress) a null pointer for the start cookie offset, intended
to convey this.

Fixes: http://tracker.ceph.com/issues/18991
Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>

rgw_file: allow setattr on placeholder directories

When a POSIX path <bucket>/foo/ is known only as an implicit path
segment from other objects (e.g., <bucket>/foo/bar.txt), a case
that would usually arise from S3 upload of such an object, an
RGWFileHandle object representing "<bucket>/foo/" will be constructed
as needed, with no backing in RGW.

This is by design, but subsequently, if a setattr is performed on
such a handle, we must be ready to create the object inline with
storing the attributes.

Fixes: http://tracker.ceph.com/issues/18989
Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>

rgw_file: invalid use of RGWFileHandle::FLAG_EXACT_MATCH

The change which introduced this flag also caused it to be
given as the flags argument to RGWLibFS::stat_leaf() when called
from rgw_lookup().

This was incorrect:  in particular, when a directory is known only
as a common prefix of other objects, the AWS namespace mapping
convention requires lookup("foo") to match a non-materialized
instance of "foo/" (case 2 in RGWLibFS::stat_leaf's stat loop).

Fixes: http://tracker.ceph.com/issues/18992
Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>

msg/async/rdma: don't need to delete event when tcp connection isn't built

Signed-off-by: Haomai Wang <haomai@xsky.com>

auth: 'ceph auth import -i' overwrites caps, if caps are not specified
in given keyring file, should alert user and should not allow this import.
Because in 'ceph auth list' we keep all the keyrings with caps and importing
'client.admin' user keyring without caps locks the cluster with error[1]
because admin keyring caps are missing in 'ceph auth'.

[1] Error connecting to cluster: PermissionDeniedError

Fixes: http://tracker.ceph.com/issues/18932
Signed-off-by: Vikhyat Umrao <vumrao@redhat.com>

Merge pull request #13519 from smithfarm/wip-drop-boost-devel

rpm: drop boost build dependencies

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #13524 from tchaikov/wip-debian-drop-boost-builddeps

debian: drop boost build dependencies

Reviewed-by: Nathan Cutler <ncutler@suse.com>

test/ceph-test-objectstore: Don't always include BlueStore code

 - FreBSD does not do Bluestore, so TESTS added using
   BlueStore need to be conditional on HAVE_LIBAIO

Signed-off-by: Willem Jan Withagen <wjw@digiware.nl>

Merge pull request #10487 from yangdongsheng/rbd_export

rbd: introduce v2 format for rbd export/import

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

rbd: import/export/mergediff: use STD[IN|OUT]_FILENO instead of 0 or 1

Signed-off-by: Dongsheng Yang <dongsheng.yang@easystack.cn>

rbd: import: use read() instead of lseek() when from_stdin is true

Signed-off-by: Dongsheng Yang <dongsheng.yang@easystack.cn>

rbd: add docs for new format of rbd export.

Signed-off-by: Dongsheng Yang <dongsheng.yang@easystack.cn>

rbd: combine the namespaces of export_diff and export_full, import_diff and import.

Signed-off-by: Dongsheng Yang <dongsheng.yang@easystack.cn>

rbd: import: decode file data in le64 for compatability on different platform

Signed-off-by: Dongsheng Yang <dongsheng.yang@easystack.cn>

rbd: import_diff/export_diff: encode length for each tag.

Signed-off-by: Dongsheng Yang <dongsheng.yang@easystack.cn>

rbd: refactor export/import to v1 and v2 methods.

Signed-off-by: Dongsheng Yang <dongsheng.yang@easystack.cn>

rbd: use the unified --export-format option for rbd export and rbd import

Signed-off-by: Dongsheng Yang <dongsheng.yang@easystack.cn>

qa/workunit/rbd: import_export.sh: add image priorities exporting and importing test cases.

Signed-off-by: Dongsheng Yang <dongsheng.yang@easystack.cn>

rbd: import/export image stripe_unit and stripe_count.

Signed-off-by: Dongsheng Yang <dongsheng.yang@easystack.cn>

rbd: import/export features of image

Signed-off-by: Dongsheng Yang <dongsheng.yang@easystack.cn>

rbd: import/export image_order.

Signed-off-by: Dongsheng Yang <dongsheng.yang@easystack.cn>

qa/workunit/rbd: add test case for --import/export-format in import_export.sh

Signed-off-by: Dongsheng Yang <dongsheng.yang@easystack.cn>

rbd: introduce --export-format to export command

Signed-off-by: Dongsheng Yang <dongsheng.yang@easystack.cn>

rbd: introduce --import-format to import command

Signed-off-by: Dongsheng Yang <dongsheng.yang@easystack.cn>

rbd: refactor do_import_diff to accept fd.

Signed-off-by: Dongsheng Yang <dongsheng.yang@easystack.cn>

rbd: refactor do_export_diff to accept fd

Signed-off-by: Dongsheng Yang <dongsheng.yang@easystack.cn>

rbd: move do_import_diff out from namespace of import_diff

Signed-off-by: Dongsheng Yang <dongsheng.yang@easystack.cn>