v11.0.0

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #9889 from Yan-waller/yj-wip-rgwbucketcc

rgw: add string describing for error number

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #9888 from weiqiaomiao/wqm-wip-clean-swiftapi

rgw: duplicate check for acl op in RGWHandler_REST_Bucket_SWIFT::op_get()

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #9854 from Yan-waller/yj-wip-rgwunlinkbucket

rgw: remove unused bufferlist variable

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #9822 from Yan-waller/yj-wiph-rgwfixinitwatch

rgw: fix the missing return value of init_watch

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #9808 from Yan-waller/yj-wip-rgwtoolsinitreturn

rgw: add error return code checking for rgw_tools_init and rgw_perf_start

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #9796 from Yan-waller/yj-wip-rgwremoveuser

rgw: miscellaneous cleanups

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #9780 from theanalyst/rgw/create-bucket-cleanup

rgw_bucket: remove unused bufferlist

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #9987 from jcsp/wip-cmake-fortify-source

cmake: fix FORTIFY_SOURCE check

Reviewed-by: Kefu Chai <kchai@redhat.com>

cmake: fix FORTIFY_SOURCE check

This got broken in 6f3e3cf6.  The -Werror part wasn't
having the desired effect, so we were enabling FORTIFY_SOURCE
even when in -O0

Signed-off-by: John Spray <john.spray@redhat.com>

Merge pull request #9984 from tchaikov/wip-fix-test_crush_bucket.sh

test/test_crush_bucket.sh: source detect-build-env-vars.sh for settin…

Reviewed-by: Loic Dachary <ldachary@redhat.com>

Merge pull request #9771 from yuyuyu101/wip-async-priority

msg/async: set socket priority for heartbeat socket

Reviewed-by: Greg Farnum <gfarnum@redhat.com>

Merge pull request #9958 from dillaman/wip-16489

rbd-mirror: clean up ImageReplayer before stopping state machine

Reviewed-by: Mykola Golub <mgolub@mirantis.com>

Merge pull request #9923 from dillaman/wip-16471

librbd: removal of partially deleted image needs id lookup

Reviewed-by: Mykola Golub <mgolub@mirantis.com>

test/test_crush_bucket.sh: source detect-build-env-vars.sh for settings paths

this fixes errors like

../qa/workunits/ceph-helpers.sh:354: run_mon:  ceph-mon --id a
--mon-osd-full-ratio=.99 --mon-data-avail-crit=1
--paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0
--erasure-code-dir= --plugin-dir= --debug-mon 20 --debug-ms 20
--debug-paxos 20 --chdir= --mon-data=testdir/testcrushbucket/a
'--log-file=testdir/testcrushbucket/$name.log'
'--admin-socket=testdir/testcrushbucket/$cluster-$name.asok'
--mon-cluster-log-file=testdir/testcrushbucket/log
--run-dir=testdir/testcrushbucket
'--pid-file=testdir/testcrushbucket/$name.pid'
2016-06-28 11:24:24.105884 2b1bbd3d0380 -1 load
dlopen(/libec_jerasure.so): /libec_jerasure.so: cannot open shared
object file: No such file or directory
../qa/workunits/ceph-helpers.sh:371: run_mon:  return 1

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #9539 from cxwshawn/wip-tc9-fix

librbd: optimize header file dependency and modify related files

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #9500 from zhuangzeqiang/wip-16056

librbd: restrict mirror enable/disable actions on parents/clones

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #9623 from rjfd/wip-15239

rbd-mirror: Throttle in-flight image syncs to only a N concurrent

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #9856 from hzhang-wx/master

librbd: discard should return error if length too large

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

rbd-mirror: tests: ImageSyncThrottler unit tests

Signed-off-by: Ricardo Dias <rdias@suse.com>

rbd-mirror: Usage of image-sync throttler in BootstrapRequest

Fixes: http://tracker.ceph.com/issues/15239
Signed-off-by: Ricardo Dias <rdias@suse.com>

rbd-mirror: Implementation of image-sync throttler

Signed-off-by: Ricardo Dias <rdias@suse.com>

Merge pull request #9699 from rjfd/wip-15108

rbd-mirror:  Update the sync point object number during sync

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

msg/async: set socket priority for heartbeat socket

To prioritize the heartbeat packets in the the diffserv-aware device.

Learn/copy from https://github.com/ceph/ceph/pull/3353

Signed-off-by: Haomai Wang <haomai@xsky.com>

Merge pull request #8086 from jaemyoun/wip-doc-rados-java-api

doc: adding JAVA Example in Step 4: Closing Sessions

Reviewed-by: Kefu Chai <kchai@redhat.com>

doc: add closing session in Java

Merge pull request #9973 from xiexingguo/xxg-wip-fix-blob-ut-failure

os/bluestore: fix ut failure due to incorrect test instance

Reviewed-by: Sage Weil <sage@redhat.com>

os/bluestore: fix ut failure due to incorrect test instance

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

Merge pull request #9795 from cyb70289/yibocai

common: Improve linux dcache hash algorithm

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Samuel Just <sjust@redhat.com>

Merge pull request #9824 from stiopaa1/msg_asyncmess_removeCount

msg/async: remove unneeded use of count

Reviewed-by: Haomai Wang <haomai@xsky.com>

Merge pull request #9861 from songbaisen/a12

 mon: remove the redundant min_last_epoch_clean judge in PGMap

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Samuel Just <sjust@redhat.com>

Merge pull request #9937 from stiopaa1/msg_asyncMess_moveWorkerClass

msg/AsyncMessenger: move Worker class to cc file

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #9966 from stiopaa1/msg_asyncmess_addOverride

msg/AsyncMessenger.h: add override to virtual func

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #9919 from tchaikov/wip-cmake-ceph-disk

cmake,test: run ceph-disk tests with cmake

Reviewed-by: Ali Maredia <amaredia@redhat.com>

Merge pull request #9897 from liewegas/wip-cmake

init-ceph: make it behave when run from a cmake build dir

README.md: update a few things

Signed-off-by: Sage Weil <sage@redhat.com>

README.md: fix whitespace

Signed-off-by: Sage Weil <sage@redhat.com>

librbd: removal of partially deleted image needs id lookup

Several operations depend on the image id but if the image cannot be
opened to retrieve the id, these cleanup operations cannot be executed.

Fixes: http://tracker.ceph.com/issues/16471
Signed-off-by: Jason Dillaman <dillaman@redhat.com>

msg/AsyncMessenger.h: add override to virtual func

Signed-off-by: Michal Jarzabek <stiopa@gmail.com>

Merge pull request #9945 from rjfd/wip-16488

rbd-mirror: fix potential race condition in resync listener removal

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
Reviewed-by: Mykola Golub <mgolub@mirantis.com>

Merge pull request #9962 from liewegas/wip-bluestore-unused

os/bluestore: replace interval_set with bitmap for unused

Reviewed-by: Sage Weil <sage@redhat.com>

os/bluestore: replace bluestore_blob_t::unused from interval to bitmap

Signed-off-by: Igor Fedotov <ifedotov@miranitis.com>

os/bluestore: move add_unused call after csum initialization as it should depend on chunk size.

Signed-off-by: Igor Fedotov <ifedotov@mirantis.com>

os/bluestore: introduce bluestore_blob_t::get_chunk_size method

Signed-off-by: Igor Fedotov <ifedotov@mirantis.com>

os/bluestore: cleanup _pad_zeros to eliminate redundant length param

Signed-off-by: Igor Fedotov <ifedotov@mirantis.com>

Merge pull request #9959 from xiexingguo/xxg-wip-bluestore-cache-clean

os/bluestore: add  some sanity checks

Reviewed-by: Sage Weil <sage@redhat.com>

rbd-mirror: tests: Support for inflight image sync point update

Signed-off-by: Ricardo Dias <rdias@suse.com>

rbd-mirror: image-sync: Periodically update sync point object number

Fixes: http://tracker.ceph.com/issues/15108
Signed-off-by: Ricardo Dias <rdias@suse.com>

os/bluestore: audit buffer bytes before applying delta

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

os/bluestore: fix typo

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

os/bluestore: add sanity check when try to trim onode from cache

We have verified that the onode size to be trimmed is positive ahead,
so "p" shall never point to onode_lru.begin() below here.

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

Merge pull request #9951 from Yan-waller/yj-wip-rgwtestcompilingwarning

rgw/test: use variable type 'size_t' instead of 'int'

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #9839 from renhwztetecs/renhw-wip-cdir-cleanup

mds: unused classes and cleanup

Reviewed-by: John Spray <john.spray@redhat.com>

rbd-mirror: clean up ImageReplayer before stopping state machine

Fixes: http://tracker.ceph.com/issues/16489
Signed-off-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #9677 from stiopaa1/mds_removeUnneededLineAndAddConst

mds/SessionMap.h: remove duplicated line and add const to member functions

Reviewed-by: Greg Farnum <gfarnum@redhat.com>

Merge pull request #9877 from batrick/mds-comm-locks

mds: add missing mds_lock locks

Reviewed-by: John Spray <john.spray@redhat.com>

Merge pull request #9942 from jcsp/wip-doc-netdev

doc: add _netdev to cephfs fstab lines

Merge pull request #9949 from trociny/fix-use-after-free

librbd: potential use after free on refresh error

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

librbd: potential use after free on refresh error

Signed-off-by: Mykola Golub <mgolub@mirantis.com>

rgw/test: use variable type 'size_t' instead of 'int' seems to be more reasonable

Signed-off-by: Yan Jun <yan.jun8@zte.com.cn>

rbd-mirror: image-replayer: Fix bug in resync listener remotion

Fixes: http://tracker.ceph.com/issues/16488
Signed-off-by: Ricardo Dias <rdias@suse.com>

doc: add _netdev to cephfs fstab lines

Signed-off-by: John Spray <john.spray@redhat.com>

rbd discard return -EINVAL if len > MAX_INT32

rbd discard use 'int' to return discarded length, but the 'len' user
passed is 'uint64', in some case, the ret value will be truncated
and return a negative value which means discard failed. ret -EINVAL
if len > MAX_INT32 to indicate support len <= MAX_INT32 only.

Fixes: http://tracker.ceph.com/issues/16465
Signed-off-by: Huan Zhang <zhanghuan@chinac.com>

Merge pull request #9742 from xiexingguo/xxg-wip-server_cmode

mds/server: fix rare race when waitting for osdmap

mds/server: fix rare race when waitting for osdmap

If wait_for_map() returns true, we have got the requested osdmap
at the specified epoch, so we shall try again to do further
verification whether we have the specific pool or not.

The above case can happen because we drop the objecter internal
rwlock during the switch between the objecter->with_osdmap()
and objecter->wait_for_map() methods.

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

test: ceph-disk: run ceph-disk with cmake

* CMakeLists.txt: do not run ceph-disk/run-tox.sh in its source
  directory anymore, "ceph" cli needs to run in the build directory.
* CMakeLists.txt, run-tox.sh:
  - pass $CEPH_BUILD_DIR down to tox.ini, so tox can set the working
    directory to $CEPH_BUILD_DIR to $CMAKE_BINARY_DIR, in which the
    "ceph" cli will be able to figure out where the pybind dso are
    located, and respawn itself with the necessary paths set accordingly.
  - pass $CEPH_BUILD_VIRTUALENV to tox, so ceph-helpers.sh is able to
    set the paths to ceph-disk and ceph-detect-init right.
* run-tox.sh: specify the full path of tox.ini, since we will call
  "tox" in the build directory.
* tox.ini:
  - move the non-flake8 related settings into [testenv:py27] section.
  - always use absolute path, as we will call "tox" in the build
    directory, instead where "tox.ini" lives.

Signed-off-by: Kefu Chai <kchai@redhat.com>

test: ceph-disk.sh: remove unused paths from from $PATH

Signed-off-by: Kefu Chai <kchai@redhat.com>

test: chmod +x ceph-disk.sh

Signed-off-by: Kefu Chai <kchai@redhat.com>

msg/AsyncMessenger: move Worker class to cc file

Signed-off-by: Michal Jarzabek <stiopa@gmail.com>

Merge pull request #9922 from dillaman/wip-16470

librbd: do not propagate mirror status notification failures

Reviewed-by: Mykola Golub <mgolub@mirantis.com>

Merge pull request #9913 from xiexingguo/xxg-wip-fix-bitallocatorut

os/bluestore: fix ut failure when switching to bitmap-allocator

Reviewed-by: Ramesh Chander <Ramesh.Chander@sandisk.com>

os/bluestore: fix memory leak during bit_alloc ut

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

rbd: restrict mirror enable/disable actions on parents/clones

Fixes: http://tracker.ceph.com/issues/16056
Signed-off-by: zhuangzeqiang zhuang.zeqiang@h3c.com

os/bluestore: use std limits for int16_t/int32_t

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

os/bluestore: update comment

We have decreased the default concurrent scans to 1,
so the comment is misleading.

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

os/bluestore: allow total blocks fall into one single zone

So the blusfs ut will be happy with this.

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

os/bluestore: update output of offset/length pair to hex version

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

Merge pull request #9905 from dillaman/wip-16450

librbd: mark exclusive lock as released after journal is closed

Reviewed-by: Mykola Golub <mgolub@mirantis.com>

Merge pull request #9899 from dillaman/wip-16350

librbd: ignore missing object map during snap remove

Reviewed-by: Mykola Golub <mgolub@mirantis.com>

Merge pull request #9896 from dillaman/wip-16446

librbd: delete ExclusiveLock instance when switching to snapshot

Reviewed-by: Mykola Golub <mgolub@mirantis.com>

Merge pull request #9895 from dillaman/wip-16445

librbd: ignore snap unprotect -EBUSY errors during journal replay

Reviewed-by: Mykola Golub <mgolub@mirantis.com>

init-ceph: make it behave when run from a cmake build dir

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #9746 from xiexingguo/xxg-wip-mdcache_hoir

mds/MDCache: fix leak of MMDSOpenInoReply

Reviewed-by: Greg Farnum <gfarnum@redhat.com>

librbd: do not propagate mirror status notification failures

These should be treated as a broadcast since no ACK is required. The
peer will eventually see the notification or its watch will have timed
out and it will re-sync.

Fixes: http://tracker.ceph.com/issues/16470
Signed-off-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #9921 from liewegas/wip-bluestore-fix-discard

os/bluestore: fix discard

Reviewed-by: Igor Fedotov <ifedotov@mirantis.com>

os/bluestore: un-inline some 2Q methods

Signed-off-by: Sage Weil <sage@redhat.com>

os/bluestore: fix buffer discard accounting for 2Q

Signed-off-by: Sage Weil <sage@redhat.com>

os/bluestore: simplify and fix blob invalidation

We can't invalidate based on any particular lextent we drop, as there may
be other lextents (in this onode or in another one) that reference the
same data.  This can break even a single multi-write transaction that
relies on the buffer cache to fill out blocks.

Fix by just doing the unallocated version unconditionally.

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #9920 from zhouyuan/librd_remove

librbd: remove unused code block

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #9916 from xiexingguo/xxg-wip-fix-decode-bnode

os/bluestore: fix decoding pool from bnode key

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #9910 from tchaikov/wip-cmake-drop-ceph-ra

cmake: ocf: drop ceph ra

librbd: memory leak possible if journal op event failed

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

librbd: trivial fix to remove unused code block

Signed-off-by: Yuan Zhou <yuan.zhou@intel.com>

os/bluestore: literal tidy-ups

There error codes are improperly arranged, which looks weird.

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

os/bluestore: make fsck can catch bad bnode key too

The case may be rare but the check shall do no harm.

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

os/bluestore: fix decoding pool from bnode key

The current result is raw and not properly translated.

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

cmake: ocf: drop ceph ra

Signed-off-by: Kefu Chai <kchai@redhat.com>

os/bluestore: default to bitmap allocator

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #9906 from stiopaa1/msg_asyncmess_workepool_deleteCopycontrAndAss

msg/AsyncMessenger:delete cpy constr and assign op

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #9273 from stiopaa1/osd_removeClosingBrace_

osd/OSD.h: remove closing brace } from comment

Reviewed-by: David Zafman <dzafman@redhat.com>