rbd: process crushed, rbd creating with striping parameters

Signed-off-by: Na Xie <xie.na@h3c.com>

Merge pull request #6790 from kylinstorage/wip-subprocess

SubProcess: fix multiple definition bug

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #5921 from caibo2014/add-tool-feature

tools:support printing part cluster map in readable fashion

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #6906 from HeyoQiang/heyongqiang1212

client: modify a word in log

Reviewed-by: Loic Dachary <ldachary@redhat.com>

Merge pull request #6866 from xiaoxichen/umount_before_mount

init-ceph: do umount when the path exists.

Reviewed-by: Loic Dachary <ldachary@redhat.com>

init-ceph: do umount when the path exists.

If the specified mount point is in use, umount it instead
of skipping mounting the fs.

In previous code , if we forgot that we already mount something unrelated to
*this ceph osd * to the particular mount point, we will skip the mount and
finally get an error complaining superblock not matching OSD ID.

umount & remount is better because
1. If the wrong FS not in use, we can get the right FS we want and make ceph boot smoothly.
2. If the wrong FS is in use, we will get EBUSY on umount, which seems explain the situation
more clearly than superblock mismatch.

Signed-off-by: Xiaoxi Chen <xiaoxi.chen@intel.com>

Merge pull request #6905 from H3C/wip-bugfix-mount1

mount.ceph: memory leaks

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #6908 from nilamdyuti/wip-doc-rbdsnapshot

doc: Modified a note section in rbd-snapshot doc.

Reviewed-by: Loic Dachary <ldachary@redhat.com>

mount/mtab.cc: memory leaks
the free() should be called to free resources, in order to avoid memory leaks

Fixes:#14066
Signed-off-by: Qiankun Zheng <zheng.qiankun@h3c.com>

client: modify a word in log

This modification changed the word "comit" in line 7704
/* ldout(cct, 10) << "sync_write_comit -- no more unsafe writes, unmount can proceed" << dendl; */
by the word "commit";

Signed-off-by: YongQiang He<he.yongqiang@h3c.com>

doc: Modified a note section in rbd-snapshot doc.

Elaborates the note section a bit more as to why stopping I/O before
taking a snapshot is recommended and how to do it.

Signed-off-by: Nilamdyuti Goswami <ngoswami@redhat.com>

Merge pull request #6901 from yehudasa/wip-mstart

vstart.sh: add mstart, mstop, mrun wrappers for running multiple vstart-style test clusters out of src tree

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #6698 from mathslinux/auth-crash-fix

auth: fix crash when bad keyring is passed

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #6890 from dachary/wip-13986-test-pg-scrub

tests: fix failure for osd-scrub-snap.sh

Reviewed-by: Xinze Chi <xinze@xsky.com>
Reviewed-by: Ning Yao <zay11022@gmail.com>

vstart.sh: mon osd reporter subtree level = osd

Since everything is on one host!

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #6709 from xiaoxichen/failure_by_host

mon: support min_down_reporter by subtree level (default by host)

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Sage Weil <sage@redhat.com>

mstop.sh, mrgw.sh: more scripts to control daemons in dev env

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

mrun: a tool to run commands against specific mstart cluster

E.g.,

$ ./mstart.sh foo -n
$ ./mrun foo ceph -s

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

mstart.sh: run multiple clusters in a single source tree

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

Merge pull request #6823 from oritwas/wip-cmake-fix-authtool

cmake: add TracepointProvider.cc to libcommon

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #6827 from trociny/wip-scrub-debug

Little improvements for troubleshooting scrub issues

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #6732 from aalba6675/patch-custom-cluster-name

sysvinit: allow custom cluster names

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #6730 from kylinstorage/fix-refcounter-in-replica_scrub

osd: fix MOSDRepScrub reference counter in replica_scrub

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #6713 from zhanghuan/fix-wbthrottle

osd: fix incorrect throttle in WBThrottle

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #6731 from kylinstorage/clean_up_objecter

librados: clean up Objecter.h

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #6863 from jdurgin/wip-buffer-header

buffer: make usable outside of ceph source again

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #6750 from theanalyst/doc/mailmap/jewel

mailmap: Jewel updates

Reviewed-by: Loic Dachary <ldachary@redhat.com>

Merge pull request #6872 from dachary/wip-osd-typo

doc: osd: s/schedued/scheduled/

Reviewed-by: Abhishek Lekshmanan <abhishek.lekshmanan@gmail.com>

mailmap: YankunLi affiliation

Signed-off-by: Abhishek Lekshmanan <abhishek.lekshmanan@gmail.com>

mailmap: Nishtha Rai affiliation

Signed-off-by: Abhishek Lekshmanan <abhishek.lekshmanan@gmail.com>

mailmap: John Coyle affiliation

Signed-off-by: Abhishek Lekshmanan <abhishek.lekshmanan@gmail.com>
Conflicts:
.organizationmap
Trivial merge conflict as upstream had an additional entry

mailmap: Jashan Kamboj affiliation

Signed-off-by: Abhishek Lekshmanan <abhishek.lekshmanan@gmail.com>

mailmap: Adam Kupczyk affiliation

Signed-off-by: Abhishek Lekshmanan <abhishek.lekshmanan@gmail.com>

mailmap: Jacek J. Lakis affiliation

Signed-off-by: Abhishek Lekshmanan <abhishek.lekshmanan@gmail.com>

mailmap: runsisi affiliation

Signed-off-by: Abhishek Lekshmanan <abhishek.lekshmanan@gmail.com>

mailmap: Zeqiang Zhuang affiliation

Signed-off-by: Abhishek Lekshmanan <abhishek.lekshmanan@gmail.com>

mailmap: You Ji affiliation

Signed-off-by: Abhishek Lekshmanan <abhishek.lekshmanan@gmail.com>

mailmap: Wenjun Huang affiliation

Signed-off-by: Abhishek Lekshmanan <abhishek.lekshmanan@gmail.com>

mailmap: Wei Luo affiliation

Signed-off-by: Abhishek Lekshmanan <abhishek.lekshmanan@gmail.com>

mailmap: Sangdi Xu name normalization

Signed-off-by: Abhishek Lekshmanan <abhishek.lekshmanan@gmail.com>

mailmap: Robin H. Johnson name normalization

Signed-off-by: Abhishek Lekshmanan <abhishek.lekshmanan@gmail.com>

mailmap: Guang Yang name normalization

Signed-off-by: Abhishek Lekshmanan <abhishek.lekshmanan@gmail.com>

Merge pull request #6783 from yuyuyu101/wip-13961

osd: KeyValueStore: don't queue NULL context

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #6892 from kylinstorage/wip-fix-rbd-tracepoint

librbd: fix tracepoint parameter in diff_iterate

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #6878 from manens/pybind_rados_ioctx_snap_rollback

pybind: Implementation of rados_ioctx_snapshot_rollback

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #6889 from kylinstorage/wip-fix-merge-diff

librbd: fix merge-diff for >2GB diff-files

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #6888 from prallabh/master

rgw: fix a glaring syntax error

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>

tests: fix failure for osd-scrub-snap.sh

Reason to get a false negative only one osd, when calling kill_daemons
$dir KILL osd, which means kill -9 shutdown() would not be called to
notify monitor so that it remain active+clean and osd up until monitor
got a mon_report timeout.  Thus wait_for_clean lose it function and when
we call ceph pg scrub, we got an EAGAIN error and monitor will ignore
the scrub message because of osd not up

Signed-off-by: Ning Yao <zay11022@gmail.com>

tests: kill_daemons uses TERM instead of KILL

When called to teardown a test, kill_daemon should use KILL to ensure
all leftovers are removed as quickly as possible to leave a clean state
for the next test. However, when kill_daemons is called to shutdown a
given daemon from within a test, it should use TERM by default so the
daemon has time to notify the MON that it goes down. For instance, if
KILLing an OSD, the mon will still report it as being up although the
calling function probably expects that it will be marked out.

Signed-off-by: Loic Dachary <loic@dachary.org>

librbd: fix merge-diff for >2GB diff-files

from http://tracker.ceph.com/issues/14030

Fixes: #14030
Signed-off-by: Yunchuan Wen <yunchuan.wen@kylin-cloud.com>

rgw: fix a glaring syntax error

syntactical error is causing rgw not to start

Signed-off-by: Pavan Rallabhandi <pavan.rallabhandi@sandisk.com>

Implementation of rados_ioctx_snapshot_rollback

Signed-off-by: Florent Manens <florent@beezim.fr>

Merge pull request #6858 from jcsp/wip-cephfs-auth-doc

doc: note that cephfs auth stuff is new in jewel

Merge pull request #6768 from marcan/cython

pybind: port the rbd bindings to Cython

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Nathan Cutler <ncutler@suse.com>
Reviewed-by: Ken Dreyer <kdreyer@redhat.com>
Reviewed-by: Alfredo Deza <adeza@redhat.com>

Merge pull request #6877 from dillaman/wip-14001

librbd: fix test case race condition for journaling ops

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

librbd: fix test case race condition for journaling ops

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

Revert "LifeCycle feature"

It's not there yet. Created a branch with a few fixes, will need to go
through some more revisions.

This reverts commit 7d48f62f5c86913d8f00b44d46a04a52d338907c.

Merge pull request #6331 from chenji-kael/wip-lifecycle

LifeCycle feature

Merge pull request #6871 from dachary/wip-14027-scrub-load

tests: --osd-scrub-load-threshold=2000 for more consistency

Reviewed-by: Xinze Chi <xinze@xsky.com>

tests: --osd-scrub-load-threshold=2000 for more consistency

In a test environment, consistency is more important than
performances. Effectively disable the test that would postpone a scrub
depending on the load average. It is assumed that a machine with a load
average higher than 2000 won't be useable anyway.

http://tracker.ceph.com/issues/14027 Refs: #14027

Signed-off-by: Loic Dachary <loic@dachary.org>

pybind/rbd.pyx: misc typo bugfixes

pybind/rbd.pyx: PyString -> PyBytes

doc: osd: s/schedued/scheduled/

Signed-off-by: Loic Dachary <loic@dachary.org>

Merge pull request #6830 from vumrao/wip-12112-vumrao

rbd: add --object-size option, deprecate --order

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

buffer: make usable outside of ceph source again

Add a rados/buffer_fwd.h symlink and remove the "include/" prefix from
buffer.h.

Signed-off-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #6857 from dachary/wip-13986-test-pg-scrub

tests: osd-scrub-snaps.sh to display full osd logs on error

Reviewed-by: Xinze Chi <xinze@xsky.com>

doc: note that cephfs auth stuff is new in jewel

+ bonus whitespace fixes to the code blocks to
get them to render properly

Signed-off-by: John Spray <john.spray@redhat.com>

tests: osd-scrub-snaps.sh to display full osd logs on error

This generates a large output but is the only way to get more
information about race conditions that turn out to show up frequently.

http://tracker.ceph.com/issues/13986 Refs: #13986

Signed-off-by: Loic Dachary <loic@dachary.org>

Merge pull request #6850 from H3C/wip-zcl-ceph-mds

ceph-mds:add --help/-h

Reviewed-by: John Spray <john.spray@redhat.com>

Merge pull request #6848 from H3C/wip-zcl-layout-rst

doc:file must be empty when writing layout fields of file use "setfattr"

Reviewed-by: John Spray <john.spray@redhat.com>

rbd: add --object-size option

Object size can be specified when creating an image with the --order option,
as a number of bits in the size.

This patch is adding new option --object-size. This new option will specify
object size directly for example --object-size 2M.

It would be easier to use. --order is still present for backwards compatibility.
For simplicity, we are rounding up the object size to the nearest power of 2.

Fixes #12112

Signed-off-by: Vikhyat Umrao <vumrao@redhat.com>

ceph-mds:add --help/-h

When use ceph-mds --hlep/-h, the usage will be showed

Signed-off-by: Cilang Zhao <zhao.cilang@h3c.com>

doc:file must be empty when writing layout fields of file use "setfattr"

When the layout fields of a file are modified, this file must be empty, otherwise an error would occur.

Signed-off-by:Cilang Zhao zhao.cilang@h3c.com

Merge remote-tracking branch 'ceph/jewel'

Merge pull request #6844 from dachary/wip-13997-centos-libunwind-jewel

build/ops: enable CR in CentOS 7

Reviewed-by: Brad Hubbard <bhubbard@redhat.com>

build/ops: enable CR in CentOS 7

To get libunwind from the CR repositories until CentOS 7.2.1511 is released.

http://tracker.ceph.com/issues/13997 Fixes: #13997

Signed-off-by: Loic Dachary <loic@dachary.org>

Merge pull request #6842 from badone/wip-13997

tests: centos7 needs the Continuous Release (CR) Repository enabled for

Reviewed-by: Loic Dachary <ldachary@redhat.com>

tests: centos7 needs the Continuous Release (CR) Repository enabled for
libunwind

Fixes: #13997
Signed-off-by: Brad Hubbard <bhubbard@redhat.com>

LifeCycle feature
As same as amazon S3 interface,"PUT Bucket lifecycle" and
"DELETE Bucket lifecycle" have been implemented,
"GET Bucket lifecycle" not realized yet as S3cmd has not
realize it also.
The feature`s main point is to remove expire file per day.
Files transfer from hot layer to cold layer is not supported.
ToDo:Maybe to transfer from replicate pool to EC pool or
from ssd to sata pool will be valuable.

Now put all buckets which should do lifecycle into shard
objects in .rgw.lc pool.

lifecycle config file format:
<LifecycleConfiguration>
    <Rule>
        <ID>sample-rule</ID>
        <Prefix></Prefix>
        <Status>enable</Status>
        <Expiration>
           <Days>1</Days>
        </Expiration>
    </Rule>
</LifecycleConfiguration>

Signed-off-by: Ji Chen <insomnia@139.com>

Merge pull request #6818 from trociny/fix-cmake

cmake: update for recent rbd changes

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #6837 from dreamhost/wip-test-cephtool-filenames

qa/workunits/cephtool/test.sh: false positive fail on /tmp/obj1.

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

qa/workunits/cephtool/test.sh: false positive fail on /tmp/obj1.

If /tmp/obj1 happened to exist already, and was not writable by the
testing user, then this test failed!

Signed-off-by: Robin H. Johnson <robin.johnson@dreamhost.com>

Merge pull request #6838 from stiopaa1/Wreorder_ceph_context

common/ceph_context.cc:fix order of initialisers

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #6831 from fullerdj/wip-14010

qa/krbd: Expunge generic/247

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #6793 from dzafman/wip-hist-les-fix

osd: Allow repair of history.last_epoch_started using config

Reviewed-by: Samuel Just <sjust@redhat.com>

common/ceph_context.cc:fix order of initialisers

Fix order of member initialisers causing [-Wreorder] warning.

Signed-off-by: Michal Jarzabek <stiopa@gmail.com>

Merge branch 'unlinkcallbyrmbucket'

Conflicts:
src/rgw/rgw_bucket.cc

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>

rgw:dont update entrypoint when removing bucket

when removing bucket,we unlink the bucket bucket but we don't want
to update the entrypoint here.Because the update_entrypoint's default
value is true,so we need to indicate it to false.

Signed-off-by: Zengran Zhang <zhangzengran@h3c.com>

Merge pull request #6735 from rzarzynski/wip-rgw-clean-putmetabucket

rgw: remove unused variable in RGWPutMetadataBucket::execute

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>

Merge pull request #6829 from LenzGr/master

doc: Fixed incorrect name of a "List Multipart Upload Parts" Response Entity

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>

Merge pull request #6774 from ceph/wip-rgw-cap-fix

rgw: add a missing cap type

osd: fix arithmetic exception when scrub_min_interval is small

Signed-off-by: Mykola Golub <mgolub@mirantis.com>

osd: more scrubber fields in pg query output

Signed-off-by: Mykola Golub <mgolub@mirantis.com>

osd: use hexdump when logging CRC values

Signed-off-by: Mykola Golub <mgolub@mirantis.com>

qa/krbd: Expunge generic/247

xfstests generic/247 exercises XFS DIO and AIO to detect races. Some
races apparently exist in the XFS code as this is a known issue within
XFS. Expunge this test because it is not specifically relevant to krbd
and not a specific krbd issue.

Signed-off-by: Douglas Fuller <dfuller@redhat.com>

tests: make sure no segfault occurs when using some bad keyring

Signed-off-by: Dunrong Huang <riegamaths@gmail.com>

cmake: add TracepointProvider.cc to libcommon

Signed-off-by: Orit Wasserman <owasserm@redhat.com>

Merge pull request #6805 from zaitcev/cmake-pluginregistry

cmake: Add common/PluginRegistry.cc to CMakeLists.txt

doc: Fixed incorrect name of a "List Multipart Upload Parts" Response Entity

Fixed incorrect name of a "List Multipart Upload Parts" Response Entity:
"InitiatedMultipartUploadsResult" -> "ListPartsResult"

Fixes: #14003
Signed-off-by: Lenz Grimmer <lenz@grimmer.com>

cmake: update for recent rbd changes

Signed-off-by: Mykola Golub <mgolub@mirantis.com>