mon: modify the level of a log about OSD's condition in OSDMonitor.cc

    In actual use, we replace old hard disk by new or do performance testing regularly and need to down OSDs manually, but there are some OSDs can not be marked down, and there is no obvious information.
The log { dout(5) << "can_mark_down current up_ratio " << up_ratio << " < min "<< g_conf->mon_osd_min_up_ratio<< ", will not mark osd." << i << "down" << dendl ; }" can explain  why it happened.
   In addition, we can change the value of mon_osd_min_up_ratio more reasonable in our operating environment.so it is necessary to adjust the log's level lower.

For example:
   There are 6 OSDs , wen have marked down 5 of them and the mon_osd_min_up_ratio = 0.3.
   In this situation, when we mark the last OSD to down, it will show "ceph-osd stop/waiting", but in actually, the OSD is still up.

Signed-off-by: Yongqiang He <he.yongqiang@h3c.com>

Merge pull request #6643 from rzarzynski/wip-rgw-slo-rzarzynski

- rgw: swift bulk delete (Radoslaw Zarzynski)
- rgw: static large objects (Radoslaw Zarzynski, Yehuda Sadeh)

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>

Merge pull request #6925 from dillaman/wip-13633

librbd: optionally validate new RBD pools for snapshot support

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

tests: new integration test for validating new RBD pools

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

librbd: optionally validate RBD pool configuration (snapshot support)

Fixes: #13633
Signed-off-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #6917 from majianpeng/journal-iohint

Journal iohint for librbd

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #6923 from dillaman/wip-14060

librbd: correct race conditions discovered during unit testing

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

journal: avoid race between flush request and in-flight flushes

Previously, if the journal object was empty, a flush request could
complete instantly even if there is an in-flight 'record safe'
callback.

Fixes: #14060
Signed-off-by: Jason Dillaman <dillaman@redhat.com>

librbd: fix test case race condition for maintenance requests

Fixes: #14060
Signed-off-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #6922 from liewegas/wip-cephfs-posix

doc/cephfs/posix: update

Merge pull request #6899 from branch-predictor/bp-fix-no-rbd-build

rbd: fix build with "--without-rbd"

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

rbd: fix build with "--without-rbd"

It's not possible to build Ceph configured with "--without-rbd", as
tests and FileStore depends on components provided by RBD. This changeset
fixes this.

Fixes: #14058
Signed-off-by: Piotr Dałek <piotr.dalek@ts.fujitsu.com>

doc/cephfs/posix: update

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #6873 from javacruft/wip/debian-infernalis-patches

ceph-detect-init: Ubuntu >= 15.04 uses systemd

Reviewed-by: Loic Dachary <ldachary@redhat.com>

Merge pull request #6787 from oritwas/wip-run-cmake-check

Wip run cmake check

Reviewed-by: Loic Dachary <ldachary@redhat.com>

journal: add iohint flags for journal replayer.

Signed-off-by: Jianpeng Ma <jianpeng.ma@intel.com>

journal: Add iohint flags for journal write ops.

Signed-off-by: Jianpeng Ma <jianpeng.ma@intel.com>

Merge pull request #6790 from kylinstorage/wip-subprocess

SubProcess: fix multiple definition bug

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #5921 from caibo2014/add-tool-feature

tools:support printing part cluster map in readable fashion

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #6906 from HeyoQiang/heyongqiang1212

client: modify a word in log

Reviewed-by: Loic Dachary <ldachary@redhat.com>

Merge pull request #6866 from xiaoxichen/umount_before_mount

init-ceph: do umount when the path exists.

Reviewed-by: Loic Dachary <ldachary@redhat.com>

init-ceph: do umount when the path exists.

If the specified mount point is in use, umount it instead
of skipping mounting the fs.

In previous code , if we forgot that we already mount something unrelated to
*this ceph osd * to the particular mount point, we will skip the mount and
finally get an error complaining superblock not matching OSD ID.

umount & remount is better because
1. If the wrong FS not in use, we can get the right FS we want and make ceph boot smoothly.
2. If the wrong FS is in use, we will get EBUSY on umount, which seems explain the situation
more clearly than superblock mismatch.

Signed-off-by: Xiaoxi Chen <xiaoxi.chen@intel.com>

Merge pull request #6905 from H3C/wip-bugfix-mount1

mount.ceph: memory leaks

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #6908 from nilamdyuti/wip-doc-rbdsnapshot

doc: Modified a note section in rbd-snapshot doc.

Reviewed-by: Loic Dachary <ldachary@redhat.com>

mount/mtab.cc: memory leaks
the free() should be called to free resources, in order to avoid memory leaks

Fixes:#14066
Signed-off-by: Qiankun Zheng <zheng.qiankun@h3c.com>

client: modify a word in log

This modification changed the word "comit" in line 7704
/* ldout(cct, 10) << "sync_write_comit -- no more unsafe writes, unmount can proceed" << dendl; */
by the word "commit";

Signed-off-by: YongQiang He<he.yongqiang@h3c.com>

doc: Modified a note section in rbd-snapshot doc.

Elaborates the note section a bit more as to why stopping I/O before
taking a snapshot is recommended and how to do it.

Signed-off-by: Nilamdyuti Goswami <ngoswami@redhat.com>

Merge pull request #6901 from yehudasa/wip-mstart

vstart.sh: add mstart, mstop, mrun wrappers for running multiple vstart-style test clusters out of src tree

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #6698 from mathslinux/auth-crash-fix

auth: fix crash when bad keyring is passed

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #6890 from dachary/wip-13986-test-pg-scrub

tests: fix failure for osd-scrub-snap.sh

Reviewed-by: Xinze Chi <xinze@xsky.com>
Reviewed-by: Ning Yao <zay11022@gmail.com>

vstart.sh: mon osd reporter subtree level = osd

Since everything is on one host!

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #6709 from xiaoxichen/failure_by_host

mon: support min_down_reporter by subtree level (default by host)

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Sage Weil <sage@redhat.com>

mstop.sh, mrgw.sh: more scripts to control daemons in dev env

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

mrun: a tool to run commands against specific mstart cluster

E.g.,

$ ./mstart.sh foo -n
$ ./mrun foo ceph -s

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

mstart.sh: run multiple clusters in a single source tree

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

Merge pull request #6823 from oritwas/wip-cmake-fix-authtool

cmake: add TracepointProvider.cc to libcommon

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #6827 from trociny/wip-scrub-debug

Little improvements for troubleshooting scrub issues

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #6732 from aalba6675/patch-custom-cluster-name

sysvinit: allow custom cluster names

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #6730 from kylinstorage/fix-refcounter-in-replica_scrub

osd: fix MOSDRepScrub reference counter in replica_scrub

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #6713 from zhanghuan/fix-wbthrottle

osd: fix incorrect throttle in WBThrottle

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #6731 from kylinstorage/clean_up_objecter

librados: clean up Objecter.h

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #6863 from jdurgin/wip-buffer-header

buffer: make usable outside of ceph source again

Reviewed-by: Sage Weil <sage@redhat.com>

cmake: add run_cmake-check.sh

currently only check to cmake build process

Signed-off-by: Orit Wasserman <owasserm@redhat.com>

Merge pull request #6750 from theanalyst/doc/mailmap/jewel

mailmap: Jewel updates

Reviewed-by: Loic Dachary <ldachary@redhat.com>

Merge pull request #6872 from dachary/wip-osd-typo

doc: osd: s/schedued/scheduled/

Reviewed-by: Abhishek Lekshmanan <abhishek.lekshmanan@gmail.com>

mailmap: YankunLi affiliation

Signed-off-by: Abhishek Lekshmanan <abhishek.lekshmanan@gmail.com>

mailmap: Nishtha Rai affiliation

Signed-off-by: Abhishek Lekshmanan <abhishek.lekshmanan@gmail.com>

mailmap: John Coyle affiliation

Signed-off-by: Abhishek Lekshmanan <abhishek.lekshmanan@gmail.com>
Conflicts:
.organizationmap
Trivial merge conflict as upstream had an additional entry

mailmap: Jashan Kamboj affiliation

Signed-off-by: Abhishek Lekshmanan <abhishek.lekshmanan@gmail.com>

mailmap: Adam Kupczyk affiliation

Signed-off-by: Abhishek Lekshmanan <abhishek.lekshmanan@gmail.com>

mailmap: Jacek J. Lakis affiliation

Signed-off-by: Abhishek Lekshmanan <abhishek.lekshmanan@gmail.com>

mailmap: runsisi affiliation

Signed-off-by: Abhishek Lekshmanan <abhishek.lekshmanan@gmail.com>

mailmap: Zeqiang Zhuang affiliation

Signed-off-by: Abhishek Lekshmanan <abhishek.lekshmanan@gmail.com>

mailmap: You Ji affiliation

Signed-off-by: Abhishek Lekshmanan <abhishek.lekshmanan@gmail.com>

mailmap: Wenjun Huang affiliation

Signed-off-by: Abhishek Lekshmanan <abhishek.lekshmanan@gmail.com>

mailmap: Wei Luo affiliation

Signed-off-by: Abhishek Lekshmanan <abhishek.lekshmanan@gmail.com>

mailmap: Sangdi Xu name normalization

Signed-off-by: Abhishek Lekshmanan <abhishek.lekshmanan@gmail.com>

mailmap: Robin H. Johnson name normalization

Signed-off-by: Abhishek Lekshmanan <abhishek.lekshmanan@gmail.com>

mailmap: Guang Yang name normalization

Signed-off-by: Abhishek Lekshmanan <abhishek.lekshmanan@gmail.com>

Merge pull request #6783 from yuyuyu101/wip-13961

osd: KeyValueStore: don't queue NULL context

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #6892 from kylinstorage/wip-fix-rbd-tracepoint

librbd: fix tracepoint parameter in diff_iterate

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #6878 from manens/pybind_rados_ioctx_snap_rollback

pybind: Implementation of rados_ioctx_snapshot_rollback

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #6889 from kylinstorage/wip-fix-merge-diff

librbd: fix merge-diff for >2GB diff-files

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #6888 from prallabh/master

rgw: fix a glaring syntax error

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>

tests: fix failure for osd-scrub-snap.sh

Reason to get a false negative only one osd, when calling kill_daemons
$dir KILL osd, which means kill -9 shutdown() would not be called to
notify monitor so that it remain active+clean and osd up until monitor
got a mon_report timeout.  Thus wait_for_clean lose it function and when
we call ceph pg scrub, we got an EAGAIN error and monitor will ignore
the scrub message because of osd not up

Signed-off-by: Ning Yao <zay11022@gmail.com>

tests: kill_daemons uses TERM instead of KILL

When called to teardown a test, kill_daemon should use KILL to ensure
all leftovers are removed as quickly as possible to leave a clean state
for the next test. However, when kill_daemons is called to shutdown a
given daemon from within a test, it should use TERM by default so the
daemon has time to notify the MON that it goes down. For instance, if
KILLing an OSD, the mon will still report it as being up although the
calling function probably expects that it will be marked out.

Signed-off-by: Loic Dachary <loic@dachary.org>

librbd: fix merge-diff for >2GB diff-files

from http://tracker.ceph.com/issues/14030

Fixes: #14030
Signed-off-by: Yunchuan Wen <yunchuan.wen@kylin-cloud.com>

rgw: fix a glaring syntax error

syntactical error is causing rgw not to start

Signed-off-by: Pavan Rallabhandi <pavan.rallabhandi@sandisk.com>

Implementation of rados_ioctx_snapshot_rollback

Signed-off-by: Florent Manens <florent@beezim.fr>

Merge pull request #6858 from jcsp/wip-cephfs-auth-doc

doc: note that cephfs auth stuff is new in jewel

Merge pull request #6768 from marcan/cython

pybind: port the rbd bindings to Cython

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Nathan Cutler <ncutler@suse.com>
Reviewed-by: Ken Dreyer <kdreyer@redhat.com>
Reviewed-by: Alfredo Deza <adeza@redhat.com>

Merge pull request #6877 from dillaman/wip-14001

librbd: fix test case race condition for journaling ops

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

librbd: fix test case race condition for journaling ops

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

Revert "LifeCycle feature"

It's not there yet. Created a branch with a few fixes, will need to go
through some more revisions.

This reverts commit 7d48f62f5c86913d8f00b44d46a04a52d338907c.

Merge pull request #6331 from chenji-kael/wip-lifecycle

LifeCycle feature

Merge pull request #6871 from dachary/wip-14027-scrub-load

tests: --osd-scrub-load-threshold=2000 for more consistency

Reviewed-by: Xinze Chi <xinze@xsky.com>

tests: --osd-scrub-load-threshold=2000 for more consistency

In a test environment, consistency is more important than
performances. Effectively disable the test that would postpone a scrub
depending on the load average. It is assumed that a machine with a load
average higher than 2000 won't be useable anyway.

http://tracker.ceph.com/issues/14027 Refs: #14027

Signed-off-by: Loic Dachary <loic@dachary.org>

ceph-detect-init: Ubuntu >= 15.04 uses systemd

Ubuntu releases since Ubuntu 15.04 use systemd instead of upstart;
teach ceph-detect-init that this is the case, thus avoiding OSD
devices on systemd based installs being marked as managed by
upstart by ceph-disk.

Signed-off-by: James Page <james.page@ubuntu.com>

pybind/rbd.pyx: misc typo bugfixes

pybind/rbd.pyx: PyString -> PyBytes

doc: osd: s/schedued/scheduled/

Signed-off-by: Loic Dachary <loic@dachary.org>

Merge pull request #6830 from vumrao/wip-12112-vumrao

rbd: add --object-size option, deprecate --order

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

buffer: make usable outside of ceph source again

Add a rados/buffer_fwd.h symlink and remove the "include/" prefix from
buffer.h.

Signed-off-by: Josh Durgin <jdurgin@redhat.com>

rgw: enforce SLO part's ETag match during GET on SLO of Swift.

Signed-off-by: Radoslaw Zarzynski <rzarzynski@mirantis.com>

rgw: append X-Static-Large-Object attribute during PUT on SLO.

Signed-off-by: Radoslaw Zarzynski <rzarzynski@mirantis.com>

rgw: add support for normalisation of ranged SLO requests.

Signed-off-by: Radoslaw Zarzynski <rzarzynski@mirantis.com>

rgw: fix wrong ofs handling during GET on Swift's SLO.

Signed-off-by: Radoslaw Zarzynski <rzarzynski@mirantis.com>

rgw: handle errors during GET on Swift's SLO appropriately.

Signed-off-by: Radoslaw Zarzynski <rzarzynski@mirantis.com>

rgw: make response for multipart delete on a SLO compliant with Swift.

Signed-off-by: Radoslaw Zarzynski <rzarzynski@mirantis.com>

rgw: add support for multipart delete on a SLO of Swift.

Signed-off-by: Radoslaw Zarzynski <rzarzynski@mirantis.com>

rgw: fix iterate_slo_parts() and improve its declaration's formatting.

Signed-off-by: Radoslaw Zarzynski <rzarzynski@mirantis.com>

rgw: move RGWBulkDelete in rgw/rgw_op.h.

Signed-off-by: Radoslaw Zarzynski <rzarzynski@mirantis.com>

rgw: iterate over slo parts

Read slo parts when a read request was sent.

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
Signed-off-by: Radoslaw Zarzynski <rzarzynski@mirantis.com>
Conflicts:
src/rgw/rgw_op.cc

rgw: store slo info when putting object

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
Signed-off-by: Radoslaw Zarzynski <rzarzynski@mirantis.com>
Conflicts:
src/rgw/rgw_op.cc
src/rgw/rgw_op.h

rgw: read and parse put slo obj request

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
Signed-off-by: Radoslaw Zarzynski <rzarzynski@mirantis.com>
Conflicts:
src/rgw/rgw_op.h
src/rgw/rgw_rest_swift.cc

rgw: s/obj_manifest/dlo_manifest

Be more clear, as slo_manifest will come later.

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
Signed-off-by: Radoslaw Zarzynski <rzarzynski@mirantis.com>
Conflicts:
src/rgw/rgw_op.cc

rgw: support bucket removal in Bulk Delete API implementation.

Signed-off-by: Radoslaw Zarzynski <rzarzynski@mirantis.com>

rgw: improve response code handling for Bulk Delete API.

Signed-off-by: Radoslaw Zarzynski <rzarzynski@mirantis.com>

rgw: enable chunking in Bulk Delete API implementation.

Signed-off-by: Radoslaw Zarzynski <rzarzynski@mirantis.com>

rgw: add support for Bulk Delete API of Swift.

Signed-off-by: Radoslaw Zarzynski <rzarzynski@mirantis.com>