Revert "makefiles: remove bz2-dev from dependencies"

Merge pull request #6939 from branch-predictor/bp-remove-bz2dev

makefiles: remove bz2-dev from dependencies

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #6943 from smithfarm/wip-doc-dev-typo

doc: fix typo in developer guide

Reviewed-by: Loic Dachary <ldachary@redhat.com>

doc: fix typo in developer guide

Signed-off-by: Nathan Cutler <ncutler@suse.com>

Merge pull request #6821 from H3C/wip-cephfs-f11322

ceph-fuse: add process to ceph-fuse --help

Reviewed-by: John Spray <john.spray@redhat.com>

makefiles: remove bz2-dev from dependencies

The only thing that uses bzip2-devel is RocksDB, and it's optional, not
requirement. Drop the bzip2-devel/libbz2-dev dependency entirely, and
let RocksDB use it only if it is already present.

Fixes: #13981
Signed-off-by: Piotr Dałek <piotr.dalek@ts.fujitsu.com>

ceph-fuse: add process to ceph-fuse --help

Signed-off-by: Wei Feng <feng.wei@h3c.com>

Merge pull request #6934 from smithfarm/wip-14070

doc: Update list of admin/build-doc dependencies

Reviewed-by: Loic Dachary <ldachary@redhat.com>

doc: Update list of admin/build-doc dependencies

http://tracker.ceph.com/issues/14070 Fixes: #14070

Signed-off-by: Nathan Cutler <ncutler@suse.com>

Merge pull request #6910 from dachary/wip-dev-guide-ceph-qa-suite

doc: dev: introduction to tests

Reviewed-by: Nathan Cutler <ncutler@suse.com>

Merge pull request #6927 from tchaikov/wip-fix-ftbfs

rgw: fix the build failure

Reviewed-by: Loic Dachary <ldachary@redhat.com>

rgw: remove unused variable

to fix the compiler warning of

 rgw/rgw_formats.cc: In member function 'virtual void
 RGWFormatter_Plain::dump_format_va(const char*, const char*, bool, const
 char*, __va_list_tag*)':
 rgw/rgw_formats.cc:143:15: warning: unused variable 'format'
 [-Wunused-variable]
    const char *format;

Signed-off-by: Kefu Chai <kchai@redhat.com>

rgw: fix the build failure

* s/bucket_name_str/bucket_name/
* the member variable name of `req_state` was changed to `bucket_name`
  in f7ca00a. but some commits was still using the old name before the
  commit got merged.
* fixes for the tenant related change
* also fixes a typo

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #6859 from dillaman/wip-librbd-async-refresh

librbd: image refresh code paths converted to async state machines

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

librbd: refresh object map after rollback

An image refresh will no longer trigger an automatic object map
reload, therefore explicitly reload the object map after rollback.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

librbd: refresh image asynchronously from watch/notify path

The AIO path already ensures asynchronous image refreshes, but the
watch/notify path for maintenance ops was still using the sync refresh
path.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

librbd: the journal state machine is now asynchronous

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

tests: adjust test cases to refactored librbd

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

librbd: all object map methods are now async

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

librbd: IO ops will now asynchronously refresh the image if needed

This avoids blocking the AIO path with the previous synchronous
refresh path.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

WorkQueue: PointerWQ now supports out-of-band processing

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

librbd: flush all in-flight IO on block write and shut down

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

librbd: add shut down support to the AIO work queue

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

librbd: integrate new async image state machines

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

librbd: ImageWatcher should release lock upon request

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

librbd: implement image state tracker for open/close/refresh/etc

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

librbd: rename ImageRefresh to ImageState

This class handles more than just refreshing the image from
disk.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

librbd: object map lifespan now controlled by state machines

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

librbd: decouple ImageWatcher from exclusive lock management

Use new ExclusiveLock state machine to handle all the proper
transitions between lock states.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

librbd: separate image refresh into async state machines

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

tests: exclusive lock state machines test cases

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

librbd: separate exclusive lock handling into async state machines

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

librados_test_stub: additional definitions and mock functions

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

tests: mock test cases for new async object map lock/unlock/refresh ops

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

librbd: async object map lock/unlock/refresh state machines

Creating async versions to support an async image refresh

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

librados_test_stub: additional mocked IoCtx methods

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

librbd: convert object_map::InvalidateRequest to template

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

librbd: support updated async cls_rbd API

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

cls_rbd: async version of class helpers added

New async versions of get_id, get_immutable_metadata,
get_mutable_metadata, get_flags, get_stripe_unit_count,
snapshot_list, and old_snapshot_list.  These are needed
by the new librbd async ImageCtx op state machines.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

librbd: fix missing header dependencies for parent_types.h

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

librados_test_stub: implemented ObjectReadOperation::stat

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

tests: restrict rebuild object map test case

It should only run when the object map is enabled for the image.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

librbd: moved object name helpers to librbd::util namespace

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

tests: integrate cleaned up librbd AIO callback adapter

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

librbd: cleanup librbd AIO completion callbacks

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

librbd: cleanup Context and librados AIO completion callbacks

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #6643 from rzarzynski/wip-rgw-slo-rzarzynski

- rgw: swift bulk delete (Radoslaw Zarzynski)
- rgw: static large objects (Radoslaw Zarzynski, Yehuda Sadeh)

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>

Merge pull request #6925 from dillaman/wip-13633

librbd: optionally validate new RBD pools for snapshot support

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

tests: new integration test for validating new RBD pools

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

librbd: optionally validate RBD pool configuration (snapshot support)

Fixes: #13633
Signed-off-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #6917 from majianpeng/journal-iohint

Journal iohint for librbd

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #6923 from dillaman/wip-14060

librbd: correct race conditions discovered during unit testing

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

journal: avoid race between flush request and in-flight flushes

Previously, if the journal object was empty, a flush request could
complete instantly even if there is an in-flight 'record safe'
callback.

Fixes: #14060
Signed-off-by: Jason Dillaman <dillaman@redhat.com>

librbd: fix test case race condition for maintenance requests

Fixes: #14060
Signed-off-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #6922 from liewegas/wip-cephfs-posix

doc/cephfs/posix: update

Merge pull request #6899 from branch-predictor/bp-fix-no-rbd-build

rbd: fix build with "--without-rbd"

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

rbd: fix build with "--without-rbd"

It's not possible to build Ceph configured with "--without-rbd", as
tests and FileStore depends on components provided by RBD. This changeset
fixes this.

Fixes: #14058
Signed-off-by: Piotr Dałek <piotr.dalek@ts.fujitsu.com>

doc/cephfs/posix: update

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #6873 from javacruft/wip/debian-infernalis-patches

ceph-detect-init: Ubuntu >= 15.04 uses systemd

Reviewed-by: Loic Dachary <ldachary@redhat.com>

Merge pull request #6787 from oritwas/wip-run-cmake-check

Wip run cmake check

Reviewed-by: Loic Dachary <ldachary@redhat.com>

journal: add iohint flags for journal replayer.

Signed-off-by: Jianpeng Ma <jianpeng.ma@intel.com>

journal: Add iohint flags for journal write ops.

Signed-off-by: Jianpeng Ma <jianpeng.ma@intel.com>

Merge pull request #6790 from kylinstorage/wip-subprocess

SubProcess: fix multiple definition bug

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #5921 from caibo2014/add-tool-feature

tools:support printing part cluster map in readable fashion

Reviewed-by: Kefu Chai <kchai@redhat.com>

doc: dev: introduction to tests

Signed-off-by: Loic Dachary <loic@dachary.org>

Merge pull request #6906 from HeyoQiang/heyongqiang1212

client: modify a word in log

Reviewed-by: Loic Dachary <ldachary@redhat.com>

Merge pull request #6866 from xiaoxichen/umount_before_mount

init-ceph: do umount when the path exists.

Reviewed-by: Loic Dachary <ldachary@redhat.com>

init-ceph: do umount when the path exists.

If the specified mount point is in use, umount it instead
of skipping mounting the fs.

In previous code , if we forgot that we already mount something unrelated to
*this ceph osd * to the particular mount point, we will skip the mount and
finally get an error complaining superblock not matching OSD ID.

umount & remount is better because
1. If the wrong FS not in use, we can get the right FS we want and make ceph boot smoothly.
2. If the wrong FS is in use, we will get EBUSY on umount, which seems explain the situation
more clearly than superblock mismatch.

Signed-off-by: Xiaoxi Chen <xiaoxi.chen@intel.com>

Merge pull request #6905 from H3C/wip-bugfix-mount1

mount.ceph: memory leaks

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #6908 from nilamdyuti/wip-doc-rbdsnapshot

doc: Modified a note section in rbd-snapshot doc.

Reviewed-by: Loic Dachary <ldachary@redhat.com>

mount/mtab.cc: memory leaks
the free() should be called to free resources, in order to avoid memory leaks

Fixes:#14066
Signed-off-by: Qiankun Zheng <zheng.qiankun@h3c.com>

client: modify a word in log

This modification changed the word "comit" in line 7704
/* ldout(cct, 10) << "sync_write_comit -- no more unsafe writes, unmount can proceed" << dendl; */
by the word "commit";

Signed-off-by: YongQiang He<he.yongqiang@h3c.com>

doc: Modified a note section in rbd-snapshot doc.

Elaborates the note section a bit more as to why stopping I/O before
taking a snapshot is recommended and how to do it.

Signed-off-by: Nilamdyuti Goswami <ngoswami@redhat.com>

Merge pull request #6901 from yehudasa/wip-mstart

vstart.sh: add mstart, mstop, mrun wrappers for running multiple vstart-style test clusters out of src tree

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #6698 from mathslinux/auth-crash-fix

auth: fix crash when bad keyring is passed

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #6890 from dachary/wip-13986-test-pg-scrub

tests: fix failure for osd-scrub-snap.sh

Reviewed-by: Xinze Chi <xinze@xsky.com>
Reviewed-by: Ning Yao <zay11022@gmail.com>

vstart.sh: mon osd reporter subtree level = osd

Since everything is on one host!

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #6709 from xiaoxichen/failure_by_host

mon: support min_down_reporter by subtree level (default by host)

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Sage Weil <sage@redhat.com>

mstop.sh, mrgw.sh: more scripts to control daemons in dev env

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

mrun: a tool to run commands against specific mstart cluster

E.g.,

$ ./mstart.sh foo -n
$ ./mrun foo ceph -s

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

mstart.sh: run multiple clusters in a single source tree

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

Merge pull request #6823 from oritwas/wip-cmake-fix-authtool

cmake: add TracepointProvider.cc to libcommon

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #6827 from trociny/wip-scrub-debug

Little improvements for troubleshooting scrub issues

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #6732 from aalba6675/patch-custom-cluster-name

sysvinit: allow custom cluster names

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #6730 from kylinstorage/fix-refcounter-in-replica_scrub

osd: fix MOSDRepScrub reference counter in replica_scrub

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #6713 from zhanghuan/fix-wbthrottle

osd: fix incorrect throttle in WBThrottle

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #6731 from kylinstorage/clean_up_objecter

librados: clean up Objecter.h

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #6863 from jdurgin/wip-buffer-header

buffer: make usable outside of ceph source again

Reviewed-by: Sage Weil <sage@redhat.com>

cmake: add run_cmake-check.sh

currently only check to cmake build process

Signed-off-by: Orit Wasserman <owasserm@redhat.com>

Merge pull request #6750 from theanalyst/doc/mailmap/jewel

mailmap: Jewel updates

Reviewed-by: Loic Dachary <ldachary@redhat.com>

Merge pull request #6872 from dachary/wip-osd-typo

doc: osd: s/schedued/scheduled/

Reviewed-by: Abhishek Lekshmanan <abhishek.lekshmanan@gmail.com>

mailmap: YankunLi affiliation

Signed-off-by: Abhishek Lekshmanan <abhishek.lekshmanan@gmail.com>

mailmap: Nishtha Rai affiliation

Signed-off-by: Abhishek Lekshmanan <abhishek.lekshmanan@gmail.com>

mailmap: John Coyle affiliation

Signed-off-by: Abhishek Lekshmanan <abhishek.lekshmanan@gmail.com>
Conflicts:
.organizationmap
Trivial merge conflict as upstream had an additional entry

mailmap: Jashan Kamboj affiliation

Signed-off-by: Abhishek Lekshmanan <abhishek.lekshmanan@gmail.com>

mailmap: Adam Kupczyk affiliation

Signed-off-by: Abhishek Lekshmanan <abhishek.lekshmanan@gmail.com>

mailmap: Jacek J. Lakis affiliation

Signed-off-by: Abhishek Lekshmanan <abhishek.lekshmanan@gmail.com>

mailmap: runsisi affiliation

Signed-off-by: Abhishek Lekshmanan <abhishek.lekshmanan@gmail.com>

mailmap: Zeqiang Zhuang affiliation

Signed-off-by: Abhishek Lekshmanan <abhishek.lekshmanan@gmail.com>

mailmap: You Ji affiliation

Signed-off-by: Abhishek Lekshmanan <abhishek.lekshmanan@gmail.com>