test/librados/test.cc: clean up EC pools' crush rules too

SetUp was adding an erasure-coded pool, which automatically adds
a new crush rule named after the pool, but only removing the
pool.  Remove the crush rule as well.

http://tracker.ceph.com/issues/13878 Fixes: #13878

Signed-off-by: Dan Mick <dan.mick@redhat.com>
Signed-off-by: Loic Dachary <loic@dachary.org>

Merge pull request #6780 from smithfarm/wip-index-20151201

doc/dev/index.rst: fix headings

Reviewed-by: Loic Dachary <ldachary@redhat.com>

Merge pull request #6786 from oritwas/wip-cmake-fix-radosgw

cmake: add rgw_basic_types.cc to librgw.a

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>

doc/dev/index.rst: fix links

Signed-off-by: Nathan Cutler <ncutler@suse.com>

doc/dev/index.rst: Flesh out the IRC section

Signed-off-by: Nathan Cutler <ncutler@suse.com>

doc/dev/index.rst: Make Issue tracker a separate chapter

Signed-off-by: Nathan Cutler <ncutler@suse.com>

Merge pull request #6789 from dillaman/wip-13969

librbd: partial revert of commit 9b0e359

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #6748 from liewegas/wip-fix-cephtool

qa/workunits/cephtool/test.sh: no ./

Reviewed-by: Loic Dachary <ldachary@redhat.com>

Merge pull request #5941 from jcsp/wip-12137

#12137: cephfs-data-scan scan_frags

Merge pull request #6777 from dillaman/wip-librbd-lockdep

librbd: journaling-related lock dependency cleanup

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #6762 from dillaman/wip-13924

journal: fire replay complete event after reading last object

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #6703 from dillaman/wip-tests-librbd-gmock-state-machines

librbd: initial collection of state machine unit tests

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

librbd: partial revert of commit 9b0e359

Fixes: #13969
Signed-off-by: Jason Dillaman <dillaman@redhat.com>

doc/dev/index.rst: start writing Bugfixing chapter

Signed-off-by: Nathan Cutler <ncutler@suse.com>

Merge pull request #6690 from branch-predictor/bp-smaller-pglog-2

osd: slightly reduce actual size of pg_log_entry_t

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #6689 from 0003ydlom/doubled_cond

osd: avoid duplicate op->mark_started in ReplicatedBackend

Reviewed-by: xinxin shu <xinxin.shu@intel.com>

Merge pull request #6681 from ifed01/wip-objstore-bench-stabilization

ceph_objectstore_bench: fix race condition, bugs

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #6677 from jack-changtao/test-bench

rados: bench: fix off-by-one to avoid writing past object_size

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #6555 from mslovy/wip-reduce-omap-transaction

osd: don't update epoch and rollback_info objects attrs if there is no need

Reviewed-by: Sage Weil <sage@redhat.com>

cmake: add rgw_basic_types.cc to librgw.a

Signed-off-by: Orit Wasserman <owasserm@redhat.com>

doc/dev/index.rst: rewrite introductory material

Signed-off-by: Nathan Cutler <ncutler@suse.com>

Merge pull request #6784 from ceph/wip-5073

rgw multitenancy

Buckets and users can now have a tenant associated with them. Buckets can be specified as [<tenant>:]<bucket> and users can be specified as [<tenant>$]<uid>. The existing users and buckets are part of the default global tenant.
This is first phase. Future work will add tighter integration (e.g., swift tenant mapping, s3 subdomain tenant mapping, etc.)

Reviewed-by: Pete Zaitcev <zaitcev@redhat.com>
Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>

doc/dev/index.rst: fix headings

This commit fixes the headings so they render properly on http://ceph.com

Signed-off-by: Nathan Cutler <ncutler@suse.com>

Merge pull request #6727 from smithfarm/wip-index

doc/dev/index.rst: begin writing Contributing to Ceph

Reviewed-by: Loic Dachary <ldachary@redhat.com>

doc/dev/index.rst: begin writing Contributing to Ceph

Replace existing index.rst with the beginnings of a comprehensive
developer guide.

Move the old index.rst to index-old.rst.

Signed-off-by: Nathan Cutler <ncutler@suse.com>

common: lockdep now tracks lock ordering when backtraces disabled

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

librbd: correct lock ordering issues discovered by lockdep

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #6775 from mkoderer/doc/fixowner

doc: remove unnecessary period in headline

Reviewed-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

doc: remove unnecessary period in headline

Signed-off-by: Marc Koderer <marc@koderer.com>

journal: correct lock ordering issues discovered by lockdep

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

rgw: remove comments

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

rgw: a minor cleanup

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

rgw: don't re-set bucket tenant and name when selecting location

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

rgw: fix a typo

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

rgw: don't allow cross-tenant bucket creation

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

rgw: avoid calling rgw_make_bucket_entry_name() when not needed

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

rgw: objexp hint name backward compatibility

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

rgw: inherit bucket tenant from user if not specified

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

Merge pull request #6739 from dreamhost/wip-13920

Gentoo: _FORTIFY_SOURCE fix.

Reviewed-by: Samuel Just <sjust@redhat.com>

Merge pull request #6771 from dillaman/wip-backtrace-fixes

common: re-enable backtrace support

Reviewed-by: Loic Dachary <ldachary@redhat.com>

tests: initial set of test cases for op state machines

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

librbd: snap create doesn't properly handle race conditions

The ESTALE result code was being filtered out, preventing the retry
logic.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

librbd: use md_ctx for self-managed snapshot management

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

librbd: convert op state machines to templates

Using templates allows the creation of mocked test cases to
verify non-librados expectations.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

tests: add gmock unit tests for librbd object map state machines

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

librbd: ensure object map is invalidated on disk if forced

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #6657 from kylinstorage/wip-rbd-nbd

rbd-nbd: network block device (NBD) support for RBD

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #6769 from aiicore/configure_fix

configure.ac: macro fix

Reviewed-by: Boris Ranto <branto@redhat.com>

configure.ac: macro fix

Configure fails with autoconf 2.63 on Centos 6.6 with:

./configure: line 34026: syntax error near unexpected token `newline'
./configure: line 34026: `  yes:no:'

Signed-off-by: Igor Podoski <igor.podoski@ts.fujitsu.com>

Merge pull request #5685 from ceph/wip-12133

cephfs-data-scan: scrub tag filtering (#12133 and #12145)

tools/cephfs: enable tag filter in DataScan

Fixes: #12133
Signed-off-by: John Spray <john.spray@redhat.com>

tools/cephfs: fix datascan kwarg parsing

Signed-off-by: John Spray <john.spray@redhat.com>

cls_cephfs: add PGLSCephFSFilter

Special purpose filter for use with CephFS disaster
recovery.

Fixes: #12145
Signed-off-by: John Spray <john.spray@redhat.com>

common: re-enable backtrace support

A recent porting effort disabled support for backtrace creation.  It
also didn't initialize all variables, which would lead to a crash when
deleting a backtrace instance.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #6719 from jcsp/wip-13898

tools: Fix layout handing in cephfs-data-scan (#13898)

Update man page for new rbd nbd command

Signed-off-by: Yunchuan Wen <yunchuan.wen@kylin-cloud.com>

update test_librbd_fsx to support rbd-nbd

Signed-off-by: Yunchuan Wen <yunchuan.wen@kylin-cloud.com>

add rbd-nbd man page reference

Signed-off-by: Yunchuan Wen <yunchuan.wen@kylin-cloud.com>

add rbd-nbd wrap to rbd CLI

Signed-off-by: Yunchuan Wen <yunchuan.wen@kylin-cloud.com>

Merge pull request #6661 from xiaoxichen/rbd_misc_fix1

cls/cls_rbd.cc: fix misused metadata_name_from_key

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

cls/cls_rbd.cc: no need to skip key == after.

We already skip the pair where key == after in both LevelDBWholeSpaceIteratorImpl::upper_bound
and RocksDBWholeSpaceIteratorImpl::upper_bound.

Fixes: #13922
Signed-off-by: Xiaoxi Chen <xiaoxi.chen@intel.com>

journal: fire replay complete event after reading last object

Fixes: #13924
Signed-off-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #6742 from dillaman/wip-13913

librbd: automatically flush IO after blocking write operations

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

init-ceph: fix systemd-run cant't start ceph daemon sometimes

Fixes: #13474
Signed-off-by: Chuanhong Wang <wang.chuanhong@zte.com.cn>

Merge pull request #6749 from dachary/wip-doc-install

doc: INSTALL redirect to online documentation

Reviewed-by: Loic Dachary <ldachary@redhat.com>

tools/cephfs: fix layout handling in injection

When the stashed-layout handling stuff was added,
the guessing part got lost from the path where
we only had a single object.  Broke test_data_scan.

Signed-off-by: John Spray <john.spray@redhat.com>

Merge pull request #6729 from majianpeng/librbd-journal

librbd: only send signal when AIO completions queue empty

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #6751 from ceph/wip-cmake-strerror_r

cmake: define STRERROR_R_CHAR_P for GNU-specific strerror_r

Reviewed-by: John Spray <john.spray@redhat.com>

Merge pull request #5465 from yuyuyu101/wip-librbd-ap

librbd: support eventfd for AIO completion notifications

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

cmake: define STRERROR_R_CHAR_P for GNU-specific strerror_r

This is what autotools does with AC_FUNC_STRERROR_R bit.  It takes us
from

    $ monmaptool --print nonexistant
    monmaptool: monmap file nonexistant
    monmaptool: couldn't open nonexistant: (2)

to

    $ monmaptool --print nonexistant
    monmaptool: monmap file nonexistant
    monmaptool: couldn't open nonexistant: (2) No such file or directory

on cmake - otherwise we end up using GNU-specific version as if it was
the POSIX version.

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>

Merge pull request #6745 from drunkard/master

doc: fix typo in cephfs/quota

Merge remote-tracking branch 'gh/jewel'

doc: INSTALL redirect to online documentation

The INSTALL file has obsolete and misleading instructions to compile
from sources (-pg actually fails at link time). Replace with links to
the online installation and build from source documentations.

Signed-off-by: Loic Dachary <loic@dachary.org>

qa/workunits/cephtool/test.sh: no ./

Introduced by 1ace4d03b23b80c699f4861d004f3212ca56a323

Signed-off-by: Sage Weil <sage@redhat.com>

doc: typo fix in cephfs/quota

librbd/JournalReplay: Only Send signal when aio completions queue empty.

Signed-off-by: Jianpeng Ma <jianpeng.ma@intel.com>

Merge pull request #6697 from XinzeChi/wip-scrub-test

test: fix osd-scrub-snaps.sh

Reviewed-by: Loic Dachary <ldachary@redhat.com>

test: fix osd-scrub-snaps.sh

when we run run-make-check.sh in hdd disk, it would spend much more
time to finish scrub.

Signed-off-by: Xinze Chi <xinze@xsky.com>

test: add pg_scrub for ceph-helpers.sh

Signed-off-by: Xinze Chi <xinze@xsky.com>

librbd: Fix rebase with new io flow

Signed-off-by: Haomai Wang <haomai@xsky.com>

librbd: fix lttng tracing argument mismatch

Signed-off-by: Haomai Wang <haomai@xsky.com>

librbd: Add set_event_notify to AioImageRequestWQ

Signed-off-by: Haomai Wang <haomai@xsky.com>

librbd: Remove unneeded set_event_notify

Since xlist clear method invoked below is enough to disassociate the xlist
item from the xlist, so the remove_myself() call in the destructor is safe
to invoke since it's already been removed from the list.

Signed-off-by: Haomai Wang <haomai@xsky.com>

Makefile: Add noinst headr files

Signed-off-by: Haomai Wang <haomai@xsky.com>

librbd: Make rbd header file uses independent enum definition

Signed-off-by: Haomai Wang <haomai@xsky.com>

Librbd: Make AioCompletion complete doesn't unlock if callback

Signed-off-by: Haomai Wang <haomai@xsky.com>

Librbd: fix return code of EventSocket init and notify

Signed-off-by: Haomai Wang <haomai@xsky.com>

librbd: Add ictx check to avoid AIO_TYPE_NONE completion

Signed-off-by: Haomai Wang <haomaiwang@gmail.com>

librbd: normalize notify return code

Signed-off-by: Haomai Wang <haomaiwang@gmail.com>

librbd: Fix incorrect api declaration

Signed-off-by: Haomai Wang <haomaiwang@gmail.com>

librbd: check event_notify to avoid extra logic

Signed-off-by: Haomai Wang <haomaiwang@gmail.com>

EventSocket: Add new event type pipe support

Signed-off-by: Haomai Wang <haomaiwang@gmail.com>

test: Add tests for getting arg of completion

Signed-off-by: Haomai Wang <haomaiwang@gmail.com>

librbd: Add interface to let user can get private data from comp

Signed-off-by: Haomai Wang <haomaiwang@gmail.com>

tests: Add tests for user io event notify

Signed-off-by: Haomai Wang <haomaiwang@gmail.com>

librbd: Add event notify interfaces

Signed-off-by: Haomai Wang <haomaiwang@gmail.com>

librbd: automatically flush IO after blocking write operations

This simplifies other state machines that previously had to flush IO
after blocking write operations.

Fixes: #13913
Signed-off-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #6741 from SUSE/wip-tyepdef-typo-fix

include/rados/librados.h: fix typo

Reviewed-by: Loic Dachary <ldachary@redhat.com>

include/rados/librados.h: fix typo

This typo causes a warning when building the docs.

Signed-off-by: Nathan Cutler <ncutler@suse.com>

Merge pull request #6740 from dillaman/wip-13911

librbd: ensure copy-on-read requests are complete prior to closing parent image

Reviewed-by: Josh Durgin <jdurgin@redhat.com>