tests: --osd-scrub-load-threshold=2000 for more consistency

In a test environment, consistency is more important than
performances. Effectively disable the test that would postpone a scrub
depending on the load average. It is assumed that a machine with a load
average higher than 2000 won't be useable anyway.

http://tracker.ceph.com/issues/14027 Refs: #14027

Signed-off-by: Loic Dachary <loic@dachary.org>

Merge pull request #6857 from dachary/wip-13986-test-pg-scrub

tests: osd-scrub-snaps.sh to display full osd logs on error

Reviewed-by: Xinze Chi <xinze@xsky.com>

tests: osd-scrub-snaps.sh to display full osd logs on error

This generates a large output but is the only way to get more
information about race conditions that turn out to show up frequently.

http://tracker.ceph.com/issues/13986 Refs: #13986

Signed-off-by: Loic Dachary <loic@dachary.org>

Merge pull request #6850 from H3C/wip-zcl-ceph-mds

ceph-mds:add --help/-h

Reviewed-by: John Spray <john.spray@redhat.com>

Merge pull request #6848 from H3C/wip-zcl-layout-rst

doc:file must be empty when writing layout fields of file use "setfattr"

Reviewed-by: John Spray <john.spray@redhat.com>

ceph-mds:add --help/-h

When use ceph-mds --hlep/-h, the usage will be showed

Signed-off-by: Cilang Zhao <zhao.cilang@h3c.com>

doc:file must be empty when writing layout fields of file use "setfattr"

When the layout fields of a file are modified, this file must be empty, otherwise an error would occur.

Signed-off-by:Cilang Zhao zhao.cilang@h3c.com

Merge remote-tracking branch 'ceph/jewel'

Merge pull request #6844 from dachary/wip-13997-centos-libunwind-jewel

build/ops: enable CR in CentOS 7

Reviewed-by: Brad Hubbard <bhubbard@redhat.com>

build/ops: enable CR in CentOS 7

To get libunwind from the CR repositories until CentOS 7.2.1511 is released.

http://tracker.ceph.com/issues/13997 Fixes: #13997

Signed-off-by: Loic Dachary <loic@dachary.org>

Merge pull request #6842 from badone/wip-13997

tests: centos7 needs the Continuous Release (CR) Repository enabled for

Reviewed-by: Loic Dachary <ldachary@redhat.com>

tests: centos7 needs the Continuous Release (CR) Repository enabled for
libunwind

Fixes: #13997
Signed-off-by: Brad Hubbard <bhubbard@redhat.com>

Merge pull request #6818 from trociny/fix-cmake

cmake: update for recent rbd changes

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #6837 from dreamhost/wip-test-cephtool-filenames

qa/workunits/cephtool/test.sh: false positive fail on /tmp/obj1.

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

qa/workunits/cephtool/test.sh: false positive fail on /tmp/obj1.

If /tmp/obj1 happened to exist already, and was not writable by the
testing user, then this test failed!

Signed-off-by: Robin H. Johnson <robin.johnson@dreamhost.com>

Merge pull request #6838 from stiopaa1/Wreorder_ceph_context

common/ceph_context.cc:fix order of initialisers

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #6831 from fullerdj/wip-14010

qa/krbd: Expunge generic/247

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #6793 from dzafman/wip-hist-les-fix

osd: Allow repair of history.last_epoch_started using config

Reviewed-by: Samuel Just <sjust@redhat.com>

common/ceph_context.cc:fix order of initialisers

Fix order of member initialisers causing [-Wreorder] warning.

Signed-off-by: Michal Jarzabek <stiopa@gmail.com>

Merge branch 'unlinkcallbyrmbucket'

Conflicts:
src/rgw/rgw_bucket.cc

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>

rgw:dont update entrypoint when removing bucket

when removing bucket,we unlink the bucket bucket but we don't want
to update the entrypoint here.Because the update_entrypoint's default
value is true,so we need to indicate it to false.

Signed-off-by: Zengran Zhang <zhangzengran@h3c.com>

Merge pull request #6735 from rzarzynski/wip-rgw-clean-putmetabucket

rgw: remove unused variable in RGWPutMetadataBucket::execute

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>

Merge pull request #6829 from LenzGr/master

doc: Fixed incorrect name of a "List Multipart Upload Parts" Response Entity

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>

Merge pull request #6774 from ceph/wip-rgw-cap-fix

rgw: add a missing cap type

qa/krbd: Expunge generic/247

xfstests generic/247 exercises XFS DIO and AIO to detect races. Some
races apparently exist in the XFS code as this is a known issue within
XFS. Expunge this test because it is not specifically relevant to krbd
and not a specific krbd issue.

Signed-off-by: Douglas Fuller <dfuller@redhat.com>

Merge pull request #6805 from zaitcev/cmake-pluginregistry

cmake: Add common/PluginRegistry.cc to CMakeLists.txt

doc: Fixed incorrect name of a "List Multipart Upload Parts" Response Entity

Fixed incorrect name of a "List Multipart Upload Parts" Response Entity:
"InitiatedMultipartUploadsResult" -> "ListPartsResult"

Fixes: #14003
Signed-off-by: Lenz Grimmer <lenz@grimmer.com>

cmake: update for recent rbd changes

Signed-off-by: Mykola Golub <mgolub@mirantis.com>

Merge pull request #6816 from drunkard/doc

doc: rst style fix for pools document

Reviewed-by: Loic Dachary <ldachary@redhat.com>

Merge pull request #6820 from mikulely/add-umcloud-affiliation

Add UMCloud affiliation

Reviewed-by: Loic Dachary <ldachary@redhat.com>

Update Jiaying Ren affinity

Signed-off-by: Jiaying Ren <jiaying.ren@umcloud.com>

Update Rongze Zhu affinity

Signed-off-by: Jiaying Ren <jiaying.ren@umcloud.com>

Merge pull request #6655 from jcsp/wip-inode-claim

tools: add cephfs-table-tool 'take_inos'

Reviewed-by: Yan, Zheng <zyan@redhat.com>

Merge pull request #6567 from jcsp/wip-layout-restrict

mds: add 'p' flag in auth caps to control setting pool in layout

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #6601 from jcsp/wip-13777-caps

mon: don't require OSD W for MRemoveSnaps

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Yan, Zheng <zyan@redhat.com>
Reviewed-by: Greg Farnum <gfarnum@redhat.com>
Reviewed-by: Sage Weil <sage@redhat.com>

doc: rst style fix for pools document

Signed-off-by: Drunkard Zhang <gongfan193@gmail.com>

deb,rpm: package buffer_fwd.h

Introduced by 206b8e2cc6157451d5c7548e294461408e0b0c8f.

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #6803 from dachary/wip-13975-systemd-flock

build/ops: systemd ceph-disk unit must not assume /bin/flock

Reviewed-by: Nathan Cutler <ncutler@suse.cz>

Merge pull request #6811 from smithfarm/wip-submitting-jewel

SubmittingPatches: there is no next; only jewel

Reviewed-by: Loic Dachary <ldachary@redhat.com>

SubmittingPatches: there is no next; only jewel

Signed-off-by: Nathan Cutler <ncutler@suse.com>

Merge pull request #6792 from smithfarm/wip-index-20151202

Wip index 20151202

Reviewed-by: Loic Dachary <ldachary@redhat.com>

Merge pull request #6795 from ceph/pybind-rados-exec

pybind: support ioctx:exec

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Add common/PluginRegistry.cc to CMakeLists.txt

Else this happens:

  [ 15%] Linking CXX executable ceph-authtool
  libcommon.a(ceph_context.cc.o): In function `CephContext::CephContext(unsigned int, int)':
  ceph_context.cc:(.text+0x2620): undefined reference to `ceph::PluginRegistry::PluginRegistry(CephContext*)'
  libcommon.a(ceph_context.cc.o): In function `CephContext::~CephContext()':
  ceph_context.cc:(.text+0x3db0): undefined reference to `ceph::PluginRegistry::~PluginRegistry()'
  collect2: error: ld returned 1 exit status
  src/CMakeFiles/ceph-authtool.dir/build.make:120: recipe for target 'src/ceph-authtool' failed
  make[2]: *** [src/ceph-authtool] Error 1
  CMakeFiles/Makefile2:331: recipe for target 'src/CMakeFiles/ceph-authtool.dir/all' failed

The ceph_context.cc is the only user of PluginRegistry at
present, so adding it without conditionals should be correct.

Signed-Off-By: Pete Zaitcev <zaitcev@redhat.com>

doc/dev/index.rst: wholesale refactor

Signed-off-by: Nathan Cutler <ncutler@suse.com>

pybind: support ioctx:exec

Signed-off-by: Noah Watkins <noahwatkins@gmail.com>

common/Makefile: ship common/event_socket.h

Broken in commit 5eb64dd324a0e0ec609acf2b178c02dee901aa5d.

Signed-off-by: Sage Weil <sage@redhat.com>

build/ops: systemd ceph-disk unit must not assume /bin/flock

The flock command may be installed elsewhere, depending on the
system. Let the PATH search figure that out.

http://tracker.ceph.com/issues/13975 Fixes: #13975

Signed-off-by: Loic Dachary <loic@dachary.org>

test: use sequential journal_tid for object cacher test

This matches the real usage by librbd.

Fixes: #13877
Signed-off-by: Josh Durgin <jdurgin@redhat.com>
(cherry picked from commit 9331e031bd2a719463920581a47c13f0606e9971)

Merge pull request #6788 from dachary/wip-13878-ec-pool

test/librados/test.cc: clean up EC pools' crush rules too

Reviewed-by: Dan Mick <dmick@redhat.com>

Merge pull request #6665 from trociny/wip-librbd-journal

rbd: journal: configuration via conf, cli, api and some fixes

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #6800 from dillaman/wip-13969

librbd: fix known test case race condition failures

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Loic Dachary <ldachary@redhat.com>

tests: wait for mocked requests to complete

Fixes: #13969
Signed-off-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #6641 from aclamk/wip-log-alloc-predictor

common: log: predict log message buffer allocation size

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #6642 from 0003ydlom/decode_reorder

osd: move newest decode version of MOSDOp and MOSDOpReply to the front

Reviewed-by: Sage Weil <sage@redhat.com>

librbd: do not complete AIO callbacks within caller's thread context

Avoid rare, racy issues when individual requests associated with an AIO
completion can complete prior to marking the completion as ready-to-fire.
Pre-calculate the expected number of individual requests to avoid the
potential re-entrant callback.

Fixes: #13969
Signed-off-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #6696 from Ved-vampir/wip_common_plugin

common: add generic plugin infrastructure

Reviewed-by: Varada Kari <varada.kari@sandisk.com>

Merge pull request #6779 from ceph/wip-librados-xattr

librados: avoid malloc(0) (which can return NULL on some platforms)

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #6759 from liewegas/wip-bdev-probe

osd: make backend and block device code a bit more generic

Reviewed-by: Loic Dachary <ldachary@redhat.com>
Reviewed-by: Varada Kari <varada.kari@sandisk.com>

Merge pull request #6686 from stiopaa1/namespace_buffer

common: use namespace instead of subclasses for buffer

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #6535 from mslovy/wip-crc-cache-tracker

common: buffer: add cached_crc and cached_crc_adjust counts to perf dump

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #6058 from guangyy/wip-pg-query

osd: dump number of missing objects for each peer with pg query

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #5953 from guangyy/wip-13121

osd: support pool level recovery_priority and recovery_op_priority

Reviewed-by: David Zafman <dzafman@redhat.com>

Merge pull request #5670 from david-z/master

osd: FileJournal: reduce locking scope in write_aio_bl

Reviewed-by: Haomai Wang <haomai@xsky.com>

Merge pull request #6798 from trociny/fix-rbd_mirror-bool

librbd: include missing header for bool type

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

librbd: include missing header for bool type

Signed-off-by: Mykola Golub <mgolub@mirantis.com>

rbd: alow to specify options for created journal

Signed-off-by: Mykola Golub <mgolub@mirantis.com>

librbd: journal options on image create/clone/copy

Signed-off-by: Mykola Golub <mgolub@mirantis.com>

librbd: output operator for image options

It is going to be used for diagnostic.

Signed-off-by: Mykola Golub <mgolub@mirantis.com>

librbd: journal: mark entry committed after replay

After replying a journal entry we have to call committed(). Otherwise,
the entries remain with flag committed=false in JournalMetadata
m_pending_commit_tids forever, which prevents commit position update.

Signed-off-by: Mykola Golub <mgolub@mirantis.com>

rbd: add new journal rbd commands

Also, extend 'rbd info' to output journal spec if journaling is enabled.

Signed-off-by: Mykola Golub <mgolub@mirantis.com>

librbd: humanize ImageWatcher and Journal states in debug log

Signed-off-by: Mykola Golub <mgolub@mirantis.com>

librbd: add image pool ID to journal object names.

The data objects are named "journal_data.<pool id>.<image id>.<num>".
This is to avoid conflict when a journal objects pool is used for
several image pools.

Signed-off-by: Mykola Golub <mgolub@mirantis.com>

librbd: allow alternate pool for journal objects

Signed-off-by: Mykola Golub <mgolub@mirantis.com>

librbd: debug: log journal metadata

Signed-off-by: Mykola Golub <mgolub@mirantis.com>

librbd: new config options to tweak journal settings

Signed-off-by: Mykola Golub <mgolub@mirantis.com>

Merge pull request #6129 from dillaman/wip-13292

RBD pool mirroring configuration API + CLI

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

osd: Test osd_find_best_info_ignore_history_les config in another assert

Signed-off-by: David Zafman <dzafman@redhat.com>

Merge pull request #6778 from liewegas/wip-13962

osd: call on_new_interval on newly split child PG

Reviewed-by: Samuel Just <sjust@redhat.com>

test/librados/test.cc: clean up EC pools' crush rules too

SetUp was adding an erasure-coded pool, which automatically adds
a new crush rule named after the pool, but only removing the
pool.  Remove the crush rule as well.

http://tracker.ceph.com/issues/13878 Fixes: #13878

Signed-off-by: Dan Mick <dan.mick@redhat.com>
Signed-off-by: Loic Dachary <loic@dachary.org>

Merge pull request #6780 from smithfarm/wip-index-20151201

doc/dev/index.rst: fix headings

Reviewed-by: Loic Dachary <ldachary@redhat.com>

Merge pull request #6786 from oritwas/wip-cmake-fix-radosgw

cmake: add rgw_basic_types.cc to librgw.a

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>

Merge pull request #6767 from oritwas/wip-13529-jewel

rgw: use smart pointer for C_Reinitwatch

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>

doc/dev/index.rst: fix links

Signed-off-by: Nathan Cutler <ncutler@suse.com>

doc/dev/index.rst: Flesh out the IRC section

Signed-off-by: Nathan Cutler <ncutler@suse.com>

doc/dev/index.rst: Make Issue tracker a separate chapter

Signed-off-by: Nathan Cutler <ncutler@suse.com>

Merge pull request #6789 from dillaman/wip-13969

librbd: partial revert of commit 9b0e359

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #6748 from liewegas/wip-fix-cephtool

qa/workunits/cephtool/test.sh: no ./

Reviewed-by: Loic Dachary <ldachary@redhat.com>

Merge pull request #5941 from jcsp/wip-12137

#12137: cephfs-data-scan scan_frags

Merge pull request #6777 from dillaman/wip-librbd-lockdep

librbd: journaling-related lock dependency cleanup

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #6762 from dillaman/wip-13924

journal: fire replay complete event after reading last object

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #6703 from dillaman/wip-tests-librbd-gmock-state-machines

librbd: initial collection of state machine unit tests

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

librbd: partial revert of commit 9b0e359

Fixes: #13969
Signed-off-by: Jason Dillaman <dillaman@redhat.com>

doc/dev/index.rst: start writing Bugfixing chapter

Signed-off-by: Nathan Cutler <ncutler@suse.com>

Merge pull request #6690 from branch-predictor/bp-smaller-pglog-2

osd: slightly reduce actual size of pg_log_entry_t

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #6689 from 0003ydlom/doubled_cond

osd: avoid duplicate op->mark_started in ReplicatedBackend

Reviewed-by: xinxin shu <xinxin.shu@intel.com>

Merge pull request #6681 from ifed01/wip-objstore-bench-stabilization

ceph_objectstore_bench: fix race condition, bugs

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #6677 from jack-changtao/test-bench

rados: bench: fix off-by-one to avoid writing past object_size

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #6555 from mslovy/wip-reduce-omap-transaction

osd: don't update epoch and rollback_info objects attrs if there is no need

Reviewed-by: Sage Weil <sage@redhat.com>

cmake: add rgw_basic_types.cc to librgw.a

Signed-off-by: Orit Wasserman <owasserm@redhat.com>

doc/dev/index.rst: rewrite introductory material

Signed-off-by: Nathan Cutler <ncutler@suse.com>