init-ceph: do umount when the path exists.

If the specified mount point is in use, umount it instead
of skipping mounting the fs.

In previous code , if we forgot that we already mount something unrelated to
*this ceph osd * to the particular mount point, we will skip the mount and
finally get an error complaining superblock not matching OSD ID.

umount & remount is better because
1. If the wrong FS not in use, we can get the right FS we want and make ceph boot smoothly.
2. If the wrong FS is in use, we will get EBUSY on umount, which seems explain the situation
more clearly than superblock mismatch.

Signed-off-by: Xiaoxi Chen <xiaoxi.chen@intel.com>

Merge pull request #6830 from vumrao/wip-12112-vumrao

rbd: add --object-size option, deprecate --order

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #6857 from dachary/wip-13986-test-pg-scrub

tests: osd-scrub-snaps.sh to display full osd logs on error

Reviewed-by: Xinze Chi <xinze@xsky.com>

tests: osd-scrub-snaps.sh to display full osd logs on error

This generates a large output but is the only way to get more
information about race conditions that turn out to show up frequently.

http://tracker.ceph.com/issues/13986 Refs: #13986

Signed-off-by: Loic Dachary <loic@dachary.org>

Merge pull request #6850 from H3C/wip-zcl-ceph-mds

ceph-mds:add --help/-h

Reviewed-by: John Spray <john.spray@redhat.com>

Merge pull request #6848 from H3C/wip-zcl-layout-rst

doc:file must be empty when writing layout fields of file use "setfattr"

Reviewed-by: John Spray <john.spray@redhat.com>

rbd: add --object-size option

Object size can be specified when creating an image with the --order option,
as a number of bits in the size.

This patch is adding new option --object-size. This new option will specify
object size directly for example --object-size 2M.

It would be easier to use. --order is still present for backwards compatibility.
For simplicity, we are rounding up the object size to the nearest power of 2.

Fixes #12112

Signed-off-by: Vikhyat Umrao <vumrao@redhat.com>

ceph-mds:add --help/-h

When use ceph-mds --hlep/-h, the usage will be showed

Signed-off-by: Cilang Zhao <zhao.cilang@h3c.com>

doc:file must be empty when writing layout fields of file use "setfattr"

When the layout fields of a file are modified, this file must be empty, otherwise an error would occur.

Signed-off-by:Cilang Zhao zhao.cilang@h3c.com

Merge remote-tracking branch 'ceph/jewel'

Merge pull request #6844 from dachary/wip-13997-centos-libunwind-jewel

build/ops: enable CR in CentOS 7

Reviewed-by: Brad Hubbard <bhubbard@redhat.com>

build/ops: enable CR in CentOS 7

To get libunwind from the CR repositories until CentOS 7.2.1511 is released.

http://tracker.ceph.com/issues/13997 Fixes: #13997

Signed-off-by: Loic Dachary <loic@dachary.org>

Merge pull request #6842 from badone/wip-13997

tests: centos7 needs the Continuous Release (CR) Repository enabled for

Reviewed-by: Loic Dachary <ldachary@redhat.com>

tests: centos7 needs the Continuous Release (CR) Repository enabled for
libunwind

Fixes: #13997
Signed-off-by: Brad Hubbard <bhubbard@redhat.com>

Merge pull request #6818 from trociny/fix-cmake

cmake: update for recent rbd changes

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #6837 from dreamhost/wip-test-cephtool-filenames

qa/workunits/cephtool/test.sh: false positive fail on /tmp/obj1.

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

qa/workunits/cephtool/test.sh: false positive fail on /tmp/obj1.

If /tmp/obj1 happened to exist already, and was not writable by the
testing user, then this test failed!

Signed-off-by: Robin H. Johnson <robin.johnson@dreamhost.com>

Merge pull request #6838 from stiopaa1/Wreorder_ceph_context

common/ceph_context.cc:fix order of initialisers

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #6831 from fullerdj/wip-14010

qa/krbd: Expunge generic/247

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #6793 from dzafman/wip-hist-les-fix

osd: Allow repair of history.last_epoch_started using config

Reviewed-by: Samuel Just <sjust@redhat.com>

common/ceph_context.cc:fix order of initialisers

Fix order of member initialisers causing [-Wreorder] warning.

Signed-off-by: Michal Jarzabek <stiopa@gmail.com>

Merge branch 'unlinkcallbyrmbucket'

Conflicts:
src/rgw/rgw_bucket.cc

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>

rgw:dont update entrypoint when removing bucket

when removing bucket,we unlink the bucket bucket but we don't want
to update the entrypoint here.Because the update_entrypoint's default
value is true,so we need to indicate it to false.

Signed-off-by: Zengran Zhang <zhangzengran@h3c.com>

Merge pull request #6735 from rzarzynski/wip-rgw-clean-putmetabucket

rgw: remove unused variable in RGWPutMetadataBucket::execute

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>

Merge pull request #6829 from LenzGr/master

doc: Fixed incorrect name of a "List Multipart Upload Parts" Response Entity

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>

Merge pull request #6774 from ceph/wip-rgw-cap-fix

rgw: add a missing cap type

qa/krbd: Expunge generic/247

xfstests generic/247 exercises XFS DIO and AIO to detect races. Some
races apparently exist in the XFS code as this is a known issue within
XFS. Expunge this test because it is not specifically relevant to krbd
and not a specific krbd issue.

Signed-off-by: Douglas Fuller <dfuller@redhat.com>

Merge pull request #6805 from zaitcev/cmake-pluginregistry

cmake: Add common/PluginRegistry.cc to CMakeLists.txt

doc: Fixed incorrect name of a "List Multipart Upload Parts" Response Entity

Fixed incorrect name of a "List Multipart Upload Parts" Response Entity:
"InitiatedMultipartUploadsResult" -> "ListPartsResult"

Fixes: #14003
Signed-off-by: Lenz Grimmer <lenz@grimmer.com>

cmake: update for recent rbd changes

Signed-off-by: Mykola Golub <mgolub@mirantis.com>

Merge pull request #6816 from drunkard/doc

doc: rst style fix for pools document

Reviewed-by: Loic Dachary <ldachary@redhat.com>

Merge pull request #6820 from mikulely/add-umcloud-affiliation

Add UMCloud affiliation

Reviewed-by: Loic Dachary <ldachary@redhat.com>

Update Jiaying Ren affinity

Signed-off-by: Jiaying Ren <jiaying.ren@umcloud.com>

Update Rongze Zhu affinity

Signed-off-by: Jiaying Ren <jiaying.ren@umcloud.com>

Merge pull request #6655 from jcsp/wip-inode-claim

tools: add cephfs-table-tool 'take_inos'

Reviewed-by: Yan, Zheng <zyan@redhat.com>

Merge pull request #6567 from jcsp/wip-layout-restrict

mds: add 'p' flag in auth caps to control setting pool in layout

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #6601 from jcsp/wip-13777-caps

mon: don't require OSD W for MRemoveSnaps

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Yan, Zheng <zyan@redhat.com>
Reviewed-by: Greg Farnum <gfarnum@redhat.com>
Reviewed-by: Sage Weil <sage@redhat.com>

doc: rst style fix for pools document

Signed-off-by: Drunkard Zhang <gongfan193@gmail.com>

deb,rpm: package buffer_fwd.h

Introduced by 206b8e2cc6157451d5c7548e294461408e0b0c8f.

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #6803 from dachary/wip-13975-systemd-flock

build/ops: systemd ceph-disk unit must not assume /bin/flock

Reviewed-by: Nathan Cutler <ncutler@suse.cz>

Merge pull request #6811 from smithfarm/wip-submitting-jewel

SubmittingPatches: there is no next; only jewel

Reviewed-by: Loic Dachary <ldachary@redhat.com>

SubmittingPatches: there is no next; only jewel

Signed-off-by: Nathan Cutler <ncutler@suse.com>

Merge pull request #6792 from smithfarm/wip-index-20151202

Wip index 20151202

Reviewed-by: Loic Dachary <ldachary@redhat.com>

Merge pull request #6795 from ceph/pybind-rados-exec

pybind: support ioctx:exec

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Add common/PluginRegistry.cc to CMakeLists.txt

Else this happens:

  [ 15%] Linking CXX executable ceph-authtool
  libcommon.a(ceph_context.cc.o): In function `CephContext::CephContext(unsigned int, int)':
  ceph_context.cc:(.text+0x2620): undefined reference to `ceph::PluginRegistry::PluginRegistry(CephContext*)'
  libcommon.a(ceph_context.cc.o): In function `CephContext::~CephContext()':
  ceph_context.cc:(.text+0x3db0): undefined reference to `ceph::PluginRegistry::~PluginRegistry()'
  collect2: error: ld returned 1 exit status
  src/CMakeFiles/ceph-authtool.dir/build.make:120: recipe for target 'src/ceph-authtool' failed
  make[2]: *** [src/ceph-authtool] Error 1
  CMakeFiles/Makefile2:331: recipe for target 'src/CMakeFiles/ceph-authtool.dir/all' failed

The ceph_context.cc is the only user of PluginRegistry at
present, so adding it without conditionals should be correct.

Signed-Off-By: Pete Zaitcev <zaitcev@redhat.com>

doc/dev/index.rst: wholesale refactor

Signed-off-by: Nathan Cutler <ncutler@suse.com>

pybind: support ioctx:exec

Signed-off-by: Noah Watkins <noahwatkins@gmail.com>

common/Makefile: ship common/event_socket.h

Broken in commit 5eb64dd324a0e0ec609acf2b178c02dee901aa5d.

Signed-off-by: Sage Weil <sage@redhat.com>

build/ops: systemd ceph-disk unit must not assume /bin/flock

The flock command may be installed elsewhere, depending on the
system. Let the PATH search figure that out.

http://tracker.ceph.com/issues/13975 Fixes: #13975

Signed-off-by: Loic Dachary <loic@dachary.org>

test: use sequential journal_tid for object cacher test

This matches the real usage by librbd.

Fixes: #13877
Signed-off-by: Josh Durgin <jdurgin@redhat.com>
(cherry picked from commit 9331e031bd2a719463920581a47c13f0606e9971)

Merge pull request #6788 from dachary/wip-13878-ec-pool

test/librados/test.cc: clean up EC pools' crush rules too

Reviewed-by: Dan Mick <dmick@redhat.com>

Merge pull request #6665 from trociny/wip-librbd-journal

rbd: journal: configuration via conf, cli, api and some fixes

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #6800 from dillaman/wip-13969

librbd: fix known test case race condition failures

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Loic Dachary <ldachary@redhat.com>

tests: wait for mocked requests to complete

Fixes: #13969
Signed-off-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #6641 from aclamk/wip-log-alloc-predictor

common: log: predict log message buffer allocation size

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #6642 from 0003ydlom/decode_reorder

osd: move newest decode version of MOSDOp and MOSDOpReply to the front

Reviewed-by: Sage Weil <sage@redhat.com>

librbd: do not complete AIO callbacks within caller's thread context

Avoid rare, racy issues when individual requests associated with an AIO
completion can complete prior to marking the completion as ready-to-fire.
Pre-calculate the expected number of individual requests to avoid the
potential re-entrant callback.

Fixes: #13969
Signed-off-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #6696 from Ved-vampir/wip_common_plugin

common: add generic plugin infrastructure

Reviewed-by: Varada Kari <varada.kari@sandisk.com>

Merge pull request #6779 from ceph/wip-librados-xattr

librados: avoid malloc(0) (which can return NULL on some platforms)

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #6759 from liewegas/wip-bdev-probe

osd: make backend and block device code a bit more generic

Reviewed-by: Loic Dachary <ldachary@redhat.com>
Reviewed-by: Varada Kari <varada.kari@sandisk.com>

Merge pull request #6686 from stiopaa1/namespace_buffer

common: use namespace instead of subclasses for buffer

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #6535 from mslovy/wip-crc-cache-tracker

common: buffer: add cached_crc and cached_crc_adjust counts to perf dump

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #6058 from guangyy/wip-pg-query

osd: dump number of missing objects for each peer with pg query

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #5953 from guangyy/wip-13121

osd: support pool level recovery_priority and recovery_op_priority

Reviewed-by: David Zafman <dzafman@redhat.com>

Merge pull request #5670 from david-z/master

osd: FileJournal: reduce locking scope in write_aio_bl

Reviewed-by: Haomai Wang <haomai@xsky.com>

Merge pull request #6798 from trociny/fix-rbd_mirror-bool

librbd: include missing header for bool type

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

librbd: include missing header for bool type

Signed-off-by: Mykola Golub <mgolub@mirantis.com>

rbd: alow to specify options for created journal

Signed-off-by: Mykola Golub <mgolub@mirantis.com>

librbd: journal options on image create/clone/copy

Signed-off-by: Mykola Golub <mgolub@mirantis.com>

librbd: output operator for image options

It is going to be used for diagnostic.

Signed-off-by: Mykola Golub <mgolub@mirantis.com>

librbd: journal: mark entry committed after replay

After replying a journal entry we have to call committed(). Otherwise,
the entries remain with flag committed=false in JournalMetadata
m_pending_commit_tids forever, which prevents commit position update.

Signed-off-by: Mykola Golub <mgolub@mirantis.com>

rbd: add new journal rbd commands

Also, extend 'rbd info' to output journal spec if journaling is enabled.

Signed-off-by: Mykola Golub <mgolub@mirantis.com>

librbd: humanize ImageWatcher and Journal states in debug log

Signed-off-by: Mykola Golub <mgolub@mirantis.com>

librbd: add image pool ID to journal object names.

The data objects are named "journal_data.<pool id>.<image id>.<num>".
This is to avoid conflict when a journal objects pool is used for
several image pools.

Signed-off-by: Mykola Golub <mgolub@mirantis.com>

librbd: allow alternate pool for journal objects

Signed-off-by: Mykola Golub <mgolub@mirantis.com>

librbd: debug: log journal metadata

Signed-off-by: Mykola Golub <mgolub@mirantis.com>

librbd: new config options to tweak journal settings

Signed-off-by: Mykola Golub <mgolub@mirantis.com>

Merge pull request #6129 from dillaman/wip-13292

RBD pool mirroring configuration API + CLI

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

osd: Test osd_find_best_info_ignore_history_les config in another assert

Signed-off-by: David Zafman <dzafman@redhat.com>

Merge pull request #6778 from liewegas/wip-13962

osd: call on_new_interval on newly split child PG

Reviewed-by: Samuel Just <sjust@redhat.com>

test/librados/test.cc: clean up EC pools' crush rules too

SetUp was adding an erasure-coded pool, which automatically adds
a new crush rule named after the pool, but only removing the
pool.  Remove the crush rule as well.

http://tracker.ceph.com/issues/13878 Fixes: #13878

Signed-off-by: Dan Mick <dan.mick@redhat.com>
Signed-off-by: Loic Dachary <loic@dachary.org>

Merge pull request #6780 from smithfarm/wip-index-20151201

doc/dev/index.rst: fix headings

Reviewed-by: Loic Dachary <ldachary@redhat.com>

Merge pull request #6786 from oritwas/wip-cmake-fix-radosgw

cmake: add rgw_basic_types.cc to librgw.a

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>

Merge pull request #6767 from oritwas/wip-13529-jewel

rgw: use smart pointer for C_Reinitwatch

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>

doc/dev/index.rst: fix links

Signed-off-by: Nathan Cutler <ncutler@suse.com>

doc/dev/index.rst: Flesh out the IRC section

Signed-off-by: Nathan Cutler <ncutler@suse.com>

doc/dev/index.rst: Make Issue tracker a separate chapter

Signed-off-by: Nathan Cutler <ncutler@suse.com>

Merge pull request #6789 from dillaman/wip-13969

librbd: partial revert of commit 9b0e359

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #6748 from liewegas/wip-fix-cephtool

qa/workunits/cephtool/test.sh: no ./

Reviewed-by: Loic Dachary <ldachary@redhat.com>

Merge pull request #5941 from jcsp/wip-12137

#12137: cephfs-data-scan scan_frags

Merge pull request #6777 from dillaman/wip-librbd-lockdep

librbd: journaling-related lock dependency cleanup

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #6762 from dillaman/wip-13924

journal: fire replay complete event after reading last object

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #6703 from dillaman/wip-tests-librbd-gmock-state-machines

librbd: initial collection of state machine unit tests

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

librbd: partial revert of commit 9b0e359

Fixes: #13969
Signed-off-by: Jason Dillaman <dillaman@redhat.com>

doc/dev/index.rst: start writing Bugfixing chapter

Signed-off-by: Nathan Cutler <ncutler@suse.com>

Merge pull request #6690 from branch-predictor/bp-smaller-pglog-2

osd: slightly reduce actual size of pg_log_entry_t

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #6689 from 0003ydlom/doubled_cond

osd: avoid duplicate op->mark_started in ReplicatedBackend

Reviewed-by: xinxin shu <xinxin.shu@intel.com>

Merge pull request #6681 from ifed01/wip-objstore-bench-stabilization

ceph_objectstore_bench: fix race condition, bugs

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #6677 from jack-changtao/test-bench

rados: bench: fix off-by-one to avoid writing past object_size

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #6555 from mslovy/wip-reduce-omap-transaction

osd: don't update epoch and rollback_info objects attrs if there is no need

Reviewed-by: Sage Weil <sage@redhat.com>