librbd: provide an out-of-class definition for MAX_DESCRIPTION_OFFSET

std::min() takes its parameters by reference, so provide a definition
for OptionPrinter::MAX_DESCRIPTION_OFFSET constant.

Signed-off-by: Ilya Dryomov <idryomov@redhat.com>

cmake: fix librbd and add src/journal

librbd was missing a recently added file, src/journal was missing
entirely - this brings the build back.

Signed-off-by: Ilya Dryomov <idryomov@redhat.com>

Merge pull request #6462 from javacruft/wip/ec-modules

Build internal plugins and classes as modules

Reviewed-by: Loic Dachary <ldachary@redhat.com>

Merge pull request #6539 from yehudasa/wip-cmake-fix

cmake: fix files list

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #6316 from dillaman/wip-13494

rbd: dynamically generated bash completion

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge remote-tracking branch 'origin/jewel'

cmake: fix files list

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

Merge pull request #6471 from mslovy/wip-check-reply-guard-again

osd: avoid double-check for replaying and can_checkpoint() in FileStore::_check_replay_guard

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #6426 from xiexingguo/xxg-wip-13642

librados: fix rare race where pool op callback may hang forever

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #6407 from renhwztetecs/renhw-wip-13627

mon: initialize recorded election epoch properly even when standalone

Reviewed-by: Joao Eduardo Luis <joao@suse.de>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #6385 from Vicente-Cheng/mkfs-does-not-decode-an-existing-superblock

osd: fix reactivate (check OSDSuperblock in mkfs() when we already have the superblock)

Reviewed-by: Loic Dachary <ldachary@redhat.com>

Merge pull request #6373 from liewegas/wip-promoteon2ndread-ec

librados: fix PromoteOn2ndRead test for EC

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #6520 from trociny/fix-rbd-apply_changes

rbd: make config changes actually apply

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #6347 from xiexingguo/xxg-wip-13566

osd: fixes for several cases where op result code was not checked or set

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #5168 from majianpeng/dynamic-control-optracker

common: allow enable/disable of optracker at runtime

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #6089 from SUSE/wip-init-ceph-getopt

init-ceph: use getopt to make option processing more flexible

Reviewed-by: Kefu Chai <kchai@redhat.com>

src/init-ceph.in: remove unused variables

Signed-off-by: Nathan Cutler <ncutler@suse.com>

src/init-ceph.in: improve usage message

Signed-off-by: Nathan Cutler <ncutler@suse.com>

src/init-ceph.in: process command-line options using getopt

http://tracker.ceph.com/issues/3015 Fixes: #3015

Signed-off-by: Nathan Cutler <ncutler@suse.com>

Merge pull request #6516 from trociny/wip-13719

librbd: perf section name: use hyphen to separate components

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

rbd: make config changes actually apply

Signed-off-by: Mykola Golub <mgolub@mirantis.com>

tests: rbd/admin_socket: use xmlstarlet when parsing perf dump

Signed-off-by: Mykola Golub <mgolub@mirantis.com>

librbd: perf section name: use hyphen to separate components

"/" and "@" characters make invalid xml format output.

Fixes: #13719
Signed-off-by: Mykola Golub <mgolub@mirantis.com>

doc: Adding --cluster option to rbd man page.

Fixes #13457
Signed-off-by: Vikhyat Umrao <vumrao@redhat.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #6519 from kylinstorage/wip-revise-organization

revise organization

Reviewed-by: Loic Dachary <ldachary@redhat.com>

Merge pull request #6525 from dillaman/wip-13636-jewel

WorkQueue: new PointerWQ base class for ContextWQ

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge branch 'wip-11286' of https://github.com/dillaman/ceph

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #5987 from dillaman/wip-rbd-refactor

rbd: refactor cli command handling

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

rbd: dynamically generated bash completion

The rbd CLI bash completion is now dynamically generated by extracting
the available commands and command options from the CLI application.

Fixes: #13494
Signed-off-by: Jason Dillaman <dillaman@redhat.com>

rbd: hidden 'bash-completion' command dumps all available commands

It also extracts all available options for a given command for use
which can be used by an improved bash completion script.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

tests: updated rbd CLI --image-feature optional

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

ceph_test_rados_api_tier: fix PromoteOn2ndRead for EC case

The ec and non-ec cases are copy&pasted code.  Yuck.  This duplicates the
non-ec fix from 347ac0f8.

Signed-off-by: Sage Weil <sage@redhat.com>

rbd: corrected handling of '--image-feature' optional

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #6488 from xiexingguo/xxg-wip-rados-cleanup

librados: do cleanup

Reviewed-by: Sebastien Ponce <sebastien.ponce@cern.ch>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #6523 from dillaman/wip-13740

librbd: perf counters might not be initialized on error

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

WorkQueue: new PointerWQ base class for ContextWQ

The existing work queues do not properly function if added to a running
thread pool.  librbd uses a singleton thread pool which requires
dynamically adding/removing work queues as images are opened and closed.

Fixes: #13636
Signed-off-by: Jason Dillaman <dillaman@redhat.com>

librbd: perf counters might not be initialized on error

Fixes: #13740
Signed-off-by: Jason Dillaman <dillaman@redhat.com>

PendingReleaseNotes: document updated rbd CLI options

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #6522 from branch-predictor/bp-temporary-mon-fix

mon: revert MonitorDBStore's WholeStoreIteratorImpl::get

Reviewed-by: Loic Dachary <ldachary@redhat.com>

mon: revert MonitorDBStore's WholeStoreIteratorImpl::get

Revert MonitorDBStore's WholeStoreIteratorImpl::get method to state
before commit 66b7b920cf5a0a9c71212573ef47fb2c7ea9b5ff until better,
long-term solution for #13742 will be provided.

Signed-off-by: Piotr Dałek <piotr.dalek@ts.fujitsu.com>

revise organization

Signed-off-by: Li Wang <li.wang@kylin-cloud.com>

Merge pull request #6394 from dillaman/wip-13560

krbd: remove deprecated --quiet param from udevadm

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #6508 from dillaman/wip-13726

librbd: deadlock while attempting to flush AIO requests

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #6447 from dillaman/wip-13674

librbd: resize should only update image size within header

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

internal: remove unused local variables

Remove unused variables bl and bl2 in snap_list and ictx_refresh.
Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

librados: cast oid to object explicitly before call ioctx methods
Cast oid to object explicitly before call ioctx methods.
Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

Merge pull request #6513 from ceph/revert-6419-wip-openssl

Revert 0374bb4a2f5054d606e4aba2d97b5e6765e781b0

IoCtxImpl: remove unused variable sName

remove unused local variable sName in rollback.
Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

Revert 0374bb4a2f5054d606e4aba2d97b5e6765e781b0

Merge pull request #6419 from ceph/wip-openssl

rgw: link against system openssl (instead of dlopen at runtime)

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>

Merge pull request #6453 from trociny/wip-2468

librbd: flush and invalidate cache via admin socket

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #6511 from liewegas/wip-fix-kv

kv: fix bug in kv key optimization

Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Piotr Dałek <piotr.dalek@ts.fujitsu.com>

Merge branch 'wip-12955'

rgw: fix modification to index attrs when setting acls

Fixes: #12955
 - add ACL change after initial population
 - populate bufferlist with object attributes

Signed-off-by: Brian Felton <bjfelton@gmail.com>

kv: fix string ctor usage

When constructing a string from a char* there is only a single
length argument, no offset.  The 3 argument variant we were using
was converting to a std::string first (an prematurely terminating
on \0).

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #6047 from Sandy4999/wip-13140

rgw: fix swift API returning incorrect account metadata

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>

Merge pull request #6504 from dillaman/wip-13727

tests: fix typo in TestClsRbd.snapshots test case

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

librbd: fixed deadlock while attempting to flush AIO requests

In-flight AIO requests might force a flush if a snapshot was created
out-of-band.  The flush completion was previously invoked asynchronously,
potentially via the same thread worker handling the AIO request. This
resulted in the flush operation deadlocking since it can't complete.

Fixes: #13726
Backport: infernalis, hammer
Signed-off-by: Jason Dillaman <dillaman@redhat.com>

tests: new test case to catch deadlock on RBD image refresh

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

tests: librbd: admin socket commands to flush and invalidate cache

Signed-off-by: Mykola Golub <mgolub@mirantis.com>

librbd: flush and invalidate cache via admin socket

Fixes: #2468
Signed-off-by: Mykola Golub <mgolub@mirantis.com>

librbd: perf counter for cache invalidates

Signed-off-by: Mykola Golub <mgolub@mirantis.com>

Merge pull request #6312 from liewegas/wip-kv

kv: optimize and clean up internal key/value interface

Reviewed-by: Piotr Dałek <piotr.dalek@ts.fujitsu.com>

tests: fix typo in TestClsRbd.snapshots test case

Fixes: #13727
Signed-off-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #6300 from XinzeChi/wip-throttle-max

common: fix reset max in Throttle using perf reset command

Reviewed-by: Loic Dachary <ldachary@redhat.com>

Merge pull request #6364 from dillaman/wip-13491

osd: fix broken balance / localized read handling

Reviewed-by: Kefu Chai <kchai@redhat.com>

tracing: fix librados signed/unsigned warnings

Signed-off-by: Sage Weil <sage@redhat.com>

os/osd: disable extra iterator validation

In a number of loops using ObjectMapIterator, the iterator is validated
twice, first as an loop break condition, then during iter->next() call.
Suppress the validation in next() method in those cases for better
performance.

Signed-off-by: Piotr Dałek <piotr.dalek@ts.fujitsu.com>

os/KeyValueDB: don't call self.valid() from next() and prev()

In most loops using ObjectMapIteratorImpl, there is a call to valid()
followed by a call to next(), which calls valid() again. Calling
valid() alone is particularly expensive, so don't do it twice if
possible.

Signed-off-by: Piotr Dałek <piotr.dalek@ts.fujitsu.com>

mon/MonitorDBStore: use single-key LevelDB::get() method

Make MonitorDBStore use single-key ::get() method for some performance
increase in heavy K/V workloads.

Signed-off-by: Piotr Dałek <piotr.dalek@ts.fujitsu.com>

os/DBObjectMap: use single-key LevelDB::get method

Make FileStore use single-key ::get() method for some performance
increase in k/v-heavy workloads.

Signed-off-by: Piotr Dałek <piotr.dalek@ts.fujitsu.com>

kv/LevelDBStore: simpler code for single-key fetches

Often there's a need to fetch value for a specified, single key. Publish
a new method in LevelDB that takes key and prefix and puts just value at
specified address, without the need for interim keysets or resultsets.

Signed-off-by: Piotr Dałek <piotr.dalek@ts.fujitsu.com>

os/LevelDBStore: faster LevelDBTransactionImpl::set

This patch builds on Sage's idea to reduce bufferlist copying on
::set() calls. Initial patch reduced LevelDB's Slice generation
time from ~57ns to ~5ns in best-case scenario (bufferlist with
single bufferptr or contiguous bufferptrs), this patch reduces
slice generation time from ~57ns to ~11ns in worst case scenario
(under assumption that entire value length is at most 128KB),
leaving old code path for extremely-bad cases.

Signed-off-by: Piotr Dałek <piotr.dalek@ts.fujitsu.com>

kv/RocksDBStore: do not Delete before Put

A put implicitly overwrites the previous value.

Signed-off-by: Sage Weil <sage@redhat.com>

kv/LevelDBStore: do not Delete before Put

A put implicitly overwrites the previous value.

Signed-off-by: Sage Weil <sage@redhat.com>

osd/ReplicatedPG: use bl-based setkeys/rmkeys

FIXME: what about the tracepoint?

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #6502 from dachary/wip-13691-rados-test

tests: fix test_rados_tools.sh rados lookup

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #6403 from xinxinsh/wip-13500

optimize clone write path if object-map is enabled

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #6494 from trociny/wip-13720

librbd: start perf counters after id is initialized

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

tests: fix test_rados_tools.sh rados lookup

http://tracker.ceph.com/issues/13691 Fixes: #13691

Signed-off-by: Loic Dachary <loic@dachary.org>

Merge pull request #6464 from dachary/wip-13691-rados-test

tests: concatenate test_rados_test_tool from src and qa

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #6497 from ukernel/wip-13714

client: use null snapc to check pool permission

Reviewed-by: John Spray <john.spray@redhat.com>

client: use null snapc to check pool permission

snap inodes' ->snaprealm can be NULL, so dereferencing it in
check_pool_perm() can cause segment fault. The pool permission
check does not write any data, so it's safe to use null snapc.

Fixes: #13714
Signed-off-by: Yan, Zheng <zyan@redhat.com>

librbd: start perf counters after id is initialized

So the perf counters section name has non-empty id.

Fixes: #13720
Signed-off-by: Mykola Golub <mgolub@mirantis.com>

Merge pull request #6228 from zhouyuan/fix_rgw_delete_exipre

rgw: fix response of delete expired objects

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>

Merge pull request #6214 from jmunhoz/wip-bucket-request-payment

rgw: bucket request payment support

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>

Merge pull request #5969 from rzarzynski/wip-conf-md-add

rgw: extend rgw_extended_http_attrs to affect Swift accounts and containers as well

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>

Merge pull request #5938 from zhuangzeqiang/rgw-cerr

rgw:  clarify the error message when trying to create an existed user

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>

Merge pull request #6479 from ceph/wip-systemd-rbdmap

rbdmap: systemd support

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #6467 from fullerdj/wip-13697

cls_rbd: change object_map_update to return 0 on success, add logging

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #6059 from dwj192/dwj-master-tip

rgw:mdlog trim add usage prompt

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>

Merge pull request #6408 from dwj192/swift-civetweb-ssl

rgw:swift use Civetweb ssl can not get right url

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>

cls_rbd: change object_map_update to return 0 on success, add logging

The rbd_object_map_update class method returns the size of the data read
from the object map when no update is necessary. Change this to return 0
on success. Add error logging to differentiate error cases.

Signed-off-by: Douglas Fuller <dfuller@redhat.com>

Merge pull request #6483 from ceph/wip-fix-doc-build

doc/releases-notes: fix build error

Reviewed-by: Loic Dachary <ldachary@redhat.com>

doc/releases-notes: fix build error

also change the release year of infernalis from 2014 to 2015

Signed-off-by: Kefu Chai <kchai@redhat.com>

rbdmap: systemd support

Fixes: #13374
Signed-off-by: Boris Ranto <branto@redhat.com>

Merge pull request #6476 from xiexingguo/xxg-wip-13703

librados: wrongly passed in argument for stat command

Reviewed-by: Kefu Chai <kchai@redhat.com>

ceph.spec.in: add new cls_journal RADOS class

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

journal: FutureImpl shouldn't hold lock while invoking callbacks

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

tests: journal updates to support C++11

Signed-off-by: Jason Dillaman <dillaman@redhat.com>