librbd: handle additional journal replay states

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

librbd: avoid recursive callback when replaying journal

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

tests: correct googlemock warnings

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

tests: update tests to support replaying snap create / resize

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

librbd: initial hook points for snap create / resize journal replay

After snap create has blocked IO and after resize has ensured clipped
IO boundaries, they will resume the playback of journal events.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

librbd: journal events still possible during shutdown

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

tests: new unit test for snap rollback state machine

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

librbd: snapshot rollback should block writes

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

librbd: track simple maintenance ops to coordinate replay

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

librbd: asynchronously replay journal events

Properly track when AIO operations are safely committed to
disk before marking the journal events are committed.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

tests: unit test cases for all available journal events

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

librbd: initial integration of maintenance op journal replay

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

librbd: AioImageRequest base class is now templated

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

tests: updated librbd test cases for updated maint ops API

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

librbd: moved all maintenance ops to new Operations class

This will permit unit testing and will facilitate proper
serialization of requests (when necessary).

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

tests: journal event handling test updates

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

tests: unit test case for librbd::operation::ResizeRequest

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

librbd: properly order maintenance ops journal events

In general, the op shoud only start after the jouranl event is
safely committed.  Snapshot create and resize need special treatment
to record the event while no IO is in-progress.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

librbd: fixed "hidden" virtual methods

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

librbd: handle journal and IO commits in either sequence

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

librbd: allocate and associate op tids to maint op journal events

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

tests: unit tests for librbd Journal

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

librbd: journal replay flush should be async

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

librbd: convert journal classes to templates for unit testing

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

tests: compile RBD and RGW types into ceph-dencoder when enabled

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

librbd: move journal support classes to journal namespace

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

Merge branch 'wip-compressor'

Reviewed-by: Sage Weil <sage@redhat.com>

Compressor: compressor plugins unit tests

Signed-off-by: Alyona Kiseleva <akiselyova@mirantis.com>

Compressor: compressor code extention: plugin system added

Signed-off-by: Alyona Kiseleva <akiselyova@mirantis.com>

Merge pull request #6425 from branch-predictor/bp-delayed-pglog-index-v2

osd: delay populating in-memory PG log hashmaps

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #6797 from mathslinux/fix-parse-return

rgw: fix wrong check for parse() return

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #7087 from H3C/wip-yrf-cephdf

mon: add RAW USED column to ceph df detail

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #7106 from tchaikov/wip-11101

ceph: improve the error message

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #7122 from xiexingguo/xxg-wip-bluefs

osd: bluestore: fix several bugs

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #7196 from liewegas/wip-bluestore

osd: bluestore: fix space rebalancing, collection split, buffered reads

Merge pull request #7200 from xiexingguo/xxg-wip-14343

osd: bluestore: bluefs: fix several small bugs

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #7213 from xiexingguo/xxg-wip-14351

osd: kstore: several small fixes

Reviewed-by: Sage Weil <sage@redhat.com>

Merge remote-tracking branch 'upstream/jewel'

Merge pull request #7222 from dack/master

ceph-disk: add -f flag for btrfs mkfs

Reviewed-by: Sage Weil <sage@redhat.com>

Merge branch 'greg-fs-testing'

fsx: checkout old version until it compiles properly on miras

I sent a patch to xfstests upstream at
http://article.gmane.org/gmane.comp.file-systems.fstests/1665, but
until that's fixed we need a version that works in our test lab.

Signed-off-by: Greg Farnum <gfarnum@redhat.com>

Merge branch 'wip-13569' of git://github.com/ukernel/ceph into greg-fs-testing

https://github.com/ceph/ceph/pull/6463

Reviewed-by: Greg Farnum <gfarnum@redhat.com>

Merge branch 'wip-13127-master' into greg-fs-testing

https://github.com/ceph/ceph/pull/5990

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge remote-tracking branch 'origin/wip-libcephfs-acl' into greg-fs-testing

https://github.com/ceph/ceph/pull/5658

Reviewed-by: John Spray <jspray@redhat.com>
Conflicts:
src/common/config_opts.h

Signed-off-by: Greg Farnum <gfarnum@redhat.com>

Merge branch 'wip-root-metadata' of git://github.com/jcsp/ceph into greg-fs-testing

https://github.com/ceph/ceph/pull/6179

Reviewed-by: Greg Farnum <gfarnum@redhat.com>

doc/release-notes: v10.0.2

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #7235 from theanalyst/fix/pybind-build-failure

pybind: fix build failure, remove extraneous semicolon in method

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

pybind/rbd.pyx: drop stray semicolon

Signed-off-by: Sage Weil <sage@redhat.com>

os/bluestore/FreelistManager: fix _dump locking

Signed-off-by: Sage Weil <sage@redhat.com>

pybind: remove extraneous semicolon in method

A semicolon on `rbd_build_object_map` method was causing build failures
in current master

Fixes: #14371
Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>

osd: less chatter about tick

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #7212 from caibo2014/fix-cephin-pipe-error

ceph.in: avoid a broken pipe error when use ceph command

Reviewed-by: Kefu Chai <kchai@redhat.com>

ceph.in:avoid a broken pipe error when use ceph command

Fixes: #14354
Signed-off-by: Bo Cai <cai.bo@h3c.com>

kstore: remove unreachable code

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

Merge pull request #7210 from swamireddy/contrib_list_10

mailmap update

Reviewed-by: Loic Dachary <ldachary@redhat.com>

Merge pull request #7227 from dwj192/dwj-paxos

mon: paxos is_recovering calc error

Reviewed-by: Joao Eduardo Luis <joao@suse.de>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #7216 from jcsp/wip-14357

mds: advance clientreplay when replying

Merge pull request #7219 from aiicore/affiliation

mailmap: Igor Podoski affiliation

Reviewed-by: Loic Dachary <ldachary@redhat.com>

BlueStore: decrease max listed object number from 5 to 1

For the sake of performance.

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

kstore: decrease max listed object number from 5 to 1

For the sake of performance.

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

BlueStore: kill warnings

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

BlueStore: fix wrong verification of object key

Fixes: #14353
Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

BlueStore: fix unhandle error cases during block file setup

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

BlueStore: remove unused local variable "empty"

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

BlueStore: remove redundant omap_head check

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

BlueStore: fix potential null pointer access

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

BlueStore: fix result code overwritten

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

BlueStore: fix shared_ptr leak during fsck

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

BlueStore: fix shared_ptr leak during mount

Fixes: #14261
Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

BlueStore: fix typo

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

BlueStore: fix memory leak in several abnormal cases

Fixes: #14260
Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

BlueStore: reset result code to errno for better tracing

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

BlueStore: skip CRC verification if unable to read label

Fixes: #14259
Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

BlueStore: differentiate error code for better tracing

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

mon: paxos is_recovering calc error

Fix:#14368

Signed-off-by: Weijun Duan <duanweijun@h3c.com>

Merge pull request #7224 from dillaman/wip-13818

tests: snap rename and rebuild object map in client update test

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #7220 from dillaman/wip-13810-jewel

tests: notification slave needs to wait for master

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #7208 from dillaman/wip-14163-jewel

helgrind: fix real (and imaginary) race conditions

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

tests: snap rename and rebuild object map in client update test

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

pybind/rbd.pyx: add rebuild_object_map method

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #7215 from trociny/wip-rbd-nbd-fixes

rbd-nbd: fix up return code handling

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

add -f flag for btrfs mkfs

os/bluestore: use auto for freelist map type

Signed-off-by: Sage Weil <sage@redhat.com>

os/bluestore/BlockDevice: disable readahead of fd_buffered

Readahead can break our mix of aio/dio and buffered io.  Although the
dio path will invalidate pages before and after initiating the IO, and we
do not issue racing buffered reads, the readahead code might.  That would
result in pages in the page cache that are not coherent with an aio write.

This is easy to reproduce with ceph_test_objectstore's Synthetic case
when bluestore_default_buffered_read=true.

Signed-off-by: Sage Weil <sage@redhat.com>

unittest_rocksdb_option_static: fix signed/unsigned warnings

Signed-off-by: Sage Weil <sage@redhat.com>

os/bluestore: add flag to make reads default to buffered

Signed-off-by: Sage Weil <sage@redhat.com>

tests: notification slave needs to wait for master

If the slave instance starts before the master, race
conditions are possible.

Fixes: #13810
Backport: infernalis, hammer
Signed-off-by: Jason Dillaman <dillaman@redhat.com>

librbd: log write/discard errors from clip operation

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

mailmap: Igor Podoski affiliation

Signed-off-by: Igor Podoski <igor.podoski@ts.fujitsu.com>

10.0.2

Merge pull request #7174 from xiexingguo/xxg-wip-14332

librbd: fix potential memory leak

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #7165 from xiexingguo/xxg-wip-14316

cls_rbd: add guards for error cases

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

osdc: hold rwlock when updating crush location config

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

msg/simple: destroy spinlock in destructor

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

osdc: hold writer lock when updating session ops

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

librados: hold lock while testing state variable

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

config: avoid re-entrant locking on observer callback

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

WorkQueue: delay registering PointerWQ with thread pool

Derived classes should be responsible for registration to avoid
accessing a partially constructed object.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

helgrind: suppress race conditions in external libraries

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

helgrind: annotate false-positive race conditions

Fixes: #14163
Signed-off-by: Jason Dillaman <dillaman@redhat.com>