librbd: use task finisher per CephContext

Signed-off-by: Haomai Wang <haomai@xsky.com>

Merge pull request #7293 from xiexingguo/xxg-wip-14436

os/bluestore: fix assert

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #7197 from ceph/wip-kill-warnings

build: fix warnings

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Greg Farnum <gfarnum@redhat.com>

Merge pull request #7232 from lzhng2000/master

cls/cls_rbd: pass string by reference

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #7242 from xiexingguo/xxg-wip-14370

os/memstore: drain finisher first before we really begin a umount-process
os/bluestore: drain wal_wq on replay error

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #7247 from ifed01/wip-14383

os/bluestore: use intrusive_ptr for Dir

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #7254 from guangyy/wip-counter

common: change the type of counter total/unhealthy_workers

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #7255 from yuyuyu101/wip-14386

msg/async: fix send closed local_connection message problem

Reviewed-by: Sage Weil <sage@redhat.com>

Merge remote-tracking branch 'gh/jewel'

Merge pull request #7300 from ceph/wip-cmake

cmake: test_build_libcephfs needs ${ALLOC_LIBS}

cmake: test_build_libcephfs needs ${ALLOC_LIBS}

Signed-off-by: Ali Maredia <amaredia@redhat.com>

Merge pull request #7231 from renhwsky/renhw-wip-14369

osd: remove unused OSDMap::set_weightf()

Reviewed-by: Kefu Chai <kchai@redhat.com>

BlueStore/StupidAllocator: fix assert

Fixes: #14436
Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

Merge pull request #7164 from flaf/master

doc: fix "mon osd down out subtree limit" option name

Reviewed-by: Kefu Chai <kchai@redhat.com>

doc: fix "mon osd down out subtree limit" option name

The name of the parameter `mon osd down out subtree limit`
was incorrect (with `downout` instead of `down out`).
Furthermore, the description seemed to me a little bit
ambiguous.

Signed-off-by: François Lafont <francois.lafont@ac-versailles.fr>

Merge pull request #7246 from ukernel/jewel-14377

ceph_test_libcephfs: tolerate duplicated entries in readdir

Reviewed-by: Greg Farnum <gfarnum@redhat.com>

Merge pull request #6205 from jcsp/wip-volume-client

pybind: add ceph_volume_client interface for Manila and similar frameworks

Reviewed-by: Greg Farnum <gfarnum@redhat.com>

Merge pull request #7229 from david-z/wip-master-fix-failling-to-cache-pressure

ceph-fuse: fix double decreasing the count to trim caps

Reviewed-by: Yan, Zheng <zyan@redhat.com>

Merge pull request #7209 from ukernel/jewel-11517

client: check if Fh is readable when processing a read

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #7199 from ukernel/jewel-14254

mds: fix client cap/message replay order on restart

Reviewed-by: Greg Farnum <gfarnum@redhat.com>

Merge pull request #7279 from zaitcev/wip-5073-subuser

rgw: Parse --subuser better

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>

Merge remote-tracking branch 'origin/jewel'

Merge pull request #6496 from ceph/wip-fs-snaptests

qa/workunits/snaps: move snap tests into fs sub-directory

Reviewed-by: Greg Farnum <gfarnum@redhat.com>

Merge pull request #6868 from H3C/wip-11642-cephfuse-useage

ceph-fuse:print usage information when no parameter specified

Reviewed-by: Greg Farnum <gfarnum@redhat.com>

Merge pull request #7270 from cbodley/wip-cmake-execinfo

cmake: add missing check for HAVE_EXECINFO_H

Merge pull request #7280 from zaitcev/wip-5073-update2

rgw: Drop a debugging message

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>

rgw: Parse --subuser better

This fixes a funny message "ERROR: 'tenant$user != tenant$user'"
in case --subuser='tenant$user' is supplied.

Signed-off-by: Pete Zaitcev <zaitcev@redhat.com>

rgw: Drop a debugging message

A P3-tagged message escaped upstream.

Signed-off-by: Pete Zaitcev <zaitcev@redhat.com>

Merge pull request #7274 from dillaman/wip-helgrind-race-jewel

helgrind: additional race conditionslibrbd: journal replay should honor inter-event dependencies

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

librbd: missing lock on state transition

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

helgrind: annotate false-positive race conditions

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #7272 from dillaman/wip-14385-jewel

qa: disable rbd/qemu-iotests test case 055 on RHEL/CentOSlibrbd: journal replay should honor inter-event dependencies

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #7019 from dillaman/wip-14006

librbd: journal replay should honor inter-event dependencies

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

qa: disable rbd/qemu-iotests test case 055 on RHEL/CentOS

Fixes: #14385
Signed-off-by: Jason Dillaman <dillaman@redhat.com>

cmake: add missing check for HAVE_EXECINFO_H

Signed-off-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #7265 from ceph/wip-14397-disable-xfs-extsize

osd: disable filestore_xfs_extsize by default

Reviewed-by: Sage Weil <sage@redhat.com>

osd: disable filestore_xfs_extsize by default

This option involves a tradeoff: When disabled, fragmentation is worse,
but large sequential writes are faster. When enabled, large sequential
writes are slower, but fragmentation is reduced.

Fixes: #14397

os/kstore: fix u64 decode checks

Signed-off-by: Sage Weil <sage@redhat.com>

Merge branch 'wip-bluestore'

os/bluestore: fix _key_decode_u64 error checks

The data that follows may be binary encoded and null.

Signed-off-by: Sage Weil <sage@redhat.com>

os/bluestore/FreelistManager: switch to std::mutex

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #7211 from cxwshawn/clean-fix

osd: ReplicatedPG: clean up unused function

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #7158 from yuyuyu101/wip-async-cleanup-deleted

msg/async: cleanup dead connection and misc things

Merge pull request #6809 from yuyuyu101/wip-13985

msgr:  fix large message data content length causing overflow

Reviewed-by: Greg Farnum <gfarnum@redhat.com>

Merge pull request #7047 from xiexingguo/xxg-wip-14176

osd: KeyValueStore: fix wrongly placed assert

Reviewed-by: David Zafman <dzafman@redhat.com>

Merge pull request #7142 from xiexingguo/xxg-wip-crush

crushtool: improve usage/tip messages

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #7203 from branch-predictor/bp-use-faster-is-zero

osd: use optimized is_zero in object_stat_sum_t.is_zero()

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #7169 from trociny/fix-bluestore-without-aio

osd: bluestore: don't include when building without libaio

Reviewed-by: Sage Weil <sage@redhat.com>

BlueStore: drain wal_wq first if we exit from mounting due to wal_replay error

Because maybe we are in the middle of wal_replay and some wal transaction is already in process.

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

BlueStore: verify fsid in a more proper way during mkfs

Fixes: #14394
Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

tests: update pool alignment API

The old ones are obsolete.

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

tools: fix pool alignment API overflow issue

The original APIs may overflow and are replaced by 1f855456cae96c91a67c169d2333e333c3b59671.
Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

librados: fix rados_ioctx_pool_requires_alignment2()

we can not cast an "int *" pointer to a "bool *" and assign a bool
to it in hope to update the integer pointed by the "int *" with
1 or 0 according to the assigned boolean. because, the existing
value pointed by the "int *" pointer could be any value, if it's
non-zero, say, 0x12345678, after casting to "bool *", and assigning it
to "false", the variable would be 0x12345600. only the least
significant 8 bits are reset. so after resetting the variable pointed
by the "int *" pointer after reset it using "bool *", it still "true"!

Introduced-by: 1f85545
Signed-off-by: Kefu Chai <kchai@redhat.com>

BlueStore: negative return code if we unable to open path

Because the caller is unable to catch a positive error code.

Fixes: #14393
Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

configure: detect new gperftool

this is a follow-up of 6edaf4e, when tcmalloc is enabled,
perfglue/heap_profiler.cc is also compiled, it includes the
google/heap-profiler.h. so to silence the the warnings like
 warning: /usr/include/google/heap-profiler.h:35:2: #warning
 "google/heap-profiler.h is deprecated. Use gperftools/heap-profiler.h
 instead" [-Wcpp]
we also check the new location of these header files if tcmalloc
is enabled.

Signed-off-by: Kefu Chai <kchai@redhat.com>

os/bluestore: fix signed/unsigned comparison warnings

Signed-off-by: Kefu Chai <kchai@redhat.com>

kv/RocksDBStore: replace deprecated calls

rocksdb::DB::CompactRange(Slice*, Slice*) is deprecated, so replace
it with its equivalent.

Signed-off-by: Kefu Chai <kchai@redhat.com>

fsx: fix signed/unsigned comparison warnings

Signed-off-by: Kefu Chai <kchai@redhat.com>

erasure-code: create an internal isa lib for testing

we were linking against a '--module' shared library using its
.la file, which is not portable on some platforms accroding to
autotools. so instead of linking against the plugin so, a dedicated
internal library "libisa.la" is created for testing. and it is added
to the libec_isa.so also.

Signed-off-by: Kefu Chai <kchai@redhat.com>

unittest_ipaddr: fix compile warnings

silence warnings like
 warning: test/test_ipaddr.cc:174:16: deprecated conversion from string
 constant to char* [-Wwrite-strings]

Signed-off-by: Kefu Chai <kchai@redhat.com>

BlueStore: improve wal_replay logic a little bit

Fixes: #14392
Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

kstore: add metadata "type" to disk

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

BlueStore: fix fd leak during error fsck scan

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

BlueStore: update obsolete content of comment

Which does not match the reality anymore.

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

BlueStore: remove reap_cond

It has no consumers at present.

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

kstore: remove reap_cond

It has no consumers at present.

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

MemStore: remove apply_lock

It doesn't block anything other than the umount caller itself.

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

MemStore: drain finisher first during umount

Otherwise some caller specified contexts may be left hanging.

Fixes: #14370
Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

AsyncConnection: Dont send message when local connection is closed

Fix: #14386
Signed-off-by: Haomai Wang <haomai@xsky.com>

AsyncConnection: Fix leak event

Signed-off-by: Haomai Wang <haomai@xsky.com>

common: change the type of counter total/unhealthy_workers

Signed-off-by: Guang Yang <yguang@yahoo-inc.com>

Merge pull request #5900 from ritz303/wip-doc-12899

doc: Updated the rados command man page to include the --run-name opt…

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #7249 from ceph/wip-fix-cmake-trusty

cmake: fix the build on trusty

librbd: handle additional journal replay states

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

librbd: avoid recursive callback when replaying journal

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

tests: correct googlemock warnings

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

tests: update tests to support replaying snap create / resize

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

librbd: initial hook points for snap create / resize journal replay

After snap create has blocked IO and after resize has ensured clipped
IO boundaries, they will resume the playback of journal events.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

librbd: journal events still possible during shutdown

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

tests: new unit test for snap rollback state machine

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

librbd: snapshot rollback should block writes

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

librbd: track simple maintenance ops to coordinate replay

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

librbd: asynchronously replay journal events

Properly track when AIO operations are safely committed to
disk before marking the journal events are committed.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

tests: unit test cases for all available journal events

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

librbd: initial integration of maintenance op journal replay

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

librbd: AioImageRequest base class is now templated

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

tests: updated librbd test cases for updated maint ops API

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

librbd: moved all maintenance ops to new Operations class

This will permit unit testing and will facilitate proper
serialization of requests (when necessary).

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

tests: journal event handling test updates

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

tests: unit test case for librbd::operation::ResizeRequest

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

librbd: properly order maintenance ops journal events

In general, the op shoud only start after the jouranl event is
safely committed.  Snapshot create and resize need special treatment
to record the event while no IO is in-progress.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

librbd: fixed "hidden" virtual methods

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

librbd: handle journal and IO commits in either sequence

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

librbd: allocate and associate op tids to maint op journal events

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

tests: unit tests for librbd Journal

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

librbd: journal replay flush should be async

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

librbd: convert journal classes to templates for unit testing

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

tests: compile RBD and RGW types into ceph-dencoder when enabled

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

librbd: move journal support classes to journal namespace

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

cmake: fix the build on trusty

Signed-off-by: Kefu Chai <kchai@redhat.com>