qa/workunits/rbd: disable deep-flatten during permissions testing

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

rbd: snap list should open image as read-only

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #8116 from SUSE/wip-docdev-nuke

doc/dev: add section on interrupting a running suite

Reviewed-by: Loic Dachary <ldachary@redhat.com>

doc/dev: add section on interrupting a running suite

This commit adds a new section, entitled "Interrupt a running suite", to
the "Testing in the cloud" chapter.

Signed-off-by: Nathan Cutler <ncutler@suse.com>

Merge pull request #8105 from SUSE/wip-ceph-detect-init-tox

ceph-detect-init: add missing test case

Merge pull request #8088 from liewegas/wip-typo

osd/OSDMap: fix typo in summarize_mapping_stats

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #8098 from liewegas/wip-fix-crush

unittest_osdmap: default crush tunables now firefly

Merge pull request #8089 from liewegas/wip-mark-pg-stale

mon/PGMonitor: reliably mark PGs state

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #7943 from llua/init-ceph

init-ceph.in: allow case-insensitive true in `osd crush update on start'

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Piotr Dałek <piotr.dalek@ts.fujitsu.com>

Merge pull request #8055 from liewegas/wip-hobject

ghobject_t: use # instead of ! as a separator

Reviewed-by: Piotr Dałek <piotr.dalek@ts.fujitsu.com>

Merge pull request #8056 from rjfd/wip-13296

rbd: support for enabling/disabling mirroring on specific images

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #7971 from ceph/wip-selinux-update

selinux: Update policy to grant additional access

Merge pull request #8033 from liewegas/wip-pg-create

osd: handle dup pg_create that races with pg deletion

Reviewed-by: Samuel Just <sjust@redhat.com>

Merge pull request #8054 from ErwanAliasr1/evelu-check

common: SubProcess: Avoid buffer corruption when calling err()

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #8057 from liewegas/wip-pglog-warning

osd/PGLog: fix warning

Reviewed-by: Samuel Just <sjust@redhat.com>

Merge pull request #8059 from ceph/wip-include-scope

rados: add ceph:: namespace to bufferlist type

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #8071 from xiexingguo/xxg-wip-fixptr

osd: initialize last_recalibrate field at construction

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #8081 from renhwsky/renhw-wip-osdmonitor-poolset

osd: min_write_recency_for_promote & min_read_recency_for_promote are tiering only

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #7879 from xiexingguo/xxg-wip-doc-rados-api

doc/rados/api/librados-intro.rst: fix typo

Reviewed-by: Kefu Chai <tchaikov@gmail.com>

doc: fix typo

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

Merge pull request #8079 from dillaman/wip-14413

rbd-mirror: integrate with image sync state machine

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

test: remove broken negative test cases

These should be reimplemented using google mock instead of
virtual functions.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

rbd-mirror: integrate new bootstrap state machine

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

rbd-mirror: simple image bootstrap state machine

This state machine will handle the simple cases of creating
a local image from a remote image, registering to the remote
journal, and sync'ing the image contents.

There are several TODOs for future remaining work.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

rbd-mirror: helper state machine for opening local image

This state machine will open a local image and request the
exclusive lock.  If the open or lock fails, it will automatically
close the image.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

rbd-mirror: helper state machine for closing librbd images

librbd ImageCtx's cannot be deleted from within the librbd thread.  This
state machine will perform the deletion from the rbd-mirror work queue
thread.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #7979 from dillaman/wip-14413

rbd-mirror: remote to local cluster image sync

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #8102 from dillaman/wip-librados-aio-notify-redux

librados: cancel aio notification linger op upon completion

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #8099 from dillaman/wip-15121

cls_rbd: protect against excessively large object maps

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

ceph-detect-init: add test for squeeze

Since we return 'sysvinit' for wheezy and squeeze only, make sure we are
testing both of these.

Signed-off-by: Nathan Cutler <ncutler@suse.com>

Merge pull request #8100 from linuxbox2/ldap-cmake

cmake: avoid false-positive LDAP header detect

Reviewed-by: Orit Wasserman <owasserm@redhat.com>

Merge pull request #8104 from dillaman/wip-rbd-read-flags

qa/workunits/rbd: do not use object map during read flag testing

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

qa/workunits/rbd: do not use object map during read flag testing

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

librados: cancel aio notification linger op upon completion

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

test: cmake: Fix make check compilation error

Signed-off-by: Ricardo Dias <rdias@suse.com>

rbd-mirroring: test: Added mirror image enable/disable to RBD CLI syntax tests

Signed-off-by: Ricardo Dias <rdias@suse.com>

cmake: avoid false-positive LDAP header detect

Pre-setting a value in OPENLDAP_INCLUDE_DIR broke the subsequent
test.

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>

cls_rbd: protect against excessively large object maps

Fixes: #15121
Signed-off-by: Jason Dillaman <dillaman@redhat.com>

rbd: rbd-mirroring: Added mirroing info to image info command in RBD CLI

Signed-off-by: Ricardo Dias <rdias@suse.com>
Fixes: #13296

rbd: rbd-mirroring: Support image mirroring state retrieval in librbd

Signed-off-by: Ricardo Dias <rdias@suse.com>

rbd: rbd-mirroring: Added mirror image enable/disable commands to RBD CLI

Signed-off-by: Ricardo Dias <rdias@suse.com>

rbd: rbd-mirroring: Support mirroring image disable in librbd

Signed-off-by: Ricardo Dias <rdias@suse.com>

rbd: rbd-mirroring: Support mirroring image enable in librbd

Signed-off-by: Ricardo Dias <rdias@suse.com>

Merge pull request #8094 from dillaman/wip-librbd-image-watch-unregister

librbd: handle unregistering the image watcher when disconnected

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

ceph_test_rados_api_tmap_migrate: fix warning

In file included from ./test/librados/TestCase.h:9:0,
                 from test/librados/tmap_migrate.cc:4:
../src/gmock/gtest/include/gtest/gtest.h: In instantiation of ‘testing::Asserti$
../src/gmock/gtest/include/gtest/gtest.h:1484:23:   required from ‘static testi$
test/librados/tmap_migrate.cc:56:3:   required from here
../src/gmock/gtest/include/gtest/gtest.h:1448:16: warning: comparison between s$
   if (expected == actual) {
                ^

Signed-off-by: Sage Weil <sage@redhat.com>

ceph_test_rados_api_list: fix warning

In file included from ./test/librados/TestCase.h:9:0,
                 from test/librados/list.cc:7:
../src/gmock/gtest/include/gtest/gtest.h: In instantiation of ‘testing::Asserti$
../src/gmock/gtest/include/gtest/gtest.h:1484:23:   required from ‘static testi$
test/librados/list.cc:944:5:   required from here
../src/gmock/gtest/include/gtest/gtest.h:1448:16: warning: comparison between s$
   if (expected == actual) {
                ^

Signed-off-by: Sage Weil <sage@redhat.com>

unittest_osdmap: default crush tunables now firefly

This should have been changed in
000344ef0ca639d515e3f89401bafc9b0b13deda.

Signed-off-by: Sage Weil <sage@redhat.com>

log: do not pass negative fd to fchown

>>>     CID 1355574:  Error handling issues  (NEGATIVE_RETURNS)
>>>     "this->m_fd" is passed to a parameter that cannot be negative.

Signed-off-by: Sage Weil <sage@redhat.com>

librbd: handle unregistering the image watcher when disconnected

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

mon/PGMonitor: make pg refresh debug output more informative

Signed-off-by: Sage Weil <sage@redhat.com>

mon/PGMap: drop unused nocreating flag for stat_pg_{add,sub}

Signed-off-by: Sage Weil <sage@redhat.com>

osd: drop create events based on same_primary_since, not interval

Creates are resent when the primary changes, not the interval.
Ensure we don't drop something we shouldn't.

Signed-off-by: Sage Weil <sage@redhat.com>

osd: drop unused primary arg to handle_pg_peering_evt

Signed-off-by: Sage Weil <sage@redhat.com>

osd: use handle_pg_peering_evt for pg creations

This ensures we captures all of the PG resurrection logic.

Signed-off-by: Sage Weil <sage@redhat.com>

mon/PGMonitor: reliably mark PGs state

In beb99d708a5051405a32f46fc10cef473bfddce3 we partially addressed
the case where a acting_primary update raced with an attemp to
mark the pg state.  However, if the acting_primary changed, we
didn't check the new acting_primary to ensure that it is also
down and we should still mark things stale.

Also, printing out the acting_primary aids debugging.

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #7913 from liewegas/wip-list-encode

include/encoding: do not try to be clever with list encoding

Merge pull request #7915 from ceph/wip-librados-stat2

librados: stat2 with higher time precision

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #7964 from liewegas/wip-crush-jewel

crush: update tunable docs.  change default profile to jewel

Reviewed-by: Samuel Just <sjust@redhat.com>

Merge pull request #8035 from liewegas/wip-bluestore

osd: bluestore updates, scrub fixes

Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #8050 from liewegas/wip-15077

os/filestore: fix punch hole usage in _zero

Reviewed-by: Samuel Just <sjust@redhat.com>

Merge pull request #8064 from liewegas/wip-zlib-test

unittest_compression_zlib: do not assume buffer will be null terminated

Merge pull request #7870 from stiopaa1/blueStoreRemoveIncludes

osd: bluestore: remove unneeded includes

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #7871 from xiexingguo/xxg-wip-cleanasync

msg: async: small cleanups

Reviewed-by: Haomai Wang <haomai@xsky.com>

osd/OSDMap: fix typo in summarize_mapping_stats

From ea9abe53d0e777b7dc3b22af71639f77c4de08c8

Signed-off-by: Sage Weil <sage@redhat.com>

SubProcess: Avoid buffer corruption when calling err()

Some code like crush/CrushTester.cc, uses err() to get the output status of a
spawned command.

As per reported in bug #15011, some tests were failing because of some utf8
decoding issues. The implied buffers were generated by the output of err()
function of SubProcess.

err() is returning the str().c_str() value of 'errstr' object which is not
living much longer that the function itself.

This patch returns a std::string to avoid the temporary allocation and fixes
the associated err() calls accordingly.

Since this commit, the make check is now passing everytime.

Fixes: #15011
Signed-off-by: Erwan Velu <erwan@redhat.com>

Merge pull request #7170 from trociny/wip-freebsd-misc

FreeBSD related fixes

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #7947 from saru95/patch---2

ceph.in: Minor python3 specific changes

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #7974 from tchaikov/wip-buffer-eq-neq

buffer: add symmetry operator==() and operator!=()

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

osd: min_write_recency_for_promote & min_read_recency_for_promote are only used in cache pool

min_write_recency_for_promote & min_read_recency_for_promote are only used in cache pool

Signed-off-by: huanwen ren <ren.huanwen@zte.com.cn>

Merge pull request #8074 from tchaikov/wip-ceph-detect-init-test

ceph-detect-init: add debian/jessie test

Reviewed-by: Loic Dachary <ldachary@redhat.com>

ceph-detect-init: return None at seeing unknown debian distro

* return None at seeing unknown debian distros
* add test for this accordingly

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #8046 from xiexingguo/xxg-wip-fixpc

msg: async: fix perf counter description and simplify _send_keepalive_or_ack

Reviewed-by: Haomai Wang <haomai@xsky.com>

Merge pull request #7949 from wjin/fix

Event: fix clock skew problem

Reviewed-by: Haomai Wang <haomai@xsky.com>

Merge pull request #8076 from ceph/wip-rgw-ldap

Wip rgw ldap

This follows a revert of the previous PR. This branch has some build related fixes.

Merge pull request #8075 from ceph/revert-7985-rgw-token-7

Revert "rgw ldap"

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

Revert "rgw ldap"

test: verify proper update of object map during rbd-mirror image sync

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

rbd-mirror: update object map while performing image sync

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

rbd-mirror: temporarily disable support for cloned images

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

test: unit test cases for rbd mirror image sync

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

rbd-mirror: image sync state machine

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

test: unit tests for rbd::mirror::image_sync::ImageCopyRequest

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

rbd-mirror: state machine to iterate and copy all image objects

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

librbd: track maximum object count during mirror sync

If the sync is interrupted, it is possible the image the source
image is shrunk.  Therefore, the sync process needs to track
the maximum known object extent of the image so objects can
be properly freed.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

test: test cases for rbd-mirror image sync point state machines

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

rbd-mirror: new image sync point create/prune state machines

These are responsible for creating/removing snapshots in the
remote image and updating the client registration to account
for the snapshots.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

librbd: added equality operator to MirrorPeer journal types

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

librbd: async version of snap create and snap remove

This will be utilized by the RBD mirror daemon when creating
sync points.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

librbd: prefix local maintenance ops methods

This will help to differentiate between the methods that send an RPC
request if the lock isn't owned vs execute locally if it is owned.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

librbd: maintenance ops now send asynchronous requests

The RBD mirror daemon will need to request the creation/removal
of snapshots from a remote image.  These operations need to be
performed asynchronously as opposed to the current synchronous
style of the librbd API.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

librbd: track starting point snapshot for mirror sync point

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

test: move get_mock_io_ctx helper method to common location

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

test: unit test cases for rbd::mirror::image_sync::SnapshotCopyRequest

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

rbd-mirror: sync snapshots between remote-local images during bootstrap

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

librbd: track local to peer snapshot id mapping within journal

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

test: unit test cases for rbd::mirror::image_sync::ObjectCopyRequest

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

test: updated mock expectation for librados remove op

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

test: added missing forward declarations for librbd mocks

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

rbd_mirror: deep-copy of object and its associated snapshots

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

librbd: helper method for wrappign rados safe callbacks

Signed-off-by: Jason Dillaman <dillaman@redhat.com>