blkin: fix cmake compile options

Signed-off-by: Sage Weil <sage@redhat.com>

blkin: remove installed files

Signed-off-by: Sage Weil <sage@redhat.com>

blkin: fix -fPIC option

Signed-off-by: Sage Weil <sage@redhat.com>

blkin: drop boost dependency

Signed-off-by: Sage Weil <sage@redhat.com>

common/zipkin_trace: fix blkin_trace_info encoding

- encode the same regardless of whether it is compiled in (!)
- encode in endian- and struct alignment-safe way.

Signed-off-by: Sage Weil <sage@redhat.com>

msg/Message: make {encode,decode}_trace unconditional

These are protocol features and cannot vary based on our compilation.
Encode and decode unconditionally.  The callers have already guarded these
field additions with a message version bump and are conditionally calling
decode_trace.

Signed-off-by: Sage Weil <sage@redhat.com>

CMakeLists.txt: fix libcommon link to blkin

Signed-off-by: Sage Weil <sage@redhat.com>

CMakeLists.txt: default WITH_BLKIN=ON

Signed-off-by: Sage Weil <sage@redhat.com>

cmake: build blkin instead of using find_package

Signed-off-by: Casey Bodley <cbodley@redhat.com>

src/blkin: add blkin submodule

Signed-off-by: Sage Weil <sage@redhat.com>

librados: Overload aio_operate with blkin trace info parameter

This will enable passing trace information from rbd to librados for
aio_reads and aio_writes.

Signed-off-by: Victor Araujo <ve.ar91@gmail.com>

cmake: Add blkin libs to target_link_libraries

Signed-off-by: Victor Araujo <ve.ar91@gmail.com>

blkin: add config variable osdc_blkin_trace_all

Signed-off-by: Casey Bodley <cbodley@redhat.com>

blkin: add traces to librados and Objecter

Signed-off-by: Casey Bodley <cbodley@redhat.com>

blkin: add config variable osd_blkin_trace_all

when set, Message::decode_trace() will always create a trace for
incoming messages, even if they didn't pass trace information

Signed-off-by: Casey Bodley <cbodley@redhat.com>

filestore: Initialize Zipkin traces when there is a store trace

Avoids creating orphaned traces.

Signed-off-by: Victor Araujo <ve.ar91@gmail.com>

blkin: add traces to FileStore/Journal

Signed-off-by: Casey Bodley <cbodley@redhat.com>

blkin: add traces to ECBackend

Signed-off-by: Casey Bodley <cbodley@redhat.com>

blkin: add traces to ReplicatedBackend

Signed-off-by: Casey Bodley <cbodley@redhat.com>

blkin: set up tracing in PGs

Signed-off-by: Casey Bodley <cbodley@redhat.com>

blkin: set up tracing in the OSD

Signed-off-by: Casey Bodley <cbodley@redhat.com>

blkin: osd op messages carry trace information

Signed-off-by: Casey Bodley <cbodley@redhat.com>

blkin: add traces to AsyncMessenger

Signed-off-by: Casey Bodley <cbodley@redhat.com>

blkin: add traces to XioMessenger

Signed-off-by: Casey Bodley <cbodley@redhat.com>

blkin: add traces to SimpleMessenger

Signed-off-by: Casey Bodley <cbodley@redhat.com>

blkin: Messenger integration

Signed-off-by: Casey Bodley <cbodley@redhat.com>

msg: pass Connection to decode_message

Signed-off-by: Casey Bodley <cbodley@redhat.com>

blkin: add header and initialization

zipkin_trace.h is a wrapper around ztracer.hpp, which provides a stub
implementation when WITH_BLKIN is not defined

Signed-off-by: Casey Bodley <cbodley@redhat.com>

cmake: add module and WITH_BLKIN option for blkin

Signed-off-by: Casey Bodley <cbodley@redhat.com>

Build support for BlkKin (LTTng + Zipkin) tracing

 * Adds --with-blkin to autoconf (default without)
 * Adds BLKIN_LIBS to necessary automake files
 * Adds documentation for testing BlkKin tracing

Signed-off-by: Andrew Shewmaker <agshew@gmail.com>
Signed-off-by: Marios-Evaggelos Kogias <marioskogias@gmail.com>
Signed-off-by: Chendi.Xue <chendi.xue@intel.com>
Signed-off-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #14834 from runsisi/wip-fix-double-free

librbd: minor fixes for image trash move

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #14830 from runsisi/wip-open-cleanup

librbd: remove redundant check for image id emptiness

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #14927 from tchaikov/wip-19427

logclient: use the seq id of the 1st log entry when resetting session

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #14920 from tchaikov/wip-14579

mgr: add machinery for python modules to send MCommands to daemons

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #13681 from liewegas/wip-epochs

osd: tag fast dispatch messages with min_epoch

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #14273 from cbodley/wip-rgw-beast-new-parser

rgw: update Beast for streaming reads in asio frontend

Reviewed-by: Radoslaw Zarzynski <rzarzynski@mirantis.com>

rgw: rename frontend from asio to beast

Signed-off-by: Casey Bodley <cbodley@redhat.com>

rgw: update Beast for streaming reads in asio frontend

Signed-off-by: Casey Bodley <cbodley@redhat.com>

rgw: update Beast submodule

Signed-off-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #14931 from tchaikov/wip-19771

qa/tasks/ceph_manager: always fix pgp_num when done with thrashosd task

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #14973 from ifed01/wip-bluestore-kvsync-cond

os/bluestore: remove unused condition variable

Reviewed-by: Sage Weil <sage@redhat.com>

os/bluestore: remove unused condition variable

Signed-off-by: Igor Fedotov <ifedotov@mirantis.com>

Merge pull request #14963 from dillaman/wip-19858

rbd-mirror: failover and failback of unmodified image results in split-brain

Reviewed-by: Mykola Golub <mgolub@mirantis.com>

Merge pull request #14965 from dmick/wip-19865

test/librbd/test_librbd.cc: set *features even if RBD_FEATURES is unset

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #14465 from dillaman/wip-librbd-devstack-tempest

qa/workunits: switch to OpenStack Ocata release for RBD testing

Reviewed-by: Mykola Golub <mgolub@mirantis.com>

test/librbd/test_librbd.cc: set *features even if RBD_FEATURES is unset

If RBD_FEATURES is not in the environment, set *features to 0 in
get_features(); callers rely on a valid return value.  (This was
breaking on arm64.)

Fixes: http://tracker.ceph.com/issues/19865
Signed-off-by: Dan Mick <dan.mick@redhat.com>

librbd: add no-op event when promoting an image

The rbd-mirror process needs an event in the journal
to properly detect the transition between primary and
non-primary state between peers.

Fixes: http://tracker.ceph.com/issues/19858
Signed-off-by: Jason Dillaman <dillaman@redhat.com>

librbd: do not delay propagation of demote events

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

rbd-mirror: prevent infinite loop when computing replay status

If the image had a non-primary predecessor epoch whose tag tid
duplicates an epoch within its own journal, an infinite loop
would result.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #14373 from rzarzynski/wip-rgw-19520

rgw: fix broken /crossdomain.xml, /info and /healthcheck of Swift API.

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>

Merge pull request #9950 from weiqiaomiao/wqm-wip-fetchobj-req

rgw: should delete in_stream_req  if conn->get_obj(...) return not zero value

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #14601 from Jing-Scott/fix-bulkupload-multisite

rgw: fix forward request for bulkupload to be applied in multisite

Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: Radoslaw Zarzynski <rzarzynski@mirantis.com>

Merge pull request #14945 from dillaman/wip-19811

rbd-mirror: ensure missing images are re-synced when detected

Reviewed-by: Mykola Golub <mgolub@mirantis.com>

Merge pull request #14935 from dillaman/wip-rbd-cli-mirror-pool

rbd: removed spurious error message from mirror pool commands

Reviewed-by: Mykola Golub <mgolub@mirantis.com>

Merge pull request #14400 from ceph/wip-cd-1node

qa/tasks: few fixes to get ceph-deploy 1node to working state

Merge pull request #14936 from cbodley/wip-19834

rgw: RGWPeriodPusher spawns http thread before cr thread

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>

Merge pull request #14908 from guihecheng/rgw_file-cleanup

rgw_file: cleanup virtual keyword on derived functions

Merge pull request #14891 from tchaikov/wip-librbd-test-odr

librbd: do not instantiate templates while building tests

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

librbd: do not instantiate templates while building tests

to avoid violation of ODR

Fixes: http://tracker.ceph.com/issues/18938
Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #14844 from liewegas/wip-bluestore-bitalloc

unittest_alloc: add test_alloc_big

Reviewed-by: Igor Fedotov <ifedotov@mirantis.com>

unittest_alloc: add test_alloc_big

Ensure that big allocations work.

Verifies http://tracker.ceph.com/issues/16662

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #14953 from tchaikov/wip-librbd-silence-Wunused-var

librbd: silence -Wunused-variable warning

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #14407 from cbodley/wip-rgw-admin-remote

rgw: fix handling of --remote in radosgw-admin period commands
Reviewed-by: Orit Wasserman <owasserm@redhat.com>

librbd: silence -Wunused-variable warning

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #11179 from rzarzynski/wip-rgw-swift-formpost

rgw: add support for FormPost of Swift API
Reviewed-by: Orit Wasserman <owasserm@redhat.com>

Merge pull request #14949 from zhsj/fix-cryptopp-build

common: fix building against libcryptopp

Reviewed-by: Kefu Chai <kchai@redhat.com>

common: fix building against libcryptopp

libnspr is only needed for libnss, move it to if defined(USE_NSS)
section.

ceph::crypto::shutdown's params should be same with declaration.

Signed-off-by: Shengjing Zhu <zhsj@umcloud.com>

Merge pull request #14923 from joscollin/wip-dmclocktest-function-error

dmclock: include missing <functional> header.

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #14948 from Liuchang0812/wip-kill-duplicated-cmake-commands

cmake: kill duplicated cmake commands

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #14951 from smithfarm/wip-19853

cmake: align cmake names of library packages

Reviewed-by: Kefu Chai <kchai@redhat.com>

Update rgw_rest.cc

fix indentation.
Signed-off-by: Orit Wasserman <owasserm@redhat.com>

cmake: align cmake names of library packages

The cmake package names of libraries are case-sensitive. The name used in
FIND_PACKAGE_HANDLE_STANDARD_ARGS - e.g. "cryptopp" - must match the name used
in the module name - "Findcryptopp.cmake" - as well as the name used in
find_package() call - "find_package(cryptopp REQUIRED)"

Note that the term "package" here refers to a cmake abstraction, *not* an
actual RPM or Debian package.

Fixes: http://tracker.ceph.com/issues/19853
Signed-off-by: Nathan Cutler <ncutler@suse.com>

cmake: kill duplicated cmake commands

Signed-off-by: liuchang0812 <liuchang0812@gmail.com>

Merge pull request #14947 from dmick/wip-arm-compressor

compressor/zlib: fix plugin for non-Intel arches

Reviewed-by: Kefu Chai <kchai@redhat.com>

compressor/zlib: fix plugin for non-Intel arches

unittest_compressor was failing on arm64 because the zlib
compressor was never initialized, even though it works fine in
non-isal mode

Signed-off-by: Dan Mick <dan.mick@redhat.com>

Merge pull request #14363 from majianpeng/finisher-optimize

common/Finisher: batch handle perfcounter && only send signal when waiter existed.

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #14627 from badone/wip-peering-state-timing

osd: Implement peering state timing

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #14709 from majianpeng/luestore-txc-finalize-kv

os/blueStore: In osd_tp_thread, call _txc_finalize_kv.

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Igor Fedotov <ifedotov@mirantis.com>

Merge pull request #14932 from markhpc/wip-rocksdb-compaction-readahead

common/config_opts.h: compaction readahead for bluestore/rocksdb

Reviewed-by: Sage Weil <sage@redhat.com>

test/librbd: remove duplicated explicit template specializations

to avoid violation of ODR

Fixes: http://tracker.ceph.com/issues/18938
Signed-off-by: Kefu Chai <kchai@redhat.com>

rbd-mirror: ensure missing images are re-synced when detected

Fixes: http://tracker.ceph.com/issues/19811
Signed-off-by: Jason Dillaman <dillaman@redhat.com>

default vpm is trusty and for master we need latest which is xenial

Signed-off-by: Vasu Kulkarni <vasu@redhat.com>

install mgr on the node

Signed-off-by: Vasu Kulkarni <vasu@redhat.com>

Merge pull request #14879 from wjwithagen/wip-wjw-reduce-noise-3

cls/log/cls_log.cc: reduce logging noise

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #14928 from drunkard/master

doc: remove deprecated subcommand in man/8/ceph.rst

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #14723 from neha-ojha/objclass-sdk

osd: objclass sdk

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #14917 from yuriw/wip-add-overrite-cls

qa/added overrides

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

cls/log/cls_log.cc: reduce logging noise

 - The other reference in the source as already at 20.
      ./src/cls/timeindex/cls_timeindex.cc:85:
CLS_LOG(20, "storing entry at %s", index.c_str());

   And we need not always know where in the log items are stored.
   So it looks like a leftover debug feature.

Fixes: http://tracker.ceph.com/issues/19835
Signed-off-by: Willem Jan Withagen <wjw@digiware.nl>

Merge pull request #14913 from liewegas/wip-19826

mon: do not prime_pg_temp creating pgs; clean up pg create conditions

Reviewed-by: Kefu Chai <kchai@redhat.com>

common/Finisher: No waite_for_empty no signal.

Avoid every time to send a signal because wait_for_empty is rare event.

Signed-off-by: Jianpeng Ma <jianpeng.ma@intel.com>

rgw: RGWPeriodPusher spawns http thread before cr thread

Fixes: http://tracker.ceph.com/issues/19834
Signed-off-by: Casey Bodley <cbodley@redhat.com>

radosgw-admin: 'period pull' reads endpoints from current period

by finding remote endpoints from the current period, 'period pull' no
longer needs to initialize RGWRados for zone[group]_conn_map

Fixes: http://tracker.ceph.com/issues/19554
Signed-off-by: Casey Bodley <cbodley@redhat.com>

radosgw-admin: commit_period gets remote connection from new period

Fixes: http://tracker.ceph.com/issues/19554
Signed-off-by: Casey Bodley <cbodley@redhat.com>

radosgw-admin: send_to_remote_gateway takes RGWRESTConn*

callers are responsible for providing their own remote connection

Signed-off-by: Casey Bodley <cbodley@redhat.com>

radosgw-admin: get remote connections from given period/zonegroup

helper functions to create connections to arbitrary remotes, rather
than depending on RGWRados to construct them in zone[group]_conn_map

Signed-off-by: Casey Bodley <cbodley@redhat.com>

rgw: enable move on RGWRESTConn

added custom move construct/assign because std::atomic is not movable

Signed-off-by: Casey Bodley <cbodley@redhat.com>

rbd: removed spurious error message from mirror pool commands

If the pool was empty, the rbd CLI would report the error
"rbd: failed to list images within pool".

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

common/Finisher: batch handle perfcounter.

Signed-off-by: Jianpeng Ma <jianpeng.ma@intel.com>

common/perf_counter: make tinc() handle batch-processing

Signed-off-by: Jianpeng Ma <jianpeng.ma@intel.com>

common/config_opts.h: compaction readahead for bluestore/rocksdb

Signed-off-by: Mark Nelson <mnelson@redhat.com>