blkin: add header and initialization

zipkin_trace.h is a wrapper around ztracer.hpp, which provides a stub
implementation when WITH_BLKIN is not defined

Signed-off-by: Casey Bodley <cbodley@redhat.com>

cmake: add module and WITH_BLKIN option for blkin

Signed-off-by: Casey Bodley <cbodley@redhat.com>

Build support for BlkKin (LTTng + Zipkin) tracing

 * Adds --with-blkin to autoconf (default without)
 * Adds BLKIN_LIBS to necessary automake files
 * Adds documentation for testing BlkKin tracing

Signed-off-by: Andrew Shewmaker <agshew@gmail.com>
Signed-off-by: Marios-Evaggelos Kogias <marioskogias@gmail.com>
Signed-off-by: Chendi.Xue <chendi.xue@intel.com>
Signed-off-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #14834 from runsisi/wip-fix-double-free

librbd: minor fixes for image trash move

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #14830 from runsisi/wip-open-cleanup

librbd: remove redundant check for image id emptiness

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #14927 from tchaikov/wip-19427

logclient: use the seq id of the 1st log entry when resetting session

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #14920 from tchaikov/wip-14579

mgr: add machinery for python modules to send MCommands to daemons

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #13681 from liewegas/wip-epochs

osd: tag fast dispatch messages with min_epoch

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #14273 from cbodley/wip-rgw-beast-new-parser

rgw: update Beast for streaming reads in asio frontend

Reviewed-by: Radoslaw Zarzynski <rzarzynski@mirantis.com>

rgw: rename frontend from asio to beast

Signed-off-by: Casey Bodley <cbodley@redhat.com>

rgw: update Beast for streaming reads in asio frontend

Signed-off-by: Casey Bodley <cbodley@redhat.com>

rgw: update Beast submodule

Signed-off-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #14931 from tchaikov/wip-19771

qa/tasks/ceph_manager: always fix pgp_num when done with thrashosd task

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #14973 from ifed01/wip-bluestore-kvsync-cond

os/bluestore: remove unused condition variable

Reviewed-by: Sage Weil <sage@redhat.com>

os/bluestore: remove unused condition variable

Signed-off-by: Igor Fedotov <ifedotov@mirantis.com>

Merge pull request #14963 from dillaman/wip-19858

rbd-mirror: failover and failback of unmodified image results in split-brain

Reviewed-by: Mykola Golub <mgolub@mirantis.com>

Merge pull request #14965 from dmick/wip-19865

test/librbd/test_librbd.cc: set *features even if RBD_FEATURES is unset

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #14465 from dillaman/wip-librbd-devstack-tempest

qa/workunits: switch to OpenStack Ocata release for RBD testing

Reviewed-by: Mykola Golub <mgolub@mirantis.com>

test/librbd/test_librbd.cc: set *features even if RBD_FEATURES is unset

If RBD_FEATURES is not in the environment, set *features to 0 in
get_features(); callers rely on a valid return value.  (This was
breaking on arm64.)

Fixes: http://tracker.ceph.com/issues/19865
Signed-off-by: Dan Mick <dan.mick@redhat.com>

librbd: add no-op event when promoting an image

The rbd-mirror process needs an event in the journal
to properly detect the transition between primary and
non-primary state between peers.

Fixes: http://tracker.ceph.com/issues/19858
Signed-off-by: Jason Dillaman <dillaman@redhat.com>

librbd: do not delay propagation of demote events

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

rbd-mirror: prevent infinite loop when computing replay status

If the image had a non-primary predecessor epoch whose tag tid
duplicates an epoch within its own journal, an infinite loop
would result.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #14373 from rzarzynski/wip-rgw-19520

rgw: fix broken /crossdomain.xml, /info and /healthcheck of Swift API.

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>

Merge pull request #9950 from weiqiaomiao/wqm-wip-fetchobj-req

rgw: should delete in_stream_req  if conn->get_obj(...) return not zero value

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #14601 from Jing-Scott/fix-bulkupload-multisite

rgw: fix forward request for bulkupload to be applied in multisite

Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: Radoslaw Zarzynski <rzarzynski@mirantis.com>

Merge pull request #14945 from dillaman/wip-19811

rbd-mirror: ensure missing images are re-synced when detected

Reviewed-by: Mykola Golub <mgolub@mirantis.com>

Merge pull request #14935 from dillaman/wip-rbd-cli-mirror-pool

rbd: removed spurious error message from mirror pool commands

Reviewed-by: Mykola Golub <mgolub@mirantis.com>

Merge pull request #14400 from ceph/wip-cd-1node

qa/tasks: few fixes to get ceph-deploy 1node to working state

Merge pull request #14936 from cbodley/wip-19834

rgw: RGWPeriodPusher spawns http thread before cr thread

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>

Merge pull request #14908 from guihecheng/rgw_file-cleanup

rgw_file: cleanup virtual keyword on derived functions

Merge pull request #14891 from tchaikov/wip-librbd-test-odr

librbd: do not instantiate templates while building tests

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

librbd: do not instantiate templates while building tests

to avoid violation of ODR

Fixes: http://tracker.ceph.com/issues/18938
Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #14844 from liewegas/wip-bluestore-bitalloc

unittest_alloc: add test_alloc_big

Reviewed-by: Igor Fedotov <ifedotov@mirantis.com>

unittest_alloc: add test_alloc_big

Ensure that big allocations work.

Verifies http://tracker.ceph.com/issues/16662

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #14953 from tchaikov/wip-librbd-silence-Wunused-var

librbd: silence -Wunused-variable warning

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #14407 from cbodley/wip-rgw-admin-remote

rgw: fix handling of --remote in radosgw-admin period commands
Reviewed-by: Orit Wasserman <owasserm@redhat.com>

librbd: silence -Wunused-variable warning

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #11179 from rzarzynski/wip-rgw-swift-formpost

rgw: add support for FormPost of Swift API
Reviewed-by: Orit Wasserman <owasserm@redhat.com>

Merge pull request #14949 from zhsj/fix-cryptopp-build

common: fix building against libcryptopp

Reviewed-by: Kefu Chai <kchai@redhat.com>

common: fix building against libcryptopp

libnspr is only needed for libnss, move it to if defined(USE_NSS)
section.

ceph::crypto::shutdown's params should be same with declaration.

Signed-off-by: Shengjing Zhu <zhsj@umcloud.com>

Merge pull request #14923 from joscollin/wip-dmclocktest-function-error

dmclock: include missing <functional> header.

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #14948 from Liuchang0812/wip-kill-duplicated-cmake-commands

cmake: kill duplicated cmake commands

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #14951 from smithfarm/wip-19853

cmake: align cmake names of library packages

Reviewed-by: Kefu Chai <kchai@redhat.com>

Update rgw_rest.cc

fix indentation.
Signed-off-by: Orit Wasserman <owasserm@redhat.com>

cmake: align cmake names of library packages

The cmake package names of libraries are case-sensitive. The name used in
FIND_PACKAGE_HANDLE_STANDARD_ARGS - e.g. "cryptopp" - must match the name used
in the module name - "Findcryptopp.cmake" - as well as the name used in
find_package() call - "find_package(cryptopp REQUIRED)"

Note that the term "package" here refers to a cmake abstraction, *not* an
actual RPM or Debian package.

Fixes: http://tracker.ceph.com/issues/19853
Signed-off-by: Nathan Cutler <ncutler@suse.com>

cmake: kill duplicated cmake commands

Signed-off-by: liuchang0812 <liuchang0812@gmail.com>

Merge pull request #14947 from dmick/wip-arm-compressor

compressor/zlib: fix plugin for non-Intel arches

Reviewed-by: Kefu Chai <kchai@redhat.com>

compressor/zlib: fix plugin for non-Intel arches

unittest_compressor was failing on arm64 because the zlib
compressor was never initialized, even though it works fine in
non-isal mode

Signed-off-by: Dan Mick <dan.mick@redhat.com>

Merge pull request #14363 from majianpeng/finisher-optimize

common/Finisher: batch handle perfcounter && only send signal when waiter existed.

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #14627 from badone/wip-peering-state-timing

osd: Implement peering state timing

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #14709 from majianpeng/luestore-txc-finalize-kv

os/blueStore: In osd_tp_thread, call _txc_finalize_kv.

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Igor Fedotov <ifedotov@mirantis.com>

Merge pull request #14932 from markhpc/wip-rocksdb-compaction-readahead

common/config_opts.h: compaction readahead for bluestore/rocksdb

Reviewed-by: Sage Weil <sage@redhat.com>

test/librbd: remove duplicated explicit template specializations

to avoid violation of ODR

Fixes: http://tracker.ceph.com/issues/18938
Signed-off-by: Kefu Chai <kchai@redhat.com>

rbd-mirror: ensure missing images are re-synced when detected

Fixes: http://tracker.ceph.com/issues/19811
Signed-off-by: Jason Dillaman <dillaman@redhat.com>

default vpm is trusty and for master we need latest which is xenial

Signed-off-by: Vasu Kulkarni <vasu@redhat.com>

install mgr on the node

Signed-off-by: Vasu Kulkarni <vasu@redhat.com>

Merge pull request #14879 from wjwithagen/wip-wjw-reduce-noise-3

cls/log/cls_log.cc: reduce logging noise

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #14928 from drunkard/master

doc: remove deprecated subcommand in man/8/ceph.rst

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #14723 from neha-ojha/objclass-sdk

osd: objclass sdk

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #14917 from yuriw/wip-add-overrite-cls

qa/added overrides

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

cls/log/cls_log.cc: reduce logging noise

 - The other reference in the source as already at 20.
      ./src/cls/timeindex/cls_timeindex.cc:85:
CLS_LOG(20, "storing entry at %s", index.c_str());

   And we need not always know where in the log items are stored.
   So it looks like a leftover debug feature.

Fixes: http://tracker.ceph.com/issues/19835
Signed-off-by: Willem Jan Withagen <wjw@digiware.nl>

Merge pull request #14913 from liewegas/wip-19826

mon: do not prime_pg_temp creating pgs; clean up pg create conditions

Reviewed-by: Kefu Chai <kchai@redhat.com>

common/Finisher: No waite_for_empty no signal.

Avoid every time to send a signal because wait_for_empty is rare event.

Signed-off-by: Jianpeng Ma <jianpeng.ma@intel.com>

rgw: RGWPeriodPusher spawns http thread before cr thread

Fixes: http://tracker.ceph.com/issues/19834
Signed-off-by: Casey Bodley <cbodley@redhat.com>

radosgw-admin: 'period pull' reads endpoints from current period

by finding remote endpoints from the current period, 'period pull' no
longer needs to initialize RGWRados for zone[group]_conn_map

Fixes: http://tracker.ceph.com/issues/19554
Signed-off-by: Casey Bodley <cbodley@redhat.com>

radosgw-admin: commit_period gets remote connection from new period

Fixes: http://tracker.ceph.com/issues/19554
Signed-off-by: Casey Bodley <cbodley@redhat.com>

radosgw-admin: send_to_remote_gateway takes RGWRESTConn*

callers are responsible for providing their own remote connection

Signed-off-by: Casey Bodley <cbodley@redhat.com>

radosgw-admin: get remote connections from given period/zonegroup

helper functions to create connections to arbitrary remotes, rather
than depending on RGWRados to construct them in zone[group]_conn_map

Signed-off-by: Casey Bodley <cbodley@redhat.com>

rgw: enable move on RGWRESTConn

added custom move construct/assign because std::atomic is not movable

Signed-off-by: Casey Bodley <cbodley@redhat.com>

rbd: removed spurious error message from mirror pool commands

If the pool was empty, the rbd CLI would report the error
"rbd: failed to list images within pool".

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

common/Finisher: batch handle perfcounter.

Signed-off-by: Jianpeng Ma <jianpeng.ma@intel.com>

common/perf_counter: make tinc() handle batch-processing

Signed-off-by: Jianpeng Ma <jianpeng.ma@intel.com>

common/config_opts.h: compaction readahead for bluestore/rocksdb

Signed-off-by: Mark Nelson <mnelson@redhat.com>

Merge pull request #14903 from dillaman/wip-19413

cls_rbd: default initialize snapshot namespace for legacy clients

Reviewed-by: Mykola Golub <mgolub@mirantis.com>

Merge pull request #14875 from dillaman/wip-rbd-trash-parent

rbd: info command should indicate if parent is in trash

Reviewed-by: Mykola Golub <mgolub@mirantis.com>

Merge pull request #14929 from tchaikov/wip-disable-isal-on-arm

test/compressor: disable isal tests if not available

Reviewed-by: Sage Weil <sage@redhat.com>

qa/tasks/ceph_manager: always fix pgp_num when done with thrashosd task

Fixes: http://tracker.ceph.com/issues/19771
Signed-off-by: Kefu Chai <kchai@redhat.com>

logclient: use the seq id of the 1st log entry when resetting session

because do_log() does not update last_log (see LogClient::get_next_seq())
and log_queue atomically, and log_queue is updated *after* last_log, so
we can not use them to figure out the number of log entries to be sent.
so, we choose to trust log_queue in this change.

Fixes: http://tracker.ceph.com/issues/19427
Signed-off-by: Kefu Chai <kchai@redhat.com>

test/compressor: disable isal tests if not available

the isal-zlib compressor plugin is not supported on non-intel archs, and
if the user does not have a capable yasm compiler, neither will he/she
get this plugin.

Signed-off-by: Kefu Chai <kchai@redhat.com>

doc: cleanup in doc/man/8/ceph.rst

Subcommand doc has been removed in 904913b37f241a73aa6722a18151d3166c8c93dd,
this is cleanup.

Signed-off-by: Drunkard Zhang <gongfan193@gmail.com>

systemd: update mgr auth caps

Granting it 'allow *' on mon and osd so that
it can use MCommand to remote control daemons.

Signed-off-by: John Spray <john.spray@redhat.com>

mgr: embed Client instance for mds_command

Now plugins can talk to MDS daemons.

Signed-off-by: John Spray <john.spray@redhat.com>

client: enable using external Objecter

To enable places that already have an objecter
(such as ceph-mgr) to avoid spinning up another
one inside Client.

This will also be a path to re-using the librados
objecter in multiple instances of libcephfs in the
same process, a la nfs-ganesha.

Signed-off-by: John Spray <john.spray@redhat.com>

dmclock: ‘std::function’ has not been declared

The following error appears during make:

ceph/src/dmclock/test/test_test_client.cc:14:0:
ceph/src/dmclock/test/../sim/src/sim_recs.h:40:12: error: ‘std::function’ has not been declared
       std::function<void()> code) {

Signed-off-by: Jos Collin <jcollin@redhat.com>

Merge pull request #14822 from jan--f/fuse-normalize-wire-o_flags

fs/ceph-fuse: normalize file open flags on the wire

Merge pull request #14909 from joscollin/wip-dmclock-error

dmclock: error: ‘function’ in namespace ‘std’ does not name a template type

Reviewed-by: J. Eric Ivancich <ivancich@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #14748 from smithfarm/wip-teuthology-worker

doc: mention teuthology-worker security group

Reviewed-by: Loic Dachary <ldachary@redhat.com>

pybind/rados: enable construction with CephContext

The CephContext pointer is passed around inside
a Python capsule.  It would have initially been
created by some other (non-cython) bit of C++
code, such as ceph-mgr.

This enables mgr modules to instantiate librados
with all the same config/auth/id as the mgr daemon
that they are running within.  Useful if you
want to spin up e.g. a librbd instance.

Signed-off-by: John Spray <john.spray@redhat.com>

mgr: add get_context() to python interface

For modules that would like a CephContext* inside
a capsule, suitable for passing to librados constructor.

Signed-off-by: John Spray <john.spray@redhat.com>

vstart: give mgr full mds/osd caps for `tell`

Signed-off-by: John Spray <john.spray@redhat.com>

mgr: expose `tell` (MMonCommand) functionality

To enable plugins to talk directly to daemons when they
want to.

Signed-off-by: John Spray <john.spray@redhat.com>

mgr: expose Objecter to PyModules

Same motivation as exposing MonClient: so that modules
can get at the command sending code.

Signed-off-by: John Spray <john.spray@redhat.com>

mon/OSDMonitor: skip new pgs for prime_pg_temp

If the pg didn't previously exist we don't need to (and can't) prime
pg_temp for it.  This avoids crashing on mapping.get().

Signed-off-by: Sage Weil <sage@redhat.com>

qa/added overrides

Signed-off-by: Yuri Weinstein <yweinste@redhat.com>

qa/workunits/rbd: add additional OpenStack services

The Ocata release requires additional services to be running.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #14412 from mikulely/cleanup-civetweb-port

rgw: drop unused port var

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #13451 from xiexingguo/wip-add-up-primary-sum

mon/PGMap: add up_primary pg number field for pg-dump cmd

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #14497 from tchaikov/wip-pybind-better-error-msg

pybind: better error msg

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #14687 from jcsp/wip-mgr-suite

qa/suites: move mgr tests into rados suite

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #14810 from liewegas/wip-status

mon: show inactive % in ceph status

Reviewed-by: Kefu Chai <kchai@redhat.com>