Merge pull request #13681 from liewegas/wip-epochs

osd: tag fast dispatch messages with min_epoch

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #14273 from cbodley/wip-rgw-beast-new-parser

rgw: update Beast for streaming reads in asio frontend

Reviewed-by: Radoslaw Zarzynski <rzarzynski@mirantis.com>

rgw: rename frontend from asio to beast

Signed-off-by: Casey Bodley <cbodley@redhat.com>

rgw: update Beast for streaming reads in asio frontend

Signed-off-by: Casey Bodley <cbodley@redhat.com>

rgw: update Beast submodule

Signed-off-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #14931 from tchaikov/wip-19771

qa/tasks/ceph_manager: always fix pgp_num when done with thrashosd task

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #14973 from ifed01/wip-bluestore-kvsync-cond

os/bluestore: remove unused condition variable

Reviewed-by: Sage Weil <sage@redhat.com>

os/bluestore: remove unused condition variable

Signed-off-by: Igor Fedotov <ifedotov@mirantis.com>

Merge pull request #14963 from dillaman/wip-19858

rbd-mirror: failover and failback of unmodified image results in split-brain

Reviewed-by: Mykola Golub <mgolub@mirantis.com>

Merge pull request #14965 from dmick/wip-19865

test/librbd/test_librbd.cc: set *features even if RBD_FEATURES is unset

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #14465 from dillaman/wip-librbd-devstack-tempest

qa/workunits: switch to OpenStack Ocata release for RBD testing

Reviewed-by: Mykola Golub <mgolub@mirantis.com>

test/librbd/test_librbd.cc: set *features even if RBD_FEATURES is unset

If RBD_FEATURES is not in the environment, set *features to 0 in
get_features(); callers rely on a valid return value.  (This was
breaking on arm64.)

Fixes: http://tracker.ceph.com/issues/19865
Signed-off-by: Dan Mick <dan.mick@redhat.com>

librbd: add no-op event when promoting an image

The rbd-mirror process needs an event in the journal
to properly detect the transition between primary and
non-primary state between peers.

Fixes: http://tracker.ceph.com/issues/19858
Signed-off-by: Jason Dillaman <dillaman@redhat.com>

librbd: do not delay propagation of demote events

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

rbd-mirror: prevent infinite loop when computing replay status

If the image had a non-primary predecessor epoch whose tag tid
duplicates an epoch within its own journal, an infinite loop
would result.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #14373 from rzarzynski/wip-rgw-19520

rgw: fix broken /crossdomain.xml, /info and /healthcheck of Swift API.

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>

Merge pull request #9950 from weiqiaomiao/wqm-wip-fetchobj-req

rgw: should delete in_stream_req  if conn->get_obj(...) return not zero value

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #14601 from Jing-Scott/fix-bulkupload-multisite

rgw: fix forward request for bulkupload to be applied in multisite

Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: Radoslaw Zarzynski <rzarzynski@mirantis.com>

Merge pull request #14945 from dillaman/wip-19811

rbd-mirror: ensure missing images are re-synced when detected

Reviewed-by: Mykola Golub <mgolub@mirantis.com>

Merge pull request #14935 from dillaman/wip-rbd-cli-mirror-pool

rbd: removed spurious error message from mirror pool commands

Reviewed-by: Mykola Golub <mgolub@mirantis.com>

Merge pull request #14400 from ceph/wip-cd-1node

qa/tasks: few fixes to get ceph-deploy 1node to working state

Merge pull request #14936 from cbodley/wip-19834

rgw: RGWPeriodPusher spawns http thread before cr thread

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>

Merge pull request #14908 from guihecheng/rgw_file-cleanup

rgw_file: cleanup virtual keyword on derived functions

Merge pull request #14891 from tchaikov/wip-librbd-test-odr

librbd: do not instantiate templates while building tests

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

librbd: do not instantiate templates while building tests

to avoid violation of ODR

Fixes: http://tracker.ceph.com/issues/18938
Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #14844 from liewegas/wip-bluestore-bitalloc

unittest_alloc: add test_alloc_big

Reviewed-by: Igor Fedotov <ifedotov@mirantis.com>

unittest_alloc: add test_alloc_big

Ensure that big allocations work.

Verifies http://tracker.ceph.com/issues/16662

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #14953 from tchaikov/wip-librbd-silence-Wunused-var

librbd: silence -Wunused-variable warning

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #14407 from cbodley/wip-rgw-admin-remote

rgw: fix handling of --remote in radosgw-admin period commands
Reviewed-by: Orit Wasserman <owasserm@redhat.com>

librbd: silence -Wunused-variable warning

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #11179 from rzarzynski/wip-rgw-swift-formpost

rgw: add support for FormPost of Swift API
Reviewed-by: Orit Wasserman <owasserm@redhat.com>

Merge pull request #14949 from zhsj/fix-cryptopp-build

common: fix building against libcryptopp

Reviewed-by: Kefu Chai <kchai@redhat.com>

common: fix building against libcryptopp

libnspr is only needed for libnss, move it to if defined(USE_NSS)
section.

ceph::crypto::shutdown's params should be same with declaration.

Signed-off-by: Shengjing Zhu <zhsj@umcloud.com>

Merge pull request #14923 from joscollin/wip-dmclocktest-function-error

dmclock: include missing <functional> header.

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #14948 from Liuchang0812/wip-kill-duplicated-cmake-commands

cmake: kill duplicated cmake commands

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #14951 from smithfarm/wip-19853

cmake: align cmake names of library packages

Reviewed-by: Kefu Chai <kchai@redhat.com>

Update rgw_rest.cc

fix indentation.
Signed-off-by: Orit Wasserman <owasserm@redhat.com>

cmake: align cmake names of library packages

The cmake package names of libraries are case-sensitive. The name used in
FIND_PACKAGE_HANDLE_STANDARD_ARGS - e.g. "cryptopp" - must match the name used
in the module name - "Findcryptopp.cmake" - as well as the name used in
find_package() call - "find_package(cryptopp REQUIRED)"

Note that the term "package" here refers to a cmake abstraction, *not* an
actual RPM or Debian package.

Fixes: http://tracker.ceph.com/issues/19853
Signed-off-by: Nathan Cutler <ncutler@suse.com>

cmake: kill duplicated cmake commands

Signed-off-by: liuchang0812 <liuchang0812@gmail.com>

Merge pull request #14947 from dmick/wip-arm-compressor

compressor/zlib: fix plugin for non-Intel arches

Reviewed-by: Kefu Chai <kchai@redhat.com>

compressor/zlib: fix plugin for non-Intel arches

unittest_compressor was failing on arm64 because the zlib
compressor was never initialized, even though it works fine in
non-isal mode

Signed-off-by: Dan Mick <dan.mick@redhat.com>

Merge pull request #14363 from majianpeng/finisher-optimize

common/Finisher: batch handle perfcounter && only send signal when waiter existed.

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #14627 from badone/wip-peering-state-timing

osd: Implement peering state timing

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #14709 from majianpeng/luestore-txc-finalize-kv

os/blueStore: In osd_tp_thread, call _txc_finalize_kv.

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Igor Fedotov <ifedotov@mirantis.com>

Merge pull request #14932 from markhpc/wip-rocksdb-compaction-readahead

common/config_opts.h: compaction readahead for bluestore/rocksdb

Reviewed-by: Sage Weil <sage@redhat.com>

test/librbd: remove duplicated explicit template specializations

to avoid violation of ODR

Fixes: http://tracker.ceph.com/issues/18938
Signed-off-by: Kefu Chai <kchai@redhat.com>

rbd-mirror: ensure missing images are re-synced when detected

Fixes: http://tracker.ceph.com/issues/19811
Signed-off-by: Jason Dillaman <dillaman@redhat.com>

default vpm is trusty and for master we need latest which is xenial

Signed-off-by: Vasu Kulkarni <vasu@redhat.com>

install mgr on the node

Signed-off-by: Vasu Kulkarni <vasu@redhat.com>

Merge pull request #14879 from wjwithagen/wip-wjw-reduce-noise-3

cls/log/cls_log.cc: reduce logging noise

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #14928 from drunkard/master

doc: remove deprecated subcommand in man/8/ceph.rst

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #14723 from neha-ojha/objclass-sdk

osd: objclass sdk

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #14917 from yuriw/wip-add-overrite-cls

qa/added overrides

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

cls/log/cls_log.cc: reduce logging noise

 - The other reference in the source as already at 20.
      ./src/cls/timeindex/cls_timeindex.cc:85:
CLS_LOG(20, "storing entry at %s", index.c_str());

   And we need not always know where in the log items are stored.
   So it looks like a leftover debug feature.

Fixes: http://tracker.ceph.com/issues/19835
Signed-off-by: Willem Jan Withagen <wjw@digiware.nl>

Merge pull request #14913 from liewegas/wip-19826

mon: do not prime_pg_temp creating pgs; clean up pg create conditions

Reviewed-by: Kefu Chai <kchai@redhat.com>

common/Finisher: No waite_for_empty no signal.

Avoid every time to send a signal because wait_for_empty is rare event.

Signed-off-by: Jianpeng Ma <jianpeng.ma@intel.com>

rgw: RGWPeriodPusher spawns http thread before cr thread

Fixes: http://tracker.ceph.com/issues/19834
Signed-off-by: Casey Bodley <cbodley@redhat.com>

radosgw-admin: 'period pull' reads endpoints from current period

by finding remote endpoints from the current period, 'period pull' no
longer needs to initialize RGWRados for zone[group]_conn_map

Fixes: http://tracker.ceph.com/issues/19554
Signed-off-by: Casey Bodley <cbodley@redhat.com>

radosgw-admin: commit_period gets remote connection from new period

Fixes: http://tracker.ceph.com/issues/19554
Signed-off-by: Casey Bodley <cbodley@redhat.com>

radosgw-admin: send_to_remote_gateway takes RGWRESTConn*

callers are responsible for providing their own remote connection

Signed-off-by: Casey Bodley <cbodley@redhat.com>

radosgw-admin: get remote connections from given period/zonegroup

helper functions to create connections to arbitrary remotes, rather
than depending on RGWRados to construct them in zone[group]_conn_map

Signed-off-by: Casey Bodley <cbodley@redhat.com>

rgw: enable move on RGWRESTConn

added custom move construct/assign because std::atomic is not movable

Signed-off-by: Casey Bodley <cbodley@redhat.com>

rbd: removed spurious error message from mirror pool commands

If the pool was empty, the rbd CLI would report the error
"rbd: failed to list images within pool".

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

common/Finisher: batch handle perfcounter.

Signed-off-by: Jianpeng Ma <jianpeng.ma@intel.com>

common/perf_counter: make tinc() handle batch-processing

Signed-off-by: Jianpeng Ma <jianpeng.ma@intel.com>

common/config_opts.h: compaction readahead for bluestore/rocksdb

Signed-off-by: Mark Nelson <mnelson@redhat.com>

Merge pull request #14903 from dillaman/wip-19413

cls_rbd: default initialize snapshot namespace for legacy clients

Reviewed-by: Mykola Golub <mgolub@mirantis.com>

Merge pull request #14875 from dillaman/wip-rbd-trash-parent

rbd: info command should indicate if parent is in trash

Reviewed-by: Mykola Golub <mgolub@mirantis.com>

Merge pull request #14929 from tchaikov/wip-disable-isal-on-arm

test/compressor: disable isal tests if not available

Reviewed-by: Sage Weil <sage@redhat.com>

qa/tasks/ceph_manager: always fix pgp_num when done with thrashosd task

Fixes: http://tracker.ceph.com/issues/19771
Signed-off-by: Kefu Chai <kchai@redhat.com>

test/compressor: disable isal tests if not available

the isal-zlib compressor plugin is not supported on non-intel archs, and
if the user does not have a capable yasm compiler, neither will he/she
get this plugin.

Signed-off-by: Kefu Chai <kchai@redhat.com>

doc: cleanup in doc/man/8/ceph.rst

Subcommand doc has been removed in 904913b37f241a73aa6722a18151d3166c8c93dd,
this is cleanup.

Signed-off-by: Drunkard Zhang <gongfan193@gmail.com>

dmclock: ‘std::function’ has not been declared

The following error appears during make:

ceph/src/dmclock/test/test_test_client.cc:14:0:
ceph/src/dmclock/test/../sim/src/sim_recs.h:40:12: error: ‘std::function’ has not been declared
       std::function<void()> code) {

Signed-off-by: Jos Collin <jcollin@redhat.com>

Merge pull request #14822 from jan--f/fuse-normalize-wire-o_flags

fs/ceph-fuse: normalize file open flags on the wire

Merge pull request #14909 from joscollin/wip-dmclock-error

dmclock: error: ‘function’ in namespace ‘std’ does not name a template type

Reviewed-by: J. Eric Ivancich <ivancich@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #14748 from smithfarm/wip-teuthology-worker

doc: mention teuthology-worker security group

Reviewed-by: Loic Dachary <ldachary@redhat.com>

mon/OSDMonitor: skip new pgs for prime_pg_temp

If the pg didn't previously exist we don't need to (and can't) prime
pg_temp for it.  This avoids crashing on mapping.get().

Signed-off-by: Sage Weil <sage@redhat.com>

qa/added overrides

Signed-off-by: Yuri Weinstein <yweinste@redhat.com>

qa/workunits/rbd: add additional OpenStack services

The Ocata release requires additional services to be running.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #14412 from mikulely/cleanup-civetweb-port

rgw: drop unused port var

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #13451 from xiexingguo/wip-add-up-primary-sum

mon/PGMap: add up_primary pg number field for pg-dump cmd

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #14497 from tchaikov/wip-pybind-better-error-msg

pybind: better error msg

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #14687 from jcsp/wip-mgr-suite

qa/suites: move mgr tests into rados suite

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #14810 from liewegas/wip-status

mon: show inactive % in ceph status

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #14821 from badone/wip-no-death-coredumps

tests: Don't dump core when using EXPECT_DEATH

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #14857 from joscollin/wip-arch-cpuid

arch: use __get_cpuid instead of do_cpuid

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #12669 from hjwsm1989/unfound-objects

osd/PG: fix possible overflow on unfound objects

Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>

Merge pull request #12882 from wonzhq/pglog-fixes

osd: pglog trimming fixes

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #14444 from liewegas/wip-past-intervals

osd: simplify past_intervals representation

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #14898 from tchaikov/wip-clean-pending_created_pgs

mon/OSDMonitor: cleanup pending_created_pgs after done with it

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #14904 from tchaikov/wip-19638-followon

osd: stop mgrc earlier in shutdown()

Reviewed-by: Sage Weil <sage@redhat.com>

osd: fix legacy pg create conditions

1- in reality we don't really care if pg_temp is set.
2- compare against acting_primary since that's what actually matters.

Signed-off-by: Sage Weil <sage@redhat.com>

mon/OSDMonitor: fix prime_pg_temp vs creating condition

We want to *skip* creating PGs.

Fixes: http://tracker.ceph.com/issues/19826
Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #14858 from dmick/wip-config-dump

mon/ConfigKeyService: add 'config-key dump' to show keys and vals

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #14870 from smithfarm/wip-gperftools

build/ops: rpm: gperftools-devel >= 2.4

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Ken Dreyer <kdreyer@redhat.com>

qa/workunits: fix appdir setup issue in devstack

See https://bugs.launchpad.net/devstack/+bug/1667545

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

qa/workunits: switch to OpenStack Ocata release for RBD testing

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

dmclock: error: ‘function’ in namespace ‘std’ does not name a template type

The following error appears during make:

In file included from ceph/src/dmclock/support/src/run_every.cc:10:0:
ceph/src/dmclock/support/src/run_every.h:30:10: error: ‘function’ in namespace ‘std’ does not name a template type
     std::function<void()>     body;
          ^~~~~~~~
ceph/src/dmclock/support/src/run_every.h:46:12: error: ‘std::function’ has not been declared
       std::function<void()> _body) :
            ^~~~~~~~
ceph/src/dmclock/support/src/run_every.h:46:20: error: expected ‘,’ or ‘...’ before ‘<’ token
       std::function<void()> _body) :
                    ^
ceph/src/dmclock/support/src/run_every.h: In constructor ‘crimson::RunEvery::RunEvery(D, int)’:
ceph/src/dmclock/support/src/run_every.h:48:7: error: class ‘crimson::RunEvery’ does not have any field named ‘body’
       body(_body)
       ^~~~
ceph/src/dmclock/support/src/run_every.h:48:12: error: ‘_body’ was not declared in this scope
       body(_body)
            ^~~~~
ceph/src/dmclock/support/src/run_every.cc: In member function ‘void crimson::RunEvery::run()’:
ceph/src/dmclock/support/src/run_every.cc:70:7: error: ‘body’ was not declared in this scope
       body();
       ^~~~
ceph/src/dmclock/support/src/run_every.cc:70:7: note: suggested alternative: ‘bool’
       body();
       ^~~~
       bool

Signed-off-by: Jos Collin <jcollin@redhat.com>

rgw_file: cleanup virtual keyword on derived functions

Signed-off-by: Gui Hecheng <guihecheng@cmss.chinamobile.com>

Merge pull request #14869 from wjwithagen/wip-wjw-cmake-uuid

cmake: fix the linked lib reference of unittest_rgw_crypto

Reviewed-by: Kefu Chai <kchai@redhat.com>