rbd-nbd: relax size check for newer kernel versions

Fixes: http://tracker.ceph.com/issues/19871
Signed-off-by: Mykola Golub <mgolub@mirantis.com>

Merge pull request #14973 from ifed01/wip-bluestore-kvsync-cond

os/bluestore: remove unused condition variable

Reviewed-by: Sage Weil <sage@redhat.com>

os/bluestore: remove unused condition variable

Signed-off-by: Igor Fedotov <ifedotov@mirantis.com>

Merge pull request #14963 from dillaman/wip-19858

rbd-mirror: failover and failback of unmodified image results in split-brain

Reviewed-by: Mykola Golub <mgolub@mirantis.com>

Merge pull request #14965 from dmick/wip-19865

test/librbd/test_librbd.cc: set *features even if RBD_FEATURES is unset

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #14465 from dillaman/wip-librbd-devstack-tempest

qa/workunits: switch to OpenStack Ocata release for RBD testing

Reviewed-by: Mykola Golub <mgolub@mirantis.com>

test/librbd/test_librbd.cc: set *features even if RBD_FEATURES is unset

If RBD_FEATURES is not in the environment, set *features to 0 in
get_features(); callers rely on a valid return value.  (This was
breaking on arm64.)

Fixes: http://tracker.ceph.com/issues/19865
Signed-off-by: Dan Mick <dan.mick@redhat.com>

librbd: add no-op event when promoting an image

The rbd-mirror process needs an event in the journal
to properly detect the transition between primary and
non-primary state between peers.

Fixes: http://tracker.ceph.com/issues/19858
Signed-off-by: Jason Dillaman <dillaman@redhat.com>

librbd: do not delay propagation of demote events

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

rbd-mirror: prevent infinite loop when computing replay status

If the image had a non-primary predecessor epoch whose tag tid
duplicates an epoch within its own journal, an infinite loop
would result.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #14373 from rzarzynski/wip-rgw-19520

rgw: fix broken /crossdomain.xml, /info and /healthcheck of Swift API.

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>

Merge pull request #9950 from weiqiaomiao/wqm-wip-fetchobj-req

rgw: should delete in_stream_req  if conn->get_obj(...) return not zero value

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #14601 from Jing-Scott/fix-bulkupload-multisite

rgw: fix forward request for bulkupload to be applied in multisite

Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: Radoslaw Zarzynski <rzarzynski@mirantis.com>

Merge pull request #14945 from dillaman/wip-19811

rbd-mirror: ensure missing images are re-synced when detected

Reviewed-by: Mykola Golub <mgolub@mirantis.com>

Merge pull request #14935 from dillaman/wip-rbd-cli-mirror-pool

rbd: removed spurious error message from mirror pool commands

Reviewed-by: Mykola Golub <mgolub@mirantis.com>

Merge pull request #14400 from ceph/wip-cd-1node

qa/tasks: few fixes to get ceph-deploy 1node to working state

Merge pull request #14936 from cbodley/wip-19834

rgw: RGWPeriodPusher spawns http thread before cr thread

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>

Merge pull request #14908 from guihecheng/rgw_file-cleanup

rgw_file: cleanup virtual keyword on derived functions

Merge pull request #14891 from tchaikov/wip-librbd-test-odr

librbd: do not instantiate templates while building tests

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

librbd: do not instantiate templates while building tests

to avoid violation of ODR

Fixes: http://tracker.ceph.com/issues/18938
Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #14844 from liewegas/wip-bluestore-bitalloc

unittest_alloc: add test_alloc_big

Reviewed-by: Igor Fedotov <ifedotov@mirantis.com>

unittest_alloc: add test_alloc_big

Ensure that big allocations work.

Verifies http://tracker.ceph.com/issues/16662

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #14953 from tchaikov/wip-librbd-silence-Wunused-var

librbd: silence -Wunused-variable warning

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #14407 from cbodley/wip-rgw-admin-remote

rgw: fix handling of --remote in radosgw-admin period commands
Reviewed-by: Orit Wasserman <owasserm@redhat.com>

librbd: silence -Wunused-variable warning

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #11179 from rzarzynski/wip-rgw-swift-formpost

rgw: add support for FormPost of Swift API
Reviewed-by: Orit Wasserman <owasserm@redhat.com>

Merge pull request #14949 from zhsj/fix-cryptopp-build

common: fix building against libcryptopp

Reviewed-by: Kefu Chai <kchai@redhat.com>

common: fix building against libcryptopp

libnspr is only needed for libnss, move it to if defined(USE_NSS)
section.

ceph::crypto::shutdown's params should be same with declaration.

Signed-off-by: Shengjing Zhu <zhsj@umcloud.com>

Merge pull request #14923 from joscollin/wip-dmclocktest-function-error

dmclock: include missing <functional> header.

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #14948 from Liuchang0812/wip-kill-duplicated-cmake-commands

cmake: kill duplicated cmake commands

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #14951 from smithfarm/wip-19853

cmake: align cmake names of library packages

Reviewed-by: Kefu Chai <kchai@redhat.com>

Update rgw_rest.cc

fix indentation.
Signed-off-by: Orit Wasserman <owasserm@redhat.com>

cmake: align cmake names of library packages

The cmake package names of libraries are case-sensitive. The name used in
FIND_PACKAGE_HANDLE_STANDARD_ARGS - e.g. "cryptopp" - must match the name used
in the module name - "Findcryptopp.cmake" - as well as the name used in
find_package() call - "find_package(cryptopp REQUIRED)"

Note that the term "package" here refers to a cmake abstraction, *not* an
actual RPM or Debian package.

Fixes: http://tracker.ceph.com/issues/19853
Signed-off-by: Nathan Cutler <ncutler@suse.com>

cmake: kill duplicated cmake commands

Signed-off-by: liuchang0812 <liuchang0812@gmail.com>

Merge pull request #14947 from dmick/wip-arm-compressor

compressor/zlib: fix plugin for non-Intel arches

Reviewed-by: Kefu Chai <kchai@redhat.com>

compressor/zlib: fix plugin for non-Intel arches

unittest_compressor was failing on arm64 because the zlib
compressor was never initialized, even though it works fine in
non-isal mode

Signed-off-by: Dan Mick <dan.mick@redhat.com>

Merge pull request #14363 from majianpeng/finisher-optimize

common/Finisher: batch handle perfcounter && only send signal when waiter existed.

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #14627 from badone/wip-peering-state-timing

osd: Implement peering state timing

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #14709 from majianpeng/luestore-txc-finalize-kv

os/blueStore: In osd_tp_thread, call _txc_finalize_kv.

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Igor Fedotov <ifedotov@mirantis.com>

Merge pull request #14932 from markhpc/wip-rocksdb-compaction-readahead

common/config_opts.h: compaction readahead for bluestore/rocksdb

Reviewed-by: Sage Weil <sage@redhat.com>

test/librbd: remove duplicated explicit template specializations

to avoid violation of ODR

Fixes: http://tracker.ceph.com/issues/18938
Signed-off-by: Kefu Chai <kchai@redhat.com>

rbd-mirror: ensure missing images are re-synced when detected

Fixes: http://tracker.ceph.com/issues/19811
Signed-off-by: Jason Dillaman <dillaman@redhat.com>

default vpm is trusty and for master we need latest which is xenial

Signed-off-by: Vasu Kulkarni <vasu@redhat.com>

install mgr on the node

Signed-off-by: Vasu Kulkarni <vasu@redhat.com>

Merge pull request #14879 from wjwithagen/wip-wjw-reduce-noise-3

cls/log/cls_log.cc: reduce logging noise

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #14928 from drunkard/master

doc: remove deprecated subcommand in man/8/ceph.rst

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #14723 from neha-ojha/objclass-sdk

osd: objclass sdk

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #14917 from yuriw/wip-add-overrite-cls

qa/added overrides

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

cls/log/cls_log.cc: reduce logging noise

 - The other reference in the source as already at 20.
      ./src/cls/timeindex/cls_timeindex.cc:85:
CLS_LOG(20, "storing entry at %s", index.c_str());

   And we need not always know where in the log items are stored.
   So it looks like a leftover debug feature.

Fixes: http://tracker.ceph.com/issues/19835
Signed-off-by: Willem Jan Withagen <wjw@digiware.nl>

Merge pull request #14913 from liewegas/wip-19826

mon: do not prime_pg_temp creating pgs; clean up pg create conditions

Reviewed-by: Kefu Chai <kchai@redhat.com>

common/Finisher: No waite_for_empty no signal.

Avoid every time to send a signal because wait_for_empty is rare event.

Signed-off-by: Jianpeng Ma <jianpeng.ma@intel.com>

rgw: RGWPeriodPusher spawns http thread before cr thread

Fixes: http://tracker.ceph.com/issues/19834
Signed-off-by: Casey Bodley <cbodley@redhat.com>

radosgw-admin: 'period pull' reads endpoints from current period

by finding remote endpoints from the current period, 'period pull' no
longer needs to initialize RGWRados for zone[group]_conn_map

Fixes: http://tracker.ceph.com/issues/19554
Signed-off-by: Casey Bodley <cbodley@redhat.com>

radosgw-admin: commit_period gets remote connection from new period

Fixes: http://tracker.ceph.com/issues/19554
Signed-off-by: Casey Bodley <cbodley@redhat.com>

radosgw-admin: send_to_remote_gateway takes RGWRESTConn*

callers are responsible for providing their own remote connection

Signed-off-by: Casey Bodley <cbodley@redhat.com>

radosgw-admin: get remote connections from given period/zonegroup

helper functions to create connections to arbitrary remotes, rather
than depending on RGWRados to construct them in zone[group]_conn_map

Signed-off-by: Casey Bodley <cbodley@redhat.com>

rgw: enable move on RGWRESTConn

added custom move construct/assign because std::atomic is not movable

Signed-off-by: Casey Bodley <cbodley@redhat.com>

rbd: removed spurious error message from mirror pool commands

If the pool was empty, the rbd CLI would report the error
"rbd: failed to list images within pool".

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

common/Finisher: batch handle perfcounter.

Signed-off-by: Jianpeng Ma <jianpeng.ma@intel.com>

common/perf_counter: make tinc() handle batch-processing

Signed-off-by: Jianpeng Ma <jianpeng.ma@intel.com>

common/config_opts.h: compaction readahead for bluestore/rocksdb

Signed-off-by: Mark Nelson <mnelson@redhat.com>

Merge pull request #14903 from dillaman/wip-19413

cls_rbd: default initialize snapshot namespace for legacy clients

Reviewed-by: Mykola Golub <mgolub@mirantis.com>

Merge pull request #14875 from dillaman/wip-rbd-trash-parent

rbd: info command should indicate if parent is in trash

Reviewed-by: Mykola Golub <mgolub@mirantis.com>

Merge pull request #14929 from tchaikov/wip-disable-isal-on-arm

test/compressor: disable isal tests if not available

Reviewed-by: Sage Weil <sage@redhat.com>

test/compressor: disable isal tests if not available

the isal-zlib compressor plugin is not supported on non-intel archs, and
if the user does not have a capable yasm compiler, neither will he/she
get this plugin.

Signed-off-by: Kefu Chai <kchai@redhat.com>

doc: cleanup in doc/man/8/ceph.rst

Subcommand doc has been removed in 904913b37f241a73aa6722a18151d3166c8c93dd,
this is cleanup.

Signed-off-by: Drunkard Zhang <gongfan193@gmail.com>

dmclock: ‘std::function’ has not been declared

The following error appears during make:

ceph/src/dmclock/test/test_test_client.cc:14:0:
ceph/src/dmclock/test/../sim/src/sim_recs.h:40:12: error: ‘std::function’ has not been declared
       std::function<void()> code) {

Signed-off-by: Jos Collin <jcollin@redhat.com>

Merge pull request #14822 from jan--f/fuse-normalize-wire-o_flags

fs/ceph-fuse: normalize file open flags on the wire

Merge pull request #14909 from joscollin/wip-dmclock-error

dmclock: error: ‘function’ in namespace ‘std’ does not name a template type

Reviewed-by: J. Eric Ivancich <ivancich@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #14748 from smithfarm/wip-teuthology-worker

doc: mention teuthology-worker security group

Reviewed-by: Loic Dachary <ldachary@redhat.com>

mon/OSDMonitor: skip new pgs for prime_pg_temp

If the pg didn't previously exist we don't need to (and can't) prime
pg_temp for it.  This avoids crashing on mapping.get().

Signed-off-by: Sage Weil <sage@redhat.com>

qa/added overrides

Signed-off-by: Yuri Weinstein <yweinste@redhat.com>

qa/workunits/rbd: add additional OpenStack services

The Ocata release requires additional services to be running.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #14412 from mikulely/cleanup-civetweb-port

rgw: drop unused port var

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #13451 from xiexingguo/wip-add-up-primary-sum

mon/PGMap: add up_primary pg number field for pg-dump cmd

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #14497 from tchaikov/wip-pybind-better-error-msg

pybind: better error msg

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #14687 from jcsp/wip-mgr-suite

qa/suites: move mgr tests into rados suite

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #14810 from liewegas/wip-status

mon: show inactive % in ceph status

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #14821 from badone/wip-no-death-coredumps

tests: Don't dump core when using EXPECT_DEATH

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #14857 from joscollin/wip-arch-cpuid

arch: use __get_cpuid instead of do_cpuid

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #12669 from hjwsm1989/unfound-objects

osd/PG: fix possible overflow on unfound objects

Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>

Merge pull request #12882 from wonzhq/pglog-fixes

osd: pglog trimming fixes

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #14444 from liewegas/wip-past-intervals

osd: simplify past_intervals representation

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #14898 from tchaikov/wip-clean-pending_created_pgs

mon/OSDMonitor: cleanup pending_created_pgs after done with it

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #14904 from tchaikov/wip-19638-followon

osd: stop mgrc earlier in shutdown()

Reviewed-by: Sage Weil <sage@redhat.com>

osd: fix legacy pg create conditions

1- in reality we don't really care if pg_temp is set.
2- compare against acting_primary since that's what actually matters.

Signed-off-by: Sage Weil <sage@redhat.com>

mon/OSDMonitor: fix prime_pg_temp vs creating condition

We want to *skip* creating PGs.

Fixes: http://tracker.ceph.com/issues/19826
Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #14858 from dmick/wip-config-dump

mon/ConfigKeyService: add 'config-key dump' to show keys and vals

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #14870 from smithfarm/wip-gperftools

build/ops: rpm: gperftools-devel >= 2.4

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Ken Dreyer <kdreyer@redhat.com>

qa/workunits: fix appdir setup issue in devstack

See https://bugs.launchpad.net/devstack/+bug/1667545

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

qa/workunits: switch to OpenStack Ocata release for RBD testing

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

dmclock: error: ‘function’ in namespace ‘std’ does not name a template type

The following error appears during make:

In file included from ceph/src/dmclock/support/src/run_every.cc:10:0:
ceph/src/dmclock/support/src/run_every.h:30:10: error: ‘function’ in namespace ‘std’ does not name a template type
     std::function<void()>     body;
          ^~~~~~~~
ceph/src/dmclock/support/src/run_every.h:46:12: error: ‘std::function’ has not been declared
       std::function<void()> _body) :
            ^~~~~~~~
ceph/src/dmclock/support/src/run_every.h:46:20: error: expected ‘,’ or ‘...’ before ‘<’ token
       std::function<void()> _body) :
                    ^
ceph/src/dmclock/support/src/run_every.h: In constructor ‘crimson::RunEvery::RunEvery(D, int)’:
ceph/src/dmclock/support/src/run_every.h:48:7: error: class ‘crimson::RunEvery’ does not have any field named ‘body’
       body(_body)
       ^~~~
ceph/src/dmclock/support/src/run_every.h:48:12: error: ‘_body’ was not declared in this scope
       body(_body)
            ^~~~~
ceph/src/dmclock/support/src/run_every.cc: In member function ‘void crimson::RunEvery::run()’:
ceph/src/dmclock/support/src/run_every.cc:70:7: error: ‘body’ was not declared in this scope
       body();
       ^~~~
ceph/src/dmclock/support/src/run_every.cc:70:7: note: suggested alternative: ‘bool’
       body();
       ^~~~
       bool

Signed-off-by: Jos Collin <jcollin@redhat.com>

rgw_file: cleanup virtual keyword on derived functions

Signed-off-by: Gui Hecheng <guihecheng@cmss.chinamobile.com>

Merge pull request #14869 from wjwithagen/wip-wjw-cmake-uuid

cmake: fix the linked lib reference of unittest_rgw_crypto

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #14900 from jupiturliu/organization_map

mailmap: add Alibaba into organization map

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #14895 from liewegas/wip-19778

kv: move 'bluestore-kv' hackery out of KeyValueDB into ceph-kvstore-tool

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #14853 from liewegas/wip-aio

os/fs/aio: use small_vector for aio_t; clean up header location

Reviewed-by: Kefu Chai <kchai@redhat.com>

OSD: stop mgrc earlier in shutdown()

it's better this way, as MgrClient is more like an internal consumer
of the services offered by OSD, instead of service privider. it could
prevent future problem if we can stop it earlier in the shutdown
process.

see also: http://tracker.ceph.com/issues/19638
Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #14897 from tchaikov/wip-19814

mon/OSDMonitor: update creating_pgs using pending_creatings

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #14889 from joscollin/wip-test-write-operations-warning

test: c_write_operations.cc: silence warning from -Wsign-compare