qa/suites/rbd: reduce monitor requirements for mirroring tests

Fixes: http://tracker.ceph.com/issues/21251
Signed-off-by: Jason Dillaman <dillaman@redhat.com>

qa/suites/rbd: fixed cache override

Fixes: http://tracker.ceph.com/issues/21251
Signed-off-by: Jason Dillaman <dillaman@redhat.com>

qa/suites/rbd: reduce OSD debug log levels

The teuthology machines are periodically running out of space
due to the aggressive log settings.

Fixes: http://tracker.ceph.com/issues/21251
Signed-off-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #17141 from theanalyst/doc-civetweb-ports

doc: rgw: mention the civetweb support for binding to multiple ports

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #17436 from ashishkumsingh/wip-doc-fix-snapshot-flatten-example

doc: Fixes rbd snapshot flatten example

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #17457 from batrick/githubmap

githubmap: add map for GitHub contributor lookup

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #17489 from mychoxin/yuanxin_deep_copy

include/buffer.h: fix typo in comment

Reviewed-by: Jos Collin <jcollin@redhat.com>

include/buffer.h: fix comment 'no not allow' -> 'do not allow'

Signed-off-by: mychoxin <mychoxin@gmail.com>

Merge pull request #17430 from Songweibin/wip-osd-misc

osd: misc cleanups

Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>

Merge pull request #16476 from renhwztetecs/renhw-wip-mgr-standby-ip

mgr: add the ip addr of standbys

Reviewed-by: John Spray <john.spray@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #17431 from tchaikov/wip-da-SCA-20170510

Coverity and SCA fixes

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #17437 from tchaikov/wip-refactor-build_initial_pg_history

osd: update info only if new_interval

Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>

Merge pull request #17393 from tchaikov/wip-cmake-disable-vta

cmake: disable VTA on options.cc

Reviewed-by: Sage Weil <sage@redhat.com>

githubmap: add map for GitHub contributor lookup

Idea of this is to allow scripts to lookup the contributor name/email by GitHub
username. This is useful in particular for adding appropriate "Reviewed-by"s
for each GitHub style "review".

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #15705 from xiexingguo/wip-fix-blob-tracker-check

os/bluestore: pass strict flag to bluestore_blob_use_tracker_t::equal()

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Igor Fedotov <ifedotov@mirantis.com>

doc: Fixes rbd snapshot flatten example

Snapshot flatten example has incorrect image name, fixing the same.

Fixes: #17723
Signed-off-by: Ashish Singh <assingh@redhat.com>

Merge pull request #15363 from ShiqiCooperation/master

test/unittest_bluefs: check whether rmdir success

Reviewed-by: Sage Weil <sage@redhat.com>

osd: update info only if new_interval

refactor OSD::build_initial_pg_history() so we update the info only if a
new interval is identified. also, this is consistent with
OSD::build_past_intervals_parallel().

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #17435 from liupan1111/wip-fix-spdk-perf

bluestore/NVMEDevice: accurate the latency perf counter of queue latency

Reviewed-by: Haomai Wang <haomai@xsky.com>

bluestore/NVMEDevice: accurate the latency of queue latency.

Signed-off-by: Pan Liu <wanjun.lp@alibaba-inc.com>
Signed-off-by: Ziye Yang <optimistyzy@gmail.com>

bluestore/NVMEDevice: remove mm_pause according to the example givn by
spdk.

Signed-off-by: Pan Liu <wanjun.lp@alibaba-inc.com>
Signed-off-by: Ziye Yang <optimistyzy@gmail.com>

mgr: add the ip addr of standbys

we need to manage the ip addr of the "standbys" state,
because the hostname/gid is insufficient to locate the
Standby node. we add ip of the mgr standby to metadata.

Signed-off-by: huanwen ren <ren.huanwen@zte.com.cn>

bluestore/NVMEDevice: rename the perf counter *aio_write* to *write*,
because it will also statistic the latency of sync write

Signed-off-by: Pan Liu <wanjun.lp@alibaba-inc.com>
Signed-off-by: Ziye Yang <optimistyzy@gmail.com>

bluestore/NVMEDevice.cc: rename the thread from ceph-osd to
nvme-device-manager, to avoid name duplication with main thread
"ceph-osd"

Signed-off-by: Pan Liu <wanjun.lp@alibaba-inc.com>
Signed-off-by: Ziye Yang <optimistyzy@gmail.com>

Merge pull request #17409 from liewegas/wip-21207

os/bluestore: separate finisher for deferred_try_submit

Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>
Reviewed-by: Varada Kari <varada.kari@sandisk.com>

Merge pull request #17395 from theanalyst/cli-rados-stat2

cli: rados: support for high precision time using stat2

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #17100 from mychoxin/yuanxin_vstart_default_bluestore

vstart.sh: use bluestore as default osd objectstore backend

Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>

Merge pull request #16855 from amitkumar50/rgw_module

rgw: Initializes uninitialized members

Reviewed-by: Jos Collin <jcollin@redhat.com>
Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #17129 from C2python/clear_para

rgw: Remove unused Parameter in Function RGWConf::init()

Reviewed-by: Jos Collin <jcollin@redhat.com>
Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>

Merge pull request #15341 from chardan/jfw-wip-rand_makes_puppies_cry

common: implement random number generator (following N3551)

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Adam Emerson <aemerson@redhat.com>

Merge pull request #15565 from gmayyyha/ceph-osd-help-output

ceph-osd: some flags are not documented in the help output

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #16942 from jcsp/wip-20888

mon: rate limit on health check update logging

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #17404 from rzarzynski/wip-bs-ldout-in-stupid

os/bluestore: replace dout with ldout in StupidAllocator.

Merge pull request #17406 from liewegas/wip-21206

qa/tasks/ceph_manager: revive osds before doing final rerr reset

Merge pull request #17416 from xiexingguo/wip-handle-small-block

os/bluestore: handle small main device properly

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #17423 from liewegas/wip-21203

osd: fix build_initial_pg_history

Reviewed-by: Sage Weil <sage@redhat.com>

test/librados/aio.cc: silence coverity with scope_guard

Fix for:

CID 1313405 (#1 of 1): Resource leak (RESOURCE_LEAK)
 leaked_storage: Variable my_completion2 going out of scope
 leaks the storage it points to.

CID 1219522 (#1 of 1): Resource leak (RESOURCE_LEAK)
 leaked_storage: Variable my_completion going out of scope
 leaks the storage it points to.

CID 1219501 (#1 of 1): Resource leak (RESOURCE_LEAK)
 leaked_storage: Variable my_completion going out of scope
 leaks the storage it points to.

CID 1313406 (#1 of 1): Resource leak (RESOURCE_LEAK)
 leaked_storage: Variable my_completion2 going out of scope
 leaks the storage it points to.

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

c_read_operations.cc: silence coverity with scope_guard

CID 1219593 (#1 of 1): Resource leak (RESOURCE_LEAK)
 leaked_storage: Variable completion going out of scope leaks
 the storage it points to.

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

LibradosTestStub.cc: memset struct sockaddr_in

Fix for:

CID 1402624 (#1 of 1): Uninitialized scalar variable (UNINIT)
 uninit_use: Using uninitialized value sin. Field sin.sin_zero is
 uninitialized.

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

common/Timer.h: ~SafeTimer needs to be virtual

Fix for:

CID 1396232 (#1 of 1): Non-virtual destructor (VIRTUAL_DTOR)
 nonvirtual_dtor: Class librbd::<unnamed>::SafeTimerSingleton has a
 destructor and a pointer to it is upcast to class SafeTimer which
 doesn't have a virtual destructor.

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

test_ipaddr.cc: memset with 0 and not '0'

Fix for:

CID 1405070 (#1 of 1): Memset fill value of '0' (NO_EFFECT)
CID 1405071 (#1 of 1): Memset fill value of '0' (NO_EFFECT)
CID 1405073 (#1 of 1): Memset fill value of '0' (NO_EFFECT)
CID 1405074 (#1 of 1): Memset fill value of '0' (NO_EFFECT)
CID 1405075 (#1 of 1): Memset fill value of '0' (NO_EFFECT)
CID 1405077 (#1 of 1): Memset fill value of '0' (NO_EFFECT)
CID 1405083 (#1 of 1): Memset fill value of '0' (NO_EFFECT)
CID 1405086 (#1 of 1): Memset fill value of '0' (NO_EFFECT)
CID 1405087 (#1 of 1): Memset fill value of '0' (NO_EFFECT)
 bad_memset: "memset" with fill value "'0'" (the zero character).
 memset(&net, 48, 28UL). (CWE-665)

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

client/MetaRequest.h: fix UNINIT_CTOR

Fix for:

CID 717207 (#1 of 1): Uninitialized pointer field (UNINIT_CTOR)
 uninit_member: Non-static class member dirp is not initialized
 in this constructor nor in any functions that it calls.

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

client/Client.cc: fix UNINIT_CTOR

Fix for:

CID 1406088 (#1 of 1): Uninitialized pointer field (UNINIT_CTOR)
 uninit_member: Non-static class member root_ancestor is not
 initialized in this constructor nor in any functions that it calls.

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

crush/CrushCompiler.cc: fix error handling

calloc() does not take negative values, check return value from
get_max_buckets() and handle it correctly.

Fix for:

CID 1405301 (#1 of 1): Argument cannot be negative (NEGATIVE_RETURNS)
 negative_returns: arg_map.size is passed to a parameter that cannot
 be negative.

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

client/Client.cc: fix USE_AFTER_FREE

Don't call put_request() twice, it's already called by
unregister_request()

Fix for:

CID 1405360 (#1 of 1): Use after free (USE_AFTER_FREE)
 deref_arg: Calling put_request dereferences freed pointer request

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

tools/rbd/Utils.cc: yank features_set_specified and related logic

since it isn't used

Fix for:

CID 1394854 (#1 of 1): 'Constant' variable guards dead code (DEADCODE)
 dead_error_line: Execution cannot reach this statement:
 opts->set(RBD_IMAGE_OPTION_....
 Local variable features_set_specified is assigned only once, to a
 constant value, making it effectively constant throughout its scope.

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

Merge pull request #17304 from kungf/calc_replicated_acting

osd/PG: remove unused parameter in calc_ec_acting

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #17088 from zmedico/interval_set_intersection_of_optimization

interval_set: optimize intersection_of

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #15816 from chardan/jfw-wip-consolidate-spinlocks

common: consolidate spinlocks

Reviewed-by: Adam Emerson <aemerson@redhat.com>

Merge pull request #17265 from zmedico/interval-set-intersection-insert-optimization

interval_set: optimize intersect_of insert operations

Reviewed-by: Adam Emerson <aemerson@redhat.com>

Merge pull request #17391 from myoungwon/wip-proxywrite-cleanup

osd: remove unused variable in do_proxy_write

Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>

Merge pull request #17394 from Songweibin/wip-common-cleanup

common: drop unused variables "bluestore_csum_*_block" in opts

Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>

Merge pull request #17398 from xiexingguo/wip-21175

os/bluestore: don't re-initialize csum-setting for existing blobs

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #17160 from ceph/wip-luminous-x-master

qa: add upgrade/luminous-x suite

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #17405 from tchaikov/wip-remove-boost-submodule

boost: remove boost submodule

Reviewed-by: Alfredo Deza <adeza@redhat.com>

Merge pull request #17425 from ceph/wip-ceph-volume-docs

docs: ceph-volume CLI updates

Reviewed-by: Andrew Schoen <aschoen@redhat.com>

Merge pull request #17426 from liewegas/wip-integration

script/build-integration-branch: print pr url list with titles

Reviewed-by: Yuri Weinstein <yweinstei@redhat.com>

script/build-integration-branch: print pr url list with titles

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #17353 from scienceluo/wip-luo-fix-rgw-memory-leak-branch

rgw: fix resource leak in rgw_bucket.cc and rgw_user.cc

Reviewed-by: Jos Collin <jcollin@redhat.com>
Reviewed-by: Casey Bodley <cbodley@redhat.com>

doc/ceph-volume prepare partitions with PARTUUID

Signed-off-by: Alfredo Deza <adeza@redhat.com>

Initial check in

Signed-off-by: Yuri Weinstein <yweinste@redhat.com>

doc/ceph-volume prepare needs both volume group and lv names

Signed-off-by: Alfredo Deza <adeza@redhat.com>

doc/ceph-volume activate re-links the journal always

Signed-off-by: Alfredo Deza <adeza@redhat.com>

doc/ceph-volume emphasize the danger of calling prepare for an existing OSD

Signed-off-by: Alfredo Deza <adeza@redhat.com>

rgw: fix resource leak in rgw_bucket.cc and rgw_user.cc

Fixes: http://tracker.ceph.com/issues/21214
Signed-off-by: Luo Kexue <luo.kexue@zte.com.cn>

Merge pull request #17421 from nrdmn/dashboard-colors

mgr/dashboard: change raw usage chart's color depending on usage

Reviewed-by: John Spray <john.spray@redhat.com>

Merge pull request #17424 from theanalyst/release-eol-updates-l

doc/releases: the Kraken sleepeth, faintest sunlights flee

Reviewed-by: Nathan Cutler <ncutler@suse.com>

mon: rate limit on health check update logging

Fixes: http://tracker.ceph.com/issues/20888
Signed-off-by: John Spray <john.spray@redhat.com>

doc/releases: the Kraken sleepeth, faintest sunlights flee

updating releases to retire Kraken, Hammer. Also updated to mention
Jewel will be around until at least N is finalized

Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>

osd: fix build_initial_pg_history

We need to update our info about the previous interval in order to
detect interval changes properly.

Fixes: http://tracker.ceph.com/issues/21203
Signed-off-by: w11979 <wang.wenfeng@h3c.com>
Signed-off-by: Sage Weil <sage@redhat.com>

doc/ceph-volume update intro to mention dm-cache

Signed-off-by: Alfredo Deza <adeza@redhat.com>

Merge pull request #17364 from theanalyst/release-updates

doc: update releases to the current state

mgr/dashboard: change raw usage chart's color depending on usage

Signed-off-by: Nick Erdmann <n@nirf.de>

mon: add helper for emitting logs by health status

We were starting to repeat our little "if(status==HEALTH_ERR"
blocks in too many places.

Signed-off-by: John Spray <john.spray@redhat.com>

doc: update releases to the current state

Adding Luminous release date, and dropping links for dev releases (since
they've been merged to 12.2.0), also rearranged the table so that newer
releases come left

Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>

os/bluestore: handle small main device properly

Originally reported by MarcinDulak on 'ceph-users' channel.

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

os/bluestore: formatting nits

So we don't output anything if csum is currently off:

_do_alloc_write blob Blob(0x7fad7cc25950 blob([0x42a80000~10000] none/0x1)

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

os/bluestore: don't re-initialize csum-setting for existing blobs

The global checksum setting may change, e.g., from NONE to CRC32,
which can cause improper re-initialization of the csum-settings of
existing blobs(e.g., partial write/overwrite may turn out to shrink
'csum_data').

We could develop some complicated solutions but for now let's not
bother since the above scenario is rare.

Fixes: http://tracker.ceph.com/issues/21175
Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

Merge pull request #17397 from ZVampirEM77/wip-doc

doc: fix doc/radosgw/admin.rst typos

Reviewed-by: Jos Collin <jcollin@redhat.com>

make-dist: download and repackage boost

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #16521 from myoungwon/wip-objecter-finishers

osd: add multiple objecter finishers

Reviewed-by: Liu-Chunmei <chunmei.liu@intel.com>
Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #16586 from tianshan/fix_logrotate

logrotate: add systemd reload in logrotate in case of centos minimal without killall

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #16643 from Aran85/wip-mon-deadlock-on-shutdown

kv/leveldb: fix deadlock when close db

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #16733 from liupan1111/wip-bluefs-fio

os: add compile option to build libbluefs.so

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #17244 from liewegas/wip-21121

osd: subscribe to new osdmap while waiting_for_healthy

Reviewed-by: Kefu Chai <kchai@redhat.com>

os/bluestore: separate finisher for deferred_try_submit

Reusing finishers[0], which is used for completions back into the OSD,
is deadlock-prone: the OSD code might block trying to submit new IO or
while waiting for some other bluestore work to complete.

Fixes: http://tracker.ceph.com/issues/21207
Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #16925 from gmayyyha/mgr-osd-metadata

mgr: implement completion of osd MetadataUpdate

Reviewed-by: John Spray <john.spray@redhat.com>

Merge pull request #17036 from majianpeng/add-pmem-submodule

submodule: make libmpem as a submodule.

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #17354 from liewegas/wip-bluefs-compaction

os/bluestore/BlueFS: compact log even when sync_metadata sees no work

Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>
Reviewed-by: Varada Kari <varada.kari@gmail.com>

Merge pull request #17357 from Liuchang0812/wip-bluefs-not-assert-eexist

os/bluestore: do not core dump when BlueRocksEnv gets EEXIST error

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #17360 from tchaikov/wip-crush-cleanup

crush: various CrushWrapper cleanups

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #17379 from liewegas/wip-div-p

qa/suites/rados/singleton/diverget_priors*: broaden whitelist

qa/tasks/ceph_manager: revive osds before doing final rerr reset

We assume below that rerrosd is up, but it may not be when we exit the
loop.

Fixes: http://tracker.ceph.com/issues/21206
Signed-off-by: Sage Weil <sage@redhat.com>

boost: remove boost submodule

Signed-off-by: Kefu Chai <kchai@redhat.com>

os/bluestore: replace dout with ldout in StupidAllocator.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

Merge pull request #17401 from liewegas/wip-disable-pidfile

test/CMakeLists: disable test-pidfile.sh

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #17390 from tchaikov/wip-test-coredump

test: Don't dump core when using EXPECT_DEATH

Reviewed-by: Brad Hubbard <bhubbard@redhat.com>

rgw: Initializes uninitialized members of rgw

Fixes the coverity issues:

** 1352181 Uninitialized scalar field
2. uninit_member: Non-static class member field fh_hk.bucket is
not initialized in this constructor nor in any functions that it calls.
CID 1352181 (#1 of 1): Uninitialized scalar field (UNINIT_CTOR)
4. uninit_member: Non-static class member field fh_hk.object is
not initialized in this constructor nor in any functions that it calls.

** 1353424 Uninitialized scalar field
CID 1353424 (#1 of 1): Uninitialized scalar field (UNINIT_CTOR)
5. uninit_member: Non-static class member watch_handle is not initialized
 in this constructor nor in any functions that it calls.

** 1355240 Uninitialized scalar field
CID 1355240 (#1 of 1): Uninitialized scalar field (UNINIT_CTOR)
2. uninit_member: Non-static class member index_type is not initialized
in this constructor nor in any functions that it calls.

Signed-off-by: Amit Kumar <amitkuma@redhat.com>

doc: fix doc/radosgw/admin.rst typos

Signed-off-by: Enming Zhang <enming.zhang@umcloud.com>

test/CMakeLists: disable test-pidfile.sh

Signed-off-by: Sage Weil <sage@redhat.com>