install-deps.sh: install `python*-devel` for python*rpm-macros

in 087ea813, we installed '*rpm-macros' for the macros, so we can have
access to the latest python packaging related macros for preparing the
build dependencies.

but we could run into https://bugs.centos.org/view.php?id=16379, if
we already have an old version of python-devel installed. as the newer
version of python-rpm-macros conflicts with it.

it was a chicken-and-egg problem, as we don't know the exact name of
*rpm-macros packages. that's why we chose to install all of them. but
we have to upgrade the existing python-devel package to resolve the
conflict. but the since there is no python3-devel in RHEL7/CentOS7,
what they have is python36-devel. so we have to hardwire the
`%{python3_pkgversion}` to "36" even before we have access to this
macro, and upgrade the python36-devel package beforehand. but this
renders installing the rpm-macro package less useful -- we intend to
use the macro offered by the package to figure out "36".

as a workaround, we pretend that we know the "main" version of python3
in current RHEL/CentOS. and always install python36-devel for
python-rpm-macros. as the former requires the latter.

once all python3*-devel on all builders are upgraded, we will be safe
to install '*rpm-macros' again without installing python36-devel first.
by then, we could revert this change, or continue installing
python36-devel until the distro bumps up the "main" python version to 3.7

Fixes: https://tracker.ceph.com/issues/41603
Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #30028 from tchaikov/wip-rpath

cmake: selectively rewrite install rpath

Reviewed-by: Sridhar Seshasayee <sseshasa@redhat.com>

Merge pull request #30015 from neha-ojha/wip-41429

mon/Monitor.cc: fix condition that checks for unrecognized auth mode

Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>
Reviewed-by: Joao Eduardo Luis <joao@suse.com>
Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #30023 from tchaikov/wip-get-req

mon,osd: use get_req<> instead of static_cast<>(get_req())

Reviewed-by: Greg Farnum <gfarnum@redhat.com>

Merge pull request #30210 from ctrlaltdel/doc-typo

doc: Fix a typo in balancer documentation

Reviewed-by: Lenz Grimmer <LGrimmer@suse.com>

Merge pull request #29520 from pcuzner/add-events-mgr-module

mgr/k8sevents: Add mgr module for kubernetes event integration

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>
Reviewed-by: Travis Nielsen <tnielsen@redhat.com>

doc: Fix a typo in balancer documentation

Signed-off-by: Francois Deppierraz <francois@ctrlaltdel.ch>

Merge PR #29786 into master

* refs/pull/29786/head:
allow for multiple "build" dirs to exist

Reviewed-by: Sage Weil <sage@redhat.com>

allow for multiple "build" dirs to exist

Signed-off-by: Yuval Lifshitz <yuvalif@yahoo.com>

Merge pull request #29985 from xiexingguo/wip-41514-plus

osd/PrimaryLogPG: cancel in-flight manifest ops on interval changing; fix race with scrub

Reviewed-by: Myoungwon Oh <ommw@sk.com>

Merge pull request #30184 from tchaikov/wip-make-check

run-make-check.sh: extract run-make.sh

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

Merge pull request #30085 from xiexingguo/wip-41601

osd/PrimaryLogPG: update oi.size on write op implicitly truncating object up

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge PR #29842 into master

* refs/pull/29842/head:
kv/KeyValueDB: fix estimate_prefix_size()

Reviewed-by: Sage Weil <sage@redhat.com>

Merge PR #29855 into master

* refs/pull/29855/head:
pybind/mgr: remove type conversions for ceph native options
mgr: return get_ceph_option result as typed Py object (not string)

Reviewed-by: Tim Serong <tserong@suse.com>

Merge pull request #30013 from ivancich/wip-beanjoy-update-attrib-tags

rgw: fix refcount tags to match and update object's idtag

Reviewed-by: Adam C. Emerson <aemerson@redhat.com>

Merge pull request #30062 from david-z/wip-fix-test-assert

test: remove useless ASSERT_XXX macros for rgw test

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #29673 from rafaelquint/imagetrash

mgr/dashboard: RBD Image Purge Trash, Move to Trash and Restore

Reviewed-by: Tiago Melo <tmelo@suse.com>

Merge pull request #28755 from dzafman/wip-network

feature: Health warnings on long network ping times, add "dump_osd_network" to get a report

Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>

run-make-check.sh: extract run-make.sh

so we can reuse run-make.sh for building the artifact used by other
tests than "make check", for instance, dashboard's E2E test and
crimson's performance test.

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #29402 from Devp00l/wip-40828

mgr/dashboard: Use onCancel on any modal event

Reviewed-by: Tiago Melo <tmelo@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>

Merge pull request #30064 from bk201/wip-41368

mgr/dashboard: fix cdEncode decorator is not working on class

Reviewed-by: Tiago Melo <tmelo@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>

Merge pull request #30145 from theanalyst/doc/releases/14.2.3

doc: releases: 14.2.3 dashboard note

https://github.com/ceph/ceph/pull/24179

mgr/dashboard: Empty datatable rendered before data has been f… (#30103)

mgr/dashboard: Empty datatable rendered before data has been fetched

Reviewed-by: Patrick Seidensal <pnawracay@suse.com>
Reviewed-by: Tiago Melo <tmelo@suse.com>

test: remove useless ASSERT_XXX macros for rgw test

Signed-off-by: Zhi Zhang <zhangz.david@outlook.com>

Merge pull request #29981 from falcon78921/wip-ceph-conf-usage

ceph-conf: added --show-config-value to usage

Reviewed-by: Kefu Chai <kchai@redhat.com>

mgr/k8sevents: Initial ceph -> k8s events integration

Adds a mgr module to provide integration between
Ceph and the kubernetes events API within the rook-ceph
namespace. It provides several commands to view event
status including;

ceph k8sevents list .... show all k8s related events
ceph k8sevents ceph .... show events generated from this
                         module
ceph k8sevents status .. show status of the tracker threads
                         and counts of tracked events

Events sent to kubernetes are sourced from the
a) clog: ceph healtchecks and admin commands
b) explicit checks: hosts, pools and OSD states are checked
   every 'n' seconds

Signed-off-by: Paul Cuzner <pcuzner@redhat.com>

ceph-conf: added --show-config-value to ceph-conf usage

--show-config-value is another way to lookup ceph.conf
values. Users may want to utilize --show-config-value instead of
--lookup because --show-config-value also shows global defaults.

Signed-off-by: James McClune <jmcclune@mcclunetechnologies.net>

doc: Document network performance monitoring

Signed-off-by: David Zafman <dzafman@redhat.com>

Merge pull request #29847 from iotcg/mon_doc

doc: set ceph_perf_msgr_server arguments

Reviewed-by: Greg Farnum <gfarnum@redhat.com>

Merge pull request #30136 from yuvalif/build_docs_on_fedora30

build-doc: allow building docs on fedora 30

Merge pull request #26755 from cbodley/wip-rgw-bilog-complete

rgw multisite: avoid writing bilog entries on PREPARE and CANCEL

Reviewed-by: J. Eric Ivancich <ivancich@redhat.com>

osd doc mon mgr: To milliseconds for config value, user input and threshold out

Signed-off-by: David Zafman <dzafman@redhat.com>

Merge pull request #29692 from mkogan1/wip-rgw-37615

rgw: fix cls_bucket_list_unordered() partial results

Reviewed-by: Matt Benjamin <mbenjamin@redhat.com>
Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: J. Eric Ivancich <ivancich@redhat.com>

mgr/dashboard: Refactoring

I came across a yoda case and a test case that didn't use 'done' while
using an async call.

Fixes: https://tracker.ceph.com/issues/40828
Signed-off-by: Stephan Müller <smueller@suse.com>

mgr/dashboard: Use onCancel on any modal event

The problem was, that if 'onCancel' was provided, it wasn't used in all
modal exit cases. Only for the close buttons (the X and the labeled
one), but not if you clicked outside the modal or hit escape.

'onCancel' is currently only used inside the user management form, where
the problem caused, that the submit button was not longer click able if
the modal was hidden through clicking outside of it or hitting escape.

Fixes: https://tracker.ceph.com/issues/40828
Signed-off-by: Stephan Müller <smueller@suse.com>

mgr/dashboard: Add tests for confirmation modal

Fixes: https://tracker.ceph.com/issues/40828
Signed-off-by: Stephan Müller <smueller@suse.com>

doc: releases: 14.2.3 dashboard note

The comment unfortunately got buried in a resolved blog

Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>

Merge pull request #29973 from theanalyst/doc/releases/14.2.3

doc: release notes for v14.2.3 nautilus

Reviewed-By: Nathan Cutler <nculter@suse.com>
Reviewed-By: Neha Ojha <nojha@redhat.com>
Reviewed-By: Josh Durgin <jdurgin@redhat.com>
Reviewed-By: Lenz Grimmer <lgrimmer@suse.com>

doc: add a final set of notable changes to 14.2.3

Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>

build-doc: allow building docs on fedora 30

Signed-off-by: Yuval Lifshitz <yuvalif@yahoo.com>

Merge pull request #29882 from cyx1231st/wip-seastar-msgr-test-lossless-crimson

crimson/net: bug fixes from v2 failover tests

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #30075 from rhcs-dashboard/fix-41600-master

mgr/dashboard: change warn_explicit to warn

Reviewed-by: Sebastian Wagner <swagner@suse.com>

Merge pull request #29929 from tspmelo/wip-ng-analytics

mgr/dashboard: Prevent angular of getting stuck during installation

Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Kiefer Chang <kiefer.chang@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>

Merge pull request #29557 from Devp00l/wip-41166

mgr/dashboard: Fix CephFS chart

Reviewed-by: Ni-Feng Chang <kiefer.chang@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>
Reviewed-by: Patrick Seidensal <pseidensal@suse.com>

Merge pull request #29996 from iliul/export-standby-metadata

mgr/prometheus: export standby mds metadata

Merge pull request #30122 from tchaikov/wip-40646

install-deps: revert 47d4351d

Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Brad Hubbard <bhubbard@redhat.com>

Merge pull request #30119 from neha-ojha/wip-41385

osd/MissingLoc, PeeringState: remove osd from missing loc in purge_strays()

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #30072 from wjwithagen/wjw-wip_split_run-rbd-unit-tests

test/librbd: allow parallel runs of run-rbd-unit-tests

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

install-deps: revert 47d4351d

this change reverts 47d4351d3ad2787ae7781d868608eb0e3c7797c7

see also 33dfffc8

Fixes: https://tracker.ceph.com/issues/40646
Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #30018 from xiexingguo/wip-38483

osd: prime splits/merges for any potential fabricated split/merge participant

Reviewed-by: Sage Weil <sage@redhat.com>

test/librbd: allow parallel runs of run-rbd-unit-tests

Running all tests sequential makes it the longest test of
`make check`, with each partial test taking around 500 sec.
Running 6 tests thus takes almost an hour.

Cut this down if ctest runs tests in parallel

Default behaviour of src/test/run-rbd-unit-tests.sh is kept:
  Without parameters the tests are run in sequence

To run unitttest_librbd with RBD_FEATURES, use `N` as parameter

Signed-off-by: Willem Jan Withagen <wjw@digiware.nl>

Merge pull request #29990 from aclamk/fix-doc-ceph_kvstore_tool-stats

doc/ceph-kvstore-tool: add description for 'stats' command

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

doc/man/ceph-kvstore-tool: fix typo

Signed-off-by: Josh Durgin <jdurgin@redhat.com>

osd/MissingLoc, PeeringState: remove osd from missing loc in purge_strays()

We should always try to keep osds in missing_loc consistent with peer_missing
and peer_info. When we remove an osd from peer_missing and peer_info, we
should also remove it from missing_loc during purging strays.

Signed-off-by: Neha Ojha <nojha@redhat.com>

Merge pull request #29639 from baixueyu/wip-crypto-bug

rgw: get barbican secret key request maybe return error code

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #29776 from cbodley/wip-rgw-beast-static-flat-buffer

rgw: increase beast parse buffer size to 64k

Reviewed-by: Abhishek Lekshmanan <abhishek@suse.com>

Merge pull request #29852 from cbodley/wip-41395

rgw: move bucket reshard checks out of write path

Reviewed-by: J. Eric Ivancich <ivancich@redhat.com>

Merge pull request #29856 from cbodley/wip-41412

rgw: RGWCoroutine::call(nullptr) sets retcode=0

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>

Merge pull request #29894 from markhpc/wip-rgw-reshard-json

rgw/rgw_reshard: Don't dump RGWBucketReshard JSON in process_single_logshard

Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: J. Eric Ivancich <ivancich@redhat.com>

Merge pull request #30088 from tchaikov/wip-40646

ceph.spec.in: use g++ >= 8.3.1-3.1

Reviewed-by: Alfredo Deza <adeza@redhat.com>
Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #29742 from dang/push

rgw: make sure modelines are correct for all files

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #29199 from tianshan/fix_gc_tag_remove

rgw: gc remove tag after all sub io finish

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #29628 from NancySu05/lihuanhuan

RGW: add bucket permission verify when copy obj

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #29943 from markhpc/wip-rgw-optimize-get_val2

rgw/rgw_op: Remove get_val from hotpath via legacy options

Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: Matt Benjamin <mbenjamin@redhat.com>

Merge pull request #29654 from Songweibin/wip-rbd-list-descendants

 librbd: skip stale child with non-existent pool for list descendants

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #30113 from tchaikov/wip-dmclock

dmclock:  pick up change to use specified C++ settings if any

Reviewed-by: Casey Bodley <cbodley@redhat.com>

dmclock:  pick up change to use specified C++ settings if any

see https://github.com/ceph/dmclock/pull/73

so dmclock won't add `-std=c++11` to CXX_FLAGS even if the parent project already
have `-std=c++17` in CXX_FLAGS.

Signed-off-by: Kefu Chai <kchai@redhat.com>

pybind/mgr: remove type conversions for ceph native options

We now return the correct Py types, just like we do for module options.

Signed-off-by: Sage Weil <sage@redhat.com>

mgr: return get_ceph_option result as typed Py object (not string)

We already return module options as the proper Py type; do the same for
native ceph options.

Signed-off-by: Sage Weil <sage@redhat.com>

mgr/dashboard: Prevent angular of getting stuck during installation

During npm packages installation Angular was getting stuck waiting for input.
Using "NG_CLI_ANALYTICS=false" will prevent that,

Signed-off-by: Tiago Melo <tmelo@suse.com>

Merge pull request #29613 from trociny/wip-41206

librbd: behave more gracefully when data pool removed

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge PR #29835 into master

* refs/pull/29835/head:
osd/PeeringState: fix wrong history of merge target

Reviewed-by: Sage Weil <sage@redhat.com>

mgr/dashboard: Empty datatable rendered before data has been fetched

Fixes: https://tracker.ceph.com/issues/41615
Signed-off-by: Volker Theile <vtheile@suse.com>

Merge pull request #30097 from jan--f/ceph-backport-cp-indiviudal-commits

ceph-backport.sh: cherry-pick individual commits

Reviewed-by: Nathan Cutler <ncutler@suse.com>

Merge pull request #30063 from Rjerk/wip-silence-unused-warning

rgw/services: silence -Wunused-variable warning

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #30065 from tchaikov/wip-rgw-test-wo-global

test: do not include unnecessary includes

Reviewed-by: Casey Bodley <cbodley@redhat.com>

ceph-backport.sh: cherry-pick individual commits

In git 2.23.0 (possibly earlier too) cherry-pick seems to interact with
rev-list and submodules in unexpected ways, where the cherry-pick of a
commit range (<sha1>..<sha1>) fails with "changes would be overwritten"
in a submodule. CP'ing individual commits avoids this.

Signed-off-by: Jan Fajerski <jfajerski@suse.com>

Merge pull request #30060 from c744402859/master

rgw: fixed "unrecognized arg" error when using "radosgw-admin zone rm".

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #30091 from joke-lee/doc-rgw-object-version

rgw: add --object-version in radosgw-admin help info

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #29957 from smithfarm/wip-ceph-backport-sh-improvements

script/ceph-backport.sh: wholesale refactor

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Mykola Golub <mgolub@suse.com>

doc/ceph-kvsore-tool: add description for 'stats' command

Signed-off-by: Adam Kupczyk <akupczyk@redhat.com>

Merge pull request #29665 from rishabh-d-dave/cv-remove-redundant-calls

ceph-volume: minor optimizations related to class Volumes's use

mgr/dashboard: fix cdEncode decorator is not working on class

Object.keys does not return all methods of a class.

Fixes: https://tracker.ceph.com/issues/41368
Signed-off-by: Kiefer Chang <kiefer.chang@suse.com>

ceph.spec.in: use g++ >= 8.3.1-3.1

since https://bugzilla.redhat.com/show_bug.cgi?id=1726630 has been
fixed. and we have devtoolset-8-gcc-c++-8.3.1-3.1.el7.x86_64.rpm.

no reason to stick with devtoolset-8-gcc-c++-8.2.1-3 anymore.

Fixes: https://tracker.ceph.com/issues/40646
Signed-off-by: Kefu Chai <kchai@redhat.com>

rgw: add --object-version in radosgw-admin help info

Signed-off-by: yuliyang <yuliyang@cmss.chinamobile.com>

Merge pull request #30086 from uniqueyehu/fix-complie-errors-dpdk

msg/async/dpdk: fix complie errors from fix FTBFS

Reviewed-by: Kefu Chai <kchai@redhat.com>

msg/async/dpdk: fix complie errors from fix FTBFS

Signed-off-by: yehu <yehu5@huawei.com>

cmake: SKIP_RPATH if RPATH is not necessary

some executables like ceph_test_mon_memory_target do not link against
libraries built from source tree, like librados and libceph-common. so
cmake does not set RPATH for them. hence cmake complains like:

before this change, `CMAKE_INSTALL_RPATH` is set globally. so cmake is
asked to rewrite the RPATH for all installed targets. but this is not
needed. as some executables do not link against libceph-common. hence,
cmake complains when installing them, like:

CMake Error at src/test/mon/cmake_install.cmake:90 (file):
  file RPATH_CHANGE could not write new RPATH:
    /usr/lib64/ceph
   to the file:
    /home/abuild/rpmbuild/BUILDROOT/ceph-15.0.0-4347.g85a07b9.x86_64/usr/bin/ceph_test_log_rss_usage
   No valid ELF RPATH or RUNPATH entry exists in the file;

after this change, `SKIP_RPATH` is set for those executables which do
not link against any libraries created from ceph source tree. so we can
avoid setting the RPATH for these executables when `make install`.

the same applies to libceph-common.

Fixes: https://tracker.ceph.com/issues/41524
Signed-off-by: Kefu Chai <kchai@redhat.com>

osd/PrimaryLogPG: update oi.size on write op implicitly truncating object up

See "BlueStore::_do_truncate", bluestore will reset ondisk object size
unconditionally to track truncated size. Hence we must adjust logical
size (and usage) accordingly to match the specified metrics.

Fixes: https://tracker.ceph.com/issues/41601
Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

Merge pull request #30010 from wjwithagen/wjw-fix-rbd-ggate-postfork

tools/rbd-ggate: close log before running postfork

Reviewed-by: Mykola Golub <mgolub@suse.com>

mgr/dashboard: change warn_explicit to warn

Fixes: https://tracker.ceph.com/issues/41600
Signed-off-by: Ernesto Puerta <epuertat@redhat.com>

Merge pull request #29923 from xiexingguo/wip-bump-decode-version

osd/osd_types: bump up some encoding versions

Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: yanjun <yan.jun8@zte.com.cn>

Merge pull request #30066 from uniqueyehu/fix-compile-error-dpdk

async/dpdk: fix compile errors from ceph::mutex update

Reviewed-by: Kefu Chai <kchai@redhat.com>

test: do not include unnecessary includes

Signed-off-by: Kefu Chai <kchai@redhat.com>

librbd: allow remove snapshot with child from non-existent pool

Signed-off-by: songweibin <song.weibin@zte.com.cn>

rgw: fixed "unrecognized arg" error when using "radosgw-admin zone rm".

Signed-off-by: Hongang Chen <c744402859@gmail.com>
Author:    root <root@devel.localdomain>
Date:      Mon Sep 2 11:23:45 2019 +0800

Merge pull request #30034 from tchaikov/wip-install-tox-deps-from-wheelhouse

mgr: install tox deps from wheelhouse

Reviewed-by: Sebastian Wagner <swagner@suse.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>

async/dpdk: fix compile errors from ceph::mutex update

Signed-off-by: yehu <yehu5@huawei.com>

cmake: only link against necessary libs

some executables like ceph_test_mon_memory_target does not link against
gtest or gmock, so no need to link agains them.

Signed-off-by: Kefu Chai <kchai@redhat.com>

tools/rbd-ggate: close log before running postfork

Otherwise the assert in Log:open() will trigger since
all data is kept during the fork and also the `is_started` flag

Fixes: https://tracker.ceph.com/issues/41592
Signed-off-by: Willem Jan Withagen <wjw@digiware.nl>