Merge PR #38083 into master

* refs/pull/38083/head:
mds: don't access mds->mdsmap without holding mds_lock

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #38315 from badone/wip-mon-scrub-testing

Test mon scrub and exercise mon scrub error injectors

Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge pull request #38127 from tchaikov/wip-crimson-client-request-wait

crimson/osd: do not wait for sent epoch

Reviewed-by: Xuehan Xu <xxhdx1985126@gmail.com>
Reviewed-by: Samuel Just <sjust@redhat.com>

Merge pull request #38203 from tchaikov/wip-doc-build-releases.yaml

doc/conf: generate releases.json in conf.py and cleanups

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #38239 from tchaikov/wip-48320

mon/OSDMonitor: drop osd_epoch of manually marked out osd

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #38281 from tchaikov/wip-mgr-no-kv-options

mgr: do not migrate conf from config-key store to new-style conf

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #38305 from tchaikov/wip-crimson-finally

crimson: fix finally and erroratorize PG::with_*_obc()

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
Reviewed-by: Samuel Just <sjust@redhat.com>

Merge PR #38248 into master

* refs/pull/38248/head:
doc: Adding missing Ceph Foundation board members

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #38313 from wjwithagen/wjw-fix-src-HttpClient.h

librbd: fix Clang warnings

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Adam C. Emerson <aemerson@redhat.com>

Merge pull request #38113 from rhcs-dashboard/login-name-cleanup

mgr/dashboard: Change the text 'Login' to 'Log in'

Reviewed-by: Aashish Sharma <aasharma@redhat.com>
Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Tatjana Dehler <tdehler@suse.com>

Merge pull request #38209 from votdev/issue_44404_placement_column

mgr/dashboard: display placement column in service table

Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Kiefer Chang <kiefer.chang@suse.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
Reviewed-by: Tatjana Dehler <tdehler@suse.com>

Merge pull request #38163 from bk201/wip-48245

mgr/dashboard: fix exception in get_smart_data_by_daemon

Reviewed-by: Laura Paduano <lpaduano@suse.com>
Reviewed-by: p-se <pseidensal@suse.com>

Merge pull request #38330 from tchaikov/wip-librados-no-reuse

include/rados/librados.h: note that {read,write}_op cannot be reused

Reviewed-by: Brad Hubbard <bhubbard@redhat.com>

Merge pull request #37994 from s0nea/wip-dashboard-button-panel

mgr/dashboard: refactor the order of buttons shown in forms

Reviewed-by: Courtney Caldwell <ccaldwel@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>

include/rados/librados.h: note that {read,write}_op cannot be reused

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #37503 from TRYTOBE8TME/wip-rgw-sts-web-identity

Wip rgw sts web identity

qa: STS Web Identity task files

This commit comprises of 3 things:
1) Keycloak task (to bring up keycloak in teuthology inorder to execute Assume_Role_With_Web_Identity tests).
2) The required changes for this in s3tests.py which consists of writing some parameters to configuration file.
3) The yaml files which executes these tasks.

Signed-off-by: Kalpesh Pandya <kapandya@redhat.com>

Merge pull request #38277 from b-ranto/wip-prom-fixes

mgr/prometheus: Make module more stable

Reviewed-by: Mykola Golub <mgolub@suse.com>
Reviewed-by: Sebastian Wagner <swagner@suse.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #38286 from votdev/issue_48360_disable_tlsv1

mgr/dashboard: Disable TLS 1.0 and 1.1

Reviewed-by: Reviewed-by: Kiefer Chang <kiefer.chang@suse.com>
Reviewed-by: Tatjana Dehler <tdehler@suse.com>
Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>

Merge pull request #38321 from rhcs-dashboard/add-dashboard-master

mailmap: add dashboard contributors

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge PR #38221 into master

* refs/pull/38221/head:
.github: add mon and mgr labelers

Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>

mailmap: add dashboard contributors

Signed-off-by: Ernesto Puerta <epuertat@redhat.com>

mgr/dashboard: Change the text 'Login' to 'Log in'

Fixes: https://tracker.ceph.com/issues/48253
Signed-off-by: Nizamudeen A <nia@redhat.com>

Merge pull request #38304 from sebastian-philipp/cephadm-inventory-cache-docstring

mgr/cephadm: add docstring describing the HostCache

Reviewed-by: Juan Miguel Olmo Martínez <jolmomar@redhat.com>

Merge pull request #38078 from zdover23/wip-doc-docs-style-guide-prompts-nov-2020

doc/start: update prompts in documenting-ceph.rst

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #38303 from aclamk/wip-bs-types-test-leak

os/bluestore: Fixed memory leak in test

Reviewed-by: Igor Fedotov <ifedotov@suse.com>

Merge pull request #38058 from lxbsz/rbd_expand11

global: reexpand the config meta just after the fork() is done

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #38240 from ifed01/wip-ifed-is-valid-io-log

blk: log is_valid_io() parameters when unsuccessful.

Reviewed-by: Adam Kupczyk <akupczyk@redhat.com>

Merge pull request #38262 from tchaikov/wip-librbd-gcc-10

neorados: use lambda instead of std::mem_fn()

Reviewed-by: Adam C. Emerson <aemerson@redhat.com>
Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #38278 from votdev/fix_cephadm_typo

pybind/mgr/cephadm: Fix various typos

Reviewed-by: Sebastian Wagner <swagner@suse.com>

Merge pull request #38211 from ricardoasmarques/sort-host-device-ls

mgr/orchestrator: Sort 'ceph orch device ls' by host

Reviewed-by: Thomas Bechtold <tbechtold@suse.com>
Reviewed-by: Sebastian Wagner <swagner@suse.com>

Merge pull request #38138 from batrick/centos-8.2

qa: update centos release to 8.2

Reviewed-by: Yuri Weinstein <yweinste@redhat.com>

Merge pull request #38224 from Rethan/fix_pg_dump_json

mgr: fix "ceph pg dump_json/dump_pools_json" format error

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #38206 from dvanders/dvanders_upmap

mgr/balancer: fix available pgs sent to calc_pg_upmaps

Reviewed-by: David Zafman <dzafman@redhat.com>

Merge pull request #38264 from tchaikov/wip-mgr-prometheus-cleanups

pybind/mgr/prometheus: cleanups

Reviewed-by: Jan Fajerski <jfajerski@suse.com>

Merge pull request #38241 from Rethan/update_url_from_master_to_lastest

doc/dev: use http://docs.ceph.com/en/latest/ instead of /docs/master/ for docs

Reviewed-by: Laura Paduano <lpaduano@suse.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Willem Jan Withagen <wjw@digiware.nl>

Merge pull request #38306 from wjwithagen/wjw-fix-uring-functype

blk: fix parameters for non native uring

Reviewed-by: Kefu Chai <kchai@redhat.com>

qa/suites/rados/monthrash: Exercise mon scrub error injectors

Signed-off-by: Brad Hubbard <bhubbard@redhat.com>

qa/config/rados.yaml: Test mon scrub

Signed-off-by: Brad Hubbard <bhubbard@redhat.com>

librbd: remove unwanted std::move

Clang argues:
```
/home/jenkins/workspace/ceph-master-compile/src/librbd/migration/HttpClient.cc:463:16: warning: moving a temporary object prevents copy elision [-Wpessimizing-move]
    response = std::move(m_parser->release());
               ^
/home/jenkins/workspace/ceph-master-compile/src/librbd/migration/HttpClient.cc:463:16: note: remove std::move call here
    response = std::move(m_parser->release());
               ^~~~~~~~~~                   ~
1 warning generated.

```

Signed-off-by: Willem Jan Withagen <wjw@digiware.nl>

librbd: remove double declaration

Signed-off-by: Willem Jan Withagen <wjw@digiware.nl>

mgr/dashboard: Disable TLS 1.0 and 1.1

Disable these TLS versions because of security issues.

Fixes: https://tracker.ceph.com/issues/48360
Signed-off-by: Volker Theile <vtheile@suse.com>

Merge pull request #38297 from badone/wip-make-sure--e-actually-works

run-make-check.sh: Make sure a build failure will exit

Reviewed-by: Kefu Chai <kchai@redhat.com>

blk: fix parameters for non native uring

The parameterlist used when uring is availble needs
to be equal to the case where no uring is available.

fixes: https://github.com/ceph/ceph/pull/38257
Signed-off-by: Willem Jan Withagen <wjw@digiware.nl>

crimson/osd: wait for min_epoch instead of map_epoch

osdmap_epoch is the latest osdmap of the client, it is not the minimum
osdmap epoch required to serve the request from the client with this
osdmap.

as a matter of fact, `MOSDOp` inherits from `MOSDFastDispatchOp`.
the `MOSDFastDispatchOp::get_min_epoch()` returns `get_map_epoch()`, and
`MOSDOp` does not override `get_min_epoch()`.

`ClientRequest` is designed to handle `MOSDOp`. so, there is no need
to wait for `m->get_map_epoch()` osdmap before moving further, and more
importantly, we should wait until PG is updated with min_epoch osdmap.

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #38265 from Rethan/doc-prometheus-config-key

doc/mgr/prometheus: update 'config-key set' to 'config set' for consistency

Reviewed-by: Mykola Golub <mgolub@suse.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

crimson/osd: errororize PG::with_*_obc methods()

they all use `seastar::with_lock()` to ensure the obc lock is released
in the error handling path. but we cannot assume that
`seastar::with_lock()` always return `seastar::future<>`, so propogate
the error returned by the function passed to `with_lock()` using
errorator.

Signed-off-by: Kefu Chai <kchai@redhat.com>

crimson/common/errorator: fix finally

We need to specify the return type here to ensure we don't lose
the errorator properties.

Signed-off-by: Samuel Just <sjust@redhat.com>

Merge pull request #38257 from lnsyyj/wip-iouring-poll

blk/kernel: expose IORING_SETUP_{IOPOLL,SQPOLL} as options

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #38263 from tchaikov/wip-boost-1.74-asio

src,cmake: define BOOST_ASIO_USE_TS_EXECUTOR_AS_DEFAULT for Boost.Asio users and cleanups

Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: Jason Dillaman <dillaman@redhat.com>
Reviewed-by: Adam C. Emerson <aemerson@redhat.com>

mgr/cephadm: add docstring describing the HostCache

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>

os/bluestore: Fixed memory leak in test

This is only unittest issue. This (potential) problems do not appear is BlueStore itself.

Signed-off-by: Adam Kupczyk <akupczyk@redhat.com>

Merge PR #38149 into master

* refs/pull/38149/head:
ceph-volume inventory: make libstoragemgmt data retrieval optional

Reviewed-by: Brad Hubbard <bhubbard@redhat.com>
Reviewed-by: Sebastian Wagner <swagner@suse.com>
Reviewed-by: Dan Mick <dmick@redhat.com>

doc/mgr/prometheus: update 'config-key set' to 'config set' for consistency

Elsewere in mgr/prometheus use the 'config set' way, so update the only 'config-key set'
to make it consistent.

Signed-off-by: haoyixing <haoyixing@kuaishou.com>

run-make-check.sh: Make sure a build failure will exit

We 'set -e' but that is ignored because 'build tests' is executed in a
'&&' list (see 'man set') so move the echo to the following line.

Follow-up to 03ff2146f95

Signed-off-by: Brad Hubbard <bhubbard@redhat.com>

Merge pull request #38292 from badone/wip-dont-run-tests-if-build-fails

run-make-check.sh: Don't run tests if build fails

Reviewed-by: Kefu Chai <kchai@redhat.com>

run-make-check.sh: Don't run tests if build fails

When run-make was taken out we lost the 'set -e' call and therefore
continue after an error.

Signed-off-by: Brad Hubbard <bhubbard@redhat.com>

Merge PR #38121 into master

* refs/pull/38121/head:
client: do not unset the client_debug_inject_tick_delay

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge PR #38109 into master

* refs/pull/38109/head:
mds: dir->mark_new() should together with dir->mark_dirty()

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge PR #38031 into master

* refs/pull/38031/head:
client: check rdonly file handle on truncate
test/libcephfs: test truncate on rdonly fd

Reviewed-by: Jeff Layton <jlayton@redhat.com>

Merge PR #38104 into master

* refs/pull/38104/head:
client: dump which fs is used by client for multiple-fs

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

ceph-volume inventory: make libstoragemgmt data retrieval optional

Default to not retrieving libstoragemgmt data since it seems this can
cause serious issues on older hardware. Safest way is to only retrieve
lsm data when the user opts in..

Fixes: https://tracker.ceph.com/issues/48270
Signed-off-by: Jan Fajerski <jfajerski@suse.com>

mgr/prometheus: use threading.Event instead of sleep

This allows us to avoid waiting for the sleep to finish when waiting for
the thread to finish.

Signed-off-by: Boris Ranto <branto@redhat.com>

mgr/prometheus: Log collection issues

Log any issues encountered during the data collection and continue to
collect the data anyway (after a sleep).

Signed-off-by: Boris Ranto <branto@redhat.com>

Merge pull request #38068 from dillaman/wip-librbd-migration-4

librbd: S3-based migration source

Reviewed-by: Mykola Golub <mgolub@suse.com>

mgr/prometheus: Use mgr.release_name for always on modules

The host_version is not populated properly in the early stages of ceph
mgr start up process. We can use mgr.release_name instead. It is more
stable and it provides the data even if mgr_map does not contain the
versions, yet.

Signed-off-by: Boris Ranto <branto@redhat.com>
meh

mgr: do not migrate conf from config-key store to new-style conf

since all module options are using the new-style config framework.
the migration is offered for the use case of upgrade from luminous to mimic,
since pacific can only be upgraded from octopus. the mimic monitors are alreay
able to populate the configurations to mgr, not to mention the octopus
monitors, so there is no need to migrate the options stored in config-key
store anymore.

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #38195 from anthonyeleven/anthonyeleven/docs-14730

doc/rados/troubleshooting: description of upgrade procedure - stopping/starting OSDs

Reviewed-by: Zac Dover <zac.dover@gmail.com>

Merge pull request #38276 from tchaikov/wip-labeler

github/labeler: add "rgw" label to rgw related changes

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #38091 from anthonyeleven/anthonyeleven/docs-46367

doc/dev/developer_guide: verb disagreement on "Testing: Unit Tests" page

Reviewed-by: Zac Dover <zac.dover@gmail.com>

Merge pull request #38143 from anthonyeleven/anthonyeleven/docs-46364

doc/dev/developer_guide: What Is Merged and When?" could be less colloquial

Reviewed-by: Zac Dover <zac.dover@gmail.com>

mgr: Fix various typos

Signed-off-by: Volker Theile <vtheile@suse.com>

Merge pull request #38079 from anthonyeleven/anthonyeleven/docs-24143

doc/rados/configuration/common.rst:  enhance the running multiple clusters section

Reviewed-by: Zac Dover <zac.dover@gmail.com>

mgr/prometheus: Clean up collection thread

We need to clean up the metrics collection thread.

Signed-off-by: Boris Ranto <branto@redhat.com>

Merge pull request #38088 from anthonyeleven/anthonyeleven/docs-47627

doc/mgr: document dashboard configuration unsetting

Reviewed-by: Zac Dover <zac.dover@gmail.com>

Merge PR #38156 into master

* refs/pull/38156/head:
ceph-volume batch: reject partitions in argparser

Reviewed-by: Dimitri Savineau <dsavinea@redhat.com>
Reviewed-by: Guillaume Abrioux <gabrioux@redhat.com>

github/labeler: label src/test/** with "tests"

Signed-off-by: Kefu Chai <kchai@redhat.com>

github/labeler: add "rgw" label to rgw related changes

Signed-off-by: Kefu Chai <kchai@redhat.com>

pybind/mgr/prometheus: use get_module_option() wrapper

for better readability

Signed-off-by: Kefu Chai <kchai@redhat.com>

pybind/mgr/prometheus: catch ImportError

silences following warning from flake8

E722 do not use bare 'except'

Signed-off-by: Kefu Chai <kchai@redhat.com>

pybind/mgr/mgr_module: have two lines before class definition

* have two lines before class definition, to silence E302 warning from flake8
* have single line before function definition, to silence E303 warning from flake8
* remove trailing space, to silence W291 warning from flake8

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #38274 from dillaman/wip-github-labeler-rbd

github/labeler: add rbd label to src/tools/rbd_[ggate/wnbd]

Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

github/labeler: add rbd label to src/tools/rbd_[ggate/wnbd]

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #38213 from petrutlucian94/rbd_exit

rbd: minor rbd-wnbd improvements

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge branch 'master' into anthonyeleven/docs-46367

.github: add mon and mgr labelers

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #38254 from neha-ojha/wip-48299

doc/releases/nautilus.rst: add release notes for 14.2.15

Reviewed-by: David Galloway <dgallowa@redhat.com>
Reviewed-by: David Galloway <dgallowa@redhat.com>
Reviewed-by: Yuri Weinstein <yweinste@redhat.com>

Merge pull request #38144 from anthonyeleven/anthonyeleven/docs-46365

doc/dev/developer_guide: prefer Latinate verbs to English phrasal verbs

Reviewed-by: Zac Dover <zac.dover@gmail.com>

Merge pull request #38260 from bk201/wip-48338

mgr/dashboard: fix tox py3 environment error

Reviewed-by: Laura Paduano <lpaduano@suse.com>
Reviewed-by: Tatjana Dehler <tdehler@suse.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

doc/mgr: document dashboard configuration unsetting

Fixes: https://tracker.ceph.com/issues/47627
Signed-off-by: Anthony D'Atri <anthony.datri@gmail.com>

Merge pull request #38253 from cbodley/wip-rgw-gc-disable-defer

rgw: temporarily disable calls to defer_gc() in RGWGetObj

Reviewed-by: Matt Benjamin <mbenjamin@redhat.com>
Reviewed-by: J. Eric Ivancich <ivancich@redhat.com>

doc/releases/nautilus.rst: add release notes for 14.2.15

Signed-off-by: Neha Ojha <nojha@redhat.com>

Merge pull request #38142 from theanalyst/doc/releases/v14.2.14

doc: release notes for octopus & nautilus backports

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Yuri Weinstein <yweinste@redhat.com>
Reviewed-by: Ilya Dryomov <idryomov@gmail.com>

doc: Adding missing Ceph Foundation board members

These companies have been members since the beginning.
Signed-off-by: Mike Perez <thingee@gmail.com>

Merge pull request #38247 from dillaman/wip-librbd-test-rename

test/librbd: rename tests to match proper namespaces

Reviewed-by: Mykola Golub <mgolub@suse.com>

doc: releases: v15.2.6 Octopus release notes

Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>
Co-authored-by: Ilya Dryomov <idryomov@gmail.com>

doc: releases: v14.2.14 Nautilus release notes

Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>
Co-authored-by: Ilya Dryomov <idryomov@gmail.com>

blk/kernel: expose IORING_SETUP_{IOPOLL,SQPOLL} as options

Signed-off-by: JiangYu <lnsyyj@hotmail.com>

Merge pull request #38261 from tchaikov/wip-45919

pybind/mgr/dashboard/run-backend-api-tests: print more log from mgr

Reviewed-by: Alfonso Martínez <almartin@redhat.com>

Merge pull request #38217 from ideepika/wip-fix-lua-failure

cmake: require Lua 5.3 and up

Reviewed-by: Yuval Lifshitz <ylifshit@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>