osd: Don't clear m_planned_scrub until scrub finishes successfully

Fixes: https://tracker.ceph.com/issues/48984
Signed-off-by: David Zafman <dzafman@redhat.com>

Merge pull request #39456 from cbodley/wip-qa-rgw-multisite-upgrade

qa/rgw: run multisite tests with mixed cluster versions

Reviewed-by: Yuri Weinstein <yweins@redhat.com>

Merge pull request #39363 from rhcs-dashboard/fix-orch-host-e2e

mgr/dashboard: check .badge instead of text for expected label

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Aashish Sharma <aasharma@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Avan Thakkar <athakkar@redhat.com>

Merge PR #38610 into master

* refs/pull/38610/head:
ceph-volume: pass --filter-for-batch from drive-group subcommand

Reviewed-by: Guillaume Abrioux <gabrioux@redhat.com>
Reviewed-by: Rishabh Dave <ridave@redhat.com>

qa/rgw: add pacific-x upgrade suite for multisite

Signed-off-by: Casey Bodley <cbodley@redhat.com>

qa/rgw: add octopus-x upgrade suite for multisite

Signed-off-by: Casey Bodley <cbodley@redhat.com>

qa/rgw: add 'branch' option to rgw-multisite-tests task

Fixes: https://tracker.ceph.com/issues/49330
Signed-off-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #39467 from sebastian-philipp/doc-cephadm-mon-quorum

doc/cephadm: Restoring the MON quorum

Reviewed-by: Juan Miguel Olmo Martínez <jolmomar@redhat.com>

Merge pull request #39496 from liewegas/cephadm-refresh-health-update

mgr/cephadm: fix host refresh

Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>

Merge pull request #39372 from rhcs-dashboard/avoid-document-write

mgr/dashboard: avoid using document.write()

Reviewed-by: Aashish Sharma <aasharma@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>

Merge pull request #39462 from rhcs-dashboard/fix-alerts-mtuMismatch

mgr/dashboard: fix MTU Mismatch alert

Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>

mgr/dashboard: check .badge instead of text for expected label

this change fixes a regression introduced by
8c5e31ec1a13bc53394eb2cb6880d74db169fac4 which broke the 01-hosts.e2e-spec.ts test
driven by test_dashboard_e2e.sh

Fixes: https://tracker.ceph.com/issues/49205
Signed-off-by: Nizamudeen A <nia@redhat.com>

Merge pull request #39464 from sebastian-philipp/doc-cephadm-unmanaged

doc/cephadm: Disable automatic deployment of daemons

Reviewed-by: Juan Miguel Olmo Martínez <jolmomar@redhat.com>
Reviewed-by: Zac Dover <zac.dover@gmail.com>

Merge pull request #39459 from tchaikov/wip-doc-api-command

doc: wrap example command

Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>

Merge PR #39490 into master

* refs/pull/39490/head:
cephadm: fix selinux mount mis-indent

Reviewed-by: Boris Ranto <branto@redhat.com>
Reviewed-by: Sebastian Wagner <swagner@suse.com>
Reviewed-by: Juan Miguel Olmo <jolmomar@redhat.com>

Merge pull request #39497 from zdover23/wip-doc-dev-teuthology-intro-workbench-removal-16-Feb-2021

doc/dev: removing workbench docs (part 2)

Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge pull request #39499 from zdover23/wip-doc-dev-teuthology-teuthology-describe-tests-repair-2021-Feb-16

doc/dev: correct "teuthology-describe-tests"

Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge pull request #38030 from p-se/prom-alert-package-drops-leeway

mgr/dashboard: prometheus alerting: add some leeway for package drops and errors

Reviewed-by: Stephan Müller <smueller@suse.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>

Merge pull request #39422 from kamoltat/wip-ksirivad-autoscaler-release-notes

PendingReleaseNotes: mgr/pg_autoscaler

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>

doc/dev: correct "teuthology-describe-tests"

This tiny commit does this:

s/teuthology-describe-tests/teuthology-describe/

The name of this feature was changed from "teuthology-
describe-tests" to "teuthology-describe", and this
commit brings up to date the title of the section
detailing it.

Signed-off-by: Zac Dover <zac.dover@gmail.com>

doc/dev: removing workbench docs (part 2)

This removes a paragraph in the section "How
Integration Tests Are Run" that described
ceph-workbench. ceph-workbench is deprecated,
so that paragraph is now gone.

Signed-off-by: Zac Dover <zac.dover@gmail.com>

mgr/cephadm: fix host refresh

Fixes: 01f60cf4e0a751c314120c02956d4ff941eb71b4
Signed-off-by: Sage Weil <sage@newdream.net>

Merge pull request #39323 from sebastian-philipp/cephadm-simplify-cephadmservice-config

mgr/cephadm: Add config() to CephadmService

Reviewed-by: Juan Miguel Olmo Martínez <jolmomar@redhat.com>

Merge pull request #39174 from zdover23/wip-doc-37949-grammar-edit

doc/dev: Teuthology guide PR#37949 grammar edit

Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge pull request #39392 from tschoonj/amqp-ssl

rgw: add support for SSL encrypted AMQP connections

Reviewed-by: Yuval Lifshitz <ylifshit@redhat.com>

doc/cephadm: Disable automatic deployment of daemons

Fixes: https://tracker.ceph.com/issues/45767
Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>

doc/dev: Teuthology guide PR#37949 grammar edit

This PR improves the wording of the technical
information added to the documentation in PR#37949.
This is the second is a series of two PRs, which series
is dedicated to testing a workflow wherein developers
add technical information to the documentation and then
technical writers improve its presentation.

Signed-off-by: Zac Dover <zac.dover@gmail.com>

Merge pull request #39430 from liewegas/bug-48142

qa/suites/rados/cephadm/upgrade: change starting version by distro

Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>

Merge pull request #39493 from rzarzynski/wip-crimson-cond-mon-config

crimson/osd: fetch configuration from monitors conditionally.

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #39492 from zdover23/wip-doc-dev-teuthology-workflow-cephfs-shell-typo-15-Feb-2021

doc/dev: cephfs-shell typo repair

Reviewed-by: Kefu Chai <kchai@redhat.com>

mgr/dashboard: prometheus alerting: add some leeway for package drops and errors (1%)

Fixes: https://tracker.ceph.com/issues/48201
Signed-off-by: Patrick Seidensal <pseidensal@suse.com>

crimson/osd: fetch configuration from monitors conditionally.

Before this change we were always attempting to fetch the config
from monitors, even if `--no-mon-config` had been passed. This
was the reason for failures at teuthology like:
http://pulpito.front.sepia.ceph.com/rzarzynski-2021-02-15_21:09:02-rados-master-distro-basic-smithi/5885250

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

doc/cephadm: Restoring the MON quorum

Fixes: https://tracker.ceph.com/issues/49214
Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>

Merge pull request #39473 from sebastian-philipp/doc-cephamd-trouble-events

doc/cephadm: Per-service and per-daemon events

Reviewed-by: Juan Miguel Olmo Martínez <jolmomar@redhat.com>
Reviewed-by: Patrick Seidensal <pseidensal@suse.com>

mgr/cephadm: _add_daemon: remove config_func

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>

mgr/cephadm: Add config() to CephadmService

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>

Merge pull request #39465 from sebastian-philipp/install-deps-rm-libboost1.72.list

install-deps: Cope with nonexistent /etc/apt/sources.list.d/ceph-libboost1.72.list

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #39249 from varshar16/wip-fix-vstart-ganesha-error

vstart: fix ganesha cluster id and rados url errors

Reviewed-by: Jeff Layton <jlayton@redhat.com>
Reviewed-by: Laura Paduano <lpaduano@suse.com>

doc/dev: cephfs-shell typo repair

This corrects the formatting for a mention of
cephfs-shell. Previously, it was not properly
formatted. This commit adds two backticks before
the string "cephfs-shell", and adds to backticks
after it.

Signed-off-by: Zac Dover <zac.dover@gmail.com>

cephadm: fix selinux mount mis-indent

We want to map this path even if we had to create the empty directory.

Signed-off-by: Sage Weil <sage@newdream.net>

Merge PR #39150 into master

* refs/pull/39150/head:
qa/suites/rados/cephadm/thrash: add cephadm thrashing test

Reviewed-by: Sebastian Wagner <swagner@suse.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge PR #39482 into master

* refs/pull/39482/head:
Revert "Merge pull request #37764 from mgfritch/cephadm-no-container-init"

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge PR #39424 into master

* refs/pull/39424/head:
cephadm: Mounting <empty> folder for selinux only if it is needed

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Boris Ranto <branto@redhat.com>
Reviewed-by: Sebastian Wagner <swagner@suse.com>
Reviewed-by: Ken Dreyer <kdreyer@redhat.com>

Merge PR #39185 into master

* refs/pull/39185/head:
os/bluestore: Add option to check BlueFS reads

Reviewed-by: Igor Fedotov <ifedotov@suse.com>

Merge PR #39281 into master

* refs/pull/39281/head:
mgr/MgrClient; less copying of mgr report struct

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge PR #39282 into master

* refs/pull/39282/head:
osdc/Objecter: remove pool_full_try interface
librados: set FULL_TRY flag on ioctx, not objecter
librados/IoCtxImpl: pass extra_op_flags everywhere

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge PR #39283 into master

* refs/pull/39283/head:
qa/suites/rados/dashboard: whitelist TELEMETRY_CHANGED

Reviewed-by: Laura Paduano <lpaduano@suse.com>
Reviewed-by: Yaarit Hatuka <yaarithatuka@gmail.com>

Merge PR #39381 into master

* refs/pull/39381/head:
mgr/cephadm: based sleep interval on configured intervals/timeouts

Reviewed-by: Sebastian Wagner <swagner@suse.com>

cephadm: Mounting <empty> folder for selinux only if it is needed

There are OSs without </usr/share/empty> folder. And selinux can be
enabled or not.

Signed-off-by: Juan Miguel Olmo Martínez <jolmomar@redhat.com>

Revert "Merge pull request #37764 from mgfritch/cephadm-no-container-init"

This reverts commit f635555fe76322336a1e0e8cc3c199562642ad0e, reversing
changes made to d4d3d17b23fcf6d8808a96b992f56924f495b174.

This PR seems to be (indirectly?) responsible for
  https://tracker.ceph.com/issues/49237

Also, it was causing the rados.py task's follow-up step to wait
for snap trimming to fail: it would time out a 'ceph osd dump --format=json'
command.  :/

Signed-off-by: Sage Weil <sage@newdream.net>

Merge pull request #39461 from yuvalif/wip-yuval-fix-49261

rgw/test: allow for duplicate events in pubsub

Reviewed-by: Casey Bodley <cbodley@redhat.com>

doc/cephadm: Per-service and per-daemon events

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>

Merge pull request #39386 from zdover23/wip-doc-dev-triggering-tests-rewrite-10-Feb-2021

doc/dev: teuthology: improve triggering tests

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

doc/dev: teuthology: improve triggering tests

- This PR cleans up the language in the "Triggering
  Tests" section of the "Workflow" chapter of the
  "Tests: Integration Tests" (Teuthology) guide.
- Added bash prompts.

Signed-off-by: Zac Dover <zac.dover@gmail.com>

Merge pull request #39407 from zdover23/wip-doc-dev-testing-qa-changes-rewrite-11-Feb-2021

doc/dev: rewrite 'testing qa changes'

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

doc/dev: rewrite 'testing qa changes'

This PR rewrites the "Testing QA Changes (Without
Re-Building Binaries)" section of the Developer
Guide. I also tried to unbury the lede.

Signed-off-by: Zac Dover <zac.dover@gmail.com>

Merge PR #38562 into master

* refs/pull/38562/head:
Update batch.py

Reviewed-by: Jan Fajerski <jfajerski@suse.com>
Reviewed-by: Guillaume Abrioux <gabrioux@redhat.com>
Reviewed-by: Rishabh Dave <ridave@redhat.com>
Reviewed-by: Dimitri Savineau <dsavinea@redhat.com>

mgr/dashboard: avoid using document.write()

Fixes: https://tracker.ceph.com/issues/49220
Signed-off-by: Avan Thakkar <athakkar@redhat.com>

install-deps: Cope with nonexistent /etc/apt/sources.list.d/ceph-libboost1.72.list

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>

rgw: add support for SSL encrypted AMQP connections

Fixes: https://tracker.ceph.com/issues/42902
Signed-off-by: Tom Schoonjans <Tom.Schoonjans@rfi.ac.uk>

Merge pull request #39458 from sebastian-philipp/cephadm-spec-store-fix-race

mgr/cephadm: SpecStore: use d-under accessor methods

Reviewed-by: Adam King <adking@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

mgr/dashboard:fix MTU Mismatch alert

This PR intends to fix the expression used for MTU Mismatch alert in prometheus

Signed-off-by: Aashish Sharma <aasharma@redhat.com>

mgr/cephadm: SpecStore: use d-under accessor methods

This fixes a conflict between:

* 1fcdad17685a8884135a2cda0cdfe3525e325474
* 2c7f899124207d0d4866e16d9eb5609777ea22e8

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>

rgw/test: allow for duplicate events in pubsub

when tested via radosgw-admin.
also, add verification for the content of the events

Fixes: https://tracker.ceph.com/issues/49261
Signed-off-by: Yuval Lifshitz <ylifshit@redhat.com>

Merge pull request #39379 from multi-arch/master

ceph.spec.in: Enable tcmalloc and lttng on IBM Power and Z

Reviewed-by: Ken Dreyer <kdreyer@redhat.com>
Reviewed-by: Luke Browning <lbrownin@redhat.com>

Merge pull request #39389 from tchaikov/wip-cmake-ccache

cmake: s/CCACHE_FOUND/CCACHE_EXECUTABLE/

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

doc: wrap example command

otherwise the field name in the field list would be rendered with a
small width.

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #39411 from tchaikov/wip-rtd-theme

doc: cleanups and group tell commands

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #39454 from rzarzynski/wip-crimson-crc-for-temp-msgr

crimson: configure CRC for the temp_mon_client's messenger.

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #39453 from rzarzynski/wip-crimson-monc-pick-from-addrs

crimson/monc: consider v1 addresses when connecting to a monitor.

Reviewed-by: Kefu Chai <kchai@redhat.com>

doc: define command properties using field list

the rendering is more compacted this way, and the parameters can be
aligned.

Signed-off-by: Kefu Chai <kchai@redhat.com>

doc: silence warnings from openapi sphinx extension

this is a follow-up of 1debd9856565905439ae56f2cfdc16b0549055ec,
we should use the logging from sphinx.util instead from Python standard
library's logging.

Signed-off-by: Kefu Chai <kchai@redhat.com>

doc: group commands by prefix

as there are lots commands, the toc in sidebar is clutterred with them,
so would be better to group them.

Signed-off-by: Kefu Chai <kchai@redhat.com>

doc: split the argdesc pair at the first '='

there is chance that the value of of the argdesc's kv pair contains
"=". for instance, the goodchars regexp could contain '='. so we should
stop at the first '='.

this change fixes the parsing of "osd pool application set" command.

Signed-off-by: Kefu Chai <kchai@redhat.com>

doc: drop ceph.js completely

these are leftovers of 7a62303b57150814ff465eb611b69b3b2e9f03f1

Signed-off-by: Kefu Chai <kchai@redhat.com>

doc: decrease padding in the "docubetter" div

for better UX

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #39385 from adk3798/error-redeploy

cephadm: allow redeploy of daemons in error state if container running

Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>

Merge pull request #38549 from adk3798/prepare-redeploy

mgr/cephadm: make generate_config internal, prepare before redeploy

Reviewed-by: Juan Miguel Olmo Martínez <jolmomar@redhat.com>
Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>

crimson: configure CRC for the temp_mon_client's messenger.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

crimson/monc: improve debugs around Connection::reply.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

crimson/monc: consider v1 addresses when connecting to a monitor.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

Merge pull request #39450 from cbodley/wip-rgw-notify-double-dpp

rgw: remove duplicated ldout in rgw_notify.cc

Reviewed-by: Ali Maredia <amaredia@redhat.com>

rgw: remove duplicated ldout in rgw_notify.cc

Signed-off-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #39373 from varshar16/wip-fix-rook-timezone-error

mgr/rook: Add timezone info

Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>
Reviewed-by: Travis Nielsen <tnielsen@redhat.com>

Merge pull request #39362 from rhcs-dashboard/49216-delete-eof-read-pwd-from-file

mgr/dashboard: delete EOF when reading passwords from file

Reviewed-by: Laura Paduano <lpaduano@suse.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
Reviewed-by: sebastian-philipp <swagner@suse.com>

Merge pull request #39434 from zdover23/wip-doc-dev-about-suites-and-filters-rewrite-2021-Feb-12

doc/dev: rewrite "about suites and filters"

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #39327 from kotreshhr/qa-fix-mgr-vol-test-cases

qa: Fix a few mgr/volume test cases

Reviewed-by: Ramana Raja <rraja@redhat.com>

Merge pull request #39409 from trociny/wip-49245

librbd: don't log error if get mirror status fails due to mirroring disabled

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #38883 from sebastian-philipp/cephadm-keep-deleted-specs

mgr/cephadm: Purge deleted services

Reviewed-by: Adam King <adking@redhat.com>

Merge pull request #39058 from sebastian-philipp/cephadm-use_repo_digest-default-true

mgr/cephadm: enable use_repo_digest by default for pacific

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #39364 from mgfritch/cephadm-log-errs

cephadm: add error messages to log file

Reviewed-by: Juan Miguel Olmo Martínez <jolmomar@redhat.com>
Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>
Reviewed-by: Varsha Rao <varao@redhat.com>

doc/dev: rewrite "about suites and filters"

This improves the English in the section
"About Suites and Filters" in the new
Teuthology Guide in the Developer Guide.

Signed-off-by: Zac Dover <zac.dover@gmail.com>

Merge PR #39404 into master

* refs/pull/39404/head:
qa/suites/upgrade/octopus-x/stress-split: cephadm-based stress-split upgrade

Reviewed-by: Yuri Weinstein <yweins@redhat.com>

Merge pull request #39375 from idryomov/wip-rbd-rm-exclusive

librbd: refuse to release exclusive lock when removing

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #39368 from MahatiC/wip-ssd-bug-fix

librbd/cache/pwl: Fix user request completion

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
Reviewed-by: Xiaoyan Li <xiaoyan.li@intel.com>

qa/suites/rados/cephadm/upgrade: change starting version by distro

centos/rhel have podman 2, which does not like conflicting --cap-add and
--privileged arguments.  cephadm versions prior to 15.2.5 use both args,
however, which means the rhel/centos upgrade test has to start at 15.2.5
to work at all on those distros (with the updated podman).

Fixes: https://tracker.ceph.com/issues/48142
Signed-off-by: Sage Weil <sage@newdream.net>

PendingReleaseNotes: mgr/pg_autoscaler

Added details describing the changes that
occured after merging #39248 into target
upstream.

Signed-off-by: Kamoltat <ksirivad@redhat.com>

qa: Fix a few mgr/volume test cases

Recovering dirty auth metadata file might not retain the order,
fixed the comparison in 'test_recover_auth_metadata_during_authorize'
and 'test_recover_auth_metadata_during_deauthorize'.

Fixes: https://tracker.ceph.com/issues/49192
Signed-off-by: Kotresh HR <khiremat@redhat.com>

librbd: don't log error if get mirror status fails due to mirroring disabled

Fixes: https://tracker.ceph.com/issues/49245
Signed-off-by: Mykola Golub <mgolub@suse.com>

Merge pull request #39037 from liu-chunmei/seastore_collection

crimson/seastore: add collection implementation

Reviewed-by: Samuel Just <sjust@redhat.com>