Change osd op queue cut off default to high

Discussion: https://www.mail-archive.com/ceph-users@ceph.io/msg00166.html

Signed-off-by: Anthony D'Atri <anthony.datri@gmail.com>
(cherry picked from commit be4582c26a82ff9d6b82ff6178cd78223715bba9)

doc/rados: Fix osd_scrub_during_recovery default value

Since 8dca17c, `osd_scrub_during_recovery` defaults to `false`, but the
documentation was still stating that its default value is `true`.

Fixes: https://tracker.ceph.com/issues/46531
Signed-off-by: Benoît Knecht <bknecht@protonmail.ch>
(cherry picked from commit 535b103d1848f8b5322af0815e1bf163267d7f2a)

Merge pull request #38302 from ceph/wip-mimic-doc

mimic: doc: enable Read the Docs

Reviewed-by: Neha Ojha <nojha@redhat.com>

doc: add in-doc search from read the docs

readthedocs-sphinx-search features better user experience than the
builtin search offered by sphinx

Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit 8bd8a8badbf992347a0883a537cce414432c867e)

doc: use plantweb as fallback of sphinx-ditaa

RTD does not support installing system packages, the only ways to install
dependencies are setuptools and pip. while ditaa is a tool written in
Java. so we need to find a native python tool allowing us to render ditaa
images. plantweb is able to the web service for rendering the ditaa
diagram. so let's use it as a fallback if "ditaa" is not around.

also start a new line after the directive, otherwise planweb server will
return 500 at seeing the diagram.

Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit 0cb56e0f13dc57167271ec7f20f11421416196a2)

doc/conf.py: exclude pybindings docs from build for RTD

because it'd difficult to prepare (dummy) librados,libcephfs and librbd for
their python bindings in the building environment offered by Read the Docs.

Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit 847e4ef941401e1b580e93d7058e8413bd131e21)

Conflicts:
doc/conf.py: remove ceph-common as it does not exist back in
                     mimic.

readthedocs: add .readthedocs.yml

Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit 99b75c14d2c84bad8a6b491e4981eeb481751c40)

Merge pull request #37196 from guits/wip-47505-mimic

mimic: ceph-volume: fix simple activate when legacy osd

ceph-volume: fix wrong type passed in terminal.warning()

`terminal.warning()` excepts a `str`.
Passing `e` means we pass a type `exceptions.RuntimeError`

Changing to `terminal.warning(e.message)` fixes the issue.

Closes: https://bugzilla.redhat.com/show_bug.cgi?id=1877672
Resolves: rhbz#1877672

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
(cherry picked from commit a1f42c8d7b3fe08da82c528038d8db9ccdd5c98a)

ceph-volume: fix simple activate when legacy osd

`ceph-volume simple activate --all` relies on the presence of json files
in `/etc/ceph/osd` that was created with `ceph-volume simple scan`
command.

In a cluster lifecycle, it is very likely an OSD which was deployed with
ceph-disk at some point gets removed or replaced. It means the corresponding
json file in `/etc/ceph/osd` becomes unrelevant. It makes `ceph-volume
simple activate --all` fails because it tries to mount non existing
partitions.
The idea here is to simply warn the user that the osd described in the
json file doesn't exist anymore and exit properly instead of throwing an
error.

Closes: https://bugzilla.redhat.com/show_bug.cgi?id=1877672
Closes: https://tracker.ceph.com/issues/47493
Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
(cherry picked from commit a3e9e215bda110b3224e165bee6565943b3f3c14)

Merge pull request #36616 from rishabh-d-dave/cv-add-tests-for-new-lvm-functions-m

mimic: ceph-volume: add tests for new functions that run LVM commands

Merge pull request #36612 from rishabh-d-dave/cv-rm-container-classes-m

mimic:  ceph-volume: remove container classes from api/lvm.py

Merge pull request #32887 from shyukri/wip-42140-mimic

Reviewed-by: Ernesto Puerta <epuertat@redhat.com>

ceph-volume: add tests for new functions that run LVM commands

... and for supporting functions. Specifically, these functions are
get_pvs(), get_vgs(), get_lvs(), get_first_pv(), get_first_vg(),
get_first_lv() and make_filters_lvmcmd_ready().

Signed-off-by: Rishabh Dave <ridave@redhat.com>
(cherry picked from commit 97c3a0c80a2c03478d948fed7e0ffa8913d394a2)

ceph-volume: relocate the new LVM methods

Signed-off-by: Rishabh Dave <ridave@redhat.com>
(cherry picked from commit e6d2dd0c893c30dda1c51d02f51df7ae7093d008)

ceph-volume: remove api.lvm.get_api_lvs() and related tests

Signed-off-by: Rishabh Dave <ridave@redhat.com>
(cherry picked from commit b7ca04dc647b2e7b3b2388c48884af9bd69db2c4)

ceph-volume: remove api.lvm.get_api_vgs() and related tests

Signed-off-by: Rishabh Dave <ridave@redhat.com>
(cherry picked from commit ab7a2c3df8a9997c693fdd6ace38b89c67650197)

ceph-volume: remove api.lvm.get_api_pvs()

Signed-off-by: Rishabh Dave <ridave@redhat.com>
(cherry picked from commit 93bae7afdd881fe345e0001a6c02975979ab188c)

ceph-volume: remove class Volumes from api/lvm.py

And remove methods is_lv(), get_lv() and get_lv_from_argument() since
these methods uses class Volumes and get_first_lv() or get_lv() can be
easily used instead when needed in future.

Also, update tests and comments accordingly.

Signed-off-by: Rishabh Dave <ridave@redhat.com>
(cherry picked from commit e09d3cd1a5bcfb42972b6958fb7e6c42a645549d)

ceph-volume: remove class VolumeGroups from api/lvm.py

And remove get_vg() since its based on VolumesGroups and get_first_vg()
or get_vg() can be easily used instead in future when needed. Also,
update tests accordingly.

iSigned-off-by: Rishabh Dave <ridave@redhat.com>
(cherry picked from commit 72068493ac9b3d281aeda3f0481ad63c24d796e3)

ceph-volume: remove PVolumes from api/lvm.py

And method get_pv() since it's based on PVolumes and get_first_pv() and
get_pvs() can be easily used instead in future. Also, update tests
accordingly.

Signed-off-by: Rishabh Dave <ridave@redhat.com>
(cherry picked from commit 009e9dcebf6e13b6fb66bc0368c907d0b988d127)

Merge pull request #35900 from rishabh-d-dave/wip-32493-notrack-mimic

mimic: ceph-volume: don't use container classes in api/lvm.py

Merge pull request #36452 from jan--f/wip-46114-mimic

mimic: ceph-volume: report correct rejected reason in inventory if device type is invalid

ceph-volume: filter by osd-id or osd-fsid when zapping

2f5c10c12c37e6865ce54bb4940d3779353cba4f introduced a bug:

`ceph-volume lvm zap` command fails under certain conditions.

when passing `--osd-id` or `--osd-fsid` to `ceph-volume lvm zap` command
it tries to zap additionnal devices that have nothing to do with the osd
being zapped.

When calling `api.get_lvs()` in `ensure_associated_lvs()` we have to
pass the osd-id/osd-fsid information so only related devices are
returned by `get_lvs()` method

Closes: https://tracker.ceph.com/issues/46627
Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
(cherry picked from commit e94aef6f96189648d0f204d8118501577dde5f52)

ceph-volume: delete devices.lvm.prepare.Prepare.get_lv

Since it calls get_lv_from_argument() internally and use get_first_lv()
instead and update tests.

Signed-off-by: Rishabh Dave <ridave@redhat.com>
(cherry picked from commit ca1d3e694c6390632c86e6dfcf01a2b3c6a67f0d)

ceph-volume: don't use api.lvm.VolumeGroups

Use api.lvm.get_vgs() and api.lvm.get_first_vg() instead and update
tests.

Signed-off-by: Rishabh Dave <ridave@redhat.com>
(cherry picked from commit dfe6b1b90384482e8ec38e3be7024971fb252968)

ceph-volume: don't use api.lvm.PVolumes

Update api.lvm.get_first_pv() and api.lvm.get_pvs() instead and update
tests.

Signed-off-by: Rishabh Dave <ridave@redhat.com>
(cherry picked from commit a6e3c69c6cd561ac6680fceb3017b9a2afe3d9de)

ceph-volume: don't use api.lvm.get_lv

Use api.lvm.get_lvs() and api.lvm.get_first_lv() instead and update the
tests.

Signed-off-by: Rishabh Dave <ridave@redhat.com>
(cherry picked from commit b2254050c01d872ebc906708a3b4c8d4fa2105d7)

ceph-volume: don't use api.lvm.get_lv_from_argument

Use api.lvm.get_lvs() and api.lvm.get_first_lv() instead and update
tests.

Signed-off-by: Rishabh Dave <ridave@redhat.com>
(cherry picked from commit 5301c4959e5169b10cc92716de6222f7516770e6)

ceph-volume: don't use class api.lvm.Volumes

Replace code using class Volumes by methods get_lvs() and
get_first_lv(). Also, update the tests accordingly.

Signed-off-by: Rishabh Dave <ridave@redhat.com>
(cherry picked from commit 2f5c10c12c37e6865ce54bb4940d3779353cba4f)

Merge pull request #36454 from jan--f/wip-45482-mimic

mimic: ceph-volume: add ceph.osdspec_affinity tag

ceph-volume: add osdspec_affinity flag to ceph-osd

Signed-off-by: Joshua Schmid <jschmid@suse.de>
(cherry picked from commit 815f8e6470c6e72d9e2f51630dd3c26d388847aa)

ceph-volume: add ceph.osdspec_affinity tag

Signed-off-by: Joshua Schmid <jschmid@suse.de>
(cherry picked from commit b7bc6f7607d79af6d43e788ea2a26785f4ed5d45)

ceph-volume: show correct rejected reason in inventory if device type is not acceptable

If device type is not acceptable in `c-v inventory`, its rejected reason
becomes "Insufficient space (<5GB)" by mistake. It's because sys_api is
empty due to skipping devices that are neither `disk` nor `device`. We
should report the target device is not acceptable in this case.

Fixes: https://tracker.ceph.com/issues/46102
Signed-off-by: Satoru Takeuchi <satoru.takeuchi@gmail.com>
(cherry picked from commit 3e5d91d41f7275d4656019c1ca3fc80927d214c9)

ceph-volume: cleanup code

Simplify the logic and fix a typo.

Signed-off-by: Satoru Takeuchi <satoru.takeuchi@gmail.com>
(cherry picked from commit 0169b72fff48134ef01802ade38c55281b9f4510)

Merge pull request #36451 from jan--f/wip-45785-mimic

mimic: ceph-volume: add and delete lvm tags in a single lvchange call.

ceph-volume: fix Volume mock in lvm test case

Signed-off-by: Jan Fajerski <jfajerski@suse.com>
(cherry picked from commit cb432fe41d4ea8cb71aa592e0727d2da1978121f)

ceph-volume: add and delete lvm tags in a single lvchange call.

Otherwise we can end up in race-y situations when a concurrent c-v calls
sees only one tag but expects all tags to be present. Say if the
ceph.type tag is present, c-v expects ceph.osd_id to be present. By
setting/deleting tags in bulk, we use lvchange (and lvms internal
locking) as a sync mechanism.

Fixes: https://tracker.ceph.com/issues/44852
Signed-off-by: Jan Fajerski <jfajerski@suse.com>
(cherry picked from commit 20ecc309371e53fda5d6a5b6cf6de6110dbe5497)

Merge pull request #35489 from jdurgin/wip-radosbench-qa-mimic

mimic: qa/tasks/radosbench: use long form of option for compatibility

Reviewed-by: Yuri Weinstein <yweinste@redhat.com>

qa/tasks/radosbench: use long form of option for compatibility

Since the short version of --object-size changed from -o to -O, it
does not work with upgrade tests.

Signed-off-by: Josh Durgin <jdurgin@redhat.com>
(cherry picked from commit 9673d37f1b72681f7583bf36537575d876da9712)

Conflicts:
qa/tasks/radosbench.py - trivial context difference

Merge pull request #34737 from smithfarm/wip-44647-mimic

mimic: qa/workunits/rbd: wait for nbd map to close after unmap

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #35164 from athanatos/sjust/wip-revert-wrong-pr-merge

Revert "doc: clarify WAL in bluestore-config-ref.rst"

Revert "doc: clarify WAL in bluestore-config-ref.rst"

This was merged to mimic by mistake.

This reverts commit 637ee39fafb3d0521bbd51681583504a2f6fd636.

Signed-off-by: Samuel Just <sjust@redhat.com>

Merge pull request #35156 from CodingSpiderFox/patch-1

Clarify abbreviation that I was not aware of

Reviewed-by: Samuel Just <sjust@redhat.com>

doc: clarify WAL in bluestore-config-ref.rst

Other people might not know this abbreviation too when diving into storage topics for the first time

Signed-off-by: CodingSpiderFox <codingspiderfox@gmail.com>

Merge pull request #34277 from jan--f/wip-44687-mimic

mimic: ceph-volume: prepare: use *-slots arguments for implicit sizing

qa/workunits/rbd: wait for nbd map to close after unmap

The unmap action only sends a signal to the kernel to notify the
rbd-nbd daemon to disconnect. Therefore, it's possible that an
unmap followed by an immediate re-map to the same device might
fail since the unmap is still in-progress.

Fixes: https://tracker.ceph.com/issues/44567
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 80a3f18cafb4add1624cc690bba436a1284dc634)

Merge remote-tracking branch 'gh/mimic' into mimic

Merge pull request #34713 from ceph/wip-yuriw-clients-upgrades-mimic

qa/tests: removed 2-workload/devstack-tempest-gate.yaml  tests

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Jason Dillaman <dillaman@redhat.com>

qa/tests: removed 2-workload/devstack-tempest-gate.yaml  tests

Signed-off-by: Yuri Weinstein <yweinste@redhat.com>

13.2.10

Merge pull request #34715 from ceph/mimic-rgw-post-obj-tagging

rgw: check for tagging element in POST Obj requests

Reviewed-By: Casey Bodley <cbodley@redhat.com>
Reviewed-By: Matt Benjamin <mbenjami@redhat.com>

rgw: check for tagging element in POST Obj requests

Check for null element when reading the tagging field from POST obj XML

Fixes: https://tracker.ceph.com/issues/44967
Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>

Merge pull request #34521 from smithfarm/wip-45057-mimic

mimic: qa/suites/rados/rest: don't pass empty dict as data arg

Reviewed-by: Brad Hubbard <bhubbard@redhat.com>

Merge pull request #34370 from smithfarm/wip-44083-mimic

mimic: qa/suites/rados/multimon/tasks/mon_clock_with_skews: disable ntpd etc

Reviewed-by: Neha Ojha <nojha@redhat.com>

13.2.9

qa/suites/rados/rest: Don't pass empty dict as data arg

Passing an empty 'args' dict as a data argument when calling
requests.get somehow confuses the transaction, causing it to fail. Pass
'None' instead.

Fixes: https://tracker.ceph.com/issues/43720
Signed-off-by: Brad Hubbard <bhubbard@redhat.com>
(cherry picked from commit e20e6fc3295da836fb5f098d46d6190ab28195d5)

Merge pull request #34504 from ceph/wip-mimic-rgw-fixes

Reviewed-By: Yuri Weinstein <yuriw@redhat.com>
Reviewed-By: Casey Bodley <cbodley@redhat.com>
Reviewed-By: Nathan Cutler <ncutler@suse.com>

Merge pull request #34458 from badone/wip-44983-mimic

mimic: selinux: Allow ceph-mgr access to httpd dir

Reviewed-by: Boris Ranto <branto@redhat.com>
Reviewed-by: Yuri Weinstein <yweinste@redhat.com>

Merge pull request #34393 from tchaikov/mimic-cbt

mimic: qa/suites: use "mimic" branch for cbt based testing

Reviewed-by: Yuri Weinstein <yweinste@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge pull request #34479 from jan--f/wip-45004-mimic

mimic: ceph-volume/batch: check lvs list before access

Reviewed-by: Yuri Weinstein <yweinste@redhat.com>

Merge pull request #34476 from jan--f/wip-45000-mimic

mimic: ceph-volume/batch: return success when all devices are filtered

Reviewed-by: Yuri Weinstein <yweinste@redhat.com>

ceph-volume/batch: check lvs list before access

Fixes: https://tracker.ceph.com/issues/44989
Signed-off-by: Jan Fajerski <jfajerski@suse.com>
(cherry picked from commit c6dcf3161b2d69b88acdda473b85c9dd2c8433e3)

rgw: reject control characters in response-header actions

S3 GetObject permits overriding response header values, but those inputs
need to be validated to insure only characters that are valid in an HTTP
header value are present.

Credit: Initial vulnerability discovery by William Bowling (@wcbowling)
Credit: Further vulnerability discovery by Robin H. Johnson <rjohnson@digitalocean.com>
Signed-off-by: Robin H. Johnson <rjohnson@digitalocean.com>

rgw: EPERM to ERR_INVALID_REQUEST

As per Robin's comments and S3 spec

Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>

rgw: reject unauthenticated response-header actions

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
Reviewed-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit d8dd5e513c0c62bbd7d3044d7e2eddcd897bd400)

ceph-volume/batch: return success when all devices are filtered

batch should only return an error if some (but not all) devices are
filtered. When only some devices are filtered the resulting osd layout
could look very different from what a user expects. If all devies are
filtered just return success.

Fixes: https://tracker.ceph.com/issues/44994
Signed-off-by: Jan Fajerski <jfajerski@suse.com>
(cherry picked from commit cb3eade3d00fdcdb34c4156f3c488bbb8747cd6f)

Merge pull request #34444 from jan--f/wip-44973-mimic

mimic: devices/simple/scan: Fix string in log statement

Reviewed-by: Yuri Weinstein <yweinste@redhat.com>

devices/simple/scan: Fix string in log statement

A multiline string enclosed by () gained a comma, making it into a
tuple, which has no format() method.

Fixes: https://tracker.ceph.com/issues/44949
Signed-off-by: Jan Fajerski <jfajerski@suse.com>
(cherry picked from commit 292ae0305db3e48a87db45bcc7ca8cb837a08ead)

selinux: Allow ceph-mgr access to httpd dir

ceph-mgr loads modules which require read access and this causes a
denial on el7.

Fixes: https://tracker.ceph.com/issues/44216
Signed-off-by: Brad Hubbard <bhubbard@redhat.com>
(cherry picked from commit 35a7fc8249337c3c59f0c561632abf578f5d20fc)

Merge pull request #34221 from ceph/wip-yuriw-clients-upgrades-mimic-octopus

qa/tests: initial check in for client-upgrade-mimic-octopus

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #34379 from cbodley/wip-44898

qa/swift: fix py3 backport regression

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Kyr Shatskyy <kyrylo.shatskyy@suse.com>

qa/suites: use "mimic" branch for cbt based testing

cbt is using features like the f string introduced by python3.6, so we
need either stop testing on ubuntu 16.04, which came with python3.5, or
specify the a cbt branch which support python3.5. to address this issue
once and for all, it's simpler just pin the branch to "mimic" when
performing cbt based tests.

this change is not cherry-picked from master, as in master, we only
tests on distros which offers python3.6 and up.

Signed-off-by: Kefu Chai <kchai@redhat.com>

qa/tests: initial check in for client-upgrade-mimic-octopus

Signed-off-by: Yuri Weinstein <yweinste@redhat.com>

qa/swift: fix py3 backport regression

this applies only to the mimic branch because it corrects a mimic
backport

Fixes: https://tracker.ceph.com/issues/44898
Signed-off-by: Casey Bodley <cbodley@redhat.com>

qa/suites/rados/multimon/tasks/mon_clock_with_skews: whitelist MOST_DOWN

The skewed clock makes some mons miss elections.

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 08b6a2bc008c006ff815dcb85bfb24b73072c7ab)

qa/suites/rados/multimon/tasks/mon_clock_with_skews: disable ntpd etc

Fixes: https://tracker.ceph.com/issues/43889
Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 9f2a854b175f156d4ab7fba955aff515052c9d93)

qa/suites/rados/multimon: skew clocks 2s (< paxos lease)

If the leader is the one with the accurate clock, it can still
form quorum, but if the leader has the skewed clock, all other mons appear
skewed from its perspective and no quorum is formed.  This leads to
intermittent failures, depending on the non-deterministic teuthology
deployment order and how the mon IPs sort.

Fix by reducing the skew.  This is enough skew to trigger a warning, but
not enough that it will break quorum.  This ensures that the parts of the
teuthology test that issue random mon commands won't fail (e.g., 'ceph osd
dump').

Fixes: http://tracker.ceph.com/issues/40112
Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit fd00136eb3b181ca34937fc7ea99427338ae6069)

qa/suites/rados/multimon: create_rbd_pool: false

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 8d137b9345afd7e77f788e7142ff28a7f422c4f9)

qa/suites/rados/multimon: no osds when skewing clock

Sometimes the clock skew prevents the mon quorum from making progress and
processing the osd boot messages.

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 1991faaafe6e450896377bb69c04fbce84b1f989)

Merge pull request #34193 from tchaikov/mimic-cbt-py3

mimic: qa/tasks/cbt: install python3 deps

Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Yuri Weinstein <yweinste@redhat.com>

ceph-volume: pass *-slots arguments to LV creation

Fixes: https://tracker.ceph.com/issues/44494
Signed-off-by: Jan Fajerski <jfajerski@suse.com>
(cherry picked from commit 61f0c3af0b0f36a03748733b16de1b11048342e6)

use extent count for slots conversion instead of free count

Signed-off-by: Jan Fajerski <jfajerski@suse.com>
(cherry picked from commit 4a81b4a2fbb4cafe45a8fe8cceeece636e51adb6)

ceph-volume: available_lvm: vg space takes precedence

This changes available_lvm to check for generic reasons only if no VGs
were found. A VG can contain a (mounted) lv, which triggers the
ro/locked test, despite the VG having space available.

Signed-off-by: Jan Fajerski <jfajerski@suse.com>
(cherry picked from commit c6d71eb0022935731bb9380098c975d9c5f27a86)

qa/tasks/cbt: install python3 deps

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 299375852a4b3210cf5b95d3559d69080f14339f)

Conflict: qa/tasks/cbt.py: change python3-yaml to python36-PyYAML,
  as in EL7, python3-* packages are still named python36-*

Merge pull request #33327 from smithfarm/wip-43257-mimic

mimic: common/config: update values when they are removed via mon

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge pull request #31875 from smithfarm/wip-42258-mimic

mimic: doc, qa: remove invalid option mon_pg_warn_max_per_osd

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #32911 from smithfarm/wip-43502-mimic

mimic: mount.ceph: give a hint message when no mds is up or cluster is laggy

Reviewed-by: Ilya Dryomov <idryomov@redhat.com>
Reviewed-by: Jeff Layton <jlayton@redhat.com>

Merge pull request #33999 from shyukri/wip-42323-mimic

mimic: qa: get rid of iterkeys for py3 compatibility

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #33598 from trociny/wip-44262-mimic

mimic: rbd-mirror: improve detection of blacklisted state

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #33966 from smithfarm/wip-44465-mimic

mimic: rgw: get barbican secret key request maybe return error code

Reviewed-by: Casey Bodley <cbodley@redhat.com>

mrg/dashboard: update protractor to fix chromedriver errors

E/launcher - session not created: Chrome version must be between 71 and 75
(Driver info: chromedriver=2.46.628388 (4a34a70827ac54148e092aafb70504c4ea7ae926),platform=Linux 4.15.0-72-generic x86_64)

Disable pylint warning

Signed-off-by: Shyukri Shyukriev <shshyukriev@suse.com>

Merge pull request #33986 from toabctl/mimic-testtools

mimic: ceph-disk,ceph-detect-init: Pin testtools test requirement for old py versions

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Nathan Cutler <ncutler@suse.com>
Reviewed-by: Laura Paduano <lpaduano@suse.com>

qa: get rid of iterkeys for py3 compatibility

Fixes: https://tracker.ceph.com/issues/42287
Signed-off-by: Kyr Shatskyy <kyrylo.shatskyy@suse.com>
(cherry picked from commit 5f95b532aa26249cecea4586166399a0bf10ad47)

Ensured all iterkeys -> keys under qa/tasks
Removed ceph_crash / install_packages methods by PR comments

ceph-detect-init & ceph-disk: Pin testtools test requirement

When running make-check on python 3.4, the test requirements
installation fails with:

ERROR: testtools requires Python '>=2.7, !=3.0.*, !=3.1.*, !=3.2.*, \
  !=3.3.*, !=3.4.*' but the running Python is 3.4.6

This happens because there is a new testtols release on pypi from Mar
14, 2020. Pinning to an older version fixes this.

This is not needed for Master, Octopus and Nautilus  given that python
3.4 is already EOL[1] before Nautilus got released.

[1] https://www.python.org/downloads/release/python-3410/

Signed-off-by: Thomas Bechtold <tbechtold@suse.com>

Merge pull request #33594 from neha-ojha/wip-44022-mimic

mimic: osd/PeeringState.h: ignore MLogRec in Peering/GetInfo

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>

Merge pull request #33748 from cbodley/wip-44445

mimic: rgw: making implicit_tenants backwards compatible

Reviewed-by: Casey Bodley <cbodley@redhat.com>

get barbican secret key request return error code

Signed-off-by: Richard Bai(白学余) <baixueyu@inspur.com>
(cherry picked from commit fbe2be57474df43996dd45bf04d1a1137a02c729)

Conflicts:
src/rgw/rgw_http_client.cc
- mimic uses Mutex::Locker instead of std::lock_guard

rgw: making implicit_tenants backwards compatible.

In jewel, "rgw keystone implicit tenants" only applied to swift. As of
luminous), this option applies to s3 also.
Sites that used this feature with jewel now have outstanding data that
depends on the old behavior.

The fix here is to expand "rgw keystone implicit tenants" so that it
can be set to any of "none", "all", "s3" or "swift" (also 0=false=none,
1=true=all).  When set to "s3" or "swift", the actual id lookup
is also partitioned.

Formerly "rgw keystone implicit tenants" was a legacy opt.
This change converts it to the new style of option,
including support for dynamically changing it.

Fixes: http://tracker.ceph.com/issues/24348
Signed-off-by: Marcus Watts <mwatts@redhat.com>
(cherry picked from commit a28a38f6e91da3abe59c34fad0e059eeaf29a65f)
(cherry picked from commit d2bdea8e7bc8eae64e4b72f581e424a9a6871aed)

Conflicts:
  - forward port from luminous was easier than backport from master

Merge pull request #31384 from less-is-morr/wip-42159-mimic

mimic: cephfs: osdc/objecter: Fix last_sent in scientific format and add age to ops

Reviewed-by: Milind Changire <mchangir@redhat.com>