test/rgw: pass encoding to Key.get_content_as_string()

we assume that boto.Key.get_content_as_string() returns str instead of
bytes, and compare the return value with a string, so, to ensure that
lhs and rhs are both strings, we need to decode the returned content.

since we always store strings composed with ASCII, it's safe to use
'ascii' to decode them.

Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit fcf4de3855f01e80b90d246529916a5915fe413d)

test/rgw/rgw_multi/tests.py: decode retval of bucket.get_policy()

return value of bucket.get_policy() is a bytes, so we need to decode it
before comparing it with a string, since the content of policy is ASCII,
it'd be suffice to decode it with 'ascii'.

Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit ea63b52b1705e80977ce1fe6d2dc276ea9477344)

qa/tasks/cephadm.py: decode data returned by teuthology.get_file()

it's used as part of the command line sent to shell, so we need to
decode it first.

Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit 6eb57a2a4fc63771096f8fd08c02b2bda49c9301)

test/rgw/rgw_multi: return (str, int) from make_request()

so its consumer can feed the returned str to json.loads()

Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit 23ddf03d26c2c8550b8c9a0c5334fe7b8f4e1682)

test/rgw/rgw_multi: pass bytes to hmac.new()

in Python3, `key` passed to `hmac.new()` should be bytes or bytearray

Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit adfcf08588e6031be837602dc4d1f8d6d14d795b)

qa/tasks/mgr: s/urllib.quote_plus/parse.quote_plus/

in Python3, urllib.quote_plus is moved into urllib.parse, let's use
six.moves at this moment.

Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit 6c2f40a5281a1cccf0cab82c620ef1c6a40dd6ef)

qa/tasks/ceph_objectstore_tool.py: use str.startswit

in Python3, string module does not offer `string.find()` anymore, let's
use `str.find()` method instead.

Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit 77ec9ce88df54b7758e56eeef1b3cf3f14952513)

Conflicts:
qa/tasks/ceph_objectstore_tool.py: trivial resolution

qa/tasks/ceph_manager.py: do not return a filter

as the caller might want to `len(manager.get_osd_status()['raw'])`, and
`len()` does not accept a `filter` object.

also, the filtered osd statuses are printed out using `self.log()`, so
we should materialize the `filter` object before sending it to logging
facility. otherwise we will have something like:
```
2020-04-08T02:58:37.001 INFO:tasks.ceph.ceph_manager.ceph:<filter object at 0x7f5a080e1518>
```
in the logging message.

Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit 6bc09c5041d025c7ed3638ccfbf6496e1626abfb)

qa/tasks/openssl_keys.py: sort cert configs before creating certs

we cannot rely on the order in which items are arranged in a dict, the
order varies from version to another. in Python2, it happens to work,
and we can always have the self-signed cert added first. but in Python3,
it does not. and an exception is thrown
```
teuthology.exceptions.ConfigError: ssl: ca root not found for
certificate rgw.client.0
```

in this change, before creating certs, the settings are reordered so
that the self-signed ones are created first.

Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit f28a5fef3b8ddb97962f91cc78174fd6e1431fed)

pybind/mgr/dashboard/.pylintrc: drop xrange from the whitelist

since dashboard is now xrange-free, there is no need to have
xrange-builtin in whitelist, moreover, in python3 the error message
cannot be emitted at seeing xrange.

Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit 0930652318f0f2bbbefbe7dd829bc914405c9013)

qa/tasks: do not random.choice(a_view)

use `random.sample()` instead of `random.choice(list(a_view))` for better performance.

Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit 9ca45bd942ef1ed0d7ed61d87caffdfc97e54e5e)

test/rgw: use "a // b" instead of "a / b"

for expressions where the value is expected to be integer. as in
python3, `a / b` returns a float.

Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit 253fc931800a0fac8f86c9228f58ba096f58dfeb)

qa,tes: s/xrange/range/

use six.moves.range for python3 copatibility, we can drop six after
migrating to python3.

Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit dd8d929c9ccd9864e4a5005e44eac2739e5fa762)

qa/tasks: use "a // b" instead of "a / b"

for expressions where the value is expected to be integer. as in
python3, `a / b` returns a float.

Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit 2089bf04b9c3b4bb065bf4c0bafa97419f2d87ff)

qa/tasks/tox: use the latest stable tox

in hope to avoid any ancient bugs

Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit b7152e8e943b713e779aec0c2414986bd556a423)

qa/valgrind.supp: less specific when suppressing issues/22052

Fixes: https://tracker.ceph.com/issues/22052
Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit a8e9fd71d79f7696a47773acdcc738f47a88d8d8)

qa/tasks/rbd_fio: update NamedTemporaryFile usage for python3

Open the file in text mode to avoid 'bytes' objects.

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
(cherry picked from commit 4e3160a019b5af51e498793aaa24a2a7b82b4b54)

qa/tasks/rbd: update NamedTemporaryFile usage for python3

- open the file in text mode to avoid 'bytes' objects

- 'bufsize' is now spelled 'buffering' and switching buffering off
  with 'buffering=0' is only allowed in binary mode.  As we need the
  file to be either in the page cache or on disk before we copy it
  to the remote, call flush().

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
(cherry picked from commit aba9910388cce7c03aa90726b5de1263298ac8ec)

qa/test_exports: fix TestExports failure under new python3 compability changes

self.mount_a.client_remote.sh() returns an 'str' object rather than a StringIO object. Hence the p.stdout.getvalue() produces an error. This commit fixes this and also fix str and byte mismatch as byte and string were the same object in Python2 but this is not the case in Python3.

Signed-off-by: Sidharth Anupkrishnan <sanupkri@redhat.com>
(cherry picked from commit b98f3e388b6a38e91c2ee064466ae8f130a5bcda)

qa/tasks: use list comprehension for checking the length

instead of using filter(), use `sum()` for counting its
length, as in Python3, `filter()` actually returns a `filter` object
instead of a list.

in this change, `filter()` calls are replaced with `sum()`
for Python3 compatibility.

Signed-off-by: Kyr Shatskyy <kyrylo.shatskyy@suse.com>
(cherry picked from commit 726c59be58473cfe53fa3ea63cf35b58ed797886)

qa/tasks/qemu: open file in text mode

in python3, if a file is opened in 'rb' mode, we will have bytes when
reading from it, as a result, we have following failure like:
```
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/contextutil.py", line 32, in nested
    vars.append(enter())
  File "/usr/lib/python3.6/contextlib.py", line 81, in __enter__
    return next(self.gen)
  File "/home/teuthworker/src/github.com_tchaikov_ceph_nautilus-py3/qa/tasks/qemu.py", line 124, in generate_iso
    test_setup = ''.join(f.readlines())
TypeError: sequence item 0: expected str instance, bytes found
```

Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit 0b21c597d438fcba43629d0795b2d4ff6112c9cf)

mgr/insights: fix python3 DeprecationWarning

insights/module.py:167: DeprecationWarning: invalid escape sequence \d
    r = "ceph version (?P<release>\d+)\.(?P<major>\d+)\.(?P<minor>\d+)"

Signed-off-by: Michael Fritch <mfritch@suse.com>
(cherry picked from commit 41ffb701eca6280df00cd526edfce3a9c851b53b)

vstart: remove commented-out vault config options

we don't have any vstart automation for rgw encryption with vault. if
anyone wants to do local testing, they can follow instructions in
the documentation

Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit 9f0ccb5f17e8723eb62974b87c4d5ed4a621f33a)

test/rgw: Cluster.admin() returns output as utf-8 str

fixes the TypeError:

  File "rgw_multi/tests.py", line 1107, in test_bucket_sync_disable
    check_buckets_sync_status_obj_not_exist(zone, buckets)
  File "rgw_multi/tests.py", line 438, in check_buckets_sync_status_obj_not_exist
    if log_list.find(':'+bucket+":") >= 0:
TypeError: argument should be integer or bytes-like object, not 'str'

Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit ff5ce2cf1457e1945aea0fe4a023bc48bffe1174)

radosgw-admin: period commit writes messages to stderr

the period's json output is written to stdout. any other messages should
be written to stderr

Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit e48a20076274174705c5e6efde3730e993a0538d)

qa/tasks/cephfs: remove sys.exc_clear() calls

sys.exc_clear() was removed in Python3, see
https://docs.python.org/3/whatsnew/3.0.html#index-22. so we should not
call it.

Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit 702de9590cdfe6c95ef8e87cb74cd05944e380c9)

vstart_runner.py: use tuple instead of set

Using python3 leads to an error that says
"TypeError: unhashable type: 'Raw'".

Fixes: https://tracker.ceph.com/issues/45446
Signed-off-by: Rishabh Dave <ridave@redhat.com>
(cherry picked from commit cb0bfc8fab4372b0caf5102823883b0b913d4fcb)

mgr: logger.warn() -> logger.warning()

logger.warn() is the same as logger.warning(), but was removed in py3.3

Signed-off-by: Michael Fritch <mfritch@suse.com>
(cherry picked from commit ef25650721e07f97da4a71cd0b1f9d3bfc862769)

pybind/ceph_volume_client: logger.warn() -> logger.warning()

logger.warn() is the same as logger.warning(), but was removed in py3.3

Signed-off-by: Michael Fritch <mfritch@suse.com>
(cherry picked from commit fe4c7fd7471bf724c4a1f027580d9bae3ddc24eb)

qa/tasks: logger.warn() -> logger.warning()

logger.warn() is the same as logger.warning(), but was removed in py3.3

Signed-off-by: Michael Fritch <mfritch@suse.com>
(cherry picked from commit 44bcee8d1a637b1429a0c9576397f70f0a660ec2)

Merge pull request #35475 from sebastian-philipp/octopus-backport-34879-34860-35069-35262-35320-35322-35342-35359-35271-35375-35365-3541

octopus: cephadm batch backport June (2)

Reviewed-by: Michael Fritch <mfritch@suse.com>

Merge pull request #35487 from jdurgin/wip-radosbench-qa-octopus

octopus: qa/tasks/radosbench: use long form of option for compatibility

Reviewed-by: Yuri Weinstein <yweinste@redhat.com>

Merge pull request #35445 from smithfarm/wip-45884-octopus

octopus: qa/*/osd-scrub-repair.sh: Convert to python3 print syntax

Reviewed-by: Brad Hubbard <bhubbard@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #35443 from smithfarm/wip-45779-octopus

octopus: qa/workunits/rados/test_envlibrados_rocksdb: Build correct rocksdb

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Brad Hubbard <bhubbard@redhat.com>

Merge pull request #35441 from smithfarm/wip-45673-octopus

octopus: qa: run install task only once

Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

qa/tasks/cephadm: minor py2 fix

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>

Merge pull request #35440 from smithfarm/wip-45885-octopus

octopus: rbd-mirror: stop local journal replayer first during shut down

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
Reviewed-by: Mykola Golub <mgolub@mirantis.com>

Merge pull request #35439 from smithfarm/wip-45801-octopus

octopus: rbd: librbd: Watcher should not attempt to re-watch after detecting blacklisting

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #35438 from smithfarm/wip-45799-octopus

octopus: rbd: librbd: make rbd_read_from_replica_policy actually work

Reviewed-by: Mykola Golub <mgolub@mirantis.com>
Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #35437 from smithfarm/wip-45763-octopus

octopus: rbd-mirror: wait for in-flight start/stop/restart

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
Reviewed-by: Mykola Golub <mgolub@mirantis.com>

Merge pull request #35436 from smithfarm/wip-45597-octopus

octopus: mgr/rbd_support: rename "rbd_trash_trash_purge_schedule" oid

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #35435 from smithfarm/wip-45580-octopus

octopus: rbd: pybind: RBD.create() method's 'old_format' parameter now defaults to False

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
Reviewed-by: Mykola Golub <mgolub@mirantis.com>

Merge pull request #35430 from smithfarm/wip-45881-octopus

octopus: rbd-mirror: don't hold (stale) copy of local image journal pointer

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #35433 from smithfarm/wip-45782-octopus

octopus: rgw: amqp: fix the "routable" delivery mode

Reviewed-by: Yuval Lifshitz <yuvalif@yahoo.com>

Merge pull request #35432 from smithfarm/wip-45777-octopus

octopus: rgw: url: fix amqp urls with vhosts

Reviewed-by: Yuval Lifshitz <yuvalif@yahoo.com>

Merge pull request #35431 from smithfarm/wip-45645-octopus

octopus: rgw: lc: enable thread-parallelism in RGWLC

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #35434 from smithfarm/wip-45844-octopus

octopus: rgw: gc: Clearing off urgent data in bufferlist, before

Reviewed-by: Yuval Lifshitz <yuvalif@yahoo.com>
Reviewed-by: Matt Benjamin <mbenjami@redhat.com>

qa/tasks/radosbench: use long form of option for compatibility

Since the short version of --object-size changed from -o to -O, it
does not work with upgrade tests.

Signed-off-by: Josh Durgin <jdurgin@redhat.com>
(cherry picked from commit 9673d37f1b72681f7583bf36537575d876da9712)

Revert "qa/tasks/radosbench: only set object size if != block size"

This reverts commit fb991fa5f62c4aaa6c564a6d15aeee4bfaa33b39.

Signed-off-by: Josh Durgin <jdurgin@redhat.com>
Conflicts:
qa/tasks/radosbench.py - extra context from -t, trivial resolution

(cherry picked from commit a78781d82e55bb01a8a1fb42eb072b944e336973)

Revert "qa/tasks/radosbench: fix usage of -O"

This reverts commit da19fd14dbe8bdb62ce376d9df7bf5578f169b9c.

Signed-off-by: Josh Durgin <jdurgin@redhat.com>
Conflicts:
qa/tasks/radosbench.py - concurrency arg added in context, trival resolution

(cherry picked from commit 5a8c7f7d7f28a697611bab4fb816771b45cecd3e)

Merge pull request #34979 from rhcs-dashboard/wip-45460-octopus

octopus: mgr/test_orchestrator: fix _get_ceph_daemons()

Reviewed-by: Laura Paduano <lpaduano@suse.com>
Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.

Merge pull request #35085 from tspmelo/wip-45539-octopus

octopus: mgr/dashboard: Fix HomeTest setup

Reviewed-by: Laura Paduano <lpaduano@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>

Merge pull request #35241 from smithfarm/wip-45704-octopus

octopus: cls/queue: fix empty markers when listing entries

Reviewed-by: Yuval Lifshitz <yuvalif@yahoo.com>
Reviewed-by: Pritha Srivastava <prsrivas@redhat.com>

Merge pull request #35036 from badone/wip-octopus-fix-ftbfs-rocksdb_env_librados_test

octopus: rocksdb: Update to ceph-octopus-v5.8-1436

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge pull request #35105 from smithfarm/wip-45585-octopus

octopus: doc: drop obsolete cache tier options

Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>

cephadm: More robust way to deduce the systemd unit

Generate a better error message when using wrong args.

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
(cherry picked from commit 182d926a4b8a2d1c5c66809dbc89e3521b24d941)

doc/cephadm: Adopt single-site RGW

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
(cherry picked from commit 10b4f7a6d822f175bf0aab6d8b26dd09cb737de0)

qa/cephadm: update adopt corpus

to 9cd9ad020d93b0b420924fec55da307aff8bd422

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
(cherry picked from commit e45e9bf84e84a6e6c57c45b1f30283682ea40b07)

doc/cephadm: Add alternative error for missing CIDR network

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
(cherry picked from commit bd43551304575a9eb22e0c01ba36ef535a65fa50)

doc/mgr: add docs for unmanaged=True

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
(cherry picked from commit a46d582422bf9e24af920420f73e4b447dafbc61)

mgr/cephadm: refactor templating in services

Using Jinja2 to render configs for services:
- Grafana
- Alertmanager
- Prometheus
- NFS
- iSCSI

Signed-off-by: Kiefer Chang <kiefer.chang@suse.com>
(cherry picked from commit 544629833949a4104e236f19054fc614e0077f96)

mgr/cephadm: add template engine Jinja2

Signed-off-by: Kiefer Chang <kiefer.chang@suse.com>
(cherry picked from commit 64bb27f2f62010de44d50bc1a876436159770b56)

doc/cephadm: align to all-available-devices flag

Signed-off-by: Joshua Schmid <jschmid@suse.de>
(cherry picked from commit e9600666e0ab85203172b3d033322635197cd5c3)

mgr/orch: use keyword rather than positional arg

use keyword `service_id` from the orch cli

Signed-off-by: Michael Fritch <mfritch@suse.com>
(cherry picked from commit 248888dab4bc3b173210c48fbdba031091fd3f19)

mgr/cephadm: use keyword rather than positional arg

use keyword `service_id` during the ServiceSpec tests

cephadm/tests/test_cephadm.py:51: in <module>
    class TestCephadm(object):
cephadm/tests/test_cephadm.py:508: in TestCephadm
    ???
../../python-common/ceph/deployment/service_spec.py:606: in __init__
    assert service_type == 'iscsi'
E   AssertionError

Signed-off-by: Michael Fritch <mfritch@suse.com>
(cherry picked from commit a4a523d5fb9753945f677d9bd8eabb2e970691af)

mgr/cephadm: fix RGWSpec mypy errors

cephadm/module.py:34: note: In module imported here:
cephadm/services/cephadmservice.py: note: In member "config" of class "RgwService":
cephadm/services/cephadmservice.py:167: error: Incompatible types in assignment (expression has type "List[str]", variable has type "str")

Signed-off-by: Michael Fritch <mfritch@suse.com>
(cherry picked from commit bd6ea9c6a92ffbfa8676212b85d590e12decdbf5)

python-common: fix pytest failure

523: error: Unsupported operand types for + ("str" and "None")
523: note: Right operand is of type "Optional[str]"

Signed-off-by: Michael Fritch <mfritch@suse.com>
(cherry picked from commit 30a9c2f51747deb4861d1b6eaba667585e6957d0)

python-common: add ServiceSpec type checking

- add missing type checking to NFSServiceSpec and IscsiServiceSpec
- clean-up type checking for ServiceSpec and RGWSpec

Signed-off-by: Michael Fritch <mfritch@suse.com>
(cherry picked from commit 5ca009c86857b9af301024db6a9c2d63ebb7cb3a)

cephadm: fix package is not running when enabler is not none

Signed-off-by: shinhwagk <shanghai_sl@hotmail.com>
(cherry picked from commit 76919e79bf7f9c462bf6df8bc00225e56ed9ba97)

mgr/cephadm: drop py2 compatible bits

since we've dropped Python3 support in octopus, there is no need to keep
the workaround for python2 anymore.

Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit 2c71d733d6327a29617ba05b5d56891d45ad28d8)

cephadm: Add a bunch of type annotations

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
(cherry picked from commit 888754ba2c74fba42bc26c9006361150ed7ddc16)

cephadm: Make easy manager modules development

A volume replaces content  in </usr/share/mgr> folder in manager containers.
The new argument <shared_mgr_folder> sets the source folder for the volume

Signed-off-by: Juan Miguel Olmo Martínez <jolmomar@redhat.com>
(cherry picked from commit f615591808a9ba2fc7b2f088016d33b72068e3e5)

qa/cephadm: Add RGW

Fixes: https://tracker.ceph.com/issues/43911
Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
(cherry picked from commit ffef64d046cb175732b4b97f898dabac5d83df10)

python-common: improve host matching funcs

Signed-off-by: Joshua Schmid <jschmid@suse.de>
(cherry picked from commit d2aa0543d924d659fa0c5321b3fa9c607a44b5e3)

mgr/test_orchestrator: placementspec->host matching adaption

Signed-off-by: Joshua Schmid <jschmid@suse.de>
(cherry picked from commit 646ff0f749a2c7bd7ffb1356bb6cec674f7c2c57)

mgr/rook: placementspec->host matching adaption

Signed-off-by: Joshua Schmid <jschmid@suse.de>
(cherry picked from commit d488de90ad7fbcf4ea838d8ca33511fdae7823d8)

mgr/cephadm: placementspec->host matching adaption

Signed-off-by: Joshua Schmid <jschmid@suse.de>
(cherry picked from commit 02231859860d096689c4f2e3f5ea4d502d8dc761)

python-common: only validate host_pattern if present

Signed-off-by: Joshua Schmid <jschmid@suse.de>
(cherry picked from commit 3b771811859b56bf004f5016fa0fb9173c417865)

mgr/cephadm: cephadm bootstrap add --apply-spec <cluster.yaml>

Have a single command when setting up a cluster for Day 1

example spec

---
service_type: host
addr: jmo-node-00
hostname: jmo-node-00
labels:
- test
- test2
---
service_type: host
addr: jmo-node-01
hostname: jmo-node-01
labels:
- grafana
---
service_type: host
addr: jmo-node-02
hostname: jmo-node-02
---
service_type: grafana
placement:
  label: "grafana"
---
service_type: osd
placement:
  host_pattern: '*'
data_devices:
  all: true

Signed-off-by: Daniel-Pivonka <dpivonka@redhat.com>
(cherry picked from commit fed98dfab1436a136d0e017b51197f0ff10d78df)

mgr/orch: allow '.' char in hostname

Fixes: https://tracker.ceph.com/issues/45399
Signed-off-by: Michael Fritch <mfritch@suse.com>
(cherry picked from commit 9c14f0d1cdf123ae9216029c9aaef1e09fd81c24)

PendingReleaseNotes: reset version number to 15.2.4

None of these release notes have actually been published yet, and the
next octopus release is supposed to have version number 15.2.4.

Signed-off-by: Nathan Cutler <ncutler@suse.com>

rbd-mirror: track journal replay flush requests to prevent race

If a journal replay flush is in-progress when the ImageReplayer is stopped,
it can race and result in an assertion failure due to two attempted shutdowns
of the same journal replay state machine.

Fixes: https://tracker.ceph.com/issues/45409
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 975a0d253a22bf22d3ec10e738ddae15834d7994)

qa/*/osd-scrub-repair.sh: Convert to python3 print syntax

Fixes: https://tracker.ceph.com/issues/45733
Signed-off-by: Brad Hubbard <bhubbard@redhat.com>
(cherry picked from commit 80e7b7c19bf28557e72bcfebcc8fbaa10b8b945b)

qa/workunits/rados/test_envlibrados_rocksdb: Build correct rocksdb

We should be building the version of rocksdb the release is pinned to,
not master. Let's just update the rocksdb submodule and clone that.

Fixes: https://tracker.ceph.com/issues/44981
Signed-off-by: Brad Hubbard <bhubbard@redhat.com>
(cherry picked from commit 1361913f5995f8a4c8e1b0f5c661ad09e66fe9aa)

qa: run install task only once

Fixes: https://tracker.ceph.com/issues/45612
Fixes: 8d77d134c73d3471515f113320e9db1f0e59eb1f
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit 41b61203992ec825c70dca51fba4effd93109e2d)

test/rbd_mirror: fix 'Uninteresting mock function call' warnings

Signed-off-by: Mykola Golub <mgolub@suse.com>
(cherry picked from commit 4a23b4be68d00ac45216f7208ef19731006887da)

librbd: Watcher should not attempt to re-watch after detecting blacklisting

Currently, the Watcher state machine will spin as fast as it can sending
re-watch requests to the OSD and then retrying after it fails with the
EBLACKLISTED error. Treat a blacklisting similarly to how removal of the
object is treated: stop attempting to re-watch.

Fixes: https://tracker.ceph.com/issues/45715
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 6be1d49c35be4c937664939947a52f33696b0d8f)

librbd: make rbd_read_from_replica_policy actually work

Commit d92be8b5856d ("librbd: Improved support for balanced and
localized reads") introduced read_flags, but used low-level Objecter
flags (CEPH_OSD_FLAG_*).  This is wrong because librbd sits on top of
librados, not Objecter.  "balance" ends up being translated into
IGNORE_REDIRECT and "localize" being ignored.

Fixes: https://tracker.ceph.com/issues/45798
Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
(cherry picked from commit fc3c0c4dca2800140bd66100770403b1346cf738)

rbd-mirror: wait for in-flight start/stop/restart

when stopping instance replayer on shut down.

Signed-off-by: Mykola Golub <mgolub@suse.com>
(cherry picked from commit e55b64eaecb750e4ad6db89a741c8d0d3f03a670)

rbd-mirror: make stop properly cancel restart

Previously, if stop was issued when restart was at "stopping"
stage, the stop was just ignored.

Signed-off-by: Mykola Golub <mgolub@suse.com>
(cherry picked from commit 0a3794e56256be33a71e363da34ee84ffc34fef7)

mgr/rbd_support: rename "rbd_trash_trash_purge_schedule" oid

to "rbd_trash_purge_schedule"

Fixes: https://tracker.ceph.com/issues/45589
Signed-off-by: Mykola Golub <mgolub@suse.com>
(cherry picked from commit 5b6849e08e80506d558200dcf877cf430b0e1ac1)

pybind/rbd: RBD.create() method's 'old_format' parameter now defaults to False

The RBD v1 format has been deprecated for numerous releases and creation of
v1 format images has been disabled since the Mimic release. This fixes
the Python API's image create() method to ensure v2 images are created by
default (and no longer throw an exception that creation of v1 images are
disabled).

Fixes: https://tracker.ceph.com/issues/45504
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 45349355f4b41c6f9de594ef34a8657230113e6b)

cls/rgw_gc: Fixing carriage returns in log statement.

Signed-off-by: Pritha Srivastava <prsrivas@redhat.com>
(cherry picked from commit 9a68cddda3f1aa94a7adb415ff186a615fb79fba)

cls/rgw_gc: Clearing off urgent data in bufferlist, before
encoding the updated one.

If this is not done, urgent data keeps getting appended
to the head, finally overwriting queue data that results
in invalid entries.

Signed-off-by: Pritha Srivastava <prsrivas@redhat.com>
(cherry picked from commit 182382cf6368eaa6f9d1efbb883ae7813a9bf6d0)

rgw/amqp: fix the "routable" delivery mode

this option was not exposed to the configuration API
however, it was still set, as hardcoded value in the code
(details:
https://www.rabbitmq.com/confirms.html#publisher-confirms)

Fixes: https://tracker.ceph.com/issues/44915
Signed-off-by: Yuval Lifshitz <yuvalif@yahoo.com>
(cherry picked from commit 73db460e52717ede87e9c0041d097853fde27721)

rgw/url: fix commensts from review #34717

Signed-off-by: Yuval Lifshitz <ylifshit@redhat.com>
Fixes: https://tracker.ceph.com/issues/45269
(cherry picked from commit 32a93ecf50a236ea654448f761371e7cce89c262)

src/rgw/rgw_url.cc, also cope with a vhost appended to a AMQP-URL

Signed-off-by: Yuval Lifshitz <ylifshit@redhat.com>
Fixes: https://tracker.ceph.com/issues/45269
(cherry picked from commit a984d2a08c013c7cf78a25234d9fafb78c6cdc9a)

rbd-mirror: don't hold (stale) copy of local image journal pointer

The exclusive-lock manages its life cycle and can close the journal
at any point. This can result in rbd-mirror deferencing a freed pointer
or a journal state machine that is in an unexpected state.

Fixes: https://tracker.ceph.com/issues/45803
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 295483fbdc377d633addf5ec8ae2600a741db78c)

Merge pull request #34995 from smithfarm/wip-44986-octopus

octopus: mgr: synchronize ClusterState's health and mon_status.

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #34822 from votdev/wip-45324-octopus

Reviewed-by: Stephan Müller <smueller@suse.com>
Reviewed-by: Tiago Melo <tmelo@suse.com>