Merge pull request #39662 from pponnuvel/wip-49441-octopus

octopus: rgw: radosgw-admin: clarify error when email address already in use

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #39577 from dillaman/wip-49252-octopus

octopus: librbd: correct incremental deep-copy object-map inconsistencies

Reviewed-by: Mykola Golub <mgolub@mirantis.com>

Merge pull request #39376 from dillaman/wip-48091-octopus

octopus: mgr/rbd_support: create mirror snapshots asynchronously

Reviewed-by: Mykola Golub <mgolub@mirantis.com>

Merge pull request #39103 from k0ste/wip-49022-octopus

octopus: mgr/dashboard: trigger alert if some nodes have a MTU different than the median value

Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>

Merge pull request #39027 from rhcs-dashboard/wip-48960-octopus

octopus: mgr/dashboard: Fix for incorrect validation in rgw user form

Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>

Merge pull request #39306 from rhcs-dashboard/wip-48829-octopus

octopus: mgr/dashboard/monitoring: upgrade Grafana version due to CVE-2020-13379

Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
Reviewed-by: p-se <NOT@FOUND>

Merge pull request #39271 from jan--f/wip-49141-octopus

octopus: ceph-volume: add some flexibility to bytes_to_extents

Merge pull request #39534 from ideepika/wip-backport-blocklist-ignorelist-qa

octopus: whitelist -> ignorelist for qa/* only

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge pull request #39235 from k0ste/wip-49107-octopus

octopus: mgr/zabbix: format ceph.[{#POOL},percent_used as float

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #39277 from smithfarm/wip-49131-octopus

octopus: rgw: fix trailing null in object names of multipart reuploads

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #39583 from liewegas/pr-39561-octopus

octopus: qa: use normal build for valgrind

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

rgw/radosgw-admin clarify error when email address already in use

The error message if you try and create an S3 user with an email
address that is already associated with another S3 account is very
confusing; this patch makes it much clearer

To reproduce:

radosgw-admin user create --uid=foo --display-name="Foo test" --email=bar@domain.invalid
radosgw-admin user create --uid=test --display-name="AN test" --email=bar@domain.invalid
could not create user: unable to parse parameters, user id mismatch, operation id: foo does not match: test

With this patch:

radosgw-admin user create --uid=test --display-name="AN test" --email=bar@domain.invalid
could not create user: unable to create user test because user id foo already exists with email bar@domain.invalid

Fixes: https://tracker.ceph.com/issues/49137
Fixes: https://tracker.ceph.com/issues/19411
Signed-off-by: Matthew Vernon <mv3@sanger.ac.uk>
(cherry picked from commit 05318d6f71e45a42a46518a0ef17047dfab83990)

15.2.9

qa/valgrind.supp: suppress free[] warnings

This is apparently not going to get fixed any time soon.

https://github.com/gperftools/gperftools/issues/792

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit 28fe149ea276693b47cc4d600780971f723672e4)

qa/suites: do not use notcmalloc flavor

teuthology now knows how to run valgrind against a tcmalloc binary

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit dc64ccf063f1ddf843fae4d2621ec25187f56135)

# Conflicts:
# qa/suites/rgw/sts/tasks/0-install.yaml
  - test not present in octopus
# qa/suites/rgw/verify/validater/valgrind.yaml

Merge pull request #39289 from kamoltat/wip-ksirivad-octopus-turn-off-backports

octopus: mgr/progress: introduce turn off/on feature

Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #39256 from ifed01/wip-ifed-fsck-spanning-blobs-octopus

octopus: os/bluestore: detect and fix "zombie" spanning blobs using fsck.

Reviewed-by: Adam Kupczyk <akupczyk@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge pull request #39253 from ifed01/wip-ifed-fix-huge-bluefs-reads-oct

octopus: os/bluestore: fix huge (>4GB) bluefs reads

Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Adam Kupczyk <akupczyk@redhat.com>

Merge pull request #39560 from neha-ojha/wip-revert-39248-octopus

octopus: Revert "mgr/pg_autoscaler: avoid scale-down until there is pressure"

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Kamoltat (Junior) Sirivadhna <ksirivad@redhat.com>

rbd-mirror: don't prune older mirror snapshots when pruning incomplete snapshot

Since we normally prune in order, we need to ensure that we don't prune older
snapshots when we need to delete an incomplete mirror snapshot since the
older snapshot might be the only remaining mirror snapshot.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 7ba9214ea5b73d0436af6c2896abf4836d741de9)

qa/workunits/rbd: show snapshot deltas during stress test failure

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit f079116e87161b55acaa08c55bf8b8e79cee8670)

qa/suites/rbd: add snapshot-based mirroring stress test

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 094bfeaf8efe1c4794a2b301314eddedfa5246f9)

librbd/deep_copy: added new migrating flag to object copy

The migration operation and the copyup state machine will set
this flag when attempting to perform a deep-copy due to a
live-migration.

This flag will prevent a possible race condition between the
start of the object deep-copy when migration was enabled and
the writing portion of the deep-copy when migration might
have completed via external means.

Fixes: https://tracker.ceph.com/issues/45694
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 1baba64e213cb808804796575d3f7969cf37a3c6)

Conflicts:
src/librbd/deep_copy/ObjectCopyRequest.cc: trivial resolution

librbd/deep_copy: added bitwise flag parameter to object copy

This initial version subsumes the original "flatten" boolean flag.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit e79f6b1c157e042f57b577bc510debb21e004ea7)

Conflicts:
src/librbd/deep_copy/ObjectCopyRequest.cc: trivial resolution
src/librbd/io/CopyupRequest.cc: trivial resolution
src/test/librbd/deep_copy/test_mock_ObjectCopyRequest.cc: trivial resolution

librbd/deep-copy: object-copy state machine must update object map

If there was no data to copy, the object-copy state machine was bypassing
the object-map update states and prematurely completing. Since the
object-map is default-initialized to all non-existent objects, this results
in incorrect state for OBJECT_EXISTS_CLEAN objects.

This commit was derived from ca0b9bfc28ef7287ca139ca9640c876223eda87b

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

librbd: deep-copy should update object-map before writing to object

For the original use-case of RBD mirroring it was (maybe) more
acceptable to write to the object before updating the object map
because an interrupted sync will be retried. However, when using
the deep-copy object copy state machine as part of copyup, it's
more likely that the object-map has the potential to become
out-of-sync with reality if it's updated after the object is
written.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit e782b85bfda8ae6487c637af0059ab94fba332d6)

Conflicts:
src/librbd/deep_copy/ObjectCopyRequest.cc: trivial resolution
src/test/librbd/deep_copy/test_mock_ObjectCopyRequest.cc: trivial resolution

librbd/object_map: diff state machine should track object existence

The deep-copy snapshot-create state machine initializes the object-map
state to non-existent for all objects. There was an assumption that the
deep-copy object-copy state machine would always update the object map
but that was being skipped for clean objects as an optimization. This
change will support a future commit to run the object-copy state machine
for existing objects.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit b81cd2460de748c71210520f8c819895f257f0c7)

Conflicts:
src/librbd/api/DiffIterate.cc: trivial resolution due to renames

test/librbd: print difference if deep-copy or migration test fails

It may appear to be useful to track the sporadic test failures
observed on jenkins, not reproducible locally.

Previously it was disabled because the output could be too
large. But after the hexdump was improved to skip repeating bytes
the output will hopefully be much smaller.

Signed-off-by: Mykola Golub <mgolub@suse.com>
(cherry picked from commit bb77f740df749de1bba0e91b03c4eb23d5586e43)

Revert "PendingReleaseNotes: mgr/pg_autoscaler"

This reverts commit 1177acb6de32331c67d707032a2ecaeff6bf8365.

Signed-off-by: Neha Ojha <nojha@redhat.com>

Revert "mgr/pg_autoscaler: avoid scale-down until there is pressure"

This reverts commit 49dba6edaa807f2c1047c40ce96a76fc5f409b82.

Reason: Needs to bake more given that it may cause pg budget issues
when multiple pools are created at the same time.

More details in https://tracker.ceph.com/issues/49364.

Signed-off-by: Neha Ojha <nojha@redhat.com>

qa/suites/rados/thrash: log-whitelist -> log-ignorelist

Signed-off-by: Neha Ojha <nojha@redhat.com>
(cherry picked from commit c0a358670a8cda2d9e319f0528d6cb9d74b6c605)

qa/suites/krbd/thrash: log-whitelist -> log-ignorelist

Signed-off-by: Neha Ojha <nojha@redhat.com>
(cherry picked from commit 01fb7e7f7b1f5c6d2c8371560ccf295567c0eb07)

qa/suites/multimds: log-whitelist -> log-ignorelist

Signed-off-by: Neha Ojha <nojha@redhat.com>
(cherry picked from commit a8c7efab2d51a249e716dad710e2ca3d01482f16)

qa: log-whitelist -> log-ignorelist

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit 2ee9365d0bc220523632753f02c61eb5c2f4fbdd)

Merge pull request #39540 from tchaikov/octopus-pr-35352

octopus: qa/tasks/vstart_runner: do not teardown test_path if "create-cluster-only"

Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>

qa/tasks/vstart_runner: do not teardown test_path if "create-cluster-only"

otherwise we could be removing a "None" directory when tearing down the cluster,
and have following failure:

Exception ignored in: <bound method LocalContext.__del__ of <__main__.LocalContext object at 0x7f99fd4a6cc0>>
Traceback (most recent call last):
  File "../qa/tasks/vstart_runner.py", line 1189, in __del__
    shutil.rmtree(self.teuthology_config['test_path'])
  File "/tmp/tmp.mmM2ugspuR/venv/lib/python3.6/shutil.py", line 477, in rmtree
    onerror(os.lstat, path, sys.exc_info())
  File "/tmp/tmp.mmM2ugspuR/venv/lib/python3.6/shutil.py", line 475, in rmtree
    orig_st = os.lstat(path)
TypeError: lstat: path should be string, bytes or os.PathLike, not NoneType

Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit 954e9a6fa67cce8e3eb8105ee858340b60b84b15)

Merge pull request #39532 from liewegas/pr-39496-octopus

octopus: mgr/cephadm: fix host refresh

Reviewed-by: Yuri Weinstein <yweinste@redhat.com>

mgr/cephadm: fix host refresh

Fixes: 01f60cf4e0a751c314120c02956d4ff941eb71b4
Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit 9df5a6d73ed21b394c01afe6c9800b6e50737c90)

Merge pull request #39393 from kamoltat/wip-ksirivad-octopus-release-notes

octopus: PendingReleaseNotes: mgr/pg_autoscaler

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #39230 from ifed01/wip-ifed-fix-pin-octopus

octopus: os/bluestore: fixing onode pinning and more

Reviewed-by: Adam Kupczyk <akupczyk@redhat.com>

Merge pull request #39004 from Vicente-Cheng/wip-48568-octopus

octopus: qa/tasks/cephfs/nfs: Check if host ip is in cluster info output

Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>
Reviewed-by: Varsha Rao <varao@redhat.com>

PendingReleaseNotes: mgr/pg_autoscaler

Added details describing the changes that
occured in after merging #39248 into octopus
upstream.

Signed-off-by: Kamoltat <ksirivad@redhat.com>

mgr/rbd_support: create mirror snapshots asynchronously

To scale up with number of images.

Fixes: https://tracker.ceph.com/issues/47827
Signed-off-by: Mykola Golub <mgolub@suse.com>
(cherry picked from commit 08151a191fd41427a751d966d523cacead5fe8fe)

Conflicts:
src/pybind/mgr/rbd_support/mirror_snapshot_schedule.py: fixed close_image bug

mgr/rbd_support: attempt to unregister watcher on mgr exit

Signed-off-by: Mykola Golub <mgolub@suse.com>
(cherry picked from commit 0bcaa261510f5bf2c69f2fe411ea3e2efe37ba60)

mgr/rbd_support: make mirror_snapshot_schedule rescan only updated pools

Signed-off-by: Mykola Golub <mgolub@suse.com>
(cherry picked from commit 69259c8d3722830c13da1e33f61fb6a72151b803)

pybind/rbd: add async mirror image get mode and info methods

Signed-off-by: Mykola Golub <mgolub@suse.com>
(cherry picked from commit 13ce488d7aaf60f85429c2e4c036362d61e5f763)

librbd: add missing declaration of rbd_aio_mirror_image_get_mode

Signed-off-by: Mykola Golub <mgolub@suse.com>
(cherry picked from commit fd5e642cd74c02e8ef00b7018baaa3093167a01f)

pybind/rbd: add image async open and close methods

Signed-off-by: Mykola Golub <mgolub@suse.com>
(cherry picked from commit b27db87c0391231b39ecd4b5aacf8ac24193c1cf)

pybind/rbd: async API for creating mirror snapshots

Signed-off-by: Mykola Golub <mgolub@suse.com>
(cherry picked from commit d1571ed2743f97280a2f3a2a70fff59e0cafc0b9)

Conflicts:
src/pybind/rbd/rbd.pyx: trival resolution re: quiesce flags

librbd: async API for creating mirror snapshots

Signed-off-by: Mykola Golub <mgolub@suse.com>
(cherry picked from commit d8468ec7e49d93860c3b8b06ff36bde539d10a94)

Conflicts:
src/librbd/api/Mirror.h/cc: no quiesce flags
src/test/rbd_mirror/test_ImageReplayer.cc: no quiesce flags

Merge pull request #39000 from Vicente-Cheng/wip-48521-octopus

octopus: cephfs: client: add ceph.{cluster_fsid/client_id} vxattrs suppport

Reviewed-by: Ramana Raja <rraja@redhat.com>

Merge pull request #38949 from Vicente-Cheng/wip-48644-octopus

octopus: cephfs: client: ensure we take Fs caps when fetching directory link count from cached inode

Reviewed-by: Ramana Raja <rraja@redhat.com>

Merge pull request #38947 from Vicente-Cheng/wip-48642-octopus

octopus: cephfs: client: set CEPH_STAT_RSTAT mask for dir in readdir_r_cb

Reviewed-by: Ramana Raja <rraja@redhat.com>

Merge pull request #38612 from ShyamsundarR/wip-47158-octopus

octopus: mgr/volumes: Add a per subvolume trash

Reviewed-by: Ramana Raja <rraja@redhat.com>
Reviewed-by: Kotresh HR <khiremat@redhat.com>

Merge pull request #38466 from Vicente-Cheng/wip-48458-octopus

octopus: cephfs: client: do not use g_conf().get_val<>() in libcephfs

Reviewed-by: Ramana Raja <rraja@redhat.com>

pybind/rados: don't try to close watch in dealoc if rados not connected

Signed-off-by: Mykola Golub <mgolub@suse.com>
(cherry picked from commit eeee5578877b2c96aa46d5ee427ea5c369a42501)

pybind/rados: add watch object bindings

Signed-off-by: Mykola Golub <mgolub@suse.com>
(cherry picked from commit dfef23d6d3fdf162f48f909b90c10577f03116bc)

pybind/rados: add Ioctx dup method

Signed-off-by: Mykola Golub <mgolub@suse.com>
(cherry picked from commit 0ec9cfcf13a1346a628962d315ed19a37e398eba)

pybind/rados: correct the declarations of C APIs

so they are consistent with the ones declared in include/rados/librados.h

Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit 31db9ce5dc21c769d78e615c2335bd2dbd94f321)

Conflicts:
src/pybind/rados/rados.pyx: trivial resolution

Merge pull request #39161 from smithfarm/wip-48496-octopus

octopus: mon: paxos: Delete logger in destructor

Reviewed-by: Brad Hubbard <bhubbard@redhat.com>

Merge pull request #39120 from rhcs-dashboard/wip-48739-octopus

octopus: mgr/dashboard: Use secure cookies to store JWT Token

Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Laura Paduano <lpaduano@suse.com>

Merge pull request #39248 from kamoltat/wip-ksirivad-octopus-backports

octopus: mgr/pg_autoscaler: avoid scale-down until there is pressure

Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge pull request #39122 from smithfarm/wip-48692-octopus

octopus: librbd: clear implicitly enabled feature bits when creating images

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
Reviewed-by: Mykola Golub <mgolub@mirantis.com>

Merge pull request #38474 from ifed01/wip-ifed-fix-avl-octopus

octopus: os/bluestore: fix inappropriate ENOSPC from avl/hybrid allocator

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Adam Kupczyk <akupczyk@redhat.com>

Merge pull request #38040 from kshtsk/wip-octopuse-cephadm-bootstrap-remote

octopus: tests: qa/task/cephadm: run cephadm only on bootstrap_remote

Reviewed-by: Thomas Bechtold <tbechtold@suse.com>
Reviewed-by: Nathan Cutler <ncutler@suse.com>

Merge pull request #38184 from smithfarm/wip-48101-octopus

octopus: rgw/rgw-admin: fixes BucketInfo for missing buckets

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #39169 from sebastian-philipp/octopus-backport-38978

octopus: mgr/cephadm: raise HEALTH_WARN when cephadm daemon in 'error' state

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #39321 from idryomov/wip-krbd-stable-writes-attr-octopus

octopus: qa: krbd_stable_pages_required.sh: move to stable_writes attribute

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #38422 from smithfarm/wip-48285-octopus

octopus: qa: ignore evicted client warnings

Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge pull request #39297 from sebastian-philipp/octopus-backport-39106

octopus: cephadm: use `apt-get` for package install/update

Reviewed-by: Michael Fritch <mfritch@suse.com>
Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #39300 from sebastian-philipp/octopus-backport-38998-38927

octopus: mgr/cephadm: try again calling ceph-volume without --filter-for-batch

Reviewed-by: Juan Miguel Olmo Martínez <jolmomar@redhat.com>
Reviewed-by: Sage Weil <sage@redhat.com>

qa: krbd_stable_pages_required.sh: move to stable_writes attribute

bdi/stable_pages_required attribute was deprecated in 5.10 and now
always returns 0.  The replacement is queue/stable_writes.  (It is
also writeable, so we can simplify these test cases somewhat in the
future.)

Fixes: https://tracker.ceph.com/issues/48232
Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
(cherry picked from commit 5adfc15b873bc16d698e7398d3ef2c2a46e8a9df)

Merge pull request #39203 from idryomov/wip-krbd-msgr2-octopus

octopus: krbd: add support for msgr2 (kernel 5.11)

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #38893 from smithfarm/wip-48519-octopus

octopus: pybind/cephfs: fix missing terminating NULL char in readlink()'s C string

Reviewed-by: Ramana Raja <rraja@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #38424 from smithfarm/wip-48375-octopus

octopus: cephfs: client: check rdonly file handle on truncate

Reviewed-by: Ramana Raja <rraja@redhat.com>

Merge pull request #38352 from smithfarm/wip-48370-octopus

octopus: cephfs: mds: dir->mark_new() should together with dir->mark_dirty()

Reviewed-by: Ramana Raja <rraja@redhat.com>

Merge pull request #38349 from smithfarm/wip-48129-octopus

octopus: cephfs: release client dentry_lease before send caps release to mds

Reviewed-by: Ramana Raja <rraja@redhat.com>

mgr/dashboard/monitoring: upgrade Grafana version due to CVE-2020-13379

Fixes: https://tracker.ceph.com/issues/48685
Signed-off-by: Alfonso Martínez <almartin@redhat.com>
(cherry picked from commit 9441fda4dc9f0257c3b686de811a01f047ef9f53)

 Conflicts:
monitoring/grafana/build/Makefile
          - Remove file as it does not exist on octopus.
src/pybind/mgr/cephadm/module.py
          - Resolve conflicts.

Merge pull request #39296 from sebastian-philipp/octopus-backport-39113

octopus: python-common: fix test_datetime_to_str_2 on non-UTC hosts

Reviewed-by: Sage Weil <sage@redhat.com>

mgr/cephadm: try again calling ceph-volume without --filter-for-batch

Fixes: https://tracker.ceph.com/issues/48870
This deals with a cephadm upgrade issue:

1. user calls `ceph orch upgrade`
2. mgr/cephadm calls `ceph orch config set mgr.x container_image <new-container>`
3. standby mgr gets upgraded
4. mgr failover to new mgr
5. mgr/cephadm calls `_refresh_host_devices`
6. `_refresh_host_devices` calls` ceph orch config get osd container_image`.
  But this returns the old image
7. `_refresh_host_devices` calls `ceph-volume ... --filter-for-batch`
  with an image that doesn't support `filter-for-batch`

The idea is to simply retiry calling ceph-volume inventory without `--filter-for-batch`

(also removed `out` being used without being declared)

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
(cherry picked from commit ede3d6d147dd7b99d37aee5c5fb9340f2878db18)

Conflicts:
  src/pybind/mgr/cephadm/tests/test_cephadm.py

mgr/cephadm: Properly handle JSON Decode error

Fixes 6d759fb5deac0c52b3c738a2e695738228749420

I.e. don't use `out`, until it is acutally defined

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
(cherry picked from commit eb1a40c3ea8e19259d8ab68a6eeb16d27e4cdbda)

Conflicts:
src/pybind/mgr/cephadm/serve.py

cephadm: use `apt-get` for package install/update

avoids errors during prepare-host:
```
apt: stderr WARNING: apt does not have a stable CLI interface. Use with caution in scripts.
```

Fixes: https://tracker.ceph.com/issues/49032
Signed-off-by: Michael Fritch <mfritch@suse.com>
(cherry picked from commit fa4706039cfece36815df46cd4452dc25448c340)

python-common: fix test_datetime_to_str_2 on non-UTC hosts

The old test parsed to a datetime without a tz, which was interpreted as
the local time zone when rendering back to a string.  Specify that it's a
UTC datetime so that behavior is consistent regardless of the test host
timezone.

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit 78aca4db249c409d0cd5a24bfae81e55cf930bc3)

Merge pull request #39170 from sebastian-philipp/octopus-backport-38910

octopus: cephadm: fix rgw osd cap tag

Merge pull request #39166 from sebastian-philipp/octopus-backport-38804-39003

octopus: cephadm: silence "Failed to evict container" log msg

Reviewed-by: Michael Fritch <mfritch@suse.com>

Merge pull request #39167 from sebastian-philipp/octopus-backport-38850

octopus: mgr/cephadm: tolerate old host inventory without 'hostname' key

Reviewed-by: Michael Fritch <mfritch@suse.com>

Merge pull request #39168 from sebastian-philipp/octopus-backport-38945

octopus: qa/cephadm: Add yaml output to smoke test

Reviewed-by: Michael Fritch <mfritch@suse.com>

Merge pull request #39171 from sebastian-philipp/octopus-backport-39083

octopus: python-common/drivegroups: avoid dropping "rotational: 0" from Device Selection

Reviewed-by: Michael Fritch <mfritch@suse.com>

mgr/progress: introduce turn off/on feature

progress module can be turned off/on by using
the commands: 'progress off' and 'progress on'

As well as refractoring teuthology test suite
to prevent future bugs that can possibly occur

fixes: https://tracker.ceph.com/issues/47238

Signed-off-by: kamoltat <ksirivad@redhat.com>
(cherry picked from commit 993bb02b30cf73a1c1c70da1ef266be8373d56dd)

Conflicts:
PendingReleaseNotes - update relesae notes
qa/tasks/mgr/test_progress.py - trivial fix

rgw: fix trailing null in object names of multipart reuploads

the overload being called had a comment saying:
/* size should be the required string size + 1 */

this calls a different overload that returns std::string and handles
this null termination correctly

Fixes: https://tracker.ceph.com/issues/48874
Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit 2ede30782b00fe28ac714047f55b11b76ae1593f)

ceph-volume: mock VGs with string properties in tests

The real things uses string properties as well.

Signed-off-by: Jan Fajerski <jfajerski@suse.com>
(cherry picked from commit c4bbb1b999b08bd84d793537f2602fc8e4c93bae)

ceph-volume: add some flexibility to bytes_to_extents

For batch we want to report the projected sizes before creating any LVM
structures, we use the device size for that. This means we pass this
projected size to lvm/api.py::create_lv. This could result in situations
where we request more extents for the new LV than are actually
available, as LVM takes up some available space on the block device
(plus alignment).
We fix this by adjusting the extent count downwards if its less than 1% off the
requested value.

Fixes: https://tracker.ceph.com/issues/47758
Signed-off-by: Jan Fajerski <jfajerski@suse.com>
(cherry picked from commit ece6e7eb9a92cb944b2f201a05a5acfb877c2dba)

os/bluestore: fix huge (>4GB) bluefs reads

Signed-off-by: Igor Fedotov <ifedotov@suse.com>
(cherry picked from commit fe558c4dd12b25b2d523987f22fde5b5898cfb83)

 Conflicts:
 (trivial) src/os/bluestore/BlueFS.h
 (trivial) src/os/bluestore/BlueRocksEnv.cc
 (trivial) src/test/objectstore/test_bluefs.cc

mgr/pg_autoscaler: avoid scale-down until there is pressure

The autoscaler will start out with scaling each
pools to have a full complements of pgs from the start
and will only decrease it when pools need more due to
increased usage.

Introduced a unit test that tests only the
function get_final_pg_target_and_ratio() which
deals with the distrubtion of pgs amongst the
pools

Edited workunit script to reflect the change
of how pgs are calculated and distrubted.

Signed-off-by: Kamoltat <ksirivad@redhat.com>
(cherry picked from commit daeb6f6ac0c8f77ae07147f9d1e2ed18d6d8e4cc)

Conflicts:
src/pybind/mgr/pg_autoscaler/module.py - trivial fix

mgr/zabbix: format ceph.[{#POOL},percent_used as float

also, multiply it with 100, as we are using 1.0 for 100% here.

for more details regarding the mapping from value_type to its values.
see
https://www.zabbix.com/documentation/4.0/manual/api/reference/item/object

Fixes: https://tracker.ceph.com/issues/48825
Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit c8e5a051a016fa29a621ef021f942d9dac97205a)

Merge pull request #38430 from smithfarm/wip-48281-octopus

octopus: osd: fix bluestore bitmap allocator calculate wrong last_pos with hint

Reviewed-by: Igor Fedotov <ifedotov@suse.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge pull request #38427 from smithfarm/wip-48283-octopus

octopus: rpm,deb: change sudoers file mode to 440

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #38333 from b-ranto/wip-prom-fixes-octopus

octopus: mgr/prometheus: Sync and backport prometheus fixes

Reviewed-by: Kefu Chai <kchai@redhat.com>

os/bluestore: detect and fix "zombie" spanning blobs using fsck.

Signed-off-by: Igor Fedotov <ifedotov@suse.com>
(cherry picked from commit 18d120dc2c15c0cf8d3b27ddba3c7f2057856555)

 Conflicts:
 (trivial)      src/os/bluestore/BlueStore.cc
 (trivial)      src/os/bluestore/BlueStore.h

Merge PR #38425 into octopus

* refs/pull/38425/head:
lvm/create.py: fix a typo in the help message

Reviewed-by: Jan Fajerski <jfajerski@suse.com>