cephadm: fixup expect-hostname message

show either a success or a failure message

Signed-off-by: Michael Fritch <mfritch@suse.com>
(cherry picked from commit ff3dd995b5f07d1a4d48e1f62d5054dadea9eb72)

cephadm: add --shared_ceph_folder to shell cmd

It can be useful to share ceph src when running the `cephadm shell`
command in order to do quick tests.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
(cherry picked from commit 37ea06cfd0937df6230383d16de9fbda9a5a6f3f)

doc/cephadm: core dumps

Signed-off-by: Sebastian Wagner <sewagner@redhat.com>
(cherry picked from commit 223af588a994eabc75b9f6d2bb33df761d6944b8)

qa/suites/orch/cephadm: verify /var/log/ceph/$fsid ownership

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit 4f71cbba1a41426e3a858b11f03fd0c5f795dc95)

cephadm: only make_log_dir for ceph daemons

For non-ceph daemons, (1) they don't log to /var/log/ceph, and (2) the
container image isn't a ceph image, which means the uid/gid extraction
won't have the correct uid/gid and we'll end up with a log directory that
ceph daemons no longer have write permissions for.

Fixes: https://tracker.ceph.com/issues/53257
Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit 393960c164bf84b21ffee6af65c19d968b416fb2)

mgr/cephadm: check_for_moved_osds: don't run post_remove

Signed-off-by: Sebastian Wagner <sewagner@redhat.com>
(cherry picked from commit 70d27f24f837f2d962ce7d1f2672b0d71f5f2f7a)

mgr/cephadm: test_cephadm: make use of with_osd_daemon

Signed-off-by: Sebastian Wagner <sewagner@redhat.com>
(cherry picked from commit 91574d47e9ba6cd0426370bf16a68017cb00f8c4)

Conflicts:
src/pybind/mgr/cephadm/tests/test_cephadm.py

mgr/cephadm: check_for_moved_osds: verify OSDs are up

Signed-off-by: Sebastian Wagner <sewagner@redhat.com>
(cherry picked from commit 47dd258cf56039d510c8d8a0d275c9104b9f49eb)

mgr/orch: Add DaemonDescriptionStatus `starting` and `unknown`

Signed-off-by: Sebastian Wagner <sewagner@redhat.com>
(cherry picked from commit 501ecf035e982af6f2020b3bb77d0ac505313b73)

mgr/cephadm: test_cephadm: Add some more tests for OrchestratorEvent

Signed-off-by: Sebastian Wagner <sewagner@redhat.com>
(cherry picked from commit 7d1bea2bb8777ac6c4d08b9042af1f6597ed2906)

mgr/cephadm: check_for_moved_osds: Add OrchEvent

Signed-off-by: Sebastian Wagner <sewagner@redhat.com>
(cherry picked from commit 25d27fb6c1bd3520564e624965fa3b6f81a21ac0)

mgr/cephadm: Add `_check_for_moved_osds`

Fixes: https://tracker.ceph.com/issues/49571
Signed-off-by: Sebastian Wagner <sewagner@redhat.com>
(cherry picked from commit 0ef6788fe6e4c3cfbebb401748eb9da29a56975c)

Conflicts:
src/pybind/mgr/cephadm/serve.py

mgr/cephadm: osd: prime new daemons with proper daemon_id type

No one noticed this before, cause it's going to fix itself next time we
check teh daemons on that host

Signed-off-by: Sebastian Wagner <sewagner@redhat.com>
(cherry picked from commit 3fa14458a853b68856e45376833e8c12038be7e6)

mgr/orch: Deprecate `ceph orch apply -i <osd_spec_file>`

* Remove usage
* Simplify error messages

Fixes: https://tracker.ceph.com/issues/47401
Signed-off-by: Sebastian Wagner <sewagner@redhat.com>
(cherry picked from commit 3fef006e0ea02f8d4b508f9d817624ee0129eadc)

qa/suites/orch/cephadm: verify that 'orch ls' reports OSDs properly

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit e1cd91098f4e566813dc98daa85e6473ccc6eed6)

mgr/cephadm: show unmanaged OSDs under 'osd' service

1- If the unit.meta file service_name = osd.NNN (which is true for lots of
OSDs deployed on older version of cephadm) then ignore the field entirely.

2- If an OSD has not service_name (see above) then show it under the 'osd'
service (instead of 'osd.unmanaged').

Sample 'ceph orch ls' output with a drivegroup + unmanaged OSD:

NAME      PORTS  RUNNING  REFRESHED  AGE  PLACEMENT
...
osd                    1  85s ago    -    <unmanaged>
osd.hdds               5  85s ago    2s   *
...

Fixes: https://tracker.ceph.com/issues/53235
Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit f13a3e5b3ca1547bba9deae3615252165bc7cac9)

qa/suites/orch/cephadm: add repave-all test case

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit 4abe4c6a78a941c8a87f9f3bdf1c731618415993)

Merge pull request #44389 from aaSharma14/wip-53689-pacific

pacific: mgr/dashboard: Cluster Expansion - Review Section: fixes and improvements

Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
Reviewed-by: Pere Diaz Bou <pdiazbou@redhat.com>

Merge pull request #44400 from aaSharma14/wip-53717-pacific

pacific: mgr/dashboard: introduce separate front-end component for API docs

Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>

mgr/dashboard: introduce seperate frontend component for API docs

This PR intends to add a separate frontend component for the API docs that are currently being rendered from the python code.

Fixes: https://tracker.ceph.com/issues/50955
Signed-off-by: Aashish Sharma <aasharma@redhat.com>
(cherry picked from commit 2c396be7cc47762290199609b99fba09f3573899)

Conflicts:
src/pybind/mgr/dashboard/frontend/package-lock.json
src/pybind/mgr/dashboard/frontend/package.json (added new swagger packages)

mgr/dashboard: Cluster Expansion - Review Section: fixes and improvements

Ensure "Storage capacity" keeps the "Description : Value" approach ("Number of devices: X" and "Raw Capacity: Y" in different lines).Correct issue with "host by services" host count

Fixes: https://tracker.ceph.com/issues/53128
Signed-off-by: Aashish Sharma <aasharma@redhat.com>
(cherry picked from commit 2ba1a5a2666674a935c9288f5b547532ba214f54)

Merge pull request #44376 from rhcs-dashboard/wip-53687-pacific

pacific: mgr/dashboard: Implement drain host functionality in dashboard

Reviewed-by: Aashish Sharma <aasharma@redhat.com>
Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>

Merge pull request #44369 from ceph/wip-yuri-pacific-p2p-pacific

qa/tests: replaced 16.2.6 with 16.2.7 version

Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge pull request #44377 from rhcs-dashboard/wip-53679-pacific

pacific: mgr/dashboard: run-backend-api-tests.sh: Older setuptools

Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: David Galloway <dgallowa@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>

pybind/mgr/dashboard/run-backend-api-tests: Older setuptools

https://github.com/pypa/setuptools/issues/2353

Signed-off-by: David Galloway <dgallowa@redhat.com>
(cherry picked from commit 4ab2df179b0bf0b123a59c649b779bba41e4bb3d)

mgr/dashboard: Implement drain host functionality in dashboard

Fixes: https://tracker.ceph.com/issues/51587
Signed-off-by: Nizamudeen A <nia@redhat.com>
(cherry picked from commit 524c340562d6b44c329ffa635cf3203e059171ad)

Merge pull request #44046 from rhcs-dashboard/wip-53344-pacific

pacific: mgr/dashboard: NFS non-existent files cleanup

Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
Reviewed-by: Pere Diaz Bou <pdiazbou@redhat.com>
Reviewed-by: Yuri Weinstein <yweins@redhat.com>

qa/tests: replaced 16.2.6 with 16.2.7 version

Signed-off-by: Yuri Weinstein <yweinste@redhat.com>

Merge pull request #44316 from rhcs-dashboard/wip-53296-pacific

pacific: monitoring/grafana: Grafana query tester

Reviewed-by: Aashish Sharma <aasharma@redhat.com>
Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
Reviewed-by: Pere Diaz Bou <pdiazbou@redhat.com>

Merge pull request #44319 from guits/wip-53617-pacific

pacific: ceph-volume: make it possible to skip needs_root()

Merge pull request #43964 from cfsnyder/wip-51484-pacific

pacific: pybind/mgr/balancer: define Plan.{dump,show}()

Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>

pybind/mgr/balancer: define Plan.{dump,show}()

as they are called by the commands

Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit 0d48b03)

Conflicts:
src/pybind/mgr/balancer/module.py

Cherry-pick notes:
- Conflicts due to missing type annotations on Pacific

mgr/dashboard: disable Promql test in ARM

Temporarily disable this test while debugging the issue (since https://github.com/ceph/ceph/pull/43669
originally passed the ARM check).

Fixes: https://tracker.ceph.com/issues/53451
Signed-off-by: Ernesto Puerta <epuertat@redhat.com>
(cherry picked from commit d10b0b7e728ccd28e1002271ce71e6f94a074256)

ceph-volume: make it possible to skip needs_root()

Add the possibility to skip the `needs_root()` decorator.
See linked tracker for details.

Fixes: https://tracker.ceph.com/issues/53511
Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
(cherry picked from commit 068a1d2a300bc21e9aa08142c4195970ef480e41)

monitoring/grafana: Grafana query tester

Signed-off-by: Pere Diaz Bou <pdiazbou@redhat.com>
(cherry picked from commit 44d3e4c264506154373ffaeb13d6c924c580e6b5)

Merge pull request #44297 from rhcs-dashboard/wip-53510-pacific

pacific: mgr/dashboard: fix frontend deps' vulnerabilities

Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>

Merge pull request #44273 from rhcs-dashboard/wip-53576-pacific

pacific: mgr/dashboard: introduce HAProxy metrics for RGW

Reviewed-by: Aashish Sharma <aasharma@redhat.com>
Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Avan Thakkar <athakkar@redhat.com>

Merge pull request #43187 from rhcs-dashboard/wip-51979-pacific

pacific: mgr/prometheus: Fix metric types from gauge to counter

Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: p-se <NOT@FOUND>

Merge pull request #44091 from aaSharma14/wip-53383-pacific

pacific: mgr/dashboard: dashboard does not show degraded objects if they are less than 0.5% under "Dashboard->Capacity->Objects block

Reviewed-by: Aashish Sharma <aasharma@redhat.com>
Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
Reviewed-by: Pere Diaz Bou <pdiazbou@redhat.com>

Merge pull request #43954 from guits/wip-53280-pacific

pacific: cephadm/ceph-volume: do not use lvm binary in containers

Merge pull request #44149 from rhcs-dashboard/wip-53437-pacific

pacific: mgr/dashboard: avoid tooltip if disk_usage=null and fast-diff enabled

Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Pere Diaz Bou <pdiazbou@redhat.com>

mgr/dashboard: fix frontend deps' vulnerabilities

- Remove npm-force-resolutions: no resolution needed anymore and this is modifying package-lock.json every time it is run (striping last empty line).
- Add .npmrc: save exact version by default; do not launch audit report when installing.

Fixes: https://tracker.ceph.com/issues/48005
Signed-off-by: Alfonso Martínez <almartin@redhat.com>
(cherry picked from commit f08c0db689dc6bd29323ac03a91c69e2fe7365a2)

 Conflicts:
src/pybind/mgr/dashboard/frontend/package-lock.json
    - Accept version from master branch.
src/pybind/mgr/dashboard/frontend/package.json
    - Accept version from master branch.

Merge pull request #44086 from rhcs-dashboard/wip-53380-pacific

pacific: mgr/dashboard: upgrade Cypress to the latest stable version

Reviewed-by: Waad Alkhoury <walkhour@redhat.com>
Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
Reviewed-by: Yuri Weinstein <yweins@redhat.com>

ceph-volume: remove --root param from nsenter cmd

This is redundant and makes nsenter throw messages like following:
```
  Failed to find sysfs mount point
  dev/block/11:0/holders/: opendir failed: Not a directory
  dev/block/252:0/holders/: opendir failed: Not a directory
  dev/block/253:0/holders/: opendir failed: Not a directory
  dev/block/252:1/holders/: opendir failed: Not a directory
  dev/block/253:1/holders/: opendir failed: Not a directory
  dev/block/252:2/holders/: opendir failed: Not a directory
  dev/block/253:2/holders/: opendir failed: Not a directory
  dev/block/252:3/holders/: opendir failed: Not a directory
  dev/block/253:3/holders/: opendir failed: Not a directory
  dev/block/252:16/holders/: opendir failed: Not a directory
  dev/block/252:32/holders/: opendir failed: Not a directory
  dev/block/252:48/holders/: opendir failed: Not a directory
  dev/block/252:64/holders/: opendir failed: Not a directory
  ```

Fixes: https://tracker.ceph.com/issues/52926
Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
(cherry picked from commit e4667e81301295f4c81328505e4376d2aef66fb2)

qa/tasks/nvme_loop: loop until 'nvme list' shows new devs

Sometimes this doesn't happen immediately.

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit d4a1ec2d0611192f4016006afcf72d57a31a6a1e)

qa/suites/roch/cephadm/smoke: use nvme_loop

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit 6436b7e05f56027fccdfef7fb0a3349139141e09)

qa/suites/orch/cephadm/smoke-roleless: no osd roles, use nvme loop

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit 4206b28dd7097ee079eb389e53f16a5afd55e21f)

qa/overrides/nvme_loop: reusable fragment to use nvme_loop devs

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit 8d4b87031638e48fddc27769003b09042643e11e)

qa/tasks/cephadm: wait for osds to start explicitly

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit dda15a792431a509d18210bb00471015493d6ce6)

qa/tasks/cephadm: if no osd roles, --all-available-devices

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit d3c9486ed9406d09904dd542eae05680d5f09498)

qa/tasks/nvme_loop: set up nvme_loop on scratch_devs

Using an nvme loop device makes the LVs look like "real" disks,
which means we can exercise all of the normal code paths for
provisioning, deprovisioning, and zapping.

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit 65cf69c6ff5ee44319e3b2ca2afb36cea7b7c81c)

qa/suites/orch/cephadm/osds: add osd rm/zap/add test

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit 96319a3d39d9ca254b86fabbab8cd19cf53ec4f8)

cephadm: mount rootfs in osd containers

See ceph-volume tracker for details [1]

[1] https://tracker.ceph.com/issues/52926

Fixes: https://tracker.ceph.com/issues/51592
Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
(cherry picked from commit 48b369e2caf3222bf594dc09f87b5969a53dfbe7)

ceph-volume: implement lvm wrapper

ceph-volume should run pv/vg/lv commands in the host namespace rather than
running them inside the container in order to avoid lvm metadata corruption.

Fixes: https://tracker.ceph.com/issues/52926
Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
(cherry picked from commit 4d33630deeaee51578868fb29337da802e9cb231)

Merge pull request #44209 from guits/wip-53371-pacific

pacific: ceph-volume: human_readable_size() refactor

Merge pull request #43649 from ideepika/wip-53019-pacific

pacific: test/librbd: harden RemoveFullTry tests

Reviewed-by: Mykola Golub mgolub@suse.com

mgr/dashboard: introduce HAProxy metrics for RGW

Fixes: https://tracker.ceph.com/issues/53311
Signed-off-by: Avan Thakkar <athakkar@redhat.com>
(cherry picked from commit 8d83126e5168d03b44ec476de08502e7a12d3bf2)

Merge pull request #44199 from ideepika/wip-pwl-backports-december-pacific

pacific: librbd/cache/pwl: misc backports

Reviewed-by: Jianpeng Ma <jianpeng.ma@intel.com>
Reviewed-by: Mykola Golub mgolub@suse.com
Reviewed-by: Sunny Kumar <sunkumar@redhat.com>

librbd: Read request need exclusive-lock when enable  pwl-cache.

TestLibRBD.TestFUA descript the following workload:
a)write/read the same image w/ pwl-cache
  write_image = open(image_name);
  read_image  = open(image_name);
b)i/o workload is:
   write(write_image)
      write need EXLock and require EXLOCK

  read(read_image)
     in ExclusiveLock<I>::init(), firstly read need EXLOCK
     so will require EXLOCK. write_image release EXLOCK(will
     flush data to osd and remove cache). read_image init pwl-cache
     and read-io firstly enter pwl-cache and missed and then read
     from osd.

   write(write_image)
     write need EXLOCK and require EXLOCK. This make read_image remove
     empty cache. write_image init cache pool and write data to cache.

   read(read_image)
       In send_set_require_lock(), it set write need EXLOCK.
       So read don't require EXLOCK and dirtyly read from osd.

Because second-read  don't need EXLOCK and make write_image don't
release EXLOCK(flush dirty data to osd and  shutdown pwl-cache).
This make second-read don't read the latest data.

So we should make read also need EXLOCK when enable pwl-cache.

Fixes: https://tracker.ceph.com/issues/51438
Tested-by: Feng Hualong <hualong.feng@intel.com>
Signed-off-by: Jianpeng Ma <jianpeng.ma@intel.com>
(cherry picked from commit 621facb6e66ce92ca36d566c78bc065a9666639e)

librbd: send FLUSH_SOURCE_INTERNAL when do copy/deep_copy.

copy/deep_copy use object_map to judge whether object exist.
If w/ librbdo pwl cache, flush can't flush data to osd which
change objectmap state. So we should send flush w/ FLUSH_SOURCE_INTERNAL
to make data flush to osd.

Fixes:https://tracker.ceph.com/issues/53057
Signed-off-by: Jianpeng Ma <jianpeng.ma@intel.com>
(cherry picked from commit a2ae83f8aab18933eae77cf3034b740082a39e4f)

qa/suites/rbd: added SSD PWL cache mode to tests

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit e2c9c5cd4146c9a86100ca3aaf407127b1df1310)

librbd/cache/pwl: it should in apply_metadata set discard_granularity for pwl cache.

Function apply_meta can overwrite discard_granularity_bytes
based on option.

Fixes:https://tracker.ceph.com/issues/53434

Signed-off-by: Jianpeng Ma <jianpeng.ma@intel.com>
(cherry picked from commit 86223d039b587e768d71ce1266e85dde8a46a50a)

librbd/cache/pwl: revert patch 76f4d29d92be3f9f45767cb1ac6cc50.

In fact, After BlockGuard to fix flush reorder issue, patch
76f4d29d9 don't need. So revert.

Signed-off-by: Jianpeng Ma <jianpeng.ma@intel.com>
(cherry picked from commit cc8dae1d89a6cc44f6133a10424d767fc3291dd2)

librbd/cache/pwl: Using BlockGuard control overlap ops  order when flush to osd.

In process of tests, we met some inconsistent-data problem. Test case
mainly use write,then discard to detect data consistent.
W/o pwl, write/discard are synchronous ops. After write, data already
located into osd. But w/ pwl, we use asynchronous api to send ops to
osd.
Although we mare sure send order. But send-order don't makre sure
complete order. This mean pwl keep order of write/discard. But it
don't keep the same semantics which use synchronous api. W/ pwl, it make
synchronous to asynchronous. For normal ops, it's not problem. But if
connected-commands w/ overlap, it make data inconsistent.
So we use BlockGuard to solve this issue.

Fixes: https://tracker.ceph.com/issues/49876
Fixes: https://tracker.ceph.com/issues/53108
Signed-off-by: Jianpeng Ma <jianpeng.ma@intel.com>
(cherry picked from commit 8e8f3ef516e98da011f3086f8e78a2fa261293ed)

Merge pull request #43745 from mfoliveira/wip-53101-pacific

pacific: os/bluestore/AvlAllocator: introduce bluestore_avl_alloc_ff_max_* options

Reviewed-by: Igor Fedotov <ifedotov@suse.com>

Merge pull request #44226 from ceph/pacific-m2r

doc: Use older mistune

16.2.7

Merge pull request #44201 from sebastian-philipp/qa-suites-orch-cephadm-mgr-nfs-upgrade-0-distro-pacific

qa/suites/orch/cephadm: mgr-nfs-upgrade: add missing 0-distro dir

Reviewed-by: Michael Fritch <mfritch@suse.com>

Merge pull request #44164 from cfsnyder/wip-52450-pacific

pacific: rpm, debian: move smartmontools and nvme-cli to ceph-base

Reviewed-by: Yaarit Hatuka <yaarithatuka@gmail.com>

Merge pull request #44154 from neha-ojha/wip-perf-pacific

pacific: qa: miscellaneous perf suite fixes

Reviewed-by: Yuri Weinstein <yweinste@redhat.com>

doc: Use older mistune

https://github.com/miyakogi/m2r/issues/66

Signed-off-by: David Galloway <dgallowa@redhat.com>
(cherry picked from commit ed2ad24a4ba3ad3f8103926bfea2466b9eb61222)

pacific: follow up on get_first_*() refactor

The backport for the lvm migrate feature in pacific was merged after the
get_first_*() refactor backport.
So we have still some old references to `get_single_lv()`

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>

ceph-volume: human_readable_size() refactor

This commit refactors the `human_readable_size()` function.

The current implementation has a couple of issues:

in a 'human readable' mindset, I would expect `human_readable_size(1024)` to
return '1.00 KB' instead of '1024.00 KB'.

```
In [1]: from ceph_volume.util.disk import human_readable_size

In [2]: human_readable_size(1024)
Out[2]: '1024.00 B'

In [3]: human_readable_size(1024*1024)
Out[3]: '1024.00 KB'

```

Also, it doesn't support PB unit:

```
In [4]: human_readable_size(1024*1024*1024*1024*1024)
Out[4]: '1024.00 TB'

In [5]: human_readable_size(1024*1024*1024*1024*1024*1024)
---------------------------------------------------------------------------
IndexError                                Traceback (most recent call last)
<ipython-input-31-0859861661dc> in <module>
----> 1 human_readable_size(1024*1024*1024*1024*1024*1024)

~/GIT/ceph/src/ceph-volume/ceph_volume/util/disk.py in human_readable_size(size)
    640     return "{size:.2f} {suffix}".format(
    641         size=size,
--> 642         suffix=suffixes[suffix_index])
    643
    644

IndexError: list index out of range
```

This commit fixes this.

Fixes: https://tracker.ceph.com/issues/48492
Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
(cherry picked from commit 6940856f233f4d365a119eed90ff88fd918f6916)

rpm, debian: move smartmontools and nvme-cli to ceph-base

We wish to be able to scrape SMART and NVMe metrics from OSD and MON
nodes. For this we require / recommend smartmontools and nvme-cli
dependencies for both the ceph-osd and ceph-mon packages.  However, the
sudoers file (which is required for invoking `smartctl` by user 'ceph')
was installed only in the ceph-osd package.  Since different packages
cannot own the same file, and because we want to be able to scrape from
every daemon, we move the dependencies and the sudoers installation to
ceph-base. For generalization, we rename:
sudoers.d/ceph-osd-smartctl -> sudoers.d/ceph-smartctl

Fixes: https://tracker.ceph.com/issues/50657
Signed-off-by: Yaarit Hatuka <yaarit@redhat.com>
(cherry picked from commit 7ca39fa92b47427af2f1c6000c653bb4dffc47fe)

Conflicts:
ceph.spec.in

Cherry-pick notes:
- Conflict in ceph.spec.in because setuptools not required for OSD in pacific

qa/suites/orch/cephadm: mgr-nfs-upgrade: add missing 0-distro dir

move qa/suites/orch/cephadm/mgr-nfs-upgrade/0-*

to qa/suites/orch/cephadm/mgr-nfs-upgrade/0-distro/*

Fixes: https://tracker.ceph.com/issues/53365
Signed-off-by: Sebastian Wagner <sewagner@redhat.com>

qa/suites/rados/perf/ceph.yaml: remove rgw

This is no longer required because we removed cosbench workloads in
fd350fd0150a2d4072f055658c20314a435a19ba. This is also required to prevent
failures like the following or any other changes that break the rgw task:

```
2021-08-06T20:13:25.812 INFO:teuthology.orchestra.run.smithi060.stderr:curl: (7) Failed to connect to smithi060.front.sepia.ceph.com port 80: Connection refused
2021-08-06T20:15:33.813 ERROR:teuthology.contextutil:Saw exception from nested tasks
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_04c2febe7099917d97a71271f17abb5710030132/teuthology/contextutil.py", line 31, in nested
    vars.append(enter())
  File "/usr/lib/python3.6/contextlib.py", line 81, in __enter__
    return next(self.gen)
  File "/home/teuthworker/src/github.com_ceph_ceph-c_3c0f8c8164075af7aac4d1f2805d3f4580709461/qa/tasks/rgw.py", line 191, in start_rgw
    wait_for_radosgw(url, remote)
  File "/home/teuthworker/src/github.com_ceph_ceph-c_3c0f8c8164075af7aac4d1f2805d3f4580709461/qa/tasks/util/rgw.py", line 94, in wait_for_radosgw
    assert exit_status == 0
AssertionError
```

Signed-off-by: Neha Ojha <nojha@redhat.com>
(cherry picked from commit 119544bb29e253322af33e593ffd09e325c2af8a)

qa: remove cosbench workloads from perf suites

Due to https://tracker.ceph.com/issues/49139

Signed-off-by: Neha Ojha <nojha@redhat.com>
(cherry picked from commit fd350fd0150a2d4072f055658c20314a435a19ba)

qa: use ubuntu_latest for perf suites

Signed-off-by: Neha Ojha <nojha@redhat.com>
(cherry picked from commit 5957d1797a4f67b4545c2554dff240463af87359)

mgr/dashboard: avoid tooltip if disk_usage=null and fast-diff enabled

Fixes: https://tracker.ceph.com/issues/53404
Signed-off-by: Avan Thakkar <athakkar@redhat.com>
(cherry picked from commit 071c3b68a131fa9ff3403ccc6ca0e1075d95a048)

vstart,init-ceph: include build bin/ in $PATH

This allows mgr/cephadm to run radosgw-admin.

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit 1ad8811d1b37ca5d46fd7ae807974e72c12d83a4)

 Conflicts:
src/init-ceph.in
    - Resolve conflicts.

mgr/dashboard: upgrade Cypress to the latest stable version

- Remove unneeded dependency that was causing UI performance issues: zone.js
- Ignore 'ResizeObserver loop limit exceeded' error.
- run-frontend-e2e-tests.sh refactoring: create rgw dashboard user through
  'ceph dashboard set-rgw-credentials' and use it on rgw buckets' tests.

Fixes: https://tracker.ceph.com/issues/53357
Signed-off-by: Alfonso Martínez <almartin@redhat.com>
(cherry picked from commit 3e4e29590aa1742fc3b44d21389325a13cca8199)

Conflicts:
        src/pybind/mgr/dashboard/frontend/package-lock.json
        - Regenerate file to align to pacific.
Signed-off-by: Alfonso Martínez <almartin@redhat.com>

Merge pull request #44047 from rhcs-dashboard/wip-53343-pacific

pacific: mgr/dashboard: rgw daemon list: add realm column

Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>

Merge pull request #44077 from rhcs-dashboard/wip-53377-pacific

pacific: mgr/dashboard: Predefine labels in create host form

Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>

Merge pull request #44051 from rhcs-dashboard/wip-53364-pacific

pacific: mgr/dashboard: provisioned values is misleading in RBD image table

Reviewed-by: Aashish Sharma <aasharma@redhat.com>
Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Avan Thakkar <athakkar@redhat.com>

mgr/dashboard: dashboard does not show degraded objects if they are less than 0.5% under "Dashboard->Capacity->Objects block

This PR is intended to fix this issue

Fixes: https://tracker.ceph.com/issues/53242
Signed-off-by: Aashish Sharma <aasharma@redhat.com>
(cherry picked from commit 4def9d89c51ec2715066d119330a0f692fcf2f88)

Merge pull request #43811 from liewegas/pr-53134-pacific

pacific: backport mgr/nfs bits

Reviewed-by: Neha Ojha <nojha@redhat.com>

mgr/dashboard: Predfine labels in create host form

Also retains the previously created labels by user in the form

Fixes: https://tracker.ceph.com/issues/53315
Signed-off-by: Nizamudeen A <nia@redhat.com>
(cherry picked from commit e228bf7563a13cc9f0f49a086c5df391c74002b1)

Merge pull request #43931 from rhcs-dashboard/wip-52528-pacific

pacific: mgr/prometheus: offer ability to disable cache

Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: p-se <NOT@FOUND>

Merge pull request #44057 from rhcs-dashboard/wip-53363-pacific

pacific: mgr/dashboard: Daemon Events listing using bootstrap class

Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
Reviewed-by: Pere Diaz Bou <pdiazbou@redhat.com>

Merge pull request #43960 from guits/wip-53284-pacific

pacific: ceph-volume: `get_first_lv()` refactor

Merge pull request #44056 from rhcs-dashboard/wip-53366-pacific

pacific: mgr/dashboard: fix flaky inventory e2e test

Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>

Merge pull request #44053 from rhcs-dashboard/wip-53297-pacific

pacific: mgr/dashboard: Device health status is not getting listed under hosts section

Reviewed-by: Aashish Sharma <aasharma@redhat.com>
Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>

mgr/dashboard: Daemon Events listing using bootstrap class

Fixes: https://tracker.ceph.com/issues/53282
Signed-off-by: Nizamudeen A <nia@redhat.com>
(cherry picked from commit 1bad306efd628dd4940b1082c08199bbbaf0b581)

mgr/dashboard: fix flaky inventory e2e test

When `inventory.getTableCount('total').should('be.eq', totalDiskCount);`
this line is executed the table was not loaded properly and hence the
getTableCount returns 0 on the first try but on second try it passes
since the table is loaded. But in orch e2es the retries are set to 0. I
am not sure if it makes sense to set it to 1. Anyway I am adapting the
test a bit to expect the count to be equal to totalDiskCount so that the
test will wait a bit.

Fixes: https://tracker.ceph.com/issues/53353
Signed-off-by: Nizamudeen A <nia@redhat.com>
(cherry picked from commit cdce5f78ad4a9f4a91837322b83cee89a393a0ca)

mgr/dashboard: Device health status is not getting listed under hosts section

Device health is shown as failed to retrieve data under Hosts > Device Health section. This PR intends to fix this issue.

Fixes: https://tracker.ceph.com/issues/53209
Signed-off-by: Aashish Sharma <aasharma@redhat.com>
(cherry picked from commit 8b868d20c470289c2f23049ac5dbbd2ba0307385)

Merge pull request #43972 from gregsfortytwo/wip-53259-pacific

pacific: mon: MonMap: display disallowed_leaders whenever they're set

Reviewed-by: Laura Flores <lflores@redhat.com>
Reviewed-by: Samuel Just <sjust@redhat.com>

Merge pull request #43971 from gregsfortytwo/wip-53239-pacific

pacific: mon: MonMap: do not increase mon_info_t's compatv in stretch mode, really

Reviewed-by: Laura Flores <lflores@redhat.com>
Reviewed-by: Samuel Just <sjust@redhat.com>

mgr/dashboard: provisioned values is misleading in RBD image table

Fixes: https://tracker.ceph.com/issues/46617
Signed-off-by: Avan Thakkar <athakkar@redhat.com>
Adding hint in image table similar to the one in rbd-details.

(cherry picked from commit 4ee2f2a9b79e246dfa56ad371b0718f0171eee3e)

Merge pull request #44032 from kamoltat/wip-ksirivad-backport-43999

pacific: pybind/mgr/pg_autoscale: revert to default profile scale-up

Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge PR #43925 into pacific

* refs/pull/43925/head:
doc: document subvolume (group) pins

Reviewed-by: Yuri Weinstein <yweins@redhat.com>