Merge pull request #44531 from sebastian-philipp/pacific-backport-44020

pacific: mgr/orchestrator: add filtering and count option for orch host ls

Reviewed-by: Adam King <adking@redhat.com>

Merge pull request #44530 from sebastian-philipp/pacific-backport-44336

pacific: mgr/cephadm: Fix test_facts

Reviewed-by: Adam King <adking@redhat.com>

Merge pull request #44597 from rhcs-dashboard/wip-53881-pacific

pacific: mgr/dashboard: fix: get SMART data from single-daemon device

Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Pere Diaz Bou <pdiazbou@redhat.com>

mgr/dashboard: fix: get SMART data from single-daemon device

Return SMART data even when a device is only associated with a single daemon.

Fixes: https://tracker.ceph.com/issues/53858
Signed-off-by: Alfonso Martínez <almartin@redhat.com>
(cherry picked from commit 6cd3729e2737f9012569cffc6fd69cc5eed287ed)

Merge pull request #44296 from batrick/i53445

pacific: mds: opening connection to up:replay/up:creating daemon causes message drop

Reviewed-by: Milind Changire <mchangir@redhat.com>

Merge pull request #44272 from nmshelke/wip-53332-pacific

pacific: doc: prerequisites fix for cephFS mount

Reviewed-by: Milind Changire <mchangir@redhat.com>

Merge pull request #44168 from cfsnyder/wip-50851-pacific

pacific: mds: PurgeQueue.cc fix for 32bit compilation

Reviewed-by: Milind Changire <mchangir@redhat.com>

Merge pull request #43979 from lxbsz/wip-53218

pacific: qa: increase the timeout value to wait a litte longer

Reviewed-by: Nikhilkumar Shelke <nshelke@redhat.com>
Reviewed-by: Milind Changire <mchangir@redhat.com>

mgr/orchestrator: add filtering and count option for orch host ls

Filter orch host ls output for only hosts whose name
contains a certain substring or who have a certain label

Add a count flag that causes the command to return the number
of hosts found (either overall or matching the substring and/or
label) instead of a list of all the matching hosts

Fixes: https://tracker.ceph.com/issues/47774
Fixes: https://tracker.ceph.com/issues/53452
Signed-off-by: Adam King <adking@redhat.com>
(cherry picked from commit edd9bf38c3f07f5fdb6714e7f66515820c736d2e)

mgr/cephadm: Fix test_facts

Wasn't executed before

Signed-off-by: Sebastian Wagner <sewagner@redhat.com>
(cherry picked from commit a03a34a01a70ce4d4ac8927a37d27e9853e46f8a)

Merge pull request #44446 from sebastian-philipp/pacific-backport-43827-43894-42906-43095-43929-43969-43873-43888-44092-44080-

pacific: cephadm: November batch 2

Reviewed-by: Adam King <adking@redhat.com>

Merge pull request #44468 from rhcs-dashboard/wip-53716-pacific

pacific: mgr/dashboard: fix timeout error in dashboard cephadm e2e job

Reviewed-by: Waad Alkhoury <walkhour@redhat.com>
Reviewed-by: Pere Diaz Bou <pdiazbou@redhat.com>

Merge pull request #44171 from cfsnyder/wip-52073-pacific

pacific: rgw: user stats showing 0 value for "size_utilized" and "size_kb_utilized" fields

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #44166 from cfsnyder/wip-53289-pacific

pacific: rgw: fix `bi put` not using right bucket index shard

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #43968 from cfsnyder/wip-53256-pacific

pacific: librgw: treat empty root path as "/" on mount

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #43966 from cfsnyder/wip-53225-pacific

pacific: qa/rgw: bump tempest version to resolve dependency issue

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #43951 from cfsnyder/wip-53098-pacific

pacific: qa/rgw: Fix vault token file access.

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #43946 from cfsnyder/wip-53271-pacific

pacific: rgw/beast: optimizations for request timeout

Reviewed-by: Casey Bodley <cbodley@redhat.com>

mgr/dashboard: stabilizing the cephadm dashboard e2e

Reordering the tests and adding some more tests to verify the cluster is
healthy before proceeding to do some complex tasks like maintenance and
drain host

Fixes: https://tracker.ceph.com/issues/53742
Signed-off-by: Nizamudeen A <nia@redhat.com>
(cherry picked from commit fbc9f46459537d0799c448e27f80623d7d4805c8)

mgr/dashboard: dashboard cephadm e2e improvement

Fixes: https://tracker.ceph.com/issues/53742
Signed-off-by: Nizamudeen A <nia@redhat.com>
(cherry picked from commit 26f86f6cd32fc86297e250472e3205d4d65744fb)

mgr/dashboard: fix timeout error in dashboard cephadm e2e job

1. Fix the timeout error happening in the dashboard e2e job
2. Take care of the flaky force maintenance check

Most of the time our test is getting timed out while searching for an item
in the table. Its because `.clear().type()` is not clearing the content
in the search field sometimes and that creates a wrong data to be
entered into the search field and it starts searching based on this
wrong name. To avoid this I am explicitly clearing the search area
before typing.

Fixes: https://tracker.ceph.com/issues/53672
Signed-off-by: Nizamudeen A <nia@redhat.com>
(cherry picked from commit fed358d7c5a21cc76cae7042975f7e47ac3f8d50)

Merge pull request #44105 from rhcs-dashboard/wip-52571-pacific

pacific: mgr/dashboard: use -f for npm ci to skip fsevents error

Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Pere Diaz Bou <pdiazbou@redhat.com>

Merge pull request #44382 from rhcs-dashboard/wip-53690-pacific

pacific: mgr/dashboard: cephadm e2e job: improvements

Reviewed-by: Aashish Sharma <aasharma@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>

python-common/ceph/deployment/drive_group: fix 'orch ls --format yaml'

In f76c02a658ed1de54a380381932bb58b6ab91d41 we may leave out the placement
key when it is empty.  If we take that path, we need to tolerate specs
that are missing that key.

Fixes: f76c02a658ed1de54a380381932bb58b6ab91d41
Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit 7e51fe6227a92b92640d37216bdd99824c84769a)

cephadm: only infer conf from mon if fsid matches

fixes: https://tracker.ceph.com/issues/53394

Signed-off-by: Adam King <adking@redhat.com>
(cherry picked from commit f64b8a34c4808af5f09ddc5de8085d78b50a9666)

mgr/cephadm: grafana.ini: Set `cookie_secure = true`

Signed-off-by: Sebastian Wagner <sewagner@redhat.com>
(cherry picked from commit e90433b8f9bb4d60b5a6da636d880ce0b7b8b243)

mgr/cephadm: Add GrafanaSpec.initial_admin_password

By default, we're not creating any admin accout for Grafana now,
but we're adding an option to set the grafana password manually using:

```yaml
service_type: grafana
spec:
  initial_admin_password: mypassword
```

Users can then easily log into Grafana with the given password.

Fixes: https://tracker.ceph.com/issues/48291
Signed-off-by: Sebastian Wagner <sewagner@redhat.com>
(cherry picked from commit fdae665a2fe645d3f76b212f440ffe2c96281bc0)

Conflicts:
src/pybind/mgr/cephadm/tests/test_services.py

python-common: Reparent AlertManagerSpec to MonitoringSpec

And remove duplicated members

Signed-off-by: Sebastian Wagner <sewagner@redhat.com>
(cherry picked from commit f2b0b451766c94505674cc1e315989c52b6bcaf5)

python-common: Move AlertManagerSpec below MonitoringSpec

Signed-off-by: Sebastian Wagner <sewagner@redhat.com>
(cherry picked from commit 208ce50b9232b83f80ad68bd71681c9d3781a76f)

python-common: test_yaml(): add a few tests

Signed-off-by: Sebastian Wagner <sewagner@redhat.com>
(cherry picked from commit e7035f1a54c8b48d918c7579dfcf5665ae351dbc)

python-common: prettify `yaml.dump(MonitoringSpec())`

Signed-off-by: Sebastian Wagner <sewagner@redhat.com>
(cherry picked from commit f76c02a658ed1de54a380381932bb58b6ab91d41)

pyhton-common: move some tests from cephadm/test_spec.py

Cause they don't have any dependencies to cephadm

Signed-off-by: Sebastian Wagner <sewagner@redhat.com>
(cherry picked from commit 6f90f0fa2efc830eec07373b94534e3c96f24945)

python-common: add string representation for Device and DeviceSelection classes

Signed-off-by: Adam King <adking@redhat.com>
(cherry picked from commit 5b60b8e75889208a0c43e6cf7e9a6abcd849bd9f)

mgr/cephadm: skip osd_stats check if osd removal queue is empty

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit 0bbbe5d31d834d5626c11fd9c49b572b111ead8f)

mgr/cephadm: drop osdspec_affinity tracking

We identify which drivespec legacy OSDs belong(ed) to by metadata they
report to the mgr.  Modern cephadm does this instead by looking at the
'service' property in the unit.meta file.  Having cephadm query the osd
metadata is expensive for large clusters, so let's avoid this and rely
entirely on unit.meta.

Worst case, some upgraded clusters will show OSDs as service 'osd' instead
of service 'osd.whatever' for whatever drivespec created them.

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit 9c2b8f557ef374212b5d11464b04f443cb8d84e6)

Conflicts:
src/pybind/mgr/cephadm/module.py
    src/pybind/mgr/cephadm/serve.py

Signed-off-by: Sebastian Wagner <sewagner@redhat.com>

mgr/cephadm: simplify HostCache.get_daemon_types

Signed-off-by: Sebastian Wagner <sewagner@redhat.com>
(cherry picked from commit 32fdb84956afe926253bc6f390701bbf56b1aaa7)

mgr/cephadm: Inventory: Fix `dictionary changed size during iteration`

Use `.copy()` for that.

Signed-off-by: Sebastian Wagner <sewagner@redhat.com>
(cherry picked from commit d770bb5c45d6f45c80543b6873c450922f3a13e5)

Conflicts:
src/pybind/mgr/cephadm/inventory.py

mgr/cephadm: record when finished with scheduled daemon action

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit 124ac26c27467e28d99e8d89ce1d7e0f1832c147)

doc/cephadm: deployment scenarios single host and isolated environment

This PR adds a deployment scenarios section to the cephadm docs to document the single-host-defaults flag, and explain how to deploy in an isolated environment.

Signed-off-by: Melissa Li <melissali@redhat.com>
(cherry picked from commit e6e034498102625ebd777bacb770b15a6c575824)

doc/cephadm: isolated environment and other deployment scenarios

This PR adds a section to the cephadm docs to describe how to install cephadm in different deployment scenarios (set cluster on single host, and deployment in an isolated environment or private network).

Signed-off-by: Melissa Li <melissali@redhat.com>
(cherry picked from commit a311e837d61f7677999eba7887b77d0752909834)

cephadm: fixup expect-hostname message

show either a success or a failure message

Signed-off-by: Michael Fritch <mfritch@suse.com>
(cherry picked from commit ff3dd995b5f07d1a4d48e1f62d5054dadea9eb72)

cephadm: add --shared_ceph_folder to shell cmd

It can be useful to share ceph src when running the `cephadm shell`
command in order to do quick tests.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
(cherry picked from commit 37ea06cfd0937df6230383d16de9fbda9a5a6f3f)

doc/cephadm: core dumps

Signed-off-by: Sebastian Wagner <sewagner@redhat.com>
(cherry picked from commit 223af588a994eabc75b9f6d2bb33df761d6944b8)

qa/suites/orch/cephadm: verify /var/log/ceph/$fsid ownership

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit 4f71cbba1a41426e3a858b11f03fd0c5f795dc95)

cephadm: only make_log_dir for ceph daemons

For non-ceph daemons, (1) they don't log to /var/log/ceph, and (2) the
container image isn't a ceph image, which means the uid/gid extraction
won't have the correct uid/gid and we'll end up with a log directory that
ceph daemons no longer have write permissions for.

Fixes: https://tracker.ceph.com/issues/53257
Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit 393960c164bf84b21ffee6af65c19d968b416fb2)

mgr/cephadm: check_for_moved_osds: don't run post_remove

Signed-off-by: Sebastian Wagner <sewagner@redhat.com>
(cherry picked from commit 70d27f24f837f2d962ce7d1f2672b0d71f5f2f7a)

mgr/cephadm: test_cephadm: make use of with_osd_daemon

Signed-off-by: Sebastian Wagner <sewagner@redhat.com>
(cherry picked from commit 91574d47e9ba6cd0426370bf16a68017cb00f8c4)

Conflicts:
src/pybind/mgr/cephadm/tests/test_cephadm.py

mgr/cephadm: check_for_moved_osds: verify OSDs are up

Signed-off-by: Sebastian Wagner <sewagner@redhat.com>
(cherry picked from commit 47dd258cf56039d510c8d8a0d275c9104b9f49eb)

mgr/orch: Add DaemonDescriptionStatus `starting` and `unknown`

Signed-off-by: Sebastian Wagner <sewagner@redhat.com>
(cherry picked from commit 501ecf035e982af6f2020b3bb77d0ac505313b73)

mgr/cephadm: test_cephadm: Add some more tests for OrchestratorEvent

Signed-off-by: Sebastian Wagner <sewagner@redhat.com>
(cherry picked from commit 7d1bea2bb8777ac6c4d08b9042af1f6597ed2906)

mgr/cephadm: check_for_moved_osds: Add OrchEvent

Signed-off-by: Sebastian Wagner <sewagner@redhat.com>
(cherry picked from commit 25d27fb6c1bd3520564e624965fa3b6f81a21ac0)

mgr/cephadm: Add `_check_for_moved_osds`

Fixes: https://tracker.ceph.com/issues/49571
Signed-off-by: Sebastian Wagner <sewagner@redhat.com>
(cherry picked from commit 0ef6788fe6e4c3cfbebb401748eb9da29a56975c)

Conflicts:
src/pybind/mgr/cephadm/serve.py

mgr/cephadm: osd: prime new daemons with proper daemon_id type

No one noticed this before, cause it's going to fix itself next time we
check teh daemons on that host

Signed-off-by: Sebastian Wagner <sewagner@redhat.com>
(cherry picked from commit 3fa14458a853b68856e45376833e8c12038be7e6)

mgr/orch: Deprecate `ceph orch apply -i <osd_spec_file>`

* Remove usage
* Simplify error messages

Fixes: https://tracker.ceph.com/issues/47401
Signed-off-by: Sebastian Wagner <sewagner@redhat.com>
(cherry picked from commit 3fef006e0ea02f8d4b508f9d817624ee0129eadc)

qa/suites/orch/cephadm: verify that 'orch ls' reports OSDs properly

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit e1cd91098f4e566813dc98daa85e6473ccc6eed6)

mgr/cephadm: show unmanaged OSDs under 'osd' service

1- If the unit.meta file service_name = osd.NNN (which is true for lots of
OSDs deployed on older version of cephadm) then ignore the field entirely.

2- If an OSD has not service_name (see above) then show it under the 'osd'
service (instead of 'osd.unmanaged').

Sample 'ceph orch ls' output with a drivegroup + unmanaged OSD:

NAME      PORTS  RUNNING  REFRESHED  AGE  PLACEMENT
...
osd                    1  85s ago    -    <unmanaged>
osd.hdds               5  85s ago    2s   *
...

Fixes: https://tracker.ceph.com/issues/53235
Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit f13a3e5b3ca1547bba9deae3615252165bc7cac9)

qa/suites/orch/cephadm: add repave-all test case

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit 4abe4c6a78a941c8a87f9f3bdf1c731618415993)

Merge pull request #44389 from aaSharma14/wip-53689-pacific

pacific: mgr/dashboard: Cluster Expansion - Review Section: fixes and improvements

Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
Reviewed-by: Pere Diaz Bou <pdiazbou@redhat.com>

Merge pull request #44400 from aaSharma14/wip-53717-pacific

pacific: mgr/dashboard: introduce separate front-end component for API docs

Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>

mgr/dashboard: introduce seperate frontend component for API docs

This PR intends to add a separate frontend component for the API docs that are currently being rendered from the python code.

Fixes: https://tracker.ceph.com/issues/50955
Signed-off-by: Aashish Sharma <aasharma@redhat.com>
(cherry picked from commit 2c396be7cc47762290199609b99fba09f3573899)

Conflicts:
src/pybind/mgr/dashboard/frontend/package-lock.json
src/pybind/mgr/dashboard/frontend/package.json (added new swagger packages)

mgr/dashboard: Cluster Expansion - Review Section: fixes and improvements

Ensure "Storage capacity" keeps the "Description : Value" approach ("Number of devices: X" and "Raw Capacity: Y" in different lines).Correct issue with "host by services" host count

Fixes: https://tracker.ceph.com/issues/53128
Signed-off-by: Aashish Sharma <aasharma@redhat.com>
(cherry picked from commit 2ba1a5a2666674a935c9288f5b547532ba214f54)

Merge pull request #44376 from rhcs-dashboard/wip-53687-pacific

pacific: mgr/dashboard: Implement drain host functionality in dashboard

Reviewed-by: Aashish Sharma <aasharma@redhat.com>
Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>

mgr/dashboard: cephadm e2e job: improvements

- Avoid jenkins env. error: Input file /workdir/./src/pybind/mgr/dashboard/ci/cephadm/ceph_cluster.yml not found.Leaving...
- More meaningful log output.

Fixes: https://tracker.ceph.com/issues/53666
Signed-off-by: Alfonso Martínez <almartin@redhat.com>
(cherry picked from commit 9aedeaf2012851fd3598bb9c8f4a347bdc58a330)

Merge pull request #44369 from ceph/wip-yuri-pacific-p2p-pacific

qa/tests: replaced 16.2.6 with 16.2.7 version

Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge pull request #44377 from rhcs-dashboard/wip-53679-pacific

pacific: mgr/dashboard: run-backend-api-tests.sh: Older setuptools

Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: David Galloway <dgallowa@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>

pybind/mgr/dashboard/run-backend-api-tests: Older setuptools

https://github.com/pypa/setuptools/issues/2353

Signed-off-by: David Galloway <dgallowa@redhat.com>
(cherry picked from commit 4ab2df179b0bf0b123a59c649b779bba41e4bb3d)

mgr/dashboard: Implement drain host functionality in dashboard

Fixes: https://tracker.ceph.com/issues/51587
Signed-off-by: Nizamudeen A <nia@redhat.com>
(cherry picked from commit 524c340562d6b44c329ffa635cf3203e059171ad)

Merge pull request #44046 from rhcs-dashboard/wip-53344-pacific

pacific: mgr/dashboard: NFS non-existent files cleanup

Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
Reviewed-by: Pere Diaz Bou <pdiazbou@redhat.com>
Reviewed-by: Yuri Weinstein <yweins@redhat.com>

qa/tests: replaced 16.2.6 with 16.2.7 version

Signed-off-by: Yuri Weinstein <yweinste@redhat.com>

Merge pull request #44316 from rhcs-dashboard/wip-53296-pacific

pacific: monitoring/grafana: Grafana query tester

Reviewed-by: Aashish Sharma <aasharma@redhat.com>
Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
Reviewed-by: Pere Diaz Bou <pdiazbou@redhat.com>

Merge pull request #44319 from guits/wip-53617-pacific

pacific: ceph-volume: make it possible to skip needs_root()

Merge pull request #43964 from cfsnyder/wip-51484-pacific

pacific: pybind/mgr/balancer: define Plan.{dump,show}()

Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>

pybind/mgr/balancer: define Plan.{dump,show}()

as they are called by the commands

Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit 0d48b03)

Conflicts:
src/pybind/mgr/balancer/module.py

Cherry-pick notes:
- Conflicts due to missing type annotations on Pacific

mgr/dashboard: disable Promql test in ARM

Temporarily disable this test while debugging the issue (since https://github.com/ceph/ceph/pull/43669
originally passed the ARM check).

Fixes: https://tracker.ceph.com/issues/53451
Signed-off-by: Ernesto Puerta <epuertat@redhat.com>
(cherry picked from commit d10b0b7e728ccd28e1002271ce71e6f94a074256)

ceph-volume: make it possible to skip needs_root()

Add the possibility to skip the `needs_root()` decorator.
See linked tracker for details.

Fixes: https://tracker.ceph.com/issues/53511
Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
(cherry picked from commit 068a1d2a300bc21e9aa08142c4195970ef480e41)

monitoring/grafana: Grafana query tester

Signed-off-by: Pere Diaz Bou <pdiazbou@redhat.com>
(cherry picked from commit 44d3e4c264506154373ffaeb13d6c924c580e6b5)

Merge pull request #44297 from rhcs-dashboard/wip-53510-pacific

pacific: mgr/dashboard: fix frontend deps' vulnerabilities

Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>

Merge pull request #44273 from rhcs-dashboard/wip-53576-pacific

pacific: mgr/dashboard: introduce HAProxy metrics for RGW

Reviewed-by: Aashish Sharma <aasharma@redhat.com>
Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Avan Thakkar <athakkar@redhat.com>

Merge pull request #43187 from rhcs-dashboard/wip-51979-pacific

pacific: mgr/prometheus: Fix metric types from gauge to counter

Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: p-se <NOT@FOUND>

Merge pull request #44091 from aaSharma14/wip-53383-pacific

pacific: mgr/dashboard: dashboard does not show degraded objects if they are less than 0.5% under "Dashboard->Capacity->Objects block

Reviewed-by: Aashish Sharma <aasharma@redhat.com>
Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
Reviewed-by: Pere Diaz Bou <pdiazbou@redhat.com>

Merge pull request #43954 from guits/wip-53280-pacific

pacific: cephadm/ceph-volume: do not use lvm binary in containers

Merge pull request #44149 from rhcs-dashboard/wip-53437-pacific

pacific: mgr/dashboard: avoid tooltip if disk_usage=null and fast-diff enabled

Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Pere Diaz Bou <pdiazbou@redhat.com>

mds: defer messages to bootstrapping ranks

This is necessary with msgr protocol v2. The protocol no longer updates
the server "myname" for each message. It's associated for a connection.
For this reason, a newly starting rank (failover or new) updating its
messenger "myname" races with other ranks trying to connect to it. We
need those ranks to hold off on connecting until the rank reaches a
known good state.

Fixes: https://tracker.ceph.com/issues/53194
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit 23a6c256162563b41c0212f2233d940cf22ce5b1)

Conflicts:
src/mds/MDSRank.h: trivial

mds: test connections to bootstrapping MDS

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit 402919cbe65c9fa219a9ec7a0f6534c6e1479f03)

Conflicts:
src/common/options/mds.yaml.in: legacy option conversion

mgr/dashboard: fix frontend deps' vulnerabilities

- Remove npm-force-resolutions: no resolution needed anymore and this is modifying package-lock.json every time it is run (striping last empty line).
- Add .npmrc: save exact version by default; do not launch audit report when installing.

Fixes: https://tracker.ceph.com/issues/48005
Signed-off-by: Alfonso Martínez <almartin@redhat.com>
(cherry picked from commit f08c0db689dc6bd29323ac03a91c69e2fe7365a2)

 Conflicts:
src/pybind/mgr/dashboard/frontend/package-lock.json
    - Accept version from master branch.
src/pybind/mgr/dashboard/frontend/package.json
    - Accept version from master branch.

mds: correct whitespace

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit 8dc54deed97add473d12fbcfdb7a6e3f65fa79f1)

Conflicts:
src/mds/MDSRank.h: trivial

Merge pull request #44086 from rhcs-dashboard/wip-53380-pacific

pacific: mgr/dashboard: upgrade Cypress to the latest stable version

Reviewed-by: Waad Alkhoury <walkhour@redhat.com>
Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
Reviewed-by: Yuri Weinstein <yweins@redhat.com>

ceph-volume: remove --root param from nsenter cmd

This is redundant and makes nsenter throw messages like following:
```
  Failed to find sysfs mount point
  dev/block/11:0/holders/: opendir failed: Not a directory
  dev/block/252:0/holders/: opendir failed: Not a directory
  dev/block/253:0/holders/: opendir failed: Not a directory
  dev/block/252:1/holders/: opendir failed: Not a directory
  dev/block/253:1/holders/: opendir failed: Not a directory
  dev/block/252:2/holders/: opendir failed: Not a directory
  dev/block/253:2/holders/: opendir failed: Not a directory
  dev/block/252:3/holders/: opendir failed: Not a directory
  dev/block/253:3/holders/: opendir failed: Not a directory
  dev/block/252:16/holders/: opendir failed: Not a directory
  dev/block/252:32/holders/: opendir failed: Not a directory
  dev/block/252:48/holders/: opendir failed: Not a directory
  dev/block/252:64/holders/: opendir failed: Not a directory
  ```

Fixes: https://tracker.ceph.com/issues/52926
Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
(cherry picked from commit e4667e81301295f4c81328505e4376d2aef66fb2)

qa/tasks/nvme_loop: loop until 'nvme list' shows new devs

Sometimes this doesn't happen immediately.

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit d4a1ec2d0611192f4016006afcf72d57a31a6a1e)

qa/suites/roch/cephadm/smoke: use nvme_loop

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit 6436b7e05f56027fccdfef7fb0a3349139141e09)

qa/suites/orch/cephadm/smoke-roleless: no osd roles, use nvme loop

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit 4206b28dd7097ee079eb389e53f16a5afd55e21f)

qa/overrides/nvme_loop: reusable fragment to use nvme_loop devs

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit 8d4b87031638e48fddc27769003b09042643e11e)

qa/tasks/cephadm: wait for osds to start explicitly

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit dda15a792431a509d18210bb00471015493d6ce6)

qa/tasks/cephadm: if no osd roles, --all-available-devices

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit d3c9486ed9406d09904dd542eae05680d5f09498)

qa/tasks/nvme_loop: set up nvme_loop on scratch_devs

Using an nvme loop device makes the LVs look like "real" disks,
which means we can exercise all of the normal code paths for
provisioning, deprovisioning, and zapping.

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit 65cf69c6ff5ee44319e3b2ca2afb36cea7b7c81c)

qa/suites/orch/cephadm/osds: add osd rm/zap/add test

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit 96319a3d39d9ca254b86fabbab8cd19cf53ec4f8)

cephadm: mount rootfs in osd containers

See ceph-volume tracker for details [1]

[1] https://tracker.ceph.com/issues/52926

Fixes: https://tracker.ceph.com/issues/51592
Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
(cherry picked from commit 48b369e2caf3222bf594dc09f87b5969a53dfbe7)

ceph-volume: implement lvm wrapper

ceph-volume should run pv/vg/lv commands in the host namespace rather than
running them inside the container in order to avoid lvm metadata corruption.

Fixes: https://tracker.ceph.com/issues/52926
Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
(cherry picked from commit 4d33630deeaee51578868fb29337da802e9cb231)

Merge pull request #44209 from guits/wip-53371-pacific

pacific: ceph-volume: human_readable_size() refactor

Merge pull request #43649 from ideepika/wip-53019-pacific

pacific: test/librbd: harden RemoveFullTry tests

Reviewed-by: Mykola Golub mgolub@suse.com