mgr/cephadm: ceph-volume verbose only when fails

Resolves: https://tracker.ceph.com/issues/50526

Signed-off-by: Juan Miguel Olmo Martínez <jolmomar@redhat.com>
(cherry picked from commit 7f67a758d001e3f92be87b262dd81ad48afdc86b)

qa/workunits/cephadm/test_cephadm: test zap-osds

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit 4b4f4ad7bf5a6662d5d8555f73de7fb05ad5d47c)

cephadm: add --zap-osds argument to rm_cluster

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit 14c95b54fb325240635be01dd95517d11f37106b)

cephadm: implement zap-osds --fsid ... command

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit a6a20bafc4a9e780cfd86887418850a6b4ee782c)

doc/cephadm: add podman version note to install

This PR adds a note to the cephadm installation
guide that informs users that only podman version
2.0.0 and higher work with Ceph, with the
exception of podman version 2.2.1, which does
not work with Ceph. There is also a note regarding
kubic stable 3.0.1 working, but only with newer
kernels.

Signed-off-by: Zac Dover <zac.dover@gmail.com>
(cherry picked from commit 3043aaaa8a880107fab9a755df948c0e9e8db358)

mgr/cephadm: check hostname resolution before adding host

Signed-off-by: Daniel Pivonka <dpivonka@redhat.com>
(cherry picked from commit 459ffb52a15f8870dd45d3ec163ed27c1467623e)

cephadm: provide a way to checkhost connection without /etc/hosts passed the shell

Signed-off-by: Daniel Pivonka <dpivonka@redhat.com>
(cherry picked from commit 35da4de8ee888d54d74d05a9d16345cd5c112d23)

doc/cephadm: remove /etc/hosts from list of hostname resoltion methods

Signed-off-by: Daniel Pivonka <dpivonka@redhat.com>
(cherry picked from commit c18932ef9fc9f56dee37a03068f3b079c03dfb33)

qa/suites/rados/cephadm/smoke-roleless: test client-keyring

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit e41931d042a1ae854523fc6229c749f0dcc563f4)

qa/tasks/cephadm.py: adjust client.admin key mode; place on all hosts

Except during upgrades, since it is not supported there.

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit b0dcaf2cfa02be0695ec8be4cadeaee465a5b2fb)

cephadm: distribute client.admin keyring+conf to label:_admin on bootstrap

If we are placing ceph.conf in /etc/ceph (the default), tell the cluster
to continue doing this going forward to hosts with the '_admin' label.

This doesn't induce the user to add the admin label to other hosts too,
unfortunately--e probably want them to add the admin label to other mons,
for instance--but it is a start.

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit 08d21ccc949aa19c08c7e806a5b58c0180f5cbe2)

doc/cephadm: document the default 'admin' label

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit b6b85edfe5f1dacd84ed96b4dd0980b5d1774302)

mgr/cephadm: 'ceph orch client-keyring ...' commands to manage keyring files

Teach cephadm to manage keyring files on cluster hosts.  These keys must
already exist in the mon auth database--cephadm does not create them if
they don't exist (and will issue warnings to the log if they do not).

A ceph.conf is pushed implicitly along with the keyring file.

Each keyring added will be pushed to the hosts described by the placement
spec with the appropriate ownership and mode.  If the ownership, mode, or
path are modified, the files are rewritten or removed as need.

If the client-keyring entry is removed, the keyring files are removed.

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit b7e4f6017875dfbe6dc914136c7de64c8c71254e)

mgr/cephadm: reimplement ceph.conf pushing

Use a more generic inventory map of paths to digests to track what we've
pushed.

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit 77411e7df313f1488e9636c871973818474f9401)

mgr/cephadm: use _write_remote_file for ceph.conf

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit 67718f53d189a9afa5fc48016df7c3acdbea5899)

mgr/cephadm: _write_remote_file helper

This is careful is ownership, mode, and fsyncs before renaming into
position.

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit 64926f3da07b821ef394abb36321917327c82806)

mgr/cephadm: add placementspec for which hosts get ceph.conf

Add a config option to control which hosts (by default, *) get a
ceph.conf (if the bool manage_etc_ceph_ceph_conf option is enabled).

We don't modify the existing option because changing a type makes for a
messy migration: we have to sort out which section the config option is
in to change it.  Also, a simple on/off which is more friendly than
specifying "*" to enable something.

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit f1096030d9b6caeeaf7c1982b770b5f3fbe913bc)

mgr/cephadm: skip ok-to-stop for mons in upgrade if < 3 mons

Fixes: https://tracker.ceph.com/issues/50443
Signed-off-by: Adam King <adking@redhat.com>
(cherry picked from commit 4c7bfdbdd133820a1c0e8d129980ed810a77168d)

mgr/cephadm: don't allow upgrade start with less than 2 mgrs

Fixes: https://tracker.ceph.com/issues/50443
Signed-off-by: Adam King <adking@redhat.com>
(cherry picked from commit 06d767ee6d36dc5f74044dc3a030c5c46cb09cbe)

cephadm: re-assimilate user provided conf after mgr created

Fixes: https://tracker.ceph.com/issues/50544
Signed-off-by: Adam King <adking@redhat.com>
(cherry picked from commit adc8a9c0a812c3dc8de118a698cc5b9ca3bc8500)

cephadm: allow several public networks be matched

Fixes: https://tracker.ceph.com/issues/50548
Signed-off-by: Stanislav Datskevych <me@nek0.net>
(cherry picked from commit cdd7d20b35676c4627a5caa58e4838fc59e57554)

mgr/cephadm: The command of 'ceph orch daemon restart mgr.xxx' may case mgr daemon loop to restart
Scene:
The mgr daemon is active. After execing restart command, it may be save "scheduled_daemon_actions": {"mgr.xxx": "restart"}}" to config-key.
So the mgr daemon will restart before call rm_scheduled_daemon_action which case mgr daemon will load restart forever.

Fix mgr infinite restart issue refering to the same solution as 'ceph orch daemon redeploy'.

Signed-off-by: jianglong01 <jianglong01@qianxin.com>
(cherry picked from commit cc5b77ee870e60c2c9d241d5826810eedb295e26)

doc/cephadm: add a single word

s/If the nfs-ganesha doesn't/If the
nfs-ganesha pool doesn't/

Signed-off-by: Zac Dover <zac.dover@gmail.com>
(cherry picked from commit 3af9fe3fd8de16e2b2766fd5c7b19ccb4583e5db)

doc/cephadm: adding "device" to a sentence

This tiny PR fixes an issue that I thought
I had squashed into PR#40914: a missing
word, "device", which was caught by Josh
Durgin during review.

Alas. I had not then caught it.

Here then it is, corrected at last.

Signed-off-by: Zac Dover <zac.dover@gmail.com>
(cherry picked from commit 72445fb724044e42a1ceacbbf076ea3b8a88e2ae)

doc/cephadm: rewrite "nfs.rst"

This PR makes a few (very small) stylistic
corrections, mostly typographical, to the text.

Signed-off-by: Zac Dover <zac.dover@gmail.com>
(cherry picked from commit 36426d6df56cadb33f20605a47e369bd8c01a813)

mgr/cephadm: s/_hosts_with_daemon_inventory/_schedulable_hosts/

`_hosts_with_daemon_inventory` was renamed to `_schedulable_hosts`
in f260c48f723641a5f82c64c0e31451199aa191f8 which was merged before
eebb842d0487660c93baf9eafda28a2f87e482f3 got merged, hence the
regression.

this change just updates the test with the new method name.

Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit 782fc0e7cac7229b2425e706c8f0fae6f4bb6884)

mgr/cephadm: don't remove daemons from hosts in maintenance or offline  mode

Fixes: https://tracker.ceph.com/issues/50364
Signed-off-by: Adam King <adking@redhat.com>
(cherry picked from commit eebb842d0487660c93baf9eafda28a2f87e482f3)

mgr/cephadm: default status for daemons on maintenance hosts to stopped

we do not refresh the daemons on maintenance hosts so our info
on them is always outdated. Therefore, the best option is to
assume maintenance mode is working correctly and the daemons
are stopped

Signed-off-by: Adam King <adking@redhat.com>
(cherry picked from commit 56641742a75bc753b473209b63fcefabcde7615b)

qa/tasks/cephadm: fix ctx archive check for teuthology

Signed-off-by: Deepika Upadhyay <dupadhya@redhat.com>
(cherry picked from commit ec2a402f55e1f61c82d65f7609be16c675bfd7aa)

python-common: use OrderedDict instead of Set to remove duplicates from host labels list

this preserves the order of the list instead of randomizing the order

Signed-off-by: Daniel Pivonka <dpivonka@redhat.com>
(cherry picked from commit 4d3ff90e8676e2403f0a0ca2f968aa49aab9bbdd)

mgr/cephadm: less noise about osd specs

We don't want this message every time around the serve() loop

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit 98c8baf165445ee06de9cbbe2e9c6c337a397681)

mgr/cephadm: kick serve loop when adding/removing labels

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit 695dc80535a389dd58d9ebbe9239855ea948e571)

mgr/cephadm: do not place osds on _no_schedule hosts

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit f5cee666d2277fcf06c15930dc00617b86aa0c0e)

doc/cephadm: document _no_schedule label

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit c7dae3ceb78999e4b06d108a43cedf2b2693c273)

mgr/cephadm: fix 'orch ls' count to reflect schedulable hosts

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit ba135842e359fe6991e35d7177cfe3eac31af16d)

mgr/cephadm: do not schedule on _no_schedule hosts

Reuse and rename existing helper to return schedulable hosts.

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit f260c48f723641a5f82c64c0e31451199aa191f8)

doc/cephadm: osd.rst -- removing colons

This PR removes colons that weren't necessary.
This is a companion PR to PR#40996.

Signed-off-by: Zac Dover <zac.dover@gmail.com>
(cherry picked from commit 8648e04a6eb69eba0c2813a93fde010605483131)

doc/cephadm: osd: rewrite "additional opts"

This PR rewrites the "Additional Options"
subsection of the OSD chapter of the cephadm
guide.

Signed-off-by: Zac Dover <zac.dover@gmail.com>
(cherry picked from commit 91de0becbd5b9e571a98b790da5d8dc5522f5446)

doc/cephadm: rewrite "advanced osd s. specs"

This PR improves the readbility and elegance of
the "Advanced OSD Service Specifications" section
of the OSD chapter of the cephadm guide.

Signed-off-by: Zac Dover <zac.dover@gmail.com>
(cherry picked from commit 4106e331fb0c828f3fa455558b63c3626b77a499)

doc/cephadm: rewrite "delcarative state" in osd.rst

This commit improves the wording of the "Declarative
State" section of the "OSD Service" chapter of the
Cephadm Guide.

Signed-off-by: Zac Dover <zac.dover@gmail.com>
(cherry picked from commit 9f2d892cc150cbbecc60d4c0d0ffb9544f5acc40)

mgr/MgrStandby: fix config observer

Otherwise handle_conf_change() is never called.

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit 73af779d314902f2af9d62e86a5adb17fb9f5d00)

mgr/MgrStandby: respawn if mgr_standby_modules changes

If we have standby modules and this goes false, or do not have modules
and this goes true, then respawn.

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit efc29fb3630aaab4e04d99621c40561d38a99c03)

qa/tasks/mgr/test_dashboard: skip test_standby if mgr_standby_modules=false

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit a2c9333a0b5ed061839aa798f8293c5fcdc6e7f5)

qa/suites/rados/cephadm/smoke-*: use cephadm.wait_for_service

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit 9c1f1288857814cd417f3b88d934b9bb0f8cfb7f)

qa/suites/rados/cephadm/smoke-singlehost: test --single-host-defaults

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit c20323d114e6c8072ecdf26f77a58ae734d38d0e)

cephadm: add --single-host-defaults option to bootstrap

[global] osd_crush_choose_leaf_type = 0
[global] osd_pool_default_size = 2
[mgr] mgr_standby_modules = false

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit ae899d37692284c7991229a7a47a2fce1bfbeca7)

mgr/cephadm: allow mgr colo if mgr_standby_modules=false

If the standby mgr daemons' modules aren't listening on any ports, then we
can schedule multiple on the same host.

Note that this may make 'orch ps' output misleading, as ports will be
reported for each mgr instance, but only one of them will actually be
listening at any one time (if they are behaving, at least!).  Treat a
mgr port check error as non-fatal.

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit 77e0f5b45c6c33c6c4d857bfab3e33ec8d9ed51c)

mgr/MgrStandby: add mgr_standby_modules option

Add config option to control whether the standby modules are started.
Default to true (no change in behavior), but if set to false the standby
mgr modules don't do the redirect business.

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit 35d2146c1a801a2b478a19b9d62af0f10279fd34)

# Conflicts:
# src/common/options/global.yaml.in
 - added to options.cc instead

cephadm: ignore apparmor if profiles file is empty

Signed-off-by: Joao Eduardo Luis <joao@suse.com>
(cherry picked from commit 8395885794df85a6141da80d4e7a13873c3f6770)

Merge PR #40922 into pacific

* refs/pull/40922/head:
pybind/ceph_argparse: print --format flag name in help descs
mgr/cephadm: don't list non ceph daemons as needing upgrade in upgrade check
qa/tasks/cephadm: ignore --keep-logs failure
qa/tasks/cephadm: use yaml.dump_all()
qa/suites/rados/cephadm/smoke-*: use cephadm.wait_for_service
qa/tasks/cephadm: tear down clsuter before gathering logs
qa/suites/rados/cephadm/smoke-roleless: test rgw-ingress
mgr/cephadm: remove virtual_ip check during scheduling
mgr/orchestrator: orch ls: leave off virtual_ip prefixlen
qa/tasks/cephadm: add wait_for_service
qa/tasks/cephadm: allow skip_monitor_stack=true
qa/tasks/cephadm: do subst_vip for cephadm.shell and .apply
qa/tasks/vip: add vip task to allocate virtual IPs
qa/suites/rados/cephadm/smoke-roleless: add rgw-ingress test case
qa/tasks/cephadm: shell: take 'all-roles' or 'all-hosts'
qa/tasks/cephadm: let cephadm.shell take string or list
doc/cephadm: wrong command for single daemon events
mgr/cephadm: place maximum on placement count based on host count
mgr/cephadm: fix nfs-rgw stray daemon
mgr/cephadm: skip-ssh flag enables cephadm mgr module
mgr/cephadm: report exception during upgrade in upgrade status
qa/suites/rados/thrash: shorten radosbench
mgr/cephadm: remove old haproxy and keepalived templates
mgr/orchestrator: validate lists in spec jsons
python-common: Verify service spec is not None
python-common: Verify data_devices is not None
mgr/orchestrator: DG loads properly the unmanaged attribute
mgr/orchestractor: rgw realm and zone flags must both be provided
mgr/cephadm: make prometheus scrape ingress haproxy
doc/cephadm: remove big warning about stability
doc/cepham/compatibility: rgw-ha -> ingress; note possibility of breaking changes
doc/cephadm: rewrite "dry run" section in osd.rst
doc/cephadm: rewrite part of "deploy osds"
doc/cephadm: rewrite osd.rst "Remove an OSD"
doc/cephadm: rewrite osd.rst - list devices
doc/cephadm: break mon section into sections
doc/cephadm: rewrite "deploying add. mons"
doc: fixes for cephadm documentation
doc/cephadm: remove warning about cephadm in production
doc/cephadm: Add Compatibility with Podman Versions
doc/cephadm: rewrite "index.rst"
doc/cephadm: explicitly show host requirments in adding host section
mgr/cephadm: ingress: add optional virtual_interface_networks
doc/cephadm/rgw: clean up example spec
mgr/cephadm/services/ingress: less verbose about prepare_create
doc/cephadm/rgw: add note about which ethernet interface is used
cephadm: make keepalived unit fiddle sysctl settings
mgr/orchestrator: report external endpoints from 'orch ls'
mgr/orchestrator: drop - when no ports
doc/cephadm/rgw: update docs for ingress service
mgr/cephadm: use per_host_daemon feature in scheduler
cephadm: fix a typo
mgr/cephadm/schedule: add per_host_daemon_type support
mgr/cephadm: HA_RGW -> Ingress
mgr/cephadm: include daemon_type in DaemonPlacement
mgr/cephadm: update list-networks to report interface names too
mgr/orchestrator: streamline 'orch ps' PORTS formatting
mgr/cephadm/schedule: handle multiple ports per daemon
mgr/cephadm/utils: resolve_ip(): prefer IPv4
cephadm: cleanup extra slash in runtime dir
cephadm: use split cgroup strategy for podman
cephadm: use class to represent container engine
mgr/cephadm: don't cleanup the daemon keyring on failed redeploy
mgr/cephadm: fix orch host add with multiple labels and no addr
doc/cephadm: remove keepalived_user from haproxy docs
rpm: re-disable SUSE lttng build on z390x
ceph.spec.in: enable tcmalloc and lttng on s390x

Reviewed-by: Sebastian Wagner <swagner@suse.com>

Merge pull request #41054 from leseb/bkp-41032

pacific: ceph-volume: fix "device" output

ceph-volume: fix "device" output

During
https://github.com/ceph/ceph/commit/ec0f5f3b22d24754c16131a1996e42b787e4255f
I changed "device" by "dev", let's revert that!

Fixes: https://tracker.ceph.com/issues/50478
Signed-off-by: Sébastien Han <seb@redhat.com>
(cherry picked from commit da1f9d215fa9e86fa30d82de7032df05c6cbefb1)

Merge pull request #40612 from rhcs-dashboard/wip-50145-pacific

pacific: qa/tasks/vstart_runner.py: start max required mgrs

Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: rishabh-d-dave <NOT@FOUND>

Merge pull request #40887 from rhcs-dashboard/wip-50350-pacific

pacific: mgr/dashboard: improve telemetry opt-in reminder notification message

Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Pere Diaz Bou <pdiazbou@redhat.com>

Merge pull request #40929 from rhcs-dashboard/wip-49658-pacific

pacific: mgr/dashboard: test prometheus rules through promtool

Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>

Merge pull request #40815 from rhcs-dashboard/wip-50171-pacific

pacific: mgr/dashboard: debug nodeenv hangs

Reviewed-by: Waad Alkhoury <walkhour@redhat.com>
Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>

Merge pull request #40980 from rhcs-dashboard/wip-50418-pacific

pacific: mgr/dashboard: filesystem pool size should use stored stat

Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
Reviewed-by: Pere Diaz Bou <pdiazbou@redhat.com>

Merge pull request #40691 from singuliere/wip-50124-pacific

pacific: mon: Modifying trim logic to change paxos_service_trim_max dynamically

Reviewed-by: Aishwarya Mathuria <amathuri@redhat.com>

Merge pull request #40989 from trociny/wip-50480-pacific

pacific: os/FileStore: don't propagate split/merge error to "create"/"remove"

Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge pull request #40759 from smithfarm/wip-50154-pacific

pacific: osd/PeeringState: fix acting_set_writeable min_size check

Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge pull request #40690 from singuliere/wip-50131-pacific

pacific: monmaptool: Don't call set_port on an invalid address

Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Brad Hubbard <bhubbard@redhat.com>

Merge pull request #40679 from singuliere/wip-50121-pacific

pacific:  crush/CrushLocation: do not print logging message in constructor

Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge pull request #40677 from singuliere/wip-50212-pacific

pacific: os/bluestore/BlueFS: do not _flush_range deleted files

Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge pull request #40853 from batrick/i50285

pacific: qa: test standby_replay in workloads

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #40852 from batrick/i50287

pacific: qa: "log [ERR] : error reading sessionmap 'mds2_sessionmap'"

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #40825 from batrick/i50253

pacific: mds: "cluster [WRN] Scrub error on inode 0x1000000039d (/client.0/tmp/blogbench-1.0/src/blogtest_in) see mds.a log and `damage ls` output for details"

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #40680 from singuliere/wip-50082-pacific

pacific: rbd-mirror: fix UB while registering perf counters

Reviewed-by: Mykola Golub <mgolub@mirantis.com>

Merge pull request #40981 from rhcs-dashboard/wip-50476-pacific

pacific: mgr/dashboard: Remove username and password from request body

Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>

Merge pull request #40990 from rhcs-dashboard/wip-50485-pacific

pacific: mgr/dashboard: fix duplicated rows when creating NFS export.

Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>

Merge pull request #40688 from singuliere/wip-50086-pacific

pacific: qa/tasks/cephfs: create enough subvolumes

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #40686 from singuliere/wip-50180-pacific

pacific: client: only check pool permissions for regular files

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #40684 from singuliere/wip-50185-pacific

pacific: test: disable mgr/mirroring for `test_mirroring_init_failure_with_recovery` test

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Venky Shankar <vshankar@redhat.com>

Merge pull request #40683 from singuliere/wip-50190-pacific

pacific: qa: fix ino_release_cb racy behavior

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #40682 from singuliere/wip-50225-pacific

pacific: mds: skip the buffer in UnknownPayload::decode()

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #40678 from singuliere/wip-50199-pacific

pacific: tools/cephfs_mirror/PeerReplayer.cc: add missing include

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #40630 from batrick/i50127

pacific: pybind/mgr/volumes: deadlock on async job hangs finisher thread

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #40627 from petrutlucian94/wip-50187-pacific

pacific: cephfs: minor ceph-dokan improvements

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

pybind/ceph_argparse: print --format flag name in help descs

Fixes: https://tracker.ceph.com/issues/49757
Signed-off-by: Adam King <adking@redhat.com>
(cherry picked from commit 97825298b99121d8ac877eaec16d3983549f30ee)

mgr/cephadm: don't list non ceph daemons as needing upgrade in upgrade check

Fixes: https://tracker.ceph.com/issues/50401
Signed-off-by: Adam King <adking@redhat.com>
(cherry picked from commit 5bdf50e5316b742bb41252832a8211928f451352)

qa/tasks/cephadm: ignore --keep-logs failure

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit 65f2848e978b3ef3908124dfbfbeed1f7bf52087)

qa/tasks/cephadm: use yaml.dump_all()

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit 97c143c3442e96b36de075ea7eefd6e17ede67bd)

qa/suites/rados/cephadm/smoke-*: use cephadm.wait_for_service

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit 16b30f2858aa8edb42cff79c4d40b40ec5ae72cc)

qa/tasks/cephadm: tear down clsuter before gathering logs

We dont' always stop all services, because teuthology doesn't know about
things it didn't start.  Use rm-cluster to tear things down, but do not
remove the logs themselves.  After we get logs, we'll clean up completely.

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit deec9074bb2fc42d29d6fa14c22b6b14b97c352f)

qa/suites/rados/cephadm/smoke-roleless: test rgw-ingress

Test this properly by downing each rgw and haproxy in turn and ensuring
that things remain up.

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit 3ff3f697b474c9669bc4f51c472a9cad35e72266)

mgr/cephadm: remove virtual_ip check during scheduling

In 2f33c6ebbc8e2a6c3844a6921c857fb0796a1552 we made the keepalived task
set the necessary sysctls  to add a virtual_ip, so we don't need this
check anymore.

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit 2382603162ec6785681700134e3c5764bd5aa99f)

mgr/orchestrator: orch ls: leave off virtual_ip prefixlen

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit 4498bbe77f58c59583bcb8b9ca1aae33296b329f)

qa/tasks/cephadm: add wait_for_service

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit ced2f7fe4a04ebaa09896376c342b7b866ab5bc7)

qa/tasks/cephadm: allow skip_monitor_stack=true

(Useful for roleless when we want to go faster)

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit bb825157dccd5dee65fc75e63fde856c8bcc12e6)

qa/tasks/cephadm: do subst_vip for cephadm.shell and .apply

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit 0d561e2741718498efa319e0c2b5ec3a902c67ca)

qa/tasks/vip: add vip task to allocate virtual IPs

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit 2a8ab2d2b87f76dad9b8ecd7f8ce8370f8004b3c)

qa/suites/rados/cephadm/smoke-roleless: add rgw-ingress test case

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit 7e16bf3468b53d218ac02a81c01fdbbc002b5f1b)

qa/tasks/cephadm: shell: take 'all-roles' or 'all-hosts'

'all' is ambiguous

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit 08039576950697e7b9dd55c6e44068440d2a1553)

qa/tasks/cephadm: let cephadm.shell take string or list

Make it a bit more forgiving.

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit 779af3da6fd6405e13a2e522c57fa1d1512595a9)

doc/cephadm: wrong command for single daemon events

Fixes: https://tracker.ceph.com/issues/50257
Signed-off-by: Adam King <adking@redhat.com>
(cherry picked from commit 31959dd6d361273cb125420338a47a6fcbf3998e)

mgr/cephadm: place maximum on placement count based on host count

Fixes: https://tracker.ceph.com/issues/49960
Signed-off-by: Adam King <adking@redhat.com>
(cherry picked from commit 73532088915b8e7daf4a45d8b7968cdab55de9d1)

mgr/cephadm: fix nfs-rgw stray daemon

nfs-rgw registers under a gid cephadm needs covert that to its known name during the stray daemon check

Signed-off-by: Daniel Pivonka <dpivonka@redhat.com>
(cherry picked from commit f94e0baf9e1897f803160eff8ba36df57aa433ac)

mgr/cephadm: skip-ssh flag enables cephadm mgr module

This commit fixes the use of skip-ssh flag. It disables ssh config and enables the cephadm mgr module.

Fixes: http://tracker.ceph.com/issues/49737
Signed-off-by: Shreyaa Sharma <shreyasharma.ss305@gmail.com>
(cherry picked from commit 777f236ad885b03b551dd820f41a00b9c89761eb)

mgr/cephadm: report exception during upgrade in upgrade status

Fixes: https://tracker.ceph.com/issues/50361
Signed-off-by: Adam King <adking@redhat.com>
(cherry picked from commit 6119294b2871977e0a70b138d48dc5afc8abd45d)

qa/suites/rados/thrash: shorten radosbench

This is the longest of the thrash workloads; reducing it will bring
this test in line with the others (<= 45 min).

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit babbab14f4ed8b000741fb92b76b97459510c689)

mgr/cephadm: remove old haproxy and keepalived templates

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit 0536364645ff06f285c3dd698df02898a920f1b9)