mgr/cephadm: use --osd-ids instead of --osd-id

Signed-off-by: Joshua Schmid <jschmid@suse.de>
(cherry picked from commit fa88483f660f2bf79d1c1ca77d7104b3a42ed05d)

mgr/cephadm: add support for osd_id_claims

Signed-off-by: Joshua Schmid <jschmid@suse.de>
(cherry picked from commit 5b32c3e51b517332b7ac26b161b7b6b4fae05b85)

cephadm: warn when no valid version arguments are supplied to add-repo

Currently add-repo will add a broken repository by default, this commit gives a pretty error to instruct the user to use one of the version arguments.

Fixes #45029

Signed-off-by: Maran Hidskes <maran@protonmail.com>
(cherry picked from commit 115b3d7b7f3b4cf4149240fc14e374697a7cd6b1)

cephadm: add-repo: Don't test for podman

Fixes:

```
root@buster:/cephadm# ./cephadm add-repo
Unable to locate any of ['podman', 'docker']
```

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
(cherry picked from commit 0d22708cc2f858491604b09f87e7af742884cb3c)

mgr/cephadm: corrected documentation for starting an upgrade

Signed-off-by: Andreas Haase <andreas.haase@community4you.de>
(cherry picked from commit bf3d06d180135901903251e6e73b66f4fbac289b)

mgr/cephadm: Fixed warn_on_stray_hosts setting

Fixes: https://tracker.ceph.com/issues/45065
Signed-off-by: Andreas Haase <andreas.haase@community4you.de>
(cherry picked from commit b07c2da63f9fb9e70a647a989675bbdf309c14f3)

mgr/orch: Fix DaemonDescription().daemon_id() for RGWs

Turns out, users put dots into their RGW service names.

Fixes: https://tracker.ceph.com/issues/44934
Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
(cherry picked from commit 39fdb5c67c4d3834c5157e8d6da20933a6f7bfb4)

mgr/cephadm: enable osd previews

Signed-off-by: Joshua Schmid <jschmid@suse.de>
(cherry picked from commit 462a68ab87446af8299d4f71c5ff22f77dfeea64)

cephadm: use CephContainer during command `enter`

instead of the `get_container` helper func

Signed-off-by: Michael Fritch <mfritch@suse.com>
(cherry picked from commit 3c762d0ed92d38a2c47bf273bf17600f101ed1bb)

Merge pull request #34495 from votdev/wip-44953-octopus

Reviewed-by: Laura Paduano <lpaduano@suse.com>
Reviewed-by: Stephan Müller <smueller@suse.com>

Merge pull request #34654 from ceph/wip-yuriw-octopus-client-upgrade-octopus-pacific

qa/tests: initial checkin for clients upgrades tests

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

qa/tests: initial checkin for clients upgrades tests

for future release 'pacific'

(Note to self: moved from wrongly pushed to master branch)

Signed-off-by: Yuri Weinstein <yweinste@redhat.com>

Merge pull request #34543 from ceph/wip-yuriw-optopus-p2p-octopus

qa/tests: octopus - initial checkin for octopus-p2p tests

qa/tests: octopus - initial checkin for octopus-p2p tests

Signed-off-by: Yuri Weinstein <yweinste@gmail.com>

Merge pull request #34309 from Exotelis/octopus-translations

octopus: mgr/dashboard: Update translations on octopus

Reviewed-by: Tatjana Dehler <tdehler@suse.com>
Reviewed-by: Stephan Müller <smueller@suse.com>

Merge pull request #34474 from rhcs-dashboard/wip-44996-octopus

octopus: rpm: add python3-saml as install dependency

Reviewed-by: Ricardo Marques <rimarques@suse.com>
Reviewed-by: Nathan Cutler <ncutler@suse.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #34350 from sebastian-philipp/octopus-backport-34180

octopus: cephadm: Fix check_ip_port to work with IPv6

Reviewed-by: Michael Fritch <mfritch@suse.com>

Merge pull request #34554 from sebastian-philipp/octopus-backport-31885-33954-34357-34053-34439-34383-34220-34485-34249-34454-34223-34382-34084-34525-34344

octopus: cephadm: Batch backport April (1)

Reviewed-by: Kiefer Chang <kiefer.chang@suse.com>
Reviewed-by: Michael Fritch <mfritch@suse.com>

Merge pull request #34299 from liewegas/bp-34150-octopus

octopus: cephadm: rm-cluster clean up /etc/ceph

Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>

Merge pull request #34387 from smithfarm/wip-44897-octopus

octopus: pybind/rbd: fix no lockers are obtained, ImageNotFound exception will be output

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
Reviewed-by: Mykola Golub <mgolub@mirantis.com>

mgr/rook: Fix wrong data output when listing devices

Information used to build the <device ls> output is not the right one.
Rook provides in the devices configmap the item <cephVolumeData> which contains the information returned by <ceph-volume>.
This is the right source information to build the devices list.

Signed-off-by: Juan Miguel Olmo Martínez <jolmomar@redhat.com>
(cherry picked from commit e3a5fa49005a7d3ca374beca1dd8cd3e416819a9)

Update index.rst

clean up "only only"

Signed-off-by: Tim <elatllat@gmail.com>
(cherry picked from commit d7c7ccb4390a91c0c791b924141963305809b2ab)

mgr/cephadm: add test for "Offline" host status

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
(cherry picked from commit 7368d77b31234c59b576303e1949ae85025fafd9)

mgr/cephadm: Daemons running of unreachable servers are offline

Fixes: https://tracker.ceph.com/issues/44602
Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
(cherry picked from commit adca3e08e8e9f4a683f7c6cef8e96705cc340188)

mgr/cephadm: Set SSH connection timeout to 30s

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
(cherry picked from commit 152b4b4b298bb7f2f79159a49a4f75dfa9b739c7)

mgr/cephadm: No daemon and device refresh if host if out

To make things faster

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
(cherry picked from commit 201a9c17eeb1051ce562205d316f4baeebb58b5f)

mgr/cephadm: in-memory set of offline hosts

Fixes: https://tracker.ceph.com/issues/43839
Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
(cherry picked from commit 05cf5796f36bc2d826a29aff4a124124df87cbc6)

mgr/cephadm: remove unused method `_check_hosts`

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
(cherry picked from commit e093c3754382f64133178b78d30d113e0f660bb9)

mgr/cephadm: type annotation for HostCache.mgr

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
(cherry picked from commit 52e05f8d20a0042c5ba7bc2d30a9dbaf259f8809)

mgr/cephadm: add CACHEINODE and EXPORT_DEFAULTS sections

Signed-off-by: Michael Fritch <mfritch@suse.com>
(cherry picked from commit 25f4dedd3e75c81b19911fd33b171e613ab1c559)

mgr/cephadm: add config required for ganesha grace

adds minimal config for ganesha-rados-grace

Signed-off-by: Michael Fritch <mfritch@suse.com>
(cherry picked from commit 8055d34455bf49fe5b93f501a60308ec0c681f42)

cephadm: add userid during grace add/remove

ganesha-rados-grace needs a rados userid to add/remove from the grace db

Mar 27 10:05:25 host1 bash[11684]: rados_connect: -13
Mar 27 10:05:25 host1 bash[11684]: Can't connect to cluster: -13

Signed-off-by: Michael Fritch <mfritch@suse.com>
(cherry picked from commit ea5b0430f38393d057166d6e238890a77157fca1)

mgr/cephadm: disable dashboard's grafana cert ssl_verify if we generate it

This will help dashboard work out of the box.

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 81b78265a32cdf2d316dc341fdeb5974dc8c5e29)

qa/workunits/cephadm/test_cephadm.sh: use fsid during dump_log

needed when running with multiple ceph clusters

Signed-off-by: Michael Fritch <mfritch@suse.com>
(cherry picked from commit 7e26083d9262e54e9646cf6a73da4e0f3b807702)

qa/workunits/cephadm/test_cephadm.sh: add fsid to ceph-volume cmds

needed when testing with a multi-cluster setup

Signed-off-by: Michael Fritch <mfritch@suse.com>
(cherry picked from commit 3848ec26bf04b94838404169a57a739f43c2098b)

mgr/cephadm: expose NFS "common" config

populate `rados_config_location` in the ServiceDescription with the
RADOS url of the "common" config

Signed-off-by: Michael Fritch <mfritch@suse.com>
(cherry picked from commit 05a9a1bfdaef1fff69ca8d0e959c84d157190db2)

mgr/cephadm: create a "common" config per nfs service

instead of a per-daemon RADOS conf object

Signed-off-by: Michael Fritch <mfritch@suse.com>
(cherry picked from commit 35a4b24a6ff6ca344a6eaa0b14dc8be9ff3173b3)

mgr/cephadm: remove stray regexp

Signed-off-by: Michael Fritch <mfritch@suse.com>
(cherry picked from commit 988b3cb76e3f8a9bbf2c83485c1d009305a4d025)

cephadm: show error when no command is specified

Traceback (most recent call last):
  File "./cephadm", line 4452, in <module>
    if args.func != command_check_host:
AttributeError: 'Namespace' object has no attribute 'func'

Signed-off-by: Michael Fritch <mfritch@suse.com>
(cherry picked from commit 2ebb9373b52342eb8c523866e5852d2b8bf44e58)

mgr/cephadm: allow config for an nfs `container_image`

ceph config set <entity> container_image <container_image>

Signed-off-by: Michael Fritch <mfritch@suse.com>
(cherry picked from commit 659a6936e277a7205742e02a489d8a56ae1bd9ed)

tox: Fix the tox.ini's to support older versions of tox

The src/cephadm/tox.ini and src/pybind/mgr/tox.ini both don't run
on older versions of tox.
When using tox 2.9.1 both fail for different reasons.

`src/cephadm/tox.ini` fails because `skipsdist=true` only works if it's
directly under the `[tox]` section.

`src/pybind/mgr/tox.ini` fails because older versions of tox can't find
the requirements.txt because they don't like whitespace between the `-r`
and `requirements.txt`.

This patch changes the tox.ini's to be backwards compatible for those
who happen to be running slightly older version of tox.

Signed-off-by: Matthew Oliver <moliver@suse.com>
(cherry picked from commit a1fd9d11e7e11a2478f268251a4a02f2d260fc1c)

doc/cephadm: <encrypted> is a global flag

Signed-off-by: Joshua Schmid <jschmid@suse.de>
(cherry picked from commit d524ad72306b6b2da688c85fddf997afbcd69f57)

cephadm: ceph-iscsi first draft

This if the first draft of the ceph-iscsi in cephadm.
There are a few gotchas when running `rbd-target-api` in a container:

 1. We need both the ceph.conf and iscsi-gateway.cfg, so needed to
ability to pass extra config. This latter is based off the spec, so now
the daemon config func api allows you to return a dict of configs:

  { 'config': '<str>' # will be appended to the ceph.conf
    '<conf name>': 'str', # Will be dumped in datadir/<conf name>
    ...
  }

It will be up to cephadm to know to bind mount it to the right location.
The first 'config' isn't used by this patch, but makes it possible for
specs or config funcs to append anything? maybe it's overkill.

 2. We need the kernel's configfs in the container so we can configure
LIO. There is a chicken and egg problem, configfs isn't mounted on the
host to bind mount when the container starts. So now a check is added to
the `unit.run` and cleanup in the `unit.poststop` scripts for
daemon_type iscsi.

 3. rbd-target-api is python and hardcodes a few things, like logging
through `/dev/log` which happens to be a domain socket. So `/dev/log`
also needed to be bind mounted into the continer.

 4. The daemon expects the keyring to be in `/etc/ceph` so this needed to
be specifically bind mounted to the correct location too.

As this currently stands this is deploying and starting the api on port
5000, so seems to be "working", also gateway.conf does exist in the
pool. I have yet to set up an iscsi device, but will test that next.

The `rbd-target-api` daemon expects the ssl key and cert to be named a
certain name in the contianer. So SSL isn't working yet. However, I do
hav a PR in ceph-iscsi to look in the mon config-key store for them[0].

[0] - https://github.com/ceph/ceph-iscsi/pull/173

Signed-off-by: Matthew Oliver <moliver@suse.com>
(cherry picked from commit 4179b960a4b1aa9c72ff51302eb6a83193b860b4)

mgr/orch: fix python3 DeprecationWarning

test_orchestrator/module.py:181: DeprecationWarning: invalid escape sequence \s
    patterns = ['-i\s(\w+)', '--id[\s=](\w+)']

Signed-off-by: Michael Fritch <mfritch@suse.com>
(cherry picked from commit f77defe55174d6f42a8a2d824c22530d11db4f7d)

python-common: add pyyaml to requirements file

Signed-off-by: Joshua Schmid <jschmid@suse.de>
(cherry picked from commit 04f0d3395c70252a0830f6f3326f85695c9b9af0)

python-common: python-common: fix /hosts/ parsing in servicespecs

Signed-off-by: Joshua Schmid <jschmid@suse.de>
(cherry picked from commit 494728c5dc729de2dcc9c40d92e154c2bd5d8008)

doc: add a basic client configuration document

Signed-off-by: Jeff Layton <jlayton@redhat.com>
(cherry picked from commit 85df3a5fb2d388045c0b01bc5bd069a53b0216c5)

Merge pull request #34440 from sebastian-philipp/octopus-backport-34406

octopus: qa/suites/rados/cephadm/upgrade: start from v15.2.0

Reviewed-by: Michael Fritch <mfritch@suse.com>

Merge pull request #34438 from sebastian-philipp/octopus-backport-34062-34061-34248-34295-34250-34206-34361-34330-34367-34384-34296-34385-34374

octopus: cephadm: batch backport March

Reviewed-by: Joshua Schmid <jschmid@suse.de>
Reviewed-by: Laura Paduano <lpaduano@suse.com>

Merge pull request #34423 from smithfarm/wip-44893-octopus

octopus: cephadm: ceph-volume: disallow concurrent execution

Reviewed-by: Jan Fajerski <jfajerski@suse.com>
Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>

Merge pull request #34407 from liewegas/pr-34296-octopus

octopus: qa/tasks/cephadm: add 'roleless' mode

Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>

Merge pull request #34523 from smithfarm/wip-45053-octopus

octopus: doc/releases/nautilus: restart OSDs to make them bind to v2 addr

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

doc/releases/nautilus: restart OSDs to make them bind to v2 addr

Fixes: https://tracker.ceph.com/issues/43896
Signed-off-by: Nathan Cutler <ncutler@suse.com>
(cherry picked from commit 4facd9daa524f1e2d77511ee6b1a18e108c07d31)

dashboard: Convert FQDN to hostname in grafana panels

The $ceph_hosts variable contained the FQDN for hosts
while the instance label created by ceph only has
the hostname.

Fixes: https://tracker.ceph.com/issues/44784
Signed-off-by: Kristoffer Grönlund <kgronlund@suse.com>
(cherry picked from commit b7abaab5bd2b72f48f41b3d2a9732743afdcecfc)

dashboard: Resolve FQDN / hostname mismatch in hosts overview panel

In the AVG Disk Utilization panel, the result is calculated
by combining the output of node_disk_io_time_seconds_total
with the output of ceph_disk_occupation. However, the
first vector encodes the instance label with the full FQDN
while the ceph label only contains the hostname:port. In
order for these to match correctly, the domain name and port
has to be stripped from the labels.

Fixes: https://tracker.ceph.com/issues/44784
Signed-off-by: Kristoffer Grönlund <kgronlund@suse.com>
(cherry picked from commit 136d21e21dc3c05ec8c586a47eed1904ffbda578)

dashboard: Use exported_instance to identify OSDs

When moving to LVM-based ceph-volume setups, several
grafana dashboards stopped working. The problem is that
(device, instance) no longer results in unique labels
which causes errors like:

"many-to-many matching not allowed: matching labels must be unique on one side"

Fixes: https://tracker.ceph.com/issues/44784
Signed-off-by: Kristoffer Grönlund <kgronlund@suse.com>
(cherry picked from commit 8b61b8d3d781b9d762098cf45b41b596a32509ab)

dashboard: AVG RAM Utilization panel always showed "N/A"

The references to `$osd_hosts` etc. were encoded as
`[[osd_hosts]]` in the PromQL expression divisor, and
the panel always displayed N/A as the result of the
query.

Replacing the `[[...]]` with `$...` makes the expression
work again.

Fixes: https://tracker.ceph.com/issues/44784
Signed-off-by: Kristoffer Grönlund <kgronlund@suse.com>
(cherry picked from commit 4444333243aaa81de1d7347bf0c57e039b657d43)

15.2.1

Merge pull request #34482 from ceph/octopus-fixes

Octopus fixes

Reviewed-By: Casey Bodley <cbodley@redhat.com>
Reviewed-By: Radoslaw Zarynski <rzarynski@redhat.com>
Reviewed-By: Josh Durgin <jdurgin@redhat.com>

rgw: reject control characters in response-header actions

S3 GetObject permits overriding response header values, but those inputs
need to be validated to insure only characters that are valid in an HTTP
header value are present.

Credit: Initial vulnerability discovery by William Bowling (@wcbowling)
Credit: Further vulnerability discovery by Robin H. Johnson <rjohnson@digitalocean.com>
Signed-off-by: Robin H. Johnson <rjohnson@digitalocean.com>

rgw: EPERM to ERR_INVALID_REQUEST

As per Robin's comments and S3 spec

Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>

rgw: reject unauthenticated response-header actions

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
Reviewed-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit d8dd5e513c0c62bbd7d3044d7e2eddcd897bd400)

msg/async/crypto_onwire: fix endianness of nonce_t

As a AES-GCM IV, nonce_t is implicitly shared between server and
client.  Currently, if their endianness doesn't match, they are unable
to communicate in secure mode because each gets its own idea of what
the next nonce should be after the counter is incremented.

Several RFCs state that the nonce counter should be BE, but since we
use LE for everything on-disk and on-wire, make it LE.

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
Reviewed-by: Sage Weil <sage@redhat.com>

msg/async/ProtocolV2: avoid AES-GCM nonce reuse vulnerabilities

The secure mode uses AES-128-GCM with 96-bit nonces consisting of a
32-bit counter followed by a 64-bit salt.  The counter is incremented
after processing each frame, the salt is fixed for the duration of
the session.  Both are initialized from the session key generated
during session negotiation, so the counter starts with essentially
a random value.  It is allowed to wrap, and, after 2**32 frames, it
repeats, resulting in nonce reuse (the actual sequence numbers that
the messenger works with are 64-bit, so the session continues on).

Because of how GCM works, this completely breaks both confidentiality
and integrity aspects of the secure mode.  A single nonce reuse reveals
the XOR of two plaintexts and almost completely reveals the subkey
used for producing authentication tags.  After a few nonces get used
twice, all confidentiality and integrity goes out the window and the
attacker can potentially encrypt-authenticate plaintext of their
choice.

We can't easily change the nonce format to extend the counter to
64 bits (and possibly XOR it with a longer salt).  Instead, just
remember the initial nonce and cut the session before it repeats,
forcing renegotiation.

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
Reviewed-by: Sage Weil <sage@redhat.com>
Conflicts:
src/msg/async/ProtocolV2.h [ context: commit ed3ec4c01d17
  ("msg: Build target 'common' without using namespace in
  headers") not in octopus ]

rpm: add python3-saml as install dependency

`python.*-saml` is required for Ceph-Dashboard SSO support (optional
feature).

Fixes: https://tracker.ceph.com/issues/44721
Signed-off-by: Ernesto Puerta <epuertat@redhat.com>
(cherry picked from commit d8d3b33633c0a995aef034fdabc55d47c3872566)

Merge pull request #34372 from rhcs-dashboard/wip-44697-octopus

octopus: mgr/dashboard: add popover list of Stand-by Managers & Metadata Servers (MDS) in landing page

Reviewed-by: Laura Paduano <lpaduano@suse.com>
Reviewed-by: Stephan Müller <smueller@suse.com>

Merge pull request #34358 from smithfarm/wip-44786-octopus

octopus: qa/tasks/mgr/dashboard/test_health: update mdsmap schema

Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Kiefer Chang <kiefer.chang@suse.com>
Reviewed-by: Laura Paduano <lpaduano@suse.com>

Merge pull request #34402 from rhcs-dashboard/wip-44932-octopus

octopus: mgr/dashboard: fix error when enabling SSO with cert. file

Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Laura Paduano <lpaduano@suse.com>

pybind/mgr: tox.ini include cython bindings.

(cherry picked from commit a44de38b61d598fb0512ea48da0de4179d39b804)

Conflicts:
src/pybind/mgr/tox.ini

The rest of this commit was not picked:

> mgr_util: add CephfsClient implementation
>
> This pulls parts of the VolumesClient implementation into mgr_util to
> make the CephFS specific pieces available to other mgr modules. To
> reduce code duplication the VolumeClient now extends the CephfsClient
> class to add the volume specific methods.

> Signed-off-by: Jan Fajerski <jfajerski@suse.com>

Merge pull request #34403 from rhcs-dashboard/wip-44933-octopus

octopus: mgr/dashboard: fix notifications E2E tests

Reviewed-by: Laura Paduano <lpaduano@suse.com>
Reviewed-by: Tiago Melo <tmelo@suse.com>

Merge pull request #34397 from votdev/wip-44921-octopus

octopus: mgr/dashboard: Add more debug information to Dashboard RGW backend

Reviewed-by: Stephan Müller <smueller@suse.com>
Reviewed-by: Laura Paduano <lpaduano@suse.com>

qa/suites/rados/cephadm/upgrade: start from v15.2.0

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 44ca20017a23625d47aa32c20f277fa4c69de76a)

pybind/mgr: Fix `run_tox.sh mgr -- cephadm`

Failed with a cryptic error.

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
(cherry picked from commit dbd696466799bc47e0716a9dc0c5cf916e99213e)

doc/dev/cephadm: a few notes on developing with cephadm

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 086acd20b956e4ec5e19f4c72bd3b2214dc46d2a)

qa/suites/rados/cephadm/smoke-roleless: add smoke test

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 76b559598a4fe38e77615d05b2dac60138a18d3e)

qa/tasks/cephadm: add 'roleless' mode

Allow cephadm to start up with roles like:

roles:
- - host.a
  - client.0
  - osd.0
  - osd.1
- - host.b
  - osd.2
  - osd.3

Cephadm will pick the mon names (based on host) and provision all
services by default.

The cephadm task can still provision other daemons, but it may
fight with mgr/cephadm.

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 9e609c9ed7f8699f08e682dc6d83611bd394fce7)

cephadm: create /var/run/ceph dir via unit.run, not unit file

The systemd unit file is shared with non-ceph daemons, which (1) don't
need the /var/run directory, and (2) are based on a uid/gid from a
different container image, which means we can't figure out the right
ceph uid/gid from them to set the ownership properly.

Instead, put it in the unit.run file... and only for ceph daemons when
we have the uid/gid we need.

Fixes: https://tracker.ceph.com/issues/44894
Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit a3be5f2aca8ac8163906e3015740327440a375b1)

CODEOWNERS: update cephadm paths

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
(cherry picked from commit 3366f6b616b9bf40a5b953276ba380fb83035d67)

cephadm: add `extra_args` to nfs daemon

Signed-off-by: Michael Fritch <mfritch@suse.com>
(cherry picked from commit 246a80af7a51a768fbc81b36d5963306b5062a6b)

cephadm: check hostnames case insensitive

Salt converts minion names to lowercase.

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
(cherry picked from commit 8194d4972fe3e58c801e4f0554fdff98ed7dfaa4)

mgr/cephadm: add useful error if python3 is not on remote host

show 'Error ENOENT: New host example (example) failed check: ["Can't communicate with
remote host, possibly because python3 is not installed there"]' instead of traceback
with OSError: cannot send(already closed?) when adding host if python3 is not on host

Signed-off-by: Daniel-Pivonka <dpivonka@redhat.com>
(cherry picked from commit bb4554a3a67c4fc29b5dad597fbb880ea52394ac)

python-common: raise on emtpy drive groups

adds a test for https://tracker.ceph.com/issues/44758

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
(cherry picked from commit 874ba5049a820d6d8aa772b34c971048641e2143)

qa/tasks/cephadm: no need to explicitly reconfig

cephadm does this magically now.

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 65ad142d59b40f2c87fbfa9b593a7f4c367d3c42)

qa/tasks/cephadm: fetch final ceph.conf

Otherwise we'll distribute the initial ceph.conf, which only has the
bootstrap mon in it.

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 29d2d7d64b01daed19ba18d971101d756b57ced6)

qa/tasks/cephadm: distribute ceph.conf and admin keyring to all nodes

Revert part of 96220c0c0574eb5b896023e1552f528bef9e1ca5 so that we still
distribute a *final* ceph.conf and admin keyring to all nodes, right after
all of the mons are up.

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 4c78dfe26fd0ec80a517d200f4685b216272f682)

mgr/orch: Add `ceph orch ls --export`

* defaults to `--format yaml`
* don't include `status`

```
$  ceph orch ls --export
placement:
  host_pattern: '*'
service_name: crash
service_type: crash
```

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
(cherry picked from commit 3543aeb84cfc3d7b8e9874cc70dad4d96cfefaf8)

mgr/dashboard: adapt to new `ServiceDescription.to_json()`

`ServiceDescription.to_json()` is now based on `ServiceSpec.to_json()`

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
(cherry picked from commit c5e6ecb54758673828a4a0ed3c817c53bde0d9ee)

python-common: add `service_name` to `ServiceSpec.to_json`

To make it compatible to `Orchestrator.describe_service`.
Otherwise we have the awkward situation that users need to
pass `service_name` to `describe_service`, but `service_id` to apply

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
(cherry picked from commit 6bffff2f1be55f09b43b7abdb36646f0939f1f98)

python-common: make ServiceSpec and ServiceDescription compatible

`ServiceSpec.from_json(ServiceDescription().to_json())` now works

as does

`ceph orch ls  --format yaml | ceph orch apply -i -`

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
(cherry picked from commit 7f8c7ad3ca1d5c3a70033aa85c6adbd7cfa6f547)

src/ceph.in: add yaml to known formats

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
(cherry picked from commit 4a4dac27254e97d0d41e201dc200518654a520fa)

mgr/orch: add yaml to `orch ls`

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
(cherry picked from commit dcde4899ce42836b8c15dcd6076d2521bca24ff7)

mgr/orch: remove `orch spec dump`

as this is now a subset of `orch ls`

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
(cherry picked from commit 446ede26f3dcbaa5d9a631730c43ab9b11b38b52)

python-common: reorder RGWSpec arguments

to match the argumens from `ServiceSpec`

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
(cherry picked from commit 9335b7e5647b6afafdcdfe30c8023fc1e7655a34)

python-common: prevent ServiceSpec of wrong type

Some Python foo to make sure, we don't have an object
like `ServiceSpec('rgw')` of type `ServiceSpec`. Now we have:

>>> type(ServiceSpec('rgw')) == type(RGWSpec('rgw'))

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
(cherry picked from commit e39088c30e315fa8d00e6baf1090839f71bf711a)

pybind/mgr: tox.ini: omit cov report

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
(cherry picked from commit 672ee984ec1ca53ee9a9fe79e826053c3fa40b45)

Conflicts:
src/pybind/mgr/tox.ini

Merge pull request #34428 from badone/wip-octopus-specify-rule-for-pool-creation

octopus: specify rule for pool creation

Reviewed-by: Yuri Weinstein <yweinste@redhat.com>

mgr/cephadm: test describe_service

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
(cherry picked from commit 372d2aca35495c74a8e7d60c8cee36ed05250cf4)

mgr/orch: ServiceDescription: change json representation

In order to match the new paradigm

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
(cherry picked from commit 24e375a53687ff4ff7e92cbb0b3e8e4a382a8e0c)

mgr/orch: ServiceDescription: Make spec a requirement

Because, a ServiceDescription is superset of a spec

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
(cherry picked from commit b4c88196ffbfdf993834b14a888b30a50ec64a89)

cephadm: update check-host() to return all problems

if checks fail, they show one at a time, forcing the admin to repeat
the command to get passed each check.

All checks should run and report once, so the admin can fix all issues
in one go, and not be forced to do repeated commands

Signed-off-by: Daniel-Pivonka <dpivonka@redhat.com>
(cherry picked from commit daed94a539e84f2efb7b6c05daceabf8ea96318f)