osd/PrimaryLogPG: fix SPARSE_READ stat

22960192 use readv to reimplement SPARSE_READ, however it is
still using total_read to accumulate total bytes it reads from
bluestore, which is always zero in code.

Fix by dropping the redundant local total_read counter.
By definition objects_readv_sync should return the correct bytes
it has read. Use that instead.

Signed-off-by: Yan Jun <yan.jun8@zte.com.cn>
(cherry picked from commit 528fb95326e76518e93fe43274f7a8e86049851d)

Merge pull request #34760 from callithea/wip-45281-octopus

Reviewed-by: Stephan Müller <smueller@suse.com>
Reviewed-by: Tiago Melo <tmelo@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>

Merge pull request #34467 from rhcs-dashboard/wip-44992-octopus

Reviewed-by: Laura Paduano <lpaduano@suse.com>

Merge pull request #34708 from rhcs-dashboard/wip-45233-octopus

Reviewed-by: Brad Hubbard <bhubbard@redhat.com>
Reviewed-by: Laura Paduano <lpaduano@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>

Merge pull request #34631 from shyukri/wip-44834-octopus

octopus: mgr/dashboard: 'Prometheus / All Alerts' page shows progress bar

Reviewed-by: Sebastian Krah <skrah@suse.com>
Reviewed-by: Tatjana Dehler <tdehler@suse.com>
Reviewed-by: Laura Paduano <lpaduano@suse.com>

Merge pull request #34547 from tspmelo/wip-45083-octopus

octopus: mgr/dashboard: Fix iSCSI's username and password validation

Reviewed-by: Ricardo Marques <rimarques@suse.com>
Reviewed-by: Stephan Müller <smueller@suse.com>
Reviewed-by: Laura Paduano <lpaduano@suse.com>

Merge pull request #34498 from votdev/wip-45020-octopus

octopus: mgr/dashboard: use FQDN for failover redirection

Reviewed-by: Lenz Grimmer <lgrimmer@suse.com>
Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Stephan Müller <smueller@suse.com>

Merge pull request #34211 from s0nea/wip-44712-octopus

octopus: mgr/dashboard: add crush rule test suite

Reviewed-by: Stephan Müller <smueller@suse.com>
Reviewed-by: Laura Paduano <lpaduano@suse.com>

mgr/dashboard: Fix ServiceDetails and PoolDetails unit tests

While testing the upgrade to Angular 9,
these 2 unit tests were consistently failling.

Fixes: https://tracker.ceph.com/issues/42929
Signed-off-by: Tiago Melo <tmelo@suse.com>
(cherry picked from commit d7e7d2b09e463a978d2f881b6b1978fd2606651e)

Merge pull request #34750 from yuriw/wip-yuriw-distro-octupus

qa/tests: clean up distro and point to centos 8.1

qa/tests: clean up distro and point to centos 8.1

Signed-off-by: Yuri Weinstein <yweinste@redhat.com>

Merge pull request #34687 from sebastian-philipp/octopus-backport-34558-34216-34415-34555-34556-34490-34529-34346-32951-34584-34612-34565-34568-34621

octopus: cephadm: Batch backport April (2)

Reviewed-by: Joshua Schmid <jschmid@suse.de>
Reviewed-by: Michael Fritch <mfritch@suse.com>

Merge pull request #34548 from tspmelo/wip-45084-octopus

Reviewed-by: Laura Paduano <lpaduano@suse.com>
Reviewed-by: Ricardo Marques <rimarques@suse.com>

Merge pull request #34625 from shyukri/wip-45051-octopus

Reviewed-by: Laura Paduano <lpaduano@suse.com>
Reviewed-by: Stephan Müller <smueller@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>

Merge pull request #34630 from shyukri/wip-44837-octopus

Reviewed-by: Laura Paduano <lpaduano@suse.com>
Reviewed-by: Stephan Müller <smueller@suse.com>

Merge pull request #34686 from sebastian-philipp/octopus-backport-34443

octopus: cephadm: Update images used

Reviewed-by: Michael Fritch <mfritch@suse.com>

Merge pull request #34638 from smithfarm/wip-45036-octopus

octopus: rbd: ignore tx-only mirror peers when adding new peers

Reviewed-by: Mykola Golub <mgolub@mirantis.com>
Reviewed-by: Jason Dillaman <dillaman@redhat.com>

mgr/dashboard: fix tasks.mgr.dashboard.test_rgw.RgwBucketTest.test_all

Increased the time for getting 2nd TOTP.

Fixes: https://tracker.ceph.com/issues/44405
Signed-off-by: Alfonso Martínez <almartin@redhat.com>
(cherry picked from commit ee2b2d19781e531770350ad2abe53ce876cebe3b)

Merge pull request #34526 from smithfarm/wip-45034-octopus

octopus: spec: address some warnings raised by RPM 4.15.1

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #34519 from smithfarm/wip-45059-octopus

octopus: qa/suites/rados/rest: don't pass empty dict as data arg

Reviewed-by: Brad Hubbard <bhubbard@redhat.com>

Merge pull request #34395 from votdev/wip-44918-octopus

octopus: monitoring: alert for prediction of disk and pool fill up broken

Reviewed-by: Patrick Seidensal <pnawracay@suse.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>

Merge pull request #34648 from shyukri/wip-44892-octopus

Reviewed-by: Kiefer Chang <kiefer.chang@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>

Merge pull request #34418 from votdev/wip-44955-octopus

Reviewed-by: Laura Paduano <lpaduano@suse.com>
Reviewed-by: Tiago Melo <tmelo@suse.com>

test_cephadm.sh: remove unmaintained test_cephadm.sh

the script has become unmaintained and duplicates logic found in
qa/workunits/cephadm/test_cephadm.sh

Signed-off-by: Michael Fritch <mfritch@suse.com>
(cherry picked from commit 879e6850992685115d0bd0a38b4391722b4e646f)

qa/workunits/cephadm/test_cephadm.sh: add CLEANUP var

preserve the current test state during EXIT:
`CLEANUP=false ../qa/workunits/cephadm/test_cephadm.sh`

Signed-off-by: Michael Fritch <mfritch@suse.com>
(cherry picked from commit 6b211447230d3b6d6d8d49b780750cf6aeeace6d)

mgr/orch: upgrade: better input validation

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
(cherry picked from commit 67a29846f80fe8903f2dbce5dca0f5a0134d55b8)

mgr/orch: cleanup python imports

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
(cherry picked from commit 62ed783dc243e076e00be93ea8ab2dd1fb262d58)

cephadm: bail out early during adopt if data dir not present

In case someone tries to run this again on an already adopted daemon...

Signed-off-by: Tim Serong <tserong@suse.com>
(cherry picked from commit bed711d3d1256758bdf7428339316bcca6f9dae0)

cephadm: add --force-start option to adopt

Current behaviour is to only start a newly adopted ceph daemon if it was
already running before the adopt.  Adding a --force-start option allows
the adopt command to start newly adopted daemons that weren't originally
running, to save the user having to manually invoke `systemctl start
ceph-$FSID@$DAEMMON.$ID`.

Signed-off-by: Tim Serong <tserong@suse.com>
(cherry picked from commit 225b25512916ec4a9577ca7f2ab39faaf122827f)

cephadm: don't throw when disabling ceph-volume simple unit

When adopting OSDs, if a ceph-volume simple service is already disabled
(or otherwise missing) the previous implementation would raise an error,
thus killing the adopt.

Signed-off-by: Tim Serong <tserong@suse.com>
(cherry picked from commit 6834ad983762b7cbf950c6a6e71aa44ec6657c32)

cephadm: handle adopting offline OSDSs

The current adopt behavior expects OSDs to be online, in order to read
/var/lib/ceph/osd/ceph-$ID/fsid.  To handle the case where OSDs
are offline, this change first checks to see if that file is present,
and if not, falls back to calling `ceph-volume lvm list` to see if
there's a matching OSD there, and if that doesn't work, it checks
/etc/ceph/osd/*.json to see if there's a matching old-style simple
OSD present.

For LVM OSDs, the only thing we need is the ODS's fsid; the remainer
of the adopt procedure "just works", as the various other files
in /var/lib/ceph/$FSID/osd.$ID are created by magic anyway when the
OSD is activated, so it doesn't matter if they're not present at
adoption time.

For simple (ceph-disk created) OSDs, we actually need all the files under
/var/lib/ceph/osd/ceph-$ID/ to be moved to /var/lib/ceph/$FSID/osd.$ID
so if a simple OSD is found, it's mounted first, so the existing
move_files() a bit further down around line 3200 continues to work.

Fixes: https://tracker.ceph.com/issues/45095
Signed-off-by: Tim Serong <tserong@suse.com>
(cherry picked from commit 9169119bd3c0ac976871e7f3321d9d7a53335c82)

Add troubleshooting guidance for ssh connection failures.

For more thorough information about this commit, see:
https://tracker.ceph.com/issues/44905

Signed-off-by: Zac Dover <zac.dover@gmail.com>
Update doc/cephadm/troubleshooting.rst

Co-Authored-By: Michael Fritch <mfritch@suse.com>
ibid

Signed-off-by: Zac Dover <zac.dover@gmail.com>
Update doc/cephadm/troubleshooting.rst

Co-Authored-By: Sebastian Wagner <sebastian@spawnhost.de>
Update doc/cephadm/troubleshooting.rst

Co-Authored-By: Sebastian Wagner <sebastian@spawnhost.de>
Added mgfritch's suggestion regarding checking to see if the public key
is listed in the authorized_keys file.

Signed-off-by: Zac Dover <zac.dover@gmail.com>
(cherry picked from commit c4765afb81489b1df71d87be32c566ed55d29a28)

mgr/test_orchestrator: fix error when listing services

- Adapting attribute change of ServiceSpec.
- Update dummy data.

Fixes: https://tracker.ceph.com/issues/45108
Signed-off-by: Kiefer Chang <kiefer.chang@suse.com>
(cherry picked from commit 223c8d816f7f1d3d5ea737c540bae4553334aabf)

mgr/orch: Fixes some deserialization errors

Fixes errors when calling `from_json` of these classes:
- InventoryHost: parsing labels
- ServiceDescription: `last_refresh` and `created` fields should be parsed
  to datetime type.

Signed-off-by: Kiefer Chang <kiefer.chang@suse.com>
(cherry picked from commit 45fd5ee275aa94409bbe9d9c3874c9aad82850a8)

cephadm: add RGW SSL support

This patch adds SSL support to RGW when using cephadm.
If an SSL certificate is provided, inside the json supplied with:

  cpeh orchestrator rgw create -i rgw.json

Then the SSL cert and/or key will be added to pushed into the mon config-key database
using the key `rgw/cert/<rgw_realm>/<rgw_zone>.[crt|key]`.
Which will then be referenced in the config:

  rgw_frontends = beast port=80 ssl_port=443 ssl_certificate=config://rgw/cert/<rgw_realm>/<rgw_zone>.crt

And if an ssl key is also supplied this becomes something like:

  rgw_frontends = beast port=80 ssl_port=443 ssl_certificate=config://rgw/cert/<rgw_realm>/<rgw_zone>.crt ssl_key=config://rgw/cert/<rgw_realm>/<rgw_zone>.key

Of course you could also just upload the cert and key yourself to
config-key location, and ssl will be enabled as well. But this patch
let's you either supply them via `-i` or as a manual upload step.

Co-Authored-By: Michael Fritch <mfritch@suse.com>
Co-Authored-By: Sebastian Wagner <sebastian@spawnhost.de>
Signed-off-by: Matthew Oliver <moliver@suse.com>
(cherry picked from commit 8204b1d524b155fac56c3937d1aec41a18ab2d52)

mgr/cephadm: add replacement flag to _create_osd

Signed-off-by: Joshua Schmid <jschmid@suse.de>
(cherry picked from commit e670b1ea83f66d977c919ef956cc4ae0db9cb453)

mgr/cephadm: use --osd-ids instead of --osd-id

Signed-off-by: Joshua Schmid <jschmid@suse.de>
(cherry picked from commit fa88483f660f2bf79d1c1ca77d7104b3a42ed05d)

mgr/cephadm: add support for osd_id_claims

Signed-off-by: Joshua Schmid <jschmid@suse.de>
(cherry picked from commit 5b32c3e51b517332b7ac26b161b7b6b4fae05b85)

cephadm: warn when no valid version arguments are supplied to add-repo

Currently add-repo will add a broken repository by default, this commit gives a pretty error to instruct the user to use one of the version arguments.

Fixes #45029

Signed-off-by: Maran Hidskes <maran@protonmail.com>
(cherry picked from commit 115b3d7b7f3b4cf4149240fc14e374697a7cd6b1)

cephadm: add-repo: Don't test for podman

Fixes:

```
root@buster:/cephadm# ./cephadm add-repo
Unable to locate any of ['podman', 'docker']
```

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
(cherry picked from commit 0d22708cc2f858491604b09f87e7af742884cb3c)

mgr/cephadm: corrected documentation for starting an upgrade

Signed-off-by: Andreas Haase <andreas.haase@community4you.de>
(cherry picked from commit bf3d06d180135901903251e6e73b66f4fbac289b)

mgr/cephadm: Fixed warn_on_stray_hosts setting

Fixes: https://tracker.ceph.com/issues/45065
Signed-off-by: Andreas Haase <andreas.haase@community4you.de>
(cherry picked from commit b07c2da63f9fb9e70a647a989675bbdf309c14f3)

mgr/orch: Fix DaemonDescription().daemon_id() for RGWs

Turns out, users put dots into their RGW service names.

Fixes: https://tracker.ceph.com/issues/44934
Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
(cherry picked from commit 39fdb5c67c4d3834c5157e8d6da20933a6f7bfb4)

mgr/cephadm: enable osd previews

Signed-off-by: Joshua Schmid <jschmid@suse.de>
(cherry picked from commit 462a68ab87446af8299d4f71c5ff22f77dfeea64)

cephadm: use CephContainer during command `enter`

instead of the `get_container` helper func

Signed-off-by: Michael Fritch <mfritch@suse.com>
(cherry picked from commit 3c762d0ed92d38a2c47bf273bf17600f101ed1bb)

cephadm: Update images used

Print warning when using cephadm from master

See also "use quay octopus tip until 15.2 tag is available"
* a9b15c7e1a0c14376cd66f166370694294398494.

See also  "update default container images"
* 1f05f7578794380f969a7e93db07345626b3e4df.

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
(cherry picked from commit de448fafe0fe7d6889f3f75ac37f870cde00cf1c)
Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>

Merge pull request #34495 from votdev/wip-44953-octopus

Reviewed-by: Laura Paduano <lpaduano@suse.com>
Reviewed-by: Stephan Müller <smueller@suse.com>

Merge pull request #34654 from ceph/wip-yuriw-octopus-client-upgrade-octopus-pacific

qa/tests: initial checkin for clients upgrades tests

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

qa/tests: initial checkin for clients upgrades tests

for future release 'pacific'

(Note to self: moved from wrongly pushed to master branch)

Signed-off-by: Yuri Weinstein <yweinste@redhat.com>

mgr/dashboard: shorten `Container ID` and `Container image ID` in Services page

Fixes: https://tracker.ceph.com/issues/44539
Signed-off-by: Volker Theile <vtheile@suse.com>
(cherry picked from commit ed51f8039abad07fe8c44c233c8e30dc4810fe8b)

qa/workunits/rbd: retry the addition of a mirror pool peer

We might race with the remote rbd-mirror daemon creating a
tx-only peer when adding a new peer. Therefore, delete the
tx-only peer and attempt to re-create it.

Fixes: https://tracker.ceph.com/issues/44938
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit fb4311f597a98b6870d7895e6403fb32356bfbe9)

librbd: mirror peer add error message should include site name

It previously included the pointer to string holding the generated
uuid (neither of which would mean much to an end user).

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit ef68f686be6d5cb9ca221e65deeff311bc397aa4)

rbd: improved error message when adding a duplicate mirror pool peer

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 74cded6b0de90660db46d0dea0cd35ca0bba563e)

rbd: ignore tx-only mirror peers when adding new peers

There is a restriction for supporting only a single RX peer but
we should support multiple TX-only peers.

Fixes: https://tracker.ceph.com/issues/44938
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 366ecdb26d90b9dfa483458ce5c28712e47a6341)

mgr/dashboard: 'Prometheus / All Alerts' page shows progress bar

Fixes: https://tracker.ceph.com/issues/44805
Signed-off-by: Volker Theile <vtheile@suse.com>
(cherry picked from commit 1efe1a76da4671bd2df301a9e7080eda2bb01cd2)

mgr/dashboard: Improve iSCSI CHAP messages

iSCSI username and password can also contain numbers

Fixes: https://tracker.ceph.com/issues/44618
Signed-off-by: Ricardo Marques <rimarques@suse.com>
(cherry picked from commit 6ec153f3329c8c6b7c6b6b874083b1bd5d6d3392)

mgr/dashboard: lint error on plugins/debug.py

Make pylint for Python 3.8 and older versions happy.

Fixes: https://tracker.ceph.com/issues/44589
Signed-off-by: Volker Theile <vtheile@suse.com>
(cherry picked from commit 45076ed13a809c8d0837e72700647f1cecc3298d)

Merge pull request #34543 from ceph/wip-yuriw-optopus-p2p-octopus

qa/tests: octopus - initial checkin for octopus-p2p tests

qa/tests: octopus - initial checkin for octopus-p2p tests

Signed-off-by: Yuri Weinstein <yweinste@gmail.com>

Merge pull request #34309 from Exotelis/octopus-translations

octopus: mgr/dashboard: Update translations on octopus

Reviewed-by: Tatjana Dehler <tdehler@suse.com>
Reviewed-by: Stephan Müller <smueller@suse.com>

Merge pull request #34474 from rhcs-dashboard/wip-44996-octopus

octopus: rpm: add python3-saml as install dependency

Reviewed-by: Ricardo Marques <rimarques@suse.com>
Reviewed-by: Nathan Cutler <ncutler@suse.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #34350 from sebastian-philipp/octopus-backport-34180

octopus: cephadm: Fix check_ip_port to work with IPv6

Reviewed-by: Michael Fritch <mfritch@suse.com>

Merge pull request #34554 from sebastian-philipp/octopus-backport-31885-33954-34357-34053-34439-34383-34220-34485-34249-34454-34223-34382-34084-34525-34344

octopus: cephadm: Batch backport April (1)

Reviewed-by: Kiefer Chang <kiefer.chang@suse.com>
Reviewed-by: Michael Fritch <mfritch@suse.com>

Merge pull request #34299 from liewegas/bp-34150-octopus

octopus: cephadm: rm-cluster clean up /etc/ceph

Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>

Merge pull request #34387 from smithfarm/wip-44897-octopus

octopus: pybind/rbd: fix no lockers are obtained, ImageNotFound exception will be output

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
Reviewed-by: Mykola Golub <mgolub@mirantis.com>

mgr/rook: Fix wrong data output when listing devices

Information used to build the <device ls> output is not the right one.
Rook provides in the devices configmap the item <cephVolumeData> which contains the information returned by <ceph-volume>.
This is the right source information to build the devices list.

Signed-off-by: Juan Miguel Olmo Martínez <jolmomar@redhat.com>
(cherry picked from commit e3a5fa49005a7d3ca374beca1dd8cd3e416819a9)

Update index.rst

clean up "only only"

Signed-off-by: Tim <elatllat@gmail.com>
(cherry picked from commit d7c7ccb4390a91c0c791b924141963305809b2ab)

mgr/cephadm: add test for "Offline" host status

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
(cherry picked from commit 7368d77b31234c59b576303e1949ae85025fafd9)

mgr/cephadm: Daemons running of unreachable servers are offline

Fixes: https://tracker.ceph.com/issues/44602
Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
(cherry picked from commit adca3e08e8e9f4a683f7c6cef8e96705cc340188)

mgr/cephadm: Set SSH connection timeout to 30s

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
(cherry picked from commit 152b4b4b298bb7f2f79159a49a4f75dfa9b739c7)

mgr/cephadm: No daemon and device refresh if host if out

To make things faster

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
(cherry picked from commit 201a9c17eeb1051ce562205d316f4baeebb58b5f)

mgr/cephadm: in-memory set of offline hosts

Fixes: https://tracker.ceph.com/issues/43839
Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
(cherry picked from commit 05cf5796f36bc2d826a29aff4a124124df87cbc6)

mgr/cephadm: remove unused method `_check_hosts`

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
(cherry picked from commit e093c3754382f64133178b78d30d113e0f660bb9)

mgr/cephadm: type annotation for HostCache.mgr

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
(cherry picked from commit 52e05f8d20a0042c5ba7bc2d30a9dbaf259f8809)

mgr/cephadm: add CACHEINODE and EXPORT_DEFAULTS sections

Signed-off-by: Michael Fritch <mfritch@suse.com>
(cherry picked from commit 25f4dedd3e75c81b19911fd33b171e613ab1c559)

mgr/cephadm: add config required for ganesha grace

adds minimal config for ganesha-rados-grace

Signed-off-by: Michael Fritch <mfritch@suse.com>
(cherry picked from commit 8055d34455bf49fe5b93f501a60308ec0c681f42)

cephadm: add userid during grace add/remove

ganesha-rados-grace needs a rados userid to add/remove from the grace db

Mar 27 10:05:25 host1 bash[11684]: rados_connect: -13
Mar 27 10:05:25 host1 bash[11684]: Can't connect to cluster: -13

Signed-off-by: Michael Fritch <mfritch@suse.com>
(cherry picked from commit ea5b0430f38393d057166d6e238890a77157fca1)

mgr/cephadm: disable dashboard's grafana cert ssl_verify if we generate it

This will help dashboard work out of the box.

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 81b78265a32cdf2d316dc341fdeb5974dc8c5e29)

qa/workunits/cephadm/test_cephadm.sh: use fsid during dump_log

needed when running with multiple ceph clusters

Signed-off-by: Michael Fritch <mfritch@suse.com>
(cherry picked from commit 7e26083d9262e54e9646cf6a73da4e0f3b807702)

qa/workunits/cephadm/test_cephadm.sh: add fsid to ceph-volume cmds

needed when testing with a multi-cluster setup

Signed-off-by: Michael Fritch <mfritch@suse.com>
(cherry picked from commit 3848ec26bf04b94838404169a57a739f43c2098b)

mgr/cephadm: expose NFS "common" config

populate `rados_config_location` in the ServiceDescription with the
RADOS url of the "common" config

Signed-off-by: Michael Fritch <mfritch@suse.com>
(cherry picked from commit 05a9a1bfdaef1fff69ca8d0e959c84d157190db2)

mgr/cephadm: create a "common" config per nfs service

instead of a per-daemon RADOS conf object

Signed-off-by: Michael Fritch <mfritch@suse.com>
(cherry picked from commit 35a4b24a6ff6ca344a6eaa0b14dc8be9ff3173b3)

mgr/cephadm: remove stray regexp

Signed-off-by: Michael Fritch <mfritch@suse.com>
(cherry picked from commit 988b3cb76e3f8a9bbf2c83485c1d009305a4d025)

cephadm: show error when no command is specified

Traceback (most recent call last):
  File "./cephadm", line 4452, in <module>
    if args.func != command_check_host:
AttributeError: 'Namespace' object has no attribute 'func'

Signed-off-by: Michael Fritch <mfritch@suse.com>
(cherry picked from commit 2ebb9373b52342eb8c523866e5852d2b8bf44e58)

mgr/cephadm: allow config for an nfs `container_image`

ceph config set <entity> container_image <container_image>

Signed-off-by: Michael Fritch <mfritch@suse.com>
(cherry picked from commit 659a6936e277a7205742e02a489d8a56ae1bd9ed)

tox: Fix the tox.ini's to support older versions of tox

The src/cephadm/tox.ini and src/pybind/mgr/tox.ini both don't run
on older versions of tox.
When using tox 2.9.1 both fail for different reasons.

`src/cephadm/tox.ini` fails because `skipsdist=true` only works if it's
directly under the `[tox]` section.

`src/pybind/mgr/tox.ini` fails because older versions of tox can't find
the requirements.txt because they don't like whitespace between the `-r`
and `requirements.txt`.

This patch changes the tox.ini's to be backwards compatible for those
who happen to be running slightly older version of tox.

Signed-off-by: Matthew Oliver <moliver@suse.com>
(cherry picked from commit a1fd9d11e7e11a2478f268251a4a02f2d260fc1c)

doc/cephadm: <encrypted> is a global flag

Signed-off-by: Joshua Schmid <jschmid@suse.de>
(cherry picked from commit d524ad72306b6b2da688c85fddf997afbcd69f57)

cephadm: ceph-iscsi first draft

This if the first draft of the ceph-iscsi in cephadm.
There are a few gotchas when running `rbd-target-api` in a container:

 1. We need both the ceph.conf and iscsi-gateway.cfg, so needed to
ability to pass extra config. This latter is based off the spec, so now
the daemon config func api allows you to return a dict of configs:

  { 'config': '<str>' # will be appended to the ceph.conf
    '<conf name>': 'str', # Will be dumped in datadir/<conf name>
    ...
  }

It will be up to cephadm to know to bind mount it to the right location.
The first 'config' isn't used by this patch, but makes it possible for
specs or config funcs to append anything? maybe it's overkill.

 2. We need the kernel's configfs in the container so we can configure
LIO. There is a chicken and egg problem, configfs isn't mounted on the
host to bind mount when the container starts. So now a check is added to
the `unit.run` and cleanup in the `unit.poststop` scripts for
daemon_type iscsi.

 3. rbd-target-api is python and hardcodes a few things, like logging
through `/dev/log` which happens to be a domain socket. So `/dev/log`
also needed to be bind mounted into the continer.

 4. The daemon expects the keyring to be in `/etc/ceph` so this needed to
be specifically bind mounted to the correct location too.

As this currently stands this is deploying and starting the api on port
5000, so seems to be "working", also gateway.conf does exist in the
pool. I have yet to set up an iscsi device, but will test that next.

The `rbd-target-api` daemon expects the ssl key and cert to be named a
certain name in the contianer. So SSL isn't working yet. However, I do
hav a PR in ceph-iscsi to look in the mon config-key store for them[0].

[0] - https://github.com/ceph/ceph-iscsi/pull/173

Signed-off-by: Matthew Oliver <moliver@suse.com>
(cherry picked from commit 4179b960a4b1aa9c72ff51302eb6a83193b860b4)

mgr/orch: fix python3 DeprecationWarning

test_orchestrator/module.py:181: DeprecationWarning: invalid escape sequence \s
    patterns = ['-i\s(\w+)', '--id[\s=](\w+)']

Signed-off-by: Michael Fritch <mfritch@suse.com>
(cherry picked from commit f77defe55174d6f42a8a2d824c22530d11db4f7d)

python-common: add pyyaml to requirements file

Signed-off-by: Joshua Schmid <jschmid@suse.de>
(cherry picked from commit 04f0d3395c70252a0830f6f3326f85695c9b9af0)

python-common: python-common: fix /hosts/ parsing in servicespecs

Signed-off-by: Joshua Schmid <jschmid@suse.de>
(cherry picked from commit 494728c5dc729de2dcc9c40d92e154c2bd5d8008)

doc: add a basic client configuration document

Signed-off-by: Jeff Layton <jlayton@redhat.com>
(cherry picked from commit 85df3a5fb2d388045c0b01bc5bd069a53b0216c5)

Merge pull request #34440 from sebastian-philipp/octopus-backport-34406

octopus: qa/suites/rados/cephadm/upgrade: start from v15.2.0

Reviewed-by: Michael Fritch <mfritch@suse.com>

Merge pull request #34438 from sebastian-philipp/octopus-backport-34062-34061-34248-34295-34250-34206-34361-34330-34367-34384-34296-34385-34374

octopus: cephadm: batch backport March

Reviewed-by: Joshua Schmid <jschmid@suse.de>
Reviewed-by: Laura Paduano <lpaduano@suse.com>

mgr/dashboard: Prevent iSCSI target recreation when editing controls

Also prevents updating target auth, when this was not modified.

Fixes: https://tracker.ceph.com/issues/44659
Signed-off-by: Tiago Melo <tmelo@suse.com>
(cherry picked from commit 7d370ce22259a5b6a6c25995a1857d960dc5d27d)

mgr/dashboard: Remove duplicate code in iscsi.py

Signed-off-by: Tiago Melo <tmelo@suse.com>
(cherry picked from commit 7291e82ffc878394debbbb00e81dd565c6f6a128)

mgr/dashboard: Fix iSCSI's username and password validation

When using regex, instead of strings, angular doesn't assume the pattern must
validate the entire string.

Add validation to the backend.

Fixes: https://tracker.ceph.com/issues/44624
Signed-off-by: Tiago Melo <tmelo@suse.com>
(cherry picked from commit 532b271572f7b6410c099362dc046148f5a686a1)

Merge pull request #34423 from smithfarm/wip-44893-octopus

octopus: cephadm: ceph-volume: disallow concurrent execution

Reviewed-by: Jan Fajerski <jfajerski@suse.com>
Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>

Merge pull request #34407 from liewegas/pr-34296-octopus

octopus: qa/tasks/cephadm: add 'roleless' mode

Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>

Merge pull request #34523 from smithfarm/wip-45053-octopus

octopus: doc/releases/nautilus: restart OSDs to make them bind to v2 addr

Reviewed-by: Josh Durgin <jdurgin@redhat.com>