Merge pull request #37247 from Vicente-Cheng/wip-46637-octopus

octopus: mds: optimize random threshold lookup for dentry load

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #37243 from Vicente-Cheng/wip-46636-octopus

octopus: mds: fix nullptr dereference in MDCache::finish_rollback

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #37240 from Vicente-Cheng/wip-46634-octopus

octopus: mds: fix mds forwarding request 'no_available_op_found'

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge PR #37520 into octopus

* refs/pull/37520/head:
ceph-volume: don't exit before empty report can be printed
PendingReleaseNotes: add note about batch refactor
pybind/mgr/orchestrator: fix preview for new c-v batch output
ceph-volume batch: return valid empty json reports
python-common: do not skip unavailable devices
ceph-volume: pass filter_for_batch as keyword argument
ceph-volume batch: fix very_fast_allocation plan and add tests
ceph-volume: batch: call the right prepare method
python-common drive-groups: allow strings for wal and db size
ceph-volume inventory: add option to filter unwanted devices
ceph-volume: address review comments
ceph-volume: batch: fix size retrieval for lvs
ceph-volume: include encryption in batch report
ceph-volume lvm batch: use namedtuple instead of tuple
ceph-volume: address review comments, mostly tidying, clarification
idempotency must result in the same outcome
ceph-volume: batch test should pass --journal-devices with filestore
ceph-volume: make --journal optional, add --journal-slots
doc: update ceph-volume lvm batch docs
py-common/deployment: drop workarounds for old batch
ceph-volume batch: add deprecation warning for auto behaviour
ceph-volume batch: add ceph.conf mocking to pass tests
ceph-volume batch: use disk.Size for size args
ceph-volume batch: Fix osd_ids passing and improve plan formatting
ceph-volume batch: track rel_size in percent, more tests
ceph-volume batch: improve backwards compatibility
ceph-volume: batch - enable legacy auto behaviour
ceph-volume: batch - major refactor
ceph-volume: Device - available_lvm if 10 extents are free.
ceph-volume: Device - add vg_free property
ceph-volume: prepare/create - size args as Size class
ceph-volume: disk.Size - add cast to bool
ceph-volume: api/lvm - add VolumeGroup.free_percent property
ceph-volume: util.device -  add vg_free_percent property
ceph-volume: api/lvm - query LV units in bytes
ceph-volume: lvm/common - refactor common arg specification
ceph-volume: batch: fix argument help message

Reviewed-by: Rishabh Dave <ridave@redhat.com>

Merge pull request #37562 from smithfarm/wip-47762-octopus

octopus: mgr/prometheus: add pool compression stats

Reviewed-by: Boris Ranto <branto@redhat.com>

Merge pull request #37504 from smithfarm/wip-arm-isa-l-octopus

octopus: erasure-code: enable isa-l EC for aarch64 platform

Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #37450 from callithea/wip-47621-octopus

octopus: mgr/dashboard: fix security scopes of some NFS-Ganesha endpoints

Reviewed-by: Volker Theile <vtheile@suse.com>
Reviewed-by: Tatjana Dehler <tdehler@suse.com>

Merge pull request #37383 from ukernel/octopus-47608

octopus: mds: reduce memory usage of open file table prefetch #37382

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #37372 from smithfarm/wip-47604-octopus

octopus: mds: fix purge_queue's _calculate_ops is inaccurate

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #37371 from smithfarm/wip-47601-octopus

octopus: mgr/volumes/nfs: Check if orchestrator spec service_id is valid

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #37370 from smithfarm/wip-47260-octopus

octopus: cephfs: client: handle readdir reply without Fs cap

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #37369 from smithfarm/wip-47255-octopus

octopus: cephfs: client: make Client::open() pass proper cap mask to path_walk

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #37368 from smithfarm/wip-47253-octopus

octopus: cephfs: osdc/Journaler: do not call onsafe->complete() if onsafe is 0

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

mds/OpenFileTable: add option that disables dirfrags prefetch

Dirfrags prefetch may speedup rejoin stage of mds recovery. But it may
use lots of memory and cause mds out of memory.

Signed-off-by: "Yan, Zheng" <zyan@redhat.com>
(cherry picked from commit cc19fc624b1ee4d7e3248d1dfc8f89f8879a46bf)

mds: use parent/d_name in cap_reconnect_t to speedup MDCache::open_ino()

Signed-off-by: "Yan, Zheng" <zyan@redhat.com>
(cherry picked from commit 3e7c5b4077b68ec519f9e3e98fea8fd9dcf9cbc3)

client: encode inode's parent/d_name in cap_reconnect_t

Signed-off-by: "Yan, Zheng" <zyan@redhat.com>
(cherry picked from commit 7f43840fd3f9a8f88f809dcad5d3d381e76ba7f3)

mds: explicitly track files with client writeable ranges

Signed-off-by: "Yan, Zheng" <zyan@redhat.com>
(cherry picked from commit 44f67be02b8e8aa8fc69283bf12107c2df3f8277)

 Conflicts:
src/mds/CInode.cc
src/mds/Locker.cc
src/mds/MDCache.cc
src/mds/Server.cc
src/mds/StrayManager.cc

mds: properly set CLIENTWRITEABLE flag for imported caps

Signed-off-by: "Yan, Zheng" <zyan@redhat.com>
(cherry picked from commit 8c56adabb36f2983437fa73f88c00b7d62d0a880)

mds: in-place update projected inode's client ranges

Signed-off-by: "Yan, Zheng" <zyan@redhat.com>
(cherry picked from commit 994017b6155af7fbb5a53f76f6a4525b363ba07f)

 Conflicts:
src/mds/Locker.cc
src/mds/Server.cc

mds: only add inodes that clients want Frw or excl caps to open file table

Signed-off-by: "Yan, Zheng" <zyan@redhat.com>
(cherry picked from commit e6de138f3334adc26cac4c01dfbbdf8a2472679a)

mds/OpenFileTable: reduce anchor map lookup during prefetch

reduce one anchor map lookup for each inode

Signed-off-by: "Yan, Zheng" <zyan@redhat.com>
(cherry picked from commit 1adc21e1b46744fa810b9d29530a04369af303e7)

mds/OpenFileTable: merge dirfrag set into anchor map

Track dirfrags whose child inodes have caps in corresponding dir
inodes' anchor objects.

Signed-off-by: "Yan, Zheng" <zyan@redhat.com>
(cherry picked from commit 12691168dfed4c6cfb97ad3f4841a078b84a1f11)

Merge pull request #37090 from shyukri/wip-46961-octopus

octopus: mgr/dashboard: Improve notification badge

Reviewed-by: Aashish Sharma <aasharma@redhat.com>
Reviewed-by: Laura Paduano <lpaduano@suse.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>

Merge pull request #37300 from alvistack/octopus-aio_graceful_reboot

octopus: systemd: Support Graceful Reboot for AIO Node

Reviewed-by: Deepika Upadhyay <dupadhya@redhat.com>

Merge pull request #37533 from smithfarm/wip-47739-octopus

octopus: mgr/devicehealth: device_health_metrics pool gets created even without any OSDs in the cluster

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge pull request #37225 from smithfarm/wip-47531-octopus

octopus: ceph.in: ignore failures to flush stdout

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #37128 from k0ste/wip-47426-octopus

octopus: mgr/zabbix: indent the output of "zabbix config-show"

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #37048 from trociny/wip-47363-octopus

octopus: bluestore: fix collection_list ordering

Reviewed-by: Igor Fedotov <ifedotov@suse.com>

Merge pull request #37436 from sebastian-philipp/octopus-backport-36800-36894-36929-36995-36946-36432-36903-36766-37046-37071-36980-3693

octopus: cephadm batch backport September (2)

Reviewed-by: Michael Fritch <mfritch@suse.com>
Reviewed-by: Nathan Cutler <ncutler@suse.com>

ceph-volume: don't exit before empty report can be printed

get_plan() called exit in case of an empty plan. This prevented a report
being printed under these circumstances. Avoid exit in this case. Also
adds tests to ensure an empty report is printed.

Fixes: https://tracker.ceph.com/issues/47760
Signed-off-by: Jan Fajerski <jfajerski@suse.com>
(cherry picked from commit 0cc5604843b215709a681fa402145c9fa403b1dd)

PendingReleaseNotes: add note about batch refactor

Signed-off-by: Jan Fajerski <jfajerski@suse.com>
(cherry picked from commit 485f0d797e4b21ffb7ac742f0949e8c0a23d43f2)

 Conflicts:
PendingReleaseNotes
        Sort ceph-volume entry under 15.2.6 heading

pybind/mgr/orchestrator: fix preview for new c-v batch output

Signed-off-by: Jan Fajerski <jfajerski@suse.com>
(cherry picked from commit 996df56a4aebe29e32fd529ae926ae484d240943)

 Conflicts:
src/pybind/mgr/orchestrator/tests/test_orchestrator.py
        drop test_handle_command(), not added by us

Merge PR #37345 into octopus

* refs/pull/37345/head:
ceph-volume: avoid format strings for now

Reviewed-by: Guillaume Abrioux <gabrioux@redhat.com>
Reviewed-by: Rishabh Dave <ridave@redhat.com>

doc: replace luminous with |stable-release|

"luminous" is a magic number in these contexts, and we should use a
constant for representing it.

the "sphinx_substitution_extensions" sphinx extension is introduced for
performing the global subsitution.

Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit 7ca85e77d040e990fc60d7fe99447d3fb1e7f69e)

ceph-volume batch: return valid empty json reports

Fixes: https://tracker.ceph.com/issues/47729
Signed-off-by: Jan Fajerski <jfajerski@suse.com>
(cherry picked from commit ab59269a6ca5bb80c28e94beef0338f23fc10fff)

python-common: do not skip unavailable devices

Fixes: https://tracker.ceph.com/issues/47726
We should assemble the device set based on the drivegroups in
mgr/cephadm (regardless of their availability) and pass it to c-v.

This patch is the missing piece to actually do this.

related changes: https://github.com/ceph/ceph/pull/34740/commits/7d168ad7bdbb6d6d5231a4ae540ab03040b49a38

Signed-off-by: Joshua Schmid <jschmid@suse.de>
(cherry picked from commit b9e3da39193419c3fd606a4cba586bea58c6e961)

ceph-volume: pass filter_for_batch as keyword argument

This PR also removes an unused ctor argument in the Devices class.

Fixes: 7d168ad7bdbb6d6d5231a4ae540ab03040b49a38
Signed-off-by: Jan Fajerski <jfajerski@suse.com>
(cherry picked from commit c5a711e5b6499915b7c2a7b7869f890fa7dc7e2d)

ceph-volume batch: fix very_fast_allocation plan and add tests

Signed-off-by: Jan Fajerski <jfajerski@suse.com>
(cherry picked from commit 8178d5c48ac1a7f3915f0003abab6d625385bd78)

ceph-volume: batch: call the right prepare method

Signed-off-by: Jan Fajerski <jfajerski@suse.com>
(cherry picked from commit e75ef77f23ae07463510ec213ac4007f29cbe2da)

python-common drive-groups: allow strings for wal and db size

This enables user to pass sizes like "10G", which batch now understands.

Signed-off-by: Jan Fajerski <jfajerski@suse.com>
(cherry picked from commit 28e33f3025d03008b46b56989e5d939682b5720d)

ceph-volume inventory: add option to filter unwanted devices

Some device we never want to pass to the batch subcommand. For now this
includes devices that have a partition or are mounted on the machine.
One goal is to filter the root device, so it is not included on a batch
command and thus would contribute to its implicit sizing calculation.

Signed-off-by: Jan Fajerski <jfajerski@suse.com>
(cherry picked from commit 7d168ad7bdbb6d6d5231a4ae540ab03040b49a38)

ceph-volume: address review comments

Signed-off-by: Jan Fajerski <jfajerski@suse.com>
(cherry picked from commit 6f1592a1146529d352184c795aae8ce12f66e554)

ceph-volume: batch: fix size retrieval for lvs

Signed-off-by: Jan Fajerski <jfajerski@suse.com>
(cherry picked from commit 24e4aa1296608ef861d4ea4b6a1892246a53ef76)

ceph-volume: include encryption in batch report

Fixes: https://tracker.ceph.com/issues/44783
Signed-off-by: Jan Fajerski <jfajerski@suse.com>
(cherry picked from commit fce184cf9b2d8f15543e1adee49e1fe6cc17437d)

ceph-volume lvm batch: use namedtuple instead of tuple

Signed-off-by: Jan Fajerski <jfajerski@suse.com>
(cherry picked from commit 98c991fc6fd17b18d5bfbebe4b8febe5ff8fa2f0)

ceph-volume: address review comments, mostly tidying, clarification

Signed-off-by: Jan Fajerski <jfajerski@suse.com>
(cherry picked from commit d0735ce1c90c952a6d2e1b805c1326d13ff7b06c)

idempotency must result in the same outcome

...not should

Co-authored-by: Joshua Schmid <jschmid@suse.de>
Signed-off-by: Jan Fajerski <jfajerski@suse.com>
(cherry picked from commit fcacd0b96ab195e939f6f879b0a0362a06385f9a)

ceph-volume: batch test should pass --journal-devices with filestore

Signed-off-by: Jan Fajerski <jfajerski@suse.com>
(cherry picked from commit 55ba8630176041dce898e8b979b5849f13e01ca5)

ceph-volume: make --journal optional, add --journal-slots

Signed-off-by: Jan Fajerski <jfajerski@suse.com>
(cherry picked from commit 7f766846328aac82d75175ed2c1c0bf3438a99e0)

doc: update ceph-volume lvm batch docs

Signed-off-by: Jan Fajerski <jfajerski@suse.com>
(cherry picked from commit 7695d1ec539f54b7b7cea8d19925f0c320223f03)

py-common/deployment: drop workarounds for old batch

This commit drops two workarounds in drive groups for the old batch
behaviour. OSDs on LVs no longer need to be created individually by
create (related to functional testing) and ceph-volume batch now filters
unavailable drives itself.

Signed-off-by: Jan Fajerski <jfajerski@suse.com>
(cherry picked from commit 0f6f2b46a347c41a15f7ab2d532efb299f226698)

ceph-volume batch: add deprecation warning for auto behaviour

Signed-off-by: Jan Fajerski <jfajerski@suse.com>
(cherry picked from commit 1239f77c8871a45e21bd1e3e81be8bb8854f24da)

ceph-volume batch: add ceph.conf mocking to pass tests

Signed-off-by: Jan Fajerski <jfajerski@suse.com>
(cherry picked from commit eef9dc7a1da6d5dde0d1b02b71301c1d7b7926a9)

ceph-volume batch: use disk.Size for size args

Signed-off-by: Jan Fajerski <jfajerski@suse.com>
(cherry picked from commit 0bc7f7424cdd0a2c5f2cd777467814dbb3959fb4)

ceph-volume batch: Fix osd_ids passing and improve plan formatting

Signed-off-by: Jan Fajerski <jfajerski@suse.com>
(cherry picked from commit 2124aa57b355fb29dcfe13c643bc78b4011f15a6)

ceph-volume batch: track rel_size in percent, more tests

Signed-off-by: Jan Fajerski <jfajerski@suse.com>
(cherry picked from commit 2327e92abae74518d463a55ef4d42dbb816c9200)

ceph-volume batch: improve backwards compatibility

This restores legacy batch behavior and also adds some initial test and
adjusts existing tests to changes.

Signed-off-by: Jan Fajerski <jfajerski@suse.com>
(cherry picked from commit a23a02df02ec4a8f65df0864f3224fb311d25b11)

ceph-volume: batch - enable legacy auto behaviour

Signed-off-by: Jan Fajerski <jfajerski@suse.com>
(cherry picked from commit d32e0e4320b54302ab989f0a93b57a0404e2094b)

ceph-volume: batch - major refactor

This completely refactors the batch code in order to make use of the
create/prepare code path for creating OSDs instead of having a second
code path doing this. This not only eases the maintenance burden but
also adds various features and fixes bugs. This subcommand can now
handle LVs, replace OSDs, reuse VGs and has a better notion of
idempotency.

Signed-off-by: Jan Fajerski <jfajerski@suse.com>
(cherry picked from commit b0b797363fd66baa40eb54cf35dd6cfd11150be9)

ceph-volume: Device - available_lvm if 10 extents are free.

This changes the available_lvm notion to only require 10 free extents
instead of 5GB.

Signed-off-by: Jan Fajerski <jfajerski@suse.com>
(cherry picked from commit 6cb0841658ae45f23c2457a8f6a489457012d93e)

ceph-volume: Device - add vg_free property

This new property returns the free space in any VGs present. If no VGs
are on the device we project how much space a VG will have.

Signed-off-by: Jan Fajerski <jfajerski@suse.com>
(cherry picked from commit b34f130f30daeca034b6b2365cc5a832ba8faa56)

ceph-volume: prepare/create - size args as Size class

This add the disk.Size class as all size related argument types. We
often create this class form args like this anyway and it enables users
to pass not only bytes but also strings like 50G.

Signed-off-by: Jan Fajerski <jfajerski@suse.com>
(cherry picked from commit 2ff37321640a614fd965d7df6e20d8f9d1430fa3)

ceph-volume: disk.Size - add cast to bool

Signed-off-by: Jan Fajerski <jfajerski@suse.com>
(cherry picked from commit e7cdeab9dee2684c06b3543a482dd44a1db83c16)

ceph-volume: api/lvm - add VolumeGroup.free_percent property

Signed-off-by: Jan Fajerski <jfajerski@suse.com>
(cherry picked from commit bad54e97817dfef0e7dfcb834753cbb728c3de46)

ceph-volume: util.device -  add vg_free_percent property

Signed-off-by: Jan Fajerski <jfajerski@suse.com>
(cherry picked from commit f48d225454e3ec347952d85262aa03b71bfb9111)

ceph-volume: api/lvm - query LV units in bytes

Signed-off-by: Jan Fajerski <jfajerski@suse.com>
(cherry picked from commit de7d67603a135825d8b4e37a06dd4b6a12dbcf1c)

ceph-volume: lvm/common - refactor common arg specification

This makes it easier to create valid Namespace objects/arg lists when
programmatically calling create/prepare.

Signed-off-by: Jan Fajerski <jfajerski@suse.com>
(cherry picked from commit fa62a7bf5f926db17dd2d7685878c175fc23500e)

ceph-volume: batch: fix argument help message

Signed-off-by: Jan Fajerski <jfajerski@suse.com>
(cherry picked from commit 51ca694bbe1552fd52b0385c43aad7e28fc5626f)

Merge pull request #37337 from smithfarm/wip-47319-octopus

octopus: rgw: RGWObjVersionTracker tracks version over increments

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #37332 from callithea/wip-47569-octopus

octopus: mgr/dashboard: table detail rows overflow

Reviewed-by: Nizamudeen A <nia@redhat.com>
Reviewed-by: Tatjana Dehler <tdehler@suse.com>
Reviewed-by: Tiago Melo <tmelo@suse.com>

admin/build-doc: use "ceph" for prefix of functions defined by libcephfs

if some python module tries to `import cephfs`, it should not fail.

Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit b9e282f0e3eed2cacf1aac0853dc6607053607a3)

admin/build-doc: pass -rpath to linker

to make sure the dummy cythonized python binding is able to find the dummy C binding.

Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit 1183f267192681df2148d44e51ad56a531b5a45b)

mgr/cephadm: fixup expected extra ceph conf test result

fix test failure introduced by:
ff7e76348e5457fa6acb23545fcef56d6640c50a

```
E           AssertionError: expected call not found.
E           Expected: _run_cephadm('test', 'mon.test', 'deploy', ['--name', 'mon.test', '--reconfig', '--config-json', '-'], stdin='{"config": "\\n\\n[mon]\\nk=v\\n", "keyring": ""}')
E           Actual: _run_cephadm('test', 'mon.test', 'deploy',
['--name', 'mon.test', '--reconfig', '--config-json', '-'],
stdin='{"config": "\\n\\n[mon]\\nk=v\\n", "keyring": ""}', image='')
```

Signed-off-by: Michael Fritch <mfritch@suse.com>
(cherry picked from commit b3205210a18c7270b93551925d0dc91e050969cb)

cephadm: capadd and privileged are mutex

Signed-off-by: Joshua Schmid <jschmid@suse.de>
(cherry picked from commit 76e5020b106e14284f63bd7cee81822ad6b1fbf0)

docs: s/svc_id/osd_id

Fixes: https://tracker.ceph.com/issues/47402
Signed-off-by: Joshua Schmid <jschmid@suse.de>
(cherry picked from commit d7d1267686474ca9d16410ebe21ba89038785c56)

cephadm: support multiple mounts when running shell

This commit adds the multiple mounts support when running the
interactive shell.

ie:
```
--mount /foo /bar:/bar:z
```

Keeping default destination `/mnt` when no destination is passed for
backward compatibility. In the above example `/foo` will be mounted in
`/mnt/foo` and `/bar` in `/bar`

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
(cherry picked from commit 3ed3cff0d0ea2744dff816270d49539fd73c1691)

mgr/cephadm: Add extra-ceph-conf

Due to limitations of `config generate-minimal-conf`,
we need to provide a way for users to modify the
ceph.conf for the daemons.

This now provides a way to append config to
the daemons config. Like. e.g. `cluster_network`.

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
(cherry picked from commit ff7e76348e5457fa6acb23545fcef56d6640c50a)

mgr/cephadm: Refactor (to,from) datetime

`datetime.datetime.strptime(input, DATEFMT)` just
looks too ugly to have it everywhere.

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
(cherry picked from commit d65092f0fa616e623993a422176fb51ecf1245bf)

mgr/cephadm: Refactor `get_minimal_ceph_conf()`

Create a single source for the minimal ceph.conf

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
(cherry picked from commit fd4611836d3aa36882a18167d27af1b06715a0a6)

mgr/cephadm: CephadmDaemonSpec: add ceph_conf attribute

* `extra_config` should actually be called `extra_files`
* Special treatment for `ceph_conf` to ease refactorization

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
(cherry picked from commit 75aa97ee3fec290682afde9eb5a3254db2104ece)

mgr/cephadm: Fix race between host_add and _apply_all_specs

This mitigates a potential race, where new host was added *after*
``_refresh_host_daemons()`` was called, but *before*
``_apply_all_specs()`` was called. thus we end up with a hosts
where daemons might be running, but we have not yet detected them.

Imagine:

1. the initial MGR was deployed manually
2. `serve()` is called
3. `_refresh_hosts_and_daemons()` is called
4. `add_host()` is called with the initial host
5. `apply_all_specs()` now deploys an additional MGR on the initial host

Fixes: https://tracker.ceph.com/issues/45093
Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
(cherry picked from commit 72e3990382d65205de7871cc86b5063ed16732e5)

python-common: deprecate filter_matching_hosts

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
(cherry picked from commit 7b7aeafad1bb4e3f0818417008f5df616de8f90c)

mgr/cephadm: Simplified test_scheduling

Removed `get_hosts_func`

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
(cherry picked from commit 5750f32e9ddfdbe148b60d98f1196c90e134478a)

mgr/cephadm: Simplify HostAssignment arguments

`get_hosts_func` -> `hosts`, as `get_hosts_func` is
only called once in __init__

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
(cherry picked from commit 908bd443f335979f428833059161673b5330e745)

mgr/cephadm: Scheduler: fetch hosts only once

Otherwise we might be prone to races, e.g.
a new hosts gets added in between calls to
`get_hosts_func`

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
(cherry picked from commit 3b841fc7fa5c13dbc212053aeb8deda012a3de0d)

cephadm: Introduce 'container' specification to deploy custom containers

Fixes: https://tracker.ceph.com/issues/46666
Signed-off-by: Volker Theile <vtheile@suse.com>
(cherry picked from commit 7219377cfcef7ccec89b34e934388b8753608e15)

Conflicts:
src/pybind/mgr/dashboard/frontend/src/app/ceph/cluster/services/service-form/service-form.component.ts

mgr/cephadm: fix RemoveUtil.load_from_store()

* Run `json.loads` only once
* `nodename` -> `hostname`
* Remove network calls from `repr(OSD(...)`

Fixes: https://tracker.ceph.com/issues/47438
Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
(cherry picked from commit 0bcd29a247f4a01ba69c6d43db668ef3dcc1610d)

qa: spawn MDS daemons before creating fs

This avoids unnecessary MDS_ALL_DOWN messages because the MDS daemons
have not yet been spawned.

Fixes: https://tracker.ceph.com/issues/47518
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit 61db7a9c2e47169017f92df05c843f5b8d0526ee)

doc/cephadm: use appropriate directive for formatting codeblocks

Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit 967104103c6c611b122e3e37227a04486cd6d2b1)

mgr/rook: Pass pod namespace to list_namespaced_pod()

As list_namespaced_pod method requires pod namespace instead of cluster name.

Fixes: https://tracker.ceph.com/issues/47511
Signed-off-by: Varsha Rao <varao@redhat.com>
(cherry picked from commit a6c6c114afa9cdc89508ae90bbe837421914d342)

mgr/rook: fix listing daemons

Pass the parameters to the `_list_daemons` method explicitly to avoid missing
parameters.

Fixes: https://tracker.ceph.com/issues/47387
Signed-off-by: Kiefer Chang <kiefer.chang@suse.com>
(cherry picked from commit 8098ca765fadce18c3738df04f9eb5ddf786c113)

cephadm: filter dangling images in get_last_local_ceph_image()

if `podman images --filter label=ceph=True --format {{.Repository}} {{.Tag}}`
returns something like following:

```
<none>                <none>
docker.io/ceph/ceph   v15
```

it will end up throwing an error:

```
    r = args.func()
  File "./cephadm", line 1147, in _infer_fsid
    return func()
  File "./cephadm", line 1184, in _infer_config
    return func()
  File "./cephadm", line 1230, in _infer_image
    return func()
  File "./cephadm", line 3283, in command_shell
    make_log_dir(args.fsid)
  File "./cephadm", line 1325, in make_log_dir
    uid, gid = extract_uid_gid()
  File "./cephadm", line 1913, in extract_uid_gid
    raise RuntimeError('uid/gid not found')
RuntimeError: uid/gid not found
```

adding `--filter dangling=false` can help to avoid this.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
(cherry picked from commit 01880cebcd64bc83ea722217e2925ab2b6304c65)

doc/mgr: fix outdated `orch apply` docs.

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
(cherry picked from commit f3bd8f2569a8315755fe89fb16199eacc9bcb6c7)

mgr/cephadm: CephadmService.create -> CephadmService.prepare_create

Refactored this to simplify the control flow. It became obvious that

* `CephadmService.make_daemon_spec`
* `CephadmService.prepare_create`
* `CephadmService.generate_config`

are basically the same thing. They're all called directly
before the deployment of a daemon. All of them should be
unified. This PR makes this refactorization simpler

By renaming `create` to `prepare_create`, we make `create`
no longer being the entrypoint to call
`create_daemon`. Thus all the functions above
return some data structures.

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
(cherry picked from commit 245cf50132e1731982d19e1114116e4150b9ea97)

python-common/deployment: respect existing osds

Fixes: https://tracker.ceph.com/issues/44888
Signed-off-by: Joshua Schmid <jschmid@suse.de>
(cherry picked from commit 6bfc4589f55c76d50cc87f75c4018860167f7708)

mgr/cephadm: allow to map existing daemons to osdspecs

Fixes: https://tracker.ceph.com/issues/44888
Signed-off-by: Joshua Schmid <jschmid@suse.de>
(cherry picked from commit 6a374c583c2963a0bfc28fd6b54f5f91b6f8d26f)

mgr/cephadm: allow iscsi to be named by teuthology

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
(cherry picked from commit 1bd86d33166246d0b49997aabe6d146be2171c1a)

qa/cephadm: Also fetch iscsi logs

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
(cherry picked from commit 130c7a438ec4201638bb182e026cc359d1a4cbd9)

qa/cephadm: Add iSCSI

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
(cherry picked from commit 430f2c51320ed0092c17273ea12e6b527ae735ed)

mgr/cephadm: merge update_keyring_caps -> create_keyring

keyring is removed during daemon remove, which obsoletes the need to
update the keyring caps after get-or-create

Signed-off-by: Michael Fritch <mfritch@suse.com>
(cherry picked from commit eea25b5ac722422369653daebb669749b9cf1bba)