cephadm: return repo_digests (plural) in pull/inspect output

When we inspect a (pulled) image, return all of the repo digests.  Update
the mgr/cephadm code accordingly to match.

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit e9333972c1b84ebdfe43c8f686d74d4c859eedf1)

mgr/cephadm: include container_image_digests in inventory

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit 9f61c94ae7d56f0b12d753642e031fcd5cd9d145)

cephadm: include image_digests list in 'ls' output

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit 0826c45e0cb5d60fcf8cd71cd14edd34a6997cd4)

vstart.sh: only extract first container digest

Otherwise the container_image value includes the same hash twice, separated
by a newline, causing all sorts of confusion.

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit 6741082eb3e03db608899a8a5fef44fb05d356b7)

mgr/cephadm: move release -> major translation to helper

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit 43ac324c3817b378beb9fe12437565ddbecb5370)

mgr/cephadm/upgrade: tolerate old upgrade_state.target_versoin

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit a3c9df91975708f42cd75266e840f0b0b019e7c7)

mgr/cephadm/upgrade: set require-osd-release when done with OSDs

After we've upgraded all of the osds, complete the major version upgrade
for core RADOS by setting the OSD cluster min.

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit 8abd45c4c95a58ae67c29905a3c4f6c77a7383e9)

mgr: add lookup_release_name(int) to mgr interface

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit ceac0861434ae761d8866889980dcf95d1f21a93)

mgr/cephadm: verify container image version after we pull it

Apply the version checks to make sure this upgrade/downgrade is possible.

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit 5147d0f9305d97bf251671da80200f92536d4862)

mgr/cephadm: only save version portion of version string

We don't want to 'ceph version ' prefix or sha1 suffix.  This matches us
up with 'orch upgrade start'

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit dcaf514bbf1ac19c0ea9f5bfa724bca76ff2d126)

cephadm: fix 'inspect' and 'pull'

With podman 2.2.1 I would get

ERRO[0000] Error printing inspect output: template: all inspect:1: function "json" not defined

Removing the 'json' portion of the query resolves it.

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit d31bed79411ca493ec48eeed4e9cbb7ad92295c3)

mgr/cephadm/upgrade: implement N-2 version checks on upgrade start

- Prevent major upgrades that span > 2 releases
- Prevent downgrades to an rc or dev release

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit 95dd015bce4153e3c47adac89936a300a0dd3152)

cephadm:updates following bootstrap code clean up

Patch to align to new(ish) bootstrap code changes

Signed-off-by: Paul Cuzner <pcuzner@redhat.com>
(cherry picked from commit 804f34e475be699807f565d48ecd13320d246782)

cephadm: updated doc text

The phrase "recommended"  has been removed.

Signed-off-by: Paul Cuzner <pcuzner@redhat.com>
(cherry picked from commit af00a333224df4e748e662efc2bd48593af25d26)

cephadm: unit test updates for cluster-network handling

Tests updated to handle comma separated subnet lists

Signed-off-by: Paul Cuzner <pcuzner@redhat.com>
(cherry picked from commit bbe4fd83c556efb35c16f6da0bf2b00e11a2b3c9)

cephadm: updated handling of cluster_network parameter

Parameter now supports a comma separated list of subnets and
detects the ipv4 or ipv6 setting on the cluster network to enable
osds to bind correctly.

Signed-off-by: Paul Cuzner <pcuzner@redhat.com>
(cherry picked from commit abd9287db0e4f4f7873864119f5ce62519af1d48)

cephadm: install doc updated to include cluster-network parameter

Install guide updated to include a description of the --cluster-network
parameter. The text also links to the complete definition for cluster-network
on the rados/configuration/network-config-ref page.

Signed-off-by: Paul Cuzner <pcuzner@redhat.com>
(cherry picked from commit 509f171a08a82a3bc132739b43500064266b8e9e)

cephadm: unit test updates for new subnet function

Adds tests to validate the check_subnet function will work as
expected with various inputs

Signed-off-by: Paul Cuzner <pcuzner@redhat.com>
(cherry picked from commit 3f352c8af4ddedd6ae7fc69045e42d07dad261c2)

cephadm: Add cluster network parameter to bootstrap cmd

Adds a --cluster-network parameter to the bootstrap cmd to
set the internal cluster network.

Signed-off-by: Paul Cuzner <pcuzner@redhat.com>
(cherry picked from commit c391ccb97f1bc81bf9c45e9b313078535b6ad4c5)

doc/mgr/orchestrator: Unify the content of command and yaml

Signed-off-by: Mitsumasa KONDO <kondo.mitsumasa@gmail.com>
(cherry picked from commit 53639c2b01c80806e8271a623ad21b40d89a6d73)

cephadm: use `apt-get` for package install/update

avoids errors during prepare-host:
```
apt: stderr WARNING: apt does not have a stable CLI interface. Use with caution in scripts.
```

Fixes: https://tracker.ceph.com/issues/49032
Signed-off-by: Michael Fritch <mfritch@suse.com>
(cherry picked from commit fa4706039cfece36815df46cd4452dc25448c340)

cephadm: fix get_data_dir calls for ha-rgw daemons

calls need to use the context's data dir rather than daemon type
puts these calls in line with the other daemons

Signed-off-by: Adam King <adking@redhat.com>
(cherry picked from commit 10214317aee39ecca5e7744b2ee7f528a4b80cb5)

cephadm: add fsid if --name is not specified

address an regression introduced by
9e81a5dfc5abdd8caeecf42faa17111fc309b8d3

Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit 556b34a6679fe617743a46f5a0cf78fcb46a39b2)

cephadm: add CephadmContext.__contains__()

more pythonic this way.

Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit b9fb0d9f3de9960ecdb8901cadcf5a735b3bc4d2)

mgr/orchestrator: remove 'host' arg for 'orch ls'

The host arg is unused, and also missing from the documentation.

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit 38b3ca2a736995baf962997995f1334786c6e2cd)

Merge pull request #39309 from liewegas/pr-39150-pacific

pacific: qa/suites/rados/cephadm/thrash: add cephadm thrashing test

Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge pull request #39181 from liewegas/pr-39039-pacific

pacific: quincy feature, custom git_url for cephadm_branch

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>

Merge pull request #39190 from liewegas/pr-39097-pacific

pacific: qa/workunits/cephadm/test_cephadm: fixes and cleanup

Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>

Merge pull request #39252 from sebastian-philipp/pacific-cephadm-daemon-base-devel-pacific

pacific: cephadm: use ceph/daemon-base:latest-pacific-devel image

Reviewed-by: Dimitri Savineau <dsavinea@redhat.com>
Reviewed-by: Juan Miguel Olmo Martínez <jolmomar@redhat.com>

Merge pull request #39038 from kotreshhr/fix-test-idempotency-test

pacific: pybind/ceph_volume_client: Fix test_idempotency test failure

Reviewed-by: Ramana Raja <rraja@redhat.com>

Merge pull request #39109 from kotreshhr/wip-49027-pacific

pacific: mgr/volumes: Evict clients based on auth-IDs and subvolume path

Reviewed-by: Ramana Raja <rraja@redhat.com>

Merge pull request #39284 from neha-ojha/wip-49145-pacific

pacific: osd: fix the scrubber behavior on multiple preemption attempts

Reviewed-by: Ronen Friedman <rfriedma@redhat.com>

Merge pull request #39264 from neha-ojha/wip-49134-pacific

pacific: qa/*/test_envlibrados_for_rocksdb: use osd_client_message_cap to prevent slow requests

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #39257 from ifed01/wip-ifed-fsck-spanning-blobs-pac

pacific: os/bluestore: detect and fix "zombie" spanning blobs using fsck.

Reviewed-by: Adam Kupczyk <akupczyk@redhat.com>

Merge pull request #39247 from kamoltat/wip-ksirivad-pacific-backports

pacific: mgr/pg_autoscaler: avoid scale-down until there is pressure

Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge pull request #39237 from liewegas/pr-39206-pacific

pacific: mgr/MetricTypes: condition encoding on feature bits

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #39102 from k0ste/wip-49021-pacific

pacific: mgr/dashboard: trigger alert if some nodes have a MTU different than the median value

Reviewed-by: Aashish Sharma <aasharma@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>

Merge pull request #39026 from rhcs-dashboard/wip-48962-pacific

pacific: mgr/dashboard: Fix for incorrect validation in rgw user form

Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>

cephadm: use ceph/daemon-base:latest-pacific-devel image

Fixes: https://tracker.ceph.com/issues/49041
Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>

Merge pull request #39050 from vshankar/wip-mgr-mirroring-interface-pacific

pacific: pybind/mgr/mirroring: directory snapshot mirror support

Merge pull request #39045 from smithfarm/wip-48970-pacific

pacific: ocf: add support for mapping images within an RBD namespace

Reviewed-by: Mykola Golub <mgolub@suse.com>
Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #39177 from cbodley/wip-qa-rgw-pacific-branches

pacific: qa: rgw tests target ceph-pacific branches instead of ceph-master

Reviewed-by: Yuri Weinstein <yweinste@redhat.com>

qa/suites/rados/cephadm/thrash: add cephadm thrashing test

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit cc6412c99127c05dda639ff753cb1102bdb995bb)

doc: cleanup cephfs-mirroring document

Remove references to in-progress feature, remove internal interfaces
section and minor cleanups.

Signed-off-by: Venky Shankar <vshankar@redhat.com>
(cherry picked from commit bf19651d6ad0a11e939c5d9f4bb744e90ad8a9d7)

test: cephfs-mirror teuthology task and test yamls

Signed-off-by: Venky Shankar <vshankar@redhat.com>
(cherry picked from commit 3478b2a06204bc58dcefae7f1d1f2497dd08fccf)

test: add tests for mirroring module w/ daemon verification

Signed-off-by: Venky Shankar <vshankar@redhat.com>
(cherry picked from commit f81e8f1e8831d6e79f24531ec73483343e3e2b9f)

test: optionally create a backup filesystem on startup

Also filter out client-id's starting with "mirror" when
cleaning leftover auth-ids since teuthology would be
configured to create client.mirror and client.mirror_remote
clients before executing mirroring tests.

Signed-off-by: Venky Shankar <vshankar@redhat.com>
(cherry picked from commit 8334bea5a64bb4b863e3ff1a6b46807bf03f3270)

pybind/mgr/mirroring: interface to mirror CephFS directory snapshots

Signed-off-by: Venky Shankar <vshankar@redhat.com>
(cherry picked from commit b7acf7fc77d8c7865935643c3eb484be2f9999a1)

pybind/mgr/mgr_util: make RTimer calss reusable

Signed-off-by: Venky Shankar <vshankar@redhat.com>
(cherry picked from commit 296e879009a16130138c9f7aa09bfdbf29ff9847)

osd: fix the scrubber behavior on multiple preemption attempts

Latest scrub code creates a time window in which a specific scrub
is marked as "preempted", but future preemptions are prohibited.
Write operations handled are then blocked but not restarted on time.

Fixes: https://tracker.ceph.com/issues/48793
Signed-off-by: Ronen Friedman <rfriedma@redhat.com>
(cherry picked from commit 4cdb293e08f0ea066511c9e33f48635d20d58dd9)

osd: replacing list::size() with list::empty() in some scrub code

Signed-off-by: Ronen Friedman <rfriedma@redhat.com>
(cherry picked from commit f100f2f7f4e332ac784db44a35bf23cf53a43dff)

Merge pull request #39280 from neha-ojha/wip-49139-pacific

pacific: qa/suites/rados/perf: pin to 18.04

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

qa/suites/perf-basic: pin to 18.04

This will have same problems as https://tracker.ceph.com/issues/49139

Signed-off-by: Neha Ojha <nojha@redhat.com>
(cherry picked from commit e5895dd8012f646ae9d67b9d1b331e7c06fe89d3)

qa/suites/rados/perf: pin to 18.04

Caused by a07ca4c5c0f04c43c71231a898c2a89e99a80606

Fixes: https://tracker.ceph.com/issues/49139
Signed-off-by: Neha Ojha <nojha@redhat.com>
(cherry picked from commit 6a03b4cec9be39b2c3217e5f28372e1db3999781)

Merge pull request #39165 from sebastian-philipp/pacific-backport-39035-38935-39028-39087-38998-38854-38982-39101-39043-39113-

pacific: cephadm: Batch backport January (2)

Reviewed-by: Jan Fajerski <jfajerski@suse.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Laura Paduano <lpaduano@suse.com>
Reviewed-by: Juan Miguel Olmo Martínez <jolmomar@redhat.com>

qa/*/test_envlibrados_for_rocksdb: use osd_client_message_cap to prevent slow requests

EnvLibradosMutipoolTest.DBBulkLoadKeysInRandomOrder can overload OSDs and cause
heartbeat timeouts. Tests in test_envlibrados_for_rocksdb also generate slow
requests on OSDs. Use osd_client_message_cap to prevent this.
Since this option is disabled by default, this may be a good way to exercise it.

Fixes: https://tracker.ceph.com/issues/49064
Signed-off-by: Neha Ojha <nojha@redhat.com>
(cherry picked from commit f02cd4d8b0c9f1566462b7b950f121a44efc1d06)

Merge pull request #39233 from idryomov/wip-unbreak-rbd-task-pacific

pacific: qa/tasks/rbd: fix regressions introduced with rbd encryption support

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

mgr/pg_autoscaler: avoid scale-down until there is pressure

The autoscaler will start out with scaling each
pools to have a full complements of pgs from the start
and will only decrease it when pools need more due to
increased usage.

Introduced a unit test that tests only the
function get_final_pg_target_and_ratio() which
deals with the distrubtion of pgs amongst the
pools

Edited workunit script to reflect the change
of how pgs are calculated and distrubted.

Signed-off-by: Kamoltat <ksirivad@redhat.com>
(cherry picked from commit daeb6f6ac0c8f77ae07147f9d1e2ed18d6d8e4cc)

mgr/MetricTypes: condition encoding on feature bits

Do not encode new metric|config types for older peers.

The combination of boost::optional and boost::variant make this super
awkward to condition on the features. :(

Fixes: https://tracker.ceph.com/issues/49069
Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit 5cf13459813228a2b8d78bba2d28a10528490c1f)

qa/tasks/rbd: don't wait for krbd symlink to be created/removed

Commit 3754c665a11e (":qa/tasks/rbd: test qemu on top of rbd
encryption") broke dev_create() for krbd by messing up the "wait
for the symlink to be created by udev" loop.  The rbd tool has been
synchronizing with udev internally since 2014, so rather than fixing
let's just drop it.

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
(cherry picked from commit d89613f52dee83c308a2b169ba1c2869c16ac9d7)

qa/tasks/rbd: dev_create() expects a properties dict

In order to add encryption_format property, commit 3754c665a11e
(":qa/tasks/rbd: test qemu on top of rbd encryption") changed the
dev_create() contract.  It now expects a properties dict, but the
main task routine wasn't updated and still passes role_images.

On top of that, role_images itself got broken.  It is supposed
to be a mapping from role to a corresponding image name, not to
a dict with an image_name key.  This affected generic_mount().

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
(cherry picked from commit 0357c0215b2fbbf7fa801331df1f92b101ad9d5e)

os/bluestore: detect and fix "zombie" spanning blobs using fsck.

Signed-off-by: Igor Fedotov <ifedotov@suse.com>
(cherry picked from commit 18d120dc2c15c0cf8d3b27ddba3c7f2057856555)

qa/workunits/cephadm/test_cephadm: fix argument

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit 8c625c077b067380acbe005c2db818322011f56e)

qa/workunits/cephadm/test_cephadm: show exporter output

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit a78dab9e53ea19488a7c8719852a46311841ae75)

qa/workunits/cephadm/test_cephadm: don't bother pulling octopus

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit f394eafd647fa6117aa93379464a089b8b5a2261)

qa/workunits/cephadm/test_cephadm: behave on terminal

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit c82770f77c77a14334dab668a1e0e2c992974849)

qa/workunits/cepham/test_cephadm: pull only recent images, from quay.ceph.io

No reason to pull nautilus/mimic, which don't even support cephadm.

This test will mostly run in the sepia lab, so pull from there.

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit 1dccdd2fa182dcf666829a54bc6bc599a1f9da69)

qa/workunits/cephadm/test_cephadm: don't respawn for python versions

We aren't supporting python2 anymore.

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit 90676eb9b1a6e1593c37d158b8592e94ae30d655)

qa/tasks/cephadm: allow custom git_url for cephadm_branch pull

Normally the git_url is git://git.ceph.com/ceph-ci.git, which mirrors
upstream ceph-ci.git.  However, the release branches aren't present there.
Allow a custom git_url so we can pull these from the main ceph.git.

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit 5842058d128d48495285242b46e41eba18916084)

# Conflicts:
# qa/suites/upgrade/pacific-x/parallel/1-tasks.yaml

qa: rgw tests target ceph-pacific branches instead of ceph-master

s3test and ragweed tests now use ceph-pacific branch. this commit
diverges from master, so is applied directly to the pacific branch

Signed-off-by: Casey Bodley <cbodley@redhat.com>

cmake: install rook-client-python using ExternalProject

so we don't need to rerun the generate_rook_ceph_client.sh script
everytime when building the script. cmake creates a stamp file for
tracking the dependencies and the time of modification of dependencies.

Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit 7fb44404bb8bed2a60177ba3948b318f14d28521)

python-common: fix test_datetime_to_str_2 on non-UTC hosts

The old test parsed to a datetime without a tz, which was interpreted as
the local time zone when rendering back to a string.  Specify that it's a
UTC datetime so that behavior is consistent regardless of the test host
timezone.

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit 78aca4db249c409d0cd5a24bfae81e55cf930bc3)

cephadm:add missing kernel_security property

The propery decorator had gone missing which meant
the dump of host facts was missing the kernel security
(LSM) settings. This patch just adds the @property
decorator back

Signed-off-by: Paul Cuzner <pcuzner@redhat.com>
(cherry picked from commit 1bc953cfbfe015d3eccf9ead002b9eb0253a6825)

cephadm: refactor call() using asyncio.asyncio.StreamReader

simpler this way, also fix a couple issues:

* create a child watcher explicitly, see
  https://bugs.python.org/issue35621
* use StringIO for collecting outputs for better performance,
  instead of appending the lines to an existing str
* catch ValueError when reading from the stream reader,
  because StreamReader.readline() could raise ValueError when
  it reaches the buffer limit while looking for a separator.
  in this case, we should try again, in hope that the spawned
  process can feed the reader with more data which contains
  the separator (i.e., b'\n').
* backport ThreadedChildWatcher from Python 3.8 so we can
  run create_subprocess_exec() in non-main threads.

Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit 30070be24860c2dfc0d27c79b0958d6c09316d87)

mgr/cephadm: Fix wrong argument type to HandleCommandResult

the argument must be a string

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
(cherry picked from commit 677b3d1b0f925cd36fccbdf9fb12251b77afd030)

cephadm: Fix iscsi client caps (allow mgr <service status> calls)

iSCSI daemons need to execute <ceph service status> command

Fixes: https://tracker.ceph.com/issues/48925
Signed-off-by: Juan Miguel Olmo Martínez <jolmomar@redhat.com>
(cherry picked from commit 9b9934f75b648a9ee01848710a6f7150a371e26e)

mgr/cephadm: force flag for ok-to-stop and ok-to-stop for monitoring stack

Daemons that could cause data loss when stopped will always block.

Daemons that will only cause loss in availability should block but have
a workaround in the form of a force flag if the user is okay with the service
being down.

Also implements ok-to-stop for monitoring stack daemons that uses this system
of blocking on availability loss unless force flag is provided

Signed-off-by: Adam King <adking@redhat.com>
Signed-off-by: Daniel-Pivonka <dpivonka@redhat.com>
(cherry picked from commit aeeffb07f1bb226b64a362de4eecd7d63fc0b0d1)

mgr/cephadm: minor pep8 fixes

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
(cherry picked from commit 4ca11ae50d734c1d059d88b8b14a3318f11f4eea)

mgr/cephadm: Properly handle JSON Decode error

Fixes 6d759fb5deac0c52b3c738a2e695738228749420

I.e. don't use `out`, until it is acutally defined

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
(cherry picked from commit eb1a40c3ea8e19259d8ab68a6eeb16d27e4cdbda)

qa/suites/rados/cephadm: drop 20.04 podman

This version is also affected

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit dccd5861733144514cb28b3325f2d5ed6c56a67e)

qa/suites/rados/cephadm: stop testing 18.04 + podman

The current bionice version triggers a podman/conmon bug that
truncates output, affecting both cephadm bootstrap when 'mgr dump' is
large, and teuthology 'pg dump' when it is large.

See https://tracker.ceph.com/issues/48993

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit e5e4710990e19625a074f6ad9ab13003fbaccba1)

cephadm: retry for 30s instead of 10s

No real reason not to wait longer, and I'm worried about things not coming
up quickly enough on slower or loady machines.

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit 963f54aebf151ea4624063d930ab85f0a2b1fa79)

cephadm: try 'mgr stat' before 'mgr dump' to get the epoch

This is less noise in the debug log.  (Also, there seems to be a problem
when the mgr stat output gets too big.)

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit c1cee0a9526b21c848e8eff2bed585ee856e4497)

mon: add 'mgr stat' command

This is analogous to 'osd stat' and 'mds stat'.

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit bb702eda199cf2def81e8277d0a222daafe9df28)

pybind/mgr/cephadm: return -ENOENT if self.ssh_user is None

there is chance that self.ssh_user is None when handling "cephadm
get-user" command, so return -ENOENT and error messages in that case.

Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit 36dde8861a1b8ae0aa53926c773e21bfa065fb07)

pybind/mgr/dashboard: use _get_localized_key for composing store key

Just for keeping this 'encapsulated', there's _get_localized_key helper
in mgr_module (there's also a set_localized_store but it only deals with
the active manager)

As suggested by Ernesto Puerta <epuertat@redhat.com>

Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit 34e96c55397da7029254864ebf7a3d88563e22c9)

pybind/mgr/prometheus: add assert to appease mypy

self.get_osd_perf_counters() might return None, add assert() to assure
mypy that it does not return None here.

Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit 3ca955c4d5f55c9c15797199b991dd0e81095386)

pybind/mgr/cephadm/serve: add assert to appease mypy

Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit 2a5a50f3bace6b853fe2feff85a641ade91825e1)

pybind/mgr/cephadm/module: assure self.ssh_user is not None

to appease mypy

Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit 2fd0a03192bbbe15fb5fb7095f67e1a373661c89)

pybind/mgr/cephadm/inventory: cast variable to expected type

to silence mypy warnings like:

cephadm/module.py:55: note: In module imported here,
cephadm/__init__.py:6: note: ... from here:
cephadm/inventory.py: note: In member "load" of class "SpecStore":
cephadm/inventory.py:130: error: Invalid index type "str" for "str"; expected type "Union[int, slice]"
cephadm/inventory.py:131: error: Invalid index type "str" for "str"; expected type "Union[int, slice]"

Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit cb3a50cd3626d6a67beb39d2c9dd897f43379d70)

tools/setup-virtualenv.sh: do not pass --use-feature=2020-resolver to pip

pip use the new resolver by default now. so no need to pass this option
to it anymore.

This reverts commit fa9e2bfd4b3648f08ed3a88ce737d432ab97cce1.

Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit cb2de05a39b28342fdf44688cd0d1c3e424a02ff)

pybind/mgr/mgr_module: annotate variable type

to silence following mypy error:

cephadm/module.py:1155: error: Incompatible types in assignment
(expression has type "None", variable has type "str")

Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit 00f5a80201923c85c50f8e8bb64e3e82458fd932)

pybind/mgr/cephadm: rename variables to avoid name reuse

to applease mypy, otherwise, for instance, "err" would be reused for
holding the return value of self.mgr.check_mon_command(...)

Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit 53415832cdbef94d2f3777e8f3abb8560b15323e)

pybind/mgr: annotate OSDMap and friends

Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit eb6603f1b52eda73d3b1ef92b03833c61ddd2223)

pybind/mgr: correct annotation for BasePyOSDMap._apply_incremental()

it's incorrect. as the underlying C implementation,
"osdmap_apply_incremental()", actually expects an instance of
`BasePyOSDMapIncrementalType`. which is mapped to
`BasePyOSDMapIncremental` in Python. and this class is wrapped using
`OSDMapIncremental` in mgr_module.py.

Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit 7b2dd7eba4422338acd5e6d8e24f88114f8694e6)

pybind/mgr: define HandleCommandResult using NamedTuple

simpler this way, so we don't need to define __new__

Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit d139166ea7514d113a41c28dae59184e3d2101d0)

pybind/mgr: use numeric values of logging level for index

less magic values in code this way

Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit ab0243b31946e058ecb41a2a04aedd8459b89bbb)

mgr/cephadm: use dict instead of defaultdict to silence mypy warning

otherwise we have:

1: cephadm/module.py:30: note: In module imported here:
1: cephadm/serve.py: note: In member "_check_for_strays" of class "CephadmServe":
1: cephadm/serve.py:395: error: Argument "default" to "get_metadata" of "MgrModule" has incompatible type "defaultdict[str, None]"; expected "Optional[Dict[str, str]]"

Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit 6c3d8174358b30f3c9eef1b9342df437749d75fd)

mgr/rook: handle the case where given osd is not found

Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit 5272748e179f67c8d5464d3b38e96eb8829b6b00)

mgr/dashboard: pass str as the 2nd param of set_store()

MgrModule.set_store() only accepts str or None as its second
parameter.

Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit 2d20390b17649beae72a0be1eb6da286ed9788ae)