qa/suites/krbd: add msgr2 modes to most subsuites

basic, rbd and rbd-nomount subsuites are expanded to run with each
of ms_mode=legacy, ms_mode=crc and ms_mode=secure.  This increases
the total number of jobs in the suite from 100 to 220.

fsx, singleton and thrash subsuites choose ms_mode at random (from
the above plus ms_mode=prefer-crc).

unmap and wac subsuites remain msgr1-only.

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
(cherry picked from commit 65948736a41f424d8152b208d013419f8d6038a4)

doc: deprecate [no]cephx_require_signatures map options

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
(cherry picked from commit fd5f5722a29964bb33b305a381bcf9a48cdcbb47)

krbd: add support for msgr2

Recognize ms_mode map option and filter initial monitor addresses
accordingly: if ms_mode is not given or ms_mode=legacy, discard v2
addresses, otherwise discard v1 addresses.

Note that nothing was discarded (i.e. v2 addresses were passed to
the kernel) previously.  The intent was to preserve that behaviour
in case ms_mode is not given, allowing to change the kernel default
in the future.  However, it turns out that mount.ceph helper has
been misguidedly discarding v2 addresses since commit eae01275134e
("mount.ceph: fork a child to get info from local configuration"),
so that ship has sailed.

Fixes: https://tracker.ceph.com/issues/48976
Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
(cherry picked from commit 08f714964b7fe5024504818f01328a41acc24965)

Conflicts:
src/tools/rbd/action/Kernel.cc [ commit 34f539d8af33 ("rbd:
  delay parsing of default kernel map options") not in octopus ]

Merge pull request #38336 from votdev/wip-48398-octopus

octopus: mgr/dashboard: display placement column in service table

Reviewed-by: Laura Paduano <lpaduano@suse.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
Reviewed-by: Tatjana Dehler <tdehler@suse.com>

Merge pull request #39018 from sebastian-philipp/octopus-backport-38766

octopus: cephadm: make "ceph orch {restart|...}" asynchronous

Reviewed-by: Michael Fritch <mfritch@suse.com>

Merge pull request #39019 from sebastian-philipp/octopus-backport-38815

octopus: mgr/cephadm: lock multithreaded access to OSDRemovalQueue

Reviewed-by: Michael Fritch <mfritch@suse.com>

Merge pull request #39020 from sebastian-philipp/octopus-backport-38904

octopus: cephadm: Don't make sysctl spam the log file

Reviewed-by: Michael Fritch <mfritch@suse.com>

Merge pull request #39071 from smithfarm/wip-49003-octopus

octopus: test/run-cli-tests: use cram from github

Reviewed-by: Kefu Chai <kchai@redhat.com>

cephadm: Don't make sysctl spam the log file

`sysctl -a` really spams the log file in Teuthology

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
(cherry picked from commit 70722a2efb5207f7d6fc7e8ecf4aab71e2a1a39d)

Conflicts:
  src/cephadm/cephadm

test/downloads: drop cram-0.5 tar ball

since we install cram from github, drop the tarball from repo

Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit d8cbe4eec11ef7f9fbbb61666b9ec7a262c1a85c)

test/run-cli-tests: use cram from github

to address the failure of

  WARNING: Built wheel for cram is invalid: Metadata 1.2 mandates PEP 440 version, but '0.5.0ceph.2011-01-14' is not
Failed to build cram

Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit c74e614813d69ef38139e5e745873dc2a31b1c59)

mgr/dashboard: display placement column in service table

Fixes: https://tracker.ceph.com/issues/44404
Signed-off-by: Volker Theile <vtheile@suse.com>
(cherry picked from commit 1c722aa89ec1efbf5cc76ea968a1f9a725a86e57)

Conflicts:
src/pybind/mgr/dashboard/frontend/src/app/ceph/cluster/services/services.component.ts
src/pybind/mgr/dashboard/frontend/src/app/ceph/cluster/services/placement.pipe.ts
Both files need to be adapted to replaced $localize with i18n.

Merge pull request #38761 from p-se/wip-48607-octopus

octopus: mgr/dashboard: enable different URL for users of browser to Grafana

Reviewed-by: Tatjana Dehler <tdehler@suse.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
Reviewed-by: Volker Theile <vtheile@suse.com>

Merge pull request #38337 from k0ste/wip-48399-octopus

octopus: mgr/balancer: fix available pgs sent to calc_pg_upmaps

Reviewed-by: Deepika Upadhyay <dupadhya@redhat.com>

Merge pull request #38943 from smithfarm/wip-48889-octopus

octopus: do_cmake.sh: use python-3.9 with fedora version 33

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #38888 from votdev/wip-46963-octopus

octopus: mgr/dashboard: Create Ceph services via Orchestrator by using ServiceSpec

Reviewed-by: Tatjana Dehler <tdehler@suse.com>
Reviewed-by: Laura Paduano <lpaduano@suse.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>

Merge pull request #38847 from smithfarm/wip-48828-octopus

octopus: cmake: add empty RPATH to ceph-diff-sorted

Reviewed-by: J. Eric Ivancich <ivancich@redhat.com>

Merge pull request #38810 from rhcs-dashboard/wip-48794-octopus

octopus: mgr/dashboard: Temporary User Lockout if 10 Invalid Login attempts

Reviewed-by: Volker Theile <vtheile@suse.com>
Reviewed-by: Tatjana Dehler <tdehler@suse.com>
Reviewed-by: Laura Paduano <lpaduano@suse.com>
Reviewed-by: Aashish Sharma <aasharma@redhat.com>

Merge pull request #38801 from callithea/wip-48714-octopus

octopus: mgr/ActivePyModules.cc: always release GIL before attempting to acquire a lock

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #38755 from smithfarm/wip-48737-octopus

octopus: rpm: require smartmontools on SUSE

Reviewed-by: Volker Theile <vtheile@suse.com>

Merge pull request #38595 from callithea/wip-48557-octopus

octopus: mgr/restful: fix TypeError occurring in _gather_osds()

Reviewed-by: Kefu Chai <kchai@redhat.com>

mgr/cephadm: lock multithreaded access to OSDRemovalQueue

Since the set can be changed also from the CLI thread

Fixes: https://tracker.ceph.com/issues/47700
Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
(cherry picked from commit db7c6a56125a3a36fa50904557280d69829c69d6)

mgr/cephadm: OSDRemovalQueue: remove some trivial wrappers

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
(cherry picked from commit 1f39532e0398c85ccd0c238feb8e3a947758b54c)

mgr/cephadm: move RemoveUtil into OSDRemovalQueue

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
(cherry picked from commit df16b86c7b6e1c11d35de4290a9e164962b4d057)

mgr/cephadm: move process_removal_queue into OSDRemovalQueue

`process_removal_queue` belongs to OSDRemovalQueue
instead of RemoveUtil

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
(cherry picked from commit af52ba47a16703aa1aadbd59d84e8876bdad2f17)

mgr/cephadm: make OSDRemovalQueue not inherent from set

Instead, let's make it a member.

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
(cherry picked from commit 086afa9a69cb99258abfab270420b4968d7c3717)

mgr/cephadm: rename OSDQueue -> OSDRemovalQueue

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
(cherry picked from commit e0f9983b50444d53d6750f9bf1f04556510419eb)

mgr/cephadm: make `ceph orch {restart|...}` asynchronous

Othwerwiese we'd block the CLI handler thread.

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
(cherry picked from commit c6e4b9ab9ca317c840613167df9ec17fe6173592)

Merge pull request #38808 from badone/wip-48528-octopus

octopus: install-deps.sh: Make powertools repo case insensitive

Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #38973 from rhcs-dashboard/wip-48928-octopus

octopus: mgr/dashboard: add `--ssl` to `ng serve`

Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Laura Paduano <lpaduano@suse.com>
Reviewed-by: Tatjana Dehler <tdehler@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>

Merge pull request #38867 from callithea/wip-48809-octopus

octopus: mgr/dashboard: fix Reads/Writes ratio of Clients IOPS donut chart

Reviewed-by: Nizamudeen A <nia@redhat.com>
Reviewed-by: Tatjana Dehler <tdehler@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>

Merge pull request #38593 from callithea/wip-48578-octopus

octopus: mgr/prometheus: Fix 'pool filling up' with >50% usage

Reviewed-by: Kiefer Chang <kiefer.chang@suse.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
Reviewed-by: p-se <pseidensal@suse.com>

Merge pull request #38496 from bk201/wip-48460-octopus

octopus: mgr/dashboard: allow selecting all daemons for Orchestrator NFS clusters

Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Tatjana Dehler <tdehler@suse.com>

Merge pull request #38347 from smithfarm/wip-48378-octopus

octopus: mon: validate crush-failure-domain

Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge pull request #37930 from rhcs-dashboard/wip-48079-octopus

octopus: mgr/dashboard: Prevent table items from getting selected while expanding

Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Laura Paduano <lpaduano@suse.com>

Merge pull request #38951 from smithfarm/wip-48888-octopus

octopus: cmake: detect and use sigdescr_np() if available

Reviewed-by: David Disseldorp <ddiss@suse.de>
Reviewed-by: David Disseldorp <ddiss@suse.de>
Reviewed-by: Brad Hubbard <bhubbard@redhat.com>

Merge pull request #38421 from smithfarm/wip-48429-octopus

octopus: rgw: avoid expiration early triggering caused by overflow

Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: Matt Benjamin <mbenjami@redhat.com>

Merge pull request #38420 from smithfarm/wip-48427-octopus

octopus: rgw: S3 Put Bucket Policy should return 204 on success

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #38339 from smithfarm/wip-48345-octopus

octopus: rgw: fix invalid payload issue when serving s3website error page

Reviewed-by: Casey Bodley <cbodley@redhat.com>

mgr/dashboard: add `--ssl` to `ng serve`

Add the `--ssl` option to the `ng serve` command in order to
reflect the changes to the cookies (https://tracker.ceph.com/issues/44591)
and be able to log into the dashboard by using the development server.

Fixes: https://tracker.ceph.com/issues/48847
Signed-off-by: Tatjana Dehler <tdehler@suse.com>
(cherry picked from commit 3971cdb0a64c077c9d136135b745778d875e0dae)

 Conflicts:
src/pybind/mgr/dashboard/frontend/package.json
     - Adopted master branch changes.

cmake: detect and use sigdescr_np() if available

sys_siglist is deprecated with glibc 2.32. A new thread-safe and
async-signal safe sigdescr_np() function is provided, so use it if
available.

Fixes: https://tracker.ceph.com/issues/47187
Signed-off-by: David Disseldorp <ddiss@suse.de>
(cherry picked from commit b9b6faf66ae67648626470cb4fc3f0850ac4d842)

do_cmake.sh: use python-3.9 with fedora version 33

Fedora-33 (rawhide) now has python-3.9.
Build breaks on Fedoa-33 with Traceback:
CMake Error at /usr/share/cmake/Modules/FindPackageHandleStandardArgs.cmake:165 (message):
  Could NOT find Python3: Found unsuitable version "3.9.0", but required is
  exact version "3.8" (found /usr/bin/python3, found components: Interpreter
  Development)

Fixes: https://tracker.ceph.com/issues/47971
Signed-off-by: Sunny Kumar <sunkumar@redhat.com>
(cherry picked from commit 2cc747cf4a55fb38a524ec12f6b6e5888fadf07a)

Merge pull request #38626 from batrick/i48635

octopus: qa: tox failures

Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge pull request #36803 from ShyamsundarR/wip-46820-octopus

octopus: mgr/volumes: Implement subvolume version v2

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #38688 from batrick/4ca8aaafa32bb7397eec80a8eb366d2dd781a72f-octopus

octopus: qa: unmount volumes before removal

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

mgr/dashboard: Create Ceph services via Orchestrator by using ServiceSpec

Fixes: https://tracker.ceph.com/issues/44831
Signed-off-by: Volker Theile <vtheile@suse.com>
(cherry picked from commit dc5e5a5980456cffa468f88bc0d51cbb3c32dd06)

Conflicts:
- src/pybind/mgr/dashboard/frontend/src/app/ceph/cluster/services/service-form/service-form.component.ts
  * Use i18n instead of $localize
  * Remove CdForm class
  * Adapt code to ngx-bootstrap. Typeahead works a little bit different than in Pacific/ng-bootstrap.
  * Adapt to older TypeScript version
- src/pybind/mgr/dashboard/frontend/src/app/ceph/cluster/services/service-form/service-form.component.spec.ts
  * Adapt code to ngx-bootstrap.
- src/pybind/mgr/dashboard/frontend/src/app/ceph/cluster/services/services.component.ts
  * Use i18n instead of $localize
  * Replace ModalService by BsModalService
- src/pybind/mgr/dashboard/frontend/src/app/shared/services/task-message.service.ts
  * Use i18n instead of $localize
- src/pybind/mgr/dashboard/frontend/src/styles/ceph-custom/_basics.scss
  * Relocate changes to src/pybind/mgr/dashboard/frontend/src/styles.scss

Merge pull request #38594 from callithea/wip-48587-octopus

octopus: mgr/dashboard: RGW User Form is validating disabled fields

Reviewed-by: Aashish Sharma <aasharma@redhat.com>
Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
Reviewed-by: Tatjana Dehler <tdehler@suse.com>

mgr/dashboard: enable different URL for users of browser to Grafana

Fixes: https://tracker.ceph.com/issues/47386
Signed-off-by: Patrick Seidensal <pseidensal@suse.com>
(cherry picked from commit 676f82923d875cfe9528a401963c09f29a6de7f4)

Conflicts:
        doc/mgr/dashboard.rst
        src/pybind/mgr/dashboard/controllers/grafana.py

Resolved some trivial conflicts and replaced the heading of the newly
added section in the documentation to another level, as the
documentation between master and octopus differs quite much.

mgr/dashboard: fix Reads/Writes ratio of Clients IOPS donut chart

Use the percentage of READ ops instead of its value.

Fixes: https://tracker.ceph.com/issues/48717
Signed-off-by: Kiefer Chang <kiefer.chang@suse.com>
(cherry picked from commit 9ef1a729b5100b0ee32c0df367ef4b33ba762d4a)

mgr/dashboard: Updating the docs regarding the user lockout

Fixes: https://tracker.ceph.com/issues/40914
Signed-off-by: Nizamudeen A <nia@redhat.com>
(cherry picked from commit 3f50b05e2530dfcdc44e5f13abac9fbdf20400c1)

mgr/dashboard: Temporary User Lockout if 10 Invalid Login attempts

Implemented a user lockout mechanism if the user enters 10 invalid attempts. The attempt count gets resetted to 0 once the user succesfully logins before getting disabled. Once the user gets disabled administrator has to manually enable the user which will also resets the number of attempts.

Fixes: https://tracker.ceph.com/issues/40914
Signed-off-by: Nizamudeen A <nia@redhat.com>
(cherry picked from commit 41941f0d28f51cb391ff7bacae84a5d511bafb36)

Merge pull request #38759 from tchaikov/octopus-38263

octopus: cmake: define BOOST_ASIO_USE_TS_EXECUTOR_AS_DEFAULT for Boost.Asio users

Reviewed-by: Deepika Upadhyay <dupadhya@redhat.com>

Merge pull request #38770 from votdev/wip-48629-octopus

octopus: mgr/dashboard: The /rgw/status endpoint does not check for running service

Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>

Merge pull request #38592 from callithea/wip-48574-octopus

octopus: mgr/crash: Serialize command handling

Reviewed-by: Brad Hubbard <bhubbard@redhat.com>
Reviewed-by: Boris Ranto <branto@redhat.com>

Merge pull request #38555 from kamoltat/wip-ksriivad-octopus-backport

octopus: qa/tasks/mgr/test_progress: update test suite to check for specific progress events

Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge pull request #38455 from trociny/wip-48470-octopus

octopus: librbd: fix regression in object map diff request

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #38429 from smithfarm/wip-48194-octopus

octopus: bluestore: Make mempool assignment same after bufferlist rebuild

Reviewed-by: Adam Kupczyk <akupczyk@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge pull request #38428 from smithfarm/wip-48094-octopus

octopus: os/bluestore: fix segfault on out-of-bound offset provided to  claim_free_to_right() call

Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Igor Fedotov <ifedotov@suse.com>
Reviewed-by: Adam Kupczyk <akupczyk@redhat.com>

cmake: add empty RPATH to ceph-diff-sorted

This fixes a transient FTBFS on openSUSE:

[ 5365s] -- Installing: /home/abuild/rpmbuild/BUILDROOT/ceph-15.2.4.337+g55cec95eaf-1.1.x86_64/usr/bin/ceph-diff-sorted
[ 5365s] CMake Error at src/tools/cmake_install.cmake:230 (file):
[ 5365s]   file RPATH_CHANGE could not write new RPATH:
[ 5365s]
[ 5365s]     /usr/lib64/ceph
[ 5365s]
[ 5365s]   to the file:
[ 5365s]
[ 5365s]     /home/abuild/rpmbuild/BUILDROOT/ceph-15.2.4.337+g55cec95eaf-1.1.x86_64/usr/bin/ceph-diff-sorted
[ 5365s]
[ 5365s]   No valid ELF RPATH or RUNPATH entry exists in the file;

Fixes: https://tracker.ceph.com/issues/46553
Signed-off-by: Nathan Cutler <ncutler@suse.com>
(cherry picked from commit cb20ce39315dc218e1b6338aae6a676a18453124)

Merge pull request #38086 from orztt/wip-rgw-versioning-octopus

octopus: rgw: cls/rgw/cls_rgw.cc: fix multiple lastest version problem

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #38782 from sebastian-philipp/octopus-backport-37607-38169-37696-37576-38211-38278-38304-38285-38317-38130-38064-3803

octopus: cephadm: batch backport January (1)

Reviewed-by: Michael Fritch <mfritch@suse.com>

Merge pull request #38798 from votdev/wip-48652-octopus

octopus: mgr/dashboard: Display a warning message in Dashboard when debug mode is enabled

Reviewed-by: Nizamudeen A <nia@redhat.com>
Reviewed-by: Tatjana Dehler <tdehler@suse.com>

Merge pull request #38797 from rhcs-dashboard/wip-48626-octopus

octopus: mgr/dashboard: Fixing dashboard logs e2e test

Reviewed-by: Aashish Sharma <aasharma@redhat.com>
Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>

Merge pull request #38607 from rhcs-dashboard/wip-48605-octopus

octopus: mgr/dashboard: Fix CRUSH map viewer VirtualScroll

Reviewed-by: Aashish Sharma <aasharma@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
Reviewed-by: Volker Theile <vtheile@suse.com>

Merge pull request #38583 from votdev/wip-48592-octopus

octopus: mgr/dashboard: Drop invalid RGW client instances, improve logging

Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Avan Thakkar <athakkar@redhat.com>

Merge pull request #38598 from callithea/wip-48494-octopus

octopus: mgr/dashboard: Fix for misleading "Orchestrator is not available" error

Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Kiefer Chang <kiefer.chang@suse.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
Reviewed-by: Volker Theile <vtheile@suse.com>

Merge pull request #38597 from callithea/wip-48539-octopus

octopus: mgr/dashboard: adjust refresh intervals of Services and Daemons

Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Kiefer Chang <kiefer.chang@suse.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
Reviewed-by: Tatjana Dehler <tdehler@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>

Merge pull request #38503 from callithea/wip-48511-octopus

octopus: mgr/dashboard: remove pyOpenSSL version pinning

Reviewed-by: Kiefer Chang <kiefer.chang@suse.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
Reviewed-by: Tatjana Dehler <tdehler@suse.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #37969 from bk201/wip-48132-octopus

octopus: mgr/dashboard: disable cluster selection in NFS export editing form

Reviewed-by: Laura Paduano <lpaduano@suse.com>
Reviewed-by: Tatjana Dehler <tdehler@suse.com>

install-deps.sh: Make powertools repo case insensitive

It looks like CentOS 8.3 will see all repos converted to lower case and
this has been pre-empted in the CentOS stream repos so we need to be
able to enable a repo called 'PowerTools' or 'powertools'

See https://git.centos.org/rpms/centos-repos/c/b759b17

Fixes: https://tracker.ceph.com/issues/48174
Signed-off-by: Brad Hubbard <bhubbard@redhat.com>
(cherry picked from commit 7a68845579c701f43c9526c810e586680cf76fe0)

 Conflicts:
install-deps.sh - minor formatiing difference

mgr/ActivePyModules.cc: always release GIL before attempting to acquire a lock

A thread that holds the GIL while attempting to acquire a mutex will cause a deadlock
if another thread owns the mutex and is waiting on the GIL. The GIL must not be treated
like an ordinary mutex since it may be preempted at any time or released when doing
blocking I/O. Such deadlocks are severe since they starve all threads from access to the
GIL and therefore prevent any Python code from running until the mgr process is restarted.

Fixes: https://tracker.ceph.com/issues/39264
Signed-off-by: Cory Snyder <csnyder@iland.com>
(cherry picked from commit 0601b31a53a455f0b67c981460d198cb3a97f3de)

mgr/cephadm: fix bug in orch apply osd --dry-run where empty table printed

Signed-off-by: Daniel-Pivonka <dpivonka@redhat.com>
(cherry picked from commit 459fa5777d400d5aa9eb7d4fe520369d47c775b9)

qa/distros: configure podman to pull from docker-mirror

This also requires a change to cephadm.py to search for
registry entries for docker.io by 'prefix' or 'location',
and to only add the mirror if a mirror isn't already present

Signed-off-by: Dan Mick <dmick@redhat.com>
(cherry picked from commit f30c55a242f381f0d191b55c76d3fc5d027e602e)

cephadm: update permissions on newly created dirs

Use 755 instead of 700 on new directories created during
bootstrap

Signed-off-by: Paul Cuzner <pcuzner@redhat.com>
(cherry picked from commit 9ef60e9111691c6511baa93ca1d080250588d6c0)

cephadm: create default output dir during bootstrap

If the output dir doesn't exist the old behaviour was to
abort bootstrap. Since we're running with UID=0, this
patch will create the missing dirs for the user to keep the
bootstrap process going

Signed-off-by: Paul Cuzner <pcuzner@redhat.com>
(cherry picked from commit 27b5eada81189ad01b31ff34bf0e362353cfbd92)

mgr/cephadm: correct connection failure message

When add host fails with py3 missing, wasn't used in
the output returned to the admin - instead they saw
the generic failure message which "points them in the
wrong direction". This patch shows the py3 missing
message

Signed-off-by: Paul Cuzner <pcuzner@redhat.com>
(cherry picked from commit 9e27995bb0a7de0072742e5f18926343acf80e05)

pybind/mgr/cephadm: limit rgw osd caps

Using tagged pools ensures RGW only can access pools used for RGW.

Fixes: https://tracker.ceph.com/issues/48594
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit 373cc847cf0f8b4ec7aefbfe64c01c3f18a4e021)

doc/cephadm: Add remove host to operations

Co-authored-by: Alexandra Settle <asettle@suse.com>
Co-authored-by: Kefu Chai <tchaikov@gmail.com>
Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
(cherry picked from commit 453000af19b25082aef1b473bd9284311ac98204)

mgr/cephadm: improve iscsi dashboard config

updating iscsi config dashboard to match what was done with ceph ansible.
This includes:
  - set iscsi api ssl verification to false when api_secure is set
  - using https rather than http in gateway url when spi_secure is set
  - default api_user and api_password to "admin"
  - convert hostname to ip address when creating gateway url

Signed-off-by: Adam King <adking@redhat.com>
(cherry picked from commit ba6d22b492400588de22b93189eee8818f647807)

mgr/cephadm: disallow_untyped_defs=True

simplifies mypy.ini

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
(cherry picked from commit a823ee6b1f5a8eb2030c1edf0a7014f86eb719ab)

Conflicts:
src/mypy.ini

orchestrator: drop redundant try/except

YAML is validated up front, so the additional try/except
clause is not needed.

Signed-off-by: Paul Cuzner <pcuzner@redhat.com>
(cherry picked from commit a8690f1b50f0952155e327ffb17952d7cecfbe98)

orchestrator: validate osd yaml

Validate the yaml document ahead of usage, to catch
any basic errors in the yaml doc. If errors are found the
exception is shown in the mgr log, and the user gets a
more friendly error message, without the traceback.

Signed-off-by: Paul Cuzner <pcuzner@redhat.com>
(cherry picked from commit 0f236c7911bdcc4b0266833e1b7da01a3c7e4eac)

cephadm: command_unit: call systemctl with verbose=True

Otherwise `cephadm unit ... status` won't print anything

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
(cherry picked from commit 35fcd04fc65a7f7dc491fd0cb2619a9572f0b4f5)

mgr/cephadm: make type annotations for module.py mandatory

Fixing bogus json representation for registry-login

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
(cherry picked from commit 7bbc77cd48bf48da2c120220ad31678f650c05b1)

Conflicts:
src/mypy.ini
src/pybind/mgr/cephadm/module.py

doc/dev/cephadm: Add link to the index

Adds a link to the index for the compliance-check doc

Signed-off-by: Paul Cuzner <pcuzner@redhat.com>
(cherry picked from commit f2791c0e26a81b8d0f1a7d070f67dd79cd6ed25c)

Conflicts:
doc/dev/cephadm/index.rst

doc/dev/cephadm: Updates following feedback

Minor grammatical and formatting changes

Signed-off-by: Paul Cuzner <pcuzner@redhat.com>
(cherry picked from commit 1facc5e9a725d33e57370c65d3086e952f13b05c)

doc: moved design doc under doc/dev

Signed-off-by: Paul Cuzner <pcuzner@redhat.com>
(cherry picked from commit 7168cbf0521aa29701f0133b4e38b858888e46b4)

docs: high level design for a compliance check feature

Document defining the high level goals of a compliance check
feature that could be added to mgr/cephadm.

Signed-off-by: Paul Cuzner <pcuzner@redhat.com>
(cherry picked from commit 2407a7bf996fced3b053727db5c1dc17ffbc9624)

mgr/cephadm: make annotations for schedule.py mandatory

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
(cherry picked from commit ba3189b13882ff919ad39a2e377fa6b29c04c79a)

 Conflicts:
src/mypy.ini

mgr/cephadm: make annotations for inventory.py mandatory

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
(cherry picked from commit 77791a21db6597bc17a33411e02b12c03bd857a8)

Conflicts:
src/mypy.ini

mgr/cephadm: make annotations for serve.py mandatory

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
(cherry picked from commit 95bf93a9e4f214cfd3bba2b4358174d9ddc8f0ea)

Conflicts:
src/mypy.ini

mgr/cephadm: make annotations for upgrade.py mandatory

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
(cherry picked from commit 3f9bf2b3f35566ec5537fd36a5163a17ab7778c6)

mgr/orchestrator: Remove 'CONTAINER ID' col from orch ps if unknown

Signed-off-by: Varsha Rao <varao@redhat.com>
(cherry picked from commit 9c241e946a9808b23e5f923ab9283666355883fd)

mgr/cephadm: improve logging during JSONDecodeError

improve logging around JSONDecodeErrors that can occur during a
cephadm image pull, load of the extra_ceph_conf, and parsing the cluster
health status

Fixes: https://tracker.ceph.com/issues/48120
Signed-off-by: Michael Fritch <mfritch@suse.com>
(cherry picked from commit d4c9d5e680cb17379940a1ceaa038c6224b9b039)

mgr/cephadm: handle JSONDecodeError in OSD service

avoid exceptions when attempting an osd deployment etc.

Fixes: https://tracker.ceph.com/issues/48119
Signed-off-by: Michael Fritch <mfritch@suse.com>
(cherry picked from commit 46692e26c12ed169a9f3569857a13304d09b41c1)

mgr/cephadm: handle JSONDecodeError in the serve loop

avoid exceptions when attempting to parse invalid JSON output as this
will later cause the cephadm module to fail with a MGR_MODULE_ERROR err

Fixes: https://tracker.ceph.com/issues/48118
Signed-off-by: Michael Fritch <mfritch@suse.com>
(cherry picked from commit 6d759fb5deac0c52b3c738a2e695738228749420)

cephadm: reference the last local image by digest

use the image digest rather than repo/tag combo when inferring the last
used local ceph image

Fixes: https://tracker.ceph.com/issues/48205
Signed-off-by: Michael Fritch <mfritch@suse.com>
(cherry picked from commit 4cf49997c3af7428d2767e72f62f6b875e55c51d)

mgr/cephadm: verify the `host` key exists during refresh

various parts of the host cache might not have a key for the desired
host due to ssh communication issues during host add/remove

Fixes: https://tracker.ceph.com/issues/48223
Signed-off-by: Michael Fritch <mfritch@suse.com>
(cherry picked from commit 218d00c8ed8e3d1df2f08ce5d89ec9ddd4f07630)

mgr/cephadm: upgrade: fail gracefully, if daemon redeploy fails

Current behaviour kills the `serve()` loop of mgr/cephadm

Fixes: https://tracker.ceph.com/issues/47684
Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
(cherry picked from commit 1ceb6dfb15781569ece627fc53bbd1eb294bf2b7)

mgr/cephadm: OSD service: reduce log noise

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
(cherry picked from commit f0ee6284a7ab0ea85ae486d236e072e13b658a78)