Revert "Merge PR #34091 into master"

This reverts commit f865f3e0a0f3a646b093b3571ea76713eca1916c, reversing
changes made to 7ef5458e26ec7c0565509a7882fa31fa064eb49d.

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #33992 from rhcs-dashboard/44626-fix-notifications-e2e-tests

mgr/dashboard: fix notifications E2E tests

Reviewed-by: Tiago Melo <tmelo@suse.com>
Reviewed-by: Laura Paduano <lpaduano@suse.com>

Merge pull request #34209 from smithfarm/wip-doc-multi-cluster-follow-on-fix

doc: fix cephadm link

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #34235 from toabctl/add-sle-15-sp2

qa/distros: add SLE-15-SP2

Reviewed-by: Nathan Cutler <ncutler@suse.com>

Merge pull request #32749 from james58899/fix-capacity

monitoring: Fix pool capacity incorrect

Reviewed-by: Jan Fajerski <jfajerski@suse.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>

Merge pull request #34233 from tchaikov/wip-qa-py3

qa/tasks: change to be py3 compatible and cleanups

Reviewed-by: Thomas Bechtold <tbechtold@suse.com>

qa/distros: add SLE-15-SP2

Ceph octopus is known to run on SLE-15-SP2 so add it to
qa/distros/all.

Signed-off-by: Thomas Bechtold <tbechtold@suse.com>

cmake: add import-tasks to run-tox-qa test

to ensure that qa suites are python3 compatible

Signed-off-by: Kefu Chai <kchai@redhat.com>

qa/tasks/mgr: use relative import

for better readability, and to ease the pain of developer to track back
to the top level python package for referencing a submodule

Signed-off-by: Kefu Chai <kchai@redhat.com>

qa/tasks/cephfs: fix print() to be py3 compatible

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #33949 from toabctl/import-tasks-with-py3

qa: Add a tox env that can test importing files

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #34141 from tchaikov/wip-test-threadsafe

test/objectstore: set "threadsafe" flag for ASSERT_DEATH tests

Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge PR #32093 into master

* refs/pull/32093/head:
doc/start: Update hardware recommendations.

Reviewed-by: Sage Weil <sage@redhat.com>

doc/start: Update hardware recommendations.

Signed-off-by: Mark Nelson <mnelson@redhat.com>

Merge PR #34218 into master

* refs/pull/34218/head:
mgr/cephadm: Add missing decorator

Reviewed-by: Michael Fritch <mfritch@suse.com>

mgr/cephadm: Add missing decorator

Caused by a conflict between

* #33581 (770816dc9f5b440fd1fc6cc507a06cd8573f6ea1) and
* #34091 (a78f42e225bbc041324dc3e565deb207feddad26)

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>

Merge pull request #33873 from lxbsz/qa_vs_failed

qa/tasks/vstart_runner: fix sh to make it compatible with teth

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #33882 from rosinL/wip-fix-rdma-segfault

msg/async/rdma: add return value check after qp created

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #34195 from tchaikov/wip-tasks-mgr-submodule

qa/tasks/mgr: partial revert of 'import with full path'

Reviewed-by: Sebastian Wagner <swagner@suse.com>
Reviewed-by: Laura Paduano <lpaduano@suse.com>

Merge pull request #34196 from majianpeng/bluestore-remove-incorrectly-comments

os/bluestore: remove incorrect comments.

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #34183 from neha-ojha/wip-doc-fix

doc/releases/octopus.rst: fix min_mon_release string

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

doc: fix cephadm link

Fixes: 8fe460e7562b28e007179eeb533ef68a3e99b4c8
Signed-off-by: Nathan Cutler <ncutler@suse.com>

Merge PR #34091 into master

* refs/pull/34091/head:
mgr/cephadm: cleanup async_map_completion etc.
mgr/cephadm: replace async_completion with trivial_completion
mgr/cephadm: replace async_map_completion with a simple wrapper

Merge PR #34188 into master

* refs/pull/34188/head:
doc: do not encourage readers to run multiple clusters on same HW

Reviewed-by: Greg Farnum <gfarnum@redhat.com>

Merge pull request #30839 from ifed01/wip-ifed-aloc_replay_improve

test/bmap_allocator_replay_test: improving allocator replay tool.

Signed-off-by: Igor Fedotov <ifedotov@suse.com>

Merge pull request #34187 from liewegas/fix-44759

osd/PeeringState: drop mimic assert

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #31453 from oneoneonepig/patch-2

doc: add sudo in ceph status command

Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #29443 from simon-rock/metatool_dev

tool: add cephfs-meta-injection to show and amend info of inode

Merge pull request #33198 from simon-rock/simon_work_get_dirfrag_bound

mds: optimize function, get_force_dirfrag_bound_set

Merge pull request #33643 from simon-rock/simon_destructor_of_session

mds : core dump if the resource is not released

Merge pull request #31668 from shenhang/deny_reconnect_on_reboot

mds: short reboot time

Merge pull request #33570 from shenhang/modifyactiverequest

mds: Using begin() and empty() to iterate the xlist

Merge pull request #33923 from systemcrash/patch-1

doc: fix spelling and grammar issues in ceph-deploy-osd.rst

Reviewed-by: Varsha Rao <varao@redhat.com>

Merge pull request #31055 from smithfarm/wip-42357

tests: qa/tasks/ceph.py: fail if any osd role doesn't get a device

Reviewed-by: Greg Farnum <gfarnum@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #33366 from jmolmo/rook_blinking_lights

mgr/rook: Blinking lights

Reviewed-by: Sebastian Wagner <swagner@suse.com>

qa: Add a tox env that can test importing files

While switching to python3, we need to make sure that we can import
the qa/tasks (and others, but this starts with qa/tasks) on a python3
environment.
To test this, we need to install teuthology into the test
venv. Currently, teuthology is not py3 ready so this will fail.

To test the current state of the qa/tasks directory with the ongoing
work for python3 within teuthology, you can now do:

TEUTHOLOGY_GIT=git+https://github.com/kshtsk/teuthology.git@wip-py3-compat \
    tox -eimport-tasks

This is using the current branch from
https://github.com/ceph/teuthology/pull/1362 which does the work to
make teuthology python3 ready.

NOTE: This tox env is not activated by default. It's currently failing
but it provides a way to iterate over the failures and once we have
them fixed, we can activate the tox env during make-check.

Signed-off-by: Thomas Bechtold <tbechtold@suse.com>

os/bluestore: remove incorrectly comments.

Signed-off-by: Jianpeng Ma <jianpeng.ma@intel.com>

mgr/rook: Blinking lights

Blinking lights implementation

Signed-off-by: Juan Miguel Olmo Martínez <jolmomar@redhat.com>

qa/tasks/mgr: partial revert of 'import with full path'

this change partially reverts #34139

using relative import helps with readability and ease the pain to write
down the full parent module name

in #34139, all relative imports were replaced with full path, because we
were using following code to verify if the code is python3 compatible:

```
  mod_spec = importlib.util.spec_from_file_location(mod_name, path)
  mod = importlib.util.module_from_spec(mod_spec)
  mod_spec.loader.exec_module(mod)
```

but this does not work with submodule which can import using relative
import without specifying the name of the package and its parent module.

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #34184 from cbodley/wip-qa-rgw-test-multi-fix

qa/rgw: multisite test fixes

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge PR #33581 into master

* refs/pull/33581/head:
mgr/cephadm: add watch_url to NFS bootstrap config
mgr/cephadm: check for an existing NFS config object
mgr/cephadm: combine `get_daemon_name` and `get_rados_user` funcs
mon: allow `auth caps` for `mgr` profile
mgr/cephadm: add deployment example to NFS docs
mgr/cephadm: add utils.py
mgr/cephadm: create/update keyring during nfs config
mgr/cephadm: pass NFSServiceSpec instead of pool/namespace args
mgr/cephadm: add nfs apply test
mgr/cephadm: fix nfs redeploy/reconfig
mgr/cephadm: ensure nfs uses the correct entity
mgr/cephadm: fix service name for nfs type
mgr/cephadm: apply nfs service spec
mgr/cephadm: don't create a unique nfs daemon name
mgr/cephadm: update docs
mgr/cephadm: add minimal caps for nfs client keyring
mgr/cephadm: add logging to NFSGanesha class
mgr/cephadm: create empty nfs rados config obj
mgr/cephadm: add nfs tests
mgr/cephadm: add a minimal ganesha.conf
mgr/cephadm: deploy nfs-ganesha service
mgr/cephadm: split out `config-and-keyrings` json
mgr/orchestrator: rearrange interface to match impl. order

Reviewed-by: Sebastian Wagner <swagner@suse.com>

Merge PR #34181 into master

* refs/pull/34181/head:
mgr/cephadm: bind grafana to all interfaces

Reviewed-by: Michael Fritch <mfritch@suse.com>

Merge PR #34182 into master

* refs/pull/34182/head:
mgr/cephadm: fix 'orch ps --refresh'

Reviewed-by: Michael Fritch <mfritch@suse.com>

Merge pull request #33910 from xiexingguo/wip-44532-2

osd/PeeringState: do not trim pg log past last_update_ondisk

Reviewed-by: Yan Jun <yan.jun8@zte.com.cn>
Reviewed-by: Samuel Just <sjust@redhat.com>

doc: do not encourage readers to run multiple clusters on same HW

This material was first added on December 22, 2012 by
46d0334456c14da2bc8e012f121c545b1b33b95f

Signed-off-by: Nathan Cutler <ncutler@suse.com>

mgr/cephadm: add watch_url to NFS bootstrap config

watch for changes to the per-daemon RADOS conf file

Signed-off-by: Michael Fritch <mfritch@suse.com>

mgr/cephadm: check for an existing NFS config object

assume an existing NFS config if the named rados object already exists

Signed-off-by: Michael Fritch <mfritch@suse.com>

mgr/cephadm: combine `get_daemon_name` and `get_rados_user` funcs

Signed-off-by: Michael Fritch <mfritch@suse.com>

mon: allow `auth caps` for `mgr` profile

cephadm orchestrator needs to update the caps during reconfig/redeploy

Signed-off-by: Michael Fritch <mfritch@suse.com>

mgr/cephadm: add deployment example to NFS docs

Signed-off-by: Michael Fritch <mfritch@suse.com>

mgr/cephadm: add utils.py

move `name_to_config_section` and `assert_valid_host` into utils.py

Signed-off-by: Michael Fritch <mfritch@suse.com>

mgr/cephadm: create/update keyring during nfs config

the keyring might already exist from a prior config/reconfig/redeploy.

attempt to get_or_create the keyring first and than update the keyring
caps afterward

Signed-off-by: Michael Fritch <mfritch@suse.com>

mgr/cephadm: pass NFSServiceSpec instead of pool/namespace args

Signed-off-by: Michael Fritch <mfritch@suse.com>

mgr/cephadm: add nfs apply test

Signed-off-by: Michael Fritch <mfritch@suse.com>

mgr/cephadm: fix nfs redeploy/reconfig

search for the `NFSServiceSpec` instead of passing empty
`cephadm_args` during `_create_daemon`

Signed-off-by: Michael Fritch <mfritch@suse.com>

mgr/cephadm: ensure nfs uses the correct entity

mon entity needs to start with `client.`

Signed-off-by: Michael Fritch <mfritch@suse.com>

mgr/cephadm: fix service name for nfs type

the `nfs` service was incorrectly reported as `mds`

Signed-off-by: Michael Fritch <mfritch@suse.com>

mgr/cephadm: apply nfs service spec

orch apply nfs <svc_id> [<placement>]

Signed-off-by: Michael Fritch <mfritch@suse.com>

mgr/cephadm: don't create a unique nfs daemon name

only one nfs daemon will be deployed per host

Signed-off-by: Michael Fritch <mfritch@suse.com>

mgr/cephadm: update docs

Signed-off-by: Michael Fritch <mfritch@suse.com>

mgr/cephadm: add minimal caps for nfs client keyring

Signed-off-by: Michael Fritch <mfritch@suse.com>

mgr/cephadm: add logging to NFSGanesha class

Signed-off-by: Michael Fritch <mfritch@suse.com>

mgr/cephadm: create empty nfs rados config obj

Signed-off-by: Michael Fritch <mfritch@suse.com>

mgr/cephadm: add nfs tests

Signed-off-by: Michael Fritch <mfritch@suse.com>

mgr/cephadm: add a minimal ganesha.conf

we only need to specify the rados pool and config object url

Signed-off-by: Michael Fritch <mfritch@suse.com>

mgr/cephadm: deploy nfs-ganesha service

orch daemon add nfs <svc_arg> <pool> [<namespace>] [<placement>]

Signed-off-by: Michael Fritch <mfritch@suse.com>

mgr/cephadm: split out `config-and-keyrings` json

Signed-off-by: Michael Fritch <mfritch@suse.com>

mgr/orchestrator: rearrange interface to match impl. order

rgw impl. happens before rbd impl

Signed-off-by: Michael Fritch <mfritch@suse.com>

osd/PeeringState: drop mimic assert

This code can still execute for a cluster that has require_osd_release >=
nautilus because a PG is processing an older OSDMap.  Drop this assert
since the osd boot checks on require_osd_release are sufficient.

Fixes: https://tracker.ceph.com/issues/44759
Signed-off-by: Sage Weil <sage@redhat.com>

Merge PR #34146 into master

* refs/pull/34146/head:
mgr/cephadm: use docker.io in default container image name
mgr/cephadm: fix 'upgrade start' message when specifying a version

Reviewed-by: Michael Fritch <mfritch@suse.com>

Merge PR #34166 into master

* refs/pull/34166/head:
mgr/cephadm: DriveGroupSpec needs to know about unmanaged attribute
mgr/cephadm: DriveGroupSpec needs to support/ignore _unmanaged_

Reviewed-by: Sebastian Wagner <swagner@suse.com>

Merge PR #34180 into master

* refs/pull/34180/head:
cephadm: Fix check_ip_port to work with IPv6

Reviewed-by: Sage Weil <sage@redhat.com>

qa/rgw: partial revert of 'import with full path'

Signed-off-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #34179 from cbodley/wip-rgw-admin-user-axe

rgw: remove unused librgw_admin_user

Reviewed-by: Daniel Gryniewicz <dang@redhat.com>
Reviewed-by: Matt Benjamin <mbenjami@redhat.com>

rgw: remove unused librgw_admin_user

unused and not worth the effort to maintain

Signed-off-by: Casey Bodley <cbodley@redhat.com>

doc/releases/octopus.rst: fix min_mon_release string

Signed-off-by: Neha Ojha <nojha@redhat.com>

mgr/cephadm: fix 'orch ps --refresh'

The use of host in the refresh loop was clobbering the argument value.

Fixes: https://tracker.ceph.com/issues/44513
Signed-off-by: Sage Weil <sage@redhat.com>

mgr/cephadm: bind grafana to all interfaces

Fixes: https://tracker.ceph.com/issues/44608
Signed-off-by: Sage Weil <sage@redhat.com>

cephadm: Fix check_ip_port to work with IPv6

Fixes: https://tracker.ceph.com/issues/43816
Signed-off-by: Ricardo Marques <rimarques@suse.com>

Merge pull request #34173 from tchaikov/wip-test-rgw

test/rgw: should import parse not urlparse

Reviewed-by: Thomas Bechtold <tbechtold@suse.com>

Merge pull request #34117 from smithfarm/wip-43746-follow-on

pybind/mgr/*: fix config_notify handling of default values

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #34136 from tchaikov/test-msgr-stdout

test/msgr: print out informative messages to stdout

Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge pull request #34015 from yuyuyu101/wip-pin-rocksdb

kv/RocksDBStore: using PinnableSlice to avoid extra value copy

Reviewed-by: Adam Kupczyk <akucpzyk@redhat.com>
Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #34140 from theanalyst/doc/releases/oct-distro-fix

doc: releases: add a note on debian packages & those built by the community

Reviewed-By: Kefu Chai <kchai@redhat.com>
Reviewed-By: Sage Weil <sweil@redhat.com>
Reviewed-By: Nathan Cutler <ncutler@suse.com>

test/rgw: should import parse not urlparse

urlparse is a function while parse is a submodule where parse_qsl() is
provided

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #34138 from bk201/wip-44725

qa/tasks/mgr/dashboard/test_health: update mdsmap schema

Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Laura Paduano <lpaduano@suse.com>

Merge pull request #34139 from tchaikov/wip-qa-test-py3

qa,test: py3 compatible changes and fix

Reviewed-by: Thomas Bechtold <tbechtold@suse.com>

mgr/cephadm: cleanup async_map_completion etc.

This breaks the progress integration, but we haven't
used that anyway till now.

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>

mgr/cephadm: replace async_completion with trivial_completion

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>

mgr/cephadm: replace async_map_completion with a simple wrapper

There is no need to wrap everything into completions.

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>

Merge pull request #34131 from tspmelo/wip-force-npm-resolutions

mgr/dashboard: Check for missing npm resolutions

Reviewed-by: Laura Paduano <lpaduano@suse.com>
Reviewed-by: Nathan Cutler <ncutler@suse.com>

mgr/cephadm: DriveGroupSpec needs to know about unmanaged attribute

Signed-off-by: Joshua Schmid <jschmid@suse.de>

Merge pull request #34165 from tchaikov/wip-doc-release-octopus-indent

doc/releases/octopus: fix indents

Reviewed-by: Abhishek Lekshmanan <abhishek@suse.com>

qa/tasks/ceph.py: fail test if osd devices not found

Fixes: https://tracker.ceph.com/issues/42357
Signed-off-by: Nathan Cutler <ncutler@suse.com>

mgr/cephadm: DriveGroupSpec needs to support/ignore _unmanaged_

Signed-off-by: Joshua Schmid <jschmid@suse.de>

qa/tasks/ceph.py: use .format to log dicts

The ".format" builtin logs dicts nicely right out of the box.

Also, some of the log messages were too cryptic - fixed them in this commit as
well.

Signed-off-by: Nathan Cutler <ncutler@suse.com>

qa/tasks/ceph.py: drop roles_to_journals and remote_to_roles_to_journals

These do not seem to get any use anymore.

Signed-off-by: Nathan Cutler <ncutler@suse.com>

qa/tasks/ceph.py: drop block_journal, tmpfs_journal

I looked, but did not find any tests that actually use these options.

Signed-off-by: Nathan Cutler <ncutler@suse.com>

qa/tasks/ceph.py: cleanup: stop calling get_wwn_id_map()

Nowadays, get_wwn_id_map is essentially a noop - it does:

    return dict((d, d) for d in devs)

This reverts another bit of 8f720454cbb536ac6f410fb6c18b2b0945b70160 from 2013.

References: https://tracker.ceph.com/issues/42313
Signed-off-by: Nathan Cutler <ncutler@suse.com>

doc/releases/octopus: fix indents

reduce the indent in subsection of "RGW object storage", some of the
nested items should be indented with two spaces instead of four.

Signed-off-by: Kefu Chai <kchai@redhat.com>