qa/distros: add SLE-15-SP2

Ceph octopus is known to run on SLE-15-SP2 so add it to
qa/distros/all.

Signed-off-by: Thomas Bechtold <tbechtold@suse.com>

Merge pull request #33847 from tchaikov/wip-crimson-tell

crimson/osd: add tell command support

Reviewed-by: Ronen Friedman <rfriedma@redhat.com>

crimson/admin: revise doxygen docs

Signed-off-by: Kefu Chai <kchai@redhat.com>

crimson/admin: drop unused functions

Signed-off-by: Kefu Chai <kchai@redhat.com>

crimson/osd: add tell command support

* change the function signature of `call()` to accept a bufferlist,
  as MCommand is supposed to carry a bufferlist which can be consumed
  by the hook.
* drop the wrapper of `tell()`, instead always use `call()`. simpler
  this way.
* promote `tell_result_t` to top level of `crimson::admin` as it is used
  as part of the interface between hook and `AdminSocket`
* replace `maybe_parsed_t` with `std::variant<parsed_command_t, tell_result_t>`,
  as the latter is able to convey more detailed error information if
  something goes wrong.
* osd now handles `MCommand`.

Signed-off-by: Kefu Chai <kchai@redhat.com>

crimson/asok: pass vector<string> to parse_cmd()

prepare for the tell command, as the payload of MCommand is a
`vector<string>`, and `cmdmap_from_json()` expects a `vector<string>`

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #33828 from dillaman/wip-cls-rbd-mirror-status-summary

cls/rbd: use explicitly sized integer in mirror status summary

Reviewed-by: Mykola Golub <mgolub@suse.com>

Merge pull request #33653 from TRYTOBE8TME/wip-rgw-sts-doc-correction

rgw: STS Lite documentation correction

Reviewed-by: Pritha Srivastava <prsivas@redhat.com>

Merge pull request #33921 from tchaikov/wip-cmake-node-gyp

cmake: use specified python for running node-gyp

Reviewed-by: Laura Paduano <lpaduano@suse.com>

Merge remote-tracking branch 'gh/octopus'

Merge pull request #33918 from tchaikov/wip-seastar

seastar: pickup change to always enable smp poller

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

Merge PR #33930 into octopus

* refs/pull/33930/head:
mgr/cephadm: implement pause/resume to suspect non-monitoring background work
mgr/orch: add pause/resume hooks

Reviewed-by: Michael Fritch <mfritch@suse.com>

Merge PR #33868 into octopus

* refs/pull/33868/head:
mgr/rook: implement remove_daemons()
mgr/rook: fix mds/fs create/apply/update
mgr/rook: use ceph_daemon_id label consistently
mgr/rook: show filesystems in 'orch ls'
mgr/orch: 'age' -> 'ago' in 'orch ls' output
mgr/rook: initial pass at 'orch ls'
mgr/rook: include timestamps in 'orch ps'
mgr/orch: shorten container_id (hash) in cephadm, not orch ps
mgr/rook: include container_image_name in 'orch ps'
mgr/rook: fix list_daemons host arg

Reviewed-by: Juan Miguel Olmo <jolmomar@redhat.com>
Reviewed-by: Sebastian Wagner <swagner@suse.com>

Merge PR #33932 into octopus

* refs/pull/33932/head:
cephadm: fix datetime regexp to capture at most 6 digits

Reviewed-by: Sage Weil <sage@redhat.com>

seastar: pickup change to always enable smp poller

this is the first commit which is in sync with upstream without any
addition change in our branch.

Signed-off-by: Kefu Chai <kchai@redhat.com>

crimson/mgr: include necessary header

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #33900 from dillaman/wip-44459

qa/suites/rbd: removed OpenStack tempest test cases

Reviewed-by: Mykola Golub <mgolub@suse.com>

Merge pull request #33840 from trociny/wip-schedule-optimize-scan

mgr/rbd_support: don't scan pools that don't have schedules

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge PR #33914 into octopus

* refs/pull/33914/head:
mgr/orch: Cleanup python imports of the interface
mgr/orch: Properly handle NotImplementedError
pybind/mgr: Rm PersistentStoreDict

Reviewed-by: Sage Weil <sage@redhat.com>

mgr/rook: implement remove_daemons()

Signed-off-by: Sage Weil <sage@redhat.com>

mgr/rook: fix mds/fs create/apply/update

Signed-off-by: Sage Weil <sage@redhat.com>

mgr/rook: use ceph_daemon_id label consistently

In particular, for mds's, it's $fsname-$rank for mds.$fsname-$rank,
which is what we want.

Signed-off-by: Sage Weil <sage@redhat.com>

mgr/rook: show filesystems in 'orch ls'

The count vs size discrepancy is weird since rook things in terms of
ranks (*2 if standby) vs just a count.

Signed-off-by: Sage Weil <sage@redhat.com>

mgr/orch: 'age' -> 'ago' in 'orch ls' output

Signed-off-by: Sage Weil <sage@redhat.com>

mgr/rook: initial pass at 'orch ls'

This doesn't cover mds.* or rgw.* yet.

Signed-off-by: Sage Weil <sage@redhat.com>

mgr/rook: include timestamps in 'orch ps'

Signed-off-by: Sage Weil <sage@redhat.com>

mgr/orch: shorten container_id (hash) in cephadm, not orch ps

This lets us present a longer, human-readable id for rook.

Signed-off-by: Sage Weil <sage@redhat.com>

mgr/rook: include container_image_name in 'orch ps'

Signed-off-by: Sage Weil <sage@redhat.com>

mgr/rook: fix list_daemons host arg

Signed-off-by: Sage Weil <sage@redhat.com>

cephadm: fix datetime regexp to capture at most 6 digits

trailing `0` digits are not shown in the precision

Signed-off-by: Michael Fritch <mfritch@suse.com>

Merge pull request #33821 from votdev/issue_43446_bucket_lock

mgr/dashboard: Create bucket with x-amz-bucket-object-lock-enabled

Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Stephan Müller <smueller@suse.com>

Merge pull request #31997 from bk201/wip-43062

mgr/dashboard: support removing OSDs in OSDs page

Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Joshua Schmid <jschmid@suse.de>
Reviewed-by: Laura Paduano <lpaduano@suse.com>
Reviewed-by: Sebastian Wagner <swagner@suse.com>
Reviewed-by: Tiago Melo <tmelo@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>

Merge pull request #33920 from LenzGr/update-404-image

mgr/dashboard: Updated octopus image on 404 page

Reviewed-by: Tatjana Dehler <tdehler@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>

mgr/cephadm: implement pause/resume to suspect non-monitoring background work

If the user does 'orch pause', suspend all background work that makes
actual changes.

Continue to do read-only operations, like checking host connectivity
and scraping daemon and device status.

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #33858 from tspmelo/wip-use-cd-modal

mgr/dashboard: Use ModalComponent in all modals

Reviewed-by: Ernesto Puertat <epuertat@redhat.com>
Reviewed-by: Stephan Müller <smueller@suse.com>

mgr/orch: add pause/resume hooks

Signed-off-by: Sage Weil <sage@redhat.com>

Merge PR #33926 into octopus

* refs/pull/33926/head:
python: fixate mypy requirement to mypy==0.770

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge PR #33874 into octopus

* refs/pull/33874/head:
qa/suites/powercycle: install build deps for building xfstest

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Xiubo Li <xiubli@redhat.com>
Reviewed-by: Greg Farnum <gfarnum@redhat.com>

Merge PR #33813 into octopus

* refs/pull/33813/head:
osd/PGStateUtils: initialize NamedState::enter_time.
osd/PGStatueUtils: remove unused member variable

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge PR #33814 into octopus

* refs/pull/33814/head:
osd/OSDMap: stop encoding osd_state with >8 bits wide states only for old client

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: yanjun <yan.jun8@zte.com.cn>

Merge PR #33064 into octopus

* refs/pull/33064/head:
cephadm: add version to `command_ls` output
cephadm: add type checking to `update_filewalld`
cephadm: allow prepare-host to start an enabled service
cephadm: add type checking for `check_host` and `prepare_host`
cephadm: generalize logic for checking and enabling units
cephadm: add 'CEPH_CONF' to the NFS ganesha container envs
cephadm: trim nfs.json sample
qa/workunits/cephadm/test_cephadm.sh: systemctl stop nfs-server
qa/workunits/cephadm/test_cephadm.sh: make pgs available
cephadm: add some log lines
cephadm: check port in use
cephadm: add/remove nfs ganesha grace
cephadm: update firewalld with nfs service
qa/workunits/cephadm/test_cephadm.sh: add nfs-ganesha test
cephadm: add ganasha.conf
cephadm: add NFSGanesha deployment type
cephadm: consolidate list of supported daemons
cephadm: use keyword instead of positional args

Reviewed-by: Sebastian Wagner <swagner@suse.com>

Merge PR #33855 into octopus

* refs/pull/33855/head:
mgr/cephadm: reconfigure ceph daemons when monmap changes

Reviewed-by: Sebastian Wagner <swagner@suse.com>

Merge PR #33829 into octopus

* refs/pull/33829/head:
cephadm: log when we infer the image
cephadm: Avoid `inspect` call by using ceph label
cephadm: Infer ceph image

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #33919 from LenzGr/octopus-documentation

documentation: Updated Dashboard Features, improved flow

Reviewed-by: Kiefer Chang <kiefer.chang@suse.com>
Reviewed-by: Tatjana Dehler <tdehler@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>

Merge pull request #33887 from rhcs-dashboard/39714-decimal-places

mgr/dashboard: Pool read/write OPS shows too many decimal places

Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Stephan Müller <smueller@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>

Merge PR #33871 into octopus

* refs/pull/33871/head:
qa/suites/rados/verify: debug_ms = 1, osd_heartbeat_grace = 60

Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>
Reviewed-by: Brad Hubbard <bhubbard@redhat.com>

Merge PR #33890 into octopus

* refs/pull/33890/head:
doc/install: rethink docs install page (similar to PR#33613)
doc/install: Rethink doc/install

Reviewed-by: Sebastian Wagner <swagner@suse.com>

python: fixate mypy requirement to mypy==0.770

I don't like suprises when upstream publishes a new mypy version.

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>

mgr/dashboard: Append "Modal" to all modal components names

Fixes: https://tracker.ceph.com/issues/44547
Signed-off-by: Tiago Melo <tmelo@suse.com>

mgr/dashboard: Use ModalComponent in all modals

Fixes: https://tracker.ceph.com/issues/44547
Signed-off-by: Tiago Melo <tmelo@suse.com>

Merge pull request #33902 from dillaman/wip-test-rbd-integration

test/cli-integration: fixed spacing issue for RBD formatted tables

Reviewed-by: Mykola Golub <mgolub@suse.com>

mgr/cephadm: reconfigure ceph daemons when monmap changes

This keeps ceph.conf fresh.

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #33898 from dillaman/wip-44567

qa/workunits/rbd: wait for nbd map to close after unmap

Reviewed-by: Mykola Golub <mgolub@suse.com>

Merge pull request #33906 from tchaikov/wip-no-req-for-python2

mgr,python-common: drop modules for python2 in requirements.txt

Reviewed-by: Ernesto Puerta <epuertat@redhat.com>

cmake: use specified python for running node-gyp

to make sure node-gyp is able to find a compatible python3

see also
https://github.com/nodejs/node-gyp#configuring-python-dependency

Signed-off-by: Kefu Chai <kchai@redhat.com>

cephadm: add version to `command_ls` output

Signed-off-by: Michael Fritch <mfritch@suse.com>

cephadm: add type checking to `update_filewalld`

fixes mypy errors:
cephadm:1682: error: Incompatible types in assignment (expression has type "str", variable has type "int")
cephadm:1683: error: List item 3 has incompatible type "int"; expected "str"
cephadm:1686: error: List item 3 has incompatible type "int"; expected "str"
Found 3 errors in 1 file (checked 1 source file)

Signed-off-by: Michael Fritch <mfritch@suse.com>

cephadm: allow prepare-host to start an enabled service

the service might be enabled, but not running

Signed-off-by: Michael Fritch <mfritch@suse.com>

cephadm: add type checking for `check_host` and `prepare_host`

Signed-off-by: Michael Fritch <mfritch@suse.com>

cephadm: generalize logic for checking and enabling units

Signed-off-by: Michael Fritch <mfritch@suse.com>

cephadm: add 'CEPH_CONF' to the NFS ganesha container envs

Signed-off-by: Michael Fritch <mfritch@suse.com>

cephadm: trim nfs.json sample

we only need to specify the rados pool and config object url

Signed-off-by: Michael Fritch <mfritch@suse.com>

qa/workunits/cephadm/test_cephadm.sh: systemctl stop nfs-server

stop any running nfs servers before attempting a deploy of nfs-ganesha

Signed-off-by: Michael Fritch <mfritch@suse.com>

qa/workunits/cephadm/test_cephadm.sh: make pgs available

allow rados obj. create to succeed during a single node test

Signed-off-by: Michael Fritch <mfritch@suse.com>

cephadm: add some log lines

Signed-off-by: Michael Fritch <mfritch@suse.com>

cephadm: check port in use

Signed-off-by: Michael Fritch <mfritch@suse.com>

cephadm: add/remove nfs ganesha grace

container unit pre-start/post-stop for ganesha grace db

Signed-off-by: Michael Fritch <mfritch@suse.com>

cephadm: update firewalld with nfs service

Signed-off-by: Michael Fritch <mfritch@suse.com>

qa/workunits/cephadm/test_cephadm.sh: add nfs-ganesha test

Signed-off-by: Michael Fritch <mfritch@suse.com>

cephadm: add ganasha.conf

also allows for any arbitrary nfs conf to be deployed by adding items to
the `files` dict of the config-json

Signed-off-by: Michael Fritch <mfritch@suse.com>

cephadm: add NFSGanesha deployment type

Signed-off-by: Michael Fritch <mfritch@suse.com>

cephadm: consolidate list of supported daemons

Signed-off-by: Michael Fritch <mfritch@suse.com>

cephadm: use keyword instead of positional args

Signed-off-by: Michael Fritch <mfritch@suse.com>

mgr/rbd_support: fix recursive non-global level schedule listing

For the case when the non-global level does not have a schedule
and a higher level is used as the parent, it wrongly listed
schedules from all branches under the parent, instead of only the
interested one.

Signed-off-by: Mykola Golub <mgolub@suse.com>

mgr/dashboard: Updated octopus image on 404 page

Replaced the image of the Nautilus octopus with another octopus
in preparation for the "Octopus" release.

The image was taken from Museums Victoria
(https://collections.museumvictoria.com.au/species/8696) and is
licensed under the Creative Commons "Attribution 4.0 International"
(CC BY 4.0) license.

Deleted older, now obsolete images from the assets directory.

Fixes: https://tracker.ceph.com/issues/43581
Signed-off-by: Lenz Grimmer <lgrimmer@suse.com>

doc: Improved Dashboard installation workflow

Moved the "Accessing the Dashboard" section in front of
the paragraphs that talk about enabling additional services.

Signed-off-by: Lenz Grimmer <lgrimmer@suse.com>

doc: Updated dashboard feature overview for Octopus

Added new Dashboard features added for the Octopus
release.

Signed-off-by: Lenz Grimmer <lgrimmer@suse.com>

mgr/orch: Cleanup python imports of the interface

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>

mgr/orch: Properly handle NotImplementedError

They are masked as RuntimeError by the mgr.

Fixes: https://tracker.ceph.com/issues/44569
Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>

pybind/mgr: Rm PersistentStoreDict

`PersistentStoreDict` works great, except that is
too expensive to be used.

In addition, no one makes use of it.

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>

cephadm: log when we infer the image

This is similar to how we log when we infer the fsid.

Signed-off-by: Sage Weil <sage@redhat.com>

cephadm: Avoid `inspect` call by using ceph label

Signed-off-by: Ricardo Marques <rimarques@suse.com>

cephadm: Infer ceph image

Fixes: https://tracker.ceph.com/issues/44440
Signed-off-by: Ricardo Marques <rimarques@suse.com>

Merge PR #33897 into octopus

* refs/pull/33897/head:
mgr/PyModule: initialize options on standby class too
Revert "pybind/mgr/mgr_module: fix standby module logging options"

Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Sebastian Wagner <swagner@suse.com>

osd/PGStateUtils: initialize NamedState::enter_time.

This bug introduce by d96c907.

Signed-off-by: Jianpeng Ma <jianpeng.ma@intel.com>

osd/PGStatueUtils: remove unused member variable

Signed-off-by: Jianpeng Ma <jianpeng.ma@intel.com>

qa/workunits/rbd: wait for schedule status updated

Signed-off-by: Mykola Golub <mgolub@suse.com>

Merge PR #33870 into octopus

* refs/pull/33870/head:
mgr/cephadm: fix redeploy for the crash daemon
cephadm: prefer config/keyring args
cephadm: raise Error when no config/keyring
cephadm: consolidate logic for reading `config-json`
cephadm: merge `--config-and-keyring` and `--config-json` args

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Sebastian Wagner <swagner@suse.com>

Merge pull request #33842 from s1eke/master

doc: Fix link errors

Reviewed-by: Varsha Rao <varao@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

mgr/dashboard: Create bucket with x-amz-bucket-object-lock-enabled

Fixes: https://tracker.ceph.com/issues/43446
Signed-off-by: Volker Theile <vtheile@suse.com>

Merge pull request #33831 from dillaman/wip-44396

rbd-mirror: prevent I/O modifications against a non-primary image

Reviewed-by: Mykola Golub <mgolub@suse.com>

Merge pull request #32419 from rhcs-dashboard/fix-43419-master

mgr/dashboard: add feature toggle for NFS and fix feature toggles regression

Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Laura Paduano <lpaduano@suse.com>
Reviewed-by: Stephan Müller <smueller@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>

doc: Fix three link errors about mount cephfs

Some links about mount cephfs did not jump to the right place.

Signed-off-by: Sean Fang <silence.boy@live.cn>

mgr,python-common: drop modules for python2 in requirements.txt

we don't support python2 anymore, so no need to install them for
python2 anymore.

also, this helps to silence the messages from pip like

```
Ignoring pytest: markers 'python_version < "3.5"' don't match your environment
Ignoring mock: markers 'python_version < "3.3"' don't match your environment
```

Signed-off-by: Kefu Chai <kchai@redhat.com>

mgr/PyModule: initialize options on standby class too

Create a separate callback (_register_options) and call that on both
the regular MgrModule and MgrStandbyModule.

Fixes: https://tracker.ceph.com/issues/44562
Signed-off-by: Sage Weil <sage@redhat.com>

Merge PR #33817 into octopus

* refs/pull/33817/head:
mgr/dashboard: Adapt tests to new DriveGroupSpec
fixup mgr/test_orchestrator: validate drive group matches anything.
mgr/orch: CLI: No Tracebacks for ServiceSpecValidationError
mgr/test_orchestrator: validate drive group matches anything.
python-common: don't run flake8 on tests.
python-common: Add support for legacy serialization format for Drive Groups
doc: Move Move ServiceSpec to python-common
python-common: Add `host_pattern` to `PlacementSpec.from_string()`
cephadm: add host_pattern to supported scheduling
python-common: Joined ServiceSpec and DriveGroupSpec from_json()
python-common: Make DriveGroupSpec a sub type of ServiceSpec
pybind/mgr: Move ServiceSpec to python-common: Fix imports
python-common, orch: Move ServiceSpec to python-common: Fix imports
python-common, orch: Move ServiceSpec tests to python-common
python-common: Move ServiceSpec to python-common: fix linting
python-common, orch: Move ServiceSpec (+deps) to python-common

Reviewed-by: Joshua Schmid <jschmid@suse.de>

Merge PR #33895 into octopus

* refs/pull/33895/head:
cephadm: update unit.* atomically

Reviewed-by: Michael Fritch <mfritch@suse.com>

Merge pull request #32675 from rosinL/openssl-evp

crypto/openssl: update to use evp functions

Reviewed-by: Kefu Chai <kchai@redhat.com>

cephadm: update unit.* atomically

Some of these are run as bash scripts, which means that updating them
can lead to the running bash picking up at a weird position mid-script
when it goes to the next command.  This produces weird errors like

bash[9321]: /var/lib/ceph/f1758250-639e-11ea-9a42-001a4aab830c/mon.c/unit.run: line 2: -to-stderr=true: command not found

Signed-off-by: Sage Weil <sage@redhat.com>

test/cli-integration: fixed spacing issue for RBD formatted tables

The tables now use two spaces between columns instead of one.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>