doc/rbd: s/use use/use/

just a typo, really.

Signed-off-by: Peter Gervai <grin@grin.hu>

Merge pull request #33836 from xxhdx1985126/wip-crimson-osd-report-failure

crimson/heartbeat: report to monitors about osd failure

Reviewed-by: Kefu Chai <kchai@redhat.com>

crimson/os/heartbeat: report peer failure to monitors

Signed-off-by: Xuehan Xu <xxhdx1985126@163.com>

Merge pull request #33907 from tchaikov/wip-cmake-mgr-rook-client

cmake: rook-ceph-client related cleanups.

Reviewed-by: Sebastian Wagner <swagner@suse.com>

Merge pull request #33984 from optimistyzy/316_increase_que_size_upstream

NVMEDevice: set the queue_size of the ctrlr opts.

Reviewed-by: Kefu Chai <kchai@redhat.com>

NVMEDevice: set the queue_size of the ctrlr opts.

Purpose: Allocate more resoures for the requests in the io qpair,
and the real value will be determined by the inner logic in SPDK NVMe Driver.
But if we do not set it, the default size is 512, and the value is too low.
If there are many big I/Os with high QD submitted, with this setting,
it could improve the performance.

Signed-off-by: Ziye Yang <ziye.yang@intel.com>

Merge remote-tracking branch 'gh/octopus'

Merge PR #33979 into octopus

* refs/pull/33979/head:
cephadm: implement 'install' command

Reviewed-by: Michael Fritch <mfritch@suse.com>

Merge pull request #33988 from dillaman/wip-44391

rbd-mirror: hold lock while updating local image name

Reviewed-by: Mykola Golub <mgolub@suse.com>

Merge pull request #33856 from rhcs-dashboard/44545-fix-vstart-runner

vstart_runner.py: fix OSError when checking if non-existent path is mounted

Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

cephadm: implement 'install' command

This was present, but a no-op.

By default, install cephadm.

Optionally take a list of packages to install instead (e.g., ceph-common).

Signed-off-by: Sage Weil <sage@redhat.com>

rbd-mirror: hold lock while updating local image name

There is a potential for an independently scheduled status update to
request the local image name from the bootstrap state machine during its
initialization.

Fixes: https://tracker.ceph.com/issues/44391
Signed-off-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #33987 from tchaikov/wip-octopus-formatting

doc/releases/octopus.rst: formatting tweaks

Reviewed-by: Sage Weil <sage@redhat.com>

Merge PR #33980 into octopus

* refs/pull/33980/head:
cephadm: bootstrap: allow --output-dir
cephadm: do not infer image for bootstrap
cephadm: write output files to /etc/ceph by default
cephadm: verify the output files' containing directory exists

Reviewed-by: Michael Fritch <mfritch@suse.com>

doc/releases/octopus.rst: use note directive for notes

so the note stands out in the text

Signed-off-by: Kefu Chai <kchai@redhat.com>

doc/releases/octopus.rst: add '$' as prompt

to help the BashSession lexer to render the text starting with "#" as comments

see https://pygments.org/docs/lexers/#lexers-for-various-shells

Signed-off-by: Kefu Chai <kchai@redhat.com>

doc/releases/octopus: decrease the indent level

for less and better horizontal spacing

Signed-off-by: Kefu Chai <kchai@redhat.com>

doc/_static/css: fine tune the spacing

* `ul.simple > li` is also used in front page, where the items are too
  sparse now. so we need to restore the default spacing.
* `ul.simple > li > ul > li:last-child` is used to control the spacing
  between nested unordered list. we need have more spacing there.
* `div.section > ul > li > p` is added to decrease the spacing of nested
  list items.

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #33847 from tchaikov/wip-crimson-tell

crimson/osd: add tell command support

Reviewed-by: Ronen Friedman <rfriedma@redhat.com>

Merge pull request #33971 from tchaikov/octopus-doc-release

doc/releases/octopus.rst: format tweaks

Reviewed-by: Sage Weil <sage@redhat.com>

Merge PR #33968 into octopus

* refs/pull/33968/head:
mgr/cephadm: do not include 'osd' service in 'orch ls' output

Reviewed-by: Michael Fritch <mfritch@suse.com>

Merge PR #33970 into octopus

* refs/pull/33970/head:
python-common: 'all:true' -> '*'

Reviewed-by: Michael Fritch <mfritch@suse.com>

Merge PR #33973 into octopus

* refs/pull/33973/head:
mgr/orch,cephadm: elevate --force flag for daemons to interface
cephadm: backup data when we remove stateful daemons (mon, osd, prometheus)
cephadm: do not zap disk after removing an OSD daemon

Reviewed-by: Michael Fritch <mfritch@suse.com>

Merge PR #33935 into octopus

* refs/pull/33935/head:
mgr/cephadm: fix host connection exception
doc/cephadm: use ssh-copy-id

Reviewed-by: Joshua Schmid <jschmid@suse.de>

doc/releases/octopus.rst: reindent top-level items

reduce the indent level, so these items are better formated

Signed-off-by: Kefu Chai <kchai@redhat.com>

doc/releases/octopus.rst: formatting tweaks

* use console lexer for highlighting command lines
  otherwise they will be rendered as python code
* use bold font instead italic in subsection headers,
  they look more formal this way
* fix a typo: delineated

Signed-off-by: Kefu Chai <kchai@redhat.com>

doc/releases: fix hyper link

Signed-off-by: Kefu Chai <kchai@redhat.com>

doc/release: fix "CephFS" subsection heading

it's a regression introduced by 26db06cf212be029e5628a40de3410632c0b86a0

Signed-off-by: Kefu Chai <kchai@redhat.com>

cephadm: bootstrap: allow --output-dir

This is easier than specifying each of output-config, output-keyring, and
output-pub-ssh-key.

Signed-off-by: Sage Weil <sage@redhat.com>

cephadm: do not infer image for bootstrap

Signed-off-by: Sage Weil <sage@redhat.com>

cephadm: write output files to /etc/ceph by default

This streamlines the install for most users by putting the config and
keyring files in /etc/ceph, where other ceph CLI commands will find them.

The cost is making isolated installs a bit more painful, by requiring
that users specify a local location for --output-config etc.

Signed-off-by: Sage Weil <sage@redhat.com>

cephadm: verify the output files' containing directory exists

Signed-off-by: Sage Weil <sage@redhat.com>

Merge PR #33977 into octopus

* refs/pull/33977/head:
pybind/mgr: apply_drivegroups should return Sequence[Completion]

Reviewed-by: Sage Weil <sage@redhat.com>

crimson/admin: revise doxygen docs

Signed-off-by: Kefu Chai <kchai@redhat.com>

crimson/admin: drop unused functions

Signed-off-by: Kefu Chai <kchai@redhat.com>

crimson/osd: add tell command support

* change the function signature of `call()` to accept a bufferlist,
  as MCommand is supposed to carry a bufferlist which can be consumed
  by the hook.
* drop the wrapper of `tell()`, instead always use `call()`. simpler
  this way.
* promote `tell_result_t` to top level of `crimson::admin` as it is used
  as part of the interface between hook and `AdminSocket`
* replace `maybe_parsed_t` with `std::variant<parsed_command_t, tell_result_t>`,
  as the latter is able to convey more detailed error information if
  something goes wrong.
* osd now handles `MCommand`.

Signed-off-by: Kefu Chai <kchai@redhat.com>

crimson/asok: pass vector<string> to parse_cmd()

prepare for the tell command, as the payload of MCommand is a
`vector<string>`, and `cmdmap_from_json()` expects a `vector<string>`

Signed-off-by: Kefu Chai <kchai@redhat.com>

pybind/mgr: apply_drivegroups should return Sequence[Completion]

* change annotation of `orchestrator.Orchestrator.apply_drivegroups()` so it returns
  `Sequence[Completion]`
  see
  https://mypy.readthedocs.io/en/latest/generics.html#variance-of-generic-types
* to follow the interface defined by `orchestrator.Orchestrator`. change
  all its implementations. see `orchestrator/_interface.py`.

Signed-off-by: Kefu Chai <kchai@redhat.com>

mgr/orch,cephadm: elevate --force flag for daemons to interface

Apply this check in the generic orchestrator code so that it applies
consistently to all implementations.

Signed-off-by: Sage Weil <sage@redhat.com>

cephadm: backup data when we remove stateful daemons (mon, osd, prometheus)

This avoids doing permanent damage when removing stateful daemons.

We include OSDs here even though in most cases no precious data lives in
the osd data directory because there might be some important metadata
there--particularly for adopted OSDs from older clusters.

Signed-off-by: Sage Weil <sage@redhat.com>

cephadm: do not zap disk after removing an OSD daemon

Signed-off-by: Sage Weil <sage@redhat.com>

Merge PR #33922 into octopus

* refs/pull/33922/head:
qa/tasks/mgr/test_orchestrator_cli: adapt
qa/suites/rados/cephadm/upgrade: start from compatible build
mgr/cephadm: refine log message
python-common: correctly dictify DriveGroupSpec objects when paths are used
mgr/cephadm: adapt test_cephadm.sh to new syntax
mgr/cephadm: adapt qa->test_cephadm.sh
qa/tasks/cephadm: use 'orch daemon osd add'
mgr/cephadm: adapt osd deployment to service_apply

Reviewed-by: Sage Weil <sage@redhat.com>

qa/tasks/mgr/test_orchestrator_cli: adapt

Signed-off-by: Sage Weil <sage@redhat.com>

qa/suites/rados/cephadm/upgrade: start from compatible build

The osd create command is changing, so upgrade from something that the
latest cephadm.py task can use.

Signed-off-by: Sage Weil <sage@redhat.com>

mgr/cephadm: fix host connection exception

1- catch exception thrown by _get_connection itself
2- make the error message more informative.  the old one was

Error ENOENT: New host dael (10.3.64.25) failed to connect: `ssh -F /tmp/cephadm-conf-9tvrgrls -i /tmp/cephadm-identity-sj66ado1 root@10.3.64.25`

which was both misleading and had irrelevant information.  Now,

Error ENOENT: Failed to connect to dael (10.3.64.25).  Check that the host is reachable and accepts connections using the cephadm SSH key

Signed-off-by: Sage Weil <sage@redhat.com>

doc/cephadm: use ssh-copy-id

Signed-off-by: Sage Weil <sage@redhat.com>

python-common: 'all:true' -> '*'

'all:true' is no longer a valid PlacementSpec.

Signed-off-by: Sage Weil <sage@redhat.com>

mgr/cephadm: do not include 'osd' service in 'orch ls' output

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #33828 from dillaman/wip-cls-rbd-mirror-status-summary

cls/rbd: use explicitly sized integer in mirror status summary

Reviewed-by: Mykola Golub <mgolub@suse.com>

Merge remote-tracking branch 'gh/octopus' into octopus

Merge PR #33520 into octopus

* refs/pull/33520/head:
mgr/dashboard: Isolate each RBD component

Reviewed-by: Stephan Müller <smueller@suse.com>

Merge PR #33758 into octopus

* refs/pull/33758/head:
mgr/dashboard: Replace ng2-tree with angular-tree-component

Reviewed-by: Stephan Müller <smueller@suse.com>
Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Laura Paduano <lpaduano@suse.com>

Merge PR #33941 into octopus

* refs/pull/33941/head:
mgr/dashboard: hide daemon table when orchestrator is disabled

Reviewed-by: Volker Theile <vtheile@suse.com>

Merge PR #33948 into octopus

* refs/pull/33948/head:
python-common: PlacementSpec.from_string: drop `host_pattern:` prefix
python-common: remove `all_hosts` from `PlacementSpec`

Reviewed-by: Sage Weil <sage@redhat.com>

mgr/cephadm: refine log message

Avoid a lot of these

2020-03-13T14:58:38.785360-0500 mgr.x [INF] No data_devices, skipping DriveGroup: osd.all

every time we consider the OSD spec and find nothing to do.  Instead, log
something like this each time we actually have work to do:

2020-03-13T15:04:22.803356-0500 mgr.x [INF] Applying osd.all on host gnit...
2020-03-13T15:04:33.723731-0500 mgr.x [INF] Deploying daemon osd.5 on gnit

Signed-off-by: Sage Weil <sage@redhat.com>

Merge PR #33777 into octopus

* refs/pull/33777/head:
rgw: svc.config_key_rados: get() warns if potentially insecure connection
rgw: svc.rados: add clog_warn()
mon: log monitor: add command level param
auth registry: helpers for checking secure method/mode

Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: Sage Weil <sage@redhat.com>

Merge PR #33950 into octopus

* refs/pull/33950/head:
ceph.in: use os._exit when we don't shut down

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge PR #33951 into octopus

* refs/pull/33951/head:
mgr/orch: dump service spec by name

Reviewed-by: Joshua Schmid <jschmid@suse.de>
Reviewed-by: Sebastian Wagner <swagner@suse.com>

ceph.in: use os._exit when we don't shut down

If we experience a timeout, we don't bother shutting down rados, because
it can't currently handle one thread blocked (or running) library init
at the same time that another thread calls rados_shutdown().  This
behavior was recently introduced by c8f353c50bcd2a30be9e3600dba91912b8cd0429

However, sys.exit() runs all kinds of shutdown work that will also
interfere with the running librados threads.

Fix by using os._exit instead of sys.exit.

Fixes: https://tracker.ceph.com/issues/44566
Signed-off-by: Sage Weil <sage@redhat.com>

15.1.1

mgr/orch: dump service spec by name

add optional arg to allow a dump by name:
`ceph orch spec dump [svc_name]`

Signed-off-by: Michael Fritch <mfritch@suse.com>

Merge PR #33953 into octopus

* refs/pull/33953/head:
doc: Move ceph-deploy docs to doc/install/ceph-deploy

Reviewed-by: Sage Weil <sage@redhat.com>

doc: Move ceph-deploy docs to doc/install/ceph-deploy

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>

Merge PR #33924 into octopus

* refs/pull/33924/head:
doc/mgr: Enhance placement specs
doc/cephadm: Fix toc of Drive Groups

Reviewed-by: Sage Weil <sage@redhat.com>

Merge PR #33891 into octopus

* refs/pull/33891/head:
mgr/cephadm: fix dump output by formatting to json first

Reviewed-by: Sebastian Wagner <swagner@suse.com>
Reviewed-by: Sage Weil <sage@redhat.com>

Merge PR #33905 into octopus

* refs/pull/33905/head:
install-deps.sh: remove the workaround to bootstrap a new venv
install-deps.sh: install additional repo for RHEL8
install-deps.sh: install rhel-7-server-optional-rpms on el7 only

Reviewed-by: David Galloway <dgallowa@redhat.com>

Merge pull request #33653 from TRYTOBE8TME/wip-rgw-sts-doc-correction

rgw: STS Lite documentation correction

Reviewed-by: Pritha Srivastava <prsivas@redhat.com>

python-common: correctly dictify DriveGroupSpec objects when paths are used

Signed-off-by: Joshua Schmid <jschmid@suse.de>

Merge pull request #33940 from tchaikov/octopus-release-notes-header

doc: use subsection for representing components in release notes

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #33921 from tchaikov/wip-cmake-node-gyp

cmake: use specified python for running node-gyp

Reviewed-by: Laura Paduano <lpaduano@suse.com>

doc/mgr: Enhance placement specs

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>

doc/releases: use h3 for rendering subsections in release notes

to highlight the components with subsection header, and allow readers to
navigate through them with TOC.

Signed-off-by: Kefu Chai <kchai@redhat.com>

doc: overide the default margin for "ul.simple > li"

by default, the user agent's css sets `margin-block-end` of "ul ul" to
"0", which renders the unordered lists in release notes cluttered. as we
are using nested unordered list to present the changes in different
components.

in this change a customized css is added to reset the begin and end
margin of ul to 1em, which is the default value of top level ul.

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge remote-tracking branch 'gh/octopus'

python-common: PlacementSpec.from_string: drop `host_pattern:` prefix

* `host_pattern` is works as any other host spec.
* Improved type annotations
* Added tests
* `mgr/cephadm/test_scheduling`: Remove tests that don't have a valid spec.

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>

mgr/cephadm: adapt test_cephadm.sh to new syntax

Signed-off-by: Joshua Schmid <jschmid@suse.de>

mgr/cephadm: adapt qa->test_cephadm.sh

Signed-off-by: Joshua Schmid <jschmid@suse.de>

qa/tasks/cephadm: use 'orch daemon osd add'

Signed-off-by: Sage Weil <sage@redhat.com>

mgr/cephadm: adapt osd deployment to service_apply

Signed-off-by: Joshua Schmid <jschmid@suse.de>

Merge pull request #33918 from tchaikov/wip-seastar

seastar: pickup change to always enable smp poller

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

Merge pull request #33867 from dillaman/wip-43590

librbd: optimize image copy state machine to use fast-diff

Reviewed-by: Mykola Golub <mgolub@suse.com>

mgr/dashboard: hide daemon table when orchestrator is disabled

The refresh time of services/daemons is also increased to 1 minute since
they are not updated so frequently.

Fixes: https://tracker.ceph.com/issues/44558
Signed-off-by: Kiefer Chang <kiefer.chang@suse.com>

Merge PR #33930 into octopus

* refs/pull/33930/head:
mgr/cephadm: implement pause/resume to suspect non-monitoring background work
mgr/orch: add pause/resume hooks

Reviewed-by: Michael Fritch <mfritch@suse.com>

Merge PR #33868 into octopus

* refs/pull/33868/head:
mgr/rook: implement remove_daemons()
mgr/rook: fix mds/fs create/apply/update
mgr/rook: use ceph_daemon_id label consistently
mgr/rook: show filesystems in 'orch ls'
mgr/orch: 'age' -> 'ago' in 'orch ls' output
mgr/rook: initial pass at 'orch ls'
mgr/rook: include timestamps in 'orch ps'
mgr/orch: shorten container_id (hash) in cephadm, not orch ps
mgr/rook: include container_image_name in 'orch ps'
mgr/rook: fix list_daemons host arg

Reviewed-by: Juan Miguel Olmo <jolmomar@redhat.com>
Reviewed-by: Sebastian Wagner <swagner@suse.com>

Merge PR #33932 into octopus

* refs/pull/33932/head:
cephadm: fix datetime regexp to capture at most 6 digits

Reviewed-by: Sage Weil <sage@redhat.com>

seastar: pickup change to always enable smp poller

this is the first commit which is in sync with upstream without any
addition change in our branch.

Signed-off-by: Kefu Chai <kchai@redhat.com>

crimson/mgr: include necessary header

Signed-off-by: Kefu Chai <kchai@redhat.com>

mgr/dashboard: Isolate each RBD component

Before when you accessed any RBD component, you were loading all 4 components
at the same time and each was doing frequent requests to the server.

Now it only loads 1 at a time.

Fixes: https://tracker.ceph.com/issues/44278
Signed-off-by: Tiago Melo <tmelo@suse.com>

Merge pull request #33900 from dillaman/wip-44459

qa/suites/rbd: removed OpenStack tempest test cases

Reviewed-by: Mykola Golub <mgolub@suse.com>

Merge pull request #33840 from trociny/wip-schedule-optimize-scan

mgr/rbd_support: don't scan pools that don't have schedules

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge PR #33914 into octopus

* refs/pull/33914/head:
mgr/orch: Cleanup python imports of the interface
mgr/orch: Properly handle NotImplementedError
pybind/mgr: Rm PersistentStoreDict

Reviewed-by: Sage Weil <sage@redhat.com>

mgr/rook: implement remove_daemons()

Signed-off-by: Sage Weil <sage@redhat.com>

mgr/dashboard: Replace ng2-tree with angular-tree-component

Fixes: https://tracker.ceph.com/issues/44450
Signed-off-by: Tiago Melo <tmelo@suse.com>

mgr/rook: fix mds/fs create/apply/update

Signed-off-by: Sage Weil <sage@redhat.com>

mgr/rook: use ceph_daemon_id label consistently

In particular, for mds's, it's $fsname-$rank for mds.$fsname-$rank,
which is what we want.

Signed-off-by: Sage Weil <sage@redhat.com>

mgr/rook: show filesystems in 'orch ls'

The count vs size discrepancy is weird since rook things in terms of
ranks (*2 if standby) vs just a count.

Signed-off-by: Sage Weil <sage@redhat.com>

mgr/orch: 'age' -> 'ago' in 'orch ls' output

Signed-off-by: Sage Weil <sage@redhat.com>

mgr/rook: initial pass at 'orch ls'

This doesn't cover mds.* or rgw.* yet.

Signed-off-by: Sage Weil <sage@redhat.com>

mgr/rook: include timestamps in 'orch ps'

Signed-off-by: Sage Weil <sage@redhat.com>

mgr/orch: shorten container_id (hash) in cephadm, not orch ps

This lets us present a longer, human-readable id for rook.

Signed-off-by: Sage Weil <sage@redhat.com>