mds/MDSDaemon: fix asok 'exit' and 'respawn' commands

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #32226 from neha-ojha/wip-four-percent

doc/rados: Better block.db size recommendations for bluestore

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Igor Fedotov <ifedotov@suse.com>

doc/rados: Better block.db size recommendations for bluestore

Signed-off-by: Neha Ojha <nojha@redhat.com>

Merge pull request #32040 from smithfarm/wip-rn-13-2-8-mimic

doc/releases: add release notes for mimic v13.2.8

Reviewed-by: Abhishek Lekshmanan <abhishek@suse.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>

doc/releases: add Mimic v13.2.8 to releases timeline

Signed-off-by: Nathan Cutler <ncutler@suse.com>

Merge PR #32236 into master

* refs/pull/32236/head:
pybind/mgr: add_tox_test: Add mypy to TOX_ENVS

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #32005 from s0nea/wip-dashboard-reactivate-suites

mgr/dashboard: reactivate dashboard test suites

Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Laura Paduano <lpaduano@suse.com>

pybind/mgr: add_tox_test: Add mypy to TOX_ENVS

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>

Merge pull request #31851 from votdev/issue_42609_device_lights

mgr/dashboard: Add support for blinking enclosure LEDs

Reviewed-by: Lenz Grimmer <lgrimmer@suse.com>
Reviewed-by: Kiefer Chang <kiefer.chang@suse.com>
Reviewed-by: Tiago Melo <tmelo@suse.com>

Merge pull request #32186 from rhcs-dashboard/43263-fix-rgw-subuser-key

mgr/dashboard: fix RGW subuser auto-generate key.

Reviewed-by: Ernesto Puertat <epuertat@redhat.com>
Reviewed-by: Volker Theile <vtheile@suse.com>
Reviewed-by: Laura Paduano <lpaduano@suse.com>

Merge pull request #31753 from smithfarm/wip-bci-resolve-fix

backport-create-issue: resolve parent only if parent has backport issues

Reviewed-by: Tiago Melo <tmelo@suse.com>

mgr/dashboard: Simplify PrometheusService::getRules implementa… (#32130)

mgr/dashboard: Simplify PrometheusService::getRules implementation

Reviewed-by: Stephan Müller <smueller@suse.com>
Reviewed-by: Tiago Melo <tmelo@suse.com>

Merge pull request #32177 from Takios/ceph_volume_fix

ceph-volume: use correct extents if using db-devices and >1 osds_per_device

mgr/dashboard: reactivate dashboard test suites

Reactivate the dashboard test suites that were commented
out in https://github.com/ceph/ceph/pull/30864 because
https://tracker.ceph.com/issues/41538 has been resolved
in the meanwhile.

Fixes: https://tracker.ceph.com/issues/42652
Signed-off-by: Tatjana Dehler <tdehler@suse.com>

Merge pull request #32224 from tspmelo/wip-e2e-insecure

mgr/dashboard: Fix e2e chromedriver problem

Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Kiefer Chang <kiefer.chang@suse.com>
Reviewed-by: Laura Paduano <lpaduano@suse.com>

Merge PR #31255 into master

* refs/pull/31255/head:
mds/MDSDaemon: make con disposable before passing message to asok
mds/SessionMap: interpret bare id as a valid filter
mds: get rid of unused send_command_reply helpers
mds: remove old handle_command machinery
mds/MDSRank: remove C_ExecAndReply helpers
mds/MDSRank: too many words about op_tracker
mds: convert 'cache drop'
mds: convert 'damage ls' and 'damage rm'
mds: convert '{session,client} config'
mds: convert 'scrub {start,abort,pause,resume,status}'
mds: convert 'session ls' and 'client ls'
mds: convert 'session evict' and 'session kill'
mds: convert 'heap' and 'cpu_profiler'
mds: convert 'exit', 'respawn'
mds/MDSDaemon: remove old duplicated tell commands
mds: send tell commands to asok infrastructure
common/admin_socket: switch tell queue to cref_t (from ref_t)
mds: switch asok calls over to call_async interface

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #32230 from shellwedance/ceph-volume_typo_fix

doc: fix a typo in a command

Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge pull request #32170 from rosinL/wip-rss

msg/async/DPDK: refactor set_rss_table to support DPDK 19.05

Reviewed-by: Kefu Chai <kchai@redhat.com>

doc: fix a typo in a command

Co-authored-by: donggyu_park <donggyu_park@tmax.co.kr>
Signed-off-by: taeuk_kim <taeuk_kim@tmax.co.kr>

Merge PR #32179 into master

* refs/pull/32179/head:
lvm: add deactivate subcommand
util/system: add unmount_tmpfs helper
api/lvm: add get_lv_by_osd_id method
api/lvm: add deactivate method to Volume class

Reviewed-by: Sage Weil <sage@redhat.com>

Merge PR #32225 into master

* refs/pull/32225/head:
qa: add force-branch to suites running s3readwrite & s3roundtrip tasks

Reviewed-by: Yuri Weinstein <yweins@redhat.com>
Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge PR #32003 into master

* refs/pull/32003/head:
mgr/ssh fix tests for update_mds/mgr/mds/rgw/rbd-mirror
Allow List[HostSpec] in PlacementSpec
mgr/rook: adapt to arg passing change in orchestrator_cli.py
mgr/ssh: Add SimpleScheduler and streamline arg passing
orch_cli: adapt to changes in PlacementSpec and ServiceSpec
orch: extend PlacementSpec and add StatefulServiceSpec

Reviewed-by: Sebastian Wagner <swagner@suse.com>

Merge PR #32218 into master

* refs/pull/32218/head:
cephadm: use raw string during `re.sub`
cephadm: fix ConfigParser DeprecationWarning in py3

Reviewed-by: Sage Weil <sage@redhat.com>

Merge PR #32187 into master

* refs/pull/32187/head:
ceph.in: (py3) use binary stdout
pybind/mgr: index=4, sort_keys=True for json.dumps throughput

Reviewed-by: Sebastian Wagner <swagner@suse.com>
Reviewed-by: Alfredo Deza <adeza@redhat.com>

Merge pull request #32013 from liewegas/doc-foundation

doc/foundation: add cloudbase and vexxhost

Merge pull request #32197 from neha-ojha/wip-report-doc-bug

doc/_templates/page.html: redirect to etherpad

Reviewed-by: Zac Dover zac.dover@gmail.com
Reviewed-by: Sage Weil sage@redhat.com

qa: add force-branch to suites running s3readwrite & s3roundtrip tasks

Signed-off-by: Ali Maredia <amaredia@redhat.com>

Merge PR #32220 into master

* refs/pull/32220/head:
qa/workunits/cephadm/test_cephadm.sh: need --fsid always

Reviewed-by: Michael Fritch <mfritch@suse.com>

mds/MDSDaemon: make con disposable before passing message to asok

Signed-off-by: Sage Weil <sage@redhat.com>

mds/SessionMap: interpret bare id as a valid filter

This works around the fact that the old asok 'session evict' took a bare
id and the tell version took a filter.

Signed-off-by: Sage Weil <sage@redhat.com>

mds: get rid of unused send_command_reply helpers

Signed-off-by: Sage Weil <sage@redhat.com>

mds: remove old handle_command machinery

Signed-off-by: Sage Weil <sage@redhat.com>

mds/MDSRank: remove C_ExecAndReply helpers

Signed-off-by: Sage Weil <sage@redhat.com>

mds/MDSRank: too many words about op_tracker

Signed-off-by: Sage Weil <sage@redhat.com>

mds: convert 'cache drop'

Signed-off-by: Sage Weil <sage@redhat.com>

mds: convert 'damage ls' and 'damage rm'

Signed-off-by: Sage Weil <sage@redhat.com>

Merge PR #31629 into master

* refs/pull/31629/head:
cephfs-shell: rearrange code for convenience

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

mgr/dashboard: Fix e2e chromedriver problem

New version of Chrome/ChromeDriver seems to have a problem with insecure
certificates.

Since we use a vStart cluster with no real certificate,
we always face this problem when running e2e tests.

This will fix the current problem and hopefully any future problems related to
certificates.

Fixes: https://tracker.ceph.com/issues/43254
Signed-off-by: Tiago Melo <tmelo@suse.com>

qa/workunits/cephadm/test_cephadm.sh: need --fsid always

Otherwise the test breaks if there are other cluster(s) on the host (like
there normally are in my dev environment).

Signed-off-by: Sage Weil <sage@redhat.com>

backport-create-issue: resolve parent only if parent has backport issues

This fixes an edge case. If a parent issue is in "Pending Backport" status
but without any backport issues, the script (when running with --resolve-parent)
was wrongly changing the status to "Resolved".

Signed-off-by: Nathan Cutler <ncutler@suse.com>

Merge PR #32129 into master

* refs/pull/32129/head:
qa: Enable flake8 tox and fix failures

ceph.in: (py3) use binary stdout

This fixes an error like

$ ceph -s -o -
Traceback (most recent call last):
  File "bin/ceph", line 1267, in <module>
    retval = main()
  File "bin/ceph", line 1229, in main
    outf.write(outbuf)
TypeError: write() argument must be str, not bytes

Signed-off-by: Sage Weil <sage@redhat.com>

pybind/mgr: index=4, sort_keys=True for json.dumps throughput

- fixes balancer test on py3 by making the pool ls order deterministic
- makes everybody else behave consistently

I left internal json storage alone--no need to sort/normalize.  I also
left out some CLI commands returning non-indented json.

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #32094 from cbodley/wip-40585

test/rgw: raise timer durations for unittest_rgw_reshard_wait

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #32153 from smithfarm/wip-23981

install-deps.sh: install python2-{virtualenv,devel} on SUSE if needed

Reviewed-by: Thomas Bechtold <tbechtold@suse.com>

cephadm: use raw string during `re.sub`

cephadm:198: DeprecationWarning: invalid escape sequence \s
  nice_conf = re.sub('\n(\s)+', '\n', raw_conf)

Signed-off-by: Michael Fritch <mfritch@suse.com>

cephadm: fix ConfigParser DeprecationWarning in py3

DeprecationWarning: This method will be removed in future versions.
Use 'parser.read_file()' instead.

Signed-off-by: Michael Fritch <mfritch@suse.com>

Merge pull request #32151 from liewegas/fix-scipy

install-deps.sh: preload wheel for all mgr requirements.txt files

Reviewed-by: Laura Paduano <lpaduano@suse.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

lvm: add deactivate subcommand

This new subcommand unmounts and OSDs tmpfs mount and closes crypt
devices if there are any.

Signed-off-by: Jan Fajerski <jfajerski@suse.com>

util/system: add unmount_tmpfs helper

Thsi unmounts a path if and only if it's a tmpfs mount.

Signed-off-by: Jan Fajerski <jfajerski@suse.com>

msg/async/dpdk: refactor set_rss_table to support DPDK 19.05

The filter API was deprecated in DPDK 19.05, using the flow
API to set rss table.

Signed-off-by: Chunsong Feng <fengchunsong@huawei.com>
Signed-off-by: luo rixin <luorixin@huawei.com>

mgr/dashboard: Add support for blinking enclosure LEDs

Fixes: https://tracker.ceph.com/issues/42609
Signed-off-by: Volker Theile <vtheile@suse.com>

install-deps.sh: install python2-{virtualenv,devel} on SUSE if needed

Recent versions of openSUSE are fully Python 3 systems, but Python 2 can still
be installed on them. When Python 2 *and* Python 3 are installed, tox will try
to run unit tests ("make check" tests) on both. But the spec-file-based
mechanism used by install-deps.sh for determining "make check" dependencies only
installs python3-virtualenv and python3-devel.

Fixes: https://tracker.ceph.com/issues/23981
Signed-off-by: Nathan Cutler <ncutler@suse.com>

Merge pull request #32172 from wjwithagen/wjw-fix-test_cls_rbd-compat

test/cls_rbd: include compat.h for ERESTART

Reviewed-by: Mykola Golub <mgolub@suse.com>

Merge pull request #32182 from sebastian-philipp/upstream-pull-31757

mgr/test_orchestrator: Add dummy data

Reviewed-by: Volker Theile <vtheile@suse.com>

Merge pull request #31920 from ricardoasmarques/fix-sso-certs

mgr/dashboard: Invalid SSO configuration when certificate path does not exist

Reviewed-by: Ernesto Puerta <epuertat@redhat.com>

mgr/dashboard: Use badges for counters in tabs (#31938)

mgr/dashboard: Use badges for counters in tabs

Reviewed-by: Stephan Müller <smueller@suse.com>
Reviewed-by: Tiago Melo <tmelo@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>

Merge pull request #31787 from shyukri/wip-36728-c_v-util-which-path

ceph-volume: util: look for executable in $PATH

mgr/ssh fix tests for update_mds/mgr/mds/rgw/rbd-mirror

Signed-off-by: Joshua Schmid <jschmid@suse.de>

Allow List[HostSpec] in PlacementSpec

Signed-off-by: Joshua Schmid <jschmid@suse.de>

qa: Enable flake8 tox and fix failures

There were a couple of problems found by flake8 in the qa/
directory (most of them fixed now). Enabling flake8 during the usual
check runs hopefully avoids adding new issues in the future.

Signed-off-by: Thomas Bechtold <tbechtold@suse.com>

Merge pull request #32080 from smithfarm/wip-43171

ceph.spec.in: move distro-conditional deps to dedicated section

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Brad Hubbard <bhubbard@redhat.com>

mgr/rook: adapt to arg passing change in orchestrator_cli.py

Signed-off-by: Joshua Schmid <jschmid@suse.de>

mgr/ssh: Add SimpleScheduler and streamline arg passing

* adding a mechanism to augment specs.placement.nodes
  even when no nodes are provided via dynamic determination
  from host_pools. (NodeAssignment)

* added SimpleScheduler
  implements most simple placment algorithm possible.
  Choose from a shuffled list of hosts.

* streamlined args passed to the public update_$service
  functions. They all take `spec` now.

Signed-off-by: Joshua Schmid <jschmid@suse.de>

orch_cli: adapt to changes in PlacementSpec and ServiceSpec

* adapt to changes in PlacementSpec (count moved to PlacementSpec)
* added label param to update_& functions

Signed-off-by: Joshua Schmid <jschmid@suse.de>

orch: extend PlacementSpec and add StatefulServiceSpec

* add public set_nodes()
* add StatefulServiceSpec

Signed-off-by: Joshua Schmid <jschmid@suse.de>

Merge pull request #32175 from rzarzynski/wip-crimson-errorator-do_with-friendship

crimson: make seastar::do_with() a friend of errorated futures.

Reviewed-by: Liu-Chunmei <chunmei.liu@intel.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

debian/control: install scipy for build too

Needed for make check

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #32143 from tchaikov/wip-qa-rocksdb-test

qa/workunits/rados/test_envlibrados_for_rocksdb: accomodate rocksdb c…

Reviewed-by: Neha Ojha <nojha@redhat.com>

cephfs-shell: rearrange code for convenience

Signed-off-by: Rishabh Dave <ridave@redhat.com>

doc/_templates/page.html: redirect to etherpad

This simplifies things for users who are not able to add cards to
https://github.com/ceph/ceph/projects/4 due to insufficient permissions.

Signed-off-by: Neha Ojha <nojha@redhat.com>

Merge pull request #32099 from hjwsm1989/wip-cancel-invalid-upmap-2

crush: remove invalid upmap items

Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>
Reviewed-by: David Zafman <dzafman@redhat.com>

Merge PR #32193 into master

* refs/pull/32193/head:
qa/tasks/ceph2 -> cephadm
mgr/ssh -> mgr/cephadm
mgr/ssh,cephadm: ceph-daemon -> cephadm
doc/bootstrap: ceph-daemon -> cephadm
ceph-daemon-adoption-corpus -> cephadm-adoption-corpus
qa: rename ceph-daemon tests -> cephadm
qa/tasks: ceph-daemon -> cephadm throughput var names and comments
mgr/ssh,qa/tasks/ceph2: fix mode to be cephadm-package (vs root)
mgr/ssh: rename option ceph_daemon_path -> cephadm_path
files,rpm,deb: rename ceph-daemon -> cephadm

Reviewed-by: Michael Fritch <mfritch@suse.com>
Reviewed-by: Sebastian Wagner <swagner@suse.com>

qa/tasks/ceph2 -> cephadm

Signed-off-by: Sage Weil <sage@redhat.com>

mgr/ssh -> mgr/cephadm

Signed-off-by: Sage Weil <sage@redhat.com>

mgr/ssh,cephadm: ceph-daemon -> cephadm

Signed-off-by: Sage Weil <sage@redhat.com>

doc/bootstrap: ceph-daemon -> cephadm

Signed-off-by: Sage Weil <sage@redhat.com>

ceph-daemon-adoption-corpus -> cephadm-adoption-corpus

Signed-off-by: Sage Weil <sage@redhat.com>

qa: rename ceph-daemon tests -> cephadm

Also move the workunit to a better location.

Signed-off-by: Sage Weil <sage@redhat.com>

qa/tasks: ceph-daemon -> cephadm throughput var names and comments

Signed-off-by: Sage Weil <sage@redhat.com>

mgr/ssh,qa/tasks/ceph2: fix mode to be cephadm-package (vs root)

At the same time align the option names with ceph2.py, yay.

Signed-off-by: Sage Weil <sage@redhat.com>

mgr/ssh: rename option ceph_daemon_path -> cephadm_path

Signed-off-by: Sage Weil <sage@redhat.com>

files,rpm,deb: rename ceph-daemon -> cephadm

This is just renaming the files and adjusting the packages.  Lots of
cleanup to do still.

Signed-off-by: Sage Weil <sage@redhat.com>

Merge PR #32189 into master

* refs/pull/32189/head:
mgr/orchestrator_cli: set type for 'orchestrator' option

Reviewed-by: Sebastian Wagner <swagner@suse.com>

Merge PR #32157 into master

* refs/pull/32157/head:
ceph-daemon: fix pod stop

Reviewed-by: Sebastian Wagner <swagner@suse.com>
Reviewed-by: Michael Fritch <mfritch@suse.com>

python-common: Also test mgr/test_orchestrator/dummy_data.json

Make sure, `pybind/mgr/test_orchestrator/dummy_data.json` is not getting out of date.

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>

mgr/orchestrator_cli: set type for 'orchestrator' option

If we don't specify the type, the mgr_module.py will str() the default
value, turning it into "None", breaking the self-test.

Signed-off-by: Sage Weil <sage@redhat.com>

Merge PR #31729 into master

* refs/pull/31729/head:
qa: reduce cache size further
mds: obsoleting 'mds_cache_size'

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge PR #31718 into master

* refs/pull/31718/head:
mds: Reorganize class members in ScrubStack header

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge PR #31717 into master

* refs/pull/31717/head:
mds: Reorganize class members in ScrubHeader header

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge PR #31716 into master

* refs/pull/31716/head:
mds: Reorganize class members in ScatterLock header

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge PR #31635 into master

* refs/pull/31635/head:
mds: Reorganize class members in RecoveryQueue header

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge PR #31597 into master

* refs/pull/31597/head:
mds: Reorganize class members in OpenFileTable header

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge PR #30201 into master

* refs/pull/30201/head:
qa/tasks: remove subvolume, subvolumegroup and their snapshots with --force
qa/tasks: Fix the commands success
qa/tasks: Fix raises that doesn't re-raise

Reviewed-by: Ramana Raja <rraja@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge PR #32178 into master

* refs/pull/32178/head:
qa/standalone/test_ceph_daemon.sh: disable adoption for the moment

Reviewed-by: Sebastian Wagner <swagner@suse.com>

mgr/dashboard: fix RGW subuser auto-generate key.

Fixes: https://tracker.ceph.com/issues/43263
Signed-off-by: Alfonso Martínez <almartin@redhat.com>

Merge pull request #32176 from idryomov/wip-doc-fancy-striping

doc: clarify support for rbd fancy striping

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge PR #32163 into master

* refs/pull/32163/head:
qa: fix output check to not be sensitive to debugging

Reviewed-by: Rishabh Dave <ridave@redhat.com>

ceph.spec.in: move distro-conditional deps to dedicated section

Post e92cb7a0336406a2981e9241031497b1749b26aa cleanup. Restore previous
structure of "distro-conditional make check dependencies" section.

Fixes: https://tracker.ceph.com/issues/43171
Signed-off-by: Nathan Cutler <ncutler@suse.com>

Merge pull request #32083 from smithfarm/wip-42612

rpm: add rpm-build to SUSE-specific make check deps

Reviewed-by: Lenz Grimmer <lgrimmer@suse.com>
Reviewed-by: Thomas Bechtold <tbechtold@suse.com>
Reviewed-by: Sebastian Wagner <swagner@suse.com>