cephfs-shell: fix warnings reported by flake8

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #34746 from tspmelo/wip-form-loading

Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Stephan Müller <smueller@suse.com>

Merge pull request #34888 from jmolmo/fix_command_parse

cephadm: Fix error parsing shell command arguments

Reviewed-by: Michael Fritch <mfritch@suse.com>
Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>

Merge pull request #34945 from sebastian-philipp/cephadm-remove-async-map-resurrect

mgr/cephadm: replace async_map_completion with a simple wrapper (again)

Reviewed-by: Michael Fritch <mfritch@suse.com>

Merge pull request #34970 from sebastian-philipp/bluestore-revert-sharding

Revert "os/bluestore: Enable sharding by default"

Reviewed-by: Adam Kupczyk <akupczyk@redhat.com>

Merge pull request #34977 from mgfritch/cephadm-prom-encoding

mgr/cephadm: ceph_default_alerts.yml can be utf-8 encoded

Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>

Merge pull request #34955 from tchaikov/wip-cbt-with-bluestore

scripts/run-cbt.sh: use bluestore for testing classic osd

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge PR #34962 into master

* refs/pull/34962/head:
vstart_runner.py: use tuple instead of set

Reviewed-by: Sidharth Anupkrishnan <sanupkri@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #34938 from mgfritch/cephadm-systemd-stop-timeout

cephadm: extend systemd stop timeout

Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>

Merge pull request #34905 from tspmelo/wip-mirroring

Reviewed-by: Alfonso Martínez <almartin@redhat.com>

vstart_runner.py: use tuple instead of set

Using python3 leads to an error that says
"TypeError: unhashable type: 'Raw'".

Fixes: https://tracker.ceph.com/issues/45446
Signed-off-by: Rishabh Dave <ridave@redhat.com>

Merge pull request #34926 from sebastian-philipp/mgr-mock-mon-store

pybind/mgr: properly mock the mon store for pytest

Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Laura Paduano <lpaduano@suse.com>

Revert "os/bluestore: Enable sharding by default"

This reverts commit aa1a116f9eef5d6fbe09d6ed99910223451431f1.

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>

mgr/cephadm: ceph_default_alerts.yml can be utf-8 encoded

non-ascii chars introduced by 653c3f66823

Fixes: https://tracker.ceph.com/issues/45458
Signed-off-by: Michael Fritch <mfritch@suse.com>

Merge PR #34948 into master

* refs/pull/34948/head:
doc/cephfs-shell: Add quota command usage
doc/cephfs-shell: Add du command usage
doc/cephfs-shell: Add df command usage

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Rishabh Dave <ridave@redhat.com>

Merge pull request #34958 from yuvalif/fix_versioning_in_notif

rgw/notifications: version id was not sent in versioned buckets

Merge pull request #34967 from tchaikov/wip-rpm-fmt

rpm: BuildRequires fmt-devel on el8

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge pull request #34781 from lxbsz/run_shell1

qa/tasks/cephfs/mount.py: run_shell to use StringIO as default

Reviewed-by: Jeff Layton <jlayton@redhat.com>
Reviewed-by: Rishabh Dave <ridave@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #34834 from sidharthanup/qa-test-exports-fix

qa/test_exports: fix TestExports failure under new py3 compability changes

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #34871 from tchaikov/wip-qa-tasks-mgr-set-application

qa/tasks/mgr: always add "application" label to pool

Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Stephan Müller <smueller@suse.com>

Merge pull request #34895 from mgfritch/mgr-insights-py3-warn

mgr/insights: fix python3 DeprecationWarning

Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>

Merge pull request #34875 from tchaikov/wip-qa-tasks-qemu

qa/tasks/qemu: open file in text mode

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge PR #34941 into master

* refs/pull/34941/head:
qa/cephfs: use StringIO instead of BytesIO

Reviewed-by: Xiubo Li <xiubli@redhat.com>

Merge pull request #34037 from ivancich/wip-adjust-ordered-list-attempts

rgw: clean up attempt loop in ordered listing

Reviewed-By: Abhishek Lekshmanan <abhishek@suse.com>
Reviewed-by: Adam C. Emerson <aemerson@redhat.com>
Reviewed-by: Ali Maredia <amaredia@redhat.com>

Merge pull request #33865 from ivancich/wip-fix-ordered-marker-bug

rgw: fix bug where bucket listing end marker not always set correctly

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #30511 from tianshan/fix_41953

rgw: skip add empty multipart chain to gc in abort_multipart_upload

Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: Matt Benjamin <mbenjami@redhat.com>

Merge pull request #29886 from tianshan/fix_bucket_handle_leak

rgw: fix some list buckets handle leak

Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: Abhishek Lekshmanan <abhishek@suse.com>

Merge pull request #34961 from sebastian-philipp/cephadm-fix-node-exporter-auth-entity

mgr/cephadm: node-exporter is not a ceph service

Reviewed-by: Michael Fritch <mfritch@suse.com>

Merge pull request #34932 from sebastian-philipp/cephadm-hostassignment-schedule.py

mgr/cephadm: move HostAssignment to new module

Reviewed-by: Joshua Schmid <jschmid@suse.de>

Merge pull request #34689 from rhcs-dashboard/45186-fix-test-orch

mgr/test_orchestrator: fix _get_ceph_daemons()

Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>

rpm: BuildRequires fmt-devel on el8

it's included in EPEL8 now.

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #34950 from sebastian-philipp/cephadm-upgrade.py

mgr/cephadm: move upgrade code to upgrade.py

Reviewed-by: Michael Fritch <mfritch@suse.com>

Merge pull request #34922 from SUSE/wip-fix-45393

mgr/cephadm: refresh osd config when mons change

Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>

Merge pull request #34903 from sebastian-philipp/cephadm-adopt-qa-grafana

qa/workunits/cephadm: test grafana adoption

Reviewed-by: Michael Fritch <mfritch@suse.com>

Merge pull request #34949 from tspmelo/wip-i18n-fix

Reviewed-by: Stephan Müller <smueller@suse.com>
Reviewed-by: Tatjana Dehler <tdehler@suse.com>

doc/cephfs-shell: Add quota command usage

Fixes: https://tracker.ceph.com/issues/44788
Signed-off-by: Varsha Rao <varao@redhat.com>

doc/cephfs-shell: Add du command usage

Fixes: https://tracker.ceph.com/issues/44788
Signed-off-by: Varsha Rao <varao@redhat.com>

doc/cephfs-shell: Add df command usage

Fixes: https://tracker.ceph.com/issues/44788
Signed-off-by: Varsha Rao <varao@redhat.com>

Merge pull request #34923 from ofriedma/wip-ofriedma-fcgi

rgw: unable to compile fcgi frontend

Reviewed-by: Matt Benjamin <mbenjamin@redhat.com>

Merge pull request #34709 from votdev/issue_45204_max_buckets

Reviewed-by: Stephan Müller <smueller@suse.com>
Reviewed-by: Tiago Melo <tmelo@suse.com>

Merge pull request #34854 from BenoitKnecht/fix-alert-description

Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Patrick Seidensal <pnawracay@suse.com>

mgr/cephadm: add extra function for the auth entity

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>

mgr/cephadm: node-exporter is not a ceph service

Fixes: https://tracker.ceph.com/issues/45427
Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>

pybind/mgr: properly mock the mon store for pytest

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>

Merge pull request #34843 from mgfritch/cephadm-cli-servicespec

mgr/orch: Clean-up ServiceSpec handling during add/remove (mds, rgw, iscsi, nfs)

Reviewed-by: Joshua Schmid <jschmid@suse.de>
Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>

mgr/cephadm: add some more type checks

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>

mgr/cephadm: cleanup async_map_completion etc.

This breaks the progress integration, but we haven't
used that anyway till now.

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>

mgr/cephadm: replace async_completion with trivial_completion

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>

mgr/cephadm: replace async_map_completion with a simple wrapper

There is no need to wrap everything into completions.

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
(cherry picked from commit a78f42e225bbc041324dc3e565deb207feddad26)

mgr/cephadm: move HostAssignment to new module

(speeds up my editor)

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>

Merge pull request #34909 from sebastian-philipp/cephadm-doc-ceph.conf

doc/cephadm: add section about setting config settings

Reviewed-by: Alexandra Settle <asettle@suse.com>
Reviewed-by: Michael Fritch <mfritch@suse.com>

Merge pull request #34891 from sebastian-philipp/cephadm-cephadmservice

mgr/cephadm: Add CephadmService class

Reviewed-by: Joshua Schmid <jschmid@suse.de>
Reviewed-by: Kiefer Chang <kiefer.chang@suse.com>
Reviewed-by: Michael Fritch <mfritch@suse.com>

cephadm: Fix error parsing <command> arguments

Optional arguments in the <command> parameter used in:
 <cephadm shell>
 <cephadm enter>
 <cephadm ceph-volume>
are not parsed properly. This fixes this problem.

Signed-off-by: Juan Miguel Olmo Martínez <jolmomar@redhat.com>

rgw/notifications: version id was not sent in versioned buckets

Signed-off-by: Yuval Lifshitz <ylifshit@redhat.com>
Fixes: https://tracker.ceph.com/issues/45440

Merge pull request #34853 from yison/rwl-image-writeback-cache-seq9

rbd/cache: Replicated Write Log core codes - compare_and_write

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #34952 from cbodley/wip-44958

doc/rgw: document 'rgw gc max concurrent io'

Reviewed-by: Matt Benjamin <mbenjamin@redhat.com>

scripts/run-cbt.sh: use bluestore for testing classic osd

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #34929 from tspmelo/wip-e2e-script

mgr/dashboard: Improve run-frontend-e2e-tests.sh

Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Laura Paduano <lpaduano@suse.com>

doc/rgw: document 'rgw gc max concurrent io'

Fixes: https://tracker.ceph.com/issues/44958
Signed-off-by: Casey Bodley <cbodley@redhat.com>

mgr/cephadm: move upgrade code to upgrade.py

Make it easier for everyone to read the upgrade algorithm.

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>

mgr/dashboard: Fix I18N errors

We were just checking if the translation sources were correct and ignore if the
extraction simply failed.

Fixes: https://tracker.ceph.com/issues/45428
Signed-off-by: Tiago Melo <tmelo@suse.com>

Merge pull request #34820 from sebastian-philipp/akamac-cephadm-fix-typo-in-postinst

debian: fix typo in cephadm user gecos

Reviewed-by: Michael Fritch <mfritch@suse.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

mgr/cephadm: Add IscsiService

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>

mgr/cephadm: Add CrashService

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>

mgr/cephadm: Add NodeExporterService

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>

mgr/cephadm: Add PrometheusService

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>

mgr/cephadm: Add AlertmanagerService

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>

mgr/cephadm: Add GrafanaService

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>

mgr/cephadm: Add RbdMirrorService

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>

mgr/cephadm: Add RgwService

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>

mgr/cephadm: Add MdsService

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>

mgr/cephadm: Add MgrService

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>

mgr/cephadm: Add MonService

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>

mgr/cephadm: Add NFSService

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>

mgr/cephadm: move OSD related code to OSDService

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>

mgr/cephadm: move nfs and osd to services/

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>

mgr/dashboard: Improve run-frontend-e2e-tests.sh

Allow to configure Cypress using env vars.

Fixes: https://tracker.ceph.com/issues/45408
Add Signed-off-by: Tiago Melo <tmelo@suse.com>

Merge pull request #34855 from s0nea/wip-dashboard-troubleshooting

Reviewed-by: Enno Gotthold <egotthold@suse.de>
Reviewed-by: Volker Theile <vtheile@suse.com>

debian: fix typo in cephadm user gecos

Signed-off-by: Alexey Miasoedov <alexey.miasoedov@gmail.com>

Merge pull request #34591 from votdev/issue_45109

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Laura Paduano <lpaduano@suse.com>
Reviewed-by: Tatjana Dehler <tdehler@suse.com>

qa/cephfs: use StringIO instead of BytesIO

Fixes: https://tracker.ceph.com/issues/45425
Signed-off-by: Rishabh Dave <ridave@redhat.com>

librbd: add aio_compare_and_write test cases

Signed-off-by: Peterson, Scott <scott.d.peterson@intel.com>
Signed-off-by: Li, Xiaoyan <xiaoyan.li@intel.com>
Signed-off-by: Lu, Yuan <yuan.y.lu@intel.com>
Signed-off-by: Chamarthy, Mahati <mahati.chamarthy@intel.com>

librbd: add aio_compare_and_write

Signed-off-by: Peterson, Scott <scott.d.peterson@intel.com>
Signed-off-by: Li, Xiaoyan <xiaoyan.li@intel.com>
Signed-off-by: Lu, Yuan <yuan.y.lu@intel.com>
Signed-off-by: Chamarthy, Mahati <mahati.chamarthy@intel.com>

librbd: add CompAndWriteRequest

Signed-off-by: Peterson, Scott <scott.d.peterson@intel.com>
Signed-off-by: Li, Xiaoyan <xiaoyan.li@intel.com>
Signed-off-by: Lu, Yuan <yuan.y.lu@intel.com>
Signed-off-by: Chamarthy, Mahati <mahati.chamarthy@intel.com>

Merge pull request #34921 from tchaikov/wip-seastar

crimson: pick up a recent seastar version and cleanups

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

cephadm: extend systemd stop timeout

daemon stop + unit.poststop might take longer to complete

Fixes: https://tracker.ceph.com/issues/45417
Signed-off-by: Michael Fritch <mfritch@suse.com>

Merge pull request #34851 from yison/rwl-image-writeback-cache-seq8

rbd/cache: Replicated Write Log core codes - retire entries and invalidate

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

mgr/dashboard: Forms now wait for all data to load until displayed

Add directive to handle the display of the form.
It will display the form only when loading is finished.
Otherwise it will show a loading message or error message.

Add a new class that should be extended by all form components.
For now it only has methods for dealing with loading,
but this could be improved later.

Fixes: https://tracker.ceph.com/issues/44912
Signed-off-by: Tiago Melo <tmelo@suse.com>

monitoring: Fix "10% OSDs down" alert description

The alert was triggered when less than 90% of OSDs were _up_, but then the
description took that value and described it as the percentage of OSDs being
_down_. So with 12% of OSDs down, the alert description would read:

```
88% or 88 of 100 OSDs are down (>=10%).
```

which can be panic-inducing.

This commit changes the alert expression to actually compute the ratio of OSDs
being down, which makes the correct value appear in the description.

Signed-off-by: Benoît Knecht <bknecht@protonmail.ch>

crimson: avoid using variadic future

it is deprecated by seastar. let's use future<tuple<...>> instead.

Signed-off-by: Kefu Chai <kchai@redhat.com>

seastar: pick up a recent seastar

to alleviate the pain of keeping in sync with seastar upstream.

Signed-off-by: Kefu Chai <kchai@redhat.com>

crimson/common: friend with do_with_impl()

seastar changed the implementatino of do_with(), so it's do_with_impl()
who tries to up-cast erroratorized future to seastar::future<>.

Signed-off-by: Kefu Chai <kchai@redhat.com>

crimson: drop Dispatch::ms_verify_authorizer()

as it's not used at all. also, variadic future was deprecated by
seastar, so let's avoid using it.

Signed-off-by: Kefu Chai <kchai@redhat.com>

crimson: s/reactor::cpu_id/seastar::this_shard_id/

the former was deprecated since seastar
02956fd436b413d6bbe562cf91533ada36553e54

Signed-off-by: Kefu Chai <kchai@redhat.com>

doc/cephadm: add section about setting config settings

Fixes: https://tracker.ceph.com/issues/44284
Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>

librbd: add invalidate test case

Signed-off-by: Peterson, Scott <scott.d.peterson@intel.com>
Signed-off-by: Li, Xiaoyan <xiaoyan.li@intel.com>
Signed-off-by: Lu, Yuan <yuan.y.lu@intel.com>
Signed-off-by: Chamarthy, Mahati <mahati.chamarthy@intel.com>

librbd: add invalidate

Signed-off-by: Peterson, Scott <scott.d.peterson@intel.com>
Signed-off-by: Li, Xiaoyan <xiaoyan.li@intel.com>
Signed-off-by: Lu, Yuan <yuan.y.lu@intel.com>
Signed-off-by: Chamarthy, Mahati <mahati.chamarthy@intel.com>

librbd: retire entries

Signed-off-by: Peterson, Scott <scott.d.peterson@intel.com>
Signed-off-by: Li, Xiaoyan <xiaoyan.li@intel.com>
Signed-off-by: Lu, Yuan <yuan.y.lu@intel.com>
Signed-off-by: Chamarthy, Mahati <mahati.chamarthy@intel.com>

Merge pull request #34899 from smanjara/wip-dynamic-resharding

qa/rgw: add ssl option in reshard tests

Reviewed-by: Casey Bodley <cbodley@redhat.com>

doc/mgr/dashboard: Add troubleshooting guide

Fixes: https://tracker.ceph.com/issues/25196
Signed-off-by: Tatjana Dehler <tdehler@suse.com>