Merge pull request #34888 from jmolmo/fix_command_parse

cephadm: Fix error parsing shell command arguments

Reviewed-by: Michael Fritch <mfritch@suse.com>
Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>

Merge pull request #34945 from sebastian-philipp/cephadm-remove-async-map-resurrect

mgr/cephadm: replace async_map_completion with a simple wrapper (again)

Reviewed-by: Michael Fritch <mfritch@suse.com>

Merge pull request #34970 from sebastian-philipp/bluestore-revert-sharding

Revert "os/bluestore: Enable sharding by default"

Reviewed-by: Adam Kupczyk <akupczyk@redhat.com>

Merge pull request #34977 from mgfritch/cephadm-prom-encoding

mgr/cephadm: ceph_default_alerts.yml can be utf-8 encoded

Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>

Merge pull request #34955 from tchaikov/wip-cbt-with-bluestore

scripts/run-cbt.sh: use bluestore for testing classic osd

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge PR #34962 into master

* refs/pull/34962/head:
vstart_runner.py: use tuple instead of set

Reviewed-by: Sidharth Anupkrishnan <sanupkri@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #34938 from mgfritch/cephadm-systemd-stop-timeout

cephadm: extend systemd stop timeout

Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>

Merge pull request #34905 from tspmelo/wip-mirroring

Reviewed-by: Alfonso Martínez <almartin@redhat.com>

vstart_runner.py: use tuple instead of set

Using python3 leads to an error that says
"TypeError: unhashable type: 'Raw'".

Fixes: https://tracker.ceph.com/issues/45446
Signed-off-by: Rishabh Dave <ridave@redhat.com>

Merge pull request #34926 from sebastian-philipp/mgr-mock-mon-store

pybind/mgr: properly mock the mon store for pytest

Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Laura Paduano <lpaduano@suse.com>

Revert "os/bluestore: Enable sharding by default"

This reverts commit aa1a116f9eef5d6fbe09d6ed99910223451431f1.

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>

mgr/cephadm: ceph_default_alerts.yml can be utf-8 encoded

non-ascii chars introduced by 653c3f66823

Fixes: https://tracker.ceph.com/issues/45458
Signed-off-by: Michael Fritch <mfritch@suse.com>

Merge PR #34948 into master

* refs/pull/34948/head:
doc/cephfs-shell: Add quota command usage
doc/cephfs-shell: Add du command usage
doc/cephfs-shell: Add df command usage

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Rishabh Dave <ridave@redhat.com>

Merge pull request #34958 from yuvalif/fix_versioning_in_notif

rgw/notifications: version id was not sent in versioned buckets

Merge pull request #34967 from tchaikov/wip-rpm-fmt

rpm: BuildRequires fmt-devel on el8

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge pull request #34781 from lxbsz/run_shell1

qa/tasks/cephfs/mount.py: run_shell to use StringIO as default

Reviewed-by: Jeff Layton <jlayton@redhat.com>
Reviewed-by: Rishabh Dave <ridave@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #34834 from sidharthanup/qa-test-exports-fix

qa/test_exports: fix TestExports failure under new py3 compability changes

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #34871 from tchaikov/wip-qa-tasks-mgr-set-application

qa/tasks/mgr: always add "application" label to pool

Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Stephan Müller <smueller@suse.com>

Merge pull request #34895 from mgfritch/mgr-insights-py3-warn

mgr/insights: fix python3 DeprecationWarning

Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>

Merge pull request #34875 from tchaikov/wip-qa-tasks-qemu

qa/tasks/qemu: open file in text mode

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge PR #34941 into master

* refs/pull/34941/head:
qa/cephfs: use StringIO instead of BytesIO

Reviewed-by: Xiubo Li <xiubli@redhat.com>

Merge pull request #34037 from ivancich/wip-adjust-ordered-list-attempts

rgw: clean up attempt loop in ordered listing

Reviewed-By: Abhishek Lekshmanan <abhishek@suse.com>
Reviewed-by: Adam C. Emerson <aemerson@redhat.com>
Reviewed-by: Ali Maredia <amaredia@redhat.com>

Merge pull request #33865 from ivancich/wip-fix-ordered-marker-bug

rgw: fix bug where bucket listing end marker not always set correctly

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #30511 from tianshan/fix_41953

rgw: skip add empty multipart chain to gc in abort_multipart_upload

Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: Matt Benjamin <mbenjami@redhat.com>

Merge pull request #29886 from tianshan/fix_bucket_handle_leak

rgw: fix some list buckets handle leak

Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: Abhishek Lekshmanan <abhishek@suse.com>

Merge pull request #34961 from sebastian-philipp/cephadm-fix-node-exporter-auth-entity

mgr/cephadm: node-exporter is not a ceph service

Reviewed-by: Michael Fritch <mfritch@suse.com>

Merge pull request #34932 from sebastian-philipp/cephadm-hostassignment-schedule.py

mgr/cephadm: move HostAssignment to new module

Reviewed-by: Joshua Schmid <jschmid@suse.de>

Merge pull request #34689 from rhcs-dashboard/45186-fix-test-orch

mgr/test_orchestrator: fix _get_ceph_daemons()

Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>

rpm: BuildRequires fmt-devel on el8

it's included in EPEL8 now.

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #34950 from sebastian-philipp/cephadm-upgrade.py

mgr/cephadm: move upgrade code to upgrade.py

Reviewed-by: Michael Fritch <mfritch@suse.com>

Merge pull request #34922 from SUSE/wip-fix-45393

mgr/cephadm: refresh osd config when mons change

Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>

Merge pull request #34903 from sebastian-philipp/cephadm-adopt-qa-grafana

qa/workunits/cephadm: test grafana adoption

Reviewed-by: Michael Fritch <mfritch@suse.com>

Merge pull request #34949 from tspmelo/wip-i18n-fix

Reviewed-by: Stephan Müller <smueller@suse.com>
Reviewed-by: Tatjana Dehler <tdehler@suse.com>

doc/cephfs-shell: Add quota command usage

Fixes: https://tracker.ceph.com/issues/44788
Signed-off-by: Varsha Rao <varao@redhat.com>

doc/cephfs-shell: Add du command usage

Fixes: https://tracker.ceph.com/issues/44788
Signed-off-by: Varsha Rao <varao@redhat.com>

doc/cephfs-shell: Add df command usage

Fixes: https://tracker.ceph.com/issues/44788
Signed-off-by: Varsha Rao <varao@redhat.com>

Merge pull request #34923 from ofriedma/wip-ofriedma-fcgi

rgw: unable to compile fcgi frontend

Reviewed-by: Matt Benjamin <mbenjamin@redhat.com>

Merge pull request #34709 from votdev/issue_45204_max_buckets

Reviewed-by: Stephan Müller <smueller@suse.com>
Reviewed-by: Tiago Melo <tmelo@suse.com>

Merge pull request #34854 from BenoitKnecht/fix-alert-description

Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Patrick Seidensal <pnawracay@suse.com>

mgr/cephadm: add extra function for the auth entity

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>

mgr/cephadm: node-exporter is not a ceph service

Fixes: https://tracker.ceph.com/issues/45427
Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>

pybind/mgr: properly mock the mon store for pytest

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>

Merge pull request #34843 from mgfritch/cephadm-cli-servicespec

mgr/orch: Clean-up ServiceSpec handling during add/remove (mds, rgw, iscsi, nfs)

Reviewed-by: Joshua Schmid <jschmid@suse.de>
Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>

mgr/cephadm: add some more type checks

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>

mgr/cephadm: cleanup async_map_completion etc.

This breaks the progress integration, but we haven't
used that anyway till now.

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>

mgr/cephadm: replace async_completion with trivial_completion

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>

mgr/cephadm: replace async_map_completion with a simple wrapper

There is no need to wrap everything into completions.

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
(cherry picked from commit a78f42e225bbc041324dc3e565deb207feddad26)

mgr/cephadm: move HostAssignment to new module

(speeds up my editor)

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>

Merge pull request #34909 from sebastian-philipp/cephadm-doc-ceph.conf

doc/cephadm: add section about setting config settings

Reviewed-by: Alexandra Settle <asettle@suse.com>
Reviewed-by: Michael Fritch <mfritch@suse.com>

Merge pull request #34891 from sebastian-philipp/cephadm-cephadmservice

mgr/cephadm: Add CephadmService class

Reviewed-by: Joshua Schmid <jschmid@suse.de>
Reviewed-by: Kiefer Chang <kiefer.chang@suse.com>
Reviewed-by: Michael Fritch <mfritch@suse.com>

cephadm: Fix error parsing <command> arguments

Optional arguments in the <command> parameter used in:
 <cephadm shell>
 <cephadm enter>
 <cephadm ceph-volume>
are not parsed properly. This fixes this problem.

Signed-off-by: Juan Miguel Olmo Martínez <jolmomar@redhat.com>

rgw/notifications: version id was not sent in versioned buckets

Signed-off-by: Yuval Lifshitz <ylifshit@redhat.com>
Fixes: https://tracker.ceph.com/issues/45440

Merge pull request #34853 from yison/rwl-image-writeback-cache-seq9

rbd/cache: Replicated Write Log core codes - compare_and_write

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #34952 from cbodley/wip-44958

doc/rgw: document 'rgw gc max concurrent io'

Reviewed-by: Matt Benjamin <mbenjamin@redhat.com>

scripts/run-cbt.sh: use bluestore for testing classic osd

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #34929 from tspmelo/wip-e2e-script

mgr/dashboard: Improve run-frontend-e2e-tests.sh

Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Laura Paduano <lpaduano@suse.com>

doc/rgw: document 'rgw gc max concurrent io'

Fixes: https://tracker.ceph.com/issues/44958
Signed-off-by: Casey Bodley <cbodley@redhat.com>

mgr/cephadm: move upgrade code to upgrade.py

Make it easier for everyone to read the upgrade algorithm.

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>

mgr/dashboard: Fix I18N errors

We were just checking if the translation sources were correct and ignore if the
extraction simply failed.

Fixes: https://tracker.ceph.com/issues/45428
Signed-off-by: Tiago Melo <tmelo@suse.com>

Merge pull request #34820 from sebastian-philipp/akamac-cephadm-fix-typo-in-postinst

debian: fix typo in cephadm user gecos

Reviewed-by: Michael Fritch <mfritch@suse.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

mgr/cephadm: Add IscsiService

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>

mgr/cephadm: Add CrashService

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>

mgr/cephadm: Add NodeExporterService

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>

mgr/cephadm: Add PrometheusService

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>

mgr/cephadm: Add AlertmanagerService

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>

mgr/cephadm: Add GrafanaService

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>

mgr/cephadm: Add RbdMirrorService

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>

mgr/cephadm: Add RgwService

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>

mgr/cephadm: Add MdsService

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>

mgr/cephadm: Add MgrService

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>

mgr/cephadm: Add MonService

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>

mgr/cephadm: Add NFSService

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>

mgr/cephadm: move OSD related code to OSDService

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>

mgr/cephadm: move nfs and osd to services/

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>

mgr/dashboard: Improve run-frontend-e2e-tests.sh

Allow to configure Cypress using env vars.

Fixes: https://tracker.ceph.com/issues/45408
Add Signed-off-by: Tiago Melo <tmelo@suse.com>

Merge pull request #34855 from s0nea/wip-dashboard-troubleshooting

Reviewed-by: Enno Gotthold <egotthold@suse.de>
Reviewed-by: Volker Theile <vtheile@suse.com>

debian: fix typo in cephadm user gecos

Signed-off-by: Alexey Miasoedov <alexey.miasoedov@gmail.com>

Merge pull request #34591 from votdev/issue_45109

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Laura Paduano <lpaduano@suse.com>
Reviewed-by: Tatjana Dehler <tdehler@suse.com>

qa/cephfs: use StringIO instead of BytesIO

Fixes: https://tracker.ceph.com/issues/45425
Signed-off-by: Rishabh Dave <ridave@redhat.com>

librbd: add aio_compare_and_write test cases

Signed-off-by: Peterson, Scott <scott.d.peterson@intel.com>
Signed-off-by: Li, Xiaoyan <xiaoyan.li@intel.com>
Signed-off-by: Lu, Yuan <yuan.y.lu@intel.com>
Signed-off-by: Chamarthy, Mahati <mahati.chamarthy@intel.com>

librbd: add aio_compare_and_write

Signed-off-by: Peterson, Scott <scott.d.peterson@intel.com>
Signed-off-by: Li, Xiaoyan <xiaoyan.li@intel.com>
Signed-off-by: Lu, Yuan <yuan.y.lu@intel.com>
Signed-off-by: Chamarthy, Mahati <mahati.chamarthy@intel.com>

librbd: add CompAndWriteRequest

Signed-off-by: Peterson, Scott <scott.d.peterson@intel.com>
Signed-off-by: Li, Xiaoyan <xiaoyan.li@intel.com>
Signed-off-by: Lu, Yuan <yuan.y.lu@intel.com>
Signed-off-by: Chamarthy, Mahati <mahati.chamarthy@intel.com>

Merge pull request #34921 from tchaikov/wip-seastar

crimson: pick up a recent seastar version and cleanups

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

cephadm: extend systemd stop timeout

daemon stop + unit.poststop might take longer to complete

Fixes: https://tracker.ceph.com/issues/45417
Signed-off-by: Michael Fritch <mfritch@suse.com>

Merge pull request #34851 from yison/rwl-image-writeback-cache-seq8

rbd/cache: Replicated Write Log core codes - retire entries and invalidate

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

monitoring: Fix "10% OSDs down" alert description

The alert was triggered when less than 90% of OSDs were _up_, but then the
description took that value and described it as the percentage of OSDs being
_down_. So with 12% of OSDs down, the alert description would read:

```
88% or 88 of 100 OSDs are down (>=10%).
```

which can be panic-inducing.

This commit changes the alert expression to actually compute the ratio of OSDs
being down, which makes the correct value appear in the description.

Signed-off-by: Benoît Knecht <bknecht@protonmail.ch>

crimson: avoid using variadic future

it is deprecated by seastar. let's use future<tuple<...>> instead.

Signed-off-by: Kefu Chai <kchai@redhat.com>

seastar: pick up a recent seastar

to alleviate the pain of keeping in sync with seastar upstream.

Signed-off-by: Kefu Chai <kchai@redhat.com>

crimson/common: friend with do_with_impl()

seastar changed the implementatino of do_with(), so it's do_with_impl()
who tries to up-cast erroratorized future to seastar::future<>.

Signed-off-by: Kefu Chai <kchai@redhat.com>

crimson: drop Dispatch::ms_verify_authorizer()

as it's not used at all. also, variadic future was deprecated by
seastar, so let's avoid using it.

Signed-off-by: Kefu Chai <kchai@redhat.com>

crimson: s/reactor::cpu_id/seastar::this_shard_id/

the former was deprecated since seastar
02956fd436b413d6bbe562cf91533ada36553e54

Signed-off-by: Kefu Chai <kchai@redhat.com>

doc/cephadm: add section about setting config settings

Fixes: https://tracker.ceph.com/issues/44284
Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>

librbd: add invalidate test case

Signed-off-by: Peterson, Scott <scott.d.peterson@intel.com>
Signed-off-by: Li, Xiaoyan <xiaoyan.li@intel.com>
Signed-off-by: Lu, Yuan <yuan.y.lu@intel.com>
Signed-off-by: Chamarthy, Mahati <mahati.chamarthy@intel.com>

librbd: add invalidate

Signed-off-by: Peterson, Scott <scott.d.peterson@intel.com>
Signed-off-by: Li, Xiaoyan <xiaoyan.li@intel.com>
Signed-off-by: Lu, Yuan <yuan.y.lu@intel.com>
Signed-off-by: Chamarthy, Mahati <mahati.chamarthy@intel.com>

librbd: retire entries

Signed-off-by: Peterson, Scott <scott.d.peterson@intel.com>
Signed-off-by: Li, Xiaoyan <xiaoyan.li@intel.com>
Signed-off-by: Lu, Yuan <yuan.y.lu@intel.com>
Signed-off-by: Chamarthy, Mahati <mahati.chamarthy@intel.com>

Merge pull request #34899 from smanjara/wip-dynamic-resharding

qa/rgw: add ssl option in reshard tests

Reviewed-by: Casey Bodley <cbodley@redhat.com>

doc/mgr/dashboard: Add troubleshooting guide

Fixes: https://tracker.ceph.com/issues/25196
Signed-off-by: Tatjana Dehler <tdehler@suse.com>

Merge pull request #34887 from sebastian-philipp/cephadm-move-specstore

mgr/cephadm: move SpecStore and HostCache to inventoy.py

Reviewed-by: Joshua Schmid <jschmid@suse.de>

Merge pull request #34898 from matthewoliver/cephadm-iscsi-priv

cephadm: give ceph-iscsi permissions to configfs

Reviewed-by: Boris Ranto <branto@redhat.com>

crimson/common: make do_with() noexcept

to be consistent with seasetar change of
85aa178d9539b2c9301d4fb0815596a741fb82c4, which make
`seasetar::do_with()` noexcept

Signed-off-by: Kefu Chai <kchai@redhat.com>