seastar: pickup the recent future optimizations.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

Merge pull request #32207 from bk201/wip-43136

mgr/dashboard: Fix OSD IDs are not displayed when using cephadm backend

Reviewed-by: Lenz Grimmer <lgrimmer@suse.com>
Reviewed-by: Ernesto Puertat <epuertat@redhat.com>
Reviewed-by: Laura Paduano <lpaduano@suse.com>

Merge pull request #32266 from xiexingguo/wip-balancer-efficiency-1

mgr/balancer: avoid pulling pg_dump twice

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #32237 from tchaikov/wip-cmake-rdkafka

cmake: check version of librdkafka

Reviewed-by: Yuval Lifshitz <yuvalif@yahoo.com>

Merge pull request #31144 from badone/wip-use-dnf-on-rhel-or-centos-8

install-deps.sh: Use dnf for rhel/centos 8

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge PR #31884 into master

* refs/pull/31884/head:
qa/tasks: Fix the volume ls in test_volume_rm
qa/tasks: tests for 'fs volume create' and 'fs volume ls'

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge PR #29059 into master

* refs/pull/29059/head:
mds: complete all the replay op when mds is restarted ,no matter the session is closed or not.

Reviewed-by: Zheng Yan <zyan@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge PR #31662 into master

* refs/pull/31662/head:
mds: fix revoking caps after after stale->resume circle
client: fix incorrect debug message

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge PR #32162 into master

* refs/pull/32162/head:
qa: note timeout in debug message

Reviewed-by: Sage Weil <sage@redhat.com>

Merge PR #31118 into master

* refs/pull/31118/head:
mds: Reorganize class members in MDSMap header

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge PR #31123 into master

* refs/pull/31123/head:
mds : optimization functions,get_dirfrags_under, to speed up processing directories with tens of millions of files

Reviewed-by: Zheng Yan <zyan@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge PR #31595 into master

* refs/pull/31595/head:
mds: optimize function, fragset_t::simplify, to improve the efficiency of  merging fragment

Reviewed-by: Zheng Yan <zyan@redhat.com>

Merge PR #32020 into master

* refs/pull/32020/head:
mds: fix assert(omap_num_objs <= MAX_OBJECTS) of OpenFileTable

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

qa: note timeout in debug message

Some tests set this to a dynamic value, it'd be helpful to know how long
a test is planning to wait.

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

Merge PR #32042 into master

* refs/pull/32042/head:
mds: update default MDS cache size memory limit
common/options: note runtime effect for some MDS configs

Reviewed-by: Rishabh Dave <ridave@redhat.com>
Reviewed-by: Jeff Layton <jlayton@redhat.com>

Merge PR #32275 into master

* refs/pull/32275/head:
qa/suites/upgrade: all upgrades to octopus on ubuntu only

Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge pull request #32274 from dillaman/wip-rbd-mirror-test

rbd-mirror: ensure deterministic ordering of method calls

Reviewed-by: Willem Jan Withagen <wjw@digiware.nl>

Merge PR #32212 into master

* refs/pull/32212/head:
rpm/cephadm: start description and summary with capital letter
rpm/cephadm: move HOMEDIR to /var/lib and make scriptlets idempotent on SUSE

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Thomas Bechtold <tbechtold@suse.com>

Merge PR #32277 into master

* refs/pull/32277/head:
lvm/deactivate: add unit tests, remove --all

Reviewed-by: Sage Weil <sage@redhat.com>

qa/suites/upgrade: all upgrades to octopus on ubuntu only

We cannot do a traditional upgrade (install old package, start cluster,
install new package, ...) because nautilus is el7-only and octopus is
el8-only.

So, do these tests on ubuntu.

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #32217 from trociny/wip-41557-2

librbd: explicitly specify mode on mirror image enable

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

lvm/deactivate: add unit tests, remove --all

Remove the --all flag until its actually implemented.

Fixes: https://tracker.ceph.com/issues/43330
Signed-off-by: Jan Fajerski <jfajerski@suse.com>

Merge PR #32198 into master

* refs/pull/32198/head:
cephadm: Fix Python2.7 isssue
cephadm: fix how uid/gid is determined
cephadm: add sample configuration for prometheus
cephadm: support deployment of prometheus

Reviewed-by: Sebastian Wagner <swagner@suse.com>

rpm/cephadm: start description and summary with capital letter

This commit fixes RPMLINT issue:

cephadm.x86_64: E: summary-not-capitalized (Badness: 20) C cephadm utility to bootstrap Ceph clusters
Summary doesn't begin with a capital letter.

Signed-off-by: Nathan Cutler <ncutler@suse.com>

rpm/cephadm: move HOMEDIR to /var/lib and make scriptlets idempotent on SUSE

Also introduce a cephadm group. Since the cephadm package does not
require ceph-common, the ceph group is not available to be used.

Drop the -r option to useradd, because cephadm does not qualify as a system
user.

Finally, fix the SUSE RPM build. SUSE RPM builds using "osc" implement a check
that compares the cephadm package's "before" and "after" file lists, to enforce
idempotency of the scriptlets. The "userdel cephadm" in the %postun breaks this
check, so omit it from the SUSE RPM builds.

Fixes: https://tracker.ceph.com/issues/43285
Signed-off-by: Nathan Cutler <ncutler@suse.com>

rbd-mirror: ensure deterministic ordering of method calls

The mock tests will require method calls in a known order. Previously
different environments could evaluate the parameters in different
orders resulting in out-of-order method calls.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #32211 from rhcs-dashboard/43284-fix-frontend-e2e-script

mgr/dashboard: run-frontend-e2e-tests.sh: allow user defined BASE_URL…

Reviewed-by: Tiago Melo <tmelo@suse.com>
Reviewed-by: Laura Paduano <lpaduano@suse.com>

Merge pull request #31819 from ricardoasmarques/wip-42687

mgr/dashboard: iSCSI targets not available if any gateway is down

Reviewed-by: Tiago Melo <tmelo@suse.com>
Reviewed-by: Laura Paduano <lpaduano@suse.com>

Merge PR #32246 into master

* refs/pull/32246/head:
mgr/cephadm: remove redundant /dev when blinking device light

Reviewed-by: Sebastian Wagner <swagner@suse.com>

Merge pull request #32257 from dillaman/wip-41556

rbd-mirror: extract journal replaying logic from image replayer

Reviewed-by: Mykola Golub <mgolub@suse.com>

cephadm: Fix Python2.7 isssue

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>

mgr/dashboard: CephFS quota management (#31811)

mgr/dashboard: CephFS quota management

Reviewed-by: Patrick Seidensal <pnawracay@suse.com>
Reviewed-by: Ricardo Marques <rimarques@suse.com>
Reviewed-by: Tatjana Dehler <tdehler@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>

mgr/dashboard: add password expiration date (#30939)

mgr/dashboard: add password expiration date

Reviewed-by: Ricardo Marques <rimarques@suse.com>
Reviewed-by: Tiago Melo <tmelo@suse.com>

mgr/dashboard: Display OSD device class in OSD list (#32092)

mgr/dashboard: Display OSD device class in OSD list

Reviewed-by: Laura Paduano <lpaduano@suse.com>
Reviewed-by: Tatjana Dehler <tdehler@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>

Merge pull request #31815 from tchaikov/wip-mgr-dashboard-gevent

mgr/dashboard: install teuthology using pip

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Ernesto Puertat <epuertat@redhat.com>

rbd: show mirror mode in image info

Signed-off-by: Mykola Golub <mgolub@suse.com>

rbd: improve mirror snapshot list formatting for status command

Signed-off-by: Mykola Golub <mgolub@suse.com>

rbd: allow to specify mode on mirror image enable

Signed-off-by: Mykola Golub <mgolub@suse.com>

librbd: explicitly specify mode on mirror image enable

- don't use journaling feature to chose the mode;
- provide new API function mirror_image_enable2;
- return back the old behavior to automatically enable/disable
  journaling feature on enabling/disabling image mirroring.

Signed-off-by: Mykola Golub <mgolub@suse.com>

mgr/balancer: avoid pulling pg_dump twice

For large cluster with thousands of osds/pgs, it may take
seconds even mitutes to refresh the relevant stats.
Since we already have pulling them down at initialization,
make a direct use of them whenever possible.

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

rbd-mirror: switch image replayer to new standalone journal replayer

Remove all the original journal replaying code embedded in the image
replayer and instead rely on the new journal replayer class.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

rbd-mirror: extract journal replay logic to its own class

This will help to greatly reduce the journal-specific code in the
current image replayer.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

rbd-mirror: moved image asok/perf counter name helper to utils

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

rbd-mirror: moved event pre-processor to sub-namespace

The current implementation is tied to journal-based replaying so
move it down into the journal sub-namespace.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

rbd-mirror: moved replay status formatter to sub-namespace

The current implementation is tied to journal-based replaying so
move it down into a new journal sub-namespace.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

rbd-mirror: drop 'using namespace' in image replayer state machine

This is going to cause conflicts w/ the '::journal' namespace and
future 'rbd::mirror::image_replayer::journal' namespace.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #32142 from changchengx/doc_libvirt

doc/rbd: simplify libvirt usage

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #32243 from trociny/wip-43289

rbd-mirror: fix race on namespace replayer initialization failure

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #32184 from trociny/wip-43004

rbd-mirror: rename per-image replication perf counters

Reviewed-by: Jan Fajerski <jfajerski@suse.com>
Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #32113 from lixiaoy1/master

librbd: finish write request early

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge PR #32251 into master

* refs/pull/32251/head:
mds/MDSDaemon: fix asok 'exit' and 'respawn' commands

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge PR #32260 into master

* refs/pull/32260/head:
.organizationmap: Wido 42on -> 42on

Reviewed-by: Kefu Chai <kchai@redhat.com>

cephadm: fix how uid/gid is determined

The uid/gid is determined by looking at file ownership. It
was looking at /etc/ceph, but on current builds this is owned by root so switched to using /var/lib/ceph which
is the homedir of the ceph account.

Signed-off-by: Paul Cuzner <pcuzner@redhat.com>

cephadm: add sample configuration for prometheus

Provides an example definition that sets up an
initial prometheus server with a scrape of itself.

Signed-off-by: Paul Cuzner <pcuzner@redhat.com>

cephadm: support deployment of prometheus

Initial support to enable cephadm to deploy
monitoring containers like prometheus. This
patch adds support for prometheus. It is the callers
responsibility to provide a valid prometheus.yml
file passed within the --config-json parameter.

Signed-off-by: Paul Cuzner <pcuzner@redhat.com>

.organizationmap: Wido 42on -> 42on

Signed-off-by: Sage Weil <sage@redhat.com>

cmake: check version of librdkafka

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge PR #32158 into master

* refs/pull/32158/head:
cephadm: fix exception when no time sync is running
cephadm: properly extract osd fsid during adoption
cephadm: do ceph-volume activate+deactivate as part of systemd unit
ceph-daemon: behave with no /etc/ceph/ceph.conf

Reviewed-by: Jan Fajerski <jfajerski@suse.com>
Reviewed-by: Sebastian Wagner <swagner@suse.com>

cephadm: fix exception when no time sync is running

Signed-off-by: Sage Weil <sage@redhat.com>

cephadm: properly extract osd fsid during adoption

This needs to be passed to the deploy_daemon_units method so that we can
set up the pre and post c-v calls properly.

Signed-off-by: Sage Weil <sage@redhat.com>

cephadm: do ceph-volume activate+deactivate as part of systemd unit

1- Do the activate as part of the unit on start, since it may include
setup that disappears on a reboot, like starting up the dmcrypted volume.

2- Do the matching deactivate step when the container stops, like shutting
down the dmcrypt volume.

Right now the OSD is the only thing that needs this special behavior.

Signed-off-by: Sage Weil <sage@redhat.com>

ceph-daemon: behave with no /etc/ceph/ceph.conf

Signed-off-by: Sage Weil <sage@redhat.com>

Merge PR #32244 into master

* refs/pull/32244/head:
qa/workunits/cephadm/test_cephadm.sh: re-enable `adopt` tests

Reviewed-by: Sage Weil <sage@redhat.com>

mds/MDSDaemon: fix asok 'exit' and 'respawn' commands

Signed-off-by: Sage Weil <sage@redhat.com>

Merge PR #32253 into master

* refs/pull/32253/head:
mgr/orchestrator: make hosts and label args consistent

Reviewed-by: Sebastian Wagner <swagner@suse.com>

Merge PR #32209 into master

* refs/pull/32209/head:
mgr/cephadm: Add unittest for service_action

Reviewed-by: Sage Weil <sage@redhat.com>

mgr/orchestrator: make hosts and label args consistent

- hosts is always n=N; labels is not
- hosts always before labels

Signed-off-by: Sage Weil <sage@redhat.com>

Merge PR #32192 into master

* refs/pull/32192/head:
common: handle return value from read(2)

Reviewed-by: Jeff Layton <jlayton@redhat.com>
Reviewed-by: Sage Weil <sage@redhat.com>

Merge PR #32195 into master

* refs/pull/32195/head:
osd/PeeringState.cc: skip peer_purged when discovering all missing

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: David Zafman <dzafman@redhat.com>

Merge PR #32203 into master

* refs/pull/32203/head:
Revert "mgr/DaemonServer: safe-to-destroy - do not consider irrelevant pgs"

Reviewed-by: Sage Weil <sage@redhat.com>

Merge PR #32219 into master

* refs/pull/32219/head:
common: move xattr -> os/filestore/os_xattr

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge PR #32250 into master

* refs/pull/32250/head:
mgr/cephadm: fix temp file prefixes
qa: fix lingering ceph-mgr-ssh -> ceph-mgr-cephadm refs

Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Sebastian Wagner <swagner@suse.com>

Merge PR #32144 into master

* refs/pull/32144/head:
doc: mention multimds suite in dev guide
doc: clarify difference between fs and kcephfs suite

Reviewed-by: Varsha Rao <varao@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge PR #32024 into master

* refs/pull/32024/head:
stop: Unmount cephFS volumes

Reviewed-by: Jeff Layton <jlayton@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

mgr/cephadm: fix temp file prefixes

Signed-off-by: Sage Weil <sage@redhat.com>

qa: fix lingering ceph-mgr-ssh -> ceph-mgr-cephadm refs

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #32226 from neha-ojha/wip-four-percent

doc/rados: Better block.db size recommendations for bluestore

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Igor Fedotov <ifedotov@suse.com>

doc/rados: Better block.db size recommendations for bluestore

Signed-off-by: Neha Ojha <nojha@redhat.com>

Merge pull request #32040 from smithfarm/wip-rn-13-2-8-mimic

doc/releases: add release notes for mimic v13.2.8

Reviewed-by: Abhishek Lekshmanan <abhishek@suse.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>

mgr/cephadm: remove redundant /dev when blinking device light

Fixes: https://tracker.ceph.com/issues/43223
Signed-off-by: Sage Weil <sage@redhat.com>

doc/releases: add Mimic v13.2.8 to releases timeline

Signed-off-by: Nathan Cutler <ncutler@suse.com>

Merge PR #32236 into master

* refs/pull/32236/head:
pybind/mgr: add_tox_test: Add mypy to TOX_ENVS

Reviewed-by: Sage Weil <sage@redhat.com>

qa/workunits/cephadm/test_cephadm.sh: re-enable `adopt` tests

Signed-off-by: Michael Fritch <mfritch@suse.com>

mgr/dashboard: Avoid escaped quotes using TSLint

The problem was that the TSLint linting hasn't matched the
linting of prettier regarding the quotes of a string.

Explanation:
'\'' will be converted to "'" with prettier, but TSLint thrown an
error that it wanted the string to look like '\''.

Now TSLint will not error if it sees an single quote inside a double
quote to match the behavior of prettier.

Fixes: https://tracker.ceph.com/issues/38287
Signed-off-by: Stephan Müller <smueller@suse.com>

mgr/dashboard: Allow string descriptions in confirmation modal

Fixes: https://tracker.ceph.com/issues/38287
Signed-off-by: Stephan Müller <smueller@suse.com>

mgr/dashboard: Add binary min and max validators

Adds simple validators that resemble the min and max validators for
numbers.

As a binary size (the string) has first be converted into bytes
(the number) to validate against some other number and in case of an
error the other number want's to be converted into a size to display,
the whole process needs a bunch of conversion, but these validators take
care of all that.

Fixes: https://tracker.ceph.com/issues/38287
Signed-off-by: Stephan Müller <smueller@suse.com>

mgr/dashboard: Make form modal more flexible

Now any input type is supported and the special binary type which will
use the cd-binary input and automatically format the binary size into
bytes when submitting.

Now any field can have custom validators and error messages.
The form will fallback on predefined error messages if not configured.
It will use the error messages provided by the binary min and max
validators. It still provides the error message for a required field.

Fixes: https://tracker.ceph.com/issues/38287
Signed-off-by: Stephan Müller <smueller@suse.com>

Merge pull request #32005 from s0nea/wip-dashboard-reactivate-suites

mgr/dashboard: reactivate dashboard test suites

Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Laura Paduano <lpaduano@suse.com>

pybind/mgr: add_tox_test: Add mypy to TOX_ENVS

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>

mgr/dashboard: CephFS quota management

Now both CephFS quotas can be changed with a validation against the next
tree maximum in the file tree, that prevents setting the quotas in a way
that would not be usable.

Fixes: https://tracker.ceph.com/issues/38287
Signed-off-by: Stephan Müller <smueller@suse.com>

rbd-mirror: fix race on namespace replayer initialization failure

Fixes: https://tracker.ceph.com/issues/43289
Signed-off-by: Mykola Golub <mgolub@suse.com>

Merge pull request #31851 from votdev/issue_42609_device_lights

mgr/dashboard: Add support for blinking enclosure LEDs

Reviewed-by: Lenz Grimmer <lgrimmer@suse.com>
Reviewed-by: Kiefer Chang <kiefer.chang@suse.com>
Reviewed-by: Tiago Melo <tmelo@suse.com>

stop: Unmount cephFS volumes

When stop.sh is run while the cephFS is mounted,
mount processes are retained and can't be killed.
And also the mount and df commands hang after this.
This patch unmounts the mounted volumes cleanly.

Signed-off-by: Kotresh HR <khiremat@redhat.com>

Merge pull request #32186 from rhcs-dashboard/43263-fix-rgw-subuser-key

mgr/dashboard: fix RGW subuser auto-generate key.

Reviewed-by: Ernesto Puertat <epuertat@redhat.com>
Reviewed-by: Volker Theile <vtheile@suse.com>
Reviewed-by: Laura Paduano <lpaduano@suse.com>

Merge pull request #31753 from smithfarm/wip-bci-resolve-fix

backport-create-issue: resolve parent only if parent has backport issues

Reviewed-by: Tiago Melo <tmelo@suse.com>

mgr/dashboard: Simplify PrometheusService::getRules implementa… (#32130)

mgr/dashboard: Simplify PrometheusService::getRules implementation

Reviewed-by: Stephan Müller <smueller@suse.com>
Reviewed-by: Tiago Melo <tmelo@suse.com>

mgr/dashboard: iSCSI targets not available if any gateway is down

Fixes: https://tracker.ceph.com/issues/42687
Signed-off-by: Ricardo Marques <rimarques@suse.com>

mgr/cephadm: Add unittest for service_action

(That was missing previously)

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>

Merge pull request #32177 from Takios/ceph_volume_fix

ceph-volume: use correct extents if using db-devices and >1 osds_per_device

mgr/dashboard: reactivate dashboard test suites

Reactivate the dashboard test suites that were commented
out in https://github.com/ceph/ceph/pull/30864 because
https://tracker.ceph.com/issues/41538 has been resolved
in the meanwhile.

Fixes: https://tracker.ceph.com/issues/42652
Signed-off-by: Tatjana Dehler <tdehler@suse.com>