Merge branch 'master' into wip-doc-prompts-update

Merge pull request #31777 from smithfarm/wip-mimic-v13.2.7-release-notes

doc/releases: add release notes for mimic v13.2.7

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Jan Fajerski <jfajerski@suse.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>

doc: Update commands in bootstrap.rst

This commit adds information about which prompt, and which node, commands
must be run from during the bootstrapping process.

Signed-off-by: Zac Dover <zac.dover@gmail.com>

Merge pull request #30387 from rzarzynski/wip-crimson-errorator

crimson: bring ceph::errorator with its first appliances

Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge PR #31793 into master

* refs/pull/31793/head:
mds: release free heap pages after trim

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #31799 from yuriw/wip-yuriw-crontab-master

qa/tests: trying to fix syntax error that prevented mimic-x to be add…

qa/tests: trying to fix syntax error that prevented mimic-x to be added to the schedule

Signed-off-by: Yuri Weinstein <yweinste@redhat.com>

Merge PR #31465 into master

* refs/pull/31465/head:
doc/install/containers: add summary of containers and branches

Reviewed-by: Dan Mick <dmick@redhat.com>

Merge PR #31702 into master

* refs/pull/31702/head:
ceph-daemon: infer fsid for shell, enter, ceph-volume, unit, logs

Reviewed-by: Michael Fritch <mfritch@suse.com>
Reviewed-by: Sebastian Wagner <swagner@suse.com>

doc/releases: add release notes for mimic v13.2.7

Signed-off-by: Nathan Cutler <ncutler@suse.com>

Merge pull request #30960 from yuvalif/wip-yuval-add-kafka-notif-endpoint

rgw/pubsub: add kafka notification endpoint

Reviewed-by: Matt Benjamin <mbenjamin@redhat.com>

mds: release free heap pages after trim

MDS free heap space can grow to large for some workloads (like smallfile
and recursive deletes). This can cause the MDS mapped memory to grow
well beyond memory targets.

When we finally use the PriorityCache in the MDS, this will not be
necessary anymore as the PriorityCache already does this.

Fixes: https://tracker.ceph.com/issues/42938
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

Merge PR #31747 into master

* refs/pull/31747/head:
qa/suites/rados/singleton-nomsgr/all/balancer: whitelist PG_AVAILABILITY

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #31734 from smithfarm/wip-cbs-fork-option

ceph-backport.sh: allow user to specify --fork explicitly

Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>

Merge PR #31120 into master

* refs/pull/31120/head:
mds: Reorganize class members in MDSRank header

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

mgr/dashboard: Cross sign button not working for some modals (#31783)

mgr/dashboard: Cross sign button not working for some modals

Reviewed-by: Volker Theile <vtheile@suse.com>

mgr/dashboard: sort monitors by open sessions correctly. (#31752)

mgr/dashboard: sort monitors by open sessions correctly.

Reviewed-by: Ernesto Puerta <epuertat@redhat.com>

Merge pull request #31738 from ricardoasmarques/wip-41776

mgr/dashboard: Unable to set boolean values to false when default is true

Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Tatjana Dehler <tdehler@suse.com>

Merge pull request #31773 from idryomov/wip-krbd-snapshot-removal-test

qa: kernel.sh: update for read-only changes

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

mgr/dashboard: Don't use any xlf file when building the defaul… (#31727)

mgr/dashboard: Don't use any xlf file when building the default language

Reviewed-by: Tiago Melo <tmelo@suse.com>

Merge PR #31748 into master

* refs/pull/31748/head:
qa/standalone/test_ceph_daemon.sh: remove old vg before creating
qa/standalone/test_ceph_daemon.sh: sudo for untar
qa/standalone/test_ceph_daemon.sh: sudo for losetup etc
qa/standalone/test_ceph_daemon.sh: fix overwrites of temp files

Reviewed-by: Michael Fritch <mfritch@suse.com>

mgr/dashboard: Update dev documentation URL (#31724)

mgr/dashboard: Update dev documentation URL

Reviewed-by: Tiago Melo <tmelo@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>

mgr/dashboard: Use new ImageSpec class (#31622)

mgr/dashboard: Use new ImageSpec class

Reviewed-by: Kiefer Chang <kiefer.chang@suse.com>
Reviewed-by: Ricardo Marques <rimarques@suse.com>

mgr/dashboard: Dashboard can't handle self-signed cert on Graf… (#31129)

mgr/dashboard: Dashboard can't handle self-signed cert on Grafana API

Reviewed-by: Stephan Müller <smueller@suse.com>

mgr/dashboard: open files with UTF-8 encoding in Grafana check… (#31750)

mgr/dashboard: open files with UTF-8 encoding in Grafana checking script

Reviewed-by: Laura Paduano <lpaduano@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>

mgr/dashboard: Cross sign button not working for some modals

Fixes: https://tracker.ceph.com/issues/42179
Signed-off-by: Ricardo Marques <rimarques@suse.com>

mds: Reorganize class members in MDSRank header

Fixes: https://tracker.ceph.com/issues/42465
Signed-off-by: Varsha Rao <varao@redhat.com>

qa/standalone/test_ceph_daemon.sh: remove old vg before creating

Signed-off-by: Sage Weil <sage@redhat.com>

qa/standalone/test_ceph_daemon.sh: sudo for untar

The deepsea.tgz tar contains actual device nodes for the OSD block devices
(not symlinks or files).  Must be root to untar.

Signed-off-by: Sage Weil <sage@redhat.com>

qa/standalone/test_ceph_daemon.sh: sudo for losetup etc

Signed-off-by: Sage Weil <sage@redhat.com>

qa/standalone/test_ceph_daemon.sh: fix overwrites of temp files

mktemp creates these files, so we have to pass --allow-overwrite (or
delete them after we get the unique name but before we write to them--this
is easier).

Broken by c7fe27a72a61d1345a66b8830fd17e7b922abd44

Signed-off-by: Sage Weil <sage@redhat.com>

Merge PR #31418 into master

* refs/pull/31418/head:
test: use distinct subvolume/group/snapshot names

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Ramana Raja <rraja@redhat.com>

Merge PR #30553 into master

* refs/pull/30553/head:
qa/tasks: check if fs mounted in umount_wait

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge PR #31534 into master

* refs/pull/31534/head:
mds: fix invalid access of mdr->dn[0].back()

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge PR #31548 into master

* refs/pull/31548/head:
mds: properly evaluate unstable locks when evicting client

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge PR #31531 into master

* refs/pull/31531/head:
cmake: link ceph-fuse against librt

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

qa: kernel.sh: update for read-only changes

Snapshot existence validation code was removed from krbd.  It was racy
and relied on having watch established for snapshots.

Fixes: https://tracker.ceph.com/issues/42916
Signed-off-by: Ilya Dryomov <idryomov@gmail.com>

crimson: avoid seastar::do_with() due to performance reasons.

`seastar::do_with(T&& rvalue, F&& f) takes object for lifetime
extension by rvalue reference. This imposes materialization of
a temporary to move from even when `do_with()` is being called
like:

  `do_with(OpsExecuter{...}, [] { /* ... */)`.

The reason behind that is following language rule:

  "Temporary objects are created when a prvalue is materialized
  so that it can be used as a glvalue, which occurs (since C++17)
  in the following situations:

   * binding a reference to a prvalue"
  (from: "Temporary object lifetime", cppreference.com)

As OpsExecuter is pretty heavy-weight, it is reasonable to avoid
`do_with()` and perform the lifetime extension with smart pointer.
Additional benefit is squeezing plain-to-errorated conversion in
`seastar::internal::do_with_state::get_future()`.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

crimson: OpsExecuter::submit_changes() operates on errorated future.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

crimson: introduce make_exception_future to errorator.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

crimson: drop extra copy in errorator's appliance.

Save few unneeded instructions on the read path. I expect
negligible impact on performance. The patch is rather for
easier reading of assembler.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

crimson: dissect error handling path from ::safe_then().

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

crimson: drop ceph::do_with(). Specialize sestar::do_with() instead.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

crimson: specialize seastar::futurize to avoid copying.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

crimson: _load_{os,ss} return errorated future to avoid copying.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

crimson: drop as_plain_future() from errorated futures.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

crimson: replace make_plain_exception_future() with errorize{}.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

crimson: clean-up errorator's error set checks.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

crimson/osd: apply errorator along the stat path.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

crimson: introduce passing-further variant of safe_then() to errorator.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

crimson/osd: switch small occurances of throwing to errorator.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

crimson/osd: apply errorator along the get_object_state() path.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

crimson: avoid plainifying/errorating in crimson::for_for_each().

We don't want to convert between `errorator::future` and
`seastar::future` back and forth due to performance penalty.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

crimson: implement ceph::do_{for_each(),do_with()} helpers.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

crimson/osd: apply errorator to op-taking PGBackend::getxattr().

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

crimson/osd: apply the errorator along execute_osd_op() path.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

crimson/osd: delegate executing osd op in objclass.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

crimson: add support for unificating errorators into compounds.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

crimson: deduplicate errorator's error set.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

crimson: use std::error_code instances for errors.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

crimson: unthrowable_wrapper::handle() accepts type-taking lambda.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

crimson/osd: erroratorize the do_op_call() path more.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

crimson/osd: apply errorator to do_op_call() path.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

crimson: introduce make_plain_exception_future() to errorator.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

crimson: introduce stateful errors to errorator.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

crimson/osd: handle attr retrieval errors with all_same_way().

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

crimson: introduce all_same_way() to errorator.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

crimson: don't allocate exception_ptr when returning error code.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

crimson: formalize the interface between errors and errorators.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

crimson/osd: apply errorator along the read path, part 5.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

crimson: rework discarding errors in errorator.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

crimson: introduce syntactic sugar for visiting errorator.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

crimson: migrate make_error<T> to unthrowable_wrapper.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

crimson: abstract the error type in unthrowable_wrapper.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

crimson: decouple errorator from unthrowable_wrapper.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

crimson: decouple errorator from unthrowable_wrapper::instance.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

crimson: add more static asserts to errorator.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

crimson: dissect maybe_handler_error_t from errorator template.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

crimson/osd: apply errorator along the read path, part 4.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

crimson: let an errorator to be converted into more errorated one.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

crimson/osd: apply errorator along the read path, part 3.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

crimson: facilitate errorator extending with new errors.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

crimson/osd: apply errorator along the read path, part 2.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

crimson/os, crimson/osd: apply errorator along the read path, part 1.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

crimson: bring discard_all{} to errorator.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

crimson: bring compile time-verified chaining of errorators.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

crimson/osd: dissect crc checking on reads into separated procedure.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

crimson/os: alias errorator for CyanStore::get_attrs().

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

crimson/os: drop the old FuturizedStore::Exception class.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

crimson: bring crimson::errorator with its first appliance.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

Merge PR #31745 into master

* refs/pull/31745/head:
qa/suites/rados/thrash-old-clients: skip TestClsRbd.mirror

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #31392 from dillaman/wip-42598

journal: fix flush by age and in-flight byte tracking

Reviewed-by: Mykola Golub <mgolub@suse.com>

mgr/dashboard: Use new ImageSpec class

This class should be used when dealing with RBD image specs.
It allows the creation of an ImageSpec given the names of the pool, image and
namespace. Alternatively you can also create one with an already existing
image spec string.

With it you keep the access to each individual component and can also convert it
to a well formated string.

Fixes: https://tracker.ceph.com/issues/42787
Signed-off-by: Tiago Melo <tmelo@suse.com>

mgr/dashboard: fix restored RBD image naming issue (#31590)

mgr/dashboard: fix restored RBD image naming issue

Reviewed-by: Laura Paduano <lpaduano@suse.com>
Reviewed-by: Tiago Melo <tmelo@suse.com>

Merge pull request #31758 from votdev/fix_typo

python-common: Fix typo in device type

Reviewed-by: Joshua Schmid <jschmid@suse.de>
Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>

mgr/dashboard: Don't use any xlf file when building the default language

The build-i18n script no longer uses a xlf file when building the default
language. This means that we don't need to keep the messages.en-US.xlf in the repository anymore.

Fixes: https://tracker.ceph.com/issues/42693
Signed-off-by: Sebastian Krah <skrah@suse.com>

Merge pull request #31410 from jan--f/c-v-create-vgs-on-demand

ceph-volume: allow raw block devices everywhere

mgr/dashboard: Fixes removal of custom tags during pool edit (#31373)

mgr/dashboard: Fixes removal of custom tags during pool edit

Reviewed-by: Ricardo Marques <rimarques@suse.com>
Reviewed-by: Tiago Melo <tmelo@suse.com>

python-common: Fix typo in device type

Signed-off-by: Volker Theile <vtheile@suse.com>

Merge pull request #31751 from tchaikov/wip-run-tox.sh-python3

script/run_tox.sh: do not use python2 if we have python3

Reviewed-by: Sebastian Wagner <swagner@suse.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>