Merge pull request #52967 from zdover23/wip-doc-2023-08-14-backport-52966-to-reef

reef: doc: update rados.cc

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>

Merge pull request #52970 from idryomov/wip-cmake-fmt-versionbump-reef

reef: cmake: ensure fmtlib is at least 8.1.1

Reviewed-by: Kefu Chai <tchaikov@gmail.com>

cmake: ensure fmt lib is atleast 8.1.1

Since we depend on newer api features like `group_digits` ensure that the system
version of fmt we use is at least as new as the submodules we bring.

Signed-off-by: Abhishek Lekshmanan <abhishek.l@cern.ch>
(cherry picked from commit eead42d4b1ec1bc1d7d2267b3ce97b8f34a7fb9b)

doc: update rados.cc

Clarify a line in ceph/src/tools/rados/rados.cc.

Signed-off-by: Zac Dover <zac.dover@proton.me>
(cherry picked from commit 152a0fdc6858f95dadb6c86fbb2f626f4a608309)

Merge pull request #52958 from zdover23/wip-doc-2023-08-12-backport-52957-to-reef

reef: doc/rados: update monitoring-osd-pg.rst

doc/rados: update monitoring-osd-pg.rst

Ingest Anthony D'Atri's notes from
https://github.com/ceph/ceph/pull/50856#discussion_r1289532902
which should have been included earlier.

Signed-off-by: Zac Dover <zac.dover@proton.me>
(cherry picked from commit 73503d4e2665bee9c49a055c11be1ffe2db253a1)

Merge pull request #52615 from trociny/wip-59064-reef

reef: rgw: fix multipart upload object leaks due to re-upload

Reviewed-by: Shilpa Jagannath <smanjara@redhat.com>

Merge pull request #52604 from cbodley/wip-62137-reef

reef: rgw: s3 object lock avoids overflow in retention date

Reviewed-by: Adam Emerson <aemerson@redhat.com>

Merge pull request #52912 from zdover23/wip-doc-2023-08-10-backport-52702-to-reef

reef: doc/start: update linking conventions

Merge pull request #52910 from zdover23/wip-doc-2023-08-10-backport-52715-to-reef

reef: doc/radosgw: update rate limit management

doc/start: update linking conventions

Update linking conventions in doc/start/documenting-ceph.rst, as
requested in
https://github.com/ceph/ceph/pull/49889#issuecomment-1657750905 by
bluikko. This commit introduces a convention that could be read as
inconsistent with other conventions mentioned on the page altered by
this commit. Future PRs will attenuate this inconsistency.

Signed-off-by: Zac Dover <zac.dover@proton.me>
(cherry picked from commit 52e8ec076f7bc65c13f61608b2a4ee0093569947)

doc/radosgw: update rate limit management

Improve the grammar and readability of the "Rate Limit Management"
section of doc/radosgw/admin.rst. This commit follows from
https://github.com/ceph/ceph/pull/52131.

Signed-off-by: Zac Dover <zac.dover@proton.me>
(cherry picked from commit 51b7ff76fce00264181f7d91b8333235be9562ab)

Merge pull request #52905 from zdover23/wip-doc-2023-08-09-backport-52490-to-reef

reef: doc: Add Reef 18.2.0 release notes

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>

Merge pull request #52835 from zdover23/wip-doc-2023-08-06-backport-52824-to-reef

reef: doc/radosgw: Improve language and formatting in config-ref.rst

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>

doc: Add Reef 18.2.0 release notes

Add release notes for the first stable release of Ceph Reef (18.2.0).

Co-authored-by: Anthony D'Atri <anthony.datri@gmail.com>
Co-authored-by: Ville Ojamo <14869000+bluikko@users.noreply.github.com>
Co-authored-by: Laura Flores <lflores@redhat.com>
Signed-off-by: Zac Dover <zac.dover@proton.me>
(cherry picked from commit 411017878edf7804f84e8e7e763a9a307edc92ca)

Merge pull request #52887 from zdover23/wip-doc-2023-08-09-backport-52884-to-reef

reef: doc/cephadm: fix typo in cephadm initial crush location section

Merge pull request #52889 from zdover23/wip-doc-2023-08-09-backport-52745-to-reef

reef: doc/rados: edit operations/add-or-rm-mons (1 of x)

doc/rados: edit operations/add-or-rm-mons (1 of x)

Edit the first part of doc/rados/operations/add-or-rm-mons.rst.

Co-authored-by: Anthony D'Atri <anthony.datri@gmail.com>
Signed-off-by: Zac Dover <zac.dover@proton.me>
(cherry picked from commit a0e638129da344cecb551b603491d94c4483099a)

doc/cephadm: fix typo in cephadm initial crush location section

A minor typo fix found while skimming through the cephadm docs:
"will no remove" -> "will not remove".

Signed-off-by: John Mulligan <jmulligan@redhat.com>
(cherry picked from commit 6e22b7b2d3a4ee11dfcc752f60563aabfe0cb9b3)

doc/radosgw: Improve language and formatting in config-ref.rst

The "Topic persistency settings" section includes convoluted
and incomplete sentences so attempt to improve the
language. The section lacks context so add a link to the
relevant documentation about Notifications.

The "Garbage Collection settings" note includes a sentence
that looks incomplete so try to improve the language.

Don't capitalize "Ops" in "RGW Ops" especially when the
same sentence talks about "op queue" with lower case "o".

Also improve formatting: indentation of a note and wrap
several very long lines.

Signed-off-by: Ville Ojamo <14869000+bluikko@users.noreply.github.com>
(cherry picked from commit 9dd86773b39582322a5577d7cc1521486c1bdaa9)

Merge pull request #52872 from zdover23/wip-doc-2023-08-08-backport-52860-to-reef

reef: doc/man/8/ceph-monstore-tool: add documentation

doc/man/8/ceph-monstore-tool: add documentation

Signed-off-by: Matan Breizman <mbreizma@redhat.com>
(cherry picked from commit ccfe7e50b6985dc12352437368ea66d80c2f95df)

Merge pull request #52536 from rhcs-dashboard/wip-62017-reef

reef: mgr/dashboard: disable promote on mirroring not enabled

Reviewed-by: Pegonzal <NOT@FOUND>

Merge pull request #52705 from rhcs-dashboard/wip-62238-reef

reef: mgr/dashboard: allow PUT in CORS

Reviewed-by: Avan Thakkar <athakkar@redhat.com>

Merge pull request #52457 from kamoltat/wip-ksirivad-backport-48209

reef: osd/OSDMap: Check for uneven weights & != 2 buckets post stretch mode

Reviewed-by: Ronen Friedman <rfriedma@redhat.com>

Merge pull request #52507 from lxbsz/wip-61984

reef: client: force sending cap revoke ack always

Reviewed-by: Venky Shankar <vshankar@redhat.com>

Merge pull request #50813 from dparmar18/wip-59260-reef

reef: mds: fix stray evaluation using scrub and introduce new option

Reviewed-by: Venky Shankar <vshankar@redhat.com>

Merge pull request #52841 from zdover23/wip-doc-2023-08-07-backport-52703-to-reef

reef: doc/start: update linking conventions

doc/start: update linking conventions

Update the "External Links" section of doc/start/documenting-ceph.rst.

Signed-off-by: Zac Dover <zac.dover@proton.me>
(cherry picked from commit 63750514d6ef7bf9dce6aa7c622664ea4310b59e)

Merge pull request #52839 from zdover23/wip-doc-2023-08-07-backport-52827-to-reef

reef: doc/rados: edit troubleshooting-mon.rst (2 of x)

doc/rados: edit troubleshooting-mon.rst (2 of x)

Edit the second part of
doc/rados/troubleshooting/troubleshooting-mon.rst.

Signed-off-by: Zac Dover <zac.dover@proton.me>
(cherry picked from commit 9938955c091274313c345d680001558810e33d2d)

Merge pull request #52833 from cbodley/wip-doc-rgw-zonegroup-feature-default-reef

reef: doc/rgw: correct statement about default zone features

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>

doc/rgw: correct statement about default zone features

this sentence wasn't updated when we added the Default column to the
table below

Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit bb5ad067ed939c83b6ed5894eda0577e781bb5b3)

Merge pull request #52825 from zdover23/wip-doc-2023-08-05-backport-52778-to-reef

reef: doc/rados: edit operations/add-or-rm-mons (2 of x)

doc/rados: edit operations/add-or-rm-mons (2 of x)

Edit the second part of doc/rados/operations/add-or-rm-mons.rst.

Signed-off-by: Zac Dover <zac.dover@proton.me>
(cherry picked from commit 4e790603d9e4645234d3953e88a34dd3a8a10068)

Merge pull request #52213 from batrick/wip-61836-reef

reef: pybind/mgr/devicehealth: do not crash if db not ready

Reviewed-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Laura Flores <lflores@redhat.com>

Merge pull request #51698 from dparmar18/wip-61204-reef

reef: MDS imported_inodes metric is not updated.

Reviewed-by: Venky Shankar <vshankar@redhat.com>

Merge pull request #52578 from rishabh-d-dave/wip-59410-reef

reef: client: correct quota check in Client::_rename()

Reviewed-by: Venky Shankar <vshankar@redhat.com>

Merge pull request #52515 from neesingh-rh/wip-61987-reef

reef:  mds: session ls command appears twice in command listing

Merge pull request #52512 from joscollin/wip-62054-reef

reef: mds: MDLog::_recovery_thread: handle the errors gracefully

Reviewed-by: Venky Shankar <vshankar@redhat.com>

Merge pull request #52504 from lxbsz/wip-62011

reef: client: wait rename to finish

Reviewed-by: Venky Shankar <vshankar@redhat.com>

Merge pull request #52501 from lxbsz/wip-62041

reef: client: do not send metrics until the MDS rank is ready

Reviewed-by: Venky Shankar <vshankar@redhat.com>

Merge pull request #52497 from lxbsz/wip-62044

reef: client: trigger to flush the buffer when making snapshot

Reviewed-by: Venky Shankar <vshankar@redhat.com>

Merge pull request #52235 from batrick/wip-59373-reef

reef: qa: wait for MDSMonitor tick to replace daemons

Reviewed-by: Venky Shankar <vshankar@redhat.com>

Merge pull request #52774 from rhcs-dashboard/wip-62223-reef

reef: mgr/dashboard: empty grafana panels for performance of daemons

Reviewed-by: Juan Miguel Olmo <jolmomar@redhat.com>

Merge pull request #52801 from ceph/reef-release

v18.2.0

Merge pull request #52805 from yaarith/release-checklists-telemetry

reef: doc/dev/release-checklist: check telemetry validation

Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>

mgr/prometheus: introduce fetch_perf_counters_metrics module option

Fixes: https://tracker.ceph.com/issues/58164
Signed-off-by: Avan Thakkar <athakkar@redhat.com>
Introducing module option in prometheus module to enable/disable support for exporting ceph daemons
perf counters as prometheus metrics, by default this option will be disabled. The use case for this option
is in case if ceph-exporter deployment failed for any reason then user still can have option to fetch metrics
from promethues exporter.

Merge pull request #52803 from zdover23/wip-doc-2023-08-04-backport-52800-to-reef

reef: doc/rgw: several response headers are supported

doc/dev/release-checklist: check telemetry validation

Signed-off-by: Yaarit Hatuka <yaarit@redhat.com>

doc/rgw: several response headers are supported

Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit ab6ec66a2f82c3dfb3807d5e76d3290ff30bf613)

Merge pull request #52769 from adamemerson/wip-62097-reef

reef: build: Remove ceph-libboost* packages in install-deps

Reviewed-by: Ilya Dryomov <idryomov@gmail.com>

18.2.0

Signed-off-by: Ceph Release Team <ceph-maintainers@ceph.io>

Merge pull request #52716 from neha-ojha/wip-reef-stable

reef: src/ceph_release, doc: mark reef stable

Reviewed-by: Yuri Weinstein <yweinste@redhat.com>
Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>
Reviewed-by: Zac Dover <zac.dover@gmail.com>

build: Remove old ceph-libboost* packages in install-deps

Here, we extract `clean_boost_on_ubuntu()` and call it before other
installs on Debian distributions so that if we install a system boost,
a potentially newer `ceph-libboost` won't get in the way.

As the sources.list.d being removed in the original cleanup code isn't
the one we're currently installing in the install code, add a removal
for the currently used source, then do apt-update so packages from the
removed source are no longer included as available.

Two subsidiary dev packages from conflicting boost libraries can be
installed, but it leaves apt in an inconsistent state. To clean this
up, add `--fix-missing` to the removal line and call
`clean_boost_on_ubuntu()` before other uses of apt.

Fixes: https://tracker.ceph.com/issues/62097
Signed-off-by: Adam Emerson <aemerson@redhat.com>
(cherry picked from commit 0c3f511e14af639b6509e69b889258b2f718f8fd)

Conflicts:
install-deps.sh
 - Different boost version for Reef than Squid
 - ci_debug does not exist in Reef
 - No INSTALL_EXTRA

Signed-off-by: Adam Emerson <aemerson@redhat.com>

Merge pull request #52779 from zdover23/wip-doc-2023-08-03-backport-52775-to-reef

reef: doc: for EC we recommend K+1

doc: for EC we recommend K+1

Update the doc to match the reality in the code. I don't know where
the recommendation to have min_size = k+2 came from, but for awhile
now we've defaulted to K+1. See PR #8008.

Signed-off-by: Dan van der Ster <dan.vanderster@clyso.com>
(cherry picked from commit 68faf88e6babf7e5c2cce2b4d9b80a159b38c30d)

exporter: fix instance_id labeld value for rgw in exporter

Signed-off-by: avanthakkar <avanjohn@gmail.com>
(cherry picked from commit ca5691834cc30c4f9175bb0b4730750ed9df59cc)

mgr/dashboard: empty grafana panels for performance of daemons

Fixes: https://tracker.ceph.com/issues/61792
Signed-off-by: avanthakkar <avanjohn@gmail.com>
Removing the `ceph-` prefix from ceph_daemon label to adopt it with the label
format used by queries in grafana dashboards. Also changing the
`instance_id` label for rgw to match the values coming from
exporter and prometheus module

(cherry picked from commit fa2d1f9f0745ffecc671af3a3f0a93e7edaa1222)

Merge pull request #52710 from cbodley/wip-62247-reef

reef: qa/rgw: pacific upgrade disables centos9

Reviewed-by: Yuri Weinstein <yweinste@redhat.com>

Merge pull request #52758 from zdover23/wip-doc-2023-08-02-backport-51960-to-reef

reef: doc: improve doc/dev/encoding.rst

doc: improve doc/dev/encoding.rst

Signed-off-by: Radosław Zarzyński <rzarzyns@redhat.com>
(cherry picked from commit 83bef6668d319685923df927f32590b4e1795746)

Merge pull request #52731 from rhcs-dashboard/flake8-fix-reef

reef: mgr: fix some flake8 complaints

Reviewed-by: Adam King <adking@redhat.com>

mgr: fix some flake8 complaints

Signed-off-by: Nizamudeen A <nia@redhat.com>
(cherry picked from commit d199782fb5f3c22f1cd5fda6ddf1d64f40a92726)

Merge pull request #52712 from zdover23/wip-doc-2023-08-01-backport-52131-to-reef

reef: doc/radosgw: correct emphasis in rate limit section

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>

src/ceph_release, doc: mark reef stable

Signed-off-by: Neha Ojha <nojha@redhat.com>

doc/radosgw: correct emphasis in rate limit section

Signed-off-by: Piotr Parczewski <piotr@stackhpc.com>
(cherry picked from commit 50bfbab6d9a30b3ba6ec4f4c8df9059298c28a7f)

qa/rgw: pacific upgrade disables centos9

we don't build centos 9 packages for pacific, so the pacific upgrade
suite was failing with errors like:

Failed to fetch package version from https://shaman.ceph.com/api/search/?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=pacific

Fixes: https://tracker.ceph.com/issues/62135
Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit b16bc1cd07a28d2a5677c2fc107686f0b96648ce)

mgr/dashboard: allow PUT in CORS

Fixes: https://tracker.ceph.com/issues/62222
Signed-off-by: Nizamudeen A <nia@redhat.com>
(cherry picked from commit 8da2a2c0e296a94a2bfab6ab68480d2e33e7d84e)

Merge pull request #52679 from zdover23/wip-doc-2023-07-28-backport-52671-to-reef

reef: doc: update README.md install procedure

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>

Merge pull request #52693 from zdover23/wip-doc-2023-07-29-backport-52668-to-reef

reef: doc/radosgw: Add missing space to date option spec in admin.rst

Merge pull request #52695 from zdover23/wip-doc-2023-07-29-backport-52685-to-reef

reef: doc/mgr/ceph_api: Promptify example commands in index.rst

doc/mgr/ceph_api: Promptify example commands in index.rst

Use the more modern prompt block instead of
using code blocks for example commands.

Signed-off-by: Ville Ojamo <14869000+bluikko@users.noreply.github.com>
(cherry picked from commit 4f9bd62f374d2f6bf74296d87e5205960464c349)

doc/radosgw: Add missing space to date option spec in admin.rst

The start time and end time CLI option specification is missing a space between the date and the optional time value. Also expand the text to talk about "optional time" after the date.

Signed-off-by: Ville Ojamo <14869000+bluikko@users.noreply.github.com>
(cherry picked from commit dd16ad9055f134433d9bd6410cd31eb0fc918723)

Merge pull request #52687 from rhcs-dashboard/wip-62220-reef

reef: exporter: ceph-exporter scrapes failing on multi-homed server

Reviewed-by: Juan Miguel Olmo <jolmomar@redhat.com>

Merge pull request #52618 from vshankar/tr-62147

reef: qa: use centos 8.stream for fs:upgrade tests

Reviewed-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Yuri Weinstein <yweinste@redhat.com>

Merge pull request #52664 from idryomov/wip-61565-reef

reef: qa/workunits/rbd: use jammy version of qemu-iotests for centos 9

Reviewed-by: Ramana Raja <rraja@redhat.com>

exporter: ceph-exporter scrapes failing on multi-homed server

Fixes: https://tracker.ceph.com/issues/61669
Signed-off-by: Avan Thakkar <athakkar@redhat.com>
Adopted ceph-exporter deployment in cephadm
to bind to all addresses on a machine.

(cherry picked from commit 9a6f01d0d229177a23175b5985d12aa140ca6271)

doc: update README.md install procedure

Add instructions directing the reader to install the "python3-routes"
package. This package is required in order to launch the dashboard after
the installation procedure has completed, but is not yet included in the
install-deps.sh script.

Signed-off-by: Zac Dover <zac.dover@proton.me>
(cherry picked from commit 0b0009bf1fb0cd54cdf0014bd724b00805a3dbd1)

Merge pull request #52641 from zdover23/wip-doc-2023-07-26-backport-52630-to-reef

reef: doc: update README.md

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>

qa/workunits/rbd: use jammy version of qemu-iotests for centos 9

It's the one we are using for all recent distros.

While at it, get rid of custom bin directory -- it appears that both
v2.3.0 and v2.11.0 tests are happy with just symlinks in the current
directory.

Fixes: https://tracker.ceph.com/issues/61565
Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
(cherry picked from commit d49df8d74ca96a3c0fc00f1d7e303d4e12f2f1e3)

doc: update README.md

Correct and improve ceph/README.md.

Signed-off-by: Zac Dover <zac.dover@proton.me>
(cherry picked from commit ad1202be685c1d35026b5ba96ba3f5cf3aefdb3a)

Merge pull request #52616 from ceph/reef-release

v18.1.3

Merge pull request #52612 from zdover23/wip-doc-2023-07-25-backport-49889-to-reef

reef: doc/rados/configuration: add links to MON DNS

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>

Merge pull request #52608 from zdover23/wip-doc-2023-07-25-backport-52601-to-reef

reef: doc/rados: remove redundant cache tiering deprecation note

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>

qa: use centos 8.stream for fs:upgrade tests

fs:upgrade tests use n/o/p release packages which aren't
available for centos 9.stream - switch using 8.stream instead.

Fixes: http://tracker.ceph.com/issues/62146
Signed-off-by: Venky Shankar <vshankar@redhat.com>
(cherry picked from commit b5756b98f989e8610f310ade4777cf6c87d72e23)

qa/rgw: test that multipart re-upload does not leave any orphans

Runs a boto script that reuploads one part multiple times before
completing and then we check for any orphans.

Original boto script contributed by Matt Benjamin
<mbenjami@redhat.com> on top of which modifications were made.

Signed-off-by: J. Eric Ivancich <ivancich@redhat.com>
(cherry picked from commit aeffd1b5986ad13f7a1f962cbff2fc72a5b21abe)

18.1.3

Signed-off-by: Ceph Release Team <ceph-maintainers@ceph.io>

doc/rados/configuration: add links to MON DNS

Add two links to MON DNS lookup and an anchor in the destination document.

Signed-off-by: Ville Ojamo <14869000+bluikko@users.noreply.github.com>
(cherry picked from commit 06cb5652157a53b25e8ae588f375548b1df72d2d)

Merge pull request #52377 from soumyakoduri/wip-skoduri-reef-dbstore

[reef]rgw/dbstore: Do not update mtime for attr change

Reviewed-by: Daniel Gryniewicz <dang@redhat.com>

doc/rados: remove redundant cache tiering deprecation note

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
(cherry picked from commit 12ccd294663b03f34a44424c6006248bb7469b73)

rgw: object lock uses std::chrono date/year conversions

Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit 1396796ca9e142202c202406792e31cacfd65aec)

rgw: s3 object lock avoids overflow in retention date

Fixes: https://tracker.ceph.com/issues/56993
Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit dd87a0ff845549ce94b740881c614c77c1857865)

Merge pull request #52574 from cbodley/wip-62106-reef

reef: valgrind: replace suppression for EVP_DecryptFinal_ex

Reviewed-by: Ilya Dryomov <idryomov@redhat.com>

Merge pull request #52567 from cbodley/wip-qa-distros-centos9-reef

reef: qa/distros: add centos stream 9 as supported distro

Reviewed-by: Ilya Dryomov <idryomov@redhat.com>

Merge pull request #52566 from cbodley/wip-62104-reef

reef: test/pybind: replace nose with pytest

Reviewed-by: Ilya Dryomov <idryomov@redhat.com>

Merge pull request #52078 from guits/wip-61700-reef

reef: ceph-volume: set lvm membership for mpath type devices

Merge pull request #52593 from zdover23/wip-doc-2023-07-23-backport-52587-to-reef

reef: doc/rados: edit ceph-conf.rst (4 of x)

doc/rados: edit ceph-conf.rst (4 of x)

Edit doc/rados/configuration/ceph-conf.rst.

Signed-off-by: Zac Dover <zac.dover@proton.me>
(cherry picked from commit ac47899863390ff9f2adf658273e1d61760a68c2)

Merge pull request #52588 from zdover23/wip-doc-2023-07-22-backport-52542-to-reef

reef: doc/rados: edit ceph-conf.rst (3 of x)