doc/cephadm: edit sentence in mgr.rst

Improve the grammar in the first sentence of
doc/cephadm/services/mgr.rst. This is a small change, but I could not
resist.

Signed-off-by: Zac Dover <zac.dover@proton.me>
(cherry picked from commit 3b8877481a8de0a92a7b68a775915eb079744f23)

Merge pull request #52228 from batrick/wip-61691-quincy

quincy:  mon/MDSMonitor: batch last_metadata update with pending

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #53060 from zdover23/wip-doc-2023-08-21-backport-52963-to-quincy

quincy: doc/troubleshooting: edit cpu-profiling.rst

doc/troubleshooting: edit cpu-profiling.rst

Improve the language in the "oprofile" section of the RADOS
Troubleshooting Guide.

Signed-off-by: Zac Dover <zac.dover@proton.me>
(cherry picked from commit 513c436aefc3c0b2e7d784c5c00e744c255a95b1)

Merge pull request #52503 from lxbsz/wip-62010

quincy: client: wait rename to finish

Merge pull request #52498 from lxbsz/wip-62045

quincy: client: trigger to flush the buffer when making snapshot

Merge pull request #50991 from lxbsz/wip-59199

quincy: qa: enable kclient test for newop test

Merge pull request #53058 from zdover23/wip-doc-2023-08-20-backport-53055-to-quincy

quincy: doc/README.md - edit "Building Ceph"

doc/README.md - edit "Building Ceph"

Improve the "Building Ceph" section of ceph/README.md.

Co-authored-by: Anthony D'Atri <anthony.datri@gmail.com>
Signed-off-by: Zac Dover <zac.dover@proton.me>
(cherry picked from commit 5bb3942365c4829f0b6580dec5d37e45f2e51e9d)

Merge pull request #53050 from zdover23/wip-doc-2023-08-18-backport-53035-to-quincy

quincy: doc: edit README.md - contributing code

doc: edit README.md - contributing code

Edit the "Contributing Code" section of the "ceph/README.md" file.

Signed-off-by: Zac Dover <zac.dover@proton.me>
(cherry picked from commit 9433a3e62f7453926e4be2512639f94beb887a4c)

Merge pull request #52516 from neesingh-rh/wip-61988-quincy

quincy: mds: session ls command appears twice in command listing

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #52514 from joscollin/wip-62056-quincy

quincy: mds: MDLog::_recovery_thread: handle the errors gracefully

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #52508 from lxbsz/wip-61985

quincy: client: force sending cap revoke ack always

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #52502 from lxbsz/wip-62042

quincy: client: do not send metrics until the MDS rank is ready

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #52303 from lxbsz/wip-61796

quincy: client: only wait for write MDS OPs when unmounting

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #52245 from batrick/wip-59303-quincy

quincy: tools/cephfs: add basic detection/cleanup tool for dentry first damage

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #52241 from batrick/wip-59558-quincy

quincy: qa: check each fs for health

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #52236 from batrick/wip-59371-quincy

quincy: qa: wait for MDSMonitor tick to replace daemons

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #52127 from joscollin/wip-61736-quincy

quincy: mds: display sane hex value (0x0) for empty feature bit

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #52074 from joscollin/wip-61695-quincy

quincy: debian: install cephfs-mirror systemd unit files and man page

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #51697 from dparmar18/wip-61203-quincy

quincy: MDS imported_inodes metric is not updated.

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #51690 from vshankar/tr-59722

quincy: qa: run scrub post file system recovery

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #51685 from lxbsz/wip-61348

quincy: mds: wait reintegrate to finish when unlinking

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #50922 from batrick/i59367

quincy: qa: test_rebuild_simple checks status on wrong file system

Reviewed-by: Venky Shankar <vshankar@redhat.com>

Merge pull request #50815 from dparmar18/wip-59262-quincy

quincy: mds: fix stray evaluation using scrub and introduce new option

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #50807 from dparmar18/wip-59250-quincy

quincy: mgr/nfs: disallow non-existent paths when creating export

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #50786 from vshankar/tr-59002

quincy: mgr/volumes: avoid returning -ESHUTDOWN back to cli

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #53025 from zdover23/wip-doc-2023-08-16-backport-51170-to-quincy

quincy: doc: Fix doc for mds cap acquisition throttle

doc: Fix doc for mds cap acquisition throttle

The config values picks the definition from the
description. Removed the duplicate description
in the doc.

Signed-off-by: Kotresh HR <khiremat@redhat.com>
(cherry picked from commit 65b0e45dd702fdedcf66f64dd6c5b8ae11df430b)

Merge pull request #53001 from zdover23/wip-doc-2023-08-16-backport-52991-to-quincy

quincy: doc: improve submodule update command - README.md

Merge pull request #52999 from zdover23/wip-doc-2023-08-15-backport-52761-to-quincy

quincy: doc/rados/configuration: Avoid repeating "support" in msgr2.rst

doc: improve submodule update command - README.md

Improve the portion of README.md that directs the reader to update the
submodules. Previously, the text assumed that the reader knew that they
should cd into the ceph directory. Now the text is explicit about the
necessity of this action.

Signed-off-by: Zac Dover <zac.dover@proton.me>
(cherry picked from commit 578fdb27a09c629003ab20c0d27267d4d97b5d6c)

doc/rados/configuration: Avoid repeating "support" in msgr2.rst

Instead of saying "[...] support is not supported"
change the first "support" to "operation".

Signed-off-by: Ville Ojamo <14869000+bluikko@users.noreply.github.com>
(cherry picked from commit df43ab5ba626c49aad6d3aca59c9737b06f8bfd8)

Merge pull request #52992 from idryomov/wip-62402-quincy

quincy: qa/suites/upgrade/octopus-x: skip TestClsRbd.mirror_snapshot test

Reviewed-by: Mykola Golub <mgolub@suse.com>

Merge pull request #52947 from zdover23/wip-doc-2023-08-11-backport-52945-to-quincy

quincy: doc: Add a note on possible deadlock on volume deletion

Reviewed-by: Cole Mitchell <cole.mitchell.ceph@gmail.com>

Merge pull request #52968 from zdover23/wip-doc-2023-08-14-backport-52966-to-quincy

quincy: doc: update rados.cc

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>

qa/suites/upgrade/octopus-x: skip TestClsRbd.mirror_snapshot test

The behavior of the class method changed in reef; the change was
backported to pacific and quincy.  An octopus test binary used against
quincy OSDs produces an expected failure:

    [ RUN      ] TestClsRbd.mirror_snapshot
    .../15.2.17/rpm/el8/BUILD/ceph-15.2.17/src/test/cls_rbd/test_cls_rbd.cc:2279: Failure
    Expected equality of these values:
      -85
      mirror_image_snapshot_unlink_peer(&ioctx, oid, 1, "peer2")
        Which is: 0
    [  FAILED  ] TestClsRbd.mirror_snapshot (6 ms)

Fixes: https://tracker.ceph.com/issues/62402
Signed-off-by: Ilya Dryomov <idryomov@gmail.com>

doc: update rados.cc

Clarify a line in ceph/src/tools/rados/rados.cc.

Signed-off-by: Zac Dover <zac.dover@proton.me>
(cherry picked from commit 152a0fdc6858f95dadb6c86fbb2f626f4a608309)

Merge pull request #52959 from zdover23/wip-doc-2023-08-12-backport-52957-to-quincy

quincy: doc/rados: update monitoring-osd-pg.rst

doc/rados: update monitoring-osd-pg.rst

Ingest Anthony D'Atri's notes from
https://github.com/ceph/ceph/pull/50856#discussion_r1289532902
which should have been included earlier.

Signed-off-by: Zac Dover <zac.dover@proton.me>
(cherry picked from commit 73503d4e2665bee9c49a055c11be1ffe2db253a1)

Merge pull request #52917 from rhcs-dashboard/wip-62375-quincy

quincy: mgr/dashboard: paginate hosts

Reviewed-by: Nizamudeen A <nia@redhat.com>

Merge pull request #52256 from ronen-fr/wip-rf-59537-quincy

Quincy: osd/scrub: verify SnapMapper consistency

Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

Merge pull request #52161 from pdvian/wip-61569-quincy

quincy: mon: Fix ceph versions command

Reviewed-by: Laura Flores <lflores@redhat.com>

Merge pull request #48171 from ifed01/wip-ifed-fix-blufs-unlink-fsync-qui

quincy: os/bluestore: do not signal deleted dirty file to bluefs log

Reviewed-by: Adam Kupczyk <akupczyk@ibm.com>

Merge pull request #48279 from ifed01/wip-ifed-fix-57688

quincy: os/bluestore: use direct write in BlueStore::_write_bdev_label

Reviewed-by: Adam Kupczyk <akupczyk@ibm.com>

doc: Add a note on possible deadlock on volume deletion

Fixes: https://tracker.ceph.com/issues/62407
Signed-off-by: Kotresh HR <khiremat@redhat.com>
(cherry picked from commit b93b3364ae00946b9eb1fa1406be0ee22e60baac)

Merge pull request #52079 from guits/wip-61699-quincy

quincy: ceph-volume: set lvm membership for mpath type devices

Merge pull request #52620 from guits/wip-62149-quincy

quincy: ceph-volume: fix raw list for lvm devices

Merge pull request #52929 from guits/wip-62037-quincy

quincy: ceph-volume: add --osd-id option to raw prepare

mgr/dashboard: invalidate rbd image cache on CRUD ops

Signed-off-by: Pere Diaz Bou <pdiazbou@redhat.com>
Fixes: https://tracker.ceph.com/issues/58710

mgr/dashboard: paginate hosts

Signed-off-by: Pere Diaz Bou <pere-altea@hotmail.com>
Fixes: https://tracker.ceph.com/issues/56513
(cherry picked from commit 720bb10e30bb7f686eb528d19c112bcf7ae7765c)

ceph-volume: add --osd-id option to raw prepare

This adds the support of the --osd-id option to the raw prepare command.

Fixes: https://tracker.ceph.com/issues/61995
Signed-off-by: Guillaume Abrioux <gabrioux@ibm.com>
(cherry picked from commit 16729656aa208deeedd631dcf871d501d6eba7f5)

Merge pull request #52642 from zdover23/wip-doc-2023-07-26-backport-52630-to-quincy

quincy: doc: update README.md

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>

Merge pull request #52913 from zdover23/wip-doc-2023-08-10-backport-52702-to-quincy

quincy: doc/start: update linking conventions

Merge pull request #52911 from zdover23/wip-doc-2023-08-10-backport-52715-to-quincy

quincy: doc/radosgw: update rate limit management

doc/start: update linking conventions

Update linking conventions in doc/start/documenting-ceph.rst, as
requested in
https://github.com/ceph/ceph/pull/49889#issuecomment-1657750905 by
bluikko. This commit introduces a convention that could be read as
inconsistent with other conventions mentioned on the page altered by
this commit. Future PRs will attenuate this inconsistency.

Signed-off-by: Zac Dover <zac.dover@proton.me>
(cherry picked from commit 52e8ec076f7bc65c13f61608b2a4ee0093569947)

doc/radosgw: update rate limit management

Improve the grammar and readability of the "Rate Limit Management"
section of doc/radosgw/admin.rst. This commit follows from
https://github.com/ceph/ceph/pull/52131.

Signed-off-by: Zac Dover <zac.dover@proton.me>
(cherry picked from commit 51b7ff76fce00264181f7d91b8333235be9562ab)

doc: update README.md

Correct and improve ceph/README.md.

Signed-off-by: Zac Dover <zac.dover@proton.me>
(cherry picked from commit ad1202be685c1d35026b5ba96ba3f5cf3aefdb3a)

Merge pull request #52836 from zdover23/wip-doc-2023-08-06-backport-52824-to-quincy

quincy: doc/radosgw: Improve language and formatting in config-ref.rst

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>

Merge pull request #52398 from mchangir/wip-61960-quincy

quincy: mon: block osd pool mksnap for fs pools

Reviewed-by: Venky Shankar <vshankar@redhat.com>

Merge pull request #52888 from zdover23/wip-doc-2023-08-09-backport-52884-to-quincy

quincy: doc/cephadm: fix typo in cephadm initial crush location section

Merge pull request #52890 from zdover23/wip-doc-2023-08-09-backport-52745-to-quincy

quincy: doc/rados: edit operations/add-or-rm-mons (1 of x)

doc/rados: edit operations/add-or-rm-mons (1 of x)

Edit the first part of doc/rados/operations/add-or-rm-mons.rst.

Co-authored-by: Anthony D'Atri <anthony.datri@gmail.com>
Signed-off-by: Zac Dover <zac.dover@proton.me>
(cherry picked from commit a0e638129da344cecb551b603491d94c4483099a)

doc/radosgw: Improve language and formatting in config-ref.rst

The "Topic persistency settings" section includes convoluted
and incomplete sentences so attempt to improve the
language. The section lacks context so add a link to the
relevant documentation about Notifications.

The "Garbage Collection settings" note includes a sentence
that looks incomplete so try to improve the language.

Don't capitalize "Ops" in "RGW Ops" especially when the
same sentence talks about "op queue" with lower case "o".

Also improve formatting: indentation of a note and wrap
several very long lines.

Signed-off-by: Ville Ojamo <14869000+bluikko@users.noreply.github.com>
(cherry picked from commit 9dd86773b39582322a5577d7cc1521486c1bdaa9)

doc/cephadm: fix typo in cephadm initial crush location section

A minor typo fix found while skimming through the cephadm docs:
"will no remove" -> "will not remove".

Signed-off-by: John Mulligan <jmulligan@redhat.com>
(cherry picked from commit 6e22b7b2d3a4ee11dfcc752f60563aabfe0cb9b3)

qa/workunites/fs: add first damage test script

Not part of the QA test suites because we don't want to install the
associated script.

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit e9730461a20739d2368a3473ede5ae4a7b0be2ff)

tools/cephfs: add basic detection/cleanup tool for dentry first damage

This tool iterates through the metadata pool looking for dentries with
non-sensical "first" (snapshot) metadata. It only specifically looks for
damage of the kind where the "first" value is greater than the latest
generated snapshot for the file system. It does not detect other kinds
of genuine damage to the "first" metadata.

Fixes: https://tracker.ceph.com/issues/56140
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit 5fa0825bf3be437f924edf10efc93304e605ac1c)

Merge pull request #52537 from rhcs-dashboard/wip-62018-quincy

quincy: mgr/dashboard: disable promote on mirroring not enabled

Reviewed-by: Pegonzal <NOT@FOUND>
Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Pere Diaz Bou <pdiazbou@redhat.com>

Merge pull request #52728 from rhcs-dashboard/wip-61998-quincy

quincy: mgr/dashboard: Add more decimals in latency graph

Reviewed-by: Nizamudeen A <nia@redhat.com>

Merge pull request #52842 from zdover23/wip-doc-2023-08-07-backport-52703-to-quincy

quincy: doc/start: update linking conventions

Merge pull request #52680 from zdover23/wip-doc-2023-07-28-backport-52671-to-quincy

quincy: doc: update README.md install procedure

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>

doc/start: update linking conventions

Update the "External Links" section of doc/start/documenting-ceph.rst.

Signed-off-by: Zac Dover <zac.dover@proton.me>
(cherry picked from commit 63750514d6ef7bf9dce6aa7c622664ea4310b59e)

Merge pull request #52840 from zdover23/wip-doc-2023-08-07-backport-52827-to-quincy

quincy: doc/rados: edit troubleshooting-mon.rst (2 of x)

doc/rados: edit troubleshooting-mon.rst (2 of x)

Edit the second part of
doc/rados/troubleshooting/troubleshooting-mon.rst.

Signed-off-by: Zac Dover <zac.dover@proton.me>
(cherry picked from commit 9938955c091274313c345d680001558810e33d2d)

Merge pull request #52826 from zdover23/wip-doc-2023-08-05-backport-52778-to-quincy

quincy: doc/rados: edit operations/add-or-rm-mons (2 of x)

doc/rados: edit operations/add-or-rm-mons (2 of x)

Edit the second part of doc/rados/operations/add-or-rm-mons.rst.

Signed-off-by: Zac Dover <zac.dover@proton.me>
(cherry picked from commit 4e790603d9e4645234d3953e88a34dd3a8a10068)

Merge pull request #52320 from mkogan1/wip-59739-quincy

quincy: rgw: D3N cache objects which oid contains slash

Reviewed-by: Casey Bodley <cbodley@redhat.com>

os/bluestore: do not signal removed dirty file to bluefs log

Fixes: https://tracker.ceph.com/issues/56533
Signed-off-by: Igor Fedotov <igor.fedotov@croit.io>
(cherry picked from commit 576cf8c076b37be978cea06fc5b7348aeffe6829)

test/test_bluefs: reproduce a weird file unlink+fsync scenario.

This triggers misordered op_update record in bluefs log which follows
op_unlink one.

Signed-off-by: Igor Fedotov <igor.fedotov@croit.io>
(cherry picked from commit 4240a315d663e2c68d796884c30be171dc73c397)

Merge pull request #52322 from batrick/wip-61898-quincy

quincy: pybind: drop GIL during library callouts

Reviewed-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Ilya Dryomov <idryomov@redhat.com>

Merge pull request #52243 from batrick/wip-61425-quincy

quincy: mon/MDSMonitor: ignore extraneous up:boot messages

Reviewed-by: Venky Shankar <vshankar@redhat.com>

Merge pull request #52239 from batrick/wip-61415-quincy

quincy: mon/MDSMonitor: do not propose on error in prepare_update

Reviewed-by: Venky Shankar <vshankar@redhat.com>

Merge pull request #52234 from batrick/wip-61412-quincy

quincy: mon/MDSMonitor: check fscid in pending exists in current

Reviewed-by: Venky Shankar <vshankar@redhat.com>

Merge pull request #52158 from cbodley/wip-57642-quincy

quincy: rgw: avoid string_view to temporary in RGWBulkUploadOp

Reviewed-by: Daniel Gryniewicz <dang@redhat.com>

Merge pull request #51102 from RaminNietzsche/wip-ifed-fix-require-osd-release-to-quincy

quincy: mon: avoid exception when setting require-osd-release more than 2

Reviewed-by: Igor Fedotov <ifedotov@suse.com>

Merge pull request #51002 from ifed01/wip-ifed-bluefs-duplicate-dir-link-qui

quincy: os/bluestore/bluefs: fix dir_link might add link that already exists in compact log

Reviewed-by: Adam Kupczyk <akupczyk@redhat.com>

Merge pull request #52804 from zdover23/wip-doc-2023-08-04-backport-52800-to-quincy

quincy: doc/rgw: several response headers are supported

doc/rgw: several response headers are supported

Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit ab6ec66a2f82c3dfb3807d5e76d3290ff30bf613)

Merge pull request #52564 from adamemerson/wip-62097-quincy

quincy: build: Remove ceph-libboost* packages in install-deps

Reviewed-by: Ilya Dryomov <idryomov@gmail.com>

Merge pull request #52780 from zdover23/wip-doc-2023-08-03-backport-52775-to-quincy

quincy: doc: for EC we recommend K+1

doc: for EC we recommend K+1

Update the doc to match the reality in the code. I don't know where
the recommendation to have min_size = k+2 came from, but for awhile
now we've defaulted to K+1. See PR #8008.

Signed-off-by: Dan van der Ster <dan.vanderster@clyso.com>
(cherry picked from commit 68faf88e6babf7e5c2cce2b4d9b80a159b38c30d)

mgr: fix some flake8 complaints

Signed-off-by: Nizamudeen A <nia@redhat.com>
(cherry picked from commit d199782fb5f3c22f1cd5fda6ddf1d64f40a92726)

Merge pull request #52759 from zdover23/wip-doc-2023-08-02-backport-51960-to-quincy

quincy: doc: improve doc/dev/encoding.rst

doc: improve doc/dev/encoding.rst

Signed-off-by: Radosław Zarzyński <rzarzyns@redhat.com>
(cherry picked from commit 83bef6668d319685923df927f32590b4e1795746)

mgr/dashboard: Add more decimals in latency graph
Fixes: https://tracker.ceph.com/issues/61930
Signed-off-by: Pedro Gonzalez Gomez <pegonzal@redhat.com>
(cherry picked from commit 8c295e38ffd50c58c0d7cda0dbfe51864ab0e906)

Merge pull request #52713 from zdover23/wip-doc-2023-08-01-backport-52131-to-quincy

quincy: doc/radosgw: correct emphasis in rate limit section

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>

doc/radosgw: correct emphasis in rate limit section

Signed-off-by: Piotr Parczewski <piotr@stackhpc.com>
(cherry picked from commit 50bfbab6d9a30b3ba6ec4f4c8df9059298c28a7f)

build: install-deps.sh installs system boost on Jammy

Since on Jammy system boost is new enough for Quincy and we don't have
Jammy packages for older boost (we only have those for Focal), just
install the system packages rather than fetching ceph-libboost.

No analogous commit exists in main as while main's Jammy case installs
ceph-libboost, we just need a system package here.

Fixes: https://tracker.ceph.com/issues/62102
Signed-off-by: Adam Emerson <aemerson@redhat.com>

build: Remove old ceph-libboost* packages in install-deps

Here, we extract `clean_boost_on_ubuntu()` and call it before other
installs on Debian distributions so that if we install a system boost,
a potentially newer `ceph-libboost` won't get in the way.

As the sources.list.d being removed in the original cleanup code isn't
the one we're currently installing in the install code, add a removal
for the currently used source, then do apt-update so packages from the
removed source are no longer included as available.

Two subsidiary dev packages from conflicting boost libraries can be
installed, but it leaves apt in an inconsistent state. To clean this
up, add `--fix-missing` to the removal line and call
`clean_boost_on_ubuntu()` before other uses of apt.

Fixes: https://tracker.ceph.com/issues/62097
Signed-off-by: Adam Emerson <aemerson@redhat.com>
(cherry picked from commit 0c3f511e14af639b6509e69b889258b2f718f8fd)

Conflicts:
install-deps.sh
 - Different boost version for Quincy than Squid.
 - ci_debug does not exist in Quincy
 - whitespace
 - No INSTALL_EXTRA

Fixes: https://tracker.ceph.com/issues/62102
Signed-off-by: Adam Emerson <aemerson@redhat.com>