doc/rados: add link to messenger v2 info in mon-lookup-dns.rst

Add a link to the page about Messenger v2 to the end of
doc/rados/configuration/mon-lookup-dns.rst.

Fixes: https://tracker.ceph.com/issues/58752
Signed-off-by: Zac Dover <zac.dover@proton.me>
(cherry picked from commit 81f9d064f714f42ff27bf620df526f87b393402b)

Merge pull request #59769 from zdover23/wip-doc-2024-09-13-backport-59438-to-quincy

quincy: doc/rgw/d3n: pass cache dir volume to extra_container_args

Merge pull request #59651 from guits/wip-63919-quincy

quincy: ceph-volume: Fix unbound var in disk.get_devices()

Merge pull request #59355 from k0ste/wip-64276-quincy

quincy: rgw: swift: tempurl fixes for ceph

Merge pull request #59338 from k0ste/wip-61633-quincy

quincy: rgw : fix add initialization for RGWGC::process()

doc/rgw/d3n: pass cache dir volume to extra_container_args

Fixes: https://tracker.ceph.com/issues/66492
Signed-off-by: Mark Kogan <mkogan@ibm.com>
(cherry picked from commit 4594e38bf19a8f73191904f5f373a3707cddd2f3)

Merge pull request #59758 from zdover23/wip-doc-2024-09-12-backport-59753-to-quincy

quincy: doc/install: Keep the name field of the created user consistent with …

doc/install: Keep the name field of the created user consistent with the node name in the Start RADOSGW service command

If the user name does not match the name of the node that started the RADOSGW service, this will cause confusion for those who are new to ceph. Because they can't start the radosgw service normally as shown in the tutorial.

Signed-off-by: hejindong <wodeshijie3394@126.com>
(cherry picked from commit 128e6451d8c4764025d4a2565f15ec175c857bb2)

Merge pull request #55336 from guits/wip-64196-quincy

quincy: ceph-volume: use 'no workqueue' options with dmcrypt

Merge pull request #59656 from zdover23/wip-doc-2024-09-08-backport-59628-to-quincy

quincy: doc/dev: add "activate latest release" RTD step

Merge pull request #59735 from zdover23/wip-doc-2024-09-11-backport-59635-to-quincy

quincy: doc/rados: add "pgs not deep scrubbed in time" info

doc/rados: add "pgs not deep scrubbed in time" info

Add a procedure to doc/rados/operations/health-warnings.rst that
explains how to remedy the "X PGs not deep-scrubbed in time" health
warning.

This procedure was developed by Eugen Block, and is at the time of this
commit available on his blog at
https://heiterbiswolkig.blogs.nde.ag/2024/09/06/pgs-not-deep-scrubbed-in-time/

Co-authored-by: Eugen Block <eblock@nde.ag>
Signed-off-by: Zac Dover <zac.dover@proton.me>
(cherry picked from commit d620a51c306d1ebcdfc974dc0b85dc46cfde8d40)

Merge pull request #58221 from guits/wip-66025-quincy

quincy: ceph-volume: create LVs when using partitions

Merge pull request #58999 from guits/wip-64643-quincy

quincy: src/ceph-volume/ceph_volume/devices/lvm/listing.py : lvm list filters with vg name

ceph-volume: fix regex usage in `set_dmcrypt_no_workqueue`

- Updated the regex pattern to `r'(\d+\.?)+'` to more accurately
  capture version numbers.

- Replaced `re.match` with `re.search` to properly match the cryptsetup
  version in the output.

- `re.match` only checks for a match at the beginning of the string,
   while `re.search` looks for a match anywhere in the string.

This fix ensures that the function correctly retrieves the
cryptsetup version from the output.

Fixes: https://tracker.ceph.com/issues/66393
Signed-off-by: Guillaume Abrioux <gabrioux@ibm.com>
(cherry picked from commit 607eb34b2c278566c386efcbf3018629cf08ccfd)

ceph-volume: fix set_dmcrypt_no_workqueue()

`set_dmcrypt_no_workqueue()` from `ceph_volume.util.encryption`

The function `set_dmcrypt_no_workqueue` in `encryption.py` now
dynamically retrieves the installed cryptsetup version using `cryptsetup
--version` command. It then parses the version string using a regular
expression to accommodate varying digit counts. If the retrieved version
is greater than or equal to the specified target version,
`conf.dmcrypt_no_workqueue` is set to True, allowing for flexible version
handling.

Fixes: https://tracker.ceph.com/issues/66393
Signed-off-by: Guillaume Abrioux <gabrioux@ibm.com>
(cherry picked from commit dc28b77a6ea50b3390663ac02eeb80367650b7ed)

ceph-volume: fix partitions support in disk.get_devices()

The following:
```
is_part = get_file_contents(os.path.join(_sys_dev_block_path, item, 'partition')) == "1"
```
assumes any `/sys/dev/block/x:y/partition` contains '1' which is wrong.
This file actually contains the corresponding partition number.

Fixes: https://tracker.ceph.com/issues/64195
Signed-off-by: Guillaume Abrioux <gabrioux@ibm.com>
(cherry picked from commit d5d2d1456542829495d3ebbbad6ca2febd7c344d)

ceph-volume: use 'no workqueue' options with dmcrypt

CloudFlare engineers made some testing and realized that using
workqueues with encryption on flash devices has a bad effect.

See [1] for details.

With this patch it will make ceph-volume call crypsetup with
`--perf-no_read_workqueue` and `--perf-no_write_workqueue` options
when the device is not a rotational.

[1] https://blog.cloudflare.com/speeding-up-linux-disk-encryption/

Fixes: https://tracker.ceph.com/issues/64195
Signed-off-by: Guillaume Abrioux <gabrioux@ibm.com>
Co-Authored-by: Stefan Kooman <stefan@kooman.org>
(cherry picked from commit 0985e201342fa53c014a811156aed661b4b8f994)
(cherry picked from commit 956305eb5caf323cfadb772a9a1f910a90aa7740)

Merge pull request #59702 from zdover23/wip-doc-2024-09-10-backport-59675-to-quincy

quincy: doc/README.md: improve formatting

Merge pull request #59684 from zdover23/wip-doc-2024-09-10-backport-59677-to-quincy

quincy: doc/man: edit ceph-bluestore-tool.rst

doc/README.md: improve formatting

Improve the formatting of the section "Building Ceph" in the file
README.md.

Signed-off-by: Zac Dover <zac.dover@proton.me>
(cherry picked from commit 7171f73c81d603f8ff62198fc1e5f50c20a0aa05)

doc/man: edit ceph-bluestore-tool.rst

Edit the section "bluefs-bdev-migrate" in
doc/man/8/ceph-bluestore-tool.rst to add the information that this
operation expands the target storage by updating its size label, making
"bluefs-bdev-expand" unnecessary.

Improve the subject-verb agreement in this section, and supply some
absent definite articles.

Co-authored-by: Peter Gervai <grin@drop.grin.hu>
Signed-off-by: Zac Dover <zac.dover@proton.me>
(cherry picked from commit 6b34707f827b2b197f53fe2e430d173b30b81401)

Merge pull request #59665 from zdover23/wip-doc-2024-09-09-backport-59642-to-quincy

quincy: doc/cephadm: add malformed-JSON removal instructions

Merge pull request #59639 from zdover23/wip-doc-2024-09-06-backport-59630-to-quincy

quincy: doc/README.md: edit "Build Prerequisites"

doc/cephadm: add malformed-JSON removal instructions

Add instructions for removing malformed JSON that causes the cephadm
module to fail to start, which in turn makes the orchestrator fail to
start.

Signed-off-by: Zac Dover <zac.dover@proton.me>
(cherry picked from commit 93d90ea358cf17b08e502caf8529061a10008fad)

doc/dev: add "activate latest release" RTD step

Add a step to the section called "First Stable Release" that explains
how to activate the latest release in Read the Docs.

Signed-off-by: Zac Dover <zac.dover@proton.me>
(cherry picked from commit e7bd4bcdd074a3defae5866a6c035559bf210271)

ceph-volume: Fix unbound var in disk.get_devices()

00ba00fdfab8 looks to have regressed.

Signed-off-by: Zack Cerza <zack@redhat.com>
(cherry picked from commit 0e95b27402e46c34586f460d2140af48d03fa305)

doc/README.md: edit "Build Prerequisites"

Make the sentences in the "Build Prerequisites" section of README.md
more direct.

Signed-off-by: Zac Dover <zac.dover@proton.me>
(cherry picked from commit 0b277fe3809c2eae7bfced002ac85f7e1629fa30)

Merge pull request #59600 from idryomov/wip-67752-quincy

quincy: rbd: amend "rbd {group,} rename" and "rbd mirror pool" command descriptions

Reviewed-by: Ramana Raja <rraja@redhat.com>

Merge pull request #56616 from prazumovsky/wip-63620

quincy: rgw/swift: preserve dashes/underscores in swift user metadata names

Merge pull request #56145 from yuvalif/wip-64793-quincy

quincy: rgw/notification: Kafka persistent notifications not retried and removed even when the broker is down

Merge pull request #56447 from pr0ton11/wip-65005

quincy: RGW: allow user disabling presigned urls in rgw configuration

Reviewed-by: Casey Bodley <cbodley@redhat.com>

rbd: mention namespace in "rbd mirror pool" command descriptions

Commit 5e64748927d0 ("doc/rbd: add namespace information for mirror
commands") did this for the man page, update the built-in help as well.

The "by default" bit in the description of "rbd mirror pool enable" and
"rbd mirror pool disable" commands is specific to pool mode which is in
turn specific to journal-based mirroring, so it's removed.

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
(cherry picked from commit 7e02946a3f60da0dcc99e0b4a4b6b3d359db1c15)

rbd: clarify "rbd {group,} rename" command description

Make it clearer that, despite a full image or group spec being taken
for source and destination, an image or a group can be renamed only
within its pool or namespace.

Rename across pools or namespaces within the same pool is unsupported.

Fixes: https://tracker.ceph.com/issues/67752
Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
(cherry picked from commit 5f064a52707af58fd6007734a768899aeb5a32fc)

Merge pull request #59584 from zdover23/wip-doc-2024-09-04-backport-59546-to-quincy

quincy: doc/rados/operations: Improve health-checks.rst

Merge pull request #59587 from zdover23/wip-doc-2024-09-04-backport-59577-to-quincy

quincy: doc/mds: improve wording

Merge pull request #59561 from zdover23/wip-doc-2024-09-02-backport-59556-to-quincy

quincy: doc: Correct link to Prometheus docs

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>

doc/mds: improve wording

Signed-off-by: Piotr Parczewski <piotr@stackhpc.com>
(cherry picked from commit 332804bad58c892d01d2d2da557e42104365ef8a)

doc/rados/operations: Improve health-checks.rst

Signed-off-by: Anthony D'Atri <anthonyeleven@users.noreply.github.com>
(cherry picked from commit 2aa82539ede8ade095ad0b22e973eac538354711)

rgw/notification: Fix Kafka persistent notification  breakage that are not retried.

Signed-off-by: kchheda3 <kchheda3@bloomberg.net>
(cherry picked from commit b2fadb65ae6a9eb3eaade56e04b08f2b694b220c)

doc: Correct link to Prometheus docs

The link is to the `#http_sd_config` anchor in the prometheus config docs; that link only works without the trailing `/`.

This correction would ideally get backported to at least reef & squid.

Signed-off-by: Matthew Vernon <mvernon@wikimedia.org>
(cherry picked from commit 84a30ba6b94b34806faac8217ccaa299c9ee68d6)

Merge pull request #59550 from zdover23/wip-doc-2024-09-01-backport-59544-to-quincy

quincy: doc: update tests-integration-testing-teuthology-workflow.rst

doc: update tests-integration-testing-teuthology-workflow.rst

* add "Infrastructure" section.

* move "Naming the ceph-ci branch" section under
   "Getting binaries - Build Ceph". Also mention
   about centos9-only trick.

* in "Teuthology Archives", mention about developer
   playground machines and ceph log files.

Signed-off-by: Vallari Agrawal <val.agl002@gmail.com>
(cherry picked from commit 9bfcb8e17db8c61e523e10856d12b237433d831a)

Merge pull request #59542 from zdover23/wip-doc-2024-08-31-backport-59528-to-quincy

quincy: doc/ceph-volume: add spillover fix procedure

doc/ceph-volume: add spillover fix procedure

Add a procedure that explains how, after an upgrade, to move bytes that
have spilled over to a relatively slow device back to the faster device.

This procedure was developed by Chris Dunlop on the [ceph-users] mailing
list, here: https://lists.ceph.io/hyperkitty/list/ceph-users@ceph.io/message/POPUFSZGXR3P2RPYPJ4WJ4HGHZ3QESF6/

Eugen Block requested the addition of this procedure to the
documentation on 30 Aug 2024.

Co-authored-by: Anthony D'Atri <anthony.datri@gmail.com>
Signed-off-by: Zac Dover <zac.dover@proton.me>
(cherry picked from commit 98618aaa1c8b786c7d240a210b62cc737fdb048d)

Merge pull request #59359 from k0ste/wip-64380-quincy

quincy: rgw/auth/s3: validate x-amz-content-sha256 for empty payloads

Reviewed-by: Daniel Gryniewicz <dang@redhat.com>

Merge pull request #59132 from k0ste/wip-67133-quincy

quincy: osd/ECTransaction: Remove incorrect asserts in generate_transactions

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
Reviewed-by: Laura Flores <lflores@redhat.com>

Merge pull request #59283 from k0ste/wip-65041-quincy

quincy: librados: use CEPH_OSD_FLAG_FULL_FORCE for IoCtxImpl::remove

Reviewed-by: Ilya Dryomov <idryomov@redhat.com>

Merge pull request #59272 from k0ste/wip-63064-quincy

quincy: test/cls_lock: expired lock before unlock and start check

Reviewed-by: Ilya Dryomov <idryomov@redhat.com>
Reviewed-by: Sunil Angadi <Sunil.Angadi@ibm.com>

Merge pull request #59320 from k0ste/wip-58592-quincy

quincy: rgw/rgw-gap-list: refactoring and adding more error checking

Reviewed-by: Daniel Gryniewicz <dang@redhat.com>

Merge pull request #59316 from k0ste/wip-57415-quincy

quincy: rgw: Use STANDARD storage class in objects appending operation when the

Merge pull request #59315 from k0ste/wip-55483-quincy

quincy: librgw: teach librgw about rgw_backend_store

Merge pull request #59314 from k0ste/wip-55289-quincy

quincy: rgw: disable RGWDataChangesLog::add_entry() when log_data is off

Merge pull request #59313 from k0ste/wip-54464-quincy

quincy: rgw: cmake configure error on fedora-37/rawhide

Merge pull request #59337 from k0ste/wip-62759-quincy

quincy: rgw: when there are a large number of multiparts, the unorder list result may miss objects

Merge pull request #59336 from k0ste/wip-59499-quincy

quincy: rgw/lua: fix CopyFrom crash

Merge pull request #59322 from k0ste/wip-58364-quincy

quincy: rgw/admin/notifications: support admin operations on topics with tenants

Merge pull request #59321 from k0ste/wip-58507-quincy

quincy: rgw/rgw-orphan-list: refactor and add more checks to the tool

Merge pull request #59319 from k0ste/wip-57421-quincy

quincy: rgw: avoid use-after-move in RGWDataSyncSingleEntryCR ctor

Merge pull request #59378 from k0ste/wip-64576-quincy

quincy: osd: always send returnvec-on-errors for client's retry

Reviewed-by: Ilya Dryomov <idryomov@redhat.com>

Merge pull request #57614 from sajibreadd/wip-66147-quincy

quincy: osd: fix for segmentation fault on OSD fast shutdown

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
Reviewed-by: Samuel Just <sjust@redhat.com>

Merge pull request #57585 from NitzanMordhai/wip-63979-quincy

quincy: mgr/BaseMgrModule: Optimize CPython Call in Finish Function

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

Merge pull request #57199 from batrick/wip-65736-quincy

quincy: pybind/mgr: disable sqlite3/python autocommit

Reviewed-by: John Mulligan <jmulligan@redhat.com>

Merge pull request #55607 from jzhu116-bloomberg/wip-64447-quincy

quincy: rgw: do not copy olh attributes in versioning suspended bucket

Merge pull request #59325 from cbodley/wip-67618-quincy

quincy: qa: barbican: restrict python packages with upper-constraints

Merge pull request #54874 from trociny/wip-63776-quincy

quincy: [rgw][lc][rgw_lifecycle_work_time] adjust timing if the configured end time is less than the start time

Merge pull request #54873 from trociny/wip-56628-quincy

quincy: rgwlc: permit lifecycle to reduce data conditionally in archive zone

Merge pull request #56163 from yuvalif/wip-64886-quincy

quincy: rgw/kafka: set message timeout to 5 seconds

Merge pull request #55882 from kamoltat/wip-64673-quincy

quincy: qa/tasks/ceph_manager.py: Rewrite test_pool_min_size
Laura Flores <lflores@redhat.com>

Merge pull request #59432 from zdover23/wip-doc-2024-08-26-backport-59428-to-quincy

quincy: doc/cephadm: how to get exact size_spec from device

Merge pull request #59412 from adk3798/wip-67683-quincy

quincy: mgr/cephadm: add "original_weight" parameter to OSD class

Reviewed-by: John Mulligan <jmulligan@redhat.com>

doc/cephadm: how to get exact size_spec from device

Add instructions for retrieving the exact size of block devices.

Fixes: https://tracker.ceph.com/issues/66754
Signed-off-by: Zac Dover <zac.dover@proton.me>
(cherry picked from commit d00d1b52d50b5575d918c3be7b7a8249ef31f0a8)

Merge pull request #59426 from zdover23/wip-doc-2024-08-25-backport-59418-to-quincy

quincy: doc/glossary: add "object storage"

doc/glossary: add "object storage"

Add a (very basic) definition of object storage.

Signed-off-by: Zac Dover <zac.dover@proton.me>
(cherry picked from commit 43057b88538e868b817acb04d5e6c4e95b4c716e)

Merge pull request #59360 from k0ste/wip-67483-quincy

quincy: rgw: catches nobjects_begin() exceptions

Reviewed-by: Daniel Gryniewicz <dang@redhat.com>

Merge pull request #59357 from k0ste/wip-64304-quincy

quincy: rgw: fix issue with concurrent versioned deletes leaving behind olh entries

Reviewed-by: Daniel Gryniewicz <dang@redhat.com>

Merge pull request #59353 from k0ste/wip-63584-quincy

quincy: rgw/notifications: cleanup all coroutines after sending the notification

Reviewed-by: Yuval Lifshitz <ylifshit@redhat.com>

Merge pull request #59350 from k0ste/wip-63424-quincy

quincy: rgwfile: fix lock_guard decl

Reviewed-by: Daniel Gryniewicz <dang@redhat.com>

Merge pull request #59347 from k0ste/wip-63404-quincy

quincy: rgw/multisite: fix sync_error_trim command

Reviewed-by: Daniel Gryniewicz <dang@redhat.com>

Merge pull request #59345 from k0ste/wip-59605-quincy

quincy: rgw/sts: changing identity to boost::none, when role policy

Reviewed-by: Daniel Gryniewicz <dang@redhat.com>

Merge pull request #59344 from k0ste/wip-59608-quincy

quincy: rgw: add crypt attrs for iam policy to PostObj and Init/CompleteMultipart

Reviewed-by: Daniel Gryniewicz <dang@redhat.com>

Merge pull request #59342 from k0ste/wip-61190-quincy

quincy: rgw/lua: add lib64 to the package search path

Reviewed-by: Yuval Lifshitz <ylifshit@redhat.com>

Merge pull request #59340 from k0sReviewed-by: Yuval Lifshitz <ylifshit@redhat.com>te/wip-61521-quincy

quincy: test/rgw/notifications: fix kafka consumer shutdown issue

Reviewed-by: Yuval Lifshitz <ylifshit@redhat.com>

Merge pull request #59382 from zdover23/wip-doc-2024-08-21-backport-59348-to-quincy

quincy: doc/rados: document unfound object cache-tiering scenario

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>

mgr/cephadm: add "original_weight" parameter to OSD class

Fixes: https://tracker.ceph.com/issues/67329
Signed-off-by: Adam King <adking@redhat.com>
(cherry picked from commit 04330f5df92994882efcd4879d5c37279138e97b)

Merge pull request #54795 from k0ste/wip-62414-quincy

quincy: rgw/auth: Fix the return code returned by AuthStrategy,

Reviewed-by: Adam Emerson <aemerson@redhat.com>

Merge pull request #54792 from k0ste/wip-63625-quincy

quincy: rgw: SignatureDoesNotMatch for certain RGW Admin Ops endpoints w/v4 auth

Reviewed-by: Adam Emerson <aemerson@redhat.com>

Merge pull request #54170 from yuvalif/wip-57888-quincy

quincy: rgw/amqp: store CA location string in connection object

Reviewed-by: Adam Emerson <aemerson@redhat.com>

Merge pull request #54128 from BBoozmen/wip-63256-quincy

quincy: RGW/STS: when generating keys, take the trailing null character into account

Reviewed-by: Adam Emerson <aemerson@redhat.com>

Merge pull request #54017 from k0ste/wip-57198-quincy

quincy: rgw: 'bucket check' deletes index of multipart meta when its pending_map is nonempty

Reviewed-by: Adam Emerson <aemerson@redhat.com>

Merge pull request #59330 from ljflores/wip-tracker-67562

quincy: qa/workunits/rados: enable crb and install generic package for c9

Merge pull request #57407 from k0ste/wip-62926-quincy

quincy: mon: stuck peering since warning is misleading

Merge pull request #55359 from ljflores/wip-revert-pr-55264

quincy: doc/rados: revert "doc/rados/operations: document `ceph balancer status detail`"

Merge pull request #59076 from tobias-urdin/quincy-keystone-admin-token

quincy: rgw: invalidate and retry keystone admin token

Reviewed-by: Adam Emerson <aemerson@redhat.com>
Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #57899 from cbodley/wip-59357-quincy

quincy: rgw/keystone: use secret key from EC2 for sigv4 streaming mode

Reviewed-by: Adam Emerson <aemerson@redhat.com>

Merge pull request #57149 from k0ste/wip-57658-quincy

quincy: rgw: set requestPayment in slave zone

Reviewed-by: Adam Emerson <aemerson@redhat.com>

Merge pull request #57133 from k0ste/wip-59376-quincy

quincy: rgw: handle old clients with transfer-encoding: chunked.

Reviewed-by: Adam Emerson <aemerson@redhat.com>

Merge pull request #57132 from k0ste/wip-59144-quincy

quincy: Do not duplicate query-string in ops-log

Reviewed-by: Adam Emerson <aemerson@redhat.com>

Merge pull request #57131 from k0ste/wip-58327-quincy

quincy: rgw: fix ListOpenIDConnectProviders XML format

Reviewed-by: Adam Emerson <aemerson@redhat.com>

doc/rados: Revert "doc/rados/operations: document `ceph balancer status detail`"

This reverts commit 7fff8e9a981c366390f61cf1eaeba6524fd6d365.

Signed-off-by: Laura Flores <lflores@ibm.com>