]> git.apps.os.sepia.ceph.com Git - ceph.git/log
ceph.git
10 months agorgw/notification: Fix Kafka persistent notification breakage that are not retried. 56145/head
kchheda3 [Thu, 26 Oct 2023 19:57:54 +0000 (15:57 -0400)]
rgw/notification: Fix Kafka persistent notification  breakage that are not retried.

Signed-off-by: kchheda3 <kchheda3@bloomberg.net>
(cherry picked from commit b2fadb65ae6a9eb3eaade56e04b08f2b694b220c)

10 months agoMerge pull request #59550 from zdover23/wip-doc-2024-09-01-backport-59544-to-quincy
Anthony D'Atri [Sun, 1 Sep 2024 15:15:58 +0000 (11:15 -0400)]
Merge pull request #59550 from zdover23/wip-doc-2024-09-01-backport-59544-to-quincy

quincy: doc: update tests-integration-testing-teuthology-workflow.rst

10 months agodoc: update tests-integration-testing-teuthology-workflow.rst 59550/head
Vallari Agrawal [Sat, 31 Aug 2024 14:27:25 +0000 (19:57 +0530)]
doc: update tests-integration-testing-teuthology-workflow.rst

* add "Infrastructure" section.

* move "Naming the ceph-ci branch" section under
   "Getting binaries - Build Ceph". Also mention
   about centos9-only trick.

* in "Teuthology Archives", mention about developer
   playground machines and ceph log files.

Signed-off-by: Vallari Agrawal <val.agl002@gmail.com>
(cherry picked from commit 9bfcb8e17db8c61e523e10856d12b237433d831a)

10 months agoMerge pull request #59542 from zdover23/wip-doc-2024-08-31-backport-59528-to-quincy
Anthony D'Atri [Sat, 31 Aug 2024 14:29:34 +0000 (10:29 -0400)]
Merge pull request #59542 from zdover23/wip-doc-2024-08-31-backport-59528-to-quincy

quincy: doc/ceph-volume: add spillover fix procedure

10 months agodoc/ceph-volume: add spillover fix procedure 59542/head
Zac Dover [Fri, 30 Aug 2024 11:16:57 +0000 (21:16 +1000)]
doc/ceph-volume: add spillover fix procedure

Add a procedure that explains how, after an upgrade, to move bytes that
have spilled over to a relatively slow device back to the faster device.

This procedure was developed by Chris Dunlop on the [ceph-users] mailing
list, here: https://lists.ceph.io/hyperkitty/list/ceph-users@ceph.io/message/POPUFSZGXR3P2RPYPJ4WJ4HGHZ3QESF6/

Eugen Block requested the addition of this procedure to the
documentation on 30 Aug 2024.

Co-authored-by: Anthony D'Atri <anthony.datri@gmail.com>
Signed-off-by: Zac Dover <zac.dover@proton.me>
(cherry picked from commit 98618aaa1c8b786c7d240a210b62cc737fdb048d)

10 months agoMerge pull request #59359 from k0ste/wip-64380-quincy
Yuri Weinstein [Fri, 30 Aug 2024 19:04:40 +0000 (12:04 -0700)]
Merge pull request #59359 from k0ste/wip-64380-quincy

quincy: rgw/auth/s3: validate x-amz-content-sha256 for empty payloads

Reviewed-by: Daniel Gryniewicz <dang@redhat.com>
10 months agoMerge pull request #59132 from k0ste/wip-67133-quincy
Yuri Weinstein [Fri, 30 Aug 2024 19:03:24 +0000 (12:03 -0700)]
Merge pull request #59132 from k0ste/wip-67133-quincy

quincy: osd/ECTransaction: Remove incorrect asserts in generate_transactions

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
Reviewed-by: Laura Flores <lflores@redhat.com>
10 months agoMerge pull request #59283 from k0ste/wip-65041-quincy
Yuri Weinstein [Fri, 30 Aug 2024 17:53:12 +0000 (10:53 -0700)]
Merge pull request #59283 from k0ste/wip-65041-quincy

quincy: librados: use CEPH_OSD_FLAG_FULL_FORCE for IoCtxImpl::remove

Reviewed-by: Ilya Dryomov <idryomov@redhat.com>
10 months agoMerge pull request #59272 from k0ste/wip-63064-quincy
Yuri Weinstein [Fri, 30 Aug 2024 17:52:40 +0000 (10:52 -0700)]
Merge pull request #59272 from k0ste/wip-63064-quincy

quincy: test/cls_lock: expired lock before unlock and start check

Reviewed-by: Ilya Dryomov <idryomov@redhat.com>
Reviewed-by: Sunil Angadi <Sunil.Angadi@ibm.com>
10 months agoMerge pull request #59320 from k0ste/wip-58592-quincy
Yuri Weinstein [Fri, 30 Aug 2024 15:06:14 +0000 (08:06 -0700)]
Merge pull request #59320 from k0ste/wip-58592-quincy

quincy: rgw/rgw-gap-list: refactoring and adding more error checking

Reviewed-by: Daniel Gryniewicz <dang@redhat.com>
10 months agoMerge pull request #59316 from k0ste/wip-57415-quincy
Shilpa Jagannath [Thu, 29 Aug 2024 22:58:07 +0000 (15:58 -0700)]
Merge pull request #59316 from k0ste/wip-57415-quincy

quincy: rgw: Use STANDARD storage class in objects appending operation when the

10 months agoMerge pull request #59315 from k0ste/wip-55483-quincy
Shilpa Jagannath [Thu, 29 Aug 2024 22:57:39 +0000 (15:57 -0700)]
Merge pull request #59315 from k0ste/wip-55483-quincy

quincy: librgw: teach librgw about rgw_backend_store

10 months agoMerge pull request #59314 from k0ste/wip-55289-quincy
Shilpa Jagannath [Thu, 29 Aug 2024 22:57:20 +0000 (15:57 -0700)]
Merge pull request #59314 from k0ste/wip-55289-quincy

quincy: rgw: disable RGWDataChangesLog::add_entry() when log_data is off

10 months agoMerge pull request #59313 from k0ste/wip-54464-quincy
Shilpa Jagannath [Thu, 29 Aug 2024 22:56:34 +0000 (15:56 -0700)]
Merge pull request #59313 from k0ste/wip-54464-quincy

quincy: rgw: cmake configure error on fedora-37/rawhide

10 months agoMerge pull request #59337 from k0ste/wip-62759-quincy
Shilpa Jagannath [Thu, 29 Aug 2024 22:55:39 +0000 (15:55 -0700)]
Merge pull request #59337 from k0ste/wip-62759-quincy

quincy: rgw: when there are a large number of multiparts, the unorder list result may miss objects

10 months agoMerge pull request #59336 from k0ste/wip-59499-quincy
Shilpa Jagannath [Thu, 29 Aug 2024 22:54:30 +0000 (15:54 -0700)]
Merge pull request #59336 from k0ste/wip-59499-quincy

quincy: rgw/lua: fix CopyFrom crash

10 months agoMerge pull request #59322 from k0ste/wip-58364-quincy
Shilpa Jagannath [Thu, 29 Aug 2024 22:53:47 +0000 (15:53 -0700)]
Merge pull request #59322 from k0ste/wip-58364-quincy

quincy: rgw/admin/notifications: support admin operations on topics with tenants

10 months agoMerge pull request #59321 from k0ste/wip-58507-quincy
Shilpa Jagannath [Thu, 29 Aug 2024 22:52:17 +0000 (15:52 -0700)]
Merge pull request #59321 from k0ste/wip-58507-quincy

quincy: rgw/rgw-orphan-list: refactor and add more checks to the tool

10 months agoMerge pull request #59319 from k0ste/wip-57421-quincy
Shilpa Jagannath [Thu, 29 Aug 2024 22:51:30 +0000 (15:51 -0700)]
Merge pull request #59319 from k0ste/wip-57421-quincy

quincy: rgw: avoid use-after-move in RGWDataSyncSingleEntryCR ctor

10 months agoMerge pull request #59378 from k0ste/wip-64576-quincy
Yuri Weinstein [Thu, 29 Aug 2024 20:12:42 +0000 (13:12 -0700)]
Merge pull request #59378 from k0ste/wip-64576-quincy

quincy: osd: always send returnvec-on-errors for client's retry

Reviewed-by: Ilya Dryomov <idryomov@redhat.com>
10 months agoMerge pull request #57614 from sajibreadd/wip-66147-quincy
Yuri Weinstein [Thu, 29 Aug 2024 20:10:43 +0000 (13:10 -0700)]
Merge pull request #57614 from sajibreadd/wip-66147-quincy

quincy: osd: fix for segmentation fault on OSD fast shutdown

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
Reviewed-by: Samuel Just <sjust@redhat.com>
10 months agoMerge pull request #57585 from NitzanMordhai/wip-63979-quincy
Yuri Weinstein [Thu, 29 Aug 2024 20:10:07 +0000 (13:10 -0700)]
Merge pull request #57585 from NitzanMordhai/wip-63979-quincy

quincy: mgr/BaseMgrModule: Optimize CPython Call in Finish Function

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
10 months agoMerge pull request #57199 from batrick/wip-65736-quincy
Yuri Weinstein [Thu, 29 Aug 2024 20:09:29 +0000 (13:09 -0700)]
Merge pull request #57199 from batrick/wip-65736-quincy

quincy: pybind/mgr: disable sqlite3/python autocommit

Reviewed-by: John Mulligan <jmulligan@redhat.com>
10 months agoMerge pull request #55607 from jzhu116-bloomberg/wip-64447-quincy
Shilpa Jagannath [Thu, 29 Aug 2024 18:22:23 +0000 (11:22 -0700)]
Merge pull request #55607 from jzhu116-bloomberg/wip-64447-quincy

quincy: rgw: do not copy olh attributes in versioning suspended bucket

10 months agoMerge pull request #59325 from cbodley/wip-67618-quincy
Shilpa Jagannath [Thu, 29 Aug 2024 18:20:24 +0000 (11:20 -0700)]
Merge pull request #59325 from cbodley/wip-67618-quincy

quincy: qa: barbican: restrict python packages with upper-constraints

10 months agoMerge pull request #54874 from trociny/wip-63776-quincy
Shilpa Jagannath [Thu, 29 Aug 2024 18:19:41 +0000 (11:19 -0700)]
Merge pull request #54874 from trociny/wip-63776-quincy

quincy: [rgw][lc][rgw_lifecycle_work_time] adjust timing if the configured end time is less than the start time

10 months agoMerge pull request #54873 from trociny/wip-56628-quincy
Shilpa Jagannath [Thu, 29 Aug 2024 18:18:45 +0000 (11:18 -0700)]
Merge pull request #54873 from trociny/wip-56628-quincy

quincy: rgwlc: permit lifecycle to reduce data conditionally in archive zone

10 months agoMerge pull request #56163 from yuvalif/wip-64886-quincy
Shilpa Jagannath [Thu, 29 Aug 2024 15:34:13 +0000 (08:34 -0700)]
Merge pull request #56163 from yuvalif/wip-64886-quincy

quincy: rgw/kafka: set message timeout to 5 seconds

10 months agoMerge pull request #55882 from kamoltat/wip-64673-quincy
Kamoltat (Junior) Sirivadhna [Tue, 27 Aug 2024 01:12:46 +0000 (21:12 -0400)]
Merge pull request #55882 from kamoltat/wip-64673-quincy

quincy: qa/tasks/ceph_manager.py: Rewrite test_pool_min_size
Laura Flores <lflores@redhat.com>

10 months agoMerge pull request #59432 from zdover23/wip-doc-2024-08-26-backport-59428-to-quincy
Anthony D'Atri [Mon, 26 Aug 2024 13:23:42 +0000 (09:23 -0400)]
Merge pull request #59432 from zdover23/wip-doc-2024-08-26-backport-59428-to-quincy

quincy: doc/cephadm: how to get exact size_spec from device

10 months agoMerge pull request #59412 from adk3798/wip-67683-quincy
Adam King [Mon, 26 Aug 2024 11:55:47 +0000 (07:55 -0400)]
Merge pull request #59412 from adk3798/wip-67683-quincy

quincy: mgr/cephadm: add "original_weight" parameter to OSD class

Reviewed-by: John Mulligan <jmulligan@redhat.com>
10 months agodoc/cephadm: how to get exact size_spec from device 59432/head
Zac Dover [Sun, 25 Aug 2024 20:03:34 +0000 (06:03 +1000)]
doc/cephadm: how to get exact size_spec from device

Add instructions for retrieving the exact size of block devices.

Fixes: https://tracker.ceph.com/issues/66754
Signed-off-by: Zac Dover <zac.dover@proton.me>
(cherry picked from commit d00d1b52d50b5575d918c3be7b7a8249ef31f0a8)

10 months agoMerge pull request #59426 from zdover23/wip-doc-2024-08-25-backport-59418-to-quincy
Anthony D'Atri [Sat, 24 Aug 2024 22:31:31 +0000 (18:31 -0400)]
Merge pull request #59426 from zdover23/wip-doc-2024-08-25-backport-59418-to-quincy

quincy: doc/glossary: add "object storage"

10 months agodoc/glossary: add "object storage" 59426/head
Zac Dover [Fri, 23 Aug 2024 12:36:16 +0000 (22:36 +1000)]
doc/glossary: add "object storage"

Add a (very basic) definition of object storage.

Signed-off-by: Zac Dover <zac.dover@proton.me>
(cherry picked from commit 43057b88538e868b817acb04d5e6c4e95b4c716e)

10 months agoMerge pull request #59360 from k0ste/wip-67483-quincy
Yuri Weinstein [Fri, 23 Aug 2024 20:14:19 +0000 (13:14 -0700)]
Merge pull request #59360 from k0ste/wip-67483-quincy

quincy: rgw: catches nobjects_begin() exceptions

Reviewed-by: Daniel Gryniewicz <dang@redhat.com>
10 months agoMerge pull request #59357 from k0ste/wip-64304-quincy
Yuri Weinstein [Fri, 23 Aug 2024 20:12:26 +0000 (13:12 -0700)]
Merge pull request #59357 from k0ste/wip-64304-quincy

quincy: rgw: fix issue with concurrent versioned deletes leaving behind olh entries

Reviewed-by: Daniel Gryniewicz <dang@redhat.com>
10 months agoMerge pull request #59353 from k0ste/wip-63584-quincy
Yuri Weinstein [Fri, 23 Aug 2024 20:10:39 +0000 (13:10 -0700)]
Merge pull request #59353 from k0ste/wip-63584-quincy

quincy: rgw/notifications: cleanup all coroutines after sending the notification

Reviewed-by: Yuval Lifshitz <ylifshit@redhat.com>
10 months agoMerge pull request #59350 from k0ste/wip-63424-quincy
Yuri Weinstein [Fri, 23 Aug 2024 20:10:04 +0000 (13:10 -0700)]
Merge pull request #59350 from k0ste/wip-63424-quincy

quincy: rgwfile: fix lock_guard decl

Reviewed-by: Daniel Gryniewicz <dang@redhat.com>
10 months agoMerge pull request #59347 from k0ste/wip-63404-quincy
Yuri Weinstein [Fri, 23 Aug 2024 20:09:34 +0000 (13:09 -0700)]
Merge pull request #59347 from k0ste/wip-63404-quincy

quincy: rgw/multisite: fix sync_error_trim command

Reviewed-by: Daniel Gryniewicz <dang@redhat.com>
10 months agoMerge pull request #59345 from k0ste/wip-59605-quincy
Yuri Weinstein [Fri, 23 Aug 2024 20:08:58 +0000 (13:08 -0700)]
Merge pull request #59345 from k0ste/wip-59605-quincy

quincy: rgw/sts: changing identity to boost::none, when role policy

Reviewed-by: Daniel Gryniewicz <dang@redhat.com>
10 months agoMerge pull request #59344 from k0ste/wip-59608-quincy
Yuri Weinstein [Fri, 23 Aug 2024 20:08:11 +0000 (13:08 -0700)]
Merge pull request #59344 from k0ste/wip-59608-quincy

quincy: rgw: add crypt attrs for iam policy to PostObj and Init/CompleteMultipart

Reviewed-by: Daniel Gryniewicz <dang@redhat.com>
10 months agoMerge pull request #59342 from k0ste/wip-61190-quincy
Yuri Weinstein [Fri, 23 Aug 2024 20:06:59 +0000 (13:06 -0700)]
Merge pull request #59342 from k0ste/wip-61190-quincy

quincy: rgw/lua: add lib64 to the package search path

Reviewed-by: Yuval Lifshitz <ylifshit@redhat.com>
10 months agoMerge pull request #59340 from k0sReviewed-by: Yuval Lifshitz <ylifshit@redhat.com...
Yuri Weinstein [Fri, 23 Aug 2024 20:06:12 +0000 (13:06 -0700)]
Merge pull request #59340 from k0sReviewed-by: Yuval Lifshitz <ylifshit@redhat.com>te/wip-61521-quincy

quincy: test/rgw/notifications: fix kafka consumer shutdown issue

Reviewed-by: Yuval Lifshitz <ylifshit@redhat.com>
10 months agoMerge pull request #59382 from zdover23/wip-doc-2024-08-21-backport-59348-to-quincy
Zac Dover [Fri, 23 Aug 2024 11:36:28 +0000 (21:36 +1000)]
Merge pull request #59382 from zdover23/wip-doc-2024-08-21-backport-59348-to-quincy

quincy: doc/rados: document unfound object cache-tiering scenario

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>
10 months agomgr/cephadm: add "original_weight" parameter to OSD class 59412/head
Adam King [Mon, 19 Aug 2024 16:30:24 +0000 (12:30 -0400)]
mgr/cephadm: add "original_weight" parameter to OSD class

Fixes: https://tracker.ceph.com/issues/67329
Signed-off-by: Adam King <adking@redhat.com>
(cherry picked from commit 04330f5df92994882efcd4879d5c37279138e97b)

10 months agoMerge pull request #54795 from k0ste/wip-62414-quincy
Yuri Weinstein [Thu, 22 Aug 2024 15:39:56 +0000 (08:39 -0700)]
Merge pull request #54795 from k0ste/wip-62414-quincy

quincy: rgw/auth: Fix the return code returned by AuthStrategy,

Reviewed-by: Adam Emerson <aemerson@redhat.com>
10 months agoMerge pull request #54792 from k0ste/wip-63625-quincy
Yuri Weinstein [Thu, 22 Aug 2024 15:39:13 +0000 (08:39 -0700)]
Merge pull request #54792 from k0ste/wip-63625-quincy

quincy: rgw: SignatureDoesNotMatch for certain RGW Admin Ops endpoints w/v4 auth

Reviewed-by: Adam Emerson <aemerson@redhat.com>
10 months agoMerge pull request #54170 from yuvalif/wip-57888-quincy
Yuri Weinstein [Thu, 22 Aug 2024 15:38:45 +0000 (08:38 -0700)]
Merge pull request #54170 from yuvalif/wip-57888-quincy

quincy: rgw/amqp: store CA location string in connection object

Reviewed-by: Adam Emerson <aemerson@redhat.com>
10 months agoMerge pull request #54128 from BBoozmen/wip-63256-quincy
Yuri Weinstein [Thu, 22 Aug 2024 15:38:01 +0000 (08:38 -0700)]
Merge pull request #54128 from BBoozmen/wip-63256-quincy

quincy: RGW/STS: when generating keys, take the trailing null character into account

Reviewed-by: Adam Emerson <aemerson@redhat.com>
10 months agoMerge pull request #54017 from k0ste/wip-57198-quincy
Yuri Weinstein [Thu, 22 Aug 2024 15:37:20 +0000 (08:37 -0700)]
Merge pull request #54017 from k0ste/wip-57198-quincy

quincy: rgw: 'bucket check' deletes index of multipart meta when its pending_map is nonempty

Reviewed-by: Adam Emerson <aemerson@redhat.com>
10 months agoMerge pull request #59330 from ljflores/wip-tracker-67562
Laura Flores [Thu, 22 Aug 2024 15:01:46 +0000 (10:01 -0500)]
Merge pull request #59330 from ljflores/wip-tracker-67562

quincy: qa/workunits/rados: enable crb and install generic package for c9

10 months agoMerge pull request #57407 from k0ste/wip-62926-quincy
Laura Flores [Wed, 21 Aug 2024 19:27:46 +0000 (14:27 -0500)]
Merge pull request #57407 from k0ste/wip-62926-quincy

quincy: mon: stuck peering since warning is misleading

10 months agoMerge pull request #55359 from ljflores/wip-revert-pr-55264
Laura Flores [Wed, 21 Aug 2024 19:04:43 +0000 (14:04 -0500)]
Merge pull request #55359 from ljflores/wip-revert-pr-55264

quincy: doc/rados: revert "doc/rados/operations: document `ceph balancer status detail`"

10 months agoMerge pull request #59076 from tobias-urdin/quincy-keystone-admin-token
Yuri Weinstein [Wed, 21 Aug 2024 18:14:33 +0000 (11:14 -0700)]
Merge pull request #59076 from tobias-urdin/quincy-keystone-admin-token

quincy: rgw: invalidate and retry keystone admin token

Reviewed-by: Adam Emerson <aemerson@redhat.com>
Reviewed-by: Casey Bodley <cbodley@redhat.com>
10 months agoMerge pull request #57899 from cbodley/wip-59357-quincy
Yuri Weinstein [Wed, 21 Aug 2024 18:14:01 +0000 (11:14 -0700)]
Merge pull request #57899 from cbodley/wip-59357-quincy

quincy: rgw/keystone: use secret key from EC2 for sigv4 streaming mode

Reviewed-by: Adam Emerson <aemerson@redhat.com>
10 months agoMerge pull request #57149 from k0ste/wip-57658-quincy
Yuri Weinstein [Wed, 21 Aug 2024 18:13:24 +0000 (11:13 -0700)]
Merge pull request #57149 from k0ste/wip-57658-quincy

quincy: rgw: set requestPayment in slave zone

Reviewed-by: Adam Emerson <aemerson@redhat.com>
10 months agoMerge pull request #57133 from k0ste/wip-59376-quincy
Yuri Weinstein [Wed, 21 Aug 2024 18:12:55 +0000 (11:12 -0700)]
Merge pull request #57133 from k0ste/wip-59376-quincy

quincy: rgw: handle old clients with transfer-encoding: chunked.

Reviewed-by: Adam Emerson <aemerson@redhat.com>
10 months agoMerge pull request #57132 from k0ste/wip-59144-quincy
Yuri Weinstein [Wed, 21 Aug 2024 18:12:23 +0000 (11:12 -0700)]
Merge pull request #57132 from k0ste/wip-59144-quincy

quincy: Do not duplicate query-string in ops-log

Reviewed-by: Adam Emerson <aemerson@redhat.com>
10 months agoMerge pull request #57131 from k0ste/wip-58327-quincy
Yuri Weinstein [Wed, 21 Aug 2024 18:11:35 +0000 (11:11 -0700)]
Merge pull request #57131 from k0ste/wip-58327-quincy

quincy: rgw: fix ListOpenIDConnectProviders XML format

Reviewed-by: Adam Emerson <aemerson@redhat.com>
10 months agodoc/rados: Revert "doc/rados/operations: document `ceph balancer status detail`" 55359/head
Laura Flores [Mon, 29 Jan 2024 17:22:08 +0000 (11:22 -0600)]
doc/rados: Revert "doc/rados/operations: document `ceph balancer status detail`"

This reverts commit 7fff8e9a981c366390f61cf1eaeba6524fd6d365.

Signed-off-by: Laura Flores <lflores@ibm.com>
10 months agodoc/rados: document unfound object cache-tiering scenario 59382/head
Zac Dover [Tue, 20 Aug 2024 12:45:29 +0000 (22:45 +1000)]
doc/rados: document unfound object cache-tiering scenario

Explain how to deal with "unfound objects" when restarting OSDs in a
cache-tiered environment.

Fixes: https://tracker.ceph.com/issues/44286
Signed-off-by: Zac Dover <zac.dover@proton.me>
(cherry picked from commit f01d7a8d5b85170c034acb962b9833913853a1c5)

10 months agoosd: always send returnvec-on-errors for client's retry 59378/head
Radoslaw Zarzynski [Wed, 14 Feb 2024 00:38:46 +0000 (00:38 +0000)]
osd: always send returnvec-on-errors for client's retry

Currently there is a discrepancy in terms of the returnvec's
presence between MOSDOpReplys sent for original requests and
those on dups. The former always contain the returnvec if
an error happened, even if `allows_returnvec()` is `false`.

This commit extends the behavior on dups.

For RCA please see: https://tracker.ceph.com/issues/64192#note-9

Fixes: https://tracker.ceph.com/issues/64192
Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
(cherry picked from commit 613575719191ce55bc8997577965f3a03e22453c)

10 months agorgw: RGWSI_RADOS::Pool::List::init() catches IoCtx::nobjects_begin() exceptions 59360/head
licc [Tue, 7 Feb 2023 08:02:19 +0000 (09:02 +0100)]
rgw: RGWSI_RADOS::Pool::List::init() catches IoCtx::nobjects_begin() exceptions

Signed-off-by: lichaochao <lichaochao2_yewu@cmss.chinamobile.com>
(cherry picked from commit 63c7d2a44b421cb95bb62ec3db9b2ff84e72d059)

10 months agorgw:log_list_init() catches IoCtx::nobjects_begin() exceptions
licc [Tue, 7 Feb 2023 07:31:44 +0000 (08:31 +0100)]
rgw:log_list_init() catches IoCtx::nobjects_begin() exceptions

Signed-off-by: lichaochao <lichaochao2_yewu@cmss.chinamobile.com>
(cherry picked from commit 14176f00f44d51b7ecbf9952a1fbbe056f7988eb)

10 months agorgw/auth/s3: validate x-amz-content-sha256 for empty payloads 59359/head
Casey Bodley [Fri, 19 Jan 2024 18:56:21 +0000 (13:56 -0500)]
rgw/auth/s3: validate x-amz-content-sha256 for empty payloads

when is_v4_payload_empty(), we return a null completer so never try to
validate the x-amz-content-sha256 for signed payloads. add this
checksum comparison to get_auth_data_v4() before we create the completer

Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit 4bb49478fae09ead4646c1baada3bbc9a2555130)

10 months agorgw: fix issue with concurrent versioned deletes leaving behind olh entries 59357/head
Cory Snyder [Fri, 12 Jan 2024 14:51:14 +0000 (14:51 +0000)]
rgw: fix issue with concurrent versioned deletes leaving behind olh entries

Fixes a scenario where BI OLH entries were left behind after racing
delete requests for the same versioned object instance.

Fixes: https://tracker.ceph.com/issues/64014
Signed-off-by: Cory Snyder <csnyder@1111systems.com>
(cherry picked from commit 66ac828f97c1b368e41c51be4e5699f3e92d47f9)

10 months agorgw: add reproducer for bug with concurrent versioned object deletes
Cory Snyder [Fri, 12 Jan 2024 14:41:31 +0000 (14:41 +0000)]
rgw: add reproducer for bug with concurrent versioned object deletes

Adds a test case to reproduce a scenario where concurrent versioned
object deletes can cause leftover OLH entries to be left behind.

Signed-off-by: Cory Snyder <csnyder@1111systems.com>
(cherry picked from commit b65046e37f62f2c65b17ba6f3434a19d3d68c983)

10 months agorgw/notifications: cleanup all coroutines after sending the notification 59353/head
Yuval Lifshitz [Thu, 16 Nov 2023 15:47:06 +0000 (15:47 +0000)]
rgw/notifications: cleanup all coroutines after sending the notification

this is fixing a regression from: 6b6592f50b6b81fa13a330bcb91273ba7f25c0c9

Fixes: https://tracker.ceph.com/issues/63580
Signed-off-by: Yuval Lifshitz <ylifshit@redhat.com>
(cherry picked from commit 63e14893cc7dc6a3aec92c0867ce5e3e67f44b29)

10 months agorgwfile: fix lock_guard decl 59350/head
Matt Benjamin [Thu, 2 Nov 2023 17:45:12 +0000 (13:45 -0400)]
rgwfile: fix lock_guard decl

Reported by Leonid Usov <leonid.usov@ibm.com>.

Fixes: https://tracker.ceph.com/issues/63409
Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
(cherry picked from commit f637e53eb62f4f940918a793865db6629fc21275)

10 months agorgw/multisite: fix sync_error_trim command. 59347/head
Shilpa Jagannath [Wed, 4 Oct 2023 19:22:41 +0000 (15:22 -0400)]
rgw/multisite: fix sync_error_trim command.
without --marker option, we trim everthing.
if --marker option is passed in, we trim upto that marker.

Signed-off-by: Shilpa Jagannath <smanjara@redhat.com>
(cherry picked from commit a0491a4f63885be9ea14a15c67cabd8903f19641)

10 months agorgw/sts: changing identity to boost::none, when role policy 59345/head
Pritha Srivastava [Thu, 2 Feb 2023 05:35:41 +0000 (11:05 +0530)]
rgw/sts: changing identity to boost::none, when role policy
is verified for putobj permissions, in case of renaming a
large file.

While renaming a large file, putobj is invoked as an intermediate
step, and role policy is verified for the source object if temp creds
are used. Since the role policy is attached to the identity (role)
itself and the role policy does not contain a Principal, there is no
need to verify the identity and hence boost::none is passed in place
of the identity.

fixes: https://tracker.ceph.com/issues/58628

Signed-off-by: Pritha Srivastava <prsrivas@redhat.com>
(cherry picked from commit c2f5716e5196073abfc50917e5f687888f6dff42)

10 months agorgw/op: Init/CompleteMultipart add crypt attrs for iam policy 59344/head
Casey Bodley [Thu, 6 Apr 2023 15:29:55 +0000 (11:29 -0400)]
rgw/op: Init/CompleteMultipart add crypt attrs for iam policy

Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit 4cc6e05a7a550b277c22ac43f2088fe7a0423716)

10 months agorgw/op: RGWPostObj adds crypt attrs for iam policy
Casey Bodley [Thu, 6 Apr 2023 15:28:25 +0000 (11:28 -0400)]
rgw/op: RGWPostObj adds crypt attrs for iam policy

Fixes: https://tracker.ceph.com/issues/58594
Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit c224bae02218b574f360cb718feb8f2a50f7ac7e)

10 months agorgw/op: factor out rgw_iam_add_crypt_attrs()
Casey Bodley [Thu, 6 Apr 2023 14:53:52 +0000 (10:53 -0400)]
rgw/op: factor out rgw_iam_add_crypt_attrs()

make the iam environment variables for server-side encryption available
to other ops

Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit 3b517a2679c62529950cd66136f9fb971ae612cc)

10 months agorgw/lua: add lib64 to the package search path 59342/head
Yuval Lifshitz [Thu, 11 May 2023 18:59:32 +0000 (21:59 +0300)]
rgw/lua: add lib64 to the package search path

Fixes: https://tracker.ceph.com/issues/59733
Signed-off-by: Yuval Lifshitz <yuvalif@yahoo.com>
(cherry picked from commit 001e0c004ebb871fe09e70ba66fddbc545705c34)

10 months agotest/rgw/notifications: fix kafka consumer shutdown issue 59340/head
Yuval Lifshitz [Mon, 29 May 2023 11:54:00 +0000 (11:54 +0000)]
test/rgw/notifications: fix kafka consumer shutdown issue

Fixes: https://tracker.ceph.com/issues/61477
Signed-off-by: Yuval Lifshitz <ylifshit@redhat.com>
(cherry picked from commit b00f92da9a6cf848705c0508c0166c12913731e1)

10 months agoRevert "rgw: update last_added_entry when count == num_entries" 59337/head
J. Eric Ivancich [Thu, 3 Aug 2023 16:50:13 +0000 (12:50 -0400)]
Revert "rgw: update last_added_entry when count == num_entries"

This reverts commit 7511f9f675ea4e43992605dc03109bc5f356a5e1.

Signed-off-by: J. Eric Ivancich <ivancich@redhat.com>
(cherry picked from commit 81d5bb74d7cd77033f9ab031fff99999e80f4a58)

10 months agorgw/lua: fix CopyFrom crash 59336/head
Yuval Lifshitz [Mon, 10 Apr 2023 14:27:05 +0000 (14:27 +0000)]
rgw/lua: fix CopyFrom crash

Fixes: https://tracker.ceph.com/issues/59381
Signed-off-by: Yuval Lifshitz <ylifshit@redhat.com>
(cherry picked from commit 41ed6b3c353c74089da2e6e6d6564365ada1cfeb)

10 months agoqa/workunits/rados: enable crb and install generic package for c9 59330/head
Laura Flores [Mon, 19 Aug 2024 20:00:24 +0000 (20:00 +0000)]
qa/workunits/rados: enable crb and install generic package for c9

This workunit was still trying to enable
"codeready-builder-for-rhel-8-x86_64-rpms", which is a repo specific
to centos 8 stream. After changing the repo name, I found that there
is no libarchive-3.3.3 package available for centos 9 stream, so I
changed it to the generic version.

Fixes: https://tracker.ceph.com/issues/67562
Signed-off-by: Laura Flores <lflores@ibm.com>
10 months agoMerge pull request #54796 from k0ste/wip-59494-quincy
Yuri Weinstein [Mon, 19 Aug 2024 18:24:01 +0000 (11:24 -0700)]
Merge pull request #54796 from k0ste/wip-59494-quincy

quincy: cmake/rgw: librgw tests depend on ALLOC_LIBS

Reviewed-by: Casey Bodley <cbodley@redhat.com>
10 months agoqa: barbican: restrict python packages with upper-constraints 59325/head
Tobias Urdin [Thu, 15 Aug 2024 15:17:14 +0000 (17:17 +0200)]
qa: barbican: restrict python packages with upper-constraints

We install barbican by doing a pip install directly on the
cloned git repository but we don't honor the upper-constraints
from the OpenStack Requirements project that handles what
versions is supported.

This changes the pip install command that we issue when
installing barbican to honor the requirements for the
version (derived from the branch) that we use, in
this case it's the 2023.1 release upper-constraints [1].

This prevents us from pulling in untested Python packages.

This only updates Barbican because for the Keystone job
we dont directly issue pip but install using tox using the
`venv` environment which already by default sets the
constraints as you can see in [2].

[1] https://releases.openstack.org/constraints/upper/2023.1
[2] https://github.com/openstack/keystone/blob/stable/2023.1/tox.ini#L12

Fixes: https://tracker.ceph.com/issues/67444
Signed-off-by: Tobias Urdin <tobias.urdin@binero.com>
(cherry picked from commit bbcb820c779ed6c48ff4fa7c10730228f43e9305)

10 months agorgw/admin/notifications: support admin operations on topics with tenants 59322/head
Yuval Lifshitz [Sun, 23 Oct 2022 18:56:18 +0000 (18:56 +0000)]
rgw/admin/notifications: support admin operations on topics with tenants

also add integration tests for topics with tenants
issue was a regression introduced in commit: 200f71a90c9e77c91452cec128c2c8be0d3d6f1f

Fixes: https://tracker.ceph.com/issues/57899
Signed-off-by: Yuval Lifshitz <ylifshit@redhat.com>
(cherry picked from commit c73f361e9c84dce8b48eb8cca7675e659f3a286c)

10 months agorgw/rgw-orphan-list: refactor / add error checking 59321/head
Michael J. Kidd [Wed, 11 Jan 2023 19:53:50 +0000 (14:53 -0500)]
rgw/rgw-orphan-list: refactor / add error checking

standardize log output
add check that supplied pool[s] exist on the cluster
add return code checks for all commands that can affect output
add check for empty intermediate files
add usage output, including a -h command line option
add command line option to specify temporary directory to use
  ( instead of /tmp )
require command line supplied pools to be encased in double quotes "
  when multiple pools are provided
add warning for indexless buckets
move major segments to separate functions

Signed-off-by: Michael J. Kidd <linuxkidd@gmail.com>
(cherry picked from commit c951c3e3bd98532b97a46cce9de334d67009fc4b)

10 months agorgw/rgw-gap-list: refactor / add more err checks 59320/head
Michael J. Kidd [Fri, 13 Jan 2023 18:13:47 +0000 (13:13 -0500)]
rgw/rgw-gap-list: refactor / add more err checks

- rename function `log_out` to `log` to prevent mental confusion with
  'logging out' of the shell, vs sending a log message.

- Centralize error checking to provide more consistent output and ease
  of error check implementation

- Add check that supplied pool name(s) exist.

- Relocate the awk script creation to just before it's needed.

- Provide usage help for specifying multiple pools on the command line

- Add hostname to log output lines.

- add checks to ensure intermediate files are not empty before
  attempting comparison.

- switch to getopt to allow  more option flexibility, removing
  dependency on positional arguments which could cause parsing / user
  confusion.
  This is a **BREAKING change** due to change in command line syntax.

- add option for temp directory specification on the command line.

- set -m option to be presence based, vs needing 0|1 param

- fix typo in usage

Signed-off-by: Michael J. Kidd <linuxkidd@gmail.com>
(cherry picked from commit b7cdf6f97940411756dcd9ca504cd088f7d5677d)

10 months agorgw: avoid use-after-move in RGWDataSyncSingleEntryCR ctor 59319/head
Casey Bodley [Thu, 1 Sep 2022 14:43:39 +0000 (10:43 -0400)]
rgw: avoid use-after-move in RGWDataSyncSingleEntryCR ctor

the constructor body below accesses the constructor's 'obligation'
parameter instead of the member variable it was moved into

Fixes: https://tracker.ceph.com/issues/57329
Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit a4c2e6310d4fc41f99208c80525afe25412218c5)

10 months agorgw: Use STANDARD storage class in objects appending operation when the 59316/head
zhangzhiming [Mon, 17 Jan 2022 10:08:29 +0000 (18:08 +0800)]
rgw: Use STANDARD storage class in objects appending operation when the
storage class attr of the object appended is empty.

Fixes: https://tracker.ceph.com/issues/53902
Signed-off-by: zhiming zhang <zhangzhm1@chinatelecom.cn>
(cherry picked from commit 3662b22e87cd53b9a0045a42b3261ff58b78602d)

10 months agoMerge pull request #54695 from k0ste/wip-63662-quincy
Laura Flores [Mon, 19 Aug 2024 16:29:13 +0000 (11:29 -0500)]
Merge pull request #54695 from k0ste/wip-63662-quincy

quincy: doc/rados/configuration/bluestore-config-ref: Fix lowcase typo

10 months agolibrgw: teach librgw about rgw_backend_store 59315/head
Matt Benjamin [Thu, 21 Apr 2022 23:26:55 +0000 (19:26 -0400)]
librgw: teach librgw about rgw_backend_store

With this change, basic i/o against dbstore works in the primary
librgw_file tests.

Fixes: https://tracker.ceph.com/issues/55430
Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
(cherry picked from commit a40574eb5e3180986f301305c10d47b089673d78)

10 months agorgw: disable RGWDataChangesLog::add_entry() when log_data is off 59314/head
Casey Bodley [Fri, 11 Mar 2022 00:03:59 +0000 (19:03 -0500)]
rgw: disable RGWDataChangesLog::add_entry() when log_data is off

this restores a check for RGWZone::log_data in add_entry(). with
per-bucket replication, this check was replaced by a call to
`RGWBucketSyncPolicyHandler::bucket_exports_data()`

this call has to consult two rados objects,
`bucket.sync-source-hints.<bucketname>` and
`bucket.sync-target-hints.<bucketname>`

but if the zone is not configured for multisite, we should avoid these
extra object reads and return early

Fixes: https://tracker.ceph.com/issues/54531
Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit 6eec5e35a4f3c8c7e9783e3706e893cabef5993f)

10 months agorgw: cmake configure error on fedora-37/rawhide 59313/head
Kaleb S. KEITHLEY [Mon, 14 Feb 2022 12:42:08 +0000 (07:42 -0500)]
rgw: cmake configure error on fedora-37/rawhide

see
https://kojipkgs.fedoraproject.org//work/tasks/6624/82706624/build.log,
excerpted below

...
CMake Error: The inter-target dependency graph contains the following
strongly connected component (cycle): "rgw_common" of type
STATIC_LIBRARY depends on "dbstore" (weak) depends on "sqlite_db" (weak)
"dbstore" of type SHARED_LIBRARY depends on "rgw_common" (weak) depends
on "sqlite_db" (weak) "sqlite_db" of type SHARED_LIBRARY depends on
"rgw_common" (weak) depends on "dbstore" (weak) At least one of these
targets is not a STATIC_LIBRARY. Cyclic dependencies are allowed only
among static libraries. CMake Generate step failed. Build files cannot
be regenerated correctly.
...

https://tracker.ceph.com/issues/54266

Signed-off-by: Kaleb S. KEITHLEY <kkeithle@redhat.com>
(cherry picked from commit 78edb6a91788133334ceeeda0c95529df5f40da8)

10 months agoMerge pull request #59296 from zdover23/wip-doc-2024-08-19-backport-59256-to-quincy
Anthony D'Atri [Mon, 19 Aug 2024 12:54:28 +0000 (05:54 -0700)]
Merge pull request #59296 from zdover23/wip-doc-2024-08-19-backport-59256-to-quincy

quincy: doc/cephfs: s/mountpoint/mount point/

10 months agoMerge pull request #58558 from rishabh-d-dave/wip-66929-quincy
Xiubo Li [Mon, 19 Aug 2024 12:08:02 +0000 (20:08 +0800)]
Merge pull request #58558 from rishabh-d-dave/wip-66929-quincy

quincy: mgr/vol: handle case where clone index entry goes missing

10 months agoMerge pull request #55301 from neesingh-rh/wip-64143-quincy
Rishabh Dave [Mon, 19 Aug 2024 10:54:36 +0000 (16:24 +0530)]
Merge pull request #55301 from neesingh-rh/wip-64143-quincy

quincy: qa: fix rank_asok() to handle errors from asok commands

Reviewed-by: Rishabh Dave <ridave@redhat.com>
10 months agoMerge pull request #55916 from mchangir/wip-64224-quincy
Rishabh Dave [Mon, 19 Aug 2024 10:54:03 +0000 (16:24 +0530)]
Merge pull request #55916 from mchangir/wip-64224-quincy

quincy: qa: bump up scrub status command timeout

Reviewed-by: Rishabh Dave <ridave@redhat.com>
10 months agoMerge pull request #57526 from batrick/wip-66045-quincy
Rishabh Dave [Mon, 19 Aug 2024 10:53:19 +0000 (16:23 +0530)]
Merge pull request #57526 from batrick/wip-66045-quincy

quincy: qa: unmount clients before damaging the fs

Reviewed-by: Rishabh Dave <ridave@redhat.com>
10 months agoMerge pull request #57564 from dparmar18/wip-66062-quincy
Rishabh Dave [Mon, 19 Aug 2024 10:53:11 +0000 (16:23 +0530)]
Merge pull request #57564 from dparmar18/wip-66062-quincy

quincy: qa: add a YAML to ignore MGR_DOWN warning

Reviewed-by: Rishabh Dave <ridave@redhat.com>
10 months agoMerge pull request #58278 from joscollin/wip-56590-quincy
Rishabh Dave [Mon, 19 Aug 2024 10:52:51 +0000 (16:22 +0530)]
Merge pull request #58278 from joscollin/wip-56590-quincy

quincy: qa/suites/fs: skip check-counters for iogen workload

Reviewed-by: Rishabh Dave <ridave@redhat.com>
10 months agodoc/cephfs: s/mountpoint/mount point/ 59296/head
Zac Dover [Sat, 17 Aug 2024 03:37:58 +0000 (13:37 +1000)]
doc/cephfs: s/mountpoint/mount point/

Change the string "mountpoint" to "mount point" in English-language
strings (as opposed to in commands, where the string "mountpoint"
sometimes appears and is correct).

cf. https://github.com/ceph/ceph/pull/58908#discussion_r1697715486
in which page 345 of The IBM Style Guide is referenced to back up this
change.

Signed-off-by: Zac Dover <zac.dover@proton.me>
(cherry picked from commit 57ed73eaa6945c69059d299ea5310eaac57e2299)

10 months agocls_lock: expired lock before unlock and start check 59272/head
NitzanMordhai [Sun, 17 Jul 2022 08:01:30 +0000 (08:01 +0000)]
cls_lock: expired lock before unlock and start check

If the lock expired, the stat check shouldn't return -ENOENT,
We will change the lock duration to prevent lock expired before the
stat check.

Fixes: https://tracker.ceph.com/issues/56575
Signed-off-by: Nitzan Mordechai <nmordec@redhat.com>
(cherry picked from commit d3457c64b1bdc26a4379197ae206019a615d2ebc)

Comment from @idryomov:

There is a delta between the original commit and the backport due
to #45762 [1] not getting backported to quincy.

[1] https://github.com/ceph/ceph/pull/45762

11 months agoMerge pull request #59144 from idryomov/wip-53674-quincy
Yuri Weinstein [Sun, 18 Aug 2024 17:46:59 +0000 (10:46 -0700)]
Merge pull request #59144 from idryomov/wip-53674-quincy

quincy: librbd/crypto: fix issue when live-migrating from encrypted export

Reviewed-by: Ramana Raja <rraja@redhat.com>