]> git.apps.os.sepia.ceph.com Git - ceph.git/log
ceph.git
21 months agoMerge pull request #52653 from joscollin/wip-62176-quincy
Yuri Weinstein [Wed, 11 Oct 2023 15:39:37 +0000 (08:39 -0700)]
Merge pull request #52653 from joscollin/wip-62176-quincy

quincy: qa: fix cephfs-mirror unwinding and 'fs volume create/rm' order

Reviewed-by: Venky Shankar <vshankar@redhat.com>
21 months agoMerge pull request #53785 from idryomov/wip-63010-quincy
Yuri Weinstein [Wed, 11 Oct 2023 15:24:35 +0000 (08:24 -0700)]
Merge pull request #53785 from idryomov/wip-63010-quincy

quincy: qa/suites/krbd: stress test for recovering from watch errors

Reviewed-by: Mykola Golub <mgolub@suse.com>
21 months agoMerge pull request #53933 from zdover23/wip-doc-2023-10-11-backport-53908-to-quincy
zdover23 [Wed, 11 Oct 2023 04:10:42 +0000 (14:10 +1000)]
Merge pull request #53933 from zdover23/wip-doc-2023-10-11-backport-53908-to-quincy

quincy: doc/rados: edit memory-profiling.rst

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>
21 months agodoc/rados: edit memory-profiling.rst 53933/head
Zac Dover [Tue, 10 Oct 2023 08:26:53 +0000 (18:26 +1000)]
doc/rados: edit memory-profiling.rst

Edit doc/rados/troubleshooting/memory-profiling.rst.

Co-authored-by: Anthony D'Atri <anthony.datri@gmail.com>
Signed-off-by: Zac Dover <zac.dover@proton.me>
(cherry picked from commit 3be932425e3020085db5abff6f914938015d8590)

21 months agoMerge pull request #53675 from kamoltat/wip-ksirivad-quincy-backport-51923
Yuri Weinstein [Tue, 10 Oct 2023 15:27:06 +0000 (08:27 -0700)]
Merge pull request #53675 from kamoltat/wip-ksirivad-quincy-backport-51923

quincy: pybind/mgr/pg_autoscaler: fix warn when not too few pgs

Reviewed-by: Laura Flores <lflores@redhat.com>
21 months agoMerge pull request #53895 from cbodley/wip-62743-quincy
Yuri Weinstein [Tue, 10 Oct 2023 15:02:55 +0000 (08:02 -0700)]
Merge pull request #53895 from cbodley/wip-62743-quincy

quincy: qa/rgw: add new POOL_APP_NOT_ENABLED failures to log-ignorelist

Reviewed-by: Casey Bodley <cbodley@redhat.com>
21 months agoMerge pull request #53821 from cfsnyder/wip-62943-quincy
Yuri Weinstein [Tue, 10 Oct 2023 14:32:16 +0000 (07:32 -0700)]
Merge pull request #53821 from cfsnyder/wip-62943-quincy

quincy: rgw: add radosgw-admin bucket check olh/unlinked commands

Reviewed-by: Casey Bodley <cbodley@redhat.com>
21 months agoMerge pull request #53774 from cbodley/wip-63056-quincy
Yuri Weinstein [Tue, 10 Oct 2023 14:31:38 +0000 (07:31 -0700)]
Merge pull request #53774 from cbodley/wip-63056-quincy

quincy: rgw: improve buffer list utilization in the chunkupload scenario

Reviewed-by: Casey Bodley <cbodley@redhat.com>
21 months agoMerge pull request #53771 from cbodley/wip-63053-quincy
Yuri Weinstein [Tue, 10 Oct 2023 14:30:54 +0000 (07:30 -0700)]
Merge pull request #53771 from cbodley/wip-63053-quincy

quincy: rgw: fix SignatureDoesNotMatch when extra headers start with 'x-amz'

Reviewed-by: Casey Bodley <cbodley@redhat.com>
21 months agoMerge pull request #53768 from cbodley/wip-63050-quincy
Yuri Weinstein [Tue, 10 Oct 2023 14:30:09 +0000 (07:30 -0700)]
Merge pull request #53768 from cbodley/wip-63050-quincy

quincy: rgw: s3website doesn't prefetch for web_dir() check

Reviewed-by: Casey Bodley <cbodley@redhat.com>
21 months agoMerge pull request #53766 from cbodley/wip-63047-quincy
Yuri Weinstein [Tue, 10 Oct 2023 14:29:19 +0000 (07:29 -0700)]
Merge pull request #53766 from cbodley/wip-63047-quincy

quincy: rgw: fix rgw rate limiting RGWRateLimitInfo class decode_json max_rea…

Reviewed-by: Casey Bodley <cbodley@redhat.com>
21 months agoMerge pull request #53761 from cbodley/wip-63059-quincy
Yuri Weinstein [Tue, 10 Oct 2023 14:28:19 +0000 (07:28 -0700)]
Merge pull request #53761 from cbodley/wip-63059-quincy

quincy: rgw: fix unwatch crash at radosgw startup

Reviewed-by: Casey Bodley <cbodley@redhat.com>
21 months agoMerge pull request #53757 from cbodley/wip-63041-quincy
Yuri Weinstein [Tue, 10 Oct 2023 14:27:18 +0000 (07:27 -0700)]
Merge pull request #53757 from cbodley/wip-63041-quincy

quincy: [CVE-2023-43040] rgw: Fix bucket validation against POST policies

Reviewed-by: Casey Bodley <cbodley@redhat.com>
21 months agoMerge pull request #53573 from kotreshhr/wip-62404-quincy
Venky Shankar [Tue, 10 Oct 2023 10:17:20 +0000 (15:47 +0530)]
Merge pull request #53573 from kotreshhr/wip-62404-quincy

quincy:    mgr/volumes: Fix pending_subvolume_deletions in volume info

Reviewed-by: Venky Shankar <vshankar@redhat.com>
21 months agoMerge pull request #53860 from ljflores/wip-63126-quincy
Yuri Weinstein [Mon, 9 Oct 2023 23:23:06 +0000 (16:23 -0700)]
Merge pull request #53860 from ljflores/wip-63126-quincy

quincy: osd: fix: slow scheduling when item_cost is large

Reviewed-by: Sridhar Seshasayee <sseshasa@redhat.com>
21 months agoMerge pull request #51412 from ifed01/wip-ifed-fix-fit-to-fast-qui
Yuri Weinstein [Mon, 9 Oct 2023 22:10:47 +0000 (15:10 -0700)]
Merge pull request #51412 from ifed01/wip-ifed-fix-fit-to-fast-qui

quincy: os/bluestore: allow 'fit_to_fast' selector for single-volume osd

Reviewed-by: Adam Kupczyk <akupczyk@redhat.com>
21 months agoMerge pull request #50636 from ifed01/wip-ifed-bound-rm-range-keys-qui
Yuri Weinstein [Mon, 9 Oct 2023 22:09:55 +0000 (15:09 -0700)]
Merge pull request #50636 from ifed01/wip-ifed-bound-rm-range-keys-qui

quincy: kv/RocksDBStore: cumulative backport for rm_range_keys and around

Reviewed-by: Adam Kupczyk <akupczyk@redhat.com>
21 months agoMerge pull request #50495 from ifed01/wip-ifed-fix-prefixed-kv-iterator-qui
Yuri Weinstein [Mon, 9 Oct 2023 22:09:05 +0000 (15:09 -0700)]
Merge pull request #50495 from ifed01/wip-ifed-fix-prefixed-kv-iterator-qui

quincy: kv/RocksDBStore: don't use real wholespace iterator for prefixed access

Reviewed-by: Adam Kupczyk <akupczyk@redhat.com>
21 months agoMerge pull request #50323 from ifed01/wip-ifed-57271-qui
Yuri Weinstein [Mon, 9 Oct 2023 22:08:24 +0000 (15:08 -0700)]
Merge pull request #50323 from ifed01/wip-ifed-57271-qui

quincy: blk/KernelDevice: Modify the rotational and discard check log message

Reviewed-by: Adam Kupczyk <akupczyk@redhat.com>
21 months agoMerge pull request #49984 from ifed01/wip-ifed-fix-read-error-code-qui
Yuri Weinstein [Mon, 9 Oct 2023 22:07:26 +0000 (15:07 -0700)]
Merge pull request #49984 from ifed01/wip-ifed-fix-read-error-code-qui

quincy: blk/kernel: Fix error code mapping in KernelDevice::read.

Reviewed-by: Adam Kupczyk <akupczyk@redhat.com>
21 months agoMerge pull request #48170 from ifed01/wip-ifed-fix-alloc-dump-qui
Yuri Weinstein [Mon, 9 Oct 2023 22:06:15 +0000 (15:06 -0700)]
Merge pull request #48170 from ifed01/wip-ifed-fix-alloc-dump-qui

quincy: os/bluestore: proper locking for Allocators' dump methods

Reviewed-by: Adam Kupczyk <akupczyk@redhat.com>
21 months agoMerge pull request #48169 from ifed01/wip-ifed-fix-bluefs-truncate-qui
Yuri Weinstein [Mon, 9 Oct 2023 22:04:39 +0000 (15:04 -0700)]
Merge pull request #48169 from ifed01/wip-ifed-fix-bluefs-truncate-qui

quincy: os/bluestore: fix no metadata update on truncate+fsync

Reviewed-by: Adam Kupczyk <akupczyk@redhat.com>
21 months agoMerge pull request #53898 from zdover23/wip-doc-2023-10-10-backport-53890-to-quincy
Anthony D'Atri [Mon, 9 Oct 2023 20:47:52 +0000 (16:47 -0400)]
Merge pull request #53898 from zdover23/wip-doc-2023-10-10-backport-53890-to-quincy

quincy: doc/rados: edit troubleshooting-mon.rst (4 of x)

21 months agoqa/rgw: add new POOL_APP_NOT_ENABLED failures to log-ignorelist 53895/head
Casey Bodley [Mon, 21 Aug 2023 19:43:04 +0000 (15:43 -0400)]
qa/rgw: add new POOL_APP_NOT_ENABLED failures to log-ignorelist

Fixes: https://tracker.ceph.com/issues/62504
Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit 4293d9bf117592a743ec70098cfb2bc1a9d38967)

21 months agodoc/rados: edit troubleshooting-mon.rst (4 of x) 53898/head
Zac Dover [Mon, 9 Oct 2023 08:55:07 +0000 (18:55 +1000)]
doc/rados: edit troubleshooting-mon.rst (4 of x)

Edit doc/rados/troubleshooting/troubleshooting-mon.rst.

Follows https://github.com/ceph/ceph/pull/53875

Co-authored-by: Anthony D'Atri <anthony.datri@gmail.com>
Signed-off-by: Zac Dover <zac.dover@proton.me>
(cherry picked from commit 99e92fb94b2c29e565e3f4e04658e103a09ef125)

21 months agoMerge pull request #53863 from vshankar/tr-62538
Yuri Weinstein [Mon, 9 Oct 2023 18:37:03 +0000 (11:37 -0700)]
Merge pull request #53863 from vshankar/tr-62538

quincy: qa: add `POOL_APP_NOT_ENABLED` to ignorelist for cephfs tests

Reviewed-by: Venky Shankar <vshankar@redhat.com>
21 months agoMerge pull request #53647 from vshankar/wip-62289-quincy
Yuri Weinstein [Mon, 9 Oct 2023 18:36:24 +0000 (11:36 -0700)]
Merge pull request #53647 from vshankar/wip-62289-quincy

[DNM] quincy: test: explicitly link to ceph-common for some libcephfs tests

Reviewed-by: Venky Shankar <vshankar@redhat.com>
21 months agoMerge pull request #53639 from vshankar/wip-62948-quincy
Yuri Weinstein [Mon, 9 Oct 2023 18:35:38 +0000 (11:35 -0700)]
Merge pull request #53639 from vshankar/wip-62948-quincy

quincy: cephfs-mirror: do not run concurrent C_RestartMirroring context

Reviewed-by: Venky Shankar <vshankar@redhat.com>
21 months agoMerge pull request #53454 from joscollin/wip-62835-quincy
Yuri Weinstein [Mon, 9 Oct 2023 18:33:38 +0000 (11:33 -0700)]
Merge pull request #53454 from joscollin/wip-62835-quincy

quincy: cephfs-top: include the missing fields in --dump output

Reviewed-by: Venky Shankar <vshankar@redhat.com>
21 months agoMerge pull request #53330 from vshankar/tr-62585
Yuri Weinstein [Mon, 9 Oct 2023 18:32:34 +0000 (11:32 -0700)]
Merge pull request #53330 from vshankar/tr-62585

quincy: mds: blocklist clients with "bloated" session metadata

Reviewed-by: Venky Shankar <vshankar@redhat.com>
21 months agoMerge pull request #53285 from vshankar/tr-61804
Yuri Weinstein [Mon, 9 Oct 2023 18:31:07 +0000 (11:31 -0700)]
Merge pull request #53285 from vshankar/tr-61804

quincy: cephfs-journal-tool: disambiguate usage of all keyword (in tool help).

Reviewed-by: Venky Shankar <vshankar@redhat.com>
21 months agoMerge pull request #53242 from batrick/wip-62660-quincy
Yuri Weinstein [Mon, 9 Oct 2023 18:30:12 +0000 (11:30 -0700)]
Merge pull request #53242 from batrick/wip-62660-quincy

quincy: mds: drop locks and retry when lock set changes

Reviewed-by: Venky Shankar <vshankar@redhat.com>
21 months agoMerge pull request #53188 from batrick/wip-62607-quincy
Yuri Weinstein [Mon, 9 Oct 2023 18:29:30 +0000 (11:29 -0700)]
Merge pull request #53188 from batrick/wip-62607-quincy

quincy: mgr: register OSDs in ms_handle_accept

Reviewed-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Ilya Dryomov <idryomov@redhat.com>
21 months agoMerge pull request #53184 from batrick/wip-62516-quincy
Yuri Weinstein [Mon, 9 Oct 2023 18:28:50 +0000 (11:28 -0700)]
Merge pull request #53184 from batrick/wip-62516-quincy

quincy: mds: acquire inode snaplock in open

Reviewed-by: Venky Shankar <vshankar@redhat.com>
21 months agoMerge pull request #53167 from leonid-s-usov/bp/cap-throttle-event/quincy
Yuri Weinstein [Mon, 9 Oct 2023 18:28:01 +0000 (11:28 -0700)]
Merge pull request #53167 from leonid-s-usov/bp/cap-throttle-event/quincy

quincy: mds/Server: mark a cap acquisition throttle event in the request

Reviewed-by: Venky Shankar <vshankar@redhat.com>
21 months agoqa/rgw: add POOL_APP_NOT_ENABLED to log-ignorelist
Casey Bodley [Mon, 15 May 2023 18:26:48 +0000 (14:26 -0400)]
qa/rgw: add POOL_APP_NOT_ENABLED to log-ignorelist

Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit f0d53e56f8d05090a5f429c8826ca552e0140798)

21 months agoMerge pull request #53414 from trociny/wip-62507-quincy
Yuri Weinstein [Mon, 9 Oct 2023 15:12:12 +0000 (08:12 -0700)]
Merge pull request #53414 from trociny/wip-62507-quincy

quincy: rgw: handle http options CORS with v4 auth

Reviewed-by: Casey Bodley <cbodley@redhat.com>
21 months agoqa: typo fix when checking for perf counter - s/md_thresh_evicted/mdthresh_evicted 53330/head
Venky Shankar [Mon, 9 Oct 2023 04:29:45 +0000 (09:59 +0530)]
qa: typo fix when checking for perf counter - s/md_thresh_evicted/mdthresh_evicted

Signed-off-by: Venky Shankar <vshankar@redhat.com>
(cherry picked from commit 5856a1e6b72dc77d3b4d9109385481b924e4fe36)

21 months agoqa: lower mds_session_metadata_threshold for tests
Venky Shankar [Sat, 7 Oct 2023 15:12:29 +0000 (20:42 +0530)]
qa: lower mds_session_metadata_threshold for tests

... and increase the number of files that are created so as to
hit the threshold with a high probability.

Fixes: http://tracker.ceph.com/issues/62873
Signed-off-by: Venky Shankar <vshankar@redhat.com>
(cherry picked from commit 92200d9d10d30ed35f640fd1120be65577a3ce64)

21 months agoMerge pull request #53880 from zdover23/wip-doc-2023-10-08-backport-53875-to-quincy
Anthony D'Atri [Sun, 8 Oct 2023 12:53:22 +0000 (08:53 -0400)]
Merge pull request #53880 from zdover23/wip-doc-2023-10-08-backport-53875-to-quincy

quincy: doc/rados: edit troubleshooting-mon.rst (3 of x)

21 months agoMerge pull request #53882 from zdover23/wip-doc-2023-10-08-backport-53874-to-quincy
Anthony D'Atri [Sun, 8 Oct 2023 12:51:29 +0000 (08:51 -0400)]
Merge pull request #53882 from zdover23/wip-doc-2023-10-08-backport-53874-to-quincy

quincy: doc/rados: edit troubleshooting/community.rst

21 months agodoc/rados: edit troubleshooting/community.rst 53882/head
Zac Dover [Sat, 7 Oct 2023 20:39:23 +0000 (06:39 +1000)]
doc/rados: edit troubleshooting/community.rst

Edit doc/rados/troubleshooting/community.rst.

Co-authored-by: Anthony D'Atri <anthony.datri@gmail.com>
Signed-off-by: Zac Dover <zac.dover@proton.me>
(cherry picked from commit fabfec273474a85cf3687ddf89183f0607c2ad71)

21 months agodoc/rados: edit troubleshooting-mon.rst (3 of x) 53880/head
Zac Dover [Sat, 7 Oct 2023 21:33:17 +0000 (07:33 +1000)]
doc/rados: edit troubleshooting-mon.rst (3 of x)

Edit doc/rados/troubleshooting/troubleshooting-mon.rst.

Follows https://github.com/ceph/ceph/pull/52827

Co-authored-by: Anthony D'Atri <anthony.datri@gmail.com>
Signed-off-by: Zac Dover <zac.dover@proton.me>
(cherry picked from commit fc45a0c4dd0247f472a940d6c92e7acdb0cdc354)

21 months agoMerge pull request #53878 from zdover23/wip-doc-2023-10-08-backport-53876-to-quincy
Anthony D'Atri [Sun, 8 Oct 2023 01:55:00 +0000 (21:55 -0400)]
Merge pull request #53878 from zdover23/wip-doc-2023-10-08-backport-53876-to-quincy

quincy: doc/architecture: repair RBD sentence

21 months agodoc/architecture: repair RBD sentence 53878/head
Zac Dover [Sat, 7 Oct 2023 21:43:43 +0000 (07:43 +1000)]
doc/architecture: repair RBD sentence

Improve an ambiguous sentence in doc/architecture.rst.

The problem presented by the original sentence is that the phrasal verb
"to provide with" is implicated in one of its possible readings.
Interpreted in that way, the sentence seems to express the incorrect
idea that RBD furnishes block devices with snapshotting and cloning, as
though snapshotting and cloning are being delivered to the block
devices. In fact, snapshotting and cloning are just features of RBD, and
are features that are described on this page:
https://docs.ceph.com/en/quincy/rbd/rbd-snapshot/.

Signed-off-by: Zac Dover <zac.dover@proton.me>
(cherry picked from commit 5abd530460e3d5a60142d9350d93544402b6dec4)

21 months agoMerge pull request #53664 from lxbsz/wip-62519
Yuri Weinstein [Sat, 7 Oct 2023 22:04:59 +0000 (15:04 -0700)]
Merge pull request #53664 from lxbsz/wip-62519

quincy: client: move the Inode to new auth mds session when changing auth cap

Reviewed-by: Venky Shankar <vshankar@redhat.com>
21 months agoMerge pull request #53663 from lxbsz/wip-62522
Yuri Weinstein [Sat, 7 Oct 2023 22:04:10 +0000 (15:04 -0700)]
Merge pull request #53663 from lxbsz/wip-62522

quincy: ceph: allow xlock state to be LOCK_PREXLOCK when putting it

Reviewed-by: Venky Shankar <vshankar@redhat.com>
21 months agoqa: use exisitng ignorelist override list for fs:mirror[-ha] 53863/head
Venky Shankar [Tue, 12 Sep 2023 10:21:05 +0000 (15:51 +0530)]
qa: use exisitng ignorelist override list for fs:mirror[-ha]

The custom ignorelist yaml would also require `POOL_APP_NOT_ENABLED`
to be ignored. The custom yaml is unnecessary anyway.

Fixes: http://tracker.ceph.com/issues/62482
Signed-off-by: Venky Shankar <vshankar@redhat.com>
(cherry picked from commit a728d170cc8e610c7dbb952197303de7b083c650)

21 months agoMerge pull request #53872 from zdover23/wip-doc-2023-10-07-backport-53790-to-quincy
zdover23 [Sat, 7 Oct 2023 05:33:26 +0000 (16:33 +1100)]
Merge pull request #53872 from zdover23/wip-doc-2023-10-07-backport-53790-to-quincy

quincy: doc/architecture: edit "Peering and Sets"

Reviewed-by: Cole Mitchell <cole.mitchell.ceph@gmail.com>
21 months agodoc/architecture: edit "Peering and Sets" 53872/head
Zac Dover [Tue, 3 Oct 2023 12:11:50 +0000 (22:11 +1000)]
doc/architecture: edit "Peering and Sets"

Edit the English in the section "Peering and Sets" in the file
doc/architecture.rst.

Co-authored-by: Anthony D'Atri <anthony.datri@gmail.com>
Signed-off-by: Zac Dover <zac.dover@proton.me>
(cherry picked from commit c69b111966f190ac4a1ebd5244a9cf9f61970dd7)

21 months agoMerge pull request #52799 from cbodley/wip-62302-quincy
Shilpa Jagannath [Fri, 6 Oct 2023 18:03:14 +0000 (14:03 -0400)]
Merge pull request #52799 from cbodley/wip-62302-quincy

quincy: rgw: retry metadata cache notifications with INVALIDATE_OBJ

21 months agoRevert "Revert "Merge PR #53077 into main""
Venky Shankar [Fri, 8 Sep 2023 08:41:13 +0000 (14:11 +0530)]
Revert "Revert "Merge PR #53077 into main""

This reverts commit af4558f0ac3b0dc23a8b3cbba63b9c8acc498ea4.

Fixes: http://tracker.ceph.com/issues/62482
Signed-off-by: Venky Shankar <vshankar@redhat.com>
(cherry picked from commit 7d5795dc37fc903121f2caa584bec55773fb61ee)

21 months agoqa: do not create rbd pool for CephFS QA
Patrick Donnelly [Thu, 17 Aug 2023 13:27:45 +0000 (09:27 -0400)]
qa: do not create rbd pool for CephFS QA

Fixes: https://tracker.ceph.com/issues/62482
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit 636d2a44c7a935426788ed61e9a77432dc49ca5e)

21 months agoRevert "Merge PR #53077 into main"
Patrick Donnelly [Mon, 28 Aug 2023 13:16:10 +0000 (09:16 -0400)]
Revert "Merge PR #53077 into main"

This reverts commit b8bf0c62b96cb76ae547d5d2f13d2a68417e79e9, reversing
changes made to fe07f64396e504c373bd1b621dda0b7893753fe8.

Silencing this health warning is unnecessary if we stop creating the rbd pool
in CephFS testing.

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit af4558f0ac3b0dc23a8b3cbba63b9c8acc498ea4)

21 months agoqa: add `POOL_APP_NOT_ENABLED` to ignorelist for cephfs tests
Venky Shankar [Tue, 22 Aug 2023 06:14:55 +0000 (11:44 +0530)]
qa: add `POOL_APP_NOT_ENABLED` to ignorelist for cephfs tests

Fixes: http://tracker.ceph.com/issues/62508
Signed-off-by: Venky Shankar <vshankar@redhat.com>
(cherry picked from commit 97b76a480b9708f8557222de14058f644746a4a6)

21 months agoosd: fix: slow scheduling when item_cost is large 53860/head
jrchyang [Tue, 12 Sep 2023 09:31:28 +0000 (09:31 +0000)]
osd: fix: slow scheduling when item_cost is large

We use the iops and bandwidth tested by
`ceph tell osd.0 bench 10737418240 204800 204800 100`
to verify the QoS function. iops was 400 and bandwidth was 80MiB/s.
When osd_mclock_scheduler_client_lim is set to 1,
the sequential write bandwidth is only half of the capacity.
Therefore, we believe that it should not unconditionally increase
osd_bandwidth_cost_per_io for each IO, but take the maximum of the two.

Fixes: https://tracker.ceph.com/issues/62812
co-author: yanghonggang <yanghonggang_yewu@cmss.chinamobile.com>
co-author: zhangjianwei <zhangjianwei2_yewu@cmss.chinamobile.com>
Signed-off-by: Jrchyang Yu <yuzhiqiang_yewu@cmss.chinamobile.com>
(cherry picked from commit 65ebf2d6f1b41c93ee2e170437780846d20f5480)

21 months agoMerge pull request #53854 from zdover23/wip-doc-2023-10-06-backport-53834-to-quincy
Anthony D'Atri [Fri, 6 Oct 2023 00:41:35 +0000 (20:41 -0400)]
Merge pull request #53854 from zdover23/wip-doc-2023-10-06-backport-53834-to-quincy

quincy: doc: remove egg fragment from dev/developer_guide/running-tests-locally

21 months agodoc: remove egg fragment from dev/developer_guide/running-tests-locally 53854/head
Dhairya Parmar [Thu, 5 Oct 2023 08:12:31 +0000 (13:42 +0530)]
doc: remove egg fragment from dev/developer_guide/running-tests-locally

DEPRECATION: git+https://github.com/ceph/teuthology#egg=teuthology
[test] contains an egg fragment with a non-PEP 508 name pip 25.0 will enforce
this behaviour change. A possible replacement is to use the req @ url syntax,
and remove the egg fragment. Discussion can be found at
https://github.com/pypa/pip/issues/11617

Signed-off-by: Dhairya Parmar <dparmar@redhat.com>
(cherry picked from commit 1f78a32a1822fea1a56a88215e452b730b7672c7)

21 months agoMerge pull request #53102 from batrick/wip-62554-quincy
Yuri Weinstein [Thu, 5 Oct 2023 18:01:06 +0000 (11:01 -0700)]
Merge pull request #53102 from batrick/wip-62554-quincy

quincy: libcephsqlite: fill 0s in unread portion of buffer

Reviewed-by: Laura Flores <lflores@redhat.com>
21 months agoMerge pull request #53566 from pdvian/wip-59177-quincy
Yuri Weinstein [Thu, 5 Oct 2023 18:00:05 +0000 (11:00 -0700)]
Merge pull request #53566 from pdvian/wip-59177-quincy

quincy: blk/kernel: Add O_EXCL for block devices

Reviewed-by: Laura Flores <lflores@redhat.com>
21 months agoMerge pull request #53158 from ljflores/wip-62592-quincy
Yuri Weinstein [Thu, 5 Oct 2023 17:59:24 +0000 (10:59 -0700)]
Merge pull request #53158 from ljflores/wip-62592-quincy

quincy: python-common: drive_selection: fix KeyError when osdspec_affinity is not set

Reviewed-by: Adam King adking@redhat.com
21 months agoMerge pull request #52952 from k0ste/wip-62118-quincy
Yuri Weinstein [Thu, 5 Oct 2023 17:58:22 +0000 (10:58 -0700)]
Merge pull request #52952 from k0ste/wip-62118-quincy

quincy: qa: use parallel gzip for compressing logs

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
21 months agoMerge pull request #52215 from batrick/wip-61834-quincy
Yuri Weinstein [Thu, 5 Oct 2023 17:57:26 +0000 (10:57 -0700)]
Merge pull request #52215 from batrick/wip-61834-quincy

quincy: pybind/mgr/devicehealth: do not crash if db not ready

Reviewed-by: Laura Flores <lflores@redhat.com>
21 months agoMerge pull request #51811 from NitzanMordhai/wip-61487-quincy
Yuri Weinstein [Thu, 5 Oct 2023 17:56:42 +0000 (10:56 -0700)]
Merge pull request #51811 from NitzanMordhai/wip-61487-quincy

quincy: pybind/argparse: blocklist ip validation

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
21 months agoMerge pull request #53838 from zdover23/wip-doc-2023-10-05-backport-53819-to-quincy
Anthony D'Atri [Thu, 5 Oct 2023 16:52:21 +0000 (12:52 -0400)]
Merge pull request #53838 from zdover23/wip-doc-2023-10-05-backport-53819-to-quincy

quincy: doc/rados: edit troubleshooting.rst

21 months agoMerge pull request #53729 from ceph/revert-50974-wip-59201-quincy
Yuri Weinstein [Thu, 5 Oct 2023 14:27:39 +0000 (07:27 -0700)]
Merge pull request #53729 from ceph/revert-50974-wip-59201-quincy

Revert "quincy: qa: add subvolume option flavors"

Reviewed-by: Milind Changire <mchangir@redhat.com>
21 months agoMerge pull request #53677 from kamoltat/wip-ksirivad-quincy-backport-52442
Yuri Weinstein [Thu, 5 Oct 2023 14:20:02 +0000 (07:20 -0700)]
Merge pull request #53677 from kamoltat/wip-ksirivad-quincy-backport-52442

quincy: pybind/mgr/pg_autoscaler: noautoscale flag retains individual pool configs

Reviewed-by: Sridhar Seshasayee <sseshasa@redhat.com>
21 months agoMerge pull request #53092 from sseshasa/wip-62547-quincy
Yuri Weinstein [Thu, 5 Oct 2023 14:19:02 +0000 (07:19 -0700)]
Merge pull request #53092 from sseshasa/wip-62547-quincy

quincy: osd/scheduler/mClockScheduler: Use same profile and client ids for all clients to ensure allocated QoS limit consumption.

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
21 months agodoc/rados: edit troubleshooting.rst 53838/head
Zac Dover [Wed, 4 Oct 2023 11:14:13 +0000 (21:14 +1000)]
doc/rados: edit troubleshooting.rst

Edit doc/rads/troubleshooting.rst to remove some language that sounds
quite close to marketing language.

Signed-off-by: Zac Dover <zac.dover@proton.me>
(cherry picked from commit 9d9d92837e44e3af233bc228850be09e8f64eb7c)

21 months agoMerge pull request #53812 from zdover23/wip-doc-2023-10-04-backport-53791-to-quincy
zdover23 [Thu, 5 Oct 2023 11:31:45 +0000 (22:31 +1100)]
Merge pull request #53812 from zdover23/wip-doc-2023-10-04-backport-53791-to-quincy

quincy: doc/rados: edit ops/control.rst (1 of x)

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>
21 months agoMerge pull request #48523 from k0ste/wip-57886-quincy
Avan [Thu, 5 Oct 2023 09:34:24 +0000 (15:04 +0530)]
Merge pull request #48523 from k0ste/wip-57886-quincy

quincy: mgr/prometheus: avoid duplicates and deleted entries for rbd_stats_pools

Reviewed-by: Avan Thakkar <athakkar@redhat.com>
21 months agoMerge pull request #52739 from theanalyst/wip-58818
Yuri Weinstein [Wed, 4 Oct 2023 20:29:11 +0000 (13:29 -0700)]
Merge pull request #52739 from theanalyst/wip-58818

quincy: rgw: swift : check for valid key in POST forms

Reviewed-by: Casey Bodley <cbodley@redhat.com>
21 months agoMerge pull request #53690 from pdvian/wip-62995-quincy
Radoslaw Zarzynski [Wed, 4 Oct 2023 19:14:50 +0000 (21:14 +0200)]
Merge pull request #53690 from pdvian/wip-62995-quincy

quincy: osd/OpRequest: Add detailed description for delayed op in osd log file

Reviewed-by: Radosław Zarzyński <rzarzyns@redhat.com>
21 months agoMerge pull request #52585 from rishabh-d-dave/wip-58991-quincy
Yuri Weinstein [Wed, 4 Oct 2023 16:18:47 +0000 (09:18 -0700)]
Merge pull request #52585 from rishabh-d-dave/wip-58991-quincy

quincy: qa/cephfs: add 'rhel' to family of RH OS in xfstest_dev.py

Reviewed-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Milind Changire <mchangir@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
21 months agoMerge pull request #53042 from pdvian/wip-62479-quincy
Yuri Weinstein [Wed, 4 Oct 2023 15:39:28 +0000 (08:39 -0700)]
Merge pull request #53042 from pdvian/wip-62479-quincy

quincy: mon, qa: issue pool application warning even if pool is empty

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
21 months agoMerge pull request #52752 from mchangir/wip-61795-quincy
Yuri Weinstein [Wed, 4 Oct 2023 15:38:45 +0000 (08:38 -0700)]
Merge pull request #52752 from mchangir/wip-61795-quincy

quincy: mgr/snap_schedule: catch all exceptions for cli

Reviewed-by: Nitzan Mordechai <nmordech@redhat.com>
21 months agoMerge pull request #52722 from NitzanMordhai/wip-62252-quincy
Yuri Weinstein [Wed, 4 Oct 2023 15:37:36 +0000 (08:37 -0700)]
Merge pull request #52722 from NitzanMordhai/wip-62252-quincy

quincy: qa/standalone/osd/divergent-prior.sh: Divergent test 3 with pg_autoscale_mode on pick divergent osd

Reviewed-by: Kamoltat (Junior) Sirivadhna <ksirivad@redhat.com>
21 months agoMerge pull request #52534 from idryomov/wip-61733-quincy
Yuri Weinstein [Wed, 4 Oct 2023 15:35:49 +0000 (08:35 -0700)]
Merge pull request #52534 from idryomov/wip-61733-quincy

quincy: mon/MonClient: resurrect original client_mount_timeout handling

Reviewed-by: Nitzan Mordechai <nmordech@redhat.com>
21 months agoMerge pull request #52467 from k0ste/wip-62030-quincy
Yuri Weinstein [Wed, 4 Oct 2023 15:34:26 +0000 (08:34 -0700)]
Merge pull request #52467 from k0ste/wip-62030-quincy

quincy: mon/ConfigMonitor: update crush_location from osd entity

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
21 months agoMerge pull request #47840 from NitzanMordhai/wip-57316-quincy
Yuri Weinstein [Wed, 4 Oct 2023 15:33:12 +0000 (08:33 -0700)]
Merge pull request #47840 from NitzanMordhai/wip-57316-quincy

quincy: osd, mon: add pglog dups length

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
21 months agoMerge pull request #53139 from mihalicyn/wip-62571-quincy
Yuri Weinstein [Wed, 4 Oct 2023 15:30:50 +0000 (08:30 -0700)]
Merge pull request #53139 from mihalicyn/wip-62571-quincy

quincy: ceph_fs.h: add separate owner_{u,g}id fields

Reviewed-by: Milind Changire <mchangir@redhat.com>
21 months agoMerge pull request #52983 from batrick/wip-61799-quincy
Yuri Weinstein [Wed, 4 Oct 2023 15:30:18 +0000 (08:30 -0700)]
Merge pull request #52983 from batrick/wip-61799-quincy

quincy: mon/MDSMonitor: plug paxos when maybe manipulating osdmap

Reviewed-by: Milind Changire <mchangir@redhat.com>
21 months agoMerge pull request #52902 from leonid-s-usov/backport/bulk-data-pool/quincy
Yuri Weinstein [Wed, 4 Oct 2023 15:29:36 +0000 (08:29 -0700)]
Merge pull request #52902 from leonid-s-usov/backport/bulk-data-pool/quincy

quincy: Consider setting "bulk" autoscale pool flag when automatically creating a data pool for CephFS

Reviewed-by: Milind Changire <mchangir@redhat.com>
Reviewed-by: Jos Collin <jcollin@redhat.com>
21 months agoMerge pull request #52851 from lxbsz/wip-61167
Yuri Weinstein [Wed, 4 Oct 2023 15:28:42 +0000 (08:28 -0700)]
Merge pull request #52851 from lxbsz/wip-61167

quincy: client: issue a cap release immediately if no cap exists

Reviewed-by: Milind Changire <mchangir@redhat.com>
21 months agoMerge pull request #52849 from lxbsz/wip-62194
Yuri Weinstein [Wed, 4 Oct 2023 15:28:09 +0000 (08:28 -0700)]
Merge pull request #52849 from lxbsz/wip-62194

quincy: mds: do not send split_realms for CEPH_SNAP_OP_UPDATE msg

Reviewed-by: Milind Changire <mchangir@redhat.com>
21 months agoMerge pull request #52845 from lxbsz/wip-62200
Yuri Weinstein [Wed, 4 Oct 2023 15:27:12 +0000 (08:27 -0700)]
Merge pull request #52845 from lxbsz/wip-62200

quincy: mds: skip forwarding request if the session were removed

Reviewed-by: Milind Changire <mchangir@redhat.com>
21 months agoMerge pull request #52749 from mchangir/wip-61991-quincy
Yuri Weinstein [Wed, 4 Oct 2023 15:26:30 +0000 (08:26 -0700)]
Merge pull request #52749 from mchangir/wip-61991-quincy

quincy: mgr/snap_schedule: allow retention spec 'n' to be user defined

Reviewed-by: Milind Changire <mchangir@redhat.com>
21 months agoMerge pull request #52743 from batrick/wip-62270-quincy
Yuri Weinstein [Wed, 4 Oct 2023 15:25:30 +0000 (08:25 -0700)]
Merge pull request #52743 from batrick/wip-62270-quincy

quincy: qa: wait for file to have correct size

Reviewed-by: Milind Changire <mchangir@redhat.com>
21 months agoMerge pull request #52725 from kotreshhr/wip-62241-quincy
Yuri Weinstein [Wed, 4 Oct 2023 15:24:10 +0000 (08:24 -0700)]
Merge pull request #52725 from kotreshhr/wip-62241-quincy

quincy: mds: Fix the linkmerge assert check

Reviewed-by: Milind Changire <mchangir@redhat.com>
21 months agoMerge pull request #52683 from batrick/wip-62191-quincy
Yuri Weinstein [Wed, 4 Oct 2023 15:15:38 +0000 (08:15 -0700)]
Merge pull request #52683 from batrick/wip-62191-quincy

quincy: mds: update mdlog perf counters during replay

Reviewed-by: Milind Changire <mchangir@redhat.com>
21 months agorgw: fix output formatting of bucket index check admin api 53821/head
Cory Snyder [Mon, 25 Sep 2023 10:06:41 +0000 (10:06 +0000)]
rgw: fix output formatting of bucket index check admin api

The bucket index check admin API was previously returning invalid
JSON.

Signed-off-by: Cory Snyder <csnyder@1111systems.com>
(cherry picked from commit 32fb6a1a68398a99324b2e64ebe3bcf3a9ccf02a)

Conflicts:
src/rgw/driver/rados/rgw_bucket.cc

Cherry-pick notes:
- Conflicts due to rgw_bucket.cc moved to driver/rados directory on main

21 months agorgw: fix radosgw-admin bucket check stat calculation bug
Cory Snyder [Fri, 22 Sep 2023 21:08:25 +0000 (21:08 +0000)]
rgw: fix radosgw-admin bucket check stat calculation bug

Fixes a regression with radosgw-admin bucket check stat
calculation and bucket reshard stat calculation when
there are objects that have transitioned from unversioned
to versioned. The bug was introduced in
152aadb71b61c53a4832a1c8cf82fce3d64b68d1.

Signed-off-by: Cory Snyder <csnyder@1111systems.com>
(cherry picked from commit 4728daa5557bfb79a608dd903b8630e2b15fcb2c)

21 months agorgw: add test case to reproduce bucket check stats bug for versioned bucket
Cory Snyder [Fri, 22 Sep 2023 21:00:46 +0000 (21:00 +0000)]
rgw: add test case to reproduce bucket check stats bug for versioned bucket

Reproduces a regression where radosgw-admin bucket check incorrectly counts
objects that started as unversioned and later transitioned to versioned.

Signed-off-by: Cory Snyder <csnyder@1111systems.com>
(cherry picked from commit 340522f9aed50d65137568c1f9dcf4b1e7945a79)

21 months agorgw: radosgw-admin bucket check should only print index entries with --check-objects...
Cory Snyder [Fri, 22 Sep 2023 08:35:16 +0000 (08:35 +0000)]
rgw: radosgw-admin bucket check should only print index entries with --check-objects flag

Printing all index entries can be very time consuming for large
buckets and the inability to switch this behavior off makes it
cumbersome to use the command for fixing bucket stats. This was
also preventing the command from outputting recalculated bucket
stats when the --fix flag wasn't specified.

Signed-off-by: Cory Snyder <csnyder@1111systems.com>
(cherry picked from commit 6b057fe55413c0eaf9959f006584cba6cc4c192a)

Conflicts:
src/rgw/driver/rados/rgw_bucket.cc

Cherry-pick notes:
- Conflicts due to rgw_bucket.cc moved to driver/rados directory on main

21 months agorgw: prevent another leftover bucket index olh entry scenario
Cory Snyder [Thu, 21 Sep 2023 19:27:51 +0000 (19:27 +0000)]
rgw: prevent another leftover bucket index olh entry scenario

If a call to bucket_index_link_olh or bucket_index_unlink_instance
fails, its associated pending xattr may have prevented the olh object
from being removed by another thread. We should do a best effort
cleanup attempt for this case by calling update_olh before returning
an error to the caller.

Signed-off-by: Cory Snyder <csnyder@1111systems.com>
(cherry picked from commit 570adec5bb8142f5baf1f05f0040e8afdb11ec05)

21 months agorgw: fix rgw versioned bucket stat accounting during reshard and check index
Cory Snyder [Thu, 7 Sep 2023 17:23:14 +0000 (17:23 +0000)]
rgw: fix rgw versioned bucket stat accounting during reshard and check index

Fixes: https://tracker.ceph.com/issues/62760
Signed-off-by: Cory Snyder <csnyder@1111systems.com>
(cherry picked from commit 152aadb71b61c53a4832a1c8cf82fce3d64b68d1)

Conflicts:
src/cls/rgw/cls_rgw.cc
src/cls/rgw/cls_rgw_types.cc

Cherry-pick notes:
- cls_rgw_types had extra generate_test_instances method on main
- cls_rgw.cc had minor changes within check_index on main

21 months agoqa/workunits/rgw: add tests that reproduce bucket stats inconsistency bugs
Cory Snyder [Thu, 7 Sep 2023 14:43:23 +0000 (14:43 +0000)]
qa/workunits/rgw: add tests that reproduce bucket stats inconsistency bugs

Signed-off-by: Cory Snyder <csnyder@1111systems.com>
(cherry picked from commit b79dcf640ac2cc3dacf1b87bbe351db823c445d0)

Conflicts:
qa/workunits/rgw/test_rgw_reshard.py

Cherry-pick notes:
- test_rgw_reshard conflicts due to some test cases from main not present on quincy

21 months agoMerge branch 'quincy' into wip-doc-2023-10-04-backport-53791-to-quincy 53812/head
zdover23 [Wed, 4 Oct 2023 14:37:34 +0000 (01:37 +1100)]
Merge branch 'quincy' into wip-doc-2023-10-04-backport-53791-to-quincy

Signed-off-by: zdover23 <zac.dover@proton.me>
21 months agoMerge pull request #53816 from zdover23/wip-doc-2023-10-04-backport-53792-to-quincy
Anthony D'Atri [Wed, 4 Oct 2023 13:01:41 +0000 (09:01 -0400)]
Merge pull request #53816 from zdover23/wip-doc-2023-10-04-backport-53792-to-quincy

quincy: doc/rados: edit ops/control.rst (2 of x)

21 months agoMerge pull request #48249 from ifed01/wip-ifed-fix-57121-qui
Igor Fedotov [Wed, 4 Oct 2023 11:55:53 +0000 (14:55 +0300)]
Merge pull request #48249 from ifed01/wip-ifed-fix-57121-qui

quincy: osd: set per_pool_stats true when OSD has no PG

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>