]> git.apps.os.sepia.ceph.com Git - ceph.git/commit
rgw: reduce per-shard entry count during ordered bucket listing
authorJ. Eric Ivancich <ivancich@redhat.com>
Mon, 14 Oct 2019 20:21:35 +0000 (16:21 -0400)
committerJ. Eric Ivancich <ivancich@redhat.com>
Wed, 6 May 2020 15:41:01 +0000 (11:41 -0400)
commitd89def43c20f574d46b07836c238f6fbd31a3ab7
treeb75878e87f5efa2bef6c733f0cf33ce5e7efdda4
parent063e0d7c663d9de88b9fa35d20762537fdee1c28
rgw: reduce per-shard entry count during ordered bucket listing

Currently, if a client requests the 1000 next entries from a bucket,
each bucket index shard will receive a request for the 1000 next
entries. When there are hundreds, thousands, or tens of thousands of
bucket index shards, this results in a huge amplification of the
request, even though only 1000 entries will be returned.

These changes reduce the per-bucket index shard requests. These also
allow re-requests in edge cases where all of one shard's returned
entries are consumed. Finally these changes improve the determination
of whether the resulting list is truncated.

Signed-off-by: J. Eric Ivancich <ivancich@redhat.com>
(cherry picked from commit 729c0ba1361b76ecace94793b956354486b3dcd8)
PendingReleaseNotes
src/rgw/rgw_rados.cc
src/rgw/rgw_rados.h