allow clients to request more than the default 1000 keys per request
bucket listing of large buckets (with many shards) can be more efficient
with larger batches. raising max-keys also improves the balls-into-bins
algorithm which we artificially cap at a minimum of 8 keys per shard
with the default max-keys=1000, we reach this min=8 at around 350
shards. above that shard count, we're always requesting more than the
optimal number of entries per shard, so wasting i/o and bandwidth
as we raise the max-keys, we push this minimum to higher shard
counts:
raising max-keys does increase the total memory usage of each bucket
listing request, which gets into megabytes (tens to hundreds?). so i
chose 5000 as a cutoff that covers most of the range up to our current
rgw_max_dynamic_shards=1999