git.apps.os.sepia.ceph.com Git

author	liubingrun <liubr1@chinatelecom.cn>
	Sun, 3 Sep 2023 18:29:47 +0000 (14:29 -0400)
committer	Casey Bodley <cbodley@redhat.com>
	Mon, 2 Oct 2023 20:22:18 +0000 (16:22 -0400)
commit	fea212bc5a283c954dd6bb6956efef6b7d76c09b
tree	92fa1177bc4d573028d6f4c1bef1d2445429aebf	tree \| snapshot
parent	987fa95080ba82385f45e0889919cddc5e2645b0	commit \| diff

rgw: improve the efficiency of buffer list utilization of chunk upload

Reduced waste of buffer::ptr by receiving multiple chunks and filling them into the buffer

AWSv4ComplMulti::recv_body() just receive one chunk and fill it into buffer.
Each 4MB buffer is actually only utilizing 64KB, leading to frequent buffer allocations.
~800GB virtual memory consumption has been observed.

Signed-off-by: liubingrun <liubr1@chinatelecom.cn>
(cherry picked from commit e47bb22f69ee83479431994ce6514bb739f5dd98)

src/rgw/rgw_auth_s3.cc		diff \| blob \| history
src/rgw/rgw_auth_s3.h		diff \| blob \| history