msg/async: improve read-prefetch logic

We shall apply prefetch policy based on the residual length
instead of the original requested length.

E.g., suppose the recv_max_prefetch is 4K, and the read sequences are 1K, 5K, 2K

**Before this change:**
- read 1K, as 1K < recv_max_prefetch, we prefetch 4K, and the 1K read
  itself is hit in the cache after the prefetch is done.
- read 5K, the first 3K is hit in the cache and the cache is now empty,
  as 5K > recv_max_prefetch, we don't prefetch and trigger a 2K read instead.
- read 2K, the cache is now empty, as 2K > recv_max_prefetch, we trigger
  another prefetch and get 2K from the cache after prefetch is done.

**After this change:**
- read 1K, as 1K < recv_max_prefetch, we prefetch 4K, and the 1K read
  itself is hit in the cache after the prefetch is done.
- read 5K, the first 3K is hit in the cache and the cache is now empty
  and we have 5K-3K = 2K to read, as 2K < recv_max_prefetch, we prefetch
  again and get 2K from the cache after prefetch is done, the cache has
  2K data remaining.
- read 2K, which is directly hit in the cache.

From the above example, we need exactly 2 (prefetch)reads now instead of
3 reads which we need before.

See-also: https://github.com/ceph/ceph/pull/10344
Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

diff --git a/src/msg/async/AsyncConnection.cc b/src/msg/async/AsyncConnection.cc
index 5bc5c69fad96737a007a5525bde14c77dedf023d..a56c86d8915113da43ce3ca9064d3aaf166cf741 100644 (file)
--- a/src/msg/async/AsyncConnection.cc
+++ b/src/msg/async/AsyncConnection.cc
@@ -213,7 +213,7 @@ ssize_t AsyncConnection::read_until(unsigned len, char *p)
 
   recv_end = recv_start = 0;
   /* nothing left in the prefetch buffer */
-  if (len > recv_max_prefetch) {
+  if (left > (uint64_t)recv_max_prefetch) {
     /* this was a large read, we don't prefetch for these */
     do {
       r = read_bulk(p+state_offset, left);