block: ensure plug merging checks the correct queue at least once

Song reports that a RAID rebuild workload runs much slower recently,
and it is seeing a lot less merging than it did previously. The reason
is that a previous commit reduced the amount of work we do for plug
merging. RAID rebuild interleaves requests between disks, so a last-entry
check in plug merging always misses a merge opportunity since we always
find a different disk than what we are looking for.

Modify the logic such that it's still a one-hit cache, but ensure that
we check enough to find the right target before giving up.

Fixes: d38a9c04c0d5 ("block: only check previous entry for plug merge attempt")
Reported-and-tested-by: Song Liu <song@kernel.org>
Reviewed-by: Song Liu <songliubraving@fb.com>
Signed-off-by: Jens Axboe <axboe@kernel.dk>

diff --git a/block/blk-merge.c b/block/blk-merge.c
index f5255991b773ca9ac4415e91a6d73b018fb0749a..8d8177f71ebdb8652a3e681c7be674bc547c0c21 100644 (file)
--- a/block/blk-merge.c
+++ b/block/blk-merge.c
@@ -1087,12 +1087,20 @@ bool blk_attempt_plug_merge(struct request_queue *q, struct bio *bio,
        if (!plug || rq_list_empty(plug->mq_list))
                return false;
 
-       /* check the previously added entry for a quick merge attempt */
-       rq = rq_list_peek(&plug->mq_list);
-       if (rq->q == q) {
-               if (blk_attempt_bio_merge(q, rq, bio, nr_segs, false) ==
-                               BIO_MERGE_OK)
-                       return true;
+       rq_list_for_each(&plug->mq_list, rq) {
+               if (rq->q == q) {
+                       if (blk_attempt_bio_merge(q, rq, bio, nr_segs, false) ==
+                           BIO_MERGE_OK)
+                               return true;
+                       break;
+               }
+
+               /*
+                * Only keep iterating plug list for merges if we have multiple
+                * queues
+                */
+               if (!plug->multiple_queues)
+                       break;
        }
        return false;
 }