xio: better way to assign connections to specific lane

Better way to assign connections to a specific lane of a portal
Avoiding lane competition/hogging.
This change resolves the slow ramping up and spiky behaviors during
clients starting/running I/Os.

Signed-off-by: Vu Pham <vu@mellanox.com>

diff --git a/src/msg/xio/XioPortal.h b/src/msg/xio/XioPortal.h
index 9d98548038c2d991b75247b345ee1c0f008250fa..090f451dfa81c8f44b0b671e25cf55d00d7edbda 100644 (file)
--- a/src/msg/xio/XioPortal.h
+++ b/src/msg/xio/XioPortal.h
@@ -67,7 +67,7 @@ private:
 
     inline Lane* get_lane(XioConnection *xcon)
       {
-       return &qlane[((uint64_t) xcon) % nlanes];
+       return &qlane[(((uint64_t) xcon) / 16) % nlanes];
       }
 
     void enq(XioConnection *xcon, XioSubmit* xs)