xio: Configure Accelio internal pool

Temporarily hardcoded all 6 allocators + growing quantum and max size
1k allocator - quantum 4k - max 256k
4k allocator - quantum 4k - max 256k
16k allocator - quantum 4k - max 256k
64k allocator - quantum 1k - max 64k
256k allocator - quantum 512 - max 16k
1m allocator - quantum 128 - max 8k

Later we need to calculate the sustainable workload and dynamically
configure Accelio's interal pool accordingly

Signed-off-by: Vu Pham <vu@mellanox.com>
Signed-off-by: Matt Benjamin <matt@cohortfs.com>

diff --git a/src/msg/xio/XioMessenger.cc b/src/msg/xio/XioMessenger.cc
index de24abfce0e4df652490ebe6ca06260859eccb2a..04fa4b21ad3c58ca6f820eba7cba79c15e89e704 100644 (file)
--- a/src/msg/xio/XioMessenger.cc
+++ b/src/msg/xio/XioMessenger.cc
@@ -303,6 +303,21 @@ XioMessenger::XioMessenger(CephContext *cct, entity_name_t name,
       xio_set_opt(NULL, XIO_OPTLEVEL_ACCELIO, XIO_OPTNAME_MAX_INLINE_HEADER,
                  &xopt, sizeof(xopt));
 
+      struct xio_mempool_config mempool_config = {
+        6,
+        {
+          {1024,  0,  4096,  262144},
+          {4096,  0,  4096,  262144},
+          {16384, 0,  4096,  262144},
+          {65536, 0,  1024,  65536},
+          {262144, 0,  512,  16384},
+          {1048576, 0, 128,  8192}
+        }
+      };
+      xio_set_opt(NULL,
+                  XIO_OPTLEVEL_ACCELIO, XIO_OPTNAME_CONFIG_MEMPOOL,
+                  &mempool_config, sizeof(mempool_config));
+
       /* and unregisterd one */
 #define XMSG_MEMPOOL_QUANTUM 4096