rsockets: Reduce the default inline size

Inline data consumes the same space used by the SGL.  Since
we reduced the default number of SGEs per SQ entry to 1,
also reduce the default inline data size to 16 bytes.
Otherwise, the SQ size won't actually be reduced.

Although this increases the latency of small messages over
16 bytes, tests show that decreasing the inline data size
from 64 bytes to 32 or 16 bytes improves large message
bandwidth 8-10%.

Sample rstream 64k_bw test results:

inline size 64: 24.85 Gbps
inline size 16: 26.5  Gbps

Signed-off-by: Sean Hefty <sean.hefty@intel.com>

diff --git a/src/rsocket.c b/src/rsocket.c
index 378afd7f60b3474c6a6972429f44b1945198429c..5eec5d9c6a8f0cfa5f34e3d945a5baec3f49fa3e 100644 (file)
--- a/src/rsocket.c
+++ b/src/rsocket.c
@@ -51,7 +51,6 @@
 #include "cma.h"
 #include "indexer.h"
 
-#define RS_INLINE 64
 #define RS_OLAP_START_SIZE 2048
 #define RS_MAX_TRANSFER 65536
 #define RS_QP_SIZE 384
@@ -223,7 +222,7 @@ static struct rsocket *rs_alloc(struct rsocket *inherited_rs)
                rs->ctrl_avail = inherited_rs->ctrl_avail;
        } else {
                rs->sbuf_size = rs->rbuf_size = RS_BUF_SIZE;
-               rs->sq_inline = RS_INLINE;
+               rs->sq_inline = RS_MIN_INLINE;
                rs->sq_size = rs->rq_size = RS_QP_SIZE;
                rs->ctrl_avail = RS_QP_CTRL_SIZE;
        }