rsockets: Reduce the default inline size

Inline data consumes the same space used by the SGL.  Since
we reduced the default number of SGEs per SQ entry to 1,
also reduce the default inline data size to 16 bytes.
Otherwise, the SQ size won't actually be reduced.

Although this increases the latency of small messages over
16 bytes, tests show that decreasing the inline data size
from 64 bytes to 32 or 16 bytes improves large message
bandwidth 8-10%.

Sample rstream 64k_bw test results:

inline size 64: 24.85 Gbps
inline size 16: 26.5  Gbps

Signed-off-by: Sean Hefty <sean.hefty@intel.com>

diff --git a/src/rsocket.c b/src/rsocket.c
index 529c3b0bfd96bea7a9393e8cd88f982a59fd8c6d..e591116ee5729cede2bec924c4d365911d7369e9 100644 (file)
--- a/src/rsocket.c
+++ b/src/rsocket.c
@@ -51,7 +51,6 @@
 #include "cma.h"
 #include "indexer.h"
 
-#define RS_INLINE 64
 #define RS_OLAP_START_SIZE 2048
 #define RS_MAX_TRANSFER 65536
 #define RS_QP_SIZE 384
@@ -223,7 +222,7 @@ static struct rsocket *rs_alloc(struct rsocket *inherited_rs)
                rs->ctrl_avail = inherited_rs->ctrl_avail;
        } else {
                rs->sbuf_size = rs->rbuf_size = RS_BUF_SIZE;
-               rs->sq_inline = RS_INLINE;
+               rs->sq_inline = RS_MIN_INLINE;
                rs->sq_size = rs->rq_size = RS_QP_SIZE;
                rs->ctrl_avail = RS_QP_CTRL_SIZE;
        }