rsocket: Fix sbuf_bytes_avail counter 'overrun' with iwarp

rsocket: Fix sbuf_bytes_avail counter 'overrun' with iwarp

Reported-by: Jonas Pfefferle1 <JPF@zurich.ibm.com>
"The problem is that on the client side sbuf_bytes_avail overflows
in rs_poll_cq.  And from what I debugged so far there are 2
completions for every send and this is because I use iWarp hardware
which does not support write with immediate so there is one completion
for the write and one for the send (both go into the default case
and add the length to sbuf_bytes_avail)."

To avoid the issue, we flag send message operations that are used
in place of immediate data.  Other send message operations are
not affected.  The completion code can then check whether the
completion is for a send message which was paired with an RDMA
write transaction and adjust the behavior accordingly.  Additionally,
such send messages only carry the opcode in their WR_ID, with the
data portion zeroed.  This avoids adding the length value twice.

Signed-off-by: Sean Hefty <sean.hefty@intel.com>