Fix data corruption triggered by wrong headroom marking order

Fix data corruption triggered by wrong headroom marking order

This is an addendum to commit 561da8d1 ("Handle new FW requirement for
send request prefetching").  We also need to handle prefetch marking
properly for S/G segments, or else the HCA may end up processing S/G
segments that are not fully written and end up sending the wrong data.

We write S/G segments in reverse order into the WQE, in order to
guarantee that the first dword of all cachelines containing S/G
segments is written last (overwriting the headroom invalidation
pattern).  The entire cacheline will thus contain valid data when the
invalidation pattern is overwritten.

Signed-off-by: Jack Morgenstein <jackm@dev.mellanox.co.il>
Signed-off-by: Roland Dreier <rolandd@cisco.com>