dapltest: Add final send/recv "sync" for transaction tests.

dapltest: Add final send/recv "sync" for transaction tests.

The transaction tests need both sides to send a sync message after running the test.  This ensures that all remote operations are complete before dapltest deregeisters memory and disconnects the endpoints.

Without this logic, we see intermittent async errors on iwarp devices because a read response or write arrives after the rmr has been destroyed.
I believe this is more likely to happen with iWARP than IB because iWARP completions only indicate the local buffer can be reused.  It doesn't imply that the message has even arrived at the peer, let alone been placed in the peer application's memory.

Changes from V1:

- allocate new send/recv buffers for the Final Sync message.

- post the Final Sync recv buffer at the beginning of the final iteration of a test.

- tests ok on cxgb4 and mlx4 devices.

Signed-off-by: Steve Wise <swise@opengridcomputing.com>