Open Fabrics Enterprise Distribution (OFED)
- Performance Tests README for OFED 1.5.1
-
- March 2010
+ Performance Tests README for OFED 1.5
+
+ December 2010
The collection conatains a set of BW and latency benchmark such as :
* Read - ib_read_bw and ib_read_lat.
- * Write - ib_write_bw and ib_wriet_lat.
+ * Write - ib_write_bw, ib_write_bw_postlist and ib_write_lat.
* Send - ib_send_bw and ib_send_lat.
* RDMA - rdma_bw and rdma_lat.
- * In additional : ib_write_bw_postlist and ib_clock_test.
-
-Please post results/observations to the openib-general mailing list.
-See "Contact Us" at http://openib.org/mailman/listinfo/openib-general and
-http://www.openib.org.
+ * Additional benchmark: ib_clock_test.
+Please post results/observations/bugs/remarks to the mailing list specified below:
+ * Maintainer - idos@dev.mellanox.co.il
+ * OFED mailing list - ewg@lists.openfabrics.org
+ or linux-rdma@vger.kernel.org
+ * http://openib.org/mailman/listinfo/openib-general
===============================================================================
2. Notes on Testing Methodology
===============================================================================
-- The benchmark used the CPU cycle counter to get time stamps without context
- switch. Some CPU architectures (e.g., Intel's 80486 or older PPC) do NOT
+The bencmarks specified below are tested of the following architectures:
+- i686
+- x86_64
+- ia64
+
+- The benchmark uses the CPU cycle counter to get time stamps without context
+ switch. Some CPU architectures (e.g., Intel's 80486 or older PPC) do NOT
have such capability.
- The benchmark measures round-trip time but reports half of that as one-way
- latency. This means that it may not be sufficiently accurate for asymmetrical
+ latency. Thus, it may not be sufficiently accurate for asymmetrical
configurations.
-- On Bw benchmarks , We calculate the BW on send side only, as he calculates
- the Bw after collecting completion from the receive side.
- In case we use the bidirectional flag , BW is calculated on both sides
+- On BW benchmarks , the BW is calculated on the send side only, as it calculates
+ the BW after collecting completion from the receive side.
+ If using the bidirectional flag , BW is calculated on both sides
- Min/Median/Max result is reported.
The median (vs average) is less sensitive to extreme scores.
Typically, the "Max" value is the first value measured.
-- Larger samples help marginally only. The default (1000) is pretty good.
+- Larger samples help marginally only. The default (1000) is sufficient.
Note that an array of cycles_t (typically unsigned long) is allocated
once to collect samples and again to store the difference between them.
- Really big sample sizes (e.g., 1 million) might expose other problems
+ Large sample sizes (e.g., 1 million) might expose other problems
with the program.
- The "-H" option will dump the histogram for additional statistical analysis.
See xgraph, ygraph, r-base (http://www.r-project.org/), pspp, or other
statistical math programs.
-Architectures tested: i686, x86_64, ia64
-
-
===============================================================================
4. Test Descriptions
===============================================================================
send_lat.c latency test with send transactions
send_bw.c BW test with send transactions
-write_lat.c latency test with RDMA write transactions
+write_lat.c latency test with RDMA write transactions
write_bw.c BW test with RDMA write transactions
read_lat.c latency test with RDMA read transactions
read_bw.c BW test with RDMA read transactions
The executable name of each test starts with the general prefix "ib_",
-e.g., ib_write_lat , exept of those of RDMA tests , in their case
+e.g., ib_write_lat, except for those of RDMA tests,
their excutable have the same name except of the .c.
Running Tests
-m, --mtu=<mtu> Mtu size (default: 1024).
-d, --ib-dev=<dev> Use IB device <dev> (default: first device found).
-i, --ib-port=<port> Use port <port> of IB device (default: 1).
+ -o, --out=<num_of_out> Number of outstanding reads. only in READ.
+ -q, --qp=<num_of_qps> Number of Qps to perform. only in write_bw.
+ -c, --connection=<c> Connection type : RC,UC,UD according to spec.
+ -g, --mcg=<num_of_qps> Number of Qps in MultiCast group. in SEND only
+ -M, --MGID=<addr> <addr> as the group MGID in format '255:1:X:X:X:X:X:X:X:X:X:X:X:X:X:X'.
-s, --size=<size> Size of message to exchange (default: 1).
-a, --all Run sizes from 2 till 2^23.
-t, --tx-depth=<dep> Size of tx queue (default: 50).
*** IMPORTANT NOTE: You need to be running a Subnet Manager on the switch or
on one of the nodes in your fabric.
-Example:
-Run "ib_rdma_lat -C" on the server side.
-Then run "ib_rdma_lat -C <server IP address>" on the client.
-
-ib_rdma_lat will exit on both server and client after printing results.