CM Performance: CPS profile for cma, scm, and ucm v2 uDAPL providers:
-----------------------------------------------------------------------
- Intel SR1600 Servers with Xeon(R) CPU X5570 @ 2.93GHz
- Urbanna Platform - 2 node, 8 cores per node, Mellanox MLX4 IB QDR, no switch.
+ Intel(R) Xeon(R) CPU E5-2690 v2 @ 3.00GHz (IVT)
+ Mellanox MLX4 IB FDR, no switch.
dtestcm (server/client):
- cma: Connections: 183.21 usec, CPS 5458.31 Total 0.18 secs, poll_cnt=3403, Num=1000
- scm: Connections: 178.80 usec, CPS 5592.93 Total 0.18 secs, poll_cnt=2344, Num=1000
- ucm: Connections: 122.43 usec, CPS 8167.93 Total 0.12 secs, poll_cnt=2609, Num=1000
-
+ cma: Connections: 313.10 usec, CPS 3193.83 Total 0.31 secs, poll_cnt=6300, Num=1000
+ scm: Connections: 167.65 usec, CPS 5964.92 Total 0.17 secs, poll_cnt=2394, Num=1000
+ ucm: Connections: 71.85 usec, CPS 13918.06 Total 0.07 secs, poll_cnt=2360, Num=1000
+
dapl_cm_bw: MPI uDAPL/CM profiling application (all-to-all connections, all ranks)
CMA
- 2 Connect times (10): Total 0.0020 per 0.0002 CPS=4997.98
- 4 Connect times (40): Total 0.0077 per 0.0002 CPS=5224.59
- 8 Connect times (240): Total 0.0276 per 0.0001 CPS=8710.76
- 16 Connect times (1120): Total 0.1194 per 0.0001 CPS=9379.37
- 32 Connect times (4800): Total 6.1949 per 0.0013 CPS=774.83
-
+ 2 Connect times (10): Total 0.0049 per 0.0005 CPS=2051.38
+ 4 Connect times (40): Total 0.0151 per 0.0004 CPS=2650.16
+ 8 Connect times (240): Total 0.0548 per 0.0002 CPS=4380.59
+ 16 Connect times (1120): Total 4.0356 per 0.0036 CPS=277.53
+ 32 Connect times (4800): Total 4.4704 per 0.0009 CPS=1073.72
+
SCM
- 2 Connect times (10): Total 0.0024 per 0.0002 CPS=4103.61
- 4 Connect times (40): Total 0.0060 per 0.0002 CPS=6622.41
- 8 Connect times (240): Total 0.0206 per 0.0001 CPS=11634.15
- 16 Connect times (1120): Total 9.0118 per 0.0080 CPS=124.28
- 32 Connect times (4800): Total 21.0198 per 0.0044 CPS=228.36
+ 2 Connect times (10): Total 0.0029 per 0.0003 CPS=3441.31
+ 4 Connect times (40): Total 0.0060 per 0.0002 CPS=6635.97
+ 8 Connect times (240): Total 0.0194 per 0.0001 CPS=12383.47
+ 16 Connect times (1120): Total 0.0649 per 0.0001 CPS=17246.93
+ 32 Connect times (4800): Total 1.0193 per 0.0002 CPS=4708.95
UCM
- 2 Connect times (10): Total 0.0014 per 0.0001 CPS=7353.27
- 4 Connect times (40): Total 0.0045 per 0.0001 CPS=8816.19
- 8 Connect times (240): Total 0.0191 per 0.0001 CPS=12582.44
- 16 Connect times (1120): Total 0.0799 per 0.0001 CPS=14017.68
- 32 Connect times (4800): Total 0.3337 per 0.0001 CPS=14385.21
-
+ 2 Connect times (10): Total 0.0014 per 0.0001 CPS=6993.91
+ 4 Connect times (40): Total 0.0045 per 0.0001 CPS=8837.87
+ 8 Connect times (240): Total 0.0155 per 0.0001 CPS=15477.13
+ 16 Connect times (1120): Total 0.0630 per 0.0001 CPS=17765.12
+ 32 Connect times (4800): Total 0.2632 per 0.0001 CPS=18236.54
BKM for build and running new DAPL library on your cluster without any impact on existing OFED install:
-------------------------------------------------------------------------------------------------------