Open Fabrics Enterprise Distribution (OFED)
- PSM in OFED 3.5 Release Notes
+ PSM in OFED 3.12 Release Notes
- December 2012
+ May 2014
======================================================================
1. Overview
The Performance Scaled Messaging (PSM) API is Intel's low-level user-level
communications interface for the Intel(R) True Scale Fabric family of products.
-The PSM libraries are included in the infinipath-psm-3.1-364.1140_open.src.rpm
+The PSM libraries are included in the infinipath-psm-3.2-2_ga8c3e3e_open.src.rpm
source RPM and get built and installed as part of a default OFED
install process.
Open MPI:
- Download a recent Open MPI tar ball from
- http://www.open-mpi.org/software/ompi/v1.6/ .
- Versions 1.6.1 & 1.6.3 has been tested with PSM from this OFED release.
+ http://www.open-mpi.org/software/ompi/v1.8/ .
+ Version 1.8.1 has been tested with PSM from this OFED release.
- Untar the file and cd to the Open MPI directory. Then
./configure --with-psm --prefix=<install directory>
make
- Download a recent MVAPICH2 tar ball from
http://mvapich.cse.ohio-state.edu/download/mvapich2/
- Version 1.8.1 has been tested with PSM from this OFED release.
-- Untar the file and go to the mvapich2-1.x.x directory
+ Version 1.9 has been tested with PSM from this OFED release.
+- Untar the file and go to the mvapich2-1.x directory
- Execute the configure and make commands as follows:
./configure --prefix=<install directory> --with-device=ch3:psm \
--enable-shared
make
make install
+Once these MPIs are built, using mpirun/mpiexec will result in the use
+of PSM by default -- no additional options needed. Also if you have two
+Intel True Scale HCAs per node, both of them will be used by default
+when you run with 2 or more ranks per node. No option is necessary on
+the mpirun/mpiexec command.
+
+If you want to use just one of the HCAs, set the IPATH_UNIT variable to:
+export IPATH_UNIT=0 (or =1)
+If you have two Intel True Scale HCAs per node, and want to get additional
+bandwidth performance even when only one MPI rank is active per node, set:
+export PSM_MULTIRAIL=0
+For most applications, it is better not to use this PSM_MULTIRAIL variable
+since it increases MPI latency somewhat, but it is helpful for a minority
+of applications that require additional bandwidth from single ranks.
* To configure with a different compiler suite than the native GCC suite on your
Open Fabrics Enterprise Distribution (OFED)
- qib in OFED 3.5 Release Notes
+ qib in OFED 3.12 Release Notes
- December 2012
+ May 2014
======================================================================
1. Overview
- QLE7280 (x16 PCIe Gen1)
-The qib driver has been updated for OFED 3.5.
+The qib driver has been updated for OFED 3.12.
The qib kernel driver obsoletes the ipath kernel driver but is
compatible with libipathverbs so no new user level components are needed.