- Open Fabrics Enterprise Distribution (OFED)
+ Open Fabrics Enterprise Distribution (OFED)
Version 1.4
- README
-
- July 2008
+ README
+
+ July 2008
This is the OpenFabrics Enterprise Distribution (OFED) version 1.4
software package supporting InfiniBand and iWARP fabrics. It is composed
constructed as an InfiniBand subnet or an iWARP network.
*** Note: If you plan to upgrade OFED on your cluster, please upgrade all
- its nodes to this new version.
+ its nodes to this new version.
This document includes the following sections:
2. OFED Package Contents
3. Installing OFED Software
4. Starting and Verifying the IB Fabric
-5. MPI (Message Passing Interface)
+5. MPI (Message Passing Interface)
6. Related Documentation
OpenFabrics Home Page: http://www.openfabrics.org
The OFED Distribution package generates RPMs for installing the following:
o OpenFabrics core and ULPs
- - HCA drivers (mthca, mlx4, ipath, ehca)
+ - HCA drivers (mthca, mlx4, ipath, ehca)
- iWARP driver (cxgb3, nes)
- core
- - Upper Layer Protocols: IPoIB, SDP, SRP Initiator and target, iSER
+ - Upper Layer Protocols: IPoIB, SDP, SRP Initiator and target, iSER
Initiator and target, RDS, uDAPL, qlgc_vnic and NFS-RDMA.
o OpenFabrics utilities
- OpenSM: InfiniBand Subnet Manager
OFED_Installation_Guide.txt under OFED-1.4/docs.
-Notes:
-1. The install script removes previously installed IB packages and
+Notes:
+1. The install script removes previously installed IB packages and
re-installs from scratch. You will be prompted to acknowledge the deletion
of the old packages. However, configuration files (.conf) will be
- preserved and saved with a ".rpmsave" extension.
+ preserved and saved with a ".rpmsave" extension.
2. After the installer completes, information about the OFED
installation such as the prefix, the kernel version, and
2) Check that the IB driver is running on all nodes: ibv_devinfo should print
"hca_id: <linux device name>" on the first line.
-
+
3) Make sure that a Subnet Manager is running by invoking the sminfo utility.
- If an SM is not running, sminfo prints:
+ If an SM is not running, sminfo prints:
sminfo: iberror: query failed
If an SM is running, sminfo prints the LID and other SM node information.
Example:
- sminfo: sm lid 0x1 sm guid 0x2c9010b7c2ae1, activity count 20 priority 1
+ sminfo: sm lid 0x1 sm guid 0x2c9010b7c2ae1, activity count 20 priority 1
To check if OpenSM is running on the management node, enter: /etc/init.d/opensm status
To start OpenSM, enter: /etc/init.d/opensm start
report a "PORT_ACTIVE" state.
5) Check the network connectivity status: run ibchecknet to see if the subnet
- is "clean" and ready for ULP/application use. The following tools display
- more information in addition to IB info: ibnetdiscover, ibhosts, and
- ibswitches.
+ is "clean" and ready for ULP/application use. The following tools display
+ more information in addition to IB info: ibnetdiscover, ibhosts, and
+ ibswitches.
6) Alternatively, instead of running steps 3 to 5 you can use the ibdiagnet
utility to perform a set of tests on your network. Upon finding an error,
ibdiagnet will print a message starting with a "-E-". For a more complete
report of the network features you should run ibdiagnet -r. If you have a
topology file describing your network you can feed this file to ibdiagnet
- (using the option: -t <file>) and all reports will use the names they
+ (using the option: -t <file>) and all reports will use the names they
appear in the file (instead of LIDs, GUIDs and directed routes).
7) To run an application over SDP set the following variables:
- env LD_PRELOAD='stack_prefix'/lib/libsdp.so
- LIBSDP_CONFIG_FILE='stack_prefix'/etc/libsdp.conf <application name>
+ env LD_PRELOAD='stack_prefix'/lib/libsdp.so
+ LIBSDP_CONFIG_FILE='stack_prefix'/etc/libsdp.conf <application name>
(or LD_PRELOAD='stack_prefix'/lib64/libsdp.so on 64 bit machines)
The default 'stack_prefix' is /usr
In Step 2 of the main menu of install.pl, options 2, 3 and 4 can
install one or more MPI stacks. Multiple MPI stacks can be installed
-simultaneously -- they will not conflict with each other.
+simultaneously -- they will not conflict with each other.
Three MPI stacks are included in this release of OFED:
- MVAPICH 1.1.0
6. Related Documentation
========================
-1) Release Notes for OFED Distribution components are to be found under
- OFED-1.4/docs and, after the package installation, under
+1) Release Notes for OFED Distribution components are to be found under
+ OFED-1.4/docs and, after the package installation, under
/usr/share/doc/ofed-docs-1.4 for RedHat
/usr/share/doc/packages/ofed-docs-1.4 for SuSE.
2) For a detailed installation guide, see OFED_Installation_Guide.txt.
3) For more information, please visit the OFED web-page http://www.openfabrics.org
-For more information contact your vendor.
+For more information contact your vendor.