o Linux Operating Systems:
- RedHat EL4 up3: 2.6.9-34.ELsmp
- RedHat EL4 up4: 2.6.9-42.ELsmp
- - RedHat EL4 up5: 2.6.9-42.ELsmp
- - RedHat EL5: 2.6.9-42.ELsmp
+ - RedHat EL4 up5: 2.6.9-55.ELsmp
+ - RedHat EL5: 2.6.18-8.el5
+ - SLES9 SP3: 2.6.5-7.244-smp
- SLES10: 2.6.16.21-0.8-smp
- - kernel.org: 2.6.19.x and 2.6.20.x
+ - kernel.org: 2.6.20.x
+ - SLES10 SP1: 2.6.16.46-0.12-smp (partialy tested)
1.3 HCAs and RNICs Supported
----------------------------
1. Intel MPI, Version 3.0 - Package ID: l_mpi_p_3.0.043
2. HP MPI, Version 2.2.5
- 1.6 OFED Sources:
- -----------------
+ 1.6 OFED Sources
+ ----------------
Source repositories:
-http://www.openfabrics.org/git/
Kernel sources: ~vlad/ofed_1_2/.git
User level Sources are located in all git trees starting with: ofed_1_2/
1. OFED installation now supports installing lib32 on 64-bit systems.
2. Hotplug removal does not hang the system when the device is used by
the uverbs interface.
- 3. MVAPICH does work on ppc64.
+ 3. MVAPICH now works on ppc64.
4. libibcm is now thread safe.
- Bugs fixed in each package are reported in the packages release notes.
+ Bugs fixed in each package are reported in the package's release notes.
+
===============================================================================
-4. Known Issues
+5. Known Issues
===============================================================================
- The following is a list of major limitations and known issues of the various
+ The following is a list of general limitations and known issues of the various
components of the OFED 1.2 release.
- 1. Memory registration by user is limited according to the administrator
+ 1. When upgrading from older OFED versions, the installation script does not
+ stop the old OFED version before uninstalling it.
+ Workaround: Stop the old OFED stack (/etc/init.d/openibd stop) before
+ upgrade to OFED 1.2.
+ 1. Memory registration by theuser is limited according to the administrator
setting. See "Pinning (Locking) User Memory Pages" in OFED_tips.txt for
system configuration.
2. Fork support from kernel 2.6.12 and above is available provided
========================
The OFED Distribution package generates RPMs for installing the following:
++<<<<<<< HEAD:README.txt
+
+ o OpenFabrics core and ULPs:
+ - HCA drivers (mthca, mlx4, ipath, ehca)
+ - core
+ - Upper Layer Protocols: IPoIB, SDP, SRP Initiator, iSER Initiator,
+ and uDAPL
+ o OpenFabrics utilities:
+ - OpenSM: InfiniBand Subnet Manager
+ - Diagnostic tools
+ - Performance tests
+ o MPI:
+ - OSU MPI stack supporting the InfiniBand interface
+ - Open MPI stack supporting the InfiniBand interface
+ - MPI benchmark tests (OSU BW/LAT, Intel MPI Benchmark, Presta)
+ o open-iscsi: open-iscsi initiator with iSER support
+ o Sources of all software modules (under conditions mentioned in the
+ modules' LICENSE files)
++=======
+
+ o OpenFabrics core and ULPs
+ - HCA drivers (mthca, ipath, ehca)
+ - iWARP driver (cxgb3)
+ - core
+ - Upper Layer Protocols: IPoIB, SDP, SRP Initiator, iSER Initiator
+ RDS, VNIC and uDAPL
+ o OpenFabrics utilities
+ - OpenSM: InfiniBand Subnet Manager
+ - Diagnostic tools
+ - Performance tests
+ o MPI
+ - OSU MVAPICH stack supporting the InfiniBand and iWARP interface
+ - Open MPI stack supporting the InfiniBand and iWARP interface
+ - OSU MVAPICH2 stack supporting the InfiniBand and iWARP interface
+ - MPI benchmark tests (OSU BW/LAT, Intel MPI Benchmark, Presta)
+ o Extra packages
+ - open-iscsi: open-iscsi initiator with iSER support
+ - ib-bonding: Bonding driver for IPoIB interface
+ o Sources of all software modules (under conditions mentioned in the
+ modules' LICENSE files)
++>>>>>>> ofed_1_2:README.txt
o Documentation
--- /dev/null
- mlx4 in OFED 1.2.c.8 Release Notes
+ Open Fabrics Enterprise Distribution (OFED)
- 2. Supported FW version
++ mlx4 in OFED 1.2.c-8 Release Notes
+
+ June 2007
+
+
+===============================================================================
+Table of Contents
+===============================================================================
+1. Overview
- mlx4 is the low level driver implementation for ConnectX HCA from Mellanox
- Technologies. Because these adapters can also be used as Ethernet NICs and Fibre
- Channel HBAs, the driver is split into two modules:
++2. Supported Firmware Version
+3. Tested ULPs
+4. Fixed Bugs
+5. Known Issues
+
+===============================================================================
+1. Overview
+===============================================================================
- mlx4_core: Handles low-level things like device initialization and
- processing firmware commands. Also controls resource allocation
- so that the InfiniBand, Ethernet and FC functions can share a
- device without stepping on each other.
-
- mlx4_ib: Handles InfiniBand-specific things; plugs into the InfiniBand midlayer.
++mlx4 is the low level driver implementation for the ConnectX adapters designed
++by Mellanox Technologies. The ConnectX can operate as an InfiniBand adapter,
++as an Ethernet NIC, or as a Fibre Channel HBA. To accommodate the three flavors,
++the driver is split into two modules:
+
- options mlx4_core parameter=<value>
-
- mlx4_core parameters:
- - msi_x attempt to use MSI-X if nonzero supported by FW)
- - debug_level Enable debug tracing if > 0
++- mlx4_core
++ Handles low-level functions like device initialization and firmware
++ commands processing. Also controls resource allocation so that the
++ InfiniBand, Ethernet and FC functions can share a device without
++ interfering with each other.
++- mlx4_ib
++ Handles InfiniBand-specific functions and plugs into the InfiniBand
++ midlayer
+
+mlx4 Available Parameters
+--------------------------
+In order to set mlx4 parameters, add the following line to /etc/modpobe.conf:
++ options mlx4_core parameter=<value>
+
- 2. Supported FW version
++ mlx4_core parameters:
++ - msi_x attempt to use MSI-X if non-zero is supported by firmware
++ - debug_level Enable debug tracing if > 0
+
+
+===============================================================================
- This release must be used with the FW version 2.0.158 or beyond.
- If someone will use older FW the driver load will succeed (since its only load
- the device driver) but ibv_devinfo will have this message:
- # ibv_devinfo
- No IB devices found
-
- And in /var/log/messages you will see this message:
-
- mlx4_core 0000:08:00.0: Installed FW version 2.0.157
- mlx4_core 0000:08:00.0: The Command i/f revision (1) of the installed FW
- mlx4_core 0000:08:00.0: is not supported by the installed driver.
- mlx4_core 0000:08:00.0: The driver supports Command i/f revision 2.
++2. Supported Firmware Version
+===============================================================================
- o The only ULPs that are thoroughly tested are: OpenSM and MPI (MVAPICH and
- Open MPI) and IPoIB UD.
- o SDP was partially tested.
++This release must be used with firmware version 2.0.158 or later. If you use
++older firmware, the driver will load will succeed (as only the device driver is
++loaded), but ibv_devinfo will list the following message:
++ # ibv_devinfo
++ No IB devices found
++Also, /var/log/messages will list a message such as the following:
++ mlx4_core 0000:08:00.0: Installed FW version 2.0.157
++ mlx4_core 0000:08:00.0: The Command i/f revision (1) of the installed FW
++ mlx4_core 0000:08:00.0: is not supported by the installed driver.
++ mlx4_core 0000:08:00.0: The driver supports Command i/f revision 2.
+
+===============================================================================
+3. Tested ULPs
+===============================================================================
- 1. Fix errors in CQ cleanup when associated QPs are closed.
- 2. Prevents the following error: Caller posts a 2-WQE list, with the second WQE
- in the list being an INLINE which is too long.
- 3. Set correctly the RNR_RETRY parameter passed to modify_qp.
- 4. Improved RDMA read latency
++- The following ULPs were thoroughly tested:
++ o OpenSM
++ o MPI (MVAPICH and Open MPI)
++ o IPoIB UD
++- SDP was partially tested
+
+===============================================================================
+4. Fixed Bugs
+===============================================================================
- 1. SRQ is not supported. One must use VIADEV_USE_SRQ=0 when lunching an MPI job
- otherwise the MPI job will hang.
- Examples:
- mpirun -np $NP -rsh -hostfile $HOSTFILE VIADEV_USE_SRQ=0 $TEST_BIN_PATH
- or
- Add the line VIADEV_USE_SRQ=0 to mvapich.conf file
- 2. IPoIB is working in UD mode only; openibd.conf was changed to set the
- default to UD.
- 3. Query QP is not supported
- 4. Fork is not supported
- 5. Resize CQ is not supported
- 6. FMRs are not supported
- 7. ibstat does not present all entries. Use ibv_devinfo instead.
- 8. In order to work with RHEL5 with PPC one need to add the following line to
- the ini file: Under the [HCA] section:
- log2_uar_bar_megabytes = 5
- After burning the new FW need also to reboot the system.
- The reason is that the default page size in RHEL5 on PPC is 64K
++- Fixed errors in CQ cleanup when associated QPs are closed
++- The following error is prevented: Caller posts a 2-WQE list with the second
++ WQE being an INLINE which is too long.
++- The RNR_RETRY parameter is now passed to modify_qp with the correct setting
++- Improved the RDMA read latency
+
+===============================================================================
+5. Known Issues
+===============================================================================
++- SRQ is not supported. Set VIADEV_USE_SRQ=0 when lunching an MPI job to avoid
++ hanging the MPI job. This setting can be provided in the command line or
++ placed in the file mvapich.conf.
++ Example:
++ mpirun -np $NP -rsh -hostfile $HOSTFILE VIADEV_USE_SRQ=0 $TEST_BIN_PATH
++
++- IPoIB works in UD mode only. openibd.conf sets the default IPoIB mode
++ accordingly (to UD).
++
++- The following commands and/or features are not supported:
++ o Query QP
++ o Fork
++ o Resize CQ
++ o FMRs
++
++- ibstat does not present all entries. Use ibv_devinfo instead.
++
++- A PPC machine with RHEL5 has 64kB as the default page size. This must be
++ changed for the driver to work. Perform the following three steps:
++ 1. Add the following line in the firmware configuration (INI) file under the
++ [HCA] section:
++ log2_uar_bar_megabytes = 5
++ 2. Burn a modified firmware image with the changed INI file
++ 3. Reboot the system
++
+
projects / ~ardavis / ofed_docs / .git / commitdiff