From 54ba4ff4c283d5142f5fc15ef747a751a2f4bbc6 Mon Sep 17 00:00:00 2001 From: Tziporet Koren Date: Sun, 27 May 2007 17:12:05 +0300 Subject: [PATCH] update of some RN --- OFED_release_notes.txt | 86 ++++++++++++++--------------------------- mthca_release_notes.txt | 28 +++++--------- sdp_release_notes.txt | 28 ++++++++++++-- 3 files changed, 63 insertions(+), 79 deletions(-) diff --git a/OFED_release_notes.txt b/OFED_release_notes.txt index 49455e9..11b0fff 100644 --- a/OFED_release_notes.txt +++ b/OFED_release_notes.txt @@ -15,7 +15,7 @@ Table of Contents - Tested Switch Platforms - Third party Test Packages - OFED sources -2. Main Changes from OFED 1.0 +2. Main Changes from OFED 1.1 3. Fixed Bugs 4. Known Issues @@ -23,10 +23,10 @@ Table of Contents =============================================================================== 1. Overview =============================================================================== -These are the release notes of Open Fabrics Enterprise Distribution (OFED) +These are the release notes of OpenFabrics Enterprise Distribution (OFED) release 1.2. The OFED software package is composed of several software modules, and is intended for use on a computer cluster constructed as an InfiniBand -network. +subnet or iWARP network. Note: If you plan to upgrade the OFED package on your cluster, please upgrade all of its nodes to this new version. @@ -36,18 +36,23 @@ all of its nodes to this new version. --------------------- The OFED package contains the following components: o OpenFabrics core and ULPs: - - IB HCA and RNIC drivers (mthca, ipath, ehca, cxgb3) + - IB HCA drivers (mthca, ipath, ehca) + - iWARP RNIC driver (cxgb3) - core - - Upper Layer Protocols: IPoIB, SDP, SRP Initiator, iSER Host, uDAPL - and VNIC. + - Upper Layer Protocols: IPoIB, SDP, SRP Initiator, iSER Host, RDS, + uDAPL and VNIC. o OpenFabrics utilities: - OpenSM (OSM): InfiniBand Subnet Manager - Diagnostic tools - Performance tests o MPI: - - OSU MPI stack supporting the InfiniBand interface - - Open MPI stack supporting the InfiniBand interface + - OSU MPI stack supporting the InfiniBand and iWARP interface + - Open MPI stack supporting the InfiniBand and iWARP interface + - OSU MVAPICH2 stack supporting the InfiniBand and iWARP interface - MPI benchmark tests (OSU benchmarks, Intel MPI benchmarks, Presta) + o Extra packages: + - open-iscsi: open-iscsi initiator with iSER support + - ib-bonding: Bonding driver for IPoIB interface o Sources of all software modules (under conditions mentioned in the modules' LICENSE files) o Documentation @@ -117,13 +122,15 @@ companies: 1.5 Third Party Packages ------------------------ The following third party packages have been tested with OFED 1.2: -1. Intel MPI, Version 2.0.1 - refresh, and Version 3.0 +1. Intel MPI, Version 3.0 - Package ID: l_mpi_p_3.0.043 +2. HP MPIi, Version 2.2.5 1.6 OFED Sources: ----------------- Source repositories: -Kernel: git://www.mellanox.co.il/~git/infiniband ofed_1_1 -User: https://openib.org/svn/gen2/branches/1.1/src/userspace +http://www.openfabrics.org/git/ +Kernel sources: ~vlad/ofed_1_2/.git +User level Sourcesi are located in all git trees starting with: ofed_1_2/ The kernel sources are based on Linux 2.6.20 mainline kernel. Its patches are included in the OFED sources directory. @@ -131,32 +138,19 @@ For details see HOWTO.build_ofed. =============================================================================== -2. Main Changes from OFED 1.0 +2. Main Changes from OFED 1.1 =============================================================================== Note: For details regarding the various changes, please see the release notes for each package in the docs directory. 2.1 General changes: - o Kernel code based on 2.6.18 - o HCA fatal - kernel flow support - o High Availability in IPoIB and SRP - o RDS was removed for the OFED package - o IBM low level driver (ehca) was added + o Kernel code based on 2.6.20 2.1 IPoIB: - o High Availability support using a user-level daemon (beta quality) 2.2 SDP: - o Beta quality (higher stability) - o Improved latency - o Implemented the Naggle algorithm - o Supports sending/receiving out of band data - o Interoperability with previous SDP implementation 2.3 SRP: - o GA quality - o DM (Device Mapper) - for high availability (beta quality). - o New srp_daemon was added 2.4 iSER: o Testing more platforms (e.g., ppc64 and ia64) @@ -166,48 +160,30 @@ for each package in the docs directory. 2.6 MPI: a. OSU MVAPICH: - o Version was changed to 0.9.7-mlx2.2.0 - o Message coalescing + o Version was changed to 0.9.9 b. Open MPI: o Version was updated to v1.1.1 o Bug fixes and general enhancements over v1.1 o See http://www.open-mpi.org/svn/new.php for details - c. MPI tests: - o Updated the tests to latest versions from LLNL, Intel, OSU + c. OSU MVAPICH2: + o MVAPICH2 version 0.98 was added to the OFED package. 2.7 OSM: - o Partition Manager (Pkey) - o Pre-computed routing load from file - o Primitive QoS - as technology preview 2.8 Management: - o Added Madeye utility - o Added saquery tool - o Enhanced ibnetdiscover tool with grouping function - o New ibutils package: - o Port error counter check - o Port performance counters dump - o Link width and Link speed check by flag 2.9 Install: - o Create both 32-bit and 64-bit user-level libraries on x86_64 and - ppc64 platforms - o OSM RPM was separated into several RPMs to enable installing - diagnostic tools without the opensm executable. - o The package kernel include files are placed under /src/openib. - These includes should be used when building kernel modules which use - the Infiniband stack. (Note that these includes, if needed, have - been "backported" to your kernel). - o The package raw (unbackported) source files are placed under - /src/openib-1.2. =============================================================================== 3. Fixed Bugs =============================================================================== 1. OFED installation now supports installing lib32 on 64-bit systems. -4. Hotplug removal does not hang the system when the device is used by +2. Hotplug removal does not hang the system when the device is used by the uverbs interface. -4. MVAPICH does work on ppc64. +3. MVAPICH does work on ppc64. +4. libibcm is not thread safe: if several threads use libibcm, the function + ib_cm_get_device will give the same device to all of the threads, which + can cause thread X to get events that were sent to thread Y. Bugs fixed in each package are reported in the packages release notes. @@ -226,9 +202,7 @@ components of the OFED 1.2 release. The former can be achieved by calling wait(childpid) the later can be achieved by application specific means. Posix system() call is supported. -4. libibcm is not thread safe: if several threads use libibcm, the function - ib_cm_get_device will give the same device to all of the threads, which - can cause thread X to get events that were sent to thread Y. -6. ipath driver is supported only on 64 bit platforms. +3. ipath driver is supported only on 64 bit platforms. +4. There are issues using Intel MPI with Qlogic card driver that cause failures Note: See the release notes of each component for additional issues. diff --git a/mthca_release_notes.txt b/mthca_release_notes.txt index 0ce1516..f1c5892 100644 --- a/mthca_release_notes.txt +++ b/mthca_release_notes.txt @@ -15,7 +15,8 @@ Table of Contents =============================================================================== 1. Overview =============================================================================== -mthca is the low level driver implementation for all Mellanox Technologies HCAs. +mthca is the low level driver implementation for Mellanox Technologies HCAs: +InfiniHost, InfiniHost III Ex and InfiniHost III Lx. mthca Available Parameters -------------------------- @@ -36,29 +37,18 @@ mthca parameters: =============================================================================== 2. New Features =============================================================================== -1. Catastrophic event reset: catastrophic event handling has been expanded - to include resetting the device. After generating the IB_EVENT_DEVICE_FATAL - async event, mthca now resets the device (assuming that the - catas_reset_disable module parameter described above is zero). - - Note that the reset entails removing then adding the device. For the device - to complete the reset, all user-level applications using device resources - directly via the user verbs layer must release those resources. Thus, such - applications should register to receive async events, should detect the - IBV_EVENT_DEVICE_FATAL event, and should release all resources for that - device upon receiving such an event. =============================================================================== 3. Fixed Bugs =============================================================================== -1. mthca no longer misses restoring the following PCI-X/PCI Express - registers after reset: - o PCI-X device: PCI-X command register - o PCI-X bridge: upstream and downstream split transaction registers - o PCI Express: PCI Express device control and link control registers -2. Fence bit is now supported properly. -3. Fixed modify_qp, modify_srq and resize_cq methods to be fully reentrant. +1. Allocating many MRs (which uses many memory pages) causes kernel Oops. (490) +2. Improve scheduling of kernel QPs in mthca. (542) +3. Wrong mthca errors (e.g. CQ entry for unknown QP bd0406). (394) +4. Kernel panic on boot when using Hide-DDR FW. (544) +5. Off-by-one in FMR handling on MemFree can cause memory corruption. (424) +6. InfiniHost III Lx: Data corruption when working with FMR (Influence on the + ULPs: iSER, SRP and Lustre). =============================================================================== 4. Known Issues diff --git a/sdp_release_notes.txt b/sdp_release_notes.txt index e7ee83f..de0015f 100644 --- a/sdp_release_notes.txt +++ b/sdp_release_notes.txt @@ -26,9 +26,22 @@ SDP in OFED is at GA level for OFED 1.2. * add proc fs support. * OFED is shipped with patched version of netstat (sdpnetstat). + SDP info is printed by default. To view only SDP sockets use the "-S" flag. + Info given from sdpnetstat (for each SDP socket): + Recv-Q + Send-Q + Local Address + Foreign Address + User + Inode + PID/Program name * Allow user, via module parameter, to bound SDP's memory use +* Implement slow start on recv buffers. The socket is created with small buffers + and when needed the socket requests the other side so resize it's buffers to + allow for more data. + =============================================================================== 3. Bug Fixes =============================================================================== @@ -96,6 +109,10 @@ SDP in OFED is at GA level for OFED 1.2. - Connection rate in lower than TCP. This may cause clients to get connection refused when server is under connection stress. +- Due to limited dma pool on PPC, sdp cannot sustain more than about 1000 sockets + with default parameters. Limiting max_large_sockets and rcvbuf_scale allows + more sockets to be opened but at a performance cost. + =============================================================================== 5. Supported Socket Options @@ -145,7 +162,7 @@ IPPROTO_IP: IP_MTU IP_ROUTER_ALERT NA IPPROTO_ICMPV6: - ICMP6_FILTER + ICMP6_FILTER NA IPPROTO_IPV6: IPV6_ADDRFORM NA IPV6_CHECKSUM NA @@ -169,7 +186,7 @@ IPPROTO_TCP: TCP_KEEPCNT TCP_KEEPIDLE TCP_KEEPINTVL - TCP_LINGER2 V + TCP_LINGER2 TCP_QUICKACK TCP_SYNCNT TCP_WINDOW_CLAMP @@ -257,11 +274,14 @@ or visible bugs were also fixed. * in "both" mode, only close the SDP socket in case the TCP connection succeeds (bugzilla 576) +* getsockname returns invalid address for sockets bind to ipv4 address + encapsulated in ipv6 (bugzilla 613) + =============================================================================== 4. Known Issues =============================================================================== -* When using server to listen on both SDP and TCP, the number of sockets is - doubled. +* When using server to listen on both SDP and TCP, the number of sockets and + file descriptors is doubled. * A rare race still exists when performing bind/listen on ANY_PORT. The race is between applications and has been greatly minimized. A test to reproduce it -- 2.46.0