From: Arlin Davis Date: Wed, 26 Sep 2018 19:13:49 +0000 (-0700) Subject: OFED 4.17 RC1 release notes updates X-Git-Url: https://openfabrics.org/gitweb/?a=commitdiff_plain;h=b57dc958f15303864e8547ce55ed01a66779b9f6;p=~ardavis%2Fofed_docs%2F.git OFED 4.17 RC1 release notes updates Signed-off-by: Arlin Davis --- diff --git a/OFED_release_notes.txt b/OFED_release_notes.txt index 3835eac..37f9fd5 100644 --- a/OFED_release_notes.txt +++ b/OFED_release_notes.txt @@ -1,12 +1,12 @@ Open Fabrics Enterprise Distribution (OFED) - Version 4.8-2 Release Notes - March 2018 + Version 4.17 RC1 Release Notes + September 2018 =============================================================================== Table of Contents =============================================================================== 1. Overview, which includes: - - OFED Distribution Rev 4.8-1 Contents + - OFED Distribution Rev 4.17 Contents - Supported Platforms and Operating Systems - Supported HCA and RNIC Adapter Cards and Firmware Versions - Tested Switch Platforms @@ -20,7 +20,7 @@ Table of Contents 1. Overview =============================================================================== These are the release notes of OpenFabrics Enterprise Distribution (OFED) -release 4.8-2. The OFED software package is composed of several software modules, +release 4.17. The OFED software package is composed of several software modules, and is intended for use on a computer cluster constructed as an InfiniBand Fabric, an iWARP Network or a RoCE Fabric. @@ -35,28 +35,24 @@ Failing to do so may lead to failing to probe vendor specific infiniband drivers with a mismatch message, or running with inbox drivers instead of the drivers that are part of OFED. -1.1 OFED 4.8-2 Contents +1.1 OFED 4.17 Contents ----------------------- The OFED package contains the following components: - OpenFabrics core and ULPs: - IB HCA drivers (mthca, mlx4, mlx5, qib, ehca) - - iWARP RNIC driver (cxgb3, cxgb4, nes, i40iw, qedr) - - RoCE drivers (mlx4, mlx5, ocrdma, qedr, bnxt_re) + - iWARP RNIC driver (cxgb3, cxgb4, i40iw, qedr) + - RoCE drivers (mlx4, mlx5, qedr, bnxt_re) - IB core - Upper Layer Protocols: - - IPoIB, SRP Initiator, iSER, uDAPL, qlgc_vnic - - NVMe-oF (kernel 4.8 only) - - NFS-RDMA (SLES12.2 RH7.3 only). + - IPoIB, SRP Initiator, iSER, uDAPL + - NVMe-oF - OpenFabrics utilities: - OpenSM (OSM): InfiniBand Subnet Manager - Diagnostic tools - Performance tests - Extra packages: - - infinipath-psm: Performance-Scaled Messaging API, an accelerated - interface to Intel(R) HCAs - libfabric - library that exports interfaces for fabric services to applications - - Sources of all software modules (under conditions mentioned in the modules' - LICENSE files) + - Sources of all software modules (under conditions mentioned in the modules LICENSE files) - Documentation @@ -68,22 +64,17 @@ The OFED package contains the following components: - ppc64 o Linux Operating Systems: - - RedHat EL7.0 3.10.0-123.el7 - - RedHat EL7.1 3.10.0-210.el7 - - RedHat EL7.2 3.10.0-327.el7 - - RedHat EL7.3 3.10.0-514.el7 - RedHat EL7.4 3.10.0-693.el7 - - SLES12 3.12.28-4 - - SLES12.1 3.12.49-11.1 - - SLES12.2 4.4.21-69-default + - RedHat EL7.5 3.10.0-862.el7 - SLES12.3 4.4.73-5-default - - kernel.org 4.8 * + - SLES15 4.12.14-23.1-default + - kernel.org 4.17 * * Minimal QA for these versions. 1.3 HCAs and RNICs Supported ---------------------------- -This release supports IB HCAs by IBM, Intel and Mellanox Technologies, iWARP RNICs by Chelsio Communications and Intel and RoCE adapters by Emulex, IBM, Mellanox, Cavium, and Broadcom. +This release supports IB HCAs by IBM, Intel and Mellanox Technologies, iWARP RNICs by Chelsio Communications and Intel and RoCE adapters by IBM, Mellanox, Cavium, and Broadcom. InfiniBand Adapters o IBM HCAs: @@ -103,6 +94,10 @@ InfiniBand Adapters o Mellanox Technologies HCAs (EDR, FDR, FDR10 Modes are Supported): - ConnectX-4 (Rev 12.12.x and above) + - ConnectX-5 (Rev xx.xx.x and above) + + o Mellanox Technologies HCAs (HDR, EDR, FDR, FDR10 Modes are Supported): + - ConnectX-6 (Rev xx.xx.x and above) For official firmware versions please see: http://www.mellanox.com/content/pages.php?pg=firmware_download @@ -116,14 +111,12 @@ iWARP Adapters - T6: T62100-CR, T62100-LP-CR, T6225-CR o Intel RNICs: - - NE020 10Gb iWARP Adapter - - i40iw 10Gb iWARP Adapter (kernel 4.8 only) + - i40iw 10Gb iWARP Adapter (kernel 4.17 only) -RoCE Adapters - o Emulex - using ocrdma - - Emulex OCe14102 2-port 10 GbE RoCE - - Emulex OCe14401 1-port 40 GbE RoCE + o Cavium + - QL4xxxx series of converged network adapters (iWARP) +RoCE Adapters o IBM - IBM Flex System EN4132 2-port 10 GbE RoCE - IBM EL27 PCIe LP 2-Port 10GbE RoCE SFP+ adapter @@ -132,6 +125,9 @@ RoCE Adapters o Mellanox - ConnectX-2 EN (Rev 2.9.1200 and above) - ConnectX-3 EN (Rev 2.31.5050 and above) + - ConnectX-4 EN (Rev x.xx.xxxx and above) + - ConnectX-5 EN (Rev x.xx.xxxx and above) + - ConnectX-6 EN (Rev x.xx.xxxx and above) o Cavium - QL4xxxx series of converged network adapters (RoCEv2) @@ -142,13 +138,13 @@ RoCE Adapters Other Drivers - VMware Paravirtual RDMA Driver (vmw_pvrdma) - Software RDMA over Ethernet (RoCEv2) - - NVMe-oF Host and Target (kernel 4.8 only) + - NVMe-oF Host and Target 1.4 Switches Supported ---------------------- -This release was tested with switches and gateways provided by the following -companies: +This release was tested with switches and gateways as follow: + InfiniBand Switches o Flextronics - F-X430044 @@ -174,10 +170,9 @@ RoCE Switches 1.5 Third Party Packages ------------------------ -The following third party packages have been tested with OFED 4.8-1: -- Open MPI - 1.8 -- Intel MPI Library 2017 -- MVAPICH2 +The following third party packages have been tested with OFED 4.17: +- Open MPI - 3.x +- Intel MPI Library 2018 1.6 OFED Sources ---------------- @@ -185,11 +180,11 @@ All sources are located under git://git.openfabrics.org/ Linux: ------ -URL: git://git.openfabrics.org/compat-rdma/linux-4.8.git +URL: git://git.openfabrics.org/compat-rdma/linux-4.17.git Branch: master - Linux kernel sub-tree that includes files relevant for the OFED project only. - Based on v4.8. Used to shorten git clone time. + Based on v4.17. Used to shorten git clone time. Note: the regular Linux git tree can be used as well. compat: @@ -221,6 +216,20 @@ http://www.openfabrics.org/downloads/MAINTAINERS =============================================================================== 2. Change log =============================================================================== + +------------------------------------------------------------------------------- +OFED-4.17 Main Changes from OFED-4.8-2 +------------------------------------------------------------------------------- +1. Add support for RHEL7.5 and SLES15 +2. Remove support for RHEL7.0,7.1,7.2,7.3, and SLES12.0,12.1,12.2 + +3. compat-rdma + +4. Updated packages + +5. ofed_scripts + +------------------------------------------------------------------------------- OFED-4.8-2 Main Changes from OFED-4.8-1 ------------------------------------------------------------------------------- 1. compat-rdma diff --git a/OFED_tips.txt b/OFED_tips.txt index 4ca8f7b..4b5f38c 100644 --- a/OFED_tips.txt +++ b/OFED_tips.txt @@ -1,6 +1,6 @@ Open Fabrics Enterprise Distribution (OFED) - Tips for Working with OFED 4.8 - May 2014 + Tips for Working with OFED 4.17 + May 2018 =============================================================================== Table of Contents @@ -8,7 +8,7 @@ Table of Contents 1. OFED Utilities 2. Debug HOWTOs 3. Pinning (Locking) User Memory Pages -4. External Module Compilation Over OFED-4.8 +4. External Module Compilation Over OFED-4.17 5. Adding vendor specific actions to the installation of OFED 6. How to compile OFED sources manually @@ -182,17 +182,17 @@ c. Using sysfs file system ------------------------------------------- The text file BUILD_ID provides data on all OFED components (whether installed or not). This file is a part of the ofed-docs RPM and installed under -/usr/share/doc/ofed-docs-4.8 on RedHat, and under -/usr/share/doc/packages/ofed-docs-4.8 on SuSE. +/usr/share/doc/ofed-docs-4.17 on RedHat, and under +/usr/share/doc/packages/ofed-docs-4.17 on SuSE. The same information can be obtained by executing the 'ofed_info' command. For example: # ofed_info -s -OFED-4.8: +OFED-4.17: # ofed_info -OFED-4.8: +OFED-4.17: compat-rdma: git://git.openfabrics.org/compat-rdma/compat-rdma.git master @@ -200,7 +200,7 @@ commit 71b57cc1149bcb2bd90659e31405b6935b604569 compat-rdma: linux: -git://git.openfabrics.org/compat-rdma/linux-4.8.git master +git://git.openfabrics.org/compat-rdma/linux-4.17.git master commit 4b9e97b1c859c8fe54a596b554215f39d22bc761 compat-rdma: @@ -257,7 +257,7 @@ Note: The file /etc/security/limits.conf contains further documentation. =============================================================================== -4. External Module Compilation Over OFED-4.8 +4. External Module Compilation Over OFED-4.17 =============================================================================== To build kernel modules depending on OFED's modules, take the Modules.symvers @@ -271,7 +271,7 @@ Example: MODULES_DIR=/lib/modules/`uname -r`/updates ./create_Module.symvers.sh See "Module versioning & Module.symvers" in the modules.txt from kernel -documentation (e.g. linux-4.8/Documentation/kbuild/modules.txt). +documentation (e.g. linux-4.17/Documentation/kbuild/modules.txt). Example of the Makefile: @@ -434,11 +434,11 @@ meaning without building the RPMs and without using the install.pl script. 6.1 Compiling the kernel modules -------------------------------- -1. tar xzf OFED-4.8.tgz -2. rpm -ihv OFED-4.8/SRPMS/compat-rdma-4.8-OFED.4.8.src.rpm +1. tar xzf OFED-4.17.tgz +2. rpm -ihv OFED-4.17/SRPMS/compat-rdma-4.17-OFED.4.17.src.rpm 3. cd /usr/src/redhat/SOURCES -4. tar xzvf compat-rdma-4.8.tgz -5. cd compat-rdma-4.8 +4. tar xzvf compat-rdma-4.17.tgz +5. cd compat-rdma-4.17 6. configure: run ./configure --help for a list of options. basic invocation is: @@ -453,12 +453,12 @@ NOTES: 6.2 Compiling the user space libraries -------------------------------------- -To install user space library from the source RPM provided by OFED-4.8 manually, +To install user space library from the source RPM provided by OFED-4.17 manually, do the following: Example for libibverbs: -1. tar xzf OFED-4.8.tgz +1. tar xzf OFED-4.17.tgz 2. rpm -ihv SRPMS/libibverbs-1.1.7-1.src.rpm 3. cd ~/rpmbuild/SOURCES (for RedHat) or diff --git a/README.txt b/README.txt index eac7d59..c2107a9 100644 --- a/README.txt +++ b/README.txt @@ -1,6 +1,6 @@ Open Fabrics Enterprise Distribution (OFED) - Version 3.12 README - May 2014 + Version 4.17 README + May 2018 ============================================================================== Table of contents @@ -22,7 +22,7 @@ Table of contents 1. Overview ============================================================================== -This is the OpenFabrics Enterprise Distribution (OFED) version 3.12 +This is the OpenFabrics Enterprise Distribution (OFED) version 4.17 software package supporting InfiniBand, iWARP and RoCE fabrics. It is composed of several software modules intended for use on a computer cluster constructed as an InfiniBand subnet or an iWARP network. @@ -126,14 +126,6 @@ Notes: See examples of ofed.conf and ofed_net.conf under OFED-X.X.X/docs. Run './install.pl -p' to get ofed.conf with all available packages included. - To enable selection of the technology preview for XEON PHI support, the - parameter --with-xeon-phi must be provided. Both kernel packages ibp_server - and ibscif needs to be selected as well as the user-mode packages ibpd and - libibscif. The interactive installation and --all will include these, - neither --hpc nor --basic will. This requires the linux version of - Intel(R) Manycore Platform Software Stack (Intel(R) MPSS) version 3.4 or - later to be installed and can be downloaded from - http://software.intel.com/mic-developer. Install Process Results: ------------------------ diff --git a/ofed.conf-example b/ofed.conf-example index fba2a75..017e0e0 100644 --- a/ofed.conf-example +++ b/ofed.conf-example @@ -1,68 +1,53 @@ -ofed-scripts=y -core=y -mthca=y -mlx4=y -mlx4_en=y -mlx5=y -cxgb3=y -cxgb4=y -nes=y -qib=y -ocrdma=y -ipoib=y -srp=y -nfsrdma=y compat-rdma=y compat-rdma-devel=y +compat-rdma-firmware=y +rdma-core=y +rdma-core-devel=y libibverbs=y -libibverbs-devel=y -libibverbs-devel-static=y libibverbs-utils=y -libmthca=y -libmthca-devel-static=y -libmlx4=y -libmlx4-devel=y -libmlx5=y -libmlx5-devel=y -libcxgb3=y -libcxgb3-devel=y -libcxgb4=y -libcxgb4-devel=y -libnes=y -libnes-devel-static=y -libocrdma=y -libocrdma-devel=y -libipathverbs=y -libipathverbs-devel=y -libibcm=y -libibcm-devel=y -libibumad=y -libibumad-devel=y -libibumad-static=y -libibmad=y -libibmad-devel=y -libibmad-static=y -ibsim=y -ibacm=y librdmacm=y librdmacm-utils=y -librdmacm-devel=y -opensm-libs=y +libibumad=y +ibacm=y +iwpmd=y +srp_daemon=y opensm=y +opensm-libs=y opensm-devel=y +opensm-debuginfo=y opensm-static=y dapl=y dapl-devel=y dapl-devel-static=y dapl-utils=y +dapl-debuginfo=y perftest=y mstflint=y -srptools=y rds-tools=y rds-devel=y ibutils=y infiniband-diags=y qperf=y +qperf-debuginfo=y ofed-docs=y -infinipath-psm=y -infinipath-psm-devel=y +ofed-scripts=y +libfabric=y +libfabric-devel=y +libfabric-debuginfo=y +fabtests=y +fabtests-debuginfo=y +core=y +mlx4=y +mlx4_en=y +mlx5=y +cxgb3=y +cxgb4=y +qib=y +ipoib=y +nvme_host=y +qedr=y +bnxt_re=y +vmw_pvrdma=y +srp=y +iser=y + diff --git a/release_notes/ipath_release_notes.txt b/release_notes/ipath_release_notes.txt deleted file mode 100644 index 0382fe9..0000000 --- a/release_notes/ipath_release_notes.txt +++ /dev/null @@ -1,13 +0,0 @@ - Open Fabrics Enterprise Distribution (OFED) - ipath in OFED 1.5 Release Notes - - December 2009 - -====================================================================== -1. Overview -====================================================================== -ipath is the low level driver implementation for the -QLogic HyperTransport HCA only (model QHT7140). - -The qib driver is the currently supported driver for all -PCI-Express based Infiniband HCAs. diff --git a/release_notes/nes_release_notes.txt b/release_notes/nes_release_notes.txt deleted file mode 100644 index dfd8ceb..0000000 --- a/release_notes/nes_release_notes.txt +++ /dev/null @@ -1,246 +0,0 @@ - Open Fabrics Enterprise Distribution (OFED) - NetEffect Ethernet Cluster Server Adapter Release Notes - June 2015 - - -The iw_nes module and libnes user library provide RDMA and L2IF -support for the NetEffect Ethernet Cluster Server Adapters. - -============================================= -Supported Architectures and Operating Systems -============================================= - -* CPU architectures: - - x86_64 - - x86 - -* Linux Operating Systems: - - RedHat EL6.5 2.6.32-431.el6 - - RedHat EL6.6 2.6.32-504.el6 - - RedHat EL7.0 3.10.0-123.el7 - - RedHat EL7.1 3.10.0-229.el7 - - SLES11 SP3 3.0.76-0.9.1 - - SLES12 3.12.28-4-default - - kernel.org 3.18 (minimal QA for this version) - -========== -What's New -========== - -OFED 3.18 contains the following new feature for iw_nes driver: - - - iWARP Port Mapper: The iWARP drivers use the port mapper API to request - mappings of tcp ports. The port mapper consists of kernel space functionaliy, - which is part of core/iw_cm and a user space service, which comes - with the libiwpm-1.0.1 package, included in OFED-3.18 - -OFED 3.12 contains the following fixes for iw_nes driver: - - - Fixes for MPA V2 implementaion with respect to the ird/ord peer negotiation - - Fixed an incorrect QP pointer passed to user space after Flush error - - NES_ABI versions have changed for nes driver and libnes (nes user space library) - - Released new libnes-1.1.4 package with updated NES_ABI version - - Added backports for Linux 2.6 and 3.0 kernels - -OFED 3.5 contains several important bug fixes for iw_nes driver: - - - Removed an incorrect macro routine from the loopback code. - - Fixed a couple of problems when unlinking skbs (interoperability with Chelsio). - - Added a lock, so that FPDUs are forwarded in order to iw_nes hardware (interoperability with Chelsio). - - Fixed an incorrect setting of a multicast address in the perfect filter table. - - Fixed a problem when adding timer and a timer is already pending. - - Fixed a problem when initializing terminate timer more than once. - - Fixed a problem when establishing iWarp connection and tx checksum offload is disabled. - - Added backports for Linux 2.6 and 3.0 kernels - - iw_nes driver version is changed to 1.5.0.1 - -======================================== -Required Setting - Power Management Mode -======================================== -If possible, disable Active State Power Management in the BIOS, e.g.: - - PCIe ASPM L0s - Advanced State Power Management: DISABLED - - -======================= -Loadable Module Options -======================= -The following options can be used when loading the iw_nes module by modifying -/etc/modprobe.conf.local file in the SLES hosts and /etc/modprobe.d/rnic.conf file in the RHEL hosts. - -wide_ppm_offset=0 - Set to 1 will increase CX4 interface clock ppm offset to 300ppm. - Default setting 0 is 100ppm. - -mpa_version=1 - MPA version to be used int MPA Req/Resp (1 or 2). - -disable_mpa_crc=0 - Disable checking of MPA CRC. - Set to 1 to enable MPA CRC. - -send_first=0 - Send RDMA Message First on Active Connection. - -nes_drv_opt=0x00000100 - Following options are supported: - - 0x00000010 - Enable MSI - 0x00000080 - No Inline Data - 0x00000100 - Disable Interrupt Moderation - 0x00000200 - Disable Virtual Work Queue - 0x00001000 - Disable extra doorbell read after write - -debug_level=0 - Specify debug output level. - -wqm_quanta=65536 - Set size of data to be transmitted at a time. - -limit_maxrdreqsz=0 - Limit PCI read request size to 256 bytes. - -nes_lro_max_aggr=64 - Max number of LRO packets to be aggregated. - -=============== -Runtime Options -=============== -The following options can be used to alter the behavior of the iw_nes module: -NOTE: Assuming NetEffect Ethernet Cluster Server Adapter is assigned eth2. - - ifconfig eth2 mtu 9000 - largest mtu supported - - ethtool -K eth2 tso on - enables TSO - ethtool -K eth2 tso off - disables TSO - - ethtool -C eth2 rx-usecs-irq 128 - set static interrupt moderation - - ethtool -C eth2 adaptive-rx on - enable dynamic interrupt moderation - ethtool -C eth2 adaptive-rx off - disable dynamic interrupt moderation - ethtool -C eth2 rx-frames-low 16 - low watermark of rx queue for dynamic interrupt moderation - ethtool -C eth2 rx-frames-high 256 - high watermark of rx queue for dynamic interrupt moderation - ethtool -C eth2 rx-usecs-low 40 - smallest interrupt moderation timer for dynamic interrupt moderation - ethtool -C eth2 rx-usecs-high 1000 - largest interrupt moderation timer for dynamic interrupt moderation - -=================== -uDAPL Configuration -=================== -Rest of the document assumes the following uDAPL settings in /etc/dat.conf: - - ofa-v2-iwarp u2.0 nonthreadsafe default libdaplofa.so.2 dapl.2.0 "eth2 0" "" - -======================== -Chelsio Interoperability -======================== -The firmware version supporting interoperability with Chelsio is 3.23 or greater. -The load time Chelsio parameter peer2peer must be set to 1. - -============== -mpd.hosts file -============== -mpd.hosts is a text file with a list of nodes, one per line, in the MPI ring. -Use either fully qualified hostname or IP address. - -=========================== -100% CPU Utilization remark -=========================== -Most of the RDMA applications use CQ Polling mode to decrease latency. -This operational mode can cause 100% CPU utilization. - -To switch to Event Driven mode and lower CPU utilization please refer to README or -Release Notes for specific application. - -======================================== -Recommended Settings for Intel MPI 4.0.x -======================================== -Add the following to mpiexec command: - - -genv I_MPI_FALLBACK_DEVICE 0 - -genv I_MPI_DEVICE rdma:ofa-v2-iwarp - -Example mpiexec command line for uDAPL-2.0: - - mpiexec -genv I_MPI_FALLBACK_DEVICE 0 - -genv I_MPI_DEVICE rdma:ofa-v2-iwarp - -ppn 1 -n 2 - /opt/intel/impi/4.0.3.008/bin64/IMB-MPI1 - -Intel MPI use CQ Polling mode as a default. -To switch to wait mode add the following to mpiexec command: - -genv I_MPI_WAIT_MODE 1 - -NOTE: Wait mode supports the sock device only. - -======================================== -Recommended Setting for MVAPICH2 and OFA -======================================== -From OFED-3.5 package MVAPICH2 is not a part of the OFED - -Download mvapich2-1.8.1 package from following location: - -http://mvapich.cse.ohio-state.edu/download/mvapich2/ - -Example mpirun_rsh command line: - - mpirun_rsh -ssh -np 2 -hostfile /root/mpd.hosts - /usr/mpi/gcc/mvapich2-1.8.1/tests/osu_benchmarks-3.1.1/osu_latency - -MVAPICH2 use CQ Polling mode as a default. -To switch to Blocking mode add the following to mpirun_rsh command: - MV2_USE_BLOCKING=1 - -========================================== -Recommended Setting for MVAPICH2 and uDAPL -========================================== -Add the following to the mpirun_rsh command for 64 or more processes: - - -env MV2_ON_DEMAND_THRESHOLD - -Example mpirun_rsh command with uDAPL-2.0: - - mpirun_rsh -ssh -np 64 -hostfile /root/mpd.hosts - MV2_DAPL_PROVIDER=ofa-v2-iwarp - MV2_ON_DEMAND_THRESHOLD=64 - /usr/mpi/gcc/mvapich2-1.8.1/tests/IMB-3.2/IMB-MPI1 - -MVAPICH2 use CQ Polling mode as a default. -To switch to Blocking mode add the following to mpirun_rsh command: - MV2_USE_BLOCKING=1 - -======================================= -Recommended Settings for Open MPI 1.6.1 -======================================= - -From OFED-3.5 package OpenMPI is not a part of the OFED - -Download openmpi-1.6.1-1 package from following location: - -http://www.open-mpi.org/software/ompi/v1.6/ - -There is more than one way to specify MCA parameters in -Open MPI. Please visit this link and use the best method -for your environment: - -http://www.open-mpi.org/faq/?category=tuning#setting-mca-params - -Example mpirun command line: - - mpirun -np 2 -hostfile /opt/mpd.hosts - -mca btl openib,self,sm - -mca btl_mpi_leave_pinned 0 - /usr/mpi/gcc/openmpi-1.6.1/tests/IMB-3.2/IMB-MPI1 - -OpenMPI use CQ Polling mode as a default. -No command parameter available to switch to Event Driven mode. - - -================================== -Intel Multicast Acceleration (IMA) -================================== - -OFED-3.5 doesn't support Intel Multicast Acceleration (IMA) functionality. - - -NetEffect is a trademark of Intel Corporation in the U.S. and other countries. - diff --git a/release_notes/ocrdma_release_notes.txt b/release_notes/ocrdma_release_notes.txt deleted file mode 100644 index 24f1d10..0000000 --- a/release_notes/ocrdma_release_notes.txt +++ /dev/null @@ -1,347 +0,0 @@ - Open Fabrics Enterprise Distribution (OFED) - - Emulex OCe14000 Adapter RELEASE NOTES - - June 2015 - - - - - -The ocrdma and be2net modules provide RDMA and NIC support for the - -Emulex OCe14000 family of adapters. Make sure you choose the 'ocrdma' - -and 'libocrdma' options while building OFED rpms. - - - -============================================ - -New for OFED-3.18 - -============================================ - -- Bug Fixes in ocrdma and libocrdma packages - -=========================== - -Supported Operating Systems - -=========================== - - o CPU architectures: - - - x86_64 - - - x86 - - o Linux Operating Systems: - - - - RedHat EL6.5 2.6.32-431.el6 - - - RedHat EL6.6 2.6.32-504.el6 - - - RedHat EL7.0 3.10.0-123.el7 - - - RedHat EL7.1 3.10.0-229.el7 - - - SLES11 SP3 3.0.76-0.9.1 - - - SLES12 3.12.28-4-default - - - kernel.org 3.18 * - - - - * Minimal QA for these versions. - -================ - -Supported Cards - -================ - -The OCe14000 family of adapters are supported. - -========================================= - -Priority Flow Control recommendations - -========================================= - -- If the HCA port is connected to a PFC enabled switch, the OCe14000 adapter - is programmed to use priority 5. Configure the priority-groups - and switch ports accordingly. - -- If the HCA port is connected to a PFC enabled switch, it is - highly recommended that you use a VLAN interface for traffic. On a non-VLAN - interface driver would insert VID 0 for all outgoing packets. - -- In back-to-back configurations of OCe14000 series, PFC is enabled by default. - - - -==================== - -Errata (Open Issues) - -==================== - - -- For a 40Gb adapter, ibv_devinfo -v will display: - - active_width: 4X (2) - - active_speed: 10.0 Gbps (4) - - - 4x * 10 = 40G - - link-width is not applicable to RoCE. ibv_devinfo needs a fix. - - -================= - -Updating Firmware - -================= - - -Firmware version 10.2.x or greater includes new features that required new - -flash regions to support them. Firmware versions earlier than 10.0.803.37 - -did not have the ability to configure the flash regions to support these - -new features. - - - -Note 1: Do not use any inbox Linux drivers to update from any version of - - 10.0.x firmware to any version of 10.2.x or greater firmware. - - -Note 2: If you are using out-of-box Linux drivers and you are updating - - from a firmware version earlier than 10.0.803.37, you must first - - update the out-of-box Linux driver to version 10.2.x, and then - - perform the firmware update procedure twice. - - -Note 3: If you are using out-of-box Linux drivers and you are updating - - from firmware version 10.0.803.37 or later, you must first update - - the out-of-box Linux driver to version 10.2.x. It is not necessary - - to perform the firmware update procedure twice. - - - -Note 4: After you have updated the firmware, you must not downgrade it to - - a version earlier than 10.0.803.37. - - - -- Get oc14-10.2.XX.X.ufi or later firmware from www.emulex.com. - -- Copy the firmware file into /lib/firmware - -- Run: ethtool -f ethX - -- Reboot the machine. - - - -The driver in this distribution has been tested with versions 10.6.88.0 and -10.6.103.8 - -eg: - -# cp /oc14-10.6.88.0.ufi /lib/firmware/ - -# ethtool -f eth2 oc14-10.6.88.0.ufi - -# reboot - - -============= - -OpenMPI: - -============= - -From the OFED-3.5 package onwards, OpenMPI is not a part of the OFED - -package. You must manually download and install it. - - - -- Download the latest OpenMPI released SRPM from the following location. - - http://www.open-mpi.org/software/ompi - - - -- Build OpenMPI using the following steps: - - ./configure --with-openib - - make - - make install - - - -- Use the following parameters while running an MPI application: - - --mca btl openib,self,sm --mca pml ob1 --mca btl_openib_flags 306 --mca - - btl_openib_receive_queues P,65536,120,64,32 --mca btl_openib_cpc_include - - rdmacm - - - - eg: - - mpirun -np 8 -hostfile --mca btl openib,self,sm --mca pml ob1 \ - - --mca btl_openib_flags 306 --mca btl_openib_receive_queues P,65536,120,64,32 \ - - --mca btl_openib_cpc_include rdmacm - - - -- For further details refer to the OpenMPI documentation. - - - -============= - -MVAPICH2: - -============= - -From the OFED-3.5 package onwards, MVAPICH2 is not a part of the OFED - -package. You must manually download and install it. - - - -- Download the latest MVAPICH2 released SRPM from the following location: - - http://mvapich.cse.ohio-state.edu/download/mvapich2/ - - - -- Build MVAPICH2 with the following configure options - - ./configure --prefix=/usr --with-device=ch3:mrail --with-rdma=gen2 --sysconfdir=/etc --libdir=/usr/lib64 - - make - - make install - -- Use the following -env options to run the MPI job in the mpirun command: - - "-env MV2_USE_RDMAOE 1 -env MV2_USE_RDMA_CM 1 -env MV2_USE_SHARED_MEM 1 - - -env MV2_USE_BLOCKING 0 -env MV2_IBA_HCA ocrdma0" - - eg: - - mpirun -np 8 -f -env MV2_USE_RDMAOE 1 -env MV2_USE_RDMA_CM 1 \ - - -env MV2_DEFAULT_MAX_CQ_SIZE 128 -env MV2_USE_SRQ 0 -env MV2_MX_SEND_WR 64 \ - - -env MV2_MAX_RECV_WR 64 -env MV2_USE_SHARED_MEM 1 -env MV2_USE_BLOCKING 0 \ - - -env MV2_IBA_HCA ocrdma0 - - - -- For further details refer to the MVAPICH2 documentation. - - - -============================================ - -Testing connectivity with ping and rping: - -============================================ - - - -- modprobe be2net - -- Configure the Ethernet interfaces for the Emulex device. - -- Configure the IP address to the interfaces. - -- modprobe ocrdma - -- ibv_devinfo should display the ocrdma device in an active state. - -- ping should succeed. - - - -Test RDMA connectivity, using the rping command. It is included in the - -librdmacm-utils rpm: - - - -On the server machine: - - - -# rping -s -p 9999 - - - -On the client machine: - - - -# rping -c -VvC10 -a server_ip_addr -p 9999 - - - -You should see ping data like this on the client: - - - -ping data: rdma-ping-0: ABCDEFGHIJKLMNOPQRSTUVWXYZ[\]^_`abcdefghijklmnopqr - -ping data: rdma-ping-1: BCDEFGHIJKLMNOPQRSTUVWXYZ[\]^_`abcdefghijklmnopqrs - -ping data: rdma-ping-2: CDEFGHIJKLMNOPQRSTUVWXYZ[\]^_`abcdefghijklmnopqrst - -ping data: rdma-ping-3: DEFGHIJKLMNOPQRSTUVWXYZ[\]^_`abcdefghijklmnopqrstu - -ping data: rdma-ping-4: EFGHIJKLMNOPQRSTUVWXYZ[\]^_`abcdefghijklmnopqrstuv - -ping data: rdma-ping-5: FGHIJKLMNOPQRSTUVWXYZ[\]^_`abcdefghijklmnopqrstuvw - -ping data: rdma-ping-6: GHIJKLMNOPQRSTUVWXYZ[\]^_`abcdefghijklmnopqrstuvwx - -ping data: rdma-ping-7: HIJKLMNOPQRSTUVWXYZ[\]^_`abcdefghijklmnopqrstuvwxy - -ping data: rdma-ping-8: IJKLMNOPQRSTUVWXYZ[\]^_`abcdefghijklmnopqrstuvwxyz - -ping data: rdma-ping-9: JKLMNOPQRSTUVWXYZ[\]^_`abcdefghijklmnopqrstuvwxyzA - -client DISCONNECT EVENT... - - - - - diff --git a/release_notes/psm_release_notes.txt b/release_notes/psm_release_notes.txt deleted file mode 100644 index 8fb9401..0000000 --- a/release_notes/psm_release_notes.txt +++ /dev/null @@ -1,72 +0,0 @@ - Open Fabrics Enterprise Distribution (OFED) - PSM in OFED 3.12 Release Notes - - May 2014 - -====================================================================== -1. Overview -====================================================================== - -The Performance Scaled Messaging (PSM) API is Intel's low-level user-level -communications interface for the Intel(R) True Scale Fabric family of products. - -The PSM libraries are included in the infinipath-psm-3.2-2_ga8c3e3e_open.src.rpm -source RPM and get built and installed as part of a default OFED -install process. - -The primary way to use PSM is by compiling applications with an MPI that has -been built to use the PSM layer as its interface to Intel HCAs. -PSM is the high-performance interface to the Intel(R) True Scale HCAs. - -Minimal instructions* for building two MPIs tested with OFED -with PSM support are as follows: - - -Open MPI: - -- Download a recent Open MPI tar ball from - http://www.open-mpi.org/software/ompi/v1.8/ . - Version 1.8.1 has been tested with PSM from this OFED release. -- Untar the file and cd to the Open MPI directory. Then - ./configure --with-psm --prefix= - make - make install - -MVAPICH2: - -- Download a recent MVAPICH2 tar ball from - http://mvapich.cse.ohio-state.edu/download/mvapich2/ - Version 1.9 has been tested with PSM from this OFED release. -- Untar the file and go to the mvapich2-1.x directory -- Execute the configure and make commands as follows: - ./configure --prefix= --with-device=ch3:psm \ - --enable-shared - make - make install - -Once these MPIs are built, using mpirun/mpiexec will result in the use -of PSM by default -- no additional options needed. Also if you have two -Intel True Scale HCAs per node, both of them will be used by default -when you run with 2 or more ranks per node. No option is necessary on -the mpirun/mpiexec command. - -If you want to use just one of the HCAs, set the IPATH_UNIT variable to: -export IPATH_UNIT=0 (or =1) -If you have two Intel True Scale HCAs per node, and want to get additional -bandwidth performance even when only one MPI rank is active per node, set: -export PSM_MULTIRAIL=0 -For most applications, it is better not to use this PSM_MULTIRAIL variable -since it increases MPI latency somewhat, but it is helpful for a minority -of applications that require additional bandwidth from single ranks. - - -* To configure with a different compiler suite than the native GCC suite on your - Linux machine, set the configure or script variables: CC, CXX, F77, F90 to be - assigned to the appropriate compiler names for your suite, such as for the - Intel Compiler suite: - CC=icc - CXX=icpc - F77=ifort - F90=ifort - - diff --git a/release_notes/uDAPL_release_notes.txt b/release_notes/uDAPL_release_notes.txt index 20eeea2..a77dbf9 100644 --- a/release_notes/uDAPL_release_notes.txt +++ b/release_notes/uDAPL_release_notes.txt @@ -1,21 +1,18 @@ - README/Release Notes - OFED 3.18-2 DAPL Release 2.1.9-2 - April 2016 + README/Release Notes + OFED 4.17 DAPL Release 2.1.10-1 + November 2018 User space libraries/utilities for Direct Access Transport (DAT) v2.0. DAT is a transport-independent, platform-independent Application Programming Interface that supports RDMA (remote direct memory access) devices. Note: v1.2 is no longer supported and will not be included with OFED releases - MIC support is provided with the new MCM provider and MPXYD service, since dapl-2.1.0. - MCM requires the Intel(R) MPSS 3.x (YOCTO) release for Linux to be installed on your system. - MPSS 3.x for Linux can be downloaded from: http://software.intel.com/mic-developer - For latest documentation and packages: //www.openfabrics.org/downloads/dapl/ ================= 1.0 Release Notes ================= + dapl-2.1.10-1 CCL Proxy optimization for post_send, dtest/dtest_suite improvements, bug fixes, UCM log for timers dapl-2.1.9-2 changes include bug fixes, MFO fixes, GID query attributes dapl-2.1.9-1 changes include dtestcm improvement, bug fixes for MIC, Fix SCM interop issue with MTU