From 7f869f2ce31f89f488b75fe1e1084cbe92686476 Mon Sep 17 00:00:00 2001 From: Vladimir Sokolovsky Date: Tue, 23 Jun 2015 14:24:41 +0300 Subject: [PATCH] Added ocrdma_release_notes Signed-off-by: Vladimir Sokolovsky --- release_notes/ocrdma_release_notes.txt | 345 +++++++++++++++++++++++++ 1 file changed, 345 insertions(+) create mode 100644 release_notes/ocrdma_release_notes.txt diff --git a/release_notes/ocrdma_release_notes.txt b/release_notes/ocrdma_release_notes.txt new file mode 100644 index 0000000..1803683 --- /dev/null +++ b/release_notes/ocrdma_release_notes.txt @@ -0,0 +1,345 @@ + Open Fabrics Enterprise Distribution (OFED) + + Emulex OCe14000 Adapter RELEASE NOTES + + June 2015 + + + + + +The ocrdma and be2net modules provide RDMA and NIC support for the + +Emulex OCe14000 family of adapters. Make sure you choose the 'ocrdma' + +and 'libocrdma' options while building OFED rpms. + + + +============================================ + +New for OFED-3.18 + +============================================ + +- Bug Fixes in ocrdma and libocrdma packages + +=========================== + +Supported Operating Systems + +=========================== + + o CPU architectures: + + - x86_64 + + - x86 + + o Linux Operating Systems: + + + - RedHat EL6.5 2.6.32-431.el6 + + - RedHat EL 6.6 2.6.32-504.el6 + + - RedHat EL 7.0 3.10.0-229.el7 + + - SLES11 SP3 3.0.76-0.9.1 + + - SLES12 3.12.28-4-default + + - kernel.org 3.18 * + + + + * Minimal QA for these versions. + +================ + +Supported Cards + +================ + +The OCe14000 family of adapters are supported. + +========================================= + +Priority Flow Control recommendations + +========================================= + +- If the HCA port is connected to a PFC enabled switch, the OCe14000 adapter + is programmed to use priority 5. Configure the priority-groups + and switch ports accordingly. + +- If the HCA port is connected to a PFC enabled switch, it is + highly recommended that you use a VLAN interface for traffic. On a non-VLAN + interface driver would insert VID 0 for all outgoing packets. + +- In back-to-back configurations of OCe14000 series, PFC is enabled by default. + + + +==================== + +Errata (Open Issues) + +==================== + + +- For a 40Gb adapter, ibv_devinfo -v will display: + + active_width: 4X (2) + + active_speed: 10.0 Gbps (4) + + + 4x * 10 = 40G + + link-width is not applicable to RoCE. ibv_devinfo needs a fix. + + +================= + +Updating Firmware + +================= + + +Firmware version 10.2.x or greater includes new features that required new + +flash regions to support them. Firmware versions earlier than 10.0.803.37 + +did not have the ability to configure the flash regions to support these + +new features. + + + +Note 1: Do not use any inbox Linux drivers to update from any version of + + 10.0.x firmware to any version of 10.2.x or greater firmware. + + +Note 2: If you are using out-of-box Linux drivers and you are updating + + from a firmware version earlier than 10.0.803.37, you must first + + update the out-of-box Linux driver to version 10.2.x, and then + + perform the firmware update procedure twice. + + +Note 3: If you are using out-of-box Linux drivers and you are updating + + from firmware version 10.0.803.37 or later, you must first update + + the out-of-box Linux driver to version 10.2.x. It is not necessary + + to perform the firmware update procedure twice. + + + +Note 4: After you have updated the firmware, you must not downgrade it to + + a version earlier than 10.0.803.37. + + + +- Get oc14-10.2.XX.X.ufi or later firmware from www.emulex.com. + +- Copy the firmware file into /lib/firmware + +- Run: ethtool -f ethX + +- Reboot the machine. + + + +The driver in this distribution has been tested with versions 10.6.88.0 and +10.6.103.8 + +eg: + +# cp /oc14-10.6.88.0.ufi /lib/firmware/ + +# ethtool -f eth2 oc14-10.6.88.0.ufi + +# reboot + + +============= + +OpenMPI: + +============= + +From the OFED-3.5 package onwards, OpenMPI is not a part of the OFED + +package. You must manually download and install it. + + + +- Download the latest OpenMPI released SRPM from the following location. + + http://www.open-mpi.org/software/ompi + + + +- Build OpenMPI using the following steps: + + ./configure --with-openib + + make + + make install + + + +- Use the following parameters while running an MPI application: + + --mca btl openib,self,sm --mca pml ob1 --mca btl_openib_flags 306 --mca + + btl_openib_receive_queues P,65536,120,64,32 --mca btl_openib_cpc_include + + rdmacm + + + + eg: + + mpirun -np 8 -hostfile --mca btl openib,self,sm --mca pml ob1 \ + + --mca btl_openib_flags 306 --mca btl_openib_receive_queues P,65536,120,64,32 \ + + --mca btl_openib_cpc_include rdmacm + + + +- For further details refer to the OpenMPI documentation. + + + +============= + +MVAPICH2: + +============= + +From the OFED-3.5 package onwards, MVAPICH2 is not a part of the OFED + +package. You must manually download and install it. + + + +- Download the latest MVAPICH2 released SRPM from the following location: + + http://mvapich.cse.ohio-state.edu/download/mvapich2/ + + + +- Build MVAPICH2 with the following configure options + + ./configure --prefix=/usr --with-device=ch3:mrail --with-rdma=gen2 --sysconfdir=/etc --libdir=/usr/lib64 + + make + + make install + +- Use the following -env options to run the MPI job in the mpirun command: + + "-env MV2_USE_RDMAOE 1 -env MV2_USE_RDMA_CM 1 -env MV2_USE_SHARED_MEM 1 + + -env MV2_USE_BLOCKING 0 -env MV2_IBA_HCA ocrdma0" + + eg: + + mpirun -np 8 -f -env MV2_USE_RDMAOE 1 -env MV2_USE_RDMA_CM 1 \ + + -env MV2_DEFAULT_MAX_CQ_SIZE 128 -env MV2_USE_SRQ 0 -env MV2_MX_SEND_WR 64 \ + + -env MV2_MAX_RECV_WR 64 -env MV2_USE_SHARED_MEM 1 -env MV2_USE_BLOCKING 0 \ + + -env MV2_IBA_HCA ocrdma0 + + + +- For further details refer to the MVAPICH2 documentation. + + + +============================================ + +Testing connectivity with ping and rping: + +============================================ + + + +- modprobe be2net + +- Configure the Ethernet interfaces for the Emulex device. + +- Configure the IP address to the interfaces. + +- modprobe ocrdma + +- ibv_devinfo should display the ocrdma device in an active state. + +- ping should succeed. + + + +Test RDMA connectivity, using the rping command. It is included in the + +librdmacm-utils rpm: + + + +On the server machine: + + + +# rping -s -p 9999 + + + +On the client machine: + + + +# rping -c -VvC10 -a server_ip_addr -p 9999 + + + +You should see ping data like this on the client: + + + +ping data: rdma-ping-0: ABCDEFGHIJKLMNOPQRSTUVWXYZ[\]^_`abcdefghijklmnopqr + +ping data: rdma-ping-1: BCDEFGHIJKLMNOPQRSTUVWXYZ[\]^_`abcdefghijklmnopqrs + +ping data: rdma-ping-2: CDEFGHIJKLMNOPQRSTUVWXYZ[\]^_`abcdefghijklmnopqrst + +ping data: rdma-ping-3: DEFGHIJKLMNOPQRSTUVWXYZ[\]^_`abcdefghijklmnopqrstu + +ping data: rdma-ping-4: EFGHIJKLMNOPQRSTUVWXYZ[\]^_`abcdefghijklmnopqrstuv + +ping data: rdma-ping-5: FGHIJKLMNOPQRSTUVWXYZ[\]^_`abcdefghijklmnopqrstuvw + +ping data: rdma-ping-6: GHIJKLMNOPQRSTUVWXYZ[\]^_`abcdefghijklmnopqrstuvwx + +ping data: rdma-ping-7: HIJKLMNOPQRSTUVWXYZ[\]^_`abcdefghijklmnopqrstuvwxy + +ping data: rdma-ping-8: IJKLMNOPQRSTUVWXYZ[\]^_`abcdefghijklmnopqrstuvwxyz + +ping data: rdma-ping-9: JKLMNOPQRSTUVWXYZ[\]^_`abcdefghijklmnopqrstuvwxyzA + +client DISCONNECT EVENT... + + + + + -- 2.46.0