From 777f9f835bb55452ed8a2fa40aa744f50e0751a2 Mon Sep 17 00:00:00 2001
From: Stan Smith
-The Windows OpenFabrics (WinOF) package is composed of software modules intended
+The OpenFabrics Enterprise Distribution for Windows package is composed of software modules intended
for use on Microsoft Windows based computer systems connected via an InfiniBand
fabric. The Windows OpenFabrics software package contains the
+ The OpenFabrics
+Enterprise Distribution for Windows software package contains the
following: Release Notes
The OpenFabrics Alliance Windows release contains a set of
- user mode tools which are designed to faciliate the smooth operation of an
- Windows OpenFabrics installation. These tools are available from a command
+ The OpenFabrics Alliance Enterprise for Windows release contains a set of
+ user mode tools which are designed to faciliate the smooth operation of an OpenFabrics
+ Enterprise Distribution installation. These tools are available from a command
window (cmd.exe) as the installation path '%SystemDrive%\Program
- Files\WinOF' is appended to the system wide search path registry entry.
- A start menu short-cut 'WinOF Cmd Window' is provided to faciliate
+ Files\OFED' is appended to the system wide search path registry entry.
+ A start menu short-cut 'OFED Cmd Window' is provided to faciliate
correction tool operation. vstat Display HCA attributes (lids), statistics and error counters. ibaddr
Query InfiniBand address(es)
-Windows OpenFabrics
+OpenFabrics Enterprise Distribution (for Windows)
User's Manual
-Release 2.2
+Release 2.3
-12/29/2009
+03/10/2010
Overview
OpenFabrics Infiniband core drivers and Upper Level Protocols (ULPs):WinOF Features
+OFED Features
Tools & Diagnostics
-
IPoIB Partition Management
@@ -167,11 +168,6 @@ TCP performance measurements
an Internet Protocol address associated with an IB GUID.
-
- OFED
- Diagnostics
-
COMMON OPTIONS
-Most WinOF diagnostics take the following common flags. The exact list
+Most OFED diagnostics take the following common flags. The exact list
of supported flags per utility can be found in the usage message and
can be shown using the util_name -h syntax.
@@ -1441,7 +1437,7 @@ simple format; don
--node-name-map <node-name-map>
Specify a node name map. The node name map file maps GUIDs to
-more user friendly names. See ibnetdiscover(8) for node name
+more user friendly names. See ibnetdiscover(8) for node name
map file format.
@@ -2429,7 +2425,7 @@ and thus make an Infiniband subnet useable. For most cases, InfiniBand
Subnet Management as a Windows service is sufficient to correctly configure most
InfiniBand fabrics.
The Infiniband subnet management process (opensm) may exist on a
-Windows (WinOF) node or a Linux (OFED) node.
+Windows (OFED) node or a Linux (OFED) node.
Limit the number of OpenSM processes per IB fabric; one SM is sufficient
although redundant SMs are supported. You do not need a Subnet Manager per
@@ -2509,9 +2505,7 @@ Management service.
opensm is an InfiniBand compliant Subnet Manager and Administration, -and runs on top of WinOF. - -opensm provides an implementation of an InfiniBand Subnet Manager and +and runs on top of OFED for Windows. opensm provides an implementation of an InfiniBand Subnet Manager and Administration. Such a software entity is required to run for in order to initialize the InfiniBand hardware (at least one per each InfiniBand subnet). @@ -2545,7 +2539,7 @@ setup the subnet correctly.
The default name of OpenSM partitions configuration file is -%ProgramFiles\WinOF\OpenSM\partitions.conf. The default may be changed +%ProgramFiles\OFED\OpenSM\partitions.conf. The default may be changed by using the --Pconfig (-P) option with OpenSM. The default partition will be created by OpenSM unconditionally even @@ -3501,25 +3495,25 @@ Both or one of options -U and -M can be specified together with '-R file'.
WinOF WSD is not supported in the Windows XP environment.
+OFED WSD is not supported in the Windows XP environment.
The WSD Protocol seamlessly transports TCP data using Infiniband data packets in 'buffered' mode or Infiniband RDMA in 'direct' mode. Either way the user mode socket application sees no @@ -4008,7 +4002,7 @@ latest WSD status.
WSD service
is automatically installed and started as part of the 'default' installation;
except on XP systems - WSD not supported.
-Manual control is performed via the \Program Files\WinOF\installsp.exe utility.
usage: installsp [-i | -r | -l]
-
@@ -4066,11 +4060,11 @@ order provide a set of transport-independent, platform-independent Application Programming Interfaces that exploit the RDMA (remote direct memory access) capabilities of next-generation interconnect technologies such as InfiniBand, and iWARP.WinOF uDAT and uDAPL are based on the +
+ (e.g., "C:\\Programs Files\\OFED\\dapl.dll").OFED uDAT and uDAPL are based on the 2.0 DAT specification. The DAPL (Direct Access Provider Library) which now fully supports Infiniband RDMA and IPoIB.
-Previous WinOF +
Previous OFED releases supported the uDAT/uDAPL 1.1 provider which has now been deprecated.
uDAT/uDAPL version 2.0 runtime libraries along with an optional v2.0 application build environment are the only options.
@@ -4158,18 +4152,18 @@ includeDAT/DAPL 2.0 (free-build) libraries are identified in %SystemRoot% as dat2.dll and dapl2.dll. Debug versions of the v2.0 runtime libraries - are located in '%SystemDrive%\%ProgramFiles%\WinOF'.
+ are located in '%SystemDrive%\%ProgramFiles%\OFED'.IA32 (aka, 32-bit) versions of DAT/DAPL 2.0 runtime libraries, found only on 64-bit systems, - are identified in '%ProgramFiles%\WinOF' as dat32.dll + are identified in '%ProgramFiles%\OFED' as dat32.dll and dapl32.dll.
In order for DAT/DAPL programs to execute correctly, the runtime library files 'dat2.dll and dapl2.dll' must be present in one of the following folders: current directory, %SystemRoot% or in the library search path.
-The default WinOF +
The default OFED installation places the runtime library files dat2.dll and dapl2.dll in the '%SystemRoot%' folder; - symbol files (.pdb) are located in '%ProgramFiles%\WinOF\'.
+ symbol files (.pdb) are located in '%ProgramFiles%\OFED\'.The default DAPL configuration file is defined as '%SystemDrive%\DAT\dat.conf'. This default specification can be overriden by use of the environment variable @@ -4186,9 +4180,9 @@ include
the '\' (backslash character) is treated as an 'escape' character. Hence all backslashes in the library path must be duplicated when enclosed in double-quotes - (e.g., "C:\\Programs Files\\WinOF\\dapl.dll").A sample InfiniBand dat.conf file is - installed as '\Program Files\WinOF\dat.conf'. If dat.conf does not + installed as '\Program Files\OFED\dat.conf'. If dat.conf does not exist in the DAT default configuration folder '%SystemDrive%\DAT\', dat.conf will be copied there.
@@ -4276,7 +4270,7 @@ include assigned to an IPoIB device instance. The 'rdma_dev0' is used to instruct the rdma-cm provider to use the 1st RDMA device (IPoIB) with an assigned IP4v address such that the dat.conf file does not - have to be specifically tailored for each WinOF installation. + have to be specifically tailored for each OFED installation.
OFED RDMA CM manager can be used to establish connections between Windows and/or Linux systems.
@@ -4706,7 +4700,7 @@ BUGS (and To Do List) Protocol (SRP) is an emerging industry standard protocol for utilizing block storage devices over an InfiniBand™ fabric. SRP is being defined in the ANSI T-10 committee. -WinOF SRP is a storage +
OFED SRP is a storage driver implementation that enables the SCSI RDMA protocol over an InfiniBand fabric.
The implementation conforms @@ -4714,7 +4708,7 @@ to the T10 Working Group draft http://www.t10.org/ftp/t10/drafts/srp/srp-r16a.pdf.Software Dependencies
-The SRP driver depends on the installation of the WinOF stack +
The SRP driver depends on the installation of the OFED stack with a Subnet
Manager running somewhere on the IB fabric.
@@ -4738,13 +4732,13 @@ Testing included SRP target drive format, read, write and dismount/offline operations.
Installation
-The WinOF installer does not install the SRP driver as part of a default +
The OFED installer does not install the SRP driver as part of a default installation. If the SRP feature is selected in the custom features installation view, an InfiniBand SRP Miniport driver will be installed; see the device manager view under SCSI and RAID controllers.
The system device 'InfiniBand I/O Unit' (IOU) -device is required for correct SRP operation. The WinOF installer will +device is required for correct SRP operation. The OFED installer will install and load the IOU driver if the SRP feature is selected. See the device manager view System Devices --> InfiniBand I/O Unit for conformation of correct IOU driver loading.
@@ -4771,9 +4765,9 @@ XP. Letter.The consequences of not releasing the SRP target drive(s) are that after the -WinOF uninstall reboot there are lingering InfiniBand driver files. These driver +OFED uninstall reboot there are lingering InfiniBand driver files. These driver files remain because while the SRP target is active they have references, thus -when the WinOF uninstall attempts to delete the files the operation fails.
+when the OFED uninstall attempts to delete the files the operation fails.SRP supports WPP tracing tools by using the GUID: '5AF07B3C-D119-4233-9C81-C07EF481CBE6'. The flags and level of debug can be controlled at load-time or run-time; see ib_srp.inf file for details.
@@ -5060,7 +5054,7 @@ Executing qlgcvnic_config without any option or with -l option will list the IOCInfiniBand Software Development Kit
-If selected during a WinOF install, the IB Software Development Kit will +
If selected during a OFED install, the IB Software Development Kit will be installed as '%SystemDrive%\IBSDK'. Underneath the IBSDK\ folder you will find an include folder 'Inc\', library definition files 'Lib\' along with a 'Samples' folder.
@@ -5102,12 +5096,13 @@ diagnostic tools.
devices.
(Open source iWarp drivers for Windows are not yet available.) A corresponding
userspace library installs as part of the Winverbs driver installation package.
Additionally, a Windows port of the OFED libibverbs library and several test
programs are also included.As of the WinOF 2.1 release, Winverbs and Winmad are are fully integrated into the HCA driver stack load.
-- 2.46.0
-That's to say, Winverbs and Winmad are now integral components of the WinOF +That's to say, Winverbs and Winmad are now integral components of the OFED stack.
Available libibverbs test programs and their usage are listed
below. Note that not all listed options apply to all applications
ibv_rc_pingpong, ibv_uc_pingpong, ibv_ud_pingpong
no args start a server and wait for connection
-h <host> connect to server at <host>
-p <port> listen on/connect to port <port> (default 18515)
-d <dev> use IB device <dev> (default first device found)
-i <port> use port <port> of IB device (default 1)
-s <size> size of message to exchange (default 4096)
-m <size> path MTU (default 1024)
-r <dep> number of receives to post at a time (default 500)
-n <iters> number of exchanges (default 1000)
-l <sl> service level value
-e sleep on CQ events (default poll)
ibv_send_bw, ibv_send_lat
ibv_read_bw, ibv_read_lat
ibv_write_bw, ibv_write_lat
no args start a server and wait for connection
-h <host> connect to server at <host>
-p <port> listen on/connect to port <port> (default 18515)
-d <dev> use IB device <dev> (default first device found)
-i <port> use port <port> of IB device (default 1)
-c <RC/UC/UD> connection type RC/UC/UD (default RC)
-m <mtu> mtu size (256 - 4096. default for hermon is 2048)
-s <size> size of message to exchange (default 65536)
-a Run sizes from 2 till 2^23
-t <dep> size of tx queue (default 300)
-g send messages to multicast group (UD only)
-r <dep> make rx queue bigger than tx (default 600)
-n <iters> number of exchanges (at least 2, default 1000)
-I <size> max size of message to be sent in inline mode (default 400)
-b measure bidirectional bandwidth (default unidirectional)
-V display version number
-e sleep on CQ events (default poll)
-N cancel peak-bw calculation (default with peak-bw)
-
To verify correct WinVerbs and libibverbs installation, run ibstat or ibv_devinfo. It
should report all RDMA devices in the system, along with limited port
attributes. Because of limitations in the WinOF stack +
To verify correct WinVerbs and libibverbs installation, run ibstat or ibv_devinfo. It
should report all RDMA devices in the system, along with limited port
attributes. Because of limitations in the +OFED for Windows stack in comparision to the Linux OFED stack, it is normal for the programs to
list several values as unknown.