<h1 align="left">\r
<img border="0" src="openfabrics.gif" width="107" height="93"> \r
<a href="#TOP"></a> </h1>\r
-<h1 align="center">Windows OpenFabrics </h1>\r
+<h1 align="center">OpenFabrics Enterprise Distribution (for Windows) </h1>\r
<h1 align="center">User's Manual</h1>\r
-<h2 align="center">Release 2.2</h2>\r
+<h2 align="center">Release 2.3</h2>\r
<h3 align="center">\r
-<!--webbot bot="Timestamp" S-Type="EDITED" S-Format="%m/%d/%Y" startspan -->12/29/2009<!--webbot bot="Timestamp" endspan i-checksum="12659" --></h3>\r
+<!--webbot bot="Timestamp" S-Type="EDITED" S-Format="%m/%d/%Y" startspan -->03/10/2010<!--webbot bot="Timestamp" endspan i-checksum="12491" --></h3>\r
<h2 align="left"><u>Overview</u></h2>\r
<p align="left"><span style="FONT-SIZE: 12pt; FONT-FAMILY: 'Times New Roman'">\r
-The Windows OpenFabrics (WinOF) package is composed of software modules intended \r
+The OpenFabrics Enterprise Distribution for Windows package is composed of software modules intended \r
for use on Microsoft Windows based computer systems connected via an InfiniBand \r
fabric.</span></p>\r
-<p align="left">The Windows OpenFabrics software package contains the \r
+<p align="left">The OpenFabrics <span style="FONT-SIZE: 12pt; FONT-FAMILY: 'Times New Roman'">\r
+Enterprise Distribution for Windows </span>software package contains the \r
following:<br>\r
<br>\r
OpenFabrics Infiniband core drivers and Upper Level Protocols (ULPs):</p>\r
<p align="left">Release Notes</li>\r
</ul>\r
<p align="left"> </p>\r
-<h2 align="left"><u><a name="TOP"></a>WinOF Features</u></h2>\r
+<h2 align="left"><u><a name="TOP"></a>OFED Features</u></h2>\r
<ul>\r
<li>\r
<h3 align="left"><a href="#Tools">Tools & Diagnostics</a></h3></li>\r
<hr>\r
<blockquote>\r
<blockquote>\r
- <p align="left">The OpenFabrics Alliance Windows release contains a set of \r
- user mode tools which are designed to faciliate the smooth operation of an \r
- Windows OpenFabrics installation. These tools are available from a command \r
+ <p align="left">The OpenFabrics Alliance Enterprise for Windows release contains a set of \r
+ user mode tools which are designed to faciliate the smooth operation of an OpenFabrics \r
+ Enterprise Distribution installation. These tools are available from a command \r
window (cmd.exe) as the installation path '%SystemDrive%\Program \r
- Files\WinOF' is appended to the system wide search path registry entry. \r
- A start menu short-cut 'WinOF Cmd Window' is provided to faciliate \r
+ Files\OFED' is appended to the system wide search path registry entry. \r
+ A start menu short-cut 'OFED Cmd Window' is provided to faciliate \r
correction tool operation.</p>\r
<h4 align="left"><u>IPoIB Partition Management</u></h4>\r
<ul>\r
an Internet Protocol address associated with an IB GUID.</li>\r
<li>\r
<p align="left"><a href="#vstat">vstat</a> Display HCA attributes (lids), statistics and error counters.</li>\r
- </ul>\r
- <h4 align="left"><u><font color="#0000FF"><a href="#diags">\r
- <font color="#000000">OFED </font> </a> </font></u>\r
- <a href="#diags"><font color="#000000">Diagnostics</font></a></h4>\r
- <ul>\r
<li>\r
<p align="left"><a href="#IBADDR">ibaddr</a> \r
Query InfiniBand address(es)</li>\r
<br>\r
<br>\r
COMMON OPTIONS<br>\r
-Most WinOF diagnostics take the following common flags. The exact list<br>\r
+Most OFED diagnostics take the following common flags. The exact list<br>\r
of supported flags per utility can be found in the usage message and<br>\r
can be shown using the util_name -h syntax.<br>\r
<br>\r
<br>\r
--node-name-map <node-name-map><br>\r
Specify a node name map. The node name map file maps GUIDs to<br>\r
-more user friendly names. See ibnetdiscover(8) for node name<br>\r
+more user friendly names. See <a href="#IBNETDISCOVER">ibnetdiscover</a>(8) for node name<br>\r
map file format.<br>\r
<br>\r
<br>\r
Subnet Management as a Windows service is sufficient to correctly configure most \r
InfiniBand fabrics.</p>\r
<p align="left">The Infiniband subnet management process (opensm) may exist on a \r
-Windows (WinOF) node or a Linux (OFED) node.<br>\r
+Windows (OFED) node or a Linux (OFED) node.<br>\r
<br>\r
Limit the number of OpenSM processes per IB fabric; one SM is sufficient \r
although redundant SMs are supported. You do not need a Subnet Manager per \r
<P>
opensm is an InfiniBand compliant Subnet Manager and Administration,
-and runs on top of WinOF.
-
-opensm provides an implementation of an InfiniBand Subnet Manager and
+and runs on top of OFED for Windows. opensm provides an implementation of an InfiniBand Subnet Manager and
Administration. Such a software entity is required to run for in order
to initialize the InfiniBand hardware (at least one per each
InfiniBand subnet).
Prints OpenSM version and exits.
<DT><B>-F</B>, <B>--config</B> <config file><DD>
The name of the OpenSM config file. When not specified
-<b>%ProgramFiles%\WinOF\OpenSM\opensm.conf</b> will be used (if exists).
+<b>%ProgramFiles%\OFED\OpenSM\opensm.conf</b> will be used (if exists).
<DT><B>-c</B>, <B>--create-config</B> <file name><DD>
OpenSM will dump its configuration to the specified file and exit.
This is a way to generate OpenSM configuration file template.
is accumulative.
<DT><B>-P</B>, <B>--Pconfig</B> <partition config file><DD>
This option defines the optional partition configuration file.
-The default name is <b>%ProgramFiles%\WinOF\OpenSM\partitions.conf</b>.
+The default name is <b>%ProgramFiles%\OFED\OpenSM\partitions.conf</b>.
<DT><B>--prefix_routes_file</B> <file name><DD>
Prefix routes control how the SA responds to path record queries for
off-subnet DGIDs. By default, the SA fails such queries. The
<B>PREFIX ROUTES</B>
section below describes the format of the configuration file.
-The default path is <b>%ProgramFiles%\WinOF\OpenSM\prefix-routes.conf</b>.
+The default path is <b>%ProgramFiles%\OFED\OpenSM\prefix-routes.conf</b>.
<DT><B>-Q</B>, <B>--qos</B><DD>
This option enables QoS setup. It is disabled by default.
<DT><B>-Y</B>, <B>--qos_policy_file</B> <file name><DD>
This option defines the optional QoS policy file. The default
-name is <b>%ProgramFiles%\WinOF\OpenSM\qos-policy.conf</b>. See
+name is <b>%ProgramFiles%\OFED\OpenSM\qos-policy.conf</b>. See
QoS_management_in_OpenSM.txt in opensm doc for more information on
configuring QoS policy via this file.
<DT><B>-N</B>, <B>--no_part_enforce</B><DD>
<li>OSM_TMP_DIR - controls the directory in which the temporary files generated by
opensm are created.<br>\r
These files are: opensm-subnet.lst, opensm.fdbs, and
-opensm.mcfdbs. By default, this directory is %ProgramFiles%\WinOF\OpenSM\.</li>\r
+opensm.mcfdbs. By default, this directory is %ProgramFiles%\OFED\OpenSM\.</li>\r
<li>OSM_CACHE_DIR - opensm stores certain data to the disk such that \r
- subsequent runs are consistent. The default directory used is %ProgramFiles%\WinOF\OpenSM\.
+ subsequent runs are consistent. The default directory used is %ProgramFiles%\OFED\OpenSM\.
The following file is included in it: <BR> guid2lid - stores the LID range assigned to each GUID<br>\r
</li>\r
</ul>\r
<P>
The default name of OpenSM partitions configuration file is
-<B>%ProgramFiles\WinOF\OpenSM\partitions.conf</B>. The default may be changed
+<B>%ProgramFiles\OFED\OpenSM\partitions.conf</B>. The default may be changed
by using the --Pconfig (-P) option with OpenSM.
The default partition will be created by OpenSM unconditionally even
<h3>FILES</h3>
<DL COMPACT>
-<DT><B>%ProgramFiles\WinOF\OpenSM\opensm.conf</B>
+<DT><B>%ProgramFiles\OFED\OpenSM\opensm.conf</B>
<DD>
default OpenSM config file.
-<DT><B>%ProgramFiles\WinOF\OpenSM\ib-node-name-map.conf</B><DD>
+<DT><B>%ProgramFiles\OFED\OpenSM\ib-node-name-map.conf</B><DD>
default node name map file. See ibnetdiscover for more information on format.
-<DT><B>%ProgramFiles\WinOF\OpenSM\partitions.conf</B>
+<DT><B>%ProgramFiles\OFED\OpenSM\partitions.conf</B>
<DD>
default partition config file
-<DT><B>%ProgramFiles\WinOF\OpenSM\qos-policy.conf</B>
+<DT><B>%ProgramFiles\OFED\OpenSM\qos-policy.conf</B>
<DD>
default QOS policy config file
-<DT><B>%ProgramFiles\WinOF\OpenSM\prefix-routes.conf</B>
+<DT><B>%ProgramFiles\OFED\OpenSM\prefix-routes.conf</B>
<DD>
default prefix routes file.
the regular TCP/IP stack, or offload it to a WSD provider. The WSD provider is a \r
hardware specific DLL that implements connection management and data transfers \r
over particular RDMA hardware.</p>\r
-<p align="left">WinOF WSD is not supported in the Windows XP environment.</p>\r
+<p align="left">OFED WSD is not supported in the Windows XP environment.</p>\r
<p align="left">The WSD Protocol seamlessly transports TCP \r
data using Infiniband data packets in 'buffered' mode or Infiniband \r
RDMA in 'direct' mode. Either way the user mode socket application sees no \r
<p align="left">WSD service \r
is automatically installed and started as part of the 'default' installation; \r
except on XP systems - WSD not supported.<br>\r
-Manual control is performed via the \Program Files\WinOF\installsp.exe utility.</p>\r
+Manual control is performed via the \Program Files\OFED\installsp.exe utility.</p>\r
<blockquote>\r
<p align="left">usage: installsp [-i | -r | -l]<br>\r
<br>\r
Programming Interfaces that exploit the RDMA (remote direct memory access) \r
capabilities of next-generation interconnect technologies such as InfiniBand, \r
and iWARP.</font></p>\r
-<p align="left"><font face="Times New Roman">WinOF uDAT and uDAPL are based on the \r
+<p align="left"><font face="Times New Roman">OFED uDAT and uDAPL are based on the \r
2.0 DAT specification. The DAPL \r
(Direct Access Provider Library) which now fully supports Infiniband RDMA and \r
IPoIB.</font></p>\r
-<p align="left"><font face="Times New Roman">Previous WinOF \r
+<p align="left"><font face="Times New Roman">Previous OFED \r
releases supported the uDAT/uDAPL 1.1 provider which has now been deprecated.<br>\r
uDAT/uDAPL version 2.0 runtime libraries along with an optional \r
v2.0 application build environment are the only options.<br>\r
<blockquote>\r
<p align="left"><font face="Courier New" size="2">DAT/DAPL 2.0 (free-build) libraries are identified in %SystemRoot% as \r
dat2.dll and dapl2.dll. Debug versions of the v2.0 runtime libraries \r
- are located in '%SystemDrive%\%ProgramFiles%\WinOF'.</font></p>\r
+ are located in '%SystemDrive%\%ProgramFiles%\OFED'.</font></p>\r
<p align="left"><font face="Courier New" size="2">IA32 (aka, 32-bit) \r
versions of DAT/DAPL 2.0 runtime libraries, found only on 64-bit systems, \r
- are identified in '%ProgramFiles%\WinOF' as dat32.dll \r
+ are identified in '%ProgramFiles%\OFED' as dat32.dll \r
and dapl32.dll.</font></p>\r
<p align="left"><font face="Courier New" size="2">In order for DAT/DAPL \r
programs to execute correctly, the runtime library files 'dat2.dll and \r
dapl2.dll' must be present in one of the following folders: current \r
directory, %SystemRoot% or in the library search path.</font></p>\r
- <p align="left"><font face="Courier New" size="2">The default WinOF \r
+ <p align="left"><font face="Courier New" size="2">The default OFED \r
installation places the runtime library files dat2.dll and dapl2.dll in the '%SystemRoot%' folder; \r
- symbol files (.pdb) are located in '%ProgramFiles%\WinOF\'.</font></p>\r
+ symbol files (.pdb) are located in '%ProgramFiles%\OFED\'.</font></p>\r
<p align="left"><font face="Courier New" size="2">The default DAPL configuration \r
file is defined as '%SystemDrive%\DAT\dat.conf'. This default \r
specification can be overriden by use of the environment variable \r
the '\' (backslash character) is treated as an 'escape' character. Hence \r
all backslashes in the library path must be duplicated when enclosed in \r
double-quotes\r
- (e.g., "C:\\Programs Files\\WinOF\\dapl.dll").</font></p>\r
+ (e.g., "C:\\Programs Files\\OFED\\dapl.dll").</font></p>\r
<p align="left"><font face="Courier New" size="2">A sample InfiniBand dat.conf file is \r
- installed as '\Program Files\WinOF\dat.conf'. If dat.conf does not \r
+ installed as '\Program Files\OFED\dat.conf'. If dat.conf does not \r
exist in the DAT default configuration folder '%SystemDrive%\DAT\', dat.conf \r
will be copied there.<br>\r
</font></p>\r
assigned to an IPoIB device instance. The 'rdma_dev0' is used to \r
instruct the rdma-cm provider to use the 1st RDMA device (IPoIB) \r
with an assigned IP4v address such that the dat.conf file does not \r
- have to be specifically tailored for each WinOF installation.</font></p></li>\r
+ have to be specifically tailored for each OFED installation.</font></p></li>\r
</ul>\r
<p align="left"><font face="Courier New" size="2">OFED RDMA CM manager \r
can be used to establish connections between Windows and/or Linux systems.</font></p></li>\r
Protocol </a> (SRP) is an emerging industry standard protocol for utilizing \r
block storage devices over an InfiniBand\99 fabric. SRP is being defined in the \r
ANSI T-10 committee.</font></span></p>\r
-<p align="left"><font face="Times New Roman">WinOF </font>SRP is a storage \r
+<p align="left"><font face="Times New Roman">OFED </font>SRP is a storage \r
driver implementation that enables the SCSI RDMA protocol over an InfiniBand \r
fabric.<br>\r
The implementation conforms\r
<a href="http://www.t10.org/ftp/t10/drafts/srp/srp-r16a.pdf">\r
http://www.t10.org/ftp/t10/drafts/srp/srp-r16a.pdf</a>.</p>\r
<h4 align="left"><u>Software Dependencies</u></h4>\r
-<p align="left">The SRP driver depends on the installation of the WinOF stack \r
+<p align="left">The SRP driver depends on the installation of the OFED stack \r
with a Subnet<br>\r
Manager running somewhere on the IB fabric.<br>\r
<br>\r
operations.<br>\r
</p>\r
<h4 align="left"><u>Installation</u></h4>\r
-<p align="left">The WinOF installer does not install the SRP driver as part of a default \r
+<p align="left">The OFED installer does not install the SRP driver as part of a default \r
installation. \r
If the SRP feature is selected in the custom features installation view, an InfiniBand \r
SRP Miniport driver will be installed; see the device manager view under SCSI \r
and RAID controllers.</p>\r
<p align="left">The system device 'InfiniBand I/O Unit' (IOU) \r
-device is required for correct SRP operation. The WinOF installer will \r
+device is required for correct SRP operation. The OFED installer will \r
install and load the IOU driver if the SRP feature is selected. See the device \r
manager view System Devices --> InfiniBand I/O Unit for conformation of correct \r
IOU driver loading.</p>\r
Letter.</li>\r
</ul>\r
<p>The consequences of not releasing the SRP target drive(s) are that after the \r
-WinOF uninstall reboot there are lingering InfiniBand driver files. These driver \r
+OFED uninstall reboot there are lingering InfiniBand driver files. These driver \r
files remain because while the SRP target is active they have references, thus \r
-when the WinOF uninstall attempts to delete the files the operation fails.</p>\r
+when the OFED uninstall attempts to delete the files the operation fails.</p>\r
<p align="left">SRP supports WPP tracing tools by using the GUID: \r
'5AF07B3C-D119-4233-9C81-C07EF481CBE6'. The flags and level of debug can be controlled at load-time or run-time; \r
see ib_srp.inf file for details.</p>\r
<h2><a name="InfiniBand_Software_Development_Kit">InfiniBand Software \r
Development Kit</a></h2>\r
<hr>\r
-<p>If selected during a WinOF install, the IB Software Development Kit will \r
+<p>If selected during a OFED install, the IB Software Development Kit will \r
be installed as '%SystemDrive%\IBSDK'. Underneath the IBSDK\ folder you will find an \r
include folder 'Inc\', library definition files 'Lib\' along with a \r
'Samples' folder.</p>\r
devices.<br>(Open source iWarp drivers for Windows are not yet available.) A corresponding<br>userspace library installs as part of the Winverbs driver installation package.<br>Additionally, a Windows port of the OFED libibverbs library and several test<br>programs are also included.</p>\r
<p>As of the WinOF 2.1 release, Winverbs and Winmad are are fully integrated \r
into the HCA driver stack load.<br>\r
-That's to say, Winverbs and Winmad are now integral components of the WinOF \r
+That's to say, Winverbs and Winmad are now integral components of the OFED \r
stack.<br>\r
<br>Available libibverbs test programs and their usage are listed<br>below. Note that not all listed options apply to all applications<br>\r
<br>ibv_rc_pingpong, ibv_uc_pingpong, ibv_ud_pingpong<br>no args start a server and wait for connection<br>-h <host> connect to server at <host><br>-p <port> listen on/connect to port <port> (default 18515)<br>-d <dev> use IB device <dev> (default first device found)<br>-i <port> use port <port> of IB device (default 1)<br>-s <size> size of message to exchange (default 4096)<br>-m <size> path MTU (default 1024)<br>-r <dep> number of receives to post at a time (default 500)<br>-n <iters> number of exchanges (default 1000)<br>-l <sl> service level value<br>-e sleep on CQ events (default poll)<br>\r
<br>ibv_send_bw, ibv_send_lat<br>ibv_read_bw, ibv_read_lat<br>ibv_write_bw, ibv_write_lat<br>no args start a server and wait for connection<br>-h <host> connect to server at <host><br>-p <port> listen on/connect to port <port> (default 18515)<br>-d <dev> use IB device <dev> (default first device found)<br>-i <port> use port <port> of IB device (default 1)<br>-c <RC/UC/UD> connection type RC/UC/UD (default RC)<br>-m <mtu> mtu size (256 - 4096. default for hermon is 2048)<br>-s <size> size of message to exchange (default 65536)<br>-a Run sizes from 2 till 2^23<br>-t <dep> size of tx queue (default 300)<br>-g send messages to multicast group (UD only)<br>-r <dep> make rx queue bigger than tx (default 600)<br>-n <iters> number of exchanges (at least 2, default 1000)<br>-I <size> max size of message to be sent in inline mode (default 400)<br>-b measure bidirectional bandwidth (default unidirectional)<br>-V display version number<br>-e sleep on CQ events (default poll)<br>-N cancel peak-bw calculation (default with peak-bw)<br>\r
-<br>To verify correct WinVerbs and libibverbs installation, run ibstat or ibv_devinfo. It<br>should report all RDMA devices in the system, along with limited port<br>attributes. Because of limitations in the WinOF stack \r
+<br>To verify correct WinVerbs and libibverbs installation, run ibstat or ibv_devinfo. It<br>should report all RDMA devices in the system, along with limited port<br>attributes. Because of limitations in the \r
+OFED for Windows stack \r
in comparision to the Linux OFED stack, it is normal for the programs to<br>list several values as unknown.</p>\r
<p><a href="#TOP"><font color="#000000"><<b>return-to-top</b>></font></a></p>\r
\r
projects / ~shefty / rdma-win.git / commitdiff