From 6362c96e60a8c29d00ce4d6995212249bcbe07ba Mon Sep 17 00:00:00 2001
From: stansmith
The Windows OpenFabrics (WinOF) package is composed of software modules intended
@@ -90,7 +90,7 @@ style='background-position: 0% 0%; mso-highlight:yellow; background-image:none;
WinOF DAT and DAPL are based on the 1.1 DAT specification. The DAPL
+ WinOF uDAT and uDAPL are based on the
+2.0 DAT specification. The DAPL
(Direct Access Provider Library) which now fully supports Infiniband RDMA and
IPoIB. WinOF 1.0.1, and future WinOF
-releases, will include DAT/DAPL version 2.0 runtime libraries along with an optional
-v2.0 application build environment. Previous WinOF
+releases supported the uDAT/uDAPL 1.1 provider which has now been deprecated. DAT/DAPL v1.1 (free-build)
- runtime libraries are installed into %SystemRoot%, with the v1.1 Debug
- versions located in '%SystemDrive%\%ProgramFiles(x86)%\WinOF'. Debug
- libraries are identified as datd.dll and dapld.dll. IA32 (aka, 32-bit)
- versions of DAT/DAPL 1.1 runtime libraries, found only on 64-bit systems,
- are identified in '%SystemDrive%\%ProgramFiles(x86)%\WinOF' as dat32.dll and
- dapl32.dll. DAT/DAPL 2.0 (free-build) libraries are identified in %SystemRoot% as
dat2.dll and dapl2.dll. Debug versions of the v2.0 runtime libraries
- are located in '%SystemDrive%\%ProgramFiles(x86)%\WinOF'. IA32 (aka, 32-bit)
versions of DAT/DAPL 2.0 runtime libraries, found only on 64-bit systems,
- are identified in '%SystemDrive%\%ProgramFiles(x86)%\WinOF' as dat232.dll
- and dapl232.dll. In order for DAT/uDAPL
- programs to execute correctly, the runtime library files 'dat.dll and
- dapl.dll' must be present in one of the following folders: current
+ are identified in '%ProgramFiles%\WinOF' as dat32.dll
+ and dapl32.dll. In order for DAT/DAPL
+ programs to execute correctly, the runtime library files 'dat2.dll and
+ dapl2.dll' must be present in one of the following folders: current
directory, %SystemRoot% or in the library search path. The default WinOF
- installation places the runtime library files dat.dll and dapl.dll in the '%SystemRoot%' folder;
- symbol files (.pdb) are located in '%SystemDrive%\%ProgramFiles(x86)%\WinOF'.Windows OpenFabrics
User's Manual
-Release 2.1
+Release 2.2
-12/10/2009
+12/29/2009
Overview
SRP (SCSI RDMA) Protocol Driver
SRP (SCSI over RDMA) Protocol Driver
QLogic VNIC_Driver
-DAT 2.0 is configured with InfiniBand extensions enabled. The IB extensions
+
+uDAT/uDAPL version 2.0 runtime libraries along with an optional
+v2.0 application build environment are the only options.
+uDAT 2.0 is configured with InfiniBand extensions enabled. The IB extensions
include
-
-
- DAT/DAPL 2.0 (free-build) libraries are identified in %SystemRoot% as
+
The default DAPL configuration file is defined as '%SystemDrive%\DAT\dat.conf'. This default specification can be overriden by use of the environment variable @@ -4184,7 +4177,7 @@ include
Within the dat.conf file, the DAPL library specification can be located as the 5th whitespace separated line argument. By default the DAPL library file is installed as - '%SystemRoot%\dapl.dll'.
+ '%SystemRoot%\dapl2.dll'.Should you choose to relocated the DAPL library file to a path where whitespace appears in the full library path specification, then the full library file specification @@ -4210,9 +4203,7 @@ include
CM is IBAL).Each non-comment line in
the dat.conf file describes a DAPL provider interface.
The 2nd to the last field on the right (7th from the left) describes the
@@ -4295,10 +4286,9 @@ include
DAT application build environment:
DAT library header files - are selectively installed in the DAT default configuration folder as
'%SystemDrive%\DAT\v1-1' - or - '%SystemDrive%\DAT\v2-0'. Your C language based DAT 1.1 application compilation command line - should include'/I%SystemDrive%\DAT\v1-1' with C code referencing '#include <DAT\udat.h>'.
+ are selectively installed in the DAT default configuration folder as
'%SystemDrive%\DAT\v2-0'. + Your C language based DAT application compilation command line + should include'/I%SystemDrive%\DAT\v2-0' with C code referencing '#include <DAT\udat.h>'.
The 'default' DAT/DAPL C language calling convention is '__stdcall', not the 'normal' Visual Studio C compiler default. __stdcall was chosen as MS @@ -4307,7 +4297,7 @@ include
Visual Studio 2005 command window - (nmake) Makefile Fragments:-DAT_PATH=%SystemDrive%\DAT\v1-1
+ When linking a DEBUG/Checked version make sure to use dat2d.lib .
+DAT_PATH=%SystemDrive%\DAT\v2-0
CC = cl
INC_FLAGS = /I $(DAT_PATH)
@@ -4321,8 +4311,7 @@ include
LINK_FLAGS = /nologo /subsystem:console /machine:X64 /libpath:$(DAT_PATH) $(LIBS)
- When linking a DEBUG/Checked version make sure to use datd.lib or - dat2d.lib for DAT v2.0.
- dapltest - test for the Direct Access Provider Library (DAPL) + dapltest - test for the Direct Access Provider Library (DAPL) v2.0 DESCRIPTION @@ -4392,8 +4381,8 @@ DESCRIPTION and verify the DAPL interfaces during development and porting. At least two instantiations of the test must be run. One acts as the server, fielding requests and spawning server-side test - threads as needed. Other client invocations connect to the - server and issue test requests. + threads as needed. Other client invocation connects to the + Dapltest server and issue test requests. The server side of the test, once invoked, listens continuously for client connection requests, until quit or killed. Upon @@ -4629,35 +4618,35 @@ USAGE - Limit test client EXAMPLES - dapltest -T S -d -D ibnic0 + dapltest -T S -d -D ibnic0v2 Starts a local dapltest server process with debug verbosity. Server loops (listen for dapltest request, process request). - dapltest -T T -d -s winIB -D ibnic0 -i 100 client SR 4096 2 server SR 4096 2 + dapltest -T T -d -s winIB -D ibnic0v2 -i 100 client SR 4096 2 server SR 4096 2 Runs a transaction test, with both sides sending one buffer with two 4KB segments, one hundred times; dapltest server is on host winIB. - dapltest -T P -d -s winIB -D ibnic0 -i 100 RW 4096 2 + dapltest -T P -d -s winIB -D ibnic0v2 -i 100 RW 4096 2 Runs a performance test, with the client RDMA writing one buffer with two 4KB segments, one hundred times. - dapltest -T Q -s winIB -D ibnic0 + dapltest -T Q -s winIB -D ibnic0v2 Asks the dapltest server at host 'winIB' to clean up and exit. - dapltest -T L -D ibnic0 -d -w 16 -m 1000 + dapltest -T L -D ibnic0v2 -d -w 16 -m 1000 Runs all of the limit tests, setting up 16 complete sets of DAPL objects, and creating at most a thousand instances when trying to exhaust resources. - dapltest -T T -V -d -t 2 -w 4 -i 55555 -s winIB -D ibnic0 \ + dapltest -T T -V -d -t 2 -w 4 -i 55555 -s winIB -D ibnic0v2 \ client RW 4096 1 server RW 2048 4 \ client SR 1024 4 server SR 4096 2 \ client SR 1024 3 -f server SR 2048 1 -f @@ -4668,7 +4657,7 @@ USAGE - Limit test client for a larger number of iterations, validating the data received.
-dt-svr.bat - DAPLtest server script; starts a DAPL2test.exe server on the local node. +dt-svr.bat - DAPLtest server script; starts a DAPL2test.exe server on the local system. dt-svr DAPL-provider [-D [hex-debug-bitmask] ]
@@ -4695,9 +4684,9 @@ dt-svr.bat - DAPLtest server script; starts a DAPL2test.exe server on the local dt-cli DAPL-provider host-IPv4-address testname [-D [hex-debug-bitmask] ] example: dt-cli ibnic0v2 10.10.2.20 trans dt-cli -h # outputs help text. - dt-svr ibnic0v2 # IBAL on HCA0 + dt-svr ibnic0v2 # IBAL dapltest server listening on port HCA0-Verify dt-*.bat script is running same dapltest.exe(v1.1) or dapl2test.exe(v2.0) +Verify dt-*.bat script is running same dapl2test.exe(DAPL v2.0) BUGS (and To Do List) @@ -4730,15 +4719,21 @@ with a Subnet
Manager running somewhere on the IB fabric.
- Supported Operating Systems and Service Packs:
- o Windows XP SP3 x86 & x64
+ o Windows 7 (x86 & x64)
+ o Windows Server 2008 R2 (x86, x64)
o Windows Server 2008/Vista (x86, x64)
- o Windows Server 2008 HPC (x64)
+ o Windows Server 2008 HPC (x64,x86)
o Windows Server 2003 SP2/R2 (x86, x64, IA64)Testing Levels
The SRP driver has undergone basic testing against Mellanox Technologies' SRP Targets MTD1000 and MTD2000.
@@ -4782,6 +4777,168 @@ when the WinOF uninstall attempts to delete the files the operation fails.
-Additionally the Linux OFED 1.4 SRP target has been tested.
+Additionally the Linux OFED 1.4.1 SRP target with scst 1.0.0.0 (vdisk with +blockio) has been tested. Note: ONLY the scst-1.0.0.0 release will work as +the OFED 1.4.1 ib_srpt driver has a symbol dependency on scst. later releases of +scst.ko no longer export the required symbol hence ib_srpt fails to load.
+When ib_srpt is updated later versions of scst can be used.
+
Testing included SRP target drive format, read, write and dismount/offline operations.
SRP supports WPP tracing tools by using the GUID: '5AF07B3C-D119-4233-9C81-C07EF481CBE6'. The flags and level of debug can be controlled at load-time or run-time; see ib_srp.inf file for details.
+Constructing a RHEL 5.1 OFED 1.4.1 SRP vdisk BLOCKIO target
+Example assumptions:
++
+- +
+Red Hat Enterprise Linux Server release 5 (Tikanga) EL 5.1 + (2.6.18-8.el5)
+ Later RHEL 5.4 releases will work with some minor scst compile time fixes + and later OFED releases.- +
+Linux SRP Target has /dev/sdb[123]
+ sizeof(sdb1) < sdb2 < sdb3; test convention only.- +
+scst-1.0.0.tgz is required as OFED 1.4.1 ib_srpt has symbol + dependency on
+ scst.ko....sigh. Later scst release do not export the required ib_srpt
+ symbol; stick with scst-1.0.0 until OFED ib_srpt is updated.- +
+Use out of the box scst defines which include (#undef +STRICT_SERIALIZING),
+
+'no' kernel mods are required for BLOCKIO access to /dev/sdb[123].
+
+cd scst-1.0.0.0
+make all
+make install
+
+cd OFED-1.4.1
+build OFED select #3 for 'all' OFED components
+ - no SRP loads in /etc/infiniband/openib.conf, edit prior to reboot.REBOOT.
+
+
+./LOAD & ./UNLOAD scripts are manual versions of what scstAdmin (separate scst +package) will do minus
+loading the OFED driver ib_srpt.
+
+SRP targets formatted from Windows using default NTFS allocation size.
+Partition size & numbering is derrived from local test conventions; your setup +will be different.
+
+/dev/sdb1 NTFS < 1GB
+/dev/sdb2 NTFS > 1GB
+/dev/sdb3 NTFS > sdb2+
Manual SRP Target LOAD script
+++#!/bin/sh + +GRP=Default + +if [ ! -e /proc/scsi_tgt ] ; then + echo -n Loading scst driver + modprobe scst + if [ $? -ne 0 ] ; then + echo + echo err $? modprobe scst + exit $? + fi + echo ...OK +fi + +if [ ! -e /proc/scsi_tgt/vdisk ] ; then + echo -n Loading scst_vdisk driver + modprobe scst_vdisk + if [ $? -ne 0 ] ; then + echo + echo err $? modprobe scst_vdisk + exit $? + fi + echo ...OK +fi + +if [ ! -e /proc/scsi_tgt/vdisk ] ; then + echo -n Loading scst_vdisk driver + modprobe scst_vdisk + if [ $? -ne 0 ] ; then + echo + echo err $? modprobe scst_vdisk + exit $? + fi + echo ...OK +fi + +fgrep -q ib_srpt /proc/modules +if [ $? -ne 0 ] ; then + modprobe ib_srpt + echo ib_srpt...OK +fi + +echo -n Open SRP devices srp[123] +echo "open srp1 /dev/sdb1 512 BLOCKIO" > /proc/scsi_tgt/vdisk/vdisk +if [ $? -ne 0 ] ; then + echo err $? open srp1 /dev/sdb1 + exit $? +fi +echo "open srp2 /dev/sdb2 512 BLOCKIO" > /proc/scsi_tgt/vdisk/vdisk +if [ $? -ne 0 ] ; then + echo err $? open srp2 /dev/sdb2 + exit $? +fi +echo "open srp3 /dev/sdb3 512 BLOCKIO" > /proc/scsi_tgt/vdisk/vdisk +if [ $? -ne 0 ] ; then + echo err $? open srp3 /dev/sdb3 + exit $? +fi +echo ...OK + +echo -n Set allowed hosts access... +echo "add *" > /proc/scsi_tgt/groups/Default/names +echo ...OK + +echo -n Adding targets srp[123] as LUNs [012] in group $GRP +echo "add srp1 0" > /proc/scsi_tgt/groups/Default/devices +if [ $? -ne 0 ] ; then + echo + echo err $? add srp1 0 + exit $? +fi +echo "add srp2 1" > /proc/scsi_tgt/groups/Default/devices +if [ $? -ne 0 ] ; then + echo + echo err $? add srp2 1 + exit $? +fi +echo "add srp3 2" > /proc/scsi_tgt/groups/Default/devices +if [ $? -ne 0 ] ; then + echo + echo err $? add srp3 2 + exit $? +fi +echo ...OK +++Manual SRP Target UNLOAD script
+#!/bin/sh + +if [ -w /proc/scsi_tgt/vdisk/vdisk ] ; then + echo -n Closing SRP Targets srp[321]... + echo "close srp3" > /proc/scsi_tgt/vdisk/vdisk + echo "close srp2" > /proc/scsi_tgt/vdisk/vdisk + echo "close srp1" > /proc/scsi_tgt/vdisk/vdisk + echo Done. +fi + +fgrep -q scst_vdisk /proc/modules +if [ $? -eq 0 ] ; then + modprobe -r scst_vdisk +fi + +fgrep -q ib_srpt /proc/modules +if [ $? -eq 0 ] ; then + modprobe -r ib_srpt +fi + +fgrep -q scst /proc/modules +if [ $? -eq 0 ] ; then + modprobe -r scst +fi ++<return-to-top>
diff --git a/trunk/docs/Manual.htm b/trunk/docs/Manual.htm index d9562227..0498658c 100644 --- a/trunk/docs/Manual.htm +++ b/trunk/docs/Manual.htm @@ -13,9 +13,9 @@ div.Section1
Windows OpenFabrics
User's Manual
-Release 2.1
+Release 2.2
-12/10/2009
+12/29/2009Overview
The Windows OpenFabrics (WinOF) package is composed of software modules intended @@ -90,7 +90,7 @@ style='background-position: 0% 0%; mso-highlight:yellow; background-image:none;
-
- +SRP (SCSI RDMA) Protocol Driver
SRP (SCSI over RDMA) Protocol Driver
QLogic VNIC_Driver
@@ -4066,13 +4066,15 @@ order provide a set of transport-independent, platform-independent Application Programming Interfaces that exploit the RDMA (remote direct memory access) capabilities of next-generation interconnect technologies such as InfiniBand, and iWARP. WinOF DAT and DAPL are based on the 1.1 DAT specification. The DAPL +
WinOF uDAT and uDAPL are based on the +2.0 DAT specification. The DAPL (Direct Access Provider Library) which now fully supports Infiniband RDMA and IPoIB.
-WinOF 1.0.1, and future WinOF -releases, will include DAT/DAPL version 2.0 runtime libraries along with an optional -v2.0 application build environment.
DAT ENVIRONMENT:
-DAT 2.0 is configured with InfiniBand extensions enabled. The IB extensions +Previous WinOF +releases supported the uDAT/uDAPL 1.1 provider which has now been deprecated.
+uDAT/uDAPL version 2.0 runtime libraries along with an optional +v2.0 application build environment are the only options.
+uDAT 2.0 is configured with InfiniBand extensions enabled. The IB extensions include
- @@ -4154,29 +4156,20 @@ include
-DAT/DAPL v1.1 (free-build) - runtime libraries are installed into %SystemRoot%, with the v1.1 Debug - versions located in '%SystemDrive%\%ProgramFiles(x86)%\WinOF'. Debug - libraries are identified as datd.dll and dapld.dll.
-IA32 (aka, 32-bit) - versions of DAT/DAPL 1.1 runtime libraries, found only on 64-bit systems, - are identified in '%SystemDrive%\%ProgramFiles(x86)%\WinOF' as dat32.dll and - dapl32.dll.
-
- DAT/DAPL 2.0 (free-build) libraries are identified in %SystemRoot% as +DAT/DAPL 2.0 (free-build) libraries are identified in %SystemRoot% as dat2.dll and dapl2.dll. Debug versions of the v2.0 runtime libraries - are located in '%SystemDrive%\%ProgramFiles(x86)%\WinOF'.
+ are located in '%SystemDrive%\%ProgramFiles%\WinOF'.IA32 (aka, 32-bit) versions of DAT/DAPL 2.0 runtime libraries, found only on 64-bit systems, - are identified in '%SystemDrive%\%ProgramFiles(x86)%\WinOF' as dat232.dll - and dapl232.dll.
-In order for DAT/uDAPL - programs to execute correctly, the runtime library files 'dat.dll and - dapl.dll' must be present in one of the following folders: current + are identified in '%ProgramFiles%\WinOF' as dat32.dll + and dapl32.dll.
+In order for DAT/DAPL + programs to execute correctly, the runtime library files 'dat2.dll and + dapl2.dll' must be present in one of the following folders: current directory, %SystemRoot% or in the library search path.
The default WinOF - installation places the runtime library files dat.dll and dapl.dll in the '%SystemRoot%' folder; - symbol files (.pdb) are located in '%SystemDrive%\%ProgramFiles(x86)%\WinOF'.
+ installation places the runtime library files dat2.dll and dapl2.dll in the '%SystemRoot%' folder; + symbol files (.pdb) are located in '%ProgramFiles%\WinOF\'.The default DAPL configuration file is defined as '%SystemDrive%\DAT\dat.conf'. This default specification can be overriden by use of the environment variable @@ -4184,7 +4177,7 @@ include
Within the dat.conf file, the DAPL library specification can be located as the 5th whitespace separated line argument. By default the DAPL library file is installed as - '%SystemRoot%\dapl.dll'.
+ '%SystemRoot%\dapl2.dll'.Should you choose to relocated the DAPL library file to a path where whitespace appears in the full library path specification, then the full library file specification @@ -4210,9 +4203,7 @@ include
CM is IBAL).
ibnic1v2-scm - InfiniBand HCA #one, DAPL version 2.0, CM is 'socket-CM'
ibnic0v2-cma - InfiniBand HCA #zero, DAPL version 2.0, CM is - 'rdma-CM'
- ibnic0-scm - InfiniBand HCA #zero, DAPL version 1.1, CM is - 'IBAL' + 'rdma-CM'Each non-comment line in the dat.conf file describes a DAPL provider interface.
The 2nd to the last field on the right (7th from the left) describes the @@ -4295,10 +4286,9 @@ includeDAT application build environment:
DAT library header files - are selectively installed in the DAT default configuration folder as
'%SystemDrive%\DAT\v1-1' - or - '%SystemDrive%\DAT\v2-0'. Your C language based DAT 1.1 application compilation command line - should include'/I%SystemDrive%\DAT\v1-1' with C code referencing '#include <DAT\udat.h>'.
+ are selectively installed in the DAT default configuration folder as
'%SystemDrive%\DAT\v2-0'. + Your C language based DAT application compilation command line + should include'/I%SystemDrive%\DAT\v2-0' with C code referencing '#include <DAT\udat.h>'.
The 'default' DAT/DAPL C language calling convention is '__stdcall', not the 'normal' Visual Studio C compiler default. __stdcall was chosen as MS @@ -4307,7 +4297,7 @@ include
Visual Studio 2005 command window - (nmake) Makefile Fragments:-DAT_PATH=%SystemDrive%\DAT\v1-1
+ When linking a DEBUG/Checked version make sure to use dat2d.lib .
+DAT_PATH=%SystemDrive%\DAT\v2-0
CC = cl
INC_FLAGS = /I $(DAT_PATH)
@@ -4321,8 +4311,7 @@ include
LINK_FLAGS = /nologo /subsystem:console /machine:X64 /libpath:$(DAT_PATH) $(LIBS)
- When linking a DEBUG/Checked version make sure to use datd.lib or - dat2d.lib for DAT v2.0.DAT library environment variables:
@@ -4384,7 +4373,7 @@ debugging information, valid values are
DAPLTEST
- dapltest - test for the Direct Access Provider Library (DAPL) + dapltest - test for the Direct Access Provider Library (DAPL) v2.0 DESCRIPTION @@ -4392,8 +4381,8 @@ DESCRIPTION and verify the DAPL interfaces during development and porting. At least two instantiations of the test must be run. One acts as the server, fielding requests and spawning server-side test - threads as needed. Other client invocations connect to the - server and issue test requests. + threads as needed. Other client invocation connects to the + Dapltest server and issue test requests. The server side of the test, once invoked, listens continuously for client connection requests, until quit or killed. Upon @@ -4629,35 +4618,35 @@ USAGE - Limit test client EXAMPLES - dapltest -T S -d -D ibnic0 + dapltest -T S -d -D ibnic0v2 Starts a local dapltest server process with debug verbosity. Server loops (listen for dapltest request, process request). - dapltest -T T -d -s winIB -D ibnic0 -i 100 client SR 4096 2 server SR 4096 2 + dapltest -T T -d -s winIB -D ibnic0v2 -i 100 client SR 4096 2 server SR 4096 2 Runs a transaction test, with both sides sending one buffer with two 4KB segments, one hundred times; dapltest server is on host winIB. - dapltest -T P -d -s winIB -D ibnic0 -i 100 RW 4096 2 + dapltest -T P -d -s winIB -D ibnic0v2 -i 100 RW 4096 2 Runs a performance test, with the client RDMA writing one buffer with two 4KB segments, one hundred times. - dapltest -T Q -s winIB -D ibnic0 + dapltest -T Q -s winIB -D ibnic0v2 Asks the dapltest server at host 'winIB' to clean up and exit. - dapltest -T L -D ibnic0 -d -w 16 -m 1000 + dapltest -T L -D ibnic0v2 -d -w 16 -m 1000 Runs all of the limit tests, setting up 16 complete sets of DAPL objects, and creating at most a thousand instances when trying to exhaust resources. - dapltest -T T -V -d -t 2 -w 4 -i 55555 -s winIB -D ibnic0 \ + dapltest -T T -V -d -t 2 -w 4 -i 55555 -s winIB -D ibnic0v2 \ client RW 4096 1 server RW 2048 4 \ client SR 1024 4 server SR 4096 2 \ client SR 1024 3 -f server SR 2048 1 -f @@ -4668,7 +4657,7 @@ USAGE - Limit test client for a larger number of iterations, validating the data received.-dt-svr.bat - DAPLtest server script; starts a DAPL2test.exe server on the local node. +dt-svr.bat - DAPLtest server script; starts a DAPL2test.exe server on the local system. dt-svr DAPL-provider [-D [hex-debug-bitmask] ]@@ -4695,9 +4684,9 @@ dt-svr.bat - DAPLtest server script; starts a DAPL2test.exe server on the local dt-cli DAPL-provider host-IPv4-address testname [-D [hex-debug-bitmask] ] example: dt-cli ibnic0v2 10.10.2.20 trans dt-cli -h # outputs help text. - dt-svr ibnic0v2 # IBAL on HCA0 + dt-svr ibnic0v2 # IBAL dapltest server listening on port HCA0-Verify dt-*.bat script is running same dapltest.exe(v1.1) or dapl2test.exe(v2.0) +Verify dt-*.bat script is running same dapl2test.exe(DAPL v2.0) BUGS (and To Do List) @@ -4730,15 +4719,21 @@ with a Subnet
Manager running somewhere on the IB fabric.
- Supported Operating Systems and Service Packs:
- o Windows XP SP3 x86 & x64
+ o Windows 7 (x86 & x64)
+ o Windows Server 2008 R2 (x86, x64)
o Windows Server 2008/Vista (x86, x64)
- o Windows Server 2008 HPC (x64)
+ o Windows Server 2008 HPC (x64,x86)
o Windows Server 2003 SP2/R2 (x86, x64, IA64)Testing Levels
The SRP driver has undergone basic testing against Mellanox Technologies' SRP Targets MTD1000 and MTD2000.
@@ -4782,6 +4777,168 @@ when the WinOF uninstall attempts to delete the files the operation fails.
-Additionally the Linux OFED 1.4 SRP target has been tested.
+Additionally the Linux OFED 1.4.1 SRP target with scst 1.0.0.0 (vdisk with +blockio) has been tested. Note: ONLY the scst-1.0.0.0 release will work as +the OFED 1.4.1 ib_srpt driver has a symbol dependency on scst. later releases of +scst.ko no longer export the required symbol hence ib_srpt fails to load.
+When ib_srpt is updated later versions of scst can be used.
+
Testing included SRP target drive format, read, write and dismount/offline operations.
SRP supports WPP tracing tools by using the GUID: '5AF07B3C-D119-4233-9C81-C07EF481CBE6'. The flags and level of debug can be controlled at load-time or run-time; see ib_srp.inf file for details.
+Constructing a RHEL 5.1 OFED 1.4.1 SRP vdisk BLOCKIO target
+Example assumptions:
++
+- +
+Red Hat Enterprise Linux Server release 5 (Tikanga) EL 5.1 + (2.6.18-8.el5)
+ Later RHEL 5.4 releases will work with some minor scst compile time fixes + and later OFED releases.- +
+Linux SRP Target has /dev/sdb[123]
+ sizeof(sdb1) < sdb2 < sdb3; test convention only.- +
+scst-1.0.0.tgz is required as OFED 1.4.1 ib_srpt has symbol + dependency on
+ scst.ko....sigh. Later scst release do not export the required ib_srpt
+ symbol; stick with scst-1.0.0 until OFED ib_srpt is updated.- +
+Use out of the box scst defines which include (#undef +STRICT_SERIALIZING),
+
+'no' kernel mods are required for BLOCKIO access to /dev/sdb[123].
+
+cd scst-1.0.0.0
+make all
+make install
+
+cd OFED-1.4.1
+build OFED select #3 for 'all' OFED components
+ - no SRP loads in /etc/infiniband/openib.conf, edit prior to reboot.REBOOT.
+
+
+./LOAD & ./UNLOAD scripts are manual versions of what scstAdmin (separate scst +package) will do minus
+loading the OFED driver ib_srpt.
+
+SRP targets formatted from Windows using default NTFS allocation size.
+Partition size & numbering is derrived from local test conventions; your setup +will be different.
+
+/dev/sdb1 NTFS < 1GB
+/dev/sdb2 NTFS > 1GB
+/dev/sdb3 NTFS > sdb2+
Manual SRP Target LOAD script
+++#!/bin/sh + +GRP=Default + +if [ ! -e /proc/scsi_tgt ] ; then + echo -n Loading scst driver + modprobe scst + if [ $? -ne 0 ] ; then + echo + echo err $? modprobe scst + exit $? + fi + echo ...OK +fi + +if [ ! -e /proc/scsi_tgt/vdisk ] ; then + echo -n Loading scst_vdisk driver + modprobe scst_vdisk + if [ $? -ne 0 ] ; then + echo + echo err $? modprobe scst_vdisk + exit $? + fi + echo ...OK +fi + +if [ ! -e /proc/scsi_tgt/vdisk ] ; then + echo -n Loading scst_vdisk driver + modprobe scst_vdisk + if [ $? -ne 0 ] ; then + echo + echo err $? modprobe scst_vdisk + exit $? + fi + echo ...OK +fi + +fgrep -q ib_srpt /proc/modules +if [ $? -ne 0 ] ; then + modprobe ib_srpt + echo ib_srpt...OK +fi + +echo -n Open SRP devices srp[123] +echo "open srp1 /dev/sdb1 512 BLOCKIO" > /proc/scsi_tgt/vdisk/vdisk +if [ $? -ne 0 ] ; then + echo err $? open srp1 /dev/sdb1 + exit $? +fi +echo "open srp2 /dev/sdb2 512 BLOCKIO" > /proc/scsi_tgt/vdisk/vdisk +if [ $? -ne 0 ] ; then + echo err $? open srp2 /dev/sdb2 + exit $? +fi +echo "open srp3 /dev/sdb3 512 BLOCKIO" > /proc/scsi_tgt/vdisk/vdisk +if [ $? -ne 0 ] ; then + echo err $? open srp3 /dev/sdb3 + exit $? +fi +echo ...OK + +echo -n Set allowed hosts access... +echo "add *" > /proc/scsi_tgt/groups/Default/names +echo ...OK + +echo -n Adding targets srp[123] as LUNs [012] in group $GRP +echo "add srp1 0" > /proc/scsi_tgt/groups/Default/devices +if [ $? -ne 0 ] ; then + echo + echo err $? add srp1 0 + exit $? +fi +echo "add srp2 1" > /proc/scsi_tgt/groups/Default/devices +if [ $? -ne 0 ] ; then + echo + echo err $? add srp2 1 + exit $? +fi +echo "add srp3 2" > /proc/scsi_tgt/groups/Default/devices +if [ $? -ne 0 ] ; then + echo + echo err $? add srp3 2 + exit $? +fi +echo ...OK +++Manual SRP Target UNLOAD script
+#!/bin/sh + +if [ -w /proc/scsi_tgt/vdisk/vdisk ] ; then + echo -n Closing SRP Targets srp[321]... + echo "close srp3" > /proc/scsi_tgt/vdisk/vdisk + echo "close srp2" > /proc/scsi_tgt/vdisk/vdisk + echo "close srp1" > /proc/scsi_tgt/vdisk/vdisk + echo Done. +fi + +fgrep -q scst_vdisk /proc/modules +if [ $? -eq 0 ] ; then + modprobe -r scst_vdisk +fi + +fgrep -q ib_srpt /proc/modules +if [ $? -eq 0 ] ; then + modprobe -r ib_srpt +fi + +fgrep -q scst /proc/modules +if [ $? -eq 0 ] ; then + modprobe -r scst +fi ++<return-to-top>
-- 2.46.0