[IPOIB] Changes for Connected Mode (RFC 4755). So far tested on x64/2003/mthca platforms only.
Known issues:
- payload MTU sizes > 16k cause problems on receive side. That also hurts performance.
- With WSD registered system can crash under heavy load while handling IOCTLs.
- Interop. with Linux still questionable because of large payload MTU issue (since Linux reports only one size - 65520 bytes and not really run-time adjustable). Connection establish algorithm doesn't work properly with Linux.
The patch fixes accessing p_user_attr field which lead to access violation, the actual field is p_pnp_attr.
It also moves port checking to the head of the function stack.
The patch fixes accessing p_user_attr field which lead to access violation, the actual field is p_pnp_attr.
It also moves port checking to the head of the function stack.
stansmith [Fri, 29 Aug 2008 22:53:16 +0000 (22:53 +0000)]
[WinOF]
buildrelease.bat now uses a .pfx file for 'OpenFabrics Alliance' driver signing.
[IPOIB]
ipoib_port.h MAX_SEND_SGE was boosted from 8 (svn.1498) to 30 in svn.1499 with only x64 testing. Result was x86 and ia64 ipoib has been broken since svn.1499. IPoIB could not create a UD QP with 30 SGE entries for x86 & ia64 (mthca). I temp set MAX_SEND_SGE == 16 which works for all supported arches.
stansmith [Fri, 29 Aug 2008 16:53:44 +0000 (16:53 +0000)]
[WinOF] RC0
Migrated driver signing to use a .pfx file for OpenFabrics Alliance signing.
WIX_tools binaries updated to 2.0.5805.0 along with Makefile references
Added WIX_tools\README.txt to explain WIX binaries download & unpack.
CustomActions.vbs - remove dpinst.exe usage as it proved to be unstable with not enough testing.
stansmith [Wed, 27 Aug 2008 17:50:19 +0000 (17:50 +0000)]
[WinOF] cleanup - deleted inactive folders from branches\ to ease svn-update pain for WOF2-0
ConnectX
MTHCA
Ndi
RDMA
WOF1-1
Remember these folders can be retrieved from previous svn versions.
For Tavor, we currently reserve separate MPT and MTT space for FMRs to
avoid abusing the vmalloc space on 32 bit kernels. No such problem
exists on 64 bit kernels so let's not do it there.
This way we have a shared pool for MR and FMR resources, used on
demand. This will also make it possible to write MTTs for regular
regions directly from driver.
MTTs are allocated in non-cache-coherent memory, so we must give
reserved MTTs their own cache line, to prevent both device and
CPU from writing into the same cache line at the same time.
The reserved_mtts field has different meaning in Tavor and Arbel, so
we are wasting mtt entries on memfree. Fix the Arbel case to match
Tavor semantics.
mthca_table_find() will return the wrong address when the table entry
being searched for is exactly at the beginning of a sglist entry
(other than the first), because it uses >= when it should use >.
Example: assume we have 2 entries in scatterlist, 4K each, offset is
4K. The current code will return first entry + 4K when we really want
the second entry.
In particular this means mapping an FMR on a memfree HCA may end up
writing the page table into the wrong place, leading to memory
corruption and also causing the HCA to use an incorrect address
translation table.
stansmith [Tue, 26 Aug 2008 00:20:11 +0000 (00:20 +0000)]
[IBBUS] bus_port_mgr.c: resolve mysterious/unannounced port_guid_pkey --> port_guid_pkey_t typedef change.
bus_driver.c: missing function bus_rem_pkey() - hacked in missing stub routine in order to build.
stansmith [Thu, 21 Aug 2008 18:08:40 +0000 (18:08 +0000)]
[WinOF] support ND on all architectures. Even though ia64 is not technically supported, ndinstall.exe is installed via netipoib.inf, so x86 versions of ND files are utilized for ia64 until full ia64 support arrives.
ND is a selectable install feature; netipoib.inf always installs the base ND files. For WinSvr-2008 'ndinstall.exe -i' is auto-run, all other OS/arch flavors no auto-run.
Winverbs is a selectable install feature; off by default.
CustomActions.vbs - HCA install has changed to be simpler for the common case, a single HCA, by installing PCI\VEN_15B3 thus allowing the .inf file to deal with matching the correct Device ID.
For multiple mixed HCA installs (mthca & mlx4), VEN_15B3 is not unique, so the install reverts to the previous WinOF 1.1 install method of installing by device ID; with a small optimization being applied.
The 1.1 install method was to traverse a list of DeviceIDs installing until success or EOL; once for mlx4 and then again for mthca. The optimization is to query and install for the resident DeviceID. The idea is to optimize for success instead of accepting install failures until the resident DeviceID is found.
Additionally the installer doesn't have to hard-core ConnectX DeviceIDs, although mthca DeviceIDs are hard coded - I believe mthca DeviceIDs are not likely to be changing. The method is for mlx4, verify the DeviceID is not mthca and then install (future proof).
stansmith [Thu, 21 Aug 2008 16:08:42 +0000 (16:08 +0000)]
[CORE\IOU] added following files:
ib_iou.cdf - for driver signing
ib_iou.inx - for ibiou.sys installation; IOU relevant items
makefile.inx - for creation of and date/time stamping ib_iou.inf file.
Added to SOURCES files the rules required to create ib_iou.inf file.
stansmith [Thu, 21 Aug 2008 15:59:00 +0000 (15:59 +0000)]
[ND] added fre_svr-03_ia64 binaries, which are really fre_svr-03_x86 binaries.
replaced fre_svr-08_ia64 binaries with fre_svr-08_x86 binaries.
Reason for replacement is due to changes in netipoib.inf, now that ndinstall.exe is always installed, so when uninstalling, the installer doesn't know if ND was started or not (ndinstall.exe is always present) so it assumes a need to remove ND ala 'ndinstall -r'. Since ND is yet to be supported on ia64 and I did not want to special case ia64 in the uninstall path, it turns out the x86 ndinstall.exe executes fine on ia64.
stansmith [Fri, 15 Aug 2008 17:25:56 +0000 (17:25 +0000)]
[ND] added XP_x86 binaries to fake build env.
renamed objfre_svr* --> fre_svr* to make it easier to identify ND binary folders from build generated folders.
leonidk [Thu, 14 Aug 2008 18:45:08 +0000 (18:45 +0000)]
[MLX4] some fixes to support the mixed IB/Eth mode. (mlnx: 2947,2949)
The only mixed configuration, supported now, is port1-IB, port2-Eth.
To work with it one has to burn FW with parameter 'dpdp_en=true' in INI file, for example MHGH28-XSC_A4-A5.ini.
ib_query_ca returns now the number of IB- and not physical ports. So VSTAT will show only one port and IBBUS will create only one IPoIB adapter.
shefty [Wed, 13 Aug 2008 19:31:04 +0000 (19:31 +0000)]
The mlx4 hca driver should install as an InfiniBandHca class driver,
rather than inventing its own class. This allows filter drivers to
reside above both the mthca and mlx4 drivers.
Signed-off-by: Sean Hefty <sean.hefty@intel.com>
git-svn-id: svn://openib.tc.cornell.edu/gen1@1481 ad392aa1-c5ef-ae45-8dd8-e69d62a5ef86
leonidk [Tue, 12 Aug 2008 08:23:29 +0000 (08:23 +0000)]
[IBAL] Fix UAL_NDI_NOOP handling
The NOOP IOCTL allows the ND provider to complete requests asynchronously when a synchronous operation completes in user-land. This is used for memory registration and deregistration, as well as for connection establishment when ARP resolution is in progress (IBAT returns E_PENDING) in the NetworkDirect provider. In the latter case, the expectation is that the IOCTL will complete asynchronously with STATUS_TIMEOUT. However the code processes the IOCTL synchronously, so the IOCTL fails with an immediate error rather than an asynchronous error.
This patch completes the IRP so that it's completion will be reported asynchronously.
Signed-off-by: Fab Tillier <ftillier@microsoft.com>
git-svn-id: svn://openib.tc.cornell.edu/gen1@1479 ad392aa1-c5ef-ae45-8dd8-e69d62a5ef86
shefty [Mon, 11 Aug 2008 16:17:04 +0000 (16:17 +0000)]
mlx4: add winverbs support
Export the mlx4 channel interface via PnP query interface calls. This allows the driver to export its channel interface to multiple drivers, such as WinVerbs.
Signed-off-by: Sean Hefty <sean.hefty@intel.com>
git-svn-id: svn://openib.tc.cornell.edu/gen1@1477 ad392aa1-c5ef-ae45-8dd8-e69d62a5ef86
stansmith [Sat, 9 Aug 2008 00:11:54 +0000 (00:11 +0000)]
[ND] add fake ND build support; to be removed when ND src hits svn.
trunk/BuildRelease.bat
Turn down tracing volume
New Commands: assist in handling large # of WinOF installers.
msi-label {tag}
rename .msi files to WOF_os*_arch*tag; frequently svn version #
msi-del
delete .msi installers containing this tag
msi-dir
directory listing of current WinOF .msi installers
trunk\ulp\nd
dirs - added user to the build
user\ - ugly hack until ND sources show up in svn - pending legal issues with MS
SOURCES
fake.c
makefile
makefile.inc
objfre_svr-03_amd64 - ND binaries from Mellanox
objfre_svr-08_amd64
objfre_svr-03_x86
objfre_svr-08_x86
objfre_svr-08_ia64 - side-steps ia64 ipoib.cdf & ipoib.inx install issues; actually x64 copies.
trunk\ulp\ipoib\kernel
ipoib32.cdf - support ND signing for 32 bit systems
ipoib32-xp.cdf - support XP32 signing
ipoib.cdf - ND signing
netipoib.inx - ND install
netipoib-xp32.inf - XP32 install, no WSD/ND
stansmith [Wed, 6 Aug 2008 21:08:11 +0000 (21:08 +0000)]
[IBBUS] fixed duplicate bus relations reporting, which also fixed the problem of a required reboot. Now the IB stack can be uninstalled and installed without requiring a reboot.
leonidk [Mon, 4 Aug 2008 16:18:23 +0000 (16:18 +0000)]
[MLX4] added HCA Soft Reset mechanism (used by Ethernet interface so far)
Soft Reset here is HCA re-initialization without bus driver reloading.
A reset can be initiated by clients (mlx4_eth, mlx4_hca) and/or driver (mlx4_bus).
Driver issues reset upon card fatal error, which prevents the following work with the card.
Clients may request the reset at any moment upon their will.
Clients have to register event callback after getting bus interface.
When a reset event comes, the bus driver will:
- bar the following work with card, returning –EFAULT to all, but destroy_xx, commands;
- reset the card to stop incoming traffic (only in case of client-initiated reset);
- notify all registered clients about pending reset.
Getting this notification clients have to:
- wait for all issued commands to end;
- reset its own clients, if any, and bar their work;
- release all the device resources, they were using till now;
- send “I’m reset-ready” notification to the bus driver;
The driver starts to perform device reset only after receiving the “I’m reset-ready” notifications from all the registered clients. It re-initializes the device and notifies all the clients.
Having received this notification, clients have to:
- dereference the old bus interface;
- get the new interface from bus driver;
- register new event handler;
- resume/restart itself;
- wake up its own clients, if any;
Mellanox HCA cards do not have Power Management support so far.
So MLX4 driver resets the card on power-down and starts its back on power-up.
Pay attention, one can go to Standby/Hibernate only when there is no running IB applications (including WSD).
Otherwise he will get an error window with a message like: "Driver mlx4_hca.sys refuses to go Hibernate/Standby".
MTHCA driver behaves upon Hibernate/Standby the same way.
leonidk [Sun, 3 Aug 2008 08:44:29 +0000 (08:44 +0000)]
[OPENSM] adjust OPENSM to changed types of the stack.
To recall, OPENSM uses it's own pair of files ib_types.h/ib_types_extended.h instead of the stack's ib_types.h.
So all the changes of 'trunk\inc\iba\ib_types.h' should be repeated in the OPENSM's files.