MTTs are allocated in non-cache-coherent memory, so we must give
reserved MTTs their own cache line, to prevent both device and
CPU from writing into the same cache line at the same time.
The reserved_mtts field has different meaning in Tavor and Arbel, so
we are wasting mtt entries on memfree. Fix the Arbel case to match
Tavor semantics.
mthca_table_find() will return the wrong address when the table entry
being searched for is exactly at the beginning of a sglist entry
(other than the first), because it uses >= when it should use >.
Example: assume we have 2 entries in scatterlist, 4K each, offset is
4K. The current code will return first entry + 4K when we really want
the second entry.
In particular this means mapping an FMR on a memfree HCA may end up
writing the page table into the wrong place, leading to memory
corruption and also causing the HCA to use an incorrect address
translation table.
stansmith [Tue, 26 Aug 2008 00:20:11 +0000 (00:20 +0000)]
[IBBUS] bus_port_mgr.c: resolve mysterious/unannounced port_guid_pkey --> port_guid_pkey_t typedef change.
bus_driver.c: missing function bus_rem_pkey() - hacked in missing stub routine in order to build.
stansmith [Thu, 21 Aug 2008 18:08:40 +0000 (18:08 +0000)]
[WinOF] support ND on all architectures. Even though ia64 is not technically supported, ndinstall.exe is installed via netipoib.inf, so x86 versions of ND files are utilized for ia64 until full ia64 support arrives.
ND is a selectable install feature; netipoib.inf always installs the base ND files. For WinSvr-2008 'ndinstall.exe -i' is auto-run, all other OS/arch flavors no auto-run.
Winverbs is a selectable install feature; off by default.
CustomActions.vbs - HCA install has changed to be simpler for the common case, a single HCA, by installing PCI\VEN_15B3 thus allowing the .inf file to deal with matching the correct Device ID.
For multiple mixed HCA installs (mthca & mlx4), VEN_15B3 is not unique, so the install reverts to the previous WinOF 1.1 install method of installing by device ID; with a small optimization being applied.
The 1.1 install method was to traverse a list of DeviceIDs installing until success or EOL; once for mlx4 and then again for mthca. The optimization is to query and install for the resident DeviceID. The idea is to optimize for success instead of accepting install failures until the resident DeviceID is found.
Additionally the installer doesn't have to hard-core ConnectX DeviceIDs, although mthca DeviceIDs are hard coded - I believe mthca DeviceIDs are not likely to be changing. The method is for mlx4, verify the DeviceID is not mthca and then install (future proof).
stansmith [Thu, 21 Aug 2008 16:08:42 +0000 (16:08 +0000)]
[CORE\IOU] added following files:
ib_iou.cdf - for driver signing
ib_iou.inx - for ibiou.sys installation; IOU relevant items
makefile.inx - for creation of and date/time stamping ib_iou.inf file.
Added to SOURCES files the rules required to create ib_iou.inf file.
stansmith [Thu, 21 Aug 2008 15:59:00 +0000 (15:59 +0000)]
[ND] added fre_svr-03_ia64 binaries, which are really fre_svr-03_x86 binaries.
replaced fre_svr-08_ia64 binaries with fre_svr-08_x86 binaries.
Reason for replacement is due to changes in netipoib.inf, now that ndinstall.exe is always installed, so when uninstalling, the installer doesn't know if ND was started or not (ndinstall.exe is always present) so it assumes a need to remove ND ala 'ndinstall -r'. Since ND is yet to be supported on ia64 and I did not want to special case ia64 in the uninstall path, it turns out the x86 ndinstall.exe executes fine on ia64.
stansmith [Fri, 15 Aug 2008 17:25:56 +0000 (17:25 +0000)]
[ND] added XP_x86 binaries to fake build env.
renamed objfre_svr* --> fre_svr* to make it easier to identify ND binary folders from build generated folders.
leonidk [Thu, 14 Aug 2008 18:45:08 +0000 (18:45 +0000)]
[MLX4] some fixes to support the mixed IB/Eth mode. (mlnx: 2947,2949)
The only mixed configuration, supported now, is port1-IB, port2-Eth.
To work with it one has to burn FW with parameter 'dpdp_en=true' in INI file, for example MHGH28-XSC_A4-A5.ini.
ib_query_ca returns now the number of IB- and not physical ports. So VSTAT will show only one port and IBBUS will create only one IPoIB adapter.
shefty [Wed, 13 Aug 2008 19:31:04 +0000 (19:31 +0000)]
The mlx4 hca driver should install as an InfiniBandHca class driver,
rather than inventing its own class. This allows filter drivers to
reside above both the mthca and mlx4 drivers.
Signed-off-by: Sean Hefty <sean.hefty@intel.com>
git-svn-id: svn://openib.tc.cornell.edu/gen1@1481 ad392aa1-c5ef-ae45-8dd8-e69d62a5ef86
leonidk [Tue, 12 Aug 2008 08:23:29 +0000 (08:23 +0000)]
[IBAL] Fix UAL_NDI_NOOP handling
The NOOP IOCTL allows the ND provider to complete requests asynchronously when a synchronous operation completes in user-land. This is used for memory registration and deregistration, as well as for connection establishment when ARP resolution is in progress (IBAT returns E_PENDING) in the NetworkDirect provider. In the latter case, the expectation is that the IOCTL will complete asynchronously with STATUS_TIMEOUT. However the code processes the IOCTL synchronously, so the IOCTL fails with an immediate error rather than an asynchronous error.
This patch completes the IRP so that it's completion will be reported asynchronously.
Signed-off-by: Fab Tillier <ftillier@microsoft.com>
git-svn-id: svn://openib.tc.cornell.edu/gen1@1479 ad392aa1-c5ef-ae45-8dd8-e69d62a5ef86
shefty [Mon, 11 Aug 2008 16:17:04 +0000 (16:17 +0000)]
mlx4: add winverbs support
Export the mlx4 channel interface via PnP query interface calls. This allows the driver to export its channel interface to multiple drivers, such as WinVerbs.
Signed-off-by: Sean Hefty <sean.hefty@intel.com>
git-svn-id: svn://openib.tc.cornell.edu/gen1@1477 ad392aa1-c5ef-ae45-8dd8-e69d62a5ef86
stansmith [Sat, 9 Aug 2008 00:11:54 +0000 (00:11 +0000)]
[ND] add fake ND build support; to be removed when ND src hits svn.
trunk/BuildRelease.bat
Turn down tracing volume
New Commands: assist in handling large # of WinOF installers.
msi-label {tag}
rename .msi files to WOF_os*_arch*tag; frequently svn version #
msi-del
delete .msi installers containing this tag
msi-dir
directory listing of current WinOF .msi installers
trunk\ulp\nd
dirs - added user to the build
user\ - ugly hack until ND sources show up in svn - pending legal issues with MS
SOURCES
fake.c
makefile
makefile.inc
objfre_svr-03_amd64 - ND binaries from Mellanox
objfre_svr-08_amd64
objfre_svr-03_x86
objfre_svr-08_x86
objfre_svr-08_ia64 - side-steps ia64 ipoib.cdf & ipoib.inx install issues; actually x64 copies.
trunk\ulp\ipoib\kernel
ipoib32.cdf - support ND signing for 32 bit systems
ipoib32-xp.cdf - support XP32 signing
ipoib.cdf - ND signing
netipoib.inx - ND install
netipoib-xp32.inf - XP32 install, no WSD/ND
stansmith [Wed, 6 Aug 2008 21:08:11 +0000 (21:08 +0000)]
[IBBUS] fixed duplicate bus relations reporting, which also fixed the problem of a required reboot. Now the IB stack can be uninstalled and installed without requiring a reboot.
leonidk [Mon, 4 Aug 2008 16:18:23 +0000 (16:18 +0000)]
[MLX4] added HCA Soft Reset mechanism (used by Ethernet interface so far)
Soft Reset here is HCA re-initialization without bus driver reloading.
A reset can be initiated by clients (mlx4_eth, mlx4_hca) and/or driver (mlx4_bus).
Driver issues reset upon card fatal error, which prevents the following work with the card.
Clients may request the reset at any moment upon their will.
Clients have to register event callback after getting bus interface.
When a reset event comes, the bus driver will:
- bar the following work with card, returning –EFAULT to all, but destroy_xx, commands;
- reset the card to stop incoming traffic (only in case of client-initiated reset);
- notify all registered clients about pending reset.
Getting this notification clients have to:
- wait for all issued commands to end;
- reset its own clients, if any, and bar their work;
- release all the device resources, they were using till now;
- send “I’m reset-ready” notification to the bus driver;
The driver starts to perform device reset only after receiving the “I’m reset-ready” notifications from all the registered clients. It re-initializes the device and notifies all the clients.
Having received this notification, clients have to:
- dereference the old bus interface;
- get the new interface from bus driver;
- register new event handler;
- resume/restart itself;
- wake up its own clients, if any;
Mellanox HCA cards do not have Power Management support so far.
So MLX4 driver resets the card on power-down and starts its back on power-up.
Pay attention, one can go to Standby/Hibernate only when there is no running IB applications (including WSD).
Otherwise he will get an error window with a message like: "Driver mlx4_hca.sys refuses to go Hibernate/Standby".
MTHCA driver behaves upon Hibernate/Standby the same way.
leonidk [Sun, 3 Aug 2008 08:44:29 +0000 (08:44 +0000)]
[OPENSM] adjust OPENSM to changed types of the stack.
To recall, OPENSM uses it's own pair of files ib_types.h/ib_types_extended.h instead of the stack's ib_types.h.
So all the changes of 'trunk\inc\iba\ib_types.h' should be repeated in the OPENSM's files.
[HW] The patch add version stamping to mlx4_bus and mlx4_hca. [MLNX: 2871] Signed-off-by: Reuven Amitai <reuven@mellanox.co.il>
git-svn-id: svn://openib.tc.cornell.edu/gen1@1451 ad392aa1-c5ef-ae45-8dd8-e69d62a5ef86
libibverbs/tests: fix buffer overrun issue that caused random connection failures
libibverbs tests occasionally had failures connecting over sockets in order to exchange QP information. (The ping-pong tests failed more than the perftests.) The failure was caused by copying a port number into a buffer that was too small, then using it to look up an address. Increase the size of the buffer and make sure that it's NULL terminated.
Signed-off-by: Sean Hefty <sean.hefty@intel.com>
git-svn-id: svn://openib.tc.cornell.edu/gen1@1447 ad392aa1-c5ef-ae45-8dd8-e69d62a5ef86
uvp/av: change create/modify calls to take SGID index
The uvp requires the SGID index to create/modify address handles.
Modify the API and UVPs to take the SGID index as input. This avoids
the UVP needing to cache the CA attributes, which can lead to accessing
a stale GID table. The SGID index is passed into the UVP through the
resv1 field of the GRH.
Signed-off-by: Sean Hefty <sean.hefty@intel.com>
git-svn-id: svn://openib.tc.cornell.edu/gen1@1444 ad392aa1-c5ef-ae45-8dd8-e69d62a5ef86
Modify the channel interface to allow clients to register different
event handlers. This is in addition to the event handler specified when
opening the channel adapter. The latter can be replaced in a subsequent
patch, but is left as is to minimize changes.
This change allows the channel interface to support multiple direct
call clients, such as winverbs and ibal.
Signed-off-by: Sean Hefty <sean.hefty@intel.com>
git-svn-id: svn://openib.tc.cornell.edu/gen1@1443 ad392aa1-c5ef-ae45-8dd8-e69d62a5ef86
uvp/av: complete all processing in pre_create/destroy_av() calls
According to the documentation, if a UVP returns IB_VERBS_PROCESSING_DONE
from a pre() IOCTL call, then all verbs related processing has completed.
The mthca pre_create_av() and pre_destroy_av() uvp implementation can
both return IB_VERBS_PROCESSING_DONE, but requires that users still call
post_create_av() and post_destroy_av(). Fix the mthca UVP to complete all
processing in the pre() IOCTL calls when returning IB_VERBS_PROCESSING_DONE,
and update IBAL to remove unnecessary post() IOCTL calls.
Signed-off-by: Sean Hefty <sean.hefty@intel.com>
git-svn-id: svn://openib.tc.cornell.edu/gen1@1442 ad392aa1-c5ef-ae45-8dd8-e69d62a5ef86
The UVP controls the ci_umv_buf, but does not allocate the p_inout_buf if it
is non-zero. However, the uvp frees the buffer later. It should always
allocate the buffer, since the caller does not know the required size and
cannot allocate it on the uvp's behalf. This avoids an unneeded
initialization by the caller, and the if check by the uvp.
Signed-off-by: Sean Hefty <sean.hefty@intel.com>
git-svn-id: svn://openib.tc.cornell.edu/gen1@1441 ad392aa1-c5ef-ae45-8dd8-e69d62a5ef86
verbs: optimize work request/completion structures
Change work completion types to quickly allow a user to determine if a
completions is for a send or receive. Align work request and work
completion type values to be the same for consistency.
Update work request structure to avoid unnecessary internal padding and
allocating extra unused space for pointers and structures that can be
part of a union. Arrange structure layout to allow casting between
the UVP, IBAL, WinVerbs, and libibverbs work requests.
Update debug print functions to match changes.
Signed-off-by: Sean Hefty <sean.hefty@intel.com>
git-svn-id: svn://openib.tc.cornell.edu/gen1@1440 ad392aa1-c5ef-ae45-8dd8-e69d62a5ef86
The current uvp_get_interface() routine takes a structure as input.
Included in the structure is a version number.
A better solution to supporting multiple interfaces is to define
interfaces using GUIDs. This not only allows different versions of
the same interface (for example the UVP interface), but also provides
a mechanism that can be used to add new interfaces without affecting
existing ABIs or APIs.
This will make it easier to extend the existing interfaces in the future.
Signed-off-by: Sean Hefty <sean.hefty@intel.com>
git-svn-id: svn://openib.tc.cornell.edu/gen1@1438 ad392aa1-c5ef-ae45-8dd8-e69d62a5ef86
Allow polling for a list of work completions using an array of
completion structures, rather than a linked list. This avoids needing
to walk the list to link the structures together before calling poll,
which is a fast path operation.
A new completion structure is added that provides the qp_context
associated with a completion. This avoids changes to the existing
ULPs, while taking advantage of the underlying UVP capabilities.
Providing the qp_context is useful when dealing with SRQ.
Signed-off-by: Sean Hefty <sean.hefty@intel.com>
git-svn-id: svn://openib.tc.cornell.edu/gen1@1437 ad392aa1-c5ef-ae45-8dd8-e69d62a5ef86
uvp/interface: add support for reporting QP context in WCs
Extend the uvp_interface to include support for reporting the QP context in
work completions. The required changes are to allow specifying a qp_context
in the pre_create_qp() IOCTL call, and reporting the qp_context through the
work completion structure.
Signed-off-by: Sean Hefty <sean.hefty@intel.com>
git-svn-id: svn://openib.tc.cornell.edu/gen1@1436 ad392aa1-c5ef-ae45-8dd8-e69d62a5ef86
The underlying hardware drivers both support setting callbacks on a per
widget (CQ, QP, or SRQ) basis, but the verbs interface only allows one
global event handler and one completion handler for the HCA. Modify the
verbs interface to allow setting events callbacks directly on the HCA
objects. This allows the HCA to support multiple consumers of its
interface, and users can optimize their event handling based on the call.
For example, the HCA drivers can invoke callbacks directly to the ULPs
without the calls first being filtered by IBAL. This will improve
completion event reporting.
To minimize changes to the existing stack, the current event handler is
used for all events, regardless of type. Optimizations are left for a
separate patch.
Signed-off-by: Sean Hefty <sean.hefty@intel.com>
git-svn-id: svn://openib.tc.cornell.edu/gen1@1435 ad392aa1-c5ef-ae45-8dd8-e69d62a5ef86
Removing false Prefast alarms by:
Adding __analysis_assume() to ASSERT when DBG and _PREFAST_ are used for use in all the WinOF tree.
Adding __pragma(warning(suppress:6326)) to suppress constants compare warning (performance decrease is acceptable since this is chked code/ heavy windows event logging.
Signed by:Doron Tsur
[HW, TOOLS] Improve VSTAT output in case of a driver startup failure. [MLNX: 2819]
When low level drivers fail on startup, which usually happens upon an incorrect FW, they report to all the above stack a dev_id of the burning device ("livefish") and imitate such a device thereafter. It was done in order to allow to all the upper level drivers to start their work.
VSTAT shows in this case the dev_id of a "livefish", which is a bit misleading.
This patch fixes this behavior. VSTAT will now show the real dev_id. The drivers still imitate the "livefish" device. The detailed message about the startup failure can be found in System Event Log (as it always was).
[MLX4] added a possibility to register event handler with MLX4_BUS driver. [MLNX:2786]
It is an extension of the Ethernet interface of mlx4_bus driver.
The mlx4_eth handler is called on DPC level upon asynchronous events like PORT_UP/PORT_DOWN, passing event data and the callback context parameter, set upon registration.
uvp/av: change create/modify calls to take SGID index
The uvp requires the SGID index to create/modify address handles.
Modify the API and UVPs to take the SGID index as input. This avoids
the UVP needing to cache the CA attributes, which can lead to accessing
a stale GID table. The SGID index is passed into the UVP through the
resv1 field of the GRH.
This adds the patch to make this change.
Signed-off-by: Sean Hefty <sean.hefty@intel.com>
git-svn-id: svn://openib.tc.cornell.edu/gen1@1420 ad392aa1-c5ef-ae45-8dd8-e69d62a5ef86
[WinOF] buildrelease no longer contains uglt hack to work around XP32 WSD build problems.
Added a 2nd cmd line arg to set OPENIB_REV value when building installers. Fix eliminates folder structure dependency, making the script more useful to a wider audience.
[DAPL2] DAPL2 debug cleanup, along with patches to IBAL provider to correctly handle accept side private_data.
Fixed the async QP & CQ error handlers as they had not been updated with latest typedef's (ib_async_qp_handler_t & ib_async_cq_handler_t), hence calling arg mismatch vs. handling definitions. Resulted in bad EP pointer dereference during Async QP error handling.
Added debug value DAPL_DBG_EVD_DEQUEUE to throttle debug output when watching EVD dequeue operations.
The SGID index can change dynamically, so look up the correct index when creating an address vector. The index is passed into the UVP in the resv2 field of the grh.
Signed-off-by: Sean Hefty <sean.hefty@intel.com>
git-svn-id: svn://openib.tc.cornell.edu/gen1@1416 ad392aa1-c5ef-ae45-8dd8-e69d62a5ef86