shefty [Sat, 1 Aug 2009 01:02:29 +0000 (01:02 +0000)]
netdirect/winverbs: fixes to run nd sample tests
Now that the WV ND provider can be installed, the ND sample tests can be run. This provides various fixes to get at least a couple of the sample tests running.
Signed-off-by: Sean Hefty <sean.hefty@intel.com>
git-svn-id: svn://openib.tc.cornell.edu/gen1@2324 ad392aa1-c5ef-ae45-8dd8-e69d62a5ef86
shefty [Sat, 1 Aug 2009 01:01:57 +0000 (01:01 +0000)]
libibverbs: add reference counts to track CQ events
We need to track CQ events to block destroying CQ objects until all outstanding events have been acknowledged. Otherwise, the caller could reference freed memory after calling ibv_get_cq_event.
Signed-off-by: Sean Hefty <sean.hefty@intel.com>
git-svn-id: svn://openib.tc.cornell.edu/gen1@2323 ad392aa1-c5ef-ae45-8dd8-e69d62a5ef86
shefty [Sat, 1 Aug 2009 00:51:05 +0000 (00:51 +0000)]
libibverbs: add reference counts to track CQ events
We need to track CQ events to block destroying CQ objects until all outstanding events have been acknowledged. Otherwise, the caller could reference freed memory after calling ibv_get_cq_event.
Signed-off-by: Sean Hefty <sean.hefty@intel.com>
git-svn-id: svn://openib.tc.cornell.edu/gen1@2322 ad392aa1-c5ef-ae45-8dd8-e69d62a5ef86
shefty [Sat, 1 Aug 2009 00:45:47 +0000 (00:45 +0000)]
netdirect/winverbs: fixes to run nd sample tests
Now that the WV ND provider can be installed, the ND sample tests can be run. This provides various fixes to get at least a couple of the sample tests running.
Signed-off-by: Sean Hefty <sean.hefty@intel.com>
git-svn-id: svn://openib.tc.cornell.edu/gen1@2321 ad392aa1-c5ef-ae45-8dd8-e69d62a5ef86
Running the ndmw test results in a crash in the winverbs driver. The
crash is caused by improper cleanup in the winverbs driver as a result
of a failure trying to allocate a memory window.
In studying the crash, the same situation could arise if other objects
also fail to be created (QPs, CQs, PDs, etc.) To fix the crash and
simplify the code, explicit calls to used to increment the reference
count on referenced objects, rather than simply holding onto the
reference that was acquired when the object was validated. (The latter
is slightly more efficient, but results in less maintainable code
once any fix is applied.)
Signed-off-by: Sean Hefty <sean.hefty@intel.com>
git-svn-id: svn://openib.tc.cornell.edu/gen1@2320 ad392aa1-c5ef-ae45-8dd8-e69d62a5ef86
Running the ndmw test results in a crash in the winverbs driver. The
crash is caused by improper cleanup in the winverbs driver as a result
of a failure trying to allocate a memory window.
In studying the crash, the same situation could arise if other objects
also fail to be created (QPs, CQs, PDs, etc.) To fix the crash and
simplify the code, explicit calls to used to increment the reference
count on referenced objects, rather than simply holding onto the
reference that was acquired when the object was validated. (The latter
is slightly more efficient, but results in less maintainable code
once any fix is applied.)
Signed-off-by: Sean Hefty <sean.hefty@intel.com>
git-svn-id: svn://openib.tc.cornell.edu/gen1@2319 ad392aa1-c5ef-ae45-8dd8-e69d62a5ef86
[ND] remove old, unused fake build folders and fake.c from the days before we could build ND from src. makefile.inc modified to reflect folder changes.
[WINOF] add new HPC example scripts; added new WIX include file to create %ProgramFiles(x86)%\WinOF\HPC install folder and HPC examples scripts; change wof.wxs to include ..\..\HPC\HPC.inc .
DAPL introduced the concept of directly waiting on the CQ for
events by adding a compile time flag and special handling in the
common code. Rather than using the compile time flag and modifying the
common code, let the provider implement the best way to wait for
CQ events.
This simplifies the code and allows the common openib providers to
optimize for Linux and Windows platforms independently, rather than
assuming a specific implementation for signaling events.
Signed-off-by: Sean Hefty <sean.hefty@intel.com>
git-svn-id: svn://openib.tc.cornell.edu/gen1@2310 ad392aa1-c5ef-ae45-8dd8-e69d62a5ef86
[DAPL2] Implement a malloc() threshold for the completion reaping byte vector allocation in functions:
DT_handle_send_op, DT_handle_rdma_op & DT_handle_recv_op.
When allocation size is under the threshold, use a stack local allocation instead of malloc/free.
Move redundant bzero() to be called only in the case of using local stack allocation as DT_Mdep_malloc() already does a bzero().
Consolidate error handling return and free()check to a single point by using goto.
Signed-off-by: Stan Smith <stan.smith@intel.com>
git-svn-id: svn://openib.tc.cornell.edu/gen1@2305 ad392aa1-c5ef-ae45-8dd8-e69d62a5ef86
[MLX4] added new Registry parameter for number of FC_EXCHs and added a sanity check, that the number of reserved QPs is less then all requested QPs. [mlnx: 4562]
It came to fix a bug: if the number of reserved QPs is greater then the number of QPs requested, the bug causes eventual allocation of reserved QPs, which is forbidden.
The bug happened due two reasons:
- absense of sanity check, comparing reserved and total number of QPs;
- a large built-in number of FC_EXCH QPs, which are not in use at all today.
The patch replaces the built-in value to Registry taken one and set it to O.
It also changes the number of MACs from 1 to 0 for there is no need in reservation for MACs.
[MLX4] 1) added support for sending large multi-parameter messages to System Event Log. [mlnx: 4561]
2) improved the wording of several most popular messages.
To print to Event Log, one has to use the new function WriteEventLogEntryData().
[WinOF] Remove disucssions for Server 2003 & XP w.r.t. selecting HCA device driver to hardware - DIFxApp/PNP simplifies and unifies HCA selection; user selects HCA vendor; Mellanox is the default.
[DAPL2] fix a problem in socket-cm with FD_SETSIZE not being uniformly defined. Add UD test options to dtestcm. Use correct -D flags argument in dt-svr.bat, do not be silent in dt-cli stop as errors were discarded. Use trunk\etc\user\gtod.c for 'gettimeofday()' function.
leonidk [Sun, 28 Jun 2009 08:37:31 +0000 (08:37 +0000)]
[IBAL] remove unnecessary query_qp request during modify_qp
HCA API of modify_qp contains an optional parameter of qp_attr.
If it is coded, modify_qp makes internally a query_qp request to low-level driver and returns QP attributes.
IBAL codes this parameter, but do not use the returned QP attributes.
It causes unnecessary query_qp request to low-level driver and possible racy scenarios.
stansmith [Wed, 24 Jun 2009 19:11:33 +0000 (19:11 +0000)]
[WinOF] Make installs with NODRV=1 work as expected: *.wxs files observe NODRV as does customactions.vbs.
Consolidate standard 'features' into a single include file; XP non-support issues handled via WIX #ifdef's.
shefty [Wed, 24 Jun 2009 17:55:07 +0000 (17:55 +0000)]
etc/dlist: eliminate dlist.c
dlist.c only contains 2 functions. Remove it and implement the
functions inline in the header file. This simplifies the use of
dlist and matches the implementation of linux list.h.
Signed-off-by: Sean Hefty <sean.hefty@intel.com>
git-svn-id: svn://openib.tc.cornell.edu/gen1@2264 ad392aa1-c5ef-ae45-8dd8-e69d62a5ef86
stansmith [Mon, 22 Jun 2009 20:49:46 +0000 (20:49 +0000)]
[WinOF] remove DPInst.exe from SVN tree as svn.2261 etc\makebin.bat mods now copy DPInst.exe from WDK tree. Bottom line - having DPInst.exe in our svn tree was a poor choice on my part.
stansmith [Mon, 22 Jun 2009 20:45:17 +0000 (20:45 +0000)]
[WinOF] Use DIFxApp (Driver Install Frameworks for Applications) to install drivers instead of home-grown CustomActions using devman or dpinst.exe. DIFxApp mods resulted in rearranging WIX src files into separate as DIFxApp can not tolerate multiple .inf files in a single directory.
The big win is to use WDK supplied driver install functions which then allows all supported OS flavors to use the Driver Store which results in the user not having to choose which HCA model to install (specifically Server 2003 & XP); The 'default' driver choice is Mellanox HCA.
stansmith [Sat, 20 Jun 2009 23:18:13 +0000 (23:18 +0000)]
[WINOF] Change makebin.bat to expect a full pathname for the DDK root instead of just a DDK version; support those who install WDK in non-standard location.
leonidk [Wed, 17 Jun 2009 13:47:59 +0000 (13:47 +0000)]
[MLX4] return different MLX4 bus locations for several cards in multi-homed machine. [mlnx: 4468]
One can see on multi-homed computers, that mlx4_bus creates all its children (mlx4_hca, IPoIB et al) with built-in location string "MLX4 Bus 0" which complicates differentiation between them.
This patch makes mlx4_bus to return a location string of the structure "MLX4 bus x, device y, function z", where x,y,z is HCA card's PCI bus location.
shefty [Thu, 11 Jun 2009 04:11:24 +0000 (04:11 +0000)]
winverbs: remove QP from EP Disconnect call
The Endpoint class takes a QP parameter for Connect, Accept, and
Disconnect. To simplify the interface, store the QP internally and
remove it from the Disconnect call. Also allow it to be optional
for Accept on the active side of the connection.
Update librdmacm accordingly.
Signed-off-by: Sean Hefty <sean.hefty@intel.com>
git-svn-id: svn://openib.tc.cornell.edu/gen1@2247 ad392aa1-c5ef-ae45-8dd8-e69d62a5ef86
shefty [Thu, 11 Jun 2009 04:06:23 +0000 (04:06 +0000)]
winverbs: make PD derive from IWVOverlapped
Memory registration is asynchronous and part of the PD interface.
Make the PD derive from overlapped, rather than unknown base class, and
add support for canceling asynchronous operations (i.e. register
memory) on the PD.
This is an oversight of the initial API caught by implementing the ND
provider.
Signed-off-by: Sean Hefty <sean.hefty@intel.com>
git-svn-id: svn://openib.tc.cornell.edu/gen1@2246 ad392aa1-c5ef-ae45-8dd8-e69d62a5ef86
shefty [Thu, 11 Jun 2009 04:02:27 +0000 (04:02 +0000)]
winverbs: convert RDMA address to network-byte order
Change the winverbs interface to accept remote RDMA addresses in network
byte order. This keeps it consistent with all other remote/wire data fields.
Unfortunately, this means that the remote address must be swizzled between
winverbs and the HCA libraries, which take the address in host order, but
the other fields in network order. Longer term, the HCA libraries can provide
optimized interfaces for winverbs, like posting a single work request.
For now, to avoid compatibility issues later, define the winverbs interface
using what's most reasonable. Update libibverbs accordingly.
Signed-off-by: Sean Hefty <sean.hefty@intel.com>
git-svn-id: svn://openib.tc.cornell.edu/gen1@2245 ad392aa1-c5ef-ae45-8dd8-e69d62a5ef86
stansmith [Wed, 10 Jun 2009 17:11:17 +0000 (17:11 +0000)]
[DOCS] trunk\docs\Manual.htm
Document new winof dapl providers: socket-cm & rdma-cm
Explain dat.conf dapl provider device specification.
Incorporate BLOCKQUOTES so selected links appear at the top of the window
Document OFED Diagnostic tools - OFED man pages reworked.
Clarify Winverbs transition from technology preview to standard WinOF/IBcore component.
shefty [Tue, 9 Jun 2009 16:40:24 +0000 (16:40 +0000)]
winverbs: process connect and accept asynchronously
Allow processing of EP:Connect and EP:Accept calls asynchronously. The
librdmacm uses events to report the completion of rdma_connect and
rdma_accept calls, which allows users of that interface to take advantage
of asynchronous operation. Modify the winverbs kernel driver to
queue connect/accept calls to a system thread for better parallelism.
This improves the measured connection rate of rdma_cmatose by 3%. The
connection rate includes address resolution, route resolution, PD/CQ/QP
creation and state transitions, memory registration, posting of receive
buffers, and CM message exchanges. This patch effectively only improves
the parallelism of modify QP.
Signed-off-by: Sean Hefty <sean.hefty@intel.com>
git-svn-id: svn://openib.tc.cornell.edu/gen1@2240 ad392aa1-c5ef-ae45-8dd8-e69d62a5ef86
shefty [Tue, 9 Jun 2009 16:40:06 +0000 (16:40 +0000)]
winverbs: process connect and accept asynchronously
Allow processing of EP:Connect and EP:Accept calls asynchronously. The
librdmacm uses events to report the completion of rdma_connect and
rdma_accept calls, which allows users of that interface to take advantage
of asynchronous operation. Modify the winverbs kernel driver to
queue connect/accept calls to a system thread for better parallelism.
This improves the measured connection rate of rdma_cmatose by 3%. The
connection rate includes address resolution, route resolution, PD/CQ/QP
creation and state transitions, memory registration, posting of receive
buffers, and CM message exchanges. This patch effectively only improves
the parallelism of modify QP.
Signed-off-by: Sean Hefty <sean.hefty@intel.com>
git-svn-id: svn://openib.tc.cornell.edu/gen1@2239 ad392aa1-c5ef-ae45-8dd8-e69d62a5ef86
shefty [Thu, 4 Jun 2009 18:51:56 +0000 (18:51 +0000)]
netdirect: replace 'localhost' with '..localmachine'
Use of 'localhost' does not work with getaddrinfo to return all available IP addresses, but '..localmachine' does. A bug was submitted to MS regarding the issue.
Signed-off-by: Sean Hefty <sean.hefty@intel.com>
git-svn-id: svn://openib.tc.cornell.edu/gen1@2235 ad392aa1-c5ef-ae45-8dd8-e69d62a5ef86
shefty [Wed, 3 Jun 2009 17:44:16 +0000 (17:44 +0000)]
winverbs: export WvGetObject as extern C
If a standard C program or library tries to link against winverbs, it will fail
because of C++ name decoration. Export the WvGetObject call using C naming
conventions.
This allows DAPL to link directly against winverbs.
Signed-off-by: Sean Hefty <sean.hefty@intel.com>
git-svn-id: svn://openib.tc.cornell.edu/gen1@2232 ad392aa1-c5ef-ae45-8dd8-e69d62a5ef86
shefty [Wed, 3 Jun 2009 17:43:33 +0000 (17:43 +0000)]
winverbs: export WvGetObject as extern C
If a standard C program or library tries to link against winverbs, it will fail
because of C++ name decoration. Export the WvGetObject call using C naming
conventions.
This allows DAPL to link directly against winverbs.
Signed-off-by: Sean Hefty <sean.hefty@intel.com>
git-svn-id: svn://openib.tc.cornell.edu/gen1@2231 ad392aa1-c5ef-ae45-8dd8-e69d62a5ef86
leonidk [Tue, 2 Jun 2009 17:49:32 +0000 (17:49 +0000)]
[IBAL] work around for reference count leakage bugs. [mlnx: 4404]
IBAL still has bugs, which cause reference count leakage, which stops the cascading destroying resources of IBAL.
It causes in turn a freeze of IBBUS on HCA disable or system power down.
On checked builds IBAL forces destroying of the objects after some timeout.
On free version it waits endlessly.
This patch makes the behavior of free version to be like in checked version while sending a message to System Event Log.
leonidk [Tue, 2 Jun 2009 17:48:10 +0000 (17:48 +0000)]
[IBBUS] fix bug in Control device implementation (patches 4218,4280). [mlnx: 4396]
Control Device was created in DriverEntry and removed in DrvUnload routine.
But PnP Manager won't call DrvUnload before Control Device is removed.
So IBBUS gets never unloaded.
leonidk [Tue, 2 Jun 2009 17:32:41 +0000 (17:32 +0000)]
[IBAL] Summary: Ill-defined mechanism of event propagation.[mlnx: 4412]
Bug description and reproduction:
1. Connect to machines (A and B) via IB switch
2. Run subnet manager (say, opensm) on B
3. Kill opensm and clear arp tables
4. Rerun opensm - ping will not longer work
5. That's because new opensm instance will clear old multicast groups, and side A will be not aware about opensm restart and will not request to join new MCAST group
Explanations:
There are 2 types of events relevant in our case: PnP and AE.
The problem had happened due to:
1. During opensm restart, port will generate AE event: IB_EVENT_LID_CHANGE or (in other cases) IB_EVENT_CLIENT_REREGISTER
These events will be generated even in the case when SM was restarted, but LID will not actually change.
2. All PnP events were handled properly; but these events were mapped to IB_AE_FATAL
This patch fixes it and maps IB_EVENT_* events to appropriate IB_AE_* events and then to IB_PNP_* events
3. unction force_smi_poll() will now update it's subscribers about LID change event iff LID was changed.
So, we still have the problem when opensm was restarted and no one of the port attributes was changed.
This patch generated appropriate IB_PNP event to resolve this issue.