shefty [Wed, 26 Aug 2009 16:45:53 +0000 (16:45 +0000)]
librdmacm: fix race in ucma_init()
There's a potential race with ucma_init() and calls that check whether
the library is ready for use. For example, in rdma_create_id, this
check is performed:
Since the check for cma_dev_cnt is outside of any synchronization, if it
is non-zeroy, then the code will assume that initialization is complete.
However, ucma_init() can set cma_dev_cnt to non-zeroy before it has finished
executing. The full impact of this race is unknown, but it can't be good.
In the worst case, the application may crash.
Fix this by ensuring that all initialization is complete before cma_dev_cnt
is incremented. Note that ucma_init() provides synchronization to protect
against duplicate threads handling init. The cma_dev_cnt check is used to
avoid synchronization on multiple calls into the library after initialization.
Signed-off-by: Sean Hefty <sean.hefty@intel.com>
git-svn-id: svn://openib.tc.cornell.edu/gen1@2383 ad392aa1-c5ef-ae45-8dd8-e69d62a5ef86
shefty [Wed, 26 Aug 2009 16:44:24 +0000 (16:44 +0000)]
ofed libs: prevent double initialization of critical section
DllMain is called multiple times for a given process. Prevent double
initialization of critical sections by only initializing it during process
attach. This avoids corrupting the critical section while it may be in use.
Signed-off-by: Sean Hefty <sean.hefty@intel.com>
git-svn-id: svn://openib.tc.cornell.edu/gen1@2382 ad392aa1-c5ef-ae45-8dd8-e69d62a5ef86
stansmith [Wed, 19 Aug 2009 21:49:50 +0000 (21:49 +0000)]
[WIX] update usage instructions to reflect WinOF is now under Trunk\ and no longer under branches\ . Include win7\x64\wof.wxs as I missed it in the last commit.
shefty [Tue, 18 Aug 2009 19:22:47 +0000 (19:22 +0000)]
ib/port_attr: report active width as part of port attributes
Return the active width in the port attributes reported by the HCA.
Because the kernel and user space share the same source
files, to avoid breaking the ABI, rename link_width_supported
field to active_width and use that field to return the value.
link_width_supported is currently uninitialized and in the
svn tree unused, so repurposing the field should have minimal
impact. However, this change will break the API.
Applications that reference the link_width_supported field
from struct _ib_port_attr will require a minor update if recompiled.
Update winverbs to report this value and fill in other
missing fields from the port attribute structure. This fix
allows reporting the correct values through the libibverbs
port and ibv_devinfo example utility.
Signed-off-by: Sean Hefty <sean.hefty@intel.com>
git-svn-id: svn://openib.tc.cornell.edu/gen1@2366 ad392aa1-c5ef-ae45-8dd8-e69d62a5ef86
stansmith [Mon, 17 Aug 2009 23:17:41 +0000 (23:17 +0000)]
[IPoIB] IPoIB PXE boot support: Clear remainder of chaddr
The IPoIB PXE boot firmware (gPXE) now sends the 8-byte port GUID in
the DHCP chaddr field. WinOF replaces the first 6 bytes of chaddr
with the Ethernet-style MAC address, but leaves the remainder
untouched. This results in trailing garbage after the Ethernet-style
MAC in the modified chaddr.
Fix by explicitly zeroing the remainder of chaddr.
Signed-off-by: Michael Brown <mcb30@etherboot.org>
git-svn-id: svn://openib.tc.cornell.edu/gen1@2364 ad392aa1-c5ef-ae45-8dd8-e69d62a5ef86
leonidk [Sun, 16 Aug 2009 12:44:11 +0000 (12:44 +0000)]
[ND provider] This patch is a fix to 2333. It removes a facility to define MaxDataInlineSize from application, because it breaks MS API
[ND porvider] Improved latency of ND provider by using INLINE send. [winof: 2333, 2352]
This patch adds usage of INLINE DATA facility of Mellanox HCAs for improving latency of ND provider.
Here are the ideas of the patch:
- by default, ND provider will create QP with inline data of 160 bytes;
(this can enlarge user's QP size)
- one can change this default by defining environment variable IBNDPROV_MAX_INLINE_SIZE;
leonidk [Sun, 16 Aug 2009 12:40:46 +0000 (12:40 +0000)]
[ND provider] This patch is a fix to 2333. It removes a facility to define MaxDataInlineSize from application, because it breaks MS API
Improved latency of ND provider by using INLINE send
This patch adds usage of INLINE DATA facility of Mellanox HCAs for improving latency of ND provider.
Here are the ideas of the patch:
- by default, ND provider will create QP with inline data of 160 bytes;
(this can enlarge user's QP size)
- one can change this default by defining environment variable IBNDPROV_MAX_INLINE_SIZE;
stansmith [Thu, 13 Aug 2009 17:34:29 +0000 (17:34 +0000)]
[IPOIB] Prevent a BSOD which happens when restarting the opensm more than once (if the local endpoint was not in the lid_endpts list). signed off by stan.smith@intel.com & Tzachi Dar [tzachid@mellanox.co.il]
shefty [Mon, 10 Aug 2009 18:55:38 +0000 (18:55 +0000)]
winverbs/driver: fix check for IP version
merge commit 2339/2340
The IP version is in the upper 4 bits of an 8 bit field. It is set
correctly, but not ready out correctly. The result is that an IPv4
address reported to user space is listed as an IPv6 address instead.
Signed-off-by: Sean Hefty <sean.hefty@intel.com>
git-svn-id: svn://openib.tc.cornell.edu/gen1@2350 ad392aa1-c5ef-ae45-8dd8-e69d62a5ef86
shefty [Mon, 10 Aug 2009 18:49:25 +0000 (18:49 +0000)]
winverbs/driver: fix check for IP version (reapply overwritten commit)
The IP version is in the upper 4 bits of an 8 bit field. It is set
correctly, but not ready out correctly. The result is that an IPv4
address reported to user space is listed as an IPv6 address instead.
Signed-off-by: Sean Hefty <sean.hefty@intel.com>
git-svn-id: svn://openib.tc.cornell.edu/gen1@2348 ad392aa1-c5ef-ae45-8dd8-e69d62a5ef86
shefty [Mon, 10 Aug 2009 18:44:36 +0000 (18:44 +0000)]
winverbs/driver: change status for rejected connections
merge commit 2339
Complete connection requests with STATUS_CONNECTION_REFUSED rather
than STATUS_REQUEST_NOT_ACCEPTED for rejected connections. This aligns
the status value with ND.
Signed-off-by: Sean Hefty <sean.hefty@intel.com>
git-svn-id: svn://openib.tc.cornell.edu/gen1@2347 ad392aa1-c5ef-ae45-8dd8-e69d62a5ef86
shefty [Mon, 10 Aug 2009 18:43:51 +0000 (18:43 +0000)]
ibal/cm: allow RDMA writes for QP attributes by default
merge commit 2334
Enable remote RDMA writes as the default QP attributes for INIT
transitions. Previously, RDMA writes were only enabled if the CM
messages carried a responder resources > 0. However, responder
resources indicates whether RDMA reads/atomics should be enabled,
not RDMA writes.
Without this patch, users of winverbs that connect in-band that
want to perform RDMA writes without performing RDMA reads will
see QP errors when performing writes.
Signed-off-by: Sean Hefty <sean.hefty@intel.com>
git-svn-id: svn://openib.tc.cornell.edu/gen1@2346 ad392aa1-c5ef-ae45-8dd8-e69d62a5ef86
shefty [Mon, 10 Aug 2009 18:41:43 +0000 (18:41 +0000)]
libibverbs: add reference counts to track CQ events
merge commit 2323
We need to track CQ events to block destroying CQ objects until all outstanding events have been acknowledged. Otherwise, the caller could reference freed memory after calling ibv_get_cq_event.
Signed-off-by: Sean Hefty <sean.hefty@intel.com>
git-svn-id: svn://openib.tc.cornell.edu/gen1@2344 ad392aa1-c5ef-ae45-8dd8-e69d62a5ef86
shefty [Mon, 10 Aug 2009 18:40:06 +0000 (18:40 +0000)]
winverbs: fix cleanup in error handling cases
merge svn 2320 from trunk
Running the ndmw test results in a crash in the winverbs driver. The
crash is caused by improper cleanup in the winverbs driver as a result
of a failure trying to allocate a memory window.
In studying the crash, the same situation could arise if other objects
also fail to be created (QPs, CQs, PDs, etc.) To fix the crash and
simplify the code, explicit calls to used to increment the reference
count on referenced objects, rather than simply holding onto the
reference that was acquired when the object was validated. (The latter
is slightly more efficient, but results in less maintainable code
once any fix is applied.)
Signed-off-by: Sean Hefty <sean.hefty@intel.com>
git-svn-id: svn://openib.tc.cornell.edu/gen1@2343 ad392aa1-c5ef-ae45-8dd8-e69d62a5ef86
shefty [Mon, 10 Aug 2009 17:22:32 +0000 (17:22 +0000)]
winverbs/driver: change status for rejected connections
Complete connection requests with STATUS_CONNECTION_REFUSED rather
than STATUS_REQUEST_NOT_ACCEPTED for rejected connections. This aligns
the status value with ND.
Signed-off-by: Sean Hefty <sean.hefty@intel.com>
git-svn-id: svn://openib.tc.cornell.edu/gen1@2341 ad392aa1-c5ef-ae45-8dd8-e69d62a5ef86
shefty [Sat, 8 Aug 2009 00:21:54 +0000 (00:21 +0000)]
winverbs/driver: fix check for IP version
The IP version is in the upper 4 bits of an 8 bit field. It is set
correctly, but not ready out correctly. The result is that an IPv4
address reported to user space is listed as an IPv6 address instead.
Signed-off-by: Sean Hefty <sean.hefty@intel.com>
git-svn-id: svn://openib.tc.cornell.edu/gen1@2340 ad392aa1-c5ef-ae45-8dd8-e69d62a5ef86
shefty [Sat, 8 Aug 2009 00:20:44 +0000 (00:20 +0000)]
winverbs/driver: change status for rejected connections
Complete connection requests with STATUS_CONNECTION_REFUSED rather
than STATUS_REQUEST_NOT_ACCEPTED for rejected connections. This aligns
the status value with ND.
Signed-off-by: Sean Hefty <sean.hefty@intel.com>
git-svn-id: svn://openib.tc.cornell.edu/gen1@2339 ad392aa1-c5ef-ae45-8dd8-e69d62a5ef86
stansmith [Fri, 7 Aug 2009 19:47:44 +0000 (19:47 +0000)]
[ETC] The completion manager abstraction can hang during cleanup. The hang
occurs when the user calls CompManagerCancel before calling CompManagerClose,
and the user is not waiting for events. In this case, the completion manager
thread will pull the cancel request from the IO completion port and queue it
with the manager. When CompManagerClose is called, it calls CompManagerCancel
to signal the thread to check the running state. However, the completion
manager's event structure is still marked busy from the user's CompManagerCancel
call.
The result is that the completion manager thread does not receive the
signal to check the running flag and remains asleep. Fix this by using a
different completion entry to signal the thread during destruction than that
used to cancel a CompManagerPoll event.
This fixes occasional hangs running dapltest with both the rdma_cm and socket
cm providers.
Signed-off-by: Sean Hefty <sean.hefty@intel.com>
git-svn-id: svn://openib.tc.cornell.edu/gen1@2336 ad392aa1-c5ef-ae45-8dd8-e69d62a5ef86
shefty [Fri, 7 Aug 2009 18:49:07 +0000 (18:49 +0000)]
ibal/cm: allow RDMA writes for QP attributes by default
Enable remote RDMA writes as the default QP attributes for INIT
transitions. Previously, RDMA writes were only enabled if the CM
messages carried a responder resources > 0. However, responder
resources indicates whether RDMA reads/atomics should be enabled,
not RDMA writes.
Without this patch, users of winverbs that connect in-band that
want to perform RDMA writes without performing RDMA reads will
see QP errors when performing writes.
Signed-off-by: Sean Hefty <sean.hefty@intel.com>
git-svn-id: svn://openib.tc.cornell.edu/gen1@2335 ad392aa1-c5ef-ae45-8dd8-e69d62a5ef86
shefty [Fri, 7 Aug 2009 18:48:52 +0000 (18:48 +0000)]
ibal/cm: allow RDMA writes for QP attributes by default
Enable remote RDMA writes as the default QP attributes for INIT
transitions. Previously, RDMA writes were only enabled if the CM
messages carried a responder resources > 0. However, responder
resources indicates whether RDMA reads/atomics should be enabled,
not RDMA writes.
Without this patch, users of winverbs that connect in-band that
want to perform RDMA writes without performing RDMA reads will
see QP errors when performing writes.
Signed-off-by: Sean Hefty <sean.hefty@intel.com>
git-svn-id: svn://openib.tc.cornell.edu/gen1@2334 ad392aa1-c5ef-ae45-8dd8-e69d62a5ef86
leonidk [Thu, 6 Aug 2009 17:13:25 +0000 (17:13 +0000)]
[ND provider] improved latency of ND provider by using INLINE send
This patch adds usage of INLINE DATA facility of Mellanox HCAs for improving latency of ND provider.
Here are the ideas of the patch:
- by default, ND provider will create QP with inline data of 160 bytes;
(this can enlarge user's QP size)
- one can change this default by defining environment variable IBNDPROV_MAX_INLINE_SIZE;
- an ND application, while creating QP, can define the necessary INLINE DATA size; this value takes precedence over the default one.
shefty [Wed, 5 Aug 2009 23:43:36 +0000 (23:43 +0000)]
libibverbs/comp_mgr: fix hang during destruction
The completion manager abstraction can hang during cleanup. The hang
occurs when the user calls CompManagerCancel before calling CompManagerClose,
and the user is not waiting for events. In this case, the completion manager
thread will pull the cancel request from the IO completion port and queue it
with the manager. When CompManagerClose is called, it calls CompManagerCancel
to signal the thread to check the running state. However, the completion
manager's event structure is still marked busy from the user's CompManagerCancel
call.
The result is that the completion manager thread does not receive the
signal to check the running flag and remains asleep. Fix this by using a
different completion entry to signal the thread during destruction than that
used to cancel a CompManagerPoll event.
This fixes occasional hangs running dapltest with both the rdma_cm and socket
cm providers.
Signed-off-by: Sean Hefty <sean.hefty@intel.com>
git-svn-id: svn://openib.tc.cornell.edu/gen1@2330 ad392aa1-c5ef-ae45-8dd8-e69d62a5ef86
shefty [Wed, 5 Aug 2009 23:43:10 +0000 (23:43 +0000)]
libibverbs/comp_mgr: fix hang during destruction
The completion manager abstraction can hang during cleanup. The hang
occurs when the user calls CompManagerCancel before calling CompManagerClose,
and the user is not waiting for events. In this case, the completion manager
thread will pull the cancel request from the IO completion port and queue it
with the manager. When CompManagerClose is called, it calls CompManagerCancel
to signal the thread to check the running state. However, the completion
manager's event structure is still marked busy from the user's CompManagerCancel
call.
The result is that the completion manager thread does not receive the
signal to check the running flag and remains asleep. Fix this by using a
different completion entry to signal the thread during destruction than that
used to cancel a CompManagerPoll event.
This fixes occasional hangs running dapltest with both the rdma_cm and socket
cm providers.
Signed-off-by: Sean Hefty <sean.hefty@intel.com>
git-svn-id: svn://openib.tc.cornell.edu/gen1@2329 ad392aa1-c5ef-ae45-8dd8-e69d62a5ef86
stansmith [Tue, 4 Aug 2009 17:13:27 +0000 (17:13 +0000)]
[DAPL2] Fix socket CM completion handling and CNQ handling; Windows variants. Add #include <winsock2.h>
#include <ws2tcpip.h> for IPv4 & IPv6 user application support (VC env vs. WDK).
shefty [Sat, 1 Aug 2009 01:02:29 +0000 (01:02 +0000)]
netdirect/winverbs: fixes to run nd sample tests
Now that the WV ND provider can be installed, the ND sample tests can be run. This provides various fixes to get at least a couple of the sample tests running.
Signed-off-by: Sean Hefty <sean.hefty@intel.com>
git-svn-id: svn://openib.tc.cornell.edu/gen1@2324 ad392aa1-c5ef-ae45-8dd8-e69d62a5ef86
shefty [Sat, 1 Aug 2009 01:01:57 +0000 (01:01 +0000)]
libibverbs: add reference counts to track CQ events
We need to track CQ events to block destroying CQ objects until all outstanding events have been acknowledged. Otherwise, the caller could reference freed memory after calling ibv_get_cq_event.
Signed-off-by: Sean Hefty <sean.hefty@intel.com>
git-svn-id: svn://openib.tc.cornell.edu/gen1@2323 ad392aa1-c5ef-ae45-8dd8-e69d62a5ef86
shefty [Sat, 1 Aug 2009 00:51:05 +0000 (00:51 +0000)]
libibverbs: add reference counts to track CQ events
We need to track CQ events to block destroying CQ objects until all outstanding events have been acknowledged. Otherwise, the caller could reference freed memory after calling ibv_get_cq_event.
Signed-off-by: Sean Hefty <sean.hefty@intel.com>
git-svn-id: svn://openib.tc.cornell.edu/gen1@2322 ad392aa1-c5ef-ae45-8dd8-e69d62a5ef86
shefty [Sat, 1 Aug 2009 00:45:47 +0000 (00:45 +0000)]
netdirect/winverbs: fixes to run nd sample tests
Now that the WV ND provider can be installed, the ND sample tests can be run. This provides various fixes to get at least a couple of the sample tests running.
Signed-off-by: Sean Hefty <sean.hefty@intel.com>
git-svn-id: svn://openib.tc.cornell.edu/gen1@2321 ad392aa1-c5ef-ae45-8dd8-e69d62a5ef86
Running the ndmw test results in a crash in the winverbs driver. The
crash is caused by improper cleanup in the winverbs driver as a result
of a failure trying to allocate a memory window.
In studying the crash, the same situation could arise if other objects
also fail to be created (QPs, CQs, PDs, etc.) To fix the crash and
simplify the code, explicit calls to used to increment the reference
count on referenced objects, rather than simply holding onto the
reference that was acquired when the object was validated. (The latter
is slightly more efficient, but results in less maintainable code
once any fix is applied.)
Signed-off-by: Sean Hefty <sean.hefty@intel.com>
git-svn-id: svn://openib.tc.cornell.edu/gen1@2320 ad392aa1-c5ef-ae45-8dd8-e69d62a5ef86
Running the ndmw test results in a crash in the winverbs driver. The
crash is caused by improper cleanup in the winverbs driver as a result
of a failure trying to allocate a memory window.
In studying the crash, the same situation could arise if other objects
also fail to be created (QPs, CQs, PDs, etc.) To fix the crash and
simplify the code, explicit calls to used to increment the reference
count on referenced objects, rather than simply holding onto the
reference that was acquired when the object was validated. (The latter
is slightly more efficient, but results in less maintainable code
once any fix is applied.)
Signed-off-by: Sean Hefty <sean.hefty@intel.com>
git-svn-id: svn://openib.tc.cornell.edu/gen1@2319 ad392aa1-c5ef-ae45-8dd8-e69d62a5ef86
[ND] remove old, unused fake build folders and fake.c from the days before we could build ND from src. makefile.inc modified to reflect folder changes.
[WINOF] add new HPC example scripts; added new WIX include file to create %ProgramFiles(x86)%\WinOF\HPC install folder and HPC examples scripts; change wof.wxs to include ..\..\HPC\HPC.inc .
DAPL introduced the concept of directly waiting on the CQ for
events by adding a compile time flag and special handling in the
common code. Rather than using the compile time flag and modifying the
common code, let the provider implement the best way to wait for
CQ events.
This simplifies the code and allows the common openib providers to
optimize for Linux and Windows platforms independently, rather than
assuming a specific implementation for signaling events.
Signed-off-by: Sean Hefty <sean.hefty@intel.com>
git-svn-id: svn://openib.tc.cornell.edu/gen1@2310 ad392aa1-c5ef-ae45-8dd8-e69d62a5ef86
[DAPL2] Implement a malloc() threshold for the completion reaping byte vector allocation in functions:
DT_handle_send_op, DT_handle_rdma_op & DT_handle_recv_op.
When allocation size is under the threshold, use a stack local allocation instead of malloc/free.
Move redundant bzero() to be called only in the case of using local stack allocation as DT_Mdep_malloc() already does a bzero().
Consolidate error handling return and free()check to a single point by using goto.
Signed-off-by: Stan Smith <stan.smith@intel.com>
git-svn-id: svn://openib.tc.cornell.edu/gen1@2305 ad392aa1-c5ef-ae45-8dd8-e69d62a5ef86
[MLX4] added new Registry parameter for number of FC_EXCHs and added a sanity check, that the number of reserved QPs is less then all requested QPs. [mlnx: 4562]
It came to fix a bug: if the number of reserved QPs is greater then the number of QPs requested, the bug causes eventual allocation of reserved QPs, which is forbidden.
The bug happened due two reasons:
- absense of sanity check, comparing reserved and total number of QPs;
- a large built-in number of FC_EXCH QPs, which are not in use at all today.
The patch replaces the built-in value to Registry taken one and set it to O.
It also changes the number of MACs from 1 to 0 for there is no need in reservation for MACs.
[MLX4] 1) added support for sending large multi-parameter messages to System Event Log. [mlnx: 4561]
2) improved the wording of several most popular messages.
To print to Event Log, one has to use the new function WriteEventLogEntryData().
[WinOF] Remove disucssions for Server 2003 & XP w.r.t. selecting HCA device driver to hardware - DIFxApp/PNP simplifies and unifies HCA selection; user selects HCA vendor; Mellanox is the default.
projects / ~shefty / rdma-win.git / log