librdmacm currently opens a device context per configured HCA. This is
usually done in rdma_create_event_channel() or first time whenever
ucma_init() is called. If a process is only going to use one of the
configured HCAs/RDMA IPs then the remaining device contexts are not
used/required. Opening a device context on each device apriori limits the
maximum number of processes that can be supported on a node to the maximum
number of open context supported per HCA regardless of number of HCAs present
in the system.
Signed-off-by: Shamir Rabinovitch <shamir.rabinovitch@oracle.com> Signed-off-by: Sean Hefty <sean.hefty@intel.com>
librdmacm currently opens a device context per configured HCA. This is
usually done in rdma_create_event_channel() or first time whenever
ucma_init() is called. If a process is only going to use one of the
configured HCAs/RDMA IPs then the remaining device contexts are not
used/required. Opening a device context on each device apriori limits the
maximum number of processes that can be supported on a node to the maximum
number of open context supported per HCA regardless of number of HCAs present
in the system.
Signed-off-by: Shamir Rabinovitch <shamir.rabinovitch@oracle.com> Signed-off-by: Sean Hefty <sean.hefty@intel.com>
librdmacm currently opens a device context per configured HCA. This is
usually done in rdma_create_event_channel() or first time whenever
ucma_init() is called. If a process is only going to use one of the
configured HCAs/RDMA IPs then the remaining device contexts are not
used/required. Opening a device context on each device apriori limits the
maximum number of processes that can be supported on a node to the maximum
number of open context supported per HCA regardless of number of HCAs present
in the system.
Signed-off-by: Shamir Rabinovitch <shamir.rabinovitch@oracle.com> Signed-off-by: Sean Hefty <sean.hefty@intel.com>
librdmacm currently opens a device context per configured HCA. This is
usually done in rdma_create_event_channel() or first time whenever
ucma_init() is called. If a process is only going to use one of the
configured HCAs/RDMA IPs then the remaining device contexts are not
used/required. Opening a device context on each device apriori limits the
maximum number of processes that can be supported on a node to the maximum
number of open context supported per HCA regardless of number of HCAs present
in the system.
Signed-off-by: Shamir Rabinovitch <shamir.rabinovitch@oracle.com> Signed-off-by: Sean Hefty <sean.hefty@intel.com>
librdmacm currently opens a device context per configured HCA. This is
usually done in rdma_create_event_channel() or first time whenever
ucma_init() is called. If a process is only going to use one of the
configured HCAs/RDMA IPs then the remaining device contexts are not
used/required. Opening a device context on each device apriori limits the
maximum number of processes that can be supported on a node to the maximum
number of open context supported per HCA regardless of number of HCAs present
in the system.
Signed-off-by: Shamir Rabinovitch <shamir.rabinovitch@oracle.com> Signed-off-by: Sean Hefty <sean.hefty@intel.com>
Sean Hefty [Wed, 9 Apr 2014 19:19:25 +0000 (12:19 -0700)]
librdmacm: Support lazy initialization
librdmacm currently opens a device context per configured HCA. This is
usually done in rdma_create_event_channel() or first time whenever
ucma_init() is called. If a process is only going to use one of the
configured HCAs/RDMA IPs then the remaining device contexts are not
used/required. Opening a device context on each device apriori limits the
maximum number of processes that can be supported on a node to the maximum
number of open context supported per HCA regardless of number of HCAs present
in the system.
Signed-off-by: Shamir Rabinovitch <shamir.rabinovitch@oracle.com> Signed-off-by: Sean Hefty <sean.hefty@intel.com>
Sean Hefty [Wed, 9 Apr 2014 19:19:25 +0000 (12:19 -0700)]
From patchwork Tue Mar 25 06:58:44 2014
Content-Type: text/plain; charset="utf-8"
MIME-Version: 1.0
Content-Transfer-Encoding: 7bit
Subject: librdmacm: lazy initialization for ib devices
From: Shamir Rabinovitch <shamir.rabinovitch@oracle.com>
X-Patchwork-Id: 3885371
Message-Id: <20140325065839.GA3794@shamir-pc>
To: linux-rdma@vger.kernel.org
Date: Tue, 25 Mar 2014 08:58:44 +0200
Hi Sean,
Resending the patch with proper mail client.
There was a request by teams in Oracle to decouple the HCA limited
ability to open device context and the rising amount of processes that
run on the servers.
It appear that the lowest capable HCA become the limit to all the
processes that attempt to use the library.
This patch try to decouple this by allowing the process to use the
library w/o implicitly opening the device.
Device will be opened only when needed.
BR, Shamir
LIBRDMACM currently opens a device context per configured HCA. This is
usually done in rdma_create_event_channel() or first time whenever
ucma_init() is called. If a process is only going to use one of the
configured HCAs/RDMA IPs then the remaining device contexts are not
used/required. Opening a device context on each device apriori limits the
maximum number of processes that can be supported on a node to the maximum
number of open context supported per HCA regardless of number of HCAs present
in the system.
Sean Hefty [Thu, 6 Mar 2014 21:42:31 +0000 (13:42 -0800)]
rsocket: Fix sbuf_bytes_avail counter 'overrun' with iwarp
Reported-by: Jonas Pfefferle1 <JPF@zurich.ibm.com>
"The problem is that on the client side sbuf_bytes_avail overflows
in rs_poll_cq. And from what I debugged so far there are 2
completions for every send and this is because I use iWarp hardware
which does not support write with immediate so there is one completion
for the write and one for the send (both go into the default case
and add the length to sbuf_bytes_avail)."
To avoid the issue, we flag send message operations that are used
in place of immediate data. Other send message operations are
not affected. The completion code can then check whether the
completion is for a send message which was paired with an RDMA
write transaction and adjust the behavior accordingly. Additionally,
such send messages only carry the opcode in their WR_ID, with the
data portion zeroed. This avoids adding the length value twice.
Sean Hefty [Mon, 27 Jan 2014 19:30:34 +0000 (11:30 -0800)]
udaddy: Remove support for port space IB
UD support for the IB port space requires that the application
use rdma_create_ep, rather than rdma_create_id. However, using
rdma_create_ep results in address and route resolution being
performed synchronously as part of the rdma_create_ep call.
Since udaddy is an example, we want to show how it can be used
with asynchronous events. So, rather than update udaddy to
use rdma_create_ep in order to support the IB port space, it
would be better to remove that support.
projects / ~shefty / librdmacm.git / log