From: Sean Hefty <sean.hefty@intel.com>
Date: Thu, 23 Aug 2012 18:20:08 +0000 (-0700)
Subject: rspreload: Avoid rsocket calls until after fork
X-Git-Tag: v1.0.17~34
X-Git-Url: https://openfabrics.org/gitweb/?a=commitdiff_plain;h=dce5bd27a522646b83d347e5483fe7be04f95f71;p=~shefty%2Flibrdmacm.git

rspreload: Avoid rsocket calls until after fork

When an rsocket call is made before an application calls fork(),
the forked applications can hang.  This can be seen by running
netserver and two netperf clients simultaneously.  The second
netperf client will eventually stop performing data transfers.

LD_PRELOAD=librspreload.so netserver -D

LD_PRELOAD=librspreload.so netperf -v2 -c -C -H 192.168.0.101 -l30
LD_PRELOAD=librspreload.so netperf -v2 -c -C -H 192.168.0.101 -l30

It's not clear what the specific problem is.  The best guess is
that libibverbs or the provider library (e.g. libmlx4) perform
some initialization, such as mmap'ing device memory, which does not
work when fork is called.

As a work-around, avoid calling rsocket routines until immediately
before they are needed.  This allows the process to fork before
the libraries are initialized.

Signed-off-by: Sean Hefty <sean.hefty@intel.com>
---

diff --git a/src/preload.c b/src/preload.c
index 0645f6dd..4ba38f5f 100644
--- a/src/preload.c
+++ b/src/preload.c
@@ -412,20 +412,21 @@ int socket(int domain, int type, int protocol)
 	if (index < 0)
 		return index;
 
+	if (fork_support && (domain == PF_INET || domain == PF_INET6) &&
+	    (type == SOCK_STREAM) && (!protocol || protocol == IPPROTO_TCP)) {
+		ret = real.socket(domain, type, protocol);
+		if (ret < 0)
+			return ret;
+		fd_store(index, ret, fd_normal, fd_fork);
+		return index;
+	}
+
 	recursive = 1;
 	ret = rsocket(domain, type, protocol);
 	recursive = 0;
 	if (ret >= 0) {
-		if (fork_support) {
-			rclose(ret);
-			ret = real.socket(domain, type, protocol);
-			if (ret < 0)
-				return ret;
-			fd_store(index, ret, fd_normal, fd_fork);
-		} else {
-			fd_store(index, ret, fd_rsocket, fd_ready);
-			set_rsocket_options(ret);
-		}
+		fd_store(index, ret, fd_rsocket, fd_ready);
+		set_rsocket_options(ret);
 		return index;
 	}
 	fd_close(index, &ret);