-commit 9fe7506ff9ddf1ae6297cfc6a9dd4d6a57e1939e\r
-Author: Arlin Davis <arlin.r.davis@intel.com>\r
-Date: Fri Oct 30 12:57:22 2009 -0800\r
-\r
- winof: Utilize WinOF version of inet_ntop() for Windows OSes which do not support inet_ntop().\r
- \r
- Signed-off-by: stan smith <stan.smith@intel.com>\r
-\r
-commit d56c645a2bf234e9e0cf215b112c2aa9d5e01945\r
-Author: Arlin Davis <arlin.r.davis@intel.com>\r
-Date: Fri Oct 30 07:17:26 2009 -0800\r
-\r
- ucm: windows build issue with new CQ completion channel\r
- \r
- Signed-off-by: Arlin Davis <arlin.r.davis@intel.com>\r
-\r
-commit 2d2e7e1e185c08542ee31b0e77561a1eeb4bde6c\r
-Author: Arlin Davis <arlin.r.davis@intel.com>\r
-Date: Fri Oct 30 06:35:33 2009 -0800\r
-\r
- winof: add ucm provider to windows build\r
- \r
- Signed-off-by: Arlin Davis <arlin.r.davis@intel.com>\r
-\r
-commit 94b2206093607214e0a9709651460692e8196e1c\r
-Author: Arlin Davis <arlin.r.davis@intel.com>\r
-Date: Fri Oct 30 06:32:56 2009 -0800\r
-\r
- winof: add missing build files for ibal, scm\r
- \r
- Signed-off-by: Arlin Davis <arlin.r.davis@intel.com>\r
-\r
-commit 66b76d7a8035b9164b69781d7630a0c77ce1bb5a\r
-Author: Arlin Davis <arlin.r.davis@intel.com>\r
-Date: Wed Oct 28 09:52:50 2009 -0800\r
-\r
- scm: connection peer resets under heavy load, incorrect event on error\r
- \r
- Under heavy load, we get a peer reset from the remote stack. In this\r
- case retry the socket connection for this QP setup.\r
- \r
- Add debugging with PID's and socket ports to help isolate\r
- these types of socket scaling issues.\r
- \r
- Report correct UD event during error, check remote_ah creation.\r
- \r
- Fix dapl_poll return codes for single event type only.\r
- \r
- Signed-off-by: Arlin Davis <arlin.r.davis@intel.com>\r
-\r
-commit f8108a9bda0200355107fdd6c43cb5885f47d648\r
-Author: Arlin Davis <arlin.r.davis@intel.com>\r
-Date: Wed Oct 28 09:47:37 2009 -0800\r
-\r
- ucm: increase default reply and rtu timeout values.\r
- \r
- Signed-off-by: Arlin Davis <arlin.r.davis@intel.com>\r
-\r
-commit 9c13d0d01c78eeb5071e802fbb53811cdb377059\r
-Author: Arlin Davis <arlin.r.davis@intel.com>\r
-Date: Wed Oct 28 07:48:20 2009 -0800\r
-\r
- ucm: change some debug message levels and add check for valid UD REPLY during retries.\r
- \r
- Signed-off-by: Arlin Davis <arlin.r.davis@intel.com>\r
-\r
-commit 1c404bb3dcc0a45e21ef3aa973d59714413beae0\r
-Author: Arlin Davis <arlin.r.davis@intel.com>\r
-Date: Tue Oct 27 10:37:45 2009 -0800\r
-\r
- ucm: increase timers during subsequent retries\r
- \r
- check/process create_ah errors during connect phase\r
- cleanup some debug messaging.\r
- \r
- Signed-off-by: Arlin Davis <arlin.r.davis@intel.com>\r
-\r
-commit cd8c48586f53e846de4fbe10994b73ba457f6406\r
-Author: Arlin Davis <arlin.r.davis@intel.com>\r
-Date: Mon Oct 19 10:38:36 2009 -0700\r
-\r
- ucm, scm: address handles need destroyed when freeing Endpoints with UD QP's.\r
- \r
- Signed-off-by: Arlin Davis <arlin.r.davis@intel.com>\r
-\r
-commit ce19f5744c0dd9461c09d999b309e8f0e2242767\r
-Author: Arlin Davis <arlin.r.davis@intel.com>\r
-Date: Fri Oct 16 14:42:00 2009 -0700\r
-\r
- openib_common: ignore pd free errors, clear pd_handle and return.\r
- \r
- some older adapters have some issues\r
- with pd free so just clear handle and return\r
- \r
- Signed-off-by: Arlin Davis <arlin.r.davis@intel.com>\r
-\r
-commit 81f5ac17d9039e2edcd8324f7d5ed5f66fcff9f2\r
-Author: Arlin Davis <arlin.r.davis@intel.com>\r
-Date: Fri Oct 16 08:52:21 2009 -0700\r
-\r
- ucm: using UD type QP's, ucm reports wrong reject event when user rejects AH resolution request.\r
- \r
- During rejects, both usr and ucm internal, the qp_type does not get initialized\r
- so the check for UD type QP messages fail on active side and the wrong\r
- event gets generated. Initialize saddr.ib information before sending reject\r
- back to active side.\r
- \r
- Signed-off-by: Arlin Davis <arlin.r.davis@intel.com>\r
-\r
-commit f0214e5a7a81a68819d308cb921eb75f5246207d\r
-Author: Arlin Davis <arlin.r.davis@intel.com>\r
-Date: Fri Oct 16 07:57:25 2009 -0700\r
-\r
- ucm, scm, cma: Fix CNO support on DTO type EVD's\r
- \r
- EVD wait_object should be used for CNO processing\r
- and not the direct CQ event channels. Add proper\r
- checking for DTO type EVD's with CNO at wait\r
- and wakeup.\r
- \r
- UCM missing support for collective EVD's under a\r
- CNO. Add support to create common channel for\r
- collective EVD's during device open. Add support\r
- in cm_thread to check this channel. Also,\r
- during disconnect, move QP to error to properly\r
- flush queue instead of moving to reset and init.\r
- \r
- Signed-off-by: Arlin Davis <arlin.r.davis@intel.com>\r
-\r
-commit 960950a7d9f5437dd831bd56ca2ad0c06cb4e324\r
-Author: Arlin Davis <arlin.r.davis@intel.com>\r
-Date: Thu Oct 15 09:19:45 2009 -0700\r
-\r
- ucm: fix lock init bug in ucm_cm_find\r
- \r
- the lock should be setup as pointer to lock\r
- not lock structure. Cleanup lock and list\r
- in cm_find function and cm_print function.\r
- \r
- Add debug aid by passing process id in\r
- msg resv area. cleanup cr references\r
- and change to cm for consistency.\r
- \r
- Signed-off-by: Arlin Davis <arlin.r.davis@intel.com>\r
-\r
-commit f86fec772f2d82eaf60228d288b295e0b7b86c59\r
-Author: Arlin Davis <arlin.r.davis@intel.com>\r
-Date: Wed Oct 14 10:03:47 2009 -0700\r
-\r
- ucm: fix build problem with latest windows ucm changes\r
- \r
- define dapls_thread_signal as inline\r
- \r
- Signed-off-by: Arlin Davis <arlin.r.davis@intel.com>\r
-\r
-commit 87b6c8ba92f3063a35d49bdb49d6cd0a5100a36c\r
-Author: Sean Hefty <sean.hefty@intel.com>\r
-Date: Wed Oct 14 09:34:22 2009 -0700\r
-\r
- Signed-off-by: Sean Hefty <sean.hefty@intel.com>\r
-\r
-commit 9fdd8d74f2cba83e9cf513256933f5241495c1da\r
-Author: Sean Hefty <sean.hefty@intel.com>\r
-Date: Wed Oct 14 09:34:18 2009 -0700\r
-\r
- The HCA should not be closed until all resources have been released.\r
- This results in a hang on windows, since closing the device frees\r
- the event processing thread.\r
- \r
- Signed-off-by: Sean Hefty <sean.hefty@intel.com>\r
-\r
-commit f9833db469f2d686842bb1d52d1ea53b74fa72a8\r
-Author: Sean Hefty <sean.hefty@intel.com>\r
-Date: Wed Oct 14 09:34:13 2009 -0700\r
-\r
- Fix build warning when compiling on 32-bit systems.\r
- \r
- Signed-off-by: Sean Hefty <sean.hefty@intel.com>\r
-\r
-commit c80515bd4b1bd11a125dc17e3f7db44240ee1fff\r
-Author: Sean Hefty <sean.hefty@intel.com>\r
-Date: Wed Oct 14 09:34:07 2009 -0700\r
-\r
- Trying to deregister the same memory region twice leads to an\r
- application crash on windows.\r
- \r
- Signed-off-by: Sean Hefty <sean.hefty@intel.com>\r
-\r
-commit 6aa2c0d901daa9cfca7e771c1df2ead074d230bd\r
-Author: Arlin Davis <arlin.r.davis@intel.com>\r
-Date: Wed Oct 14 07:59:23 2009 -0700\r
-\r
- dat: reduce debug message level when parsing for location of dat.conf\r
- \r
- Don't output failover to default /etc/dat.conf from\r
- sysconfdir at ERROR level. Reduce to DAT_OS_DBG_TYPE_SR.\r
- \r
- Signed-off-by: Arlin Davis <arlin.r.davis@intel.com>\r
-\r
-commit e4038e078747201b57203f16ba793b7fc22c12f2\r
-Author: Arlin Davis <arlin.r.davis@intel.com>\r
-Date: Thu Oct 8 16:23:22 2009 -0700\r
-\r
- ucm: update ucm provider for windows environment\r
- \r
- add dapls_thread_signal abstraction and a new\r
- cm_thread function specific for windows.\r
- \r
- Signed-off-by: Sean Hefty <sean.hefty@intel.com>\r
- Signed-off-by: Arlin Davis <arlin.r.davis@intel.com>\r
-\r
-commit d80ce42390eb57b9c4f816b4df063f90bd5699bc\r
-Author: Arlin Davis <arlin.r.davis@intel.com>\r
-Date: Thu Oct 8 16:02:52 2009 -0700\r
-\r
- ucm: add timer/retry CM logic to the ucm provider\r
- \r
- add reply, rtu and retry count options via\r
- environment variables. Times in msecs.\r
- DAPL_UCM_RETRY 10\r
- DAPL_UCM_REP_TIME 400\r
- DAPL_UCM_RTU_TIME 200\r
- \r
- Add RTU_PENDING and DISC_RECV states\r
- \r
- Add check timer code to the cm_thread\r
- and the option to the select abstaction\r
- to take timeout values in msecs.\r
- DREQ, REQ, and REPLY will all be timed\r
- and retried.\r
- \r
- Split out reply code and disconnect_final\r
- code to better facilitate retry timers.\r
- Add checking for duplicate messages.\r
- \r
- Added new UD extension events for errors.\r
- DAT_IB_UD_CONNECTION_REJECT_EVENT\r
- DAT_IB_UD_CONNECTION_ERROR_EVENT\r
- \r
- Signed-off-by: Arlin Davis <arlin.r.davis@intel.com>\r
-\r
-commit 1186bfc949f4bb7278c30c2c59b7fcb6d5142638\r
-Author: Arlin Davis <arlin.r.davis@intel.com>\r
-Date: Fri Oct 2 14:49:52 2009 -0700\r
-\r
- Release 2.0.23\r
- \r
- Signed-off-by: Arlin Davis <arlin.r.davis@intel.com>\r
-\r
-commit a5f1220cfd96983c9c89a595d80fab7ddcb1a954\r
-Author: Arlin Davis <arlin.r.davis@intel.com>\r
-Date: Fri Oct 2 14:48:15 2009 -0700\r
-\r
- cma: cannot reuse the cm_id and qp for new connection, must reallocate a new one.\r
- \r
- When merging common code base the dapls_ib_reinit_ep mistakely\r
- modified QP to reset then init for all providers. Will\r
- not work for rdma_cm (cma provider) since the cm_id cannot\r
- be reused. Add build check for _OPENIB_CMA_ to pull in correct\r
- free and reallocate method for reinit_ep.\r
- \r
- Signed-off-by: Arlin Davis <arlin.r.davis@intel.com>\r
-\r
-commit 7b07435495de0938e59be064fe8642cfd739f1ac\r
-Author: Arlin Davis <arlin.r.davis@intel.com>\r
-Date: Fri Oct 2 13:50:12 2009 -0700\r
-\r
- scm, cma: update DAPL cm protocol revision with latest address/port changes\r
- \r
- CM protocol changed, roll revision to 6.\r
- The socket cm could be competing with address space if\r
- application is using sockets above to exchange information\r
- like dapltest, and MPI consumers. Adjust port on listen\r
- and connect to reduce the chance of port collision with\r
- application above.\r
- \r
- Signed-off-by: Arlin Davis <arlin.r.davis@intel.com>\r
-\r
-commit 9cc1e76b672f7ff1231b4d113bd4dd1a016d1410\r
-Author: Arlin Davis <arlin.r.davis@intel.com>\r
-Date: Fri Oct 2 12:47:37 2009 -0700\r
-\r
- ucm: modify IB address format to align better with sockaddr_in6\r
- \r
- Restructure the dcm_addr union to map the IB side\r
- closer to sockaddr6 and initialize family to\r
- AF_INET6 to insure callee allocates enough memory\r
- for ucm dat_ia_address type. Put qpn in flowinfo\r
- and gid in sin6_addr. Change the test suites\r
- to print address information based on AF_INET\r
- or AF_INET6 instead of using specific IB address\r
- union from the provider.\r
- \r
- Signed-off-by: Arlin Davis <arlin.r.davis@intel.com>\r
-\r
-commit f50dbe6e82bd471845adf27829b2e07234a0a86a\r
-Author: Sean Hefty <sean.hefty@intel.com>\r
-Date: Wed Sep 30 14:29:03 2009 -0700\r
-\r
- Add definition for getpid similar to that used by the other dtest apps.\r
- \r
- Signed-off-by: Sean Hefty <sean.hefty@intel.com>\r
-\r
-commit 51147412fad72c00115b595955760e80ff0be7d5\r
-Author: Sean Hefty <sean.hefty@intel.com>\r
-Date: Wed Sep 30 14:28:57 2009 -0700\r
-\r
- WinOF provides a common implementation of gettimeofday that should\r
- be used instead.\r
- \r
- Signed-off-by: Sean Hefty <sean.hefty@intel.com>\r
-\r
-commit 732fc84cee036126282715ea157bdb619fc11ec7\r
-Author: Sean Hefty <sean.hefty@intel.com>\r
-Date: Wed Sep 30 14:27:50 2009 -0700\r
-\r
- The completion manager was updated to provide an abstraction that\r
- better mimicked how fd's were used. Update dapl to use this\r
- abstraction, rather than the older completion manager api.\r
- \r
- This helps minimize changes between linux and windows.\r
- \r
- Signed-off-by: Sean Hefty <sean.hefty@intel.com>\r
-\r
-commit fc1855b7738838d3865389405f017b87d223f743\r
-Author: Arlin Davis <arlin.r.davis@intel.com>\r
-Date: Wed Sep 30 14:26:47 2009 -0700\r
-\r
- dtestcm: remove IB verb definitions\r
- \r
- Remove gid and qp_type references from test app.\r
- Print address infomation in sockaddr and\r
- ucm provider format with qpn and lid.\r
- \r
- Signed-off-by: Arlin Davis <arlin.r.davis@intel.com>\r
-\r
-commit 36a885d4267be7b3c6c1086372d321121ae03dee\r
-Author: Arlin Davis <arlin.r.davis@intel.com>\r
-Date: Wed Sep 30 10:44:14 2009 -0700\r
-\r
- dtest, dtestx: remove IB verb definitions\r
- \r
- remove gid and qp_type checking from test suite.\r
- Print address infomation in sockaddr and\r
- ucm provider format with qpn and lid.\r
- \r
- Signed-off-by: Arlin Davis <arlin.r.davis@intel.com>\r
-\r
-commit 23445bbbd3a09f7f5e666a000d5c7c3b22dd95b3\r
-Author: Arlin Davis <arlin.r.davis@intel.com>\r
-Date: Mon Sep 28 10:59:36 2009 -0700\r
-\r
- scm: tighten up socket options to insure similiar behavior on Windows and Linux.\r
- \r
- Add IPPROTO_TCP to create socket. Specify device IP address\r
- when binding instead of INADDR_ANY and remove setsocketopt\r
- REUSEADDR on the listen socket to avoid any issues with\r
- portability. Don't want duplicate port bindings.\r
- \r
- Signed-off-by: Arlin Davis <arlin.r.davis@intel.com>\r
-\r
-commit b31b64d60abe7c6c1a83484db97176d225076b30\r
-Author: Arlin Davis <arlin.r.davis@intel.com>\r
-Date: Mon Sep 28 10:46:26 2009 -0700\r
-\r
- cma: improve serialization of destroy and event processing\r
- \r
- WinOF testing with slightly different scheduler and verbs\r
- showed some issues with cleanup. Add better protection around\r
- destroy and event processing thread.\r
- \r
- Remove destroy flag and add refs counting to conn objects\r
- to block destroy until all references are cleared. Add\r
- locking aroung ref counting and passive and active\r
- event processing.\r
- \r
- Signed-off-by: Arlin Davis <arlin.r.davis@intel.com>\r
-\r
-commit a4adf463695ba75beaa1fe79d514346bf5fe2cb5\r
-Author: Arlin Davis <arlin.r.davis@intel.com>\r
-Date: Mon Sep 28 10:42:52 2009 -0700\r
-\r
- scm: improve serialization of destroy and state changes\r
- \r
- WinOF testing with slightly different scheduler and verbs\r
- showed some issues with cleanup. Add better protection around\r
- destroy and move state change before socket send to insure\r
- correct state in multi-thread environment targeting the same\r
- device on send and recv.\r
- \r
- Change DCM_RTU_PENDING to DCM_REP_PENDING and\r
- and add static definition to local routines for better\r
- readability.\r
- \r
- Signed-off-by: Arlin Davis <arlin.r.davis@intel.com>\r
-\r
-commit 29358ccc587db55fe5f5a1b14eed9e0e31f5f02c\r
-Author: Arlin Davis <arlin.r.davis@intel.com>\r
-Date: Thu Sep 17 08:56:06 2009 -0700\r
-\r
- common: no cleanup/release code for timer thread\r
- \r
- dapl_set_timer() creates a thread to process timers for dat_ep_connect\r
- but provides no mechanism to destroy/exit during dapl library unload.\r
- Timers are initialized in library init code and should be released\r
- in the fini code. Add a dapl_timer_release call to the dapl_fini\r
- function to check state of timer thread and destroy before exiting.\r
- \r
- Signed-off-by: Arlin Davis <arlin.r.davis@intel.com>\r
-\r
-commit e287ce12f5b0108ef83186aae305f25af8cdd02c\r
-Author: Arlin Davis <arlin.r.davis@intel.com>\r
-Date: Thu Sep 17 08:53:29 2009 -0700\r
-\r
- scm, cma: dapli_thread doesn't always get teminated on library close.\r
- \r
- DAPL doesn't actually wait for the async processing thread to exit before\r
- allowing the library to close. It will wait up to 10 seconds, which under\r
- heavy load isn't enough time. Since the thread is created by an application\r
- level thread, it will continue to run as long as the application runs. But\r
- if the application closes the library, then all library data and code is\r
- invalid, which can result in the thread running something that's not\r
- library code and accessing freed memory.\r
- \r
- With this change, I was able to run mpi ping-pong, 16 ranks on a single\r
- system (scm provider) without crashes 1300 times.\r
- \r
- Signed-off-by: Sean Hefty <sean.hefty@intel.com>\r
-\r
-commit 5bffe521c6ad7a5ce10457a5d7b25814522a0cde\r
-Author: Arlin Davis <arlin.r.davis@intel.com>\r
-Date: Wed Sep 9 13:10:35 2009 -0700\r
-\r
- ucm: tighten up locking with CM processing, state changes\r
- \r
- tighten up locking on CM processing and state changes\r
- and reduce the send completion threshold to 50 from 100\r
- to replenish the request message faster.\r
- \r
- Signed-off-by: Arlin Davis <arlin.r.davis@intel.com>\r
-\r
-commit e38daa8448bc3b73c53e54ffc9ac47b9cfc2af89\r
-Author: Arlin Davis <arlin.r.davis@intel.com>\r
-Date: Wed Sep 9 09:44:03 2009 -0700\r
-\r
- ucm: For UD type QP's, return CR p_data with CONN_EST event on passive side.\r
- \r
- Intel MPI uses the p_data provided with CONN_EST as a reference to the\r
- UD pair and remote rank. The ucm provider was overwriting the CR p_data\r
- with the ACCEPT p_data. Change to save CR p_data but also provide\r
- storage for user provided ACCEPT p_data in case the REPLY is lost\r
- and needs retransmitted.\r
- \r
- p_data size was provided to event processing in network order\r
- instead of host order.\r
- \r
- For new QP's create new address handles and do not use\r
- existing AH's created for the CM. Different PD's are\r
- associated with each.\r
- \r
- Signed-off-by: Arlin Davis <arlin.r.davis@intel.com>\r
-\r
-commit 0bc6a0954bc1a1a61e1b1cbc5c280f69ed168453\r
-Author: Arlin Davis <arlin.r.davis@intel.com>\r
-Date: Tue Sep 8 09:14:46 2009 -0700\r
-\r
- ucm: cleanup extra cr/lf\r
- \r
- Signed-off-by: Arlin Davis <arlin.r.davis@intel.com>\r
-\r
-commit 6e7df65a884b4e068135e64dcb3ec660f4c7ab14\r
-Author: Arlin Davis <arlin.r.davis@intel.com>\r
-Date: Tue Sep 8 09:11:37 2009 -0700\r
-\r
- ucm: fix issues with UD QP's.\r
- \r
- private data size not in host order when processing\r
- connection events.\r
- \r
- ud extentions event should include original ia_addr\r
- and qpn used during connection and not the IB qpn.\r
- \r
- ucm QP service resource cleanup in wrong order.\r
- \r
- cleanup extra cr/lf device.c\r
- \r
- Signed-off-by: Arlin Davis <arlin.r.davis@intel.com>\r
-\r
-commit e7c3090945c13f80abb929643fc975465afeeb53\r
-Author: Arlin Davis <arlin.r.davis@intel.com>\r
-Date: Thu Sep 3 10:45:56 2009 -0700\r
-\r
- winof: Convert windows version of dapl and dat libaries to use private heaps.\r
- \r
- This allows for better support of memory registration caching by upper\r
- level libaries (MPI) that use SecureMemoryCacheCallback.\r
- \r
- It also makes it easier to debug heap corruption issues.\r
- \r
- Signed-off-by: Sean Hefty <sean.hefty@intel.com>\r
-\r
-commit bc3c8a34558880c8d09a03e3eb7c3f50d7762c67\r
-Author: Arlin Davis <arlin.r.davis@intel.com>\r
-Date: Wed Sep 2 14:01:51 2009 -0700\r
-\r
- dtest, dtestx: modifications for UD QP testing with ucm provider.\r
- \r
- remote_addr is wrong for IP remote address.\r
- \r
- The dtestx requires the server connect back to the client\r
- for the UD test. With the ucm provider you need to provide\r
- the QPN and the LID which you cannot get until the dtest\r
- client starts. So, for now, don't support UD testing\r
- on UCM providers.\r
- \r
- Signed-off-by: Arlin Davis <arlin.r.davis@intel.com>\r
-\r
-commit e4555143ca71ebe18d8c9f027e2e7a5282088a30\r
-Author: Arlin Davis <arlin.r.davis@intel.com>\r
-Date: Wed Sep 2 13:54:59 2009 -0700\r
-\r
- scm, ucm: UD QP support was broken when porting to common openib code base.\r
- \r
- create remote_ah was moved out of modify_qp_state function but not\r
- included in the RTU and ACCEPT code for UD QP's. qp type check\r
- should be on daddr not saddr in ucm cm code.\r
- \r
- QP number must be converted to host order before supplying remote_ah,\r
- and qp number to consumer.\r
- \r
- Modify QP state to RTR for UD QP mask setting incorrect.\r
- \r
- Signed-off-by: Arlin Davis <arlin.r.davis@intel.com>\r
-\r
-commit 375d368494bb2f9bb44c82073abc60e01ac28615\r
-Author: Arlin Davis <arlin.r.davis@intel.com>\r
-Date: Tue Sep 1 13:02:24 2009 -0700\r
-\r
- cma: cleanup warning with unused local variable, ret, in disconnect\r
- \r
- Signed-off-by: Arlin Davis <arlin.r.davis@intel.com>\r
-\r
-commit 72d1c0c984919809ecd94b2e4ede31ec56668518\r
-Author: Arlin Davis <arlin.r.davis@intel.com>\r
-Date: Tue Sep 1 12:36:31 2009 -0700\r
-\r
- cma: remove debug message after rdma_disconnect failure\r
- \r
- DAPL automatically calls rdma_disconnect() when a disconnect request is\r
- received. If the user also calls disconnect, that calls rdma_disconnect() as\r
- well, but the connection has already been disconnected by DAPL and is no longer\r
- valid. The result is that the user's call to rdma_disconnect() will fail. Do\r
- not display an error message if this occurs.\r
- \r
- Locking could be added to prevent calling rdma_disconnect() multiple times, but\r
- since the librdmacm provides synchronization to trap this, we might as well take\r
- advantage of it.\r
- \r
- Signed-off-by: Sean Hefty <sean.hefty@intel.com>\r
-\r
-commit 290c568030da6df398a4afb932b5d4f9bd585c26\r
-Author: Arlin Davis <arlin.r.davis@intel.com>\r
-Date: Tue Sep 1 12:27:43 2009 -0700\r
-\r
- scm: socket errno check needs O/S dependent wrapper\r
- \r
- Intel MPI checks the uDAPL error code when calling dat_psp_create() to see if\r
- the port number that it provides is in use or not. Convert winsock error codes\r
- to unix errno values.\r
- \r
- This fixes the following error reported by Intel MPI:\r
- 'DAPL provider is not found and fallback device is not enabled'\r
- \r
- Signed-off-by: Sean Hefty <sean.hefty@intel.com>\r
-\r
-commit 03b6d1b58ae03058a509c54ffe27147cc44d1851\r
-Author: Arlin Davis <arlin.r.davis@intel.com>\r
-Date: Tue Sep 1 12:13:16 2009 -0700\r
-\r
- dapltest: update script files for WinOF\r
- \r
- Cleanup 64-bit paths now that WinOF is always installed into '\Program Files\WinOF'.\r
- \r
- Signed-off-by: Arlin Davis <arlin.r.davis@intel.com>\r
-\r
-commit e38585be89e9fa8f139002a3fe1a82eb86438f93\r
-Author: Arlin Davis <arlin.r.davis@intel.com>\r
-Date: Tue Sep 1 12:10:21 2009 -0700\r
-\r
- cma: conditional check for new rdma_cm definition.\r
- \r
- RDMA_CM_EVENT_TIMEWAIT_EXIT is new to OFED 1.4\r
- add conditional check so dapl can build and run\r
- against older OFED 1.3 stacks\r
- \r
- Signed-off-by: Arlin Davis <arlin.r.davis@intel.com>\r
-\r
-commit 83af586b77aa905b63de6dd6010cd60e91487a9e\r
-Author: Arlin Davis <arlin.r.davis@intel.com>\r
-Date: Thu Aug 20 09:13:43 2009 -0700\r
-\r
- Release 2.0.22\r
- \r
- Signed-off-by: Arlin Davis <arlin.r.davis@intel.com>\r
-\r
-commit 48bc0dc95093ccbfe5e01cef4877c128e2caaf77\r
-Author: Arlin Davis <arlin.r.davis@intel.com>\r
-Date: Thu Aug 20 09:12:47 2009 -0700\r
-\r
- dapltest: add mdep processor yield and use with dapltest\r
- \r
- Be thread scheduler friendly and release the current thread thus allowing other threads to run.\r
- \r
- Signed off by Stan Smith stan.smith@intel.com\r
-\r
-commit e13f255a06b9d2a2ed4eec9a62f9fe105d0c26d7\r
-Author: Arlin Davis <arlin.r.davis@intel.com>\r
-Date: Tue Aug 18 10:15:15 2009 -0700\r
-\r
- ucm: Add new provider using a DAPL based IB-UD cm mechanism for MPI implementations.\r
- \r
- New provider uses it's own CM protocol on top of IB-UD queue pairs.\r
- During device open, this provider creates a UD queue pair and\r
- returns local address information via dat_ia_query. This 24 byte\r
- opaque address must be exchange out-of-band before connecting to a\r
- server via dat_ep_connect. This provider is targeted for MPI\r
- implementations that already exchange address information\r
- during mpi_init phase.\r
- \r
- Future release may provide some ARP mechanism via multicast.\r
- \r
- dtest, dtestx, and dtestcm was modified to report the lid and qpn\r
- information on the server side so you can provide appropriate\r
- destination address information for the client test suite.\r
- \r
- dapltest will not work with this provider.\r
- \r
- Signed-off-by: Arlin Davis <arlin.r.davis@intel.com>\r
-\r
-commit 38d224dd95896c7e60f0bc0ffa52b26ab78f489b\r
-Author: Arlin Davis <arlin.r.davis@intel.com>\r
-Date: Tue Aug 4 20:54:12 2009 -0700\r
-\r
- Release 2.0.21\r
- \r
- Signed-off-by: Arlin Davis <arlin.r.davis@intel.com>\r
-\r
-commit ed4999a26043c9c3c73c792b21d24ced1df1553c\r
-Author: Arlin Davis <arlin.r.davis@intel.com>\r
-Date: Tue Aug 4 20:49:09 2009 -0700\r
-\r
- scm: Fix disconnect. QP's need to move to ERROR state in\r
- order to flush work requests and notify consumer. Moving to\r
- RESET removed all requests but did not notify consumer.\r
- \r
- Signed-off-by: Arlin Davis <arlin.r.davis@intel.com>\r
-\r
-commit 512f1d7a480f06a1fa491d21870e560ad111c4d0\r
-Author: Arlin Davis <arlin.r.davis@intel.com>\r
-Date: Tue Aug 4 20:48:03 2009 -0700\r
-\r
- modify dtest.c to cleanup CNO wait code and consolidate into\r
- collect_event() call. After waking up from CNO wait the\r
- consumer must check all EVD's. The EVD's under the CNO\r
- could be dropped if already triggered or could come in any order.\r
- DT_RetToString changed to DT_RetToStr and DT_EventToSTr\r
- changed to DT_EventToStr for consistency.\r
- \r
- Signed-off-by: Arlin Davis <arlin.r.davis@intel.com>\r
-\r
-commit 024e36975d37a1556bf68145e1573f637d269bfc\r
-Author: Arlin Davis <arlin.r.davis@intel.com>\r
-Date: Tue Aug 4 20:47:17 2009 -0700\r
-\r
- CNO events, once triggered will not be returned during the cno wait.\r
- Check for triggered state before going to sleep in cno_wait. Reset\r
- triggered EVD reference after reporting.\r
- \r
- Signed-off-by: Arlin Davis <arlin.r.davis@intel.com>\r
-\r
-commit 6d6c72a49158d10825929111d6b4df1c6d2bb589\r
-Author: Arlin Davis <arlin.r.davis@intel.com>\r
-Date: Sun Aug 2 14:21:09 2009 -0700\r
-\r
- CNO support broken in both CMA and SCM providers.\r
- \r
- CQ thread/callback mechanism was removed by mistake. Still\r
- need indirect DTO callbacks when CNO is attached to EVD's.\r
- \r
- Add CQ event channel to cma provider's thread and add\r
- to select for rdma_cm and async channels.\r
- \r
- For scm provider there is not easy way to add this channel\r
- to the select across sockets on windows. So, for portablity\r
- reasons 2 thread is started to process the ASYNC and\r
- CQ channels for events.\r
- \r
- Must disable EVD (evd_endabled=FALSE) during destroy\r
- to prevent EVD events firing for CNOs and re-arming CQ while\r
- CQ is being destroyed.\r
- \r
- Change dtest to check EVD after CNO timesout.\r
- \r
- Signed-off-by: Arlin Davis <arlin.r.davis@intel.com>\r
-\r
-commit 6fe8bd1d8f44777211e816b72e0b2a6d22900207\r
-Author: Arlin Davis <arlin.r.davis@intel.com>\r
-Date: Thu Jul 30 08:02:30 2009 -0700\r
-\r
- common osd: include winsock2.h for IPv6 definitions.\r
- \r
- Signed-off-by: Arlin Davis <arlin.r.davis@intel.com>\r
-\r
-commit bd26383900d18962aeeff54fa59922009091ecfc\r
-Author: Arlin Davis <arlin.r.davis@intel.com>\r
-Date: Wed Jul 29 08:02:15 2009 -0700\r
-\r
- common osd: include w2tcpip.h for sockaddr_in6 definitions.\r
- \r
- Signed-off-by: Arlin Davis <arlin.r.davis@intel.com>\r
-\r
-commit f25544f14554200a6714accef5f761b0269b5819\r
-Author: Sean Hefty <sean.hefty@intel.com>\r
-Date: Mon Jul 27 15:07:33 2009 -0700\r
-\r
- DAPL introduced the concept of directly waiting on the CQ for\r
- events by adding a compile time flag and special handling in the common\r
- code. Rather than using the compile time flag and modifying the\r
- common code, let the provider implement the best way to wait for\r
- CQ events.\r
- \r
- This simplifies the code and allows the common openib providers to\r
- optimize for Linux and Windows platforms independently, rather than\r
- assuming a specific implementation for signaling events.\r
- \r
- Signed-off-by: Sean Hefty <sean.hefty@intel.com>\r
-\r
-commit 1548405a377d2bd17938df69419e9bcf3364d91a\r
-Author: Arlin Davis <arlin.r.davis@intel.com>\r
-Date: Thu Jul 16 12:41:22 2009 -0700\r
-\r
- dapltest: Implement a malloc() threshold for the completion reaping.\r
- \r
- change byte vector allocation to stack in functions:\r
- DT_handle_send_op, DT_handle_rdma_op & DT_handle_recv_op.\r
- \r
- When allocation size is under the threshold, use a stack local\r
- allocation instead of malloc/free. Move redundant bzero() to\r
- be called only in the case of using local stack allocation as\r
- DT_Mdep_malloc() already does a bzero(). Consolidate error handling\r
- return and free()check to a single point by using goto.\r
- \r
- Signed-off-by: Stan Smith <stan.smith@intel.com>\r
-\r
-commit f6311ca7295230bf9efbcddc639fa8e1065b1f3d\r
-Author: Arlin Davis <arlin.r.davis@intel.com>\r
-Date: Thu Jul 16 12:32:09 2009 -0700\r
-\r
- scm: handle connected state when freeing CM objects\r
- \r
- The QP could be freed before being disconnected\r
- so the provider needs process disconnect before freeing\r
- the CM object. The disconnect clean will finish\r
- the destroy process during the disc callback.\r
- \r
- Signed-off-by: Arlin Davis <arlin.r.davis@intel.com>\r
-\r
-commit 4387359106ce398b29847982883016f7fd48b372\r
-Author: Arlin Davis <arlin.r.davis@intel.com>\r
-Date: Wed Jul 8 12:49:43 2009 -0700\r
-\r
- scm, dtest: changes for winof gettimeofday and FD_SETSIZE settings.\r
- \r
- scm changes to set FD_SETSIZE with expected value and\r
- prevent windows override.\r
- \r
- dtest: remove gettimeofday implementation for windows\r
- specific implemenation etc\user\gtod.c\r
- \r
- general EOL cleanup\r
- \r
- Signed-off-by: Arlin Davis <arlin.r.davis@intel.com>\r
-\r
-commit 3542a83d8a31f5ac68adf3aa44e3ebf1265068df\r
-Author: Arlin Davis <arlin.r.davis@intel.com>\r
-Date: Mon Jul 6 09:24:07 2009 -0700\r
-\r
- scm: set TCP_NODELAY sockopt on the server side for sends.\r
- \r
- scm provider sends small messages from both server and client\r
- sides. Set NODELAY on both sides to avoid send delays either\r
- way.\r
- \r
- Signed-off-by: Arlin Davis <arlin.r.davis@intel.com>\r
-\r
-commit 9d591180392856935b9c3befbab2243dd8daf628\r
-Author: Arlin Davis <arlin.r.davis@intel.com>\r
-Date: Thu Jul 2 14:16:52 2009 -0700\r
-\r
- windows: remove obsolete files in dapl/udapl source tree\r
- \r
- SOURCES,makefile,udapl.r,udapl_exports.src,udapl_sources.c\r
- \r
- Signed-off-by: Arlin Davis <arlin.r.davis@intel.com>\r
-\r
-commit 85c238ee0a41dd0a4a24b3d422f34674b0183161\r
-Author: Arlin Davis <arlin.r.davis@intel.com>\r
-Date: Thu Jul 2 14:11:20 2009 -0700\r
-\r
- dtestcm: add UD type QP option to test\r
- \r
- Add -u for UD type QP's during connection setup.\r
- Will setup UD QPs and provide remote AH\r
- in connect establishment event. Measures\r
- setup/exchange rates.\r
- \r
- Signed-off-by: Arlin Davis <arlin.r.davis@intel.com>\r
-\r
-commit 89a2211526e37b1db58fc0ea663b330bc19125c8\r
-Author: Arlin Davis <arlin.r.davis@intel.com>\r
-Date: Thu Jul 2 14:07:36 2009 -0700\r
-\r
- scm: destroy QP called before disconnect\r
- \r
- Handle the case where QP is destroyed before\r
- disconnect processing. Windows supports\r
- reinit_qp during a disconnect call by\r
- destroying the QP and recreating the\r
- QO instead of state change from reset\r
- to init. Call disconnect in destroy\r
- CM code to handle this unexpected state.\r
- \r
- Signed-off-by: Arlin Davis <arlin.r.davis@intel.com>\r
-\r
-commit 6eb35b7d69a896c256b1031337d3353575cd07b4\r
-Author: Arlin Davis <arlin.r.davis@intel.com>\r
-Date: Thu Jul 2 14:03:12 2009 -0700\r
-\r
- cma: add support for rdma_cm TIME_WAIT event.\r
- \r
- Nothing to process, simply ack the event.\r
- \r
- Signed-off-by: Arlin Davis <arlin.r.davis@intel.com>\r
-\r
-commit b6c56b3052ecd3e36c32092ee62ff0c724da5ad4\r
-Author: Arlin Davis <arlin.r.davis@intel.com>\r
-Date: Wed Jul 1 07:58:32 2009 -0700\r
-\r
- scm: remove old udapl_scm code replaced by openib_scm.\r
- \r
- Signed-off-by: Arlin Davis <arlin.r.davis@intel.com>\r
-\r
-commit 5bbae42a56e1cca678d590ac4c841dd61e839d74\r
-Author: Arlin Davis <arlin.r.davis@intel.com>\r
-Date: Wed Jul 1 07:53:18 2009 -0700\r
-\r
- winof: fix build issues after consolidating cma, scm code base.\r
- \r
- Signed-off-by: Arlin Davis <arlin.r.davis@intel.com>\r
-\r
-commit 6bd1d931c4d0d4cbafac383f225140120aee4c51\r
-Author: Arlin Davis <arlin.r.davis@intel.com>\r
-Date: Wed Jul 1 07:51:59 2009 -0700\r
-\r
- cma: lock held when exiting as a result of a rdma_create_event_channel failure.\r
- \r
- Signed-off-by: Arlin Davis <arlin.r.davis@intel.com>\r
-\r
-commit b8a14ff1cc257defa2f74373d143600f5f471823\r
-Author: Sean Hefty <sean.hefty@intel.com>\r
-Date: Mon Jun 29 12:34:54 2009 -0700\r
-\r
- windows: all dlist functions have been moved to the header file.\r
- remove references to dlist.c\r
- \r
- Signed-off-by: Sean Hefty <sean.hefty@intel.com>\r
-\r
-commit 1a081a0a467e4773a641e8edc876a7a4d7a30ca8\r
-Author: Arlin Davis <arlin.r.davis@intel.com>\r
-Date: Mon Jun 29 12:13:48 2009 -0700\r
-\r
- dtestcm windows: add build infrastructure for new dtestcm test suite\r
- \r
- Signed-off-by: Arlin Davis <arlin.r.davis@intel.com>\r
-\r
-commit c37d7a25dca97011ea76e2a541f936d10ca658e0\r
-Author: Arlin Davis <arlin.r.davis@intel.com>\r
-Date: Mon Jun 29 08:57:46 2009 -0700\r
-\r
- openib_common: reorganize provider code base to share common mem, cq, qp, dto functions\r
- \r
- add new openib_common directory with cq, qp, util, dto, mem function calls\r
- and definitions. This basically leaves the unique CM and Device definitions\r
- and functions to the individual providers directory of openib_scm and openib_cma.\r
- \r
- modifications to dapl_cr_accept required. ep->cm_handle is allocated\r
- and managed entirely in provider so dapl common code should not update\r
- ep_handle->cm_handle from the cr->cm_handle automatically. The provider\r
- should determine which cm_handle is required for the accept.\r
- \r
- openib_cma defines _OPENIB_CMA_ and openib_scm defines _OPENIB_SCM_ for provider\r
- specific build needs in common code.\r
-\r
-commit 961a4083ffb646c070137abd33e9ba2ea9482685\r
-Author: Arlin Davis <arlin.r.davis@intel.com>\r
-Date: Fri Jun 26 14:45:34 2009 -0700\r
-\r
- scm: fixes and optimizations for connection scaling\r
- \r
- Prioritize accepts on listen ports via FD_READ\r
- process the accepts ahead of other work to avoid\r
- socket half_connection (SYN_RECV) stalls.\r
- \r
- Fix dapl_poll to return DAPL_FD_ERROR on\r
- all event error types.\r
- \r
- Add new state for socket released, but CR\r
- not yet destroyed. This enables scm to release\r
- the socket resources immediately after exchanging\r
- all QP information. Also, add state to str call.\r
- \r
- Only add the CR reference to the EP if it is\r
- RC type. UD has multiple CR's per EP so when\r
- a UD EP disconnect_clean was called, from a\r
- timeout, it destroyed the wrong CR.\r
- \r
- Signed-off-by: Arlin Davis <arlin.r.davis@intel.com>\r
-\r
-commit a60a9e1fce5588cb23f41391b48acf04edd82499\r
-Author: Arlin Davis <arlin.r.davis@intel.com>\r
-Date: Fri Jun 26 14:31:19 2009 -0700\r
-\r
- scm: double the default fd_set_size\r
- \r
- Signed-off-by: Arlin Davis <arlin.r.davis@intel.com>\r
-\r
-commit 17d5e1692db4ae1eb09aa919d5607f22851d7ec5\r
-Author: Arlin Davis <arlin.r.davis@intel.com>\r
-Date: Fri Jun 26 14:28:30 2009 -0700\r
-\r
- scm: EP reference in CR should be cleared during ep_destroy\r
- \r
- The EP reference in the CR should be set to null\r
- during the EP free call to insure no further\r
- reference back to a mem freed EP.\r
- \r
- Signed-off-by: Arlin Davis <arlin.r.davis@intel.com>\r
-\r
-commit ebb820364cec9d72285c005a0874e7d459a9ff7d\r
-Author: Arlin Davis <arlin.r.davis@intel.com>\r
-Date: Fri Jun 26 14:23:35 2009 -0700\r
-\r
- dtestx: fix conn establishment event checking\r
- \r
- not catching error cases on client side\r
- when checking for event number and UD type\r
- && should have been ||\r
- \r
- Signed-off-by: Arlin Davis <arlin.r.davis@intel.com>\r
-\r
-commit 747b793898042e3011fbad4b2d1285d2c040cb13\r
-Author: Arlin Davis <arlin.r.davis@intel.com>\r
-Date: Fri Jun 26 14:18:37 2009 -0700\r
-\r
- dtestcm: new test to measure dapl connection rates.\r
- \r
- new test suite added to measure connection\r
- rates of providers. Used to compare cma, scm,\r
- and other providers under development.\r
- \r
- dtestcm USAGE\r
- \r
- s: server\r
- c: connections (default = 1000)\r
- b: burst rate of conn_reqs (default = 100)\r
- m: multi-listens (set to burst setting )\r
- v: verbose\r
- w: wait on event (default, polling)\r
- d: delay before accept\r
- h: hostname/address of server, specified on client\r
- P: provider name (default = OpenIB-v2-ib0)\r
- \r
- Signed-off-by: Arlin Davis <arlin.r.davis@intel.com>\r
-\r
-commit d58fbc3a870a060ead882e1d15c6d245cdf39096\r
-Author: Arlin Davis <arlin.r.davis@intel.com>\r
-Date: Fri Jun 19 20:59:16 2009 -0700\r
-\r
- Release 2.0.20\r
- \r
- Signed-off-by: Arlin Davis <arlin.r.davis@intel.com>\r
-\r
-commit beebe0066b47d7bf476925ff280bad2a3db38324\r
-Author: Arlin Davis <arlin.r.davis@intel.com>\r
-Date: Fri Jun 19 20:52:51 2009 -0700\r
-\r
- common,scm: add debug capabilities to print in-process CM lists\r
- \r
- Add a new debug bit DAPL_DBG_TYPE_CM_LIST.\r
- If set, the pending CM requests will be\r
- dumped when dat_print_counters is called.\r
- Only provided when built with -DDAPL_COUNTERS\r
- \r
- Add new dapl_cm_state_str() call for state\r
- to string conversion for debug prints.\r
- \r
- Signed-off-by: Arlin Davis <arlin.r.davis@intel.com>\r
-\r
-commit b1c51f1e68993d9306e3ebd48bd3a1f0e9878fa3\r
-Author: Arlin Davis <arlin.r.davis@intel.com>\r
-Date: Tue Jun 16 09:22:31 2009 -0700\r
-\r
- scm: disconnect EP before cleaning up orphaned CR's during dat_ep_free\r
- \r
- There is the possibility of dat_ep_free being called\r
- with RC CR's still in connected state. Call disconnect\r
- on the CR before marking for destroy.\r
- \r
- Signed-off-by: Arlin Davis <arlin.r.davis@intel.com>\r
-\r
-commit 531d223455a88c885d6c5f7b1d7e158c1079fbce\r
-Author: Arlin Davis <arlin.r.davis@intel.com>\r
-Date: Wed Jun 10 12:05:17 2009 -0700\r
-\r
- dapltest: windows scripts updated\r
- \r
- Support added for provider specification and general simplification of internal workings.\r
- \r
- Signed-off-by: Stan Smith <stan.smith@intel.com>\r
-\r
-commit 049d1ea08643d4c4eff761741641d37bb3f01fc1\r
-Author: Arlin Davis <arlin.r.davis@intel.com>\r
-Date: Wed Jun 10 09:18:09 2009 -0700\r
-\r
- scm: private data is not handled properly via CR rejects.\r
- \r
- For both RC and UD connect requests, the private\r
- data is not being received on socket and passed\r
- back via the active side REJECT event.\r
- \r
- UD requires new extended reject event type of\r
- DAT_IB_UD_CONNECTION_REJECT_EVENT to distiquish\r
- between RC and UD type rejects.\r
- \r
- cr_thread exit/cleanup processing fixed to insure\r
- all items are off the list before exiting.\r
- \r
- Signed-off-by: Arlin Davis <arlin.r.davis@intel.com>\r
-\r
-commit 3c26870e276a934e2009090e0fca8bdc36c1be67\r
-Author: Arlin Davis <arlin.r.davis@intel.com>\r
-Date: Wed Jun 10 09:09:56 2009 -0700\r
-\r
- scm: cleanup orphaned UD CR's when destroying the EP\r
- \r
- UD CR objects are kept active because of direct private data references\r
- from CONN events. The cr->socket is closed and marked inactive but the\r
- object remains allocated and queued on the CR resource list. There can\r
- be multiple CR's associated with a given EP and there is no way to\r
- determine when consumer is finished with event until the dat_ep_free.\r
- Schedule destruction for all CR's associated with this EP during\r
- free call. cr_thread will complete cleanup with state of SCM_DESTROY.\r
- \r
- Signed-off-by: Arlin Davis <arlin.r.davis@intel.com>\r
-\r
-commit 73abd3f58fa7b14241fad98912ef27c7b4fdb47e\r
-Author: Arlin Davis <arlin.r.davis@intel.com>\r
-Date: Wed Jun 10 09:05:32 2009 -0700\r
-\r
- scm: provider specific query for default UD MTU is wrong.\r
- \r
- Change the provider specific query DAT_IB_TRANSPORT_MTU\r
- to report 2048 for new default MTU size.\r
- \r
- Signed-off-by: Arlin Davis <arlin.r.davis@intel.com>\r
-\r
-commit 27c0d7edc4c931b808a7c5a24bd5aa2625b48aa1\r
-Author: Arlin Davis <arlin.r.davis@intel.com>\r
-Date: Wed Jun 10 10:06:59 2009 -0700\r
-\r
- scm: update CM code to shutdown before closing socket\r
- \r
- data could be lost without calling shutdown on the socket\r
- before closing. Update to shutdown and then close. Add\r
- definition for SHUT_RW to SD_BOTH for windows.\r
- \r
- Signed-off-by: Arlin Davis <arlin.r.davis@intel.com>\r
- ---\r
-\r
-commit 536ec3103c15c1fed4367326c9117660345e0eab\r
-Author: Arlin Davis <arlin.r.davis@intel.com>\r
-Date: Thu Jun 4 13:48:18 2009 -0700\r
-\r
- dapltest: windows script dt-cli.bat updated\r
- \r
- scn should be scm\r
- \r
- Signed-off-by: Arlin Davis <arlin.r.davis@intel.com>\r
-\r
-commit e8991b8f0877b0e2e857717e1140c679e9266abe\r
-Author: Sean Hefty <sean.hefty@intel.com>\r
-Date: Thu Jun 4 08:19:12 2009 -0700\r
-\r
- dapl/windows cma provider: add support for network devices based on index\r
- \r
- The linux cma provider provides support for named network devices, such\r
- as 'ib0' or 'eth0'. This allows the same dapl configuration file to\r
- be used easily across a cluster.\r
- \r
- To allow similar support on Windows, allow users to specify the device\r
- name 'rdma_devN' in the dapl.conf file. The given index, N, is map to a\r
- corresponding IP address that is associated with an RDMA device.\r
- \r
- Signed-off-by: Sean Hefty <sean.hefty@intel.com>\r
-\r
-commit 79fa3e7d241f740bc886dd075f24fcbc611306de\r
-Author: Arlin Davis <arlin.r.davis@intel.com>\r
-Date: Thu Jun 4 08:00:29 2009 -0700\r
-\r
- openib: remove 1st gen provider, replaced with openib_cma and openib_scm\r
- \r
- Signed-off-by: Arlin Davis <arlin.r.davis@intel.com>\r
-\r
-commit 624039247cdc0db7aa040dfbb4dced00f2cf9006\r
-Author: Arlin Davis <arlin.r.davis@intel.com>\r
-Date: Fri May 29 08:21:10 2009 -0700\r
-\r
- dapltest: update windows script files\r
- \r
- Enhancement to take DAPL provider name as cmd-line arguement.\r
- \r
- Signed-off-by: Arlin Davis <arlin.r.davis@intel.com>\r
-\r
-commit b93baa07b7bbaeb7a55fa817c354d0c94783d61f\r
-Author: Arlin Davis <arlin.r.davis@intel.com>\r
-Date: Thu May 28 15:30:05 2009 -0700\r
-\r
- dapltest: update windows batch files in sripts directory\r
- \r
- Signed-off-by: Arlin Davis <arlin.r.davis@intel.com>\r
-\r
-commit 2f185c6b5e464c4fc9e84ad3e90cc2b86aebf9aa\r
-Author: Arlin Davis <arlin.r.davis@intel.com>\r
-Date: Mon May 18 14:00:02 2009 -0700\r
-\r
- windows_osd/linux_osd: new dapl_os_gettid macro to return thread id\r
- \r
- Change dapl_os_getpid inline to macro on windows and add dapl_os_gettid\r
- macros on linux and windows to return thread id.\r
- \r
- Signed-off-by: Arlin Davis <arlin.r.davis@intel.com>\r
-\r
-commit 984303824cd0c3e248a789066cf665ced8e1ae5b\r
-Author: Arlin Davis <arlin.r.davis@intel.com>\r
-Date: Mon May 18 13:53:59 2009 -0700\r
-\r
- windows: missing build files for common and udapl sub-directories\r
- \r
- Add dapl/dapl_common_src.c and dapl/dapl_udapl_src.c\r
- \r
- Signed-off-by: Arlin Davis <arlin.r.davis@intel.com>\r
-\r
-commit 3be4ccf9681a975e74a5aa05e3f7912477f342a7\r
-Author: Arlin Davis <arlin.r.davis@intel.com>\r
-Date: Mon May 18 09:06:19 2009 -0700\r
-\r
- windows: add build files for openib_scm, remove /Wp64 build option.\r
- \r
- Add build files for windows socket cm and change build\r
- option on windows providers. The new Win7 WDK issues a\r
- depreciated compiler option warning for /Wp64\r
- (Enable 64-bit porting warnings)\r
- \r
- Signed-off-by: Arlin Davis <arlin.r.davis@intel.com>\r
-\r
-commit 163112cfeb6e409886b3cb7f85da7ce003300d5c\r
-Author: Arlin Davis <arlin.r.davis@intel.com>\r
-Date: Mon May 18 08:50:35 2009 -0700\r
-\r
- scm: multi-hca CM processing broken. Need cr thread wakeup mechanism per HCA.\r
- \r
- Currently there is only one pipe across all\r
- device opens. This results in some posted CR work\r
- getting delayed or not processed at all. Provide\r
- pipe for each device open and cr thread created\r
- and manage on a per device level.\r
- \r
- Signed-off-by: Arlin Davis <arlin.r.davis@intel.com>\r
-\r
-commit e6e799f623df6ef136ffc5388251d3f3a38c8a91\r
-Author: Arlin Davis <arlin.r.davis@intel.com>\r
-Date: Fri May 15 11:06:19 2009 -0700\r
-\r
- dtest: add connection timers on client side\r
- \r
- Add timers for active connections and print\r
- results. Allow polling or wait on conn event.\r
- \r
- Signed-off-by: Arlin Davis <arlin.r.davis@intel.com>\r
-\r
-commit d656bbf619123deaed6e8985e52207e5415f359f\r
-Author: Arlin Davis <arlin.r.davis@intel.com>\r
-Date: Fri May 15 09:48:38 2009 -0700\r
-\r
- linux_osd: use pthread_self instead of getpid for debug messages\r
- \r
- getpid provides process ids which are not unique. Use unique thread\r
- id's in debug messages to help isolate issues across many device\r
- opens with multiple CM threads.\r
- \r
- Signed-off-by: Arlin Davis <arlin.r.davis@intel.com>\r
-\r
-commit 92bb0d2933d3d1546e18f0479475f3daf5b92052\r
-Author: Arlin Davis <arlin.r.davis@intel.com>\r
-Date: Fri May 1 10:18:05 2009 -0700\r
-\r
- windows ibal-scm: dapl/dirs file needs updated to remove ibal-scm\r
- \r
- Signed-off-by: Arlin Davis <arlin.r.davis@intel.com>\r
-\r
-commit 9c37d9d667fb7e8f21841bbec4a84b2c652fffe1\r
-Author: Arlin Davis <arlin.r.davis@intel.com>\r
-Date: Wed Apr 29 23:13:36 2009 -0700\r
-\r
- Release 2.0.19\r
- \r
- Signed-off-by: Arlin Davis <arlin.r.davis@intel.com>\r
-\r
-commit 0ef94459e0a0175233b43b3fcbaaac2596e1042d\r
-Author: Arlin Davis <arlin.r.davis@intel.com>\r
-Date: Wed Apr 29 14:33:28 2009 -0700\r
-\r
- scm, cma: dat max_lmr_block_size is 32 bit, verbs max_mr_size is 64 bit\r
- \r
- mismatch of device attribute size restricts max_lmr_block_size to 32 bit\r
- value. Add check, if larger then limit to 4G-1 until DAT v2 spec changes.\r
- \r
- Consumers should use max_lmr_virtual_address for actual max\r
- registration block size until attribute interface changes.\r
- \r
- Signed-off-by: Arlin Davis <arlin.r.davis@intel.com>\r
-\r
-commit f91f27eaaab28b13a631adf75b933b7be3afbc0f\r
-Author: Arlin Davis <arlin.r.davis@intel.com>\r
-Date: Wed Apr 29 10:51:03 2009 -0700\r
-\r
- scm: increase default MTU size from 1024 to 2048\r
- \r
- Signed-off-by: Arlin Davis <arlin.r.davis@intel.com>\r
-\r
-commit 8d6846056f4c86b6a06346147df55d37c4ba9933\r
-Author: Arlin Davis <arlin.r.davis@intel.com>\r
-Date: Wed Apr 29 10:49:09 2009 -0700\r
-\r
- openib_scm, cma: use direct SGE mappings from dat_lmr_triplet to ibv_sge\r
- \r
- no need to rebuild scatter gather list given that DAT v2.0\r
- is now aligned with verbs ibv_sge. Fix ib_send_op_type_t typedef.\r
- \r
- Signed-off-by: Arlin Davis <arlin.r.davis@intel.com>\r
-\r
-commit c61f75b3412935e7d4a7a1acc9c495dcb4ac7e24\r
-Author: Arlin Davis <arlin.r.davis@intel.com>\r
-Date: Wed Apr 29 08:39:37 2009 -0700\r
-\r
- dtest: add flush EVD call after data transfer errors\r
- \r
- Flush and print entries on async, request, and receive\r
- queues after any data transfer error. Will help\r
- identify failing operation during operations\r
- without completion events requested.\r
- Fix -B0 so burst size of 0 works.\r
- \r
- Signed-off-by: Arlin Davis <arlin.r.davis@intel.com>\r
-\r
-commit c88a191c7a408b0fb3dfb418a77a5b3b5afc778e\r
-Author: Arlin Davis <arlin.r.davis@intel.com>\r
-Date: Wed Apr 22 13:16:19 2009 -0700\r
-\r
- dtest/dapltest: Cleanup code with Lindent\r
- \r
- Signed-off-by: Arlin Davis <ardavis@ichips.intel.com>\r
-\r
-commit 8699a9f1bd2bb45b04b87f887698707ba7b62d0a\r
-Author: Arlin Davis <arlin.r.davis@intel.com>\r
-Date: Tue Apr 21 15:51:24 2009 -0700\r
-\r
- ibal-scm: remove, obsolete\r
- \r
- Signed-off-by: Arlin Davis <ardavis@ichips.intel.com>\r
-\r
-commit 67ddd6bfba46f1f7a61b772257132f1257d05c96\r
-Author: Arlin Davis <arlin.r.davis@intel.com>\r
-Date: Tue Apr 21 15:44:15 2009 -0700\r
-\r
- scm, cma provider: Cleanup code with Lindent\r
- \r
- Signed-off-by: Arlin Davis <ardavis@ichips.intel.com>\r
-\r
-commit d0898091090ff19be7929fed0d14f1ca696d5e53\r
-Author: Arlin Davis <arlin.r.davis@intel.com>\r
-Date: Tue Apr 21 15:39:01 2009 -0700\r
-\r
- udapl: Cleanup code with Lindent\r
- \r
- Signed-off-by: Arlin Davis <ardavis@ichips.intel.com>\r
-\r
-commit a688d1cfb52fde256c5bfd95a27f940dd17e7ced\r
-Author: Arlin Davis <arlin.r.davis@intel.com>\r
-Date: Tue Apr 21 15:31:20 2009 -0700\r
-\r
- dapl common: Cleanup code with Lindent\r
- \r
- Signed-off-by: Arlin Davis <ardavis@ichips.intel.com>\r
-\r
-commit 2bded73c7f68cfb870e432ab3ebae7427d595cbe\r
-Author: Arlin Davis <arlin.r.davis@intel.com>\r
-Date: Tue Apr 21 12:52:29 2009 -0700\r
-\r
- dat: Cleanup code with Lindent\r
- \r
- Signed-off-by: Arlin Davis <ardavis@ichips.intel.com>\r
-\r
-commit 31e7b9210fc5334ff3be62558e74e3fdf01d6cbd\r
-Author: Arlin Davis <arlin.r.davis@intel.com>\r
-Date: Mon Apr 20 12:28:08 2009 -0700\r
-\r
- Release 2.0.18\r
- \r
- Signed-off-by: Arlin Davis <ardavis@ichips.intel.com>\r
-\r
-commit 29bf0a24f54c45d2742026756f31f1a1f26fb6f3\r
-Author: Arlin Davis <arlin.r.davis@intel.com>\r
-Date: Thu Apr 16 14:35:18 2009 -0700\r
-\r
- dapltest: reset server listen ports to avoid collisions during long runs\r
- \r
- If server is running continuously the port number increments\r
- from base without reseting between tests. This will\r
- eventually cause collisions in port space.\r
- \r
- Signed-off-by: Arlin Davis <ardavis@ichips.intel.com>\r
-\r
-commit c27af8de0501d132b8152ec8546023cdba212de5\r
-Author: Sean Hefty <sean.hefty@intel.com>\r
-Date: Thu Apr 16 10:21:51 2009 -0700\r
-\r
- To avoid duplicating port numbers between different tests, the next port\r
- number to use must increment based on the number of endpoints per thread *\r
- the number of threads.\r
- \r
- Signed-off-by: Sean Hefty <sean.hefty@intel.com>\r
-\r
-commit 3084310197c20aaa50abe82260fc835786f591f5\r
-Author: Sean Hefty <sean.hefty@intel.com>\r
-Date: Thu Apr 16 10:21:45 2009 -0700\r
-\r
- dapltest assumes that events across multiple endpoints occur in a specific\r
- order. Since this is a false assumption, avoid this by directing events to\r
- per endpoint EVDs, rather than using shared EVDs.\r
- \r
- Signed-off-by: Sean Hefty <sean.hefty@intel.com>\r
-\r
-commit ef87a0a462f4fa07ac252e28d3aeb44af73cc202\r
-Author: Sean Hefty <sean.hefty@intel.com>\r
-Date: Thu Apr 16 10:21:41 2009 -0700\r
-\r
- Synchronization is missing between removing items from an EVD and queuing\r
- them. Since the removal thread is the user's, but the queuing thread is\r
- not, the synchronization must be provided by DAPL. Hold the evd lock\r
- around any calls to dapls_rbuf_*.\r
- \r
- Signed-off-by: Sean Hefty <sean.hefty@intel.com>\r
-\r
-commit f5e86d28f803162ffdf94b41ec7435dec92f728d\r
-Author: Sean Hefty <sean.hefty@intel.com>\r
-Date: Thu Apr 16 10:21:26 2009 -0700\r
-\r
- Communication to the CR thread is done using an internal socket. When a\r
- new connection request is ready for processing, an object is placed on\r
- the CR list, and data is written to the internal socket. The write causes\r
- the CR thread to wake-up and process anything on its cr list.\r
- \r
- If multiple objects are placed on the CR list around the same time, then\r
- the CR thread will read in a single character, but process the entire list.\r
- This results in additional data being left on the internal socket. When\r
- the CR does a select(), it will find more data to read, read the data, but\r
- not have any real work to do. The result is that the thread spins in a\r
- loop checking for changes when none have occurred until all data on the\r
- internal socket has been read.\r
- \r
- Avoid this overhead by reading all data off the internal socket before\r
- processing the CR list.\r
- \r
- Signed-off-by: Sean Hefty <sean.hefty@intel.com>\r
-\r
-commit 2ab52e9b1ab37c6eb44206c135e0568a8c2d01fa\r
-Author: Sean Hefty <sean.hefty@intel.com>\r
-Date: Thu Apr 16 10:21:13 2009 -0700\r
-\r
- The dapl connect call takes as input an address (sockaddr) and a port number\r
- as separate input parameters. It modifies the sockaddr address to set the\r
- port number before trying to connect. This leads to a situation in\r
- dapltest with multiple threads that reference the same buffer for their\r
- address, but specify different port numbers, where the different threads\r
- end up trying to connect to the same remote port.\r
- \r
- To solve this, do not modify the caller's address buffer and instead use\r
- a local buffer. This fixes an issue seen running multithreaded tests with\r
- dapltest.\r
- \r
- Signed-off-by: Sean Hefty <sean.hefty@intel.com>\r
-\r
-commit 7947026ede478f08b4a7b8cb607f457765bf2afa\r
-Author: Sean Hefty <sean.hefty@intel.com>\r
-Date: Thu Apr 16 10:21:03 2009 -0700\r
-\r
- Windows socket calls should check return values against SOCKET_ERROR to\r
- determine if an error occurred.\r
- \r
- Signed-off-by: Sean Hefty <sean.hefty@intel.com>\r
-\r
-commit e66e42fc44c50c8202f7c98f76d799a69aa3f1b6\r
-Author: Arlin Davis <arlin.r.davis@intel.com>\r
-Date: Fri Apr 10 08:33:41 2009 -0700\r
-\r
- Build: add new file dapl/openib_cma/linux/openib_osd.h to EXTRA_DIST\r
- \r
- Fix rpmbuild problem with new cma osd include file.\r
- \r
- Signed-off-by: Arlin Davis <ardavis@ichips.intel.com>\r
-\r
-commit acb213adb3268e9bf6999e2bf040d4a71212b701\r
-Author: Arlin Davis <arlin.r.davis@intel.com>\r
-Date: Fri Apr 10 08:32:24 2009 -0700\r
-\r
- dapl scm: reduce wait time for thread startup.\r
- \r
- thread startup wait reduce to 2ms to reduce open times.\r
- \r
- Signed-off-by: Arlin Davis <ardavis@ichips.intel.com>\r
-\r
-commit 55459699fa9c0e5fb7e2b17822f0916412c64b35\r
-Author: Arlin Davis <arlin.r.davis@intel.com>\r
-Date: Fri Apr 10 08:31:22 2009 -0700\r
-\r
- dapl-scm: getsockopt optlen needs initialized to size of optval\r
- \r
- Signed-off-by: Arlin Davis <ardavis@ichips.intel.com>\r
-\r
-commit d710c5327e05a40796341d16b45a2b098b03f588\r
-Author: Sean Hefty <sean.hefty@intel.com>\r
-Date: Fri Apr 10 08:17:32 2009 -0700\r
-\r
- The connection request thread adds sockets to a select list unless\r
- the cr->socket is invalid and the cr request state is set to destroy. If the\r
- cr->socket is invalid, but the cr->state is not destroy, then the cr->socket\r
- is added to an FD set for select/poll. This results in select/poll\r
- returning an error when select is called. As a result, the cr thread never\r
- actually blocks during this state.\r
- \r
- Fix this by only destroying a cr based on its state being set to destroy\r
- and skip adding cr->sockets to the FD set when they are invalid.\r
- \r
- Signed-off-by: Sean Hefty <sean.hefty@intel.com>\r
-\r
-commit 0be961c432f897d4f92d9a24dcb7c42ad30ea160\r
-Author: Sean Hefty <sean.hefty@intel.com>\r
-Date: Fri Apr 10 08:08:16 2009 -0700\r
-\r
- Make sure all locks are initialized properly and don't zero their memory\r
- once they are.\r
- \r
- Signed-off-by: Sean Hefty <sean.hefty@intel.com>\r
-\r
-commit 9abdc26cd6154aa55588759ba54c9ca69e3fe2b5\r
-Author: Sean Hefty <sean.hefty@intel.com>\r
-Date: Fri Apr 10 08:08:13 2009 -0700\r
-\r
- The lock functions are defined just a few lines beneath the prototypes\r
- as inline. Remove the duplicate prototypes.\r
- \r
- Signed-off-by: Sean Hefty <sean.hefty@intel.com>\r
-\r
-commit 9578c4aeb9878d98374e4b7abc02db182aef82c6\r
-Author: Sean Hefty <sean.hefty@intel.com>\r
-Date: Fri Apr 10 08:08:07 2009 -0700\r
-\r
- Make sure all locks are initialized and don't zero out their memory once\r
- they are.\r
- \r
- Signed-off-by: Sean Hefty <sean.hefty@intel.com>\r
-\r
-commit 97edcbb662b489303ef68c0da02831efaddeed91\r
-Author: Sean Hefty <sean.hefty@intel.com>\r
-Date: Fri Apr 10 08:08:03 2009 -0700\r
-\r
- The IBAL library allocates a small number of threads for callbacks to the\r
- user. If the user blocks all of the callback threads, no additional\r
- callbacks can be invoked. The DAPL IBAL provider cancels listen requests\r
- from within an IBAL callback, then waits for a second callback to confirm\r
- that the listen has been canceled. If there is a single IBAL callback\r
- thread, or multiple listens are canceled simultaneously, then the provider\r
- can deadlock waiting for a cancel callback that never occurs.\r
- \r
- This problem is seen when running dapltest with multiple threads.\r
- \r
- Signed-off-by: Sean Hefty <sean.hefty@intel.com>\r
-\r
-commit 3e56e63bcb68de352edadafdcfcc4cb222c08c7b\r
-Author: Sean Hefty <sean.hefty@intel.com>\r
-Date: Fri Apr 10 08:07:57 2009 -0700\r
-\r
- We need to check the return value from select for errors before checking\r
- the FD sets. An item may be in an FD set but select could have returned\r
- an error.\r
- \r
- Signed-off-by: Sean Hefty <sean.hefty@intel.com>\r
-\r
-commit a8a977becaeefe0d7f8e01e01631a11988d2d54e\r
-Author: Sean Hefty <sean.hefty@intel.com>\r
-Date: Fri Apr 10 08:07:53 2009 -0700\r
-\r
- Signed-off-by: Sean Hefty <sean.hefty@intel.com>\r
-\r
-commit ecc79cc0a1ae2bdbb3dfd19e15b3b562ac9a2957\r
-Author: Sean Hefty <sean.hefty@intel.com>\r
-Date: Fri Apr 10 08:07:49 2009 -0700\r
-\r
- Enable building with CQ_WAIT_OBJECTS support to directly wait on CQ\r
- completion channels in the Windows version of the openib_scm provider.\r
- Also minor fixup to use DAPL_DBG_TYPE_UTIL for debug log messages\r
- instead of DAPL_DBG_TYPE_CM.\r
- \r
- Signed-off-by: Sean Hefty <sean.hefty@intel.com>\r
-\r
-commit 73728763666a46df5789af93b50db53cdf64afd6\r
-Author: Sean Hefty <sean.hefty@intel.com>\r
-Date: Fri Apr 10 08:07:44 2009 -0700\r
-\r
- The IBAL-SCM provider will run into an inifinite loop if the check for\r
- cr->socket > SCM_MAX_CONN - 1 fails. The code continues back to the start\r
- of the while loop without moving to the next connection request entry\r
- in the list.\r
- \r
- Signed-off-by: Sean Hefty <sean.hefty@intel.com>\r
-\r
-commit 9b1b396539926d36ffacfff04fbe7c081e436b45\r
-Author: Sean Hefty <sean.hefty@intel.com>\r
-Date: Fri Apr 10 08:07:40 2009 -0700\r
-\r
- next_cr is set just before and inside the check\r
- if ((cr->socket == DAPL_INVALID_SOCKET && cr->state == SCM_DESTROY)\r
- Remove setting it inside the if statement.\r
- \r
- Signed-off-by: Sean Hefty <sean.hefty@intel.com>\r
-\r
-commit 7b49a9850f62276bb7bfccb2d85a1e94e311813c\r
-Author: Sean Hefty <sean.hefty@intel.com>\r
-Date: Fri Apr 10 08:07:35 2009 -0700\r
-\r
- Some errors on windows are more easily interpretted in hex than decimal.\r
- \r
- Signed-off-by: Sean Hefty <sean.hefty@intel.com>\r
-\r
-commit 08ee072a1396ac2c28983878dbc6b02feb035787\r
-Author: Sean Hefty <sean.hefty@intel.com>\r
-Date: Fri Apr 10 08:07:32 2009 -0700\r
-\r
- The WinOF HCA driver cannot handle transitioning from RTS -> RESET ->\r
- INIT -> ERROR. Simply delete the QP and re-create it to reinitialize\r
- the endpoint until the bug is fixed.\r
- \r
- Signed-off-by: Sean Hefty <sean.hefty@intel.com>\r
-\r
-commit 51ef5d96ce67d6141ec02b2a318f1b6e12be1bcf\r
-Author: Sean Hefty <sean.hefty@intel.com>\r
-Date: Fri Apr 10 08:07:23 2009 -0700\r
-\r
- Signed-off-by: Sean Hefty <sean.hefty@intel.com>\r
-\r
-commit a8582be0e1fc89e856f1d0b43a3c1b271295a352\r
-Author: Sean Hefty <sean.hefty@intel.com>\r
-Date: Fri Apr 10 08:07:18 2009 -0700\r
-\r
- Convert the openib_cma provider to common code between linux and windows.\r
- \r
- Signed-off-by: Sean Hefty <sean.hefty@intel.com>\r
-\r
-commit cc2a71dfe0c35a70f6b1ba66070a3a06059a8bb5\r
-Author: Sean Hefty <sean.hefty@intel.com>\r
-Date: Fri Apr 10 08:06:53 2009 -0700\r
-\r
- Move from using pipes to sockets for internal communication. This\r
- avoids issues with windows only supporting select() on sockets.\r
- \r
- Remove windows specific definition of dapl_dbg_log.\r
- \r
- Update to latest windows libibverbs implementation using completion\r
- channel abstraction to improve windows scalability and simplify\r
- porting where FD's are accessed directly in Linux.\r
- \r
- Signed-off-by: Sean Hefty <sean.hefty@intel.com>\r
-\r
-commit b3ad2ed97399a24a869841e17d1314e11c379aae\r
-Author: Arlin Davis <arlin.r.davis@intel.com>\r
-Date: Tue Mar 31 05:41:50 2009 -0800\r
-\r
- Release 2.0.17\r
- \r
- Signed-off-by: Arlin Davis <ardavis@ichips.intel.com>\r
-\r
-commit 5d732929f8a90a490994e8e35a3666c3647ad4fe\r
-Author: Arlin Davis <arlin.r.davis@intel.com>\r
-Date: Tue Mar 31 05:22:11 2009 -0800\r
-\r
- dapl: ia64 build problem on SuSE 11, atomic.h no longer exists.\r
- \r
- Add autotools check for SuSE 11 and include intrinsics.h\r
- \r
- Signed-off-by: Arlin Davis <ardavis@ichips.intel.com>\r
-\r
-commit d7b8654db3a1f4ead16cb2e6d15f0902a322a188\r
-Author: Arlin Davis <arlin.r.davis@intel.com>\r
-Date: Mon Mar 16 13:23:50 2009 -0800\r
-\r
- Release 2.0.16\r
- \r
- Fix changelog year in spec file.\r
- \r
- Signed-off-by: Arlin Davis <ardavis@ichips.intel.com>\r
-\r
-commit 08d9e26a85911f99d47cbb92ec8bccfc7f073be0\r
-Author: Arlin Davis <arlin.r.davis@intel.com>\r
-Date: Mon Mar 16 13:15:22 2009 -0800\r
-\r
- Release 2.0.16\r
- \r
- Signed-off-by: Arlin Davis <ardavis@ichips.intel.com>\r
-\r
-commit 42c97b2a11d63ac6ba8a15fe8c82061e7da6e136\r
-Author: Arlin Davis <arlin.r.davis@intel.com>\r
-Date: Fri Mar 13 12:39:12 2009 -0800\r
-\r
- uDAPL: scm provider, remove query gid/lid from connection setup phase\r
- \r
- move lid/gid queries from the connection setup phase\r
- and put them in the open call to avoid overhead\r
- of more fd's during connections. No need\r
- to query during connection setup since uDAPL\r
- binds to specific hca/ports via dat_ia_open.\r
- \r
- Signed-off-by: Arlin Davis <ardavis@ichips.intel.com>\r
-\r
-commit 775394b73980a7bc0af018a33d2a5bb795469c78\r
-Author: Arlin Davis <arlin.r.davis@intel.com>\r
-Date: Thu Mar 12 12:44:43 2009 -0800\r
-\r
- Build: missing new linux/osd include file in EXTRA_DIST\r
- \r
- Add dapl/openib_scm/linux/openib_osd.h to EXTRA_DIST\r
- \r
- Signed-off-by: Arlin Davis <ardavis@ichips.intel.com>\r
-\r
-commit 647e288c1c9af5261495a5ed88e6ecbe1daf6d6e\r
-Author: Arlin Davis <arlin.r.davis@intel.com>\r
-Date: Thu Mar 12 12:11:21 2009 -0800\r
-\r
- Build: spec files missing Requires(post) statements for sed/coreutils\r
- \r
- needed for anaconda install\r
- \r
- Signed-off-by: Arlin Davis <ardavis@ichips.intel.com>\r
-\r
-commit 1f5b3b7cab0785b64e8dab035dd4cd27111497d3\r
-Author: Arlin Davis <arlin.r.davis@intel.com>\r
-Date: Wed Mar 4 10:04:13 2009 -0800\r
-\r
- dapl scm: remove unecessary thread when using direct objects\r
- \r
- A thread is created for processing events on devices without\r
- direct event objecti support. Since all openfabrics devices support\r
- direct events there is no need to start a thread. Move this under\r
- \r
- Signed-off-by: Arlin Davis <ardavis@ichips.intel.com>\r
-\r
-commit 66e4236e2c57dbaf860b7c20809b65a4fbbafa6f\r
-Author: Arlin Davis <arlin.r.davis@intel.com>\r
-Date: Tue Mar 3 11:08:12 2009 -0800\r
-\r
- dtestx: add missing F64u definition for windows\r
- \r
- Signed-off-by: Arlin Davis <ardavis@ichips.intel.com>\r
-\r
-commit d9e771da16ec2b360a222ceccbbca5d088e20ee5\r
-Author: Arlin Davis <arlin.r.davis@intel.com>\r
-Date: Tue Mar 3 09:25:26 2009 -0800\r
-\r
- uDAPL common: add 64 bit counters for IA, EP, and EVD's.\r
- \r
- -DDAPL_COUNTERS to build-in counters for cma and scm providers.\r
- New extension calls in dat_ib_extensions.h for counters\r
- dat_print_counters, dat_query_counters\r
- Counters for operations, async errors, and data\r
- Update dtestx (-p) with print and query counter examples\r
- \r
- Signed-off-by: Arlin Davis <ardavis@ichips.intel.com>\r
-\r
-commit 5e6ad3cdaa1d72523ad6b38d8306e0a2d0f1ada6\r
-Author: Sean Hefty <sean.hefty@intel.com>\r
-Date: Tue Feb 17 07:24:27 2009 -0800\r
-\r
- Modify the openib_scm provider to support both OFED and WinOF releases.\r
- This takes advantage of having a libibverbs compatibility library.\r
- \r
- Signed-off-by: Sean Hefty <sean.hefty@intel.com>\r
-\r
-commit 0425c24c3b66e672bba346a1d0946b11c7b8b11e\r
-Author: Stan Smith <stan.smith@intel.com>\r
-Date: Fri Jan 30 09:52:33 2009 -0800\r
-\r
- Update the dapl.git tree with the latest SVN version of the\r
- ibal-scm provider.\r
- \r
- Signed-off-by: Sean Hefty <sean.hefty@intel.com>\r
-\r
-commit 83543b6cca342e25fd6408454f1261ec6835a172\r
-Author: Stan Smith <stan.smith@intel.com>\r
-Date: Fri Jan 30 09:52:25 2009 -0800\r
-\r
- Merge SVN IBAL provider code back into the main git tree.\r
- \r
- Signed-off-by: Sean Hefty <sean.hefty@intel.com>\r
-\r
-commit be0b6122d0fe4f93afc8cf3ec961702faf82fb44\r
-Author: Sean Hefty <sean.hefty@intel.com>\r
-Date: Fri Jan 30 09:52:11 2009 -0800\r
-\r
- Changes to dtest to support building on Windows.\r
- \r
- Signed-off-by: Sean Hefty <sean.hefty@intel.com>\r
-\r
-commit 9f87610c4b37e4db4d74205c14028582a2f6a79e\r
-Author: Stan Smith <stan.smith@intel.com>\r
-Date: Fri Jan 30 09:48:26 2009 -0800\r
-\r
- Add return codes to various functions.\r
- Add script (batch file) for Windows testing.\r
- \r
- Signed-off-by: Sean Hefty <sean.hefty@intel.com>\r
-\r
-commit 723067550265defdcfe6e00460a4f89f7a81fbf1\r
-Author: Sean Hefty <sean.hefty@intel.com>\r
-Date: Fri Jan 30 09:46:40 2009 -0800\r
-\r
- Merge OFED and WinOF trees for common dapl code.\r
- \r
- Signed-off-by: Sean Hefty <sean.hefty@intel.com>\r
-\r
-commit 5b37e080e1ecc6903b3ab1ac9b0d4c9f8a18b790\r
-Author: Arlin Davis <arlin.r.davis@intel.com>\r
-Date: Fri Jan 16 08:16:13 2009 -0800\r
-\r
- dtest/dapltest: use $(top_builddir) for .la files during test builds\r
- \r
- Signed-off-by: Arlin Davis <ardavis@ichips.intel.com>\r
-\r
-commit bd655d4fdd7e63e7fdeed3979926da5582f71e4b\r
-Author: Arlin Davis <arlin.r.davis@intel.com>\r
-Date: Wed Nov 26 07:12:30 2008 -0800\r
-\r
- Release 2.0.15\r
- \r
- Signed-off-by: Arlin Davis <ardavis@ichips.intel.com>\r
-\r
-commit 5d5dec42717c963d1644ee3e716459dc5d58e930\r
-Author: Arlin Davis <arlin.r.davis@intel.com>\r
-Date: Mon Oct 27 08:48:53 2008 -0800\r
-\r
- dapltest: transaction test moves to cleanup stage before rdma_read processing is complete\r
- \r
- With multiple treads, the transaction server tread can move to cleanup\r
- stage and unregister memory before the remote client process has\r
- completed the rdma read. In lieu of a rewrite to add sync messages\r
- at the end of transaction test phase, just add a delay before cleanup.\r
- \r
- Signed-off-by: Arlin Davis <ardavis@ichips.intel.com>\r
-\r
-commit 101a843e59b881bc131dfd5c2aec7e54d44f35c0\r
-Author: Arlin Davis <arlin.r.davis@intel.com>\r
-Date: Tue Oct 14 11:56:35 2008 -0700\r
-\r
- Current static registration (SR) assumes DAT_OVERRIDE or /etc/dat.conf.\r
- Change SR to include sysconfdir. SR file access in the following order:\r
- \r
- - DAT_OVERRIDE\r
- - sysconfdir\r
- - /etc\r
- \r
- if DAT_OVERRIDE is set, assume administration override\r
- and do not failover to other locations. Add debug\r
- messages for each failure and retries.\r
- \r
- Signed-off-by: Arlin Davis <ardavis@ichips.intel.com>\r
- Acked-by: Doug Ledford <dledford@redhat.com>\r
-\r
-commit c98d2169b839a73d76691acf510dd8976ddc850a\r
-Author: Arlin Davis <arlin.r.davis@intel.com>\r
-Date: Fri Oct 3 08:00:56 2008 -0700\r
-\r
- Release 2.0.14\r
- \r
- Signed-off-by: Arlin Davis <ardavis@ichips.intel.com>\r
-\r
-commit c26d0bb065f3734f09058e1e6d26dde4a3738e55\r
-Author: Arlin Davis <arlin.r.davis@intel.com>\r
-Date: Fri Oct 3 05:40:04 2008 -0700\r
-\r
- dat.conf: add ofa-v2-iwarp entry for iwarp devices\r
- \r
- Signed-off-by: Arlin Davis <ardavis@ichips.intel.com>\r
-\r
-commit d54c35c8a9ba33a464ca15f4a65b914688e5194d\r
-Author: Arlin Davis <arlin.r.davis@intel.com>\r
-Date: Fri Oct 3 05:30:10 2008 -0700\r
-\r
- dapl: adjust max_rdma_read_iov to 1 for query on iWARP devices\r
- \r
- iWarp spec allows only one iov on rdma reads\r
- \r
- Signed-off-by: Arlin Davis <ardavis@ichips.intel.com>\r
-\r
-commit 9584df22d0cb3aa5a2a2a04bf427524d17650ef7\r
-Author: Arlin Davis <arlin.r.davis@intel.com>\r
-Date: Wed Oct 1 08:17:49 2008 -0700\r
-\r
- dtest: reduce default IOV's during dat_ep_create for iWARP devices\r
- \r
- iWarp adapters tend to have less IOV resources then IB adapters.\r
- \r
- Signed-off-by: Arlin Davis <ardavis@ichips.intel.com>\r
-\r
-commit 8ca9898621acf5b9769d46ea34ee8ca1eecaf2ff\r
-Author: Arlin Davis <arlin.r.davis@intel.com>\r
-Date: Fri Sep 26 12:43:13 2008 -0700\r
-\r
- dtest: fix 32-bit build issues in dtest and dtestx examples.\r
- \r
- Signed-off-by: Arlin Davis <ardavis@ichips.intel.com>\r
-\r
-commit 42a3a4edf30115a35d9d599b51f8756814e62368\r
-Author: Arlin Davis <arlin.r.davis@intel.com>\r
-Date: Fri Sep 26 08:48:31 2008 -0700\r
-\r
- Revert "Release 2.0.14"\r
- \r
- This reverts commit 816d1c3acfb4a457c6b1cc07d657f018312f2a63.\r
- \r
- missed some fixes for package release.\r
-\r
-commit 816d1c3acfb4a457c6b1cc07d657f018312f2a63\r
-Author: Arlin Davis <arlin.r.davis@intel.com>\r
-Date: Wed Sep 24 12:13:37 2008 -0700\r
-\r
- Release 2.0.14\r
- \r
- Signed-off-by: Arlin Davis <ardavis@ichips.intel.com>\r
-\r
-commit c00d858da3113ce5463d408ab5e13e17cc9529e4\r
-Author: Arlin Davis <arlin.r.davis@intel.com>\r
-Date: Wed Sep 24 08:33:32 2008 -0700\r
-\r
- build: $(DESTDIR) prepend needed on install hooks for dat.conf\r
- \r
- All install directives that automake creates automatically\r
- have $(DESTDIR) prepended to them so that a make\r
- DESTDIR=<some_path> install will work. The hand written\r
- install hooks for dat.conf was missing DESTDIR.\r
- \r
- Signed-off-by: Doug Ledford <dledford@redhat.com>\r
-\r
-commit 860db3be4907c8ff290ce7c6b631b2117f5080bd\r
-Author: Arlin Davis <arlin.r.davis@intel.com>\r
-Date: Wed Sep 24 08:26:28 2008 -0700\r
-\r
- dapl scm: UD shares EP's which requires serialization\r
- \r
- add locking around the modify_qp state changes to avoid\r
- unnecessary modify_qp calls during multiple resolve\r
- remote AH connection events on a single EP.\r
- \r
- Signed-off-by: Arlin Davis <ardavis@ichips.intel.com>\r
-\r
-commit f2c214647b5ce53e52052d6b6bea3fbace7cc20a\r
-Author: Arlin Davis <arlin.r.davis@intel.com>\r
-Date: Sat Sep 20 16:02:00 2008 -0700\r
-\r
- dtestx: Add new options to test UD.\r
- \r
- - many to one/many EP remote AH resolution, data flow\r
- - bi-directional EP remote AH resolution, data flow\r
- \r
- Signed-off by: Arlin Davis ardavis@ichips.intel.com\r
-\r
-commit 3c218b5ed65d0b7349a86eb0fe6f6bf57e3eccd6\r
-Author: Arlin Davis <arlin.r.davis@intel.com>\r
-Date: Sat Sep 20 15:58:59 2008 -0700\r
-\r
- dapl: fixes for IB UD extensions in common code and socket cm provider.\r
- \r
- - Manage EP states base on attribute service type.\r
- - Allow multiple connections (remote_ah resolution)\r
- and accepts on UD type endpoints.\r
- - Supply private data on CR conn establishment\r
- - Add UD extension conn event type - DAT_IB_UD_PASSIVE_REMOTE_AH\r
- \r
- Signed-off by: Arlin Davis ardavis@ichips.intel.com\r
-\r
-commit 01cdd688ddebec8df6f17f92c3b72a410f50dd6f\r
-Author: Arlin Davis <arlin.r.davis@intel.com>\r
-Date: Sat Sep 20 15:49:40 2008 -0700\r
-\r
- dapl: add provider specific attribute query option for IB UD MTU size\r
- \r
- Signed-off by: Arlin Davis ardavis@ichips.intel.com\r
-\r
-commit ee3d9ce2389a14c85405dfdff67f04a5f5679e32\r
-Author: Arlin Davis <arlin.r.davis@intel.com>\r
-Date: Mon Sep 1 15:52:37 2008 -0700\r
-\r
- Release 2.0.13\r
- \r
- Signed-off by: Arlin Davis ardavis@ichips.intel.com\r
-\r
-commit f37589927fabf0feb3a09c4c7c03e18df5749fef\r
-Author: Arlin Davis <arlin.r.davis@intel.com>\r
-Date: Mon Sep 1 12:24:49 2008 -0700\r
-\r
- dapl build: add correct CFLAGS, set non-debug build by default for v2\r
- \r
- Signed-off by: Arlin Davis ardavis@ichips.intel.com\r
-\r
-commit 280f3aec6d0fa9d7e36f75711471e35333ee34cf\r
-Author: Arlin Davis <arlin.r.davis@intel.com>\r
-Date: Mon Sep 1 12:22:19 2008 -0700\r
-\r
- dapl providers: fix compiler warnings in cma and scm providers\r
- \r
- Signed-off by: Arlin Davis ardavis@ichips.intel.com\r
-\r
-commit 60c0589a2aba520ae67b1c8eaad5a167edb6fba3\r
-Author: Arlin Davis <arlin.r.davis@intel.com>\r
-Date: Mon Sep 1 12:20:08 2008 -0700\r
-\r
- dat: fix compiler warnings in dat common code\r
- \r
- Signed-off by: Arlin Davis ardavis@ichips.intel.com\r
-\r
-commit fb3e368db07d02a3daa1d12d71f62ac4e7a5ef23\r
-Author: Arlin Davis <arlin.r.davis@intel.com>\r
-Date: Mon Sep 1 12:18:48 2008 -0700\r
-\r
- dapl: fix compiler warnings in common code\r
- \r
- Signed-off by: Arlin Davis ardavis@ichips.intel.com\r
-\r
-commit 38a53ae75bce5059a84262fe1b40eacf92b22287\r
-Author: Arlin Davis <arlin.r.davis@intel.com>\r
-Date: Mon Sep 1 12:16:58 2008 -0700\r
-\r
- dtest/dapltest: fix compiler warnings\r
- \r
- Signed-off by: Arlin Davis ardavis@ichips.intel.com\r
-\r
-commit 45900087764917b354411fdd2b3880473d553ab8\r
-Author: Arlin Davis <arlin.r.davis@intel.com>\r
-Date: Fri Aug 22 14:51:22 2008 -0700\r
-\r
- dapl cma: debug message during query needs definition for inet_ntoa\r
- \r
- Signed-off by: Arlin Davis ardavis@ichips.intel.com\r
-\r
-commit 99d46313a03af18771966cf86fcc934d179627b5\r
-Author: Arlin Davis <arlin.r.davis@intel.com>\r
-Date: Thu Aug 21 12:54:58 2008 -0700\r
-\r
- Release 2.0.12\r
- \r
- Signed-off by: Arlin Davis ardavis@ichips.intel.com\r
-\r
-commit 0b2a3fac8d28d5b3c2e1416fa696fe4cbc672f00\r
-Author: Arlin Davis <arlin.r.davis@intel.com>\r
-Date: Wed Aug 20 18:51:00 2008 -0700\r
-\r
- dapl scm: fix corner case that delivers duplicate disconnect events\r
- \r
- Signed-off by: Arlin Davis ardavis@ichips.intel.com\r
-\r
-commit 96f6822b90fa880a6c6a64b1e183064a449f7237\r
-Author: Arlin Davis <arlin.r.davis@intel.com>\r
-Date: Wed Aug 20 18:47:19 2008 -0700\r
-\r
- dat: include stddef.h for NULL definition in dat_platform_specific.h\r
- \r
- Signed-off by: Arlin Davis ardavis@ichips.intel.com\r
-\r
-commit 283e37c09ad072d69d29e28225e9a6e8bf3f75f0\r
-Author: Arlin Davis <arlin.r.davis@intel.com>\r
-Date: Wed Aug 20 18:27:08 2008 -0700\r
-\r
- dapl: add debug messages during async and overflow events\r
- \r
- Signed-off by: Arlin Davis ardavis@ichips.intel.com\r
-\r
-commit 4a7965ac07b7f2ee1deb5b144ed50b30c1749d38\r
-Author: Arlin Davis <arlin.r.davis@intel.com>\r
-Date: Wed Aug 20 18:24:33 2008 -0700\r
-\r
- dapltest: add check for duplicate disconnect events in transaction test\r
- \r
- Signed-off by: Arlin Davis ardavis@ichips.intel.com\r
-\r
-commit 7e8986f2c6496851b724a007458881c3248ac998\r
-Author: Arlin Davis <arlin.r.davis@intel.com>\r
-Date: Wed Aug 20 18:22:42 2008 -0700\r
-\r
- dtestx: fix stack corruption problem with hostname strcpy\r
- \r
- Signed-off by: Arlin Davis ardavis@ichips.intel.com\r
-\r
-commit ee2a553762abd6ebede99db5d26d1ba1d74ba3cb\r
-Author: Arlin Davis <arlin.r.davis@intel.com>\r
-Date: Thu Aug 14 09:42:57 2008 -0700\r
-\r
- dapl scm: use correct device attribute for max_rdma_read_out, max_qp_init_rd_atom\r
- \r
- Signed-off by: Arlin Davis ardavis@ichips.intel.com\r
-\r
-commit 142a8e4a99259fa6fe31f74ce6d0dac1017f381b\r
-Author: Arlin Davis <arlin.r.davis@intel.com>\r
-Date: Thu Aug 14 09:19:53 2008 -0700\r
-\r
- dapl scm: change IB RC qp inline and timer defaults.\r
- \r
- rnr nak can be the result of any operation not just\r
- message send recevier not ready. Timer is much too\r
- large given this case.\r
- \r
- Signed-off by: Arlin Davis ardavis@ichips.intel.com\r
-\r
-commit 36088a92d87e833bae535fcea0c45417dec34e65\r
-Author: Arlin Davis <arlin.r.davis@intel.com>\r
-Date: Thu Aug 14 09:12:38 2008 -0700\r
-\r
- dapl scm: add mtu adjustments via environment, default = 1024.\r
- \r
- DAPL_IB_MTU adjusts path mtu setting for RC qp's. Default setting\r
- is min of 1024 and active mtu on IB device.\r
- \r
- Signed-off by: Arlin Davis ardavis@ichips.intel.com\r
-\r
-commit a7dabdc4c1ddc62cc0384d60e8157ee829f12898\r
-Author: Arlin Davis <arlin.r.davis@intel.com>\r
-Date: Wed Aug 13 14:17:40 2008 -0700\r
-\r
- dapl scm: change connect and accept to non-blocking to avoid blocking user thread.\r
- \r
- The connect socket that is used to exchange QP information is now non-blocking\r
- and the data exchange is done via the cr thread. New state RTU_PENDING added.\r
- On the passive side there is a new state ACCEPT_DATA used to avoid read blocking\r
- on the user accept call.\r
- \r
- Signed-off by: Arlin Davis ardavis@ichips.intel.com\r
-\r
-commit 7e25c0f21d755cce3aa7aff993fb0baddaafc0e8\r
-Author: Arlin Davis <arlin.r.davis@intel.com>\r
-Date: Tue Jul 29 08:18:25 2008 -0700\r
-\r
- dapl scm: update max_rdma_read_iov, max_rdma_write_iov EP attributes during query\r
- \r
- Signed-off by: Arlin Davis ardavis@ichips.intel.com\r
-\r
-commit fd9909618fdfff0eb2c8ce97bff61ea98ec44a8e\r
-Author: Arlin Davis <arlin.r.davis@intel.com>\r
-Date: Sun Jul 20 13:20:45 2008 -0700\r
-\r
- Release 2.0.11\r
- \r
- Signed-off by: Arlin Davis ardavis@ichips.intel.com\r
-\r
-commit 0003bb7866af3ba73cc79c703b565a8012439bb1\r
-Author: Arlin Davis <arlin.r.davis@intel.com>\r
-Date: Sun Jul 20 13:17:22 2008 -0700\r
-\r
- dtestx: add -d option to test new IB UD extension.\r
- \r
- modify dtestx to connect peer UD QP's and exchange/verify messages as an example for new extension.\r
- \r
- Signed-off by: Arlin Davis ardavis@ichips.intel.com\r
-\r
-commit 9816ac15f98f6e73cf7b371ac6e1e312d4171c0b\r
-Author: Arlin Davis <arlin.r.davis@intel.com>\r
-Date: Sun Jul 20 13:13:09 2008 -0700\r
-\r
- dapl scm: add support for UD extensions in Openfabrics socket cm provider\r
- \r
- add qp_type in connection information exchange\r
- add new post_send_ud call\r
- changes to connection manager to support qp types beyond RC.\r
- changes to connection events to use new extended event calls.\r
- exchange address handle information during connection phase.\r
- changes to modify_qp to handle both RC and UD types.\r
- \r
- Signed-off by: Arlin Davis ardavis@ichips.intel.com\r
-\r
-commit 927dac5b61e64868089acd49d468b98327e14a1a\r
-Author: Arlin Davis <arlin.r.davis@intel.com>\r
-Date: Sun Jul 20 13:07:34 2008 -0700\r
-\r
- dapl: add support for UD extensions in common code.\r
- \r
- allow EP create for extended service types.\r
- extend connection event types to include UD AH resolution/exchange.\r
- add new extended connect and connect request upcalls for providers.\r
- - dapls_evd_post_cr_event_ext\r
- - dapls_evd_post_connection_event_ext\r
- \r
- Signed-off by: Arlin Davis ardavis@ichips.intel.com\r
-\r
-commit be3d6a53f3340294697706ce50a05faf151aacc7\r
-Author: Arlin Davis <arlin.r.davis@intel.com>\r
-Date: Sun Jul 20 12:57:49 2008 -0700\r
-\r
- dat: New definitions for IB unreliable datagram extension\r
- \r
- Extend EP dat_service_type, with DAT_IB_SERVICE_TYPE_UD\r
- Add IB extension call dat_ib_post_send_ud().\r
- Add address handle definition for UD calls.\r
- Add IB event definitions to provide remote AH via connect and connect requests\r
- Roll IB extension version to 2.0.2\r
- \r
- Signed-off by: Arlin Davis ardavis@ichips.intel.com\r
-\r
-commit 8f65d4c96a7e13f7092d82f2f5aad477a61a57c1\r
-Author: Arlin Davis <arlin.r.davis@intel.com>\r
-Date: Fri Jul 18 07:46:20 2008 -0700\r
-\r
- dat: allow TYPE_ERR messages to be turned off with DAT_DBG_TYPE\r
- \r
- Signed-off by: Arlin Davis ardavis@ichips.intel.com\r
-\r
-commit 4fefc6bca4e21c9d757923bf13bc93c62dbff17d\r
-Author: Arlin Davis <arlin.r.davis@intel.com>\r
-Date: Wed Jun 25 08:02:11 2008 -0700\r
-\r
- dapltest: manpage - rdma write example incorrect\r
- \r
- parameter for rdma write should be RW and not WR\r
- \r
- Signed-off by: Arlin Davis ardavis@ichips.intel.com\r
-\r
-commit 68638bde71b529a142c13ac332cd44435cabc896\r
-Author: Arlin Davis <arlin.r.davis@intel.com>\r
-Date: Mon Jun 23 15:26:30 2008 -0700\r
-\r
- dapl: remove needless terminating 0 in dto_op_str functions.\r
- \r
- Signed-off by: Arlin Davis ardavis@ichips.intel.com\r
-\r
-commit cb1f2a144ecd375d2143d6e176a6a92a18556d7e\r
-Author: Arlin Davis <arlin.r.davis@intel.com>\r
-Date: Mon Jun 23 10:52:46 2008 -0700\r
-\r
- Release 2.0.10\r
- \r
- Signed-off by: Arlin Davis ardavis@ichips.intel.com\r
-\r
-commit d9b3c06ae98ac4d3b264384f6510137166d78cb0\r
-Author: Arlin Davis <arlin.r.davis@intel.com>\r
-Date: Mon Jun 23 10:35:17 2008 -0700\r
-\r
- remove reference to doc/dat.conf in makefile.am\r
- \r
- Signed-off by: Arlin Davis ardavis@ichips.intel.com\r
-\r
-commit b052d402e09ac78281a25af2c8fe902fa71f5c6f\r
-Author: Arlin Davis <arlin.r.davis@intel.com>\r
-Date: Thu Jun 19 13:34:49 2008 -0700\r
-\r
- dapl scm: fix ibv_destroy_cq busy error condition during dat_evd_free.\r
- \r
- Problem surfaced while running Intel MPI 3.1 and mpich2-test suite.\r
- dapli_destroy_cm was incorrectly removing reference to qp_handle in endpoint\r
- and destroy_cq and destroy_pd code was ignoring verbs errors.\r
- \r
- Signed-off by: Arlin Davis ardavis@ichips.intel.com\r
-\r
-commit e466d8e330c45176c5f00efda79ad745bf3f71a4\r
-Author: Arlin Davis <arlin.r.davis@intel.com>\r
-Date: Wed Jun 18 14:21:28 2008 -0700\r
-\r
- dapl scm: add stdout logging for uname and gethostbyname errors during open.\r
- \r
- Signed-off by: Arlin Davis ardavis@ichips.intel.com\r
-\r
-commit 8e1f4db702cacdd2b9b95083db54ec38c9d2f5e5\r
-Author: Arlin Davis <arlin.r.davis@intel.com>\r
-Date: Wed Jun 18 14:19:51 2008 -0700\r
-\r
- dapl scm: support global routing and set mtu based on active_mtu\r
- \r
- Signed-off by: Arlin Davis ardavis@ichips.intel.com\r
-\r
-commit fc65a08727c59c304dad20337a8bff803f2302c0\r
-Author: Arlin Davis <arlin.r.davis@intel.com>\r
-Date: Wed Jun 18 13:59:44 2008 -0700\r
-\r
- dapl scm: Adding socket cm provider for better scalability on large homogeneous clusters.\r
- \r
- Bring socket cm provider back to life with some changes:\r
- \r
- better threading support for exchanging QP information.\r
- Avoid blocking during connect to support dynamic connection\r
- model with MPI implementations.\r
- \r
- consumer control of ack timeout/retries.\r
- \r
- disconnect/reject capabilities via socket exchange.\r
- \r
- version support for wire protocol to insure compatibility\r
- with peer scm provider. Add gids to exchange.\r
- \r
- validated with Intel MPI on a 14,000+ core fabric using IB DDR.\r
- \r
- Signed-off by: Arlin Davis ardavis@ichips.intel.com\r
-\r
-commit 0855af175fec2f1bec8391ebae2a2cdff26a3359\r
-Author: Arlin Davis <arlin.r.davis@intel.com>\r
-Date: Wed Jun 11 10:43:24 2008 -0700\r
-\r
- dapl: add opcode to string function to report opcode during failures.\r
- \r
- Need to use cookie opcode during failures in lieu of cqe opcode.\r
- \r
- Signed-off by: Arlin Davis ardavis@ichips.intel.com\r
-\r
-commit ec6296e7f0a843c69231f8284ae780014fa26fbe\r
-Author: Arlin Davis <arlin.r.davis@intel.com>\r
-Date: Mon Jun 16 14:59:43 2008 -0700\r
-\r
- dapl: remove unused iov buffer allocation on the endpoint\r
- \r
- provider's manage iov space on stack during posting.\r
- \r
- Signed-off by: Arlin Davis ardavis@ichips.intel.com\r
-\r
-commit c678414ced8bc9cbe21b2fc1d54aa2af73ba6528\r
-Author: Arlin Davis <arlin.r.davis@intel.com>\r
-Date: Mon Jun 16 13:59:11 2008 -0700\r
-\r
- dapl: endpoint pending request count is wrong\r
- \r
- The code assumes every cookie allocated during posting of\r
- requests gets completed. This incorrect assumption results in\r
- wrong pending count. Remove request_pending field and replace\r
- with direct call, dapl_cb_pending, to provide accurate\r
- data to consumer.\r
- \r
- Add debug print if consumer overruns request queue.\r
- \r
- Signed-off by: Arlin Davis ardavis@ichips.intel.com\r
-\r
-commit c446a3a3f3ecbd91ab583fee35bf613ab10defcd\r
-Author: Arlin Davis <arlin.r.davis@intel.com>\r
-Date: Mon Jun 2 12:40:45 2008 -0700\r
-\r
- dapl extension: dapli_post_ext should always allocate cookie for requests.\r
- \r
- extension didn't allocate cookie if completion was suppressed which resulted\r
- segfault during provider post call. Provider's expect cookie for wr_id,\r
- even with surpressed completions, to handle events during errors.\r
- \r
- Signed-off by: Arlin Davis ardavis@ichips.intel.com\r
-\r
-commit 2583f07d9d0f55eee14e0b0e6074bc6fd0712177\r
-Author: Arlin Davis <arlin.r.davis@intel.com>\r
-Date: Tue May 20 14:35:43 2008 -0700\r
-\r
- Release 2.0.9\r
- \r
- Signed-off by: Arlin Davis ardavis@ichips.intel.com\r
-\r
-commit 9b38e7b792c48ea63c2078fc6c10bb843d96bd06\r
-Author: Arlin Davis <arlin.r.davis@intel.com>\r
-Date: Tue May 20 21:56:06 2008 -0700\r
-\r
- dtest,dtestx,dapltest: fix build issues with Redhat EL5.1\r
- \r
- need include files/definitions for sleep, getpid, gettimeofday\r
- \r
- Signed-off by: Arlin Davis ardavis@ichips.intel.com\r
-\r
-commit 8084ebf39729bac310447467b518df4248e9e2b6\r
-Author: Arlin Davis <arlin.r.davis@intel.com>\r
-Date: Tue May 20 14:31:09 2008 -0700\r
-\r
- dapl: Fix long delays with the cma provider open call when DNS is not configure on server.\r
- \r
- Open call should default to netdev names when resolving local IP address for cma binding to match dat.conf settings. The open code attempts to resolve with IP or Hostname first and if there is no DNS services setup the failover to netdev name resolution is delayed for as much as 20 seconds.\r
- \r
- Signed-off by: Arlin Davis ardavis@ichips.intel.com\r
-\r
-commit 2f603b03f4cebe7c414cbaeecb7155f7bf1fb115\r
-Author: Arlin Davis <arlin.r.davis@intel.com>\r
-Date: Tue May 20 14:30:05 2008 -0700\r
-\r
- dapl: change cma provider to use max_rdma_read_in,out from ep_attr instead of HCA max values when connecting.\r
- \r
- Signed-off by: Arlin Davis ardavis@ichips.intel.com\r
-\r
-commit 7a0e5fad6de7c6affc4ba3c608b0c56f4206c48c\r
-Author: Arlin Davis <arlin.r.davis@intel.com>\r
-Date: Wed Apr 30 14:48:21 2008 -0700\r
-\r
- Release 2.0.8\r
- \r
- Signed-off by: Arlin Davis ardavis@ichips.intel.com\r
-\r
-commit 561f09ab6f9fd46a6075ecb54ca7693f2b12f311\r
-Author: Arlin Davis <arlin.r.davis@intel.com>\r
-Date: Wed Apr 30 13:51:41 2008 -0700\r
-\r
- dapl: fix post_send, post_recv to handle 0 byte's and NULL iov handles\r
- \r
- Signed-off by: Arlin Davis ardavis@ichips.intel.com\r
-\r
-commit 0289daa3fcc4451bace8cc6b6e20ddb7bbade07e\r
-Author: Arlin Davis <arlin.r.davis@intel.com>\r
-Date: Wed Apr 30 13:25:53 2008 -0700\r
-\r
- dat: udat_sr_parser ia_name will fail on comments, turn down debug message\r
- \r
- Signed-off by: Arlin Davis ardavis@ichips.intel.com\r
-\r
-commit b93b91d48186d100c48f1a479776d56476847607\r
-Author: Arlin Davis <arlin.r.davis@intel.com>\r
-Date: Tue Apr 29 16:15:44 2008 -0700\r
-\r
- dat: cleanup error handling with static registry parsing of dat.conf\r
- \r
- change asserts to return codes, add log messages, and\r
- report errors via open instead of asserts during dat\r
- library load.\r
- \r
- Signed-off by: Arlin Davis ardavis@ichips.intel.com\r
-\r
-commit b6b1f152efabe064ab8bdcdeecbd1edd30eb732f\r
-Author: Arlin Davis <arlin.r.davis@intel.com>\r
-Date: Mon Apr 28 10:14:20 2008 -0700\r
-\r
- dapl: cma provider needs to support lower inline send default for iWARP\r
- \r
- IB and iWARP work best with different defaults. Add transport check\r
- and set default accordingly. 64 for iWARP, 200 for IB.\r
- \r
- DAPL_MAX_INLINE environment variable is still used to override.\r
- \r
- Signed-off by: Arlin Davis ardavis@ichips.intel.com\r
-\r
-commit be32d7d5beeeceac5dbb1974d3217265dc4d5461\r
-Author: Arlin Davis <arlin.r.davis@intel.com>\r
-Date: Mon Apr 28 09:44:12 2008 -0700\r
-\r
- dtestx: need to include string.h for memset/strcpy declarations\r
- \r
- Signed-off by: Arlin Davis ardavis@ichips.intel.com\r
-\r
-commit a192465714e7607529303a80d8f9a312e0c7aec6\r
-Author: Arlin Davis <arlin.r.davis@intel.com>\r
-Date: Mon Apr 28 08:41:05 2008 -0700\r
-\r
- dapl: add vendor_err with DTO error logging\r
- \r
- DAPL_GET_CQE_VENDOR_ERR added to get vendor_err via cq entry.\r
- \r
- Signed-off by: Arlin Davis ardavis@ichips.intel.com\r
-\r
-commit 6ac657a4e7e5e27254a024fca7fdead569043f9a\r
-Author: Arlin Davis <arlin.r.davis@intel.com>\r
-Date: Fri Apr 25 15:12:34 2008 -0700\r
-\r
- dapl: add check before destroying cm event channel in release\r
- \r
- library may be loaded and unloaded without calling open\r
- in which case the cm event channel is not created.\r
- \r
- Signed-off by: Arlin Davis ardavis@ichips.intel.com\r
-\r
-commit c1eb23352f44aec6faeef37d8f0c3180f6259cf8\r
-Author: Arlin Davis <arlin.r.davis@intel.com>\r
-Date: Tue Apr 22 12:55:13 2008 -0700\r
-\r
- dapl: evd_alloc doesn't check for direct_object_create errors.\r
- \r
- Fix error check in dapls_ib_wait_object_create() and dat_evd_alloc.\r
- When attempting to create large number of evd's that exceed\r
- open files limit the error was not propagated up causing\r
- a segfault. Note: there are 3 FD's required for each EVD\r
- 2 for pipe, and one for cq event_channel.\r
- \r
- Change the error reporting to indicate correct return\r
- code and send to log with non-debug builds.\r
- \r
- Signed-off by: Arlin Davis ardavis@ichips.intel.com\r
-\r
-commit c5c6229b3e7744782cace0ca1f5f1fa89198bd3a\r
-Author: Arlin Davis <arlin.r.davis@intel.com>\r
-Date: Mon Apr 14 13:10:13 2008 -0700\r
-\r
- dapl: change packaging to modify OFA provider contents of dat.conf instead of file replacement.\r
- \r
- Change the packaging to update only the OFA provider contents in dat.conf. This allows other\r
- dapl providers, other then OFA, to co-exist and configure properly. Adding manpage to explain\r
- syntax of this static configuration file.\r
- \r
- Signed-off by: Arlin Davis ardavis@ichips.intel.com\r
-\r
-commit d32b27d991bc1314eea055ce3f55bb585b11aaac\r
-Author: Arlin Davis <arlin.r.davis@intel.com>\r
-Date: Fri Apr 11 11:37:48 2008 -0700\r
-\r
- dapl openib_cma: fix hca query to use correct max_rd_atom values\r
- \r
- Signed-off by: Arlin Davis ardavis@ichips.intel.com\r
-\r
-commit ac4ccfd144c01b7f4285b0cecc6218c58d0482a8\r
-Author: Arlin Davis <arlin.r.davis@intel.com>\r
-Date: Wed Apr 9 17:26:06 2008 -0700\r
-\r
- dat: add logging by default during library load failures.\r
- \r
- Signed-off by: Arlin Davis ardavis@ichips.intel.com\r
-\r
-commit 1794e94754a1e58fcf214c2802e950124bbd1316\r
-Author: Arlin Davis <arlin.r.davis@intel.com>\r
-Date: Tue Apr 8 17:32:03 2008 -0700\r
-\r
- dtest: add private data validation with connect and accept.\r
- \r
- Include code, with build option, to validate private data with\r
- consumer rejects.\r
- \r
- Signed-off by: Arlin Davis ardavis@ichips.intel.com\r
-\r
-commit 46f21284bc05e76e82b9ad6cd9f1dbc9bcde0a28\r
-Author: Arlin Davis <arlin.r.davis@intel.com>\r
-Date: Tue Apr 8 17:25:11 2008 -0700\r
-\r
- dapl: add hooks in evd connection callback code to deliver private data with consumer reject.\r
- \r
- PEER rejects can include private data. The common code didn't support delivery\r
- via the connect event data structure. Add the necessary hooks in\r
- dapl_evd_connection_callback function and include checks in openib_cma\r
- provider to check and delivery properly. Also, fix the private data size\r
- check in dapls_ib_reject_connection function.\r
- \r
- Signed-off by: Arlin Davis ardavis@ichips.intel.com\r
-\r
-commit 90e04c6f76dd5cfebd2f2867bfe22e85b0c2f461\r
-Author: Arlin Davis <arlin.r.davis@intel.com>\r
-Date: Mon Apr 7 15:47:57 2008 -0700\r
-\r
- dapl: increase reject private data size to avoid odd byte offets.\r
- \r
- remove reject type checking on passive side since it will\r
- always be non-consumer from active side.\r
- \r
- Signed-off by: Arlin Davis ardavis@ichips.intel.com\r
-\r
-commit 353a1c8a00bb2a1380fd7a372973a5a70828da35\r
-Author: Arlin Davis <arlin.r.davis@intel.com>\r
-Date: Fri Apr 4 16:04:11 2008 -0800\r
-\r
- dapl: update vendor information for OFA v2 provider.\r
- \r
- Signed-off by: Arlin Davis ardavis@ichips.intel.com\r
-\r
-commit dbf1ea37f43caec61911dea06af801c2f906db0a\r
-Author: Arlin Davis <arlin.r.davis@intel.com>\r
-Date: Fri Apr 4 16:03:03 2008 -0800\r
-\r
- dapl: add provider vendor revision data in private data with reject\r
- \r
- Add 1 byte header containing provider/vendor major revision\r
- to distinguish between consumer and non-consumer rejects.\r
- Validate size of consumer reject privated data.\r
- \r
- Signed-off by: Arlin Davis ardavis@ichips.intel.com\r
-\r
-commit 0f71b9be594739a1fba7d74929eacd42a8cee392\r
-Author: Arlin Davis <arlin.r.davis@intel.com>\r
-Date: Thu Apr 3 17:06:27 2008 -0800\r
-\r
- dapl: add support for logging errors in non-debug build.\r
- \r
- Add debug logging (stdout, syslog) for error cases during\r
- device open, cm, async, and dto operations. Default settings\r
- are ERR for DAPL_DBG_TYPE, and stdout for DAPL_DBG_DEST.\r
- \r
- Change default configuration to build non-debug.\r
- \r
- Signed-off by: Arlin Davis ardavis@ichips.intel.com\r
-\r
-commit bea882ad9b11ac7188628a939f5227e22c914169\r
-Author: Arlin Davis <arlin.r.davis@intel.com>\r
-Date: Thu Apr 3 16:23:29 2008 -0800\r
-\r
- dapl: add support for private data in CR reject.\r
- \r
- Private data support via dat_cr_reject was added to\r
- the v2 DAT specification but dapl was never extended\r
- to support at the provider level.\r
- \r
- Signed-off by: Arlin Davis ardavis@ichips.intel.com\r
-\r
-commit b5b096b6b6949fcb9fa711ab4784f41f1bb87525\r
-Author: Arlin Davis <arlin.r.davis@intel.com>\r
-Date: Tue Apr 1 11:02:37 2008 -0800\r
-\r
- dapl: calculate private data size based on transport type and cma_hdr overhead\r
- \r
- Need to adjust CM private date size based on different transport types.\r
- Add hca_ptr to dapls_ib_private_data_size call for transport type\r
- validation via verbs device. Add definitions to include iWARP size\r
- of 512 and subtract 36 bytes for cma_hdr overhead.\r
- \r
- Signed-off by: Arlin Davis ardavis@ichips.intel.com\r
-\r
-commit 632af34f0ea15e12b572c656fe2ef561a0ad62b7\r
-Author: Arlin Davis <arlin.r.davis@intel.com>\r
-Date: Wed Mar 26 17:58:17 2008 -0800\r
-\r
- Remove improperly licensed GETOPT code from dtest source tree.\r
- \r
- Signed-off by: Arlin Davis ardavis@ichips.intel.com\r
-\r
-commit 88069fdd21f92923388dec7adbde0d1bc334e7c4\r
-Author: Arlin Davis <arlin.r.davis@intel.com>\r
-Date: Tue Mar 18 15:04:34 2008 -0800\r
-\r
- remove unnecessary assert from dapl_ep_free.\r
- \r
- dat_ep_free must handle the case where a consumer calls\r
- free in CONNECTED or DISCONNECT_PENDING states. After\r
- free calls disconnect, there may be a pending event,\r
- in which case the providers dapls_ib_qp_free will block\r
- accordingly and handle pending events.\r
- \r
- Signed-off by: Arlin Davis ardavis@ichips.intel.com\r
-\r
-commit b3f020e5cba765077049a8cf61c4ce5009fa1031\r
-Author: Patrick Marchand Latifi <patrick.latifi@qlogic.com>\r
-Date: Fri Mar 14 14:31:40 2008 -0700\r
-\r
- fix openib_scm compiler warning\r
- \r
- Cast to socklen_t since accept(2) expects an unsigned argument.\r
- \r
- Makes the openib_scm provider now build successfully when using\r
- make VERBS=<provider> (the -Werror flag was causing the build\r
- failure)\r
- \r
- Signed-off-by: Patrick Marchand Latifi <patrick.latifi@qlogic.com>\r
-\r
-commit 441996f05d3cc8b09c94c166ef736bc50c24de7e\r
-Author: Patrick Marchand Latifi <patrick.latifi@qlogic.com>\r
-Date: Fri Mar 14 14:31:34 2008 -0700\r
-\r
- fix provider-specific compiler warnings\r
- \r
- Initialize ds_array_start_p otherwise the compiler would claim\r
- that this variable could be used with an uninitialized value.\r
- \r
- Makes the uDAPL providers now build successfully when using make\r
- VERBS=<provider> (the -Werror flag was causing the build failure)\r
- \r
- Signed-off-by: Patrick Marchand Latifi <patrick.latifi@qlogic.com>\r
-\r
-commit 5a710fc43ad004ecb4603db1359abb4a4fcd77e3\r
-Author: Arlin Davis <arlin.r.davis@intel.com>\r
-Date: Tue Mar 11 09:25:07 2008 -0800\r
-\r
- uDAPL: fix query to adjust max sge for default inline of 64 bytes\r
- and include missing max_rdma_write_iov ep_attr field\r
- \r
- Signed-off by: Arlin Davis ardavis@ichips.intel.com\r
-\r
-commit e2c6bf57f78dfebc21e168df01e5876202053e08\r
-Author: Patrick Marchand Latifi <patrick.latifi@qlogic.com>\r
-Date: Fri Mar 7 09:39:22 2008 -0800\r
-\r
- uDAT: fix reuse of va_list in debugging mode\r
- \r
- Make sure we reinitialize the va_list since va_list is undefined\r
- if a function traverses the va_list with va_arg.\r
- \r
- This patch fixes the uDAT debugging case when both stdout and\r
- syslog output is wanted.\r
- \r
- Signed-off-by: Patrick Marchand Latifi <patrick.latifi@qlogic.com>\r
-\r
-commit 4c9cd7a7268c0d8afb5b4d9b31537bc50cac18fe\r
-Author: Arlin Davis <arlin.r.davis@intel.com>\r
-Date: Thu Mar 6 15:40:35 2008 -0800\r
-\r
- Add hostname and process id to debug output to aid\r
- scale-up and out debug.\r
- \r
- Signed-off by: Arlin Davis ardavis@ichips.intel.com\r
-\r
-commit 460aa6f089fcdb34d78af1c8391cbecbcc6b0db3\r
-Author: Patrick Marchand Latifi <patrick.latifi@qlogic.com>\r
-Date: Sat Feb 23 21:03:21 2008 -0800\r
-\r
- fix reuse of va_list in debugging mode\r
- \r
- Make sure we reinitialize the va_list since va_list is undefined\r
- if a function traverses the va_list with va_arg.\r
- \r
- This patch fixes the debugging case when both stdout and syslog\r
- output is wanted.\r
- \r
- Signed-off-by: Patrick Marchand Latifi <patrick.latifi@qlogic.com>\r
-\r
-commit da80af8c76f220508407a2e171c8b28b43b35bf9\r
-Author: Patrick Marchand Latifi <patrick.latifi@qlogic.com>\r
-Date: Tue Feb 19 16:54:45 2008 -0800\r
-\r
- Fix memory leak in provider specific post send/recv if there's\r
- more than DEFAULT_DS_ENTRIES iovecs.\r
- \r
- Signed-off-by: Patrick Marchand Latifi <patrick.latifi@qlogic.com>\r
-\r
-commit d78ee9f95fbe48f71eb247b01c598994edfa4d17\r
-Author: Patrick Marchand Latifi <patrick.latifi@qlogic.com>\r
-Date: Tue Feb 19 03:19:42 2008 -0800\r
-\r
- Guarantee NUL termination if hostname gets truncated.\r
- \r
- Signed-off-by: Patrick Marchand Latifi <patrick.latifi@qlogic.com>\r
-\r
-commit 3484fa0d2b1a5549d83ecc82d89b72bb86f8fe4e\r
-Author: Patrick Marchand Latifi <patrick.latifi@qlogic.com>\r
-Date: Tue Feb 19 03:19:37 2008 -0800\r
-\r
- Make sure we don't leak the hash table if dapl_hca_alloc fails.\r
- \r
- Signed-off-by: Patrick Marchand Latifi <patrick.latifi@qlogic.com>\r
-\r
-commit e4ed56be6bd64684564169d89df7ca30faffdb53\r
-Author: Patrick Marchand Latifi <patrick.latifi@qlogic.com>\r
-Date: Tue Feb 19 03:19:32 2008 -0800\r
-\r
- Fix memory leak.\r
- \r
- Signed-off-by: Patrick Marchand Latifi <patrick.latifi@qlogic.com>\r
-\r
-commit 9efd005b185cfd970d2f8c58f05cf2eaae0dcdc8\r
-Author: Patrick Marchand Latifi <patrick.latifi@qlogic.com>\r
-Date: Tue Feb 19 03:19:27 2008 -0800\r
-\r
- Fix memory leak in error path.\r
- \r
- Signed-off-by: Patrick Marchand Latifi <patrick.latifi@qlogic.com>\r
-\r
-commit d971e799bb5385a8c847cf3f863f19854e95c1b2\r
-Author: Patrick Marchand Latifi <patrick.latifi@qlogic.com>\r
-Date: Tue Feb 19 03:19:22 2008 -0800\r
-\r
- Fix memory leak\r
- \r
- Signed-off-by: Patrick Marchand Latifi <patrick.latifi@qlogic.com>\r
-\r
-commit c21f2f455af7934675a58ff825bed6cf54d457c9\r
-Author: Patrick Marchand Latifi <patrick.latifi@qlogic.com>\r
-Date: Tue Feb 19 03:19:17 2008 -0800\r
-\r
- Fix memory leak.\r
- \r
- Signed-off-by: Patrick Marchand Latifi <patrick.latifi@qlogic.com>\r
-\r
-commit ee46aa47e2a4deeded347fe18bd6321db61c6594\r
-Author: Arlin Davis <arlin.r.davis@intel.com>\r
-Date: Fri Feb 15 10:10:01 2008 -0800\r
-\r
- Release 2.0.7\r
- \r
- Signed-off by: Arlin Davis <ardavis@ichips.intel.com>\r
-\r
-commit f0ca504bc2639cb7a48528d45e9026b54dab3e57\r
-Author: Arlin Davis <arlin.r.davis@intel.com>\r
-Date: Thu Feb 14 09:46:56 2008 -0800\r
-\r
- uDAT: Make sure we initialize the dictionary entry early enough so that\r
- we can base our cleanup decisions on that variable being\r
- initialized.\r
- \r
- Signed-off-by: Patrick Marchand Latifi <patrick.latifi@qlogic.com>\r
-\r
-commit 6b11838043a6012668c7e1a22a869e9e84dc40d6\r
-Author: Arlin Davis <arlin.r.davis@intel.com>\r
-Date: Thu Feb 14 09:46:18 2008 -0800\r
-\r
- uDAT: Make sure we stay within bounds when manipulating the ia_name.\r
- \r
- Signed-off-by: Patrick Marchand Latifi <patrick.latifi@qlogic.com>\r
- Signed-off by: Arlin Davis <ardavis@ichips.intel.com>\r
-\r
-commit 3eab70b4d34f850661b51dc1b856cd8d672a5cc9\r
-Author: Arlin Davis <arlin.r.davis@intel.com>\r
-Date: Thu Feb 14 09:44:56 2008 -0800\r
-\r
- uDAT: Make sure we stay within bounds when manipulating the ia handle.\r
- Fix typo in comment.\r
- \r
- Signed-off-by: Patrick Marchand Latifi <patrick.latifi@qlogic.com>\r
-\r
-commit 7fe64f10874667062e067efb7ea8c3d385ae90b7\r
-Author: Arlin Davis <arlin.r.davis@intel.com>\r
-Date: Thu Feb 14 09:43:39 2008 -0800\r
-\r
- uDAT: Zero-out memory otherwise we might base our cleanup decisions on\r
- uninitialized memory.\r
- \r
- Signed-off-by: Patrick Marchand Latifi <patrick.latifi@qlogic.com>\r
-\r
-commit f510e7e8f5b72a6262a7f8b255926cf96c65b654\r
-Author: Arlin Davis <arlin.r.davis@intel.com>\r
-Date: Wed Feb 13 20:39:26 2008 -0800\r
-\r
- Modify default configure options to match rpm spec file\r
- --enable-debug, --enable-ext-type=ib\r
- \r
- Signed-off by: Arlin Davis <ardavis@ichips.intel.com>\r
-\r
-commit 75273f1a5c599777bb43add93f30563689fdbc10\r
-Author: Arlin Davis <arlin.r.davis@intel.com>\r
-Date: Mon Feb 11 15:43:03 2008 -0800\r
-\r
- udapl OFA provider: set listen backlog to default cma backlog max.\r
- \r
- Signed-off by: Arlin Davis <ardavis@ichips.intel.com>\r
-\r
-commit 7f173ecd7e18f25bc21a42651603922e7d71f7c7\r
-Author: Arlin Davis <arlin.r.davis@intel.com>\r
-Date: Mon Feb 11 14:50:33 2008 -0800\r
-\r
- The OFA dapl provider is checking for incorrect return code\r
- from rdma_bind_addr and rdma_listen calls. This causes an error\r
- to be returned back to the consumer instead of correctly\r
- incrementing the seed port and re-calling the OFA provider\r
- until a valid port is issued. The seed value (1000) is also\r
- incorrect and should start a non-privledged port (1024) to\r
- avoid EPERM errors when seeding the starting port value.\r
- \r
- Signed-off by: Arlin Davis <ardavis@ichips.intel.com>\r
-\r
-commit 7bcb67ba7c9b37d7c122773f542c7f7f718d4a49\r
-Author: Arlin Davis <arlin.r.davis@intel.com>\r
-Date: Mon Feb 4 16:16:10 2008 -0800\r
-\r
- Release 2.0.6 - ChangeLog\r
-\r
-commit bead5f36542a8a4c790bda8ecc8dde3e630c15e6\r
-Author: Arlin Davis <arlin.r.davis@intel.com>\r
-Date: Mon Feb 4 16:14:02 2008 -0800\r
-\r
- Release 2.0.6\r
- \r
- Signed-off by: Arlin Davis <ardavis@ichips.intel.com>\r
-\r
-commit 8c5beb870048aca286f7396549771ccb075c5c1b\r
-Author: Arlin Davis <arlin.r.davis@intel.com>\r
-Date: Mon Feb 4 16:12:47 2008 -0800\r
-\r
- Fix OFED v2 package to build against and target /dat2/include directory.\r
- \r
- Prevous patch missed dat_osd.h, dapltest/dtest incorrect.\r
- \r
- Signed-off by: Arlin Davis <ardavis@ichips.intel.com>\r
-\r
-commit 42a63bb0271f91541e7b3c3967a9a977ef6660ae\r
-Author: Arlin Davis <arlin.r.davis@intel.com>\r
-Date: Mon Feb 4 13:00:45 2008 -0800\r
-\r
- uDAT/DAPL: Fix package to build against and target /dat2/include directory.\r
- \r
- Signed-off by: Arlin Davis <ardavis@ichips.intel.com>\r
-\r
-commit b3294c738d61f44ae4d0888662bdd6b64f6d6b1f\r
-Author: Arlin Davis <arlin.r.davis@intel.com>\r
-Date: Tue Jan 29 16:34:49 2008 -0800\r
-\r
- Release 2.0.5 - ChangeLog updated\r
- \r
- Signed-off by: Arlin Davis <ardavis@ichips.intel.com>\r
-\r
-commit 356cf91905d39ade06d76ab9ace6203cd7907d93\r
-Author: Arlin Davis <arlin.r.davis@intel.com>\r
-Date: Tue Jan 29 16:33:10 2008 -0800\r
-\r
- libdat: rename libdat.so to libdat2.so to coexist with v1 devel\r
- \r
- Signed-off by: Arlin Davis <ardavis@ichips.intel.com>\r
-\r
-commit d9a9f46ee220ec9c479756acc306ed68060a662f\r
-Author: Arlin Davis <arlin.r.davis@intel.com>\r
-Date: Mon Jan 28 13:55:35 2008 -0800\r
-\r
- Release 2.0.4 - README\r
- \r
- Signed-off by: Arlin Davis <ardavis@ichips.intel.com>\r
-\r
-commit 533983cec914a7ecc6829934a56f867d7870e301\r
-Author: Arlin Davis <arlin.r.davis@intel.com>\r
-Date: Mon Jan 28 12:06:03 2008 -0800\r
-\r
- Release 2.0.4 Changelog\r
- \r
- Signed-off by: Arlin Davis <ardavis@ichips.intel.com>\r
-\r
-commit bfc5bd4b4190c7302b08c3922c17e2131bfbe605\r
-Author: Arlin Davis <arlin.r.davis@intel.com>\r
-Date: Mon Jan 28 12:04:18 2008 -0800\r
-\r
- Relase package 2.0.4\r
-\r
-commit 95935648f81ac5177ae7120b35e55483902b8c64\r
-Author: Arlin Davis <arlin.r.davis@intel.com>\r
-Date: Wed Jan 23 16:40:48 2008 -0800\r
-\r
- dapltest does not include definitions for inet_ntoa.\r
- At load time the symbol was resolved but with the\r
- default definition of int, instead of char*, it caused\r
- segfault. Add correct include files in dapl_mdep_user.h\r
- for linux.\r
- \r
- Signed-off by: Arlin Davis <ardavis@ichips.intel.com>\r
-\r
-commit 3c1bb0e56c14c5ad50876820e25933c1f4c3dde4\r
-Author: Arlin Davis <arlin.r.davis@intel.com>\r
-Date: Wed Jan 23 14:46:30 2008 -0800\r
-\r
- Add freeaddrinfo to deallocate getaddrinfo buffer.\r
- Cleanup a few printf messages.\r
- \r
- Signed-off by: Arlin Davis <ardavis@ichips.intel.com>\r
-\r
-commit a4a8ad5eb1b15be19b3a343dee71ad3013d3f4bd\r
-Author: Arlin Davis <arlin.r.davis@intel.com>\r
-Date: Wed Jan 23 14:25:21 2008 -0800\r
-\r
- Fix for uDAPL v2 using extended operation. After extension completion,\r
- the DTO cookie must be checked for type before deallocating to handle\r
- inbound immediate data in receive. The sample dtestx client will fail\r
- when running loopback if the rdma immediate is received from remote\r
- before the rdma immediate request completion fires.\r
- \r
- Bug causes following dtestx client error:\r
- dat_ib_post_rdma_write_immed\r
- returned DAT_INSUFFICIENT_RESOURCES : DAT_RESOURCE_MEMORY\r
- \r
- Signed-off by: Arlin Davis <ardavis@ichips.intel.com>\r
-\r
-commit 6dcf1763c153c27c29ba76bac35be4f6935ddd96\r
-Author: Arlin Davis <arlin.r.davis@intel.com>\r
-Date: Thu Jan 17 11:30:27 2008 -0800\r
-\r
- WinOF: update dapltest,dtest directories for windows build\r
- WinOF: add README.windows\r
- \r
- Signed-off by: Stan Smith <stan.smith@intel.com>\r
- Signed-off by: Arlin Davis <ardavis@ichips.intel.com>\r
-\r
-commit 9bececbb32dd31d4a3528e8f000a773e5c593430\r
-Author: Arlin Davis <arlin.r.davis@intel.com>\r
-Date: Thu Jan 17 11:24:46 2008 -0800\r
-\r
- WinOF: update dtestx for windows build\r
- \r
- Signed-off by: Stan Smith <stan.smith@intel.com>\r
- Signed-off by: Arlin Davis <ardavis@ichips.intel.com>\r
-\r
-commit 4dc8c5defef0f52da751a1eca4d4f35de911c3c0\r
-Author: Arlin Davis <arlin.r.davis@intel.com>\r
-Date: Thu Jan 17 11:23:21 2008 -0800\r
-\r
- WinOF: add build enviroment, cleanup old makefiles\r
- \r
- Signed-off by: Stan Smith <stan.smith@intel.com>\r
- Signed-off by: Arlin Davis <ardavis@ichips.intel.com>\r
-\r
-commit 5a9e035fa33a6307b3d2b370f64639b14dfca87e\r
-Author: Arlin Davis <arlin.r.davis@intel.com>\r
-Date: Tue Jan 15 16:59:34 2008 -0800\r
-\r
- WinOF: name collision with ibal verbs ib_cm_handle_t\r
- \r
- Signed-off by: Stan Smith <stan.smith@intel.com>\r
- Signed-off by: Arlin Davis <ardavis@ichips.intel.com>\r
-\r
-commit 7c5418a781c1dfdb37b09450b5df40363dce84b4\r
-Author: Arlin Davis <arlin.r.davis@intel.com>\r
-Date: Tue Jan 15 16:06:10 2008 -0800\r
-\r
- dtest: dat_evd_query for correct size\r
- \r
- Signed-off by: Arlin Davis <ardavis@ichips.intel.com>\r
-\r
-commit 6de5e635c01e78bde6e153b727926da900677d52\r
-Author: Arlin Davis <arlin.r.davis@intel.com>\r
-Date: Tue Jan 15 16:03:27 2008 -0800\r
-\r
- dapltest: WinOF support.\r
- \r
- Signed-off by: Stan Smith <stan.smith@intel.com>\r
-\r
-commit c680e95b96e57bd8b69b1c73e95730854fcea028\r
-Author: Arlin Davis <arlin.r.davis@intel.com>\r
-Date: Tue Jan 15 14:54:57 2008 -0800\r
-\r
- openib_cma: fix cleanup issues in destroy_cm_id\r
- add macros to convert SID and PORT\r
- fix init/responder settings in accept\r
- \r
- Signed-off by: Arlin Davis <ardavis@ichips.intel.com>\r
-\r
-commit f8f867e52ffc36a31830dd4003eb2a4b4d265c93\r
-Author: Arlin Davis <arlin.r.davis@intel.com>\r
-Date: Tue Jan 15 14:49:54 2008 -0800\r
-\r
- Modifications for WinOF build and endian support for RMR info\r
- \r
- Signed-off by: Stan Smith <stan.smith@intel.com>\r
- Signed-off by: Arlin Davis <ardavis@ichips.intel.com>\r
-\r
-commit 4c1fc48d2825efcffaa8cdf7efbdd2d41aeb56ab\r
-Author: Arlin Davis <arlin.r.davis@intel.com>\r
-Date: Tue Jan 15 14:39:18 2008 -0800\r
-\r
- WinOF: DAT_API changes, white space and tab cleanup\r
- IBAL provider code added\r
- \r
- Common code: initialize cookie pool buffer\r
- add dapl extension DTO counter\r
- add get_cqe_op_str debug call\r
- remove excess dbg in evd_dequeue\r
- add dbg assert to sp_search_cr\r
- IBAL provider support _VENDOR_IBAL_\r
- \r
- Signed-off by: Stan Smith <stan.smith@intel.com>\r
- Signed-off by: Arlin Davis <ardavis@ichips.intel.com>\r
-\r
-commit 68f2ad0af3623dec27b1223aeaca6357348eef4b\r
-Author: Arlin Davis <arlin.r.davis@intel.com>\r
-Date: Fri Jan 4 08:32:39 2008 -0800\r
-\r
- Windows specific -\r
- IBAL support in evd_create\r
- Build IB extensions by default\r
- Common code -\r
- check return status, evd_free, evd_wait\r
- add dapl_event_str function\r
- definitions for dat_os_library_error, dat_os_ungetc\r
- \r
- Signed-off by: Stan Smith <stan.smith@intel.com>\r
- Signed-off by: Arlin Davis <ardavis@ichips.intel.com>\r
-\r
-commit d21f36406408245ac979f0b9594e7d1d0b9a3852\r
-Author: Arlin Davis <arlin.r.davis@intel.com>\r
-Date: Fri Jan 4 08:32:19 2008 -0800\r
-\r
- Common code -\r
- Missing DAT_API defs\r
- casting to fix build issues\r
- bitmaps for extension debug\r
- DAPL_BAD_PTR macro\r
- \r
- Signed-off by: Stan Smith <stan.smith@intel.com>\r
- Signed-off by: Arlin Davis <ardavis@ichips.intel.com>\r
-\r
-commit 258678a381c6a0170020c48b0ba627e820abd3e7\r
-Author: Arlin Davis <arlin.r.davis@intel.com>\r
-Date: Fri Jan 4 08:31:59 2008 -0800\r
-\r
- Common code -\r
- Add DAT_API definitions for dat_redirection.h, udat_redirection.h\r
- \r
- Signed-off by: Stan Smith <stan.smith@intel.com>\r
- Signed-off by: Arlin Davis <ardavis@ichips.intel.com>\r
-\r
-commit 87d0b46a0c0f25e8828d7425e0173a0304a23f8b\r
-Author: Arlin Davis <arlin.r.davis@intel.com>\r
-Date: Fri Jan 4 08:31:27 2008 -0800\r
-\r
- Windows specific -\r
- Add dapl_ep fields ibal_cm_handle, recv_disc, sent_disc for IBAL provider\r
- Support for direct object on CQ\r
- INIT and FINI changes\r
- setup dat.conf default path, fix sr parsing\r
- Common code -\r
- Add Stan Smith as contributor\r
- O/S independent dat_os_library_error()\r
- \r
- Signed-off by: Stan Smith <stan.smith@intel.com>\r
- Signed-off by: Arlin Davis <ardavis@ichips.intel.com>\r
-\r
-commit b3ba3dc9743baf3bff243e8969edf3395d1118dd\r
-Author: Arlin Davis <arlin.r.davis@intel.com>\r
-Date: Tue Dec 11 14:44:24 2007 -0800\r
-\r
- 2/2 uDAPL changes to sync common code base with WinOF\r
- - add DAT_API to specify calling conventions (windows=__stdcall, linux= )\r
- - cleanup platform specific definitions for windows\r
- - c++ support\r
- - add handle check macros DAT_IA_HANDLE_TO_UL and DAT_UL_TO_IA_HANDLE\r
-\r
-commit 92d7eef38877ad472a91e5e9f88e4e49657716bf\r
-Author: Arlin Davis <arlin.r.davis@intel.com>\r
-Date: Tue Dec 11 14:43:05 2007 -0800\r
-\r
- 1/2 uDAT changes to sync common code base with WinOF\r
- - add DAT_API to specify calling conventions (windows=__stdcall, linux= )\r
- - cleanup platform specific definitions for windows\r
- - c++ support\r
- - add handle check macros DAT_IA_HANDLE_TO_UL and DAT_UL_TO_IA_HANDLE\r
- \r
- Signed-off by: Arlin Davis <ardavis@ichips.intel.com>\r
- Signed-off by: Stan Smith <stan.smith@intel.com>\r
-\r
-commit 95764c6da28284d29071cf01d1a09bdcb967a971\r
-Author: Arlin Davis <arlin.r.davis@intel.com>\r
-Date: Tue Dec 4 13:19:27 2007 -0800\r
-\r
- - Fix size of evd request queue on creation\r
- - Add query and checking of DTO request queue\r
- \r
- Signed-off by: Arlin Davis <ardavis@ichips.intel.com>\r
-\r
-commit 9bc97e65c1240224d7dc9d6ac9a48e7aed199ee6\r
-Merge: 11a165a... abb4356...\r
-Author: Arlin Davis <arlin.r.davis@intel.com>\r
-Date: Tue Nov 27 13:31:32 2007 -0800\r
-\r
- master_dat2.0_merge\r
-\r
-commit abb4356cd765d38cf5cff2dfbdb610b380f944a2\r
-Author: Arlin Davis <arlin.r.davis@intel.com>\r
-Date: Tue Nov 20 12:10:39 2007 -0800\r
-\r
- Release 2.0.3\r
-\r
-commit d2c66eb7363234c5a9fb82aa92df1c132e46477e\r
-Author: Arlin Davis <arlin.r.davis@intel.com>\r
-Date: Tue Nov 20 12:07:58 2007 -0800\r
-\r
- - Lower default settings (rdma ops, inline sends) for latest iWARP/IB devices\r
- - Add missing ia_query for max_iov_segments_per_rdma_write\r
- - Cleanup CMA code no longer supported by rdma_cm\r
- \r
- Signed-off by: Arlin Davis <ardavis@ichips.intel.com>\r
-\r
-commit 3a3519167bd65bd999424788f139b930b099b405\r
-Author: Arlin Davis <arlin.r.davis@intel.com>\r
-Date: Mon Nov 19 15:26:44 2007 -0800\r
-\r
- Change dapltest timers to use gettimeofday instead of get_cycles for better portability.\r
- \r
- Signed-off by: Arlin Davis <ardavis@ichips.intel.com>\r
-\r
-commit 11a165a1868b1748fe476e08fc40af620f961cd2\r
-Author: Arlin Davis <arlin.r.davis@intel.com>\r
-Date: Wed Oct 31 10:58:51 2007 -0800\r
-\r
- - DAT/DAPL Version 1.2.3 Release 1\r
- \r
- Signed-off by: Arlin Davis <ardavis@ichips.intel.com>\r
-\r
-commit bc2d39a78e31c5e9463c8fa16f0ecaf49f75a15f\r
-Author: Arlin Davis <arlin.r.davis@intel.com>\r
-Date: Wed Oct 31 10:55:48 2007 -0800\r
-\r
- ChangeLog update\r
-\r
-commit 7d3ec3d68a756f895a6c6ba8ed3d7a1d602468e9\r
-Author: Arlin Davis <arlin.r.davis@intel.com>\r
-Date: Wed Oct 31 10:55:05 2007 -0800\r
-\r
- - DAT/DAPL Version 2.0.2 Release 1\r
- \r
- Signed-off by: Arlin Davis <ardavis@ichips.intel.com>\r
-\r
-commit 43931378b9d4f5f721da828623f1e391f32f446b\r
-Author: Arlin Davis <arlin.r.davis@intel.com>\r
-Date: Tue Oct 30 09:06:24 2007 -0800\r
-\r
- - Add ChangeLog\r
- - update cma provider to report remote and local ports via dat_ep_query.\r
- \r
- Signed-off by: Arlin Davis <ardavis@ichips.intel.com>\r
-\r
-commit a65da8a86ed637bacc32e3518d6c37eeb3b496bf\r
-Author: Arlin Davis <arlin.r.davis@intel.com>\r
-Date: Thu Oct 25 14:37:14 2007 -0700\r
-\r
- Fix dapltest endian issue with mem_handle, mem_address.\r
- \r
- Signed-off-by: Shirley Ma <xma@us.ibm.com>\r
-\r
-commit 8196f1655fe6088c66dafa6ad8e4474ea8ebe1d9\r
-Author: Arlin Davis <arlin.r.davis@intel.com>\r
-Date: Thu Oct 25 14:36:12 2007 -0700\r
-\r
- Fix dapltest endian issue with mem_handle, mem_address.\r
- \r
- Signed-off-by: Shirley Ma <xma@us.ibm.com>\r
-\r
-commit 39667dbb0160d395fb20eb53a1b4e995e2e623dd\r
-Author: Arlin Davis <arlin.r.davis@intel.com>\r
-Date: Tue Oct 16 14:23:17 2007 -0700\r
-\r
- Fix dtest to build properly with extensions.\r
- \r
- Modify dat.h dat_event to include event_extension_data[8].\r
- Extend struct dat_event outside of extension build\r
- switch to enable non-extended applications to work\r
- with extended libraries. Otherwise, there is a potential\r
- for the event callee to write back too much event data\r
- and exceed callers non-extended event buffer.\r
- \r
- Signed-off by: Arlin Davis <ardavis@ichips.intel.com>\r
-\r
-commit d7134fb2bcad6f4f68410af997f8791edd788cfb\r
-Author: Arlin Davis <arlin.r.davis@intel.com>\r
-Date: Tue Oct 16 14:10:52 2007 -0700\r
-\r
- Use inet_ntoa instead of open coding it. IP addresses were being\r
- reversed on PowerPC.\r
- \r
- On PowerPC the timebase ticks at a different frequency to the CPU.\r
- dapltest currently assumes a 1:1 relationship, and gives bogus values\r
- when scaling timebase to real time.\r
- \r
- To fix this, look at the timebase field in /proc/cpuinfo instead. To\r
- keep things consistent with x86, scale that value to MHz.\r
- \r
- Signed-off-by: Anton Blanchard <anton@samba.org>\r
-\r
-commit c6710c291a4f7c0845a4535767d41d66f092fabf\r
-Author: Arlin Davis <arlin.r.davis@intel.com>\r
-Date: Tue Oct 16 14:09:56 2007 -0700\r
-\r
- Use inet_ntoa instead of open coding it. IP addresses were being\r
- reversed on PowerPC.\r
- \r
- On PowerPC the timebase ticks at a different frequency to the CPU.\r
- dapltest currently assumes a 1:1 relationship, and gives bogus values\r
- when scaling timebase to real time.\r
- \r
- To fix this, look at the timebase field in /proc/cpuinfo instead. To\r
- keep things consistent with x86, scale that value to MHz.\r
- \r
- Signed-off-by: Anton Blanchard <anton@samba.org>\r
-\r
-commit 9446029979a55e6e477fb31cfdf9ce0dc77ffa8f\r
-Author: Arlin Davis <arlin.r.davis@intel.com>\r
-Date: Tue Oct 16 14:02:36 2007 -0700\r
-\r
- Minor clean-up of cr/lf\r
- \r
- Signed-off by: Arlin Davis <ardavis@ichips.intel.com>\r
-\r
-commit 33fd0628497911df11dea640aea4660e54989ed6\r
-Author: Arlin Davis <arlin.r.davis@intel.com>\r
-Date: Tue Oct 2 16:01:37 2007 -0700\r
-\r
- Final changes for 2.0.1-1 package, OFED 1.3 ALPHA\r
- Fix build issue with SLES 10, gcc++ compiler\r
- \r
- Signed-off-by: Jimmy Hill <jimmy.hill@us.ibm.com>\r
- Signed-off-by: Arlin Davis <ardavis@ichips.intel.com>\r
-\r
-commit bc5f16991d75ff9d09e3e3a3cc8c2d6801a9d61f\r
-Author: Arlin Davis <arlin.r.davis@intel.com>\r
-Date: Tue Oct 2 14:50:02 2007 -0700\r
-\r
- Final changes for package 1.2.2-1, OFED 1.3 ALPHA\r
- \r
- Signed-off by: Arlin Davis <ardavis@ichips.intel.com>\r
-\r
-commit 52bc16939e87587f8208e775dd061f54196a9acb\r
-Author: Arlin Davis <arlin.r.davis@intel.com>\r
-Date: Tue Oct 2 11:58:46 2007 -0700\r
-\r
- Change v2 dat.conf provider names to associate with ib net devices\r
- \r
- Signed-off by: Arlin Davis <ardavis@ichips.intel.com>\r
-\r
-commit d22e62f989dd16d503d5430ffe6f55075139e057\r
-Author: Arlin Davis <arlin.r.davis@intel.com>\r
-Date: Tue Oct 2 11:43:34 2007 -0700\r
-\r
- Change DT_Mdep_GetTime to use gettimeofday() which has more resolution\r
- than times().\r
- \r
- Signed-off-by: Anton Blanchard <anton@samba.org>\r
-\r
-commit a64eae5bd36598a5c93010e07869e95599aa8ceb\r
-Author: Arlin Davis <arlin.r.davis@intel.com>\r
-Date: Tue Oct 2 11:41:40 2007 -0700\r
-\r
- Change v2 dat.conf provider names to associate with ib net devices\r
- \r
- Signed-off by: Arlin Davis <ardavis@ichips.intel.com>\r
-\r
-commit 870764dfad8df0ffe6d3d449e7a8e296cfee8ef5\r
-Author: Arlin Davis <arlin.r.davis@intel.com>\r
-Date: Tue Oct 2 11:35:21 2007 -0700\r
-\r
- Change DT_Mdep_GetTime to use gettimeofday() which has more resolution\r
- than times().\r
- \r
- Signed-off-by: Anton Blanchard <anton@samba.org>\r
-\r
-commit 66bf23e3e53f370c92803f162144947f29ce06d8\r
-Author: Arlin Davis <arlin.r.davis@intel.com>\r
-Date: Tue Oct 2 11:30:15 2007 -0700\r
-\r
- Change DT_Mdep_GetTime to use gettimeofday() which has more resolution\r
- than times().\r
- \r
- Signed-off-by: Anton Blanchard <anton@samba.org>\r
-\r
-commit c220760bbb1f6357b6e187ff6c5e576dd74fd504\r
-Author: Arlin Davis <arlin.r.davis@intel.com>\r
-Date: Tue Oct 2 10:39:09 2007 -0700\r
-\r
- Fix dapl to compile as both 32bit and 64bit on PowerPC. Instead of using\r
- the kernel atomic routines, code them explicitely like x86 does.\r
- \r
- Signed-off-by: Anton Blanchard <anton@samba.org>\r
-\r
-commit 9ffdf3722e5a9cde3fd6add5d7b3940a7ea9f71f\r
-Author: Arlin Davis <arlin.r.davis@intel.com>\r
-Date: Fri Sep 28 17:04:54 2007 -0700\r
-\r
- Clean up packaging, modify dat.conf for v2.0 cma provider name change to ofa\r
- \r
- Signed-off by: Arlin Davis <ardavis@ichips.intel.com>\r
-\r
-commit 493e65b5b47f47e4824f775959fd98e56aeaccc4\r
-Author: Arlin Davis <arlin.r.davis@intel.com>\r
-Date: Fri Sep 28 17:03:41 2007 -0700\r
-\r
- Clean up packaging, modify dat.conf for cma provider name change to ofa\r
- \r
- Signed-off by: Arlin Davis <ardavis@ichips.intel.com>\r
-\r
-commit 99f0a0bf0a0d99fee0729fba0fcdf6f3e89e2ec4\r
-Author: Arlin Davis <arlin.r.davis@intel.com>\r
-Date: Thu Sep 20 12:25:55 2007 -0700\r
-\r
- Modifications to coexist with 2.0 libraries\r
- - fix RPM specfile, configure.in, 1.2.2 package\r
- - modify dat.conf\r
- \r
- Signed-off by: Arlin Davis <ardavis@ichips.intel.com>\r
-\r
-commit 7ff4a8a8e861b0701b5b2a6fc95e6aa8b36d2662\r
-Author: Arlin Davis <arlin.r.davis@intel.com>\r
-Date: Thu Sep 20 10:55:19 2007 -0700\r
-\r
- Modifications to coexist with 1.2 libraries\r
- - cleanup CR-LF in dtestx\r
- - fix RPM specfile, 2.0.1 package\r
- \r
- Signed-off by: Arlin Davis <ardavis@ichips.intel.com>\r
-\r
-commit 230767742b8287490373c09d1bd346337b375b48\r
-Author: Arlin Davis <arlin.r.davis@intel.com>\r
-Date: Fri Jun 22 11:48:20 2007 -0700\r
-\r
- Update copyright in specfile\r
-\r
-commit 5294cbe5e58f67d0a98862edea3684fff6e773bb\r
-Author: Arlin Davis <arlin.r.davis@intel.com>\r
-Date: Fri Jun 22 11:47:14 2007 -0700\r
-\r
- Update Copyright in specfile\r
-\r
-commit 3654c6ef425f94b9f27a593b0b8c1f3d7cc39029\r
-Author: Arlin Davis <arlin.r.davis@intel.com>\r
-Date: Wed Jun 6 13:20:38 2007 -0700\r
-\r
- Update specfile to reflect DAT/DAPL 1.2.1 release\r
-\r
-commit babb95eff1bcef88bed46e92c323193d8f039eff\r
-Author: Arlin Davis <arlin.r.davis@intel.com>\r
-Date: Wed Jun 6 11:48:07 2007 -0700\r
-\r
- More changes to the release notes\r
-\r
-commit 0f299bf1deb9198b2f008c3ffa717bef252b6097\r
-Author: Arlin Davis <arlin.r.davis@intel.com>\r
-Date: Tue Jun 5 15:56:17 2007 -0700\r
-\r
- Update release notes\r
-\r
-commit ad70f98a228ade4a863ca349990eaa7ab1e82ec2\r
-Author: Arlin Davis <arlin.r.davis@intel.com>\r
-Date: Tue Jun 5 15:52:18 2007 -0700\r
-\r
- Add release notes updated for OFED 1.2 release\r
-\r
-commit f332badb80f0b1d88bf2d70dba0c90afc40f088f\r
-Author: Arlin Davis <arlin.r.davis@intel.com>\r
-Date: Thu May 31 12:43:28 2007 -0700\r
-\r
- Add provider specific interface to uDAPL for extensions support.\r
- Fix memory leak with extensions, missing cookie deallocation when processing extended DTO events\r
- Remove unnecessary check on dats_set_ia_handle in dat_ia_openv\r
- Clean up specfile and some extra LF's\r
-\r
-commit 4d7e30586402149228a30bea3036466395577ec4\r
-Author: Arlin Davis <arlin.r.davis@intel.com>\r
-Date: Wed May 16 12:56:39 2007 -0700\r
-\r
- add iwarp extension include file\r
-\r
-commit d9963cc9984c06f147b92877945e847f657cd512\r
-Author: Arlin Davis <arlin.r.davis@intel.com>\r
-Date: Wed May 16 12:52:38 2007 -0700\r
-\r
- clean up some CR's in include files\r
-\r
-commit 80f4e8c4be02bac5d472e1e6c4937079882a0388\r
-Author: Arlin Davis <arlin.r.davis@intel.com>\r
-Date: Wed May 9 16:21:16 2007 -0700\r
-\r
- Take out references to specific 1.1 verbs definitions (ibv_comp_channel).\r
-\r
-commit 544fbb873d5320e9606f176c4b71bcba0e257d7d\r
-Author: Arlin Davis <arlin.r.davis@intel.com>\r
-Date: Wed May 9 12:51:53 2007 -0700\r
-\r
- Bug Fixes:\r
- - 606: Return local and remote ports with dat_ep_query\r
- - 585: Add bonding example to dat.conf\r
-\r
-commit e64079f7b961aa9a672711f0e933a77f3999d302\r
-Author: Arlin Davis <arlin.r.davis@intel.com>\r
-Date: Mon May 7 15:54:59 2007 -0700\r
-\r
- Update dapltest to use default device OpenIB-cma\r
- Fix dapltest manpage, example for performance test wrong\r
-\r
-commit 7cda2304a6880371ec2d9451a4f83a7a254bc474\r
-Author: Arlin Davis <arlin.r.davis@intel.com>\r
-Date: Mon May 7 12:49:18 2007 -0700\r
-\r
- Fix issues with IB DTO extensions\r
- - debug print_cqe updated for latest IBV definitions\r
- - dapli_evd_cq_to_event modified to handle both post and receive extensions\r
- - dtestx example tested with rdma_write_imm, fetch_add, and cmp_swap\r
- - modify cookie DTO types to support extensions properly\r
-\r
-commit 82a6750d31cd432c7e09298fc98c0e7e74abd012\r
-Author: Arlin Davis <arlin.r.davis@intel.com>\r
-Date: Fri May 4 17:17:41 2007 -0700\r
-\r
- Add new dapl functions for 2.0 (stubs for now)\r
- - dapl_ep_post_send_invalidate\r
- - dapl_ep_post_rdma_read_to_rmr\r
- - dapl_ia_ha\r
- - dapl_csp_create, query, free\r
- - dapl_cno_trigger\r
- - dapl_cno_fd_create\r
- - dapl_extensions\r
- \r
- Add new 2.0 parameters to existing API's\r
- - dapl_cr_reject\r
- - dapl_rmr_bind\r
-\r
-commit 8679aaf56c781715adc132a38a731e36194570f1\r
-Author: Arlin Davis <arlin.r.davis@intel.com>\r
-Date: Thu May 3 09:44:43 2007 -0700\r
-\r
- update dtestx.c, default provider OpenIB-cma\r
-\r
-commit 527f6d19125e9eec7ecef03a8633626b0043a2f4\r
-Author: Arlin Davis <arlin.r.davis@intel.com>\r
-Date: Wed May 2 17:27:44 2007 -0700\r
-\r
- Added IB extensions and dtestx as an example program using\r
- rdma write with immediate and atomic operations\r
-\r
-commit 83ac961b505346708f12d59152146f3b04c8306f\r
-Author: Arlin Davis <arlin.r.davis@intel.com>\r
-Date: Mon Apr 30 10:55:59 2007 -0700\r
-\r
- Fixes atomic operation build problem with ia64 and RHEL5.\r
-\r
-commit 04da88bb70ee33b249a4cf2f5a92122eeec83e3c\r
-Author: Arlin Davis <arlin.r.davis@intel.com>\r
-Date: Thu Apr 26 17:29:37 2007 -0700\r
-\r
- Update README and dapltest manpage\r
-\r
-commit 9a951d0a8713657da90568c0613eb48f5010cf1e\r
-Author: Arlin Davis <arlin.r.davis@intel.com>\r
-Date: Tue Apr 3 16:35:17 2007 -0700\r
-\r
- Cleanup RPM specfile for the dapl package, move to 1.2-1 release.\r
-\r
-commit a93f0ffcd6a46735c97ec34de564a7a91f9fc5c2\r
-Author: Arlin Davis <arlin.r.davis@intel.com>\r
-Date: Tue Apr 3 14:38:31 2007 -0700\r
-\r
- Add support for multiple IB devices to dat.conf to support IPoIB HA failover.\r
-\r
-commit 5434b720b36de2f262a02ff9dfccd99953c09e59\r
-Author: Arlin Davis <arlin.r.davis@intel.com>\r
-Date: Thu Mar 15 10:46:33 2007 -0800\r
-\r
- Fix ia64 builds on SUSE\r
-\r
-commit b0f9eef1aa7f279802da233480cf6c495e16565b\r
-Author: Arlin Davis <arlin.r.davis@intel.com>\r
-Date: Wed Mar 14 13:29:08 2007 -0800\r
-\r
- DAT 2.0 support for new DAT_VA_TYPE added to uDAPL and openib-cma provider\r
-\r
-commit 61858b925f4c1a6f9edba6389a5bd601daf936e9\r
-Author: Arlin Davis <arlin.r.davis@intel.com>\r
-Date: Wed Mar 14 11:08:19 2007 -0800\r
-\r
- change DAT_VERSION major and minor numbers to 2.0\r
-\r
-commit 6dcb2e083bda8f2521bd3515b329c5465e1ac724\r
-Author: Arlin Davis <arlin.r.davis@intel.com>\r
-Date: Wed Mar 14 10:43:56 2007 -0800\r
-\r
- add provider support to dtest, set default to OpenIB-cma\r
-\r
-commit ff8eb667c1000be6c68ca291e7ed7bd080cb73f4\r
-Author: Arlin Davis <arlin.r.davis@intel.com>\r
-Date: Tue Mar 13 16:20:20 2007 -0800\r
-\r
- add provider option to dtest, set default to OpenIB-cma\r
-\r
-commit 76a43cace54567135bac7ae54e336c6595b65fd9\r
-Author: Arlin Davis <arlin.r.davis@intel.com>\r
-Date: Fri Mar 9 13:09:14 2007 -0800\r
-\r
- Initial build with DAT 2.0 definitions and IB extensions. IB extensions configurable with --enable-ext-type=ib\r
-\r
-commit 921687efed992e6ab72dfb731687b6816324a024\r
-Author: Arlin Davis <arlin.r.davis@intel.com>\r
-Date: Thu Mar 8 16:01:29 2007 -0800\r
-\r
- Update the README\r
-\r
-commit 52ed210ae99b291f72441e71459006b5f2c851ce\r
-Author: Arlin Davis <arlin.r.davis@intel.com>\r
-Date: Wed Mar 7 15:34:41 2007 -0800\r
-\r
- - Fix bug 408, dapltest compilation failure on 32 bit arch\r
- - Update libdat.spec.in file to build uDAPL RPMs correctly\r
-\r
-commit e3f6aca57a8fa5cbaaf872bf6844feb7d5e1e66c\r
-Author: Arlin Davis <arlin.r.davis@intel.com>\r
-Date: Mon Mar 5 14:15:49 2007 -0800\r
-\r
- Fix build issues with dtest and dapltest. Define build tree path to dat/include.\r
-\r
-commit f1f829a28e645831c3bcd1eb2d465fcb7a1fd5d8\r
-Author: Arlin Davis <arlin.r.davis@intel.com>\r
-Date: Wed Feb 28 17:14:55 2007 -0800\r
-\r
- Add dapltest headers to EXTRA_DIST\r
- Modify dtest to delay before accepting\r
- \r
- Signed-off-by: Vladimir Sokolovsky <vlad@mellanox.co.il>\r
- Signed-off-by: Arlin Davis <ardavis@ichips.intel.com>\r
-\r
-commit 46b830a4664d5fee2daf1ebdc4e95ecb7c580e80\r
-Author: Arlin Davis <arlin.r.davis@intel.com>\r
-Date: Mon Feb 26 13:54:15 2007 -0800\r
-\r
- Adding dtest and dapltest to the build. Manual pages created.\r
-\r
-commit d245664e27148e54469268ad81f41b2a894a131a\r
-Author: Arlin Davis <arlin.r.davis@intel.com>\r
-Date: Fri Jan 19 16:21:30 2007 -0800\r
-\r
- uDAPL changes to support exchanging and validation of the device responder_resources and the\r
- initiator_depth during connection establishment\r
-\r
-commit 2280f833090aa9f750d5be8f9b06e7e08e642da5\r
-Author: Arlin Davis <arlin.r.davis@intel.com>\r
-Date: Wed Dec 6 11:49:27 2006 -0800\r
-\r
- Update cma provider to sync with rdma_ucm changes\r
-\r
-commit 89448545b415b6dff57e3314b020619f6b979ef8\r
-Author: Arlin Davis <arlin.r.davis@intel.com>\r
-Date: Mon Dec 4 13:54:20 2006 -0800\r
-\r
- Update autogen to create config directory\r
-\r
-commit 0a917b104eba0aae6c6ef49c7990a2dc7efc759d\r
-Author: Arlin Davis <ardavis@ichips.intel.com>\r
-Date: Tue Nov 7 20:22:05 2006 +0000\r
-\r
- r10074: Added support for new ib verbs client register event. No extra processing required at the uDAPL\r
- level.\r
- \r
- Fix some issues supporting create qp without recv cq handle or recv qp resources. IB verbs assume a\r
- recv_cq handle and uDAPL dapl_ep_create assumes there is always recv_sge resources specified.\r
- \r
- Fix some timeout and long disconnect delay issues discovered during scale-out testing. Added support\r
- to retry rdma_cm address and route resolution with configuration options. Provide a disconnect call\r
- when receiving the disconnect request to guarantee a disconnect reply and event on the remote side.\r
- The rdma_disconnect was not being called from dat_ep_disconnect() as a result of the state changing\r
- to DISCONNECTED in the event callback.\r
- \r
- Here are the new options (environment variables) with the default setting:\r
- \r
- DAPL_CM_ARP_TIMEOUT_MS 4000\r
- DAPL_CM_ARP_RETRY_COUNT 15\r
- DAPL_CM_ROUTE_TIMEOUT_MS 4000\r
- DAPL_CM_ROUTE_RETRY_COUNT 15\r
-\r
-commit c73aeb904504a0bc6cce0fb1248af9ba39521395\r
-Author: Arlin Davis <ardavis@ichips.intel.com>\r
-Date: Thu Oct 12 22:41:33 2006 +0000\r
-\r
- r9802: Remove Socket CM provider from build and dat.conf configuration. No longer needed nor supported.\r
-\r
-commit b1d94b26610f682cdd43bde2aecf5004e0865422\r
-Author: Steve Wise <swise@opengridcomputing.com>\r
-Date: Tue Sep 12 18:15:39 2006 +0000\r
-\r
- r9442: Update obsolete CLK_TCK to CLOCKS_PER_SEC\r
- Signed-off-by: Steve Wise <swise@opengridcomputing.com>\r
- Signed-off-by: James Lentini <jlentini@netapp.com>\r
-\r
-commit 99a5dddd07d4c271ebb075b5b0f800101f850a56\r
-Author: Arlin Davis <ardavis@ichips.intel.com>\r
-Date: Thu Sep 7 18:09:11 2006 +0000\r
-\r
- r9346:\r
- inadvertently added evdtest to makefile when testing dat_evd_set_unwaitable fix with openib_cma provider\r
-\r
-commit b53a87c856d9754313da9543a1dac5c6f1307085\r
-Author: Arlin Davis <ardavis@ichips.intel.com>\r
-Date: Wed Sep 6 20:36:09 2006 +0000\r
-\r
- r9315:\r
- Fill out some unitialized fields in the ia_attr structure\r
- returned by dat_ia_query().\r
- \r
- Signed-off by: Arlin Davis <ardavis@ichips.intel.com>\r
- Signed-off by: Robert Walsh <robert.walsh@qlogic.com>\r
-\r
-commit b6c4e84399d0aa44c72c6ca870409c3666d7e79b\r
-Author: Arlin Davis <ardavis@ichips.intel.com>\r
-Date: Fri Aug 11 20:44:23 2006 +0000\r
-\r
- r8895: Update dtest to support multiple segments on rdma write and change makefile to use OpenIB-cma by default.\r
-\r
-commit 4737c63d79b23c3aff329e864dd50e3cffb6a17f\r
-Author: Arlin Davis <ardavis@ichips.intel.com>\r
-Date: Wed Jul 19 17:15:06 2006 +0000\r
-\r
- r8592: Add support for dat_evd_set_unwaitable on a DTO evd in openib_cma provider\r
-\r
-commit 25fb8c376547de895a170194c09b2d72dfea789d\r
-Author: Arlin Davis <ardavis@ichips.intel.com>\r
-Date: Mon Jul 17 22:59:17 2006 +0000\r
-\r
- r8565: Added errno reporting (message and return codes) during open to help diagnose create thread issues.\r
-\r
-commit f3a1ed6232ccdee7d193e8e3b9b0013b2bd222af\r
-Author: Anton Blanchard <anton@samba.org>\r
-Date: Mon Jul 17 21:26:03 2006 +0000\r
-\r
- r8562: Fix some suspicious inline assembly:\r
- - EIEIO_ON_SMP and ISYNC_ON_SMP are in kernel UP build optimisations, we\r
- shouldnt export them to userspace. Replace it with lwsync and isync.\r
- - The comment says its implemenenting cmpxchg64 but in fact its\r
- implementing cmpxchg32. Fix the comment.\r
- \r
- Signed-off-by: Anton Blanchard <anton@samba.org>\r
- Signed-off-by: James Lentini <jlentini@netapp.com>\r
-\r
-commit 63759108a1376b6e45a4491551f71d8cafdcddc1\r
-Author: James Lentini <jlentini@netapp.com>\r
-Date: Wed Jul 12 14:56:26 2006 +0000\r
-\r
- r8503: Fix IA64 build problems reported by John Partridge <johnip@sgi.com>\r
- Signed-off-by: James Lentini <jlentini@netapp.com>\r
-\r
-commit 51362c9781cb65fd8f9a3cb5b7c12c88e4c8527a\r
-Author: Arlin Davis <ardavis@ichips.intel.com>\r
-Date: Thu Jun 22 22:02:56 2006 +0000\r
-\r
- r8182: Lower the reject debug message level so we don't see warnings\r
- when consumers reject.\r
- Signed-off-by: Arlin Davis <ardavis@ichips.intel.com>\r
- Signed-off-by: James Lentini <jlentini@netapp.com>\r
-\r
-commit f14889bb0cd22e897148ea2f6931a6b4f23143b0\r
-Author: Arlin Davis <ardavis@ichips.intel.com>\r
-Date: Thu Jun 22 21:13:37 2006 +0000\r
-\r
- r8181: Added support for active side TIMED_OUT event from a provider.\r
- Signed-off-by: Arlin Davis <ardavis@ichips.intel.com>\r
- Signed-off-by: James Lentini <jlentini@netapp.com>\r
-\r
-commit 14fc704ae20b6a3ad0d433d7de9c02ce99e095b3\r
-Author: Arlin Davis <ardavis@ichips.intel.com>\r
-Date: Thu Jun 22 20:58:06 2006 +0000\r
-\r
- r8180: Fix bug in dapls_ib_get_dat_event() call after adding new unreachable event.\r
- Signed-off-by: Arlin Davis <ardavis@ichips.intel.com>\r
- Signed-off-by: James Lentini <jlentini@netapp.com>\r
-\r
-commit 893698c31a0d8e60227806d992485a44375840cb\r
-Author: James Lentini <jlentini@netapp.com>\r
-Date: Mon Jun 19 17:20:45 2006 +0000\r
-\r
- r8112: Update for new rdma_create_id() function signature.\r
- Signed-off-by: James Lentini <jlentini@netapp.com>\r
-\r
-commit 53483d84b0d02c432d9435d2f8e840cab3ded320\r
-Author: Arlin Davis <ardavis@ichips.intel.com>\r
-Date: Wed Jun 14 16:17:39 2006 +0000\r
-\r
- r8008: Set max rdma read per EP attributes\r
- Signed-off-by: Arlin Davis <ardavis@ichips.intel.com>\r
- Signed-off-by: James Lentini <jlentini@netapp.com>\r
-\r
-commit 6cb854cd72d9492ddc1c5da01dbfb24b3f30af83\r
-Author: Arlin Davis <ardavis@ichips.intel.com>\r
-Date: Mon Jun 12 15:42:50 2006 +0000\r
-\r
- r7931: Report the proper error and timeout events.\r
- Signed-off-by: Arlin Davis <ardavis@ichips.intel.com>\r
- Signed-off-by: James Lentini <jlentini@netapp.com>\r
-\r
-commit 44a97ff1f599f4bf882a801ead7aa495ea9ba936\r
-Author: Arlin Davis <ardavis@ichips.intel.com>\r
-Date: Mon Jun 12 14:51:14 2006 +0000\r
-\r
- r7928: Socket CM fix to guard against using a loopback address\r
- as the local device address.\r
- Signed-off-by: Arlin Davis <ardavis@ichips.intel.com>\r
- Signed-off-by: James Lentini <jlentini@netapp.com>\r
-\r
-commit 152219cf83c61e459fdf3de03d4e83ddba045230\r
-Author: Arlin Davis <ardavis@ichips.intel.com>\r
-Date: Tue Jun 6 21:46:44 2006 +0000\r
-\r
- r7755: Use the uCM set_option feature to adjust connect request timeout\r
- and retry values. Also, a fix to disallow any event after a disconnect\r
- event.\r
- Signed-off-by: Arlin Davis <ardavis@ichips.intel.com>\r
- Signed-off-by: James Lentini <jlentini@netapp.com>\r
-\r
-commit 3a0686a2624ed28c7ea37b650415801f1cedbd10\r
-Author: James Lentini <jlentini@netapp.com>\r
-Date: Wed May 31 19:55:57 2006 +0000\r
-\r
- r7608: Correct comment.\r
- Signed-off-by: James Lentini <jlentini@netapp.com>\r
-\r
-commit eb760157c90f59183b424ac8e71474fe0b46094c\r
-Author: James Lentini <jlentini@netapp.com>\r
-Date: Thu May 18 21:54:12 2006 +0000\r
-\r
- r7347: Undo inadvertent change.\r
- Signed-off-by: James Lentini <jlentini@netapp.com>\r
-\r
-commit 27256222b42fecfac8a44b3f82fe2524ecc72de2\r
-Author: James Lentini <jlentini@netapp.com>\r
-Date: Thu May 18 21:50:27 2006 +0000\r
-\r
- r7346: Fix for uCMA provider to return the correct event as a result of\r
- rejects. Also, ran into a segv bug with dapl_ep_create when\r
- creating without a conn_evd.\r
- \r
- Signed-off by: Arlin Davis <ardavis@ichips.intel.com>\r
- Signed-off-by: James Lentini <jlentini@netapp.com>\r
-\r
-commit b1b6e16f3e41e123cd347bc78b01e3272076362b\r
-Author: Arlin Davis <ardavis@ichips.intel.com>\r
-Date: Fri May 12 19:50:19 2006 +0000\r
-\r
- r7141: Update the uDAPL openib_cma provider to work with the new\r
- uCMA event channel interface.\r
- Signed-off-by: Arlin Davis <ardavis@ichips.intel.com>\r
- Signed-off-by: James Lentini <jlentini@netapp.com>\r
-\r
-commit 27f9f0c106662cc7b41bcb747495860a1b6c7133\r
-Author: Steve Wise <swise@opengridcomputing.com>\r
-Date: Tue May 2 21:33:35 2006 +0000\r
-\r
- r6873: Transaction test change to comply with the iWARP MPA protocol's\r
- "Connection Startup Rules".\r
- Signed-off-by: Steve Wise <swise@opengridcomputing.com>\r
- Signed-off-by: James Lentini <jlentini@netapp.com>\r
-\r
-commit 060d09f974ffbe73672e17641b2f18d3821d31a7\r
-Author: Arlin Davis <ardavis@ichips.intel.com>\r
-Date: Fri Apr 28 13:44:17 2006 +0000\r
-\r
- r6736: getaddrinfo() fix for debug builds and some additional debug messages for\r
- connect errors and rejects.\r
- \r
- Signed-off-by: Arlin Davis <ardavis@ichips.intel.com>\r
- Signed-off-by: James Lentini <jlentini@netapp.com>\r
-\r
-commit 4e8b95bbeaf6e5c27de59ef347ec2ef0aa2e5e6b\r
-Author: James Lentini <jlentini@netapp.com>\r
-Date: Wed Apr 19 16:49:34 2006 +0000\r
-\r
- r6520: Fix compilation warning.\r
- Signed-off-by: James Lentini <jlentini@netapp.com>\r
-\r
-commit 115fcc396164066326f9447d71af798a381d063f\r
-Author: Steve Wise <swise@opengridcomputing.com>\r
-Date: Wed Apr 19 16:32:01 2006 +0000\r
-\r
- r6518: Do not always generate an event for an abrupt disconnect.\r
- Signed-off-by: Steve Wise <swise@opengridcomputing.com>\r
- Signed-off-by: James Lentini <jlentini@netapp.com>\r
-\r
-commit f959bb786cd884bf4d2a5da4d299da6297d65293\r
-Author: James Lentini <jlentini@netapp.com>\r
-Date: Wed Apr 19 16:30:37 2006 +0000\r
-\r
- r6517: Generate a disconnect event for providers that do not generate\r
- one automatically.\r
- Signed-off-by: James Lentini <jlentini@netapp.com>\r
-\r
-commit bb467511cf6e217147817ba12bbe800aae97cab5\r
-Author: James Lentini <jlentini@netapp.com>\r
-Date: Wed Apr 19 16:25:33 2006 +0000\r
-\r
- r6516: Fix compilation error.\r
- Signed-off-by: James Lentini <jlentini@netapp.com>\r
-\r
-commit 117a9856c269bf08b738a1923c92f5a1949f6cc1\r
-Author: James Lentini <jlentini@netapp.com>\r
-Date: Mon Apr 10 20:16:44 2006 +0000\r
-\r
- r6392: Fix for RDMA cm id destruction and debug builds.\r
- Signed-off by: Arlin Davis <ardavis@ichips.intel.com>\r
- Signed-off-by: James Lentini <jlentini@netapp.com>\r
-\r
-commit 5f56b2b7339c17276188464dfff12b1be9e1dbb7\r
-Author: James Lentini <jlentini@netapp.com>\r
-Date: Thu Apr 6 15:32:47 2006 +0000\r
-\r
- r6289: Set max_iov_segments_per_rdma_read and max_rdma_read_iov using the correct\r
- attribute.\r
- Signed-off-by: James Lentini <jlentini@netapp.com>\r
-\r
-commit 304f48370adcaa12463c1a7d99e513164b83810c\r
-Author: Steve Wise <swise@opengridcomputing.com>\r
-Date: Thu Apr 6 15:16:10 2006 +0000\r
-\r
- r6286: Set the IA attribute max_iov_segments_per_rdma_read and the EP attribute\r
- max_rdma_read_iov based on the openib max_sge_rd device attribute.\r
- Signed-off-by: Steve Wise <swise@opengridcomputing.com>\r
- Signed-off-by: James Lentini <jlentini@netapp.com>\r
-\r
-commit fe27222d2a00d7c5c4d98f39d2926fe14c7f32bc\r
-Author: James Lentini <jlentini@netapp.com>\r
-Date: Tue Apr 4 18:29:00 2006 +0000\r
-\r
- r6221: Change the mechanism by which library init and fini functions are specified\r
- Signed-off-by: James Lentini <jlentini@netapp.com>\r
-\r
-commit 5a0598b90ab021cb2115e3791cb38dcfc0347948\r
-Author: James Lentini <jlentini@netapp.com>\r
-Date: Mon Apr 3 17:29:55 2006 +0000\r
-\r
- r6182: Remove unused variables.\r
- Signed-off-by: James Lentini <jlentini@netapp.com>\r
-\r
-commit b8084c4edc21b5ac2191ec654a882b65bad0c77d\r
-Author: James Lentini <jlentini@netapp.com>\r
-Date: Mon Apr 3 15:29:30 2006 +0000\r
-\r
- r6179: Fix dapltest compiler warnings.\r
- Signed-off-by: James Lentini <jlentini@netapp.com>\r
-\r
-commit abdbec194670d72012d481b98b2e6f728e9c5b48\r
-Author: James Lentini <jlentini@netapp.com>\r
-Date: Mon Apr 3 14:08:48 2006 +0000\r
-\r
- r6168: Fixed debug prints of 64-bit values and removed compile warnings.\r
- Signed-off-by: James Lentini <jlentini@netapp.com>\r
-\r
-commit 52de2a35e02ddeb15887d0d690b52a4e02812e57\r
-Author: James Lentini <jlentini@netapp.com>\r
-Date: Tue Mar 21 22:28:10 2006 +0000\r
-\r
- r5939: Move libdat.spec to libdat.spec.in\r
- Signed-off-by: James Lentini <jlentini@netapp.com>\r
-\r
-commit 6b1a6356a1757ae9b9d1557bd2ae67e0913d04c2\r
-Author: Arlin Davis <ardavis@ichips.intel.com>\r
-Date: Fri Mar 17 22:02:18 2006 +0000\r
-\r
- r5879: Add GNU Autotools support and an RPM spec file\r
- Signed-off-by: Arlin Davis <ardavis@ichips.intel.com>\r
- Signed-off-by: James Lentini <jlentini@netapp.com>\r
-\r
-commit 9d00582b8be33add8d3e4173e8311b1a222b0b34\r
-Author: Arlin Davis <ardavis@ichips.intel.com>\r
-Date: Thu Mar 16 22:40:18 2006 +0000\r
-\r
- r5871: Fixes a corner case where a CMA event was not acknowledged during\r
- disconnect processing.\r
- \r
- Signed-off-by: Arlin Davis <ardavis@ichips.intel.com>\r
- Signed-off-by: James Lentini <jlentini@netapp.com>\r
-\r
-commit ede1d714a8e7ae99246eb382c1c1165f238cf7c3\r
-Author: James Lentini <jlentini@netapp.com>\r
-Date: Mon Mar 13 18:15:49 2006 +0000\r
-\r
- r5789: Reduces some debug output in the async thread and fixes listen\r
- processing of EBUSY.\r
- \r
- Signed-off by: Arlin Davis <ardavis@ichips.intel.com>\r
- Signed-off by: James Lentini <jlentini@netapp.com>\r
-\r
-commit afd558cbdada66e01aa553364b9a126da00dbe65\r
-Author: Arlin Davis <ardavis@ichips.intel.com>\r
-Date: Wed Mar 8 14:24:26 2006 +0000\r
-\r
- r5684: Remove unused NO_EP_STATE.\r
- Signed-off-by: Arlin Davis <ardavis@ichips.intel.com>\r
- Signed-off-by: James Lentini <jlentini@netapp.com>\r
-\r
-commit 4c36aca16b9b492a91c0b05002af5fc954f3b44e\r
-Author: James Lentini <jlentini@netapp.com>\r
-Date: Tue Feb 28 15:41:03 2006 +0000\r
-\r
- r5529: Fix to destroy QPs in all cases and close the HCA.\r
- \r
- Signed-off by: Arlin Davis <ardavis@ichips.intel.com>\r
- Signed-off by: James Lentini <jlentini@netapp.com>\r
-\r
-commit 5874fdb26436c58365a7096b9b68c8e45a51d7a3\r
-Author: James Lentini <jlentini@netapp.com>\r
-Date: Mon Jan 9 20:17:33 2006 +0000\r
-\r
- r4855: Make use of dat_evd_wait the default.\r
- Signed-off-by: James Lentini <jlentini@netapp.com>\r
-\r
-commit 254bd0e349bafbd970d6475efcafd7e52f05415e\r
-Author: Arlin Davis <ardavis@ichips.intel.com>\r
-Date: Mon Jan 9 16:59:38 2006 +0000\r
-\r
- r4852: Fix disconnect event processing and update dtest to validate.\r
- Signed-off-by: Arlin Davis <ardavis@ichips.intel.com>\r
- Signed-off-by: James Lentini <jlentini@netapp.com>\r
-\r
-commit f69d737ee51f6b00a1e6cf8531695a61a322651b\r
-Author: Arlin Davis <ardavis@ichips.intel.com>\r
-Date: Fri Dec 16 02:03:05 2005 +0000\r
-\r
- r4502: Query for rd_atomic values.\r
- Signed-off-by: Arlin Davis <ardavis@ichips.intel.com>\r
- Signed-off-by: James Lentini <jlentini@netapp.com>\r
-\r
-commit 86566b556920a51f1e538d245674058682012668\r
-Author: James Lentini <jlentini@netapp.com>\r
-Date: Fri Dec 16 01:57:30 2005 +0000\r
-\r
- r4501: Allow a network name, network address, or device name in the dat.conf file.\r
- Singed-off-by: Arlin Davis <ardavis@ichips.intel.com>\r
- Signed-off-by: James Lentini <jlentini@netapp.com>\r
-\r
-commit 9e0ca3d4ccb92b9c0337efbadce405076a365f0f\r
-Author: Roland Dreier <rolandd@cisco.com>\r
-Date: Wed Dec 14 20:44:36 2005 +0000\r
-\r
- r4466: Change libibverbs API for listing all known devices from\r
- ibv_get_devices() to ibv_get_device_list(), and update all\r
- in-tree uses of this API.\r
- \r
- Signed-off-by: Roland Dreier <rolandd@cisco.com>\r
-\r
-commit 73a80143ab7b3f9aad19f84f904f99b489dca6cf\r
-Author: James Lentini <jlentini@netapp.com>\r
-Date: Mon Dec 5 16:37:46 2005 +0000\r
-\r
- r4308: Fix a gcc 4.0 warning\r
- Signed-off-by: James Lentini <jlentini@netapp.com>\r
-\r
-commit aa8b16b7e83f321eaaa18b38e6c165c2f120bcec\r
-Author: Arlin Davis <ardavis@ichips.intel.com>\r
-Date: Thu Dec 1 15:03:10 2005 +0000\r
-\r
- r4279: Added CMA API support.\r
- Signed-off-by: Arlin Davis <ardavis@ichips.intel.com>\r
- Signed-off-by: James Lentini <jlentini@netapp.com>\r
-\r
-commit 74d3c32e42ab54f3d6f2eec3d0a66d08f800e075\r
-Author: James Lentini <jlentini@netapp.com>\r
-Date: Thu Nov 10 21:39:34 2005 +0000\r
-\r
- r4018: Fixed some problems with the free build openib_scm version and turned\r
- down some debugging and added some debug prints for uAT path records.\r
- Signed-off by: Arlin Davis <ardavis@ichips.intel.com>\r
- Signed-off by: James Lentini <jlentini@netapp.com>\r
-\r
-commit 70a2f23c5604e55f8f76672f78b4bf92f6a79b98\r
-Author: James Lentini <jlentini@netapp.com>\r
-Date: Mon Oct 31 18:27:13 2005 +0000\r
-\r
- r3917: Fix printing of debug statements.\r
- Signed off by: Aniruddha Bohra <bohra@cs.rutgers.edu>\r
- Signed off by: James Lentini <jlentini@netapp.com\r
-\r
-commit 4306debe3118948fca72a68ec20ef0d73b399fd1\r
-Author: James Lentini <jlentini@netapp.com>\r
-Date: Wed Oct 26 21:10:26 2005 +0000\r
-\r
- r3882: uDAPL provider for OpenIB socket CM.\r
- Signed-off by: Arlin Davis <ardavis@ichips.intel.com>\r
- Signed-off by: James Lentini <jlentini@netapp.com>\r
-\r
-commit 42a64ec2ec1d8ec71492bfebba077b006684ce97\r
-Author: James Lentini <jlentini@netapp.com>\r
-Date: Thu Oct 13 20:45:22 2005 +0000\r
-\r
- r3774: Fix the async error handling and callback mappings.\r
- Updated TODO list.\r
- Signed-off by: Arlin Davis <ardavis@ichips.intel.com>\r
- Signed-off by: James Lentini <jlentini@netapp.com>\r
-\r
-commit fea8ed1188fbeef8291cfd7e681cd48c06ff5bde\r
-Author: Robert Walsh <rjwalsh@pathscale.com>\r
-Date: Fri Oct 7 21:15:00 2005 +0000\r
-\r
- r3693: Update some more ignores.\r
-\r
-commit 303147143afa9aa72906246a9f1973e4172f75b8\r
-Author: Todd Bowman <twbowman@gmail.com>\r
-Date: Thu Oct 6 21:13:32 2005 +0000\r
-\r
- r3687: Add PPC64 support for udapl\r
- Signed-off-by: Todd Bowman <twbowman@gmail.com>\r
- Signed-off-by: James Lentini <jlentini@netapp.com>\r
-\r
-commit 103c7db321e24a7b5b06c7c26b0e0a65d1dd11ce\r
-Author: Todd Bowman <twbowman@gmail.com>\r
-Date: Thu Oct 6 15:22:08 2005 +0000\r
-\r
- r3683: Remove the dtest qualifier from the sdp range.\r
- Signed-off-by: Todd Bowman <twbowman@gmail.com>\r
- Signed-off-by: James Lentini <jlentini@netapp.com>\r
-\r
-commit 33fbf9c81ac29492394e419588d856533e7fffb8\r
-Author: James Lentini <jlentini@netapp.com>\r
-Date: Mon Oct 3 14:59:22 2005 +0000\r
-\r
- r3637: Support CQ_WAIT_OBJECT with channels and sync with latest verbs.\r
- Signed-off by: Arlin Davis <ardavis@ichips.intel.com>\r
- Signed-off by: James Lentini <jlentini@netapp.com>\r
-\r
-commit f01dac62b08d8f4fd417c336be48fb3bc8cd15c5\r
-Author: James Lentini <jlentini@netapp.com>\r
-Date: Thu Sep 29 18:13:25 2005 +0000\r
-\r
- r3619: Makefile fix.\r
- Signed-off-by: James Lentini <jlentini@netapp.com>\r
-\r
-commit 634b199218b775a8ed071c1faea519c4cc4ee4e3\r
-Author: James Lentini <jlentini@netapp.com>\r
-Date: Wed Sep 28 21:50:13 2005 +0000\r
-\r
- r3606: Fixes IA64 build problems (atomics) with the latest Redhat EL4.0 update and\r
- adds support for SuSe.\r
- Signed-off by: Arlin Davis <ardavis@ichips.intel.com>\r
- Signed-off by: James Lentini <jlentini@netapp.com>\r
-\r
-commit 3a23c7135905666ff969b86dab3e90f90ac73008\r
-Author: Robert Walsh <rjwalsh@pathscale.com>\r
-Date: Tue Sep 27 16:59:14 2005 +0000\r
-\r
- r3567: Setup svn:ignore on a bunch of directories.\r
-\r
-commit d41ea62125636a58e8748871e372810c09865b0a\r
-Author: James Lentini <jlentini@netapp.com>\r
-Date: Thu Sep 22 21:24:38 2005 +0000\r
-\r
- r3525: Improve dtest and measure RDMA reads\r
- Signed-off by: Arlin Davis <ardavis@ichips.intel.com>\r
- Signed-off-by: James Lentini <jlentini@netapp.com>\r
-\r
-commit 32258d13af6aaf76078ec6ba187d8129a0f70397\r
-Author: James Lentini <jlentini@netapp.com>\r
-Date: Wed Sep 21 19:54:07 2005 +0000\r
-\r
- r3513: Temporary workaround for the RDMA read performance anomaly\r
- Signed-off by: Arlin Davis <ardavis@ichips.intel.com>\r
- Signed-off by: James Lentini <jlentini@netapp.com>\r
-\r
-commit 990bdc4320809e0b989c90c24bef361c1bc91c7f\r
-Author: James Lentini <jlentini@netapp.com>\r
-Date: Tue Sep 20 17:00:37 2005 +0000\r
-\r
- r3494: Moved dapl_evd_modify_cno.c to match SourceForge\r
- Signed-off-by: James Lentini <jlentini@netapp.com>\r
-\r
-commit 4509fb64fdbf99db7bdcaad4d8e3884718184d86\r
-Author: James Lentini <jlentini@netapp.com>\r
-Date: Tue Sep 20 16:17:59 2005 +0000\r
-\r
- r3493: Support ib_cm_init_qp_attr(), add cm event processing on a per\r
- device basis, and add copyrights for kDAPL cm work that was\r
- used in uDAPL.\r
- Signed-off by: Arlin Davis <ardavis@ichips.intel.com>\r
- Signed-off by: James Lentini <jlentini@netapp.com>\r
-\r
-commit c1d45137c2e26024142f24207344e3e48a577b83\r
-Author: Sean Hefty <sean.hefty@intel.com>\r
-Date: Thu Sep 15 21:43:21 2005 +0000\r
-\r
- r3453: Bind communication identifiers to a device to support device removal.\r
- Export per HCA CM devices to userspace.\r
- \r
- Signed-off-by: Sean Hefty <sean.hefty@intel.com>\r
-\r
-commit 5146689f1dbbce01cc46e23d749c28828e7e3ca8\r
-Author: James Lentini <jlentini@netapp.com>\r
-Date: Wed Sep 14 17:44:44 2005 +0000\r
-\r
- r3432: Request address resolution using ATS.\r
- Signed-off-by: James Lentini <jlentini@netapp.com>\r
-\r
-commit 780b8237f2bf6884cf1abcc11190f97ed5c0a343\r
-Author: James Lentini <jlentini@netapp.com>\r
-Date: Mon Sep 12 19:14:43 2005 +0000\r
-\r
- r3378: Added DAPL documentation.\r
- Signed-off-by: James Lentini <jlentini@netapp.com>\r
-\r
-commit 8911b60eb16770d28907c14e45556444317dd276\r
-Author: James Lentini <jlentini@netapp.com>\r
-Date: Mon Sep 12 19:10:13 2005 +0000\r
-\r
- r3377: Removed executable premission.\r
- Signed-off-by: James Lentini <jlentini@netapp.com>\r
-\r
-commit dec4d2eafebdfe7e6b495a36dd16bd5a98417e04\r
-Author: Sean Hefty <sean.hefty@intel.com>\r
-Date: Fri Sep 9 21:51:58 2005 +0000\r
-\r
- r3349: Update DAPL to match the verbs and CM event processing APIs.\r
- Signed-off-by: Sean Hefty <sean.hefty@intel.com>\r
- Signed-off-by: James Lentini <jlentini@netapp.com>\r
-\r
-commit d06dcfd25e5d37310d089bcb7f3d3d75fcece75a\r
-Author: Arlin Davis <ardavis@ichips.intel.com>\r
-Date: Tue Sep 6 19:34:46 2005 +0000\r
-\r
- r3326: Changes to support async events. Also consolidated the uAT,uCM,uCQ threads into one processing thread.\r
- Signed-off-by: Arlin Davis <ardavis@ichips.intel.com>\r
- Signed-off-by: James Lentini <jlentini@netapp.com>\r
-\r
-commit 1dd8c28ce515675ee8df37cc1596bca17587eaf6\r
-Author: James Lentini <jlentini@netapp.com>\r
-Date: Mon Aug 29 15:07:44 2005 +0000\r
-\r
- r3232: validate default settings so they don't exceed device maximums\r
- Signed-off by: Arlin Davis <ardavis@ichips.intel.com>\r
- Signed-off by: James Lentini <jlentini@netapp.com>\r
-\r
-commit ac5308b4aac7de79a356439dfec2312faf7705ae\r
-Author: James Lentini <jlentini@netapp.com>\r
-Date: Mon Aug 29 14:15:23 2005 +0000\r
-\r
- r3227: Support for ibv_query_port, device, and gid.\r
- Signed-off by: Arlin Davis <ardavis@ichips.intel.com>\r
- Signed-off by: James Lentini <jlentini@netapp.com>\r
-\r
-commit ef8c7a3a4896487ff0d37a7d7234746c15fe5cf7\r
-Author: Sean Hefty <sean.hefty@intel.com>\r
-Date: Sun Aug 21 22:30:08 2005 +0000\r
-\r
- r3143: - Add user specified context to all uCM events. Users will not retrieve\r
- any events associated with the context after destroying the corresponding\r
- cm_id.\r
- - Provide the ib_cm_init_qp_attr() call to userspace clients of the CM.\r
- This call may be used to set QP attributes properly before modifying the QP.\r
- - Fixes some error handling syncrhonization and cleanup issues.\r
- - Performs some minor code cleanup.\r
- - Replaces the ucm_simple test program with a userspace version of cmpost.\r
- - Updates DAPL to the new API.\r
- \r
- Signed-off-by: Sean Hefty <sean.hefty@intel.com>\r
-\r
-commit ec0f86d7c55830c6dffded585c04754cc6ac2a83\r
-Author: James Lentini <jlentini@netapp.com>\r
-Date: Tue Aug 16 14:10:13 2005 +0000\r
-\r
- r3107: Removed unused debug counter macros\r
- Signed-off-by: James Lentini <jlentini@netapp.com>\r
-\r
-commit 49087ba27bd93769bb64d7ac5c454de1b94005bc\r
-Author: James Lentini <jlentini@netapp.com>\r
-Date: Tue Aug 16 14:07:42 2005 +0000\r
-\r
- r3106: Implemented debug counters\r
- Signed-off by: Arlin Davis <ardavis@ichips.intel.com>\r
- Signed-off by: James Lentini <jlentini@netapp.com>\r
-\r
-commit f98e3af7dfc56b288cc77a9103b90f8d6a927fc5\r
-Author: James Lentini <jlentini@netapp.com>\r
-Date: Thu Aug 11 20:23:56 2005 +0000\r
-\r
- r3072: Update from SourceForge DAPL: use the LMR context in calls to\r
- dapls_hash_remove()\r
- Signed-off-by: James Lentini <jlentini@netapp.com>\r
-\r
-commit 878e524c5cc63a62802d28fdc215a2b69ceb1141\r
-Author: James Lentini <jlentini@netapp.com>\r
-Date: Thu Aug 11 20:07:06 2005 +0000\r
-\r
- r3071: Updates from SourceForge DAPL: EVD updates\r
- Signed-off-by: James Lentini <jlentini@netapp.com>\r
-\r
-commit 8dc70f7d972615f40e624d8f1272e5e7c16ba34f\r
-Author: James Lentini <jlentini@netapp.com>\r
-Date: Thu Aug 11 19:57:40 2005 +0000\r
-\r
- r3070: Update from SourceForge DAPL: set async evd to null in dapli_ia_release_hca\r
- Signed-off-by: James Lentini <jlentini@netapp.com>\r
-\r
-commit f2801ae6caf010d660fe302970dabddc8948e1bf\r
-Author: James Lentini <jlentini@netapp.com>\r
-Date: Thu Aug 11 19:46:39 2005 +0000\r
-\r
- r3069: Updates from SourceForge DAPL: size EP IOV correctly\r
- Signed-off-by: James Lentini <jlentini@netapp.com>\r
-\r
-commit c87fd235eaf0b3a30e005422f7d347c406c14f2c\r
-Author: James Lentini <jlentini@netapp.com>\r
-Date: Thu Aug 11 19:35:05 2005 +0000\r
-\r
- r3068: Update from SourceForge DAPL: removed duplicate ia pointer in SP structure and\r
- fixed the spelling of ib_hca_transport_t.\r
- Signed-off-by: James Lentini <jlentini@netapp.com>\r
-\r
-commit a88bebc09a9655e462b3d32dfddec823024eab59\r
-Author: James Lentin <jlentini@netapp.com>\r
-Date: Thu Aug 11 19:24:56 2005 +0000\r
-\r
- r3067: Update from SourceForge DAPL: use include guard\r
- Signed-off-by: James Lentin <jlentini@netapp.com>\r
-\r
-commit d2da08920de882a9a266f0606b81150c625fa003\r
-Author: James Lentini <jlentini@netapp.com>\r
-Date: Thu Aug 11 19:21:56 2005 +0000\r
-\r
- r3066: Update from SourceForge DAPL: optimization to dapl_ep_query\r
- Signed-off-by: James Lentini <jlentini@netapp.com>\r
-\r
-commit 4ca3b0cbc59227a90b5450eea1ffeeb91826dd6d\r
-Author: James Lentini <jlentini@netapp.com>\r
-Date: Thu Aug 11 18:39:34 2005 +0000\r
-\r
- r3065: Update from DAPL SourceForge: indicate which handle is invalid\r
- Signed-off-by: James Lentini <jlentini@netapp.com>\r
-\r
-commit 6d8f34137776c32149251bdec493c017b399cd10\r
-Author: James Lentini <jlentini@netapp.com>\r
-Date: Thu Aug 11 18:35:58 2005 +0000\r
-\r
- r3064: Update from DAPL SourceForge: set ep param values.\r
- Signed-off-by: James Lentini <jlentini@netapp.com>\r
-\r
-commit 0f35002a1942303ff46cb9a2b70056f9a38aebdb\r
-Author: James Lentini <jlentini@netapp.com>\r
-Date: Thu Aug 11 18:33:47 2005 +0000\r
-\r
- r3063: Updates from DAPL SourceForge: QP state and connection event fix.\r
- Signed-off-by: James Lentini <jlentini@netapp.com>\r
-\r
-commit 3fc876339693c6f0eed5e57780e5342f301bd95c\r
-Author: James Lentini <jlentini@netapp.com>\r
-Date: Thu Aug 11 18:23:33 2005 +0000\r
-\r
- r3062: Update from DAPL SourceForge: remove unused DAPL_CNO_PROXY_CALL\r
- Signed-off-by: James Lentini <jlentini@netapp.com>\r
-\r
-commit 70f8e7a2e6bde4e757ddc8c7f59d3a5c6a13adf9\r
-Author: Hal Rosenstock <halr@voltaire.com>\r
-Date: Fri Aug 5 17:59:38 2005 +0000\r
-\r
- r2989: Fix dtest makefile\r
- Signed-off-by: Hal Rosenstock <halr@voltaire.com>\r
- Signed-off-by: James Lentini <jlentini@netapp.com>\r
-\r
-commit 864695cfef37d84359ada8838ab4cd4f4dafc6bb\r
-Author: James Lentini <jlentini@netapp.com>\r
-Date: Fri Aug 5 17:57:31 2005 +0000\r
-\r
- r2988: Remove kernel directory.\r
- Signed-off-by: James Lentini <jlentini@netapp.com>\r
-\r
-commit 9c4e246a5baf43cadc6380e91fd5a6e319777278\r
-Author: James Lentini <jlentini@netapp.com>\r
-Date: Fri Aug 5 17:56:56 2005 +0000\r
-\r
- r2987: Remove kernel code directory.\r
- Signed-off-by: James Lentini <jlentini@netapp.com>\r
-\r
-commit 26706cb0de471ba47279de0cb949ba5a41de82cc\r
-Author: James Lentini <jlentini@netapp.com>\r
-Date: Fri Aug 5 16:41:12 2005 +0000\r
-\r
- r2986: Add uDAPL to the trunk\r
- Signed-off-by: James Lentini <jlentini@netapp.com>\r
-\r
-commit 76aa2de7fe38a8595d88669842450084cfa88316\r
-Author: Roland Dreier <roland@topspin.com>\r
-Date: Thu Nov 4 17:54:50 2004 +0000\r
-\r
- r1139: Copy roland-merge branch to trunk\r
-\r
-commit 3bd72a559dfe22685aae33599c99d021d2ae4aca\r
-Author: Roland Dreier <roland@topspin.com>\r
-Date: Tue Jul 20 21:34:32 2004 +0000\r
-\r
- r502: Move 2.6-only source away from trunk\r
-\r
-commit 4f05b6ed3fd1d14161664c677264846eeb51dba5\r
-Author: Roland Dreier <roland@topspin.com>\r
-Date: Tue Jul 20 21:34:32 2004 +0000\r
-\r
- r502: Move 2.6-only source away from trunk\r
-\r
-commit 6da8b951c069072a2afc6aba03a3dca2c44db022\r
-Author: Roland Dreier <roland@topspin.com>\r
-Date: Tue Jul 20 01:41:16 2004 +0000\r
-\r
- r497: Move 2.6-only tree to gen2\r
+commit 0983c66cbd9511128c1fa221470c4c983903e420
+Author: Arlin Davis <arlin.r.davis@intel.com>
+Date: Tue Nov 24 08:58:44 2009 -0800
+
+ winof cma: windows definition for EADDRNOTAVAIL missing
+
+ Signed-off-by: stan smith <stan.smith@intel.com>
+
+commit 4fbbcfa1d7bf7a843b27e351d35f3ffbc2ac4db8
+Author: Arlin Davis <arlin.r.davis@intel.com>
+Date: Tue Nov 24 08:54:26 2009 -0800
+
+ scm: client side setsockopt NODELAY fails if data arrives before setting
+
+ Move setsockopt before connect to avoid race with data.
+ Seems to fail on windows. Not seen on linux.
+
+ Signed-off-by: Arlin Davis <arlin.r.davis@intel.com>
+
+commit 9dfa9a06d55101fed0773028ce7ab85330514c67
+Author: Arlin Davis <arlin.r.davis@intel.com>
+Date: Wed Nov 18 09:52:40 2009 -0800
+
+ cma: setup_listener Cannot assign requested address
+
+ Colliding with RDS port of 18634. rdma_cm can return
+ either EADDRINUSE or EADDRNOTAVAIL if the bind fails.
+ Add check for either and return proper DAT_CONN_QUAL_IN_USE.
+
+ Signed-off-by: Arlin Davis <arlin.r.davis@intel.com>
+
+commit 0698fb56d533a5225cbc3a5a4b8ab2e2d56b7502
+Author: Arlin Davis <arlin.r.davis@intel.com>
+Date: Wed Nov 18 09:43:38 2009 -0800
+
+ common: seg fault in dapl_evd_wait with multi-thread application using CNO's.
+
+ If we are dealing with event streams besides a CQ event stream,
+ be conservative and set producer side locking. Otherwise, no.
+ Check for CNO is missing, CNO is not considered CQ event stream.
+
+ Signed-off-by: Arlin Davis <arlin.r.davis@intel.com>
+
+commit 4d26a280572a58248b0796e4d5ed01ea5d67be46
+Author: Arlin Davis <arlin.r.davis@intel.com>
+Date: Wed Nov 18 09:37:48 2009 -0800
+
+ ucm: inbound DREQ/DREP handshake should transition QP.
+
+ During release, when receiving a disconnect request from remote peer
+ instead of a disconnect call from the client, the QP didn't get properly
+ set in ERR state and didn't flush the queue during disconnect processing.
+
+ Signed-off-by: Arlin Davis <arlin.r.davis@intel.com>
+
+commit 87965cd15bbfa38b00ed2d77a4ea5a76f76cf4a3
+Author: Arlin Davis <arlin.r.davis@intel.com>
+Date: Mon Nov 2 08:24:53 2009 -0800
+
+ winof: Remove duplicate include of comp_channel.cpp from cm.c as it is included in opensm_ucb/device.c.
+
+ Signed-off-by: stan smith <stan.smith@intel.com>
+
+commit 737fa288b72fad19a1fc3e762eb9f2c471ce6ddd
+Author: Arlin Davis <arlin.r.davis@intel.com>
+Date: Fri Oct 30 13:19:21 2009 -0800
+
+ Release 2.0.24
+
+ Signed-off-by: Arlin Davis <arlin.r.davis@intel.com>
+
+commit 9fe7506ff9ddf1ae6297cfc6a9dd4d6a57e1939e
+Author: Arlin Davis <arlin.r.davis@intel.com>
+Date: Fri Oct 30 12:57:22 2009 -0800
+
+ winof: Utilize WinOF version of inet_ntop() for Windows OSes which do not support inet_ntop().
+
+ Signed-off-by: stan smith <stan.smith@intel.com>
+
+commit d56c645a2bf234e9e0cf215b112c2aa9d5e01945
+Author: Arlin Davis <arlin.r.davis@intel.com>
+Date: Fri Oct 30 07:17:26 2009 -0800
+
+ ucm: windows build issue with new CQ completion channel
+
+ Signed-off-by: Arlin Davis <arlin.r.davis@intel.com>
+
+commit 2d2e7e1e185c08542ee31b0e77561a1eeb4bde6c
+Author: Arlin Davis <arlin.r.davis@intel.com>
+Date: Fri Oct 30 06:35:33 2009 -0800
+
+ winof: add ucm provider to windows build
+
+ Signed-off-by: Arlin Davis <arlin.r.davis@intel.com>
+
+commit 94b2206093607214e0a9709651460692e8196e1c
+Author: Arlin Davis <arlin.r.davis@intel.com>
+Date: Fri Oct 30 06:32:56 2009 -0800
+
+ winof: add missing build files for ibal, scm
+
+ Signed-off-by: Arlin Davis <arlin.r.davis@intel.com>
+
+commit 66b76d7a8035b9164b69781d7630a0c77ce1bb5a
+Author: Arlin Davis <arlin.r.davis@intel.com>
+Date: Wed Oct 28 09:52:50 2009 -0800
+
+ scm: connection peer resets under heavy load, incorrect event on error
+
+ Under heavy load, we get a peer reset from the remote stack. In this
+ case retry the socket connection for this QP setup.
+
+ Add debugging with PID's and socket ports to help isolate
+ these types of socket scaling issues.
+
+ Report correct UD event during error, check remote_ah creation.
+
+ Fix dapl_poll return codes for single event type only.
+
+ Signed-off-by: Arlin Davis <arlin.r.davis@intel.com>
+
+commit f8108a9bda0200355107fdd6c43cb5885f47d648
+Author: Arlin Davis <arlin.r.davis@intel.com>
+Date: Wed Oct 28 09:47:37 2009 -0800
+
+ ucm: increase default reply and rtu timeout values.
+
+ Signed-off-by: Arlin Davis <arlin.r.davis@intel.com>
+
+commit 9c13d0d01c78eeb5071e802fbb53811cdb377059
+Author: Arlin Davis <arlin.r.davis@intel.com>
+Date: Wed Oct 28 07:48:20 2009 -0800
+
+ ucm: change some debug message levels and add check for valid UD REPLY during retries.
+
+ Signed-off-by: Arlin Davis <arlin.r.davis@intel.com>
+
+commit 1c404bb3dcc0a45e21ef3aa973d59714413beae0
+Author: Arlin Davis <arlin.r.davis@intel.com>
+Date: Tue Oct 27 10:37:45 2009 -0800
+
+ ucm: increase timers during subsequent retries
+
+ check/process create_ah errors during connect phase
+ cleanup some debug messaging.
+
+ Signed-off-by: Arlin Davis <arlin.r.davis@intel.com>
+
+commit cd8c48586f53e846de4fbe10994b73ba457f6406
+Author: Arlin Davis <arlin.r.davis@intel.com>
+Date: Mon Oct 19 10:38:36 2009 -0700
+
+ ucm, scm: address handles need destroyed when freeing Endpoints with UD QP's.
+
+ Signed-off-by: Arlin Davis <arlin.r.davis@intel.com>
+
+commit ce19f5744c0dd9461c09d999b309e8f0e2242767
+Author: Arlin Davis <arlin.r.davis@intel.com>
+Date: Fri Oct 16 14:42:00 2009 -0700
+
+ openib_common: ignore pd free errors, clear pd_handle and return.
+
+ some older adapters have some issues
+ with pd free so just clear handle and return
+
+ Signed-off-by: Arlin Davis <arlin.r.davis@intel.com>
+
+commit 81f5ac17d9039e2edcd8324f7d5ed5f66fcff9f2
+Author: Arlin Davis <arlin.r.davis@intel.com>
+Date: Fri Oct 16 08:52:21 2009 -0700
+
+ ucm: using UD type QP's, ucm reports wrong reject event when user rejects AH resolution request.
+
+ During rejects, both usr and ucm internal, the qp_type does not get initialized
+ so the check for UD type QP messages fail on active side and the wrong
+ event gets generated. Initialize saddr.ib information before sending reject
+ back to active side.
+
+ Signed-off-by: Arlin Davis <arlin.r.davis@intel.com>
+
+commit f0214e5a7a81a68819d308cb921eb75f5246207d
+Author: Arlin Davis <arlin.r.davis@intel.com>
+Date: Fri Oct 16 07:57:25 2009 -0700
+
+ ucm, scm, cma: Fix CNO support on DTO type EVD's
+
+ EVD wait_object should be used for CNO processing
+ and not the direct CQ event channels. Add proper
+ checking for DTO type EVD's with CNO at wait
+ and wakeup.
+
+ UCM missing support for collective EVD's under a
+ CNO. Add support to create common channel for
+ collective EVD's during device open. Add support
+ in cm_thread to check this channel. Also,
+ during disconnect, move QP to error to properly
+ flush queue instead of moving to reset and init.
+
+ Signed-off-by: Arlin Davis <arlin.r.davis@intel.com>
+
+commit 960950a7d9f5437dd831bd56ca2ad0c06cb4e324
+Author: Arlin Davis <arlin.r.davis@intel.com>
+Date: Thu Oct 15 09:19:45 2009 -0700
+
+ ucm: fix lock init bug in ucm_cm_find
+
+ the lock should be setup as pointer to lock
+ not lock structure. Cleanup lock and list
+ in cm_find function and cm_print function.
+
+ Add debug aid by passing process id in
+ msg resv area. cleanup cr references
+ and change to cm for consistency.
+
+ Signed-off-by: Arlin Davis <arlin.r.davis@intel.com>
+
+commit f86fec772f2d82eaf60228d288b295e0b7b86c59
+Author: Arlin Davis <arlin.r.davis@intel.com>
+Date: Wed Oct 14 10:03:47 2009 -0700
+
+ ucm: fix build problem with latest windows ucm changes
+
+ define dapls_thread_signal as inline
+
+ Signed-off-by: Arlin Davis <arlin.r.davis@intel.com>
+
+commit 87b6c8ba92f3063a35d49bdb49d6cd0a5100a36c
+Author: Sean Hefty <sean.hefty@intel.com>
+Date: Wed Oct 14 09:34:22 2009 -0700
+
+ Signed-off-by: Sean Hefty <sean.hefty@intel.com>
+
+commit 9fdd8d74f2cba83e9cf513256933f5241495c1da
+Author: Sean Hefty <sean.hefty@intel.com>
+Date: Wed Oct 14 09:34:18 2009 -0700
+
+ The HCA should not be closed until all resources have been released.
+ This results in a hang on windows, since closing the device frees
+ the event processing thread.
+
+ Signed-off-by: Sean Hefty <sean.hefty@intel.com>
+
+commit f9833db469f2d686842bb1d52d1ea53b74fa72a8
+Author: Sean Hefty <sean.hefty@intel.com>
+Date: Wed Oct 14 09:34:13 2009 -0700
+
+ Fix build warning when compiling on 32-bit systems.
+
+ Signed-off-by: Sean Hefty <sean.hefty@intel.com>
+
+commit c80515bd4b1bd11a125dc17e3f7db44240ee1fff
+Author: Sean Hefty <sean.hefty@intel.com>
+Date: Wed Oct 14 09:34:07 2009 -0700
+
+ Trying to deregister the same memory region twice leads to an
+ application crash on windows.
+
+ Signed-off-by: Sean Hefty <sean.hefty@intel.com>
+
+commit 6aa2c0d901daa9cfca7e771c1df2ead074d230bd
+Author: Arlin Davis <arlin.r.davis@intel.com>
+Date: Wed Oct 14 07:59:23 2009 -0700
+
+ dat: reduce debug message level when parsing for location of dat.conf
+
+ Don't output failover to default /etc/dat.conf from
+ sysconfdir at ERROR level. Reduce to DAT_OS_DBG_TYPE_SR.
+
+ Signed-off-by: Arlin Davis <arlin.r.davis@intel.com>
+
+commit e4038e078747201b57203f16ba793b7fc22c12f2
+Author: Arlin Davis <arlin.r.davis@intel.com>
+Date: Thu Oct 8 16:23:22 2009 -0700
+
+ ucm: update ucm provider for windows environment
+
+ add dapls_thread_signal abstraction and a new
+ cm_thread function specific for windows.
+
+ Signed-off-by: Sean Hefty <sean.hefty@intel.com>
+ Signed-off-by: Arlin Davis <arlin.r.davis@intel.com>
+
+commit d80ce42390eb57b9c4f816b4df063f90bd5699bc
+Author: Arlin Davis <arlin.r.davis@intel.com>
+Date: Thu Oct 8 16:02:52 2009 -0700
+
+ ucm: add timer/retry CM logic to the ucm provider
+
+ add reply, rtu and retry count options via
+ environment variables. Times in msecs.
+ DAPL_UCM_RETRY 10
+ DAPL_UCM_REP_TIME 400
+ DAPL_UCM_RTU_TIME 200
+
+ Add RTU_PENDING and DISC_RECV states
+
+ Add check timer code to the cm_thread
+ and the option to the select abstaction
+ to take timeout values in msecs.
+ DREQ, REQ, and REPLY will all be timed
+ and retried.
+
+ Split out reply code and disconnect_final
+ code to better facilitate retry timers.
+ Add checking for duplicate messages.
+
+ Added new UD extension events for errors.
+ DAT_IB_UD_CONNECTION_REJECT_EVENT
+ DAT_IB_UD_CONNECTION_ERROR_EVENT
+
+ Signed-off-by: Arlin Davis <arlin.r.davis@intel.com>
+
+commit 1186bfc949f4bb7278c30c2c59b7fcb6d5142638
+Author: Arlin Davis <arlin.r.davis@intel.com>
+Date: Fri Oct 2 14:49:52 2009 -0700
+
+ Release 2.0.23
+
+ Signed-off-by: Arlin Davis <arlin.r.davis@intel.com>
+
+commit a5f1220cfd96983c9c89a595d80fab7ddcb1a954
+Author: Arlin Davis <arlin.r.davis@intel.com>
+Date: Fri Oct 2 14:48:15 2009 -0700
+
+ cma: cannot reuse the cm_id and qp for new connection, must reallocate a new one.
+
+ When merging common code base the dapls_ib_reinit_ep mistakely
+ modified QP to reset then init for all providers. Will
+ not work for rdma_cm (cma provider) since the cm_id cannot
+ be reused. Add build check for _OPENIB_CMA_ to pull in correct
+ free and reallocate method for reinit_ep.
+
+ Signed-off-by: Arlin Davis <arlin.r.davis@intel.com>
+
+commit 7b07435495de0938e59be064fe8642cfd739f1ac
+Author: Arlin Davis <arlin.r.davis@intel.com>
+Date: Fri Oct 2 13:50:12 2009 -0700
+
+ scm, cma: update DAPL cm protocol revision with latest address/port changes
+
+ CM protocol changed, roll revision to 6.
+ The socket cm could be competing with address space if
+ application is using sockets above to exchange information
+ like dapltest, and MPI consumers. Adjust port on listen
+ and connect to reduce the chance of port collision with
+ application above.
+
+ Signed-off-by: Arlin Davis <arlin.r.davis@intel.com>
+
+commit 9cc1e76b672f7ff1231b4d113bd4dd1a016d1410
+Author: Arlin Davis <arlin.r.davis@intel.com>
+Date: Fri Oct 2 12:47:37 2009 -0700
+
+ ucm: modify IB address format to align better with sockaddr_in6
+
+ Restructure the dcm_addr union to map the IB side
+ closer to sockaddr6 and initialize family to
+ AF_INET6 to insure callee allocates enough memory
+ for ucm dat_ia_address type. Put qpn in flowinfo
+ and gid in sin6_addr. Change the test suites
+ to print address information based on AF_INET
+ or AF_INET6 instead of using specific IB address
+ union from the provider.
+
+ Signed-off-by: Arlin Davis <arlin.r.davis@intel.com>
+
+commit f50dbe6e82bd471845adf27829b2e07234a0a86a
+Author: Sean Hefty <sean.hefty@intel.com>
+Date: Wed Sep 30 14:29:03 2009 -0700
+
+ Add definition for getpid similar to that used by the other dtest apps.
+
+ Signed-off-by: Sean Hefty <sean.hefty@intel.com>
+
+commit 51147412fad72c00115b595955760e80ff0be7d5
+Author: Sean Hefty <sean.hefty@intel.com>
+Date: Wed Sep 30 14:28:57 2009 -0700
+
+ WinOF provides a common implementation of gettimeofday that should
+ be used instead.
+
+ Signed-off-by: Sean Hefty <sean.hefty@intel.com>
+
+commit 732fc84cee036126282715ea157bdb619fc11ec7
+Author: Sean Hefty <sean.hefty@intel.com>
+Date: Wed Sep 30 14:27:50 2009 -0700
+
+ The completion manager was updated to provide an abstraction that
+ better mimicked how fd's were used. Update dapl to use this
+ abstraction, rather than the older completion manager api.
+
+ This helps minimize changes between linux and windows.
+
+ Signed-off-by: Sean Hefty <sean.hefty@intel.com>
+
+commit fc1855b7738838d3865389405f017b87d223f743
+Author: Arlin Davis <arlin.r.davis@intel.com>
+Date: Wed Sep 30 14:26:47 2009 -0700
+
+ dtestcm: remove IB verb definitions
+
+ Remove gid and qp_type references from test app.
+ Print address infomation in sockaddr and
+ ucm provider format with qpn and lid.
+
+ Signed-off-by: Arlin Davis <arlin.r.davis@intel.com>
+
+commit 36a885d4267be7b3c6c1086372d321121ae03dee
+Author: Arlin Davis <arlin.r.davis@intel.com>
+Date: Wed Sep 30 10:44:14 2009 -0700
+
+ dtest, dtestx: remove IB verb definitions
+
+ remove gid and qp_type checking from test suite.
+ Print address infomation in sockaddr and
+ ucm provider format with qpn and lid.
+
+ Signed-off-by: Arlin Davis <arlin.r.davis@intel.com>
+
+commit 23445bbbd3a09f7f5e666a000d5c7c3b22dd95b3
+Author: Arlin Davis <arlin.r.davis@intel.com>
+Date: Mon Sep 28 10:59:36 2009 -0700
+
+ scm: tighten up socket options to insure similiar behavior on Windows and Linux.
+
+ Add IPPROTO_TCP to create socket. Specify device IP address
+ when binding instead of INADDR_ANY and remove setsocketopt
+ REUSEADDR on the listen socket to avoid any issues with
+ portability. Don't want duplicate port bindings.
+
+ Signed-off-by: Arlin Davis <arlin.r.davis@intel.com>
+
+commit b31b64d60abe7c6c1a83484db97176d225076b30
+Author: Arlin Davis <arlin.r.davis@intel.com>
+Date: Mon Sep 28 10:46:26 2009 -0700
+
+ cma: improve serialization of destroy and event processing
+
+ WinOF testing with slightly different scheduler and verbs
+ showed some issues with cleanup. Add better protection around
+ destroy and event processing thread.
+
+ Remove destroy flag and add refs counting to conn objects
+ to block destroy until all references are cleared. Add
+ locking aroung ref counting and passive and active
+ event processing.
+
+ Signed-off-by: Arlin Davis <arlin.r.davis@intel.com>
+
+commit a4adf463695ba75beaa1fe79d514346bf5fe2cb5
+Author: Arlin Davis <arlin.r.davis@intel.com>
+Date: Mon Sep 28 10:42:52 2009 -0700
+
+ scm: improve serialization of destroy and state changes
+
+ WinOF testing with slightly different scheduler and verbs
+ showed some issues with cleanup. Add better protection around
+ destroy and move state change before socket send to insure
+ correct state in multi-thread environment targeting the same
+ device on send and recv.
+
+ Change DCM_RTU_PENDING to DCM_REP_PENDING and
+ and add static definition to local routines for better
+ readability.
+
+ Signed-off-by: Arlin Davis <arlin.r.davis@intel.com>
+
+commit 29358ccc587db55fe5f5a1b14eed9e0e31f5f02c
+Author: Arlin Davis <arlin.r.davis@intel.com>
+Date: Thu Sep 17 08:56:06 2009 -0700
+
+ common: no cleanup/release code for timer thread
+
+ dapl_set_timer() creates a thread to process timers for dat_ep_connect
+ but provides no mechanism to destroy/exit during dapl library unload.
+ Timers are initialized in library init code and should be released
+ in the fini code. Add a dapl_timer_release call to the dapl_fini
+ function to check state of timer thread and destroy before exiting.
+
+ Signed-off-by: Arlin Davis <arlin.r.davis@intel.com>
+
+commit e287ce12f5b0108ef83186aae305f25af8cdd02c
+Author: Arlin Davis <arlin.r.davis@intel.com>
+Date: Thu Sep 17 08:53:29 2009 -0700
+
+ scm, cma: dapli_thread doesn't always get teminated on library close.
+
+ DAPL doesn't actually wait for the async processing thread to exit before
+ allowing the library to close. It will wait up to 10 seconds, which under
+ heavy load isn't enough time. Since the thread is created by an application
+ level thread, it will continue to run as long as the application runs. But
+ if the application closes the library, then all library data and code is
+ invalid, which can result in the thread running something that's not
+ library code and accessing freed memory.
+
+ With this change, I was able to run mpi ping-pong, 16 ranks on a single
+ system (scm provider) without crashes 1300 times.
+
+ Signed-off-by: Sean Hefty <sean.hefty@intel.com>
+
+commit 5bffe521c6ad7a5ce10457a5d7b25814522a0cde
+Author: Arlin Davis <arlin.r.davis@intel.com>
+Date: Wed Sep 9 13:10:35 2009 -0700
+
+ ucm: tighten up locking with CM processing, state changes
+
+ tighten up locking on CM processing and state changes
+ and reduce the send completion threshold to 50 from 100
+ to replenish the request message faster.
+
+ Signed-off-by: Arlin Davis <arlin.r.davis@intel.com>
+
+commit e38daa8448bc3b73c53e54ffc9ac47b9cfc2af89
+Author: Arlin Davis <arlin.r.davis@intel.com>
+Date: Wed Sep 9 09:44:03 2009 -0700
+
+ ucm: For UD type QP's, return CR p_data with CONN_EST event on passive side.
+
+ Intel MPI uses the p_data provided with CONN_EST as a reference to the
+ UD pair and remote rank. The ucm provider was overwriting the CR p_data
+ with the ACCEPT p_data. Change to save CR p_data but also provide
+ storage for user provided ACCEPT p_data in case the REPLY is lost
+ and needs retransmitted.
+
+ p_data size was provided to event processing in network order
+ instead of host order.
+
+ For new QP's create new address handles and do not use
+ existing AH's created for the CM. Different PD's are
+ associated with each.
+
+ Signed-off-by: Arlin Davis <arlin.r.davis@intel.com>
+
+commit 0bc6a0954bc1a1a61e1b1cbc5c280f69ed168453
+Author: Arlin Davis <arlin.r.davis@intel.com>
+Date: Tue Sep 8 09:14:46 2009 -0700
+
+ ucm: cleanup extra cr/lf
+
+ Signed-off-by: Arlin Davis <arlin.r.davis@intel.com>
+
+commit 6e7df65a884b4e068135e64dcb3ec660f4c7ab14
+Author: Arlin Davis <arlin.r.davis@intel.com>
+Date: Tue Sep 8 09:11:37 2009 -0700
+
+ ucm: fix issues with UD QP's.
+
+ private data size not in host order when processing
+ connection events.
+
+ ud extentions event should include original ia_addr
+ and qpn used during connection and not the IB qpn.
+
+ ucm QP service resource cleanup in wrong order.
+
+ cleanup extra cr/lf device.c
+
+ Signed-off-by: Arlin Davis <arlin.r.davis@intel.com>
+
+commit e7c3090945c13f80abb929643fc975465afeeb53
+Author: Arlin Davis <arlin.r.davis@intel.com>
+Date: Thu Sep 3 10:45:56 2009 -0700
+
+ winof: Convert windows version of dapl and dat libaries to use private heaps.
+
+ This allows for better support of memory registration caching by upper
+ level libaries (MPI) that use SecureMemoryCacheCallback.
+
+ It also makes it easier to debug heap corruption issues.
+
+ Signed-off-by: Sean Hefty <sean.hefty@intel.com>
+
+commit bc3c8a34558880c8d09a03e3eb7c3f50d7762c67
+Author: Arlin Davis <arlin.r.davis@intel.com>
+Date: Wed Sep 2 14:01:51 2009 -0700
+
+ dtest, dtestx: modifications for UD QP testing with ucm provider.
+
+ remote_addr is wrong for IP remote address.
+
+ The dtestx requires the server connect back to the client
+ for the UD test. With the ucm provider you need to provide
+ the QPN and the LID which you cannot get until the dtest
+ client starts. So, for now, don't support UD testing
+ on UCM providers.
+
+ Signed-off-by: Arlin Davis <arlin.r.davis@intel.com>
+
+commit e4555143ca71ebe18d8c9f027e2e7a5282088a30
+Author: Arlin Davis <arlin.r.davis@intel.com>
+Date: Wed Sep 2 13:54:59 2009 -0700
+
+ scm, ucm: UD QP support was broken when porting to common openib code base.
+
+ create remote_ah was moved out of modify_qp_state function but not
+ included in the RTU and ACCEPT code for UD QP's. qp type check
+ should be on daddr not saddr in ucm cm code.
+
+ QP number must be converted to host order before supplying remote_ah,
+ and qp number to consumer.
+
+ Modify QP state to RTR for UD QP mask setting incorrect.
+
+ Signed-off-by: Arlin Davis <arlin.r.davis@intel.com>
+
+commit 375d368494bb2f9bb44c82073abc60e01ac28615
+Author: Arlin Davis <arlin.r.davis@intel.com>
+Date: Tue Sep 1 13:02:24 2009 -0700
+
+ cma: cleanup warning with unused local variable, ret, in disconnect
+
+ Signed-off-by: Arlin Davis <arlin.r.davis@intel.com>
+
+commit 72d1c0c984919809ecd94b2e4ede31ec56668518
+Author: Arlin Davis <arlin.r.davis@intel.com>
+Date: Tue Sep 1 12:36:31 2009 -0700
+
+ cma: remove debug message after rdma_disconnect failure
+
+ DAPL automatically calls rdma_disconnect() when a disconnect request is
+ received. If the user also calls disconnect, that calls rdma_disconnect() as
+ well, but the connection has already been disconnected by DAPL and is no longer
+ valid. The result is that the user's call to rdma_disconnect() will fail. Do
+ not display an error message if this occurs.
+
+ Locking could be added to prevent calling rdma_disconnect() multiple times, but
+ since the librdmacm provides synchronization to trap this, we might as well take
+ advantage of it.
+
+ Signed-off-by: Sean Hefty <sean.hefty@intel.com>
+
+commit 290c568030da6df398a4afb932b5d4f9bd585c26
+Author: Arlin Davis <arlin.r.davis@intel.com>
+Date: Tue Sep 1 12:27:43 2009 -0700
+
+ scm: socket errno check needs O/S dependent wrapper
+
+ Intel MPI checks the uDAPL error code when calling dat_psp_create() to see if
+ the port number that it provides is in use or not. Convert winsock error codes
+ to unix errno values.
+
+ This fixes the following error reported by Intel MPI:
+ 'DAPL provider is not found and fallback device is not enabled'
+
+ Signed-off-by: Sean Hefty <sean.hefty@intel.com>
+
+commit 03b6d1b58ae03058a509c54ffe27147cc44d1851
+Author: Arlin Davis <arlin.r.davis@intel.com>
+Date: Tue Sep 1 12:13:16 2009 -0700
+
+ dapltest: update script files for WinOF
+
+ Cleanup 64-bit paths now that WinOF is always installed into '\Program Files\WinOF'.
+
+ Signed-off-by: Arlin Davis <arlin.r.davis@intel.com>
+
+commit e38585be89e9fa8f139002a3fe1a82eb86438f93
+Author: Arlin Davis <arlin.r.davis@intel.com>
+Date: Tue Sep 1 12:10:21 2009 -0700
+
+ cma: conditional check for new rdma_cm definition.
+
+ RDMA_CM_EVENT_TIMEWAIT_EXIT is new to OFED 1.4
+ add conditional check so dapl can build and run
+ against older OFED 1.3 stacks
+
+ Signed-off-by: Arlin Davis <arlin.r.davis@intel.com>
+
+commit 83af586b77aa905b63de6dd6010cd60e91487a9e
+Author: Arlin Davis <arlin.r.davis@intel.com>
+Date: Thu Aug 20 09:13:43 2009 -0700
+
+ Release 2.0.22
+
+ Signed-off-by: Arlin Davis <arlin.r.davis@intel.com>
+
+commit 48bc0dc95093ccbfe5e01cef4877c128e2caaf77
+Author: Arlin Davis <arlin.r.davis@intel.com>
+Date: Thu Aug 20 09:12:47 2009 -0700
+
+ dapltest: add mdep processor yield and use with dapltest
+
+ Be thread scheduler friendly and release the current thread thus allowing other threads to run.
+
+ Signed off by Stan Smith stan.smith@intel.com
+
+commit e13f255a06b9d2a2ed4eec9a62f9fe105d0c26d7
+Author: Arlin Davis <arlin.r.davis@intel.com>
+Date: Tue Aug 18 10:15:15 2009 -0700
+
+ ucm: Add new provider using a DAPL based IB-UD cm mechanism for MPI implementations.
+
+ New provider uses it's own CM protocol on top of IB-UD queue pairs.
+ During device open, this provider creates a UD queue pair and
+ returns local address information via dat_ia_query. This 24 byte
+ opaque address must be exchange out-of-band before connecting to a
+ server via dat_ep_connect. This provider is targeted for MPI
+ implementations that already exchange address information
+ during mpi_init phase.
+
+ Future release may provide some ARP mechanism via multicast.
+
+ dtest, dtestx, and dtestcm was modified to report the lid and qpn
+ information on the server side so you can provide appropriate
+ destination address information for the client test suite.
+
+ dapltest will not work with this provider.
+
+ Signed-off-by: Arlin Davis <arlin.r.davis@intel.com>
+
+commit 38d224dd95896c7e60f0bc0ffa52b26ab78f489b
+Author: Arlin Davis <arlin.r.davis@intel.com>
+Date: Tue Aug 4 20:54:12 2009 -0700
+
+ Release 2.0.21
+
+ Signed-off-by: Arlin Davis <arlin.r.davis@intel.com>
+
+commit ed4999a26043c9c3c73c792b21d24ced1df1553c
+Author: Arlin Davis <arlin.r.davis@intel.com>
+Date: Tue Aug 4 20:49:09 2009 -0700
+
+ scm: Fix disconnect. QP's need to move to ERROR state in
+ order to flush work requests and notify consumer. Moving to
+ RESET removed all requests but did not notify consumer.
+
+ Signed-off-by: Arlin Davis <arlin.r.davis@intel.com>
+
+commit 512f1d7a480f06a1fa491d21870e560ad111c4d0
+Author: Arlin Davis <arlin.r.davis@intel.com>
+Date: Tue Aug 4 20:48:03 2009 -0700
+
+ modify dtest.c to cleanup CNO wait code and consolidate into
+ collect_event() call. After waking up from CNO wait the
+ consumer must check all EVD's. The EVD's under the CNO
+ could be dropped if already triggered or could come in any order.
+ DT_RetToString changed to DT_RetToStr and DT_EventToSTr
+ changed to DT_EventToStr for consistency.
+
+ Signed-off-by: Arlin Davis <arlin.r.davis@intel.com>
+
+commit 024e36975d37a1556bf68145e1573f637d269bfc
+Author: Arlin Davis <arlin.r.davis@intel.com>
+Date: Tue Aug 4 20:47:17 2009 -0700
+
+ CNO events, once triggered will not be returned during the cno wait.
+ Check for triggered state before going to sleep in cno_wait. Reset
+ triggered EVD reference after reporting.
+
+ Signed-off-by: Arlin Davis <arlin.r.davis@intel.com>
+
+commit 6d6c72a49158d10825929111d6b4df1c6d2bb589
+Author: Arlin Davis <arlin.r.davis@intel.com>
+Date: Sun Aug 2 14:21:09 2009 -0700
+
+ CNO support broken in both CMA and SCM providers.
+
+ CQ thread/callback mechanism was removed by mistake. Still
+ need indirect DTO callbacks when CNO is attached to EVD's.
+
+ Add CQ event channel to cma provider's thread and add
+ to select for rdma_cm and async channels.
+
+ For scm provider there is not easy way to add this channel
+ to the select across sockets on windows. So, for portablity
+ reasons 2 thread is started to process the ASYNC and
+ CQ channels for events.
+
+ Must disable EVD (evd_endabled=FALSE) during destroy
+ to prevent EVD events firing for CNOs and re-arming CQ while
+ CQ is being destroyed.
+
+ Change dtest to check EVD after CNO timesout.
+
+ Signed-off-by: Arlin Davis <arlin.r.davis@intel.com>
+
+commit 6fe8bd1d8f44777211e816b72e0b2a6d22900207
+Author: Arlin Davis <arlin.r.davis@intel.com>
+Date: Thu Jul 30 08:02:30 2009 -0700
+
+ common osd: include winsock2.h for IPv6 definitions.
+
+ Signed-off-by: Arlin Davis <arlin.r.davis@intel.com>
+
+commit bd26383900d18962aeeff54fa59922009091ecfc
+Author: Arlin Davis <arlin.r.davis@intel.com>
+Date: Wed Jul 29 08:02:15 2009 -0700
+
+ common osd: include w2tcpip.h for sockaddr_in6 definitions.
+
+ Signed-off-by: Arlin Davis <arlin.r.davis@intel.com>
+
+commit f25544f14554200a6714accef5f761b0269b5819
+Author: Sean Hefty <sean.hefty@intel.com>
+Date: Mon Jul 27 15:07:33 2009 -0700
+
+ DAPL introduced the concept of directly waiting on the CQ for
+ events by adding a compile time flag and special handling in the common
+ code. Rather than using the compile time flag and modifying the
+ common code, let the provider implement the best way to wait for
+ CQ events.
+
+ This simplifies the code and allows the common openib providers to
+ optimize for Linux and Windows platforms independently, rather than
+ assuming a specific implementation for signaling events.
+
+ Signed-off-by: Sean Hefty <sean.hefty@intel.com>
+
+commit 1548405a377d2bd17938df69419e9bcf3364d91a
+Author: Arlin Davis <arlin.r.davis@intel.com>
+Date: Thu Jul 16 12:41:22 2009 -0700
+
+ dapltest: Implement a malloc() threshold for the completion reaping.
+
+ change byte vector allocation to stack in functions:
+ DT_handle_send_op, DT_handle_rdma_op & DT_handle_recv_op.
+
+ When allocation size is under the threshold, use a stack local
+ allocation instead of malloc/free. Move redundant bzero() to
+ be called only in the case of using local stack allocation as
+ DT_Mdep_malloc() already does a bzero(). Consolidate error handling
+ return and free()check to a single point by using goto.
+
+ Signed-off-by: Stan Smith <stan.smith@intel.com>
+
+commit f6311ca7295230bf9efbcddc639fa8e1065b1f3d
+Author: Arlin Davis <arlin.r.davis@intel.com>
+Date: Thu Jul 16 12:32:09 2009 -0700
+
+ scm: handle connected state when freeing CM objects
+
+ The QP could be freed before being disconnected
+ so the provider needs process disconnect before freeing
+ the CM object. The disconnect clean will finish
+ the destroy process during the disc callback.
+
+ Signed-off-by: Arlin Davis <arlin.r.davis@intel.com>
+
+commit 4387359106ce398b29847982883016f7fd48b372
+Author: Arlin Davis <arlin.r.davis@intel.com>
+Date: Wed Jul 8 12:49:43 2009 -0700
+
+ scm, dtest: changes for winof gettimeofday and FD_SETSIZE settings.
+
+ scm changes to set FD_SETSIZE with expected value and
+ prevent windows override.
+
+ dtest: remove gettimeofday implementation for windows
+ specific implemenation etc\user\gtod.c
+
+ general EOL cleanup
+
+ Signed-off-by: Arlin Davis <arlin.r.davis@intel.com>
+
+commit 3542a83d8a31f5ac68adf3aa44e3ebf1265068df
+Author: Arlin Davis <arlin.r.davis@intel.com>
+Date: Mon Jul 6 09:24:07 2009 -0700
+
+ scm: set TCP_NODELAY sockopt on the server side for sends.
+
+ scm provider sends small messages from both server and client
+ sides. Set NODELAY on both sides to avoid send delays either
+ way.
+
+ Signed-off-by: Arlin Davis <arlin.r.davis@intel.com>
+
+commit 9d591180392856935b9c3befbab2243dd8daf628
+Author: Arlin Davis <arlin.r.davis@intel.com>
+Date: Thu Jul 2 14:16:52 2009 -0700
+
+ windows: remove obsolete files in dapl/udapl source tree
+
+ SOURCES,makefile,udapl.r,udapl_exports.src,udapl_sources.c
+
+ Signed-off-by: Arlin Davis <arlin.r.davis@intel.com>
+
+commit 85c238ee0a41dd0a4a24b3d422f34674b0183161
+Author: Arlin Davis <arlin.r.davis@intel.com>
+Date: Thu Jul 2 14:11:20 2009 -0700
+
+ dtestcm: add UD type QP option to test
+
+ Add -u for UD type QP's during connection setup.
+ Will setup UD QPs and provide remote AH
+ in connect establishment event. Measures
+ setup/exchange rates.
+
+ Signed-off-by: Arlin Davis <arlin.r.davis@intel.com>
+
+commit 89a2211526e37b1db58fc0ea663b330bc19125c8
+Author: Arlin Davis <arlin.r.davis@intel.com>
+Date: Thu Jul 2 14:07:36 2009 -0700
+
+ scm: destroy QP called before disconnect
+
+ Handle the case where QP is destroyed before
+ disconnect processing. Windows supports
+ reinit_qp during a disconnect call by
+ destroying the QP and recreating the
+ QO instead of state change from reset
+ to init. Call disconnect in destroy
+ CM code to handle this unexpected state.
+
+ Signed-off-by: Arlin Davis <arlin.r.davis@intel.com>
+
+commit 6eb35b7d69a896c256b1031337d3353575cd07b4
+Author: Arlin Davis <arlin.r.davis@intel.com>
+Date: Thu Jul 2 14:03:12 2009 -0700
+
+ cma: add support for rdma_cm TIME_WAIT event.
+
+ Nothing to process, simply ack the event.
+
+ Signed-off-by: Arlin Davis <arlin.r.davis@intel.com>
+
+commit b6c56b3052ecd3e36c32092ee62ff0c724da5ad4
+Author: Arlin Davis <arlin.r.davis@intel.com>
+Date: Wed Jul 1 07:58:32 2009 -0700
+
+ scm: remove old udapl_scm code replaced by openib_scm.
+
+ Signed-off-by: Arlin Davis <arlin.r.davis@intel.com>
+
+commit 5bbae42a56e1cca678d590ac4c841dd61e839d74
+Author: Arlin Davis <arlin.r.davis@intel.com>
+Date: Wed Jul 1 07:53:18 2009 -0700
+
+ winof: fix build issues after consolidating cma, scm code base.
+
+ Signed-off-by: Arlin Davis <arlin.r.davis@intel.com>
+
+commit 6bd1d931c4d0d4cbafac383f225140120aee4c51
+Author: Arlin Davis <arlin.r.davis@intel.com>
+Date: Wed Jul 1 07:51:59 2009 -0700
+
+ cma: lock held when exiting as a result of a rdma_create_event_channel failure.
+
+ Signed-off-by: Arlin Davis <arlin.r.davis@intel.com>
+
+commit b8a14ff1cc257defa2f74373d143600f5f471823
+Author: Sean Hefty <sean.hefty@intel.com>
+Date: Mon Jun 29 12:34:54 2009 -0700
+
+ windows: all dlist functions have been moved to the header file.
+ remove references to dlist.c
+
+ Signed-off-by: Sean Hefty <sean.hefty@intel.com>
+
+commit 1a081a0a467e4773a641e8edc876a7a4d7a30ca8
+Author: Arlin Davis <arlin.r.davis@intel.com>
+Date: Mon Jun 29 12:13:48 2009 -0700
+
+ dtestcm windows: add build infrastructure for new dtestcm test suite
+
+ Signed-off-by: Arlin Davis <arlin.r.davis@intel.com>
+
+commit c37d7a25dca97011ea76e2a541f936d10ca658e0
+Author: Arlin Davis <arlin.r.davis@intel.com>
+Date: Mon Jun 29 08:57:46 2009 -0700
+
+ openib_common: reorganize provider code base to share common mem, cq, qp, dto functions
+
+ add new openib_common directory with cq, qp, util, dto, mem function calls
+ and definitions. This basically leaves the unique CM and Device definitions
+ and functions to the individual providers directory of openib_scm and openib_cma.
+
+ modifications to dapl_cr_accept required. ep->cm_handle is allocated
+ and managed entirely in provider so dapl common code should not update
+ ep_handle->cm_handle from the cr->cm_handle automatically. The provider
+ should determine which cm_handle is required for the accept.
+
+ openib_cma defines _OPENIB_CMA_ and openib_scm defines _OPENIB_SCM_ for provider
+ specific build needs in common code.
+
+commit 961a4083ffb646c070137abd33e9ba2ea9482685
+Author: Arlin Davis <arlin.r.davis@intel.com>
+Date: Fri Jun 26 14:45:34 2009 -0700
+
+ scm: fixes and optimizations for connection scaling
+
+ Prioritize accepts on listen ports via FD_READ
+ process the accepts ahead of other work to avoid
+ socket half_connection (SYN_RECV) stalls.
+
+ Fix dapl_poll to return DAPL_FD_ERROR on
+ all event error types.
+
+ Add new state for socket released, but CR
+ not yet destroyed. This enables scm to release
+ the socket resources immediately after exchanging
+ all QP information. Also, add state to str call.
+
+ Only add the CR reference to the EP if it is
+ RC type. UD has multiple CR's per EP so when
+ a UD EP disconnect_clean was called, from a
+ timeout, it destroyed the wrong CR.
+
+ Signed-off-by: Arlin Davis <arlin.r.davis@intel.com>
+
+commit a60a9e1fce5588cb23f41391b48acf04edd82499
+Author: Arlin Davis <arlin.r.davis@intel.com>
+Date: Fri Jun 26 14:31:19 2009 -0700
+
+ scm: double the default fd_set_size
+
+ Signed-off-by: Arlin Davis <arlin.r.davis@intel.com>
+
+commit 17d5e1692db4ae1eb09aa919d5607f22851d7ec5
+Author: Arlin Davis <arlin.r.davis@intel.com>
+Date: Fri Jun 26 14:28:30 2009 -0700
+
+ scm: EP reference in CR should be cleared during ep_destroy
+
+ The EP reference in the CR should be set to null
+ during the EP free call to insure no further
+ reference back to a mem freed EP.
+
+ Signed-off-by: Arlin Davis <arlin.r.davis@intel.com>
+
+commit ebb820364cec9d72285c005a0874e7d459a9ff7d
+Author: Arlin Davis <arlin.r.davis@intel.com>
+Date: Fri Jun 26 14:23:35 2009 -0700
+
+ dtestx: fix conn establishment event checking
+
+ not catching error cases on client side
+ when checking for event number and UD type
+ && should have been ||
+
+ Signed-off-by: Arlin Davis <arlin.r.davis@intel.com>
+
+commit 747b793898042e3011fbad4b2d1285d2c040cb13
+Author: Arlin Davis <arlin.r.davis@intel.com>
+Date: Fri Jun 26 14:18:37 2009 -0700
+
+ dtestcm: new test to measure dapl connection rates.
+
+ new test suite added to measure connection
+ rates of providers. Used to compare cma, scm,
+ and other providers under development.
+
+ dtestcm USAGE
+
+ s: server
+ c: connections (default = 1000)
+ b: burst rate of conn_reqs (default = 100)
+ m: multi-listens (set to burst setting )
+ v: verbose
+ w: wait on event (default, polling)
+ d: delay before accept
+ h: hostname/address of server, specified on client
+ P: provider name (default = OpenIB-v2-ib0)
+
+ Signed-off-by: Arlin Davis <arlin.r.davis@intel.com>
+
+commit d58fbc3a870a060ead882e1d15c6d245cdf39096
+Author: Arlin Davis <arlin.r.davis@intel.com>
+Date: Fri Jun 19 20:59:16 2009 -0700
+
+ Release 2.0.20
+
+ Signed-off-by: Arlin Davis <arlin.r.davis@intel.com>
+
+commit beebe0066b47d7bf476925ff280bad2a3db38324
+Author: Arlin Davis <arlin.r.davis@intel.com>
+Date: Fri Jun 19 20:52:51 2009 -0700
+
+ common,scm: add debug capabilities to print in-process CM lists
+
+ Add a new debug bit DAPL_DBG_TYPE_CM_LIST.
+ If set, the pending CM requests will be
+ dumped when dat_print_counters is called.
+ Only provided when built with -DDAPL_COUNTERS
+
+ Add new dapl_cm_state_str() call for state
+ to string conversion for debug prints.
+
+ Signed-off-by: Arlin Davis <arlin.r.davis@intel.com>
+
+commit b1c51f1e68993d9306e3ebd48bd3a1f0e9878fa3
+Author: Arlin Davis <arlin.r.davis@intel.com>
+Date: Tue Jun 16 09:22:31 2009 -0700
+
+ scm: disconnect EP before cleaning up orphaned CR's during dat_ep_free
+
+ There is the possibility of dat_ep_free being called
+ with RC CR's still in connected state. Call disconnect
+ on the CR before marking for destroy.
+
+ Signed-off-by: Arlin Davis <arlin.r.davis@intel.com>
+
+commit 531d223455a88c885d6c5f7b1d7e158c1079fbce
+Author: Arlin Davis <arlin.r.davis@intel.com>
+Date: Wed Jun 10 12:05:17 2009 -0700
+
+ dapltest: windows scripts updated
+
+ Support added for provider specification and general simplification of internal workings.
+
+ Signed-off-by: Stan Smith <stan.smith@intel.com>
+
+commit 049d1ea08643d4c4eff761741641d37bb3f01fc1
+Author: Arlin Davis <arlin.r.davis@intel.com>
+Date: Wed Jun 10 09:18:09 2009 -0700
+
+ scm: private data is not handled properly via CR rejects.
+
+ For both RC and UD connect requests, the private
+ data is not being received on socket and passed
+ back via the active side REJECT event.
+
+ UD requires new extended reject event type of
+ DAT_IB_UD_CONNECTION_REJECT_EVENT to distiquish
+ between RC and UD type rejects.
+
+ cr_thread exit/cleanup processing fixed to insure
+ all items are off the list before exiting.
+
+ Signed-off-by: Arlin Davis <arlin.r.davis@intel.com>
+
+commit 3c26870e276a934e2009090e0fca8bdc36c1be67
+Author: Arlin Davis <arlin.r.davis@intel.com>
+Date: Wed Jun 10 09:09:56 2009 -0700
+
+ scm: cleanup orphaned UD CR's when destroying the EP
+
+ UD CR objects are kept active because of direct private data references
+ from CONN events. The cr->socket is closed and marked inactive but the
+ object remains allocated and queued on the CR resource list. There can
+ be multiple CR's associated with a given EP and there is no way to
+ determine when consumer is finished with event until the dat_ep_free.
+ Schedule destruction for all CR's associated with this EP during
+ free call. cr_thread will complete cleanup with state of SCM_DESTROY.
+
+ Signed-off-by: Arlin Davis <arlin.r.davis@intel.com>
+
+commit 73abd3f58fa7b14241fad98912ef27c7b4fdb47e
+Author: Arlin Davis <arlin.r.davis@intel.com>
+Date: Wed Jun 10 09:05:32 2009 -0700
+
+ scm: provider specific query for default UD MTU is wrong.
+
+ Change the provider specific query DAT_IB_TRANSPORT_MTU
+ to report 2048 for new default MTU size.
+
+ Signed-off-by: Arlin Davis <arlin.r.davis@intel.com>
+
+commit 27c0d7edc4c931b808a7c5a24bd5aa2625b48aa1
+Author: Arlin Davis <arlin.r.davis@intel.com>
+Date: Wed Jun 10 10:06:59 2009 -0700
+
+ scm: update CM code to shutdown before closing socket
+
+ data could be lost without calling shutdown on the socket
+ before closing. Update to shutdown and then close. Add
+ definition for SHUT_RW to SD_BOTH for windows.
+
+ Signed-off-by: Arlin Davis <arlin.r.davis@intel.com>
+ ---
+
+commit 536ec3103c15c1fed4367326c9117660345e0eab
+Author: Arlin Davis <arlin.r.davis@intel.com>
+Date: Thu Jun 4 13:48:18 2009 -0700
+
+ dapltest: windows script dt-cli.bat updated
+
+ scn should be scm
+
+ Signed-off-by: Arlin Davis <arlin.r.davis@intel.com>
+
+commit e8991b8f0877b0e2e857717e1140c679e9266abe
+Author: Sean Hefty <sean.hefty@intel.com>
+Date: Thu Jun 4 08:19:12 2009 -0700
+
+ dapl/windows cma provider: add support for network devices based on index
+
+ The linux cma provider provides support for named network devices, such
+ as 'ib0' or 'eth0'. This allows the same dapl configuration file to
+ be used easily across a cluster.
+
+ To allow similar support on Windows, allow users to specify the device
+ name 'rdma_devN' in the dapl.conf file. The given index, N, is map to a
+ corresponding IP address that is associated with an RDMA device.
+
+ Signed-off-by: Sean Hefty <sean.hefty@intel.com>
+
+commit 79fa3e7d241f740bc886dd075f24fcbc611306de
+Author: Arlin Davis <arlin.r.davis@intel.com>
+Date: Thu Jun 4 08:00:29 2009 -0700
+
+ openib: remove 1st gen provider, replaced with openib_cma and openib_scm
+
+ Signed-off-by: Arlin Davis <arlin.r.davis@intel.com>
+
+commit 624039247cdc0db7aa040dfbb4dced00f2cf9006
+Author: Arlin Davis <arlin.r.davis@intel.com>
+Date: Fri May 29 08:21:10 2009 -0700
+
+ dapltest: update windows script files
+
+ Enhancement to take DAPL provider name as cmd-line arguement.
+
+ Signed-off-by: Arlin Davis <arlin.r.davis@intel.com>
+
+commit b93baa07b7bbaeb7a55fa817c354d0c94783d61f
+Author: Arlin Davis <arlin.r.davis@intel.com>
+Date: Thu May 28 15:30:05 2009 -0700
+
+ dapltest: update windows batch files in sripts directory
+
+ Signed-off-by: Arlin Davis <arlin.r.davis@intel.com>
+
+commit 2f185c6b5e464c4fc9e84ad3e90cc2b86aebf9aa
+Author: Arlin Davis <arlin.r.davis@intel.com>
+Date: Mon May 18 14:00:02 2009 -0700
+
+ windows_osd/linux_osd: new dapl_os_gettid macro to return thread id
+
+ Change dapl_os_getpid inline to macro on windows and add dapl_os_gettid
+ macros on linux and windows to return thread id.
+
+ Signed-off-by: Arlin Davis <arlin.r.davis@intel.com>
+
+commit 984303824cd0c3e248a789066cf665ced8e1ae5b
+Author: Arlin Davis <arlin.r.davis@intel.com>
+Date: Mon May 18 13:53:59 2009 -0700
+
+ windows: missing build files for common and udapl sub-directories
+
+ Add dapl/dapl_common_src.c and dapl/dapl_udapl_src.c
+
+ Signed-off-by: Arlin Davis <arlin.r.davis@intel.com>
+
+commit 3be4ccf9681a975e74a5aa05e3f7912477f342a7
+Author: Arlin Davis <arlin.r.davis@intel.com>
+Date: Mon May 18 09:06:19 2009 -0700
+
+ windows: add build files for openib_scm, remove /Wp64 build option.
+
+ Add build files for windows socket cm and change build
+ option on windows providers. The new Win7 WDK issues a
+ depreciated compiler option warning for /Wp64
+ (Enable 64-bit porting warnings)
+
+ Signed-off-by: Arlin Davis <arlin.r.davis@intel.com>
+
+commit 163112cfeb6e409886b3cb7f85da7ce003300d5c
+Author: Arlin Davis <arlin.r.davis@intel.com>
+Date: Mon May 18 08:50:35 2009 -0700
+
+ scm: multi-hca CM processing broken. Need cr thread wakeup mechanism per HCA.
+
+ Currently there is only one pipe across all
+ device opens. This results in some posted CR work
+ getting delayed or not processed at all. Provide
+ pipe for each device open and cr thread created
+ and manage on a per device level.
+
+ Signed-off-by: Arlin Davis <arlin.r.davis@intel.com>
+
+commit e6e799f623df6ef136ffc5388251d3f3a38c8a91
+Author: Arlin Davis <arlin.r.davis@intel.com>
+Date: Fri May 15 11:06:19 2009 -0700
+
+ dtest: add connection timers on client side
+
+ Add timers for active connections and print
+ results. Allow polling or wait on conn event.
+
+ Signed-off-by: Arlin Davis <arlin.r.davis@intel.com>
+
+commit d656bbf619123deaed6e8985e52207e5415f359f
+Author: Arlin Davis <arlin.r.davis@intel.com>
+Date: Fri May 15 09:48:38 2009 -0700
+
+ linux_osd: use pthread_self instead of getpid for debug messages
+
+ getpid provides process ids which are not unique. Use unique thread
+ id's in debug messages to help isolate issues across many device
+ opens with multiple CM threads.
+
+ Signed-off-by: Arlin Davis <arlin.r.davis@intel.com>
+
+commit 92bb0d2933d3d1546e18f0479475f3daf5b92052
+Author: Arlin Davis <arlin.r.davis@intel.com>
+Date: Fri May 1 10:18:05 2009 -0700
+
+ windows ibal-scm: dapl/dirs file needs updated to remove ibal-scm
+
+ Signed-off-by: Arlin Davis <arlin.r.davis@intel.com>
+
+commit 9c37d9d667fb7e8f21841bbec4a84b2c652fffe1
+Author: Arlin Davis <arlin.r.davis@intel.com>
+Date: Wed Apr 29 23:13:36 2009 -0700
+
+ Release 2.0.19
+
+ Signed-off-by: Arlin Davis <arlin.r.davis@intel.com>
+
+commit 0ef94459e0a0175233b43b3fcbaaac2596e1042d
+Author: Arlin Davis <arlin.r.davis@intel.com>
+Date: Wed Apr 29 14:33:28 2009 -0700
+
+ scm, cma: dat max_lmr_block_size is 32 bit, verbs max_mr_size is 64 bit
+
+ mismatch of device attribute size restricts max_lmr_block_size to 32 bit
+ value. Add check, if larger then limit to 4G-1 until DAT v2 spec changes.
+
+ Consumers should use max_lmr_virtual_address for actual max
+ registration block size until attribute interface changes.
+
+ Signed-off-by: Arlin Davis <arlin.r.davis@intel.com>
+
+commit f91f27eaaab28b13a631adf75b933b7be3afbc0f
+Author: Arlin Davis <arlin.r.davis@intel.com>
+Date: Wed Apr 29 10:51:03 2009 -0700
+
+ scm: increase default MTU size from 1024 to 2048
+
+ Signed-off-by: Arlin Davis <arlin.r.davis@intel.com>
+
+commit 8d6846056f4c86b6a06346147df55d37c4ba9933
+Author: Arlin Davis <arlin.r.davis@intel.com>
+Date: Wed Apr 29 10:49:09 2009 -0700
+
+ openib_scm, cma: use direct SGE mappings from dat_lmr_triplet to ibv_sge
+
+ no need to rebuild scatter gather list given that DAT v2.0
+ is now aligned with verbs ibv_sge. Fix ib_send_op_type_t typedef.
+
+ Signed-off-by: Arlin Davis <arlin.r.davis@intel.com>
+
+commit c61f75b3412935e7d4a7a1acc9c495dcb4ac7e24
+Author: Arlin Davis <arlin.r.davis@intel.com>
+Date: Wed Apr 29 08:39:37 2009 -0700
+
+ dtest: add flush EVD call after data transfer errors
+
+ Flush and print entries on async, request, and receive
+ queues after any data transfer error. Will help
+ identify failing operation during operations
+ without completion events requested.
+ Fix -B0 so burst size of 0 works.
+
+ Signed-off-by: Arlin Davis <arlin.r.davis@intel.com>
+
+commit c88a191c7a408b0fb3dfb418a77a5b3b5afc778e
+Author: Arlin Davis <arlin.r.davis@intel.com>
+Date: Wed Apr 22 13:16:19 2009 -0700
+
+ dtest/dapltest: Cleanup code with Lindent
+
+ Signed-off-by: Arlin Davis <ardavis@ichips.intel.com>
+
+commit 8699a9f1bd2bb45b04b87f887698707ba7b62d0a
+Author: Arlin Davis <arlin.r.davis@intel.com>
+Date: Tue Apr 21 15:51:24 2009 -0700
+
+ ibal-scm: remove, obsolete
+
+ Signed-off-by: Arlin Davis <ardavis@ichips.intel.com>
+
+commit 67ddd6bfba46f1f7a61b772257132f1257d05c96
+Author: Arlin Davis <arlin.r.davis@intel.com>
+Date: Tue Apr 21 15:44:15 2009 -0700
+
+ scm, cma provider: Cleanup code with Lindent
+
+ Signed-off-by: Arlin Davis <ardavis@ichips.intel.com>
+
+commit d0898091090ff19be7929fed0d14f1ca696d5e53
+Author: Arlin Davis <arlin.r.davis@intel.com>
+Date: Tue Apr 21 15:39:01 2009 -0700
+
+ udapl: Cleanup code with Lindent
+
+ Signed-off-by: Arlin Davis <ardavis@ichips.intel.com>
+
+commit a688d1cfb52fde256c5bfd95a27f940dd17e7ced
+Author: Arlin Davis <arlin.r.davis@intel.com>
+Date: Tue Apr 21 15:31:20 2009 -0700
+
+ dapl common: Cleanup code with Lindent
+
+ Signed-off-by: Arlin Davis <ardavis@ichips.intel.com>
+
+commit 2bded73c7f68cfb870e432ab3ebae7427d595cbe
+Author: Arlin Davis <arlin.r.davis@intel.com>
+Date: Tue Apr 21 12:52:29 2009 -0700
+
+ dat: Cleanup code with Lindent
+
+ Signed-off-by: Arlin Davis <ardavis@ichips.intel.com>
+
+commit 31e7b9210fc5334ff3be62558e74e3fdf01d6cbd
+Author: Arlin Davis <arlin.r.davis@intel.com>
+Date: Mon Apr 20 12:28:08 2009 -0700
+
+ Release 2.0.18
+
+ Signed-off-by: Arlin Davis <ardavis@ichips.intel.com>
+
+commit 29bf0a24f54c45d2742026756f31f1a1f26fb6f3
+Author: Arlin Davis <arlin.r.davis@intel.com>
+Date: Thu Apr 16 14:35:18 2009 -0700
+
+ dapltest: reset server listen ports to avoid collisions during long runs
+
+ If server is running continuously the port number increments
+ from base without reseting between tests. This will
+ eventually cause collisions in port space.
+
+ Signed-off-by: Arlin Davis <ardavis@ichips.intel.com>
+
+commit c27af8de0501d132b8152ec8546023cdba212de5
+Author: Sean Hefty <sean.hefty@intel.com>
+Date: Thu Apr 16 10:21:51 2009 -0700
+
+ To avoid duplicating port numbers between different tests, the next port
+ number to use must increment based on the number of endpoints per thread *
+ the number of threads.
+
+ Signed-off-by: Sean Hefty <sean.hefty@intel.com>
+
+commit 3084310197c20aaa50abe82260fc835786f591f5
+Author: Sean Hefty <sean.hefty@intel.com>
+Date: Thu Apr 16 10:21:45 2009 -0700
+
+ dapltest assumes that events across multiple endpoints occur in a specific
+ order. Since this is a false assumption, avoid this by directing events to
+ per endpoint EVDs, rather than using shared EVDs.
+
+ Signed-off-by: Sean Hefty <sean.hefty@intel.com>
+
+commit ef87a0a462f4fa07ac252e28d3aeb44af73cc202
+Author: Sean Hefty <sean.hefty@intel.com>
+Date: Thu Apr 16 10:21:41 2009 -0700
+
+ Synchronization is missing between removing items from an EVD and queuing
+ them. Since the removal thread is the user's, but the queuing thread is
+ not, the synchronization must be provided by DAPL. Hold the evd lock
+ around any calls to dapls_rbuf_*.
+
+ Signed-off-by: Sean Hefty <sean.hefty@intel.com>
+
+commit f5e86d28f803162ffdf94b41ec7435dec92f728d
+Author: Sean Hefty <sean.hefty@intel.com>
+Date: Thu Apr 16 10:21:26 2009 -0700
+
+ Communication to the CR thread is done using an internal socket. When a
+ new connection request is ready for processing, an object is placed on
+ the CR list, and data is written to the internal socket. The write causes
+ the CR thread to wake-up and process anything on its cr list.
+
+ If multiple objects are placed on the CR list around the same time, then
+ the CR thread will read in a single character, but process the entire list.
+ This results in additional data being left on the internal socket. When
+ the CR does a select(), it will find more data to read, read the data, but
+ not have any real work to do. The result is that the thread spins in a
+ loop checking for changes when none have occurred until all data on the
+ internal socket has been read.
+
+ Avoid this overhead by reading all data off the internal socket before
+ processing the CR list.
+
+ Signed-off-by: Sean Hefty <sean.hefty@intel.com>
+
+commit 2ab52e9b1ab37c6eb44206c135e0568a8c2d01fa
+Author: Sean Hefty <sean.hefty@intel.com>
+Date: Thu Apr 16 10:21:13 2009 -0700
+
+ The dapl connect call takes as input an address (sockaddr) and a port number
+ as separate input parameters. It modifies the sockaddr address to set the
+ port number before trying to connect. This leads to a situation in
+ dapltest with multiple threads that reference the same buffer for their
+ address, but specify different port numbers, where the different threads
+ end up trying to connect to the same remote port.
+
+ To solve this, do not modify the caller's address buffer and instead use
+ a local buffer. This fixes an issue seen running multithreaded tests with
+ dapltest.
+
+ Signed-off-by: Sean Hefty <sean.hefty@intel.com>
+
+commit 7947026ede478f08b4a7b8cb607f457765bf2afa
+Author: Sean Hefty <sean.hefty@intel.com>
+Date: Thu Apr 16 10:21:03 2009 -0700
+
+ Windows socket calls should check return values against SOCKET_ERROR to
+ determine if an error occurred.
+
+ Signed-off-by: Sean Hefty <sean.hefty@intel.com>
+
+commit e66e42fc44c50c8202f7c98f76d799a69aa3f1b6
+Author: Arlin Davis <arlin.r.davis@intel.com>
+Date: Fri Apr 10 08:33:41 2009 -0700
+
+ Build: add new file dapl/openib_cma/linux/openib_osd.h to EXTRA_DIST
+
+ Fix rpmbuild problem with new cma osd include file.
+
+ Signed-off-by: Arlin Davis <ardavis@ichips.intel.com>
+
+commit acb213adb3268e9bf6999e2bf040d4a71212b701
+Author: Arlin Davis <arlin.r.davis@intel.com>
+Date: Fri Apr 10 08:32:24 2009 -0700
+
+ dapl scm: reduce wait time for thread startup.
+
+ thread startup wait reduce to 2ms to reduce open times.
+
+ Signed-off-by: Arlin Davis <ardavis@ichips.intel.com>
+
+commit 55459699fa9c0e5fb7e2b17822f0916412c64b35
+Author: Arlin Davis <arlin.r.davis@intel.com>
+Date: Fri Apr 10 08:31:22 2009 -0700
+
+ dapl-scm: getsockopt optlen needs initialized to size of optval
+
+ Signed-off-by: Arlin Davis <ardavis@ichips.intel.com>
+
+commit d710c5327e05a40796341d16b45a2b098b03f588
+Author: Sean Hefty <sean.hefty@intel.com>
+Date: Fri Apr 10 08:17:32 2009 -0700
+
+ The connection request thread adds sockets to a select list unless
+ the cr->socket is invalid and the cr request state is set to destroy. If the
+ cr->socket is invalid, but the cr->state is not destroy, then the cr->socket
+ is added to an FD set for select/poll. This results in select/poll
+ returning an error when select is called. As a result, the cr thread never
+ actually blocks during this state.
+
+ Fix this by only destroying a cr based on its state being set to destroy
+ and skip adding cr->sockets to the FD set when they are invalid.
+
+ Signed-off-by: Sean Hefty <sean.hefty@intel.com>
+
+commit 0be961c432f897d4f92d9a24dcb7c42ad30ea160
+Author: Sean Hefty <sean.hefty@intel.com>
+Date: Fri Apr 10 08:08:16 2009 -0700
+
+ Make sure all locks are initialized properly and don't zero their memory
+ once they are.
+
+ Signed-off-by: Sean Hefty <sean.hefty@intel.com>
+
+commit 9abdc26cd6154aa55588759ba54c9ca69e3fe2b5
+Author: Sean Hefty <sean.hefty@intel.com>
+Date: Fri Apr 10 08:08:13 2009 -0700
+
+ The lock functions are defined just a few lines beneath the prototypes
+ as inline. Remove the duplicate prototypes.
+
+ Signed-off-by: Sean Hefty <sean.hefty@intel.com>
+
+commit 9578c4aeb9878d98374e4b7abc02db182aef82c6
+Author: Sean Hefty <sean.hefty@intel.com>
+Date: Fri Apr 10 08:08:07 2009 -0700
+
+ Make sure all locks are initialized and don't zero out their memory once
+ they are.
+
+ Signed-off-by: Sean Hefty <sean.hefty@intel.com>
+
+commit 97edcbb662b489303ef68c0da02831efaddeed91
+Author: Sean Hefty <sean.hefty@intel.com>
+Date: Fri Apr 10 08:08:03 2009 -0700
+
+ The IBAL library allocates a small number of threads for callbacks to the
+ user. If the user blocks all of the callback threads, no additional
+ callbacks can be invoked. The DAPL IBAL provider cancels listen requests
+ from within an IBAL callback, then waits for a second callback to confirm
+ that the listen has been canceled. If there is a single IBAL callback
+ thread, or multiple listens are canceled simultaneously, then the provider
+ can deadlock waiting for a cancel callback that never occurs.
+
+ This problem is seen when running dapltest with multiple threads.
+
+ Signed-off-by: Sean Hefty <sean.hefty@intel.com>
+
+commit 3e56e63bcb68de352edadafdcfcc4cb222c08c7b
+Author: Sean Hefty <sean.hefty@intel.com>
+Date: Fri Apr 10 08:07:57 2009 -0700
+
+ We need to check the return value from select for errors before checking
+ the FD sets. An item may be in an FD set but select could have returned
+ an error.
+
+ Signed-off-by: Sean Hefty <sean.hefty@intel.com>
+
+commit a8a977becaeefe0d7f8e01e01631a11988d2d54e
+Author: Sean Hefty <sean.hefty@intel.com>
+Date: Fri Apr 10 08:07:53 2009 -0700
+
+ Signed-off-by: Sean Hefty <sean.hefty@intel.com>
+
+commit ecc79cc0a1ae2bdbb3dfd19e15b3b562ac9a2957
+Author: Sean Hefty <sean.hefty@intel.com>
+Date: Fri Apr 10 08:07:49 2009 -0700
+
+ Enable building with CQ_WAIT_OBJECTS support to directly wait on CQ
+ completion channels in the Windows version of the openib_scm provider.
+ Also minor fixup to use DAPL_DBG_TYPE_UTIL for debug log messages
+ instead of DAPL_DBG_TYPE_CM.
+
+ Signed-off-by: Sean Hefty <sean.hefty@intel.com>
+
+commit 73728763666a46df5789af93b50db53cdf64afd6
+Author: Sean Hefty <sean.hefty@intel.com>
+Date: Fri Apr 10 08:07:44 2009 -0700
+
+ The IBAL-SCM provider will run into an inifinite loop if the check for
+ cr->socket > SCM_MAX_CONN - 1 fails. The code continues back to the start
+ of the while loop without moving to the next connection request entry
+ in the list.
+
+ Signed-off-by: Sean Hefty <sean.hefty@intel.com>
+
+commit 9b1b396539926d36ffacfff04fbe7c081e436b45
+Author: Sean Hefty <sean.hefty@intel.com>
+Date: Fri Apr 10 08:07:40 2009 -0700
+
+ next_cr is set just before and inside the check
+ if ((cr->socket == DAPL_INVALID_SOCKET && cr->state == SCM_DESTROY)
+ Remove setting it inside the if statement.
+
+ Signed-off-by: Sean Hefty <sean.hefty@intel.com>
+
+commit 7b49a9850f62276bb7bfccb2d85a1e94e311813c
+Author: Sean Hefty <sean.hefty@intel.com>
+Date: Fri Apr 10 08:07:35 2009 -0700
+
+ Some errors on windows are more easily interpretted in hex than decimal.
+
+ Signed-off-by: Sean Hefty <sean.hefty@intel.com>
+
+commit 08ee072a1396ac2c28983878dbc6b02feb035787
+Author: Sean Hefty <sean.hefty@intel.com>
+Date: Fri Apr 10 08:07:32 2009 -0700
+
+ The WinOF HCA driver cannot handle transitioning from RTS -> RESET ->
+ INIT -> ERROR. Simply delete the QP and re-create it to reinitialize
+ the endpoint until the bug is fixed.
+
+ Signed-off-by: Sean Hefty <sean.hefty@intel.com>
+
+commit 51ef5d96ce67d6141ec02b2a318f1b6e12be1bcf
+Author: Sean Hefty <sean.hefty@intel.com>
+Date: Fri Apr 10 08:07:23 2009 -0700
+
+ Signed-off-by: Sean Hefty <sean.hefty@intel.com>
+
+commit a8582be0e1fc89e856f1d0b43a3c1b271295a352
+Author: Sean Hefty <sean.hefty@intel.com>
+Date: Fri Apr 10 08:07:18 2009 -0700
+
+ Convert the openib_cma provider to common code between linux and windows.
+
+ Signed-off-by: Sean Hefty <sean.hefty@intel.com>
+
+commit cc2a71dfe0c35a70f6b1ba66070a3a06059a8bb5
+Author: Sean Hefty <sean.hefty@intel.com>
+Date: Fri Apr 10 08:06:53 2009 -0700
+
+ Move from using pipes to sockets for internal communication. This
+ avoids issues with windows only supporting select() on sockets.
+
+ Remove windows specific definition of dapl_dbg_log.
+
+ Update to latest windows libibverbs implementation using completion
+ channel abstraction to improve windows scalability and simplify
+ porting where FD's are accessed directly in Linux.
+
+ Signed-off-by: Sean Hefty <sean.hefty@intel.com>
+
+commit b3ad2ed97399a24a869841e17d1314e11c379aae
+Author: Arlin Davis <arlin.r.davis@intel.com>
+Date: Tue Mar 31 05:41:50 2009 -0800
+
+ Release 2.0.17
+
+ Signed-off-by: Arlin Davis <ardavis@ichips.intel.com>
+
+commit 5d732929f8a90a490994e8e35a3666c3647ad4fe
+Author: Arlin Davis <arlin.r.davis@intel.com>
+Date: Tue Mar 31 05:22:11 2009 -0800
+
+ dapl: ia64 build problem on SuSE 11, atomic.h no longer exists.
+
+ Add autotools check for SuSE 11 and include intrinsics.h
+
+ Signed-off-by: Arlin Davis <ardavis@ichips.intel.com>
+
+commit d7b8654db3a1f4ead16cb2e6d15f0902a322a188
+Author: Arlin Davis <arlin.r.davis@intel.com>
+Date: Mon Mar 16 13:23:50 2009 -0800
+
+ Release 2.0.16
+
+ Fix changelog year in spec file.
+
+ Signed-off-by: Arlin Davis <ardavis@ichips.intel.com>
+
+commit 08d9e26a85911f99d47cbb92ec8bccfc7f073be0
+Author: Arlin Davis <arlin.r.davis@intel.com>
+Date: Mon Mar 16 13:15:22 2009 -0800
+
+ Release 2.0.16
+
+ Signed-off-by: Arlin Davis <ardavis@ichips.intel.com>
+
+commit 42c97b2a11d63ac6ba8a15fe8c82061e7da6e136
+Author: Arlin Davis <arlin.r.davis@intel.com>
+Date: Fri Mar 13 12:39:12 2009 -0800
+
+ uDAPL: scm provider, remove query gid/lid from connection setup phase
+
+ move lid/gid queries from the connection setup phase
+ and put them in the open call to avoid overhead
+ of more fd's during connections. No need
+ to query during connection setup since uDAPL
+ binds to specific hca/ports via dat_ia_open.
+
+ Signed-off-by: Arlin Davis <ardavis@ichips.intel.com>
+
+commit 775394b73980a7bc0af018a33d2a5bb795469c78
+Author: Arlin Davis <arlin.r.davis@intel.com>
+Date: Thu Mar 12 12:44:43 2009 -0800
+
+ Build: missing new linux/osd include file in EXTRA_DIST
+
+ Add dapl/openib_scm/linux/openib_osd.h to EXTRA_DIST
+
+ Signed-off-by: Arlin Davis <ardavis@ichips.intel.com>
+
+commit 647e288c1c9af5261495a5ed88e6ecbe1daf6d6e
+Author: Arlin Davis <arlin.r.davis@intel.com>
+Date: Thu Mar 12 12:11:21 2009 -0800
+
+ Build: spec files missing Requires(post) statements for sed/coreutils
+
+ needed for anaconda install
+
+ Signed-off-by: Arlin Davis <ardavis@ichips.intel.com>
+
+commit 1f5b3b7cab0785b64e8dab035dd4cd27111497d3
+Author: Arlin Davis <arlin.r.davis@intel.com>
+Date: Wed Mar 4 10:04:13 2009 -0800
+
+ dapl scm: remove unecessary thread when using direct objects
+
+ A thread is created for processing events on devices without
+ direct event objecti support. Since all openfabrics devices support
+ direct events there is no need to start a thread. Move this under
+
+ Signed-off-by: Arlin Davis <ardavis@ichips.intel.com>
+
+commit 66e4236e2c57dbaf860b7c20809b65a4fbbafa6f
+Author: Arlin Davis <arlin.r.davis@intel.com>
+Date: Tue Mar 3 11:08:12 2009 -0800
+
+ dtestx: add missing F64u definition for windows
+
+ Signed-off-by: Arlin Davis <ardavis@ichips.intel.com>
+
+commit d9e771da16ec2b360a222ceccbbca5d088e20ee5
+Author: Arlin Davis <arlin.r.davis@intel.com>
+Date: Tue Mar 3 09:25:26 2009 -0800
+
+ uDAPL common: add 64 bit counters for IA, EP, and EVD's.
+
+ -DDAPL_COUNTERS to build-in counters for cma and scm providers.
+ New extension calls in dat_ib_extensions.h for counters
+ dat_print_counters, dat_query_counters
+ Counters for operations, async errors, and data
+ Update dtestx (-p) with print and query counter examples
+
+ Signed-off-by: Arlin Davis <ardavis@ichips.intel.com>
+
+commit 5e6ad3cdaa1d72523ad6b38d8306e0a2d0f1ada6
+Author: Sean Hefty <sean.hefty@intel.com>
+Date: Tue Feb 17 07:24:27 2009 -0800
+
+ Modify the openib_scm provider to support both OFED and WinOF releases.
+ This takes advantage of having a libibverbs compatibility library.
+
+ Signed-off-by: Sean Hefty <sean.hefty@intel.com>
+
+commit 0425c24c3b66e672bba346a1d0946b11c7b8b11e
+Author: Stan Smith <stan.smith@intel.com>
+Date: Fri Jan 30 09:52:33 2009 -0800
+
+ Update the dapl.git tree with the latest SVN version of the
+ ibal-scm provider.
+
+ Signed-off-by: Sean Hefty <sean.hefty@intel.com>
+
+commit 83543b6cca342e25fd6408454f1261ec6835a172
+Author: Stan Smith <stan.smith@intel.com>
+Date: Fri Jan 30 09:52:25 2009 -0800
+
+ Merge SVN IBAL provider code back into the main git tree.
+
+ Signed-off-by: Sean Hefty <sean.hefty@intel.com>
+
+commit be0b6122d0fe4f93afc8cf3ec961702faf82fb44
+Author: Sean Hefty <sean.hefty@intel.com>
+Date: Fri Jan 30 09:52:11 2009 -0800
+
+ Changes to dtest to support building on Windows.
+
+ Signed-off-by: Sean Hefty <sean.hefty@intel.com>
+
+commit 9f87610c4b37e4db4d74205c14028582a2f6a79e
+Author: Stan Smith <stan.smith@intel.com>
+Date: Fri Jan 30 09:48:26 2009 -0800
+
+ Add return codes to various functions.
+ Add script (batch file) for Windows testing.
+
+ Signed-off-by: Sean Hefty <sean.hefty@intel.com>
+
+commit 723067550265defdcfe6e00460a4f89f7a81fbf1
+Author: Sean Hefty <sean.hefty@intel.com>
+Date: Fri Jan 30 09:46:40 2009 -0800
+
+ Merge OFED and WinOF trees for common dapl code.
+
+ Signed-off-by: Sean Hefty <sean.hefty@intel.com>
+
+commit 5b37e080e1ecc6903b3ab1ac9b0d4c9f8a18b790
+Author: Arlin Davis <arlin.r.davis@intel.com>
+Date: Fri Jan 16 08:16:13 2009 -0800
+
+ dtest/dapltest: use $(top_builddir) for .la files during test builds
+
+ Signed-off-by: Arlin Davis <ardavis@ichips.intel.com>
+
+commit bd655d4fdd7e63e7fdeed3979926da5582f71e4b
+Author: Arlin Davis <arlin.r.davis@intel.com>
+Date: Wed Nov 26 07:12:30 2008 -0800
+
+ Release 2.0.15
+
+ Signed-off-by: Arlin Davis <ardavis@ichips.intel.com>
+
+commit 5d5dec42717c963d1644ee3e716459dc5d58e930
+Author: Arlin Davis <arlin.r.davis@intel.com>
+Date: Mon Oct 27 08:48:53 2008 -0800
+
+ dapltest: transaction test moves to cleanup stage before rdma_read processing is complete
+
+ With multiple treads, the transaction server tread can move to cleanup
+ stage and unregister memory before the remote client process has
+ completed the rdma read. In lieu of a rewrite to add sync messages
+ at the end of transaction test phase, just add a delay before cleanup.
+
+ Signed-off-by: Arlin Davis <ardavis@ichips.intel.com>
+
+commit 101a843e59b881bc131dfd5c2aec7e54d44f35c0
+Author: Arlin Davis <arlin.r.davis@intel.com>
+Date: Tue Oct 14 11:56:35 2008 -0700
+
+ Current static registration (SR) assumes DAT_OVERRIDE or /etc/dat.conf.
+ Change SR to include sysconfdir. SR file access in the following order:
+
+ - DAT_OVERRIDE
+ - sysconfdir
+ - /etc
+
+ if DAT_OVERRIDE is set, assume administration override
+ and do not failover to other locations. Add debug
+ messages for each failure and retries.
+
+ Signed-off-by: Arlin Davis <ardavis@ichips.intel.com>
+ Acked-by: Doug Ledford <dledford@redhat.com>
+
+commit c98d2169b839a73d76691acf510dd8976ddc850a
+Author: Arlin Davis <arlin.r.davis@intel.com>
+Date: Fri Oct 3 08:00:56 2008 -0700
+
+ Release 2.0.14
+
+ Signed-off-by: Arlin Davis <ardavis@ichips.intel.com>
+
+commit c26d0bb065f3734f09058e1e6d26dde4a3738e55
+Author: Arlin Davis <arlin.r.davis@intel.com>
+Date: Fri Oct 3 05:40:04 2008 -0700
+
+ dat.conf: add ofa-v2-iwarp entry for iwarp devices
+
+ Signed-off-by: Arlin Davis <ardavis@ichips.intel.com>
+
+commit d54c35c8a9ba33a464ca15f4a65b914688e5194d
+Author: Arlin Davis <arlin.r.davis@intel.com>
+Date: Fri Oct 3 05:30:10 2008 -0700
+
+ dapl: adjust max_rdma_read_iov to 1 for query on iWARP devices
+
+ iWarp spec allows only one iov on rdma reads
+
+ Signed-off-by: Arlin Davis <ardavis@ichips.intel.com>
+
+commit 9584df22d0cb3aa5a2a2a04bf427524d17650ef7
+Author: Arlin Davis <arlin.r.davis@intel.com>
+Date: Wed Oct 1 08:17:49 2008 -0700
+
+ dtest: reduce default IOV's during dat_ep_create for iWARP devices
+
+ iWarp adapters tend to have less IOV resources then IB adapters.
+
+ Signed-off-by: Arlin Davis <ardavis@ichips.intel.com>
+
+commit 8ca9898621acf5b9769d46ea34ee8ca1eecaf2ff
+Author: Arlin Davis <arlin.r.davis@intel.com>
+Date: Fri Sep 26 12:43:13 2008 -0700
+
+ dtest: fix 32-bit build issues in dtest and dtestx examples.
+
+ Signed-off-by: Arlin Davis <ardavis@ichips.intel.com>
+
+commit 42a3a4edf30115a35d9d599b51f8756814e62368
+Author: Arlin Davis <arlin.r.davis@intel.com>
+Date: Fri Sep 26 08:48:31 2008 -0700
+
+ Revert "Release 2.0.14"
+
+ This reverts commit 816d1c3acfb4a457c6b1cc07d657f018312f2a63.
+
+ missed some fixes for package release.
+
+commit 816d1c3acfb4a457c6b1cc07d657f018312f2a63
+Author: Arlin Davis <arlin.r.davis@intel.com>
+Date: Wed Sep 24 12:13:37 2008 -0700
+
+ Release 2.0.14
+
+ Signed-off-by: Arlin Davis <ardavis@ichips.intel.com>
+
+commit c00d858da3113ce5463d408ab5e13e17cc9529e4
+Author: Arlin Davis <arlin.r.davis@intel.com>
+Date: Wed Sep 24 08:33:32 2008 -0700
+
+ build: $(DESTDIR) prepend needed on install hooks for dat.conf
+
+ All install directives that automake creates automatically
+ have $(DESTDIR) prepended to them so that a make
+ DESTDIR=<some_path> install will work. The hand written
+ install hooks for dat.conf was missing DESTDIR.
+
+ Signed-off-by: Doug Ledford <dledford@redhat.com>
+
+commit 860db3be4907c8ff290ce7c6b631b2117f5080bd
+Author: Arlin Davis <arlin.r.davis@intel.com>
+Date: Wed Sep 24 08:26:28 2008 -0700
+
+ dapl scm: UD shares EP's which requires serialization
+
+ add locking around the modify_qp state changes to avoid
+ unnecessary modify_qp calls during multiple resolve
+ remote AH connection events on a single EP.
+
+ Signed-off-by: Arlin Davis <ardavis@ichips.intel.com>
+
+commit f2c214647b5ce53e52052d6b6bea3fbace7cc20a
+Author: Arlin Davis <arlin.r.davis@intel.com>
+Date: Sat Sep 20 16:02:00 2008 -0700
+
+ dtestx: Add new options to test UD.
+
+ - many to one/many EP remote AH resolution, data flow
+ - bi-directional EP remote AH resolution, data flow
+
+ Signed-off by: Arlin Davis ardavis@ichips.intel.com
+
+commit 3c218b5ed65d0b7349a86eb0fe6f6bf57e3eccd6
+Author: Arlin Davis <arlin.r.davis@intel.com>
+Date: Sat Sep 20 15:58:59 2008 -0700
+
+ dapl: fixes for IB UD extensions in common code and socket cm provider.
+
+ - Manage EP states base on attribute service type.
+ - Allow multiple connections (remote_ah resolution)
+ and accepts on UD type endpoints.
+ - Supply private data on CR conn establishment
+ - Add UD extension conn event type - DAT_IB_UD_PASSIVE_REMOTE_AH
+
+ Signed-off by: Arlin Davis ardavis@ichips.intel.com
+
+commit 01cdd688ddebec8df6f17f92c3b72a410f50dd6f
+Author: Arlin Davis <arlin.r.davis@intel.com>
+Date: Sat Sep 20 15:49:40 2008 -0700
+
+ dapl: add provider specific attribute query option for IB UD MTU size
+
+ Signed-off by: Arlin Davis ardavis@ichips.intel.com
+
+commit ee3d9ce2389a14c85405dfdff67f04a5f5679e32
+Author: Arlin Davis <arlin.r.davis@intel.com>
+Date: Mon Sep 1 15:52:37 2008 -0700
+
+ Release 2.0.13
+
+ Signed-off by: Arlin Davis ardavis@ichips.intel.com
+
+commit f37589927fabf0feb3a09c4c7c03e18df5749fef
+Author: Arlin Davis <arlin.r.davis@intel.com>
+Date: Mon Sep 1 12:24:49 2008 -0700
+
+ dapl build: add correct CFLAGS, set non-debug build by default for v2
+
+ Signed-off by: Arlin Davis ardavis@ichips.intel.com
+
+commit 280f3aec6d0fa9d7e36f75711471e35333ee34cf
+Author: Arlin Davis <arlin.r.davis@intel.com>
+Date: Mon Sep 1 12:22:19 2008 -0700
+
+ dapl providers: fix compiler warnings in cma and scm providers
+
+ Signed-off by: Arlin Davis ardavis@ichips.intel.com
+
+commit 60c0589a2aba520ae67b1c8eaad5a167edb6fba3
+Author: Arlin Davis <arlin.r.davis@intel.com>
+Date: Mon Sep 1 12:20:08 2008 -0700
+
+ dat: fix compiler warnings in dat common code
+
+ Signed-off by: Arlin Davis ardavis@ichips.intel.com
+
+commit fb3e368db07d02a3daa1d12d71f62ac4e7a5ef23
+Author: Arlin Davis <arlin.r.davis@intel.com>
+Date: Mon Sep 1 12:18:48 2008 -0700
+
+ dapl: fix compiler warnings in common code
+
+ Signed-off by: Arlin Davis ardavis@ichips.intel.com
+
+commit 38a53ae75bce5059a84262fe1b40eacf92b22287
+Author: Arlin Davis <arlin.r.davis@intel.com>
+Date: Mon Sep 1 12:16:58 2008 -0700
+
+ dtest/dapltest: fix compiler warnings
+
+ Signed-off by: Arlin Davis ardavis@ichips.intel.com
+
+commit 45900087764917b354411fdd2b3880473d553ab8
+Author: Arlin Davis <arlin.r.davis@intel.com>
+Date: Fri Aug 22 14:51:22 2008 -0700
+
+ dapl cma: debug message during query needs definition for inet_ntoa
+
+ Signed-off by: Arlin Davis ardavis@ichips.intel.com
+
+commit 99d46313a03af18771966cf86fcc934d179627b5
+Author: Arlin Davis <arlin.r.davis@intel.com>
+Date: Thu Aug 21 12:54:58 2008 -0700
+
+ Release 2.0.12
+
+ Signed-off by: Arlin Davis ardavis@ichips.intel.com
+
+commit 0b2a3fac8d28d5b3c2e1416fa696fe4cbc672f00
+Author: Arlin Davis <arlin.r.davis@intel.com>
+Date: Wed Aug 20 18:51:00 2008 -0700
+
+ dapl scm: fix corner case that delivers duplicate disconnect events
+
+ Signed-off by: Arlin Davis ardavis@ichips.intel.com
+
+commit 96f6822b90fa880a6c6a64b1e183064a449f7237
+Author: Arlin Davis <arlin.r.davis@intel.com>
+Date: Wed Aug 20 18:47:19 2008 -0700
+
+ dat: include stddef.h for NULL definition in dat_platform_specific.h
+
+ Signed-off by: Arlin Davis ardavis@ichips.intel.com
+
+commit 283e37c09ad072d69d29e28225e9a6e8bf3f75f0
+Author: Arlin Davis <arlin.r.davis@intel.com>
+Date: Wed Aug 20 18:27:08 2008 -0700
+
+ dapl: add debug messages during async and overflow events
+
+ Signed-off by: Arlin Davis ardavis@ichips.intel.com
+
+commit 4a7965ac07b7f2ee1deb5b144ed50b30c1749d38
+Author: Arlin Davis <arlin.r.davis@intel.com>
+Date: Wed Aug 20 18:24:33 2008 -0700
+
+ dapltest: add check for duplicate disconnect events in transaction test
+
+ Signed-off by: Arlin Davis ardavis@ichips.intel.com
+
+commit 7e8986f2c6496851b724a007458881c3248ac998
+Author: Arlin Davis <arlin.r.davis@intel.com>
+Date: Wed Aug 20 18:22:42 2008 -0700
+
+ dtestx: fix stack corruption problem with hostname strcpy
+
+ Signed-off by: Arlin Davis ardavis@ichips.intel.com
+
+commit ee2a553762abd6ebede99db5d26d1ba1d74ba3cb
+Author: Arlin Davis <arlin.r.davis@intel.com>
+Date: Thu Aug 14 09:42:57 2008 -0700
+
+ dapl scm: use correct device attribute for max_rdma_read_out, max_qp_init_rd_atom
+
+ Signed-off by: Arlin Davis ardavis@ichips.intel.com
+
+commit 142a8e4a99259fa6fe31f74ce6d0dac1017f381b
+Author: Arlin Davis <arlin.r.davis@intel.com>
+Date: Thu Aug 14 09:19:53 2008 -0700
+
+ dapl scm: change IB RC qp inline and timer defaults.
+
+ rnr nak can be the result of any operation not just
+ message send recevier not ready. Timer is much too
+ large given this case.
+
+ Signed-off by: Arlin Davis ardavis@ichips.intel.com
+
+commit 36088a92d87e833bae535fcea0c45417dec34e65
+Author: Arlin Davis <arlin.r.davis@intel.com>
+Date: Thu Aug 14 09:12:38 2008 -0700
+
+ dapl scm: add mtu adjustments via environment, default = 1024.
+
+ DAPL_IB_MTU adjusts path mtu setting for RC qp's. Default setting
+ is min of 1024 and active mtu on IB device.
+
+ Signed-off by: Arlin Davis ardavis@ichips.intel.com
+
+commit a7dabdc4c1ddc62cc0384d60e8157ee829f12898
+Author: Arlin Davis <arlin.r.davis@intel.com>
+Date: Wed Aug 13 14:17:40 2008 -0700
+
+ dapl scm: change connect and accept to non-blocking to avoid blocking user thread.
+
+ The connect socket that is used to exchange QP information is now non-blocking
+ and the data exchange is done via the cr thread. New state RTU_PENDING added.
+ On the passive side there is a new state ACCEPT_DATA used to avoid read blocking
+ on the user accept call.
+
+ Signed-off by: Arlin Davis ardavis@ichips.intel.com
+
+commit 7e25c0f21d755cce3aa7aff993fb0baddaafc0e8
+Author: Arlin Davis <arlin.r.davis@intel.com>
+Date: Tue Jul 29 08:18:25 2008 -0700
+
+ dapl scm: update max_rdma_read_iov, max_rdma_write_iov EP attributes during query
+
+ Signed-off by: Arlin Davis ardavis@ichips.intel.com
+
+commit fd9909618fdfff0eb2c8ce97bff61ea98ec44a8e
+Author: Arlin Davis <arlin.r.davis@intel.com>
+Date: Sun Jul 20 13:20:45 2008 -0700
+
+ Release 2.0.11
+
+ Signed-off by: Arlin Davis ardavis@ichips.intel.com
+
+commit 0003bb7866af3ba73cc79c703b565a8012439bb1
+Author: Arlin Davis <arlin.r.davis@intel.com>
+Date: Sun Jul 20 13:17:22 2008 -0700
+
+ dtestx: add -d option to test new IB UD extension.
+
+ modify dtestx to connect peer UD QP's and exchange/verify messages as an example for new extension.
+
+ Signed-off by: Arlin Davis ardavis@ichips.intel.com
+
+commit 9816ac15f98f6e73cf7b371ac6e1e312d4171c0b
+Author: Arlin Davis <arlin.r.davis@intel.com>
+Date: Sun Jul 20 13:13:09 2008 -0700
+
+ dapl scm: add support for UD extensions in Openfabrics socket cm provider
+
+ add qp_type in connection information exchange
+ add new post_send_ud call
+ changes to connection manager to support qp types beyond RC.
+ changes to connection events to use new extended event calls.
+ exchange address handle information during connection phase.
+ changes to modify_qp to handle both RC and UD types.
+
+ Signed-off by: Arlin Davis ardavis@ichips.intel.com
+
+commit 927dac5b61e64868089acd49d468b98327e14a1a
+Author: Arlin Davis <arlin.r.davis@intel.com>
+Date: Sun Jul 20 13:07:34 2008 -0700
+
+ dapl: add support for UD extensions in common code.
+
+ allow EP create for extended service types.
+ extend connection event types to include UD AH resolution/exchange.
+ add new extended connect and connect request upcalls for providers.
+ - dapls_evd_post_cr_event_ext
+ - dapls_evd_post_connection_event_ext
+
+ Signed-off by: Arlin Davis ardavis@ichips.intel.com
+
+commit be3d6a53f3340294697706ce50a05faf151aacc7
+Author: Arlin Davis <arlin.r.davis@intel.com>
+Date: Sun Jul 20 12:57:49 2008 -0700
+
+ dat: New definitions for IB unreliable datagram extension
+
+ Extend EP dat_service_type, with DAT_IB_SERVICE_TYPE_UD
+ Add IB extension call dat_ib_post_send_ud().
+ Add address handle definition for UD calls.
+ Add IB event definitions to provide remote AH via connect and connect requests
+ Roll IB extension version to 2.0.2
+
+ Signed-off by: Arlin Davis ardavis@ichips.intel.com
+
+commit 8f65d4c96a7e13f7092d82f2f5aad477a61a57c1
+Author: Arlin Davis <arlin.r.davis@intel.com>
+Date: Fri Jul 18 07:46:20 2008 -0700
+
+ dat: allow TYPE_ERR messages to be turned off with DAT_DBG_TYPE
+
+ Signed-off by: Arlin Davis ardavis@ichips.intel.com
+
+commit 4fefc6bca4e21c9d757923bf13bc93c62dbff17d
+Author: Arlin Davis <arlin.r.davis@intel.com>
+Date: Wed Jun 25 08:02:11 2008 -0700
+
+ dapltest: manpage - rdma write example incorrect
+
+ parameter for rdma write should be RW and not WR
+
+ Signed-off by: Arlin Davis ardavis@ichips.intel.com
+
+commit 68638bde71b529a142c13ac332cd44435cabc896
+Author: Arlin Davis <arlin.r.davis@intel.com>
+Date: Mon Jun 23 15:26:30 2008 -0700
+
+ dapl: remove needless terminating 0 in dto_op_str functions.
+
+ Signed-off by: Arlin Davis ardavis@ichips.intel.com
+
+commit cb1f2a144ecd375d2143d6e176a6a92a18556d7e
+Author: Arlin Davis <arlin.r.davis@intel.com>
+Date: Mon Jun 23 10:52:46 2008 -0700
+
+ Release 2.0.10
+
+ Signed-off by: Arlin Davis ardavis@ichips.intel.com
+
+commit d9b3c06ae98ac4d3b264384f6510137166d78cb0
+Author: Arlin Davis <arlin.r.davis@intel.com>
+Date: Mon Jun 23 10:35:17 2008 -0700
+
+ remove reference to doc/dat.conf in makefile.am
+
+ Signed-off by: Arlin Davis ardavis@ichips.intel.com
+
+commit b052d402e09ac78281a25af2c8fe902fa71f5c6f
+Author: Arlin Davis <arlin.r.davis@intel.com>
+Date: Thu Jun 19 13:34:49 2008 -0700
+
+ dapl scm: fix ibv_destroy_cq busy error condition during dat_evd_free.
+
+ Problem surfaced while running Intel MPI 3.1 and mpich2-test suite.
+ dapli_destroy_cm was incorrectly removing reference to qp_handle in endpoint
+ and destroy_cq and destroy_pd code was ignoring verbs errors.
+
+ Signed-off by: Arlin Davis ardavis@ichips.intel.com
+
+commit e466d8e330c45176c5f00efda79ad745bf3f71a4
+Author: Arlin Davis <arlin.r.davis@intel.com>
+Date: Wed Jun 18 14:21:28 2008 -0700
+
+ dapl scm: add stdout logging for uname and gethostbyname errors during open.
+
+ Signed-off by: Arlin Davis ardavis@ichips.intel.com
+
+commit 8e1f4db702cacdd2b9b95083db54ec38c9d2f5e5
+Author: Arlin Davis <arlin.r.davis@intel.com>
+Date: Wed Jun 18 14:19:51 2008 -0700
+
+ dapl scm: support global routing and set mtu based on active_mtu
+
+ Signed-off by: Arlin Davis ardavis@ichips.intel.com
+
+commit fc65a08727c59c304dad20337a8bff803f2302c0
+Author: Arlin Davis <arlin.r.davis@intel.com>
+Date: Wed Jun 18 13:59:44 2008 -0700
+
+ dapl scm: Adding socket cm provider for better scalability on large homogeneous clusters.
+
+ Bring socket cm provider back to life with some changes:
+
+ better threading support for exchanging QP information.
+ Avoid blocking during connect to support dynamic connection
+ model with MPI implementations.
+
+ consumer control of ack timeout/retries.
+
+ disconnect/reject capabilities via socket exchange.
+
+ version support for wire protocol to insure compatibility
+ with peer scm provider. Add gids to exchange.
+
+ validated with Intel MPI on a 14,000+ core fabric using IB DDR.
+
+ Signed-off by: Arlin Davis ardavis@ichips.intel.com
+
+commit 0855af175fec2f1bec8391ebae2a2cdff26a3359
+Author: Arlin Davis <arlin.r.davis@intel.com>
+Date: Wed Jun 11 10:43:24 2008 -0700
+
+ dapl: add opcode to string function to report opcode during failures.
+
+ Need to use cookie opcode during failures in lieu of cqe opcode.
+
+ Signed-off by: Arlin Davis ardavis@ichips.intel.com
+
+commit ec6296e7f0a843c69231f8284ae780014fa26fbe
+Author: Arlin Davis <arlin.r.davis@intel.com>
+Date: Mon Jun 16 14:59:43 2008 -0700
+
+ dapl: remove unused iov buffer allocation on the endpoint
+
+ provider's manage iov space on stack during posting.
+
+ Signed-off by: Arlin Davis ardavis@ichips.intel.com
+
+commit c678414ced8bc9cbe21b2fc1d54aa2af73ba6528
+Author: Arlin Davis <arlin.r.davis@intel.com>
+Date: Mon Jun 16 13:59:11 2008 -0700
+
+ dapl: endpoint pending request count is wrong
+
+ The code assumes every cookie allocated during posting of
+ requests gets completed. This incorrect assumption results in
+ wrong pending count. Remove request_pending field and replace
+ with direct call, dapl_cb_pending, to provide accurate
+ data to consumer.
+
+ Add debug print if consumer overruns request queue.
+
+ Signed-off by: Arlin Davis ardavis@ichips.intel.com
+
+commit c446a3a3f3ecbd91ab583fee35bf613ab10defcd
+Author: Arlin Davis <arlin.r.davis@intel.com>
+Date: Mon Jun 2 12:40:45 2008 -0700
+
+ dapl extension: dapli_post_ext should always allocate cookie for requests.
+
+ extension didn't allocate cookie if completion was suppressed which resulted
+ segfault during provider post call. Provider's expect cookie for wr_id,
+ even with surpressed completions, to handle events during errors.
+
+ Signed-off by: Arlin Davis ardavis@ichips.intel.com
+
+commit 2583f07d9d0f55eee14e0b0e6074bc6fd0712177
+Author: Arlin Davis <arlin.r.davis@intel.com>
+Date: Tue May 20 14:35:43 2008 -0700
+
+ Release 2.0.9
+
+ Signed-off by: Arlin Davis ardavis@ichips.intel.com
+
+commit 9b38e7b792c48ea63c2078fc6c10bb843d96bd06
+Author: Arlin Davis <arlin.r.davis@intel.com>
+Date: Tue May 20 21:56:06 2008 -0700
+
+ dtest,dtestx,dapltest: fix build issues with Redhat EL5.1
+
+ need include files/definitions for sleep, getpid, gettimeofday
+
+ Signed-off by: Arlin Davis ardavis@ichips.intel.com
+
+commit 8084ebf39729bac310447467b518df4248e9e2b6
+Author: Arlin Davis <arlin.r.davis@intel.com>
+Date: Tue May 20 14:31:09 2008 -0700
+
+ dapl: Fix long delays with the cma provider open call when DNS is not configure on server.
+
+ Open call should default to netdev names when resolving local IP address for cma binding to match dat.conf settings. The open code attempts to resolve with IP or Hostname first and if there is no DNS services setup the failover to netdev name resolution is delayed for as much as 20 seconds.
+
+ Signed-off by: Arlin Davis ardavis@ichips.intel.com
+
+commit 2f603b03f4cebe7c414cbaeecb7155f7bf1fb115
+Author: Arlin Davis <arlin.r.davis@intel.com>
+Date: Tue May 20 14:30:05 2008 -0700
+
+ dapl: change cma provider to use max_rdma_read_in,out from ep_attr instead of HCA max values when connecting.
+
+ Signed-off by: Arlin Davis ardavis@ichips.intel.com
+
+commit 7a0e5fad6de7c6affc4ba3c608b0c56f4206c48c
+Author: Arlin Davis <arlin.r.davis@intel.com>
+Date: Wed Apr 30 14:48:21 2008 -0700
+
+ Release 2.0.8
+
+ Signed-off by: Arlin Davis ardavis@ichips.intel.com
+
+commit 561f09ab6f9fd46a6075ecb54ca7693f2b12f311
+Author: Arlin Davis <arlin.r.davis@intel.com>
+Date: Wed Apr 30 13:51:41 2008 -0700
+
+ dapl: fix post_send, post_recv to handle 0 byte's and NULL iov handles
+
+ Signed-off by: Arlin Davis ardavis@ichips.intel.com
+
+commit 0289daa3fcc4451bace8cc6b6e20ddb7bbade07e
+Author: Arlin Davis <arlin.r.davis@intel.com>
+Date: Wed Apr 30 13:25:53 2008 -0700
+
+ dat: udat_sr_parser ia_name will fail on comments, turn down debug message
+
+ Signed-off by: Arlin Davis ardavis@ichips.intel.com
+
+commit b93b91d48186d100c48f1a479776d56476847607
+Author: Arlin Davis <arlin.r.davis@intel.com>
+Date: Tue Apr 29 16:15:44 2008 -0700
+
+ dat: cleanup error handling with static registry parsing of dat.conf
+
+ change asserts to return codes, add log messages, and
+ report errors via open instead of asserts during dat
+ library load.
+
+ Signed-off by: Arlin Davis ardavis@ichips.intel.com
+
+commit b6b1f152efabe064ab8bdcdeecbd1edd30eb732f
+Author: Arlin Davis <arlin.r.davis@intel.com>
+Date: Mon Apr 28 10:14:20 2008 -0700
+
+ dapl: cma provider needs to support lower inline send default for iWARP
+
+ IB and iWARP work best with different defaults. Add transport check
+ and set default accordingly. 64 for iWARP, 200 for IB.
+
+ DAPL_MAX_INLINE environment variable is still used to override.
+
+ Signed-off by: Arlin Davis ardavis@ichips.intel.com
+
+commit be32d7d5beeeceac5dbb1974d3217265dc4d5461
+Author: Arlin Davis <arlin.r.davis@intel.com>
+Date: Mon Apr 28 09:44:12 2008 -0700
+
+ dtestx: need to include string.h for memset/strcpy declarations
+
+ Signed-off by: Arlin Davis ardavis@ichips.intel.com
+
+commit a192465714e7607529303a80d8f9a312e0c7aec6
+Author: Arlin Davis <arlin.r.davis@intel.com>
+Date: Mon Apr 28 08:41:05 2008 -0700
+
+ dapl: add vendor_err with DTO error logging
+
+ DAPL_GET_CQE_VENDOR_ERR added to get vendor_err via cq entry.
+
+ Signed-off by: Arlin Davis ardavis@ichips.intel.com
+
+commit 6ac657a4e7e5e27254a024fca7fdead569043f9a
+Author: Arlin Davis <arlin.r.davis@intel.com>
+Date: Fri Apr 25 15:12:34 2008 -0700
+
+ dapl: add check before destroying cm event channel in release
+
+ library may be loaded and unloaded without calling open
+ in which case the cm event channel is not created.
+
+ Signed-off by: Arlin Davis ardavis@ichips.intel.com
+
+commit c1eb23352f44aec6faeef37d8f0c3180f6259cf8
+Author: Arlin Davis <arlin.r.davis@intel.com>
+Date: Tue Apr 22 12:55:13 2008 -0700
+
+ dapl: evd_alloc doesn't check for direct_object_create errors.
+
+ Fix error check in dapls_ib_wait_object_create() and dat_evd_alloc.
+ When attempting to create large number of evd's that exceed
+ open files limit the error was not propagated up causing
+ a segfault. Note: there are 3 FD's required for each EVD
+ 2 for pipe, and one for cq event_channel.
+
+ Change the error reporting to indicate correct return
+ code and send to log with non-debug builds.
+
+ Signed-off by: Arlin Davis ardavis@ichips.intel.com
+
+commit c5c6229b3e7744782cace0ca1f5f1fa89198bd3a
+Author: Arlin Davis <arlin.r.davis@intel.com>
+Date: Mon Apr 14 13:10:13 2008 -0700
+
+ dapl: change packaging to modify OFA provider contents of dat.conf instead of file replacement.
+
+ Change the packaging to update only the OFA provider contents in dat.conf. This allows other
+ dapl providers, other then OFA, to co-exist and configure properly. Adding manpage to explain
+ syntax of this static configuration file.
+
+ Signed-off by: Arlin Davis ardavis@ichips.intel.com
+
+commit d32b27d991bc1314eea055ce3f55bb585b11aaac
+Author: Arlin Davis <arlin.r.davis@intel.com>
+Date: Fri Apr 11 11:37:48 2008 -0700
+
+ dapl openib_cma: fix hca query to use correct max_rd_atom values
+
+ Signed-off by: Arlin Davis ardavis@ichips.intel.com
+
+commit ac4ccfd144c01b7f4285b0cecc6218c58d0482a8
+Author: Arlin Davis <arlin.r.davis@intel.com>
+Date: Wed Apr 9 17:26:06 2008 -0700
+
+ dat: add logging by default during library load failures.
+
+ Signed-off by: Arlin Davis ardavis@ichips.intel.com
+
+commit 1794e94754a1e58fcf214c2802e950124bbd1316
+Author: Arlin Davis <arlin.r.davis@intel.com>
+Date: Tue Apr 8 17:32:03 2008 -0700
+
+ dtest: add private data validation with connect and accept.
+
+ Include code, with build option, to validate private data with
+ consumer rejects.
+
+ Signed-off by: Arlin Davis ardavis@ichips.intel.com
+
+commit 46f21284bc05e76e82b9ad6cd9f1dbc9bcde0a28
+Author: Arlin Davis <arlin.r.davis@intel.com>
+Date: Tue Apr 8 17:25:11 2008 -0700
+
+ dapl: add hooks in evd connection callback code to deliver private data with consumer reject.
+
+ PEER rejects can include private data. The common code didn't support delivery
+ via the connect event data structure. Add the necessary hooks in
+ dapl_evd_connection_callback function and include checks in openib_cma
+ provider to check and delivery properly. Also, fix the private data size
+ check in dapls_ib_reject_connection function.
+
+ Signed-off by: Arlin Davis ardavis@ichips.intel.com
+
+commit 90e04c6f76dd5cfebd2f2867bfe22e85b0c2f461
+Author: Arlin Davis <arlin.r.davis@intel.com>
+Date: Mon Apr 7 15:47:57 2008 -0700
+
+ dapl: increase reject private data size to avoid odd byte offets.
+
+ remove reject type checking on passive side since it will
+ always be non-consumer from active side.
+
+ Signed-off by: Arlin Davis ardavis@ichips.intel.com
+
+commit 353a1c8a00bb2a1380fd7a372973a5a70828da35
+Author: Arlin Davis <arlin.r.davis@intel.com>
+Date: Fri Apr 4 16:04:11 2008 -0800
+
+ dapl: update vendor information for OFA v2 provider.
+
+ Signed-off by: Arlin Davis ardavis@ichips.intel.com
+
+commit dbf1ea37f43caec61911dea06af801c2f906db0a
+Author: Arlin Davis <arlin.r.davis@intel.com>
+Date: Fri Apr 4 16:03:03 2008 -0800
+
+ dapl: add provider vendor revision data in private data with reject
+
+ Add 1 byte header containing provider/vendor major revision
+ to distinguish between consumer and non-consumer rejects.
+ Validate size of consumer reject privated data.
+
+ Signed-off by: Arlin Davis ardavis@ichips.intel.com
+
+commit 0f71b9be594739a1fba7d74929eacd42a8cee392
+Author: Arlin Davis <arlin.r.davis@intel.com>
+Date: Thu Apr 3 17:06:27 2008 -0800
+
+ dapl: add support for logging errors in non-debug build.
+
+ Add debug logging (stdout, syslog) for error cases during
+ device open, cm, async, and dto operations. Default settings
+ are ERR for DAPL_DBG_TYPE, and stdout for DAPL_DBG_DEST.
+
+ Change default configuration to build non-debug.
+
+ Signed-off by: Arlin Davis ardavis@ichips.intel.com
+
+commit bea882ad9b11ac7188628a939f5227e22c914169
+Author: Arlin Davis <arlin.r.davis@intel.com>
+Date: Thu Apr 3 16:23:29 2008 -0800
+
+ dapl: add support for private data in CR reject.
+
+ Private data support via dat_cr_reject was added to
+ the v2 DAT specification but dapl was never extended
+ to support at the provider level.
+
+ Signed-off by: Arlin Davis ardavis@ichips.intel.com
+
+commit b5b096b6b6949fcb9fa711ab4784f41f1bb87525
+Author: Arlin Davis <arlin.r.davis@intel.com>
+Date: Tue Apr 1 11:02:37 2008 -0800
+
+ dapl: calculate private data size based on transport type and cma_hdr overhead
+
+ Need to adjust CM private date size based on different transport types.
+ Add hca_ptr to dapls_ib_private_data_size call for transport type
+ validation via verbs device. Add definitions to include iWARP size
+ of 512 and subtract 36 bytes for cma_hdr overhead.
+
+ Signed-off by: Arlin Davis ardavis@ichips.intel.com
+
+commit 632af34f0ea15e12b572c656fe2ef561a0ad62b7
+Author: Arlin Davis <arlin.r.davis@intel.com>
+Date: Wed Mar 26 17:58:17 2008 -0800
+
+ Remove improperly licensed GETOPT code from dtest source tree.
+
+ Signed-off by: Arlin Davis ardavis@ichips.intel.com
+
+commit 88069fdd21f92923388dec7adbde0d1bc334e7c4
+Author: Arlin Davis <arlin.r.davis@intel.com>
+Date: Tue Mar 18 15:04:34 2008 -0800
+
+ remove unnecessary assert from dapl_ep_free.
+
+ dat_ep_free must handle the case where a consumer calls
+ free in CONNECTED or DISCONNECT_PENDING states. After
+ free calls disconnect, there may be a pending event,
+ in which case the providers dapls_ib_qp_free will block
+ accordingly and handle pending events.
+
+ Signed-off by: Arlin Davis ardavis@ichips.intel.com
+
+commit b3f020e5cba765077049a8cf61c4ce5009fa1031
+Author: Patrick Marchand Latifi <patrick.latifi@qlogic.com>
+Date: Fri Mar 14 14:31:40 2008 -0700
+
+ fix openib_scm compiler warning
+
+ Cast to socklen_t since accept(2) expects an unsigned argument.
+
+ Makes the openib_scm provider now build successfully when using
+ make VERBS=<provider> (the -Werror flag was causing the build
+ failure)
+
+ Signed-off-by: Patrick Marchand Latifi <patrick.latifi@qlogic.com>
+
+commit 441996f05d3cc8b09c94c166ef736bc50c24de7e
+Author: Patrick Marchand Latifi <patrick.latifi@qlogic.com>
+Date: Fri Mar 14 14:31:34 2008 -0700
+
+ fix provider-specific compiler warnings
+
+ Initialize ds_array_start_p otherwise the compiler would claim
+ that this variable could be used with an uninitialized value.
+
+ Makes the uDAPL providers now build successfully when using make
+ VERBS=<provider> (the -Werror flag was causing the build failure)
+
+ Signed-off-by: Patrick Marchand Latifi <patrick.latifi@qlogic.com>
+
+commit 5a710fc43ad004ecb4603db1359abb4a4fcd77e3
+Author: Arlin Davis <arlin.r.davis@intel.com>
+Date: Tue Mar 11 09:25:07 2008 -0800
+
+ uDAPL: fix query to adjust max sge for default inline of 64 bytes
+ and include missing max_rdma_write_iov ep_attr field
+
+ Signed-off by: Arlin Davis ardavis@ichips.intel.com
+
+commit e2c6bf57f78dfebc21e168df01e5876202053e08
+Author: Patrick Marchand Latifi <patrick.latifi@qlogic.com>
+Date: Fri Mar 7 09:39:22 2008 -0800
+
+ uDAT: fix reuse of va_list in debugging mode
+
+ Make sure we reinitialize the va_list since va_list is undefined
+ if a function traverses the va_list with va_arg.
+
+ This patch fixes the uDAT debugging case when both stdout and
+ syslog output is wanted.
+
+ Signed-off-by: Patrick Marchand Latifi <patrick.latifi@qlogic.com>
+
+commit 4c9cd7a7268c0d8afb5b4d9b31537bc50cac18fe
+Author: Arlin Davis <arlin.r.davis@intel.com>
+Date: Thu Mar 6 15:40:35 2008 -0800
+
+ Add hostname and process id to debug output to aid
+ scale-up and out debug.
+
+ Signed-off by: Arlin Davis ardavis@ichips.intel.com
+
+commit 460aa6f089fcdb34d78af1c8391cbecbcc6b0db3
+Author: Patrick Marchand Latifi <patrick.latifi@qlogic.com>
+Date: Sat Feb 23 21:03:21 2008 -0800
+
+ fix reuse of va_list in debugging mode
+
+ Make sure we reinitialize the va_list since va_list is undefined
+ if a function traverses the va_list with va_arg.
+
+ This patch fixes the debugging case when both stdout and syslog
+ output is wanted.
+
+ Signed-off-by: Patrick Marchand Latifi <patrick.latifi@qlogic.com>
+
+commit da80af8c76f220508407a2e171c8b28b43b35bf9
+Author: Patrick Marchand Latifi <patrick.latifi@qlogic.com>
+Date: Tue Feb 19 16:54:45 2008 -0800
+
+ Fix memory leak in provider specific post send/recv if there's
+ more than DEFAULT_DS_ENTRIES iovecs.
+
+ Signed-off-by: Patrick Marchand Latifi <patrick.latifi@qlogic.com>
+
+commit d78ee9f95fbe48f71eb247b01c598994edfa4d17
+Author: Patrick Marchand Latifi <patrick.latifi@qlogic.com>
+Date: Tue Feb 19 03:19:42 2008 -0800
+
+ Guarantee NUL termination if hostname gets truncated.
+
+ Signed-off-by: Patrick Marchand Latifi <patrick.latifi@qlogic.com>
+
+commit 3484fa0d2b1a5549d83ecc82d89b72bb86f8fe4e
+Author: Patrick Marchand Latifi <patrick.latifi@qlogic.com>
+Date: Tue Feb 19 03:19:37 2008 -0800
+
+ Make sure we don't leak the hash table if dapl_hca_alloc fails.
+
+ Signed-off-by: Patrick Marchand Latifi <patrick.latifi@qlogic.com>
+
+commit e4ed56be6bd64684564169d89df7ca30faffdb53
+Author: Patrick Marchand Latifi <patrick.latifi@qlogic.com>
+Date: Tue Feb 19 03:19:32 2008 -0800
+
+ Fix memory leak.
+
+ Signed-off-by: Patrick Marchand Latifi <patrick.latifi@qlogic.com>
+
+commit 9efd005b185cfd970d2f8c58f05cf2eaae0dcdc8
+Author: Patrick Marchand Latifi <patrick.latifi@qlogic.com>
+Date: Tue Feb 19 03:19:27 2008 -0800
+
+ Fix memory leak in error path.
+
+ Signed-off-by: Patrick Marchand Latifi <patrick.latifi@qlogic.com>
+
+commit d971e799bb5385a8c847cf3f863f19854e95c1b2
+Author: Patrick Marchand Latifi <patrick.latifi@qlogic.com>
+Date: Tue Feb 19 03:19:22 2008 -0800
+
+ Fix memory leak
+
+ Signed-off-by: Patrick Marchand Latifi <patrick.latifi@qlogic.com>
+
+commit c21f2f455af7934675a58ff825bed6cf54d457c9
+Author: Patrick Marchand Latifi <patrick.latifi@qlogic.com>
+Date: Tue Feb 19 03:19:17 2008 -0800
+
+ Fix memory leak.
+
+ Signed-off-by: Patrick Marchand Latifi <patrick.latifi@qlogic.com>
+
+commit ee46aa47e2a4deeded347fe18bd6321db61c6594
+Author: Arlin Davis <arlin.r.davis@intel.com>
+Date: Fri Feb 15 10:10:01 2008 -0800
+
+ Release 2.0.7
+
+ Signed-off by: Arlin Davis <ardavis@ichips.intel.com>
+
+commit f0ca504bc2639cb7a48528d45e9026b54dab3e57
+Author: Arlin Davis <arlin.r.davis@intel.com>
+Date: Thu Feb 14 09:46:56 2008 -0800
+
+ uDAT: Make sure we initialize the dictionary entry early enough so that
+ we can base our cleanup decisions on that variable being
+ initialized.
+
+ Signed-off-by: Patrick Marchand Latifi <patrick.latifi@qlogic.com>
+
+commit 6b11838043a6012668c7e1a22a869e9e84dc40d6
+Author: Arlin Davis <arlin.r.davis@intel.com>
+Date: Thu Feb 14 09:46:18 2008 -0800
+
+ uDAT: Make sure we stay within bounds when manipulating the ia_name.
+
+ Signed-off-by: Patrick Marchand Latifi <patrick.latifi@qlogic.com>
+ Signed-off by: Arlin Davis <ardavis@ichips.intel.com>
+
+commit 3eab70b4d34f850661b51dc1b856cd8d672a5cc9
+Author: Arlin Davis <arlin.r.davis@intel.com>
+Date: Thu Feb 14 09:44:56 2008 -0800
+
+ uDAT: Make sure we stay within bounds when manipulating the ia handle.
+ Fix typo in comment.
+
+ Signed-off-by: Patrick Marchand Latifi <patrick.latifi@qlogic.com>
+
+commit 7fe64f10874667062e067efb7ea8c3d385ae90b7
+Author: Arlin Davis <arlin.r.davis@intel.com>
+Date: Thu Feb 14 09:43:39 2008 -0800
+
+ uDAT: Zero-out memory otherwise we might base our cleanup decisions on
+ uninitialized memory.
+
+ Signed-off-by: Patrick Marchand Latifi <patrick.latifi@qlogic.com>
+
+commit f510e7e8f5b72a6262a7f8b255926cf96c65b654
+Author: Arlin Davis <arlin.r.davis@intel.com>
+Date: Wed Feb 13 20:39:26 2008 -0800
+
+ Modify default configure options to match rpm spec file
+ --enable-debug, --enable-ext-type=ib
+
+ Signed-off by: Arlin Davis <ardavis@ichips.intel.com>
+
+commit 75273f1a5c599777bb43add93f30563689fdbc10
+Author: Arlin Davis <arlin.r.davis@intel.com>
+Date: Mon Feb 11 15:43:03 2008 -0800
+
+ udapl OFA provider: set listen backlog to default cma backlog max.
+
+ Signed-off by: Arlin Davis <ardavis@ichips.intel.com>
+
+commit 7f173ecd7e18f25bc21a42651603922e7d71f7c7
+Author: Arlin Davis <arlin.r.davis@intel.com>
+Date: Mon Feb 11 14:50:33 2008 -0800
+
+ The OFA dapl provider is checking for incorrect return code
+ from rdma_bind_addr and rdma_listen calls. This causes an error
+ to be returned back to the consumer instead of correctly
+ incrementing the seed port and re-calling the OFA provider
+ until a valid port is issued. The seed value (1000) is also
+ incorrect and should start a non-privledged port (1024) to
+ avoid EPERM errors when seeding the starting port value.
+
+ Signed-off by: Arlin Davis <ardavis@ichips.intel.com>
+
+commit 7bcb67ba7c9b37d7c122773f542c7f7f718d4a49
+Author: Arlin Davis <arlin.r.davis@intel.com>
+Date: Mon Feb 4 16:16:10 2008 -0800
+
+ Release 2.0.6 - ChangeLog
+
+commit bead5f36542a8a4c790bda8ecc8dde3e630c15e6
+Author: Arlin Davis <arlin.r.davis@intel.com>
+Date: Mon Feb 4 16:14:02 2008 -0800
+
+ Release 2.0.6
+
+ Signed-off by: Arlin Davis <ardavis@ichips.intel.com>
+
+commit 8c5beb870048aca286f7396549771ccb075c5c1b
+Author: Arlin Davis <arlin.r.davis@intel.com>
+Date: Mon Feb 4 16:12:47 2008 -0800
+
+ Fix OFED v2 package to build against and target /dat2/include directory.
+
+ Prevous patch missed dat_osd.h, dapltest/dtest incorrect.
+
+ Signed-off by: Arlin Davis <ardavis@ichips.intel.com>
+
+commit 42a63bb0271f91541e7b3c3967a9a977ef6660ae
+Author: Arlin Davis <arlin.r.davis@intel.com>
+Date: Mon Feb 4 13:00:45 2008 -0800
+
+ uDAT/DAPL: Fix package to build against and target /dat2/include directory.
+
+ Signed-off by: Arlin Davis <ardavis@ichips.intel.com>
+
+commit b3294c738d61f44ae4d0888662bdd6b64f6d6b1f
+Author: Arlin Davis <arlin.r.davis@intel.com>
+Date: Tue Jan 29 16:34:49 2008 -0800
+
+ Release 2.0.5 - ChangeLog updated
+
+ Signed-off by: Arlin Davis <ardavis@ichips.intel.com>
+
+commit 356cf91905d39ade06d76ab9ace6203cd7907d93
+Author: Arlin Davis <arlin.r.davis@intel.com>
+Date: Tue Jan 29 16:33:10 2008 -0800
+
+ libdat: rename libdat.so to libdat2.so to coexist with v1 devel
+
+ Signed-off by: Arlin Davis <ardavis@ichips.intel.com>
+
+commit d9a9f46ee220ec9c479756acc306ed68060a662f
+Author: Arlin Davis <arlin.r.davis@intel.com>
+Date: Mon Jan 28 13:55:35 2008 -0800
+
+ Release 2.0.4 - README
+
+ Signed-off by: Arlin Davis <ardavis@ichips.intel.com>
+
+commit 533983cec914a7ecc6829934a56f867d7870e301
+Author: Arlin Davis <arlin.r.davis@intel.com>
+Date: Mon Jan 28 12:06:03 2008 -0800
+
+ Release 2.0.4 Changelog
+
+ Signed-off by: Arlin Davis <ardavis@ichips.intel.com>
+
+commit bfc5bd4b4190c7302b08c3922c17e2131bfbe605
+Author: Arlin Davis <arlin.r.davis@intel.com>
+Date: Mon Jan 28 12:04:18 2008 -0800
+
+ Relase package 2.0.4
+
+commit 95935648f81ac5177ae7120b35e55483902b8c64
+Author: Arlin Davis <arlin.r.davis@intel.com>
+Date: Wed Jan 23 16:40:48 2008 -0800
+
+ dapltest does not include definitions for inet_ntoa.
+ At load time the symbol was resolved but with the
+ default definition of int, instead of char*, it caused
+ segfault. Add correct include files in dapl_mdep_user.h
+ for linux.
+
+ Signed-off by: Arlin Davis <ardavis@ichips.intel.com>
+
+commit 3c1bb0e56c14c5ad50876820e25933c1f4c3dde4
+Author: Arlin Davis <arlin.r.davis@intel.com>
+Date: Wed Jan 23 14:46:30 2008 -0800
+
+ Add freeaddrinfo to deallocate getaddrinfo buffer.
+ Cleanup a few printf messages.
+
+ Signed-off by: Arlin Davis <ardavis@ichips.intel.com>
+
+commit a4a8ad5eb1b15be19b3a343dee71ad3013d3f4bd
+Author: Arlin Davis <arlin.r.davis@intel.com>
+Date: Wed Jan 23 14:25:21 2008 -0800
+
+ Fix for uDAPL v2 using extended operation. After extension completion,
+ the DTO cookie must be checked for type before deallocating to handle
+ inbound immediate data in receive. The sample dtestx client will fail
+ when running loopback if the rdma immediate is received from remote
+ before the rdma immediate request completion fires.
+
+ Bug causes following dtestx client error:
+ dat_ib_post_rdma_write_immed
+ returned DAT_INSUFFICIENT_RESOURCES : DAT_RESOURCE_MEMORY
+
+ Signed-off by: Arlin Davis <ardavis@ichips.intel.com>
+
+commit 6dcf1763c153c27c29ba76bac35be4f6935ddd96
+Author: Arlin Davis <arlin.r.davis@intel.com>
+Date: Thu Jan 17 11:30:27 2008 -0800
+
+ WinOF: update dapltest,dtest directories for windows build
+ WinOF: add README.windows
+
+ Signed-off by: Stan Smith <stan.smith@intel.com>
+ Signed-off by: Arlin Davis <ardavis@ichips.intel.com>
+
+commit 9bececbb32dd31d4a3528e8f000a773e5c593430
+Author: Arlin Davis <arlin.r.davis@intel.com>
+Date: Thu Jan 17 11:24:46 2008 -0800
+
+ WinOF: update dtestx for windows build
+
+ Signed-off by: Stan Smith <stan.smith@intel.com>
+ Signed-off by: Arlin Davis <ardavis@ichips.intel.com>
+
+commit 4dc8c5defef0f52da751a1eca4d4f35de911c3c0
+Author: Arlin Davis <arlin.r.davis@intel.com>
+Date: Thu Jan 17 11:23:21 2008 -0800
+
+ WinOF: add build enviroment, cleanup old makefiles
+
+ Signed-off by: Stan Smith <stan.smith@intel.com>
+ Signed-off by: Arlin Davis <ardavis@ichips.intel.com>
+
+commit 5a9e035fa33a6307b3d2b370f64639b14dfca87e
+Author: Arlin Davis <arlin.r.davis@intel.com>
+Date: Tue Jan 15 16:59:34 2008 -0800
+
+ WinOF: name collision with ibal verbs ib_cm_handle_t
+
+ Signed-off by: Stan Smith <stan.smith@intel.com>
+ Signed-off by: Arlin Davis <ardavis@ichips.intel.com>
+
+commit 7c5418a781c1dfdb37b09450b5df40363dce84b4
+Author: Arlin Davis <arlin.r.davis@intel.com>
+Date: Tue Jan 15 16:06:10 2008 -0800
+
+ dtest: dat_evd_query for correct size
+
+ Signed-off by: Arlin Davis <ardavis@ichips.intel.com>
+
+commit 6de5e635c01e78bde6e153b727926da900677d52
+Author: Arlin Davis <arlin.r.davis@intel.com>
+Date: Tue Jan 15 16:03:27 2008 -0800
+
+ dapltest: WinOF support.
+
+ Signed-off by: Stan Smith <stan.smith@intel.com>
+
+commit c680e95b96e57bd8b69b1c73e95730854fcea028
+Author: Arlin Davis <arlin.r.davis@intel.com>
+Date: Tue Jan 15 14:54:57 2008 -0800
+
+ openib_cma: fix cleanup issues in destroy_cm_id
+ add macros to convert SID and PORT
+ fix init/responder settings in accept
+
+ Signed-off by: Arlin Davis <ardavis@ichips.intel.com>
+
+commit f8f867e52ffc36a31830dd4003eb2a4b4d265c93
+Author: Arlin Davis <arlin.r.davis@intel.com>
+Date: Tue Jan 15 14:49:54 2008 -0800
+
+ Modifications for WinOF build and endian support for RMR info
+
+ Signed-off by: Stan Smith <stan.smith@intel.com>
+ Signed-off by: Arlin Davis <ardavis@ichips.intel.com>
+
+commit 4c1fc48d2825efcffaa8cdf7efbdd2d41aeb56ab
+Author: Arlin Davis <arlin.r.davis@intel.com>
+Date: Tue Jan 15 14:39:18 2008 -0800
+
+ WinOF: DAT_API changes, white space and tab cleanup
+ IBAL provider code added
+
+ Common code: initialize cookie pool buffer
+ add dapl extension DTO counter
+ add get_cqe_op_str debug call
+ remove excess dbg in evd_dequeue
+ add dbg assert to sp_search_cr
+ IBAL provider support _VENDOR_IBAL_
+
+ Signed-off by: Stan Smith <stan.smith@intel.com>
+ Signed-off by: Arlin Davis <ardavis@ichips.intel.com>
+
+commit 68f2ad0af3623dec27b1223aeaca6357348eef4b
+Author: Arlin Davis <arlin.r.davis@intel.com>
+Date: Fri Jan 4 08:32:39 2008 -0800
+
+ Windows specific -
+ IBAL support in evd_create
+ Build IB extensions by default
+ Common code -
+ check return status, evd_free, evd_wait
+ add dapl_event_str function
+ definitions for dat_os_library_error, dat_os_ungetc
+
+ Signed-off by: Stan Smith <stan.smith@intel.com>
+ Signed-off by: Arlin Davis <ardavis@ichips.intel.com>
+
+commit d21f36406408245ac979f0b9594e7d1d0b9a3852
+Author: Arlin Davis <arlin.r.davis@intel.com>
+Date: Fri Jan 4 08:32:19 2008 -0800
+
+ Common code -
+ Missing DAT_API defs
+ casting to fix build issues
+ bitmaps for extension debug
+ DAPL_BAD_PTR macro
+
+ Signed-off by: Stan Smith <stan.smith@intel.com>
+ Signed-off by: Arlin Davis <ardavis@ichips.intel.com>
+
+commit 258678a381c6a0170020c48b0ba627e820abd3e7
+Author: Arlin Davis <arlin.r.davis@intel.com>
+Date: Fri Jan 4 08:31:59 2008 -0800
+
+ Common code -
+ Add DAT_API definitions for dat_redirection.h, udat_redirection.h
+
+ Signed-off by: Stan Smith <stan.smith@intel.com>
+ Signed-off by: Arlin Davis <ardavis@ichips.intel.com>
+
+commit 87d0b46a0c0f25e8828d7425e0173a0304a23f8b
+Author: Arlin Davis <arlin.r.davis@intel.com>
+Date: Fri Jan 4 08:31:27 2008 -0800
+
+ Windows specific -
+ Add dapl_ep fields ibal_cm_handle, recv_disc, sent_disc for IBAL provider
+ Support for direct object on CQ
+ INIT and FINI changes
+ setup dat.conf default path, fix sr parsing
+ Common code -
+ Add Stan Smith as contributor
+ O/S independent dat_os_library_error()
+
+ Signed-off by: Stan Smith <stan.smith@intel.com>
+ Signed-off by: Arlin Davis <ardavis@ichips.intel.com>
+
+commit b3ba3dc9743baf3bff243e8969edf3395d1118dd
+Author: Arlin Davis <arlin.r.davis@intel.com>
+Date: Tue Dec 11 14:44:24 2007 -0800
+
+ 2/2 uDAPL changes to sync common code base with WinOF
+ - add DAT_API to specify calling conventions (windows=__stdcall, linux= )
+ - cleanup platform specific definitions for windows
+ - c++ support
+ - add handle check macros DAT_IA_HANDLE_TO_UL and DAT_UL_TO_IA_HANDLE
+
+commit 92d7eef38877ad472a91e5e9f88e4e49657716bf
+Author: Arlin Davis <arlin.r.davis@intel.com>
+Date: Tue Dec 11 14:43:05 2007 -0800
+
+ 1/2 uDAT changes to sync common code base with WinOF
+ - add DAT_API to specify calling conventions (windows=__stdcall, linux= )
+ - cleanup platform specific definitions for windows
+ - c++ support
+ - add handle check macros DAT_IA_HANDLE_TO_UL and DAT_UL_TO_IA_HANDLE
+
+ Signed-off by: Arlin Davis <ardavis@ichips.intel.com>
+ Signed-off by: Stan Smith <stan.smith@intel.com>
+
+commit 95764c6da28284d29071cf01d1a09bdcb967a971
+Author: Arlin Davis <arlin.r.davis@intel.com>
+Date: Tue Dec 4 13:19:27 2007 -0800
+
+ - Fix size of evd request queue on creation
+ - Add query and checking of DTO request queue
+
+ Signed-off by: Arlin Davis <ardavis@ichips.intel.com>
+
+commit 9bc97e65c1240224d7dc9d6ac9a48e7aed199ee6
+Merge: 11a165a... abb4356...
+Author: Arlin Davis <arlin.r.davis@intel.com>
+Date: Tue Nov 27 13:31:32 2007 -0800
+
+ master_dat2.0_merge
+
+commit abb4356cd765d38cf5cff2dfbdb610b380f944a2
+Author: Arlin Davis <arlin.r.davis@intel.com>
+Date: Tue Nov 20 12:10:39 2007 -0800
+
+ Release 2.0.3
+
+commit d2c66eb7363234c5a9fb82aa92df1c132e46477e
+Author: Arlin Davis <arlin.r.davis@intel.com>
+Date: Tue Nov 20 12:07:58 2007 -0800
+
+ - Lower default settings (rdma ops, inline sends) for latest iWARP/IB devices
+ - Add missing ia_query for max_iov_segments_per_rdma_write
+ - Cleanup CMA code no longer supported by rdma_cm
+
+ Signed-off by: Arlin Davis <ardavis@ichips.intel.com>
+
+commit 3a3519167bd65bd999424788f139b930b099b405
+Author: Arlin Davis <arlin.r.davis@intel.com>
+Date: Mon Nov 19 15:26:44 2007 -0800
+
+ Change dapltest timers to use gettimeofday instead of get_cycles for better portability.
+
+ Signed-off by: Arlin Davis <ardavis@ichips.intel.com>
+
+commit 11a165a1868b1748fe476e08fc40af620f961cd2
+Author: Arlin Davis <arlin.r.davis@intel.com>
+Date: Wed Oct 31 10:58:51 2007 -0800
+
+ - DAT/DAPL Version 1.2.3 Release 1
+
+ Signed-off by: Arlin Davis <ardavis@ichips.intel.com>
+
+commit bc2d39a78e31c5e9463c8fa16f0ecaf49f75a15f
+Author: Arlin Davis <arlin.r.davis@intel.com>
+Date: Wed Oct 31 10:55:48 2007 -0800
+
+ ChangeLog update
+
+commit 7d3ec3d68a756f895a6c6ba8ed3d7a1d602468e9
+Author: Arlin Davis <arlin.r.davis@intel.com>
+Date: Wed Oct 31 10:55:05 2007 -0800
+
+ - DAT/DAPL Version 2.0.2 Release 1
+
+ Signed-off by: Arlin Davis <ardavis@ichips.intel.com>
+
+commit 43931378b9d4f5f721da828623f1e391f32f446b
+Author: Arlin Davis <arlin.r.davis@intel.com>
+Date: Tue Oct 30 09:06:24 2007 -0800
+
+ - Add ChangeLog
+ - update cma provider to report remote and local ports via dat_ep_query.
+
+ Signed-off by: Arlin Davis <ardavis@ichips.intel.com>
+
+commit a65da8a86ed637bacc32e3518d6c37eeb3b496bf
+Author: Arlin Davis <arlin.r.davis@intel.com>
+Date: Thu Oct 25 14:37:14 2007 -0700
+
+ Fix dapltest endian issue with mem_handle, mem_address.
+
+ Signed-off-by: Shirley Ma <xma@us.ibm.com>
+
+commit 8196f1655fe6088c66dafa6ad8e4474ea8ebe1d9
+Author: Arlin Davis <arlin.r.davis@intel.com>
+Date: Thu Oct 25 14:36:12 2007 -0700
+
+ Fix dapltest endian issue with mem_handle, mem_address.
+
+ Signed-off-by: Shirley Ma <xma@us.ibm.com>
+
+commit 39667dbb0160d395fb20eb53a1b4e995e2e623dd
+Author: Arlin Davis <arlin.r.davis@intel.com>
+Date: Tue Oct 16 14:23:17 2007 -0700
+
+ Fix dtest to build properly with extensions.
+
+ Modify dat.h dat_event to include event_extension_data[8].
+ Extend struct dat_event outside of extension build
+ switch to enable non-extended applications to work
+ with extended libraries. Otherwise, there is a potential
+ for the event callee to write back too much event data
+ and exceed callers non-extended event buffer.
+
+ Signed-off by: Arlin Davis <ardavis@ichips.intel.com>
+
+commit d7134fb2bcad6f4f68410af997f8791edd788cfb
+Author: Arlin Davis <arlin.r.davis@intel.com>
+Date: Tue Oct 16 14:10:52 2007 -0700
+
+ Use inet_ntoa instead of open coding it. IP addresses were being
+ reversed on PowerPC.
+
+ On PowerPC the timebase ticks at a different frequency to the CPU.
+ dapltest currently assumes a 1:1 relationship, and gives bogus values
+ when scaling timebase to real time.
+
+ To fix this, look at the timebase field in /proc/cpuinfo instead. To
+ keep things consistent with x86, scale that value to MHz.
+
+ Signed-off-by: Anton Blanchard <anton@samba.org>
+
+commit c6710c291a4f7c0845a4535767d41d66f092fabf
+Author: Arlin Davis <arlin.r.davis@intel.com>
+Date: Tue Oct 16 14:09:56 2007 -0700
+
+ Use inet_ntoa instead of open coding it. IP addresses were being
+ reversed on PowerPC.
+
+ On PowerPC the timebase ticks at a different frequency to the CPU.
+ dapltest currently assumes a 1:1 relationship, and gives bogus values
+ when scaling timebase to real time.
+
+ To fix this, look at the timebase field in /proc/cpuinfo instead. To
+ keep things consistent with x86, scale that value to MHz.
+
+ Signed-off-by: Anton Blanchard <anton@samba.org>
+
+commit 9446029979a55e6e477fb31cfdf9ce0dc77ffa8f
+Author: Arlin Davis <arlin.r.davis@intel.com>
+Date: Tue Oct 16 14:02:36 2007 -0700
+
+ Minor clean-up of cr/lf
+
+ Signed-off by: Arlin Davis <ardavis@ichips.intel.com>
+
+commit 33fd0628497911df11dea640aea4660e54989ed6
+Author: Arlin Davis <arlin.r.davis@intel.com>
+Date: Tue Oct 2 16:01:37 2007 -0700
+
+ Final changes for 2.0.1-1 package, OFED 1.3 ALPHA
+ Fix build issue with SLES 10, gcc++ compiler
+
+ Signed-off-by: Jimmy Hill <jimmy.hill@us.ibm.com>
+ Signed-off-by: Arlin Davis <ardavis@ichips.intel.com>
+
+commit bc5f16991d75ff9d09e3e3a3cc8c2d6801a9d61f
+Author: Arlin Davis <arlin.r.davis@intel.com>
+Date: Tue Oct 2 14:50:02 2007 -0700
+
+ Final changes for package 1.2.2-1, OFED 1.3 ALPHA
+
+ Signed-off by: Arlin Davis <ardavis@ichips.intel.com>
+
+commit 52bc16939e87587f8208e775dd061f54196a9acb
+Author: Arlin Davis <arlin.r.davis@intel.com>
+Date: Tue Oct 2 11:58:46 2007 -0700
+
+ Change v2 dat.conf provider names to associate with ib net devices
+
+ Signed-off by: Arlin Davis <ardavis@ichips.intel.com>
+
+commit d22e62f989dd16d503d5430ffe6f55075139e057
+Author: Arlin Davis <arlin.r.davis@intel.com>
+Date: Tue Oct 2 11:43:34 2007 -0700
+
+ Change DT_Mdep_GetTime to use gettimeofday() which has more resolution
+ than times().
+
+ Signed-off-by: Anton Blanchard <anton@samba.org>
+
+commit a64eae5bd36598a5c93010e07869e95599aa8ceb
+Author: Arlin Davis <arlin.r.davis@intel.com>
+Date: Tue Oct 2 11:41:40 2007 -0700
+
+ Change v2 dat.conf provider names to associate with ib net devices
+
+ Signed-off by: Arlin Davis <ardavis@ichips.intel.com>
+
+commit 870764dfad8df0ffe6d3d449e7a8e296cfee8ef5
+Author: Arlin Davis <arlin.r.davis@intel.com>
+Date: Tue Oct 2 11:35:21 2007 -0700
+
+ Change DT_Mdep_GetTime to use gettimeofday() which has more resolution
+ than times().
+
+ Signed-off-by: Anton Blanchard <anton@samba.org>
+
+commit 66bf23e3e53f370c92803f162144947f29ce06d8
+Author: Arlin Davis <arlin.r.davis@intel.com>
+Date: Tue Oct 2 11:30:15 2007 -0700
+
+ Change DT_Mdep_GetTime to use gettimeofday() which has more resolution
+ than times().
+
+ Signed-off-by: Anton Blanchard <anton@samba.org>
+
+commit c220760bbb1f6357b6e187ff6c5e576dd74fd504
+Author: Arlin Davis <arlin.r.davis@intel.com>
+Date: Tue Oct 2 10:39:09 2007 -0700
+
+ Fix dapl to compile as both 32bit and 64bit on PowerPC. Instead of using
+ the kernel atomic routines, code them explicitely like x86 does.
+
+ Signed-off-by: Anton Blanchard <anton@samba.org>
+
+commit 9ffdf3722e5a9cde3fd6add5d7b3940a7ea9f71f
+Author: Arlin Davis <arlin.r.davis@intel.com>
+Date: Fri Sep 28 17:04:54 2007 -0700
+
+ Clean up packaging, modify dat.conf for v2.0 cma provider name change to ofa
+
+ Signed-off by: Arlin Davis <ardavis@ichips.intel.com>
+
+commit 493e65b5b47f47e4824f775959fd98e56aeaccc4
+Author: Arlin Davis <arlin.r.davis@intel.com>
+Date: Fri Sep 28 17:03:41 2007 -0700
+
+ Clean up packaging, modify dat.conf for cma provider name change to ofa
+
+ Signed-off by: Arlin Davis <ardavis@ichips.intel.com>
+
+commit 99f0a0bf0a0d99fee0729fba0fcdf6f3e89e2ec4
+Author: Arlin Davis <arlin.r.davis@intel.com>
+Date: Thu Sep 20 12:25:55 2007 -0700
+
+ Modifications to coexist with 2.0 libraries
+ - fix RPM specfile, configure.in, 1.2.2 package
+ - modify dat.conf
+
+ Signed-off by: Arlin Davis <ardavis@ichips.intel.com>
+
+commit 7ff4a8a8e861b0701b5b2a6fc95e6aa8b36d2662
+Author: Arlin Davis <arlin.r.davis@intel.com>
+Date: Thu Sep 20 10:55:19 2007 -0700
+
+ Modifications to coexist with 1.2 libraries
+ - cleanup CR-LF in dtestx
+ - fix RPM specfile, 2.0.1 package
+
+ Signed-off by: Arlin Davis <ardavis@ichips.intel.com>
+
+commit 230767742b8287490373c09d1bd346337b375b48
+Author: Arlin Davis <arlin.r.davis@intel.com>
+Date: Fri Jun 22 11:48:20 2007 -0700
+
+ Update copyright in specfile
+
+commit 5294cbe5e58f67d0a98862edea3684fff6e773bb
+Author: Arlin Davis <arlin.r.davis@intel.com>
+Date: Fri Jun 22 11:47:14 2007 -0700
+
+ Update Copyright in specfile
+
+commit 3654c6ef425f94b9f27a593b0b8c1f3d7cc39029
+Author: Arlin Davis <arlin.r.davis@intel.com>
+Date: Wed Jun 6 13:20:38 2007 -0700
+
+ Update specfile to reflect DAT/DAPL 1.2.1 release
+
+commit babb95eff1bcef88bed46e92c323193d8f039eff
+Author: Arlin Davis <arlin.r.davis@intel.com>
+Date: Wed Jun 6 11:48:07 2007 -0700
+
+ More changes to the release notes
+
+commit 0f299bf1deb9198b2f008c3ffa717bef252b6097
+Author: Arlin Davis <arlin.r.davis@intel.com>
+Date: Tue Jun 5 15:56:17 2007 -0700
+
+ Update release notes
+
+commit ad70f98a228ade4a863ca349990eaa7ab1e82ec2
+Author: Arlin Davis <arlin.r.davis@intel.com>
+Date: Tue Jun 5 15:52:18 2007 -0700
+
+ Add release notes updated for OFED 1.2 release
+
+commit f332badb80f0b1d88bf2d70dba0c90afc40f088f
+Author: Arlin Davis <arlin.r.davis@intel.com>
+Date: Thu May 31 12:43:28 2007 -0700
+
+ Add provider specific interface to uDAPL for extensions support.
+ Fix memory leak with extensions, missing cookie deallocation when processing extended DTO events
+ Remove unnecessary check on dats_set_ia_handle in dat_ia_openv
+ Clean up specfile and some extra LF's
+
+commit 4d7e30586402149228a30bea3036466395577ec4
+Author: Arlin Davis <arlin.r.davis@intel.com>
+Date: Wed May 16 12:56:39 2007 -0700
+
+ add iwarp extension include file
+
+commit d9963cc9984c06f147b92877945e847f657cd512
+Author: Arlin Davis <arlin.r.davis@intel.com>
+Date: Wed May 16 12:52:38 2007 -0700
+
+ clean up some CR's in include files
+
+commit 80f4e8c4be02bac5d472e1e6c4937079882a0388
+Author: Arlin Davis <arlin.r.davis@intel.com>
+Date: Wed May 9 16:21:16 2007 -0700
+
+ Take out references to specific 1.1 verbs definitions (ibv_comp_channel).
+
+commit 544fbb873d5320e9606f176c4b71bcba0e257d7d
+Author: Arlin Davis <arlin.r.davis@intel.com>
+Date: Wed May 9 12:51:53 2007 -0700
+
+ Bug Fixes:
+ - 606: Return local and remote ports with dat_ep_query
+ - 585: Add bonding example to dat.conf
+
+commit e64079f7b961aa9a672711f0e933a77f3999d302
+Author: Arlin Davis <arlin.r.davis@intel.com>
+Date: Mon May 7 15:54:59 2007 -0700
+
+ Update dapltest to use default device OpenIB-cma
+ Fix dapltest manpage, example for performance test wrong
+
+commit 7cda2304a6880371ec2d9451a4f83a7a254bc474
+Author: Arlin Davis <arlin.r.davis@intel.com>
+Date: Mon May 7 12:49:18 2007 -0700
+
+ Fix issues with IB DTO extensions
+ - debug print_cqe updated for latest IBV definitions
+ - dapli_evd_cq_to_event modified to handle both post and receive extensions
+ - dtestx example tested with rdma_write_imm, fetch_add, and cmp_swap
+ - modify cookie DTO types to support extensions properly
+
+commit 82a6750d31cd432c7e09298fc98c0e7e74abd012
+Author: Arlin Davis <arlin.r.davis@intel.com>
+Date: Fri May 4 17:17:41 2007 -0700
+
+ Add new dapl functions for 2.0 (stubs for now)
+ - dapl_ep_post_send_invalidate
+ - dapl_ep_post_rdma_read_to_rmr
+ - dapl_ia_ha
+ - dapl_csp_create, query, free
+ - dapl_cno_trigger
+ - dapl_cno_fd_create
+ - dapl_extensions
+
+ Add new 2.0 parameters to existing API's
+ - dapl_cr_reject
+ - dapl_rmr_bind
+
+commit 8679aaf56c781715adc132a38a731e36194570f1
+Author: Arlin Davis <arlin.r.davis@intel.com>
+Date: Thu May 3 09:44:43 2007 -0700
+
+ update dtestx.c, default provider OpenIB-cma
+
+commit 527f6d19125e9eec7ecef03a8633626b0043a2f4
+Author: Arlin Davis <arlin.r.davis@intel.com>
+Date: Wed May 2 17:27:44 2007 -0700
+
+ Added IB extensions and dtestx as an example program using
+ rdma write with immediate and atomic operations
+
+commit 83ac961b505346708f12d59152146f3b04c8306f
+Author: Arlin Davis <arlin.r.davis@intel.com>
+Date: Mon Apr 30 10:55:59 2007 -0700
+
+ Fixes atomic operation build problem with ia64 and RHEL5.
+
+commit 04da88bb70ee33b249a4cf2f5a92122eeec83e3c
+Author: Arlin Davis <arlin.r.davis@intel.com>
+Date: Thu Apr 26 17:29:37 2007 -0700
+
+ Update README and dapltest manpage
+
+commit 9a951d0a8713657da90568c0613eb48f5010cf1e
+Author: Arlin Davis <arlin.r.davis@intel.com>
+Date: Tue Apr 3 16:35:17 2007 -0700
+
+ Cleanup RPM specfile for the dapl package, move to 1.2-1 release.
+
+commit a93f0ffcd6a46735c97ec34de564a7a91f9fc5c2
+Author: Arlin Davis <arlin.r.davis@intel.com>
+Date: Tue Apr 3 14:38:31 2007 -0700
+
+ Add support for multiple IB devices to dat.conf to support IPoIB HA failover.
+
+commit 5434b720b36de2f262a02ff9dfccd99953c09e59
+Author: Arlin Davis <arlin.r.davis@intel.com>
+Date: Thu Mar 15 10:46:33 2007 -0800
+
+ Fix ia64 builds on SUSE
+
+commit b0f9eef1aa7f279802da233480cf6c495e16565b
+Author: Arlin Davis <arlin.r.davis@intel.com>
+Date: Wed Mar 14 13:29:08 2007 -0800
+
+ DAT 2.0 support for new DAT_VA_TYPE added to uDAPL and openib-cma provider
+
+commit 61858b925f4c1a6f9edba6389a5bd601daf936e9
+Author: Arlin Davis <arlin.r.davis@intel.com>
+Date: Wed Mar 14 11:08:19 2007 -0800
+
+ change DAT_VERSION major and minor numbers to 2.0
+
+commit 6dcb2e083bda8f2521bd3515b329c5465e1ac724
+Author: Arlin Davis <arlin.r.davis@intel.com>
+Date: Wed Mar 14 10:43:56 2007 -0800
+
+ add provider support to dtest, set default to OpenIB-cma
+
+commit ff8eb667c1000be6c68ca291e7ed7bd080cb73f4
+Author: Arlin Davis <arlin.r.davis@intel.com>
+Date: Tue Mar 13 16:20:20 2007 -0800
+
+ add provider option to dtest, set default to OpenIB-cma
+
+commit 76a43cace54567135bac7ae54e336c6595b65fd9
+Author: Arlin Davis <arlin.r.davis@intel.com>
+Date: Fri Mar 9 13:09:14 2007 -0800
+
+ Initial build with DAT 2.0 definitions and IB extensions. IB extensions configurable with --enable-ext-type=ib
+
+commit 921687efed992e6ab72dfb731687b6816324a024
+Author: Arlin Davis <arlin.r.davis@intel.com>
+Date: Thu Mar 8 16:01:29 2007 -0800
+
+ Update the README
+
+commit 52ed210ae99b291f72441e71459006b5f2c851ce
+Author: Arlin Davis <arlin.r.davis@intel.com>
+Date: Wed Mar 7 15:34:41 2007 -0800
+
+ - Fix bug 408, dapltest compilation failure on 32 bit arch
+ - Update libdat.spec.in file to build uDAPL RPMs correctly
+
+commit e3f6aca57a8fa5cbaaf872bf6844feb7d5e1e66c
+Author: Arlin Davis <arlin.r.davis@intel.com>
+Date: Mon Mar 5 14:15:49 2007 -0800
+
+ Fix build issues with dtest and dapltest. Define build tree path to dat/include.
+
+commit f1f829a28e645831c3bcd1eb2d465fcb7a1fd5d8
+Author: Arlin Davis <arlin.r.davis@intel.com>
+Date: Wed Feb 28 17:14:55 2007 -0800
+
+ Add dapltest headers to EXTRA_DIST
+ Modify dtest to delay before accepting
+
+ Signed-off-by: Vladimir Sokolovsky <vlad@mellanox.co.il>
+ Signed-off-by: Arlin Davis <ardavis@ichips.intel.com>
+
+commit 46b830a4664d5fee2daf1ebdc4e95ecb7c580e80
+Author: Arlin Davis <arlin.r.davis@intel.com>
+Date: Mon Feb 26 13:54:15 2007 -0800
+
+ Adding dtest and dapltest to the build. Manual pages created.
+
+commit d245664e27148e54469268ad81f41b2a894a131a
+Author: Arlin Davis <arlin.r.davis@intel.com>
+Date: Fri Jan 19 16:21:30 2007 -0800
+
+ uDAPL changes to support exchanging and validation of the device responder_resources and the
+ initiator_depth during connection establishment
+
+commit 2280f833090aa9f750d5be8f9b06e7e08e642da5
+Author: Arlin Davis <arlin.r.davis@intel.com>
+Date: Wed Dec 6 11:49:27 2006 -0800
+
+ Update cma provider to sync with rdma_ucm changes
+
+commit 89448545b415b6dff57e3314b020619f6b979ef8
+Author: Arlin Davis <arlin.r.davis@intel.com>
+Date: Mon Dec 4 13:54:20 2006 -0800
+
+ Update autogen to create config directory
+
+commit 0a917b104eba0aae6c6ef49c7990a2dc7efc759d
+Author: Arlin Davis <ardavis@ichips.intel.com>
+Date: Tue Nov 7 20:22:05 2006 +0000
+
+ r10074: Added support for new ib verbs client register event. No extra processing required at the uDAPL
+ level.
+
+ Fix some issues supporting create qp without recv cq handle or recv qp resources. IB verbs assume a
+ recv_cq handle and uDAPL dapl_ep_create assumes there is always recv_sge resources specified.
+
+ Fix some timeout and long disconnect delay issues discovered during scale-out testing. Added support
+ to retry rdma_cm address and route resolution with configuration options. Provide a disconnect call
+ when receiving the disconnect request to guarantee a disconnect reply and event on the remote side.
+ The rdma_disconnect was not being called from dat_ep_disconnect() as a result of the state changing
+ to DISCONNECTED in the event callback.
+
+ Here are the new options (environment variables) with the default setting:
+
+ DAPL_CM_ARP_TIMEOUT_MS 4000
+ DAPL_CM_ARP_RETRY_COUNT 15
+ DAPL_CM_ROUTE_TIMEOUT_MS 4000
+ DAPL_CM_ROUTE_RETRY_COUNT 15
+
+commit c73aeb904504a0bc6cce0fb1248af9ba39521395
+Author: Arlin Davis <ardavis@ichips.intel.com>
+Date: Thu Oct 12 22:41:33 2006 +0000
+
+ r9802: Remove Socket CM provider from build and dat.conf configuration. No longer needed nor supported.
+
+commit b1d94b26610f682cdd43bde2aecf5004e0865422
+Author: Steve Wise <swise@opengridcomputing.com>
+Date: Tue Sep 12 18:15:39 2006 +0000
+
+ r9442: Update obsolete CLK_TCK to CLOCKS_PER_SEC
+ Signed-off-by: Steve Wise <swise@opengridcomputing.com>
+ Signed-off-by: James Lentini <jlentini@netapp.com>
+
+commit 99a5dddd07d4c271ebb075b5b0f800101f850a56
+Author: Arlin Davis <ardavis@ichips.intel.com>
+Date: Thu Sep 7 18:09:11 2006 +0000
+
+ r9346:
+ inadvertently added evdtest to makefile when testing dat_evd_set_unwaitable fix with openib_cma provider
+
+commit b53a87c856d9754313da9543a1dac5c6f1307085
+Author: Arlin Davis <ardavis@ichips.intel.com>
+Date: Wed Sep 6 20:36:09 2006 +0000
+
+ r9315:
+ Fill out some unitialized fields in the ia_attr structure
+ returned by dat_ia_query().
+
+ Signed-off by: Arlin Davis <ardavis@ichips.intel.com>
+ Signed-off by: Robert Walsh <robert.walsh@qlogic.com>
+
+commit b6c4e84399d0aa44c72c6ca870409c3666d7e79b
+Author: Arlin Davis <ardavis@ichips.intel.com>
+Date: Fri Aug 11 20:44:23 2006 +0000
+
+ r8895: Update dtest to support multiple segments on rdma write and change makefile to use OpenIB-cma by default.
+
+commit 4737c63d79b23c3aff329e864dd50e3cffb6a17f
+Author: Arlin Davis <ardavis@ichips.intel.com>
+Date: Wed Jul 19 17:15:06 2006 +0000
+
+ r8592: Add support for dat_evd_set_unwaitable on a DTO evd in openib_cma provider
+
+commit 25fb8c376547de895a170194c09b2d72dfea789d
+Author: Arlin Davis <ardavis@ichips.intel.com>
+Date: Mon Jul 17 22:59:17 2006 +0000
+
+ r8565: Added errno reporting (message and return codes) during open to help diagnose create thread issues.
+
+commit f3a1ed6232ccdee7d193e8e3b9b0013b2bd222af
+Author: Anton Blanchard <anton@samba.org>
+Date: Mon Jul 17 21:26:03 2006 +0000
+
+ r8562: Fix some suspicious inline assembly:
+ - EIEIO_ON_SMP and ISYNC_ON_SMP are in kernel UP build optimisations, we
+ shouldnt export them to userspace. Replace it with lwsync and isync.
+ - The comment says its implemenenting cmpxchg64 but in fact its
+ implementing cmpxchg32. Fix the comment.
+
+ Signed-off-by: Anton Blanchard <anton@samba.org>
+ Signed-off-by: James Lentini <jlentini@netapp.com>
+
+commit 63759108a1376b6e45a4491551f71d8cafdcddc1
+Author: James Lentini <jlentini@netapp.com>
+Date: Wed Jul 12 14:56:26 2006 +0000
+
+ r8503: Fix IA64 build problems reported by John Partridge <johnip@sgi.com>
+ Signed-off-by: James Lentini <jlentini@netapp.com>
+
+commit 51362c9781cb65fd8f9a3cb5b7c12c88e4c8527a
+Author: Arlin Davis <ardavis@ichips.intel.com>
+Date: Thu Jun 22 22:02:56 2006 +0000
+
+ r8182: Lower the reject debug message level so we don't see warnings
+ when consumers reject.
+ Signed-off-by: Arlin Davis <ardavis@ichips.intel.com>
+ Signed-off-by: James Lentini <jlentini@netapp.com>
+
+commit f14889bb0cd22e897148ea2f6931a6b4f23143b0
+Author: Arlin Davis <ardavis@ichips.intel.com>
+Date: Thu Jun 22 21:13:37 2006 +0000
+
+ r8181: Added support for active side TIMED_OUT event from a provider.
+ Signed-off-by: Arlin Davis <ardavis@ichips.intel.com>
+ Signed-off-by: James Lentini <jlentini@netapp.com>
+
+commit 14fc704ae20b6a3ad0d433d7de9c02ce99e095b3
+Author: Arlin Davis <ardavis@ichips.intel.com>
+Date: Thu Jun 22 20:58:06 2006 +0000
+
+ r8180: Fix bug in dapls_ib_get_dat_event() call after adding new unreachable event.
+ Signed-off-by: Arlin Davis <ardavis@ichips.intel.com>
+ Signed-off-by: James Lentini <jlentini@netapp.com>
+
+commit 893698c31a0d8e60227806d992485a44375840cb
+Author: James Lentini <jlentini@netapp.com>
+Date: Mon Jun 19 17:20:45 2006 +0000
+
+ r8112: Update for new rdma_create_id() function signature.
+ Signed-off-by: James Lentini <jlentini@netapp.com>
+
+commit 53483d84b0d02c432d9435d2f8e840cab3ded320
+Author: Arlin Davis <ardavis@ichips.intel.com>
+Date: Wed Jun 14 16:17:39 2006 +0000
+
+ r8008: Set max rdma read per EP attributes
+ Signed-off-by: Arlin Davis <ardavis@ichips.intel.com>
+ Signed-off-by: James Lentini <jlentini@netapp.com>
+
+commit 6cb854cd72d9492ddc1c5da01dbfb24b3f30af83
+Author: Arlin Davis <ardavis@ichips.intel.com>
+Date: Mon Jun 12 15:42:50 2006 +0000
+
+ r7931: Report the proper error and timeout events.
+ Signed-off-by: Arlin Davis <ardavis@ichips.intel.com>
+ Signed-off-by: James Lentini <jlentini@netapp.com>
+
+commit 44a97ff1f599f4bf882a801ead7aa495ea9ba936
+Author: Arlin Davis <ardavis@ichips.intel.com>
+Date: Mon Jun 12 14:51:14 2006 +0000
+
+ r7928: Socket CM fix to guard against using a loopback address
+ as the local device address.
+ Signed-off-by: Arlin Davis <ardavis@ichips.intel.com>
+ Signed-off-by: James Lentini <jlentini@netapp.com>
+
+commit 152219cf83c61e459fdf3de03d4e83ddba045230
+Author: Arlin Davis <ardavis@ichips.intel.com>
+Date: Tue Jun 6 21:46:44 2006 +0000
+
+ r7755: Use the uCM set_option feature to adjust connect request timeout
+ and retry values. Also, a fix to disallow any event after a disconnect
+ event.
+ Signed-off-by: Arlin Davis <ardavis@ichips.intel.com>
+ Signed-off-by: James Lentini <jlentini@netapp.com>
+
+commit 3a0686a2624ed28c7ea37b650415801f1cedbd10
+Author: James Lentini <jlentini@netapp.com>
+Date: Wed May 31 19:55:57 2006 +0000
+
+ r7608: Correct comment.
+ Signed-off-by: James Lentini <jlentini@netapp.com>
+
+commit eb760157c90f59183b424ac8e71474fe0b46094c
+Author: James Lentini <jlentini@netapp.com>
+Date: Thu May 18 21:54:12 2006 +0000
+
+ r7347: Undo inadvertent change.
+ Signed-off-by: James Lentini <jlentini@netapp.com>
+
+commit 27256222b42fecfac8a44b3f82fe2524ecc72de2
+Author: James Lentini <jlentini@netapp.com>
+Date: Thu May 18 21:50:27 2006 +0000
+
+ r7346: Fix for uCMA provider to return the correct event as a result of
+ rejects. Also, ran into a segv bug with dapl_ep_create when
+ creating without a conn_evd.
+
+ Signed-off by: Arlin Davis <ardavis@ichips.intel.com>
+ Signed-off-by: James Lentini <jlentini@netapp.com>
+
+commit b1b6e16f3e41e123cd347bc78b01e3272076362b
+Author: Arlin Davis <ardavis@ichips.intel.com>
+Date: Fri May 12 19:50:19 2006 +0000
+
+ r7141: Update the uDAPL openib_cma provider to work with the new
+ uCMA event channel interface.
+ Signed-off-by: Arlin Davis <ardavis@ichips.intel.com>
+ Signed-off-by: James Lentini <jlentini@netapp.com>
+
+commit 27f9f0c106662cc7b41bcb747495860a1b6c7133
+Author: Steve Wise <swise@opengridcomputing.com>
+Date: Tue May 2 21:33:35 2006 +0000
+
+ r6873: Transaction test change to comply with the iWARP MPA protocol's
+ "Connection Startup Rules".
+ Signed-off-by: Steve Wise <swise@opengridcomputing.com>
+ Signed-off-by: James Lentini <jlentini@netapp.com>
+
+commit 060d09f974ffbe73672e17641b2f18d3821d31a7
+Author: Arlin Davis <ardavis@ichips.intel.com>
+Date: Fri Apr 28 13:44:17 2006 +0000
+
+ r6736: getaddrinfo() fix for debug builds and some additional debug messages for
+ connect errors and rejects.
+
+ Signed-off-by: Arlin Davis <ardavis@ichips.intel.com>
+ Signed-off-by: James Lentini <jlentini@netapp.com>
+
+commit 4e8b95bbeaf6e5c27de59ef347ec2ef0aa2e5e6b
+Author: James Lentini <jlentini@netapp.com>
+Date: Wed Apr 19 16:49:34 2006 +0000
+
+ r6520: Fix compilation warning.
+ Signed-off-by: James Lentini <jlentini@netapp.com>
+
+commit 115fcc396164066326f9447d71af798a381d063f
+Author: Steve Wise <swise@opengridcomputing.com>
+Date: Wed Apr 19 16:32:01 2006 +0000
+
+ r6518: Do not always generate an event for an abrupt disconnect.
+ Signed-off-by: Steve Wise <swise@opengridcomputing.com>
+ Signed-off-by: James Lentini <jlentini@netapp.com>
+
+commit f959bb786cd884bf4d2a5da4d299da6297d65293
+Author: James Lentini <jlentini@netapp.com>
+Date: Wed Apr 19 16:30:37 2006 +0000
+
+ r6517: Generate a disconnect event for providers that do not generate
+ one automatically.
+ Signed-off-by: James Lentini <jlentini@netapp.com>
+
+commit bb467511cf6e217147817ba12bbe800aae97cab5
+Author: James Lentini <jlentini@netapp.com>
+Date: Wed Apr 19 16:25:33 2006 +0000
+
+ r6516: Fix compilation error.
+ Signed-off-by: James Lentini <jlentini@netapp.com>
+
+commit 117a9856c269bf08b738a1923c92f5a1949f6cc1
+Author: James Lentini <jlentini@netapp.com>
+Date: Mon Apr 10 20:16:44 2006 +0000
+
+ r6392: Fix for RDMA cm id destruction and debug builds.
+ Signed-off by: Arlin Davis <ardavis@ichips.intel.com>
+ Signed-off-by: James Lentini <jlentini@netapp.com>
+
+commit 5f56b2b7339c17276188464dfff12b1be9e1dbb7
+Author: James Lentini <jlentini@netapp.com>
+Date: Thu Apr 6 15:32:47 2006 +0000
+
+ r6289: Set max_iov_segments_per_rdma_read and max_rdma_read_iov using the correct
+ attribute.
+ Signed-off-by: James Lentini <jlentini@netapp.com>
+
+commit 304f48370adcaa12463c1a7d99e513164b83810c
+Author: Steve Wise <swise@opengridcomputing.com>
+Date: Thu Apr 6 15:16:10 2006 +0000
+
+ r6286: Set the IA attribute max_iov_segments_per_rdma_read and the EP attribute
+ max_rdma_read_iov based on the openib max_sge_rd device attribute.
+ Signed-off-by: Steve Wise <swise@opengridcomputing.com>
+ Signed-off-by: James Lentini <jlentini@netapp.com>
+
+commit fe27222d2a00d7c5c4d98f39d2926fe14c7f32bc
+Author: James Lentini <jlentini@netapp.com>
+Date: Tue Apr 4 18:29:00 2006 +0000
+
+ r6221: Change the mechanism by which library init and fini functions are specified
+ Signed-off-by: James Lentini <jlentini@netapp.com>
+
+commit 5a0598b90ab021cb2115e3791cb38dcfc0347948
+Author: James Lentini <jlentini@netapp.com>
+Date: Mon Apr 3 17:29:55 2006 +0000
+
+ r6182: Remove unused variables.
+ Signed-off-by: James Lentini <jlentini@netapp.com>
+
+commit b8084c4edc21b5ac2191ec654a882b65bad0c77d
+Author: James Lentini <jlentini@netapp.com>
+Date: Mon Apr 3 15:29:30 2006 +0000
+
+ r6179: Fix dapltest compiler warnings.
+ Signed-off-by: James Lentini <jlentini@netapp.com>
+
+commit abdbec194670d72012d481b98b2e6f728e9c5b48
+Author: James Lentini <jlentini@netapp.com>
+Date: Mon Apr 3 14:08:48 2006 +0000
+
+ r6168: Fixed debug prints of 64-bit values and removed compile warnings.
+ Signed-off-by: James Lentini <jlentini@netapp.com>
+
+commit 52de2a35e02ddeb15887d0d690b52a4e02812e57
+Author: James Lentini <jlentini@netapp.com>
+Date: Tue Mar 21 22:28:10 2006 +0000
+
+ r5939: Move libdat.spec to libdat.spec.in
+ Signed-off-by: James Lentini <jlentini@netapp.com>
+
+commit 6b1a6356a1757ae9b9d1557bd2ae67e0913d04c2
+Author: Arlin Davis <ardavis@ichips.intel.com>
+Date: Fri Mar 17 22:02:18 2006 +0000
+
+ r5879: Add GNU Autotools support and an RPM spec file
+ Signed-off-by: Arlin Davis <ardavis@ichips.intel.com>
+ Signed-off-by: James Lentini <jlentini@netapp.com>
+
+commit 9d00582b8be33add8d3e4173e8311b1a222b0b34
+Author: Arlin Davis <ardavis@ichips.intel.com>
+Date: Thu Mar 16 22:40:18 2006 +0000
+
+ r5871: Fixes a corner case where a CMA event was not acknowledged during
+ disconnect processing.
+
+ Signed-off-by: Arlin Davis <ardavis@ichips.intel.com>
+ Signed-off-by: James Lentini <jlentini@netapp.com>
+
+commit ede1d714a8e7ae99246eb382c1c1165f238cf7c3
+Author: James Lentini <jlentini@netapp.com>
+Date: Mon Mar 13 18:15:49 2006 +0000
+
+ r5789: Reduces some debug output in the async thread and fixes listen
+ processing of EBUSY.
+
+ Signed-off by: Arlin Davis <ardavis@ichips.intel.com>
+ Signed-off by: James Lentini <jlentini@netapp.com>
+
+commit afd558cbdada66e01aa553364b9a126da00dbe65
+Author: Arlin Davis <ardavis@ichips.intel.com>
+Date: Wed Mar 8 14:24:26 2006 +0000
+
+ r5684: Remove unused NO_EP_STATE.
+ Signed-off-by: Arlin Davis <ardavis@ichips.intel.com>
+ Signed-off-by: James Lentini <jlentini@netapp.com>
+
+commit 4c36aca16b9b492a91c0b05002af5fc954f3b44e
+Author: James Lentini <jlentini@netapp.com>
+Date: Tue Feb 28 15:41:03 2006 +0000
+
+ r5529: Fix to destroy QPs in all cases and close the HCA.
+
+ Signed-off by: Arlin Davis <ardavis@ichips.intel.com>
+ Signed-off by: James Lentini <jlentini@netapp.com>
+
+commit 5874fdb26436c58365a7096b9b68c8e45a51d7a3
+Author: James Lentini <jlentini@netapp.com>
+Date: Mon Jan 9 20:17:33 2006 +0000
+
+ r4855: Make use of dat_evd_wait the default.
+ Signed-off-by: James Lentini <jlentini@netapp.com>
+
+commit 254bd0e349bafbd970d6475efcafd7e52f05415e
+Author: Arlin Davis <ardavis@ichips.intel.com>
+Date: Mon Jan 9 16:59:38 2006 +0000
+
+ r4852: Fix disconnect event processing and update dtest to validate.
+ Signed-off-by: Arlin Davis <ardavis@ichips.intel.com>
+ Signed-off-by: James Lentini <jlentini@netapp.com>
+
+commit f69d737ee51f6b00a1e6cf8531695a61a322651b
+Author: Arlin Davis <ardavis@ichips.intel.com>
+Date: Fri Dec 16 02:03:05 2005 +0000
+
+ r4502: Query for rd_atomic values.
+ Signed-off-by: Arlin Davis <ardavis@ichips.intel.com>
+ Signed-off-by: James Lentini <jlentini@netapp.com>
+
+commit 86566b556920a51f1e538d245674058682012668
+Author: James Lentini <jlentini@netapp.com>
+Date: Fri Dec 16 01:57:30 2005 +0000
+
+ r4501: Allow a network name, network address, or device name in the dat.conf file.
+ Singed-off-by: Arlin Davis <ardavis@ichips.intel.com>
+ Signed-off-by: James Lentini <jlentini@netapp.com>
+
+commit 9e0ca3d4ccb92b9c0337efbadce405076a365f0f
+Author: Roland Dreier <rolandd@cisco.com>
+Date: Wed Dec 14 20:44:36 2005 +0000
+
+ r4466: Change libibverbs API for listing all known devices from
+ ibv_get_devices() to ibv_get_device_list(), and update all
+ in-tree uses of this API.
+
+ Signed-off-by: Roland Dreier <rolandd@cisco.com>
+
+commit 73a80143ab7b3f9aad19f84f904f99b489dca6cf
+Author: James Lentini <jlentini@netapp.com>
+Date: Mon Dec 5 16:37:46 2005 +0000
+
+ r4308: Fix a gcc 4.0 warning
+ Signed-off-by: James Lentini <jlentini@netapp.com>
+
+commit aa8b16b7e83f321eaaa18b38e6c165c2f120bcec
+Author: Arlin Davis <ardavis@ichips.intel.com>
+Date: Thu Dec 1 15:03:10 2005 +0000
+
+ r4279: Added CMA API support.
+ Signed-off-by: Arlin Davis <ardavis@ichips.intel.com>
+ Signed-off-by: James Lentini <jlentini@netapp.com>
+
+commit 74d3c32e42ab54f3d6f2eec3d0a66d08f800e075
+Author: James Lentini <jlentini@netapp.com>
+Date: Thu Nov 10 21:39:34 2005 +0000
+
+ r4018: Fixed some problems with the free build openib_scm version and turned
+ down some debugging and added some debug prints for uAT path records.
+ Signed-off by: Arlin Davis <ardavis@ichips.intel.com>
+ Signed-off by: James Lentini <jlentini@netapp.com>
+
+commit 70a2f23c5604e55f8f76672f78b4bf92f6a79b98
+Author: James Lentini <jlentini@netapp.com>
+Date: Mon Oct 31 18:27:13 2005 +0000
+
+ r3917: Fix printing of debug statements.
+ Signed off by: Aniruddha Bohra <bohra@cs.rutgers.edu>
+ Signed off by: James Lentini <jlentini@netapp.com
+
+commit 4306debe3118948fca72a68ec20ef0d73b399fd1
+Author: James Lentini <jlentini@netapp.com>
+Date: Wed Oct 26 21:10:26 2005 +0000
+
+ r3882: uDAPL provider for OpenIB socket CM.
+ Signed-off by: Arlin Davis <ardavis@ichips.intel.com>
+ Signed-off by: James Lentini <jlentini@netapp.com>
+
+commit 42a64ec2ec1d8ec71492bfebba077b006684ce97
+Author: James Lentini <jlentini@netapp.com>
+Date: Thu Oct 13 20:45:22 2005 +0000
+
+ r3774: Fix the async error handling and callback mappings.
+ Updated TODO list.
+ Signed-off by: Arlin Davis <ardavis@ichips.intel.com>
+ Signed-off by: James Lentini <jlentini@netapp.com>
+
+commit fea8ed1188fbeef8291cfd7e681cd48c06ff5bde
+Author: Robert Walsh <rjwalsh@pathscale.com>
+Date: Fri Oct 7 21:15:00 2005 +0000
+
+ r3693: Update some more ignores.
+
+commit 303147143afa9aa72906246a9f1973e4172f75b8
+Author: Todd Bowman <twbowman@gmail.com>
+Date: Thu Oct 6 21:13:32 2005 +0000
+
+ r3687: Add PPC64 support for udapl
+ Signed-off-by: Todd Bowman <twbowman@gmail.com>
+ Signed-off-by: James Lentini <jlentini@netapp.com>
+
+commit 103c7db321e24a7b5b06c7c26b0e0a65d1dd11ce
+Author: Todd Bowman <twbowman@gmail.com>
+Date: Thu Oct 6 15:22:08 2005 +0000
+
+ r3683: Remove the dtest qualifier from the sdp range.
+ Signed-off-by: Todd Bowman <twbowman@gmail.com>
+ Signed-off-by: James Lentini <jlentini@netapp.com>
+
+commit 33fbf9c81ac29492394e419588d856533e7fffb8
+Author: James Lentini <jlentini@netapp.com>
+Date: Mon Oct 3 14:59:22 2005 +0000
+
+ r3637: Support CQ_WAIT_OBJECT with channels and sync with latest verbs.
+ Signed-off by: Arlin Davis <ardavis@ichips.intel.com>
+ Signed-off by: James Lentini <jlentini@netapp.com>
+
+commit f01dac62b08d8f4fd417c336be48fb3bc8cd15c5
+Author: James Lentini <jlentini@netapp.com>
+Date: Thu Sep 29 18:13:25 2005 +0000
+
+ r3619: Makefile fix.
+ Signed-off-by: James Lentini <jlentini@netapp.com>
+
+commit 634b199218b775a8ed071c1faea519c4cc4ee4e3
+Author: James Lentini <jlentini@netapp.com>
+Date: Wed Sep 28 21:50:13 2005 +0000
+
+ r3606: Fixes IA64 build problems (atomics) with the latest Redhat EL4.0 update and
+ adds support for SuSe.
+ Signed-off by: Arlin Davis <ardavis@ichips.intel.com>
+ Signed-off by: James Lentini <jlentini@netapp.com>
+
+commit 3a23c7135905666ff969b86dab3e90f90ac73008
+Author: Robert Walsh <rjwalsh@pathscale.com>
+Date: Tue Sep 27 16:59:14 2005 +0000
+
+ r3567: Setup svn:ignore on a bunch of directories.
+
+commit d41ea62125636a58e8748871e372810c09865b0a
+Author: James Lentini <jlentini@netapp.com>
+Date: Thu Sep 22 21:24:38 2005 +0000
+
+ r3525: Improve dtest and measure RDMA reads
+ Signed-off by: Arlin Davis <ardavis@ichips.intel.com>
+ Signed-off-by: James Lentini <jlentini@netapp.com>
+
+commit 32258d13af6aaf76078ec6ba187d8129a0f70397
+Author: James Lentini <jlentini@netapp.com>
+Date: Wed Sep 21 19:54:07 2005 +0000
+
+ r3513: Temporary workaround for the RDMA read performance anomaly
+ Signed-off by: Arlin Davis <ardavis@ichips.intel.com>
+ Signed-off by: James Lentini <jlentini@netapp.com>
+
+commit 990bdc4320809e0b989c90c24bef361c1bc91c7f
+Author: James Lentini <jlentini@netapp.com>
+Date: Tue Sep 20 17:00:37 2005 +0000
+
+ r3494: Moved dapl_evd_modify_cno.c to match SourceForge
+ Signed-off-by: James Lentini <jlentini@netapp.com>
+
+commit 4509fb64fdbf99db7bdcaad4d8e3884718184d86
+Author: James Lentini <jlentini@netapp.com>
+Date: Tue Sep 20 16:17:59 2005 +0000
+
+ r3493: Support ib_cm_init_qp_attr(), add cm event processing on a per
+ device basis, and add copyrights for kDAPL cm work that was
+ used in uDAPL.
+ Signed-off by: Arlin Davis <ardavis@ichips.intel.com>
+ Signed-off by: James Lentini <jlentini@netapp.com>
+
+commit c1d45137c2e26024142f24207344e3e48a577b83
+Author: Sean Hefty <sean.hefty@intel.com>
+Date: Thu Sep 15 21:43:21 2005 +0000
+
+ r3453: Bind communication identifiers to a device to support device removal.
+ Export per HCA CM devices to userspace.
+
+ Signed-off-by: Sean Hefty <sean.hefty@intel.com>
+
+commit 5146689f1dbbce01cc46e23d749c28828e7e3ca8
+Author: James Lentini <jlentini@netapp.com>
+Date: Wed Sep 14 17:44:44 2005 +0000
+
+ r3432: Request address resolution using ATS.
+ Signed-off-by: James Lentini <jlentini@netapp.com>
+
+commit 780b8237f2bf6884cf1abcc11190f97ed5c0a343
+Author: James Lentini <jlentini@netapp.com>
+Date: Mon Sep 12 19:14:43 2005 +0000
+
+ r3378: Added DAPL documentation.
+ Signed-off-by: James Lentini <jlentini@netapp.com>
+
+commit 8911b60eb16770d28907c14e45556444317dd276
+Author: James Lentini <jlentini@netapp.com>
+Date: Mon Sep 12 19:10:13 2005 +0000
+
+ r3377: Removed executable premission.
+ Signed-off-by: James Lentini <jlentini@netapp.com>
+
+commit dec4d2eafebdfe7e6b495a36dd16bd5a98417e04
+Author: Sean Hefty <sean.hefty@intel.com>
+Date: Fri Sep 9 21:51:58 2005 +0000
+
+ r3349: Update DAPL to match the verbs and CM event processing APIs.
+ Signed-off-by: Sean Hefty <sean.hefty@intel.com>
+ Signed-off-by: James Lentini <jlentini@netapp.com>
+
+commit d06dcfd25e5d37310d089bcb7f3d3d75fcece75a
+Author: Arlin Davis <ardavis@ichips.intel.com>
+Date: Tue Sep 6 19:34:46 2005 +0000
+
+ r3326: Changes to support async events. Also consolidated the uAT,uCM,uCQ threads into one processing thread.
+ Signed-off-by: Arlin Davis <ardavis@ichips.intel.com>
+ Signed-off-by: James Lentini <jlentini@netapp.com>
+
+commit 1dd8c28ce515675ee8df37cc1596bca17587eaf6
+Author: James Lentini <jlentini@netapp.com>
+Date: Mon Aug 29 15:07:44 2005 +0000
+
+ r3232: validate default settings so they don't exceed device maximums
+ Signed-off by: Arlin Davis <ardavis@ichips.intel.com>
+ Signed-off by: James Lentini <jlentini@netapp.com>
+
+commit ac5308b4aac7de79a356439dfec2312faf7705ae
+Author: James Lentini <jlentini@netapp.com>
+Date: Mon Aug 29 14:15:23 2005 +0000
+
+ r3227: Support for ibv_query_port, device, and gid.
+ Signed-off by: Arlin Davis <ardavis@ichips.intel.com>
+ Signed-off by: James Lentini <jlentini@netapp.com>
+
+commit ef8c7a3a4896487ff0d37a7d7234746c15fe5cf7
+Author: Sean Hefty <sean.hefty@intel.com>
+Date: Sun Aug 21 22:30:08 2005 +0000
+
+ r3143: - Add user specified context to all uCM events. Users will not retrieve
+ any events associated with the context after destroying the corresponding
+ cm_id.
+ - Provide the ib_cm_init_qp_attr() call to userspace clients of the CM.
+ This call may be used to set QP attributes properly before modifying the QP.
+ - Fixes some error handling syncrhonization and cleanup issues.
+ - Performs some minor code cleanup.
+ - Replaces the ucm_simple test program with a userspace version of cmpost.
+ - Updates DAPL to the new API.
+
+ Signed-off-by: Sean Hefty <sean.hefty@intel.com>
+
+commit ec0f86d7c55830c6dffded585c04754cc6ac2a83
+Author: James Lentini <jlentini@netapp.com>
+Date: Tue Aug 16 14:10:13 2005 +0000
+
+ r3107: Removed unused debug counter macros
+ Signed-off-by: James Lentini <jlentini@netapp.com>
+
+commit 49087ba27bd93769bb64d7ac5c454de1b94005bc
+Author: James Lentini <jlentini@netapp.com>
+Date: Tue Aug 16 14:07:42 2005 +0000
+
+ r3106: Implemented debug counters
+ Signed-off by: Arlin Davis <ardavis@ichips.intel.com>
+ Signed-off by: James Lentini <jlentini@netapp.com>
+
+commit f98e3af7dfc56b288cc77a9103b90f8d6a927fc5
+Author: James Lentini <jlentini@netapp.com>
+Date: Thu Aug 11 20:23:56 2005 +0000
+
+ r3072: Update from SourceForge DAPL: use the LMR context in calls to
+ dapls_hash_remove()
+ Signed-off-by: James Lentini <jlentini@netapp.com>
+
+commit 878e524c5cc63a62802d28fdc215a2b69ceb1141
+Author: James Lentini <jlentini@netapp.com>
+Date: Thu Aug 11 20:07:06 2005 +0000
+
+ r3071: Updates from SourceForge DAPL: EVD updates
+ Signed-off-by: James Lentini <jlentini@netapp.com>
+
+commit 8dc70f7d972615f40e624d8f1272e5e7c16ba34f
+Author: James Lentini <jlentini@netapp.com>
+Date: Thu Aug 11 19:57:40 2005 +0000
+
+ r3070: Update from SourceForge DAPL: set async evd to null in dapli_ia_release_hca
+ Signed-off-by: James Lentini <jlentini@netapp.com>
+
+commit f2801ae6caf010d660fe302970dabddc8948e1bf
+Author: James Lentini <jlentini@netapp.com>
+Date: Thu Aug 11 19:46:39 2005 +0000
+
+ r3069: Updates from SourceForge DAPL: size EP IOV correctly
+ Signed-off-by: James Lentini <jlentini@netapp.com>
+
+commit c87fd235eaf0b3a30e005422f7d347c406c14f2c
+Author: James Lentini <jlentini@netapp.com>
+Date: Thu Aug 11 19:35:05 2005 +0000
+
+ r3068: Update from SourceForge DAPL: removed duplicate ia pointer in SP structure and
+ fixed the spelling of ib_hca_transport_t.
+ Signed-off-by: James Lentini <jlentini@netapp.com>
+
+commit a88bebc09a9655e462b3d32dfddec823024eab59
+Author: James Lentin <jlentini@netapp.com>
+Date: Thu Aug 11 19:24:56 2005 +0000
+
+ r3067: Update from SourceForge DAPL: use include guard
+ Signed-off-by: James Lentin <jlentini@netapp.com>
+
+commit d2da08920de882a9a266f0606b81150c625fa003
+Author: James Lentini <jlentini@netapp.com>
+Date: Thu Aug 11 19:21:56 2005 +0000
+
+ r3066: Update from SourceForge DAPL: optimization to dapl_ep_query
+ Signed-off-by: James Lentini <jlentini@netapp.com>
+
+commit 4ca3b0cbc59227a90b5450eea1ffeeb91826dd6d
+Author: James Lentini <jlentini@netapp.com>
+Date: Thu Aug 11 18:39:34 2005 +0000
+
+ r3065: Update from DAPL SourceForge: indicate which handle is invalid
+ Signed-off-by: James Lentini <jlentini@netapp.com>
+
+commit 6d8f34137776c32149251bdec493c017b399cd10
+Author: James Lentini <jlentini@netapp.com>
+Date: Thu Aug 11 18:35:58 2005 +0000
+
+ r3064: Update from DAPL SourceForge: set ep param values.
+ Signed-off-by: James Lentini <jlentini@netapp.com>
+
+commit 0f35002a1942303ff46cb9a2b70056f9a38aebdb
+Author: James Lentini <jlentini@netapp.com>
+Date: Thu Aug 11 18:33:47 2005 +0000
+
+ r3063: Updates from DAPL SourceForge: QP state and connection event fix.
+ Signed-off-by: James Lentini <jlentini@netapp.com>
+
+commit 3fc876339693c6f0eed5e57780e5342f301bd95c
+Author: James Lentini <jlentini@netapp.com>
+Date: Thu Aug 11 18:23:33 2005 +0000
+
+ r3062: Update from DAPL SourceForge: remove unused DAPL_CNO_PROXY_CALL
+ Signed-off-by: James Lentini <jlentini@netapp.com>
+
+commit 70f8e7a2e6bde4e757ddc8c7f59d3a5c6a13adf9
+Author: Hal Rosenstock <halr@voltaire.com>
+Date: Fri Aug 5 17:59:38 2005 +0000
+
+ r2989: Fix dtest makefile
+ Signed-off-by: Hal Rosenstock <halr@voltaire.com>
+ Signed-off-by: James Lentini <jlentini@netapp.com>
+
+commit 864695cfef37d84359ada8838ab4cd4f4dafc6bb
+Author: James Lentini <jlentini@netapp.com>
+Date: Fri Aug 5 17:57:31 2005 +0000
+
+ r2988: Remove kernel directory.
+ Signed-off-by: James Lentini <jlentini@netapp.com>
+
+commit 9c4e246a5baf43cadc6380e91fd5a6e319777278
+Author: James Lentini <jlentini@netapp.com>
+Date: Fri Aug 5 17:56:56 2005 +0000
+
+ r2987: Remove kernel code directory.
+ Signed-off-by: James Lentini <jlentini@netapp.com>
+
+commit 26706cb0de471ba47279de0cb949ba5a41de82cc
+Author: James Lentini <jlentini@netapp.com>
+Date: Fri Aug 5 16:41:12 2005 +0000
+
+ r2986: Add uDAPL to the trunk
+ Signed-off-by: James Lentini <jlentini@netapp.com>
+
+commit 76aa2de7fe38a8595d88669842450084cfa88316
+Author: Roland Dreier <roland@topspin.com>
+Date: Thu Nov 4 17:54:50 2004 +0000
+
+ r1139: Copy roland-merge branch to trunk
+
+commit 3bd72a559dfe22685aae33599c99d021d2ae4aca
+Author: Roland Dreier <roland@topspin.com>
+Date: Tue Jul 20 21:34:32 2004 +0000
+
+ r502: Move 2.6-only source away from trunk
+
+commit 4f05b6ed3fd1d14161664c677264846eeb51dba5
+Author: Roland Dreier <roland@topspin.com>
+Date: Tue Jul 20 21:34:32 2004 +0000
+
+ r502: Move 2.6-only source away from trunk
+
+commit 6da8b951c069072a2afc6aba03a3dca2c44db022
+Author: Roland Dreier <roland@topspin.com>
+Date: Tue Jul 20 01:41:16 2004 +0000
+
+ r497: Move 2.6-only tree to gen2
-# $Id: $\r
-\r
-OSFLAGS = -DOS_RELEASE=$(shell expr `uname -r | cut -f1 -d.` \* 65536 + `uname -r | cut -f2 -d.`)\r
-# Check for RedHat, needed for ia64 udapl atomic operations (IA64_FETCHADD syntax)\r
-# and built-in atomics for RedHat EL5\r
-if OS_RHEL4\r
-OSFLAGS += -DREDHAT_EL4\r
-endif\r
-\r
-if OS_RHEL5\r
-OSFLAGS += -DREDHAT_EL5\r
-endif\r
-\r
-if OS_SUSE11\r
-OSFLAGS += -DSUSE_11\r
-endif\r
-\r
-if EXT_TYPE_IB\r
-XFLAGS = -DDAT_EXTENSIONS\r
-XPROGRAMS = dapl/openib_common/ib_extensions.c\r
-else\r
-XFLAGS =\r
-XPROGRAMS =\r
-endif\r
-\r
-if DEBUG\r
-AM_CFLAGS = -g -Wall -D_GNU_SOURCE -DDAPL_DBG -DDAT_CONF="\"$(sysconfdir)/dat.conf\""\r
-else\r
-AM_CFLAGS = -g -Wall -D_GNU_SOURCE -DDAT_CONF="\"$(sysconfdir)/dat.conf\""\r
-endif\r
-\r
-datlibdir = $(libdir)\r
-dapllibofadir = $(libdir)\r
-daplliboscmdir = $(libdir)\r
-daplliboucmdir = $(libdir)\r
-\r
-datlib_LTLIBRARIES = dat/udat/libdat2.la\r
-dapllibofa_LTLIBRARIES = dapl/udapl/libdaplofa.la\r
-daplliboscm_LTLIBRARIES = dapl/udapl/libdaploscm.la\r
-daplliboucm_LTLIBRARIES = dapl/udapl/libdaploucm.la\r
-\r
-dat_udat_libdat2_la_CFLAGS = $(AM_CFLAGS) -D_GNU_SOURCE $(OSFLAGS) $(XFLAGS) \\r
- -I$(srcdir)/dat/include/ -I$(srcdir)/dat/udat/ \\r
- -I$(srcdir)/dat/udat/linux -I$(srcdir)/dat/common/ \r
-\r
-dapl_udapl_libdaplofa_la_CFLAGS = $(AM_CFLAGS) -D_GNU_SOURCE $(OSFLAGS) $(XFLAGS) \\r
- -DOPENIB -DCQ_WAIT_OBJECT \\r
- -I$(srcdir)/dat/include/ -I$(srcdir)/dapl/include/ \\r
- -I$(srcdir)/dapl/common -I$(srcdir)/dapl/udapl/linux \\r
- -I$(srcdir)/dapl/openib_common \\r
- -I$(srcdir)/dapl/openib_cma \\r
- -I$(srcdir)/dapl/openib_cma/linux\r
- \r
-dapl_udapl_libdaploscm_la_CFLAGS = $(AM_CFLAGS) -D_GNU_SOURCE $(OSFLAGS) $(XFLAGS) \\r
- -DOPENIB -DCQ_WAIT_OBJECT \\r
- -I$(srcdir)/dat/include/ -I$(srcdir)/dapl/include/ \\r
- -I$(srcdir)/dapl/common -I$(srcdir)/dapl/udapl/linux \\r
- -I$(srcdir)/dapl/openib_common \\r
- -I$(srcdir)/dapl/openib_scm \\r
- -I$(srcdir)/dapl/openib_scm/linux\r
-\r
-dapl_udapl_libdaploucm_la_CFLAGS = $(AM_CFLAGS) -D_GNU_SOURCE $(OSFLAGS) $(XFLAGS) \\r
- -DOPENIB -DCQ_WAIT_OBJECT \\r
- -I$(srcdir)/dat/include/ -I$(srcdir)/dapl/include/ \\r
- -I$(srcdir)/dapl/common -I$(srcdir)/dapl/udapl/linux \\r
- -I$(srcdir)/dapl/openib_common \\r
- -I$(srcdir)/dapl/openib_ucm \\r
- -I$(srcdir)/dapl/openib_ucm/linux \r
-\r
-if HAVE_LD_VERSION_SCRIPT\r
- dat_version_script = -Wl,--version-script=$(srcdir)/dat/udat/libdat2.map\r
- daplofa_version_script = -Wl,--version-script=$(srcdir)/dapl/udapl/libdaplofa.map\r
- daploscm_version_script = -Wl,--version-script=$(srcdir)/dapl/udapl/libdaploscm.map\r
- daploucm_version_script = -Wl,--version-script=$(srcdir)/dapl/udapl/libdaploucm.map\r
-else\r
- dat_version_script = \r
- daplofa_version_script = \r
- daploscm_version_script =\r
- daploucm_version_script =\r
-endif\r
-\r
-#\r
-# uDAT: libdat2.so\r
-#\r
-dat_udat_libdat2_la_SOURCES = dat/udat/udat.c \\r
- dat/udat/udat_api.c \\r
- dat/udat/udat_sr_parser.c \\r
- dat/udat/linux/dat_osd.c \\r
- dat/common/dat_api.c \\r
- dat/common/dat_dictionary.c \\r
- dat/common/dat_strerror.c \\r
- dat/common/dat_init.c \\r
- dat/common/dat_dr.c \\r
- dat/common/dat_sr.c\r
-dat_udat_libdat2_la_LDFLAGS = -version-info 2:0:0 $(dat_version_script) -ldl \r
-\r
-#\r
-# uDAPL OpenFabrics rdma_cm version: libdaplofa.so\r
-#\r
-dapl_udapl_libdaplofa_la_SOURCES = dapl/udapl/dapl_init.c \\r
- dapl/udapl/dapl_evd_create.c \\r
- dapl/udapl/dapl_evd_query.c \\r
- dapl/udapl/dapl_cno_create.c \\r
- dapl/udapl/dapl_cno_modify_agent.c \\r
- dapl/udapl/dapl_cno_free.c \\r
- dapl/udapl/dapl_cno_wait.c \\r
- dapl/udapl/dapl_cno_query.c \\r
- dapl/udapl/dapl_lmr_create.c \\r
- dapl/udapl/dapl_evd_wait.c \\r
- dapl/udapl/dapl_evd_disable.c \\r
- dapl/udapl/dapl_evd_enable.c \\r
- dapl/udapl/dapl_evd_modify_cno.c \\r
- dapl/udapl/dapl_evd_set_unwaitable.c \\r
- dapl/udapl/dapl_evd_clear_unwaitable.c \\r
- dapl/udapl/linux/dapl_osd.c \\r
- dapl/common/dapl_cookie.c \\r
- dapl/common/dapl_cr_accept.c \\r
- dapl/common/dapl_cr_query.c \\r
- dapl/common/dapl_cr_reject.c \\r
- dapl/common/dapl_cr_util.c \\r
- dapl/common/dapl_cr_callback.c \\r
- dapl/common/dapl_cr_handoff.c \\r
- dapl/common/dapl_ep_connect.c \\r
- dapl/common/dapl_ep_create.c \\r
- dapl/common/dapl_ep_disconnect.c \\r
- dapl/common/dapl_ep_dup_connect.c \\r
- dapl/common/dapl_ep_free.c \\r
- dapl/common/dapl_ep_reset.c \\r
- dapl/common/dapl_ep_get_status.c \\r
- dapl/common/dapl_ep_modify.c \\r
- dapl/common/dapl_ep_post_rdma_read.c \\r
- dapl/common/dapl_ep_post_rdma_write.c \\r
- dapl/common/dapl_ep_post_recv.c \\r
- dapl/common/dapl_ep_post_send.c \\r
- dapl/common/dapl_ep_query.c \\r
- dapl/common/dapl_ep_util.c \\r
- dapl/common/dapl_evd_dequeue.c \\r
- dapl/common/dapl_evd_free.c \\r
- dapl/common/dapl_evd_post_se.c \\r
- dapl/common/dapl_evd_resize.c \\r
- dapl/common/dapl_evd_util.c \\r
- dapl/common/dapl_evd_cq_async_error_callb.c \\r
- dapl/common/dapl_evd_qp_async_error_callb.c \\r
- dapl/common/dapl_evd_un_async_error_callb.c \\r
- dapl/common/dapl_evd_connection_callb.c \\r
- dapl/common/dapl_evd_dto_callb.c \\r
- dapl/common/dapl_get_consumer_context.c \\r
- dapl/common/dapl_get_handle_type.c \\r
- dapl/common/dapl_hash.c \\r
- dapl/common/dapl_hca_util.c \\r
- dapl/common/dapl_ia_close.c \\r
- dapl/common/dapl_ia_open.c \\r
- dapl/common/dapl_ia_query.c \\r
- dapl/common/dapl_ia_util.c \\r
- dapl/common/dapl_llist.c \\r
- dapl/common/dapl_lmr_free.c \\r
- dapl/common/dapl_lmr_query.c \\r
- dapl/common/dapl_lmr_util.c \\r
- dapl/common/dapl_lmr_sync_rdma_read.c \\r
- dapl/common/dapl_lmr_sync_rdma_write.c \\r
- dapl/common/dapl_mr_util.c \\r
- dapl/common/dapl_provider.c \\r
- dapl/common/dapl_sp_util.c \\r
- dapl/common/dapl_psp_create.c \\r
- dapl/common/dapl_psp_create_any.c \\r
- dapl/common/dapl_psp_free.c \\r
- dapl/common/dapl_psp_query.c \\r
- dapl/common/dapl_pz_create.c \\r
- dapl/common/dapl_pz_free.c \\r
- dapl/common/dapl_pz_query.c \\r
- dapl/common/dapl_pz_util.c \\r
- dapl/common/dapl_rmr_create.c \\r
- dapl/common/dapl_rmr_free.c \\r
- dapl/common/dapl_rmr_bind.c \\r
- dapl/common/dapl_rmr_query.c \\r
- dapl/common/dapl_rmr_util.c \\r
- dapl/common/dapl_rsp_create.c \\r
- dapl/common/dapl_rsp_free.c \\r
- dapl/common/dapl_rsp_query.c \\r
- dapl/common/dapl_cno_util.c \\r
- dapl/common/dapl_set_consumer_context.c \\r
- dapl/common/dapl_ring_buffer_util.c \\r
- dapl/common/dapl_name_service.c \\r
- dapl/common/dapl_timer_util.c \\r
- dapl/common/dapl_ep_create_with_srq.c \\r
- dapl/common/dapl_ep_recv_query.c \\r
- dapl/common/dapl_ep_set_watermark.c \\r
- dapl/common/dapl_srq_create.c \\r
- dapl/common/dapl_srq_free.c \\r
- dapl/common/dapl_srq_query.c \\r
- dapl/common/dapl_srq_resize.c \\r
- dapl/common/dapl_srq_post_recv.c \\r
- dapl/common/dapl_srq_set_lw.c \\r
- dapl/common/dapl_srq_util.c \\r
- dapl/common/dapl_debug.c \\r
- dapl/common/dapl_ia_ha.c \\r
- dapl/common/dapl_csp.c \\r
- dapl/common/dapl_ep_post_send_invalidate.c \\r
- dapl/common/dapl_ep_post_rdma_read_to_rmr.c \\r
- dapl/openib_common/mem.c \\r
- dapl/openib_common/cq.c \\r
- dapl/openib_common/qp.c \\r
- dapl/openib_common/util.c \\r
- dapl/openib_cma/cm.c \\r
- dapl/openib_cma/device.c $(XPROGRAMS)\r
-\r
-dapl_udapl_libdaplofa_la_LDFLAGS = -version-info 2:0:0 $(daplofa_version_script) \\r
- -Wl,-init,dapl_init -Wl,-fini,dapl_fini \\r
- -lpthread -libverbs -lrdmacm \r
- \r
-#\r
-# uDAPL OpenFabrics Socket CM version for IB: libdaplscm.so\r
-#\r
-dapl_udapl_libdaploscm_la_SOURCES = dapl/udapl/dapl_init.c \\r
- dapl/udapl/dapl_evd_create.c \\r
- dapl/udapl/dapl_evd_query.c \\r
- dapl/udapl/dapl_cno_create.c \\r
- dapl/udapl/dapl_cno_modify_agent.c \\r
- dapl/udapl/dapl_cno_free.c \\r
- dapl/udapl/dapl_cno_wait.c \\r
- dapl/udapl/dapl_cno_query.c \\r
- dapl/udapl/dapl_lmr_create.c \\r
- dapl/udapl/dapl_evd_wait.c \\r
- dapl/udapl/dapl_evd_disable.c \\r
- dapl/udapl/dapl_evd_enable.c \\r
- dapl/udapl/dapl_evd_modify_cno.c \\r
- dapl/udapl/dapl_evd_set_unwaitable.c \\r
- dapl/udapl/dapl_evd_clear_unwaitable.c \\r
- dapl/udapl/linux/dapl_osd.c \\r
- dapl/common/dapl_cookie.c \\r
- dapl/common/dapl_cr_accept.c \\r
- dapl/common/dapl_cr_query.c \\r
- dapl/common/dapl_cr_reject.c \\r
- dapl/common/dapl_cr_util.c \\r
- dapl/common/dapl_cr_callback.c \\r
- dapl/common/dapl_cr_handoff.c \\r
- dapl/common/dapl_ep_connect.c \\r
- dapl/common/dapl_ep_create.c \\r
- dapl/common/dapl_ep_disconnect.c \\r
- dapl/common/dapl_ep_dup_connect.c \\r
- dapl/common/dapl_ep_free.c \\r
- dapl/common/dapl_ep_reset.c \\r
- dapl/common/dapl_ep_get_status.c \\r
- dapl/common/dapl_ep_modify.c \\r
- dapl/common/dapl_ep_post_rdma_read.c \\r
- dapl/common/dapl_ep_post_rdma_write.c \\r
- dapl/common/dapl_ep_post_recv.c \\r
- dapl/common/dapl_ep_post_send.c \\r
- dapl/common/dapl_ep_query.c \\r
- dapl/common/dapl_ep_util.c \\r
- dapl/common/dapl_evd_dequeue.c \\r
- dapl/common/dapl_evd_free.c \\r
- dapl/common/dapl_evd_post_se.c \\r
- dapl/common/dapl_evd_resize.c \\r
- dapl/common/dapl_evd_util.c \\r
- dapl/common/dapl_evd_cq_async_error_callb.c \\r
- dapl/common/dapl_evd_qp_async_error_callb.c \\r
- dapl/common/dapl_evd_un_async_error_callb.c \\r
- dapl/common/dapl_evd_connection_callb.c \\r
- dapl/common/dapl_evd_dto_callb.c \\r
- dapl/common/dapl_get_consumer_context.c \\r
- dapl/common/dapl_get_handle_type.c \\r
- dapl/common/dapl_hash.c \\r
- dapl/common/dapl_hca_util.c \\r
- dapl/common/dapl_ia_close.c \\r
- dapl/common/dapl_ia_open.c \\r
- dapl/common/dapl_ia_query.c \\r
- dapl/common/dapl_ia_util.c \\r
- dapl/common/dapl_llist.c \\r
- dapl/common/dapl_lmr_free.c \\r
- dapl/common/dapl_lmr_query.c \\r
- dapl/common/dapl_lmr_util.c \\r
- dapl/common/dapl_lmr_sync_rdma_read.c \\r
- dapl/common/dapl_lmr_sync_rdma_write.c \\r
- dapl/common/dapl_mr_util.c \\r
- dapl/common/dapl_provider.c \\r
- dapl/common/dapl_sp_util.c \\r
- dapl/common/dapl_psp_create.c \\r
- dapl/common/dapl_psp_create_any.c \\r
- dapl/common/dapl_psp_free.c \\r
- dapl/common/dapl_psp_query.c \\r
- dapl/common/dapl_pz_create.c \\r
- dapl/common/dapl_pz_free.c \\r
- dapl/common/dapl_pz_query.c \\r
- dapl/common/dapl_pz_util.c \\r
- dapl/common/dapl_rmr_create.c \\r
- dapl/common/dapl_rmr_free.c \\r
- dapl/common/dapl_rmr_bind.c \\r
- dapl/common/dapl_rmr_query.c \\r
- dapl/common/dapl_rmr_util.c \\r
- dapl/common/dapl_rsp_create.c \\r
- dapl/common/dapl_rsp_free.c \\r
- dapl/common/dapl_rsp_query.c \\r
- dapl/common/dapl_cno_util.c \\r
- dapl/common/dapl_set_consumer_context.c \\r
- dapl/common/dapl_ring_buffer_util.c \\r
- dapl/common/dapl_name_service.c \\r
- dapl/common/dapl_timer_util.c \\r
- dapl/common/dapl_ep_create_with_srq.c \\r
- dapl/common/dapl_ep_recv_query.c \\r
- dapl/common/dapl_ep_set_watermark.c \\r
- dapl/common/dapl_srq_create.c \\r
- dapl/common/dapl_srq_free.c \\r
- dapl/common/dapl_srq_query.c \\r
- dapl/common/dapl_srq_resize.c \\r
- dapl/common/dapl_srq_post_recv.c \\r
- dapl/common/dapl_srq_set_lw.c \\r
- dapl/common/dapl_srq_util.c \\r
- dapl/common/dapl_debug.c \\r
- dapl/common/dapl_ia_ha.c \\r
- dapl/common/dapl_csp.c \\r
- dapl/common/dapl_ep_post_send_invalidate.c \\r
- dapl/common/dapl_ep_post_rdma_read_to_rmr.c \\r
- dapl/openib_common/mem.c \\r
- dapl/openib_common/cq.c \\r
- dapl/openib_common/qp.c \\r
- dapl/openib_common/util.c \\r
- dapl/openib_scm/cm.c \\r
- dapl/openib_scm/device.c $(XPROGRAMS)\r
-\r
-dapl_udapl_libdaploscm_la_LDFLAGS = -version-info 2:0:0 $(daploscm_version_script) \\r
- -Wl,-init,dapl_init -Wl,-fini,dapl_fini \\r
- -lpthread -libverbs\r
- \r
-#\r
-# uDAPL OpenFabrics UD CM version for IB: libdaplucm.so\r
-#\r
-dapl_udapl_libdaploucm_la_SOURCES = dapl/udapl/dapl_init.c \\r
- dapl/udapl/dapl_evd_create.c \\r
- dapl/udapl/dapl_evd_query.c \\r
- dapl/udapl/dapl_cno_create.c \\r
- dapl/udapl/dapl_cno_modify_agent.c \\r
- dapl/udapl/dapl_cno_free.c \\r
- dapl/udapl/dapl_cno_wait.c \\r
- dapl/udapl/dapl_cno_query.c \\r
- dapl/udapl/dapl_lmr_create.c \\r
- dapl/udapl/dapl_evd_wait.c \\r
- dapl/udapl/dapl_evd_disable.c \\r
- dapl/udapl/dapl_evd_enable.c \\r
- dapl/udapl/dapl_evd_modify_cno.c \\r
- dapl/udapl/dapl_evd_set_unwaitable.c \\r
- dapl/udapl/dapl_evd_clear_unwaitable.c \\r
- dapl/udapl/linux/dapl_osd.c \\r
- dapl/common/dapl_cookie.c \\r
- dapl/common/dapl_cr_accept.c \\r
- dapl/common/dapl_cr_query.c \\r
- dapl/common/dapl_cr_reject.c \\r
- dapl/common/dapl_cr_util.c \\r
- dapl/common/dapl_cr_callback.c \\r
- dapl/common/dapl_cr_handoff.c \\r
- dapl/common/dapl_ep_connect.c \\r
- dapl/common/dapl_ep_create.c \\r
- dapl/common/dapl_ep_disconnect.c \\r
- dapl/common/dapl_ep_dup_connect.c \\r
- dapl/common/dapl_ep_free.c \\r
- dapl/common/dapl_ep_reset.c \\r
- dapl/common/dapl_ep_get_status.c \\r
- dapl/common/dapl_ep_modify.c \\r
- dapl/common/dapl_ep_post_rdma_read.c \\r
- dapl/common/dapl_ep_post_rdma_write.c \\r
- dapl/common/dapl_ep_post_recv.c \\r
- dapl/common/dapl_ep_post_send.c \\r
- dapl/common/dapl_ep_query.c \\r
- dapl/common/dapl_ep_util.c \\r
- dapl/common/dapl_evd_dequeue.c \\r
- dapl/common/dapl_evd_free.c \\r
- dapl/common/dapl_evd_post_se.c \\r
- dapl/common/dapl_evd_resize.c \\r
- dapl/common/dapl_evd_util.c \\r
- dapl/common/dapl_evd_cq_async_error_callb.c \\r
- dapl/common/dapl_evd_qp_async_error_callb.c \\r
- dapl/common/dapl_evd_un_async_error_callb.c \\r
- dapl/common/dapl_evd_connection_callb.c \\r
- dapl/common/dapl_evd_dto_callb.c \\r
- dapl/common/dapl_get_consumer_context.c \\r
- dapl/common/dapl_get_handle_type.c \\r
- dapl/common/dapl_hash.c \\r
- dapl/common/dapl_hca_util.c \\r
- dapl/common/dapl_ia_close.c \\r
- dapl/common/dapl_ia_open.c \\r
- dapl/common/dapl_ia_query.c \\r
- dapl/common/dapl_ia_util.c \\r
- dapl/common/dapl_llist.c \\r
- dapl/common/dapl_lmr_free.c \\r
- dapl/common/dapl_lmr_query.c \\r
- dapl/common/dapl_lmr_util.c \\r
- dapl/common/dapl_lmr_sync_rdma_read.c \\r
- dapl/common/dapl_lmr_sync_rdma_write.c \\r
- dapl/common/dapl_mr_util.c \\r
- dapl/common/dapl_provider.c \\r
- dapl/common/dapl_sp_util.c \\r
- dapl/common/dapl_psp_create.c \\r
- dapl/common/dapl_psp_create_any.c \\r
- dapl/common/dapl_psp_free.c \\r
- dapl/common/dapl_psp_query.c \\r
- dapl/common/dapl_pz_create.c \\r
- dapl/common/dapl_pz_free.c \\r
- dapl/common/dapl_pz_query.c \\r
- dapl/common/dapl_pz_util.c \\r
- dapl/common/dapl_rmr_create.c \\r
- dapl/common/dapl_rmr_free.c \\r
- dapl/common/dapl_rmr_bind.c \\r
- dapl/common/dapl_rmr_query.c \\r
- dapl/common/dapl_rmr_util.c \\r
- dapl/common/dapl_rsp_create.c \\r
- dapl/common/dapl_rsp_free.c \\r
- dapl/common/dapl_rsp_query.c \\r
- dapl/common/dapl_cno_util.c \\r
- dapl/common/dapl_set_consumer_context.c \\r
- dapl/common/dapl_ring_buffer_util.c \\r
- dapl/common/dapl_name_service.c \\r
- dapl/common/dapl_timer_util.c \\r
- dapl/common/dapl_ep_create_with_srq.c \\r
- dapl/common/dapl_ep_recv_query.c \\r
- dapl/common/dapl_ep_set_watermark.c \\r
- dapl/common/dapl_srq_create.c \\r
- dapl/common/dapl_srq_free.c \\r
- dapl/common/dapl_srq_query.c \\r
- dapl/common/dapl_srq_resize.c \\r
- dapl/common/dapl_srq_post_recv.c \\r
- dapl/common/dapl_srq_set_lw.c \\r
- dapl/common/dapl_srq_util.c \\r
- dapl/common/dapl_debug.c \\r
- dapl/common/dapl_ia_ha.c \\r
- dapl/common/dapl_csp.c \\r
- dapl/common/dapl_ep_post_send_invalidate.c \\r
- dapl/common/dapl_ep_post_rdma_read_to_rmr.c \\r
- dapl/openib_common/mem.c \\r
- dapl/openib_common/cq.c \\r
- dapl/openib_common/qp.c \\r
- dapl/openib_common/util.c \\r
- dapl/openib_ucm/cm.c \\r
- dapl/openib_ucm/device.c $(XPROGRAMS)\r
-\r
-dapl_udapl_libdaploucm_la_LDFLAGS = -version-info 2:0:0 $(daploscm_version_script) \\r
- -Wl,-init,dapl_init -Wl,-fini,dapl_fini \\r
- -lpthread -libverbs\r
-\r
-libdatincludedir = $(includedir)/dat2\r
-\r
-libdatinclude_HEADERS = dat/include/dat2/dat.h \\r
- dat/include/dat2/dat_error.h \\r
- dat/include/dat2/dat_platform_specific.h \\r
- dat/include/dat2/dat_redirection.h \\r
- dat/include/dat2/dat_registry.h \\r
- dat/include/dat2/dat_vendor_specific.h \\r
- dat/include/dat2/udat_config.h \\r
- dat/include/dat2/udat.h \\r
- dat/include/dat2/udat_redirection.h \\r
- dat/include/dat2/udat_vendor_specific.h \\r
- dat/include/dat2/dat_ib_extensions.h \r
- \r
-man_MANS = man/dtest.1 man/dapltest.1 man/dat.conf.5 \r
- \r
-EXTRA_DIST = dat/common/dat_dictionary.h \\r
- dat/common/dat_dr.h \\r
- dat/common/dat_init.h \\r
- dat/common/dat_sr.h \\r
- dat/udat/udat_sr_parser.h \\r
- dat/udat/linux/dat_osd.h \\r
- dat/include/dat2/dat.h \\r
- dat/include/dat2/dat_error.h \\r
- dat/include/dat2/dat_platform_specific.h \\r
- dat/include/dat2/dat_redirection.h \\r
- dat/include/dat2/dat_registry.h \\r
- dat/include/dat2/dat_vendor_specific.h \\r
- dat/include/dat2/udat_config.h \\r
- dat/include/dat2/udat.h \\r
- dat/include/dat2/udat_redirection.h \\r
- dat/include/dat2/udat_vendor_specific.h \\r
- dapl/common/dapl_adapter_util.h \\r
- dapl/common/dapl_cno_util.h \\r
- dapl/common/dapl_cookie.h \\r
- dapl/common/dapl_cr_util.h \\r
- dapl/common/dapl_ep_util.h \\r
- dapl/common/dapl_evd_util.h \\r
- dapl/common/dapl_hash.h \\r
- dapl/common/dapl_hca_util.h \\r
- dapl/common/dapl_ia_util.h \\r
- dapl/common/dapl_init.h \\r
- dapl/common/dapl_lmr_util.h \\r
- dapl/common/dapl_mr_util.h \\r
- dapl/common/dapl_name_service.h \\r
- dapl/common/dapl_provider.h \\r
- dapl/common/dapl_pz_util.h \\r
- dapl/common/dapl_ring_buffer_util.h \\r
- dapl/common/dapl_rmr_util.h \\r
- dapl/common/dapl_sp_util.h \\r
- dapl/common/dapl_srq_util.h \\r
- dapl/common/dapl_timer_util.h \\r
- dapl/udapl/linux/dapl_osd.h \\r
- dapl/include/dapl.h \\r
- dapl/include/dapl_debug.h \\r
- dapl/include/dapl_ipoib_names.h \\r
- dapl/include/dapl_vendor.h \\r
- dapl/openib_common/dapl_ib_dto.h \\r
- dapl/openib_common/dapl_ib_common.h \\r
- dapl/openib_cma/dapl_ib_util.h \\r
- dapl/openib_cma/linux/openib_osd.h \\r
- dapl/openib_scm/dapl_ib_util.h \\r
- dapl/openib_scm/linux/openib_osd.h \\r
- dapl/openib_ucm/dapl_ib_util.h \\r
- dapl/openib_ucm/linux/openib_osd.h \\r
- dat/udat/libdat2.map \\r
- dapl/udapl/libdaplofa.map \\r
- dapl/udapl/libdaploscm.map \\r
- dapl/udapl/libdaploucm.map \\r
- dapl.spec.in \\r
- $(man_MANS) \\r
- test/dapltest/include/dapl_bpool.h \\r
- test/dapltest/include/dapl_client_info.h \\r
- test/dapltest/include/dapl_common.h \\r
- test/dapltest/include/dapl_execute.h \\r
- test/dapltest/include/dapl_fft_cmd.h \\r
- test/dapltest/include/dapl_fft_util.h \\r
- test/dapltest/include/dapl_getopt.h \\r
- test/dapltest/include/dapl_global.h \\r
- test/dapltest/include/dapl_limit_cmd.h \\r
- test/dapltest/include/dapl_mdep.h \\r
- test/dapltest/include/dapl_memlist.h \\r
- test/dapltest/include/dapl_params.h \\r
- test/dapltest/include/dapl_performance_cmd.h \\r
- test/dapltest/include/dapl_performance_stats.h \\r
- test/dapltest/include/dapl_performance_test.h \\r
- test/dapltest/include/dapl_proto.h \\r
- test/dapltest/include/dapl_quit_cmd.h \\r
- test/dapltest/include/dapl_server_cmd.h \\r
- test/dapltest/include/dapl_server_info.h \\r
- test/dapltest/include/dapl_tdep.h \\r
- test/dapltest/include/dapl_tdep_print.h \\r
- test/dapltest/include/dapl_test_data.h \\r
- test/dapltest/include/dapl_transaction_cmd.h \\r
- test/dapltest/include/dapl_transaction_stats.h \\r
- test/dapltest/include/dapl_transaction_test.h \\r
- test/dapltest/include/dapl_version.h \\r
- test/dapltest/mdep/linux/dapl_mdep_user.h\r
- \r
-dist-hook: dapl.spec \r
- cp dapl.spec $(distdir)\r
- \r
-install-exec-hook:\r
- if ! test -d $(DESTDIR)$(sysconfdir); then \\r
- mkdir -p $(DESTDIR)$(sysconfdir); \\r
- fi; \\r
- if test -e $(DESTDIR)$(sysconfdir)/dat.conf; then \\r
- sed -e '/ofa-v2-.* u2/d' < $(DESTDIR)$(sysconfdir)/dat.conf > /tmp/$$$$ofadapl; \\r
- cp /tmp/$$$$ofadapl $(DESTDIR)$(sysconfdir)/dat.conf; \\r
- fi; \\r
- echo ofa-v2-mlx4_0-1 u2.0 nonthreadsafe default libdaploscm.so.2 dapl.2.0 '"mlx4_0 1" ""' >> $(DESTDIR)$(sysconfdir)/dat.conf; \\r
- echo ofa-v2-mlx4_0-2 u2.0 nonthreadsafe default libdaploscm.so.2 dapl.2.0 '"mlx4_0 2" ""' >> $(DESTDIR)$(sysconfdir)/dat.conf; \\r
- echo ofa-v2-ib0 u2.0 nonthreadsafe default libdaplofa.so.2 dapl.2.0 '"ib0 0" ""' >> $(DESTDIR)$(sysconfdir)/dat.conf; \\r
- echo ofa-v2-ib1 u2.0 nonthreadsafe default libdaplofa.so.2 dapl.2.0 '"ib1 0" ""' >> $(DESTDIR)$(sysconfdir)/dat.conf; \\r
- echo ofa-v2-mthca0-1 u2.0 nonthreadsafe default libdaploscm.so.2 dapl.2.0 '"mthca0 1" ""' >> $(DESTDIR)$(sysconfdir)/dat.conf; \\r
- echo ofa-v2-mthca0-2 u2.0 nonthreadsafe default libdaploscm.so.2 dapl.2.0 '"mthca0 2" ""' >> $(DESTDIR)$(sysconfdir)/dat.conf; \\r
- echo ofa-v2-ipath0-1 u2.0 nonthreadsafe default libdaploscm.so.2 dapl.2.0 '"ipath0 1" ""' >> $(DESTDIR)$(sysconfdir)/dat.conf; \\r
- echo ofa-v2-ipath0-2 u2.0 nonthreadsafe default libdaploscm.so.2 dapl.2.0 '"ipath0 2" ""' >> $(DESTDIR)$(sysconfdir)/dat.conf; \\r
- echo ofa-v2-ehca0-2 u2.0 nonthreadsafe default libdaploscm.so.2 dapl.2.0 '"ehca0 1" ""' >> $(DESTDIR)$(sysconfdir)/dat.conf; \\r
- echo ofa-v2-iwarp u2.0 nonthreadsafe default libdaplofa.so.2 dapl.2.0 '"eth2 0" ""' >> $(DESTDIR)$(sysconfdir)/dat.conf; \\r
- echo ofa-v2-mlx4_0-1u u2.0 nonthreadsafe default libdaploucm.so.2 dapl.2.0 '"mlx4_0 1" ""' >> $(DESTDIR)$(sysconfdir)/dat.conf; \\r
- echo ofa-v2-mlx4_0-2u u2.0 nonthreadsafe default libdaploucm.so.2 dapl.2.0 '"mlx4_0 2" ""' >> $(DESTDIR)$(sysconfdir)/dat.conf; \\r
- echo ofa-v2-mthca0-1u u2.0 nonthreadsafe default libdaploucm.so.2 dapl.2.0 '"mthca0 1" ""' >> $(DESTDIR)$(sysconfdir)/dat.conf; \\r
- echo ofa-v2-mthca0-2u u2.0 nonthreadsafe default libdaploucm.so.2 dapl.2.0 '"mthca0 2" ""' >> $(DESTDIR)$(sysconfdir)/dat.conf;\r
-\r
-uninstall-hook:\r
- if test -e $(DESTDIR)$(sysconfdir)/dat.conf; then \\r
- sed -e '/ofa-v2-.* u2/d' < $(DESTDIR)$(sysconfdir)/dat.conf > /tmp/$$$$ofadapl; \\r
- cp /tmp/$$$$ofadapl $(DESTDIR)$(sysconfdir)/dat.conf; \\r
- fi;\r
-\r
-SUBDIRS = . test/dtest test/dapltest\r
+# $Id: $
+
+OSFLAGS = -DOS_RELEASE=$(shell expr `uname -r | cut -f1 -d.` \* 65536 + `uname -r | cut -f2 -d.`)
+# Check for RedHat, needed for ia64 udapl atomic operations (IA64_FETCHADD syntax)
+# and built-in atomics for RedHat EL5
+if OS_RHEL4
+OSFLAGS += -DREDHAT_EL4
+endif
+
+if OS_RHEL5
+OSFLAGS += -DREDHAT_EL5
+endif
+
+if OS_SUSE11
+OSFLAGS += -DSUSE_11
+endif
+
+if EXT_TYPE_IB
+XFLAGS = -DDAT_EXTENSIONS
+XPROGRAMS = dapl/openib_common/ib_extensions.c
+else
+XFLAGS =
+XPROGRAMS =
+endif
+
+if DEBUG
+AM_CFLAGS = -g -Wall -D_GNU_SOURCE -DDAPL_DBG -DDAT_CONF="\"$(sysconfdir)/dat.conf\""
+else
+AM_CFLAGS = -g -Wall -D_GNU_SOURCE -DDAT_CONF="\"$(sysconfdir)/dat.conf\""
+endif
+
+datlibdir = $(libdir)
+dapllibofadir = $(libdir)
+daplliboscmdir = $(libdir)
+daplliboucmdir = $(libdir)
+
+datlib_LTLIBRARIES = dat/udat/libdat2.la
+dapllibofa_LTLIBRARIES = dapl/udapl/libdaplofa.la
+daplliboscm_LTLIBRARIES = dapl/udapl/libdaploscm.la
+daplliboucm_LTLIBRARIES = dapl/udapl/libdaploucm.la
+
+dat_udat_libdat2_la_CFLAGS = $(AM_CFLAGS) -D_GNU_SOURCE $(OSFLAGS) $(XFLAGS) \
+ -I$(srcdir)/dat/include/ -I$(srcdir)/dat/udat/ \
+ -I$(srcdir)/dat/udat/linux -I$(srcdir)/dat/common/
+
+dapl_udapl_libdaplofa_la_CFLAGS = $(AM_CFLAGS) -D_GNU_SOURCE $(OSFLAGS) $(XFLAGS) \
+ -DOPENIB -DCQ_WAIT_OBJECT \
+ -I$(srcdir)/dat/include/ -I$(srcdir)/dapl/include/ \
+ -I$(srcdir)/dapl/common -I$(srcdir)/dapl/udapl/linux \
+ -I$(srcdir)/dapl/openib_common \
+ -I$(srcdir)/dapl/openib_cma \
+ -I$(srcdir)/dapl/openib_cma/linux
+
+dapl_udapl_libdaploscm_la_CFLAGS = $(AM_CFLAGS) -D_GNU_SOURCE $(OSFLAGS) $(XFLAGS) \
+ -DOPENIB -DCQ_WAIT_OBJECT \
+ -I$(srcdir)/dat/include/ -I$(srcdir)/dapl/include/ \
+ -I$(srcdir)/dapl/common -I$(srcdir)/dapl/udapl/linux \
+ -I$(srcdir)/dapl/openib_common \
+ -I$(srcdir)/dapl/openib_scm \
+ -I$(srcdir)/dapl/openib_scm/linux
+
+dapl_udapl_libdaploucm_la_CFLAGS = $(AM_CFLAGS) -D_GNU_SOURCE $(OSFLAGS) $(XFLAGS) \
+ -DOPENIB -DCQ_WAIT_OBJECT \
+ -I$(srcdir)/dat/include/ -I$(srcdir)/dapl/include/ \
+ -I$(srcdir)/dapl/common -I$(srcdir)/dapl/udapl/linux \
+ -I$(srcdir)/dapl/openib_common \
+ -I$(srcdir)/dapl/openib_ucm \
+ -I$(srcdir)/dapl/openib_ucm/linux
+
+if HAVE_LD_VERSION_SCRIPT
+ dat_version_script = -Wl,--version-script=$(srcdir)/dat/udat/libdat2.map
+ daplofa_version_script = -Wl,--version-script=$(srcdir)/dapl/udapl/libdaplofa.map
+ daploscm_version_script = -Wl,--version-script=$(srcdir)/dapl/udapl/libdaploscm.map
+ daploucm_version_script = -Wl,--version-script=$(srcdir)/dapl/udapl/libdaploucm.map
+else
+ dat_version_script =
+ daplofa_version_script =
+ daploscm_version_script =
+ daploucm_version_script =
+endif
+
+#
+# uDAT: libdat2.so
+#
+dat_udat_libdat2_la_SOURCES = dat/udat/udat.c \
+ dat/udat/udat_api.c \
+ dat/udat/udat_sr_parser.c \
+ dat/udat/linux/dat_osd.c \
+ dat/common/dat_api.c \
+ dat/common/dat_dictionary.c \
+ dat/common/dat_strerror.c \
+ dat/common/dat_init.c \
+ dat/common/dat_dr.c \
+ dat/common/dat_sr.c
+dat_udat_libdat2_la_LDFLAGS = -version-info 2:0:0 $(dat_version_script) -ldl
+
+#
+# uDAPL OpenFabrics rdma_cm version: libdaplofa.so
+#
+dapl_udapl_libdaplofa_la_SOURCES = dapl/udapl/dapl_init.c \
+ dapl/udapl/dapl_evd_create.c \
+ dapl/udapl/dapl_evd_query.c \
+ dapl/udapl/dapl_cno_create.c \
+ dapl/udapl/dapl_cno_modify_agent.c \
+ dapl/udapl/dapl_cno_free.c \
+ dapl/udapl/dapl_cno_wait.c \
+ dapl/udapl/dapl_cno_query.c \
+ dapl/udapl/dapl_lmr_create.c \
+ dapl/udapl/dapl_evd_wait.c \
+ dapl/udapl/dapl_evd_disable.c \
+ dapl/udapl/dapl_evd_enable.c \
+ dapl/udapl/dapl_evd_modify_cno.c \
+ dapl/udapl/dapl_evd_set_unwaitable.c \
+ dapl/udapl/dapl_evd_clear_unwaitable.c \
+ dapl/udapl/linux/dapl_osd.c \
+ dapl/common/dapl_cookie.c \
+ dapl/common/dapl_cr_accept.c \
+ dapl/common/dapl_cr_query.c \
+ dapl/common/dapl_cr_reject.c \
+ dapl/common/dapl_cr_util.c \
+ dapl/common/dapl_cr_callback.c \
+ dapl/common/dapl_cr_handoff.c \
+ dapl/common/dapl_ep_connect.c \
+ dapl/common/dapl_ep_create.c \
+ dapl/common/dapl_ep_disconnect.c \
+ dapl/common/dapl_ep_dup_connect.c \
+ dapl/common/dapl_ep_free.c \
+ dapl/common/dapl_ep_reset.c \
+ dapl/common/dapl_ep_get_status.c \
+ dapl/common/dapl_ep_modify.c \
+ dapl/common/dapl_ep_post_rdma_read.c \
+ dapl/common/dapl_ep_post_rdma_write.c \
+ dapl/common/dapl_ep_post_recv.c \
+ dapl/common/dapl_ep_post_send.c \
+ dapl/common/dapl_ep_query.c \
+ dapl/common/dapl_ep_util.c \
+ dapl/common/dapl_evd_dequeue.c \
+ dapl/common/dapl_evd_free.c \
+ dapl/common/dapl_evd_post_se.c \
+ dapl/common/dapl_evd_resize.c \
+ dapl/common/dapl_evd_util.c \
+ dapl/common/dapl_evd_cq_async_error_callb.c \
+ dapl/common/dapl_evd_qp_async_error_callb.c \
+ dapl/common/dapl_evd_un_async_error_callb.c \
+ dapl/common/dapl_evd_connection_callb.c \
+ dapl/common/dapl_evd_dto_callb.c \
+ dapl/common/dapl_get_consumer_context.c \
+ dapl/common/dapl_get_handle_type.c \
+ dapl/common/dapl_hash.c \
+ dapl/common/dapl_hca_util.c \
+ dapl/common/dapl_ia_close.c \
+ dapl/common/dapl_ia_open.c \
+ dapl/common/dapl_ia_query.c \
+ dapl/common/dapl_ia_util.c \
+ dapl/common/dapl_llist.c \
+ dapl/common/dapl_lmr_free.c \
+ dapl/common/dapl_lmr_query.c \
+ dapl/common/dapl_lmr_util.c \
+ dapl/common/dapl_lmr_sync_rdma_read.c \
+ dapl/common/dapl_lmr_sync_rdma_write.c \
+ dapl/common/dapl_mr_util.c \
+ dapl/common/dapl_provider.c \
+ dapl/common/dapl_sp_util.c \
+ dapl/common/dapl_psp_create.c \
+ dapl/common/dapl_psp_create_any.c \
+ dapl/common/dapl_psp_free.c \
+ dapl/common/dapl_psp_query.c \
+ dapl/common/dapl_pz_create.c \
+ dapl/common/dapl_pz_free.c \
+ dapl/common/dapl_pz_query.c \
+ dapl/common/dapl_pz_util.c \
+ dapl/common/dapl_rmr_create.c \
+ dapl/common/dapl_rmr_free.c \
+ dapl/common/dapl_rmr_bind.c \
+ dapl/common/dapl_rmr_query.c \
+ dapl/common/dapl_rmr_util.c \
+ dapl/common/dapl_rsp_create.c \
+ dapl/common/dapl_rsp_free.c \
+ dapl/common/dapl_rsp_query.c \
+ dapl/common/dapl_cno_util.c \
+ dapl/common/dapl_set_consumer_context.c \
+ dapl/common/dapl_ring_buffer_util.c \
+ dapl/common/dapl_name_service.c \
+ dapl/common/dapl_timer_util.c \
+ dapl/common/dapl_ep_create_with_srq.c \
+ dapl/common/dapl_ep_recv_query.c \
+ dapl/common/dapl_ep_set_watermark.c \
+ dapl/common/dapl_srq_create.c \
+ dapl/common/dapl_srq_free.c \
+ dapl/common/dapl_srq_query.c \
+ dapl/common/dapl_srq_resize.c \
+ dapl/common/dapl_srq_post_recv.c \
+ dapl/common/dapl_srq_set_lw.c \
+ dapl/common/dapl_srq_util.c \
+ dapl/common/dapl_debug.c \
+ dapl/common/dapl_ia_ha.c \
+ dapl/common/dapl_csp.c \
+ dapl/common/dapl_ep_post_send_invalidate.c \
+ dapl/common/dapl_ep_post_rdma_read_to_rmr.c \
+ dapl/openib_common/mem.c \
+ dapl/openib_common/cq.c \
+ dapl/openib_common/qp.c \
+ dapl/openib_common/util.c \
+ dapl/openib_cma/cm.c \
+ dapl/openib_cma/device.c $(XPROGRAMS)
+
+dapl_udapl_libdaplofa_la_LDFLAGS = -version-info 2:0:0 $(daplofa_version_script) \
+ -Wl,-init,dapl_init -Wl,-fini,dapl_fini \
+ -lpthread -libverbs -lrdmacm
+
+#
+# uDAPL OpenFabrics Socket CM version for IB: libdaplscm.so
+#
+dapl_udapl_libdaploscm_la_SOURCES = dapl/udapl/dapl_init.c \
+ dapl/udapl/dapl_evd_create.c \
+ dapl/udapl/dapl_evd_query.c \
+ dapl/udapl/dapl_cno_create.c \
+ dapl/udapl/dapl_cno_modify_agent.c \
+ dapl/udapl/dapl_cno_free.c \
+ dapl/udapl/dapl_cno_wait.c \
+ dapl/udapl/dapl_cno_query.c \
+ dapl/udapl/dapl_lmr_create.c \
+ dapl/udapl/dapl_evd_wait.c \
+ dapl/udapl/dapl_evd_disable.c \
+ dapl/udapl/dapl_evd_enable.c \
+ dapl/udapl/dapl_evd_modify_cno.c \
+ dapl/udapl/dapl_evd_set_unwaitable.c \
+ dapl/udapl/dapl_evd_clear_unwaitable.c \
+ dapl/udapl/linux/dapl_osd.c \
+ dapl/common/dapl_cookie.c \
+ dapl/common/dapl_cr_accept.c \
+ dapl/common/dapl_cr_query.c \
+ dapl/common/dapl_cr_reject.c \
+ dapl/common/dapl_cr_util.c \
+ dapl/common/dapl_cr_callback.c \
+ dapl/common/dapl_cr_handoff.c \
+ dapl/common/dapl_ep_connect.c \
+ dapl/common/dapl_ep_create.c \
+ dapl/common/dapl_ep_disconnect.c \
+ dapl/common/dapl_ep_dup_connect.c \
+ dapl/common/dapl_ep_free.c \
+ dapl/common/dapl_ep_reset.c \
+ dapl/common/dapl_ep_get_status.c \
+ dapl/common/dapl_ep_modify.c \
+ dapl/common/dapl_ep_post_rdma_read.c \
+ dapl/common/dapl_ep_post_rdma_write.c \
+ dapl/common/dapl_ep_post_recv.c \
+ dapl/common/dapl_ep_post_send.c \
+ dapl/common/dapl_ep_query.c \
+ dapl/common/dapl_ep_util.c \
+ dapl/common/dapl_evd_dequeue.c \
+ dapl/common/dapl_evd_free.c \
+ dapl/common/dapl_evd_post_se.c \
+ dapl/common/dapl_evd_resize.c \
+ dapl/common/dapl_evd_util.c \
+ dapl/common/dapl_evd_cq_async_error_callb.c \
+ dapl/common/dapl_evd_qp_async_error_callb.c \
+ dapl/common/dapl_evd_un_async_error_callb.c \
+ dapl/common/dapl_evd_connection_callb.c \
+ dapl/common/dapl_evd_dto_callb.c \
+ dapl/common/dapl_get_consumer_context.c \
+ dapl/common/dapl_get_handle_type.c \
+ dapl/common/dapl_hash.c \
+ dapl/common/dapl_hca_util.c \
+ dapl/common/dapl_ia_close.c \
+ dapl/common/dapl_ia_open.c \
+ dapl/common/dapl_ia_query.c \
+ dapl/common/dapl_ia_util.c \
+ dapl/common/dapl_llist.c \
+ dapl/common/dapl_lmr_free.c \
+ dapl/common/dapl_lmr_query.c \
+ dapl/common/dapl_lmr_util.c \
+ dapl/common/dapl_lmr_sync_rdma_read.c \
+ dapl/common/dapl_lmr_sync_rdma_write.c \
+ dapl/common/dapl_mr_util.c \
+ dapl/common/dapl_provider.c \
+ dapl/common/dapl_sp_util.c \
+ dapl/common/dapl_psp_create.c \
+ dapl/common/dapl_psp_create_any.c \
+ dapl/common/dapl_psp_free.c \
+ dapl/common/dapl_psp_query.c \
+ dapl/common/dapl_pz_create.c \
+ dapl/common/dapl_pz_free.c \
+ dapl/common/dapl_pz_query.c \
+ dapl/common/dapl_pz_util.c \
+ dapl/common/dapl_rmr_create.c \
+ dapl/common/dapl_rmr_free.c \
+ dapl/common/dapl_rmr_bind.c \
+ dapl/common/dapl_rmr_query.c \
+ dapl/common/dapl_rmr_util.c \
+ dapl/common/dapl_rsp_create.c \
+ dapl/common/dapl_rsp_free.c \
+ dapl/common/dapl_rsp_query.c \
+ dapl/common/dapl_cno_util.c \
+ dapl/common/dapl_set_consumer_context.c \
+ dapl/common/dapl_ring_buffer_util.c \
+ dapl/common/dapl_name_service.c \
+ dapl/common/dapl_timer_util.c \
+ dapl/common/dapl_ep_create_with_srq.c \
+ dapl/common/dapl_ep_recv_query.c \
+ dapl/common/dapl_ep_set_watermark.c \
+ dapl/common/dapl_srq_create.c \
+ dapl/common/dapl_srq_free.c \
+ dapl/common/dapl_srq_query.c \
+ dapl/common/dapl_srq_resize.c \
+ dapl/common/dapl_srq_post_recv.c \
+ dapl/common/dapl_srq_set_lw.c \
+ dapl/common/dapl_srq_util.c \
+ dapl/common/dapl_debug.c \
+ dapl/common/dapl_ia_ha.c \
+ dapl/common/dapl_csp.c \
+ dapl/common/dapl_ep_post_send_invalidate.c \
+ dapl/common/dapl_ep_post_rdma_read_to_rmr.c \
+ dapl/openib_common/mem.c \
+ dapl/openib_common/cq.c \
+ dapl/openib_common/qp.c \
+ dapl/openib_common/util.c \
+ dapl/openib_scm/cm.c \
+ dapl/openib_scm/device.c $(XPROGRAMS)
+
+dapl_udapl_libdaploscm_la_LDFLAGS = -version-info 2:0:0 $(daploscm_version_script) \
+ -Wl,-init,dapl_init -Wl,-fini,dapl_fini \
+ -lpthread -libverbs
+
+#
+# uDAPL OpenFabrics UD CM version for IB: libdaplucm.so
+#
+dapl_udapl_libdaploucm_la_SOURCES = dapl/udapl/dapl_init.c \
+ dapl/udapl/dapl_evd_create.c \
+ dapl/udapl/dapl_evd_query.c \
+ dapl/udapl/dapl_cno_create.c \
+ dapl/udapl/dapl_cno_modify_agent.c \
+ dapl/udapl/dapl_cno_free.c \
+ dapl/udapl/dapl_cno_wait.c \
+ dapl/udapl/dapl_cno_query.c \
+ dapl/udapl/dapl_lmr_create.c \
+ dapl/udapl/dapl_evd_wait.c \
+ dapl/udapl/dapl_evd_disable.c \
+ dapl/udapl/dapl_evd_enable.c \
+ dapl/udapl/dapl_evd_modify_cno.c \
+ dapl/udapl/dapl_evd_set_unwaitable.c \
+ dapl/udapl/dapl_evd_clear_unwaitable.c \
+ dapl/udapl/linux/dapl_osd.c \
+ dapl/common/dapl_cookie.c \
+ dapl/common/dapl_cr_accept.c \
+ dapl/common/dapl_cr_query.c \
+ dapl/common/dapl_cr_reject.c \
+ dapl/common/dapl_cr_util.c \
+ dapl/common/dapl_cr_callback.c \
+ dapl/common/dapl_cr_handoff.c \
+ dapl/common/dapl_ep_connect.c \
+ dapl/common/dapl_ep_create.c \
+ dapl/common/dapl_ep_disconnect.c \
+ dapl/common/dapl_ep_dup_connect.c \
+ dapl/common/dapl_ep_free.c \
+ dapl/common/dapl_ep_reset.c \
+ dapl/common/dapl_ep_get_status.c \
+ dapl/common/dapl_ep_modify.c \
+ dapl/common/dapl_ep_post_rdma_read.c \
+ dapl/common/dapl_ep_post_rdma_write.c \
+ dapl/common/dapl_ep_post_recv.c \
+ dapl/common/dapl_ep_post_send.c \
+ dapl/common/dapl_ep_query.c \
+ dapl/common/dapl_ep_util.c \
+ dapl/common/dapl_evd_dequeue.c \
+ dapl/common/dapl_evd_free.c \
+ dapl/common/dapl_evd_post_se.c \
+ dapl/common/dapl_evd_resize.c \
+ dapl/common/dapl_evd_util.c \
+ dapl/common/dapl_evd_cq_async_error_callb.c \
+ dapl/common/dapl_evd_qp_async_error_callb.c \
+ dapl/common/dapl_evd_un_async_error_callb.c \
+ dapl/common/dapl_evd_connection_callb.c \
+ dapl/common/dapl_evd_dto_callb.c \
+ dapl/common/dapl_get_consumer_context.c \
+ dapl/common/dapl_get_handle_type.c \
+ dapl/common/dapl_hash.c \
+ dapl/common/dapl_hca_util.c \
+ dapl/common/dapl_ia_close.c \
+ dapl/common/dapl_ia_open.c \
+ dapl/common/dapl_ia_query.c \
+ dapl/common/dapl_ia_util.c \
+ dapl/common/dapl_llist.c \
+ dapl/common/dapl_lmr_free.c \
+ dapl/common/dapl_lmr_query.c \
+ dapl/common/dapl_lmr_util.c \
+ dapl/common/dapl_lmr_sync_rdma_read.c \
+ dapl/common/dapl_lmr_sync_rdma_write.c \
+ dapl/common/dapl_mr_util.c \
+ dapl/common/dapl_provider.c \
+ dapl/common/dapl_sp_util.c \
+ dapl/common/dapl_psp_create.c \
+ dapl/common/dapl_psp_create_any.c \
+ dapl/common/dapl_psp_free.c \
+ dapl/common/dapl_psp_query.c \
+ dapl/common/dapl_pz_create.c \
+ dapl/common/dapl_pz_free.c \
+ dapl/common/dapl_pz_query.c \
+ dapl/common/dapl_pz_util.c \
+ dapl/common/dapl_rmr_create.c \
+ dapl/common/dapl_rmr_free.c \
+ dapl/common/dapl_rmr_bind.c \
+ dapl/common/dapl_rmr_query.c \
+ dapl/common/dapl_rmr_util.c \
+ dapl/common/dapl_rsp_create.c \
+ dapl/common/dapl_rsp_free.c \
+ dapl/common/dapl_rsp_query.c \
+ dapl/common/dapl_cno_util.c \
+ dapl/common/dapl_set_consumer_context.c \
+ dapl/common/dapl_ring_buffer_util.c \
+ dapl/common/dapl_name_service.c \
+ dapl/common/dapl_timer_util.c \
+ dapl/common/dapl_ep_create_with_srq.c \
+ dapl/common/dapl_ep_recv_query.c \
+ dapl/common/dapl_ep_set_watermark.c \
+ dapl/common/dapl_srq_create.c \
+ dapl/common/dapl_srq_free.c \
+ dapl/common/dapl_srq_query.c \
+ dapl/common/dapl_srq_resize.c \
+ dapl/common/dapl_srq_post_recv.c \
+ dapl/common/dapl_srq_set_lw.c \
+ dapl/common/dapl_srq_util.c \
+ dapl/common/dapl_debug.c \
+ dapl/common/dapl_ia_ha.c \
+ dapl/common/dapl_csp.c \
+ dapl/common/dapl_ep_post_send_invalidate.c \
+ dapl/common/dapl_ep_post_rdma_read_to_rmr.c \
+ dapl/openib_common/mem.c \
+ dapl/openib_common/cq.c \
+ dapl/openib_common/qp.c \
+ dapl/openib_common/util.c \
+ dapl/openib_ucm/cm.c \
+ dapl/openib_ucm/device.c $(XPROGRAMS)
+
+dapl_udapl_libdaploucm_la_LDFLAGS = -version-info 2:0:0 $(daploscm_version_script) \
+ -Wl,-init,dapl_init -Wl,-fini,dapl_fini \
+ -lpthread -libverbs
+
+libdatincludedir = $(includedir)/dat2
+
+libdatinclude_HEADERS = dat/include/dat2/dat.h \
+ dat/include/dat2/dat_error.h \
+ dat/include/dat2/dat_platform_specific.h \
+ dat/include/dat2/dat_redirection.h \
+ dat/include/dat2/dat_registry.h \
+ dat/include/dat2/dat_vendor_specific.h \
+ dat/include/dat2/udat_config.h \
+ dat/include/dat2/udat.h \
+ dat/include/dat2/udat_redirection.h \
+ dat/include/dat2/udat_vendor_specific.h \
+ dat/include/dat2/dat_ib_extensions.h
+
+man_MANS = man/dtest.1 man/dapltest.1 man/dat.conf.5
+
+EXTRA_DIST = dat/common/dat_dictionary.h \
+ dat/common/dat_dr.h \
+ dat/common/dat_init.h \
+ dat/common/dat_sr.h \
+ dat/udat/udat_sr_parser.h \
+ dat/udat/linux/dat_osd.h \
+ dat/include/dat2/dat.h \
+ dat/include/dat2/dat_error.h \
+ dat/include/dat2/dat_platform_specific.h \
+ dat/include/dat2/dat_redirection.h \
+ dat/include/dat2/dat_registry.h \
+ dat/include/dat2/dat_vendor_specific.h \
+ dat/include/dat2/udat_config.h \
+ dat/include/dat2/udat.h \
+ dat/include/dat2/udat_redirection.h \
+ dat/include/dat2/udat_vendor_specific.h \
+ dapl/common/dapl_adapter_util.h \
+ dapl/common/dapl_cno_util.h \
+ dapl/common/dapl_cookie.h \
+ dapl/common/dapl_cr_util.h \
+ dapl/common/dapl_ep_util.h \
+ dapl/common/dapl_evd_util.h \
+ dapl/common/dapl_hash.h \
+ dapl/common/dapl_hca_util.h \
+ dapl/common/dapl_ia_util.h \
+ dapl/common/dapl_init.h \
+ dapl/common/dapl_lmr_util.h \
+ dapl/common/dapl_mr_util.h \
+ dapl/common/dapl_name_service.h \
+ dapl/common/dapl_provider.h \
+ dapl/common/dapl_pz_util.h \
+ dapl/common/dapl_ring_buffer_util.h \
+ dapl/common/dapl_rmr_util.h \
+ dapl/common/dapl_sp_util.h \
+ dapl/common/dapl_srq_util.h \
+ dapl/common/dapl_timer_util.h \
+ dapl/udapl/linux/dapl_osd.h \
+ dapl/include/dapl.h \
+ dapl/include/dapl_debug.h \
+ dapl/include/dapl_ipoib_names.h \
+ dapl/include/dapl_vendor.h \
+ dapl/openib_common/dapl_ib_dto.h \
+ dapl/openib_common/dapl_ib_common.h \
+ dapl/openib_cma/dapl_ib_util.h \
+ dapl/openib_cma/linux/openib_osd.h \
+ dapl/openib_scm/dapl_ib_util.h \
+ dapl/openib_scm/linux/openib_osd.h \
+ dapl/openib_ucm/dapl_ib_util.h \
+ dapl/openib_ucm/linux/openib_osd.h \
+ dat/udat/libdat2.map \
+ dapl/udapl/libdaplofa.map \
+ dapl/udapl/libdaploscm.map \
+ dapl/udapl/libdaploucm.map \
+ dapl.spec.in \
+ $(man_MANS) \
+ test/dapltest/include/dapl_bpool.h \
+ test/dapltest/include/dapl_client_info.h \
+ test/dapltest/include/dapl_common.h \
+ test/dapltest/include/dapl_execute.h \
+ test/dapltest/include/dapl_fft_cmd.h \
+ test/dapltest/include/dapl_fft_util.h \
+ test/dapltest/include/dapl_getopt.h \
+ test/dapltest/include/dapl_global.h \
+ test/dapltest/include/dapl_limit_cmd.h \
+ test/dapltest/include/dapl_mdep.h \
+ test/dapltest/include/dapl_memlist.h \
+ test/dapltest/include/dapl_params.h \
+ test/dapltest/include/dapl_performance_cmd.h \
+ test/dapltest/include/dapl_performance_stats.h \
+ test/dapltest/include/dapl_performance_test.h \
+ test/dapltest/include/dapl_proto.h \
+ test/dapltest/include/dapl_quit_cmd.h \
+ test/dapltest/include/dapl_server_cmd.h \
+ test/dapltest/include/dapl_server_info.h \
+ test/dapltest/include/dapl_tdep.h \
+ test/dapltest/include/dapl_tdep_print.h \
+ test/dapltest/include/dapl_test_data.h \
+ test/dapltest/include/dapl_transaction_cmd.h \
+ test/dapltest/include/dapl_transaction_stats.h \
+ test/dapltest/include/dapl_transaction_test.h \
+ test/dapltest/include/dapl_version.h \
+ test/dapltest/mdep/linux/dapl_mdep_user.h
+
+dist-hook: dapl.spec
+ cp dapl.spec $(distdir)
+
+install-exec-hook:
+ if ! test -d $(DESTDIR)$(sysconfdir); then \
+ mkdir -p $(DESTDIR)$(sysconfdir); \
+ fi; \
+ if test -e $(DESTDIR)$(sysconfdir)/dat.conf; then \
+ sed -e '/ofa-v2-.* u2/d' < $(DESTDIR)$(sysconfdir)/dat.conf > /tmp/$$$$ofadapl; \
+ cp /tmp/$$$$ofadapl $(DESTDIR)$(sysconfdir)/dat.conf; \
+ fi; \
+ echo ofa-v2-mlx4_0-1 u2.0 nonthreadsafe default libdaploscm.so.2 dapl.2.0 '"mlx4_0 1" ""' >> $(DESTDIR)$(sysconfdir)/dat.conf; \
+ echo ofa-v2-mlx4_0-2 u2.0 nonthreadsafe default libdaploscm.so.2 dapl.2.0 '"mlx4_0 2" ""' >> $(DESTDIR)$(sysconfdir)/dat.conf; \
+ echo ofa-v2-ib0 u2.0 nonthreadsafe default libdaplofa.so.2 dapl.2.0 '"ib0 0" ""' >> $(DESTDIR)$(sysconfdir)/dat.conf; \
+ echo ofa-v2-ib1 u2.0 nonthreadsafe default libdaplofa.so.2 dapl.2.0 '"ib1 0" ""' >> $(DESTDIR)$(sysconfdir)/dat.conf; \
+ echo ofa-v2-mthca0-1 u2.0 nonthreadsafe default libdaploscm.so.2 dapl.2.0 '"mthca0 1" ""' >> $(DESTDIR)$(sysconfdir)/dat.conf; \
+ echo ofa-v2-mthca0-2 u2.0 nonthreadsafe default libdaploscm.so.2 dapl.2.0 '"mthca0 2" ""' >> $(DESTDIR)$(sysconfdir)/dat.conf; \
+ echo ofa-v2-ipath0-1 u2.0 nonthreadsafe default libdaploscm.so.2 dapl.2.0 '"ipath0 1" ""' >> $(DESTDIR)$(sysconfdir)/dat.conf; \
+ echo ofa-v2-ipath0-2 u2.0 nonthreadsafe default libdaploscm.so.2 dapl.2.0 '"ipath0 2" ""' >> $(DESTDIR)$(sysconfdir)/dat.conf; \
+ echo ofa-v2-ehca0-2 u2.0 nonthreadsafe default libdaploscm.so.2 dapl.2.0 '"ehca0 1" ""' >> $(DESTDIR)$(sysconfdir)/dat.conf; \
+ echo ofa-v2-iwarp u2.0 nonthreadsafe default libdaplofa.so.2 dapl.2.0 '"eth2 0" ""' >> $(DESTDIR)$(sysconfdir)/dat.conf; \
+ echo ofa-v2-mlx4_0-1u u2.0 nonthreadsafe default libdaploucm.so.2 dapl.2.0 '"mlx4_0 1" ""' >> $(DESTDIR)$(sysconfdir)/dat.conf; \
+ echo ofa-v2-mlx4_0-2u u2.0 nonthreadsafe default libdaploucm.so.2 dapl.2.0 '"mlx4_0 2" ""' >> $(DESTDIR)$(sysconfdir)/dat.conf; \
+ echo ofa-v2-mthca0-1u u2.0 nonthreadsafe default libdaploucm.so.2 dapl.2.0 '"mthca0 1" ""' >> $(DESTDIR)$(sysconfdir)/dat.conf; \
+ echo ofa-v2-mthca0-2u u2.0 nonthreadsafe default libdaploucm.so.2 dapl.2.0 '"mthca0 2" ""' >> $(DESTDIR)$(sysconfdir)/dat.conf;
+
+uninstall-hook:
+ if test -e $(DESTDIR)$(sysconfdir)/dat.conf; then \
+ sed -e '/ofa-v2-.* u2/d' < $(DESTDIR)$(sysconfdir)/dat.conf > /tmp/$$$$ofadapl; \
+ cp /tmp/$$$$ofadapl $(DESTDIR)$(sysconfdir)/dat.conf; \
+ fi;
+
+SUBDIRS = . test/dtest test/dapltest
-dnl Process this file with autoconf to produce a configure script.\r
-\r
-AC_PREREQ(2.57)\r
-AC_INIT(dapl, 2.0.24, general@lists.openfabrics.org)\r
-AC_CONFIG_SRCDIR([dat/udat/udat.c])\r
-AC_CONFIG_AUX_DIR(config)\r
-AM_CONFIG_HEADER(config.h)\r
-AM_INIT_AUTOMAKE(dapl, 2.0.24)\r
-\r
-AM_PROG_LIBTOOL\r
-\r
-AC_ARG_ENABLE(libcheck, [ --disable-libcheck do not test for presence of ib libraries],\r
-[ if test x$enableval = xno ; then\r
- disable_libcheck=yes\r
- fi\r
-])\r
-\r
-dnl Checks for programs\r
-AC_PROG_CC\r
-\r
-dnl Checks for libraries\r
-if test "$disable_libcheck" != "yes"\r
-then\r
-AC_CHECK_LIB(ibverbs, ibv_get_device_list, [],\r
- AC_MSG_ERROR([ibv_get_device_list() not found. libdapl requires libibverbs.]))\r
-fi\r
-\r
-dnl Checks for header files.\r
-if test "$disable_libcheck" != "yes"\r
-then\r
-AC_CHECK_HEADER(infiniband/verbs.h, [],\r
- AC_MSG_ERROR([<infiniband/verbs.h> not found. Is libibverbs installed?]))\r
-fi\r
-\r
-AC_CACHE_CHECK(whether ld accepts --version-script, ac_cv_version_script,\r
- if test -n "`$LD --help < /dev/null 2>/dev/null | grep version-script`"; then\r
- ac_cv_version_script=yes\r
- else\r
- ac_cv_version_script=no\r
- fi)\r
-AM_CONDITIONAL(HAVE_LD_VERSION_SCRIPT, test "$ac_cv_version_script" = "yes")\r
-\r
-dnl Support debug mode build - if enable-debug provided the DEBUG variable is set \r
-AC_ARG_ENABLE(debug,\r
-[ --enable-debug Turn on debug mode, default=off],\r
-[case "${enableval}" in\r
- yes) debug=true ;;\r
- no) debug=false ;;\r
- *) AC_MSG_ERROR(bad value ${enableval} for --enable-debug) ;;\r
-esac],[debug=false])\r
-AM_CONDITIONAL(DEBUG, test x$debug = xtrue)\r
-\r
-dnl Support ib_extension build - if enable-ext-type == ib \r
-AC_ARG_ENABLE(ext-type,\r
-[ --enable-ext-type Enable extensions support for library: ib, none, default=ib],\r
- [ if test "x$enableval" = "xib" ; then\r
- ext_type=ib\r
- elif test "x$enableval" = "xnone" ; then\r
- ext_type=none\r
- else\r
- echo\r
- echo "Error!"\r
- echo "Unknown extension type' type"\r
- exit -1\r
- fi\r
- ],[ext_type=ib])\r
-AM_CONDITIONAL(EXT_TYPE_IB, test "$ext_type" = "ib")\r
-\r
-dnl Check for Redhat EL release 4\r
-AC_CACHE_CHECK(Check for RHEL4 system, ac_cv_rhel4,\r
- if test -f /etc/redhat-release &&\r
- test -n "`grep -e "release 4" /etc/redhat-release`"; then\r
- ac_cv_rhel4=yes\r
- else\r
- ac_cv_rhel4=no\r
- fi)\r
-AM_CONDITIONAL(OS_RHEL4, test "$ac_cv_rhel4" = "yes")\r
-\r
-dnl Check for Redhat EL release 5\r
-AC_CACHE_CHECK(Check for RHEL5 system, ac_cv_rhel5,\r
- if test -f /etc/redhat-release &&\r
- test -n "`grep -e "release 5" /etc/redhat-release`"; then\r
- ac_cv_rhel5=yes\r
- else\r
- ac_cv_rhel5=no\r
- fi)\r
-AM_CONDITIONAL(OS_RHEL5, test "$ac_cv_rhel5" = "yes")\r
-\r
-dnl Check for SuSE release 11\r
-AC_CACHE_CHECK(Check for SUSE_11 system, ac_cv_suse11,\r
- if test -f /etc/SuSE-release &&\r
- test -n "`grep -e "VERSION = 11" /etc/SuSE-release`"; then\r
- ac_cv_suse11=yes\r
- else\r
- ac_cv_suse11=no\r
- fi)\r
-AM_CONDITIONAL(OS_SUSE11, test "$ac_cv_suse11" = "yes")\r
-\r
-AC_CONFIG_FILES([Makefile test/dtest/Makefile test/dapltest/Makefile dapl.spec])\r
-\r
-AC_OUTPUT\r
+dnl Process this file with autoconf to produce a configure script.
+
+AC_PREREQ(2.57)
+AC_INIT(dapl, 2.0.25, linux-rdma@vger.kernel.org)
+AC_CONFIG_SRCDIR([dat/udat/udat.c])
+AC_CONFIG_AUX_DIR(config)
+AM_CONFIG_HEADER(config.h)
+AM_INIT_AUTOMAKE(dapl, 2.0.25)
+
+AM_PROG_LIBTOOL
+
+AC_ARG_ENABLE(libcheck, [ --disable-libcheck do not test for presence of ib libraries],
+[ if test x$enableval = xno ; then
+ disable_libcheck=yes
+ fi
+])
+
+dnl Checks for programs
+AC_PROG_CC
+
+dnl Checks for libraries
+if test "$disable_libcheck" != "yes"
+then
+AC_CHECK_LIB(ibverbs, ibv_get_device_list, [],
+ AC_MSG_ERROR([ibv_get_device_list() not found. libdapl requires libibverbs.]))
+fi
+
+dnl Checks for header files.
+if test "$disable_libcheck" != "yes"
+then
+AC_CHECK_HEADER(infiniband/verbs.h, [],
+ AC_MSG_ERROR([<infiniband/verbs.h> not found. Is libibverbs installed?]))
+fi
+
+AC_CACHE_CHECK(whether ld accepts --version-script, ac_cv_version_script,
+ if test -n "`$LD --help < /dev/null 2>/dev/null | grep version-script`"; then
+ ac_cv_version_script=yes
+ else
+ ac_cv_version_script=no
+ fi)
+AM_CONDITIONAL(HAVE_LD_VERSION_SCRIPT, test "$ac_cv_version_script" = "yes")
+
+dnl Support debug mode build - if enable-debug provided the DEBUG variable is set
+AC_ARG_ENABLE(debug,
+[ --enable-debug Turn on debug mode, default=off],
+[case "${enableval}" in
+ yes) debug=true ;;
+ no) debug=false ;;
+ *) AC_MSG_ERROR(bad value ${enableval} for --enable-debug) ;;
+esac],[debug=false])
+AM_CONDITIONAL(DEBUG, test x$debug = xtrue)
+
+dnl Support ib_extension build - if enable-ext-type == ib
+AC_ARG_ENABLE(ext-type,
+[ --enable-ext-type Enable extensions support for library: ib, none, default=ib],
+ [ if test "x$enableval" = "xib" ; then
+ ext_type=ib
+ elif test "x$enableval" = "xnone" ; then
+ ext_type=none
+ else
+ echo
+ echo "Error!"
+ echo "Unknown extension type' type"
+ exit -1
+ fi
+ ],[ext_type=ib])
+AM_CONDITIONAL(EXT_TYPE_IB, test "$ext_type" = "ib")
+
+dnl Check for Redhat EL release 4
+AC_CACHE_CHECK(Check for RHEL4 system, ac_cv_rhel4,
+ if test -f /etc/redhat-release &&
+ test -n "`grep -e "release 4" /etc/redhat-release`"; then
+ ac_cv_rhel4=yes
+ else
+ ac_cv_rhel4=no
+ fi)
+AM_CONDITIONAL(OS_RHEL4, test "$ac_cv_rhel4" = "yes")
+
+dnl Check for Redhat EL release 5
+AC_CACHE_CHECK(Check for RHEL5 system, ac_cv_rhel5,
+ if test -f /etc/redhat-release &&
+ test -n "`grep -e "release 5" /etc/redhat-release`"; then
+ ac_cv_rhel5=yes
+ else
+ ac_cv_rhel5=no
+ fi)
+AM_CONDITIONAL(OS_RHEL5, test "$ac_cv_rhel5" = "yes")
+
+dnl Check for SuSE release 11
+AC_CACHE_CHECK(Check for SUSE_11 system, ac_cv_suse11,
+ if test -f /etc/SuSE-release &&
+ test -n "`grep -e "VERSION = 11" /etc/SuSE-release`"; then
+ ac_cv_suse11=yes
+ else
+ ac_cv_suse11=no
+ fi)
+AM_CONDITIONAL(OS_SUSE11, test "$ac_cv_suse11" = "yes")
+
+AC_CONFIG_FILES([Makefile test/dtest/Makefile test/dapltest/Makefile dapl.spec])
+
+AC_OUTPUT
-# Copyright (c) 2002-2005, Network Appliance, Inc. All rights reserved.\r
-# Copyright (c) 2007, Intel Corporation. All rights reserved.\r
-#\r
-# This Software is licensed under one of the following licenses:\r
-#\r
-# 1) under the terms of the "Common Public License 1.0" a copy of which is\r
-# in the file LICENSE.txt in the root directory. The license is also\r
-# available from the Open Source Initiative, see\r
-# http://www.opensource.org/licenses/cpl.php.\r
-#\r
-# 2) under the terms of the "The BSD License" a copy of which is in the file\r
-# LICENSE2.txt in the root directory. The license is also available from\r
-# the Open Source Initiative, see\r
-# http://www.opensource.org/licenses/bsd-license.php.\r
-#\r
-# 3) under the terms of the "GNU General Public License (GPL) Version 2" a \r
-# copy of which is in the file LICENSE3.txt in the root directory. The \r
-# license is also available from the Open Source Initiative, see\r
-# http://www.opensource.org/licenses/gpl-license.php.\r
-#\r
-# Licensee has the right to choose one of the above licenses.\r
-#\r
-# Redistributions of source code must retain the above copyright\r
-# notice and one of the license notices.\r
-#\r
-# Redistributions in binary form must reproduce both the above copyright\r
-# notice, one of the license notices in the documentation\r
-# and/or other materials provided with the distribution.\r
-#\r
-#\r
-# uDAT and uDAPL 2.0 Registry RPM SPEC file\r
-#\r
-# $Id: $\r
-Name: dapl\r
-Version: @VERSION@\r
-Release: 1%{?dist}\r
-Summary: A Library for userspace access to RDMA devices using OS Agnostic DAT APIs.\r
-\r
-Group: System Environment/Libraries\r
-License: Dual GPL/BSD/CPL\r
-Url: http://openfabrics.org/\r
-Source: http://www.openfabrics.org/downloads/%{name}/%{name}-%{version}.tar.gz\r
-BuildRoot: %(mktemp -ud %{_tmppath}/%{name}-%{version}-%{release}-XXXXXX)\r
-Requires(post): /sbin/ldconfig\r
-Requires(postun): /sbin/ldconfig\r
-Requires(post): sed\r
-Requires(post): coreutils\r
-\r
-%description\r
-Along with the OpenFabrics kernel drivers, libdat and libdapl provides a userspace\r
-RDMA API that supports DAT 2.0 specification and IB transport extensions for\r
-atomic operations and rdma write with immediate data.\r
-\r
-%package devel\r
-Summary: Development files for the libdat and libdapl libraries\r
-Group: System Environment/Libraries\r
-\r
-%description devel\r
-Header files for libdat and libdapl library.\r
-\r
-%package devel-static\r
-Summary: Static development files for libdat and libdapl library\r
-Group: System Environment/Libraries\r
- \r
-%description devel-static\r
-Static libraries for libdat and libdapl library.\r
-\r
-%package utils\r
-Summary: Test suites for uDAPL library\r
-Group: System Environment/Libraries\r
-Requires: %{name} = %{version}-%{release}\r
-\r
-%description utils\r
-Useful test suites to validate uDAPL library API's.\r
-\r
-%prep\r
-%setup -q\r
-\r
-%build\r
-%configure --enable-ext-type=ib \r
-make %{?_smp_mflags}\r
-\r
-%install\r
-rm -rf %{buildroot}\r
-make DESTDIR=%{buildroot} install\r
-# remove unpackaged files from the buildroot\r
-rm -f %{buildroot}%{_libdir}/*.la\r
-rm -f %{buildroot}%{_sysconfdir}/*.conf\r
-\r
-%clean\r
-rm -rf %{buildroot}\r
-\r
-%post \r
-/sbin/ldconfig\r
-if [ -e %{_sysconfdir}/dat.conf ]; then\r
- sed -e '/ofa-v2-.* u2/d' < %{_sysconfdir}/dat.conf > /tmp/$$ofadapl\r
- mv /tmp/$$ofadapl %{_sysconfdir}/dat.conf\r
-fi\r
-echo ofa-v2-mlx4_0-1 u2.0 nonthreadsafe default libdaploscm.so.2 dapl.2.0 '"mlx4_0 1" ""' >> %{_sysconfdir}/dat.conf\r
-echo ofa-v2-mlx4_0-2 u2.0 nonthreadsafe default libdaploscm.so.2 dapl.2.0 '"mlx4_0 2" ""' >> %{_sysconfdir}/dat.conf\r
-echo ofa-v2-ib0 u2.0 nonthreadsafe default libdaplofa.so.2 dapl.2.0 '"ib0 0" ""' >> %{_sysconfdir}/dat.conf\r
-echo ofa-v2-ib1 u2.0 nonthreadsafe default libdaplofa.so.2 dapl.2.0 '"ib1 0" ""' >> %{_sysconfdir}/dat.conf\r
-echo ofa-v2-mthca0-1 u2.0 nonthreadsafe default libdaploscm.so.2 dapl.2.0 '"mthca0 1" ""' >> %{_sysconfdir}/dat.conf\r
-echo ofa-v2-mthca0-2 u2.0 nonthreadsafe default libdaploscm.so.2 dapl.2.0 '"mthca0 2" ""' >> %{_sysconfdir}/dat.conf\r
-echo ofa-v2-ipath0-1 u2.0 nonthreadsafe default libdaploscm.so.2 dapl.2.0 '"ipath0 1" ""' >> %{_sysconfdir}/dat.conf\r
-echo ofa-v2-ipath0-2 u2.0 nonthreadsafe default libdaploscm.so.2 dapl.2.0 '"ipath0 2" ""' >> %{_sysconfdir}/dat.conf\r
-echo ofa-v2-ehca0-1 u2.0 nonthreadsafe default libdaploscm.so.2 dapl.2.0 '"ehca0 1" ""' >> %{_sysconfdir}/dat.conf\r
-echo ofa-v2-iwarp u2.0 nonthreadsafe default libdaplofa.so.2 dapl.2.0 '"eth2 0" ""' >> %{_sysconfdir}/dat.conf\r
-echo ofa-v2-mlx4_0-1u u2.0 nonthreadsafe default libdaploucm.so.2 dapl.2.0 '"mlx4_0 1" ""' >> %{_sysconfdir}/dat.conf\r
-echo ofa-v2-mlx4_0-2u u2.0 nonthreadsafe default libdaploucm.so.2 dapl.2.0 '"mlx4_0 2" ""' >> %{_sysconfdir}/dat.conf\r
-echo ofa-v2-mthca0-1u u2.0 nonthreadsafe default libdaploucm.so.2 dapl.2.0 '"mthca0 1" ""' >> %{_sysconfdir}/dat.conf\r
-echo ofa-v2-mthca0-2u u2.0 nonthreadsafe default libdaploucm.so.2 dapl.2.0 '"mthca0 2" ""' >> %{_sysconfdir}/dat.conf\r
-\r
-%postun \r
-/sbin/ldconfig\r
-if [ -e %{_sysconfdir}/dat.conf ]; then\r
- sed -e '/ofa-v2-.* u2/d' < %{_sysconfdir}/dat.conf > /tmp/$$ofadapl\r
- mv /tmp/$$ofadapl %{_sysconfdir}/dat.conf\r
-fi\r
-\r
-%files\r
-%defattr(-,root,root,-)\r
-%{_libdir}/libda*.so.*\r
-%doc AUTHORS README ChangeLog\r
-\r
-%files devel\r
-%defattr(-,root,root,-)\r
-%{_libdir}/*.so\r
-%dir %{_includedir}/dat2\r
-%{_includedir}/dat2/*\r
-\r
-%files devel-static\r
-%defattr(-,root,root,-)\r
-%{_libdir}/*.a\r
-\r
-%files utils\r
-%defattr(-,root,root,-)\r
-%{_bindir}/*\r
-%{_mandir}/man1/*.1*\r
-%{_mandir}/man5/*.5*\r
-\r
-%changelog\r
-* Fri Oct 30 2009 Arlin Davis <ardavis@ichips.intel.com> - 2.0.24\r
-- DAT/DAPL Version 2.0.24 Release 1, OFED 1.5 RC2 \r
-\r
-* Fri Oct 2 2009 Arlin Davis <ardavis@ichips.intel.com> - 2.0.23\r
-- DAT/DAPL Version 2.0.23 Release 1, OFED 1.5 RC1 \r
-\r
-* Wed Aug 19 2009 Arlin Davis <ardavis@ichips.intel.com> - 2.0.22\r
-- DAT/DAPL Version 2.0.22 Release 1, OFED 1.5 ALPHA new UCM provider \r
-\r
-* Wed Aug 5 2009 Arlin Davis <ardavis@ichips.intel.com> - 2.0.21\r
-- DAT/DAPL Version 2.0.21 Release 1, WinOF 2.1, OFED 1.4.1+ \r
-\r
-* Fri Jun 19 2009 Arlin Davis <ardavis@ichips.intel.com> - 2.0.20\r
-- DAT/DAPL Version 2.0.20 Release 1, OFED 1.4.1 + UD reject/scaling fixes \r
-\r
-* Thu Apr 30 2009 Arlin Davis <ardavis@ichips.intel.com> - 2.0.19\r
-- DAT/DAPL Version 2.0.19 Release 1, OFED 1.4.1 GA Final \r
-\r
-* Fri Apr 17 2009 Arlin Davis <ardavis@ichips.intel.com> - 2.0.18\r
-- DAT/DAPL Version 2.0.18 Release 1, OFED 1.4.1 GA \r
-\r
-* Tue Mar 31 2009 Arlin Davis <ardavis@ichips.intel.com> - 2.0.17\r
-- DAT/DAPL Version 2.0.17 Release 1, OFED 1.4.1 GA\r
-\r
-* Mon Mar 16 2009 Arlin Davis <ardavis@ichips.intel.com> - 2.0.16\r
-- DAT/DAPL Version 2.0.16 Release 1, OFED 1.4.1 \r
-\r
-* Fri Nov 07 2008 Arlin Davis <ardavis@ichips.intel.com> - 2.0.15\r
-- DAT/DAPL Version 2.0.15 Release 1, OFED 1.4 GA\r
-\r
-* Fri Oct 03 2008 Arlin Davis <ardavis@ichips.intel.com> - 2.0.14\r
-- DAT/DAPL Version 2.0.14 Release 1, OFED 1.4 rc3\r
-\r
-* Mon Sep 01 2008 Arlin Davis <ardavis@ichips.intel.com> - 2.0.13\r
-- DAT/DAPL Version 2.0.13 Release 1, OFED 1.4 rc1\r
-\r
-* Thu Aug 21 2008 Arlin Davis <ardavis@ichips.intel.com> - 2.0.12\r
-- DAT/DAPL Version 2.0.12 Release 1, OFED 1.4 beta\r
-\r
-* Sun Jul 20 2008 Arlin Davis <ardavis@ichips.intel.com> - 2.0.11\r
-- DAT/DAPL Version 2.0.11 Release 1, IB UD extensions in SCM provider \r
-\r
-* Tue Jun 23 2008 Arlin Davis <ardavis@ichips.intel.com> - 2.0.10\r
-- DAT/DAPL Version 2.0.10 Release 1, socket CM provider \r
-\r
-* Tue May 20 2008 Arlin Davis <ardavis@ichips.intel.com> - 2.0.9\r
-- DAT/DAPL Version 2.0.9 Release 1, OFED 1.3.1 GA \r
-\r
-* Thu May 1 2008 Arlin Davis <ardavis@ichips.intel.com> - 2.0.8\r
-- DAT/DAPL Version 2.0.8 Release 1, OFED 1.3.1 \r
-\r
-* Thu Feb 14 2008 Arlin Davis <ardavis@ichips.intel.com> - 2.0.7\r
-- DAT/DAPL Version 2.0.7 Release 1, OFED 1.3 GA \r
-\r
-* Mon Feb 04 2008 Arlin Davis <ardavis@ichips.intel.com> - 2.0.6\r
-- DAT/DAPL Version 2.0.6 Release 1, OFED 1.3 RC4\r
-\r
-* Tue Jan 29 2008 Arlin Davis <ardavis@ichips.intel.com> - 2.0.5\r
-- DAT/DAPL Version 2.0.5 Release 1, OFED 1.3 RC3\r
-\r
-* Thu Jan 17 2008 Arlin Davis <ardavis@ichips.intel.com> - 2.0.4\r
-- DAT/DAPL Version 2.0.4 Release 1, OFED 1.3 RC2\r
-\r
-* Tue Nov 20 2007 Arlin Davis <ardavis@ichips.intel.com> - 2.0.3\r
-- DAT/DAPL Version 2.0.3 Release 1\r
-\r
-* Tue Oct 30 2007 Arlin Davis <ardavis@ichips.intel.com> - 2.0.2\r
-- DAT/DAPL Version 2.0.2 Release 1\r
-\r
-* Tue Sep 18 2007 Arlin Davis <ardavis@ichips.intel.com> - 2.0.1-1\r
-- OFED 1.3-alpha, co-exist with DAT 1.2 library package. \r
-\r
-* Wed Mar 7 2007 Arlin Davis <ardavis@ichips.intel.com> - 2.0.0.pre\r
-- Initial release of DAT 2.0 APIs, includes IB extensions \r
+# Copyright (c) 2002-2005, Network Appliance, Inc. All rights reserved.
+# Copyright (c) 2007, Intel Corporation. All rights reserved.
+#
+# This Software is licensed under one of the following licenses:
+#
+# 1) under the terms of the "Common Public License 1.0" a copy of which is
+# in the file LICENSE.txt in the root directory. The license is also
+# available from the Open Source Initiative, see
+# http://www.opensource.org/licenses/cpl.php.
+#
+# 2) under the terms of the "The BSD License" a copy of which is in the file
+# LICENSE2.txt in the root directory. The license is also available from
+# the Open Source Initiative, see
+# http://www.opensource.org/licenses/bsd-license.php.
+#
+# 3) under the terms of the "GNU General Public License (GPL) Version 2" a
+# copy of which is in the file LICENSE3.txt in the root directory. The
+# license is also available from the Open Source Initiative, see
+# http://www.opensource.org/licenses/gpl-license.php.
+#
+# Licensee has the right to choose one of the above licenses.
+#
+# Redistributions of source code must retain the above copyright
+# notice and one of the license notices.
+#
+# Redistributions in binary form must reproduce both the above copyright
+# notice, one of the license notices in the documentation
+# and/or other materials provided with the distribution.
+#
+#
+# uDAT and uDAPL 2.0 Registry RPM SPEC file
+#
+# $Id: $
+Name: dapl
+Version: @VERSION@
+Release: 1%{?dist}
+Summary: A Library for userspace access to RDMA devices using OS Agnostic DAT APIs.
+
+Group: System Environment/Libraries
+License: Dual GPL/BSD/CPL
+Url: http://openfabrics.org/
+Source: http://www.openfabrics.org/downloads/%{name}/%{name}-%{version}.tar.gz
+BuildRoot: %(mktemp -ud %{_tmppath}/%{name}-%{version}-%{release}-XXXXXX)
+Requires(post): /sbin/ldconfig
+Requires(postun): /sbin/ldconfig
+Requires(post): sed
+Requires(post): coreutils
+
+%description
+Along with the OpenFabrics kernel drivers, libdat and libdapl provides a userspace
+RDMA API that supports DAT 2.0 specification and IB transport extensions for
+atomic operations and rdma write with immediate data.
+
+%package devel
+Summary: Development files for the libdat and libdapl libraries
+Group: System Environment/Libraries
+
+%description devel
+Header files for libdat and libdapl library.
+
+%package devel-static
+Summary: Static development files for libdat and libdapl library
+Group: System Environment/Libraries
+
+%description devel-static
+Static libraries for libdat and libdapl library.
+
+%package utils
+Summary: Test suites for uDAPL library
+Group: System Environment/Libraries
+Requires: %{name} = %{version}-%{release}
+
+%description utils
+Useful test suites to validate uDAPL library API's.
+
+%prep
+%setup -q
+
+%build
+%configure --enable-ext-type=ib
+make %{?_smp_mflags}
+
+%install
+rm -rf %{buildroot}
+make DESTDIR=%{buildroot} install
+# remove unpackaged files from the buildroot
+rm -f %{buildroot}%{_libdir}/*.la
+rm -f %{buildroot}%{_sysconfdir}/*.conf
+
+%clean
+rm -rf %{buildroot}
+
+%post
+/sbin/ldconfig
+if [ -e %{_sysconfdir}/dat.conf ]; then
+ sed -e '/ofa-v2-.* u2/d' < %{_sysconfdir}/dat.conf > /tmp/$$ofadapl
+ mv /tmp/$$ofadapl %{_sysconfdir}/dat.conf
+fi
+echo ofa-v2-mlx4_0-1 u2.0 nonthreadsafe default libdaploscm.so.2 dapl.2.0 '"mlx4_0 1" ""' >> %{_sysconfdir}/dat.conf
+echo ofa-v2-mlx4_0-2 u2.0 nonthreadsafe default libdaploscm.so.2 dapl.2.0 '"mlx4_0 2" ""' >> %{_sysconfdir}/dat.conf
+echo ofa-v2-ib0 u2.0 nonthreadsafe default libdaplofa.so.2 dapl.2.0 '"ib0 0" ""' >> %{_sysconfdir}/dat.conf
+echo ofa-v2-ib1 u2.0 nonthreadsafe default libdaplofa.so.2 dapl.2.0 '"ib1 0" ""' >> %{_sysconfdir}/dat.conf
+echo ofa-v2-mthca0-1 u2.0 nonthreadsafe default libdaploscm.so.2 dapl.2.0 '"mthca0 1" ""' >> %{_sysconfdir}/dat.conf
+echo ofa-v2-mthca0-2 u2.0 nonthreadsafe default libdaploscm.so.2 dapl.2.0 '"mthca0 2" ""' >> %{_sysconfdir}/dat.conf
+echo ofa-v2-ipath0-1 u2.0 nonthreadsafe default libdaploscm.so.2 dapl.2.0 '"ipath0 1" ""' >> %{_sysconfdir}/dat.conf
+echo ofa-v2-ipath0-2 u2.0 nonthreadsafe default libdaploscm.so.2 dapl.2.0 '"ipath0 2" ""' >> %{_sysconfdir}/dat.conf
+echo ofa-v2-ehca0-1 u2.0 nonthreadsafe default libdaploscm.so.2 dapl.2.0 '"ehca0 1" ""' >> %{_sysconfdir}/dat.conf
+echo ofa-v2-iwarp u2.0 nonthreadsafe default libdaplofa.so.2 dapl.2.0 '"eth2 0" ""' >> %{_sysconfdir}/dat.conf
+echo ofa-v2-mlx4_0-1u u2.0 nonthreadsafe default libdaploucm.so.2 dapl.2.0 '"mlx4_0 1" ""' >> %{_sysconfdir}/dat.conf
+echo ofa-v2-mlx4_0-2u u2.0 nonthreadsafe default libdaploucm.so.2 dapl.2.0 '"mlx4_0 2" ""' >> %{_sysconfdir}/dat.conf
+echo ofa-v2-mthca0-1u u2.0 nonthreadsafe default libdaploucm.so.2 dapl.2.0 '"mthca0 1" ""' >> %{_sysconfdir}/dat.conf
+echo ofa-v2-mthca0-2u u2.0 nonthreadsafe default libdaploucm.so.2 dapl.2.0 '"mthca0 2" ""' >> %{_sysconfdir}/dat.conf
+
+%postun
+/sbin/ldconfig
+if [ -e %{_sysconfdir}/dat.conf ]; then
+ sed -e '/ofa-v2-.* u2/d' < %{_sysconfdir}/dat.conf > /tmp/$$ofadapl
+ mv /tmp/$$ofadapl %{_sysconfdir}/dat.conf
+fi
+
+%files
+%defattr(-,root,root,-)
+%{_libdir}/libda*.so.*
+%doc AUTHORS README ChangeLog
+
+%files devel
+%defattr(-,root,root,-)
+%{_libdir}/*.so
+%dir %{_includedir}/dat2
+%{_includedir}/dat2/*
+
+%files devel-static
+%defattr(-,root,root,-)
+%{_libdir}/*.a
+
+%files utils
+%defattr(-,root,root,-)
+%{_bindir}/*
+%{_mandir}/man1/*.1*
+%{_mandir}/man5/*.5*
+
+%changelog
+* Tue Nov 24 2009 Arlin Davis <ardavis@ichips.intel.com> - 2.0.25
+- DAT/DAPL Version 2.0.25 Release 1, OFED 1.5 RC3
+
+* Fri Oct 30 2009 Arlin Davis <ardavis@ichips.intel.com> - 2.0.24
+- DAT/DAPL Version 2.0.24 Release 1, OFED 1.5 RC2
+
+* Fri Oct 2 2009 Arlin Davis <ardavis@ichips.intel.com> - 2.0.23
+- DAT/DAPL Version 2.0.23 Release 1, OFED 1.5 RC1
+
+* Wed Aug 19 2009 Arlin Davis <ardavis@ichips.intel.com> - 2.0.22
+- DAT/DAPL Version 2.0.22 Release 1, OFED 1.5 ALPHA new UCM provider
+
+* Wed Aug 5 2009 Arlin Davis <ardavis@ichips.intel.com> - 2.0.21
+- DAT/DAPL Version 2.0.21 Release 1, WinOF 2.1, OFED 1.4.1+
+
+* Fri Jun 19 2009 Arlin Davis <ardavis@ichips.intel.com> - 2.0.20
+- DAT/DAPL Version 2.0.20 Release 1, OFED 1.4.1 + UD reject/scaling fixes
+
+* Thu Apr 30 2009 Arlin Davis <ardavis@ichips.intel.com> - 2.0.19
+- DAT/DAPL Version 2.0.19 Release 1, OFED 1.4.1 GA Final
+
+* Fri Apr 17 2009 Arlin Davis <ardavis@ichips.intel.com> - 2.0.18
+- DAT/DAPL Version 2.0.18 Release 1, OFED 1.4.1 GA
+
+* Tue Mar 31 2009 Arlin Davis <ardavis@ichips.intel.com> - 2.0.17
+- DAT/DAPL Version 2.0.17 Release 1, OFED 1.4.1 GA
+
+* Mon Mar 16 2009 Arlin Davis <ardavis@ichips.intel.com> - 2.0.16
+- DAT/DAPL Version 2.0.16 Release 1, OFED 1.4.1
+
+* Fri Nov 07 2008 Arlin Davis <ardavis@ichips.intel.com> - 2.0.15
+- DAT/DAPL Version 2.0.15 Release 1, OFED 1.4 GA
+
+* Fri Oct 03 2008 Arlin Davis <ardavis@ichips.intel.com> - 2.0.14
+- DAT/DAPL Version 2.0.14 Release 1, OFED 1.4 rc3
+
+* Mon Sep 01 2008 Arlin Davis <ardavis@ichips.intel.com> - 2.0.13
+- DAT/DAPL Version 2.0.13 Release 1, OFED 1.4 rc1
+
+* Thu Aug 21 2008 Arlin Davis <ardavis@ichips.intel.com> - 2.0.12
+- DAT/DAPL Version 2.0.12 Release 1, OFED 1.4 beta
+
+* Sun Jul 20 2008 Arlin Davis <ardavis@ichips.intel.com> - 2.0.11
+- DAT/DAPL Version 2.0.11 Release 1, IB UD extensions in SCM provider
+
+* Tue Jun 23 2008 Arlin Davis <ardavis@ichips.intel.com> - 2.0.10
+- DAT/DAPL Version 2.0.10 Release 1, socket CM provider
+
+* Tue May 20 2008 Arlin Davis <ardavis@ichips.intel.com> - 2.0.9
+- DAT/DAPL Version 2.0.9 Release 1, OFED 1.3.1 GA
+
+* Thu May 1 2008 Arlin Davis <ardavis@ichips.intel.com> - 2.0.8
+- DAT/DAPL Version 2.0.8 Release 1, OFED 1.3.1
+
+* Thu Feb 14 2008 Arlin Davis <ardavis@ichips.intel.com> - 2.0.7
+- DAT/DAPL Version 2.0.7 Release 1, OFED 1.3 GA
+
+* Mon Feb 04 2008 Arlin Davis <ardavis@ichips.intel.com> - 2.0.6
+- DAT/DAPL Version 2.0.6 Release 1, OFED 1.3 RC4
+
+* Tue Jan 29 2008 Arlin Davis <ardavis@ichips.intel.com> - 2.0.5
+- DAT/DAPL Version 2.0.5 Release 1, OFED 1.3 RC3
+
+* Thu Jan 17 2008 Arlin Davis <ardavis@ichips.intel.com> - 2.0.4
+- DAT/DAPL Version 2.0.4 Release 1, OFED 1.3 RC2
+
+* Tue Nov 20 2007 Arlin Davis <ardavis@ichips.intel.com> - 2.0.3
+- DAT/DAPL Version 2.0.3 Release 1
+
+* Tue Oct 30 2007 Arlin Davis <ardavis@ichips.intel.com> - 2.0.2
+- DAT/DAPL Version 2.0.2 Release 1
+
+* Tue Sep 18 2007 Arlin Davis <ardavis@ichips.intel.com> - 2.0.1-1
+- OFED 1.3-alpha, co-exist with DAT 1.2 library package.
+
+* Wed Mar 7 2007 Arlin Davis <ardavis@ichips.intel.com> - 2.0.0.pre
+- Initial release of DAT 2.0 APIs, includes IB extensions
dapl_os_lock(&sp_ptr->header.lock);
if (sp_ptr->listening == DAT_FALSE) {
dapl_os_unlock(&sp_ptr->header.lock);
- dapl_dbg_log(DAPL_DBG_TYPE_CM,
+ dapl_dbg_log(DAPL_DBG_TYPE_ERR,
"---> dapls_cr_callback: conn event on down SP\n");
(void)dapls_ib_reject_connection(ib_cm_handle,
DAT_CONNECTION_EVENT_UNREACHABLE,
break;
}
+ case DAT_CONNECTION_EVENT_UNREACHABLE:
+ dapl_dbg_log(DAPL_DBG_TYPE_ERR,
+ "dapls_cr_callback - unreachable\n");
case DAT_CONNECTION_EVENT_NON_PEER_REJECTED:
case DAT_CONNECTION_EVENT_PEER_REJECTED:
- case DAT_CONNECTION_EVENT_UNREACHABLE:
{
/*
* After posting an accept the requesting node has
-/*\r
- * Copyright (c) 2002-2003, Network Appliance, Inc. All rights reserved.\r
- *\r
- * This Software is licensed under one of the following licenses:\r
- *\r
- * 1) under the terms of the "Common Public License 1.0" a copy of which is\r
- * available from the Open Source Initiative, see\r
- * http://www.opensource.org/licenses/cpl.php.\r
- *\r
- * 2) under the terms of the "The BSD License" a copy of which is\r
- * available from the Open Source Initiative, see\r
- * http://www.opensource.org/licenses/bsd-license.php.\r
- *\r
- * 3) under the terms of the "GNU General Public License (GPL) Version 2" a\r
- * copy of which is available from the Open Source Initiative, see\r
- * http://www.opensource.org/licenses/gpl-license.php.\r
- *\r
- * Licensee has the right to choose one of the above licenses.\r
- *\r
- * Redistributions of source code must retain the above copyright\r
- * notice and one of the license notices.\r
- *\r
- * Redistributions in binary form must reproduce both the above copyright\r
- * notice, one of the license notices in the documentation\r
- * and/or other materials provided with the distribution.\r
- */\r
-\r
-#include "dapl.h"\r
-#if !defined(__KDAPL__)\r
-#include <stdarg.h>\r
-#include <stdlib.h>\r
-#endif /* __KDAPL__ */\r
-\r
-DAPL_DBG_TYPE g_dapl_dbg_type; /* initialized in dapl_init.c */\r
-DAPL_DBG_DEST g_dapl_dbg_dest; /* initialized in dapl_init.c */\r
-\r
-static char *_ptr_host_ = NULL;\r
-static char _hostname_[128];\r
-\r
-void dapl_internal_dbg_log(DAPL_DBG_TYPE type, const char *fmt, ...)\r
-{\r
- va_list args;\r
-\r
- if (_ptr_host_ == NULL) {\r
- gethostname(_hostname_, sizeof(_hostname_));\r
- _ptr_host_ = _hostname_;\r
- }\r
-\r
- if (type & g_dapl_dbg_type) {\r
- if (DAPL_DBG_DEST_STDOUT & g_dapl_dbg_dest) {\r
- va_start(args, fmt);\r
- fprintf(stdout, "%s:%x: ", _ptr_host_,\r
- dapl_os_getpid());\r
- dapl_os_vprintf(fmt, args);\r
- va_end(args);\r
- }\r
-\r
- if (DAPL_DBG_DEST_SYSLOG & g_dapl_dbg_dest) {\r
- va_start(args, fmt);\r
- dapl_os_syslog(fmt, args);\r
- va_end(args);\r
- }\r
- }\r
-}\r
-\r
-#ifdef DAPL_COUNTERS\r
-\r
-/*\r
- * The order of this list must match the DAT counter definitions \r
- */\r
-static char *ia_cntr_names[] = {\r
- "DCNT_IA_PZ_CREATE",\r
- "DCNT_IA_PZ_FREE",\r
- "DCNT_IA_LMR_CREATE",\r
- "DCNT_IA_LMR_FREE",\r
- "DCNT_IA_RMR_CREATE",\r
- "DCNT_IA_RMR_FREE",\r
- "DCNT_IA_PSP_CREATE",\r
- "DCNT_IA_PSP_CREATE_ANY",\r
- "DCNT_IA_PSP_FREE",\r
- "DCNT_IA_RSP_CREATE",\r
- "DCNT_IA_RSP_FREE",\r
- "DCNT_IA_EVD_CREATE",\r
- "DCNT_IA_EVD_FREE",\r
- "DCNT_IA_EP_CREATE",\r
- "DCNT_IA_EP_FREE",\r
- "DCNT_IA_SRQ_CREATE",\r
- "DCNT_IA_SRQ_FREE",\r
- "DCNT_IA_SP_CR",\r
- "DCNT_IA_SP_CR_ACCEPTED",\r
- "DCNT_IA_SP_CR_REJECTED",\r
- "DCNT_IA_MEM_ALLOC",\r
- "DCNT_IA_MEM_ALLOC_DATA",\r
- "DCNT_IA_MEM_FREE",\r
- "DCNT_IA_ASYNC_ERROR",\r
- "DCNT_IA_ASYNC_QP_ERROR",\r
- "DCNT_IA_ASYNC_CQ_ERROR"\r
-};\r
-\r
-static char *ep_cntr_names[] = {\r
- "DCNT_EP_CONNECT",\r
- "DCNT_EP_DISCONNECT",\r
- "DCNT_EP_POST_SEND",\r
- "DCNT_EP_POST_SEND_DATA",\r
- "DCNT_EP_POST_SEND_UD",\r
- "DCNT_EP_POST_SEND_UD_DATA",\r
- "DCNT_EP_POST_RECV",\r
- "DCNT_EP_POST_RECV_DATA",\r
- "DCNT_EP_POST_WRITE",\r
- "DCNT_EP_POST_WRITE_DATA",\r
- "DCNT_EP_POST_WRITE_IMM",\r
- "DCNT_EP_POST_WRITE_IMM_DATA",\r
- "DCNT_EP_POST_READ",\r
- "DCNT_EP_POST_READ_DATA",\r
- "DCNT_EP_POST_CMP_SWAP",\r
- "DCNT_EP_POST_FETCH_ADD",\r
- "DCNT_EP_RECV",\r
- "DCNT_EP_RECV_DATA",\r
- "DCNT_EP_RECV_UD",\r
- "DCNT_EP_RECV_UD_DATA",\r
- "DCNT_EP_RECV_IMM",\r
- "DCNT_EP_RECV_IMM_DATA",\r
- "DCNT_EP_RECV_RDMA_IMM",\r
- "DCNT_EP_RECV_RDMA_IMM_DATA",\r
-};\r
-\r
-static char *evd_cntr_names[] = {\r
- "DCNT_EVD_WAIT",\r
- "DCNT_EVD_WAIT_BLOCKED",\r
- "DCNT_EVD_WAIT_NOTIFY",\r
- "DCNT_EVD_DEQUEUE",\r
- "DCNT_EVD_DEQUEUE_FOUND",\r
- "DCNT_EVD_DEQUEUE_NOT_FOUND",\r
- "DCNT_EVD_DEQUEUE_POLL",\r
- "DCNT_EVD_DEQUEUE_POLL_FOUND",\r
- "DCNT_EVD_CONN_CALLBACK",\r
- "DCNT_EVD_DTO_CALLBACK",\r
-};\r
-\r
-DAT_RETURN dapl_query_counter(DAT_HANDLE dh,\r
- int counter, void *p_cntrs_out, int reset)\r
-{\r
- int i, max;\r
- DAT_UINT64 *p_cntrs;\r
- DAT_HANDLE_TYPE type = 0;\r
-\r
- dat_get_handle_type(dh, &type);\r
-\r
- switch (type) {\r
- case DAT_HANDLE_TYPE_IA:\r
- max = DCNT_IA_ALL_COUNTERS;\r
- p_cntrs = ((DAPL_IA *) dh)->cntrs;\r
- break;\r
- case DAT_HANDLE_TYPE_EP:\r
- max = DCNT_EP_ALL_COUNTERS;\r
- p_cntrs = ((DAPL_EP *) dh)->cntrs;\r
- break;\r
- case DAT_HANDLE_TYPE_EVD:\r
- max = DCNT_EVD_ALL_COUNTERS;\r
- p_cntrs = ((DAPL_EVD *) dh)->cntrs;\r
- break;\r
- default:\r
- return DAT_INVALID_HANDLE;\r
- }\r
-\r
- for (i = 0; i < max; i++) {\r
- if ((counter == i) || (counter == max)) {\r
- ((DAT_UINT64 *) p_cntrs_out)[i] = p_cntrs[i];\r
- if (reset)\r
- p_cntrs[i] = 0;\r
- }\r
- }\r
- return DAT_SUCCESS;\r
-}\r
-\r
-char *dapl_query_counter_name(DAT_HANDLE dh, int counter)\r
-{\r
- DAT_HANDLE_TYPE type = 0;\r
-\r
- dat_get_handle_type(dh, &type);\r
-\r
- switch (type) {\r
- case DAT_HANDLE_TYPE_IA:\r
- if (counter < DCNT_IA_ALL_COUNTERS)\r
- return ia_cntr_names[counter];\r
- break;\r
- case DAT_HANDLE_TYPE_EP:\r
- if (counter < DCNT_EP_ALL_COUNTERS)\r
- return ep_cntr_names[counter];\r
- break;\r
- case DAT_HANDLE_TYPE_EVD:\r
- if (counter < DCNT_EVD_ALL_COUNTERS)\r
- return evd_cntr_names[counter];\r
- break;\r
- default:\r
- return NULL;\r
- }\r
- return NULL;\r
-}\r
-\r
-void dapl_print_counter(DAT_HANDLE dh, int counter, int reset)\r
-{\r
- int i, max;\r
- DAT_UINT64 *p_cntrs;\r
- DAT_HANDLE_TYPE type = 0;\r
-\r
- dat_get_handle_type(dh, &type);\r
-\r
- switch (type) {\r
- case DAT_HANDLE_TYPE_IA:\r
- max = DCNT_IA_ALL_COUNTERS;\r
- p_cntrs = ((DAPL_IA *) dh)->cntrs;\r
- break;\r
- case DAT_HANDLE_TYPE_EP:\r
- max = DCNT_EP_ALL_COUNTERS;\r
- p_cntrs = ((DAPL_EP *) dh)->cntrs;\r
- break;\r
- case DAT_HANDLE_TYPE_EVD:\r
- max = DCNT_EVD_ALL_COUNTERS;\r
- p_cntrs = ((DAPL_EVD *) dh)->cntrs;\r
- break;\r
- default:\r
- return;\r
- }\r
-\r
- for (i = 0; i < max; i++) {\r
- if ((counter == i) || (counter == max)) {\r
- printf(" %s = " F64u " \n",\r
- dapl_query_counter_name(dh, i), p_cntrs[i]);\r
- if (reset)\r
- p_cntrs[i] = 0;\r
- }\r
- }\r
-\r
- /* Print in process CR's for this IA, if debug type set */\r
- if ((type == DAT_HANDLE_TYPE_IA) && \r
- (g_dapl_dbg_type & DAPL_DBG_TYPE_CM_LIST)) {\r
- dapls_print_cm_list((DAPL_IA*)dh);\r
- }\r
- return;\r
-}\r
-\r
-#endif /* DAPL_COUNTERS */\r
+/*
+ * Copyright (c) 2002-2003, Network Appliance, Inc. All rights reserved.
+ *
+ * This Software is licensed under one of the following licenses:
+ *
+ * 1) under the terms of the "Common Public License 1.0" a copy of which is
+ * available from the Open Source Initiative, see
+ * http://www.opensource.org/licenses/cpl.php.
+ *
+ * 2) under the terms of the "The BSD License" a copy of which is
+ * available from the Open Source Initiative, see
+ * http://www.opensource.org/licenses/bsd-license.php.
+ *
+ * 3) under the terms of the "GNU General Public License (GPL) Version 2" a
+ * copy of which is available from the Open Source Initiative, see
+ * http://www.opensource.org/licenses/gpl-license.php.
+ *
+ * Licensee has the right to choose one of the above licenses.
+ *
+ * Redistributions of source code must retain the above copyright
+ * notice and one of the license notices.
+ *
+ * Redistributions in binary form must reproduce both the above copyright
+ * notice, one of the license notices in the documentation
+ * and/or other materials provided with the distribution.
+ */
+
+#include "dapl.h"
+#if !defined(__KDAPL__)
+#include <stdarg.h>
+#include <stdlib.h>
+#endif /* __KDAPL__ */
+
+DAPL_DBG_TYPE g_dapl_dbg_type; /* initialized in dapl_init.c */
+DAPL_DBG_DEST g_dapl_dbg_dest; /* initialized in dapl_init.c */
+
+static char *_ptr_host_ = NULL;
+static char _hostname_[128];
+
+void dapl_internal_dbg_log(DAPL_DBG_TYPE type, const char *fmt, ...)
+{
+ va_list args;
+
+ if (_ptr_host_ == NULL) {
+ gethostname(_hostname_, sizeof(_hostname_));
+ _ptr_host_ = _hostname_;
+ }
+
+ if (type & g_dapl_dbg_type) {
+ if (DAPL_DBG_DEST_STDOUT & g_dapl_dbg_dest) {
+ va_start(args, fmt);
+ fprintf(stdout, "%s:%x: ", _ptr_host_,
+ dapl_os_getpid());
+ dapl_os_vprintf(fmt, args);
+ va_end(args);
+ }
+
+ if (DAPL_DBG_DEST_SYSLOG & g_dapl_dbg_dest) {
+ va_start(args, fmt);
+ dapl_os_syslog(fmt, args);
+ va_end(args);
+ }
+ }
+}
+
+#ifdef DAPL_COUNTERS
+
+/*
+ * The order of this list must match the DAT counter definitions
+ */
+static char *ia_cntr_names[] = {
+ "DCNT_IA_PZ_CREATE",
+ "DCNT_IA_PZ_FREE",
+ "DCNT_IA_LMR_CREATE",
+ "DCNT_IA_LMR_FREE",
+ "DCNT_IA_RMR_CREATE",
+ "DCNT_IA_RMR_FREE",
+ "DCNT_IA_PSP_CREATE",
+ "DCNT_IA_PSP_CREATE_ANY",
+ "DCNT_IA_PSP_FREE",
+ "DCNT_IA_RSP_CREATE",
+ "DCNT_IA_RSP_FREE",
+ "DCNT_IA_EVD_CREATE",
+ "DCNT_IA_EVD_FREE",
+ "DCNT_IA_EP_CREATE",
+ "DCNT_IA_EP_FREE",
+ "DCNT_IA_SRQ_CREATE",
+ "DCNT_IA_SRQ_FREE",
+ "DCNT_IA_SP_CR",
+ "DCNT_IA_SP_CR_ACCEPTED",
+ "DCNT_IA_SP_CR_REJECTED",
+ "DCNT_IA_MEM_ALLOC",
+ "DCNT_IA_MEM_ALLOC_DATA",
+ "DCNT_IA_MEM_FREE",
+ "DCNT_IA_ASYNC_ERROR",
+ "DCNT_IA_ASYNC_QP_ERROR",
+ "DCNT_IA_ASYNC_CQ_ERROR"
+};
+
+static char *ep_cntr_names[] = {
+ "DCNT_EP_CONNECT",
+ "DCNT_EP_DISCONNECT",
+ "DCNT_EP_POST_SEND",
+ "DCNT_EP_POST_SEND_DATA",
+ "DCNT_EP_POST_SEND_UD",
+ "DCNT_EP_POST_SEND_UD_DATA",
+ "DCNT_EP_POST_RECV",
+ "DCNT_EP_POST_RECV_DATA",
+ "DCNT_EP_POST_WRITE",
+ "DCNT_EP_POST_WRITE_DATA",
+ "DCNT_EP_POST_WRITE_IMM",
+ "DCNT_EP_POST_WRITE_IMM_DATA",
+ "DCNT_EP_POST_READ",
+ "DCNT_EP_POST_READ_DATA",
+ "DCNT_EP_POST_CMP_SWAP",
+ "DCNT_EP_POST_FETCH_ADD",
+ "DCNT_EP_RECV",
+ "DCNT_EP_RECV_DATA",
+ "DCNT_EP_RECV_UD",
+ "DCNT_EP_RECV_UD_DATA",
+ "DCNT_EP_RECV_IMM",
+ "DCNT_EP_RECV_IMM_DATA",
+ "DCNT_EP_RECV_RDMA_IMM",
+ "DCNT_EP_RECV_RDMA_IMM_DATA",
+};
+
+static char *evd_cntr_names[] = {
+ "DCNT_EVD_WAIT",
+ "DCNT_EVD_WAIT_BLOCKED",
+ "DCNT_EVD_WAIT_NOTIFY",
+ "DCNT_EVD_DEQUEUE",
+ "DCNT_EVD_DEQUEUE_FOUND",
+ "DCNT_EVD_DEQUEUE_NOT_FOUND",
+ "DCNT_EVD_DEQUEUE_POLL",
+ "DCNT_EVD_DEQUEUE_POLL_FOUND",
+ "DCNT_EVD_CONN_CALLBACK",
+ "DCNT_EVD_DTO_CALLBACK",
+};
+
+DAT_RETURN dapl_query_counter(DAT_HANDLE dh,
+ int counter, void *p_cntrs_out, int reset)
+{
+ int i, max;
+ DAT_UINT64 *p_cntrs;
+ DAT_HANDLE_TYPE type = 0;
+
+ dat_get_handle_type(dh, &type);
+
+ switch (type) {
+ case DAT_HANDLE_TYPE_IA:
+ max = DCNT_IA_ALL_COUNTERS;
+ p_cntrs = ((DAPL_IA *) dh)->cntrs;
+ break;
+ case DAT_HANDLE_TYPE_EP:
+ max = DCNT_EP_ALL_COUNTERS;
+ p_cntrs = ((DAPL_EP *) dh)->cntrs;
+ break;
+ case DAT_HANDLE_TYPE_EVD:
+ max = DCNT_EVD_ALL_COUNTERS;
+ p_cntrs = ((DAPL_EVD *) dh)->cntrs;
+ break;
+ default:
+ return DAT_INVALID_HANDLE;
+ }
+
+ for (i = 0; i < max; i++) {
+ if ((counter == i) || (counter == max)) {
+ ((DAT_UINT64 *) p_cntrs_out)[i] = p_cntrs[i];
+ if (reset)
+ p_cntrs[i] = 0;
+ }
+ }
+ return DAT_SUCCESS;
+}
+
+char *dapl_query_counter_name(DAT_HANDLE dh, int counter)
+{
+ DAT_HANDLE_TYPE type = 0;
+
+ dat_get_handle_type(dh, &type);
+
+ switch (type) {
+ case DAT_HANDLE_TYPE_IA:
+ if (counter < DCNT_IA_ALL_COUNTERS)
+ return ia_cntr_names[counter];
+ break;
+ case DAT_HANDLE_TYPE_EP:
+ if (counter < DCNT_EP_ALL_COUNTERS)
+ return ep_cntr_names[counter];
+ break;
+ case DAT_HANDLE_TYPE_EVD:
+ if (counter < DCNT_EVD_ALL_COUNTERS)
+ return evd_cntr_names[counter];
+ break;
+ default:
+ return NULL;
+ }
+ return NULL;
+}
+
+void dapl_print_counter(DAT_HANDLE dh, int counter, int reset)
+{
+ int i, max;
+ DAT_UINT64 *p_cntrs;
+ DAT_HANDLE_TYPE type = 0;
+
+ dat_get_handle_type(dh, &type);
+
+ switch (type) {
+ case DAT_HANDLE_TYPE_IA:
+ max = DCNT_IA_ALL_COUNTERS;
+ p_cntrs = ((DAPL_IA *) dh)->cntrs;
+ break;
+ case DAT_HANDLE_TYPE_EP:
+ max = DCNT_EP_ALL_COUNTERS;
+ p_cntrs = ((DAPL_EP *) dh)->cntrs;
+ break;
+ case DAT_HANDLE_TYPE_EVD:
+ max = DCNT_EVD_ALL_COUNTERS;
+ p_cntrs = ((DAPL_EVD *) dh)->cntrs;
+ break;
+ default:
+ return;
+ }
+
+ for (i = 0; i < max; i++) {
+ if ((counter == i) || (counter == max)) {
+ printf(" %s = " F64u " \n",
+ dapl_query_counter_name(dh, i), p_cntrs[i]);
+ if (reset)
+ p_cntrs[i] = 0;
+ }
+ }
+
+ /* Print in process CR's for this IA, if debug type set */
+ if ((type == DAT_HANDLE_TYPE_IA) &&
+ (g_dapl_dbg_type & DAPL_DBG_TYPE_CM_LIST)) {
+ dapls_print_cm_list((DAPL_IA*)dh);
+ }
+ return;
+}
+
+#endif /* DAPL_COUNTERS */
* this happens, return SUCCESS and generate the event
*/
if (dat_status == DAT_ERROR(DAT_INTERNAL_ERROR, 1)) {
+ dapl_dbg_log(DAPL_DBG_TYPE_ERR,
+ "dapl_ep_connect - unreachable\n");
dapls_evd_post_connection_event((DAPL_EVD *) ep_ptr->
param.
connect_evd_handle,
"dapl_evd_connection_callback PEER REJ pd=%p sz=%d\n",
prd_ptr, private_data_size);
}
- case DAT_CONNECTION_EVENT_DISCONNECTED:
case DAT_CONNECTION_EVENT_UNREACHABLE:
+ dapl_dbg_log(DAPL_DBG_TYPE_ERR,
+ "dapl_evd_connection_callback - unreachable\n");
+ case DAT_CONNECTION_EVENT_DISCONNECTED:
case DAT_CONNECTION_EVENT_NON_PEER_REJECTED:
{
ep_ptr->param.ep_state = DAT_EP_STATE_DISCONNECTED;
dapl_os_lock(&sp_ptr->header.lock);
if (sp_ptr->listening == DAT_FALSE) {
dapl_os_unlock(&sp_ptr->header.lock);
- dapl_dbg_log(DAPL_DBG_TYPE_CM,
+ dapl_dbg_log(DAPL_DBG_TYPE_ERR,
"---> post_cr_event_ext: conn event on down SP\n");
(void)dapls_ib_reject_connection(ib_cm_handle,
DAT_CONNECTION_EVENT_UNREACHABLE,
-DIRS = ibal openib_common openib_scm openib_cma openib_ucm \r
+DIRS = ibal openib_common openib_scm openib_cma openib_ucm
-/*\r
- * Copyright (c) 2005 Voltaire Inc. All rights reserved.\r
- * Copyright (c) 2005-2007 Intel Corporation. All rights reserved.\r
- * Copyright (c) 2004-2005, Mellanox Technologies, Inc. All rights reserved. \r
- * Copyright (c) 2003 Topspin Corporation. All rights reserved. \r
- * Copyright (c) 2005 Sun Microsystems, Inc. All rights reserved.\r
- *\r
- * This Software is licensed under one of the following licenses:\r
- *\r
- * 1) under the terms of the "Common Public License 1.0" a copy of which is\r
- * available from the Open Source Initiative, see\r
- * http://www.opensource.org/licenses/cpl.php.\r
- *\r
- * 2) under the terms of the "The BSD License" a copy of which is\r
- * available from the Open Source Initiative, see\r
- * http://www.opensource.org/licenses/bsd-license.php.\r
- *\r
- * 3) under the terms of the "GNU General Public License (GPL) Version 2" a\r
- * copy of which is available from the Open Source Initiative, see\r
- * http://www.opensource.org/licenses/gpl-license.php.\r
- *\r
- * Licensee has the right to choose one of the above licenses.\r
- *\r
- * Redistributions of source code must retain the above copyright\r
- * notice and one of the license notices.\r
- *\r
- * Redistributions in binary form must reproduce both the above copyright\r
- * notice, one of the license notices in the documentation\r
- * and/or other materials provided with the distribution.\r
- */\r
-\r
-/**********************************************************************\r
- *\r
- * MODULE: dapl_ib_cm.c\r
- *\r
- * PURPOSE: The OFED provider - uCMA, name and route resolution\r
- *\r
- * $Id: $\r
- *\r
- **********************************************************************/\r
-\r
-#include "dapl.h"\r
-#include "dapl_adapter_util.h"\r
-#include "dapl_evd_util.h"\r
-#include "dapl_cr_util.h"\r
-#include "dapl_name_service.h"\r
-#include "dapl_ib_util.h"\r
-#include "dapl_vendor.h"\r
-#include "dapl_osd.h"\r
-\r
-extern struct rdma_event_channel *g_cm_events;\r
-\r
-/* local prototypes */\r
-static struct dapl_cm_id *dapli_req_recv(struct dapl_cm_id *conn,\r
- struct rdma_cm_event *event);\r
-static void dapli_cm_active_cb(struct dapl_cm_id *conn,\r
- struct rdma_cm_event *event);\r
-static void dapli_cm_passive_cb(struct dapl_cm_id *conn,\r
- struct rdma_cm_event *event);\r
-static void dapli_addr_resolve(struct dapl_cm_id *conn);\r
-static void dapli_route_resolve(struct dapl_cm_id *conn);\r
-\r
-/* cma requires 16 bit SID, in network order */\r
-#define IB_PORT_MOD 32001\r
-#define IB_PORT_BASE (65535 - IB_PORT_MOD)\r
-#define SID_TO_PORT(SID) \\r
- (SID > 0xffff ? \\r
- htons((unsigned short)((SID % IB_PORT_MOD) + IB_PORT_BASE)) :\\r
- htons((unsigned short)SID))\r
-\r
-#define PORT_TO_SID(p) ntohs(p)\r
-\r
-/* private data header to validate consumer rejects versus abnormal events */\r
-struct dapl_pdata_hdr {\r
- DAT_UINT32 version;\r
-};\r
-\r
-static void dapli_addr_resolve(struct dapl_cm_id *conn)\r
-{\r
- int ret;\r
-#ifdef DAPL_DBG\r
- struct rdma_addr *ipaddr = &conn->cm_id->route.addr;\r
-#endif\r
- dapl_dbg_log(DAPL_DBG_TYPE_CM,\r
- " addr_resolve: cm_id %p SRC %x DST %x\n",\r
- conn->cm_id, ntohl(((struct sockaddr_in *)\r
- &ipaddr->src_addr)->sin_addr.s_addr),\r
- ntohl(((struct sockaddr_in *)\r
- &ipaddr->dst_addr)->sin_addr.s_addr));\r
-\r
- ret = rdma_resolve_route(conn->cm_id, conn->route_timeout);\r
- if (ret) {\r
- dapl_log(DAPL_DBG_TYPE_ERR,\r
- " dapl_cma_connect: rdma_resolve_route ERR 0x%x %s\n",\r
- ret, strerror(errno));\r
- dapl_evd_connection_callback(conn,\r
- IB_CME_LOCAL_FAILURE,\r
- NULL, conn->ep);\r
- }\r
-}\r
-\r
-static void dapli_route_resolve(struct dapl_cm_id *conn)\r
-{\r
- int ret;\r
-#ifdef DAPL_DBG\r
- struct rdma_addr *ipaddr = &conn->cm_id->route.addr;\r
- struct ib_addr *ibaddr = &conn->cm_id->route.addr.addr.ibaddr;\r
-#endif\r
-\r
- dapl_dbg_log(DAPL_DBG_TYPE_CM,\r
- " route_resolve: cm_id %p SRC %x DST %x PORT %d\n",\r
- conn->cm_id, ntohl(((struct sockaddr_in *)\r
- &ipaddr->src_addr)->sin_addr.s_addr),\r
- ntohl(((struct sockaddr_in *)\r
- &ipaddr->dst_addr)->sin_addr.s_addr),\r
- ntohs(((struct sockaddr_in *)\r
- &ipaddr->dst_addr)->sin_port));\r
-\r
- dapl_dbg_log(DAPL_DBG_TYPE_CM,\r
- " route_resolve: SRC GID subnet %016llx id %016llx\n",\r
- (unsigned long long)\r
- ntohll(ibaddr->sgid.global.subnet_prefix),\r
- (unsigned long long)\r
- ntohll(ibaddr->sgid.global.interface_id));\r
-\r
- dapl_dbg_log(DAPL_DBG_TYPE_CM,\r
- " route_resolve: DST GID subnet %016llx id %016llx\n",\r
- (unsigned long long)\r
- ntohll(ibaddr->dgid.global.subnet_prefix),\r
- (unsigned long long)\r
- ntohll(ibaddr->dgid.global.interface_id));\r
-\r
- dapl_dbg_log(DAPL_DBG_TYPE_CM,\r
- " route_resolve: cm_id %p pdata %p plen %d rr %d ind %d\n",\r
- conn->cm_id,\r
- conn->params.private_data,\r
- conn->params.private_data_len,\r
- conn->params.responder_resources,\r
- conn->params.initiator_depth);\r
-\r
- ret = rdma_connect(conn->cm_id, &conn->params);\r
- if (ret) {\r
- dapl_log(DAPL_DBG_TYPE_ERR,\r
- " dapl_cma_connect: rdma_connect ERR %d %s\n",\r
- ret, strerror(errno));\r
- goto bail;\r
- }\r
- return;\r
-\r
- bail:\r
- dapl_evd_connection_callback(conn,\r
- IB_CME_LOCAL_FAILURE, NULL, conn->ep);\r
-}\r
-\r
-dp_ib_cm_handle_t dapls_ib_cm_create(DAPL_EP *ep)\r
-{\r
- dp_ib_cm_handle_t conn;\r
- struct rdma_cm_id *cm_id;\r
-\r
- /* Allocate CM and initialize lock */\r
- if ((conn = dapl_os_alloc(sizeof(*conn))) == NULL)\r
- return NULL;\r
-\r
- dapl_os_memzero(conn, sizeof(*conn));\r
- dapl_os_lock_init(&conn->lock);\r
- conn->refs++;\r
-\r
- /* create CM_ID, bind to local device, create QP */\r
- if (rdma_create_id(g_cm_events, &cm_id, (void *)conn, RDMA_PS_TCP)) {\r
- dapl_os_free(conn, sizeof(*conn));\r
- return NULL;\r
- }\r
- conn->cm_id = cm_id;\r
-\r
- /* setup timers for address and route resolution */\r
- conn->arp_timeout = dapl_os_get_env_val("DAPL_CM_ARP_TIMEOUT_MS",\r
- IB_ARP_TIMEOUT);\r
- conn->arp_retries = dapl_os_get_env_val("DAPL_CM_ARP_RETRY_COUNT",\r
- IB_ARP_RETRY_COUNT);\r
- conn->route_timeout = dapl_os_get_env_val("DAPL_CM_ROUTE_TIMEOUT_MS",\r
- IB_ROUTE_TIMEOUT);\r
- conn->route_retries = dapl_os_get_env_val("DAPL_CM_ROUTE_RETRY_COUNT",\r
- IB_ROUTE_RETRY_COUNT);\r
- if (ep != NULL) {\r
- conn->ep = ep;\r
- conn->hca = ((DAPL_IA *)ep->param.ia_handle)->hca_ptr;\r
- }\r
-\r
- return conn;\r
-}\r
-\r
-/* \r
- * Only called from consumer thread via dat_ep_free()\r
- * accept, reject, or connect.\r
- * Cannot be called from callback thread.\r
- * rdma_destroy_id will block until rdma_get_cm_event is acked.\r
- */\r
-void dapls_ib_cm_free(dp_ib_cm_handle_t conn, DAPL_EP *ep)\r
-{\r
- dapl_dbg_log(DAPL_DBG_TYPE_CM,\r
- " destroy_conn: conn %p id %d\n", \r
- conn, conn->cm_id);\r
-\r
- dapl_os_lock(&conn->lock);\r
- conn->refs--;\r
- dapl_os_unlock(&conn->lock);\r
-\r
- /* block until event thread complete */\r
- while (conn->refs) \r
- dapl_os_sleep_usec(10000);\r
- \r
- if (ep) {\r
- ep->cm_handle = NULL;\r
- ep->qp_handle = NULL;\r
- ep->qp_state = IB_QP_STATE_ERROR;\r
- }\r
-\r
- if (conn->cm_id) {\r
- if (conn->cm_id->qp)\r
- rdma_destroy_qp(conn->cm_id);\r
- rdma_destroy_id(conn->cm_id);\r
- }\r
-\r
- dapl_os_free(conn, sizeof(*conn));\r
-}\r
-\r
-static struct dapl_cm_id *dapli_req_recv(struct dapl_cm_id *conn,\r
- struct rdma_cm_event *event)\r
-{\r
- struct dapl_cm_id *new_conn;\r
-#ifdef DAPL_DBG\r
- struct rdma_addr *ipaddr = &event->id->route.addr;\r
-#endif\r
-\r
- if (conn->sp == NULL) {\r
- dapl_dbg_log(DAPL_DBG_TYPE_ERR,\r
- " dapli_rep_recv: on invalid listen " "handle\n");\r
- return NULL;\r
- }\r
-\r
- /* allocate new cm_id and merge listen parameters */\r
- new_conn = dapl_os_alloc(sizeof(*new_conn));\r
- if (new_conn) {\r
- (void)dapl_os_memzero(new_conn, sizeof(*new_conn));\r
- dapl_os_lock_init(&new_conn->lock);\r
- new_conn->cm_id = event->id; /* provided by uCMA */\r
- event->id->context = new_conn; /* update CM_ID context */\r
- new_conn->sp = conn->sp;\r
- new_conn->hca = conn->hca;\r
- new_conn->refs++;\r
-\r
- /* Get requesters connect data, setup for accept */\r
- new_conn->params.responder_resources =\r
- DAPL_MIN(event->param.conn.responder_resources,\r
- conn->hca->ib_trans.rd_atom_in);\r
- new_conn->params.initiator_depth =\r
- DAPL_MIN(event->param.conn.initiator_depth,\r
- conn->hca->ib_trans.rd_atom_out);\r
-\r
- new_conn->params.flow_control = event->param.conn.flow_control;\r
- new_conn->params.rnr_retry_count =\r
- event->param.conn.rnr_retry_count;\r
- new_conn->params.retry_count = event->param.conn.retry_count;\r
-\r
- /* save private data */\r
- if (event->param.conn.private_data_len) {\r
- dapl_os_memcpy(new_conn->p_data,\r
- event->param.conn.private_data,\r
- event->param.conn.private_data_len);\r
- new_conn->params.private_data = new_conn->p_data;\r
- new_conn->params.private_data_len =\r
- event->param.conn.private_data_len;\r
- }\r
-\r
- dapl_dbg_log(DAPL_DBG_TYPE_CM, " passive_cb: "\r
- "REQ: SP %p PORT %d LID %d "\r
- "NEW CONN %p ID %p pdata %p,%d\n",\r
- new_conn->sp, ntohs(((struct sockaddr_in *)\r
- &ipaddr->src_addr)->sin_port),\r
- event->listen_id, new_conn, event->id,\r
- event->param.conn.private_data,\r
- event->param.conn.private_data_len);\r
-\r
- dapl_dbg_log(DAPL_DBG_TYPE_CM, " passive_cb: "\r
- "REQ: IP SRC %x PORT %d DST %x PORT %d "\r
- "rr %d init %d\n", ntohl(((struct sockaddr_in *)\r
- &ipaddr->src_addr)->\r
- sin_addr.s_addr),\r
- ntohs(((struct sockaddr_in *)\r
- &ipaddr->src_addr)->sin_port),\r
- ntohl(((struct sockaddr_in *)\r
- &ipaddr->dst_addr)->sin_addr.s_addr),\r
- ntohs(((struct sockaddr_in *)\r
- &ipaddr->dst_addr)->sin_port),\r
- new_conn->params.responder_resources,\r
- new_conn->params.initiator_depth);\r
- }\r
- return new_conn;\r
-}\r
-\r
-static void dapli_cm_active_cb(struct dapl_cm_id *conn,\r
- struct rdma_cm_event *event)\r
-{\r
- DAPL_OS_LOCK *lock = &conn->lock;\r
- ib_cm_events_t ib_cm_event;\r
- const void *pdata = NULL;\r
-\r
- dapl_dbg_log(DAPL_DBG_TYPE_CM,\r
- " active_cb: conn %p id %d event %d\n",\r
- conn, conn->cm_id, event->event);\r
-\r
- /* There is a chance that we can get events after\r
- * the consumer calls disconnect in a pending state\r
- * since the IB CM and uDAPL states are not shared.\r
- * In some cases, IB CM could generate either a DCONN\r
- * or CONN_ERR after the consumer returned from\r
- * dapl_ep_disconnect with a DISCONNECTED event\r
- * already queued. Check state here and bail to\r
- * avoid any events after a disconnect.\r
- */\r
- if (DAPL_BAD_HANDLE(conn->ep, DAPL_MAGIC_EP))\r
- return;\r
-\r
- dapl_os_lock(&conn->ep->header.lock);\r
- if (conn->ep->param.ep_state == DAT_EP_STATE_DISCONNECTED) {\r
- dapl_os_unlock(&conn->ep->header.lock);\r
- return;\r
- }\r
- if (event->event == RDMA_CM_EVENT_DISCONNECTED)\r
- conn->ep->param.ep_state = DAT_EP_STATE_DISCONNECTED;\r
-\r
- dapl_os_unlock(&conn->ep->header.lock);\r
- dapl_os_lock(lock);\r
-\r
- switch (event->event) {\r
- case RDMA_CM_EVENT_UNREACHABLE:\r
- case RDMA_CM_EVENT_CONNECT_ERROR:\r
- dapl_log(DAPL_DBG_TYPE_WARN,\r
- "dapl_cma_active: CONN_ERR event=0x%x"\r
- " status=%d %s DST %s, %d\n",\r
- event->event, event->status,\r
- (event->status == -ETIMEDOUT) ? "TIMEOUT" : "",\r
- inet_ntoa(((struct sockaddr_in *)\r
- &conn->cm_id->route.addr.dst_addr)->\r
- sin_addr),\r
- ntohs(((struct sockaddr_in *)\r
- &conn->cm_id->route.addr.dst_addr)->\r
- sin_port));\r
-\r
- /* per DAT SPEC provider always returns UNREACHABLE */\r
- ib_cm_event = IB_CME_DESTINATION_UNREACHABLE;\r
- break;\r
- case RDMA_CM_EVENT_REJECTED:\r
- dapl_dbg_log(DAPL_DBG_TYPE_CM,\r
- " dapli_cm_active_handler: REJECTED reason=%d\n",\r
- event->status);\r
-\r
- /* valid REJ from consumer will always contain private data */\r
- if (event->status == 28 &&\r
- event->param.conn.private_data_len) {\r
- ib_cm_event = IB_CME_DESTINATION_REJECT_PRIVATE_DATA;\r
- pdata =\r
- (unsigned char *)event->param.conn.\r
- private_data +\r
- sizeof(struct dapl_pdata_hdr);\r
- } else {\r
- ib_cm_event = IB_CME_DESTINATION_REJECT;\r
- dapl_log(DAPL_DBG_TYPE_WARN,\r
- "dapl_cma_active: non-consumer REJ,"\r
- " reason=%d, DST %s, %d\n",\r
- event->status,\r
- inet_ntoa(((struct sockaddr_in *)\r
- &conn->cm_id->route.addr.\r
- dst_addr)->sin_addr),\r
- ntohs(((struct sockaddr_in *)\r
- &conn->cm_id->route.addr.\r
- dst_addr)->sin_port));\r
- }\r
- break;\r
- case RDMA_CM_EVENT_ESTABLISHED:\r
- dapl_dbg_log(DAPL_DBG_TYPE_CM,\r
- " active_cb: cm_id %d PORT %d CONNECTED to %s!\n",\r
- conn->cm_id, ntohs(((struct sockaddr_in *)\r
- &conn->cm_id->route.addr.\r
- dst_addr)->sin_port),\r
- inet_ntoa(((struct sockaddr_in *)\r
- &conn->cm_id->route.addr.dst_addr)->\r
- sin_addr));\r
-\r
- /* setup local and remote ports for ep query */\r
- conn->ep->param.remote_port_qual =\r
- PORT_TO_SID(rdma_get_dst_port(conn->cm_id));\r
- conn->ep->param.local_port_qual =\r
- PORT_TO_SID(rdma_get_src_port(conn->cm_id));\r
-\r
- ib_cm_event = IB_CME_CONNECTED;\r
- pdata = event->param.conn.private_data;\r
- break;\r
- case RDMA_CM_EVENT_DISCONNECTED:\r
- dapl_dbg_log(DAPL_DBG_TYPE_CM,\r
- " active_cb: DISC EVENT - EP %p\n",conn->ep);\r
- rdma_disconnect(conn->cm_id); /* required for DREP */\r
- ib_cm_event = IB_CME_DISCONNECTED;\r
- /* validate EP handle */\r
- if (DAPL_BAD_HANDLE(conn->ep, DAPL_MAGIC_EP))\r
- conn = NULL;\r
- break;\r
- default:\r
- dapl_dbg_log(DAPL_DBG_TYPE_ERR,\r
- " dapli_cm_active_cb_handler: Unexpected CM "\r
- "event %d on ID 0x%p\n", event->event,\r
- conn->cm_id);\r
- conn = NULL;\r
- break;\r
- }\r
-\r
- dapl_os_unlock(lock);\r
- if (conn)\r
- dapl_evd_connection_callback(conn, ib_cm_event, pdata, conn->ep);\r
-}\r
-\r
-static void dapli_cm_passive_cb(struct dapl_cm_id *conn,\r
- struct rdma_cm_event *event)\r
-{\r
- ib_cm_events_t ib_cm_event;\r
- struct dapl_cm_id *conn_recv = conn;\r
- const void *pdata = NULL;\r
- \r
- dapl_dbg_log(DAPL_DBG_TYPE_CM,\r
- " passive_cb: conn %p id %d event %d\n",\r
- conn, event->id, event->event);\r
-\r
- dapl_os_lock(&conn->lock);\r
-\r
- switch (event->event) {\r
- case RDMA_CM_EVENT_CONNECT_REQUEST:\r
- /* create new conn object with new conn_id from event */\r
- conn_recv = dapli_req_recv(conn, event);\r
- ib_cm_event = IB_CME_CONNECTION_REQUEST_PENDING;\r
- pdata = event->param.conn.private_data;\r
- break;\r
- case RDMA_CM_EVENT_UNREACHABLE:\r
- case RDMA_CM_EVENT_CONNECT_ERROR:\r
- dapl_log(DAPL_DBG_TYPE_WARN,\r
- "dapl_cm_passive: CONN_ERR event=0x%x status=%d %s,"\r
- " DST %s,%d\n",\r
- event->event, event->status,\r
- (event->status == -ETIMEDOUT) ? "TIMEOUT" : "",\r
- inet_ntoa(((struct sockaddr_in *)\r
- &conn->cm_id->route.addr.dst_addr)->\r
- sin_addr), ntohs(((struct sockaddr_in *)\r
- &conn->cm_id->route.addr.\r
- dst_addr)->sin_port));\r
- ib_cm_event = IB_CME_DESTINATION_UNREACHABLE;\r
- break;\r
- case RDMA_CM_EVENT_REJECTED:\r
- /* will alwasys be abnormal NON-consumer from active side */\r
- dapl_log(DAPL_DBG_TYPE_WARN,\r
- "dapl_cm_passive: non-consumer REJ, reason=%d,"\r
- " DST %s, %d\n",\r
- event->status,\r
- inet_ntoa(((struct sockaddr_in *)\r
- &conn->cm_id->route.addr.dst_addr)->\r
- sin_addr),\r
- ntohs(((struct sockaddr_in *)\r
- &conn->cm_id->route.addr.dst_addr)->\r
- sin_port));\r
- ib_cm_event = IB_CME_DESTINATION_REJECT;\r
- break;\r
- case RDMA_CM_EVENT_ESTABLISHED:\r
- dapl_dbg_log(DAPL_DBG_TYPE_CM,\r
- " passive_cb: cm_id %p PORT %d CONNECTED from 0x%x!\n",\r
- conn->cm_id, ntohs(((struct sockaddr_in *)\r
- &conn->cm_id->route.addr.\r
- src_addr)->sin_port),\r
- ntohl(((struct sockaddr_in *)\r
- &conn->cm_id->route.addr.dst_addr)->\r
- sin_addr.s_addr));\r
- ib_cm_event = IB_CME_CONNECTED;\r
- break;\r
- case RDMA_CM_EVENT_DISCONNECTED:\r
- rdma_disconnect(conn->cm_id); /* required for DREP */\r
- ib_cm_event = IB_CME_DISCONNECTED;\r
- /* validate SP handle context */\r
- if (DAPL_BAD_HANDLE(conn->sp, DAPL_MAGIC_PSP) &&\r
- DAPL_BAD_HANDLE(conn->sp, DAPL_MAGIC_RSP))\r
- conn_recv = NULL;\r
- break;\r
- default:\r
- dapl_dbg_log(DAPL_DBG_TYPE_ERR, " passive_cb: "\r
- "Unexpected CM event %d on ID 0x%p\n",\r
- event->event, conn->cm_id);\r
- conn_recv = NULL;\r
- break;\r
- }\r
-\r
- dapl_os_unlock(&conn->lock);\r
- if (conn_recv)\r
- dapls_cr_callback(conn_recv, ib_cm_event, pdata, conn_recv->sp);\r
-}\r
-\r
-/************************ DAPL provider entry points **********************/\r
-\r
-/*\r
- * dapls_ib_connect\r
- *\r
- * Initiate a connection with the passive listener on another node\r
- *\r
- * Input:\r
- * ep_handle,\r
- * remote_ia_address,\r
- * remote_conn_qual,\r
- * prd_size size of private data and structure\r
- * prd_prt pointer to private data structure\r
- *\r
- * Output:\r
- * none\r
- *\r
- * Returns:\r
- * DAT_SUCCESS\r
- * DAT_INSUFFICIENT_RESOURCES\r
- * DAT_INVALID_PARAMETER\r
- *\r
- */\r
-DAT_RETURN dapls_ib_connect(IN DAT_EP_HANDLE ep_handle,\r
- IN DAT_IA_ADDRESS_PTR r_addr,\r
- IN DAT_CONN_QUAL r_qual,\r
- IN DAT_COUNT p_size, IN void *p_data)\r
-{\r
- struct dapl_ep *ep_ptr = ep_handle;\r
- struct dapl_cm_id *conn = ep_ptr->cm_handle;\r
- int ret;\r
-\r
- /* Sanity check */\r
- if (NULL == ep_ptr)\r
- return DAT_SUCCESS;\r
-\r
- dapl_dbg_log(DAPL_DBG_TYPE_CM,\r
- " connect: rSID 0x%llx rPort %d, pdata %p, ln %d\n",\r
- r_qual, ntohs(SID_TO_PORT(r_qual)), p_data, p_size);\r
-\r
- /* rdma conn and cm_id pre-bound; reference via ep_ptr->cm_handle */\r
-\r
- /* Setup QP/CM parameters and private data in cm_id */\r
- (void)dapl_os_memzero(&conn->params, sizeof(conn->params));\r
- conn->params.responder_resources =\r
- ep_ptr->param.ep_attr.max_rdma_read_in;\r
- conn->params.initiator_depth = ep_ptr->param.ep_attr.max_rdma_read_out;\r
- conn->params.flow_control = 1;\r
- conn->params.rnr_retry_count = IB_RNR_RETRY_COUNT;\r
- conn->params.retry_count = IB_RC_RETRY_COUNT;\r
- if (p_size) {\r
- dapl_os_memcpy(conn->p_data, p_data, p_size);\r
- conn->params.private_data = conn->p_data;\r
- conn->params.private_data_len = p_size;\r
- }\r
-\r
- /* copy in remote address, need a copy for retry attempts */\r
- dapl_os_memcpy(&conn->r_addr, r_addr, sizeof(*r_addr));\r
-\r
- /* Resolve remote address, src already bound during QP create */\r
- ((struct sockaddr_in *)&conn->r_addr)->sin_port = SID_TO_PORT(r_qual);\r
- ((struct sockaddr_in *)&conn->r_addr)->sin_family = AF_INET;\r
-\r
- ret = rdma_resolve_addr(conn->cm_id, NULL,\r
- (struct sockaddr *)&conn->r_addr,\r
- conn->arp_timeout);\r
- if (ret) {\r
- dapl_log(DAPL_DBG_TYPE_ERR,\r
- " dapl_cma_connect: rdma_resolve_addr ERR 0x%x %s\n",\r
- ret, strerror(errno));\r
- return dapl_convert_errno(errno, "ib_connect");\r
- }\r
- dapl_dbg_log(DAPL_DBG_TYPE_CM,\r
- " connect: resolve_addr: cm_id %p -> %s port %d\n",\r
- conn->cm_id,\r
- inet_ntoa(((struct sockaddr_in *)&conn->r_addr)->sin_addr),\r
- ((struct sockaddr_in *)&conn->r_addr)->sin_port);\r
-\r
- return DAT_SUCCESS;\r
-}\r
-\r
-/*\r
- * dapls_ib_disconnect\r
- *\r
- * Disconnect an EP\r
- *\r
- * Input:\r
- * ep_handle,\r
- * disconnect_flags\r
- *\r
- * Output:\r
- * none\r
- *\r
- * Returns:\r
- * DAT_SUCCESS\r
- *\r
- */\r
-DAT_RETURN\r
-dapls_ib_disconnect(IN DAPL_EP * ep_ptr, IN DAT_CLOSE_FLAGS close_flags)\r
-{\r
- dp_ib_cm_handle_t conn = ep_ptr->cm_handle;\r
-\r
- dapl_dbg_log(DAPL_DBG_TYPE_CM,\r
- " disconnect(ep %p, conn %p, id %d flags %x)\n",\r
- ep_ptr, conn, (conn ? conn->cm_id : 0), close_flags);\r
-\r
- if ((conn == IB_INVALID_HANDLE) || (conn->cm_id == NULL))\r
- return DAT_SUCCESS;\r
-\r
- /* no graceful half-pipe disconnect option */\r
- rdma_disconnect(conn->cm_id);\r
-\r
- /* \r
- * DAT event notification occurs from the callback\r
- * Note: will fire even if DREQ goes unanswered on timeout \r
- */\r
- return DAT_SUCCESS;\r
-}\r
-\r
-/*\r
- * dapls_ib_disconnect_clean\r
- *\r
- * Clean up outstanding connection data. This routine is invoked\r
- * after the final disconnect callback has occurred. Only on the\r
- * ACTIVE side of a connection.\r
- *\r
- * Input:\r
- * ep_ptr DAPL_EP\r
- * active Indicates active side of connection\r
- *\r
- * Output:\r
- * none\r
- *\r
- * Returns:\r
- * void\r
- *\r
- */\r
-void\r
-dapls_ib_disconnect_clean(IN DAPL_EP * ep_ptr,\r
- IN DAT_BOOLEAN active,\r
- IN const ib_cm_events_t ib_cm_event)\r
-{\r
- /* nothing to do */\r
- return;\r
-}\r
-\r
-/*\r
- * dapl_ib_setup_conn_listener\r
- *\r
- * Have the CM set up a connection listener.\r
- *\r
- * Input:\r
- * ibm_hca_handle HCA handle\r
- * qp_handle QP handle\r
- *\r
- * Output:\r
- * none\r
- *\r
- * Returns:\r
- * DAT_SUCCESS\r
- * DAT_INSUFFICIENT_RESOURCES\r
- * DAT_INTERNAL_ERROR\r
- * DAT_CONN_QUAL_UNAVAILBLE\r
- * DAT_CONN_QUAL_IN_USE\r
- *\r
- */\r
-DAT_RETURN\r
-dapls_ib_setup_conn_listener(IN DAPL_IA * ia_ptr,\r
- IN DAT_UINT64 ServiceID, IN DAPL_SP * sp_ptr)\r
-{\r
- DAT_RETURN dat_status = DAT_SUCCESS;\r
- ib_cm_srvc_handle_t conn;\r
- DAT_SOCK_ADDR6 addr; /* local binding address */\r
-\r
- /* Allocate CM and initialize lock */\r
- if ((conn = dapl_os_alloc(sizeof(*conn))) == NULL)\r
- return DAT_INSUFFICIENT_RESOURCES;\r
-\r
- dapl_os_memzero(conn, sizeof(*conn));\r
- dapl_os_lock_init(&conn->lock);\r
- conn->refs++;\r
-\r
- /* create CM_ID, bind to local device, create QP */\r
- if (rdma_create_id\r
- (g_cm_events, &conn->cm_id, (void *)conn, RDMA_PS_TCP)) {\r
- dapl_os_free(conn, sizeof(*conn));\r
- return (dapl_convert_errno(errno, "setup_listener"));\r
- }\r
-\r
- /* open identifies the local device; per DAT specification */\r
- /* Get family and address then set port to consumer's ServiceID */\r
- dapl_os_memcpy(&addr, &ia_ptr->hca_ptr->hca_address, sizeof(addr));\r
- ((struct sockaddr_in *)&addr)->sin_port = SID_TO_PORT(ServiceID);\r
-\r
- if (rdma_bind_addr(conn->cm_id, (struct sockaddr *)&addr)) {\r
- if ((errno == EBUSY) || (errno == EADDRINUSE) || \r
- (errno == EADDRNOTAVAIL))\r
- dat_status = DAT_CONN_QUAL_IN_USE;\r
- else\r
- dat_status =\r
- dapl_convert_errno(errno, "setup_listener");\r
- goto bail;\r
- }\r
-\r
- dapl_dbg_log(DAPL_DBG_TYPE_CM,\r
- " listen(ia_ptr %p SID 0x%llx Port %d sp %p conn %p id %d)\n",\r
- ia_ptr, ServiceID, ntohs(SID_TO_PORT(ServiceID)),\r
- sp_ptr, conn, conn->cm_id);\r
-\r
- sp_ptr->cm_srvc_handle = conn;\r
- conn->sp = sp_ptr;\r
- conn->hca = ia_ptr->hca_ptr;\r
-\r
- dapl_dbg_log(DAPL_DBG_TYPE_EP,\r
- " listen(conn=%p cm_id=%d)\n",\r
- sp_ptr->cm_srvc_handle, conn->cm_id);\r
-\r
- if (rdma_listen(conn->cm_id, 0)) { /* max cma backlog */\r
-\r
- if ((errno == EBUSY) || (errno == EADDRINUSE) ||\r
- (errno == EADDRNOTAVAIL))\r
- dat_status = DAT_CONN_QUAL_IN_USE;\r
- else\r
- dat_status =\r
- dapl_convert_errno(errno, "setup_listener");\r
- goto bail;\r
- }\r
-\r
- /* success */\r
- return DAT_SUCCESS;\r
-\r
- bail:\r
- rdma_destroy_id(conn->cm_id);\r
- dapl_os_free(conn, sizeof(*conn));\r
- return dat_status;\r
-}\r
-\r
-/*\r
- * dapl_ib_remove_conn_listener\r
- *\r
- * Have the CM remove a connection listener.\r
- *\r
- * Input:\r
- * ia_handle IA handle\r
- * ServiceID IB Channel Service ID\r
- *\r
- * Output:\r
- * none\r
- *\r
- * Returns:\r
- * DAT_SUCCESS\r
- * DAT_INVALID_STATE\r
- *\r
- */\r
-DAT_RETURN\r
-dapls_ib_remove_conn_listener(IN DAPL_IA * ia_ptr, IN DAPL_SP * sp_ptr)\r
-{\r
- ib_cm_srvc_handle_t conn = sp_ptr->cm_srvc_handle;\r
-\r
- dapl_dbg_log(DAPL_DBG_TYPE_CM,\r
- " remove_listen(ia_ptr %p sp_ptr %p cm_ptr %p)\n",\r
- ia_ptr, sp_ptr, conn);\r
-\r
- if (conn != IB_INVALID_HANDLE) {\r
- sp_ptr->cm_srvc_handle = NULL;\r
- dapls_ib_cm_free(conn, NULL);\r
- }\r
- return DAT_SUCCESS;\r
-}\r
-\r
-/*\r
- * dapls_ib_accept_connection\r
- *\r
- * Perform necessary steps to accept a connection\r
- *\r
- * Input:\r
- * cr_handle\r
- * ep_handle\r
- * private_data_size\r
- * private_data\r
- *\r
- * Output:\r
- * none\r
- *\r
- * Returns:\r
- * DAT_SUCCESS\r
- * DAT_INSUFFICIENT_RESOURCES\r
- * DAT_INTERNAL_ERROR\r
- *\r
- */\r
-DAT_RETURN\r
-dapls_ib_accept_connection(IN DAT_CR_HANDLE cr_handle,\r
- IN DAT_EP_HANDLE ep_handle,\r
- IN DAT_COUNT p_size, IN const DAT_PVOID p_data)\r
-{\r
- DAPL_CR *cr_ptr = (DAPL_CR *) cr_handle;\r
- DAPL_EP *ep_ptr = (DAPL_EP *) ep_handle;\r
- DAPL_IA *ia_ptr = ep_ptr->header.owner_ia;\r
- struct dapl_cm_id *cr_conn = cr_ptr->ib_cm_handle;\r
- int ret;\r
- DAT_RETURN dat_status;\r
-\r
- dapl_dbg_log(DAPL_DBG_TYPE_CM,\r
- " accept(cr %p conn %p, id %p, p_data %p, p_sz=%d)\n",\r
- cr_ptr, cr_conn, cr_conn->cm_id, p_data, p_size);\r
-\r
- /* Obtain size of private data structure & contents */\r
- if (p_size > IB_MAX_REP_PDATA_SIZE) {\r
- dat_status = DAT_ERROR(DAT_LENGTH_ERROR, DAT_NO_SUBTYPE);\r
- goto bail;\r
- }\r
-\r
- if (ep_ptr->qp_state == DAPL_QP_STATE_UNATTACHED) {\r
- /* \r
- * If we are lazy attaching the QP then we may need to\r
- * hook it up here. Typically, we run this code only for\r
- * DAT_PSP_PROVIDER_FLAG\r
- */\r
- dat_status = dapls_ib_qp_alloc(ia_ptr, ep_ptr, NULL);\r
- if (dat_status != DAT_SUCCESS) {\r
- dapl_log(DAPL_DBG_TYPE_ERR,\r
- " dapl_cma_accept: qp_alloc ERR %d\n",\r
- dat_status);\r
- goto bail;\r
- }\r
- }\r
-\r
- /* \r
- * Validate device and port in EP cm_id against inbound \r
- * CR cm_id. The pre-allocated EP cm_id is already bound to \r
- * a local device (cm_id and QP) when created. Move the QP\r
- * to the new cm_id only if device and port numbers match.\r
- */\r
- if (ep_ptr->cm_handle->cm_id->verbs == cr_conn->cm_id->verbs &&\r
- ep_ptr->cm_handle->cm_id->port_num == cr_conn->cm_id->port_num) {\r
- /* move QP to new cr_conn, remove QP ref in EP cm_id */\r
- cr_conn->cm_id->qp = ep_ptr->cm_handle->cm_id->qp;\r
- ep_ptr->cm_handle->cm_id->qp = NULL;\r
- dapls_ib_cm_free(ep_ptr->cm_handle, NULL);\r
- } else {\r
- dapl_log(DAPL_DBG_TYPE_ERR,\r
- " dapl_cma_accept: ERR dev(%p!=%p) or"\r
- " port mismatch(%d!=%d)\n",\r
- ep_ptr->cm_handle->cm_id->verbs, cr_conn->cm_id->verbs,\r
- ntohs(ep_ptr->cm_handle->cm_id->port_num),\r
- ntohs(cr_conn->cm_id->port_num));\r
- dat_status = DAT_INTERNAL_ERROR;\r
- goto bail;\r
- }\r
-\r
- cr_ptr->param.local_ep_handle = ep_handle;\r
- cr_conn->params.private_data = p_data;\r
- cr_conn->params.private_data_len = p_size;\r
-\r
- ret = rdma_accept(cr_conn->cm_id, &cr_conn->params);\r
- if (ret) {\r
- dapl_log(DAPL_DBG_TYPE_ERR, " dapl_cma_accept: ERR %d %s\n",\r
- ret, strerror(errno));\r
- dat_status = dapl_convert_errno(ret, "accept");\r
- goto bail;\r
- }\r
-\r
- /* save accepted conn and EP reference, qp_handle unchanged */\r
- ep_ptr->cm_handle = cr_conn;\r
- cr_conn->ep = ep_ptr;\r
-\r
- /* setup local and remote ports for ep query */\r
- /* Note: port qual in network order */\r
- ep_ptr->param.remote_port_qual =\r
- PORT_TO_SID(rdma_get_dst_port(cr_conn->cm_id));\r
- ep_ptr->param.local_port_qual =\r
- PORT_TO_SID(rdma_get_src_port(cr_conn->cm_id));\r
-\r
- return DAT_SUCCESS;\r
- bail:\r
- rdma_reject(cr_conn->cm_id, NULL, 0);\r
- dapls_ib_cm_free(cr_conn, NULL);\r
- return dat_status;\r
-}\r
-\r
-/*\r
- * dapls_ib_reject_connection\r
- *\r
- * Reject a connection\r
- *\r
- * Input:\r
- * cr_handle\r
- *\r
- * Output:\r
- * none\r
- *\r
- * Returns:\r
- * DAT_SUCCESS\r
- * DAT_INTERNAL_ERROR\r
- *\r
- */\r
-DAT_RETURN\r
-dapls_ib_reject_connection(IN dp_ib_cm_handle_t cm_handle,\r
- IN int reason,\r
- IN DAT_COUNT private_data_size,\r
- IN const DAT_PVOID private_data)\r
-{\r
- int ret;\r
- int offset = sizeof(struct dapl_pdata_hdr);\r
- struct dapl_pdata_hdr pdata_hdr;\r
-\r
- memset(&pdata_hdr, 0, sizeof pdata_hdr);\r
- pdata_hdr.version = htonl((DAT_VERSION_MAJOR << 24) |\r
- (DAT_VERSION_MINOR << 16) |\r
- (VN_PROVIDER_MAJOR << 8) |\r
- (VN_PROVIDER_MINOR));\r
-\r
- dapl_dbg_log(DAPL_DBG_TYPE_CM,\r
- " reject: handle %p reason %x, ver=%x, data %p, sz=%d\n",\r
- cm_handle, reason, ntohl(pdata_hdr.version),\r
- private_data, private_data_size);\r
-\r
- if (cm_handle == IB_INVALID_HANDLE) {\r
- dapl_dbg_log(DAPL_DBG_TYPE_ERR,\r
- " reject: invalid handle: reason %d\n", reason);\r
- return DAT_ERROR(DAT_INVALID_HANDLE, DAT_INVALID_HANDLE_CR);\r
- }\r
-\r
- if (private_data_size >\r
- dapls_ib_private_data_size(NULL, DAPL_PDATA_CONN_REJ,\r
- cm_handle->hca))\r
- return DAT_ERROR(DAT_INVALID_PARAMETER, DAT_INVALID_ARG3);\r
-\r
- /* setup pdata_hdr and users data, in CR pdata buffer */\r
- dapl_os_memcpy(cm_handle->p_data, &pdata_hdr, offset);\r
- if (private_data_size)\r
- dapl_os_memcpy(cm_handle->p_data + offset,\r
- private_data, private_data_size);\r
-\r
- /*\r
- * Always some private data with reject so active peer can\r
- * determine real application reject from an abnormal \r
- * application termination\r
- */\r
- ret = rdma_reject(cm_handle->cm_id,\r
- cm_handle->p_data, offset + private_data_size);\r
-\r
- dapls_ib_cm_free(cm_handle, NULL);\r
- return dapl_convert_errno(ret, "reject");\r
-}\r
-\r
-/*\r
- * dapls_ib_cm_remote_addr\r
- *\r
- * Obtain the remote IP address given a connection\r
- *\r
- * Input:\r
- * cr_handle\r
- *\r
- * Output:\r
- * remote_ia_address: where to place the remote address\r
- *\r
- * Returns:\r
- * DAT_SUCCESS\r
- * DAT_INVALID_HANDLE\r
- *\r
- */\r
-DAT_RETURN\r
-dapls_ib_cm_remote_addr(IN DAT_HANDLE dat_handle, OUT DAT_SOCK_ADDR6 * raddr)\r
-{\r
- DAPL_HEADER *header;\r
- dp_ib_cm_handle_t ib_cm_handle;\r
- struct rdma_addr *ipaddr;\r
-\r
- dapl_dbg_log(DAPL_DBG_TYPE_EP,\r
- " remote_addr(cm_handle=%p, r_addr=%p)\n",\r
- dat_handle, raddr);\r
-\r
- header = (DAPL_HEADER *) dat_handle;\r
-\r
- if (header->magic == DAPL_MAGIC_EP)\r
- ib_cm_handle = ((DAPL_EP *) dat_handle)->cm_handle;\r
- else if (header->magic == DAPL_MAGIC_CR)\r
- ib_cm_handle = ((DAPL_CR *) dat_handle)->ib_cm_handle;\r
- else\r
- return DAT_INVALID_HANDLE;\r
-\r
- /* get remote IP address from cm_id route */\r
- ipaddr = &ib_cm_handle->cm_id->route.addr;\r
-\r
- dapl_dbg_log(DAPL_DBG_TYPE_CM,\r
- " remote_addr: conn %p id %p SRC %x DST %x PORT %d\n",\r
- ib_cm_handle, ib_cm_handle->cm_id,\r
- ntohl(((struct sockaddr_in *)\r
- &ipaddr->src_addr)->sin_addr.s_addr),\r
- ntohl(((struct sockaddr_in *)\r
- &ipaddr->dst_addr)->sin_addr.s_addr),\r
- ntohs(((struct sockaddr_in *)\r
- &ipaddr->dst_addr)->sin_port));\r
-\r
- dapl_os_memcpy(raddr, &ipaddr->dst_addr, sizeof(DAT_SOCK_ADDR));\r
- return DAT_SUCCESS;\r
-}\r
-\r
-/*\r
- * dapls_ib_private_data_size\r
- *\r
- * Return the size of private data given a connection op type\r
- *\r
- * Input:\r
- * prd_ptr private data pointer\r
- * conn_op connection operation type\r
- * hca_ptr hca pointer, needed for transport type\r
- *\r
- * If prd_ptr is NULL, this is a query for the max size supported by\r
- * the provider, otherwise it is the actual size of the private data\r
- * contained in prd_ptr.\r
- *\r
- *\r
- * Output:\r
- * None\r
- *\r
- * Returns:\r
- * length of private data\r
- *\r
- */\r
-int dapls_ib_private_data_size(IN DAPL_PRIVATE * prd_ptr,\r
- IN DAPL_PDATA_OP conn_op, IN DAPL_HCA * hca_ptr)\r
-{\r
- int size;\r
-\r
- if (hca_ptr->ib_hca_handle->device->transport_type\r
- == IBV_TRANSPORT_IWARP)\r
- return (IWARP_MAX_PDATA_SIZE - sizeof(struct dapl_pdata_hdr));\r
-\r
- switch (conn_op) {\r
-\r
- case DAPL_PDATA_CONN_REQ:\r
- size = IB_MAX_REQ_PDATA_SIZE;\r
- break;\r
- case DAPL_PDATA_CONN_REP:\r
- size = IB_MAX_REP_PDATA_SIZE;\r
- break;\r
- case DAPL_PDATA_CONN_REJ:\r
- size = IB_MAX_REJ_PDATA_SIZE - sizeof(struct dapl_pdata_hdr);\r
- break;\r
- case DAPL_PDATA_CONN_DREQ:\r
- size = IB_MAX_DREQ_PDATA_SIZE;\r
- break;\r
- case DAPL_PDATA_CONN_DREP:\r
- size = IB_MAX_DREP_PDATA_SIZE;\r
- break;\r
- default:\r
- size = 0;\r
-\r
- } /* end case */\r
-\r
- return size;\r
-}\r
-\r
-/*\r
- * Map all CMA event codes to the DAT equivelent.\r
- */\r
-#define DAPL_IB_EVENT_CNT 13\r
-\r
-static struct ib_cm_event_map {\r
- const ib_cm_events_t ib_cm_event;\r
- DAT_EVENT_NUMBER dat_event_num;\r
-} ib_cm_event_map[DAPL_IB_EVENT_CNT] = {\r
- /* 00 */ {\r
- IB_CME_CONNECTED, DAT_CONNECTION_EVENT_ESTABLISHED},\r
- /* 01 */ {\r
- IB_CME_DISCONNECTED, DAT_CONNECTION_EVENT_DISCONNECTED},\r
- /* 02 */ {\r
- IB_CME_DISCONNECTED_ON_LINK_DOWN,\r
- DAT_CONNECTION_EVENT_DISCONNECTED},\r
- /* 03 */ {\r
- IB_CME_CONNECTION_REQUEST_PENDING, DAT_CONNECTION_REQUEST_EVENT},\r
- /* 04 */ {\r
- IB_CME_CONNECTION_REQUEST_PENDING_PRIVATE_DATA,\r
- DAT_CONNECTION_REQUEST_EVENT},\r
- /* 05 */ {\r
- IB_CME_CONNECTION_REQUEST_ACKED, DAT_CONNECTION_REQUEST_EVENT},\r
- /* 06 */ {\r
- IB_CME_DESTINATION_REJECT,\r
- DAT_CONNECTION_EVENT_NON_PEER_REJECTED},\r
- /* 07 */ {\r
- IB_CME_DESTINATION_REJECT_PRIVATE_DATA,\r
- DAT_CONNECTION_EVENT_PEER_REJECTED},\r
- /* 08 */ {\r
- IB_CME_DESTINATION_UNREACHABLE, DAT_CONNECTION_EVENT_UNREACHABLE},\r
- /* 09 */ {\r
- IB_CME_TOO_MANY_CONNECTION_REQUESTS,\r
- DAT_CONNECTION_EVENT_NON_PEER_REJECTED},\r
- /* 10 */ {\r
- IB_CME_LOCAL_FAILURE, DAT_CONNECTION_EVENT_BROKEN},\r
- /* 11 */ {\r
- IB_CME_BROKEN, DAT_CONNECTION_EVENT_BROKEN},\r
- /* 12 */ {\r
-IB_CME_TIMEOUT, DAT_CONNECTION_EVENT_TIMED_OUT},};\r
-\r
-/*\r
- * dapls_ib_get_cm_event\r
- *\r
- * Return a DAT connection event given a provider CM event.\r
- *\r
- * Input:\r
- * dat_event_num DAT event we need an equivelent CM event for\r
- *\r
- * Output:\r
- * none\r
- *\r
- * Returns:\r
- * ib_cm_event of translated DAPL value\r
- */\r
-DAT_EVENT_NUMBER\r
-dapls_ib_get_dat_event(IN const ib_cm_events_t ib_cm_event,\r
- IN DAT_BOOLEAN active)\r
-{\r
- DAT_EVENT_NUMBER dat_event_num;\r
- int i;\r
-\r
- active = active;\r
-\r
- dat_event_num = 0;\r
- for (i = 0; i < DAPL_IB_EVENT_CNT; i++) {\r
- if (ib_cm_event == ib_cm_event_map[i].ib_cm_event) {\r
- dat_event_num = ib_cm_event_map[i].dat_event_num;\r
- break;\r
- }\r
- }\r
- dapl_dbg_log(DAPL_DBG_TYPE_CALLBACK,\r
- "dapls_ib_get_dat_event: event(%s) ib=0x%x dat=0x%x\n",\r
- active ? "active" : "passive", ib_cm_event, dat_event_num);\r
-\r
- return dat_event_num;\r
-}\r
-\r
-/*\r
- * dapls_ib_get_dat_event\r
- *\r
- * Return a DAT connection event given a provider CM event.\r
- * \r
- * Input:\r
- * ib_cm_event event provided to the dapl callback routine\r
- * active switch indicating active or passive connection\r
- *\r
- * Output:\r
- * none\r
- *\r
- * Returns:\r
- * DAT_EVENT_NUMBER of translated provider value\r
- */\r
-ib_cm_events_t dapls_ib_get_cm_event(IN DAT_EVENT_NUMBER dat_event_num)\r
-{\r
- ib_cm_events_t ib_cm_event;\r
- int i;\r
-\r
- ib_cm_event = 0;\r
- for (i = 0; i < DAPL_IB_EVENT_CNT; i++) {\r
- if (dat_event_num == ib_cm_event_map[i].dat_event_num) {\r
- ib_cm_event = ib_cm_event_map[i].ib_cm_event;\r
- break;\r
- }\r
- }\r
- return ib_cm_event;\r
-}\r
-\r
-void dapli_cma_event_cb(void)\r
-{\r
- struct rdma_cm_event *event;\r
- \r
- /* process one CM event, fairness, non-blocking */\r
- if (!rdma_get_cm_event(g_cm_events, &event)) {\r
- struct dapl_cm_id *conn;\r
-\r
- /* set proper conn from cm_id context */\r
- if (event->event == RDMA_CM_EVENT_CONNECT_REQUEST)\r
- conn = (struct dapl_cm_id *)event->listen_id->context;\r
- else\r
- conn = (struct dapl_cm_id *)event->id->context;\r
-\r
- dapl_dbg_log(DAPL_DBG_TYPE_CM,\r
- " cm_event: EVENT=%d ID=%p LID=%p CTX=%p\n",\r
- event->event, event->id, event->listen_id, conn);\r
- \r
- /* cm_free is blocked waiting for ack */\r
- dapl_os_lock(&conn->lock);\r
- if (!conn->refs) {\r
- dapl_os_unlock(&conn->lock);\r
- rdma_ack_cm_event(event);\r
- return;\r
- }\r
- conn->refs++;\r
- dapl_os_unlock(&conn->lock);\r
-\r
- switch (event->event) {\r
- case RDMA_CM_EVENT_ADDR_RESOLVED:\r
- dapli_addr_resolve(conn);\r
- break;\r
-\r
- case RDMA_CM_EVENT_ROUTE_RESOLVED:\r
- dapli_route_resolve(conn);\r
- break;\r
-\r
- case RDMA_CM_EVENT_ADDR_ERROR:\r
- dapl_log(DAPL_DBG_TYPE_WARN,\r
- "dapl_cma_active: CM ADDR ERROR: ->"\r
- " DST %s retry (%d)..\n",\r
- inet_ntoa(((struct sockaddr_in *)\r
- &conn->r_addr)->sin_addr),\r
- conn->arp_retries);\r
-\r
- /* retry address resolution */\r
- if ((--conn->arp_retries) &&\r
- (event->status == -ETIMEDOUT)) {\r
- int ret;\r
- ret = rdma_resolve_addr(conn->cm_id, NULL,\r
- (struct sockaddr *)\r
- &conn->r_addr,\r
- conn->arp_timeout);\r
- if (!ret)\r
- break;\r
- else {\r
- dapl_dbg_log(DAPL_DBG_TYPE_WARN,\r
- " ERROR: rdma_resolve_addr = "\r
- "%d %s\n",\r
- ret, strerror(errno));\r
- }\r
- }\r
- /* retries exhausted or resolve_addr failed */\r
- dapl_log(DAPL_DBG_TYPE_ERR,\r
- "dapl_cma_active: ARP_ERR, retries(%d)"\r
- " exhausted -> DST %s,%d\n",\r
- IB_ARP_RETRY_COUNT,\r
- inet_ntoa(((struct sockaddr_in *)\r
- &conn->cm_id->route.addr.dst_addr)->\r
- sin_addr),\r
- ntohs(((struct sockaddr_in *)\r
- &conn->cm_id->route.addr.dst_addr)->\r
- sin_port));\r
-\r
- dapl_evd_connection_callback(conn,\r
- IB_CME_DESTINATION_UNREACHABLE,\r
- NULL, conn->ep);\r
- break;\r
-\r
- case RDMA_CM_EVENT_ROUTE_ERROR:\r
- dapl_log(DAPL_DBG_TYPE_WARN,\r
- "dapl_cma_active: CM ROUTE ERROR: ->"\r
- " DST %s retry (%d)..\n",\r
- inet_ntoa(((struct sockaddr_in *)\r
- &conn->r_addr)->sin_addr),\r
- conn->route_retries);\r
-\r
- /* retry route resolution */\r
- if ((--conn->route_retries) &&\r
- (event->status == -ETIMEDOUT))\r
- dapli_addr_resolve(conn);\r
- else {\r
- dapl_log(DAPL_DBG_TYPE_ERR,\r
- "dapl_cma_active: PATH_RECORD_ERR,"\r
- " retries(%d) exhausted, DST %s,%d\n",\r
- IB_ROUTE_RETRY_COUNT,\r
- inet_ntoa(((struct sockaddr_in *)\r
- &conn->cm_id->route.addr.\r
- dst_addr)->sin_addr),\r
- ntohs(((struct sockaddr_in *)\r
- &conn->cm_id->route.addr.\r
- dst_addr)->sin_port));\r
-\r
- dapl_evd_connection_callback(conn,\r
- IB_CME_DESTINATION_UNREACHABLE,\r
- NULL, conn->ep);\r
- }\r
- break;\r
-\r
- case RDMA_CM_EVENT_DEVICE_REMOVAL:\r
- dapl_evd_connection_callback(conn,\r
- IB_CME_LOCAL_FAILURE,\r
- NULL, conn->ep);\r
- break;\r
- case RDMA_CM_EVENT_CONNECT_REQUEST:\r
- case RDMA_CM_EVENT_CONNECT_ERROR:\r
- case RDMA_CM_EVENT_UNREACHABLE:\r
- case RDMA_CM_EVENT_REJECTED:\r
- case RDMA_CM_EVENT_ESTABLISHED:\r
- case RDMA_CM_EVENT_DISCONNECTED:\r
- /* passive or active */\r
- if (conn->sp)\r
- dapli_cm_passive_cb(conn, event);\r
- else\r
- dapli_cm_active_cb(conn, event);\r
- break;\r
- case RDMA_CM_EVENT_CONNECT_RESPONSE:\r
-#ifdef RDMA_CM_EVENT_TIMEWAIT_EXIT\r
- case RDMA_CM_EVENT_TIMEWAIT_EXIT:\r
-#endif\r
- break;\r
- default:\r
- dapl_dbg_log(DAPL_DBG_TYPE_CM,\r
- " cm_event: UNEXPECTED EVENT=%p ID=%p CTX=%p\n",\r
- event->event, event->id,\r
- event->id->context);\r
- break;\r
- }\r
- \r
- /* ack event, unblocks destroy_cm_id in consumer threads */\r
- rdma_ack_cm_event(event);\r
-\r
- dapl_os_lock(&conn->lock);\r
- conn->refs--;\r
- dapl_os_unlock(&conn->lock);\r
- } \r
-}\r
-\r
-/*\r
- * Local variables:\r
- * c-indent-level: 4\r
- * c-basic-offset: 4\r
- * tab-width: 8\r
- * End:\r
- */\r
+/*
+ * Copyright (c) 2005 Voltaire Inc. All rights reserved.
+ * Copyright (c) 2005-2007 Intel Corporation. All rights reserved.
+ * Copyright (c) 2004-2005, Mellanox Technologies, Inc. All rights reserved.
+ * Copyright (c) 2003 Topspin Corporation. All rights reserved.
+ * Copyright (c) 2005 Sun Microsystems, Inc. All rights reserved.
+ *
+ * This Software is licensed under one of the following licenses:
+ *
+ * 1) under the terms of the "Common Public License 1.0" a copy of which is
+ * available from the Open Source Initiative, see
+ * http://www.opensource.org/licenses/cpl.php.
+ *
+ * 2) under the terms of the "The BSD License" a copy of which is
+ * available from the Open Source Initiative, see
+ * http://www.opensource.org/licenses/bsd-license.php.
+ *
+ * 3) under the terms of the "GNU General Public License (GPL) Version 2" a
+ * copy of which is available from the Open Source Initiative, see
+ * http://www.opensource.org/licenses/gpl-license.php.
+ *
+ * Licensee has the right to choose one of the above licenses.
+ *
+ * Redistributions of source code must retain the above copyright
+ * notice and one of the license notices.
+ *
+ * Redistributions in binary form must reproduce both the above copyright
+ * notice, one of the license notices in the documentation
+ * and/or other materials provided with the distribution.
+ */
+
+/**********************************************************************
+ *
+ * MODULE: dapl_ib_cm.c
+ *
+ * PURPOSE: The OFED provider - uCMA, name and route resolution
+ *
+ * $Id: $
+ *
+ **********************************************************************/
+
+#include "dapl.h"
+#include "dapl_adapter_util.h"
+#include "dapl_evd_util.h"
+#include "dapl_cr_util.h"
+#include "dapl_name_service.h"
+#include "dapl_ib_util.h"
+#include "dapl_vendor.h"
+#include "dapl_osd.h"
+
+extern struct rdma_event_channel *g_cm_events;
+
+/* local prototypes */
+static struct dapl_cm_id *dapli_req_recv(struct dapl_cm_id *conn,
+ struct rdma_cm_event *event);
+static void dapli_cm_active_cb(struct dapl_cm_id *conn,
+ struct rdma_cm_event *event);
+static void dapli_cm_passive_cb(struct dapl_cm_id *conn,
+ struct rdma_cm_event *event);
+static void dapli_addr_resolve(struct dapl_cm_id *conn);
+static void dapli_route_resolve(struct dapl_cm_id *conn);
+
+/* cma requires 16 bit SID, in network order */
+#define IB_PORT_MOD 32001
+#define IB_PORT_BASE (65535 - IB_PORT_MOD)
+#define SID_TO_PORT(SID) \
+ (SID > 0xffff ? \
+ htons((unsigned short)((SID % IB_PORT_MOD) + IB_PORT_BASE)) :\
+ htons((unsigned short)SID))
+
+#define PORT_TO_SID(p) ntohs(p)
+
+/* private data header to validate consumer rejects versus abnormal events */
+struct dapl_pdata_hdr {
+ DAT_UINT32 version;
+};
+
+static void dapli_addr_resolve(struct dapl_cm_id *conn)
+{
+ int ret;
+#ifdef DAPL_DBG
+ struct rdma_addr *ipaddr = &conn->cm_id->route.addr;
+#endif
+ dapl_dbg_log(DAPL_DBG_TYPE_CM,
+ " addr_resolve: cm_id %p SRC %x DST %x\n",
+ conn->cm_id, ntohl(((struct sockaddr_in *)
+ &ipaddr->src_addr)->sin_addr.s_addr),
+ ntohl(((struct sockaddr_in *)
+ &ipaddr->dst_addr)->sin_addr.s_addr));
+
+ ret = rdma_resolve_route(conn->cm_id, conn->route_timeout);
+ if (ret) {
+ dapl_log(DAPL_DBG_TYPE_ERR,
+ " dapl_cma_connect: rdma_resolve_route ERR 0x%x %s\n",
+ ret, strerror(errno));
+ dapl_evd_connection_callback(conn,
+ IB_CME_LOCAL_FAILURE,
+ NULL, conn->ep);
+ }
+}
+
+static void dapli_route_resolve(struct dapl_cm_id *conn)
+{
+ int ret;
+#ifdef DAPL_DBG
+ struct rdma_addr *ipaddr = &conn->cm_id->route.addr;
+ struct ib_addr *ibaddr = &conn->cm_id->route.addr.addr.ibaddr;
+#endif
+
+ dapl_dbg_log(DAPL_DBG_TYPE_CM,
+ " route_resolve: cm_id %p SRC %x DST %x PORT %d\n",
+ conn->cm_id, ntohl(((struct sockaddr_in *)
+ &ipaddr->src_addr)->sin_addr.s_addr),
+ ntohl(((struct sockaddr_in *)
+ &ipaddr->dst_addr)->sin_addr.s_addr),
+ ntohs(((struct sockaddr_in *)
+ &ipaddr->dst_addr)->sin_port));
+
+ dapl_dbg_log(DAPL_DBG_TYPE_CM,
+ " route_resolve: SRC GID subnet %016llx id %016llx\n",
+ (unsigned long long)
+ ntohll(ibaddr->sgid.global.subnet_prefix),
+ (unsigned long long)
+ ntohll(ibaddr->sgid.global.interface_id));
+
+ dapl_dbg_log(DAPL_DBG_TYPE_CM,
+ " route_resolve: DST GID subnet %016llx id %016llx\n",
+ (unsigned long long)
+ ntohll(ibaddr->dgid.global.subnet_prefix),
+ (unsigned long long)
+ ntohll(ibaddr->dgid.global.interface_id));
+
+ dapl_dbg_log(DAPL_DBG_TYPE_CM,
+ " route_resolve: cm_id %p pdata %p plen %d rr %d ind %d\n",
+ conn->cm_id,
+ conn->params.private_data,
+ conn->params.private_data_len,
+ conn->params.responder_resources,
+ conn->params.initiator_depth);
+
+ ret = rdma_connect(conn->cm_id, &conn->params);
+ if (ret) {
+ dapl_log(DAPL_DBG_TYPE_ERR,
+ " dapl_cma_connect: rdma_connect ERR %d %s\n",
+ ret, strerror(errno));
+ goto bail;
+ }
+ return;
+
+ bail:
+ dapl_evd_connection_callback(conn,
+ IB_CME_LOCAL_FAILURE, NULL, conn->ep);
+}
+
+dp_ib_cm_handle_t dapls_ib_cm_create(DAPL_EP *ep)
+{
+ dp_ib_cm_handle_t conn;
+ struct rdma_cm_id *cm_id;
+
+ /* Allocate CM and initialize lock */
+ if ((conn = dapl_os_alloc(sizeof(*conn))) == NULL)
+ return NULL;
+
+ dapl_os_memzero(conn, sizeof(*conn));
+ dapl_os_lock_init(&conn->lock);
+ conn->refs++;
+
+ /* create CM_ID, bind to local device, create QP */
+ if (rdma_create_id(g_cm_events, &cm_id, (void *)conn, RDMA_PS_TCP)) {
+ dapl_os_free(conn, sizeof(*conn));
+ return NULL;
+ }
+ conn->cm_id = cm_id;
+
+ /* setup timers for address and route resolution */
+ conn->arp_timeout = dapl_os_get_env_val("DAPL_CM_ARP_TIMEOUT_MS",
+ IB_ARP_TIMEOUT);
+ conn->arp_retries = dapl_os_get_env_val("DAPL_CM_ARP_RETRY_COUNT",
+ IB_ARP_RETRY_COUNT);
+ conn->route_timeout = dapl_os_get_env_val("DAPL_CM_ROUTE_TIMEOUT_MS",
+ IB_ROUTE_TIMEOUT);
+ conn->route_retries = dapl_os_get_env_val("DAPL_CM_ROUTE_RETRY_COUNT",
+ IB_ROUTE_RETRY_COUNT);
+ if (ep != NULL) {
+ conn->ep = ep;
+ conn->hca = ((DAPL_IA *)ep->param.ia_handle)->hca_ptr;
+ }
+
+ return conn;
+}
+
+/*
+ * Only called from consumer thread via dat_ep_free()
+ * accept, reject, or connect.
+ * Cannot be called from callback thread.
+ * rdma_destroy_id will block until rdma_get_cm_event is acked.
+ */
+void dapls_ib_cm_free(dp_ib_cm_handle_t conn, DAPL_EP *ep)
+{
+ dapl_dbg_log(DAPL_DBG_TYPE_CM,
+ " destroy_conn: conn %p id %d\n",
+ conn, conn->cm_id);
+
+ dapl_os_lock(&conn->lock);
+ conn->refs--;
+ dapl_os_unlock(&conn->lock);
+
+ /* block until event thread complete */
+ while (conn->refs)
+ dapl_os_sleep_usec(10000);
+
+ if (ep) {
+ ep->cm_handle = NULL;
+ ep->qp_handle = NULL;
+ ep->qp_state = IB_QP_STATE_ERROR;
+ }
+
+ if (conn->cm_id) {
+ if (conn->cm_id->qp)
+ rdma_destroy_qp(conn->cm_id);
+ rdma_destroy_id(conn->cm_id);
+ }
+
+ dapl_os_free(conn, sizeof(*conn));
+}
+
+static struct dapl_cm_id *dapli_req_recv(struct dapl_cm_id *conn,
+ struct rdma_cm_event *event)
+{
+ struct dapl_cm_id *new_conn;
+#ifdef DAPL_DBG
+ struct rdma_addr *ipaddr = &event->id->route.addr;
+#endif
+
+ if (conn->sp == NULL) {
+ dapl_dbg_log(DAPL_DBG_TYPE_ERR,
+ " dapli_rep_recv: on invalid listen " "handle\n");
+ return NULL;
+ }
+
+ /* allocate new cm_id and merge listen parameters */
+ new_conn = dapl_os_alloc(sizeof(*new_conn));
+ if (new_conn) {
+ (void)dapl_os_memzero(new_conn, sizeof(*new_conn));
+ dapl_os_lock_init(&new_conn->lock);
+ new_conn->cm_id = event->id; /* provided by uCMA */
+ event->id->context = new_conn; /* update CM_ID context */
+ new_conn->sp = conn->sp;
+ new_conn->hca = conn->hca;
+ new_conn->refs++;
+
+ /* Get requesters connect data, setup for accept */
+ new_conn->params.responder_resources =
+ DAPL_MIN(event->param.conn.responder_resources,
+ conn->hca->ib_trans.rd_atom_in);
+ new_conn->params.initiator_depth =
+ DAPL_MIN(event->param.conn.initiator_depth,
+ conn->hca->ib_trans.rd_atom_out);
+
+ new_conn->params.flow_control = event->param.conn.flow_control;
+ new_conn->params.rnr_retry_count =
+ event->param.conn.rnr_retry_count;
+ new_conn->params.retry_count = event->param.conn.retry_count;
+
+ /* save private data */
+ if (event->param.conn.private_data_len) {
+ dapl_os_memcpy(new_conn->p_data,
+ event->param.conn.private_data,
+ event->param.conn.private_data_len);
+ new_conn->params.private_data = new_conn->p_data;
+ new_conn->params.private_data_len =
+ event->param.conn.private_data_len;
+ }
+
+ dapl_dbg_log(DAPL_DBG_TYPE_CM, " passive_cb: "
+ "REQ: SP %p PORT %d LID %d "
+ "NEW CONN %p ID %p pdata %p,%d\n",
+ new_conn->sp, ntohs(((struct sockaddr_in *)
+ &ipaddr->src_addr)->sin_port),
+ event->listen_id, new_conn, event->id,
+ event->param.conn.private_data,
+ event->param.conn.private_data_len);
+
+ dapl_dbg_log(DAPL_DBG_TYPE_CM, " passive_cb: "
+ "REQ: IP SRC %x PORT %d DST %x PORT %d "
+ "rr %d init %d\n", ntohl(((struct sockaddr_in *)
+ &ipaddr->src_addr)->
+ sin_addr.s_addr),
+ ntohs(((struct sockaddr_in *)
+ &ipaddr->src_addr)->sin_port),
+ ntohl(((struct sockaddr_in *)
+ &ipaddr->dst_addr)->sin_addr.s_addr),
+ ntohs(((struct sockaddr_in *)
+ &ipaddr->dst_addr)->sin_port),
+ new_conn->params.responder_resources,
+ new_conn->params.initiator_depth);
+ }
+ return new_conn;
+}
+
+static void dapli_cm_active_cb(struct dapl_cm_id *conn,
+ struct rdma_cm_event *event)
+{
+ DAPL_OS_LOCK *lock = &conn->lock;
+ ib_cm_events_t ib_cm_event;
+ const void *pdata = NULL;
+
+ dapl_dbg_log(DAPL_DBG_TYPE_CM,
+ " active_cb: conn %p id %d event %d\n",
+ conn, conn->cm_id, event->event);
+
+ /* There is a chance that we can get events after
+ * the consumer calls disconnect in a pending state
+ * since the IB CM and uDAPL states are not shared.
+ * In some cases, IB CM could generate either a DCONN
+ * or CONN_ERR after the consumer returned from
+ * dapl_ep_disconnect with a DISCONNECTED event
+ * already queued. Check state here and bail to
+ * avoid any events after a disconnect.
+ */
+ if (DAPL_BAD_HANDLE(conn->ep, DAPL_MAGIC_EP))
+ return;
+
+ dapl_os_lock(&conn->ep->header.lock);
+ if (conn->ep->param.ep_state == DAT_EP_STATE_DISCONNECTED) {
+ dapl_os_unlock(&conn->ep->header.lock);
+ return;
+ }
+ if (event->event == RDMA_CM_EVENT_DISCONNECTED)
+ conn->ep->param.ep_state = DAT_EP_STATE_DISCONNECTED;
+
+ dapl_os_unlock(&conn->ep->header.lock);
+ dapl_os_lock(lock);
+
+ switch (event->event) {
+ case RDMA_CM_EVENT_UNREACHABLE:
+ case RDMA_CM_EVENT_CONNECT_ERROR:
+ dapl_log(DAPL_DBG_TYPE_ERR,
+ "dapl_cma_active: CONN_ERR event=0x%x"
+ " status=%d %s DST %s, %d\n",
+ event->event, event->status,
+ (event->status == -ETIMEDOUT) ? "TIMEOUT" : "",
+ inet_ntoa(((struct sockaddr_in *)
+ &conn->cm_id->route.addr.dst_addr)->
+ sin_addr),
+ ntohs(((struct sockaddr_in *)
+ &conn->cm_id->route.addr.dst_addr)->
+ sin_port));
+
+ /* per DAT SPEC provider always returns UNREACHABLE */
+ ib_cm_event = IB_CME_DESTINATION_UNREACHABLE;
+ break;
+ case RDMA_CM_EVENT_REJECTED:
+ dapl_dbg_log(DAPL_DBG_TYPE_CM,
+ " dapli_cm_active_handler: REJECTED reason=%d\n",
+ event->status);
+
+ /* valid REJ from consumer will always contain private data */
+ if (event->status == 28 &&
+ event->param.conn.private_data_len) {
+ ib_cm_event = IB_CME_DESTINATION_REJECT_PRIVATE_DATA;
+ pdata =
+ (unsigned char *)event->param.conn.
+ private_data +
+ sizeof(struct dapl_pdata_hdr);
+ } else {
+ ib_cm_event = IB_CME_DESTINATION_REJECT;
+ dapl_log(DAPL_DBG_TYPE_WARN,
+ "dapl_cma_active: non-consumer REJ,"
+ " reason=%d, DST %s, %d\n",
+ event->status,
+ inet_ntoa(((struct sockaddr_in *)
+ &conn->cm_id->route.addr.
+ dst_addr)->sin_addr),
+ ntohs(((struct sockaddr_in *)
+ &conn->cm_id->route.addr.
+ dst_addr)->sin_port));
+ }
+ break;
+ case RDMA_CM_EVENT_ESTABLISHED:
+ dapl_dbg_log(DAPL_DBG_TYPE_CM,
+ " active_cb: cm_id %d PORT %d CONNECTED to %s!\n",
+ conn->cm_id, ntohs(((struct sockaddr_in *)
+ &conn->cm_id->route.addr.
+ dst_addr)->sin_port),
+ inet_ntoa(((struct sockaddr_in *)
+ &conn->cm_id->route.addr.dst_addr)->
+ sin_addr));
+
+ /* setup local and remote ports for ep query */
+ conn->ep->param.remote_port_qual =
+ PORT_TO_SID(rdma_get_dst_port(conn->cm_id));
+ conn->ep->param.local_port_qual =
+ PORT_TO_SID(rdma_get_src_port(conn->cm_id));
+
+ ib_cm_event = IB_CME_CONNECTED;
+ pdata = event->param.conn.private_data;
+ break;
+ case RDMA_CM_EVENT_DISCONNECTED:
+ dapl_dbg_log(DAPL_DBG_TYPE_CM,
+ " active_cb: DISC EVENT - EP %p\n",conn->ep);
+ rdma_disconnect(conn->cm_id); /* required for DREP */
+ ib_cm_event = IB_CME_DISCONNECTED;
+ /* validate EP handle */
+ if (DAPL_BAD_HANDLE(conn->ep, DAPL_MAGIC_EP))
+ conn = NULL;
+ break;
+ default:
+ dapl_dbg_log(DAPL_DBG_TYPE_ERR,
+ " dapli_cm_active_cb_handler: Unexpected CM "
+ "event %d on ID 0x%p\n", event->event,
+ conn->cm_id);
+ conn = NULL;
+ break;
+ }
+
+ dapl_os_unlock(lock);
+ if (conn)
+ dapl_evd_connection_callback(conn, ib_cm_event, pdata, conn->ep);
+}
+
+static void dapli_cm_passive_cb(struct dapl_cm_id *conn,
+ struct rdma_cm_event *event)
+{
+ ib_cm_events_t ib_cm_event;
+ struct dapl_cm_id *conn_recv = conn;
+ const void *pdata = NULL;
+
+ dapl_dbg_log(DAPL_DBG_TYPE_CM,
+ " passive_cb: conn %p id %d event %d\n",
+ conn, event->id, event->event);
+
+ dapl_os_lock(&conn->lock);
+
+ switch (event->event) {
+ case RDMA_CM_EVENT_CONNECT_REQUEST:
+ /* create new conn object with new conn_id from event */
+ conn_recv = dapli_req_recv(conn, event);
+ ib_cm_event = IB_CME_CONNECTION_REQUEST_PENDING;
+ pdata = event->param.conn.private_data;
+ break;
+ case RDMA_CM_EVENT_UNREACHABLE:
+ case RDMA_CM_EVENT_CONNECT_ERROR:
+ dapl_log(DAPL_DBG_TYPE_ERR,
+ "dapl_cm_passive: CONN_ERR event=0x%x status=%d %s,"
+ " DST %s,%d\n",
+ event->event, event->status,
+ (event->status == -ETIMEDOUT) ? "TIMEOUT" : "",
+ inet_ntoa(((struct sockaddr_in *)
+ &conn->cm_id->route.addr.dst_addr)->
+ sin_addr), ntohs(((struct sockaddr_in *)
+ &conn->cm_id->route.addr.
+ dst_addr)->sin_port));
+ ib_cm_event = IB_CME_DESTINATION_UNREACHABLE;
+ break;
+ case RDMA_CM_EVENT_REJECTED:
+ /* will alwasys be abnormal NON-consumer from active side */
+ dapl_log(DAPL_DBG_TYPE_WARN,
+ "dapl_cm_passive: non-consumer REJ, reason=%d,"
+ " DST %s, %d\n",
+ event->status,
+ inet_ntoa(((struct sockaddr_in *)
+ &conn->cm_id->route.addr.dst_addr)->
+ sin_addr),
+ ntohs(((struct sockaddr_in *)
+ &conn->cm_id->route.addr.dst_addr)->
+ sin_port));
+ ib_cm_event = IB_CME_DESTINATION_REJECT;
+ break;
+ case RDMA_CM_EVENT_ESTABLISHED:
+ dapl_dbg_log(DAPL_DBG_TYPE_CM,
+ " passive_cb: cm_id %p PORT %d CONNECTED from 0x%x!\n",
+ conn->cm_id, ntohs(((struct sockaddr_in *)
+ &conn->cm_id->route.addr.
+ src_addr)->sin_port),
+ ntohl(((struct sockaddr_in *)
+ &conn->cm_id->route.addr.dst_addr)->
+ sin_addr.s_addr));
+ ib_cm_event = IB_CME_CONNECTED;
+ break;
+ case RDMA_CM_EVENT_DISCONNECTED:
+ rdma_disconnect(conn->cm_id); /* required for DREP */
+ ib_cm_event = IB_CME_DISCONNECTED;
+ /* validate SP handle context */
+ if (DAPL_BAD_HANDLE(conn->sp, DAPL_MAGIC_PSP) &&
+ DAPL_BAD_HANDLE(conn->sp, DAPL_MAGIC_RSP))
+ conn_recv = NULL;
+ break;
+ default:
+ dapl_dbg_log(DAPL_DBG_TYPE_ERR, " passive_cb: "
+ "Unexpected CM event %d on ID 0x%p\n",
+ event->event, conn->cm_id);
+ conn_recv = NULL;
+ break;
+ }
+
+ dapl_os_unlock(&conn->lock);
+ if (conn_recv)
+ dapls_cr_callback(conn_recv, ib_cm_event, pdata, conn_recv->sp);
+}
+
+/************************ DAPL provider entry points **********************/
+
+/*
+ * dapls_ib_connect
+ *
+ * Initiate a connection with the passive listener on another node
+ *
+ * Input:
+ * ep_handle,
+ * remote_ia_address,
+ * remote_conn_qual,
+ * prd_size size of private data and structure
+ * prd_prt pointer to private data structure
+ *
+ * Output:
+ * none
+ *
+ * Returns:
+ * DAT_SUCCESS
+ * DAT_INSUFFICIENT_RESOURCES
+ * DAT_INVALID_PARAMETER
+ *
+ */
+DAT_RETURN dapls_ib_connect(IN DAT_EP_HANDLE ep_handle,
+ IN DAT_IA_ADDRESS_PTR r_addr,
+ IN DAT_CONN_QUAL r_qual,
+ IN DAT_COUNT p_size, IN void *p_data)
+{
+ struct dapl_ep *ep_ptr = ep_handle;
+ struct dapl_cm_id *conn = ep_ptr->cm_handle;
+ int ret;
+
+ /* Sanity check */
+ if (NULL == ep_ptr)
+ return DAT_SUCCESS;
+
+ dapl_dbg_log(DAPL_DBG_TYPE_CM,
+ " connect: rSID 0x%llx rPort %d, pdata %p, ln %d\n",
+ r_qual, ntohs(SID_TO_PORT(r_qual)), p_data, p_size);
+
+ /* rdma conn and cm_id pre-bound; reference via ep_ptr->cm_handle */
+
+ /* Setup QP/CM parameters and private data in cm_id */
+ (void)dapl_os_memzero(&conn->params, sizeof(conn->params));
+ conn->params.responder_resources =
+ ep_ptr->param.ep_attr.max_rdma_read_in;
+ conn->params.initiator_depth = ep_ptr->param.ep_attr.max_rdma_read_out;
+ conn->params.flow_control = 1;
+ conn->params.rnr_retry_count = IB_RNR_RETRY_COUNT;
+ conn->params.retry_count = IB_RC_RETRY_COUNT;
+ if (p_size) {
+ dapl_os_memcpy(conn->p_data, p_data, p_size);
+ conn->params.private_data = conn->p_data;
+ conn->params.private_data_len = p_size;
+ }
+
+ /* copy in remote address, need a copy for retry attempts */
+ dapl_os_memcpy(&conn->r_addr, r_addr, sizeof(*r_addr));
+
+ /* Resolve remote address, src already bound during QP create */
+ ((struct sockaddr_in *)&conn->r_addr)->sin_port = SID_TO_PORT(r_qual);
+ ((struct sockaddr_in *)&conn->r_addr)->sin_family = AF_INET;
+
+ ret = rdma_resolve_addr(conn->cm_id, NULL,
+ (struct sockaddr *)&conn->r_addr,
+ conn->arp_timeout);
+ if (ret) {
+ dapl_log(DAPL_DBG_TYPE_ERR,
+ " dapl_cma_connect: rdma_resolve_addr ERR 0x%x %s\n",
+ ret, strerror(errno));
+ return dapl_convert_errno(errno, "ib_connect");
+ }
+ dapl_dbg_log(DAPL_DBG_TYPE_CM,
+ " connect: resolve_addr: cm_id %p -> %s port %d\n",
+ conn->cm_id,
+ inet_ntoa(((struct sockaddr_in *)&conn->r_addr)->sin_addr),
+ ((struct sockaddr_in *)&conn->r_addr)->sin_port);
+
+ return DAT_SUCCESS;
+}
+
+/*
+ * dapls_ib_disconnect
+ *
+ * Disconnect an EP
+ *
+ * Input:
+ * ep_handle,
+ * disconnect_flags
+ *
+ * Output:
+ * none
+ *
+ * Returns:
+ * DAT_SUCCESS
+ *
+ */
+DAT_RETURN
+dapls_ib_disconnect(IN DAPL_EP * ep_ptr, IN DAT_CLOSE_FLAGS close_flags)
+{
+ dp_ib_cm_handle_t conn = ep_ptr->cm_handle;
+
+ dapl_dbg_log(DAPL_DBG_TYPE_CM,
+ " disconnect(ep %p, conn %p, id %d flags %x)\n",
+ ep_ptr, conn, (conn ? conn->cm_id : 0), close_flags);
+
+ if ((conn == IB_INVALID_HANDLE) || (conn->cm_id == NULL))
+ return DAT_SUCCESS;
+
+ /* no graceful half-pipe disconnect option */
+ rdma_disconnect(conn->cm_id);
+
+ /*
+ * DAT event notification occurs from the callback
+ * Note: will fire even if DREQ goes unanswered on timeout
+ */
+ return DAT_SUCCESS;
+}
+
+/*
+ * dapls_ib_disconnect_clean
+ *
+ * Clean up outstanding connection data. This routine is invoked
+ * after the final disconnect callback has occurred. Only on the
+ * ACTIVE side of a connection.
+ *
+ * Input:
+ * ep_ptr DAPL_EP
+ * active Indicates active side of connection
+ *
+ * Output:
+ * none
+ *
+ * Returns:
+ * void
+ *
+ */
+void
+dapls_ib_disconnect_clean(IN DAPL_EP * ep_ptr,
+ IN DAT_BOOLEAN active,
+ IN const ib_cm_events_t ib_cm_event)
+{
+ /* nothing to do */
+ return;
+}
+
+/*
+ * dapl_ib_setup_conn_listener
+ *
+ * Have the CM set up a connection listener.
+ *
+ * Input:
+ * ibm_hca_handle HCA handle
+ * qp_handle QP handle
+ *
+ * Output:
+ * none
+ *
+ * Returns:
+ * DAT_SUCCESS
+ * DAT_INSUFFICIENT_RESOURCES
+ * DAT_INTERNAL_ERROR
+ * DAT_CONN_QUAL_UNAVAILBLE
+ * DAT_CONN_QUAL_IN_USE
+ *
+ */
+DAT_RETURN
+dapls_ib_setup_conn_listener(IN DAPL_IA * ia_ptr,
+ IN DAT_UINT64 ServiceID, IN DAPL_SP * sp_ptr)
+{
+ DAT_RETURN dat_status = DAT_SUCCESS;
+ ib_cm_srvc_handle_t conn;
+ DAT_SOCK_ADDR6 addr; /* local binding address */
+
+ /* Allocate CM and initialize lock */
+ if ((conn = dapl_os_alloc(sizeof(*conn))) == NULL)
+ return DAT_INSUFFICIENT_RESOURCES;
+
+ dapl_os_memzero(conn, sizeof(*conn));
+ dapl_os_lock_init(&conn->lock);
+ conn->refs++;
+
+ /* create CM_ID, bind to local device, create QP */
+ if (rdma_create_id
+ (g_cm_events, &conn->cm_id, (void *)conn, RDMA_PS_TCP)) {
+ dapl_os_free(conn, sizeof(*conn));
+ return (dapl_convert_errno(errno, "setup_listener"));
+ }
+
+ /* open identifies the local device; per DAT specification */
+ /* Get family and address then set port to consumer's ServiceID */
+ dapl_os_memcpy(&addr, &ia_ptr->hca_ptr->hca_address, sizeof(addr));
+ ((struct sockaddr_in *)&addr)->sin_port = SID_TO_PORT(ServiceID);
+
+ if (rdma_bind_addr(conn->cm_id, (struct sockaddr *)&addr)) {
+ if ((errno == EBUSY) || (errno == EADDRINUSE) ||
+ (errno == EADDRNOTAVAIL))
+ dat_status = DAT_CONN_QUAL_IN_USE;
+ else
+ dat_status =
+ dapl_convert_errno(errno, "setup_listener");
+ goto bail;
+ }
+
+ dapl_dbg_log(DAPL_DBG_TYPE_CM,
+ " listen(ia_ptr %p SID 0x%llx Port %d sp %p conn %p id %d)\n",
+ ia_ptr, ServiceID, ntohs(SID_TO_PORT(ServiceID)),
+ sp_ptr, conn, conn->cm_id);
+
+ sp_ptr->cm_srvc_handle = conn;
+ conn->sp = sp_ptr;
+ conn->hca = ia_ptr->hca_ptr;
+
+ dapl_dbg_log(DAPL_DBG_TYPE_EP,
+ " listen(conn=%p cm_id=%d)\n",
+ sp_ptr->cm_srvc_handle, conn->cm_id);
+
+ if (rdma_listen(conn->cm_id, 0)) { /* max cma backlog */
+
+ if ((errno == EBUSY) || (errno == EADDRINUSE) ||
+ (errno == EADDRNOTAVAIL))
+ dat_status = DAT_CONN_QUAL_IN_USE;
+ else
+ dat_status =
+ dapl_convert_errno(errno, "setup_listener");
+ goto bail;
+ }
+
+ /* success */
+ return DAT_SUCCESS;
+
+ bail:
+ rdma_destroy_id(conn->cm_id);
+ dapl_os_free(conn, sizeof(*conn));
+ return dat_status;
+}
+
+/*
+ * dapl_ib_remove_conn_listener
+ *
+ * Have the CM remove a connection listener.
+ *
+ * Input:
+ * ia_handle IA handle
+ * ServiceID IB Channel Service ID
+ *
+ * Output:
+ * none
+ *
+ * Returns:
+ * DAT_SUCCESS
+ * DAT_INVALID_STATE
+ *
+ */
+DAT_RETURN
+dapls_ib_remove_conn_listener(IN DAPL_IA * ia_ptr, IN DAPL_SP * sp_ptr)
+{
+ ib_cm_srvc_handle_t conn = sp_ptr->cm_srvc_handle;
+
+ dapl_dbg_log(DAPL_DBG_TYPE_CM,
+ " remove_listen(ia_ptr %p sp_ptr %p cm_ptr %p)\n",
+ ia_ptr, sp_ptr, conn);
+
+ if (conn != IB_INVALID_HANDLE) {
+ sp_ptr->cm_srvc_handle = NULL;
+ dapls_ib_cm_free(conn, NULL);
+ }
+ return DAT_SUCCESS;
+}
+
+/*
+ * dapls_ib_accept_connection
+ *
+ * Perform necessary steps to accept a connection
+ *
+ * Input:
+ * cr_handle
+ * ep_handle
+ * private_data_size
+ * private_data
+ *
+ * Output:
+ * none
+ *
+ * Returns:
+ * DAT_SUCCESS
+ * DAT_INSUFFICIENT_RESOURCES
+ * DAT_INTERNAL_ERROR
+ *
+ */
+DAT_RETURN
+dapls_ib_accept_connection(IN DAT_CR_HANDLE cr_handle,
+ IN DAT_EP_HANDLE ep_handle,
+ IN DAT_COUNT p_size, IN const DAT_PVOID p_data)
+{
+ DAPL_CR *cr_ptr = (DAPL_CR *) cr_handle;
+ DAPL_EP *ep_ptr = (DAPL_EP *) ep_handle;
+ DAPL_IA *ia_ptr = ep_ptr->header.owner_ia;
+ struct dapl_cm_id *cr_conn = cr_ptr->ib_cm_handle;
+ int ret;
+ DAT_RETURN dat_status;
+
+ dapl_dbg_log(DAPL_DBG_TYPE_CM,
+ " accept(cr %p conn %p, id %p, p_data %p, p_sz=%d)\n",
+ cr_ptr, cr_conn, cr_conn->cm_id, p_data, p_size);
+
+ /* Obtain size of private data structure & contents */
+ if (p_size > IB_MAX_REP_PDATA_SIZE) {
+ dat_status = DAT_ERROR(DAT_LENGTH_ERROR, DAT_NO_SUBTYPE);
+ goto bail;
+ }
+
+ if (ep_ptr->qp_state == DAPL_QP_STATE_UNATTACHED) {
+ /*
+ * If we are lazy attaching the QP then we may need to
+ * hook it up here. Typically, we run this code only for
+ * DAT_PSP_PROVIDER_FLAG
+ */
+ dat_status = dapls_ib_qp_alloc(ia_ptr, ep_ptr, NULL);
+ if (dat_status != DAT_SUCCESS) {
+ dapl_log(DAPL_DBG_TYPE_ERR,
+ " dapl_cma_accept: qp_alloc ERR %d\n",
+ dat_status);
+ goto bail;
+ }
+ }
+
+ /*
+ * Validate device and port in EP cm_id against inbound
+ * CR cm_id. The pre-allocated EP cm_id is already bound to
+ * a local device (cm_id and QP) when created. Move the QP
+ * to the new cm_id only if device and port numbers match.
+ */
+ if (ep_ptr->cm_handle->cm_id->verbs == cr_conn->cm_id->verbs &&
+ ep_ptr->cm_handle->cm_id->port_num == cr_conn->cm_id->port_num) {
+ /* move QP to new cr_conn, remove QP ref in EP cm_id */
+ cr_conn->cm_id->qp = ep_ptr->cm_handle->cm_id->qp;
+ ep_ptr->cm_handle->cm_id->qp = NULL;
+ dapls_ib_cm_free(ep_ptr->cm_handle, NULL);
+ } else {
+ dapl_log(DAPL_DBG_TYPE_ERR,
+ " dapl_cma_accept: ERR dev(%p!=%p) or"
+ " port mismatch(%d!=%d)\n",
+ ep_ptr->cm_handle->cm_id->verbs, cr_conn->cm_id->verbs,
+ ntohs(ep_ptr->cm_handle->cm_id->port_num),
+ ntohs(cr_conn->cm_id->port_num));
+ dat_status = DAT_INTERNAL_ERROR;
+ goto bail;
+ }
+
+ cr_ptr->param.local_ep_handle = ep_handle;
+ cr_conn->params.private_data = p_data;
+ cr_conn->params.private_data_len = p_size;
+
+ ret = rdma_accept(cr_conn->cm_id, &cr_conn->params);
+ if (ret) {
+ dapl_log(DAPL_DBG_TYPE_ERR, " dapl_cma_accept: ERR %d %s\n",
+ ret, strerror(errno));
+ dat_status = dapl_convert_errno(ret, "accept");
+ goto bail;
+ }
+
+ /* save accepted conn and EP reference, qp_handle unchanged */
+ ep_ptr->cm_handle = cr_conn;
+ cr_conn->ep = ep_ptr;
+
+ /* setup local and remote ports for ep query */
+ /* Note: port qual in network order */
+ ep_ptr->param.remote_port_qual =
+ PORT_TO_SID(rdma_get_dst_port(cr_conn->cm_id));
+ ep_ptr->param.local_port_qual =
+ PORT_TO_SID(rdma_get_src_port(cr_conn->cm_id));
+
+ return DAT_SUCCESS;
+ bail:
+ rdma_reject(cr_conn->cm_id, NULL, 0);
+ dapls_ib_cm_free(cr_conn, NULL);
+ return dat_status;
+}
+
+/*
+ * dapls_ib_reject_connection
+ *
+ * Reject a connection
+ *
+ * Input:
+ * cr_handle
+ *
+ * Output:
+ * none
+ *
+ * Returns:
+ * DAT_SUCCESS
+ * DAT_INTERNAL_ERROR
+ *
+ */
+DAT_RETURN
+dapls_ib_reject_connection(IN dp_ib_cm_handle_t cm_handle,
+ IN int reason,
+ IN DAT_COUNT private_data_size,
+ IN const DAT_PVOID private_data)
+{
+ int ret;
+ int offset = sizeof(struct dapl_pdata_hdr);
+ struct dapl_pdata_hdr pdata_hdr;
+
+ memset(&pdata_hdr, 0, sizeof pdata_hdr);
+ pdata_hdr.version = htonl((DAT_VERSION_MAJOR << 24) |
+ (DAT_VERSION_MINOR << 16) |
+ (VN_PROVIDER_MAJOR << 8) |
+ (VN_PROVIDER_MINOR));
+
+ dapl_dbg_log(DAPL_DBG_TYPE_CM,
+ " reject: handle %p reason %x, ver=%x, data %p, sz=%d\n",
+ cm_handle, reason, ntohl(pdata_hdr.version),
+ private_data, private_data_size);
+
+ if (cm_handle == IB_INVALID_HANDLE) {
+ dapl_dbg_log(DAPL_DBG_TYPE_ERR,
+ " reject: invalid handle: reason %d\n", reason);
+ return DAT_ERROR(DAT_INVALID_HANDLE, DAT_INVALID_HANDLE_CR);
+ }
+
+ if (private_data_size >
+ dapls_ib_private_data_size(NULL, DAPL_PDATA_CONN_REJ,
+ cm_handle->hca))
+ return DAT_ERROR(DAT_INVALID_PARAMETER, DAT_INVALID_ARG3);
+
+ /* setup pdata_hdr and users data, in CR pdata buffer */
+ dapl_os_memcpy(cm_handle->p_data, &pdata_hdr, offset);
+ if (private_data_size)
+ dapl_os_memcpy(cm_handle->p_data + offset,
+ private_data, private_data_size);
+
+ /*
+ * Always some private data with reject so active peer can
+ * determine real application reject from an abnormal
+ * application termination
+ */
+ ret = rdma_reject(cm_handle->cm_id,
+ cm_handle->p_data, offset + private_data_size);
+
+ dapls_ib_cm_free(cm_handle, NULL);
+ return dapl_convert_errno(ret, "reject");
+}
+
+/*
+ * dapls_ib_cm_remote_addr
+ *
+ * Obtain the remote IP address given a connection
+ *
+ * Input:
+ * cr_handle
+ *
+ * Output:
+ * remote_ia_address: where to place the remote address
+ *
+ * Returns:
+ * DAT_SUCCESS
+ * DAT_INVALID_HANDLE
+ *
+ */
+DAT_RETURN
+dapls_ib_cm_remote_addr(IN DAT_HANDLE dat_handle, OUT DAT_SOCK_ADDR6 * raddr)
+{
+ DAPL_HEADER *header;
+ dp_ib_cm_handle_t ib_cm_handle;
+ struct rdma_addr *ipaddr;
+
+ dapl_dbg_log(DAPL_DBG_TYPE_EP,
+ " remote_addr(cm_handle=%p, r_addr=%p)\n",
+ dat_handle, raddr);
+
+ header = (DAPL_HEADER *) dat_handle;
+
+ if (header->magic == DAPL_MAGIC_EP)
+ ib_cm_handle = ((DAPL_EP *) dat_handle)->cm_handle;
+ else if (header->magic == DAPL_MAGIC_CR)
+ ib_cm_handle = ((DAPL_CR *) dat_handle)->ib_cm_handle;
+ else
+ return DAT_INVALID_HANDLE;
+
+ /* get remote IP address from cm_id route */
+ ipaddr = &ib_cm_handle->cm_id->route.addr;
+
+ dapl_dbg_log(DAPL_DBG_TYPE_CM,
+ " remote_addr: conn %p id %p SRC %x DST %x PORT %d\n",
+ ib_cm_handle, ib_cm_handle->cm_id,
+ ntohl(((struct sockaddr_in *)
+ &ipaddr->src_addr)->sin_addr.s_addr),
+ ntohl(((struct sockaddr_in *)
+ &ipaddr->dst_addr)->sin_addr.s_addr),
+ ntohs(((struct sockaddr_in *)
+ &ipaddr->dst_addr)->sin_port));
+
+ dapl_os_memcpy(raddr, &ipaddr->dst_addr, sizeof(DAT_SOCK_ADDR));
+ return DAT_SUCCESS;
+}
+
+/*
+ * dapls_ib_private_data_size
+ *
+ * Return the size of private data given a connection op type
+ *
+ * Input:
+ * prd_ptr private data pointer
+ * conn_op connection operation type
+ * hca_ptr hca pointer, needed for transport type
+ *
+ * If prd_ptr is NULL, this is a query for the max size supported by
+ * the provider, otherwise it is the actual size of the private data
+ * contained in prd_ptr.
+ *
+ *
+ * Output:
+ * None
+ *
+ * Returns:
+ * length of private data
+ *
+ */
+int dapls_ib_private_data_size(IN DAPL_PRIVATE * prd_ptr,
+ IN DAPL_PDATA_OP conn_op, IN DAPL_HCA * hca_ptr)
+{
+ int size;
+
+ if (hca_ptr->ib_hca_handle->device->transport_type
+ == IBV_TRANSPORT_IWARP)
+ return (IWARP_MAX_PDATA_SIZE - sizeof(struct dapl_pdata_hdr));
+
+ switch (conn_op) {
+
+ case DAPL_PDATA_CONN_REQ:
+ size = IB_MAX_REQ_PDATA_SIZE;
+ break;
+ case DAPL_PDATA_CONN_REP:
+ size = IB_MAX_REP_PDATA_SIZE;
+ break;
+ case DAPL_PDATA_CONN_REJ:
+ size = IB_MAX_REJ_PDATA_SIZE - sizeof(struct dapl_pdata_hdr);
+ break;
+ case DAPL_PDATA_CONN_DREQ:
+ size = IB_MAX_DREQ_PDATA_SIZE;
+ break;
+ case DAPL_PDATA_CONN_DREP:
+ size = IB_MAX_DREP_PDATA_SIZE;
+ break;
+ default:
+ size = 0;
+
+ } /* end case */
+
+ return size;
+}
+
+/*
+ * Map all CMA event codes to the DAT equivelent.
+ */
+#define DAPL_IB_EVENT_CNT 13
+
+static struct ib_cm_event_map {
+ const ib_cm_events_t ib_cm_event;
+ DAT_EVENT_NUMBER dat_event_num;
+} ib_cm_event_map[DAPL_IB_EVENT_CNT] = {
+ /* 00 */ {
+ IB_CME_CONNECTED, DAT_CONNECTION_EVENT_ESTABLISHED},
+ /* 01 */ {
+ IB_CME_DISCONNECTED, DAT_CONNECTION_EVENT_DISCONNECTED},
+ /* 02 */ {
+ IB_CME_DISCONNECTED_ON_LINK_DOWN,
+ DAT_CONNECTION_EVENT_DISCONNECTED},
+ /* 03 */ {
+ IB_CME_CONNECTION_REQUEST_PENDING, DAT_CONNECTION_REQUEST_EVENT},
+ /* 04 */ {
+ IB_CME_CONNECTION_REQUEST_PENDING_PRIVATE_DATA,
+ DAT_CONNECTION_REQUEST_EVENT},
+ /* 05 */ {
+ IB_CME_CONNECTION_REQUEST_ACKED, DAT_CONNECTION_REQUEST_EVENT},
+ /* 06 */ {
+ IB_CME_DESTINATION_REJECT,
+ DAT_CONNECTION_EVENT_NON_PEER_REJECTED},
+ /* 07 */ {
+ IB_CME_DESTINATION_REJECT_PRIVATE_DATA,
+ DAT_CONNECTION_EVENT_PEER_REJECTED},
+ /* 08 */ {
+ IB_CME_DESTINATION_UNREACHABLE, DAT_CONNECTION_EVENT_UNREACHABLE},
+ /* 09 */ {
+ IB_CME_TOO_MANY_CONNECTION_REQUESTS,
+ DAT_CONNECTION_EVENT_NON_PEER_REJECTED},
+ /* 10 */ {
+ IB_CME_LOCAL_FAILURE, DAT_CONNECTION_EVENT_BROKEN},
+ /* 11 */ {
+ IB_CME_BROKEN, DAT_CONNECTION_EVENT_BROKEN},
+ /* 12 */ {
+IB_CME_TIMEOUT, DAT_CONNECTION_EVENT_TIMED_OUT},};
+
+/*
+ * dapls_ib_get_cm_event
+ *
+ * Return a DAT connection event given a provider CM event.
+ *
+ * Input:
+ * dat_event_num DAT event we need an equivelent CM event for
+ *
+ * Output:
+ * none
+ *
+ * Returns:
+ * ib_cm_event of translated DAPL value
+ */
+DAT_EVENT_NUMBER
+dapls_ib_get_dat_event(IN const ib_cm_events_t ib_cm_event,
+ IN DAT_BOOLEAN active)
+{
+ DAT_EVENT_NUMBER dat_event_num;
+ int i;
+
+ active = active;
+
+ dat_event_num = 0;
+ for (i = 0; i < DAPL_IB_EVENT_CNT; i++) {
+ if (ib_cm_event == ib_cm_event_map[i].ib_cm_event) {
+ dat_event_num = ib_cm_event_map[i].dat_event_num;
+ break;
+ }
+ }
+ dapl_dbg_log(DAPL_DBG_TYPE_CALLBACK,
+ "dapls_ib_get_dat_event: event(%s) ib=0x%x dat=0x%x\n",
+ active ? "active" : "passive", ib_cm_event, dat_event_num);
+
+ return dat_event_num;
+}
+
+/*
+ * dapls_ib_get_dat_event
+ *
+ * Return a DAT connection event given a provider CM event.
+ *
+ * Input:
+ * ib_cm_event event provided to the dapl callback routine
+ * active switch indicating active or passive connection
+ *
+ * Output:
+ * none
+ *
+ * Returns:
+ * DAT_EVENT_NUMBER of translated provider value
+ */
+ib_cm_events_t dapls_ib_get_cm_event(IN DAT_EVENT_NUMBER dat_event_num)
+{
+ ib_cm_events_t ib_cm_event;
+ int i;
+
+ ib_cm_event = 0;
+ for (i = 0; i < DAPL_IB_EVENT_CNT; i++) {
+ if (dat_event_num == ib_cm_event_map[i].dat_event_num) {
+ ib_cm_event = ib_cm_event_map[i].ib_cm_event;
+ break;
+ }
+ }
+ return ib_cm_event;
+}
+
+void dapli_cma_event_cb(void)
+{
+ struct rdma_cm_event *event;
+
+ /* process one CM event, fairness, non-blocking */
+ if (!rdma_get_cm_event(g_cm_events, &event)) {
+ struct dapl_cm_id *conn;
+
+ /* set proper conn from cm_id context */
+ if (event->event == RDMA_CM_EVENT_CONNECT_REQUEST)
+ conn = (struct dapl_cm_id *)event->listen_id->context;
+ else
+ conn = (struct dapl_cm_id *)event->id->context;
+
+ dapl_dbg_log(DAPL_DBG_TYPE_CM,
+ " cm_event: EVENT=%d ID=%p LID=%p CTX=%p\n",
+ event->event, event->id, event->listen_id, conn);
+
+ /* cm_free is blocked waiting for ack */
+ dapl_os_lock(&conn->lock);
+ if (!conn->refs) {
+ dapl_os_unlock(&conn->lock);
+ rdma_ack_cm_event(event);
+ return;
+ }
+ conn->refs++;
+ dapl_os_unlock(&conn->lock);
+
+ switch (event->event) {
+ case RDMA_CM_EVENT_ADDR_RESOLVED:
+ dapli_addr_resolve(conn);
+ break;
+
+ case RDMA_CM_EVENT_ROUTE_RESOLVED:
+ dapli_route_resolve(conn);
+ break;
+
+ case RDMA_CM_EVENT_ADDR_ERROR:
+ dapl_log(DAPL_DBG_TYPE_WARN,
+ "dapl_cma_active: CM ADDR ERROR: ->"
+ " DST %s retry (%d)..\n",
+ inet_ntoa(((struct sockaddr_in *)
+ &conn->r_addr)->sin_addr),
+ conn->arp_retries);
+
+ /* retry address resolution */
+ if ((--conn->arp_retries) &&
+ (event->status == -ETIMEDOUT)) {
+ int ret;
+ ret = rdma_resolve_addr(conn->cm_id, NULL,
+ (struct sockaddr *)
+ &conn->r_addr,
+ conn->arp_timeout);
+ if (!ret)
+ break;
+ else {
+ dapl_dbg_log(DAPL_DBG_TYPE_WARN,
+ " ERROR: rdma_resolve_addr = "
+ "%d %s\n",
+ ret, strerror(errno));
+ }
+ }
+ /* retries exhausted or resolve_addr failed */
+ dapl_log(DAPL_DBG_TYPE_ERR,
+ "dapl_cma_active: ARP_ERR, retries(%d)"
+ " exhausted -> DST %s,%d\n",
+ IB_ARP_RETRY_COUNT,
+ inet_ntoa(((struct sockaddr_in *)
+ &conn->cm_id->route.addr.dst_addr)->
+ sin_addr),
+ ntohs(((struct sockaddr_in *)
+ &conn->cm_id->route.addr.dst_addr)->
+ sin_port));
+
+ dapl_evd_connection_callback(conn,
+ IB_CME_DESTINATION_UNREACHABLE,
+ NULL, conn->ep);
+ break;
+
+ case RDMA_CM_EVENT_ROUTE_ERROR:
+ dapl_log(DAPL_DBG_TYPE_WARN,
+ "dapl_cma_active: CM ROUTE ERROR: ->"
+ " DST %s retry (%d)..\n",
+ inet_ntoa(((struct sockaddr_in *)
+ &conn->r_addr)->sin_addr),
+ conn->route_retries);
+
+ /* retry route resolution */
+ if ((--conn->route_retries) &&
+ (event->status == -ETIMEDOUT))
+ dapli_addr_resolve(conn);
+ else {
+ dapl_log(DAPL_DBG_TYPE_ERR,
+ "dapl_cma_active: PATH_RECORD_ERR,"
+ " retries(%d) exhausted, DST %s,%d\n",
+ IB_ROUTE_RETRY_COUNT,
+ inet_ntoa(((struct sockaddr_in *)
+ &conn->cm_id->route.addr.
+ dst_addr)->sin_addr),
+ ntohs(((struct sockaddr_in *)
+ &conn->cm_id->route.addr.
+ dst_addr)->sin_port));
+
+ dapl_evd_connection_callback(conn,
+ IB_CME_DESTINATION_UNREACHABLE,
+ NULL, conn->ep);
+ }
+ break;
+
+ case RDMA_CM_EVENT_DEVICE_REMOVAL:
+ dapl_evd_connection_callback(conn,
+ IB_CME_LOCAL_FAILURE,
+ NULL, conn->ep);
+ break;
+ case RDMA_CM_EVENT_CONNECT_REQUEST:
+ case RDMA_CM_EVENT_CONNECT_ERROR:
+ case RDMA_CM_EVENT_UNREACHABLE:
+ case RDMA_CM_EVENT_REJECTED:
+ case RDMA_CM_EVENT_ESTABLISHED:
+ case RDMA_CM_EVENT_DISCONNECTED:
+ /* passive or active */
+ if (conn->sp)
+ dapli_cm_passive_cb(conn, event);
+ else
+ dapli_cm_active_cb(conn, event);
+ break;
+ case RDMA_CM_EVENT_CONNECT_RESPONSE:
+#ifdef RDMA_CM_EVENT_TIMEWAIT_EXIT
+ case RDMA_CM_EVENT_TIMEWAIT_EXIT:
+#endif
+ break;
+ default:
+ dapl_dbg_log(DAPL_DBG_TYPE_CM,
+ " cm_event: UNEXPECTED EVENT=%p ID=%p CTX=%p\n",
+ event->event, event->id,
+ event->id->context);
+ break;
+ }
+
+ /* ack event, unblocks destroy_cm_id in consumer threads */
+ rdma_ack_cm_event(event);
+
+ dapl_os_lock(&conn->lock);
+ conn->refs--;
+ dapl_os_unlock(&conn->lock);
+ }
+}
+
+/*
+ * Local variables:
+ * c-indent-level: 4
+ * c-basic-offset: 4
+ * tab-width: 8
+ * End:
+ */
-/*\r
- * Copyright (c) 2009 Intel Corporation. All rights reserved.\r
- *\r
- * This Software is licensed under one of the following licenses:\r
- *\r
- * 1) under the terms of the "Common Public License 1.0" a copy of which is\r
- * available from the Open Source Initiative, see\r
- * http://www.opensource.org/licenses/cpl.php.\r
- *\r
- * 2) under the terms of the "The BSD License" a copy of which is\r
- * available from the Open Source Initiative, see\r
- * http://www.opensource.org/licenses/bsd-license.php.\r
- *\r
- * 3) under the terms of the "GNU General Public License (GPL) Version 2" a\r
- * copy of which is available from the Open Source Initiative, see\r
- * http://www.opensource.org/licenses/gpl-license.php.\r
- *\r
- * Licensee has the right to choose one of the above licenses.\r
- *\r
- * Redistributions of source code must retain the above copyright\r
- * notice and one of the license notices.\r
- *\r
- * Redistributions in binary form must reproduce both the above copyright\r
- * notice, one of the license notices in the documentation\r
- * and/or other materials provided with the distribution.\r
- */\r
-#include "openib_osd.h"\r
-#include "dapl.h"\r
-#include "dapl_adapter_util.h"\r
-#include "dapl_lmr_util.h"\r
-#include "dapl_evd_util.h"\r
-#include "dapl_ring_buffer_util.h"\r
-\r
-/*\r
- * Map all verbs DTO completion codes to the DAT equivelent.\r
- *\r
- * Not returned by verbs: DAT_DTO_ERR_PARTIAL_PACKET\r
- */\r
-static struct ib_status_map {\r
- int ib_status;\r
- DAT_DTO_COMPLETION_STATUS dat_status;\r
-} ib_status_map[] = {\r
-/* 00 */ {IBV_WC_SUCCESS, DAT_DTO_SUCCESS},\r
-/* 01 */ {IBV_WC_LOC_LEN_ERR, DAT_DTO_ERR_LOCAL_LENGTH},\r
-/* 02 */ {IBV_WC_LOC_QP_OP_ERR, DAT_DTO_ERR_LOCAL_EP},\r
-/* 03 */ {IBV_WC_LOC_EEC_OP_ERR, DAT_DTO_ERR_TRANSPORT},\r
-/* 04 */ {IBV_WC_LOC_PROT_ERR, DAT_DTO_ERR_LOCAL_PROTECTION},\r
-/* 05 */ {IBV_WC_WR_FLUSH_ERR, DAT_DTO_ERR_FLUSHED},\r
-/* 06 */ {IBV_WC_MW_BIND_ERR, DAT_RMR_OPERATION_FAILED},\r
-/* 07 */ {IBV_WC_BAD_RESP_ERR, DAT_DTO_ERR_BAD_RESPONSE},\r
-/* 08 */ {IBV_WC_LOC_ACCESS_ERR, DAT_DTO_ERR_LOCAL_PROTECTION},\r
-/* 09 */ {IBV_WC_REM_INV_REQ_ERR, DAT_DTO_ERR_REMOTE_RESPONDER},\r
-/* 10 */ {IBV_WC_REM_ACCESS_ERR, DAT_DTO_ERR_REMOTE_ACCESS},\r
-/* 11 */ {IBV_WC_REM_OP_ERR, DAT_DTO_ERR_REMOTE_RESPONDER},\r
-/* 12 */ {IBV_WC_RETRY_EXC_ERR, DAT_DTO_ERR_TRANSPORT},\r
-/* 13 */ {IBV_WC_RNR_RETRY_EXC_ERR, DAT_DTO_ERR_RECEIVER_NOT_READY},\r
-/* 14 */ {IBV_WC_LOC_RDD_VIOL_ERR, DAT_DTO_ERR_LOCAL_PROTECTION},\r
-/* 15 */ {IBV_WC_REM_INV_RD_REQ_ERR, DAT_DTO_ERR_REMOTE_RESPONDER},\r
-/* 16 */ {IBV_WC_REM_ABORT_ERR, DAT_DTO_ERR_REMOTE_RESPONDER},\r
-/* 17 */ {IBV_WC_INV_EECN_ERR, DAT_DTO_ERR_TRANSPORT},\r
-/* 18 */ {IBV_WC_INV_EEC_STATE_ERR, DAT_DTO_ERR_TRANSPORT},\r
-/* 19 */ {IBV_WC_FATAL_ERR, DAT_DTO_ERR_TRANSPORT},\r
-/* 20 */ {IBV_WC_RESP_TIMEOUT_ERR, DAT_DTO_ERR_RECEIVER_NOT_READY},\r
-/* 21 */ {IBV_WC_GENERAL_ERR, DAT_DTO_ERR_TRANSPORT},\r
-};\r
-\r
-/*\r
- * dapls_ib_get_dto_status\r
- *\r
- * Return the DAT status of a DTO operation\r
- *\r
- * Input:\r
- * cqe_ptr pointer to completion queue entry\r
- *\r
- * Output:\r
- * none\r
- *\r
- * Returns:\r
- * Value from ib_status_map table above\r
- */\r
-\r
-DAT_DTO_COMPLETION_STATUS\r
-dapls_ib_get_dto_status(IN ib_work_completion_t * cqe_ptr)\r
-{\r
- uint32_t ib_status;\r
- int i;\r
-\r
- ib_status = DAPL_GET_CQE_STATUS(cqe_ptr);\r
-\r
- /*\r
- * Due to the implementation of verbs completion code, we need to\r
- * search the table for the correct value rather than assuming\r
- * linear distribution.\r
- */\r
- for (i = 0; i <= IBV_WC_GENERAL_ERR; i++) {\r
- if (ib_status == ib_status_map[i].ib_status) {\r
- if (ib_status != IBV_WC_SUCCESS) {\r
- dapl_log(DAPL_DBG_TYPE_DTO_COMP_ERR,\r
- " DTO completion ERROR: %d:"\r
- " op %#x\n",\r
- ib_status,\r
- DAPL_GET_CQE_OPTYPE(cqe_ptr));\r
- }\r
- return ib_status_map[i].dat_status;\r
- }\r
- }\r
-\r
- return DAT_DTO_FAILURE;\r
-}\r
-\r
-DAT_RETURN dapls_ib_get_async_event(IN ib_error_record_t * err_record,\r
- OUT DAT_EVENT_NUMBER * async_event)\r
-{\r
- DAT_RETURN dat_status = DAT_SUCCESS;\r
- int err_code = err_record->event_type;\r
-\r
- switch (err_code) {\r
- /* OVERFLOW error */\r
- case IBV_EVENT_CQ_ERR:\r
- *async_event = DAT_ASYNC_ERROR_EVD_OVERFLOW;\r
- break;\r
- /* INTERNAL errors */\r
- case IBV_EVENT_DEVICE_FATAL:\r
- *async_event = DAT_ASYNC_ERROR_PROVIDER_INTERNAL_ERROR;\r
- break;\r
- /* CATASTROPHIC errors */\r
- case IBV_EVENT_PORT_ERR:\r
- *async_event = DAT_ASYNC_ERROR_IA_CATASTROPHIC;\r
- break;\r
- /* BROKEN QP error */\r
- case IBV_EVENT_SQ_DRAINED:\r
- case IBV_EVENT_QP_FATAL:\r
- case IBV_EVENT_QP_REQ_ERR:\r
- case IBV_EVENT_QP_ACCESS_ERR:\r
- *async_event = DAT_ASYNC_ERROR_EP_BROKEN;\r
- break;\r
-\r
- /* connection completion */\r
- case IBV_EVENT_COMM_EST:\r
- *async_event = DAT_CONNECTION_EVENT_ESTABLISHED;\r
- break;\r
-\r
- /* TODO: process HW state changes */\r
- case IBV_EVENT_PATH_MIG:\r
- case IBV_EVENT_PATH_MIG_ERR:\r
- case IBV_EVENT_PORT_ACTIVE:\r
- case IBV_EVENT_LID_CHANGE:\r
- case IBV_EVENT_PKEY_CHANGE:\r
- case IBV_EVENT_SM_CHANGE:\r
- default:\r
- dat_status = DAT_ERROR(DAT_NOT_IMPLEMENTED, 0);\r
- }\r
- return dat_status;\r
-}\r
-\r
-/*\r
- * dapl_ib_cq_alloc\r
- *\r
- * Alloc a CQ\r
- *\r
- * Input:\r
- * ia_handle IA handle\r
- * evd_ptr pointer to EVD struct\r
- * cqlen minimum QLen\r
- *\r
- * Output:\r
- * none\r
- *\r
- * Returns:\r
- * DAT_SUCCESS\r
- * DAT_INSUFFICIENT_RESOURCES\r
- *\r
- */\r
-DAT_RETURN\r
-dapls_ib_cq_alloc(IN DAPL_IA * ia_ptr,\r
- IN DAPL_EVD * evd_ptr, IN DAT_COUNT * cqlen)\r
-{\r
- struct ibv_comp_channel *channel;\r
- DAT_RETURN ret;\r
-\r
- dapl_dbg_log(DAPL_DBG_TYPE_UTIL,\r
- "dapls_ib_cq_alloc: evd %p cqlen=%d \n", evd_ptr, *cqlen);\r
-\r
- if (!evd_ptr->cno_ptr)\r
- channel = ibv_create_comp_channel(ia_ptr->hca_ptr->ib_hca_handle);\r
- else\r
- channel = ia_ptr->hca_ptr->ib_trans.ib_cq;\r
-\r
- if (!channel)\r
- return DAT_INSUFFICIENT_RESOURCES;\r
-\r
- evd_ptr->ib_cq_handle = ibv_create_cq(ia_ptr->hca_ptr->ib_hca_handle,\r
- *cqlen, evd_ptr, channel, 0);\r
-\r
- if (evd_ptr->ib_cq_handle == IB_INVALID_HANDLE) {\r
- ret = DAT_INSUFFICIENT_RESOURCES;\r
- goto err;\r
- }\r
-\r
- /* arm cq for events */\r
- dapls_set_cq_notify(ia_ptr, evd_ptr);\r
-\r
- /* update with returned cq entry size */\r
- *cqlen = evd_ptr->ib_cq_handle->cqe;\r
-\r
- dapl_dbg_log(DAPL_DBG_TYPE_UTIL,\r
- "dapls_ib_cq_alloc: new_cq %p cqlen=%d \n",\r
- evd_ptr->ib_cq_handle, *cqlen);\r
-\r
- return DAT_SUCCESS;\r
-\r
-err:\r
- if (!evd_ptr->cno_ptr)\r
- ibv_destroy_comp_channel(channel);\r
- return ret;\r
-}\r
-\r
-/*\r
- * dapls_ib_cq_free\r
- *\r
- * destroy a CQ\r
- *\r
- * Input:\r
- * ia_handle IA handle\r
- * evd_ptr pointer to EVD struct\r
- *\r
- * Output:\r
- * none\r
- *\r
- * Returns:\r
- * DAT_SUCCESS\r
- * DAT_INVALID_PARAMETER\r
- *\r
- */\r
-DAT_RETURN dapls_ib_cq_free(IN DAPL_IA * ia_ptr, IN DAPL_EVD * evd_ptr)\r
-{\r
- DAT_EVENT event;\r
- ib_work_completion_t wc;\r
- struct ibv_comp_channel *channel;\r
-\r
- if (evd_ptr->ib_cq_handle != IB_INVALID_HANDLE) {\r
- /* pull off CQ and EVD entries and toss */\r
- while (ibv_poll_cq(evd_ptr->ib_cq_handle, 1, &wc) == 1) ;\r
- while (dapl_evd_dequeue(evd_ptr, &event) == DAT_SUCCESS) ;\r
-\r
- channel = evd_ptr->ib_cq_handle->channel;\r
- if (ibv_destroy_cq(evd_ptr->ib_cq_handle))\r
- return (dapl_convert_errno(errno, "ibv_destroy_cq"));\r
- if (!evd_ptr->cno_ptr)\r
- ibv_destroy_comp_channel(channel);\r
- evd_ptr->ib_cq_handle = IB_INVALID_HANDLE;\r
- }\r
- return DAT_SUCCESS;\r
-}\r
-\r
-DAT_RETURN\r
-dapls_evd_dto_wakeup(IN DAPL_EVD * evd_ptr)\r
-{\r
- dapl_dbg_log(DAPL_DBG_TYPE_UTIL,\r
- " cq_object_wakeup: evd=%p\n", evd_ptr);\r
-\r
- /* EVD with CNO; waiting on OS wait object */\r
- if (evd_ptr->cno_ptr)\r
- dapl_os_wait_object_wakeup(&evd_ptr->wait_object);\r
-\r
- /* otherwise, no wake up mechanism */\r
- return DAT_SUCCESS;\r
-}\r
-\r
-#if defined(_WIN32)\r
-static int\r
-dapls_wait_comp_channel(IN struct ibv_comp_channel *channel, IN uint32_t timeout)\r
-{\r
- channel->comp_channel.Milliseconds =\r
- (timeout == DAT_TIMEOUT_INFINITE) ? INFINITE : timeout / 1000;\r
- return 0;\r
-}\r
-\r
-#else // WIN32\r
-\r
-static int\r
-dapls_wait_comp_channel(IN struct ibv_comp_channel *channel, IN uint32_t timeout)\r
-{\r
- int status, timeout_ms;\r
- struct pollfd cq_fd = {\r
- .fd = channel->fd,\r
- .events = POLLIN,\r
- .revents = 0\r
- };\r
-\r
- /* uDAPL timeout values in usecs */\r
- timeout_ms = (timeout == DAT_TIMEOUT_INFINITE) ? -1 : timeout / 1000;\r
- status = poll(&cq_fd, 1, timeout_ms);\r
- if (status > 0)\r
- return 0;\r
- else if (status == 0)\r
- return ETIMEDOUT;\r
- else\r
- return status;\r
-}\r
-#endif\r
-\r
-DAT_RETURN\r
-dapls_evd_dto_wait(IN DAPL_EVD * evd_ptr, IN uint32_t timeout)\r
-{\r
- struct ibv_comp_channel *channel = evd_ptr->ib_cq_handle->channel;\r
- struct ibv_cq *ibv_cq = NULL;\r
- void *context;\r
- int status;\r
-\r
- dapl_dbg_log(DAPL_DBG_TYPE_UTIL,\r
- " cq_object_wait: EVD %p time %d\n",\r
- evd_ptr, timeout);\r
-\r
- status = dapls_wait_comp_channel(channel, timeout);\r
- if (!status) {\r
- if (!ibv_get_cq_event(channel, &ibv_cq, &context)) {\r
- ibv_ack_cq_events(ibv_cq, 1);\r
- }\r
- }\r
-\r
- dapl_dbg_log(DAPL_DBG_TYPE_UTIL,\r
- " cq_object_wait: RET evd %p ibv_cq %p %s\n",\r
- evd_ptr, ibv_cq, strerror(errno));\r
-\r
- return dapl_convert_errno(status, "cq_wait_object_wait");\r
-}\r
-\r
-void dapli_cq_event_cb(struct _ib_hca_transport *tp)\r
-{\r
- /* check all comp events on this device */\r
- struct dapl_evd *evd = NULL;\r
- struct ibv_cq *ibv_cq = NULL;\r
-\r
- dapl_dbg_log(DAPL_DBG_TYPE_UTIL," dapli_cq_event_cb(%p)\n", tp);\r
-\r
- while (!ibv_get_cq_event(tp->ib_cq, &ibv_cq, (void*)&evd)) {\r
-\r
- if (!DAPL_BAD_HANDLE(evd, DAPL_MAGIC_EVD)) {\r
- /* Both EVD or EVD->CNO event via callback */\r
- dapl_evd_dto_callback(tp->ib_ctx, \r
- evd->ib_cq_handle, (void*)evd);\r
- }\r
-\r
- ibv_ack_cq_events(ibv_cq, 1);\r
- } \r
-}\r
-\r
-/*\r
- * dapl_ib_cq_resize\r
- *\r
- * Alloc a CQ\r
- *\r
- * Input:\r
- * ia_handle IA handle\r
- * evd_ptr pointer to EVD struct\r
- * cqlen minimum QLen\r
- *\r
- * Output:\r
- * none\r
- *\r
- * Returns:\r
- * DAT_SUCCESS\r
- * DAT_INVALID_PARAMETER\r
- *\r
- */\r
-DAT_RETURN\r
-dapls_ib_cq_resize(IN DAPL_IA * ia_ptr,\r
- IN DAPL_EVD * evd_ptr, IN DAT_COUNT * cqlen)\r
-{\r
- ib_cq_handle_t old_cq, new_cq;\r
- DAT_RETURN ret;\r
-\r
- old_cq = evd_ptr->ib_cq_handle;\r
- ret = dapls_ib_cq_alloc(ia_ptr, evd_ptr, cqlen);\r
- if (ret)\r
- goto err;\r
-\r
- new_cq = evd_ptr->ib_cq_handle;\r
- evd_ptr->ib_cq_handle = old_cq;\r
- dapls_ib_cq_free(ia_ptr, evd_ptr);\r
- evd_ptr->ib_cq_handle = new_cq;\r
- return DAT_SUCCESS;\r
-\r
-err:\r
- evd_ptr->ib_cq_handle = old_cq;\r
- return ret;\r
-}\r
-\r
-/*\r
- * dapls_set_cq_notify\r
- *\r
- * Set the CQ notification for next\r
- *\r
- * Input:\r
- * hca_handl hca handle\r
- * DAPL_EVD evd handle\r
- *\r
- * Output:\r
- * none\r
- *\r
- * Returns:\r
- * DAT_SUCCESS\r
- * dapl_convert_errno \r
- */\r
-DAT_RETURN dapls_set_cq_notify(IN DAPL_IA * ia_ptr, IN DAPL_EVD * evd_ptr)\r
-{\r
- if (ibv_req_notify_cq(evd_ptr->ib_cq_handle, 0))\r
- return (dapl_convert_errno(errno, "notify_cq"));\r
- else\r
- return DAT_SUCCESS;\r
-}\r
-\r
-/*\r
- * dapls_ib_completion_notify\r
- *\r
- * Set the CQ notification type\r
- *\r
- * Input:\r
- * hca_handl hca handle\r
- * evd_ptr evd handle\r
- * type notification type\r
- *\r
- * Output:\r
- * none\r
- *\r
- * Returns:\r
- * DAT_SUCCESS\r
- * dapl_convert_errno\r
- */\r
-DAT_RETURN dapls_ib_completion_notify(IN ib_hca_handle_t hca_handle,\r
- IN DAPL_EVD * evd_ptr,\r
- IN ib_notification_type_t type)\r
-{\r
- if (ibv_req_notify_cq(evd_ptr->ib_cq_handle, type))\r
- return (dapl_convert_errno(errno, "notify_cq_type"));\r
- else\r
- return DAT_SUCCESS;\r
-}\r
-\r
-/*\r
- * dapls_ib_completion_poll\r
- *\r
- * CQ poll for completions\r
- *\r
- * Input:\r
- * hca_handl hca handle\r
- * evd_ptr evd handle\r
- * wc_ptr work completion\r
- *\r
- * Output:\r
- * none\r
- *\r
- * Returns: \r
- * DAT_SUCCESS\r
- * DAT_QUEUE_EMPTY\r
- * \r
- */\r
-DAT_RETURN dapls_ib_completion_poll(IN DAPL_HCA * hca_ptr,\r
- IN DAPL_EVD * evd_ptr,\r
- IN ib_work_completion_t * wc_ptr)\r
-{\r
- int ret;\r
-\r
- ret = ibv_poll_cq(evd_ptr->ib_cq_handle, 1, wc_ptr);\r
- if (ret == 1)\r
- return DAT_SUCCESS;\r
-\r
- return DAT_QUEUE_EMPTY;\r
-}\r
-\r
-/*\r
- * Local variables:\r
- * c-indent-level: 4\r
- * c-basic-offset: 4\r
- * tab-width: 8\r
- * End:\r
- */\r
+/*
+ * Copyright (c) 2009 Intel Corporation. All rights reserved.
+ *
+ * This Software is licensed under one of the following licenses:
+ *
+ * 1) under the terms of the "Common Public License 1.0" a copy of which is
+ * available from the Open Source Initiative, see
+ * http://www.opensource.org/licenses/cpl.php.
+ *
+ * 2) under the terms of the "The BSD License" a copy of which is
+ * available from the Open Source Initiative, see
+ * http://www.opensource.org/licenses/bsd-license.php.
+ *
+ * 3) under the terms of the "GNU General Public License (GPL) Version 2" a
+ * copy of which is available from the Open Source Initiative, see
+ * http://www.opensource.org/licenses/gpl-license.php.
+ *
+ * Licensee has the right to choose one of the above licenses.
+ *
+ * Redistributions of source code must retain the above copyright
+ * notice and one of the license notices.
+ *
+ * Redistributions in binary form must reproduce both the above copyright
+ * notice, one of the license notices in the documentation
+ * and/or other materials provided with the distribution.
+ */
+#include "openib_osd.h"
+#include "dapl.h"
+#include "dapl_adapter_util.h"
+#include "dapl_lmr_util.h"
+#include "dapl_evd_util.h"
+#include "dapl_ring_buffer_util.h"
+
+/*
+ * Map all verbs DTO completion codes to the DAT equivelent.
+ *
+ * Not returned by verbs: DAT_DTO_ERR_PARTIAL_PACKET
+ */
+static struct ib_status_map {
+ int ib_status;
+ DAT_DTO_COMPLETION_STATUS dat_status;
+} ib_status_map[] = {
+/* 00 */ {IBV_WC_SUCCESS, DAT_DTO_SUCCESS},
+/* 01 */ {IBV_WC_LOC_LEN_ERR, DAT_DTO_ERR_LOCAL_LENGTH},
+/* 02 */ {IBV_WC_LOC_QP_OP_ERR, DAT_DTO_ERR_LOCAL_EP},
+/* 03 */ {IBV_WC_LOC_EEC_OP_ERR, DAT_DTO_ERR_TRANSPORT},
+/* 04 */ {IBV_WC_LOC_PROT_ERR, DAT_DTO_ERR_LOCAL_PROTECTION},
+/* 05 */ {IBV_WC_WR_FLUSH_ERR, DAT_DTO_ERR_FLUSHED},
+/* 06 */ {IBV_WC_MW_BIND_ERR, DAT_RMR_OPERATION_FAILED},
+/* 07 */ {IBV_WC_BAD_RESP_ERR, DAT_DTO_ERR_BAD_RESPONSE},
+/* 08 */ {IBV_WC_LOC_ACCESS_ERR, DAT_DTO_ERR_LOCAL_PROTECTION},
+/* 09 */ {IBV_WC_REM_INV_REQ_ERR, DAT_DTO_ERR_REMOTE_RESPONDER},
+/* 10 */ {IBV_WC_REM_ACCESS_ERR, DAT_DTO_ERR_REMOTE_ACCESS},
+/* 11 */ {IBV_WC_REM_OP_ERR, DAT_DTO_ERR_REMOTE_RESPONDER},
+/* 12 */ {IBV_WC_RETRY_EXC_ERR, DAT_DTO_ERR_TRANSPORT},
+/* 13 */ {IBV_WC_RNR_RETRY_EXC_ERR, DAT_DTO_ERR_RECEIVER_NOT_READY},
+/* 14 */ {IBV_WC_LOC_RDD_VIOL_ERR, DAT_DTO_ERR_LOCAL_PROTECTION},
+/* 15 */ {IBV_WC_REM_INV_RD_REQ_ERR, DAT_DTO_ERR_REMOTE_RESPONDER},
+/* 16 */ {IBV_WC_REM_ABORT_ERR, DAT_DTO_ERR_REMOTE_RESPONDER},
+/* 17 */ {IBV_WC_INV_EECN_ERR, DAT_DTO_ERR_TRANSPORT},
+/* 18 */ {IBV_WC_INV_EEC_STATE_ERR, DAT_DTO_ERR_TRANSPORT},
+/* 19 */ {IBV_WC_FATAL_ERR, DAT_DTO_ERR_TRANSPORT},
+/* 20 */ {IBV_WC_RESP_TIMEOUT_ERR, DAT_DTO_ERR_RECEIVER_NOT_READY},
+/* 21 */ {IBV_WC_GENERAL_ERR, DAT_DTO_ERR_TRANSPORT},
+};
+
+/*
+ * dapls_ib_get_dto_status
+ *
+ * Return the DAT status of a DTO operation
+ *
+ * Input:
+ * cqe_ptr pointer to completion queue entry
+ *
+ * Output:
+ * none
+ *
+ * Returns:
+ * Value from ib_status_map table above
+ */
+
+DAT_DTO_COMPLETION_STATUS
+dapls_ib_get_dto_status(IN ib_work_completion_t * cqe_ptr)
+{
+ uint32_t ib_status;
+ int i;
+
+ ib_status = DAPL_GET_CQE_STATUS(cqe_ptr);
+
+ /*
+ * Due to the implementation of verbs completion code, we need to
+ * search the table for the correct value rather than assuming
+ * linear distribution.
+ */
+ for (i = 0; i <= IBV_WC_GENERAL_ERR; i++) {
+ if (ib_status == ib_status_map[i].ib_status) {
+ if (ib_status != IBV_WC_SUCCESS) {
+ dapl_log(DAPL_DBG_TYPE_DTO_COMP_ERR,
+ " DTO completion ERROR: %d:"
+ " op %#x\n",
+ ib_status,
+ DAPL_GET_CQE_OPTYPE(cqe_ptr));
+ }
+ return ib_status_map[i].dat_status;
+ }
+ }
+
+ return DAT_DTO_FAILURE;
+}
+
+DAT_RETURN dapls_ib_get_async_event(IN ib_error_record_t * err_record,
+ OUT DAT_EVENT_NUMBER * async_event)
+{
+ DAT_RETURN dat_status = DAT_SUCCESS;
+ int err_code = err_record->event_type;
+
+ switch (err_code) {
+ /* OVERFLOW error */
+ case IBV_EVENT_CQ_ERR:
+ *async_event = DAT_ASYNC_ERROR_EVD_OVERFLOW;
+ break;
+ /* INTERNAL errors */
+ case IBV_EVENT_DEVICE_FATAL:
+ *async_event = DAT_ASYNC_ERROR_PROVIDER_INTERNAL_ERROR;
+ break;
+ /* CATASTROPHIC errors */
+ case IBV_EVENT_PORT_ERR:
+ *async_event = DAT_ASYNC_ERROR_IA_CATASTROPHIC;
+ break;
+ /* BROKEN QP error */
+ case IBV_EVENT_SQ_DRAINED:
+ case IBV_EVENT_QP_FATAL:
+ case IBV_EVENT_QP_REQ_ERR:
+ case IBV_EVENT_QP_ACCESS_ERR:
+ *async_event = DAT_ASYNC_ERROR_EP_BROKEN;
+ break;
+
+ /* connection completion */
+ case IBV_EVENT_COMM_EST:
+ *async_event = DAT_CONNECTION_EVENT_ESTABLISHED;
+ break;
+
+ /* TODO: process HW state changes */
+ case IBV_EVENT_PATH_MIG:
+ case IBV_EVENT_PATH_MIG_ERR:
+ case IBV_EVENT_PORT_ACTIVE:
+ case IBV_EVENT_LID_CHANGE:
+ case IBV_EVENT_PKEY_CHANGE:
+ case IBV_EVENT_SM_CHANGE:
+ default:
+ dat_status = DAT_ERROR(DAT_NOT_IMPLEMENTED, 0);
+ }
+ return dat_status;
+}
+
+/*
+ * dapl_ib_cq_alloc
+ *
+ * Alloc a CQ
+ *
+ * Input:
+ * ia_handle IA handle
+ * evd_ptr pointer to EVD struct
+ * cqlen minimum QLen
+ *
+ * Output:
+ * none
+ *
+ * Returns:
+ * DAT_SUCCESS
+ * DAT_INSUFFICIENT_RESOURCES
+ *
+ */
+DAT_RETURN
+dapls_ib_cq_alloc(IN DAPL_IA * ia_ptr,
+ IN DAPL_EVD * evd_ptr, IN DAT_COUNT * cqlen)
+{
+ struct ibv_comp_channel *channel;
+ DAT_RETURN ret;
+
+ dapl_dbg_log(DAPL_DBG_TYPE_UTIL,
+ "dapls_ib_cq_alloc: evd %p cqlen=%d \n", evd_ptr, *cqlen);
+
+ if (!evd_ptr->cno_ptr)
+ channel = ibv_create_comp_channel(ia_ptr->hca_ptr->ib_hca_handle);
+ else
+ channel = ia_ptr->hca_ptr->ib_trans.ib_cq;
+
+ if (!channel)
+ return DAT_INSUFFICIENT_RESOURCES;
+
+ evd_ptr->ib_cq_handle = ibv_create_cq(ia_ptr->hca_ptr->ib_hca_handle,
+ *cqlen, evd_ptr, channel, 0);
+
+ if (evd_ptr->ib_cq_handle == IB_INVALID_HANDLE) {
+ ret = DAT_INSUFFICIENT_RESOURCES;
+ goto err;
+ }
+
+ /* arm cq for events */
+ dapls_set_cq_notify(ia_ptr, evd_ptr);
+
+ /* update with returned cq entry size */
+ *cqlen = evd_ptr->ib_cq_handle->cqe;
+
+ dapl_dbg_log(DAPL_DBG_TYPE_UTIL,
+ "dapls_ib_cq_alloc: new_cq %p cqlen=%d \n",
+ evd_ptr->ib_cq_handle, *cqlen);
+
+ return DAT_SUCCESS;
+
+err:
+ if (!evd_ptr->cno_ptr)
+ ibv_destroy_comp_channel(channel);
+ return ret;
+}
+
+/*
+ * dapls_ib_cq_free
+ *
+ * destroy a CQ
+ *
+ * Input:
+ * ia_handle IA handle
+ * evd_ptr pointer to EVD struct
+ *
+ * Output:
+ * none
+ *
+ * Returns:
+ * DAT_SUCCESS
+ * DAT_INVALID_PARAMETER
+ *
+ */
+DAT_RETURN dapls_ib_cq_free(IN DAPL_IA * ia_ptr, IN DAPL_EVD * evd_ptr)
+{
+ DAT_EVENT event;
+ ib_work_completion_t wc;
+ struct ibv_comp_channel *channel;
+
+ if (evd_ptr->ib_cq_handle != IB_INVALID_HANDLE) {
+ /* pull off CQ and EVD entries and toss */
+ while (ibv_poll_cq(evd_ptr->ib_cq_handle, 1, &wc) == 1) ;
+ while (dapl_evd_dequeue(evd_ptr, &event) == DAT_SUCCESS) ;
+
+ channel = evd_ptr->ib_cq_handle->channel;
+ if (ibv_destroy_cq(evd_ptr->ib_cq_handle))
+ return (dapl_convert_errno(errno, "ibv_destroy_cq"));
+ if (!evd_ptr->cno_ptr)
+ ibv_destroy_comp_channel(channel);
+ evd_ptr->ib_cq_handle = IB_INVALID_HANDLE;
+ }
+ return DAT_SUCCESS;
+}
+
+DAT_RETURN
+dapls_evd_dto_wakeup(IN DAPL_EVD * evd_ptr)
+{
+ dapl_dbg_log(DAPL_DBG_TYPE_UTIL,
+ " cq_object_wakeup: evd=%p\n", evd_ptr);
+
+ /* EVD with CNO; waiting on OS wait object */
+ if (evd_ptr->cno_ptr)
+ dapl_os_wait_object_wakeup(&evd_ptr->wait_object);
+
+ /* otherwise, no wake up mechanism */
+ return DAT_SUCCESS;
+}
+
+#if defined(_WIN32)
+static int
+dapls_wait_comp_channel(IN struct ibv_comp_channel *channel, IN uint32_t timeout)
+{
+ channel->comp_channel.Milliseconds =
+ (timeout == DAT_TIMEOUT_INFINITE) ? INFINITE : timeout / 1000;
+ return 0;
+}
+
+#else // WIN32
+
+static int
+dapls_wait_comp_channel(IN struct ibv_comp_channel *channel, IN uint32_t timeout)
+{
+ int status, timeout_ms;
+ struct pollfd cq_fd = {
+ .fd = channel->fd,
+ .events = POLLIN,
+ .revents = 0
+ };
+
+ /* uDAPL timeout values in usecs */
+ timeout_ms = (timeout == DAT_TIMEOUT_INFINITE) ? -1 : timeout / 1000;
+ status = poll(&cq_fd, 1, timeout_ms);
+ if (status > 0)
+ return 0;
+ else if (status == 0)
+ return ETIMEDOUT;
+ else
+ return status;
+}
+#endif
+
+DAT_RETURN
+dapls_evd_dto_wait(IN DAPL_EVD * evd_ptr, IN uint32_t timeout)
+{
+ struct ibv_comp_channel *channel = evd_ptr->ib_cq_handle->channel;
+ struct ibv_cq *ibv_cq = NULL;
+ void *context;
+ int status;
+
+ dapl_dbg_log(DAPL_DBG_TYPE_UTIL,
+ " cq_object_wait: EVD %p time %d\n",
+ evd_ptr, timeout);
+
+ status = dapls_wait_comp_channel(channel, timeout);
+ if (!status) {
+ if (!ibv_get_cq_event(channel, &ibv_cq, &context)) {
+ ibv_ack_cq_events(ibv_cq, 1);
+ }
+ }
+
+ dapl_dbg_log(DAPL_DBG_TYPE_UTIL,
+ " cq_object_wait: RET evd %p ibv_cq %p %s\n",
+ evd_ptr, ibv_cq, strerror(errno));
+
+ return dapl_convert_errno(status, "cq_wait_object_wait");
+}
+
+void dapli_cq_event_cb(struct _ib_hca_transport *tp)
+{
+ /* check all comp events on this device */
+ struct dapl_evd *evd = NULL;
+ struct ibv_cq *ibv_cq = NULL;
+
+ dapl_dbg_log(DAPL_DBG_TYPE_UTIL," dapli_cq_event_cb(%p)\n", tp);
+
+ while (!ibv_get_cq_event(tp->ib_cq, &ibv_cq, (void*)&evd)) {
+
+ if (!DAPL_BAD_HANDLE(evd, DAPL_MAGIC_EVD)) {
+ /* Both EVD or EVD->CNO event via callback */
+ dapl_evd_dto_callback(tp->ib_ctx,
+ evd->ib_cq_handle, (void*)evd);
+ }
+
+ ibv_ack_cq_events(ibv_cq, 1);
+ }
+}
+
+/*
+ * dapl_ib_cq_resize
+ *
+ * Alloc a CQ
+ *
+ * Input:
+ * ia_handle IA handle
+ * evd_ptr pointer to EVD struct
+ * cqlen minimum QLen
+ *
+ * Output:
+ * none
+ *
+ * Returns:
+ * DAT_SUCCESS
+ * DAT_INVALID_PARAMETER
+ *
+ */
+DAT_RETURN
+dapls_ib_cq_resize(IN DAPL_IA * ia_ptr,
+ IN DAPL_EVD * evd_ptr, IN DAT_COUNT * cqlen)
+{
+ ib_cq_handle_t old_cq, new_cq;
+ DAT_RETURN ret;
+
+ old_cq = evd_ptr->ib_cq_handle;
+ ret = dapls_ib_cq_alloc(ia_ptr, evd_ptr, cqlen);
+ if (ret)
+ goto err;
+
+ new_cq = evd_ptr->ib_cq_handle;
+ evd_ptr->ib_cq_handle = old_cq;
+ dapls_ib_cq_free(ia_ptr, evd_ptr);
+ evd_ptr->ib_cq_handle = new_cq;
+ return DAT_SUCCESS;
+
+err:
+ evd_ptr->ib_cq_handle = old_cq;
+ return ret;
+}
+
+/*
+ * dapls_set_cq_notify
+ *
+ * Set the CQ notification for next
+ *
+ * Input:
+ * hca_handl hca handle
+ * DAPL_EVD evd handle
+ *
+ * Output:
+ * none
+ *
+ * Returns:
+ * DAT_SUCCESS
+ * dapl_convert_errno
+ */
+DAT_RETURN dapls_set_cq_notify(IN DAPL_IA * ia_ptr, IN DAPL_EVD * evd_ptr)
+{
+ if (ibv_req_notify_cq(evd_ptr->ib_cq_handle, 0))
+ return (dapl_convert_errno(errno, "notify_cq"));
+ else
+ return DAT_SUCCESS;
+}
+
+/*
+ * dapls_ib_completion_notify
+ *
+ * Set the CQ notification type
+ *
+ * Input:
+ * hca_handl hca handle
+ * evd_ptr evd handle
+ * type notification type
+ *
+ * Output:
+ * none
+ *
+ * Returns:
+ * DAT_SUCCESS
+ * dapl_convert_errno
+ */
+DAT_RETURN dapls_ib_completion_notify(IN ib_hca_handle_t hca_handle,
+ IN DAPL_EVD * evd_ptr,
+ IN ib_notification_type_t type)
+{
+ if (ibv_req_notify_cq(evd_ptr->ib_cq_handle, type))
+ return (dapl_convert_errno(errno, "notify_cq_type"));
+ else
+ return DAT_SUCCESS;
+}
+
+/*
+ * dapls_ib_completion_poll
+ *
+ * CQ poll for completions
+ *
+ * Input:
+ * hca_handl hca handle
+ * evd_ptr evd handle
+ * wc_ptr work completion
+ *
+ * Output:
+ * none
+ *
+ * Returns:
+ * DAT_SUCCESS
+ * DAT_QUEUE_EMPTY
+ *
+ */
+DAT_RETURN dapls_ib_completion_poll(IN DAPL_HCA * hca_ptr,
+ IN DAPL_EVD * evd_ptr,
+ IN ib_work_completion_t * wc_ptr)
+{
+ int ret;
+
+ ret = ibv_poll_cq(evd_ptr->ib_cq_handle, 1, wc_ptr);
+ if (ret == 1)
+ return DAT_SUCCESS;
+
+ return DAT_QUEUE_EMPTY;
+}
+
+/*
+ * Local variables:
+ * c-indent-level: 4
+ * c-basic-offset: 4
+ * tab-width: 8
+ * End:
+ */
-/*\r
- * Copyright (c) 2009 Intel Corporation. All rights reserved.\r
- *\r
- * This Software is licensed under one of the following licenses:\r
- *\r
- * 1) under the terms of the "Common Public License 1.0" a copy of which is\r
- * available from the Open Source Initiative, see\r
- * http://www.opensource.org/licenses/cpl.php.\r
- *\r
- * 2) under the terms of the "The BSD License" a copy of which is\r
- * available from the Open Source Initiative, see\r
- * http://www.opensource.org/licenses/bsd-license.php.\r
- *\r
- * 3) under the terms of the "GNU General Public License (GPL) Version 2" a\r
- * copy of which is available from the Open Source Initiative, see\r
- * http://www.opensource.org/licenses/gpl-license.php.\r
- *\r
- * Licensee has the right to choose one of the above licenses.\r
- *\r
- * Redistributions of source code must retain the above copyright\r
- * notice and one of the license notices.\r
- *\r
- * Redistributions in binary form must reproduce both the above copyright\r
- * notice, one of the license notices in the documentation\r
- * and/or other materials provided with the distribution.\r
- */\r
-\r
-/* \r
- * Definitions common to all OpenIB providers, cma, scm, ucm \r
- */\r
-\r
-#ifndef _DAPL_IB_COMMON_H_\r
-#define _DAPL_IB_COMMON_H_\r
-\r
-#include <infiniband/verbs.h>\r
-\r
-#ifdef DAT_EXTENSIONS\r
-#include <dat2/dat_ib_extensions.h>\r
-#endif\r
-\r
-#ifndef __cplusplus\r
-#define false 0\r
-#define true 1\r
-#endif /*__cplusplus */\r
-\r
-/* Typedefs to map common DAPL provider types to IB verbs */\r
-typedef struct ibv_qp *ib_qp_handle_t;\r
-typedef struct ibv_cq *ib_cq_handle_t;\r
-typedef struct ibv_pd *ib_pd_handle_t;\r
-typedef struct ibv_mr *ib_mr_handle_t;\r
-typedef struct ibv_mw *ib_mw_handle_t;\r
-typedef struct ibv_wc ib_work_completion_t;\r
-typedef struct ibv_ah *ib_ah_handle_t;\r
-typedef union ibv_gid *ib_gid_handle_t;\r
-\r
-/* HCA context type maps to IB verbs */\r
-typedef struct ibv_context *ib_hca_handle_t;\r
-typedef ib_hca_handle_t dapl_ibal_ca_t;\r
-\r
-/* QP info to exchange, wire protocol version for these CM's */\r
-#define DCM_VER 6\r
-\r
-/* CM private data areas, same for all operations */\r
-#define DCM_MAX_PDATA_SIZE 118\r
-\r
-/*\r
- * UCM DAPL IB/QP address (lid, qp_num, gid) mapping to\r
- * DAT_IA_ADDRESS_PTR, DAT_SOCK_ADDR2 (28 bytes)\r
- * For applications, like MPI, that exchange IA_ADDRESS\r
- * across the fabric before connecting, it eliminates the\r
- * overhead of name and address resolution to the destination's\r
- * CM services. UCM provider uses the following for \r
- * DAT_IA_ADDRESS. Note: family == AF_INET6 to insure proper\r
- * callee storage for address.\r
- */\r
-union dcm_addr {\r
- DAT_SOCK_ADDR6 so;\r
- struct {\r
- uint16_t family; /* sin6_family */\r
- uint16_t lid; /* sin6_port */\r
- uint32_t qpn; /* sin6_flowinfo */\r
- uint8_t gid[16]; /* sin6_addr */\r
- uint16_t port; /* sin6_scope_id */\r
- uint8_t sl;\r
- uint8_t qp_type;\r
- } ib;\r
-};\r
-\r
-/* 256 bytes total; default max_inline_send, min IB MTU size */\r
-typedef struct _ib_cm_msg\r
-{\r
- uint16_t ver;\r
- uint16_t op;\r
- uint16_t sport; /* src cm port */\r
- uint16_t dport; /* dst cm port */\r
- uint32_t sqpn; /* src cm qpn */\r
- uint32_t dqpn; /* dst cm qpn */\r
- uint16_t p_size;\r
- uint8_t resv[14];\r
- union dcm_addr saddr;\r
- union dcm_addr daddr;\r
- union dcm_addr saddr_alt;\r
- union dcm_addr daddr_alt;\r
- uint8_t p_data[DCM_MAX_PDATA_SIZE];\r
-\r
-} ib_cm_msg_t;\r
-\r
-/* CM events */\r
-typedef enum {\r
- IB_CME_CONNECTED,\r
- IB_CME_DISCONNECTED,\r
- IB_CME_DISCONNECTED_ON_LINK_DOWN,\r
- IB_CME_CONNECTION_REQUEST_PENDING,\r
- IB_CME_CONNECTION_REQUEST_PENDING_PRIVATE_DATA,\r
- IB_CME_CONNECTION_REQUEST_ACKED,\r
- IB_CME_DESTINATION_REJECT,\r
- IB_CME_DESTINATION_REJECT_PRIVATE_DATA,\r
- IB_CME_DESTINATION_UNREACHABLE,\r
- IB_CME_TOO_MANY_CONNECTION_REQUESTS,\r
- IB_CME_LOCAL_FAILURE,\r
- IB_CME_BROKEN,\r
- IB_CME_TIMEOUT\r
-} ib_cm_events_t;\r
-\r
-/* Operation and state mappings */\r
-typedef int ib_send_op_type_t;\r
-typedef struct ibv_sge ib_data_segment_t;\r
-typedef enum ibv_qp_state ib_qp_state_t;\r
-typedef enum ibv_event_type ib_async_event_type;\r
-typedef struct ibv_async_event ib_error_record_t; \r
-\r
-/* CQ notifications */\r
-typedef enum\r
-{\r
- IB_NOTIFY_ON_NEXT_COMP,\r
- IB_NOTIFY_ON_SOLIC_COMP\r
-\r
-} ib_notification_type_t;\r
-\r
-/* other mappings */\r
-typedef int ib_bool_t;\r
-typedef union ibv_gid GID;\r
-typedef char *IB_HCA_NAME;\r
-typedef uint16_t ib_hca_port_t;\r
-\r
-/* Definitions */\r
-#define IB_INVALID_HANDLE NULL\r
-\r
-/* inline send rdma threshold */\r
-#define INLINE_SEND_IWARP_DEFAULT 64\r
-#define INLINE_SEND_IB_DEFAULT 256\r
-\r
-/* qkey for UD QP's */\r
-#define DAT_UD_QKEY 0x78654321\r
-\r
-/* RC timer - retry count defaults */\r
-#define DCM_ACK_TIMER 16 /* 5 bits, 4.096us*2^ack_timer. 16== 268ms */\r
-#define DCM_ACK_RETRY 7 /* 3 bits, 7 * 268ms = 1.8 seconds */\r
-#define DCM_RNR_TIMER 12 /* 5 bits, 12 =.64ms, 28 =163ms, 31 =491ms */\r
-#define DCM_RNR_RETRY 7 /* 3 bits, 7 == infinite */\r
-#define DCM_IB_MTU 2048\r
-\r
-/* Global routing defaults */\r
-#define DCM_GLOBAL 0 /* global routing is disabled */\r
-#define DCM_HOP_LIMIT 0xff\r
-#define DCM_TCLASS 0\r
-\r
-/* DAPL uCM timers, default queue sizes */\r
-#define DCM_RETRY_CNT 10 \r
-#define DCM_REP_TIME 800 /* reply timeout in m_secs */\r
-#define DCM_RTU_TIME 400 /* rtu timeout in m_secs */\r
-#define DCM_QP_SIZE 500 /* uCM tx, rx qp size */\r
-#define DCM_CQ_SIZE 500 /* uCM cq size */\r
-\r
-/* DTO OPs, ordered for DAPL ENUM definitions */\r
-#define OP_RDMA_WRITE IBV_WR_RDMA_WRITE\r
-#define OP_RDMA_WRITE_IMM IBV_WR_RDMA_WRITE_WITH_IMM\r
-#define OP_SEND IBV_WR_SEND\r
-#define OP_SEND_IMM IBV_WR_SEND_WITH_IMM\r
-#define OP_RDMA_READ IBV_WR_RDMA_READ\r
-#define OP_COMP_AND_SWAP IBV_WR_ATOMIC_CMP_AND_SWP\r
-#define OP_FETCH_AND_ADD IBV_WR_ATOMIC_FETCH_AND_ADD\r
-#define OP_RECEIVE 7 /* internal op */\r
-#define OP_RECEIVE_IMM 8 /* rdma write with immed, internel op */\r
-#define OP_RECEIVE_MSG_IMM 9 /* recv msg with immed, internel op */\r
-#define OP_BIND_MW 10 /* internal op */\r
-#define OP_SEND_UD 11 /* internal op */\r
-#define OP_RECV_UD 12 /* internal op */\r
-#define OP_INVALID 0xff\r
-\r
-/* Definitions to map QP state */\r
-#define IB_QP_STATE_RESET IBV_QPS_RESET\r
-#define IB_QP_STATE_INIT IBV_QPS_INIT\r
-#define IB_QP_STATE_RTR IBV_QPS_RTR\r
-#define IB_QP_STATE_RTS IBV_QPS_RTS\r
-#define IB_QP_STATE_SQD IBV_QPS_SQD\r
-#define IB_QP_STATE_SQE IBV_QPS_SQE\r
-#define IB_QP_STATE_ERROR IBV_QPS_ERR\r
-\r
-/* Definitions for ibverbs/mthca return codes, should be defined in verbs.h */\r
-/* some are errno and some are -n values */\r
-\r
-/**\r
- * ibv_get_device_name - Return kernel device name\r
- * ibv_get_device_guid - Return device's node GUID\r
- * ibv_open_device - Return ibv_context or NULL\r
- * ibv_close_device - Return 0, (errno?)\r
- * ibv_get_async_event - Return 0, -1 \r
- * ibv_alloc_pd - Return ibv_pd, NULL\r
- * ibv_dealloc_pd - Return 0, errno \r
- * ibv_reg_mr - Return ibv_mr, NULL\r
- * ibv_dereg_mr - Return 0, errno\r
- * ibv_create_cq - Return ibv_cq, NULL\r
- * ibv_destroy_cq - Return 0, errno\r
- * ibv_get_cq_event - Return 0 & ibv_cq/context, int\r
- * ibv_poll_cq - Return n & ibv_wc, 0 ok, -1 empty, -2 error \r
- * ibv_req_notify_cq - Return 0 (void?)\r
- * ibv_create_qp - Return ibv_qp, NULL\r
- * ibv_modify_qp - Return 0, errno\r
- * ibv_destroy_qp - Return 0, errno\r
- * ibv_post_send - Return 0, -1 & bad_wr\r
- * ibv_post_recv - Return 0, -1 & bad_wr \r
- */\r
-\r
-/* async handler for DTO, CQ, QP, and unafiliated */\r
-typedef void (*ib_async_dto_handler_t)(\r
- IN ib_hca_handle_t ib_hca_handle,\r
- IN ib_error_record_t *err_code,\r
- IN void *context);\r
-\r
-typedef void (*ib_async_cq_handler_t)(\r
- IN ib_hca_handle_t ib_hca_handle,\r
- IN ib_cq_handle_t ib_cq_handle,\r
- IN ib_error_record_t *err_code,\r
- IN void *context);\r
-\r
-typedef void (*ib_async_qp_handler_t)(\r
- IN ib_hca_handle_t ib_hca_handle,\r
- IN ib_qp_handle_t ib_qp_handle,\r
- IN ib_error_record_t *err_code,\r
- IN void *context);\r
-\r
-typedef void (*ib_async_handler_t)(\r
- IN ib_hca_handle_t ib_hca_handle,\r
- IN ib_error_record_t *err_code,\r
- IN void *context);\r
-\r
-typedef enum\r
-{\r
- IB_THREAD_INIT,\r
- IB_THREAD_CREATE,\r
- IB_THREAD_RUN,\r
- IB_THREAD_CANCEL,\r
- IB_THREAD_EXIT\r
-\r
-} ib_thread_state_t;\r
-\r
-typedef enum dapl_cm_op\r
-{\r
- DCM_REQ = 1,\r
- DCM_REP,\r
- DCM_REJ_USER, /* user reject */\r
- DCM_REJ_CM, /* cm reject, no SID */\r
- DCM_RTU,\r
- DCM_DREQ,\r
- DCM_DREP\r
-\r
-} DAPL_CM_OP;\r
-\r
-typedef enum dapl_cm_state\r
-{\r
- DCM_INIT,\r
- DCM_LISTEN,\r
- DCM_CONN_PENDING,\r
- DCM_REP_PENDING,\r
- DCM_ACCEPTING,\r
- DCM_ACCEPTING_DATA,\r
- DCM_ACCEPTED,\r
- DCM_REJECTING,\r
- DCM_REJECTED,\r
- DCM_CONNECTED,\r
- DCM_RELEASED,\r
- DCM_DISC_PENDING,\r
- DCM_DISCONNECTED,\r
- DCM_DESTROY,\r
- DCM_RTU_PENDING,\r
- DCM_DISC_RECV\r
-\r
-} DAPL_CM_STATE;\r
-\r
-/* provider specfic fields for shared memory support */\r
-typedef uint32_t ib_shm_transport_t;\r
-\r
-/* prototypes */\r
-int32_t dapls_ib_init(void);\r
-int32_t dapls_ib_release(void);\r
-\r
-/* util.c */\r
-enum ibv_mtu dapl_ib_mtu(int mtu);\r
-char *dapl_ib_mtu_str(enum ibv_mtu mtu);\r
-DAT_RETURN getlocalipaddr(DAT_SOCK_ADDR *addr, int addr_len);\r
-\r
-/* qp.c */\r
-DAT_RETURN dapls_modify_qp_ud(IN DAPL_HCA *hca, IN ib_qp_handle_t qp);\r
-DAT_RETURN dapls_modify_qp_state(IN ib_qp_handle_t qp_handle,\r
- IN ib_qp_state_t qp_state,\r
- IN uint32_t qpn,\r
- IN uint16_t lid,\r
- IN ib_gid_handle_t gid);\r
-ib_ah_handle_t dapls_create_ah( IN DAPL_HCA *hca,\r
- IN ib_pd_handle_t pd,\r
- IN ib_qp_handle_t qp,\r
- IN uint16_t lid,\r
- IN ib_gid_handle_t gid);\r
-\r
-/* inline functions */\r
-STATIC _INLINE_ IB_HCA_NAME dapl_ib_convert_name (IN char *name)\r
-{\r
- /* use ascii; name of local device */\r
- return dapl_os_strdup(name);\r
-}\r
-\r
-STATIC _INLINE_ void dapl_ib_release_name (IN IB_HCA_NAME name)\r
-{\r
- return;\r
-}\r
-\r
-/*\r
- * Convert errno to DAT_RETURN values\r
- */\r
-STATIC _INLINE_ DAT_RETURN \r
-dapl_convert_errno( IN int err, IN const char *str )\r
-{\r
- if (!err) return DAT_SUCCESS;\r
- \r
-#if DAPL_DBG\r
- if ((err != EAGAIN) && (err != ETIMEDOUT))\r
- dapl_dbg_log (DAPL_DBG_TYPE_ERR," %s %s\n", str, strerror(err));\r
-#endif \r
-\r
- switch( err )\r
- {\r
- case EOVERFLOW : return DAT_LENGTH_ERROR;\r
- case EACCES : return DAT_PRIVILEGES_VIOLATION;\r
- case EPERM : return DAT_PROTECTION_VIOLATION; \r
- case EINVAL : return DAT_INVALID_HANDLE;\r
- case EISCONN : return DAT_INVALID_STATE | DAT_INVALID_STATE_EP_CONNECTED;\r
- case ECONNREFUSED : return DAT_INVALID_STATE | DAT_INVALID_STATE_EP_NOTREADY;\r
- case ETIMEDOUT : return DAT_TIMEOUT_EXPIRED;\r
- case ENETUNREACH: return DAT_INVALID_ADDRESS | DAT_INVALID_ADDRESS_UNREACHABLE;\r
- case EADDRINUSE : return DAT_CONN_QUAL_IN_USE;\r
- case EALREADY : return DAT_INVALID_STATE | DAT_INVALID_STATE_EP_ACTCONNPENDING;\r
- case ENOMEM : return DAT_INSUFFICIENT_RESOURCES;\r
- case EAGAIN : return DAT_QUEUE_EMPTY;\r
- case EINTR : return DAT_INTERRUPTED_CALL;\r
- case EAFNOSUPPORT : return DAT_INVALID_ADDRESS | DAT_INVALID_ADDRESS_MALFORMED;\r
- case EFAULT : \r
- default : return DAT_INTERNAL_ERROR;\r
- }\r
- }\r
-\r
-STATIC _INLINE_ char * dapl_cm_state_str(IN int st)\r
-{\r
- static char *state[] = {\r
- "CM_INIT",\r
- "CM_LISTEN",\r
- "CM_CONN_PENDING",\r
- "CM_REP_PENDING",\r
- "CM_ACCEPTING",\r
- "CM_ACCEPTING_DATA",\r
- "CM_ACCEPTED",\r
- "CM_REJECTING",\r
- "CM_REJECTED",\r
- "CM_CONNECTED",\r
- "CM_RELEASED",\r
- "CM_DISC_PENDING",\r
- "CM_DISCONNECTED",\r
- "CM_DESTROY",\r
- "CM_RTU_PENDING",\r
- "CM_DISC_RECV"\r
- };\r
- return ((st < 0 || st > 15) ? "Invalid CM state?" : state[st]);\r
-}\r
-\r
-STATIC _INLINE_ char * dapl_cm_op_str(IN int op)\r
-{\r
- static char *ops[] = {\r
- "INVALID",\r
- "REQ",\r
- "REP",\r
- "REJ_USER",\r
- "REJ_CM",\r
- "RTU",\r
- "DREQ",\r
- "DREP",\r
- };\r
- return ((op < 1 || op > 7) ? "Invalid OP?" : ops[op]);\r
-}\r
-\r
-#endif /* _DAPL_IB_COMMON_H_ */\r
+/*
+ * Copyright (c) 2009 Intel Corporation. All rights reserved.
+ *
+ * This Software is licensed under one of the following licenses:
+ *
+ * 1) under the terms of the "Common Public License 1.0" a copy of which is
+ * available from the Open Source Initiative, see
+ * http://www.opensource.org/licenses/cpl.php.
+ *
+ * 2) under the terms of the "The BSD License" a copy of which is
+ * available from the Open Source Initiative, see
+ * http://www.opensource.org/licenses/bsd-license.php.
+ *
+ * 3) under the terms of the "GNU General Public License (GPL) Version 2" a
+ * copy of which is available from the Open Source Initiative, see
+ * http://www.opensource.org/licenses/gpl-license.php.
+ *
+ * Licensee has the right to choose one of the above licenses.
+ *
+ * Redistributions of source code must retain the above copyright
+ * notice and one of the license notices.
+ *
+ * Redistributions in binary form must reproduce both the above copyright
+ * notice, one of the license notices in the documentation
+ * and/or other materials provided with the distribution.
+ */
+
+/*
+ * Definitions common to all OpenIB providers, cma, scm, ucm
+ */
+
+#ifndef _DAPL_IB_COMMON_H_
+#define _DAPL_IB_COMMON_H_
+
+#include <infiniband/verbs.h>
+
+#ifdef DAT_EXTENSIONS
+#include <dat2/dat_ib_extensions.h>
+#endif
+
+#ifndef __cplusplus
+#define false 0
+#define true 1
+#endif /*__cplusplus */
+
+/* Typedefs to map common DAPL provider types to IB verbs */
+typedef struct ibv_qp *ib_qp_handle_t;
+typedef struct ibv_cq *ib_cq_handle_t;
+typedef struct ibv_pd *ib_pd_handle_t;
+typedef struct ibv_mr *ib_mr_handle_t;
+typedef struct ibv_mw *ib_mw_handle_t;
+typedef struct ibv_wc ib_work_completion_t;
+typedef struct ibv_ah *ib_ah_handle_t;
+typedef union ibv_gid *ib_gid_handle_t;
+
+/* HCA context type maps to IB verbs */
+typedef struct ibv_context *ib_hca_handle_t;
+typedef ib_hca_handle_t dapl_ibal_ca_t;
+
+/* QP info to exchange, wire protocol version for these CM's */
+#define DCM_VER 6
+
+/* CM private data areas, same for all operations */
+#define DCM_MAX_PDATA_SIZE 118
+
+/*
+ * UCM DAPL IB/QP address (lid, qp_num, gid) mapping to
+ * DAT_IA_ADDRESS_PTR, DAT_SOCK_ADDR2 (28 bytes)
+ * For applications, like MPI, that exchange IA_ADDRESS
+ * across the fabric before connecting, it eliminates the
+ * overhead of name and address resolution to the destination's
+ * CM services. UCM provider uses the following for
+ * DAT_IA_ADDRESS. Note: family == AF_INET6 to insure proper
+ * callee storage for address.
+ */
+union dcm_addr {
+ DAT_SOCK_ADDR6 so;
+ struct {
+ uint16_t family; /* sin6_family */
+ uint16_t lid; /* sin6_port */
+ uint32_t qpn; /* sin6_flowinfo */
+ uint8_t gid[16]; /* sin6_addr */
+ uint16_t port; /* sin6_scope_id */
+ uint8_t sl;
+ uint8_t qp_type;
+ } ib;
+};
+
+/* 256 bytes total; default max_inline_send, min IB MTU size */
+typedef struct _ib_cm_msg
+{
+ uint16_t ver;
+ uint16_t op;
+ uint16_t sport; /* src cm port */
+ uint16_t dport; /* dst cm port */
+ uint32_t sqpn; /* src cm qpn */
+ uint32_t dqpn; /* dst cm qpn */
+ uint16_t p_size;
+ uint8_t resv[14];
+ union dcm_addr saddr;
+ union dcm_addr daddr;
+ union dcm_addr saddr_alt;
+ union dcm_addr daddr_alt;
+ uint8_t p_data[DCM_MAX_PDATA_SIZE];
+
+} ib_cm_msg_t;
+
+/* CM events */
+typedef enum {
+ IB_CME_CONNECTED,
+ IB_CME_DISCONNECTED,
+ IB_CME_DISCONNECTED_ON_LINK_DOWN,
+ IB_CME_CONNECTION_REQUEST_PENDING,
+ IB_CME_CONNECTION_REQUEST_PENDING_PRIVATE_DATA,
+ IB_CME_CONNECTION_REQUEST_ACKED,
+ IB_CME_DESTINATION_REJECT,
+ IB_CME_DESTINATION_REJECT_PRIVATE_DATA,
+ IB_CME_DESTINATION_UNREACHABLE,
+ IB_CME_TOO_MANY_CONNECTION_REQUESTS,
+ IB_CME_LOCAL_FAILURE,
+ IB_CME_BROKEN,
+ IB_CME_TIMEOUT
+} ib_cm_events_t;
+
+/* Operation and state mappings */
+typedef int ib_send_op_type_t;
+typedef struct ibv_sge ib_data_segment_t;
+typedef enum ibv_qp_state ib_qp_state_t;
+typedef enum ibv_event_type ib_async_event_type;
+typedef struct ibv_async_event ib_error_record_t;
+
+/* CQ notifications */
+typedef enum
+{
+ IB_NOTIFY_ON_NEXT_COMP,
+ IB_NOTIFY_ON_SOLIC_COMP
+
+} ib_notification_type_t;
+
+/* other mappings */
+typedef int ib_bool_t;
+typedef union ibv_gid GID;
+typedef char *IB_HCA_NAME;
+typedef uint16_t ib_hca_port_t;
+
+/* Definitions */
+#define IB_INVALID_HANDLE NULL
+
+/* inline send rdma threshold */
+#define INLINE_SEND_IWARP_DEFAULT 64
+#define INLINE_SEND_IB_DEFAULT 256
+
+/* qkey for UD QP's */
+#define DAT_UD_QKEY 0x78654321
+
+/* RC timer - retry count defaults */
+#define DCM_ACK_TIMER 16 /* 5 bits, 4.096us*2^ack_timer. 16== 268ms */
+#define DCM_ACK_RETRY 7 /* 3 bits, 7 * 268ms = 1.8 seconds */
+#define DCM_RNR_TIMER 12 /* 5 bits, 12 =.64ms, 28 =163ms, 31 =491ms */
+#define DCM_RNR_RETRY 7 /* 3 bits, 7 == infinite */
+#define DCM_IB_MTU 2048
+
+/* Global routing defaults */
+#define DCM_GLOBAL 0 /* global routing is disabled */
+#define DCM_HOP_LIMIT 0xff
+#define DCM_TCLASS 0
+
+/* DAPL uCM timers, default queue sizes */
+#define DCM_RETRY_CNT 10
+#define DCM_REP_TIME 800 /* reply timeout in m_secs */
+#define DCM_RTU_TIME 400 /* rtu timeout in m_secs */
+#define DCM_QP_SIZE 500 /* uCM tx, rx qp size */
+#define DCM_CQ_SIZE 500 /* uCM cq size */
+
+/* DTO OPs, ordered for DAPL ENUM definitions */
+#define OP_RDMA_WRITE IBV_WR_RDMA_WRITE
+#define OP_RDMA_WRITE_IMM IBV_WR_RDMA_WRITE_WITH_IMM
+#define OP_SEND IBV_WR_SEND
+#define OP_SEND_IMM IBV_WR_SEND_WITH_IMM
+#define OP_RDMA_READ IBV_WR_RDMA_READ
+#define OP_COMP_AND_SWAP IBV_WR_ATOMIC_CMP_AND_SWP
+#define OP_FETCH_AND_ADD IBV_WR_ATOMIC_FETCH_AND_ADD
+#define OP_RECEIVE 7 /* internal op */
+#define OP_RECEIVE_IMM 8 /* rdma write with immed, internel op */
+#define OP_RECEIVE_MSG_IMM 9 /* recv msg with immed, internel op */
+#define OP_BIND_MW 10 /* internal op */
+#define OP_SEND_UD 11 /* internal op */
+#define OP_RECV_UD 12 /* internal op */
+#define OP_INVALID 0xff
+
+/* Definitions to map QP state */
+#define IB_QP_STATE_RESET IBV_QPS_RESET
+#define IB_QP_STATE_INIT IBV_QPS_INIT
+#define IB_QP_STATE_RTR IBV_QPS_RTR
+#define IB_QP_STATE_RTS IBV_QPS_RTS
+#define IB_QP_STATE_SQD IBV_QPS_SQD
+#define IB_QP_STATE_SQE IBV_QPS_SQE
+#define IB_QP_STATE_ERROR IBV_QPS_ERR
+
+/* Definitions for ibverbs/mthca return codes, should be defined in verbs.h */
+/* some are errno and some are -n values */
+
+/**
+ * ibv_get_device_name - Return kernel device name
+ * ibv_get_device_guid - Return device's node GUID
+ * ibv_open_device - Return ibv_context or NULL
+ * ibv_close_device - Return 0, (errno?)
+ * ibv_get_async_event - Return 0, -1
+ * ibv_alloc_pd - Return ibv_pd, NULL
+ * ibv_dealloc_pd - Return 0, errno
+ * ibv_reg_mr - Return ibv_mr, NULL
+ * ibv_dereg_mr - Return 0, errno
+ * ibv_create_cq - Return ibv_cq, NULL
+ * ibv_destroy_cq - Return 0, errno
+ * ibv_get_cq_event - Return 0 & ibv_cq/context, int
+ * ibv_poll_cq - Return n & ibv_wc, 0 ok, -1 empty, -2 error
+ * ibv_req_notify_cq - Return 0 (void?)
+ * ibv_create_qp - Return ibv_qp, NULL
+ * ibv_modify_qp - Return 0, errno
+ * ibv_destroy_qp - Return 0, errno
+ * ibv_post_send - Return 0, -1 & bad_wr
+ * ibv_post_recv - Return 0, -1 & bad_wr
+ */
+
+/* async handler for DTO, CQ, QP, and unafiliated */
+typedef void (*ib_async_dto_handler_t)(
+ IN ib_hca_handle_t ib_hca_handle,
+ IN ib_error_record_t *err_code,
+ IN void *context);
+
+typedef void (*ib_async_cq_handler_t)(
+ IN ib_hca_handle_t ib_hca_handle,
+ IN ib_cq_handle_t ib_cq_handle,
+ IN ib_error_record_t *err_code,
+ IN void *context);
+
+typedef void (*ib_async_qp_handler_t)(
+ IN ib_hca_handle_t ib_hca_handle,
+ IN ib_qp_handle_t ib_qp_handle,
+ IN ib_error_record_t *err_code,
+ IN void *context);
+
+typedef void (*ib_async_handler_t)(
+ IN ib_hca_handle_t ib_hca_handle,
+ IN ib_error_record_t *err_code,
+ IN void *context);
+
+typedef enum
+{
+ IB_THREAD_INIT,
+ IB_THREAD_CREATE,
+ IB_THREAD_RUN,
+ IB_THREAD_CANCEL,
+ IB_THREAD_EXIT
+
+} ib_thread_state_t;
+
+typedef enum dapl_cm_op
+{
+ DCM_REQ = 1,
+ DCM_REP,
+ DCM_REJ_USER, /* user reject */
+ DCM_REJ_CM, /* cm reject, no SID */
+ DCM_RTU,
+ DCM_DREQ,
+ DCM_DREP
+
+} DAPL_CM_OP;
+
+typedef enum dapl_cm_state
+{
+ DCM_INIT,
+ DCM_LISTEN,
+ DCM_CONN_PENDING,
+ DCM_REP_PENDING,
+ DCM_ACCEPTING,
+ DCM_ACCEPTING_DATA,
+ DCM_ACCEPTED,
+ DCM_REJECTING,
+ DCM_REJECTED,
+ DCM_CONNECTED,
+ DCM_RELEASED,
+ DCM_DISC_PENDING,
+ DCM_DISCONNECTED,
+ DCM_DESTROY,
+ DCM_RTU_PENDING,
+ DCM_DISC_RECV
+
+} DAPL_CM_STATE;
+
+/* provider specfic fields for shared memory support */
+typedef uint32_t ib_shm_transport_t;
+
+/* prototypes */
+int32_t dapls_ib_init(void);
+int32_t dapls_ib_release(void);
+
+/* util.c */
+enum ibv_mtu dapl_ib_mtu(int mtu);
+char *dapl_ib_mtu_str(enum ibv_mtu mtu);
+DAT_RETURN getlocalipaddr(DAT_SOCK_ADDR *addr, int addr_len);
+
+/* qp.c */
+DAT_RETURN dapls_modify_qp_ud(IN DAPL_HCA *hca, IN ib_qp_handle_t qp);
+DAT_RETURN dapls_modify_qp_state(IN ib_qp_handle_t qp_handle,
+ IN ib_qp_state_t qp_state,
+ IN uint32_t qpn,
+ IN uint16_t lid,
+ IN ib_gid_handle_t gid);
+ib_ah_handle_t dapls_create_ah( IN DAPL_HCA *hca,
+ IN ib_pd_handle_t pd,
+ IN ib_qp_handle_t qp,
+ IN uint16_t lid,
+ IN ib_gid_handle_t gid);
+
+/* inline functions */
+STATIC _INLINE_ IB_HCA_NAME dapl_ib_convert_name (IN char *name)
+{
+ /* use ascii; name of local device */
+ return dapl_os_strdup(name);
+}
+
+STATIC _INLINE_ void dapl_ib_release_name (IN IB_HCA_NAME name)
+{
+ return;
+}
+
+/*
+ * Convert errno to DAT_RETURN values
+ */
+STATIC _INLINE_ DAT_RETURN
+dapl_convert_errno( IN int err, IN const char *str )
+{
+ if (!err) return DAT_SUCCESS;
+
+#if DAPL_DBG
+ if ((err != EAGAIN) && (err != ETIMEDOUT))
+ dapl_dbg_log (DAPL_DBG_TYPE_ERR," %s %s\n", str, strerror(err));
+#endif
+
+ switch( err )
+ {
+ case EOVERFLOW : return DAT_LENGTH_ERROR;
+ case EACCES : return DAT_PRIVILEGES_VIOLATION;
+ case EPERM : return DAT_PROTECTION_VIOLATION;
+ case EINVAL : return DAT_INVALID_HANDLE;
+ case EISCONN : return DAT_INVALID_STATE | DAT_INVALID_STATE_EP_CONNECTED;
+ case ECONNREFUSED : return DAT_INVALID_STATE | DAT_INVALID_STATE_EP_NOTREADY;
+ case ETIMEDOUT : return DAT_TIMEOUT_EXPIRED;
+ case ENETUNREACH: return DAT_INVALID_ADDRESS | DAT_INVALID_ADDRESS_UNREACHABLE;
+ case EADDRINUSE : return DAT_CONN_QUAL_IN_USE;
+ case EALREADY : return DAT_INVALID_STATE | DAT_INVALID_STATE_EP_ACTCONNPENDING;
+ case ENOMEM : return DAT_INSUFFICIENT_RESOURCES;
+ case EAGAIN : return DAT_QUEUE_EMPTY;
+ case EINTR : return DAT_INTERRUPTED_CALL;
+ case EAFNOSUPPORT : return DAT_INVALID_ADDRESS | DAT_INVALID_ADDRESS_MALFORMED;
+ case EFAULT :
+ default : return DAT_INTERNAL_ERROR;
+ }
+ }
+
+STATIC _INLINE_ char * dapl_cm_state_str(IN int st)
+{
+ static char *state[] = {
+ "CM_INIT",
+ "CM_LISTEN",
+ "CM_CONN_PENDING",
+ "CM_REP_PENDING",
+ "CM_ACCEPTING",
+ "CM_ACCEPTING_DATA",
+ "CM_ACCEPTED",
+ "CM_REJECTING",
+ "CM_REJECTED",
+ "CM_CONNECTED",
+ "CM_RELEASED",
+ "CM_DISC_PENDING",
+ "CM_DISCONNECTED",
+ "CM_DESTROY",
+ "CM_RTU_PENDING",
+ "CM_DISC_RECV"
+ };
+ return ((st < 0 || st > 15) ? "Invalid CM state?" : state[st]);
+}
+
+STATIC _INLINE_ char * dapl_cm_op_str(IN int op)
+{
+ static char *ops[] = {
+ "INVALID",
+ "REQ",
+ "REP",
+ "REJ_USER",
+ "REJ_CM",
+ "RTU",
+ "DREQ",
+ "DREP",
+ };
+ return ((op < 1 || op > 7) ? "Invalid OP?" : ops[op]);
+}
+
+#endif /* _DAPL_IB_COMMON_H_ */
-/*\r
- * Copyright (c) 2005-2007 Intel Corporation. All rights reserved.\r
- *\r
- * This Software is licensed under one of the following licenses:\r
- * 1) under the terms of the "Common Public License 1.0" a copy of which is\r
- * available from the Open Source Initiative, see\r
- * http://www.opensource.org/licenses/cpl.php.\r
- *\r
- * 2) under the terms of the "The BSD License" a copy of which is\r
- * available from the Open Source Initiative, see\r
- * http://www.opensource.org/licenses/bsd-license.php.\r
- *\r
- * 3) under the terms of the "GNU General Public License (GPL) Version 2" a\r
- * copy of which is available from the Open Source Initiative, see\r
- * http://www.opensource.org/licenses/gpl-license.php.\r
- *\r
- * Licensee has the right to choose one of the above licenses.\r
- *\r
- * Redistributions of source code must retain the above copyright\r
- * notice and one of the license notices.\r
- *\r
- * Redistributions in binary form must reproduce both the above copyright\r
- * notice, one of the license notices in the documentation\r
- * and/or other materials provided with the distribution.\r
- */\r
-#include "dapl.h"\r
-#include "dapl_adapter_util.h"\r
-#include "dapl_lmr_util.h"\r
-\r
-/*\r
- * dapls_convert_privileges\r
- *\r
- * Convert LMR privileges to provider \r
- *\r
- * Input:\r
- * DAT_MEM_PRIV_FLAGS\r
- *\r
- * Output:\r
- * none\r
- *\r
- * Returns:\r
- * ibv_access_flags\r
- *\r
- */\r
-STATIC _INLINE_ int dapls_convert_privileges(IN DAT_MEM_PRIV_FLAGS privileges)\r
-{\r
- int access = 0;\r
-\r
- /*\r
- * if (DAT_MEM_PRIV_LOCAL_READ_FLAG & privileges) do nothing\r
- */\r
- if (DAT_MEM_PRIV_LOCAL_WRITE_FLAG & privileges)\r
- access |= IBV_ACCESS_LOCAL_WRITE;\r
- if (DAT_MEM_PRIV_REMOTE_WRITE_FLAG & privileges)\r
- access |= IBV_ACCESS_REMOTE_WRITE;\r
- if (DAT_MEM_PRIV_REMOTE_READ_FLAG & privileges)\r
- access |= IBV_ACCESS_REMOTE_READ;\r
- if (DAT_MEM_PRIV_REMOTE_READ_FLAG & privileges)\r
- access |= IBV_ACCESS_REMOTE_READ;\r
- if (DAT_MEM_PRIV_REMOTE_READ_FLAG & privileges)\r
- access |= IBV_ACCESS_REMOTE_READ;\r
-#ifdef DAT_EXTENSIONS\r
- if (DAT_IB_MEM_PRIV_REMOTE_ATOMIC & privileges)\r
- access |= IBV_ACCESS_REMOTE_ATOMIC;\r
-#endif\r
-\r
- return access;\r
-}\r
-\r
-/*\r
- * dapl_ib_pd_alloc\r
- *\r
- * Alloc a PD\r
- *\r
- * Input:\r
- * ia_handle IA handle\r
- * pz pointer to PZ struct\r
- *\r
- * Output:\r
- * none\r
- *\r
- * Returns:\r
- * DAT_SUCCESS\r
- * DAT_INSUFFICIENT_RESOURCES\r
- *\r
- */\r
-DAT_RETURN dapls_ib_pd_alloc(IN DAPL_IA * ia_ptr, IN DAPL_PZ * pz)\r
-{\r
- /* get a protection domain */\r
- pz->pd_handle = ibv_alloc_pd(ia_ptr->hca_ptr->ib_hca_handle);\r
- if (!pz->pd_handle)\r
- return (dapl_convert_errno(ENOMEM, "alloc_pd"));\r
-\r
- dapl_dbg_log(DAPL_DBG_TYPE_UTIL,\r
- " pd_alloc: pd_handle=%p\n", pz->pd_handle);\r
-\r
- return DAT_SUCCESS;\r
-}\r
-\r
-/*\r
- * dapl_ib_pd_free\r
- *\r
- * Free a PD\r
- *\r
- * Input:\r
- * ia_handle IA handle\r
- * PZ_ptr pointer to PZ struct\r
- *\r
- * Output:\r
- * none\r
- *\r
- * Returns:\r
- * DAT_SUCCESS\r
- * DAT_INVALID_STATE\r
- *\r
- */\r
-DAT_RETURN dapls_ib_pd_free(IN DAPL_PZ * pz)\r
-{\r
- if (pz->pd_handle != IB_INVALID_HANDLE) {\r
- ibv_dealloc_pd(pz->pd_handle);\r
- pz->pd_handle = IB_INVALID_HANDLE;\r
- }\r
- return DAT_SUCCESS;\r
-}\r
-\r
-/*\r
- * dapl_ib_mr_register\r
- *\r
- * Register a virtual memory region\r
- *\r
- * Input:\r
- * ia_handle IA handle\r
- * lmr pointer to dapl_lmr struct\r
- * virt_addr virtual address of beginning of mem region\r
- * length length of memory region\r
- *\r
- * Output:\r
- * none\r
- *\r
- * Returns:\r
- * DAT_SUCCESS\r
- * DAT_INSUFFICIENT_RESOURCES\r
- *\r
- */\r
-DAT_RETURN\r
-dapls_ib_mr_register(IN DAPL_IA * ia_ptr,\r
- IN DAPL_LMR * lmr,\r
- IN DAT_PVOID virt_addr,\r
- IN DAT_VLEN length,\r
- IN DAT_MEM_PRIV_FLAGS privileges, IN DAT_VA_TYPE va_type)\r
-{\r
- ib_pd_handle_t ib_pd_handle;\r
- struct ibv_device *ibv_dev = ia_ptr->hca_ptr->ib_hca_handle->device;\r
-\r
- ib_pd_handle = ((DAPL_PZ *) lmr->param.pz_handle)->pd_handle;\r
-\r
- dapl_dbg_log(DAPL_DBG_TYPE_UTIL,\r
- " mr_register: ia=%p, lmr=%p va=%p ln=%d pv=0x%x\n",\r
- ia_ptr, lmr, virt_addr, length, privileges);\r
-\r
- /* TODO: shared memory */\r
- if (lmr->param.mem_type == DAT_MEM_TYPE_SHARED_VIRTUAL) {\r
- dapl_dbg_log(DAPL_DBG_TYPE_ERR,\r
- " mr_register_shared: NOT IMPLEMENTED\n");\r
- return DAT_ERROR(DAT_NOT_IMPLEMENTED, DAT_NO_SUBTYPE);\r
- }\r
-\r
- /* iWARP only support */\r
- if ((va_type == DAT_VA_TYPE_ZB) &&\r
- (ibv_dev->transport_type != IBV_TRANSPORT_IWARP)) {\r
- dapl_dbg_log(DAPL_DBG_TYPE_ERR,\r
- " va_type == DAT_VA_TYPE_ZB: NOT SUPPORTED\n");\r
- return DAT_ERROR(DAT_NOT_IMPLEMENTED, DAT_NO_SUBTYPE);\r
- }\r
-\r
- /* local read is default on IB */\r
- lmr->mr_handle =\r
- ibv_reg_mr(((DAPL_PZ *) lmr->param.pz_handle)->pd_handle,\r
- virt_addr, length, dapls_convert_privileges(privileges));\r
-\r
- if (!lmr->mr_handle)\r
- return (dapl_convert_errno(ENOMEM, "reg_mr"));\r
-\r
- lmr->param.lmr_context = lmr->mr_handle->lkey;\r
- lmr->param.rmr_context = lmr->mr_handle->rkey;\r
- lmr->param.registered_size = length;\r
- lmr->param.registered_address = (DAT_VADDR) (uintptr_t) virt_addr;\r
-\r
- dapl_dbg_log(DAPL_DBG_TYPE_UTIL,\r
- " mr_register: mr=%p addr=%p pd %p ctx %p "\r
- "lkey=0x%x rkey=0x%x priv=%x\n",\r
- lmr->mr_handle, lmr->mr_handle->addr,\r
- lmr->mr_handle->pd, lmr->mr_handle->context,\r
- lmr->mr_handle->lkey, lmr->mr_handle->rkey,\r
- length, dapls_convert_privileges(privileges));\r
-\r
- return DAT_SUCCESS;\r
-}\r
-\r
-/*\r
- * dapl_ib_mr_deregister\r
- *\r
- * Free a memory region\r
- *\r
- * Input:\r
- * lmr pointer to dapl_lmr struct\r
- *\r
- * Output:\r
- * none\r
- *\r
- * Returns:\r
- * DAT_SUCCESS\r
- * DAT_INVALID_STATE\r
- *\r
- */\r
-DAT_RETURN dapls_ib_mr_deregister(IN DAPL_LMR * lmr)\r
-{\r
- if (lmr->mr_handle != IB_INVALID_HANDLE) {\r
- if (ibv_dereg_mr(lmr->mr_handle))\r
- return (dapl_convert_errno(errno, "dereg_pd"));\r
- lmr->mr_handle = IB_INVALID_HANDLE;\r
- }\r
- return DAT_SUCCESS;\r
-}\r
-\r
-/*\r
- * dapl_ib_mr_register_shared\r
- *\r
- * Register a virtual memory region\r
- *\r
- * Input:\r
- * ia_ptr IA handle\r
- * lmr pointer to dapl_lmr struct\r
- * privileges \r
- * va_type \r
- *\r
- * Output:\r
- * none\r
- *\r
- * Returns:\r
- * DAT_SUCCESS\r
- * DAT_INSUFFICIENT_RESOURCES\r
- *\r
- */\r
-DAT_RETURN\r
-dapls_ib_mr_register_shared(IN DAPL_IA * ia_ptr,\r
- IN DAPL_LMR * lmr,\r
- IN DAT_MEM_PRIV_FLAGS privileges,\r
- IN DAT_VA_TYPE va_type)\r
-{\r
- dapl_dbg_log(DAPL_DBG_TYPE_ERR,\r
- " mr_register_shared: NOT IMPLEMENTED\n");\r
-\r
- return DAT_ERROR(DAT_NOT_IMPLEMENTED, DAT_NO_SUBTYPE);\r
-}\r
-\r
-/*\r
- * dapls_ib_mw_alloc\r
- *\r
- * Bind a protection domain to a memory window\r
- *\r
- * Input:\r
- * rmr Initialized rmr to hold binding handles\r
- *\r
- * Output:\r
- * none\r
- *\r
- * Returns:\r
- * DAT_SUCCESS\r
- * DAT_INSUFFICIENT_RESOURCES\r
- *\r
- */\r
-DAT_RETURN dapls_ib_mw_alloc(IN DAPL_RMR * rmr)\r
-{\r
-\r
- dapl_dbg_log(DAPL_DBG_TYPE_ERR, " mw_alloc: NOT IMPLEMENTED\n");\r
-\r
- return DAT_ERROR(DAT_NOT_IMPLEMENTED, DAT_NO_SUBTYPE);\r
-}\r
-\r
-/*\r
- * dapls_ib_mw_free\r
- *\r
- * Release bindings of a protection domain to a memory window\r
- *\r
- * Input:\r
- * rmr Initialized rmr to hold binding handles\r
- *\r
- * Output:\r
- * none\r
- *\r
- * Returns:\r
- * DAT_SUCCESS\r
- * DAT_INVALID_STATE\r
- *\r
- */\r
-DAT_RETURN dapls_ib_mw_free(IN DAPL_RMR * rmr)\r
-{\r
- dapl_dbg_log(DAPL_DBG_TYPE_ERR, " mw_free: NOT IMPLEMENTED\n");\r
-\r
- return DAT_ERROR(DAT_NOT_IMPLEMENTED, DAT_NO_SUBTYPE);\r
-}\r
-\r
-/*\r
- * dapls_ib_mw_bind\r
- *\r
- * Bind a protection domain to a memory window\r
- *\r
- * Input:\r
- * rmr Initialized rmr to hold binding handles\r
- *\r
- * Output:\r
- * none\r
- *\r
- * Returns:\r
- * DAT_SUCCESS\r
- * DAT_INVALID_PARAMETER;\r
- * DAT_INSUFFICIENT_RESOURCES\r
- *\r
- */\r
-DAT_RETURN\r
-dapls_ib_mw_bind(IN DAPL_RMR * rmr,\r
- IN DAPL_LMR * lmr,\r
- IN DAPL_EP * ep,\r
- IN DAPL_COOKIE * cookie,\r
- IN DAT_VADDR virtual_address,\r
- IN DAT_VLEN length,\r
- IN DAT_MEM_PRIV_FLAGS mem_priv, IN DAT_BOOLEAN is_signaled)\r
-{\r
- dapl_dbg_log(DAPL_DBG_TYPE_ERR, " mw_bind: NOT IMPLEMENTED\n");\r
-\r
- return DAT_ERROR(DAT_NOT_IMPLEMENTED, DAT_NO_SUBTYPE);\r
-}\r
-\r
-/*\r
- * dapls_ib_mw_unbind\r
- *\r
- * Unbind a protection domain from a memory window\r
- *\r
- * Input:\r
- * rmr Initialized rmr to hold binding handles\r
- *\r
- * Output:\r
- * none\r
- *\r
- * Returns:\r
- * DAT_SUCCESS\r
- * DAT_INVALID_PARAMETER;\r
- * DAT_INVALID_STATE;\r
- * DAT_INSUFFICIENT_RESOURCES\r
- *\r
- */\r
-DAT_RETURN\r
-dapls_ib_mw_unbind(IN DAPL_RMR * rmr,\r
- IN DAPL_EP * ep,\r
- IN DAPL_COOKIE * cookie, IN DAT_BOOLEAN is_signaled)\r
-{\r
- dapl_dbg_log(DAPL_DBG_TYPE_ERR, " mw_unbind: NOT IMPLEMENTED\n");\r
-\r
- return DAT_ERROR(DAT_NOT_IMPLEMENTED, DAT_NO_SUBTYPE);\r
-}\r
-\r
-/*\r
- * Local variables:\r
- * c-indent-level: 4\r
- * c-basic-offset: 4\r
- * tab-width: 8\r
- * End:\r
- */\r
+/*
+ * Copyright (c) 2005-2007 Intel Corporation. All rights reserved.
+ *
+ * This Software is licensed under one of the following licenses:
+ * 1) under the terms of the "Common Public License 1.0" a copy of which is
+ * available from the Open Source Initiative, see
+ * http://www.opensource.org/licenses/cpl.php.
+ *
+ * 2) under the terms of the "The BSD License" a copy of which is
+ * available from the Open Source Initiative, see
+ * http://www.opensource.org/licenses/bsd-license.php.
+ *
+ * 3) under the terms of the "GNU General Public License (GPL) Version 2" a
+ * copy of which is available from the Open Source Initiative, see
+ * http://www.opensource.org/licenses/gpl-license.php.
+ *
+ * Licensee has the right to choose one of the above licenses.
+ *
+ * Redistributions of source code must retain the above copyright
+ * notice and one of the license notices.
+ *
+ * Redistributions in binary form must reproduce both the above copyright
+ * notice, one of the license notices in the documentation
+ * and/or other materials provided with the distribution.
+ */
+#include "dapl.h"
+#include "dapl_adapter_util.h"
+#include "dapl_lmr_util.h"
+
+/*
+ * dapls_convert_privileges
+ *
+ * Convert LMR privileges to provider
+ *
+ * Input:
+ * DAT_MEM_PRIV_FLAGS
+ *
+ * Output:
+ * none
+ *
+ * Returns:
+ * ibv_access_flags
+ *
+ */
+STATIC _INLINE_ int dapls_convert_privileges(IN DAT_MEM_PRIV_FLAGS privileges)
+{
+ int access = 0;
+
+ /*
+ * if (DAT_MEM_PRIV_LOCAL_READ_FLAG & privileges) do nothing
+ */
+ if (DAT_MEM_PRIV_LOCAL_WRITE_FLAG & privileges)
+ access |= IBV_ACCESS_LOCAL_WRITE;
+ if (DAT_MEM_PRIV_REMOTE_WRITE_FLAG & privileges)
+ access |= IBV_ACCESS_REMOTE_WRITE;
+ if (DAT_MEM_PRIV_REMOTE_READ_FLAG & privileges)
+ access |= IBV_ACCESS_REMOTE_READ;
+ if (DAT_MEM_PRIV_REMOTE_READ_FLAG & privileges)
+ access |= IBV_ACCESS_REMOTE_READ;
+ if (DAT_MEM_PRIV_REMOTE_READ_FLAG & privileges)
+ access |= IBV_ACCESS_REMOTE_READ;
+#ifdef DAT_EXTENSIONS
+ if (DAT_IB_MEM_PRIV_REMOTE_ATOMIC & privileges)
+ access |= IBV_ACCESS_REMOTE_ATOMIC;
+#endif
+
+ return access;
+}
+
+/*
+ * dapl_ib_pd_alloc
+ *
+ * Alloc a PD
+ *
+ * Input:
+ * ia_handle IA handle
+ * pz pointer to PZ struct
+ *
+ * Output:
+ * none
+ *
+ * Returns:
+ * DAT_SUCCESS
+ * DAT_INSUFFICIENT_RESOURCES
+ *
+ */
+DAT_RETURN dapls_ib_pd_alloc(IN DAPL_IA * ia_ptr, IN DAPL_PZ * pz)
+{
+ /* get a protection domain */
+ pz->pd_handle = ibv_alloc_pd(ia_ptr->hca_ptr->ib_hca_handle);
+ if (!pz->pd_handle)
+ return (dapl_convert_errno(ENOMEM, "alloc_pd"));
+
+ dapl_dbg_log(DAPL_DBG_TYPE_UTIL,
+ " pd_alloc: pd_handle=%p\n", pz->pd_handle);
+
+ return DAT_SUCCESS;
+}
+
+/*
+ * dapl_ib_pd_free
+ *
+ * Free a PD
+ *
+ * Input:
+ * ia_handle IA handle
+ * PZ_ptr pointer to PZ struct
+ *
+ * Output:
+ * none
+ *
+ * Returns:
+ * DAT_SUCCESS
+ * DAT_INVALID_STATE
+ *
+ */
+DAT_RETURN dapls_ib_pd_free(IN DAPL_PZ * pz)
+{
+ if (pz->pd_handle != IB_INVALID_HANDLE) {
+ ibv_dealloc_pd(pz->pd_handle);
+ pz->pd_handle = IB_INVALID_HANDLE;
+ }
+ return DAT_SUCCESS;
+}
+
+/*
+ * dapl_ib_mr_register
+ *
+ * Register a virtual memory region
+ *
+ * Input:
+ * ia_handle IA handle
+ * lmr pointer to dapl_lmr struct
+ * virt_addr virtual address of beginning of mem region
+ * length length of memory region
+ *
+ * Output:
+ * none
+ *
+ * Returns:
+ * DAT_SUCCESS
+ * DAT_INSUFFICIENT_RESOURCES
+ *
+ */
+DAT_RETURN
+dapls_ib_mr_register(IN DAPL_IA * ia_ptr,
+ IN DAPL_LMR * lmr,
+ IN DAT_PVOID virt_addr,
+ IN DAT_VLEN length,
+ IN DAT_MEM_PRIV_FLAGS privileges, IN DAT_VA_TYPE va_type)
+{
+ ib_pd_handle_t ib_pd_handle;
+ struct ibv_device *ibv_dev = ia_ptr->hca_ptr->ib_hca_handle->device;
+
+ ib_pd_handle = ((DAPL_PZ *) lmr->param.pz_handle)->pd_handle;
+
+ dapl_dbg_log(DAPL_DBG_TYPE_UTIL,
+ " mr_register: ia=%p, lmr=%p va=%p ln=%d pv=0x%x\n",
+ ia_ptr, lmr, virt_addr, length, privileges);
+
+ /* TODO: shared memory */
+ if (lmr->param.mem_type == DAT_MEM_TYPE_SHARED_VIRTUAL) {
+ dapl_dbg_log(DAPL_DBG_TYPE_ERR,
+ " mr_register_shared: NOT IMPLEMENTED\n");
+ return DAT_ERROR(DAT_NOT_IMPLEMENTED, DAT_NO_SUBTYPE);
+ }
+
+ /* iWARP only support */
+ if ((va_type == DAT_VA_TYPE_ZB) &&
+ (ibv_dev->transport_type != IBV_TRANSPORT_IWARP)) {
+ dapl_dbg_log(DAPL_DBG_TYPE_ERR,
+ " va_type == DAT_VA_TYPE_ZB: NOT SUPPORTED\n");
+ return DAT_ERROR(DAT_NOT_IMPLEMENTED, DAT_NO_SUBTYPE);
+ }
+
+ /* local read is default on IB */
+ lmr->mr_handle =
+ ibv_reg_mr(((DAPL_PZ *) lmr->param.pz_handle)->pd_handle,
+ virt_addr, length, dapls_convert_privileges(privileges));
+
+ if (!lmr->mr_handle)
+ return (dapl_convert_errno(ENOMEM, "reg_mr"));
+
+ lmr->param.lmr_context = lmr->mr_handle->lkey;
+ lmr->param.rmr_context = lmr->mr_handle->rkey;
+ lmr->param.registered_size = length;
+ lmr->param.registered_address = (DAT_VADDR) (uintptr_t) virt_addr;
+
+ dapl_dbg_log(DAPL_DBG_TYPE_UTIL,
+ " mr_register: mr=%p addr=%p pd %p ctx %p "
+ "lkey=0x%x rkey=0x%x priv=%x\n",
+ lmr->mr_handle, lmr->mr_handle->addr,
+ lmr->mr_handle->pd, lmr->mr_handle->context,
+ lmr->mr_handle->lkey, lmr->mr_handle->rkey,
+ length, dapls_convert_privileges(privileges));
+
+ return DAT_SUCCESS;
+}
+
+/*
+ * dapl_ib_mr_deregister
+ *
+ * Free a memory region
+ *
+ * Input:
+ * lmr pointer to dapl_lmr struct
+ *
+ * Output:
+ * none
+ *
+ * Returns:
+ * DAT_SUCCESS
+ * DAT_INVALID_STATE
+ *
+ */
+DAT_RETURN dapls_ib_mr_deregister(IN DAPL_LMR * lmr)
+{
+ if (lmr->mr_handle != IB_INVALID_HANDLE) {
+ if (ibv_dereg_mr(lmr->mr_handle))
+ return (dapl_convert_errno(errno, "dereg_pd"));
+ lmr->mr_handle = IB_INVALID_HANDLE;
+ }
+ return DAT_SUCCESS;
+}
+
+/*
+ * dapl_ib_mr_register_shared
+ *
+ * Register a virtual memory region
+ *
+ * Input:
+ * ia_ptr IA handle
+ * lmr pointer to dapl_lmr struct
+ * privileges
+ * va_type
+ *
+ * Output:
+ * none
+ *
+ * Returns:
+ * DAT_SUCCESS
+ * DAT_INSUFFICIENT_RESOURCES
+ *
+ */
+DAT_RETURN
+dapls_ib_mr_register_shared(IN DAPL_IA * ia_ptr,
+ IN DAPL_LMR * lmr,
+ IN DAT_MEM_PRIV_FLAGS privileges,
+ IN DAT_VA_TYPE va_type)
+{
+ dapl_dbg_log(DAPL_DBG_TYPE_ERR,
+ " mr_register_shared: NOT IMPLEMENTED\n");
+
+ return DAT_ERROR(DAT_NOT_IMPLEMENTED, DAT_NO_SUBTYPE);
+}
+
+/*
+ * dapls_ib_mw_alloc
+ *
+ * Bind a protection domain to a memory window
+ *
+ * Input:
+ * rmr Initialized rmr to hold binding handles
+ *
+ * Output:
+ * none
+ *
+ * Returns:
+ * DAT_SUCCESS
+ * DAT_INSUFFICIENT_RESOURCES
+ *
+ */
+DAT_RETURN dapls_ib_mw_alloc(IN DAPL_RMR * rmr)
+{
+
+ dapl_dbg_log(DAPL_DBG_TYPE_ERR, " mw_alloc: NOT IMPLEMENTED\n");
+
+ return DAT_ERROR(DAT_NOT_IMPLEMENTED, DAT_NO_SUBTYPE);
+}
+
+/*
+ * dapls_ib_mw_free
+ *
+ * Release bindings of a protection domain to a memory window
+ *
+ * Input:
+ * rmr Initialized rmr to hold binding handles
+ *
+ * Output:
+ * none
+ *
+ * Returns:
+ * DAT_SUCCESS
+ * DAT_INVALID_STATE
+ *
+ */
+DAT_RETURN dapls_ib_mw_free(IN DAPL_RMR * rmr)
+{
+ dapl_dbg_log(DAPL_DBG_TYPE_ERR, " mw_free: NOT IMPLEMENTED\n");
+
+ return DAT_ERROR(DAT_NOT_IMPLEMENTED, DAT_NO_SUBTYPE);
+}
+
+/*
+ * dapls_ib_mw_bind
+ *
+ * Bind a protection domain to a memory window
+ *
+ * Input:
+ * rmr Initialized rmr to hold binding handles
+ *
+ * Output:
+ * none
+ *
+ * Returns:
+ * DAT_SUCCESS
+ * DAT_INVALID_PARAMETER;
+ * DAT_INSUFFICIENT_RESOURCES
+ *
+ */
+DAT_RETURN
+dapls_ib_mw_bind(IN DAPL_RMR * rmr,
+ IN DAPL_LMR * lmr,
+ IN DAPL_EP * ep,
+ IN DAPL_COOKIE * cookie,
+ IN DAT_VADDR virtual_address,
+ IN DAT_VLEN length,
+ IN DAT_MEM_PRIV_FLAGS mem_priv, IN DAT_BOOLEAN is_signaled)
+{
+ dapl_dbg_log(DAPL_DBG_TYPE_ERR, " mw_bind: NOT IMPLEMENTED\n");
+
+ return DAT_ERROR(DAT_NOT_IMPLEMENTED, DAT_NO_SUBTYPE);
+}
+
+/*
+ * dapls_ib_mw_unbind
+ *
+ * Unbind a protection domain from a memory window
+ *
+ * Input:
+ * rmr Initialized rmr to hold binding handles
+ *
+ * Output:
+ * none
+ *
+ * Returns:
+ * DAT_SUCCESS
+ * DAT_INVALID_PARAMETER;
+ * DAT_INVALID_STATE;
+ * DAT_INSUFFICIENT_RESOURCES
+ *
+ */
+DAT_RETURN
+dapls_ib_mw_unbind(IN DAPL_RMR * rmr,
+ IN DAPL_EP * ep,
+ IN DAPL_COOKIE * cookie, IN DAT_BOOLEAN is_signaled)
+{
+ dapl_dbg_log(DAPL_DBG_TYPE_ERR, " mw_unbind: NOT IMPLEMENTED\n");
+
+ return DAT_ERROR(DAT_NOT_IMPLEMENTED, DAT_NO_SUBTYPE);
+}
+
+/*
+ * Local variables:
+ * c-indent-level: 4
+ * c-basic-offset: 4
+ * tab-width: 8
+ * End:
+ */
-/*\r
- * This Software is licensed under one of the following licenses:\r
- *\r
- * 1) under the terms of the "Common Public License 1.0" a copy of which is\r
- * available from the Open Source Initiative, see\r
- * http://www.opensource.org/licenses/cpl.php.\r
- *\r
- * 2) under the terms of the "The BSD License" a copy of which is\r
- * available from the Open Source Initiative, see\r
- * http://www.opensource.org/licenses/bsd-license.php.\r
- *\r
- * 3) under the terms of the "GNU General Public License (GPL) Version 2" a\r
- * copy of which is available from the Open Source Initiative, see\r
- * http://www.opensource.org/licenses/gpl-license.php.\r
- *\r
- * Licensee has the right to choose one of the above licenses.\r
- *\r
- * Redistributions of source code must retain the above copyright\r
- * notice and one of the license notices.\r
- *\r
- * Redistributions in binary form must reproduce both the above copyright\r
- * notice, one of the license notices in the documentation\r
- * and/or other materials provided with the distribution.\r
- */\r
-#include "dapl.h"\r
-#include "dapl_adapter_util.h"\r
-\r
-/*\r
- * dapl_ib_qp_alloc\r
- *\r
- * Alloc a QP\r
- *\r
- * Input:\r
- * *ep_ptr pointer to EP INFO\r
- * ib_hca_handle provider HCA handle\r
- * ib_pd_handle provider protection domain handle\r
- * cq_recv provider recv CQ handle\r
- * cq_send provider send CQ handle\r
- *\r
- * Output:\r
- * none\r
- *\r
- * Returns:\r
- * DAT_SUCCESS\r
- * DAT_INSUFFICIENT_RESOURCES\r
- * DAT_INTERNAL_ERROR\r
- *\r
- */\r
-DAT_RETURN\r
-dapls_ib_qp_alloc(IN DAPL_IA * ia_ptr,\r
- IN DAPL_EP * ep_ptr, IN DAPL_EP * ep_ctx_ptr)\r
-{\r
- DAT_EP_ATTR *attr;\r
- DAPL_EVD *rcv_evd, *req_evd;\r
- ib_cq_handle_t rcv_cq, req_cq;\r
- ib_pd_handle_t ib_pd_handle;\r
- struct ibv_qp_init_attr qp_create;\r
-#ifdef _OPENIB_CMA_\r
- dp_ib_cm_handle_t conn;\r
-#endif\r
- dapl_dbg_log(DAPL_DBG_TYPE_EP,\r
- " qp_alloc: ia_ptr %p ep_ptr %p ep_ctx_ptr %p\n",\r
- ia_ptr, ep_ptr, ep_ctx_ptr);\r
-\r
- attr = &ep_ptr->param.ep_attr;\r
- ib_pd_handle = ((DAPL_PZ *) ep_ptr->param.pz_handle)->pd_handle;\r
- rcv_evd = (DAPL_EVD *) ep_ptr->param.recv_evd_handle;\r
- req_evd = (DAPL_EVD *) ep_ptr->param.request_evd_handle;\r
-\r
- /* \r
- * DAT allows usage model of EP's with no EVD's but IB does not. \r
- * Create a CQ with zero entries under the covers to support and \r
- * catch any invalid posting. \r
- */\r
- if (rcv_evd != DAT_HANDLE_NULL)\r
- rcv_cq = rcv_evd->ib_cq_handle;\r
- else if (!ia_ptr->hca_ptr->ib_trans.ib_cq_empty)\r
- rcv_cq = ia_ptr->hca_ptr->ib_trans.ib_cq_empty;\r
- else {\r
- struct ibv_comp_channel *channel;\r
-\r
- channel = ibv_create_comp_channel(ia_ptr->hca_ptr->ib_hca_handle);\r
- if (!channel)\r
- return (dapl_convert_errno(ENOMEM, "create_cq"));\r
- \r
- /* Call IB verbs to create CQ */\r
- rcv_cq = ibv_create_cq(ia_ptr->hca_ptr->ib_hca_handle,\r
- 0, NULL, channel, 0);\r
-\r
- if (rcv_cq == IB_INVALID_HANDLE) {\r
- ibv_destroy_comp_channel(channel);\r
- return (dapl_convert_errno(ENOMEM, "create_cq"));\r
- }\r
-\r
- ia_ptr->hca_ptr->ib_trans.ib_cq_empty = rcv_cq;\r
- }\r
- if (req_evd != DAT_HANDLE_NULL)\r
- req_cq = req_evd->ib_cq_handle;\r
- else\r
- req_cq = ia_ptr->hca_ptr->ib_trans.ib_cq_empty;\r
-\r
- /* \r
- * IMPLEMENTATION NOTE:\r
- * uDAPL allows consumers to post buffers on the EP after creation\r
- * and before a connect request (outbound and inbound). This forces\r
- * a binding to a device during the hca_open call and requires the\r
- * consumer to predetermine which device to listen on or connect from.\r
- * This restriction eliminates any option of listening or connecting \r
- * over multiple devices. uDAPL should add API's to resolve addresses \r
- * and bind to the device at the approriate time (before connect \r
- * and after CR arrives). Discovery should happen at connection time \r
- * based on addressing and not on static configuration during open.\r
- */\r
-\r
-#ifdef _OPENIB_CMA_\r
- /* Allocate CM and initialize lock */\r
- if ((conn = dapls_ib_cm_create(ep_ptr)) == NULL)\r
- return (dapl_convert_errno(ENOMEM, "create_cq"));\r
-\r
- /* open identifies the local device; per DAT specification */\r
- if (rdma_bind_addr(conn->cm_id,\r
- (struct sockaddr *)&ia_ptr->hca_ptr->hca_address))\r
- return (dapl_convert_errno(EAFNOSUPPORT, "create_cq"));\r
-#endif\r
- /* Setup attributes and create qp */\r
- dapl_os_memzero((void *)&qp_create, sizeof(qp_create));\r
- qp_create.send_cq = req_cq;\r
- qp_create.cap.max_send_wr = attr->max_request_dtos;\r
- qp_create.cap.max_send_sge = attr->max_request_iov;\r
- qp_create.cap.max_inline_data =\r
- ia_ptr->hca_ptr->ib_trans.max_inline_send;\r
- qp_create.qp_type = IBV_QPT_RC;\r
- qp_create.qp_context = (void *)ep_ptr;\r
-\r
-#ifdef DAT_EXTENSIONS \r
- if (attr->service_type == DAT_IB_SERVICE_TYPE_UD) {\r
-#ifdef _OPENIB_CMA_\r
- return (DAT_NOT_IMPLEMENTED);\r
-#endif\r
- qp_create.qp_type = IBV_QPT_UD;\r
- if (attr->max_message_size >\r
- (128 << ia_ptr->hca_ptr->ib_trans.mtu)) {\r
- return (DAT_INVALID_PARAMETER | DAT_INVALID_ARG6);\r
- }\r
- }\r
-#endif\r
- \r
- /* ibv assumes rcv_cq is never NULL, set to req_cq */\r
- if (rcv_cq == NULL) {\r
- qp_create.recv_cq = req_cq;\r
- qp_create.cap.max_recv_wr = 0;\r
- qp_create.cap.max_recv_sge = 0;\r
- } else {\r
- qp_create.recv_cq = rcv_cq;\r
- qp_create.cap.max_recv_wr = attr->max_recv_dtos;\r
- qp_create.cap.max_recv_sge = attr->max_recv_iov;\r
- }\r
-\r
-#ifdef _OPENIB_CMA_\r
- if (rdma_create_qp(conn->cm_id, ib_pd_handle, &qp_create)) {\r
- dapls_ib_cm_free(conn, ep_ptr);\r
- return (dapl_convert_errno(errno, "create_qp"));\r
- }\r
- ep_ptr->qp_handle = conn->cm_id->qp;\r
- ep_ptr->cm_handle = conn;\r
- ep_ptr->qp_state = IBV_QPS_INIT;\r
- \r
- /* setup up ep->param to reference the bound local address and port */\r
- ep_ptr->param.local_ia_address_ptr = \r
- &conn->cm_id->route.addr.src_addr;\r
- ep_ptr->param.local_port_qual = rdma_get_src_port(conn->cm_id);\r
-#else\r
- ep_ptr->qp_handle = ibv_create_qp(ib_pd_handle, &qp_create);\r
- if (!ep_ptr->qp_handle)\r
- return (dapl_convert_errno(ENOMEM, "create_qp"));\r
- \r
- /* Setup QP attributes for INIT state on the way out */\r
- if (dapls_modify_qp_state(ep_ptr->qp_handle,\r
- IBV_QPS_INIT, 0, 0, 0) != DAT_SUCCESS) {\r
- ibv_destroy_qp(ep_ptr->qp_handle);\r
- ep_ptr->qp_handle = IB_INVALID_HANDLE;\r
- return DAT_INTERNAL_ERROR;\r
- }\r
-#endif\r
- dapl_dbg_log(DAPL_DBG_TYPE_EP,\r
- " qp_alloc: qpn %p sq %d,%d rq %d,%d\n",\r
- ep_ptr->qp_handle->qp_num,\r
- qp_create.cap.max_send_wr, qp_create.cap.max_send_sge,\r
- qp_create.cap.max_recv_wr, qp_create.cap.max_recv_sge);\r
-\r
- return DAT_SUCCESS;\r
-}\r
-\r
-/*\r
- * dapl_ib_qp_free\r
- *\r
- * Free a QP\r
- *\r
- * Input:\r
- * ia_handle IA handle\r
- * *ep_ptr pointer to EP INFO\r
- *\r
- * Output:\r
- * none\r
- *\r
- * Returns:\r
- * DAT_SUCCESS\r
- * dapl_convert_errno\r
- *\r
- */\r
-DAT_RETURN dapls_ib_qp_free(IN DAPL_IA * ia_ptr, IN DAPL_EP * ep_ptr)\r
-{\r
- dapl_dbg_log(DAPL_DBG_TYPE_EP, " qp_free: ep_ptr %p qp %p\n",\r
- ep_ptr, ep_ptr->qp_handle);\r
-\r
- if (ep_ptr->cm_handle != NULL) {\r
- dapls_ib_cm_free(ep_ptr->cm_handle, ep_ptr);\r
- }\r
- \r
- if (ep_ptr->qp_handle != NULL) {\r
- /* force error state to flush queue, then destroy */\r
- dapls_modify_qp_state(ep_ptr->qp_handle, IBV_QPS_ERR, 0,0,0);\r
-\r
- if (ibv_destroy_qp(ep_ptr->qp_handle))\r
- return (dapl_convert_errno(errno, "destroy_qp"));\r
-\r
- ep_ptr->qp_handle = NULL;\r
- }\r
-\r
-#ifdef DAT_EXTENSIONS\r
- /* UD endpoints can have many CR associations and will not\r
- * set ep->cm_handle. Call provider with cm_ptr null to incidate\r
- * UD type multi CR's for this EP. It will parse internal list\r
- * and cleanup all associations.\r
- */\r
- if (ep_ptr->param.ep_attr.service_type == DAT_IB_SERVICE_TYPE_UD) \r
- dapls_ib_cm_free(NULL, ep_ptr);\r
-#endif\r
-\r
- return DAT_SUCCESS;\r
-}\r
-\r
-/*\r
- * dapl_ib_qp_modify\r
- *\r
- * Set the QP to the parameters specified in an EP_PARAM\r
- *\r
- * The EP_PARAM structure that is provided has been\r
- * sanitized such that only non-zero values are valid.\r
- *\r
- * Input:\r
- * ib_hca_handle HCA handle\r
- * qp_handle QP handle\r
- * ep_attr Sanitized EP Params\r
- *\r
- * Output:\r
- * none\r
- *\r
- * Returns:\r
- * DAT_SUCCESS\r
- * DAT_INSUFFICIENT_RESOURCES\r
- * DAT_INVALID_PARAMETER\r
- *\r
- */\r
-DAT_RETURN\r
-dapls_ib_qp_modify(IN DAPL_IA * ia_ptr,\r
- IN DAPL_EP * ep_ptr, IN DAT_EP_ATTR * attr)\r
-{\r
- struct ibv_qp_attr qp_attr;\r
-\r
- if (ep_ptr->qp_handle == IB_INVALID_HANDLE)\r
- return DAT_INVALID_PARAMETER;\r
-\r
- /* \r
- * EP state, qp_handle state should be an indication\r
- * of current state but the only way to be sure is with\r
- * a user mode ibv_query_qp call which is NOT available \r
- */\r
-\r
- /* move to error state if necessary */\r
- if ((ep_ptr->qp_state == IB_QP_STATE_ERROR) &&\r
- (ep_ptr->qp_handle->state != IBV_QPS_ERR)) {\r
- return (dapls_modify_qp_state(ep_ptr->qp_handle, \r
- IBV_QPS_ERR, 0, 0, 0));\r
- }\r
-\r
- /*\r
- * Check if we have the right qp_state to modify attributes\r
- */\r
- if ((ep_ptr->qp_handle->state != IBV_QPS_RTR) &&\r
- (ep_ptr->qp_handle->state != IBV_QPS_RTS))\r
- return DAT_INVALID_STATE;\r
-\r
- /* Adjust to current EP attributes */\r
- dapl_os_memzero((void *)&qp_attr, sizeof(qp_attr));\r
- qp_attr.cap.max_send_wr = attr->max_request_dtos;\r
- qp_attr.cap.max_recv_wr = attr->max_recv_dtos;\r
- qp_attr.cap.max_send_sge = attr->max_request_iov;\r
- qp_attr.cap.max_recv_sge = attr->max_recv_iov;\r
-\r
- dapl_dbg_log(DAPL_DBG_TYPE_EP,\r
- "modify_qp: qp %p sq %d,%d, rq %d,%d\n",\r
- ep_ptr->qp_handle,\r
- qp_attr.cap.max_send_wr, qp_attr.cap.max_send_sge,\r
- qp_attr.cap.max_recv_wr, qp_attr.cap.max_recv_sge);\r
-\r
- if (ibv_modify_qp(ep_ptr->qp_handle, &qp_attr, IBV_QP_CAP)) {\r
- dapl_dbg_log(DAPL_DBG_TYPE_ERR,\r
- "modify_qp: modify ep %p qp %p failed\n",\r
- ep_ptr, ep_ptr->qp_handle);\r
- return (dapl_convert_errno(errno, "modify_qp_state"));\r
- }\r
-\r
- return DAT_SUCCESS;\r
-}\r
-\r
-/*\r
- * dapls_ib_reinit_ep\r
- *\r
- * Move the QP to INIT state again.\r
- *\r
- * Input:\r
- * ep_ptr DAPL_EP\r
- *\r
- * Output:\r
- * none\r
- *\r
- * Returns:\r
- * void\r
- *\r
- */\r
-#if defined(_WIN32) || defined(_WIN64) || defined(_OPENIB_CMA_)\r
-void dapls_ib_reinit_ep(IN DAPL_EP * ep_ptr)\r
-{\r
- /* work around bug in low level driver - 3/24/09 */\r
- /* RTS -> RESET -> INIT -> ERROR QP transition crashes system */\r
- if (ep_ptr->qp_handle != IB_INVALID_HANDLE) {\r
- dapls_ib_qp_free(ep_ptr->header.owner_ia, ep_ptr);\r
- dapls_ib_qp_alloc(ep_ptr->header.owner_ia, ep_ptr, ep_ptr);\r
- }\r
-}\r
-#else // _WIN32 || _WIN64\r
-void dapls_ib_reinit_ep(IN DAPL_EP * ep_ptr)\r
-{\r
- if (ep_ptr->qp_handle != IB_INVALID_HANDLE &&\r
- ep_ptr->qp_handle->qp_type != IBV_QPT_UD) {\r
- /* move to RESET state and then to INIT */\r
- dapls_modify_qp_state(ep_ptr->qp_handle, IBV_QPS_RESET,0,0,0);\r
- dapls_modify_qp_state(ep_ptr->qp_handle, IBV_QPS_INIT,0,0,0);\r
- }\r
-}\r
-#endif // _WIN32 || _WIN64\r
-\r
-/* \r
- * Generic QP modify for init, reset, error, RTS, RTR\r
- * For UD, create_ah on RTR, qkey on INIT\r
- * CM msg provides QP attributes, info in network order\r
- */\r
-DAT_RETURN\r
-dapls_modify_qp_state(IN ib_qp_handle_t qp_handle,\r
- IN ib_qp_state_t qp_state, \r
- IN uint32_t qpn,\r
- IN uint16_t lid,\r
- IN ib_gid_handle_t gid)\r
-{\r
- struct ibv_qp_attr qp_attr;\r
- enum ibv_qp_attr_mask mask = IBV_QP_STATE;\r
- DAPL_EP *ep_ptr = (DAPL_EP *) qp_handle->qp_context;\r
- DAPL_IA *ia_ptr = ep_ptr->header.owner_ia;\r
- int ret;\r
-\r
- dapl_os_memzero((void *)&qp_attr, sizeof(qp_attr));\r
- qp_attr.qp_state = qp_state;\r
- \r
- switch (qp_state) {\r
- case IBV_QPS_RTR:\r
- dapl_dbg_log(DAPL_DBG_TYPE_EP,\r
- " QPS_RTR: type %d qpn 0x%x lid 0x%x"\r
- " port %d ep %p qp_state %d \n",\r
- qp_handle->qp_type, \r
- ntohl(qpn), ntohs(lid), \r
- ia_ptr->hca_ptr->port_num,\r
- ep_ptr, ep_ptr->qp_state);\r
-\r
- mask |= IBV_QP_AV |\r
- IBV_QP_PATH_MTU |\r
- IBV_QP_DEST_QPN |\r
- IBV_QP_RQ_PSN |\r
- IBV_QP_MAX_DEST_RD_ATOMIC | IBV_QP_MIN_RNR_TIMER;\r
-\r
- qp_attr.dest_qp_num = ntohl(qpn);\r
- qp_attr.rq_psn = 1;\r
- qp_attr.path_mtu = ia_ptr->hca_ptr->ib_trans.mtu;\r
- qp_attr.max_dest_rd_atomic =\r
- ep_ptr->param.ep_attr.max_rdma_read_out;\r
- qp_attr.min_rnr_timer =\r
- ia_ptr->hca_ptr->ib_trans.rnr_timer;\r
-\r
- /* address handle. RC and UD */\r
- qp_attr.ah_attr.dlid = ntohs(lid);\r
- if (ia_ptr->hca_ptr->ib_trans.global) {\r
- qp_attr.ah_attr.is_global = 1;\r
- qp_attr.ah_attr.grh.dgid.global.subnet_prefix = \r
- ntohll(gid->global.subnet_prefix);\r
- qp_attr.ah_attr.grh.dgid.global.interface_id = \r
- ntohll(gid->global.interface_id);\r
- qp_attr.ah_attr.grh.hop_limit =\r
- ia_ptr->hca_ptr->ib_trans.hop_limit;\r
- qp_attr.ah_attr.grh.traffic_class =\r
- ia_ptr->hca_ptr->ib_trans.tclass;\r
- }\r
- qp_attr.ah_attr.sl = 0;\r
- qp_attr.ah_attr.src_path_bits = 0;\r
- qp_attr.ah_attr.port_num = ia_ptr->hca_ptr->port_num;\r
-\r
- /* UD: already in RTR, RTS state */\r
- if (qp_handle->qp_type == IBV_QPT_UD) {\r
- mask = IBV_QP_STATE;\r
- if (ep_ptr->qp_state == IBV_QPS_RTR ||\r
- ep_ptr->qp_state == IBV_QPS_RTS)\r
- return DAT_SUCCESS;\r
- }\r
- break;\r
- case IBV_QPS_RTS:\r
- if (qp_handle->qp_type == IBV_QPT_RC) {\r
- mask |= IBV_QP_SQ_PSN |\r
- IBV_QP_TIMEOUT |\r
- IBV_QP_RETRY_CNT |\r
- IBV_QP_RNR_RETRY | IBV_QP_MAX_QP_RD_ATOMIC;\r
- qp_attr.timeout =\r
- ia_ptr->hca_ptr->ib_trans.ack_timer;\r
- qp_attr.retry_cnt =\r
- ia_ptr->hca_ptr->ib_trans.ack_retry;\r
- qp_attr.rnr_retry =\r
- ia_ptr->hca_ptr->ib_trans.rnr_retry;\r
- qp_attr.max_rd_atomic =\r
- ep_ptr->param.ep_attr.max_rdma_read_out;\r
- }\r
- /* RC and UD */\r
- qp_attr.qp_state = IBV_QPS_RTS;\r
- qp_attr.sq_psn = 1;\r
-\r
- dapl_dbg_log(DAPL_DBG_TYPE_EP,\r
- " QPS_RTS: psn %x rd_atomic %d ack %d "\r
- " retry %d rnr_retry %d ep %p qp_state %d\n",\r
- qp_attr.sq_psn, qp_attr.max_rd_atomic,\r
- qp_attr.timeout, qp_attr.retry_cnt,\r
- qp_attr.rnr_retry, ep_ptr,\r
- ep_ptr->qp_state);\r
-\r
- if (qp_handle->qp_type == IBV_QPT_UD) {\r
- /* already RTS, multi remote AH's on QP */\r
- if (ep_ptr->qp_state == IBV_QPS_RTS)\r
- return DAT_SUCCESS;\r
- else\r
- mask = IBV_QP_STATE | IBV_QP_SQ_PSN;\r
- }\r
- break;\r
- case IBV_QPS_INIT:\r
- mask |= IBV_QP_PKEY_INDEX | IBV_QP_PORT;\r
- if (qp_handle->qp_type == IBV_QPT_RC) {\r
- mask |= IBV_QP_ACCESS_FLAGS;\r
- qp_attr.qp_access_flags =\r
- IBV_ACCESS_LOCAL_WRITE |\r
- IBV_ACCESS_REMOTE_WRITE |\r
- IBV_ACCESS_REMOTE_READ |\r
- IBV_ACCESS_REMOTE_ATOMIC |\r
- IBV_ACCESS_MW_BIND;\r
- }\r
-\r
- if (qp_handle->qp_type == IBV_QPT_UD) {\r
- /* already INIT, multi remote AH's on QP */\r
- if (ep_ptr->qp_state == IBV_QPS_INIT)\r
- return DAT_SUCCESS;\r
- mask |= IBV_QP_QKEY;\r
- qp_attr.qkey = DAT_UD_QKEY;\r
- }\r
-\r
- qp_attr.pkey_index = 0;\r
- qp_attr.port_num = ia_ptr->hca_ptr->port_num;\r
-\r
- dapl_dbg_log(DAPL_DBG_TYPE_EP,\r
- " QPS_INIT: pi %x port %x acc %x qkey 0x%x\n",\r
- qp_attr.pkey_index, qp_attr.port_num,\r
- qp_attr.qp_access_flags, qp_attr.qkey);\r
- break;\r
- default:\r
- break;\r
- }\r
-\r
- ret = ibv_modify_qp(qp_handle, &qp_attr, mask);\r
- if (ret == 0) {\r
- ep_ptr->qp_state = qp_state;\r
- return DAT_SUCCESS;\r
- } else {\r
- return (dapl_convert_errno(errno, "modify_qp_state"));\r
- }\r
-}\r
-\r
-/* Modify UD type QP from init, rtr, rts, info network order */\r
-DAT_RETURN \r
-dapls_modify_qp_ud(IN DAPL_HCA *hca, IN ib_qp_handle_t qp)\r
-{\r
- struct ibv_qp_attr qp_attr;\r
-\r
- /* modify QP, setup and prepost buffers */\r
- dapl_os_memzero((void *)&qp_attr, sizeof(qp_attr));\r
- qp_attr.qp_state = IBV_QPS_INIT;\r
- qp_attr.pkey_index = 0;\r
- qp_attr.port_num = hca->port_num;\r
- qp_attr.qkey = DAT_UD_QKEY;\r
- if (ibv_modify_qp(qp, &qp_attr, \r
- IBV_QP_STATE |\r
- IBV_QP_PKEY_INDEX |\r
- IBV_QP_PORT |\r
- IBV_QP_QKEY)) {\r
- dapl_log(DAPL_DBG_TYPE_ERR,\r
- " modify_ud_qp INIT: ERR %s\n", strerror(errno));\r
- return (dapl_convert_errno(errno, "modify_qp"));\r
- }\r
- dapl_os_memzero((void *)&qp_attr, sizeof(qp_attr));\r
- qp_attr.qp_state = IBV_QPS_RTR;\r
- if (ibv_modify_qp(qp, &qp_attr,IBV_QP_STATE)) {\r
- dapl_log(DAPL_DBG_TYPE_ERR, \r
- " modify_ud_qp RTR: ERR %s\n", strerror(errno));\r
- return (dapl_convert_errno(errno, "modify_qp"));\r
- }\r
- dapl_os_memzero((void *)&qp_attr, sizeof(qp_attr));\r
- qp_attr.qp_state = IBV_QPS_RTS;\r
- qp_attr.sq_psn = 1;\r
- if (ibv_modify_qp(qp, &qp_attr, \r
- IBV_QP_STATE | IBV_QP_SQ_PSN)) {\r
- dapl_log(DAPL_DBG_TYPE_ERR,\r
- " modify_ud_qp RTS: ERR %s\n", strerror(errno));\r
- return (dapl_convert_errno(errno, "modify_qp"));\r
- }\r
- return DAT_SUCCESS;\r
-}\r
-\r
-/* Create address handle for remote QP, info in network order */\r
-ib_ah_handle_t \r
-dapls_create_ah(IN DAPL_HCA *hca,\r
- IN ib_pd_handle_t pd,\r
- IN ib_qp_handle_t qp,\r
- IN uint16_t lid,\r
- IN ib_gid_handle_t gid)\r
-{\r
- struct ibv_qp_attr qp_attr;\r
- ib_ah_handle_t ah;\r
-\r
- if (qp->qp_type != IBV_QPT_UD)\r
- return NULL;\r
-\r
- dapl_os_memzero((void *)&qp_attr, sizeof(qp_attr));\r
- qp_attr.qp_state = IBV_QP_STATE;\r
-\r
- /* address handle. RC and UD */\r
- qp_attr.ah_attr.dlid = ntohs(lid);\r
- if (gid != NULL) {\r
- dapl_log(DAPL_DBG_TYPE_CM, "dapl_create_ah: with GID\n");\r
- qp_attr.ah_attr.is_global = 1;\r
- qp_attr.ah_attr.grh.dgid.global.subnet_prefix = \r
- ntohll(gid->global.subnet_prefix);\r
- qp_attr.ah_attr.grh.dgid.global.interface_id = \r
- ntohll(gid->global.interface_id);\r
- qp_attr.ah_attr.grh.hop_limit = hca->ib_trans.hop_limit;\r
- qp_attr.ah_attr.grh.traffic_class = hca->ib_trans.tclass;\r
- }\r
- qp_attr.ah_attr.sl = 0;\r
- qp_attr.ah_attr.src_path_bits = 0;\r
- qp_attr.ah_attr.port_num = hca->port_num;\r
-\r
- dapl_log(DAPL_DBG_TYPE_CM, \r
- " dapls_create_ah: port %x lid %x pd %p ctx %p handle 0x%x\n", \r
- hca->port_num,qp_attr.ah_attr.dlid, pd, pd->context, pd->handle);\r
-\r
- /* UD: create AH for remote side */\r
- ah = ibv_create_ah(pd, &qp_attr.ah_attr);\r
- if (!ah) {\r
- dapl_log(DAPL_DBG_TYPE_ERR,\r
- " create_ah: ERR %s\n", strerror(errno));\r
- return NULL;\r
- }\r
-\r
- dapl_log(DAPL_DBG_TYPE_CM, \r
- " dapls_create_ah: AH %p for lid %x\n", \r
- ah, qp_attr.ah_attr.dlid);\r
-\r
- return ah;\r
-}\r
-\r
-/*\r
- * Local variables:\r
- * c-indent-level: 4\r
- * c-basic-offset: 4\r
- * tab-width: 8\r
- * End:\r
- */\r
+/*
+ * This Software is licensed under one of the following licenses:
+ *
+ * 1) under the terms of the "Common Public License 1.0" a copy of which is
+ * available from the Open Source Initiative, see
+ * http://www.opensource.org/licenses/cpl.php.
+ *
+ * 2) under the terms of the "The BSD License" a copy of which is
+ * available from the Open Source Initiative, see
+ * http://www.opensource.org/licenses/bsd-license.php.
+ *
+ * 3) under the terms of the "GNU General Public License (GPL) Version 2" a
+ * copy of which is available from the Open Source Initiative, see
+ * http://www.opensource.org/licenses/gpl-license.php.
+ *
+ * Licensee has the right to choose one of the above licenses.
+ *
+ * Redistributions of source code must retain the above copyright
+ * notice and one of the license notices.
+ *
+ * Redistributions in binary form must reproduce both the above copyright
+ * notice, one of the license notices in the documentation
+ * and/or other materials provided with the distribution.
+ */
+#include "dapl.h"
+#include "dapl_adapter_util.h"
+
+/*
+ * dapl_ib_qp_alloc
+ *
+ * Alloc a QP
+ *
+ * Input:
+ * *ep_ptr pointer to EP INFO
+ * ib_hca_handle provider HCA handle
+ * ib_pd_handle provider protection domain handle
+ * cq_recv provider recv CQ handle
+ * cq_send provider send CQ handle
+ *
+ * Output:
+ * none
+ *
+ * Returns:
+ * DAT_SUCCESS
+ * DAT_INSUFFICIENT_RESOURCES
+ * DAT_INTERNAL_ERROR
+ *
+ */
+DAT_RETURN
+dapls_ib_qp_alloc(IN DAPL_IA * ia_ptr,
+ IN DAPL_EP * ep_ptr, IN DAPL_EP * ep_ctx_ptr)
+{
+ DAT_EP_ATTR *attr;
+ DAPL_EVD *rcv_evd, *req_evd;
+ ib_cq_handle_t rcv_cq, req_cq;
+ ib_pd_handle_t ib_pd_handle;
+ struct ibv_qp_init_attr qp_create;
+#ifdef _OPENIB_CMA_
+ dp_ib_cm_handle_t conn;
+#endif
+ dapl_dbg_log(DAPL_DBG_TYPE_EP,
+ " qp_alloc: ia_ptr %p ep_ptr %p ep_ctx_ptr %p\n",
+ ia_ptr, ep_ptr, ep_ctx_ptr);
+
+ attr = &ep_ptr->param.ep_attr;
+ ib_pd_handle = ((DAPL_PZ *) ep_ptr->param.pz_handle)->pd_handle;
+ rcv_evd = (DAPL_EVD *) ep_ptr->param.recv_evd_handle;
+ req_evd = (DAPL_EVD *) ep_ptr->param.request_evd_handle;
+
+ /*
+ * DAT allows usage model of EP's with no EVD's but IB does not.
+ * Create a CQ with zero entries under the covers to support and
+ * catch any invalid posting.
+ */
+ if (rcv_evd != DAT_HANDLE_NULL)
+ rcv_cq = rcv_evd->ib_cq_handle;
+ else if (!ia_ptr->hca_ptr->ib_trans.ib_cq_empty)
+ rcv_cq = ia_ptr->hca_ptr->ib_trans.ib_cq_empty;
+ else {
+ struct ibv_comp_channel *channel;
+
+ channel = ibv_create_comp_channel(ia_ptr->hca_ptr->ib_hca_handle);
+ if (!channel)
+ return (dapl_convert_errno(ENOMEM, "create_cq"));
+
+ /* Call IB verbs to create CQ */
+ rcv_cq = ibv_create_cq(ia_ptr->hca_ptr->ib_hca_handle,
+ 0, NULL, channel, 0);
+
+ if (rcv_cq == IB_INVALID_HANDLE) {
+ ibv_destroy_comp_channel(channel);
+ return (dapl_convert_errno(ENOMEM, "create_cq"));
+ }
+
+ ia_ptr->hca_ptr->ib_trans.ib_cq_empty = rcv_cq;
+ }
+ if (req_evd != DAT_HANDLE_NULL)
+ req_cq = req_evd->ib_cq_handle;
+ else
+ req_cq = ia_ptr->hca_ptr->ib_trans.ib_cq_empty;
+
+ /*
+ * IMPLEMENTATION NOTE:
+ * uDAPL allows consumers to post buffers on the EP after creation
+ * and before a connect request (outbound and inbound). This forces
+ * a binding to a device during the hca_open call and requires the
+ * consumer to predetermine which device to listen on or connect from.
+ * This restriction eliminates any option of listening or connecting
+ * over multiple devices. uDAPL should add API's to resolve addresses
+ * and bind to the device at the approriate time (before connect
+ * and after CR arrives). Discovery should happen at connection time
+ * based on addressing and not on static configuration during open.
+ */
+
+#ifdef _OPENIB_CMA_
+ /* Allocate CM and initialize lock */
+ if ((conn = dapls_ib_cm_create(ep_ptr)) == NULL)
+ return (dapl_convert_errno(ENOMEM, "create_cq"));
+
+ /* open identifies the local device; per DAT specification */
+ if (rdma_bind_addr(conn->cm_id,
+ (struct sockaddr *)&ia_ptr->hca_ptr->hca_address))
+ return (dapl_convert_errno(EAFNOSUPPORT, "create_cq"));
+#endif
+ /* Setup attributes and create qp */
+ dapl_os_memzero((void *)&qp_create, sizeof(qp_create));
+ qp_create.send_cq = req_cq;
+ qp_create.cap.max_send_wr = attr->max_request_dtos;
+ qp_create.cap.max_send_sge = attr->max_request_iov;
+ qp_create.cap.max_inline_data =
+ ia_ptr->hca_ptr->ib_trans.max_inline_send;
+ qp_create.qp_type = IBV_QPT_RC;
+ qp_create.qp_context = (void *)ep_ptr;
+
+#ifdef DAT_EXTENSIONS
+ if (attr->service_type == DAT_IB_SERVICE_TYPE_UD) {
+#ifdef _OPENIB_CMA_
+ return (DAT_NOT_IMPLEMENTED);
+#endif
+ qp_create.qp_type = IBV_QPT_UD;
+ if (attr->max_message_size >
+ (128 << ia_ptr->hca_ptr->ib_trans.mtu)) {
+ return (DAT_INVALID_PARAMETER | DAT_INVALID_ARG6);
+ }
+ }
+#endif
+
+ /* ibv assumes rcv_cq is never NULL, set to req_cq */
+ if (rcv_cq == NULL) {
+ qp_create.recv_cq = req_cq;
+ qp_create.cap.max_recv_wr = 0;
+ qp_create.cap.max_recv_sge = 0;
+ } else {
+ qp_create.recv_cq = rcv_cq;
+ qp_create.cap.max_recv_wr = attr->max_recv_dtos;
+ qp_create.cap.max_recv_sge = attr->max_recv_iov;
+ }
+
+#ifdef _OPENIB_CMA_
+ if (rdma_create_qp(conn->cm_id, ib_pd_handle, &qp_create)) {
+ dapls_ib_cm_free(conn, ep_ptr);
+ return (dapl_convert_errno(errno, "create_qp"));
+ }
+ ep_ptr->qp_handle = conn->cm_id->qp;
+ ep_ptr->cm_handle = conn;
+ ep_ptr->qp_state = IBV_QPS_INIT;
+
+ /* setup up ep->param to reference the bound local address and port */
+ ep_ptr->param.local_ia_address_ptr =
+ &conn->cm_id->route.addr.src_addr;
+ ep_ptr->param.local_port_qual = rdma_get_src_port(conn->cm_id);
+#else
+ ep_ptr->qp_handle = ibv_create_qp(ib_pd_handle, &qp_create);
+ if (!ep_ptr->qp_handle)
+ return (dapl_convert_errno(ENOMEM, "create_qp"));
+
+ /* Setup QP attributes for INIT state on the way out */
+ if (dapls_modify_qp_state(ep_ptr->qp_handle,
+ IBV_QPS_INIT, 0, 0, 0) != DAT_SUCCESS) {
+ ibv_destroy_qp(ep_ptr->qp_handle);
+ ep_ptr->qp_handle = IB_INVALID_HANDLE;
+ return DAT_INTERNAL_ERROR;
+ }
+#endif
+ dapl_dbg_log(DAPL_DBG_TYPE_EP,
+ " qp_alloc: qpn %p sq %d,%d rq %d,%d\n",
+ ep_ptr->qp_handle->qp_num,
+ qp_create.cap.max_send_wr, qp_create.cap.max_send_sge,
+ qp_create.cap.max_recv_wr, qp_create.cap.max_recv_sge);
+
+ return DAT_SUCCESS;
+}
+
+/*
+ * dapl_ib_qp_free
+ *
+ * Free a QP
+ *
+ * Input:
+ * ia_handle IA handle
+ * *ep_ptr pointer to EP INFO
+ *
+ * Output:
+ * none
+ *
+ * Returns:
+ * DAT_SUCCESS
+ * dapl_convert_errno
+ *
+ */
+DAT_RETURN dapls_ib_qp_free(IN DAPL_IA * ia_ptr, IN DAPL_EP * ep_ptr)
+{
+ dapl_dbg_log(DAPL_DBG_TYPE_EP, " qp_free: ep_ptr %p qp %p\n",
+ ep_ptr, ep_ptr->qp_handle);
+
+ if (ep_ptr->cm_handle != NULL) {
+ dapls_ib_cm_free(ep_ptr->cm_handle, ep_ptr);
+ }
+
+ if (ep_ptr->qp_handle != NULL) {
+ /* force error state to flush queue, then destroy */
+ dapls_modify_qp_state(ep_ptr->qp_handle, IBV_QPS_ERR, 0,0,0);
+
+ if (ibv_destroy_qp(ep_ptr->qp_handle))
+ return (dapl_convert_errno(errno, "destroy_qp"));
+
+ ep_ptr->qp_handle = NULL;
+ }
+
+#ifdef DAT_EXTENSIONS
+ /* UD endpoints can have many CR associations and will not
+ * set ep->cm_handle. Call provider with cm_ptr null to incidate
+ * UD type multi CR's for this EP. It will parse internal list
+ * and cleanup all associations.
+ */
+ if (ep_ptr->param.ep_attr.service_type == DAT_IB_SERVICE_TYPE_UD)
+ dapls_ib_cm_free(NULL, ep_ptr);
+#endif
+
+ return DAT_SUCCESS;
+}
+
+/*
+ * dapl_ib_qp_modify
+ *
+ * Set the QP to the parameters specified in an EP_PARAM
+ *
+ * The EP_PARAM structure that is provided has been
+ * sanitized such that only non-zero values are valid.
+ *
+ * Input:
+ * ib_hca_handle HCA handle
+ * qp_handle QP handle
+ * ep_attr Sanitized EP Params
+ *
+ * Output:
+ * none
+ *
+ * Returns:
+ * DAT_SUCCESS
+ * DAT_INSUFFICIENT_RESOURCES
+ * DAT_INVALID_PARAMETER
+ *
+ */
+DAT_RETURN
+dapls_ib_qp_modify(IN DAPL_IA * ia_ptr,
+ IN DAPL_EP * ep_ptr, IN DAT_EP_ATTR * attr)
+{
+ struct ibv_qp_attr qp_attr;
+
+ if (ep_ptr->qp_handle == IB_INVALID_HANDLE)
+ return DAT_INVALID_PARAMETER;
+
+ /*
+ * EP state, qp_handle state should be an indication
+ * of current state but the only way to be sure is with
+ * a user mode ibv_query_qp call which is NOT available
+ */
+
+ /* move to error state if necessary */
+ if ((ep_ptr->qp_state == IB_QP_STATE_ERROR) &&
+ (ep_ptr->qp_handle->state != IBV_QPS_ERR)) {
+ return (dapls_modify_qp_state(ep_ptr->qp_handle,
+ IBV_QPS_ERR, 0, 0, 0));
+ }
+
+ /*
+ * Check if we have the right qp_state to modify attributes
+ */
+ if ((ep_ptr->qp_handle->state != IBV_QPS_RTR) &&
+ (ep_ptr->qp_handle->state != IBV_QPS_RTS))
+ return DAT_INVALID_STATE;
+
+ /* Adjust to current EP attributes */
+ dapl_os_memzero((void *)&qp_attr, sizeof(qp_attr));
+ qp_attr.cap.max_send_wr = attr->max_request_dtos;
+ qp_attr.cap.max_recv_wr = attr->max_recv_dtos;
+ qp_attr.cap.max_send_sge = attr->max_request_iov;
+ qp_attr.cap.max_recv_sge = attr->max_recv_iov;
+
+ dapl_dbg_log(DAPL_DBG_TYPE_EP,
+ "modify_qp: qp %p sq %d,%d, rq %d,%d\n",
+ ep_ptr->qp_handle,
+ qp_attr.cap.max_send_wr, qp_attr.cap.max_send_sge,
+ qp_attr.cap.max_recv_wr, qp_attr.cap.max_recv_sge);
+
+ if (ibv_modify_qp(ep_ptr->qp_handle, &qp_attr, IBV_QP_CAP)) {
+ dapl_dbg_log(DAPL_DBG_TYPE_ERR,
+ "modify_qp: modify ep %p qp %p failed\n",
+ ep_ptr, ep_ptr->qp_handle);
+ return (dapl_convert_errno(errno, "modify_qp_state"));
+ }
+
+ return DAT_SUCCESS;
+}
+
+/*
+ * dapls_ib_reinit_ep
+ *
+ * Move the QP to INIT state again.
+ *
+ * Input:
+ * ep_ptr DAPL_EP
+ *
+ * Output:
+ * none
+ *
+ * Returns:
+ * void
+ *
+ */
+#if defined(_WIN32) || defined(_WIN64) || defined(_OPENIB_CMA_)
+void dapls_ib_reinit_ep(IN DAPL_EP * ep_ptr)
+{
+ /* work around bug in low level driver - 3/24/09 */
+ /* RTS -> RESET -> INIT -> ERROR QP transition crashes system */
+ if (ep_ptr->qp_handle != IB_INVALID_HANDLE) {
+ dapls_ib_qp_free(ep_ptr->header.owner_ia, ep_ptr);
+ dapls_ib_qp_alloc(ep_ptr->header.owner_ia, ep_ptr, ep_ptr);
+ }
+}
+#else // _WIN32 || _WIN64
+void dapls_ib_reinit_ep(IN DAPL_EP * ep_ptr)
+{
+ if (ep_ptr->qp_handle != IB_INVALID_HANDLE &&
+ ep_ptr->qp_handle->qp_type != IBV_QPT_UD) {
+ /* move to RESET state and then to INIT */
+ dapls_modify_qp_state(ep_ptr->qp_handle, IBV_QPS_RESET,0,0,0);
+ dapls_modify_qp_state(ep_ptr->qp_handle, IBV_QPS_INIT,0,0,0);
+ }
+}
+#endif // _WIN32 || _WIN64
+
+/*
+ * Generic QP modify for init, reset, error, RTS, RTR
+ * For UD, create_ah on RTR, qkey on INIT
+ * CM msg provides QP attributes, info in network order
+ */
+DAT_RETURN
+dapls_modify_qp_state(IN ib_qp_handle_t qp_handle,
+ IN ib_qp_state_t qp_state,
+ IN uint32_t qpn,
+ IN uint16_t lid,
+ IN ib_gid_handle_t gid)
+{
+ struct ibv_qp_attr qp_attr;
+ enum ibv_qp_attr_mask mask = IBV_QP_STATE;
+ DAPL_EP *ep_ptr = (DAPL_EP *) qp_handle->qp_context;
+ DAPL_IA *ia_ptr = ep_ptr->header.owner_ia;
+ int ret;
+
+ dapl_os_memzero((void *)&qp_attr, sizeof(qp_attr));
+ qp_attr.qp_state = qp_state;
+
+ switch (qp_state) {
+ case IBV_QPS_RTR:
+ dapl_dbg_log(DAPL_DBG_TYPE_EP,
+ " QPS_RTR: type %d qpn 0x%x lid 0x%x"
+ " port %d ep %p qp_state %d \n",
+ qp_handle->qp_type,
+ ntohl(qpn), ntohs(lid),
+ ia_ptr->hca_ptr->port_num,
+ ep_ptr, ep_ptr->qp_state);
+
+ mask |= IBV_QP_AV |
+ IBV_QP_PATH_MTU |
+ IBV_QP_DEST_QPN |
+ IBV_QP_RQ_PSN |
+ IBV_QP_MAX_DEST_RD_ATOMIC | IBV_QP_MIN_RNR_TIMER;
+
+ qp_attr.dest_qp_num = ntohl(qpn);
+ qp_attr.rq_psn = 1;
+ qp_attr.path_mtu = ia_ptr->hca_ptr->ib_trans.mtu;
+ qp_attr.max_dest_rd_atomic =
+ ep_ptr->param.ep_attr.max_rdma_read_out;
+ qp_attr.min_rnr_timer =
+ ia_ptr->hca_ptr->ib_trans.rnr_timer;
+
+ /* address handle. RC and UD */
+ qp_attr.ah_attr.dlid = ntohs(lid);
+ if (ia_ptr->hca_ptr->ib_trans.global) {
+ qp_attr.ah_attr.is_global = 1;
+ qp_attr.ah_attr.grh.dgid.global.subnet_prefix =
+ ntohll(gid->global.subnet_prefix);
+ qp_attr.ah_attr.grh.dgid.global.interface_id =
+ ntohll(gid->global.interface_id);
+ qp_attr.ah_attr.grh.hop_limit =
+ ia_ptr->hca_ptr->ib_trans.hop_limit;
+ qp_attr.ah_attr.grh.traffic_class =
+ ia_ptr->hca_ptr->ib_trans.tclass;
+ }
+ qp_attr.ah_attr.sl = 0;
+ qp_attr.ah_attr.src_path_bits = 0;
+ qp_attr.ah_attr.port_num = ia_ptr->hca_ptr->port_num;
+
+ /* UD: already in RTR, RTS state */
+ if (qp_handle->qp_type == IBV_QPT_UD) {
+ mask = IBV_QP_STATE;
+ if (ep_ptr->qp_state == IBV_QPS_RTR ||
+ ep_ptr->qp_state == IBV_QPS_RTS)
+ return DAT_SUCCESS;
+ }
+ break;
+ case IBV_QPS_RTS:
+ if (qp_handle->qp_type == IBV_QPT_RC) {
+ mask |= IBV_QP_SQ_PSN |
+ IBV_QP_TIMEOUT |
+ IBV_QP_RETRY_CNT |
+ IBV_QP_RNR_RETRY | IBV_QP_MAX_QP_RD_ATOMIC;
+ qp_attr.timeout =
+ ia_ptr->hca_ptr->ib_trans.ack_timer;
+ qp_attr.retry_cnt =
+ ia_ptr->hca_ptr->ib_trans.ack_retry;
+ qp_attr.rnr_retry =
+ ia_ptr->hca_ptr->ib_trans.rnr_retry;
+ qp_attr.max_rd_atomic =
+ ep_ptr->param.ep_attr.max_rdma_read_out;
+ }
+ /* RC and UD */
+ qp_attr.qp_state = IBV_QPS_RTS;
+ qp_attr.sq_psn = 1;
+
+ dapl_dbg_log(DAPL_DBG_TYPE_EP,
+ " QPS_RTS: psn %x rd_atomic %d ack %d "
+ " retry %d rnr_retry %d ep %p qp_state %d\n",
+ qp_attr.sq_psn, qp_attr.max_rd_atomic,
+ qp_attr.timeout, qp_attr.retry_cnt,
+ qp_attr.rnr_retry, ep_ptr,
+ ep_ptr->qp_state);
+
+ if (qp_handle->qp_type == IBV_QPT_UD) {
+ /* already RTS, multi remote AH's on QP */
+ if (ep_ptr->qp_state == IBV_QPS_RTS)
+ return DAT_SUCCESS;
+ else
+ mask = IBV_QP_STATE | IBV_QP_SQ_PSN;
+ }
+ break;
+ case IBV_QPS_INIT:
+ mask |= IBV_QP_PKEY_INDEX | IBV_QP_PORT;
+ if (qp_handle->qp_type == IBV_QPT_RC) {
+ mask |= IBV_QP_ACCESS_FLAGS;
+ qp_attr.qp_access_flags =
+ IBV_ACCESS_LOCAL_WRITE |
+ IBV_ACCESS_REMOTE_WRITE |
+ IBV_ACCESS_REMOTE_READ |
+ IBV_ACCESS_REMOTE_ATOMIC |
+ IBV_ACCESS_MW_BIND;
+ }
+
+ if (qp_handle->qp_type == IBV_QPT_UD) {
+ /* already INIT, multi remote AH's on QP */
+ if (ep_ptr->qp_state == IBV_QPS_INIT)
+ return DAT_SUCCESS;
+ mask |= IBV_QP_QKEY;
+ qp_attr.qkey = DAT_UD_QKEY;
+ }
+
+ qp_attr.pkey_index = 0;
+ qp_attr.port_num = ia_ptr->hca_ptr->port_num;
+
+ dapl_dbg_log(DAPL_DBG_TYPE_EP,
+ " QPS_INIT: pi %x port %x acc %x qkey 0x%x\n",
+ qp_attr.pkey_index, qp_attr.port_num,
+ qp_attr.qp_access_flags, qp_attr.qkey);
+ break;
+ default:
+ break;
+ }
+
+ ret = ibv_modify_qp(qp_handle, &qp_attr, mask);
+ if (ret == 0) {
+ ep_ptr->qp_state = qp_state;
+ return DAT_SUCCESS;
+ } else {
+ return (dapl_convert_errno(errno, "modify_qp_state"));
+ }
+}
+
+/* Modify UD type QP from init, rtr, rts, info network order */
+DAT_RETURN
+dapls_modify_qp_ud(IN DAPL_HCA *hca, IN ib_qp_handle_t qp)
+{
+ struct ibv_qp_attr qp_attr;
+
+ /* modify QP, setup and prepost buffers */
+ dapl_os_memzero((void *)&qp_attr, sizeof(qp_attr));
+ qp_attr.qp_state = IBV_QPS_INIT;
+ qp_attr.pkey_index = 0;
+ qp_attr.port_num = hca->port_num;
+ qp_attr.qkey = DAT_UD_QKEY;
+ if (ibv_modify_qp(qp, &qp_attr,
+ IBV_QP_STATE |
+ IBV_QP_PKEY_INDEX |
+ IBV_QP_PORT |
+ IBV_QP_QKEY)) {
+ dapl_log(DAPL_DBG_TYPE_ERR,
+ " modify_ud_qp INIT: ERR %s\n", strerror(errno));
+ return (dapl_convert_errno(errno, "modify_qp"));
+ }
+ dapl_os_memzero((void *)&qp_attr, sizeof(qp_attr));
+ qp_attr.qp_state = IBV_QPS_RTR;
+ if (ibv_modify_qp(qp, &qp_attr,IBV_QP_STATE)) {
+ dapl_log(DAPL_DBG_TYPE_ERR,
+ " modify_ud_qp RTR: ERR %s\n", strerror(errno));
+ return (dapl_convert_errno(errno, "modify_qp"));
+ }
+ dapl_os_memzero((void *)&qp_attr, sizeof(qp_attr));
+ qp_attr.qp_state = IBV_QPS_RTS;
+ qp_attr.sq_psn = 1;
+ if (ibv_modify_qp(qp, &qp_attr,
+ IBV_QP_STATE | IBV_QP_SQ_PSN)) {
+ dapl_log(DAPL_DBG_TYPE_ERR,
+ " modify_ud_qp RTS: ERR %s\n", strerror(errno));
+ return (dapl_convert_errno(errno, "modify_qp"));
+ }
+ return DAT_SUCCESS;
+}
+
+/* Create address handle for remote QP, info in network order */
+ib_ah_handle_t
+dapls_create_ah(IN DAPL_HCA *hca,
+ IN ib_pd_handle_t pd,
+ IN ib_qp_handle_t qp,
+ IN uint16_t lid,
+ IN ib_gid_handle_t gid)
+{
+ struct ibv_qp_attr qp_attr;
+ ib_ah_handle_t ah;
+
+ if (qp->qp_type != IBV_QPT_UD)
+ return NULL;
+
+ dapl_os_memzero((void *)&qp_attr, sizeof(qp_attr));
+ qp_attr.qp_state = IBV_QP_STATE;
+
+ /* address handle. RC and UD */
+ qp_attr.ah_attr.dlid = ntohs(lid);
+ if (gid != NULL) {
+ dapl_log(DAPL_DBG_TYPE_CM, "dapl_create_ah: with GID\n");
+ qp_attr.ah_attr.is_global = 1;
+ qp_attr.ah_attr.grh.dgid.global.subnet_prefix =
+ ntohll(gid->global.subnet_prefix);
+ qp_attr.ah_attr.grh.dgid.global.interface_id =
+ ntohll(gid->global.interface_id);
+ qp_attr.ah_attr.grh.hop_limit = hca->ib_trans.hop_limit;
+ qp_attr.ah_attr.grh.traffic_class = hca->ib_trans.tclass;
+ }
+ qp_attr.ah_attr.sl = 0;
+ qp_attr.ah_attr.src_path_bits = 0;
+ qp_attr.ah_attr.port_num = hca->port_num;
+
+ dapl_log(DAPL_DBG_TYPE_CM,
+ " dapls_create_ah: port %x lid %x pd %p ctx %p handle 0x%x\n",
+ hca->port_num,qp_attr.ah_attr.dlid, pd, pd->context, pd->handle);
+
+ /* UD: create AH for remote side */
+ ah = ibv_create_ah(pd, &qp_attr.ah_attr);
+ if (!ah) {
+ dapl_log(DAPL_DBG_TYPE_ERR,
+ " create_ah: ERR %s\n", strerror(errno));
+ return NULL;
+ }
+
+ dapl_log(DAPL_DBG_TYPE_CM,
+ " dapls_create_ah: AH %p for lid %x\n",
+ ah, qp_attr.ah_attr.dlid);
+
+ return ah;
+}
+
+/*
+ * Local variables:
+ * c-indent-level: 4
+ * c-basic-offset: 4
+ * tab-width: 8
+ * End:
+ */
-/*\r
- * This Software is licensed under one of the following licenses:\r
- *\r
- * 1) under the terms of the "Common Public License 1.0" a copy of which is\r
- * available from the Open Source Initiative, see\r
- * http://www.opensource.org/licenses/cpl.php.\r
- *\r
- * 2) under the terms of the "The BSD License" a copy of which is\r
- * available from the Open Source Initiative, see\r
- * http://www.opensource.org/licenses/bsd-license.php.\r
- *\r
- * 3) under the terms of the "GNU General Public License (GPL) Version 2" a\r
- * copy of which is available from the Open Source Initiative, see\r
- * http://www.opensource.org/licenses/gpl-license.php.\r
- *\r
- * Licensee has the right to choose one of the above licenses.\r
- *\r
- * Redistributions of source code must retain the above copyright\r
- * notice and one of the license notices.\r
- *\r
- * Redistributions in binary form must reproduce both the above copyright\r
- * notice, one of the license notices in the documentation\r
- * and/or other materials provided with the distribution.\r
- */\r
-\r
-/***************************************************************************\r
- *\r
- * Module: uDAPL\r
- *\r
- * Filename: dapl_ib_cm.c\r
- *\r
- * Author: Arlin Davis\r
- *\r
- * Created: 3/10/2005\r
- *\r
- * Description: \r
- *\r
- * The uDAPL openib provider - connection management\r
- *\r
- ****************************************************************************\r
- * Source Control System Information\r
- *\r
- * $Id: $\r
- *\r
- * Copyright (c) 2005 Intel Corporation. All rights reserved.\r
- *\r
- **************************************************************************/\r
-\r
-#if defined(_WIN32)\r
-#define FD_SETSIZE 1024\r
-#define DAPL_FD_SETSIZE FD_SETSIZE\r
-#endif\r
-\r
-#include "dapl.h"\r
-#include "dapl_adapter_util.h"\r
-#include "dapl_evd_util.h"\r
-#include "dapl_cr_util.h"\r
-#include "dapl_name_service.h"\r
-#include "dapl_ib_util.h"\r
-#include "dapl_osd.h"\r
-\r
-#if defined(_WIN32) || defined(_WIN64)\r
-enum DAPL_FD_EVENTS {\r
- DAPL_FD_READ = 0x1,\r
- DAPL_FD_WRITE = 0x2,\r
- DAPL_FD_ERROR = 0x4\r
-};\r
-\r
-static int dapl_config_socket(DAPL_SOCKET s)\r
-{\r
- unsigned long nonblocking = 1;\r
- return ioctlsocket(s, FIONBIO, &nonblocking);\r
-}\r
-\r
-static int dapl_connect_socket(DAPL_SOCKET s, struct sockaddr *addr,\r
- int addrlen)\r
-{\r
- int err;\r
-\r
- err = connect(s, addr, addrlen);\r
- if (err == SOCKET_ERROR)\r
- err = WSAGetLastError();\r
- return (err == WSAEWOULDBLOCK) ? EAGAIN : err;\r
-}\r
-\r
-struct dapl_fd_set {\r
- struct fd_set set[3];\r
-};\r
-\r
-static struct dapl_fd_set *dapl_alloc_fd_set(void)\r
-{\r
- return dapl_os_alloc(sizeof(struct dapl_fd_set));\r
-}\r
-\r
-static void dapl_fd_zero(struct dapl_fd_set *set)\r
-{\r
- FD_ZERO(&set->set[0]);\r
- FD_ZERO(&set->set[1]);\r
- FD_ZERO(&set->set[2]);\r
-}\r
-\r
-static int dapl_fd_set(DAPL_SOCKET s, struct dapl_fd_set *set,\r
- enum DAPL_FD_EVENTS event)\r
-{\r
- FD_SET(s, &set->set[(event == DAPL_FD_READ) ? 0 : 1]);\r
- FD_SET(s, &set->set[2]);\r
- return 0;\r
-}\r
-\r
-static enum DAPL_FD_EVENTS dapl_poll(DAPL_SOCKET s, enum DAPL_FD_EVENTS event)\r
-{\r
- struct fd_set rw_fds;\r
- struct fd_set err_fds;\r
- struct timeval tv;\r
- int ret;\r
-\r
- FD_ZERO(&rw_fds);\r
- FD_ZERO(&err_fds);\r
- FD_SET(s, &rw_fds);\r
- FD_SET(s, &err_fds);\r
-\r
- tv.tv_sec = 0;\r
- tv.tv_usec = 0;\r
-\r
- if (event == DAPL_FD_READ)\r
- ret = select(1, &rw_fds, NULL, &err_fds, &tv);\r
- else\r
- ret = select(1, NULL, &rw_fds, &err_fds, &tv);\r
-\r
- if (ret == 0)\r
- return 0;\r
- else if (ret == SOCKET_ERROR)\r
- return DAPL_FD_ERROR;\r
- else if (FD_ISSET(s, &rw_fds))\r
- return event;\r
- else\r
- return DAPL_FD_ERROR;\r
-}\r
-\r
-static int dapl_select(struct dapl_fd_set *set)\r
-{\r
- int ret;\r
-\r
- dapl_dbg_log(DAPL_DBG_TYPE_CM, " dapl_select: sleep\n");\r
- ret = select(0, &set->set[0], &set->set[1], &set->set[2], NULL);\r
- dapl_dbg_log(DAPL_DBG_TYPE_CM, " dapl_select: wakeup\n");\r
-\r
- if (ret == SOCKET_ERROR)\r
- dapl_dbg_log(DAPL_DBG_TYPE_CM,\r
- " dapl_select: error 0x%x\n", WSAGetLastError());\r
-\r
- return ret;\r
-}\r
-\r
-static int dapl_socket_errno(void)\r
-{\r
- int err;\r
-\r
- err = WSAGetLastError();\r
- switch (err) {\r
- case WSAEACCES:\r
- case WSAEADDRINUSE:\r
- return EADDRINUSE;\r
- case WSAECONNRESET:\r
- return ECONNRESET;\r
- default:\r
- return err;\r
- }\r
-}\r
-#else // _WIN32 || _WIN64\r
-enum DAPL_FD_EVENTS {\r
- DAPL_FD_READ = POLLIN,\r
- DAPL_FD_WRITE = POLLOUT,\r
- DAPL_FD_ERROR = POLLERR\r
-};\r
-\r
-static int dapl_config_socket(DAPL_SOCKET s)\r
-{\r
- int ret;\r
-\r
- ret = fcntl(s, F_GETFL);\r
- if (ret >= 0)\r
- ret = fcntl(s, F_SETFL, ret | O_NONBLOCK);\r
- return ret;\r
-}\r
-\r
-static int dapl_connect_socket(DAPL_SOCKET s, struct sockaddr *addr,\r
- int addrlen)\r
-{\r
- int ret;\r
-\r
- ret = connect(s, addr, addrlen);\r
-\r
- return (errno == EINPROGRESS) ? EAGAIN : ret;\r
-}\r
-\r
-struct dapl_fd_set {\r
- int index;\r
- struct pollfd set[DAPL_FD_SETSIZE];\r
-};\r
-\r
-static struct dapl_fd_set *dapl_alloc_fd_set(void)\r
-{\r
- return dapl_os_alloc(sizeof(struct dapl_fd_set));\r
-}\r
-\r
-static void dapl_fd_zero(struct dapl_fd_set *set)\r
-{\r
- set->index = 0;\r
-}\r
-\r
-static int dapl_fd_set(DAPL_SOCKET s, struct dapl_fd_set *set,\r
- enum DAPL_FD_EVENTS event)\r
-{\r
- if (set->index == DAPL_FD_SETSIZE - 1) {\r
- dapl_log(DAPL_DBG_TYPE_ERR,\r
- "SCM ERR: cm_thread exceeded FD_SETSIZE %d\n",\r
- set->index + 1);\r
- return -1;\r
- }\r
-\r
- set->set[set->index].fd = s;\r
- set->set[set->index].revents = 0;\r
- set->set[set->index++].events = event;\r
- return 0;\r
-}\r
-\r
-static enum DAPL_FD_EVENTS dapl_poll(DAPL_SOCKET s, enum DAPL_FD_EVENTS event)\r
-{\r
- struct pollfd fds;\r
- int ret;\r
-\r
- fds.fd = s;\r
- fds.events = event;\r
- fds.revents = 0;\r
- ret = poll(&fds, 1, 0);\r
- dapl_log(DAPL_DBG_TYPE_CM, " dapl_poll: fd=%d ret=%d, evnts=0x%x\n",\r
- s, ret, fds.revents);\r
- if (ret == 0)\r
- return 0;\r
- else if (ret < 0 || (fds.revents & (POLLERR | POLLHUP | POLLNVAL))) \r
- return DAPL_FD_ERROR;\r
- else \r
- return event;\r
-}\r
-\r
-static int dapl_select(struct dapl_fd_set *set)\r
-{\r
- int ret;\r
-\r
- dapl_dbg_log(DAPL_DBG_TYPE_CM, " dapl_select: sleep, fds=%d\n", set->index);\r
- ret = poll(set->set, set->index, -1);\r
- dapl_dbg_log(DAPL_DBG_TYPE_CM, " dapl_select: wakeup, ret=0x%x\n", ret);\r
- return ret;\r
-}\r
-\r
-#define dapl_socket_errno() errno\r
-#endif\r
-\r
-dp_ib_cm_handle_t dapls_ib_cm_create(DAPL_EP *ep)\r
-{\r
- dp_ib_cm_handle_t cm_ptr;\r
-\r
- /* Allocate CM, init lock, and initialize */\r
- if ((cm_ptr = dapl_os_alloc(sizeof(*cm_ptr))) == NULL)\r
- return NULL;\r
-\r
- (void)dapl_os_memzero(cm_ptr, sizeof(*cm_ptr));\r
- if (dapl_os_lock_init(&cm_ptr->lock))\r
- goto bail;\r
-\r
- cm_ptr->msg.ver = htons(DCM_VER);\r
- cm_ptr->socket = DAPL_INVALID_SOCKET;\r
- cm_ptr->ep = ep;\r
- return cm_ptr;\r
-bail:\r
- dapl_os_free(cm_ptr, sizeof(*cm_ptr));\r
- return NULL;\r
-}\r
-\r
-/* mark for destroy, remove all references, schedule cleanup */\r
-/* cm_ptr == NULL (UD), then multi CR's, kill all associated with EP */\r
-void dapls_ib_cm_free(dp_ib_cm_handle_t cm_ptr, DAPL_EP *ep)\r
-{\r
- DAPL_IA *ia_ptr;\r
- DAPL_HCA *hca_ptr = NULL;\r
- dp_ib_cm_handle_t cr, next_cr;\r
-\r
- dapl_dbg_log(DAPL_DBG_TYPE_CM,\r
- " cm_destroy: cm %p ep %p\n", cm_ptr, ep);\r
-\r
- if (cm_ptr == NULL)\r
- goto multi_cleanup;\r
-\r
- /* to notify cleanup thread */\r
- hca_ptr = cm_ptr->hca;\r
-\r
- /* cleanup, never made it to work queue */\r
- dapl_os_lock(&cm_ptr->lock);\r
- if (cm_ptr->state == DCM_INIT) {\r
- if (cm_ptr->socket != DAPL_INVALID_SOCKET) {\r
- shutdown(cm_ptr->socket, SHUT_RDWR);\r
- closesocket(cm_ptr->socket);\r
- }\r
- dapl_os_unlock(&cm_ptr->lock);\r
- dapl_os_free(cm_ptr, sizeof(*cm_ptr));\r
- return;\r
- }\r
-\r
- /* free could be called before disconnect, disc_clean will destroy */\r
- if (cm_ptr->state == DCM_CONNECTED) {\r
- dapl_os_unlock(&cm_ptr->lock);\r
- dapli_socket_disconnect(cm_ptr);\r
- return;\r
- }\r
-\r
- cm_ptr->state = DCM_DESTROY;\r
- if ((cm_ptr->ep) && (cm_ptr->ep->cm_handle == cm_ptr)) {\r
- cm_ptr->ep->cm_handle = IB_INVALID_HANDLE;\r
- cm_ptr->ep = NULL;\r
- }\r
-\r
- dapl_os_unlock(&cm_ptr->lock);\r
- goto notify_thread;\r
-\r
-multi_cleanup:\r
-\r
- /* \r
- * UD CR objects are kept active because of direct private data references\r
- * from CONN events. The cr->socket is closed and marked inactive but the \r
- * object remains allocated and queued on the CR resource list. There can\r
- * be multiple CR's associated with a given EP. There is no way to determine \r
- * when consumer is finished with event until the dat_ep_free.\r
- *\r
- * Schedule destruction for all CR's associated with this EP, cr_thread will\r
- * complete the cleanup with state == DCM_DESTROY. \r
- */ \r
- ia_ptr = ep->header.owner_ia;\r
- dapl_os_lock(&ia_ptr->hca_ptr->ib_trans.lock);\r
- if (!dapl_llist_is_empty((DAPL_LLIST_HEAD*)\r
- &ia_ptr->hca_ptr->ib_trans.list))\r
- next_cr = dapl_llist_peek_head((DAPL_LLIST_HEAD*)\r
- &ia_ptr->hca_ptr->ib_trans.list);\r
- else\r
- next_cr = NULL;\r
-\r
- while (next_cr) {\r
- cr = next_cr;\r
- next_cr = dapl_llist_next_entry((DAPL_LLIST_HEAD*)\r
- &ia_ptr->hca_ptr->ib_trans.list,\r
- (DAPL_LLIST_ENTRY*)&cr->entry);\r
- if (cr->ep == ep) {\r
- dapl_dbg_log(DAPL_DBG_TYPE_EP,\r
- " qp_free CR: ep %p cr %p\n", ep, cr);\r
- dapli_socket_disconnect(cr);\r
- dapl_os_lock(&cr->lock);\r
- hca_ptr = cr->hca;\r
- cr->ep = NULL;\r
- if (cr->ah) {\r
- ibv_destroy_ah(cr->ah);\r
- cr->ah = NULL;\r
- }\r
- cr->state = DCM_DESTROY;\r
- dapl_os_unlock(&cr->lock);\r
- }\r
- }\r
- dapl_os_unlock(&ia_ptr->hca_ptr->ib_trans.lock);\r
-\r
-notify_thread:\r
-\r
- /* wakeup work thread, if something destroyed */\r
- if (hca_ptr != NULL) \r
- send(hca_ptr->ib_trans.scm[1], "w", sizeof "w", 0);\r
-}\r
-\r
-/* queue socket for processing CM work */\r
-static void dapli_cm_queue(struct ib_cm_handle *cm_ptr)\r
-{\r
- /* add to work queue for cr thread processing */\r
- dapl_llist_init_entry((DAPL_LLIST_ENTRY *) & cm_ptr->entry);\r
- dapl_os_lock(&cm_ptr->hca->ib_trans.lock);\r
- dapl_llist_add_tail(&cm_ptr->hca->ib_trans.list,\r
- (DAPL_LLIST_ENTRY *) & cm_ptr->entry, cm_ptr);\r
- dapl_os_unlock(&cm_ptr->hca->ib_trans.lock);\r
-\r
- /* wakeup CM work thread */\r
- send(cm_ptr->hca->ib_trans.scm[1], "w", sizeof "w", 0);\r
-}\r
-\r
-/*\r
- * ACTIVE/PASSIVE: called from CR thread or consumer via ep_disconnect\r
- * or from ep_free\r
- */\r
-DAT_RETURN dapli_socket_disconnect(dp_ib_cm_handle_t cm_ptr)\r
-{\r
- DAPL_EP *ep_ptr = cm_ptr->ep;\r
- DAT_UINT32 disc_data = htonl(0xdead);\r
-\r
- if (ep_ptr == NULL)\r
- return DAT_SUCCESS;\r
-\r
- dapl_os_lock(&cm_ptr->lock);\r
- if (cm_ptr->state != DCM_CONNECTED) {\r
- dapl_os_unlock(&cm_ptr->lock);\r
- return DAT_SUCCESS;\r
- }\r
-\r
- /* send disc date, close socket, schedule destroy */\r
- dapls_modify_qp_state(ep_ptr->qp_handle, IBV_QPS_ERR, 0,0,0);\r
- cm_ptr->state = DCM_DISCONNECTED;\r
- send(cm_ptr->socket, (char *)&disc_data, sizeof(disc_data), 0);\r
- dapl_os_unlock(&cm_ptr->lock);\r
-\r
- /* disconnect events for RC's only */\r
- if (ep_ptr->param.ep_attr.service_type == DAT_SERVICE_TYPE_RC) {\r
- if (ep_ptr->cr_ptr) {\r
- dapls_cr_callback(cm_ptr,\r
- IB_CME_DISCONNECTED,\r
- NULL,\r
- ((DAPL_CR *) ep_ptr->cr_ptr)->sp_ptr);\r
- } else {\r
- dapl_evd_connection_callback(ep_ptr->cm_handle,\r
- IB_CME_DISCONNECTED,\r
- NULL, ep_ptr);\r
- }\r
- }\r
-\r
- /* scheduled destroy via disconnect clean in callback */\r
- return DAT_SUCCESS;\r
-}\r
-\r
-/*\r
- * ACTIVE: socket connected, send QP information to peer \r
- */\r
-static void dapli_socket_connected(dp_ib_cm_handle_t cm_ptr, int err)\r
-{\r
- int ret, len, exp, opt = 1;\r
- struct iovec iov[2];\r
- struct dapl_ep *ep_ptr = cm_ptr->ep;\r
-\r
- if (err) {\r
- dapl_log(DAPL_DBG_TYPE_ERR,\r
- " CONN_PENDING: %s ERR %s -> %s %d\n",\r
- err == -1 ? "POLL" : "SOCKOPT",\r
- err == -1 ? strerror(errno) : strerror(err), \r
- inet_ntoa(((struct sockaddr_in *)\r
- &cm_ptr->addr)->sin_addr), \r
- ntohs(((struct sockaddr_in *)\r
- &cm_ptr->addr)->sin_port));\r
- goto bail;\r
- }\r
-\r
- /* no delay for small packets */\r
- ret = setsockopt(cm_ptr->socket, IPPROTO_TCP, TCP_NODELAY,\r
- (char *)&opt, sizeof(opt));\r
- if (ret)\r
- dapl_log(DAPL_DBG_TYPE_ERR,\r
- " CONN_PENDING: NODELAY setsockopt: %s\n",\r
- strerror(errno));\r
- \r
- cm_ptr->state = DCM_REP_PENDING;\r
-\r
- /* send qp info and pdata to remote peer */\r
- exp = sizeof(ib_cm_msg_t) - DCM_MAX_PDATA_SIZE;\r
- iov[0].iov_base = (void *)&cm_ptr->msg;\r
- iov[0].iov_len = exp;\r
- if (cm_ptr->msg.p_size) {\r
- iov[1].iov_base = cm_ptr->msg.p_data;\r
- iov[1].iov_len = ntohs(cm_ptr->msg.p_size);\r
- len = writev(cm_ptr->socket, iov, 2);\r
- } else {\r
- len = writev(cm_ptr->socket, iov, 1);\r
- }\r
-\r
- if (len != (exp + ntohs(cm_ptr->msg.p_size))) {\r
- dapl_log(DAPL_DBG_TYPE_ERR,\r
- " CONN_PENDING len ERR %s, wcnt=%d(%d) -> %s\n",\r
- strerror(errno), len, \r
- exp + ntohs(cm_ptr->msg.p_size), \r
- inet_ntoa(((struct sockaddr_in *)\r
- ep_ptr->param.\r
- remote_ia_address_ptr)->sin_addr));\r
- goto bail;\r
- }\r
-\r
- dapl_dbg_log(DAPL_DBG_TYPE_CM,\r
- " CONN_PENDING: sending SRC lid=0x%x,"\r
- " qpn=0x%x, psize=%d\n",\r
- ntohs(cm_ptr->msg.saddr.ib.lid),\r
- ntohl(cm_ptr->msg.saddr.ib.qpn), \r
- ntohs(cm_ptr->msg.p_size));\r
- dapl_dbg_log(DAPL_DBG_TYPE_CM,\r
- " CONN_PENDING: SRC GID subnet %016llx id %016llx\n",\r
- (unsigned long long)\r
- htonll(*(uint64_t*)&cm_ptr->msg.saddr.ib.gid[0]),\r
- (unsigned long long)\r
- htonll(*(uint64_t*)&cm_ptr->msg.saddr.ib.gid[8]));\r
- return;\r
-\r
-bail:\r
- /* close socket, free cm structure and post error event */\r
- dapls_ib_cm_free(cm_ptr, cm_ptr->ep);\r
- dapl_evd_connection_callback(NULL, IB_CME_LOCAL_FAILURE, NULL, ep_ptr);\r
-}\r
-\r
-/*\r
- * ACTIVE: Create socket, connect, defer exchange QP information to CR thread\r
- * to avoid blocking. \r
- */\r
-DAT_RETURN\r
-dapli_socket_connect(DAPL_EP * ep_ptr,\r
- DAT_IA_ADDRESS_PTR r_addr,\r
- DAT_CONN_QUAL r_qual, DAT_COUNT p_size, DAT_PVOID p_data)\r
-{\r
- dp_ib_cm_handle_t cm_ptr;\r
- int ret;\r
- socklen_t sl;\r
- DAPL_IA *ia_ptr = ep_ptr->header.owner_ia;\r
- DAT_RETURN dat_ret = DAT_INSUFFICIENT_RESOURCES;\r
-\r
- dapl_dbg_log(DAPL_DBG_TYPE_EP, " connect: r_qual %d p_size=%d\n",\r
- r_qual, p_size);\r
-\r
- cm_ptr = dapls_ib_cm_create(ep_ptr);\r
- if (cm_ptr == NULL)\r
- return dat_ret;\r
-\r
- /* create, connect, sockopt, and exchange QP information */\r
- if ((cm_ptr->socket =\r
- socket(AF_INET, SOCK_STREAM, IPPROTO_TCP)) == DAPL_INVALID_SOCKET) {\r
- dapl_log(DAPL_DBG_TYPE_ERR,\r
- " connect: socket create ERR %s\n", strerror(errno));\r
- goto bail;\r
- }\r
-\r
- ret = dapl_config_socket(cm_ptr->socket);\r
- if (ret < 0) {\r
- dapl_log(DAPL_DBG_TYPE_ERR,\r
- " connect: config socket %d ERR %d %s\n",\r
- cm_ptr->socket, ret, strerror(errno));\r
- dat_ret = DAT_INTERNAL_ERROR;\r
- goto bail;\r
- }\r
-\r
- /* save remote address */\r
- dapl_os_memcpy(&cm_ptr->addr, r_addr, sizeof(r_addr));\r
-\r
-#ifdef DAPL_DBG\r
- /* DBG: Active PID [0], PASSIVE PID [2]*/\r
- *(uint16_t*)&cm_ptr->msg.resv[0] = htons((uint16_t)dapl_os_getpid()); \r
- *(uint16_t*)&cm_ptr->msg.resv[2] = ((struct sockaddr_in *)&cm_ptr->addr)->sin_port;\r
-#endif\r
- ((struct sockaddr_in *)&cm_ptr->addr)->sin_port = htons(r_qual + 1000);\r
- ret = dapl_connect_socket(cm_ptr->socket, (struct sockaddr *)&cm_ptr->addr,\r
- sizeof(cm_ptr->addr));\r
- if (ret && ret != EAGAIN) {\r
- dat_ret = DAT_INVALID_ADDRESS;\r
- goto bail;\r
- }\r
-\r
- /* REQ: QP info in msg.saddr, IA address in msg.daddr, and pdata */\r
- cm_ptr->msg.op = ntohs(DCM_REQ);\r
- cm_ptr->msg.saddr.ib.qpn = htonl(ep_ptr->qp_handle->qp_num);\r
- cm_ptr->msg.saddr.ib.qp_type = ep_ptr->qp_handle->qp_type;\r
- cm_ptr->msg.saddr.ib.lid = ia_ptr->hca_ptr->ib_trans.lid;\r
- dapl_os_memcpy(&cm_ptr->msg.saddr.ib.gid[0], \r
- &ia_ptr->hca_ptr->ib_trans.gid, 16);\r
- \r
- /* save references */\r
- cm_ptr->hca = ia_ptr->hca_ptr;\r
- cm_ptr->ep = ep_ptr;\r
- \r
- /* get local address information from socket */\r
- sl = sizeof(cm_ptr->msg.daddr.so);\r
- if (getsockname(cm_ptr->socket, (struct sockaddr *)&cm_ptr->msg.daddr.so, &sl)) {\r
- dapl_log(DAPL_DBG_TYPE_ERR,\r
- " connect getsockname ERROR: %s -> %s r_qual %d\n",\r
- strerror(errno), \r
- inet_ntoa(((struct sockaddr_in *)r_addr)->sin_addr),\r
- (unsigned int)r_qual);;\r
- }\r
-\r
- if (p_size) {\r
- cm_ptr->msg.p_size = htons(p_size);\r
- dapl_os_memcpy(cm_ptr->msg.p_data, p_data, p_size);\r
- }\r
-\r
- /* connected or pending, either way results via async event */\r
- if (ret == 0)\r
- dapli_socket_connected(cm_ptr, 0);\r
- else\r
- cm_ptr->state = DCM_CONN_PENDING;\r
-\r
- dapl_dbg_log(DAPL_DBG_TYPE_EP, " connect: p_data=%p %p\n",\r
- cm_ptr->msg.p_data, cm_ptr->msg.p_data);\r
-\r
- dapl_dbg_log(DAPL_DBG_TYPE_EP,\r
- " connect: %s r_qual %d pending, p_sz=%d, %d %d ...\n",\r
- inet_ntoa(((struct sockaddr_in *)&cm_ptr->addr)->sin_addr), \r
- (unsigned int)r_qual, ntohs(cm_ptr->msg.p_size),\r
- cm_ptr->msg.p_data[0], cm_ptr->msg.p_data[1]);\r
-\r
- dapli_cm_queue(cm_ptr);\r
- return DAT_SUCCESS;\r
-\r
-bail:\r
- dapl_log(DAPL_DBG_TYPE_ERR,\r
- " connect ERROR: %s -> %s r_qual %d\n",\r
- strerror(errno), \r
- inet_ntoa(((struct sockaddr_in *)r_addr)->sin_addr),\r
- (unsigned int)r_qual);\r
-\r
- /* close socket, free cm structure */\r
- dapls_ib_cm_free(cm_ptr, NULL);\r
- return dat_ret;\r
-}\r
-\r
-/*\r
- * ACTIVE: exchange QP information, called from CR thread\r
- */\r
-static void dapli_socket_connect_rtu(dp_ib_cm_handle_t cm_ptr)\r
-{\r
- DAPL_EP *ep_ptr = cm_ptr->ep;\r
- int len, exp = sizeof(ib_cm_msg_t) - DCM_MAX_PDATA_SIZE;\r
- ib_cm_events_t event = IB_CME_LOCAL_FAILURE;\r
- socklen_t sl;\r
-\r
- /* read DST information into cm_ptr, overwrite SRC info */\r
- dapl_dbg_log(DAPL_DBG_TYPE_EP, " connect_rtu: recv peer QP data\n");\r
-\r
- len = recv(cm_ptr->socket, (char *)&cm_ptr->msg, exp, 0);\r
- if (len != exp || ntohs(cm_ptr->msg.ver) != DCM_VER) {\r
- dapl_log(DAPL_DBG_TYPE_WARN,\r
- " CONN_RTU read: sk %d ERR %s, rcnt=%d, v=%d -> %s PORT L-%x R-%x PID L-%x R-%x\n",\r
- cm_ptr->socket, strerror(errno), len, ntohs(cm_ptr->msg.ver),\r
- inet_ntoa(((struct sockaddr_in *)&cm_ptr->addr)->sin_addr),\r
- ntohs(((struct sockaddr_in *)&cm_ptr->msg.daddr.so)->sin_port),\r
- ntohs(((struct sockaddr_in *)&cm_ptr->addr)->sin_port),\r
- ntohs(*(uint16_t*)&cm_ptr->msg.resv[0]),\r
- ntohs(*(uint16_t*)&cm_ptr->msg.resv[2]));\r
-\r
- /* Retry; corner case where server tcp stack resets under load */\r
- if (dapl_socket_errno() == ECONNRESET) {\r
- closesocket(cm_ptr->socket);\r
- cm_ptr->socket = DAPL_INVALID_SOCKET;\r
- dapli_socket_connect(cm_ptr->ep, (DAT_IA_ADDRESS_PTR)&cm_ptr->addr, \r
- ntohs(((struct sockaddr_in *)&cm_ptr->addr)->sin_port) - 1000,\r
- ntohs(cm_ptr->msg.p_size), &cm_ptr->msg.p_data);\r
- dapls_ib_cm_free(cm_ptr, NULL);\r
- return;\r
- }\r
- goto bail;\r
- }\r
-\r
- /* keep the QP, address info in network order */\r
- \r
- /* save remote address information, in msg.daddr */\r
- dapl_os_memcpy(&ep_ptr->remote_ia_address,\r
- &cm_ptr->msg.daddr.so,\r
- sizeof(union dcm_addr));\r
-\r
- /* save local address information from socket */\r
- sl = sizeof(cm_ptr->addr);\r
- getsockname(cm_ptr->socket,(struct sockaddr *)&cm_ptr->addr, &sl);\r
-\r
- dapl_dbg_log(DAPL_DBG_TYPE_EP,\r
- " CONN_RTU: DST %s %d lid=0x%x,"\r
- " qpn=0x%x, qp_type=%d, psize=%d\n",\r
- inet_ntoa(((struct sockaddr_in *)\r
- &cm_ptr->msg.daddr.so)->sin_addr),\r
- ntohs(((struct sockaddr_in *)\r
- &cm_ptr->msg.daddr.so)->sin_port),\r
- ntohs(cm_ptr->msg.saddr.ib.lid),\r
- ntohl(cm_ptr->msg.saddr.ib.qpn), \r
- cm_ptr->msg.saddr.ib.qp_type, \r
- ntohs(cm_ptr->msg.p_size));\r
-\r
- /* validate private data size before reading */\r
- if (ntohs(cm_ptr->msg.p_size) > DCM_MAX_PDATA_SIZE) {\r
- dapl_log(DAPL_DBG_TYPE_ERR,\r
- " CONN_RTU read: psize (%d) wrong -> %s\n",\r
- ntohs(cm_ptr->msg.p_size), \r
- inet_ntoa(((struct sockaddr_in *)\r
- ep_ptr->param.\r
- remote_ia_address_ptr)->sin_addr));\r
- goto bail;\r
- }\r
-\r
- /* read private data into cm_handle if any present */\r
- dapl_dbg_log(DAPL_DBG_TYPE_EP," CONN_RTU: read private data\n");\r
- exp = ntohs(cm_ptr->msg.p_size);\r
- if (exp) {\r
- len = recv(cm_ptr->socket, cm_ptr->msg.p_data, exp, 0);\r
- if (len != exp) {\r
- dapl_log(DAPL_DBG_TYPE_ERR,\r
- " CONN_RTU read pdata: ERR %s, rcnt=%d -> %s\n",\r
- strerror(errno), len,\r
- inet_ntoa(((struct sockaddr_in *)\r
- ep_ptr->param.\r
- remote_ia_address_ptr)->sin_addr));\r
- goto bail;\r
- }\r
- }\r
-\r
- /* check for consumer or protocol stack reject */\r
- if (ntohs(cm_ptr->msg.op) == DCM_REP)\r
- event = IB_CME_CONNECTED;\r
- else if (ntohs(cm_ptr->msg.op) == DCM_REJ_USER) \r
- event = IB_CME_DESTINATION_REJECT_PRIVATE_DATA;\r
- else \r
- event = IB_CME_DESTINATION_REJECT;\r
- \r
- if (event != IB_CME_CONNECTED) {\r
- dapl_log(DAPL_DBG_TYPE_CM,\r
- " CONN_RTU: reject from %s %x\n",\r
- inet_ntoa(((struct sockaddr_in *)\r
- &cm_ptr->msg.daddr.so)->sin_addr),\r
- ntohs(((struct sockaddr_in *)\r
- &cm_ptr->msg.daddr.so)->sin_port));\r
- goto bail;\r
- }\r
-\r
- /* modify QP to RTR and then to RTS with remote info */\r
- dapl_os_lock(&ep_ptr->header.lock);\r
- if (dapls_modify_qp_state(ep_ptr->qp_handle,\r
- IBV_QPS_RTR, \r
- cm_ptr->msg.saddr.ib.qpn,\r
- cm_ptr->msg.saddr.ib.lid,\r
- NULL) != DAT_SUCCESS) {\r
- dapl_log(DAPL_DBG_TYPE_ERR,\r
- " CONN_RTU: QPS_RTR ERR %s (%d,%d,%x,%x,%x) -> %s %x\n",\r
- strerror(errno), ep_ptr->qp_handle->qp_type,\r
- ep_ptr->qp_state, ep_ptr->qp_handle->qp_num,\r
- ntohl(cm_ptr->msg.saddr.ib.qpn), \r
- ntohs(cm_ptr->msg.saddr.ib.lid),\r
- inet_ntoa(((struct sockaddr_in *)\r
- &cm_ptr->msg.daddr.so)->sin_addr),\r
- ntohs(((struct sockaddr_in *)\r
- &cm_ptr->msg.daddr.so)->sin_port));\r
- dapl_os_unlock(&ep_ptr->header.lock);\r
- goto bail;\r
- }\r
- if (dapls_modify_qp_state(ep_ptr->qp_handle,\r
- IBV_QPS_RTS, \r
- cm_ptr->msg.saddr.ib.qpn,\r
- cm_ptr->msg.saddr.ib.lid,\r
- NULL) != DAT_SUCCESS) {\r
- dapl_log(DAPL_DBG_TYPE_ERR,\r
- " CONN_RTU: QPS_RTS ERR %s (%d,%d,%x,%x,%x) -> %s %x\n",\r
- strerror(errno), ep_ptr->qp_handle->qp_type,\r
- ep_ptr->qp_state, ep_ptr->qp_handle->qp_num,\r
- ntohl(cm_ptr->msg.saddr.ib.qpn), \r
- ntohs(cm_ptr->msg.saddr.ib.lid),\r
- inet_ntoa(((struct sockaddr_in *)\r
- &cm_ptr->msg.daddr.so)->sin_addr),\r
- ntohs(((struct sockaddr_in *)\r
- &cm_ptr->msg.daddr.so)->sin_port));\r
- dapl_os_unlock(&ep_ptr->header.lock);\r
- goto bail;\r
- }\r
- dapl_os_unlock(&ep_ptr->header.lock);\r
- dapl_dbg_log(DAPL_DBG_TYPE_EP, " connect_rtu: send RTU\n");\r
-\r
- /* complete handshake after final QP state change, Just ver+op */\r
- cm_ptr->state = DCM_CONNECTED;\r
- cm_ptr->msg.op = ntohs(DCM_RTU);\r
- if (send(cm_ptr->socket, (char *)&cm_ptr->msg, 4, 0) == -1) {\r
- dapl_log(DAPL_DBG_TYPE_ERR,\r
- " CONN_RTU: write error = %s\n", strerror(errno));\r
- goto bail;\r
- }\r
- /* post the event with private data */\r
- event = IB_CME_CONNECTED;\r
- dapl_dbg_log(DAPL_DBG_TYPE_EP, " ACTIVE: connected!\n");\r
-\r
-#ifdef DAT_EXTENSIONS\r
-ud_bail:\r
- if (cm_ptr->msg.saddr.ib.qp_type == IBV_QPT_UD) {\r
- DAT_IB_EXTENSION_EVENT_DATA xevent;\r
- ib_pd_handle_t pd_handle = \r
- ((DAPL_PZ *)ep_ptr->param.pz_handle)->pd_handle;\r
-\r
- if (event == IB_CME_CONNECTED) {\r
- cm_ptr->ah = dapls_create_ah(cm_ptr->hca, pd_handle,\r
- ep_ptr->qp_handle,\r
- cm_ptr->msg.saddr.ib.lid, \r
- NULL);\r
- if (cm_ptr->ah) {\r
- /* post UD extended EVENT */\r
- xevent.status = 0;\r
- xevent.type = DAT_IB_UD_REMOTE_AH;\r
- xevent.remote_ah.ah = cm_ptr->ah;\r
- xevent.remote_ah.qpn = ntohl(cm_ptr->msg.saddr.ib.qpn);\r
- dapl_os_memcpy(&xevent.remote_ah.ia_addr,\r
- &ep_ptr->remote_ia_address,\r
- sizeof(union dcm_addr));\r
- event = DAT_IB_UD_CONNECTION_EVENT_ESTABLISHED;\r
-\r
- dapl_log(DAPL_DBG_TYPE_CM, \r
- " CONN_RTU: UD AH %p for lid 0x%x"\r
- " qpn 0x%x\n", \r
- cm_ptr->ah, \r
- ntohs(cm_ptr->msg.saddr.ib.lid),\r
- ntohl(cm_ptr->msg.saddr.ib.qpn));\r
- \r
- } else \r
- event = DAT_IB_UD_CONNECTION_ERROR_EVENT;\r
- \r
- } else if (event == IB_CME_LOCAL_FAILURE) {\r
- event = DAT_IB_UD_CONNECTION_ERROR_EVENT;\r
- } else \r
- event = DAT_IB_UD_CONNECTION_REJECT_EVENT;\r
- \r
- dapls_evd_post_connection_event_ext(\r
- (DAPL_EVD *) ep_ptr->param.connect_evd_handle,\r
- event,\r
- (DAT_EP_HANDLE) ep_ptr,\r
- (DAT_COUNT) exp,\r
- (DAT_PVOID *) cm_ptr->msg.p_data,\r
- (DAT_PVOID *) &xevent);\r
-\r
- /* done with socket, don't destroy cm_ptr, need pdata */\r
- closesocket(cm_ptr->socket);\r
- cm_ptr->socket = DAPL_INVALID_SOCKET;\r
- cm_ptr->state = DCM_RELEASED;\r
- } else\r
-#endif\r
- {\r
- ep_ptr->cm_handle = cm_ptr; /* only RC, multi CR's on UD */\r
- dapl_evd_connection_callback(cm_ptr, event, cm_ptr->msg.p_data, ep_ptr);\r
- }\r
- return;\r
-\r
-bail:\r
-\r
-#ifdef DAT_EXTENSIONS\r
- if (cm_ptr->msg.saddr.ib.qp_type == IBV_QPT_UD) \r
- goto ud_bail;\r
-#endif\r
- /* close socket, and post error event */\r
- dapls_modify_qp_state(ep_ptr->qp_handle, IBV_QPS_ERR, 0, 0, 0);\r
- closesocket(cm_ptr->socket);\r
- cm_ptr->socket = DAPL_INVALID_SOCKET;\r
- dapl_evd_connection_callback(NULL, event, cm_ptr->msg.p_data, ep_ptr);\r
-}\r
-\r
-/*\r
- * PASSIVE: Create socket, listen, accept, exchange QP information \r
- */\r
-DAT_RETURN\r
-dapli_socket_listen(DAPL_IA * ia_ptr, DAT_CONN_QUAL serviceID, DAPL_SP * sp_ptr)\r
-{\r
- struct sockaddr_in addr;\r
- ib_cm_srvc_handle_t cm_ptr = NULL;\r
- DAT_RETURN dat_status = DAT_SUCCESS;\r
-\r
- dapl_dbg_log(DAPL_DBG_TYPE_EP,\r
- " listen(ia_ptr %p ServiceID %d sp_ptr %p)\n",\r
- ia_ptr, serviceID, sp_ptr);\r
-\r
- cm_ptr = dapls_ib_cm_create(NULL);\r
- if (cm_ptr == NULL)\r
- return DAT_INSUFFICIENT_RESOURCES;\r
-\r
- cm_ptr->sp = sp_ptr;\r
- cm_ptr->hca = ia_ptr->hca_ptr;\r
-\r
- /* bind, listen, set sockopt, accept, exchange data */\r
- if ((cm_ptr->socket =\r
- socket(AF_INET, SOCK_STREAM, IPPROTO_TCP)) == DAPL_INVALID_SOCKET) {\r
- dapl_log(DAPL_DBG_TYPE_ERR, " ERR: listen socket create: %s\n",\r
- strerror(errno));\r
- dat_status = DAT_INSUFFICIENT_RESOURCES;\r
- goto bail;\r
- }\r
-\r
- addr.sin_port = htons(serviceID + 1000);\r
- addr.sin_family = AF_INET;\r
- addr.sin_addr = ((struct sockaddr_in *) &ia_ptr->hca_ptr->hca_address)->sin_addr;\r
-\r
- if ((bind(cm_ptr->socket, (struct sockaddr *)&addr, sizeof(addr)) < 0)\r
- || (listen(cm_ptr->socket, 128) < 0)) {\r
- dapl_dbg_log(DAPL_DBG_TYPE_CM,\r
- " listen: ERROR %s on conn_qual 0x%x\n",\r
- strerror(errno), serviceID + 1000);\r
- if (dapl_socket_errno() == EADDRINUSE)\r
- dat_status = DAT_CONN_QUAL_IN_USE;\r
- else\r
- dat_status = DAT_CONN_QUAL_UNAVAILABLE;\r
- goto bail;\r
- }\r
-\r
- /* set cm_handle for this service point, save listen socket */\r
- sp_ptr->cm_srvc_handle = cm_ptr;\r
- dapl_os_memcpy(&cm_ptr->addr, &addr, sizeof(addr)); \r
-\r
- /* queue up listen socket to process inbound CR's */\r
- cm_ptr->state = DCM_LISTEN;\r
- dapli_cm_queue(cm_ptr);\r
-\r
- dapl_dbg_log(DAPL_DBG_TYPE_CM,\r
- " listen: qual 0x%x cr %p s_fd %d\n",\r
- ntohs(serviceID + 1000), cm_ptr, cm_ptr->socket);\r
-\r
- return dat_status;\r
-bail:\r
- dapl_dbg_log(DAPL_DBG_TYPE_CM,\r
- " listen: ERROR on conn_qual 0x%x\n", serviceID + 1000);\r
- dapls_ib_cm_free(cm_ptr, cm_ptr->ep);\r
- return dat_status;\r
-}\r
-\r
-/*\r
- * PASSIVE: accept socket \r
- */\r
-static void dapli_socket_accept(ib_cm_srvc_handle_t cm_ptr)\r
-{\r
- dp_ib_cm_handle_t acm_ptr;\r
- int ret, len, opt = 1;\r
- socklen_t sl;\r
-\r
- /* \r
- * Accept all CR's on this port to avoid half-connection (SYN_RCV)\r
- * stalls with many to one connection storms\r
- */\r
- do {\r
- /* Allocate accept CM and initialize */\r
- if ((acm_ptr = dapls_ib_cm_create(NULL)) == NULL)\r
- return;\r
-\r
- acm_ptr->sp = cm_ptr->sp;\r
- acm_ptr->hca = cm_ptr->hca;\r
-\r
- len = sizeof(union dcm_addr);\r
- acm_ptr->socket = accept(cm_ptr->socket,\r
- (struct sockaddr *)\r
- &acm_ptr->msg.daddr.so,\r
- (socklen_t *) &len);\r
- if (acm_ptr->socket == DAPL_INVALID_SOCKET) {\r
- dapl_log(DAPL_DBG_TYPE_ERR,\r
- " ACCEPT: ERR %s on FD %d l_cr %p\n",\r
- strerror(errno), cm_ptr->socket, cm_ptr);\r
- dapls_ib_cm_free(acm_ptr, acm_ptr->ep);\r
- return;\r
- }\r
- dapl_dbg_log(DAPL_DBG_TYPE_CM, " accepting from %s %x\n",\r
- inet_ntoa(((struct sockaddr_in *)\r
- &acm_ptr->msg.daddr.so)->sin_addr),\r
- ntohs(((struct sockaddr_in *)\r
- &acm_ptr->msg.daddr.so)->sin_port));\r
-\r
- /* no delay for small packets */\r
- ret = setsockopt(acm_ptr->socket, IPPROTO_TCP, TCP_NODELAY,\r
- (char *)&opt, sizeof(opt));\r
- if (ret)\r
- dapl_log(DAPL_DBG_TYPE_ERR,\r
- " ACCEPT: NODELAY setsockopt: %s\n",\r
- strerror(errno));\r
- \r
- /* get local address information from socket */\r
- sl = sizeof(acm_ptr->addr);\r
- getsockname(acm_ptr->socket, (struct sockaddr *)&acm_ptr->addr, &sl);\r
- acm_ptr->state = DCM_ACCEPTING;\r
- dapli_cm_queue(acm_ptr);\r
- \r
- } while (dapl_poll(cm_ptr->socket, DAPL_FD_READ) == DAPL_FD_READ);\r
-}\r
-\r
-/*\r
- * PASSIVE: receive peer QP information, private data, post cr_event \r
- */\r
-static void dapli_socket_accept_data(ib_cm_srvc_handle_t acm_ptr)\r
-{\r
- int len, exp = sizeof(ib_cm_msg_t) - DCM_MAX_PDATA_SIZE;\r
- void *p_data = NULL;\r
-\r
- dapl_dbg_log(DAPL_DBG_TYPE_EP, " socket accepted, read QP data\n");\r
-\r
- /* read in DST QP info, IA address. check for private data */\r
- len = recv(acm_ptr->socket, (char *)&acm_ptr->msg, exp, 0);\r
- if (len != exp || ntohs(acm_ptr->msg.ver) != DCM_VER) {\r
- dapl_log(DAPL_DBG_TYPE_ERR,\r
- " ACCEPT read: ERR %s, rcnt=%d, ver=%d\n",\r
- strerror(errno), len, ntohs(acm_ptr->msg.ver));\r
- goto bail;\r
- }\r
-\r
- /* keep the QP, address info in network order */\r
-\r
- /* validate private data size before reading */\r
- exp = ntohs(acm_ptr->msg.p_size);\r
- if (exp > DCM_MAX_PDATA_SIZE) {\r
- dapl_log(DAPL_DBG_TYPE_ERR,\r
- " accept read: psize (%d) wrong\n",\r
- acm_ptr->msg.p_size);\r
- goto bail;\r
- }\r
-\r
- /* read private data into cm_handle if any present */\r
- if (exp) {\r
- len = recv(acm_ptr->socket, acm_ptr->msg.p_data, exp, 0);\r
- if (len != exp) {\r
- dapl_log(DAPL_DBG_TYPE_ERR,\r
- " accept read pdata: ERR %s, rcnt=%d\n",\r
- strerror(errno), len);\r
- goto bail;\r
- }\r
- p_data = acm_ptr->msg.p_data;\r
- }\r
-\r
- acm_ptr->state = DCM_ACCEPTING_DATA;\r
-\r
- dapl_dbg_log(DAPL_DBG_TYPE_CM,\r
- " ACCEPT: DST %s %x lid=0x%x, qpn=0x%x, psz=%d\n",\r
- inet_ntoa(((struct sockaddr_in *)\r
- &acm_ptr->msg.daddr.so)->sin_addr), \r
- ntohs(((struct sockaddr_in *)\r
- &acm_ptr->msg.daddr.so)->sin_port),\r
- ntohs(acm_ptr->msg.saddr.ib.lid), \r
- ntohl(acm_ptr->msg.saddr.ib.qpn), exp);\r
-\r
-#ifdef DAT_EXTENSIONS\r
- if (acm_ptr->msg.saddr.ib.qp_type == IBV_QPT_UD) {\r
- DAT_IB_EXTENSION_EVENT_DATA xevent;\r
-\r
- /* post EVENT, modify_qp created ah */\r
- xevent.status = 0;\r
- xevent.type = DAT_IB_UD_CONNECT_REQUEST;\r
-\r
- dapls_evd_post_cr_event_ext(acm_ptr->sp,\r
- DAT_IB_UD_CONNECTION_REQUEST_EVENT,\r
- acm_ptr,\r
- (DAT_COUNT) exp,\r
- (DAT_PVOID *) acm_ptr->msg.p_data,\r
- (DAT_PVOID *) &xevent);\r
- } else\r
-#endif\r
- /* trigger CR event and return SUCCESS */\r
- dapls_cr_callback(acm_ptr,\r
- IB_CME_CONNECTION_REQUEST_PENDING,\r
- p_data, acm_ptr->sp);\r
- return;\r
-bail:\r
- /* close socket, free cm structure, active will see close as rej */\r
- dapls_ib_cm_free(acm_ptr, acm_ptr->ep);\r
- return;\r
-}\r
-\r
-/*\r
- * PASSIVE: consumer accept, send local QP information, private data, \r
- * queue on work thread to receive RTU information to avoid blocking\r
- * user thread. \r
- */\r
-static DAT_RETURN\r
-dapli_socket_accept_usr(DAPL_EP * ep_ptr,\r
- DAPL_CR * cr_ptr, DAT_COUNT p_size, DAT_PVOID p_data)\r
-{\r
- DAPL_IA *ia_ptr = ep_ptr->header.owner_ia;\r
- dp_ib_cm_handle_t cm_ptr = cr_ptr->ib_cm_handle;\r
- ib_cm_msg_t local;\r
- struct iovec iov[2];\r
- int len, exp = sizeof(ib_cm_msg_t) - DCM_MAX_PDATA_SIZE;\r
- DAT_RETURN ret = DAT_INTERNAL_ERROR;\r
- socklen_t sl;\r
-\r
- if (p_size > DCM_MAX_PDATA_SIZE) {\r
- dapl_log(DAPL_DBG_TYPE_ERR,\r
- " accept_usr: psize(%d) too large\n", p_size);\r
- return DAT_LENGTH_ERROR;\r
- }\r
-\r
- /* must have a accepted socket */\r
- if (cm_ptr->socket == DAPL_INVALID_SOCKET) {\r
- dapl_log(DAPL_DBG_TYPE_ERR,\r
- " accept_usr: cm socket invalid\n");\r
- goto bail;\r
- }\r
-\r
- dapl_dbg_log(DAPL_DBG_TYPE_CM,\r
- " ACCEPT_USR: remote lid=0x%x"\r
- " qpn=0x%x qp_type %d, psize=%d\n",\r
- ntohs(cm_ptr->msg.saddr.ib.lid),\r
- ntohl(cm_ptr->msg.saddr.ib.qpn), \r
- cm_ptr->msg.saddr.ib.qp_type, \r
- ntohs(cm_ptr->msg.p_size));\r
-\r
-#ifdef DAT_EXTENSIONS\r
- if (cm_ptr->msg.saddr.ib.qp_type == IBV_QPT_UD &&\r
- ep_ptr->qp_handle->qp_type != IBV_QPT_UD) {\r
- dapl_log(DAPL_DBG_TYPE_ERR,\r
- " ACCEPT_USR: ERR remote QP is UD,"\r
- ", but local QP is not\n");\r
- ret = (DAT_INVALID_HANDLE | DAT_INVALID_HANDLE_EP);\r
- goto bail;\r
- }\r
-#endif\r
-\r
- /* modify QP to RTR and then to RTS with remote info already read */\r
- dapl_os_lock(&ep_ptr->header.lock);\r
- if (dapls_modify_qp_state(ep_ptr->qp_handle,\r
- IBV_QPS_RTR, \r
- cm_ptr->msg.saddr.ib.qpn,\r
- cm_ptr->msg.saddr.ib.lid,\r
- NULL) != DAT_SUCCESS) {\r
- dapl_log(DAPL_DBG_TYPE_ERR,\r
- " ACCEPT_USR: QPS_RTR ERR %s -> %s\n",\r
- strerror(errno), \r
- inet_ntoa(((struct sockaddr_in *)\r
- &cm_ptr->msg.daddr.so)->sin_addr));\r
- dapl_os_unlock(&ep_ptr->header.lock);\r
- goto bail;\r
- }\r
- if (dapls_modify_qp_state(ep_ptr->qp_handle,\r
- IBV_QPS_RTS, \r
- cm_ptr->msg.saddr.ib.qpn,\r
- cm_ptr->msg.saddr.ib.lid,\r
- NULL) != DAT_SUCCESS) {\r
- dapl_log(DAPL_DBG_TYPE_ERR,\r
- " ACCEPT_USR: QPS_RTS ERR %s -> %s\n",\r
- strerror(errno), \r
- inet_ntoa(((struct sockaddr_in *)\r
- &cm_ptr->msg.daddr.so)->sin_addr));\r
- dapl_os_unlock(&ep_ptr->header.lock);\r
- goto bail;\r
- }\r
- dapl_os_unlock(&ep_ptr->header.lock);\r
-\r
- /* save remote address information */\r
- dapl_os_memcpy(&ep_ptr->remote_ia_address,\r
- &cm_ptr->msg.daddr.so,\r
- sizeof(union dcm_addr));\r
-\r
- /* send our QP info, IA address, pdata. Don't overwrite dst data */\r
- local.ver = htons(DCM_VER);\r
- local.op = htons(DCM_REP);\r
- local.saddr.ib.qpn = htonl(ep_ptr->qp_handle->qp_num);\r
- local.saddr.ib.qp_type = ep_ptr->qp_handle->qp_type;\r
- local.saddr.ib.lid = ia_ptr->hca_ptr->ib_trans.lid;\r
- dapl_os_memcpy(&local.saddr.ib.gid[0], \r
- &ia_ptr->hca_ptr->ib_trans.gid, 16);\r
- \r
- /* Get local address information from socket */\r
- sl = sizeof(local.daddr.so);\r
- getsockname(cm_ptr->socket, (struct sockaddr *)&local.daddr.so, &sl);\r
-\r
-#ifdef DAPL_DBG\r
- /* DBG: Active PID [0], PASSIVE PID [2] */\r
- *(uint16_t*)&cm_ptr->msg.resv[2] = htons((uint16_t)dapl_os_getpid()); \r
- dapl_os_memcpy(local.resv, cm_ptr->msg.resv, 4); \r
-#endif\r
-\r
- cm_ptr->ep = ep_ptr;\r
- cm_ptr->hca = ia_ptr->hca_ptr;\r
- cm_ptr->state = DCM_ACCEPTED;\r
-\r
- local.p_size = htons(p_size);\r
- iov[0].iov_base = (void *)&local;\r
- iov[0].iov_len = exp;\r
- \r
- if (p_size) {\r
- iov[1].iov_base = p_data;\r
- iov[1].iov_len = p_size;\r
- len = writev(cm_ptr->socket, iov, 2);\r
- } else \r
- len = writev(cm_ptr->socket, iov, 1);\r
- \r
- if (len != (p_size + exp)) {\r
- dapl_log(DAPL_DBG_TYPE_ERR,\r
- " ACCEPT_USR: ERR %s, wcnt=%d -> %s\n",\r
- strerror(errno), len, \r
- inet_ntoa(((struct sockaddr_in *)\r
- &cm_ptr->msg.daddr.so)->sin_addr));\r
- cm_ptr->ep = NULL;\r
- goto bail;\r
- }\r
-\r
- dapl_dbg_log(DAPL_DBG_TYPE_CM,\r
- " ACCEPT_USR: local lid=0x%x qpn=0x%x psz=%d\n",\r
- ntohs(local.saddr.ib.lid),\r
- ntohl(local.saddr.ib.qpn), ntohs(local.p_size));\r
- dapl_dbg_log(DAPL_DBG_TYPE_CM,\r
- " ACCEPT_USR: SRC GID subnet %016llx id %016llx\n",\r
- (unsigned long long)\r
- htonll(*(uint64_t*)&local.saddr.ib.gid[0]),\r
- (unsigned long long)\r
- htonll(*(uint64_t*)&local.saddr.ib.gid[8]));\r
-\r
- dapl_dbg_log(DAPL_DBG_TYPE_EP, " PASSIVE: accepted!\n");\r
- return DAT_SUCCESS;\r
-bail:\r
- dapls_ib_cm_free(cm_ptr, NULL);\r
- return ret;\r
-}\r
-\r
-/*\r
- * PASSIVE: read RTU from active peer, post CONN event\r
- */\r
-static void dapli_socket_accept_rtu(dp_ib_cm_handle_t cm_ptr)\r
-{\r
- int len;\r
- ib_cm_events_t event = IB_CME_CONNECTED;\r
-\r
- /* complete handshake after final QP state change, VER and OP */\r
- len = recv(cm_ptr->socket, (char *)&cm_ptr->msg, 4, 0);\r
- if (len != 4 || ntohs(cm_ptr->msg.op) != DCM_RTU) {\r
- dapl_log(DAPL_DBG_TYPE_ERR,\r
- " ACCEPT_RTU: rcv ERR, rcnt=%d op=%x\n",\r
- len, ntohs(cm_ptr->msg.op),\r
- inet_ntoa(((struct sockaddr_in *)\r
- &cm_ptr->msg.daddr.so)->sin_addr));\r
- event = IB_CME_DESTINATION_REJECT;\r
- goto bail;\r
- }\r
-\r
- /* save state and reference to EP, queue for disc event */\r
- cm_ptr->state = DCM_CONNECTED;\r
-\r
- /* final data exchange if remote QP state is good to go */\r
- dapl_dbg_log(DAPL_DBG_TYPE_EP, " PASSIVE: connected!\n");\r
-\r
-#ifdef DAT_EXTENSIONS\r
-ud_bail:\r
- if (cm_ptr->msg.saddr.ib.qp_type == IBV_QPT_UD) {\r
- DAT_IB_EXTENSION_EVENT_DATA xevent;\r
-\r
- ib_pd_handle_t pd_handle = \r
- ((DAPL_PZ *)cm_ptr->ep->param.pz_handle)->pd_handle;\r
- \r
- if (event == IB_CME_CONNECTED) {\r
- cm_ptr->ah = dapls_create_ah(cm_ptr->hca, pd_handle,\r
- cm_ptr->ep->qp_handle,\r
- cm_ptr->msg.saddr.ib.lid, \r
- NULL);\r
- if (cm_ptr->ah) { \r
- /* post EVENT, modify_qp created ah */\r
- xevent.status = 0;\r
- xevent.type = DAT_IB_UD_PASSIVE_REMOTE_AH;\r
- xevent.remote_ah.ah = cm_ptr->ah;\r
- xevent.remote_ah.qpn = ntohl(cm_ptr->msg.saddr.ib.qpn);\r
- dapl_os_memcpy(&xevent.remote_ah.ia_addr,\r
- &cm_ptr->msg.daddr.so,\r
- sizeof(union dcm_addr));\r
- event = DAT_IB_UD_CONNECTION_EVENT_ESTABLISHED;\r
- } else \r
- event = DAT_IB_UD_CONNECTION_ERROR_EVENT;\r
- } else \r
- event = DAT_IB_UD_CONNECTION_ERROR_EVENT;\r
-\r
- dapl_log(DAPL_DBG_TYPE_CM, \r
- " CONN_RTU: UD AH %p for lid 0x%x qpn 0x%x\n", \r
- cm_ptr->ah, ntohs(cm_ptr->msg.saddr.ib.lid),\r
- ntohl(cm_ptr->msg.saddr.ib.qpn));\r
-\r
- dapls_evd_post_connection_event_ext(\r
- (DAPL_EVD *) \r
- cm_ptr->ep->param.connect_evd_handle,\r
- event,\r
- (DAT_EP_HANDLE) cm_ptr->ep,\r
- (DAT_COUNT) ntohs(cm_ptr->msg.p_size),\r
- (DAT_PVOID *) cm_ptr->msg.p_data,\r
- (DAT_PVOID *) &xevent);\r
-\r
- /* done with socket, don't destroy cm_ptr, need pdata */\r
- closesocket(cm_ptr->socket);\r
- cm_ptr->socket = DAPL_INVALID_SOCKET;\r
- cm_ptr->state = DCM_RELEASED;\r
- } else \r
-#endif\r
- {\r
- cm_ptr->ep->cm_handle = cm_ptr; /* only RC, multi CR's on UD */\r
- dapls_cr_callback(cm_ptr, event, NULL, cm_ptr->sp);\r
- }\r
- return;\r
- \r
-bail:\r
-#ifdef DAT_EXTENSIONS\r
- if (cm_ptr->msg.saddr.ib.qp_type == IBV_QPT_UD) \r
- goto ud_bail;\r
-#endif\r
- dapls_modify_qp_state(cm_ptr->ep->qp_handle, IBV_QPS_ERR, 0, 0, 0);\r
- dapls_ib_cm_free(cm_ptr, cm_ptr->ep);\r
- dapls_cr_callback(cm_ptr, event, NULL, cm_ptr->sp);\r
-}\r
-\r
-/*\r
- * dapls_ib_connect\r
- *\r
- * Initiate a connection with the passive listener on another node\r
- *\r
- * Input:\r
- * ep_handle,\r
- * remote_ia_address,\r
- * remote_conn_qual,\r
- * prd_size size of private data and structure\r
- * prd_prt pointer to private data structure\r
- *\r
- * Output:\r
- * none\r
- *\r
- * Returns:\r
- * DAT_SUCCESS\r
- * DAT_INSUFFICIENT_RESOURCES\r
- * DAT_INVALID_PARAMETER\r
- *\r
- */\r
-DAT_RETURN\r
-dapls_ib_connect(IN DAT_EP_HANDLE ep_handle,\r
- IN DAT_IA_ADDRESS_PTR remote_ia_address,\r
- IN DAT_CONN_QUAL remote_conn_qual,\r
- IN DAT_COUNT private_data_size, IN void *private_data)\r
-{\r
- DAPL_EP *ep_ptr;\r
- ib_qp_handle_t qp_ptr;\r
-\r
- dapl_dbg_log(DAPL_DBG_TYPE_EP,\r
- " connect(ep_handle %p ....)\n", ep_handle);\r
-\r
- ep_ptr = (DAPL_EP *) ep_handle;\r
- qp_ptr = ep_ptr->qp_handle;\r
-\r
- return (dapli_socket_connect(ep_ptr, remote_ia_address,\r
- remote_conn_qual,\r
- private_data_size, private_data));\r
-}\r
-\r
-/*\r
- * dapls_ib_disconnect\r
- *\r
- * Disconnect an EP\r
- *\r
- * Input:\r
- * ep_handle,\r
- * disconnect_flags\r
- *\r
- * Output:\r
- * none\r
- *\r
- * Returns:\r
- * DAT_SUCCESS\r
- */\r
-DAT_RETURN\r
-dapls_ib_disconnect(IN DAPL_EP * ep_ptr, IN DAT_CLOSE_FLAGS close_flags)\r
-{\r
- dapl_dbg_log(DAPL_DBG_TYPE_EP,\r
- "dapls_ib_disconnect(ep_handle %p ....)\n", ep_ptr);\r
-\r
- /* Transition to error state to flush queue */\r
- dapls_modify_qp_state(ep_ptr->qp_handle, IBV_QPS_ERR, 0, 0, 0);\r
- \r
- if (ep_ptr->cm_handle == NULL ||\r
- ep_ptr->param.ep_state == DAT_EP_STATE_DISCONNECTED)\r
- return DAT_SUCCESS;\r
- else\r
- return (dapli_socket_disconnect(ep_ptr->cm_handle));\r
-}\r
-\r
-/*\r
- * dapls_ib_disconnect_clean\r
- *\r
- * Clean up outstanding connection data. This routine is invoked\r
- * after the final disconnect callback has occurred. Only on the\r
- * ACTIVE side of a connection. It is also called if dat_ep_connect\r
- * times out using the consumer supplied timeout value.\r
- *\r
- * Input:\r
- * ep_ptr DAPL_EP\r
- * active Indicates active side of connection\r
- *\r
- * Output:\r
- * none\r
- *\r
- * Returns:\r
- * void\r
- *\r
- */\r
-void\r
-dapls_ib_disconnect_clean(IN DAPL_EP * ep_ptr,\r
- IN DAT_BOOLEAN active,\r
- IN const ib_cm_events_t ib_cm_event)\r
-{\r
- /* NOTE: SCM will only initialize cm_handle with RC type\r
- * \r
- * For UD there can many in-flight CR's so you \r
- * cannot cleanup timed out CR's with EP reference \r
- * alone since they share the same EP. The common\r
- * code that handles connection timeout logic needs \r
- * updated for UD support.\r
- */\r
- if (ep_ptr->cm_handle)\r
- dapls_ib_cm_free(ep_ptr->cm_handle, ep_ptr);\r
-\r
- return;\r
-}\r
-\r
-/*\r
- * dapl_ib_setup_conn_listener\r
- *\r
- * Have the CM set up a connection listener.\r
- *\r
- * Input:\r
- * ibm_hca_handle HCA handle\r
- * qp_handle QP handle\r
- *\r
- * Output:\r
- * none\r
- *\r
- * Returns:\r
- * DAT_SUCCESS\r
- * DAT_INSUFFICIENT_RESOURCES\r
- * DAT_INTERNAL_ERROR\r
- * DAT_CONN_QUAL_UNAVAILBLE\r
- * DAT_CONN_QUAL_IN_USE\r
- *\r
- */\r
-DAT_RETURN\r
-dapls_ib_setup_conn_listener(IN DAPL_IA * ia_ptr,\r
- IN DAT_UINT64 ServiceID, IN DAPL_SP * sp_ptr)\r
-{\r
- return (dapli_socket_listen(ia_ptr, ServiceID, sp_ptr));\r
-}\r
-\r
-/*\r
- * dapl_ib_remove_conn_listener\r
- *\r
- * Have the CM remove a connection listener.\r
- *\r
- * Input:\r
- * ia_handle IA handle\r
- * ServiceID IB Channel Service ID\r
- *\r
- * Output:\r
- * none\r
- *\r
- * Returns:\r
- * DAT_SUCCESS\r
- * DAT_INVALID_STATE\r
- *\r
- */\r
-DAT_RETURN\r
-dapls_ib_remove_conn_listener(IN DAPL_IA * ia_ptr, IN DAPL_SP * sp_ptr)\r
-{\r
- ib_cm_srvc_handle_t cm_ptr = sp_ptr->cm_srvc_handle;\r
-\r
- dapl_dbg_log(DAPL_DBG_TYPE_EP,\r
- "dapls_ib_remove_conn_listener(ia_ptr %p sp_ptr %p cm_ptr %p)\n",\r
- ia_ptr, sp_ptr, cm_ptr);\r
-\r
- /* close accepted socket, free cm_srvc_handle and return */\r
- if (cm_ptr != NULL) {\r
- /* cr_thread will free */\r
- dapl_os_lock(&cm_ptr->lock);\r
- cm_ptr->state = DCM_DESTROY;\r
- sp_ptr->cm_srvc_handle = NULL;\r
- send(cm_ptr->hca->ib_trans.scm[1], "w", sizeof "w", 0);\r
- dapl_os_unlock(&cm_ptr->lock);\r
- }\r
- return DAT_SUCCESS;\r
-}\r
-\r
-/*\r
- * dapls_ib_accept_connection\r
- *\r
- * Perform necessary steps to accept a connection\r
- *\r
- * Input:\r
- * cr_handle\r
- * ep_handle\r
- * private_data_size\r
- * private_data\r
- *\r
- * Output:\r
- * none\r
- *\r
- * Returns:\r
- * DAT_SUCCESS\r
- * DAT_INSUFFICIENT_RESOURCES\r
- * DAT_INTERNAL_ERROR\r
- *\r
- */\r
-DAT_RETURN\r
-dapls_ib_accept_connection(IN DAT_CR_HANDLE cr_handle,\r
- IN DAT_EP_HANDLE ep_handle,\r
- IN DAT_COUNT p_size, IN const DAT_PVOID p_data)\r
-{\r
- DAPL_CR *cr_ptr;\r
- DAPL_EP *ep_ptr;\r
-\r
- dapl_dbg_log(DAPL_DBG_TYPE_EP,\r
- "dapls_ib_accept_connection(cr %p ep %p prd %p,%d)\n",\r
- cr_handle, ep_handle, p_data, p_size);\r
-\r
- cr_ptr = (DAPL_CR *) cr_handle;\r
- ep_ptr = (DAPL_EP *) ep_handle;\r
-\r
- /* allocate and attach a QP if necessary */\r
- if (ep_ptr->qp_state == DAPL_QP_STATE_UNATTACHED) {\r
- DAT_RETURN status;\r
- status = dapls_ib_qp_alloc(ep_ptr->header.owner_ia,\r
- ep_ptr, ep_ptr);\r
- if (status != DAT_SUCCESS)\r
- return status;\r
- }\r
- return (dapli_socket_accept_usr(ep_ptr, cr_ptr, p_size, p_data));\r
-}\r
-\r
-/*\r
- * dapls_ib_reject_connection\r
- *\r
- * Reject a connection\r
- *\r
- * Input:\r
- * cr_handle\r
- *\r
- * Output:\r
- * none\r
- *\r
- * Returns:\r
- * DAT_SUCCESS\r
- * DAT_INTERNAL_ERROR\r
- *\r
- */\r
-DAT_RETURN\r
-dapls_ib_reject_connection(IN dp_ib_cm_handle_t cm_ptr,\r
- IN int reason,\r
- IN DAT_COUNT psize, IN const DAT_PVOID pdata)\r
-{\r
- struct iovec iov[2];\r
-\r
- dapl_dbg_log(DAPL_DBG_TYPE_EP,\r
- " reject(cm %p reason %x, pdata %p, psize %d)\n",\r
- cm_ptr, reason, pdata, psize);\r
-\r
- if (psize > DCM_MAX_PDATA_SIZE)\r
- return DAT_LENGTH_ERROR;\r
-\r
- dapl_os_lock(&cm_ptr->lock);\r
-\r
- /* write reject data to indicate reject */\r
- cm_ptr->msg.op = htons(DCM_REJ_USER);\r
- cm_ptr->msg.p_size = htons(psize);\r
- \r
- iov[0].iov_base = (void *)&cm_ptr->msg;\r
- iov[0].iov_len = sizeof(ib_cm_msg_t) - DCM_MAX_PDATA_SIZE;\r
- if (psize) {\r
- iov[1].iov_base = pdata;\r
- iov[1].iov_len = psize;\r
- writev(cm_ptr->socket, iov, 2);\r
- } else {\r
- writev(cm_ptr->socket, iov, 1);\r
- }\r
-\r
- /* cr_thread will destroy CR */\r
- cm_ptr->state = DCM_DESTROY;\r
- send(cm_ptr->hca->ib_trans.scm[1], "w", sizeof "w", 0);\r
- dapl_os_unlock(&cm_ptr->lock);\r
- return DAT_SUCCESS;\r
-}\r
-\r
-/*\r
- * dapls_ib_cm_remote_addr\r
- *\r
- * Obtain the remote IP address given a connection\r
- *\r
- * Input:\r
- * cr_handle\r
- *\r
- * Output:\r
- * remote_ia_address: where to place the remote address\r
- *\r
- * Returns:\r
- * DAT_SUCCESS\r
- * DAT_INVALID_HANDLE\r
- *\r
- */\r
-DAT_RETURN\r
-dapls_ib_cm_remote_addr(IN DAT_HANDLE dat_handle,\r
- OUT DAT_SOCK_ADDR6 * remote_ia_address)\r
-{\r
- DAPL_HEADER *header;\r
- dp_ib_cm_handle_t ib_cm_handle;\r
-\r
- dapl_dbg_log(DAPL_DBG_TYPE_EP,\r
- "dapls_ib_cm_remote_addr(dat_handle %p, ....)\n",\r
- dat_handle);\r
-\r
- header = (DAPL_HEADER *) dat_handle;\r
-\r
- if (header->magic == DAPL_MAGIC_EP)\r
- ib_cm_handle = ((DAPL_EP *) dat_handle)->cm_handle;\r
- else if (header->magic == DAPL_MAGIC_CR)\r
- ib_cm_handle = ((DAPL_CR *) dat_handle)->ib_cm_handle;\r
- else\r
- return DAT_INVALID_HANDLE;\r
-\r
- dapl_os_memcpy(remote_ia_address,\r
- &ib_cm_handle->msg.daddr.so, sizeof(DAT_SOCK_ADDR6));\r
-\r
- return DAT_SUCCESS;\r
-}\r
-\r
-/*\r
- * dapls_ib_private_data_size\r
- *\r
- * Return the size of private data given a connection op type\r
- *\r
- * Input:\r
- * prd_ptr private data pointer\r
- * conn_op connection operation type\r
- *\r
- * If prd_ptr is NULL, this is a query for the max size supported by\r
- * the provider, otherwise it is the actual size of the private data\r
- * contained in prd_ptr.\r
- *\r
- *\r
- * Output:\r
- * None\r
- *\r
- * Returns:\r
- * length of private data\r
- *\r
- */\r
-int dapls_ib_private_data_size(IN DAPL_PRIVATE * prd_ptr,\r
- IN DAPL_PDATA_OP conn_op, IN DAPL_HCA * hca_ptr)\r
-{\r
- int size;\r
-\r
- switch (conn_op) {\r
- case DAPL_PDATA_CONN_REQ:\r
- case DAPL_PDATA_CONN_REP:\r
- case DAPL_PDATA_CONN_REJ:\r
- case DAPL_PDATA_CONN_DREQ:\r
- case DAPL_PDATA_CONN_DREP:\r
- size = DCM_MAX_PDATA_SIZE;\r
- break;\r
- default:\r
- size = 0;\r
- } \r
- return size;\r
-}\r
-\r
-/*\r
- * Map all socket CM event codes to the DAT equivelent.\r
- */\r
-#define DAPL_IB_EVENT_CNT 10\r
-\r
-static struct ib_cm_event_map {\r
- const ib_cm_events_t ib_cm_event;\r
- DAT_EVENT_NUMBER dat_event_num;\r
-} ib_cm_event_map[DAPL_IB_EVENT_CNT] = {\r
-/* 00 */ {IB_CME_CONNECTED, \r
- DAT_CONNECTION_EVENT_ESTABLISHED},\r
-/* 01 */ {IB_CME_DISCONNECTED, \r
- DAT_CONNECTION_EVENT_DISCONNECTED},\r
-/* 02 */ {IB_CME_DISCONNECTED_ON_LINK_DOWN,\r
- DAT_CONNECTION_EVENT_DISCONNECTED},\r
-/* 03 */ {IB_CME_CONNECTION_REQUEST_PENDING, \r
- DAT_CONNECTION_REQUEST_EVENT},\r
-/* 04 */ {IB_CME_CONNECTION_REQUEST_PENDING_PRIVATE_DATA,\r
- DAT_CONNECTION_REQUEST_EVENT},\r
-/* 05 */ {IB_CME_DESTINATION_REJECT,\r
- DAT_CONNECTION_EVENT_NON_PEER_REJECTED},\r
-/* 06 */ {IB_CME_DESTINATION_REJECT_PRIVATE_DATA,\r
- DAT_CONNECTION_EVENT_PEER_REJECTED},\r
-/* 07 */ {IB_CME_DESTINATION_UNREACHABLE, \r
- DAT_CONNECTION_EVENT_UNREACHABLE},\r
-/* 08 */ {IB_CME_TOO_MANY_CONNECTION_REQUESTS,\r
- DAT_CONNECTION_EVENT_NON_PEER_REJECTED},\r
-/* 09 */ {IB_CME_LOCAL_FAILURE, \r
- DAT_CONNECTION_EVENT_BROKEN}\r
-};\r
-\r
-/*\r
- * dapls_ib_get_cm_event\r
- *\r
- * Return a DAT connection event given a provider CM event.\r
- *\r
- * Input:\r
- * dat_event_num DAT event we need an equivelent CM event for\r
- *\r
- * Output:\r
- * none\r
- *\r
- * Returns:\r
- * ib_cm_event of translated DAPL value\r
- */\r
-DAT_EVENT_NUMBER\r
-dapls_ib_get_dat_event(IN const ib_cm_events_t ib_cm_event,\r
- IN DAT_BOOLEAN active)\r
-{\r
- DAT_EVENT_NUMBER dat_event_num;\r
- int i;\r
-\r
- active = active;\r
-\r
- if (ib_cm_event > IB_CME_LOCAL_FAILURE)\r
- return (DAT_EVENT_NUMBER) 0;\r
-\r
- dat_event_num = 0;\r
- for (i = 0; i < DAPL_IB_EVENT_CNT; i++) {\r
- if (ib_cm_event == ib_cm_event_map[i].ib_cm_event) {\r
- dat_event_num = ib_cm_event_map[i].dat_event_num;\r
- break;\r
- }\r
- }\r
- dapl_dbg_log(DAPL_DBG_TYPE_CALLBACK,\r
- "dapls_ib_get_dat_event: event translate(%s) ib=0x%x dat=0x%x\n",\r
- active ? "active" : "passive", ib_cm_event, dat_event_num);\r
-\r
- return dat_event_num;\r
-}\r
-\r
-/*\r
- * dapls_ib_get_dat_event\r
- *\r
- * Return a DAT connection event given a provider CM event.\r
- * \r
- * Input:\r
- * ib_cm_event event provided to the dapl callback routine\r
- * active switch indicating active or passive connection\r
- *\r
- * Output:\r
- * none\r
- *\r
- * Returns:\r
- * DAT_EVENT_NUMBER of translated provider value\r
- */\r
-ib_cm_events_t dapls_ib_get_cm_event(IN DAT_EVENT_NUMBER dat_event_num)\r
-{\r
- ib_cm_events_t ib_cm_event;\r
- int i;\r
-\r
- ib_cm_event = 0;\r
- for (i = 0; i < DAPL_IB_EVENT_CNT; i++) {\r
- if (dat_event_num == ib_cm_event_map[i].dat_event_num) {\r
- ib_cm_event = ib_cm_event_map[i].ib_cm_event;\r
- break;\r
- }\r
- }\r
- return ib_cm_event;\r
-}\r
-\r
-/* outbound/inbound CR processing thread to avoid blocking applications */\r
-void cr_thread(void *arg)\r
-{\r
- struct dapl_hca *hca_ptr = arg;\r
- dp_ib_cm_handle_t cr, next_cr;\r
- int opt, ret;\r
- socklen_t opt_len;\r
- char rbuf[2];\r
- struct dapl_fd_set *set;\r
- enum DAPL_FD_EVENTS event;\r
-\r
- dapl_dbg_log(DAPL_DBG_TYPE_UTIL, " cr_thread: ENTER hca %p\n", hca_ptr);\r
- set = dapl_alloc_fd_set();\r
- if (!set)\r
- goto out;\r
-\r
- dapl_os_lock(&hca_ptr->ib_trans.lock);\r
- hca_ptr->ib_trans.cr_state = IB_THREAD_RUN;\r
-\r
- while (1) {\r
- dapl_fd_zero(set);\r
- dapl_fd_set(hca_ptr->ib_trans.scm[0], set, DAPL_FD_READ);\r
-\r
- if (!dapl_llist_is_empty(&hca_ptr->ib_trans.list))\r
- next_cr = dapl_llist_peek_head(&hca_ptr->ib_trans.list);\r
- else\r
- next_cr = NULL;\r
-\r
- while (next_cr) {\r
- cr = next_cr;\r
- next_cr = dapl_llist_next_entry(&hca_ptr->ib_trans.list,\r
- (DAPL_LLIST_ENTRY *) &\r
- cr->entry);\r
- dapl_os_lock(&cr->lock);\r
- if (cr->state == DCM_DESTROY\r
- || hca_ptr->ib_trans.cr_state != IB_THREAD_RUN) {\r
- dapl_os_unlock(&cr->lock);\r
- dapl_llist_remove_entry(&hca_ptr->ib_trans.list,\r
- (DAPL_LLIST_ENTRY *) &\r
- cr->entry);\r
- dapl_dbg_log(DAPL_DBG_TYPE_CM, \r
- " CR FREE: %p ep=%p st=%d sock=%d\n", \r
- cr, cr->ep, cr->state, cr->socket);\r
-\r
- if (cr->socket != DAPL_INVALID_SOCKET) {\r
- shutdown(cr->socket, SHUT_RDWR);\r
- closesocket(cr->socket);\r
- }\r
- dapl_os_free(cr, sizeof(*cr));\r
- continue;\r
- }\r
- if (cr->socket == DAPL_INVALID_SOCKET) {\r
- dapl_os_unlock(&cr->lock);\r
- continue;\r
- }\r
-\r
- event = (cr->state == DCM_CONN_PENDING) ?\r
- DAPL_FD_WRITE : DAPL_FD_READ;\r
-\r
- if (dapl_fd_set(cr->socket, set, event)) {\r
- dapl_log(DAPL_DBG_TYPE_ERR,\r
- " cr_thread: DESTROY CR st=%d fd %d"\r
- " -> %s\n", cr->state, cr->socket,\r
- inet_ntoa(((struct sockaddr_in *)\r
- &cr->msg.daddr.so)->sin_addr));\r
- dapl_os_unlock(&cr->lock);\r
- dapls_ib_cm_free(cr, cr->ep);\r
- continue;\r
- }\r
- dapl_os_unlock(&cr->lock);\r
- dapl_dbg_log(DAPL_DBG_TYPE_CM,\r
- " poll cr=%p, sck=%d\n", cr, cr->socket);\r
- dapl_os_unlock(&hca_ptr->ib_trans.lock);\r
-\r
- ret = dapl_poll(cr->socket, event);\r
-\r
- dapl_dbg_log(DAPL_DBG_TYPE_CM,\r
- " poll ret=0x%x cr->state=%d sck=%d\n",\r
- ret, cr->state, cr->socket);\r
-\r
- /* data on listen, qp exchange, and on disc req */\r
- if ((ret == DAPL_FD_READ) || \r
- (cr->state != DCM_CONN_PENDING &&\r
- ret == DAPL_FD_ERROR)) {\r
- if (cr->socket != DAPL_INVALID_SOCKET) {\r
- switch (cr->state) {\r
- case DCM_LISTEN:\r
- dapli_socket_accept(cr);\r
- break;\r
- case DCM_ACCEPTING:\r
- dapli_socket_accept_data(cr);\r
- break;\r
- case DCM_ACCEPTED:\r
- dapli_socket_accept_rtu(cr);\r
- break;\r
- case DCM_REP_PENDING:\r
- dapli_socket_connect_rtu(cr);\r
- break;\r
- case DCM_CONNECTED:\r
- dapli_socket_disconnect(cr);\r
- break;\r
- default:\r
- break;\r
- }\r
- }\r
- /* ASYNC connections, writable, readable, error; check status */\r
- } else if (ret == DAPL_FD_WRITE ||\r
- (cr->state == DCM_CONN_PENDING && \r
- ret == DAPL_FD_ERROR)) {\r
-\r
- if (ret == DAPL_FD_ERROR)\r
- dapl_log(DAPL_DBG_TYPE_ERR, " CONN_PENDING - FD_ERROR\n");\r
- \r
- opt = 0;\r
- opt_len = sizeof(opt);\r
- ret = getsockopt(cr->socket, SOL_SOCKET,\r
- SO_ERROR, (char *)&opt,\r
- &opt_len);\r
- if (!ret && !opt)\r
- dapli_socket_connected(cr, opt);\r
- else\r
- dapli_socket_connected(cr, opt ? opt : dapl_socket_errno());\r
- } \r
- dapl_os_lock(&hca_ptr->ib_trans.lock);\r
- }\r
-\r
- /* set to exit and all resources destroyed */\r
- if ((hca_ptr->ib_trans.cr_state != IB_THREAD_RUN) &&\r
- (dapl_llist_is_empty(&hca_ptr->ib_trans.list)))\r
- break;\r
-\r
- dapl_os_unlock(&hca_ptr->ib_trans.lock);\r
- dapl_select(set);\r
-\r
- /* if pipe used to wakeup, consume */\r
- while (dapl_poll(hca_ptr->ib_trans.scm[0], \r
- DAPL_FD_READ) == DAPL_FD_READ) {\r
- if (recv(hca_ptr->ib_trans.scm[0], rbuf, 2, 0) == -1)\r
- dapl_log(DAPL_DBG_TYPE_CM,\r
- " cr_thread: read pipe error = %s\n",\r
- strerror(errno));\r
- }\r
- dapl_os_lock(&hca_ptr->ib_trans.lock);\r
- \r
- /* set to exit and all resources destroyed */\r
- if ((hca_ptr->ib_trans.cr_state != IB_THREAD_RUN) &&\r
- (dapl_llist_is_empty(&hca_ptr->ib_trans.list)))\r
- break;\r
- }\r
-\r
- dapl_os_unlock(&hca_ptr->ib_trans.lock);\r
- dapl_os_free(set, sizeof(struct dapl_fd_set));\r
-out:\r
- hca_ptr->ib_trans.cr_state = IB_THREAD_EXIT;\r
- dapl_dbg_log(DAPL_DBG_TYPE_UTIL, " cr_thread(hca %p) exit\n", hca_ptr);\r
-}\r
-\r
-\r
-#ifdef DAPL_COUNTERS\r
-/* Debug aid: List all Connections in process and state */\r
-void dapls_print_cm_list(IN DAPL_IA *ia_ptr)\r
-{\r
- /* Print in process CR's for this IA, if debug type set */\r
- int i = 0;\r
- dp_ib_cm_handle_t cr, next_cr;\r
-\r
- dapl_os_lock(&ia_ptr->hca_ptr->ib_trans.lock);\r
- if (!dapl_llist_is_empty((DAPL_LLIST_HEAD*)\r
- &ia_ptr->hca_ptr->ib_trans.list))\r
- next_cr = dapl_llist_peek_head((DAPL_LLIST_HEAD*)\r
- &ia_ptr->hca_ptr->ib_trans.list);\r
- else\r
- next_cr = NULL;\r
-\r
- printf("\n DAPL IA CONNECTIONS IN PROCESS:\n");\r
- while (next_cr) {\r
- cr = next_cr;\r
- next_cr = dapl_llist_next_entry((DAPL_LLIST_HEAD*)\r
- &ia_ptr->hca_ptr->ib_trans.list,\r
- (DAPL_LLIST_ENTRY*)&cr->entry);\r
-\r
- printf( " CONN[%d]: sp %p ep %p sock %d %s %s %s %s %s %s PORT L-%x R-%x PID L-%x R-%x\n",\r
- i, cr->sp, cr->ep, cr->socket,\r
- cr->msg.saddr.ib.qp_type == IBV_QPT_RC ? "RC" : "UD",\r
- dapl_cm_state_str(cr->state), dapl_cm_op_str(ntohs(cr->msg.op)),\r
- ntohs(cr->msg.op) == DCM_REQ ? /* local address */\r
- inet_ntoa(((struct sockaddr_in *)&cr->msg.daddr.so)->sin_addr) :\r
- inet_ntoa(((struct sockaddr_in *)&cr->addr)->sin_addr),\r
- cr->sp ? "<-" : "->",\r
- ntohs(cr->msg.op) == DCM_REQ ? /* remote address */\r
- inet_ntoa(((struct sockaddr_in *)&cr->addr)->sin_addr) :\r
- inet_ntoa(((struct sockaddr_in *)&cr->msg.daddr.so)->sin_addr),\r
-\r
- ntohs(cr->msg.op) == DCM_REQ ? /* local port */\r
- ntohs(((struct sockaddr_in *)&cr->msg.daddr.so)->sin_port) :\r
- ntohs(((struct sockaddr_in *)&cr->addr)->sin_port),\r
-\r
- ntohs(cr->msg.op) == DCM_REQ ? /* remote port */\r
- ntohs(((struct sockaddr_in *)&cr->addr)->sin_port) :\r
- ntohs(((struct sockaddr_in *)&cr->msg.daddr.so)->sin_port),\r
-\r
- cr->sp ? ntohs(*(uint16_t*)&cr->msg.resv[2]) : ntohs(*(uint16_t*)&cr->msg.resv[0]),\r
- cr->sp ? ntohs(*(uint16_t*)&cr->msg.resv[0]) : ntohs(*(uint16_t*)&cr->msg.resv[2]));\r
-\r
- i++;\r
- }\r
- printf("\n");\r
- dapl_os_unlock(&ia_ptr->hca_ptr->ib_trans.lock);\r
-}\r
-#endif\r
+/*
+ * This Software is licensed under one of the following licenses:
+ *
+ * 1) under the terms of the "Common Public License 1.0" a copy of which is
+ * available from the Open Source Initiative, see
+ * http://www.opensource.org/licenses/cpl.php.
+ *
+ * 2) under the terms of the "The BSD License" a copy of which is
+ * available from the Open Source Initiative, see
+ * http://www.opensource.org/licenses/bsd-license.php.
+ *
+ * 3) under the terms of the "GNU General Public License (GPL) Version 2" a
+ * copy of which is available from the Open Source Initiative, see
+ * http://www.opensource.org/licenses/gpl-license.php.
+ *
+ * Licensee has the right to choose one of the above licenses.
+ *
+ * Redistributions of source code must retain the above copyright
+ * notice and one of the license notices.
+ *
+ * Redistributions in binary form must reproduce both the above copyright
+ * notice, one of the license notices in the documentation
+ * and/or other materials provided with the distribution.
+ */
+
+/***************************************************************************
+ *
+ * Module: uDAPL
+ *
+ * Filename: dapl_ib_cm.c
+ *
+ * Author: Arlin Davis
+ *
+ * Created: 3/10/2005
+ *
+ * Description:
+ *
+ * The uDAPL openib provider - connection management
+ *
+ ****************************************************************************
+ * Source Control System Information
+ *
+ * $Id: $
+ *
+ * Copyright (c) 2005 Intel Corporation. All rights reserved.
+ *
+ **************************************************************************/
+
+#if defined(_WIN32)
+#define FD_SETSIZE 1024
+#define DAPL_FD_SETSIZE FD_SETSIZE
+#endif
+
+#include "dapl.h"
+#include "dapl_adapter_util.h"
+#include "dapl_evd_util.h"
+#include "dapl_cr_util.h"
+#include "dapl_name_service.h"
+#include "dapl_ib_util.h"
+#include "dapl_osd.h"
+
+#if defined(_WIN32) || defined(_WIN64)
+enum DAPL_FD_EVENTS {
+ DAPL_FD_READ = 0x1,
+ DAPL_FD_WRITE = 0x2,
+ DAPL_FD_ERROR = 0x4
+};
+
+static int dapl_config_socket(DAPL_SOCKET s)
+{
+ unsigned long nonblocking = 1;
+ int ret, opt;
+
+ ret = ioctlsocket(s, FIONBIO, &nonblocking);
+
+ /* no delay for small packets */
+ if (!ret)
+ ret = setsockopt(s, IPPROTO_TCP, TCP_NODELAY,
+ (char *)&opt, sizeof(opt));
+ return ret;
+}
+
+static int dapl_connect_socket(DAPL_SOCKET s, struct sockaddr *addr,
+ int addrlen)
+{
+ int err;
+
+ err = connect(s, addr, addrlen);
+ if (err == SOCKET_ERROR)
+ err = WSAGetLastError();
+ return (err == WSAEWOULDBLOCK) ? EAGAIN : err;
+}
+
+struct dapl_fd_set {
+ struct fd_set set[3];
+};
+
+static struct dapl_fd_set *dapl_alloc_fd_set(void)
+{
+ return dapl_os_alloc(sizeof(struct dapl_fd_set));
+}
+
+static void dapl_fd_zero(struct dapl_fd_set *set)
+{
+ FD_ZERO(&set->set[0]);
+ FD_ZERO(&set->set[1]);
+ FD_ZERO(&set->set[2]);
+}
+
+static int dapl_fd_set(DAPL_SOCKET s, struct dapl_fd_set *set,
+ enum DAPL_FD_EVENTS event)
+{
+ FD_SET(s, &set->set[(event == DAPL_FD_READ) ? 0 : 1]);
+ FD_SET(s, &set->set[2]);
+ return 0;
+}
+
+static enum DAPL_FD_EVENTS dapl_poll(DAPL_SOCKET s, enum DAPL_FD_EVENTS event)
+{
+ struct fd_set rw_fds;
+ struct fd_set err_fds;
+ struct timeval tv;
+ int ret;
+
+ FD_ZERO(&rw_fds);
+ FD_ZERO(&err_fds);
+ FD_SET(s, &rw_fds);
+ FD_SET(s, &err_fds);
+
+ tv.tv_sec = 0;
+ tv.tv_usec = 0;
+
+ if (event == DAPL_FD_READ)
+ ret = select(1, &rw_fds, NULL, &err_fds, &tv);
+ else
+ ret = select(1, NULL, &rw_fds, &err_fds, &tv);
+
+ if (ret == 0)
+ return 0;
+ else if (ret == SOCKET_ERROR)
+ return DAPL_FD_ERROR;
+ else if (FD_ISSET(s, &rw_fds))
+ return event;
+ else
+ return DAPL_FD_ERROR;
+}
+
+static int dapl_select(struct dapl_fd_set *set)
+{
+ int ret;
+
+ dapl_dbg_log(DAPL_DBG_TYPE_CM, " dapl_select: sleep\n");
+ ret = select(0, &set->set[0], &set->set[1], &set->set[2], NULL);
+ dapl_dbg_log(DAPL_DBG_TYPE_CM, " dapl_select: wakeup\n");
+
+ if (ret == SOCKET_ERROR)
+ dapl_dbg_log(DAPL_DBG_TYPE_CM,
+ " dapl_select: error 0x%x\n", WSAGetLastError());
+
+ return ret;
+}
+
+static int dapl_socket_errno(void)
+{
+ int err;
+
+ err = WSAGetLastError();
+ switch (err) {
+ case WSAEACCES:
+ case WSAEADDRINUSE:
+ return EADDRINUSE;
+ case WSAECONNRESET:
+ return ECONNRESET;
+ default:
+ return err;
+ }
+}
+#else // _WIN32 || _WIN64
+enum DAPL_FD_EVENTS {
+ DAPL_FD_READ = POLLIN,
+ DAPL_FD_WRITE = POLLOUT,
+ DAPL_FD_ERROR = POLLERR
+};
+
+static int dapl_config_socket(DAPL_SOCKET s)
+{
+ int ret, opt = 1;
+
+ /* non-blocking */
+ ret = fcntl(s, F_GETFL);
+ if (ret >= 0)
+ ret = fcntl(s, F_SETFL, ret | O_NONBLOCK);
+
+ /* no delay for small packets */
+ if (!ret)
+ ret = setsockopt(s, IPPROTO_TCP, TCP_NODELAY,
+ (char *)&opt, sizeof(opt));
+ return ret;
+}
+
+static int dapl_connect_socket(DAPL_SOCKET s, struct sockaddr *addr,
+ int addrlen)
+{
+ int ret;
+
+ ret = connect(s, addr, addrlen);
+
+ return (errno == EINPROGRESS) ? EAGAIN : ret;
+}
+
+struct dapl_fd_set {
+ int index;
+ struct pollfd set[DAPL_FD_SETSIZE];
+};
+
+static struct dapl_fd_set *dapl_alloc_fd_set(void)
+{
+ return dapl_os_alloc(sizeof(struct dapl_fd_set));
+}
+
+static void dapl_fd_zero(struct dapl_fd_set *set)
+{
+ set->index = 0;
+}
+
+static int dapl_fd_set(DAPL_SOCKET s, struct dapl_fd_set *set,
+ enum DAPL_FD_EVENTS event)
+{
+ if (set->index == DAPL_FD_SETSIZE - 1) {
+ dapl_log(DAPL_DBG_TYPE_ERR,
+ "SCM ERR: cm_thread exceeded FD_SETSIZE %d\n",
+ set->index + 1);
+ return -1;
+ }
+
+ set->set[set->index].fd = s;
+ set->set[set->index].revents = 0;
+ set->set[set->index++].events = event;
+ return 0;
+}
+
+static enum DAPL_FD_EVENTS dapl_poll(DAPL_SOCKET s, enum DAPL_FD_EVENTS event)
+{
+ struct pollfd fds;
+ int ret;
+
+ fds.fd = s;
+ fds.events = event;
+ fds.revents = 0;
+ ret = poll(&fds, 1, 0);
+ dapl_log(DAPL_DBG_TYPE_CM, " dapl_poll: fd=%d ret=%d, evnts=0x%x\n",
+ s, ret, fds.revents);
+ if (ret == 0)
+ return 0;
+ else if (ret < 0 || (fds.revents & (POLLERR | POLLHUP | POLLNVAL)))
+ return DAPL_FD_ERROR;
+ else
+ return event;
+}
+
+static int dapl_select(struct dapl_fd_set *set)
+{
+ int ret;
+
+ dapl_dbg_log(DAPL_DBG_TYPE_CM, " dapl_select: sleep, fds=%d\n", set->index);
+ ret = poll(set->set, set->index, -1);
+ dapl_dbg_log(DAPL_DBG_TYPE_CM, " dapl_select: wakeup, ret=0x%x\n", ret);
+ return ret;
+}
+
+#define dapl_socket_errno() errno
+#endif
+
+dp_ib_cm_handle_t dapls_ib_cm_create(DAPL_EP *ep)
+{
+ dp_ib_cm_handle_t cm_ptr;
+
+ /* Allocate CM, init lock, and initialize */
+ if ((cm_ptr = dapl_os_alloc(sizeof(*cm_ptr))) == NULL)
+ return NULL;
+
+ (void)dapl_os_memzero(cm_ptr, sizeof(*cm_ptr));
+ if (dapl_os_lock_init(&cm_ptr->lock))
+ goto bail;
+
+ cm_ptr->msg.ver = htons(DCM_VER);
+ cm_ptr->socket = DAPL_INVALID_SOCKET;
+ cm_ptr->ep = ep;
+ return cm_ptr;
+bail:
+ dapl_os_free(cm_ptr, sizeof(*cm_ptr));
+ return NULL;
+}
+
+/* mark for destroy, remove all references, schedule cleanup */
+/* cm_ptr == NULL (UD), then multi CR's, kill all associated with EP */
+void dapls_ib_cm_free(dp_ib_cm_handle_t cm_ptr, DAPL_EP *ep)
+{
+ DAPL_IA *ia_ptr;
+ DAPL_HCA *hca_ptr = NULL;
+ dp_ib_cm_handle_t cr, next_cr;
+
+ dapl_dbg_log(DAPL_DBG_TYPE_CM,
+ " cm_destroy: cm %p ep %p\n", cm_ptr, ep);
+
+ if (cm_ptr == NULL)
+ goto multi_cleanup;
+
+ /* to notify cleanup thread */
+ hca_ptr = cm_ptr->hca;
+
+ /* cleanup, never made it to work queue */
+ dapl_os_lock(&cm_ptr->lock);
+ if (cm_ptr->state == DCM_INIT) {
+ if (cm_ptr->socket != DAPL_INVALID_SOCKET) {
+ shutdown(cm_ptr->socket, SHUT_RDWR);
+ closesocket(cm_ptr->socket);
+ }
+ dapl_os_unlock(&cm_ptr->lock);
+ dapl_os_free(cm_ptr, sizeof(*cm_ptr));
+ return;
+ }
+
+ /* free could be called before disconnect, disc_clean will destroy */
+ if (cm_ptr->state == DCM_CONNECTED) {
+ dapl_os_unlock(&cm_ptr->lock);
+ dapli_socket_disconnect(cm_ptr);
+ return;
+ }
+
+ cm_ptr->state = DCM_DESTROY;
+ if ((cm_ptr->ep) && (cm_ptr->ep->cm_handle == cm_ptr)) {
+ cm_ptr->ep->cm_handle = IB_INVALID_HANDLE;
+ cm_ptr->ep = NULL;
+ }
+
+ dapl_os_unlock(&cm_ptr->lock);
+ goto notify_thread;
+
+multi_cleanup:
+
+ /*
+ * UD CR objects are kept active because of direct private data references
+ * from CONN events. The cr->socket is closed and marked inactive but the
+ * object remains allocated and queued on the CR resource list. There can
+ * be multiple CR's associated with a given EP. There is no way to determine
+ * when consumer is finished with event until the dat_ep_free.
+ *
+ * Schedule destruction for all CR's associated with this EP, cr_thread will
+ * complete the cleanup with state == DCM_DESTROY.
+ */
+ ia_ptr = ep->header.owner_ia;
+ dapl_os_lock(&ia_ptr->hca_ptr->ib_trans.lock);
+ if (!dapl_llist_is_empty((DAPL_LLIST_HEAD*)
+ &ia_ptr->hca_ptr->ib_trans.list))
+ next_cr = dapl_llist_peek_head((DAPL_LLIST_HEAD*)
+ &ia_ptr->hca_ptr->ib_trans.list);
+ else
+ next_cr = NULL;
+
+ while (next_cr) {
+ cr = next_cr;
+ next_cr = dapl_llist_next_entry((DAPL_LLIST_HEAD*)
+ &ia_ptr->hca_ptr->ib_trans.list,
+ (DAPL_LLIST_ENTRY*)&cr->entry);
+ if (cr->ep == ep) {
+ dapl_dbg_log(DAPL_DBG_TYPE_EP,
+ " qp_free CR: ep %p cr %p\n", ep, cr);
+ dapli_socket_disconnect(cr);
+ dapl_os_lock(&cr->lock);
+ hca_ptr = cr->hca;
+ cr->ep = NULL;
+ if (cr->ah) {
+ ibv_destroy_ah(cr->ah);
+ cr->ah = NULL;
+ }
+ cr->state = DCM_DESTROY;
+ dapl_os_unlock(&cr->lock);
+ }
+ }
+ dapl_os_unlock(&ia_ptr->hca_ptr->ib_trans.lock);
+
+notify_thread:
+
+ /* wakeup work thread, if something destroyed */
+ if (hca_ptr != NULL)
+ send(hca_ptr->ib_trans.scm[1], "w", sizeof "w", 0);
+}
+
+/* queue socket for processing CM work */
+static void dapli_cm_queue(struct ib_cm_handle *cm_ptr)
+{
+ /* add to work queue for cr thread processing */
+ dapl_llist_init_entry((DAPL_LLIST_ENTRY *) & cm_ptr->entry);
+ dapl_os_lock(&cm_ptr->hca->ib_trans.lock);
+ dapl_llist_add_tail(&cm_ptr->hca->ib_trans.list,
+ (DAPL_LLIST_ENTRY *) & cm_ptr->entry, cm_ptr);
+ dapl_os_unlock(&cm_ptr->hca->ib_trans.lock);
+
+ /* wakeup CM work thread */
+ send(cm_ptr->hca->ib_trans.scm[1], "w", sizeof "w", 0);
+}
+
+/*
+ * ACTIVE/PASSIVE: called from CR thread or consumer via ep_disconnect
+ * or from ep_free
+ */
+DAT_RETURN dapli_socket_disconnect(dp_ib_cm_handle_t cm_ptr)
+{
+ DAPL_EP *ep_ptr = cm_ptr->ep;
+ DAT_UINT32 disc_data = htonl(0xdead);
+
+ if (ep_ptr == NULL)
+ return DAT_SUCCESS;
+
+ dapl_os_lock(&cm_ptr->lock);
+ if (cm_ptr->state != DCM_CONNECTED) {
+ dapl_os_unlock(&cm_ptr->lock);
+ return DAT_SUCCESS;
+ }
+
+ /* send disc date, close socket, schedule destroy */
+ dapls_modify_qp_state(ep_ptr->qp_handle, IBV_QPS_ERR, 0,0,0);
+ cm_ptr->state = DCM_DISCONNECTED;
+ send(cm_ptr->socket, (char *)&disc_data, sizeof(disc_data), 0);
+ dapl_os_unlock(&cm_ptr->lock);
+
+ /* disconnect events for RC's only */
+ if (ep_ptr->param.ep_attr.service_type == DAT_SERVICE_TYPE_RC) {
+ if (ep_ptr->cr_ptr) {
+ dapls_cr_callback(cm_ptr,
+ IB_CME_DISCONNECTED,
+ NULL,
+ ((DAPL_CR *) ep_ptr->cr_ptr)->sp_ptr);
+ } else {
+ dapl_evd_connection_callback(ep_ptr->cm_handle,
+ IB_CME_DISCONNECTED,
+ NULL, ep_ptr);
+ }
+ }
+
+ /* scheduled destroy via disconnect clean in callback */
+ return DAT_SUCCESS;
+}
+
+/*
+ * ACTIVE: socket connected, send QP information to peer
+ */
+static void dapli_socket_connected(dp_ib_cm_handle_t cm_ptr, int err)
+{
+ int len, exp;
+ struct iovec iov[2];
+ struct dapl_ep *ep_ptr = cm_ptr->ep;
+
+ if (err) {
+ dapl_log(DAPL_DBG_TYPE_ERR,
+ " CONN_PENDING: %s ERR %s -> %s %d\n",
+ err == -1 ? "POLL" : "SOCKOPT",
+ err == -1 ? strerror(errno) : strerror(err),
+ inet_ntoa(((struct sockaddr_in *)
+ &cm_ptr->addr)->sin_addr),
+ ntohs(((struct sockaddr_in *)
+ &cm_ptr->addr)->sin_port));
+ goto bail;
+ }
+
+ cm_ptr->state = DCM_REP_PENDING;
+
+ /* send qp info and pdata to remote peer */
+ exp = sizeof(ib_cm_msg_t) - DCM_MAX_PDATA_SIZE;
+ iov[0].iov_base = (void *)&cm_ptr->msg;
+ iov[0].iov_len = exp;
+ if (cm_ptr->msg.p_size) {
+ iov[1].iov_base = cm_ptr->msg.p_data;
+ iov[1].iov_len = ntohs(cm_ptr->msg.p_size);
+ len = writev(cm_ptr->socket, iov, 2);
+ } else {
+ len = writev(cm_ptr->socket, iov, 1);
+ }
+
+ if (len != (exp + ntohs(cm_ptr->msg.p_size))) {
+ dapl_log(DAPL_DBG_TYPE_ERR,
+ " CONN_PENDING len ERR %s, wcnt=%d(%d) -> %s\n",
+ strerror(errno), len,
+ exp + ntohs(cm_ptr->msg.p_size),
+ inet_ntoa(((struct sockaddr_in *)
+ ep_ptr->param.
+ remote_ia_address_ptr)->sin_addr));
+ goto bail;
+ }
+
+ dapl_dbg_log(DAPL_DBG_TYPE_CM,
+ " CONN_PENDING: sending SRC lid=0x%x,"
+ " qpn=0x%x, psize=%d\n",
+ ntohs(cm_ptr->msg.saddr.ib.lid),
+ ntohl(cm_ptr->msg.saddr.ib.qpn),
+ ntohs(cm_ptr->msg.p_size));
+ dapl_dbg_log(DAPL_DBG_TYPE_CM,
+ " CONN_PENDING: SRC GID subnet %016llx id %016llx\n",
+ (unsigned long long)
+ htonll(*(uint64_t*)&cm_ptr->msg.saddr.ib.gid[0]),
+ (unsigned long long)
+ htonll(*(uint64_t*)&cm_ptr->msg.saddr.ib.gid[8]));
+ return;
+
+bail:
+ /* close socket, free cm structure and post error event */
+ dapls_ib_cm_free(cm_ptr, cm_ptr->ep);
+ dapl_evd_connection_callback(NULL, IB_CME_LOCAL_FAILURE, NULL, ep_ptr);
+}
+
+/*
+ * ACTIVE: Create socket, connect, defer exchange QP information to CR thread
+ * to avoid blocking.
+ */
+DAT_RETURN
+dapli_socket_connect(DAPL_EP * ep_ptr,
+ DAT_IA_ADDRESS_PTR r_addr,
+ DAT_CONN_QUAL r_qual, DAT_COUNT p_size, DAT_PVOID p_data)
+{
+ dp_ib_cm_handle_t cm_ptr;
+ int ret;
+ socklen_t sl;
+ DAPL_IA *ia_ptr = ep_ptr->header.owner_ia;
+ DAT_RETURN dat_ret = DAT_INSUFFICIENT_RESOURCES;
+
+ dapl_dbg_log(DAPL_DBG_TYPE_EP, " connect: r_qual %d p_size=%d\n",
+ r_qual, p_size);
+
+ cm_ptr = dapls_ib_cm_create(ep_ptr);
+ if (cm_ptr == NULL)
+ return dat_ret;
+
+ /* create, connect, sockopt, and exchange QP information */
+ if ((cm_ptr->socket =
+ socket(AF_INET, SOCK_STREAM, IPPROTO_TCP)) == DAPL_INVALID_SOCKET) {
+ dapl_log(DAPL_DBG_TYPE_ERR,
+ " connect: socket create ERR %s\n", strerror(errno));
+ goto bail;
+ }
+
+ ret = dapl_config_socket(cm_ptr->socket);
+ if (ret < 0) {
+ dapl_log(DAPL_DBG_TYPE_ERR,
+ " connect: config socket %d ERR %d %s\n",
+ cm_ptr->socket, ret, strerror(dapl_socket_errno()));
+ dat_ret = DAT_INTERNAL_ERROR;
+ goto bail;
+ }
+
+ /* save remote address */
+ dapl_os_memcpy(&cm_ptr->addr, r_addr, sizeof(r_addr));
+
+#ifdef DAPL_DBG
+ /* DBG: Active PID [0], PASSIVE PID [2]*/
+ *(uint16_t*)&cm_ptr->msg.resv[0] = htons((uint16_t)dapl_os_getpid());
+ *(uint16_t*)&cm_ptr->msg.resv[2] = ((struct sockaddr_in *)&cm_ptr->addr)->sin_port;
+#endif
+ ((struct sockaddr_in *)&cm_ptr->addr)->sin_port = htons(r_qual + 1000);
+ ret = dapl_connect_socket(cm_ptr->socket, (struct sockaddr *)&cm_ptr->addr,
+ sizeof(cm_ptr->addr));
+ if (ret && ret != EAGAIN) {
+ dat_ret = DAT_INVALID_ADDRESS;
+ goto bail;
+ }
+
+ /* REQ: QP info in msg.saddr, IA address in msg.daddr, and pdata */
+ cm_ptr->msg.op = ntohs(DCM_REQ);
+ cm_ptr->msg.saddr.ib.qpn = htonl(ep_ptr->qp_handle->qp_num);
+ cm_ptr->msg.saddr.ib.qp_type = ep_ptr->qp_handle->qp_type;
+ cm_ptr->msg.saddr.ib.lid = ia_ptr->hca_ptr->ib_trans.lid;
+ dapl_os_memcpy(&cm_ptr->msg.saddr.ib.gid[0],
+ &ia_ptr->hca_ptr->ib_trans.gid, 16);
+
+ /* save references */
+ cm_ptr->hca = ia_ptr->hca_ptr;
+ cm_ptr->ep = ep_ptr;
+
+ /* get local address information from socket */
+ sl = sizeof(cm_ptr->msg.daddr.so);
+ if (getsockname(cm_ptr->socket, (struct sockaddr *)&cm_ptr->msg.daddr.so, &sl)) {
+ dapl_log(DAPL_DBG_TYPE_ERR,
+ " connect getsockname ERROR: %s -> %s r_qual %d\n",
+ strerror(errno),
+ inet_ntoa(((struct sockaddr_in *)r_addr)->sin_addr),
+ (unsigned int)r_qual);;
+ }
+
+ if (p_size) {
+ cm_ptr->msg.p_size = htons(p_size);
+ dapl_os_memcpy(cm_ptr->msg.p_data, p_data, p_size);
+ }
+
+ /* connected or pending, either way results via async event */
+ if (ret == 0)
+ dapli_socket_connected(cm_ptr, 0);
+ else
+ cm_ptr->state = DCM_CONN_PENDING;
+
+ dapl_dbg_log(DAPL_DBG_TYPE_EP, " connect: p_data=%p %p\n",
+ cm_ptr->msg.p_data, cm_ptr->msg.p_data);
+
+ dapl_dbg_log(DAPL_DBG_TYPE_EP,
+ " connect: %s r_qual %d pending, p_sz=%d, %d %d ...\n",
+ inet_ntoa(((struct sockaddr_in *)&cm_ptr->addr)->sin_addr),
+ (unsigned int)r_qual, ntohs(cm_ptr->msg.p_size),
+ cm_ptr->msg.p_data[0], cm_ptr->msg.p_data[1]);
+
+ dapli_cm_queue(cm_ptr);
+ return DAT_SUCCESS;
+
+bail:
+ dapl_log(DAPL_DBG_TYPE_ERR,
+ " connect ERROR: %s -> %s r_qual %d\n",
+ strerror(errno),
+ inet_ntoa(((struct sockaddr_in *)r_addr)->sin_addr),
+ (unsigned int)r_qual);
+
+ /* close socket, free cm structure */
+ dapls_ib_cm_free(cm_ptr, NULL);
+ return dat_ret;
+}
+
+/*
+ * ACTIVE: exchange QP information, called from CR thread
+ */
+static void dapli_socket_connect_rtu(dp_ib_cm_handle_t cm_ptr)
+{
+ DAPL_EP *ep_ptr = cm_ptr->ep;
+ int len, exp = sizeof(ib_cm_msg_t) - DCM_MAX_PDATA_SIZE;
+ ib_cm_events_t event = IB_CME_LOCAL_FAILURE;
+ socklen_t sl;
+
+ /* read DST information into cm_ptr, overwrite SRC info */
+ dapl_dbg_log(DAPL_DBG_TYPE_EP, " connect_rtu: recv peer QP data\n");
+
+ len = recv(cm_ptr->socket, (char *)&cm_ptr->msg, exp, 0);
+ if (len != exp || ntohs(cm_ptr->msg.ver) != DCM_VER) {
+ dapl_log(DAPL_DBG_TYPE_WARN,
+ " CONN_RTU read: sk %d ERR %s, rcnt=%d, v=%d -> %s PORT L-%x R-%x PID L-%x R-%x\n",
+ cm_ptr->socket, strerror(errno), len, ntohs(cm_ptr->msg.ver),
+ inet_ntoa(((struct sockaddr_in *)&cm_ptr->addr)->sin_addr),
+ ntohs(((struct sockaddr_in *)&cm_ptr->msg.daddr.so)->sin_port),
+ ntohs(((struct sockaddr_in *)&cm_ptr->addr)->sin_port),
+ ntohs(*(uint16_t*)&cm_ptr->msg.resv[0]),
+ ntohs(*(uint16_t*)&cm_ptr->msg.resv[2]));
+
+ /* Retry; corner case where server tcp stack resets under load */
+ if (dapl_socket_errno() == ECONNRESET) {
+ closesocket(cm_ptr->socket);
+ cm_ptr->socket = DAPL_INVALID_SOCKET;
+ dapli_socket_connect(cm_ptr->ep, (DAT_IA_ADDRESS_PTR)&cm_ptr->addr,
+ ntohs(((struct sockaddr_in *)&cm_ptr->addr)->sin_port) - 1000,
+ ntohs(cm_ptr->msg.p_size), &cm_ptr->msg.p_data);
+ dapls_ib_cm_free(cm_ptr, NULL);
+ return;
+ }
+ goto bail;
+ }
+
+ /* keep the QP, address info in network order */
+
+ /* save remote address information, in msg.daddr */
+ dapl_os_memcpy(&ep_ptr->remote_ia_address,
+ &cm_ptr->msg.daddr.so,
+ sizeof(union dcm_addr));
+
+ /* save local address information from socket */
+ sl = sizeof(cm_ptr->addr);
+ getsockname(cm_ptr->socket,(struct sockaddr *)&cm_ptr->addr, &sl);
+
+ dapl_dbg_log(DAPL_DBG_TYPE_EP,
+ " CONN_RTU: DST %s %d lid=0x%x,"
+ " qpn=0x%x, qp_type=%d, psize=%d\n",
+ inet_ntoa(((struct sockaddr_in *)
+ &cm_ptr->msg.daddr.so)->sin_addr),
+ ntohs(((struct sockaddr_in *)
+ &cm_ptr->msg.daddr.so)->sin_port),
+ ntohs(cm_ptr->msg.saddr.ib.lid),
+ ntohl(cm_ptr->msg.saddr.ib.qpn),
+ cm_ptr->msg.saddr.ib.qp_type,
+ ntohs(cm_ptr->msg.p_size));
+
+ /* validate private data size before reading */
+ if (ntohs(cm_ptr->msg.p_size) > DCM_MAX_PDATA_SIZE) {
+ dapl_log(DAPL_DBG_TYPE_ERR,
+ " CONN_RTU read: psize (%d) wrong -> %s\n",
+ ntohs(cm_ptr->msg.p_size),
+ inet_ntoa(((struct sockaddr_in *)
+ ep_ptr->param.
+ remote_ia_address_ptr)->sin_addr));
+ goto bail;
+ }
+
+ /* read private data into cm_handle if any present */
+ dapl_dbg_log(DAPL_DBG_TYPE_EP," CONN_RTU: read private data\n");
+ exp = ntohs(cm_ptr->msg.p_size);
+ if (exp) {
+ len = recv(cm_ptr->socket, cm_ptr->msg.p_data, exp, 0);
+ if (len != exp) {
+ dapl_log(DAPL_DBG_TYPE_ERR,
+ " CONN_RTU read pdata: ERR %s, rcnt=%d -> %s\n",
+ strerror(errno), len,
+ inet_ntoa(((struct sockaddr_in *)
+ ep_ptr->param.
+ remote_ia_address_ptr)->sin_addr));
+ goto bail;
+ }
+ }
+
+ /* check for consumer or protocol stack reject */
+ if (ntohs(cm_ptr->msg.op) == DCM_REP)
+ event = IB_CME_CONNECTED;
+ else if (ntohs(cm_ptr->msg.op) == DCM_REJ_USER)
+ event = IB_CME_DESTINATION_REJECT_PRIVATE_DATA;
+ else
+ event = IB_CME_DESTINATION_REJECT;
+
+ if (event != IB_CME_CONNECTED) {
+ dapl_log(DAPL_DBG_TYPE_CM,
+ " CONN_RTU: reject from %s %x\n",
+ inet_ntoa(((struct sockaddr_in *)
+ &cm_ptr->msg.daddr.so)->sin_addr),
+ ntohs(((struct sockaddr_in *)
+ &cm_ptr->msg.daddr.so)->sin_port));
+ goto bail;
+ }
+
+ /* modify QP to RTR and then to RTS with remote info */
+ dapl_os_lock(&ep_ptr->header.lock);
+ if (dapls_modify_qp_state(ep_ptr->qp_handle,
+ IBV_QPS_RTR,
+ cm_ptr->msg.saddr.ib.qpn,
+ cm_ptr->msg.saddr.ib.lid,
+ NULL) != DAT_SUCCESS) {
+ dapl_log(DAPL_DBG_TYPE_ERR,
+ " CONN_RTU: QPS_RTR ERR %s (%d,%d,%x,%x,%x) -> %s %x\n",
+ strerror(errno), ep_ptr->qp_handle->qp_type,
+ ep_ptr->qp_state, ep_ptr->qp_handle->qp_num,
+ ntohl(cm_ptr->msg.saddr.ib.qpn),
+ ntohs(cm_ptr->msg.saddr.ib.lid),
+ inet_ntoa(((struct sockaddr_in *)
+ &cm_ptr->msg.daddr.so)->sin_addr),
+ ntohs(((struct sockaddr_in *)
+ &cm_ptr->msg.daddr.so)->sin_port));
+ dapl_os_unlock(&ep_ptr->header.lock);
+ goto bail;
+ }
+ if (dapls_modify_qp_state(ep_ptr->qp_handle,
+ IBV_QPS_RTS,
+ cm_ptr->msg.saddr.ib.qpn,
+ cm_ptr->msg.saddr.ib.lid,
+ NULL) != DAT_SUCCESS) {
+ dapl_log(DAPL_DBG_TYPE_ERR,
+ " CONN_RTU: QPS_RTS ERR %s (%d,%d,%x,%x,%x) -> %s %x\n",
+ strerror(errno), ep_ptr->qp_handle->qp_type,
+ ep_ptr->qp_state, ep_ptr->qp_handle->qp_num,
+ ntohl(cm_ptr->msg.saddr.ib.qpn),
+ ntohs(cm_ptr->msg.saddr.ib.lid),
+ inet_ntoa(((struct sockaddr_in *)
+ &cm_ptr->msg.daddr.so)->sin_addr),
+ ntohs(((struct sockaddr_in *)
+ &cm_ptr->msg.daddr.so)->sin_port));
+ dapl_os_unlock(&ep_ptr->header.lock);
+ goto bail;
+ }
+ dapl_os_unlock(&ep_ptr->header.lock);
+ dapl_dbg_log(DAPL_DBG_TYPE_EP, " connect_rtu: send RTU\n");
+
+ /* complete handshake after final QP state change, Just ver+op */
+ cm_ptr->state = DCM_CONNECTED;
+ cm_ptr->msg.op = ntohs(DCM_RTU);
+ if (send(cm_ptr->socket, (char *)&cm_ptr->msg, 4, 0) == -1) {
+ dapl_log(DAPL_DBG_TYPE_ERR,
+ " CONN_RTU: write error = %s\n", strerror(errno));
+ goto bail;
+ }
+ /* post the event with private data */
+ event = IB_CME_CONNECTED;
+ dapl_dbg_log(DAPL_DBG_TYPE_EP, " ACTIVE: connected!\n");
+
+#ifdef DAT_EXTENSIONS
+ud_bail:
+ if (cm_ptr->msg.saddr.ib.qp_type == IBV_QPT_UD) {
+ DAT_IB_EXTENSION_EVENT_DATA xevent;
+ ib_pd_handle_t pd_handle =
+ ((DAPL_PZ *)ep_ptr->param.pz_handle)->pd_handle;
+
+ if (event == IB_CME_CONNECTED) {
+ cm_ptr->ah = dapls_create_ah(cm_ptr->hca, pd_handle,
+ ep_ptr->qp_handle,
+ cm_ptr->msg.saddr.ib.lid,
+ NULL);
+ if (cm_ptr->ah) {
+ /* post UD extended EVENT */
+ xevent.status = 0;
+ xevent.type = DAT_IB_UD_REMOTE_AH;
+ xevent.remote_ah.ah = cm_ptr->ah;
+ xevent.remote_ah.qpn = ntohl(cm_ptr->msg.saddr.ib.qpn);
+ dapl_os_memcpy(&xevent.remote_ah.ia_addr,
+ &ep_ptr->remote_ia_address,
+ sizeof(union dcm_addr));
+ event = DAT_IB_UD_CONNECTION_EVENT_ESTABLISHED;
+
+ dapl_log(DAPL_DBG_TYPE_CM,
+ " CONN_RTU: UD AH %p for lid 0x%x"
+ " qpn 0x%x\n",
+ cm_ptr->ah,
+ ntohs(cm_ptr->msg.saddr.ib.lid),
+ ntohl(cm_ptr->msg.saddr.ib.qpn));
+
+ } else
+ event = DAT_IB_UD_CONNECTION_ERROR_EVENT;
+
+ } else if (event == IB_CME_LOCAL_FAILURE) {
+ event = DAT_IB_UD_CONNECTION_ERROR_EVENT;
+ } else
+ event = DAT_IB_UD_CONNECTION_REJECT_EVENT;
+
+ dapls_evd_post_connection_event_ext(
+ (DAPL_EVD *) ep_ptr->param.connect_evd_handle,
+ event,
+ (DAT_EP_HANDLE) ep_ptr,
+ (DAT_COUNT) exp,
+ (DAT_PVOID *) cm_ptr->msg.p_data,
+ (DAT_PVOID *) &xevent);
+
+ /* done with socket, don't destroy cm_ptr, need pdata */
+ closesocket(cm_ptr->socket);
+ cm_ptr->socket = DAPL_INVALID_SOCKET;
+ cm_ptr->state = DCM_RELEASED;
+ } else
+#endif
+ {
+ ep_ptr->cm_handle = cm_ptr; /* only RC, multi CR's on UD */
+ dapl_evd_connection_callback(cm_ptr, event, cm_ptr->msg.p_data, ep_ptr);
+ }
+ return;
+
+bail:
+
+#ifdef DAT_EXTENSIONS
+ if (cm_ptr->msg.saddr.ib.qp_type == IBV_QPT_UD)
+ goto ud_bail;
+#endif
+ /* close socket, and post error event */
+ dapls_modify_qp_state(ep_ptr->qp_handle, IBV_QPS_ERR, 0, 0, 0);
+ closesocket(cm_ptr->socket);
+ cm_ptr->socket = DAPL_INVALID_SOCKET;
+ dapl_evd_connection_callback(NULL, event, cm_ptr->msg.p_data, ep_ptr);
+}
+
+/*
+ * PASSIVE: Create socket, listen, accept, exchange QP information
+ */
+DAT_RETURN
+dapli_socket_listen(DAPL_IA * ia_ptr, DAT_CONN_QUAL serviceID, DAPL_SP * sp_ptr)
+{
+ struct sockaddr_in addr;
+ ib_cm_srvc_handle_t cm_ptr = NULL;
+ DAT_RETURN dat_status = DAT_SUCCESS;
+
+ dapl_dbg_log(DAPL_DBG_TYPE_EP,
+ " listen(ia_ptr %p ServiceID %d sp_ptr %p)\n",
+ ia_ptr, serviceID, sp_ptr);
+
+ cm_ptr = dapls_ib_cm_create(NULL);
+ if (cm_ptr == NULL)
+ return DAT_INSUFFICIENT_RESOURCES;
+
+ cm_ptr->sp = sp_ptr;
+ cm_ptr->hca = ia_ptr->hca_ptr;
+
+ /* bind, listen, set sockopt, accept, exchange data */
+ if ((cm_ptr->socket =
+ socket(AF_INET, SOCK_STREAM, IPPROTO_TCP)) == DAPL_INVALID_SOCKET) {
+ dapl_log(DAPL_DBG_TYPE_ERR, " ERR: listen socket create: %s\n",
+ strerror(errno));
+ dat_status = DAT_INSUFFICIENT_RESOURCES;
+ goto bail;
+ }
+
+ addr.sin_port = htons(serviceID + 1000);
+ addr.sin_family = AF_INET;
+ addr.sin_addr = ((struct sockaddr_in *) &ia_ptr->hca_ptr->hca_address)->sin_addr;
+
+ if ((bind(cm_ptr->socket, (struct sockaddr *)&addr, sizeof(addr)) < 0)
+ || (listen(cm_ptr->socket, 128) < 0)) {
+ dapl_dbg_log(DAPL_DBG_TYPE_CM,
+ " listen: ERROR %s on conn_qual 0x%x\n",
+ strerror(errno), serviceID + 1000);
+ if (dapl_socket_errno() == EADDRINUSE)
+ dat_status = DAT_CONN_QUAL_IN_USE;
+ else
+ dat_status = DAT_CONN_QUAL_UNAVAILABLE;
+ goto bail;
+ }
+
+ /* set cm_handle for this service point, save listen socket */
+ sp_ptr->cm_srvc_handle = cm_ptr;
+ dapl_os_memcpy(&cm_ptr->addr, &addr, sizeof(addr));
+
+ /* queue up listen socket to process inbound CR's */
+ cm_ptr->state = DCM_LISTEN;
+ dapli_cm_queue(cm_ptr);
+
+ dapl_dbg_log(DAPL_DBG_TYPE_CM,
+ " listen: qual 0x%x cr %p s_fd %d\n",
+ ntohs(serviceID + 1000), cm_ptr, cm_ptr->socket);
+
+ return dat_status;
+bail:
+ dapl_dbg_log(DAPL_DBG_TYPE_CM,
+ " listen: ERROR on conn_qual 0x%x\n", serviceID + 1000);
+ dapls_ib_cm_free(cm_ptr, cm_ptr->ep);
+ return dat_status;
+}
+
+/*
+ * PASSIVE: accept socket
+ */
+static void dapli_socket_accept(ib_cm_srvc_handle_t cm_ptr)
+{
+ dp_ib_cm_handle_t acm_ptr;
+ int ret, len, opt = 1;
+ socklen_t sl;
+
+ /*
+ * Accept all CR's on this port to avoid half-connection (SYN_RCV)
+ * stalls with many to one connection storms
+ */
+ do {
+ /* Allocate accept CM and initialize */
+ if ((acm_ptr = dapls_ib_cm_create(NULL)) == NULL)
+ return;
+
+ acm_ptr->sp = cm_ptr->sp;
+ acm_ptr->hca = cm_ptr->hca;
+
+ len = sizeof(union dcm_addr);
+ acm_ptr->socket = accept(cm_ptr->socket,
+ (struct sockaddr *)
+ &acm_ptr->msg.daddr.so,
+ (socklen_t *) &len);
+ if (acm_ptr->socket == DAPL_INVALID_SOCKET) {
+ dapl_log(DAPL_DBG_TYPE_ERR,
+ " ACCEPT: ERR %s on FD %d l_cr %p\n",
+ strerror(errno), cm_ptr->socket, cm_ptr);
+ dapls_ib_cm_free(acm_ptr, acm_ptr->ep);
+ return;
+ }
+ dapl_dbg_log(DAPL_DBG_TYPE_CM, " accepting from %s %x\n",
+ inet_ntoa(((struct sockaddr_in *)
+ &acm_ptr->msg.daddr.so)->sin_addr),
+ ntohs(((struct sockaddr_in *)
+ &acm_ptr->msg.daddr.so)->sin_port));
+
+ /* no delay for small packets */
+ ret = setsockopt(acm_ptr->socket, IPPROTO_TCP, TCP_NODELAY,
+ (char *)&opt, sizeof(opt));
+ if (ret)
+ dapl_log(DAPL_DBG_TYPE_ERR,
+ " ACCEPT: NODELAY setsockopt: 0x%x 0x%x %s\n",
+ ret, dapl_socket_errno(), strerror(dapl_socket_errno()));
+
+ /* get local address information from socket */
+ sl = sizeof(acm_ptr->addr);
+ getsockname(acm_ptr->socket, (struct sockaddr *)&acm_ptr->addr, &sl);
+ acm_ptr->state = DCM_ACCEPTING;
+ dapli_cm_queue(acm_ptr);
+
+ } while (dapl_poll(cm_ptr->socket, DAPL_FD_READ) == DAPL_FD_READ);
+}
+
+/*
+ * PASSIVE: receive peer QP information, private data, post cr_event
+ */
+static void dapli_socket_accept_data(ib_cm_srvc_handle_t acm_ptr)
+{
+ int len, exp = sizeof(ib_cm_msg_t) - DCM_MAX_PDATA_SIZE;
+ void *p_data = NULL;
+
+ dapl_dbg_log(DAPL_DBG_TYPE_EP, " socket accepted, read QP data\n");
+
+ /* read in DST QP info, IA address. check for private data */
+ len = recv(acm_ptr->socket, (char *)&acm_ptr->msg, exp, 0);
+ if (len != exp || ntohs(acm_ptr->msg.ver) != DCM_VER) {
+ dapl_log(DAPL_DBG_TYPE_ERR,
+ " ACCEPT read: ERR %s, rcnt=%d, ver=%d\n",
+ strerror(errno), len, ntohs(acm_ptr->msg.ver));
+ goto bail;
+ }
+
+ /* keep the QP, address info in network order */
+
+ /* validate private data size before reading */
+ exp = ntohs(acm_ptr->msg.p_size);
+ if (exp > DCM_MAX_PDATA_SIZE) {
+ dapl_log(DAPL_DBG_TYPE_ERR,
+ " accept read: psize (%d) wrong\n",
+ acm_ptr->msg.p_size);
+ goto bail;
+ }
+
+ /* read private data into cm_handle if any present */
+ if (exp) {
+ len = recv(acm_ptr->socket, acm_ptr->msg.p_data, exp, 0);
+ if (len != exp) {
+ dapl_log(DAPL_DBG_TYPE_ERR,
+ " accept read pdata: ERR %s, rcnt=%d\n",
+ strerror(errno), len);
+ goto bail;
+ }
+ p_data = acm_ptr->msg.p_data;
+ }
+
+ acm_ptr->state = DCM_ACCEPTING_DATA;
+
+ dapl_dbg_log(DAPL_DBG_TYPE_CM,
+ " ACCEPT: DST %s %x lid=0x%x, qpn=0x%x, psz=%d\n",
+ inet_ntoa(((struct sockaddr_in *)
+ &acm_ptr->msg.daddr.so)->sin_addr),
+ ntohs(((struct sockaddr_in *)
+ &acm_ptr->msg.daddr.so)->sin_port),
+ ntohs(acm_ptr->msg.saddr.ib.lid),
+ ntohl(acm_ptr->msg.saddr.ib.qpn), exp);
+
+#ifdef DAT_EXTENSIONS
+ if (acm_ptr->msg.saddr.ib.qp_type == IBV_QPT_UD) {
+ DAT_IB_EXTENSION_EVENT_DATA xevent;
+
+ /* post EVENT, modify_qp created ah */
+ xevent.status = 0;
+ xevent.type = DAT_IB_UD_CONNECT_REQUEST;
+
+ dapls_evd_post_cr_event_ext(acm_ptr->sp,
+ DAT_IB_UD_CONNECTION_REQUEST_EVENT,
+ acm_ptr,
+ (DAT_COUNT) exp,
+ (DAT_PVOID *) acm_ptr->msg.p_data,
+ (DAT_PVOID *) &xevent);
+ } else
+#endif
+ /* trigger CR event and return SUCCESS */
+ dapls_cr_callback(acm_ptr,
+ IB_CME_CONNECTION_REQUEST_PENDING,
+ p_data, acm_ptr->sp);
+ return;
+bail:
+ /* close socket, free cm structure, active will see close as rej */
+ dapls_ib_cm_free(acm_ptr, acm_ptr->ep);
+ return;
+}
+
+/*
+ * PASSIVE: consumer accept, send local QP information, private data,
+ * queue on work thread to receive RTU information to avoid blocking
+ * user thread.
+ */
+static DAT_RETURN
+dapli_socket_accept_usr(DAPL_EP * ep_ptr,
+ DAPL_CR * cr_ptr, DAT_COUNT p_size, DAT_PVOID p_data)
+{
+ DAPL_IA *ia_ptr = ep_ptr->header.owner_ia;
+ dp_ib_cm_handle_t cm_ptr = cr_ptr->ib_cm_handle;
+ ib_cm_msg_t local;
+ struct iovec iov[2];
+ int len, exp = sizeof(ib_cm_msg_t) - DCM_MAX_PDATA_SIZE;
+ DAT_RETURN ret = DAT_INTERNAL_ERROR;
+ socklen_t sl;
+
+ if (p_size > DCM_MAX_PDATA_SIZE) {
+ dapl_log(DAPL_DBG_TYPE_ERR,
+ " accept_usr: psize(%d) too large\n", p_size);
+ return DAT_LENGTH_ERROR;
+ }
+
+ /* must have a accepted socket */
+ if (cm_ptr->socket == DAPL_INVALID_SOCKET) {
+ dapl_log(DAPL_DBG_TYPE_ERR,
+ " accept_usr: cm socket invalid\n");
+ goto bail;
+ }
+
+ dapl_dbg_log(DAPL_DBG_TYPE_CM,
+ " ACCEPT_USR: remote lid=0x%x"
+ " qpn=0x%x qp_type %d, psize=%d\n",
+ ntohs(cm_ptr->msg.saddr.ib.lid),
+ ntohl(cm_ptr->msg.saddr.ib.qpn),
+ cm_ptr->msg.saddr.ib.qp_type,
+ ntohs(cm_ptr->msg.p_size));
+
+#ifdef DAT_EXTENSIONS
+ if (cm_ptr->msg.saddr.ib.qp_type == IBV_QPT_UD &&
+ ep_ptr->qp_handle->qp_type != IBV_QPT_UD) {
+ dapl_log(DAPL_DBG_TYPE_ERR,
+ " ACCEPT_USR: ERR remote QP is UD,"
+ ", but local QP is not\n");
+ ret = (DAT_INVALID_HANDLE | DAT_INVALID_HANDLE_EP);
+ goto bail;
+ }
+#endif
+
+ /* modify QP to RTR and then to RTS with remote info already read */
+ dapl_os_lock(&ep_ptr->header.lock);
+ if (dapls_modify_qp_state(ep_ptr->qp_handle,
+ IBV_QPS_RTR,
+ cm_ptr->msg.saddr.ib.qpn,
+ cm_ptr->msg.saddr.ib.lid,
+ NULL) != DAT_SUCCESS) {
+ dapl_log(DAPL_DBG_TYPE_ERR,
+ " ACCEPT_USR: QPS_RTR ERR %s -> %s\n",
+ strerror(errno),
+ inet_ntoa(((struct sockaddr_in *)
+ &cm_ptr->msg.daddr.so)->sin_addr));
+ dapl_os_unlock(&ep_ptr->header.lock);
+ goto bail;
+ }
+ if (dapls_modify_qp_state(ep_ptr->qp_handle,
+ IBV_QPS_RTS,
+ cm_ptr->msg.saddr.ib.qpn,
+ cm_ptr->msg.saddr.ib.lid,
+ NULL) != DAT_SUCCESS) {
+ dapl_log(DAPL_DBG_TYPE_ERR,
+ " ACCEPT_USR: QPS_RTS ERR %s -> %s\n",
+ strerror(errno),
+ inet_ntoa(((struct sockaddr_in *)
+ &cm_ptr->msg.daddr.so)->sin_addr));
+ dapl_os_unlock(&ep_ptr->header.lock);
+ goto bail;
+ }
+ dapl_os_unlock(&ep_ptr->header.lock);
+
+ /* save remote address information */
+ dapl_os_memcpy(&ep_ptr->remote_ia_address,
+ &cm_ptr->msg.daddr.so,
+ sizeof(union dcm_addr));
+
+ /* send our QP info, IA address, pdata. Don't overwrite dst data */
+ local.ver = htons(DCM_VER);
+ local.op = htons(DCM_REP);
+ local.saddr.ib.qpn = htonl(ep_ptr->qp_handle->qp_num);
+ local.saddr.ib.qp_type = ep_ptr->qp_handle->qp_type;
+ local.saddr.ib.lid = ia_ptr->hca_ptr->ib_trans.lid;
+ dapl_os_memcpy(&local.saddr.ib.gid[0],
+ &ia_ptr->hca_ptr->ib_trans.gid, 16);
+
+ /* Get local address information from socket */
+ sl = sizeof(local.daddr.so);
+ getsockname(cm_ptr->socket, (struct sockaddr *)&local.daddr.so, &sl);
+
+#ifdef DAPL_DBG
+ /* DBG: Active PID [0], PASSIVE PID [2] */
+ *(uint16_t*)&cm_ptr->msg.resv[2] = htons((uint16_t)dapl_os_getpid());
+ dapl_os_memcpy(local.resv, cm_ptr->msg.resv, 4);
+#endif
+
+ cm_ptr->ep = ep_ptr;
+ cm_ptr->hca = ia_ptr->hca_ptr;
+ cm_ptr->state = DCM_ACCEPTED;
+
+ local.p_size = htons(p_size);
+ iov[0].iov_base = (void *)&local;
+ iov[0].iov_len = exp;
+
+ if (p_size) {
+ iov[1].iov_base = p_data;
+ iov[1].iov_len = p_size;
+ len = writev(cm_ptr->socket, iov, 2);
+ } else
+ len = writev(cm_ptr->socket, iov, 1);
+
+ if (len != (p_size + exp)) {
+ dapl_log(DAPL_DBG_TYPE_ERR,
+ " ACCEPT_USR: ERR %s, wcnt=%d -> %s\n",
+ strerror(errno), len,
+ inet_ntoa(((struct sockaddr_in *)
+ &cm_ptr->msg.daddr.so)->sin_addr));
+ cm_ptr->ep = NULL;
+ goto bail;
+ }
+
+ dapl_dbg_log(DAPL_DBG_TYPE_CM,
+ " ACCEPT_USR: local lid=0x%x qpn=0x%x psz=%d\n",
+ ntohs(local.saddr.ib.lid),
+ ntohl(local.saddr.ib.qpn), ntohs(local.p_size));
+ dapl_dbg_log(DAPL_DBG_TYPE_CM,
+ " ACCEPT_USR: SRC GID subnet %016llx id %016llx\n",
+ (unsigned long long)
+ htonll(*(uint64_t*)&local.saddr.ib.gid[0]),
+ (unsigned long long)
+ htonll(*(uint64_t*)&local.saddr.ib.gid[8]));
+
+ dapl_dbg_log(DAPL_DBG_TYPE_EP, " PASSIVE: accepted!\n");
+ return DAT_SUCCESS;
+bail:
+ dapls_ib_cm_free(cm_ptr, NULL);
+ return ret;
+}
+
+/*
+ * PASSIVE: read RTU from active peer, post CONN event
+ */
+static void dapli_socket_accept_rtu(dp_ib_cm_handle_t cm_ptr)
+{
+ int len;
+ ib_cm_events_t event = IB_CME_CONNECTED;
+
+ /* complete handshake after final QP state change, VER and OP */
+ len = recv(cm_ptr->socket, (char *)&cm_ptr->msg, 4, 0);
+ if (len != 4 || ntohs(cm_ptr->msg.op) != DCM_RTU) {
+ dapl_log(DAPL_DBG_TYPE_ERR,
+ " ACCEPT_RTU: rcv ERR, rcnt=%d op=%x\n",
+ len, ntohs(cm_ptr->msg.op),
+ inet_ntoa(((struct sockaddr_in *)
+ &cm_ptr->msg.daddr.so)->sin_addr));
+ event = IB_CME_DESTINATION_REJECT;
+ goto bail;
+ }
+
+ /* save state and reference to EP, queue for disc event */
+ cm_ptr->state = DCM_CONNECTED;
+
+ /* final data exchange if remote QP state is good to go */
+ dapl_dbg_log(DAPL_DBG_TYPE_EP, " PASSIVE: connected!\n");
+
+#ifdef DAT_EXTENSIONS
+ud_bail:
+ if (cm_ptr->msg.saddr.ib.qp_type == IBV_QPT_UD) {
+ DAT_IB_EXTENSION_EVENT_DATA xevent;
+
+ ib_pd_handle_t pd_handle =
+ ((DAPL_PZ *)cm_ptr->ep->param.pz_handle)->pd_handle;
+
+ if (event == IB_CME_CONNECTED) {
+ cm_ptr->ah = dapls_create_ah(cm_ptr->hca, pd_handle,
+ cm_ptr->ep->qp_handle,
+ cm_ptr->msg.saddr.ib.lid,
+ NULL);
+ if (cm_ptr->ah) {
+ /* post EVENT, modify_qp created ah */
+ xevent.status = 0;
+ xevent.type = DAT_IB_UD_PASSIVE_REMOTE_AH;
+ xevent.remote_ah.ah = cm_ptr->ah;
+ xevent.remote_ah.qpn = ntohl(cm_ptr->msg.saddr.ib.qpn);
+ dapl_os_memcpy(&xevent.remote_ah.ia_addr,
+ &cm_ptr->msg.daddr.so,
+ sizeof(union dcm_addr));
+ event = DAT_IB_UD_CONNECTION_EVENT_ESTABLISHED;
+ } else
+ event = DAT_IB_UD_CONNECTION_ERROR_EVENT;
+ } else
+ event = DAT_IB_UD_CONNECTION_ERROR_EVENT;
+
+ dapl_log(DAPL_DBG_TYPE_CM,
+ " CONN_RTU: UD AH %p for lid 0x%x qpn 0x%x\n",
+ cm_ptr->ah, ntohs(cm_ptr->msg.saddr.ib.lid),
+ ntohl(cm_ptr->msg.saddr.ib.qpn));
+
+ dapls_evd_post_connection_event_ext(
+ (DAPL_EVD *)
+ cm_ptr->ep->param.connect_evd_handle,
+ event,
+ (DAT_EP_HANDLE) cm_ptr->ep,
+ (DAT_COUNT) ntohs(cm_ptr->msg.p_size),
+ (DAT_PVOID *) cm_ptr->msg.p_data,
+ (DAT_PVOID *) &xevent);
+
+ /* done with socket, don't destroy cm_ptr, need pdata */
+ closesocket(cm_ptr->socket);
+ cm_ptr->socket = DAPL_INVALID_SOCKET;
+ cm_ptr->state = DCM_RELEASED;
+ } else
+#endif
+ {
+ cm_ptr->ep->cm_handle = cm_ptr; /* only RC, multi CR's on UD */
+ dapls_cr_callback(cm_ptr, event, NULL, cm_ptr->sp);
+ }
+ return;
+
+bail:
+#ifdef DAT_EXTENSIONS
+ if (cm_ptr->msg.saddr.ib.qp_type == IBV_QPT_UD)
+ goto ud_bail;
+#endif
+ dapls_modify_qp_state(cm_ptr->ep->qp_handle, IBV_QPS_ERR, 0, 0, 0);
+ dapls_ib_cm_free(cm_ptr, cm_ptr->ep);
+ dapls_cr_callback(cm_ptr, event, NULL, cm_ptr->sp);
+}
+
+/*
+ * dapls_ib_connect
+ *
+ * Initiate a connection with the passive listener on another node
+ *
+ * Input:
+ * ep_handle,
+ * remote_ia_address,
+ * remote_conn_qual,
+ * prd_size size of private data and structure
+ * prd_prt pointer to private data structure
+ *
+ * Output:
+ * none
+ *
+ * Returns:
+ * DAT_SUCCESS
+ * DAT_INSUFFICIENT_RESOURCES
+ * DAT_INVALID_PARAMETER
+ *
+ */
+DAT_RETURN
+dapls_ib_connect(IN DAT_EP_HANDLE ep_handle,
+ IN DAT_IA_ADDRESS_PTR remote_ia_address,
+ IN DAT_CONN_QUAL remote_conn_qual,
+ IN DAT_COUNT private_data_size, IN void *private_data)
+{
+ DAPL_EP *ep_ptr;
+ ib_qp_handle_t qp_ptr;
+
+ dapl_dbg_log(DAPL_DBG_TYPE_EP,
+ " connect(ep_handle %p ....)\n", ep_handle);
+
+ ep_ptr = (DAPL_EP *) ep_handle;
+ qp_ptr = ep_ptr->qp_handle;
+
+ return (dapli_socket_connect(ep_ptr, remote_ia_address,
+ remote_conn_qual,
+ private_data_size, private_data));
+}
+
+/*
+ * dapls_ib_disconnect
+ *
+ * Disconnect an EP
+ *
+ * Input:
+ * ep_handle,
+ * disconnect_flags
+ *
+ * Output:
+ * none
+ *
+ * Returns:
+ * DAT_SUCCESS
+ */
+DAT_RETURN
+dapls_ib_disconnect(IN DAPL_EP * ep_ptr, IN DAT_CLOSE_FLAGS close_flags)
+{
+ dapl_dbg_log(DAPL_DBG_TYPE_EP,
+ "dapls_ib_disconnect(ep_handle %p ....)\n", ep_ptr);
+
+ /* Transition to error state to flush queue */
+ dapls_modify_qp_state(ep_ptr->qp_handle, IBV_QPS_ERR, 0, 0, 0);
+
+ if (ep_ptr->cm_handle == NULL ||
+ ep_ptr->param.ep_state == DAT_EP_STATE_DISCONNECTED)
+ return DAT_SUCCESS;
+ else
+ return (dapli_socket_disconnect(ep_ptr->cm_handle));
+}
+
+/*
+ * dapls_ib_disconnect_clean
+ *
+ * Clean up outstanding connection data. This routine is invoked
+ * after the final disconnect callback has occurred. Only on the
+ * ACTIVE side of a connection. It is also called if dat_ep_connect
+ * times out using the consumer supplied timeout value.
+ *
+ * Input:
+ * ep_ptr DAPL_EP
+ * active Indicates active side of connection
+ *
+ * Output:
+ * none
+ *
+ * Returns:
+ * void
+ *
+ */
+void
+dapls_ib_disconnect_clean(IN DAPL_EP * ep_ptr,
+ IN DAT_BOOLEAN active,
+ IN const ib_cm_events_t ib_cm_event)
+{
+ /* NOTE: SCM will only initialize cm_handle with RC type
+ *
+ * For UD there can many in-flight CR's so you
+ * cannot cleanup timed out CR's with EP reference
+ * alone since they share the same EP. The common
+ * code that handles connection timeout logic needs
+ * updated for UD support.
+ */
+ if (ep_ptr->cm_handle)
+ dapls_ib_cm_free(ep_ptr->cm_handle, ep_ptr);
+
+ return;
+}
+
+/*
+ * dapl_ib_setup_conn_listener
+ *
+ * Have the CM set up a connection listener.
+ *
+ * Input:
+ * ibm_hca_handle HCA handle
+ * qp_handle QP handle
+ *
+ * Output:
+ * none
+ *
+ * Returns:
+ * DAT_SUCCESS
+ * DAT_INSUFFICIENT_RESOURCES
+ * DAT_INTERNAL_ERROR
+ * DAT_CONN_QUAL_UNAVAILBLE
+ * DAT_CONN_QUAL_IN_USE
+ *
+ */
+DAT_RETURN
+dapls_ib_setup_conn_listener(IN DAPL_IA * ia_ptr,
+ IN DAT_UINT64 ServiceID, IN DAPL_SP * sp_ptr)
+{
+ return (dapli_socket_listen(ia_ptr, ServiceID, sp_ptr));
+}
+
+/*
+ * dapl_ib_remove_conn_listener
+ *
+ * Have the CM remove a connection listener.
+ *
+ * Input:
+ * ia_handle IA handle
+ * ServiceID IB Channel Service ID
+ *
+ * Output:
+ * none
+ *
+ * Returns:
+ * DAT_SUCCESS
+ * DAT_INVALID_STATE
+ *
+ */
+DAT_RETURN
+dapls_ib_remove_conn_listener(IN DAPL_IA * ia_ptr, IN DAPL_SP * sp_ptr)
+{
+ ib_cm_srvc_handle_t cm_ptr = sp_ptr->cm_srvc_handle;
+
+ dapl_dbg_log(DAPL_DBG_TYPE_EP,
+ "dapls_ib_remove_conn_listener(ia_ptr %p sp_ptr %p cm_ptr %p)\n",
+ ia_ptr, sp_ptr, cm_ptr);
+
+ /* close accepted socket, free cm_srvc_handle and return */
+ if (cm_ptr != NULL) {
+ /* cr_thread will free */
+ dapl_os_lock(&cm_ptr->lock);
+ cm_ptr->state = DCM_DESTROY;
+ sp_ptr->cm_srvc_handle = NULL;
+ send(cm_ptr->hca->ib_trans.scm[1], "w", sizeof "w", 0);
+ dapl_os_unlock(&cm_ptr->lock);
+ }
+ return DAT_SUCCESS;
+}
+
+/*
+ * dapls_ib_accept_connection
+ *
+ * Perform necessary steps to accept a connection
+ *
+ * Input:
+ * cr_handle
+ * ep_handle
+ * private_data_size
+ * private_data
+ *
+ * Output:
+ * none
+ *
+ * Returns:
+ * DAT_SUCCESS
+ * DAT_INSUFFICIENT_RESOURCES
+ * DAT_INTERNAL_ERROR
+ *
+ */
+DAT_RETURN
+dapls_ib_accept_connection(IN DAT_CR_HANDLE cr_handle,
+ IN DAT_EP_HANDLE ep_handle,
+ IN DAT_COUNT p_size, IN const DAT_PVOID p_data)
+{
+ DAPL_CR *cr_ptr;
+ DAPL_EP *ep_ptr;
+
+ dapl_dbg_log(DAPL_DBG_TYPE_EP,
+ "dapls_ib_accept_connection(cr %p ep %p prd %p,%d)\n",
+ cr_handle, ep_handle, p_data, p_size);
+
+ cr_ptr = (DAPL_CR *) cr_handle;
+ ep_ptr = (DAPL_EP *) ep_handle;
+
+ /* allocate and attach a QP if necessary */
+ if (ep_ptr->qp_state == DAPL_QP_STATE_UNATTACHED) {
+ DAT_RETURN status;
+ status = dapls_ib_qp_alloc(ep_ptr->header.owner_ia,
+ ep_ptr, ep_ptr);
+ if (status != DAT_SUCCESS)
+ return status;
+ }
+ return (dapli_socket_accept_usr(ep_ptr, cr_ptr, p_size, p_data));
+}
+
+/*
+ * dapls_ib_reject_connection
+ *
+ * Reject a connection
+ *
+ * Input:
+ * cr_handle
+ *
+ * Output:
+ * none
+ *
+ * Returns:
+ * DAT_SUCCESS
+ * DAT_INTERNAL_ERROR
+ *
+ */
+DAT_RETURN
+dapls_ib_reject_connection(IN dp_ib_cm_handle_t cm_ptr,
+ IN int reason,
+ IN DAT_COUNT psize, IN const DAT_PVOID pdata)
+{
+ struct iovec iov[2];
+
+ dapl_dbg_log(DAPL_DBG_TYPE_EP,
+ " reject(cm %p reason %x, pdata %p, psize %d)\n",
+ cm_ptr, reason, pdata, psize);
+
+ if (psize > DCM_MAX_PDATA_SIZE)
+ return DAT_LENGTH_ERROR;
+
+ dapl_os_lock(&cm_ptr->lock);
+
+ /* write reject data to indicate reject */
+ cm_ptr->msg.op = htons(DCM_REJ_USER);
+ cm_ptr->msg.p_size = htons(psize);
+
+ iov[0].iov_base = (void *)&cm_ptr->msg;
+ iov[0].iov_len = sizeof(ib_cm_msg_t) - DCM_MAX_PDATA_SIZE;
+ if (psize) {
+ iov[1].iov_base = pdata;
+ iov[1].iov_len = psize;
+ writev(cm_ptr->socket, iov, 2);
+ } else {
+ writev(cm_ptr->socket, iov, 1);
+ }
+
+ /* cr_thread will destroy CR */
+ cm_ptr->state = DCM_DESTROY;
+ send(cm_ptr->hca->ib_trans.scm[1], "w", sizeof "w", 0);
+ dapl_os_unlock(&cm_ptr->lock);
+ return DAT_SUCCESS;
+}
+
+/*
+ * dapls_ib_cm_remote_addr
+ *
+ * Obtain the remote IP address given a connection
+ *
+ * Input:
+ * cr_handle
+ *
+ * Output:
+ * remote_ia_address: where to place the remote address
+ *
+ * Returns:
+ * DAT_SUCCESS
+ * DAT_INVALID_HANDLE
+ *
+ */
+DAT_RETURN
+dapls_ib_cm_remote_addr(IN DAT_HANDLE dat_handle,
+ OUT DAT_SOCK_ADDR6 * remote_ia_address)
+{
+ DAPL_HEADER *header;
+ dp_ib_cm_handle_t ib_cm_handle;
+
+ dapl_dbg_log(DAPL_DBG_TYPE_EP,
+ "dapls_ib_cm_remote_addr(dat_handle %p, ....)\n",
+ dat_handle);
+
+ header = (DAPL_HEADER *) dat_handle;
+
+ if (header->magic == DAPL_MAGIC_EP)
+ ib_cm_handle = ((DAPL_EP *) dat_handle)->cm_handle;
+ else if (header->magic == DAPL_MAGIC_CR)
+ ib_cm_handle = ((DAPL_CR *) dat_handle)->ib_cm_handle;
+ else
+ return DAT_INVALID_HANDLE;
+
+ dapl_os_memcpy(remote_ia_address,
+ &ib_cm_handle->msg.daddr.so, sizeof(DAT_SOCK_ADDR6));
+
+ return DAT_SUCCESS;
+}
+
+/*
+ * dapls_ib_private_data_size
+ *
+ * Return the size of private data given a connection op type
+ *
+ * Input:
+ * prd_ptr private data pointer
+ * conn_op connection operation type
+ *
+ * If prd_ptr is NULL, this is a query for the max size supported by
+ * the provider, otherwise it is the actual size of the private data
+ * contained in prd_ptr.
+ *
+ *
+ * Output:
+ * None
+ *
+ * Returns:
+ * length of private data
+ *
+ */
+int dapls_ib_private_data_size(IN DAPL_PRIVATE * prd_ptr,
+ IN DAPL_PDATA_OP conn_op, IN DAPL_HCA * hca_ptr)
+{
+ int size;
+
+ switch (conn_op) {
+ case DAPL_PDATA_CONN_REQ:
+ case DAPL_PDATA_CONN_REP:
+ case DAPL_PDATA_CONN_REJ:
+ case DAPL_PDATA_CONN_DREQ:
+ case DAPL_PDATA_CONN_DREP:
+ size = DCM_MAX_PDATA_SIZE;
+ break;
+ default:
+ size = 0;
+ }
+ return size;
+}
+
+/*
+ * Map all socket CM event codes to the DAT equivelent.
+ */
+#define DAPL_IB_EVENT_CNT 10
+
+static struct ib_cm_event_map {
+ const ib_cm_events_t ib_cm_event;
+ DAT_EVENT_NUMBER dat_event_num;
+} ib_cm_event_map[DAPL_IB_EVENT_CNT] = {
+/* 00 */ {IB_CME_CONNECTED,
+ DAT_CONNECTION_EVENT_ESTABLISHED},
+/* 01 */ {IB_CME_DISCONNECTED,
+ DAT_CONNECTION_EVENT_DISCONNECTED},
+/* 02 */ {IB_CME_DISCONNECTED_ON_LINK_DOWN,
+ DAT_CONNECTION_EVENT_DISCONNECTED},
+/* 03 */ {IB_CME_CONNECTION_REQUEST_PENDING,
+ DAT_CONNECTION_REQUEST_EVENT},
+/* 04 */ {IB_CME_CONNECTION_REQUEST_PENDING_PRIVATE_DATA,
+ DAT_CONNECTION_REQUEST_EVENT},
+/* 05 */ {IB_CME_DESTINATION_REJECT,
+ DAT_CONNECTION_EVENT_NON_PEER_REJECTED},
+/* 06 */ {IB_CME_DESTINATION_REJECT_PRIVATE_DATA,
+ DAT_CONNECTION_EVENT_PEER_REJECTED},
+/* 07 */ {IB_CME_DESTINATION_UNREACHABLE,
+ DAT_CONNECTION_EVENT_UNREACHABLE},
+/* 08 */ {IB_CME_TOO_MANY_CONNECTION_REQUESTS,
+ DAT_CONNECTION_EVENT_NON_PEER_REJECTED},
+/* 09 */ {IB_CME_LOCAL_FAILURE,
+ DAT_CONNECTION_EVENT_BROKEN}
+};
+
+/*
+ * dapls_ib_get_cm_event
+ *
+ * Return a DAT connection event given a provider CM event.
+ *
+ * Input:
+ * dat_event_num DAT event we need an equivelent CM event for
+ *
+ * Output:
+ * none
+ *
+ * Returns:
+ * ib_cm_event of translated DAPL value
+ */
+DAT_EVENT_NUMBER
+dapls_ib_get_dat_event(IN const ib_cm_events_t ib_cm_event,
+ IN DAT_BOOLEAN active)
+{
+ DAT_EVENT_NUMBER dat_event_num;
+ int i;
+
+ active = active;
+
+ if (ib_cm_event > IB_CME_LOCAL_FAILURE)
+ return (DAT_EVENT_NUMBER) 0;
+
+ dat_event_num = 0;
+ for (i = 0; i < DAPL_IB_EVENT_CNT; i++) {
+ if (ib_cm_event == ib_cm_event_map[i].ib_cm_event) {
+ dat_event_num = ib_cm_event_map[i].dat_event_num;
+ break;
+ }
+ }
+ dapl_dbg_log(DAPL_DBG_TYPE_CALLBACK,
+ "dapls_ib_get_dat_event: event translate(%s) ib=0x%x dat=0x%x\n",
+ active ? "active" : "passive", ib_cm_event, dat_event_num);
+
+ return dat_event_num;
+}
+
+/*
+ * dapls_ib_get_dat_event
+ *
+ * Return a DAT connection event given a provider CM event.
+ *
+ * Input:
+ * ib_cm_event event provided to the dapl callback routine
+ * active switch indicating active or passive connection
+ *
+ * Output:
+ * none
+ *
+ * Returns:
+ * DAT_EVENT_NUMBER of translated provider value
+ */
+ib_cm_events_t dapls_ib_get_cm_event(IN DAT_EVENT_NUMBER dat_event_num)
+{
+ ib_cm_events_t ib_cm_event;
+ int i;
+
+ ib_cm_event = 0;
+ for (i = 0; i < DAPL_IB_EVENT_CNT; i++) {
+ if (dat_event_num == ib_cm_event_map[i].dat_event_num) {
+ ib_cm_event = ib_cm_event_map[i].ib_cm_event;
+ break;
+ }
+ }
+ return ib_cm_event;
+}
+
+/* outbound/inbound CR processing thread to avoid blocking applications */
+void cr_thread(void *arg)
+{
+ struct dapl_hca *hca_ptr = arg;
+ dp_ib_cm_handle_t cr, next_cr;
+ int opt, ret;
+ socklen_t opt_len;
+ char rbuf[2];
+ struct dapl_fd_set *set;
+ enum DAPL_FD_EVENTS event;
+
+ dapl_dbg_log(DAPL_DBG_TYPE_UTIL, " cr_thread: ENTER hca %p\n", hca_ptr);
+ set = dapl_alloc_fd_set();
+ if (!set)
+ goto out;
+
+ dapl_os_lock(&hca_ptr->ib_trans.lock);
+ hca_ptr->ib_trans.cr_state = IB_THREAD_RUN;
+
+ while (1) {
+ dapl_fd_zero(set);
+ dapl_fd_set(hca_ptr->ib_trans.scm[0], set, DAPL_FD_READ);
+
+ if (!dapl_llist_is_empty(&hca_ptr->ib_trans.list))
+ next_cr = dapl_llist_peek_head(&hca_ptr->ib_trans.list);
+ else
+ next_cr = NULL;
+
+ while (next_cr) {
+ cr = next_cr;
+ next_cr = dapl_llist_next_entry(&hca_ptr->ib_trans.list,
+ (DAPL_LLIST_ENTRY *) &
+ cr->entry);
+ dapl_os_lock(&cr->lock);
+ if (cr->state == DCM_DESTROY
+ || hca_ptr->ib_trans.cr_state != IB_THREAD_RUN) {
+ dapl_os_unlock(&cr->lock);
+ dapl_llist_remove_entry(&hca_ptr->ib_trans.list,
+ (DAPL_LLIST_ENTRY *) &
+ cr->entry);
+ dapl_dbg_log(DAPL_DBG_TYPE_CM,
+ " CR FREE: %p ep=%p st=%d sock=%d\n",
+ cr, cr->ep, cr->state, cr->socket);
+
+ if (cr->socket != DAPL_INVALID_SOCKET) {
+ shutdown(cr->socket, SHUT_RDWR);
+ closesocket(cr->socket);
+ }
+ dapl_os_free(cr, sizeof(*cr));
+ continue;
+ }
+ if (cr->socket == DAPL_INVALID_SOCKET) {
+ dapl_os_unlock(&cr->lock);
+ continue;
+ }
+
+ event = (cr->state == DCM_CONN_PENDING) ?
+ DAPL_FD_WRITE : DAPL_FD_READ;
+
+ if (dapl_fd_set(cr->socket, set, event)) {
+ dapl_log(DAPL_DBG_TYPE_ERR,
+ " cr_thread: DESTROY CR st=%d fd %d"
+ " -> %s\n", cr->state, cr->socket,
+ inet_ntoa(((struct sockaddr_in *)
+ &cr->msg.daddr.so)->sin_addr));
+ dapl_os_unlock(&cr->lock);
+ dapls_ib_cm_free(cr, cr->ep);
+ continue;
+ }
+ dapl_os_unlock(&cr->lock);
+ dapl_dbg_log(DAPL_DBG_TYPE_CM,
+ " poll cr=%p, sck=%d\n", cr, cr->socket);
+ dapl_os_unlock(&hca_ptr->ib_trans.lock);
+
+ ret = dapl_poll(cr->socket, event);
+
+ dapl_dbg_log(DAPL_DBG_TYPE_CM,
+ " poll ret=0x%x cr->state=%d sck=%d\n",
+ ret, cr->state, cr->socket);
+
+ /* data on listen, qp exchange, and on disc req */
+ if ((ret == DAPL_FD_READ) ||
+ (cr->state != DCM_CONN_PENDING &&
+ ret == DAPL_FD_ERROR)) {
+ if (cr->socket != DAPL_INVALID_SOCKET) {
+ switch (cr->state) {
+ case DCM_LISTEN:
+ dapli_socket_accept(cr);
+ break;
+ case DCM_ACCEPTING:
+ dapli_socket_accept_data(cr);
+ break;
+ case DCM_ACCEPTED:
+ dapli_socket_accept_rtu(cr);
+ break;
+ case DCM_REP_PENDING:
+ dapli_socket_connect_rtu(cr);
+ break;
+ case DCM_CONNECTED:
+ dapli_socket_disconnect(cr);
+ break;
+ default:
+ break;
+ }
+ }
+ /* ASYNC connections, writable, readable, error; check status */
+ } else if (ret == DAPL_FD_WRITE ||
+ (cr->state == DCM_CONN_PENDING &&
+ ret == DAPL_FD_ERROR)) {
+
+ if (ret == DAPL_FD_ERROR)
+ dapl_log(DAPL_DBG_TYPE_ERR, " CONN_PENDING - FD_ERROR\n");
+
+ opt = 0;
+ opt_len = sizeof(opt);
+ ret = getsockopt(cr->socket, SOL_SOCKET,
+ SO_ERROR, (char *)&opt,
+ &opt_len);
+ if (!ret && !opt)
+ dapli_socket_connected(cr, opt);
+ else
+ dapli_socket_connected(cr, opt ? opt : dapl_socket_errno());
+ }
+ dapl_os_lock(&hca_ptr->ib_trans.lock);
+ }
+
+ /* set to exit and all resources destroyed */
+ if ((hca_ptr->ib_trans.cr_state != IB_THREAD_RUN) &&
+ (dapl_llist_is_empty(&hca_ptr->ib_trans.list)))
+ break;
+
+ dapl_os_unlock(&hca_ptr->ib_trans.lock);
+ dapl_select(set);
+
+ /* if pipe used to wakeup, consume */
+ while (dapl_poll(hca_ptr->ib_trans.scm[0],
+ DAPL_FD_READ) == DAPL_FD_READ) {
+ if (recv(hca_ptr->ib_trans.scm[0], rbuf, 2, 0) == -1)
+ dapl_log(DAPL_DBG_TYPE_CM,
+ " cr_thread: read pipe error = %s\n",
+ strerror(errno));
+ }
+ dapl_os_lock(&hca_ptr->ib_trans.lock);
+
+ /* set to exit and all resources destroyed */
+ if ((hca_ptr->ib_trans.cr_state != IB_THREAD_RUN) &&
+ (dapl_llist_is_empty(&hca_ptr->ib_trans.list)))
+ break;
+ }
+
+ dapl_os_unlock(&hca_ptr->ib_trans.lock);
+ dapl_os_free(set, sizeof(struct dapl_fd_set));
+out:
+ hca_ptr->ib_trans.cr_state = IB_THREAD_EXIT;
+ dapl_dbg_log(DAPL_DBG_TYPE_UTIL, " cr_thread(hca %p) exit\n", hca_ptr);
+}
+
+
+#ifdef DAPL_COUNTERS
+/* Debug aid: List all Connections in process and state */
+void dapls_print_cm_list(IN DAPL_IA *ia_ptr)
+{
+ /* Print in process CR's for this IA, if debug type set */
+ int i = 0;
+ dp_ib_cm_handle_t cr, next_cr;
+
+ dapl_os_lock(&ia_ptr->hca_ptr->ib_trans.lock);
+ if (!dapl_llist_is_empty((DAPL_LLIST_HEAD*)
+ &ia_ptr->hca_ptr->ib_trans.list))
+ next_cr = dapl_llist_peek_head((DAPL_LLIST_HEAD*)
+ &ia_ptr->hca_ptr->ib_trans.list);
+ else
+ next_cr = NULL;
+
+ printf("\n DAPL IA CONNECTIONS IN PROCESS:\n");
+ while (next_cr) {
+ cr = next_cr;
+ next_cr = dapl_llist_next_entry((DAPL_LLIST_HEAD*)
+ &ia_ptr->hca_ptr->ib_trans.list,
+ (DAPL_LLIST_ENTRY*)&cr->entry);
+
+ printf( " CONN[%d]: sp %p ep %p sock %d %s %s %s %s %s %s PORT L-%x R-%x PID L-%x R-%x\n",
+ i, cr->sp, cr->ep, cr->socket,
+ cr->msg.saddr.ib.qp_type == IBV_QPT_RC ? "RC" : "UD",
+ dapl_cm_state_str(cr->state), dapl_cm_op_str(ntohs(cr->msg.op)),
+ ntohs(cr->msg.op) == DCM_REQ ? /* local address */
+ inet_ntoa(((struct sockaddr_in *)&cr->msg.daddr.so)->sin_addr) :
+ inet_ntoa(((struct sockaddr_in *)&cr->addr)->sin_addr),
+ cr->sp ? "<-" : "->",
+ ntohs(cr->msg.op) == DCM_REQ ? /* remote address */
+ inet_ntoa(((struct sockaddr_in *)&cr->addr)->sin_addr) :
+ inet_ntoa(((struct sockaddr_in *)&cr->msg.daddr.so)->sin_addr),
+
+ ntohs(cr->msg.op) == DCM_REQ ? /* local port */
+ ntohs(((struct sockaddr_in *)&cr->msg.daddr.so)->sin_port) :
+ ntohs(((struct sockaddr_in *)&cr->addr)->sin_port),
+
+ ntohs(cr->msg.op) == DCM_REQ ? /* remote port */
+ ntohs(((struct sockaddr_in *)&cr->addr)->sin_port) :
+ ntohs(((struct sockaddr_in *)&cr->msg.daddr.so)->sin_port),
+
+ cr->sp ? ntohs(*(uint16_t*)&cr->msg.resv[2]) : ntohs(*(uint16_t*)&cr->msg.resv[0]),
+ cr->sp ? ntohs(*(uint16_t*)&cr->msg.resv[0]) : ntohs(*(uint16_t*)&cr->msg.resv[2]));
+
+ i++;
+ }
+ printf("\n");
+ dapl_os_unlock(&ia_ptr->hca_ptr->ib_trans.lock);
+}
+#endif
-/*\r
- * This Software is licensed under one of the following licenses:\r
- *\r
- * 1) under the terms of the "Common Public License 1.0" a copy of which is\r
- * available from the Open Source Initiative, see\r
- * http://www.opensource.org/licenses/cpl.php.\r
- *\r
- * 2) under the terms of the "The BSD License" a copy of which is\r
- * available from the Open Source Initiative, see\r
- * http://www.opensource.org/licenses/bsd-license.php.\r
- *\r
- * 3) under the terms of the "GNU General Public License (GPL) Version 2" a\r
- * copy of which is available from the Open Source Initiative, see\r
- * http://www.opensource.org/licenses/gpl-license.php.\r
- *\r
- * Licensee has the right to choose one of the above licenses.\r
- *\r
- * Redistributions of source code must retain the above copyright\r
- * notice and one of the license notices.\r
- *\r
- * Redistributions in binary form must reproduce both the above copyright\r
- * notice, one of the license notices in the documentation\r
- * and/or other materials provided with the distribution.\r
- */\r
-\r
-#ifndef _DAPL_IB_UTIL_H_\r
-#define _DAPL_IB_UTIL_H_\r
-#define _OPENIB_SCM_ \r
-\r
-#include <infiniband/verbs.h>\r
-#include "openib_osd.h"\r
-#include "dapl_ib_common.h"\r
-\r
-struct ib_cm_handle\r
-{ \r
- struct dapl_llist_entry entry;\r
- DAPL_OS_LOCK lock;\r
- int state;\r
- DAPL_SOCKET socket;\r
- struct dapl_hca *hca;\r
- struct dapl_sp *sp; \r
- struct dapl_ep *ep;\r
- ib_cm_msg_t msg;\r
- struct ibv_ah *ah;\r
- DAT_SOCK_ADDR6 addr; \r
-};\r
-\r
-typedef struct ib_cm_handle *dp_ib_cm_handle_t;\r
-typedef dp_ib_cm_handle_t ib_cm_srvc_handle_t;\r
-\r
-/* Definitions */\r
-#define IB_INVALID_HANDLE NULL\r
-\r
-/* inline send rdma threshold */\r
-#define INLINE_SEND_DEFAULT 200\r
-\r
-/* RC timer - retry count defaults */\r
-#define SCM_ACK_TIMER 16 /* 5 bits, 4.096us*2^ack_timer. 16== 268ms */\r
-#define SCM_ACK_RETRY 7 /* 3 bits, 7 * 268ms = 1.8 seconds */\r
-#define SCM_RNR_TIMER 12 /* 5 bits, 12 =.64ms, 28 =163ms, 31 =491ms */\r
-#define SCM_RNR_RETRY 7 /* 3 bits, 7 == infinite */\r
-#define SCM_IB_MTU 2048\r
-\r
-/* Global routing defaults */\r
-#define SCM_GLOBAL 0 /* global routing is disabled */\r
-#define SCM_HOP_LIMIT 0xff\r
-#define SCM_TCLASS 0\r
-\r
-/* ib_hca_transport_t, specific to this implementation */\r
-typedef struct _ib_hca_transport\r
-{ \r
- struct dapl_llist_entry entry;\r
- int destroy;\r
- union ibv_gid gid;\r
- struct ibv_device *ib_dev;\r
- struct ibv_context *ib_ctx;\r
- ib_cq_handle_t ib_cq_empty;\r
- DAPL_OS_LOCK cq_lock; \r
- int max_inline_send;\r
- ib_thread_state_t cq_state;\r
- DAPL_OS_THREAD cq_thread;\r
- struct ibv_comp_channel *ib_cq;\r
- int cr_state;\r
- DAPL_OS_THREAD thread;\r
- DAPL_OS_LOCK lock; \r
- struct dapl_llist_entry *list; \r
- ib_async_handler_t async_unafiliated;\r
- void *async_un_ctx;\r
- ib_async_cq_handler_t async_cq_error;\r
- ib_async_dto_handler_t async_cq;\r
- ib_async_qp_handler_t async_qp_error;\r
- int rd_atom_in;\r
- int rd_atom_out;\r
- uint16_t lid;\r
- uint8_t ack_timer;\r
- uint8_t ack_retry;\r
- uint8_t rnr_timer;\r
- uint8_t rnr_retry;\r
- uint8_t global;\r
- uint8_t hop_limit;\r
- uint8_t tclass;\r
- uint8_t mtu;\r
- DAT_NAMED_ATTR named_attr;\r
- DAPL_SOCKET scm[2];\r
-} ib_hca_transport_t;\r
-\r
-/* prototypes */\r
-void cr_thread(void *arg);\r
-int dapli_cq_thread_init(struct dapl_hca *hca_ptr);\r
-void dapli_cq_thread_destroy(struct dapl_hca *hca_ptr);\r
-void dapli_async_event_cb(struct _ib_hca_transport *tp);\r
-void dapli_cq_event_cb(struct _ib_hca_transport *tp);\r
-DAT_RETURN dapli_socket_disconnect(dp_ib_cm_handle_t cm_ptr);\r
-dp_ib_cm_handle_t dapls_ib_cm_create(DAPL_EP *ep);\r
-void dapls_ib_cm_free(dp_ib_cm_handle_t cm, DAPL_EP *ep);\r
-void dapls_print_cm_list(IN DAPL_IA *ia_ptr);\r
-\r
-#endif /* _DAPL_IB_UTIL_H_ */\r
+/*
+ * This Software is licensed under one of the following licenses:
+ *
+ * 1) under the terms of the "Common Public License 1.0" a copy of which is
+ * available from the Open Source Initiative, see
+ * http://www.opensource.org/licenses/cpl.php.
+ *
+ * 2) under the terms of the "The BSD License" a copy of which is
+ * available from the Open Source Initiative, see
+ * http://www.opensource.org/licenses/bsd-license.php.
+ *
+ * 3) under the terms of the "GNU General Public License (GPL) Version 2" a
+ * copy of which is available from the Open Source Initiative, see
+ * http://www.opensource.org/licenses/gpl-license.php.
+ *
+ * Licensee has the right to choose one of the above licenses.
+ *
+ * Redistributions of source code must retain the above copyright
+ * notice and one of the license notices.
+ *
+ * Redistributions in binary form must reproduce both the above copyright
+ * notice, one of the license notices in the documentation
+ * and/or other materials provided with the distribution.
+ */
+
+#ifndef _DAPL_IB_UTIL_H_
+#define _DAPL_IB_UTIL_H_
+#define _OPENIB_SCM_
+
+#include <infiniband/verbs.h>
+#include "openib_osd.h"
+#include "dapl_ib_common.h"
+
+struct ib_cm_handle
+{
+ struct dapl_llist_entry entry;
+ DAPL_OS_LOCK lock;
+ int state;
+ DAPL_SOCKET socket;
+ struct dapl_hca *hca;
+ struct dapl_sp *sp;
+ struct dapl_ep *ep;
+ ib_cm_msg_t msg;
+ struct ibv_ah *ah;
+ DAT_SOCK_ADDR6 addr;
+};
+
+typedef struct ib_cm_handle *dp_ib_cm_handle_t;
+typedef dp_ib_cm_handle_t ib_cm_srvc_handle_t;
+
+/* Definitions */
+#define IB_INVALID_HANDLE NULL
+
+/* inline send rdma threshold */
+#define INLINE_SEND_DEFAULT 200
+
+/* RC timer - retry count defaults */
+#define SCM_ACK_TIMER 16 /* 5 bits, 4.096us*2^ack_timer. 16== 268ms */
+#define SCM_ACK_RETRY 7 /* 3 bits, 7 * 268ms = 1.8 seconds */
+#define SCM_RNR_TIMER 12 /* 5 bits, 12 =.64ms, 28 =163ms, 31 =491ms */
+#define SCM_RNR_RETRY 7 /* 3 bits, 7 == infinite */
+#define SCM_IB_MTU 2048
+
+/* Global routing defaults */
+#define SCM_GLOBAL 0 /* global routing is disabled */
+#define SCM_HOP_LIMIT 0xff
+#define SCM_TCLASS 0
+
+/* ib_hca_transport_t, specific to this implementation */
+typedef struct _ib_hca_transport
+{
+ struct dapl_llist_entry entry;
+ int destroy;
+ union ibv_gid gid;
+ struct ibv_device *ib_dev;
+ struct ibv_context *ib_ctx;
+ ib_cq_handle_t ib_cq_empty;
+ DAPL_OS_LOCK cq_lock;
+ int max_inline_send;
+ ib_thread_state_t cq_state;
+ DAPL_OS_THREAD cq_thread;
+ struct ibv_comp_channel *ib_cq;
+ int cr_state;
+ DAPL_OS_THREAD thread;
+ DAPL_OS_LOCK lock;
+ struct dapl_llist_entry *list;
+ ib_async_handler_t async_unafiliated;
+ void *async_un_ctx;
+ ib_async_cq_handler_t async_cq_error;
+ ib_async_dto_handler_t async_cq;
+ ib_async_qp_handler_t async_qp_error;
+ int rd_atom_in;
+ int rd_atom_out;
+ uint16_t lid;
+ uint8_t ack_timer;
+ uint8_t ack_retry;
+ uint8_t rnr_timer;
+ uint8_t rnr_retry;
+ uint8_t global;
+ uint8_t hop_limit;
+ uint8_t tclass;
+ uint8_t mtu;
+ DAT_NAMED_ATTR named_attr;
+ DAPL_SOCKET scm[2];
+} ib_hca_transport_t;
+
+/* prototypes */
+void cr_thread(void *arg);
+int dapli_cq_thread_init(struct dapl_hca *hca_ptr);
+void dapli_cq_thread_destroy(struct dapl_hca *hca_ptr);
+void dapli_async_event_cb(struct _ib_hca_transport *tp);
+void dapli_cq_event_cb(struct _ib_hca_transport *tp);
+DAT_RETURN dapli_socket_disconnect(dp_ib_cm_handle_t cm_ptr);
+dp_ib_cm_handle_t dapls_ib_cm_create(DAPL_EP *ep);
+void dapls_ib_cm_free(dp_ib_cm_handle_t cm, DAPL_EP *ep);
+void dapls_print_cm_list(IN DAPL_IA *ia_ptr);
+
+#endif /* _DAPL_IB_UTIL_H_ */
-/*\r
- * This Software is licensed under one of the following licenses:\r
- *\r
- * 1) under the terms of the "Common Public License 1.0" a copy of which is\r
- * available from the Open Source Initiative, see\r
- * http://www.opensource.org/licenses/cpl.php.\r
- *\r
- * 2) under the terms of the "The BSD License" a copy of which is\r
- * available from the Open Source Initiative, see\r
- * http://www.opensource.org/licenses/bsd-license.php.\r
- *\r
- * 3) under the terms of the "GNU General Public License (GPL) Version 2" a\r
- * copy of which is available from the Open Source Initiative, see\r
- * http://www.opensource.org/licenses/gpl-license.php.\r
- *\r
- * Licensee has the right to choose one of the above licenses.\r
- *\r
- * Redistributions of source code must retain the above copyright\r
- * notice and one of the license notices.\r
- *\r
- * Redistributions in binary form must reproduce both the above copyright\r
- * notice, one of the license notices in the documentation\r
- * and/or other materials provided with the distribution.\r
- */\r
-\r
-/***************************************************************************\r
- *\r
- * Module: uDAPL\r
- *\r
- * Filename: dapl_ib_util.c\r
- *\r
- * Author: Arlin Davis\r
- *\r
- * Created: 3/10/2005\r
- *\r
- * Description: \r
- *\r
- * The uDAPL openib provider - init, open, close, utilities\r
- *\r
- ****************************************************************************\r
- * Source Control System Information\r
- *\r
- * $Id: $\r
- *\r
- * Copyright (c) 2005 Intel Corporation. All rights reserved.\r
- *\r
- **************************************************************************/\r
-#ifdef RCSID\r
-static const char rcsid[] = "$Id: $";\r
-#endif\r
-\r
-#include "openib_osd.h"\r
-#include "dapl.h"\r
-#include "dapl_adapter_util.h"\r
-#include "dapl_ib_util.h"\r
-#include "dapl_osd.h"\r
-\r
-#include <stdlib.h>\r
-\r
-ib_thread_state_t g_ib_thread_state = 0;\r
-DAPL_OS_THREAD g_ib_thread;\r
-DAPL_OS_LOCK g_hca_lock;\r
-struct dapl_llist_entry *g_hca_list;\r
-\r
-void dapli_thread(void *arg);\r
-DAT_RETURN dapli_ib_thread_init(void);\r
-void dapli_ib_thread_destroy(void);\r
-\r
-#if defined(_WIN64) || defined(_WIN32)\r
-#include "..\..\..\..\..\etc\user\comp_channel.cpp"\r
-#include <rdma\winverbs.h>\r
-\r
-static COMP_SET ufds;\r
-\r
-static int dapls_os_init(void)\r
-{\r
- return CompSetInit(&ufds);\r
-}\r
-\r
-static void dapls_os_release(void)\r
-{\r
- CompSetCleanup(&ufds);\r
-}\r
-\r
-static int dapls_config_verbs(struct ibv_context *verbs)\r
-{\r
- verbs->channel.Milliseconds = 0;\r
- return 0;\r
-}\r
-\r
-static int dapls_config_comp_channel(struct ibv_comp_channel *channel)\r
-{\r
- channel->comp_channel.Milliseconds = 0;\r
- return 0;\r
-}\r
-\r
-static int dapls_thread_signal(void)\r
-{\r
- CompSetCancel(&ufds);\r
- return 0;\r
-}\r
-#else // _WIN64 || WIN32\r
-int g_ib_pipe[2];\r
-\r
-static int dapls_os_init(void)\r
-{\r
- /* create pipe for waking up work thread */\r
- return pipe(g_ib_pipe);\r
-}\r
-\r
-static void dapls_os_release(void)\r
-{\r
- /* close pipe? */\r
-}\r
-\r
-static int dapls_config_fd(int fd)\r
-{\r
- int opts;\r
-\r
- opts = fcntl(fd, F_GETFL);\r
- if (opts < 0 || fcntl(fd, F_SETFL, opts | O_NONBLOCK) < 0) {\r
- dapl_log(DAPL_DBG_TYPE_ERR,\r
- " dapls_config_fd: fcntl on fd %d ERR %d %s\n",\r
- fd, opts, strerror(errno));\r
- return errno;\r
- }\r
-\r
- return 0;\r
-}\r
-\r
-static int dapls_config_verbs(struct ibv_context *verbs)\r
-{\r
- return dapls_config_fd(verbs->async_fd);\r
-}\r
-\r
-static int dapls_config_comp_channel(struct ibv_comp_channel *channel)\r
-{\r
- return dapls_config_fd(channel->fd);\r
-}\r
-\r
-static int dapls_thread_signal(void)\r
-{\r
- return write(g_ib_pipe[1], "w", sizeof "w");\r
-}\r
-#endif\r
-\r
-\r
-static int32_t create_cr_pipe(IN DAPL_HCA * hca_ptr)\r
-{\r
- DAPL_SOCKET listen_socket;\r
- struct sockaddr_in addr;\r
- socklen_t addrlen = sizeof(addr);\r
- int ret;\r
-\r
- listen_socket = socket(AF_INET, SOCK_STREAM, IPPROTO_TCP);\r
- if (listen_socket == DAPL_INVALID_SOCKET)\r
- return 1;\r
-\r
- memset(&addr, 0, sizeof addr);\r
- addr.sin_family = AF_INET;\r
- addr.sin_addr.s_addr = htonl(0x7f000001);\r
- ret = bind(listen_socket, (struct sockaddr *)&addr, sizeof addr);\r
- if (ret)\r
- goto err1;\r
-\r
- ret = getsockname(listen_socket, (struct sockaddr *)&addr, &addrlen);\r
- if (ret)\r
- goto err1;\r
-\r
- ret = listen(listen_socket, 0);\r
- if (ret)\r
- goto err1;\r
-\r
- hca_ptr->ib_trans.scm[1] = socket(AF_INET, SOCK_STREAM, IPPROTO_TCP);\r
- if (hca_ptr->ib_trans.scm[1] == DAPL_INVALID_SOCKET)\r
- goto err1;\r
-\r
- ret = connect(hca_ptr->ib_trans.scm[1], \r
- (struct sockaddr *)&addr, sizeof(addr));\r
- if (ret)\r
- goto err2;\r
-\r
- hca_ptr->ib_trans.scm[0] = accept(listen_socket, NULL, NULL);\r
- if (hca_ptr->ib_trans.scm[0] == DAPL_INVALID_SOCKET)\r
- goto err2;\r
-\r
- closesocket(listen_socket);\r
- return 0;\r
-\r
- err2:\r
- closesocket(hca_ptr->ib_trans.scm[1]);\r
- err1:\r
- closesocket(listen_socket);\r
- return 1;\r
-}\r
-\r
-static void destroy_cr_pipe(IN DAPL_HCA * hca_ptr)\r
-{\r
- closesocket(hca_ptr->ib_trans.scm[0]);\r
- closesocket(hca_ptr->ib_trans.scm[1]);\r
-}\r
-\r
-\r
-/*\r
- * dapls_ib_init, dapls_ib_release\r
- *\r
- * Initialize Verb related items for device open\r
- *\r
- * Input:\r
- * none\r
- *\r
- * Output:\r
- * none\r
- *\r
- * Returns:\r
- * 0 success, -1 error\r
- *\r
- */\r
-int32_t dapls_ib_init(void)\r
-{\r
- /* initialize hca_list */\r
- dapl_os_lock_init(&g_hca_lock);\r
- dapl_llist_init_head(&g_hca_list);\r
-\r
- if (dapls_os_init())\r
- return 1;\r
-\r
- return 0;\r
-}\r
-\r
-int32_t dapls_ib_release(void)\r
-{\r
- dapli_ib_thread_destroy();\r
- dapls_os_release();\r
- return 0;\r
-}\r
-\r
-/*\r
- * dapls_ib_open_hca\r
- *\r
- * Open HCA\r
- *\r
- * Input:\r
- * *hca_name pointer to provider device name\r
- * *ib_hca_handle_p pointer to provide HCA handle\r
- *\r
- * Output:\r
- * none\r
- *\r
- * Return:\r
- * DAT_SUCCESS\r
- * dapl_convert_errno\r
- *\r
- */\r
-DAT_RETURN dapls_ib_open_hca(IN IB_HCA_NAME hca_name, IN DAPL_HCA * hca_ptr)\r
-{\r
- struct ibv_device **dev_list;\r
- struct ibv_port_attr port_attr;\r
- int i;\r
- DAT_RETURN dat_status;\r
-\r
- dapl_dbg_log(DAPL_DBG_TYPE_UTIL,\r
- " open_hca: %s - %p\n", hca_name, hca_ptr);\r
-\r
- /* get the IP address of the device */\r
- dat_status = getlocalipaddr((DAT_SOCK_ADDR *) &hca_ptr->hca_address,\r
- sizeof(DAT_SOCK_ADDR6));\r
- if (dat_status != DAT_SUCCESS)\r
- return dat_status;\r
-\r
-#ifdef DAPL_DBG\r
- /* DBG: unused port, set process id, lower 16 bits of pid */\r
- ((struct sockaddr_in *)&hca_ptr->hca_address)->sin_port = \r
- htons((uint16_t)dapl_os_getpid());\r
-#endif\r
- /* Get list of all IB devices, find match, open */\r
- dev_list = ibv_get_device_list(NULL);\r
- if (!dev_list) {\r
- dapl_dbg_log(DAPL_DBG_TYPE_ERR,\r
- " open_hca: ibv_get_device_list() failed\n",\r
- hca_name);\r
- return DAT_INTERNAL_ERROR;\r
- }\r
-\r
- for (i = 0; dev_list[i]; ++i) {\r
- hca_ptr->ib_trans.ib_dev = dev_list[i];\r
- if (!strcmp(ibv_get_device_name(hca_ptr->ib_trans.ib_dev),\r
- hca_name))\r
- goto found;\r
- }\r
-\r
- dapl_log(DAPL_DBG_TYPE_ERR,\r
- " open_hca: device %s not found\n", hca_name);\r
- goto err;\r
-\r
-found:\r
- dapl_dbg_log(DAPL_DBG_TYPE_UTIL, " open_hca: Found dev %s %016llx\n",\r
- ibv_get_device_name(hca_ptr->ib_trans.ib_dev),\r
- (unsigned long long)\r
- ntohll(ibv_get_device_guid(hca_ptr->ib_trans.ib_dev)));\r
-\r
- hca_ptr->ib_hca_handle = ibv_open_device(hca_ptr->ib_trans.ib_dev);\r
- if (!hca_ptr->ib_hca_handle) {\r
- dapl_log(DAPL_DBG_TYPE_ERR,\r
- " open_hca: dev open failed for %s, err=%s\n",\r
- ibv_get_device_name(hca_ptr->ib_trans.ib_dev),\r
- strerror(errno));\r
- goto err;\r
- }\r
- hca_ptr->ib_trans.ib_ctx = hca_ptr->ib_hca_handle;\r
- dapls_config_verbs(hca_ptr->ib_hca_handle);\r
-\r
- /* get lid for this hca-port, network order */\r
- if (ibv_query_port(hca_ptr->ib_hca_handle,\r
- (uint8_t) hca_ptr->port_num, &port_attr)) {\r
- dapl_log(DAPL_DBG_TYPE_ERR,\r
- " open_hca: get lid ERR for %s, err=%s\n",\r
- ibv_get_device_name(hca_ptr->ib_trans.ib_dev),\r
- strerror(errno));\r
- goto err;\r
- } else {\r
- hca_ptr->ib_trans.lid = htons(port_attr.lid);\r
- }\r
-\r
- /* get gid for this hca-port, network order */\r
- if (ibv_query_gid(hca_ptr->ib_hca_handle,\r
- (uint8_t) hca_ptr->port_num,\r
- 0, &hca_ptr->ib_trans.gid)) {\r
- dapl_log(DAPL_DBG_TYPE_ERR,\r
- " open_hca: query GID ERR for %s, err=%s\n",\r
- ibv_get_device_name(hca_ptr->ib_trans.ib_dev),\r
- strerror(errno));\r
- goto err;\r
- }\r
-\r
- /* set RC tunables via enviroment or default */\r
- hca_ptr->ib_trans.max_inline_send =\r
- dapl_os_get_env_val("DAPL_MAX_INLINE", INLINE_SEND_DEFAULT);\r
- hca_ptr->ib_trans.ack_retry =\r
- dapl_os_get_env_val("DAPL_ACK_RETRY", SCM_ACK_RETRY);\r
- hca_ptr->ib_trans.ack_timer =\r
- dapl_os_get_env_val("DAPL_ACK_TIMER", SCM_ACK_TIMER);\r
- hca_ptr->ib_trans.rnr_retry =\r
- dapl_os_get_env_val("DAPL_RNR_RETRY", SCM_RNR_RETRY);\r
- hca_ptr->ib_trans.rnr_timer =\r
- dapl_os_get_env_val("DAPL_RNR_TIMER", SCM_RNR_TIMER);\r
- hca_ptr->ib_trans.global =\r
- dapl_os_get_env_val("DAPL_GLOBAL_ROUTING", SCM_GLOBAL);\r
- hca_ptr->ib_trans.hop_limit =\r
- dapl_os_get_env_val("DAPL_HOP_LIMIT", SCM_HOP_LIMIT);\r
- hca_ptr->ib_trans.tclass =\r
- dapl_os_get_env_val("DAPL_TCLASS", SCM_TCLASS);\r
- hca_ptr->ib_trans.mtu =\r
- dapl_ib_mtu(dapl_os_get_env_val("DAPL_IB_MTU", SCM_IB_MTU));\r
-\r
-\r
- /* EVD events without direct CQ channels, CNO support */\r
- hca_ptr->ib_trans.ib_cq =\r
- ibv_create_comp_channel(hca_ptr->ib_hca_handle);\r
- if (hca_ptr->ib_trans.ib_cq == NULL) {\r
- dapl_log(DAPL_DBG_TYPE_ERR,\r
- " open_hca: ibv_create_comp_channel ERR %s\n",\r
- strerror(errno));\r
- goto bail;\r
- }\r
- dapls_config_comp_channel(hca_ptr->ib_trans.ib_cq);\r
- \r
- dat_status = dapli_ib_thread_init();\r
- if (dat_status != DAT_SUCCESS) {\r
- dapl_log(DAPL_DBG_TYPE_ERR,\r
- " open_hca: failed to init cq thread lock\n");\r
- goto bail;\r
- }\r
- /* \r
- * Put new hca_transport on list for async and CQ event processing \r
- * Wakeup work thread to add to polling list\r
- */\r
- dapl_llist_init_entry((DAPL_LLIST_ENTRY *)&hca_ptr->ib_trans.entry);\r
- dapl_os_lock(&g_hca_lock);\r
- dapl_llist_add_tail(&g_hca_list,\r
- (DAPL_LLIST_ENTRY *) &hca_ptr->ib_trans.entry,\r
- &hca_ptr->ib_trans.entry);\r
- if (dapls_thread_signal() == -1)\r
- dapl_log(DAPL_DBG_TYPE_UTIL,\r
- " open_hca: thread wakeup error = %s\n",\r
- strerror(errno));\r
- dapl_os_unlock(&g_hca_lock);\r
-\r
- /* initialize cr_list lock */\r
- dat_status = dapl_os_lock_init(&hca_ptr->ib_trans.lock);\r
- if (dat_status != DAT_SUCCESS) {\r
- dapl_log(DAPL_DBG_TYPE_ERR,\r
- " open_hca: failed to init cr_list lock\n");\r
- goto bail;\r
- }\r
-\r
- /* initialize CM list for listens on this HCA */\r
- dapl_llist_init_head(&hca_ptr->ib_trans.list);\r
-\r
- /* initialize pipe, user level wakeup on select */\r
- if (create_cr_pipe(hca_ptr)) {\r
- dapl_log(DAPL_DBG_TYPE_ERR,\r
- " open_hca: failed to init cr pipe - %s\n",\r
- strerror(errno));\r
- goto bail;\r
- }\r
-\r
- /* create thread to process inbound connect request */\r
- hca_ptr->ib_trans.cr_state = IB_THREAD_INIT;\r
- dat_status = dapl_os_thread_create(cr_thread,\r
- (void *)hca_ptr,\r
- &hca_ptr->ib_trans.thread);\r
- if (dat_status != DAT_SUCCESS) {\r
- dapl_log(DAPL_DBG_TYPE_ERR,\r
- " open_hca: failed to create thread\n");\r
- goto bail;\r
- }\r
-\r
- /* wait for thread */\r
- while (hca_ptr->ib_trans.cr_state != IB_THREAD_RUN) {\r
- dapl_os_sleep_usec(1000);\r
- }\r
-\r
- dapl_dbg_log(DAPL_DBG_TYPE_UTIL,\r
- " open_hca: devname %s, port %d, hostname_IP %s\n",\r
- ibv_get_device_name(hca_ptr->ib_trans.ib_dev),\r
- hca_ptr->port_num, inet_ntoa(((struct sockaddr_in *)\r
- &hca_ptr->hca_address)->\r
- sin_addr));\r
- dapl_dbg_log(DAPL_DBG_TYPE_UTIL,\r
- " open_hca: LID 0x%x GID Subnet 0x" F64x " ID 0x" F64x\r
- "\n", ntohs(hca_ptr->ib_trans.lid), (unsigned long long)\r
- htonll(hca_ptr->ib_trans.gid.global.subnet_prefix),\r
- (unsigned long long)htonll(hca_ptr->ib_trans.gid.global.\r
- interface_id));\r
-\r
- ibv_free_device_list(dev_list);\r
- return dat_status;\r
-\r
- bail:\r
- ibv_close_device(hca_ptr->ib_hca_handle);\r
- hca_ptr->ib_hca_handle = IB_INVALID_HANDLE;\r
- err:\r
- ibv_free_device_list(dev_list);\r
- return DAT_INTERNAL_ERROR;\r
-}\r
-\r
-/*\r
- * dapls_ib_close_hca\r
- *\r
- * Open HCA\r
- *\r
- * Input:\r
- * DAPL_HCA provide CA handle\r
- *\r
- * Output:\r
- * none\r
- *\r
- * Return:\r
- * DAT_SUCCESS\r
- * dapl_convert_errno \r
- *\r
- */\r
-DAT_RETURN dapls_ib_close_hca(IN DAPL_HCA * hca_ptr)\r
-{\r
- dapl_dbg_log(DAPL_DBG_TYPE_UTIL, " close_hca: %p\n", hca_ptr);\r
-\r
- if (hca_ptr->ib_hca_handle != IB_INVALID_HANDLE) {\r
- if (ibv_close_device(hca_ptr->ib_hca_handle))\r
- return (dapl_convert_errno(errno, "ib_close_device"));\r
- hca_ptr->ib_hca_handle = IB_INVALID_HANDLE;\r
- }\r
-\r
- dapl_os_lock(&g_hca_lock);\r
- if (g_ib_thread_state != IB_THREAD_RUN) {\r
- dapl_os_unlock(&g_hca_lock);\r
- return (DAT_SUCCESS);\r
- }\r
- dapl_os_unlock(&g_hca_lock);\r
-\r
- /* destroy cr_thread and lock */\r
- hca_ptr->ib_trans.cr_state = IB_THREAD_CANCEL;\r
- send(hca_ptr->ib_trans.scm[1], "w", sizeof "w", 0);\r
- while (hca_ptr->ib_trans.cr_state != IB_THREAD_EXIT) {\r
- dapl_dbg_log(DAPL_DBG_TYPE_UTIL,\r
- " close_hca: waiting for cr_thread\n");\r
- send(hca_ptr->ib_trans.scm[1], "w", sizeof "w", 0);\r
- dapl_os_sleep_usec(1000);\r
- }\r
- dapl_os_lock_destroy(&hca_ptr->ib_trans.lock);\r
- destroy_cr_pipe(hca_ptr); /* no longer need pipe */\r
- \r
- /* \r
- * Remove hca from async event processing list\r
- * Wakeup work thread to remove from polling list\r
- */\r
- hca_ptr->ib_trans.destroy = 1;\r
- if (dapls_thread_signal() == -1)\r
- dapl_log(DAPL_DBG_TYPE_UTIL,\r
- " destroy: thread wakeup error = %s\n",\r
- strerror(errno));\r
-\r
- /* wait for thread to remove HCA references */\r
- while (hca_ptr->ib_trans.destroy != 2) {\r
- if (dapls_thread_signal() == -1)\r
- dapl_log(DAPL_DBG_TYPE_UTIL,\r
- " destroy: thread wakeup error = %s\n",\r
- strerror(errno));\r
- dapl_os_sleep_usec(1000);\r
- }\r
-\r
- return (DAT_SUCCESS);\r
-}\r
-\r
-DAT_RETURN dapli_ib_thread_init(void)\r
-{\r
- DAT_RETURN dat_status;\r
-\r
- dapl_os_lock(&g_hca_lock);\r
- if (g_ib_thread_state != IB_THREAD_INIT) {\r
- dapl_os_unlock(&g_hca_lock);\r
- return DAT_SUCCESS;\r
- }\r
-\r
- g_ib_thread_state = IB_THREAD_CREATE;\r
- dapl_os_unlock(&g_hca_lock);\r
-\r
- /* create thread to process inbound connect request */\r
- dat_status = dapl_os_thread_create(dapli_thread, NULL, &g_ib_thread);\r
- if (dat_status != DAT_SUCCESS)\r
- return (dapl_convert_errno(errno,\r
- "create_thread ERR:"\r
- " check resource limits"));\r
-\r
- /* wait for thread to start */\r
- dapl_os_lock(&g_hca_lock);\r
- while (g_ib_thread_state != IB_THREAD_RUN) {\r
- dapl_dbg_log(DAPL_DBG_TYPE_UTIL,\r
- " ib_thread_init: waiting for ib_thread\n");\r
- dapl_os_unlock(&g_hca_lock);\r
- dapl_os_sleep_usec(1000);\r
- dapl_os_lock(&g_hca_lock);\r
- }\r
- dapl_os_unlock(&g_hca_lock);\r
-\r
- return DAT_SUCCESS;\r
-}\r
-\r
-void dapli_ib_thread_destroy(void)\r
-{\r
- dapl_dbg_log(DAPL_DBG_TYPE_UTIL,\r
- " ib_thread_destroy(%d)\n", dapl_os_getpid());\r
- /* \r
- * wait for async thread to terminate. \r
- * pthread_join would be the correct method\r
- * but some applications have some issues\r
- */\r
-\r
- /* destroy ib_thread, wait for termination, if not already */\r
- dapl_os_lock(&g_hca_lock);\r
- if (g_ib_thread_state != IB_THREAD_RUN)\r
- goto bail;\r
-\r
- g_ib_thread_state = IB_THREAD_CANCEL;\r
- while (g_ib_thread_state != IB_THREAD_EXIT) {\r
- dapl_dbg_log(DAPL_DBG_TYPE_UTIL,\r
- " ib_thread_destroy: waiting for ib_thread\n");\r
- if (dapls_thread_signal() == -1)\r
- dapl_log(DAPL_DBG_TYPE_UTIL,\r
- " destroy: thread wakeup error = %s\n",\r
- strerror(errno));\r
- dapl_os_unlock(&g_hca_lock);\r
- dapl_os_sleep_usec(2000);\r
- dapl_os_lock(&g_hca_lock);\r
- }\r
-bail:\r
- dapl_os_unlock(&g_hca_lock);\r
-\r
- dapl_dbg_log(DAPL_DBG_TYPE_UTIL,\r
- " ib_thread_destroy(%d) exit\n", dapl_os_getpid());\r
-}\r
-\r
-\r
-#if defined(_WIN64) || defined(_WIN32)\r
-/* work thread for uAT, uCM, CQ, and async events */\r
-void dapli_thread(void *arg)\r
-{\r
- struct _ib_hca_transport *hca;\r
- struct _ib_hca_transport *uhca[8];\r
- int ret, idx, cnt;\r
-\r
- dapl_dbg_log(DAPL_DBG_TYPE_UTIL, " ib_thread(%d,0x%x): ENTER: \n",\r
- dapl_os_getpid(), g_ib_thread);\r
-\r
- dapl_os_lock(&g_hca_lock);\r
- for (g_ib_thread_state = IB_THREAD_RUN;\r
- g_ib_thread_state == IB_THREAD_RUN; \r
- dapl_os_lock(&g_hca_lock)) {\r
-\r
- CompSetZero(&ufds);\r
- idx = 0;\r
- hca = dapl_llist_is_empty(&g_hca_list) ? NULL :\r
- dapl_llist_peek_head(&g_hca_list);\r
-\r
- while (hca) {\r
- CompSetAdd(&hca->ib_ctx->channel, &ufds);\r
- CompSetAdd(&hca->ib_cq->comp_channel, &ufds);\r
- uhca[idx++] = hca;\r
- hca = dapl_llist_next_entry(&g_hca_list,\r
- (DAPL_LLIST_ENTRY *)\r
- &hca->entry);\r
- }\r
- cnt = idx;\r
-\r
- dapl_os_unlock(&g_hca_lock);\r
- ret = CompSetPoll(&ufds, INFINITE);\r
-\r
- dapl_dbg_log(DAPL_DBG_TYPE_UTIL,\r
- " ib_thread(%d) poll_event 0x%x\n",\r
- dapl_os_getpid(), ret);\r
-\r
-\r
- /* check and process ASYNC events, per device */\r
- for (idx = 0; idx < cnt; idx++) {\r
- if (uhca[idx]->destroy == 1) {\r
- dapl_os_lock(&g_hca_lock);\r
- dapl_llist_remove_entry(&g_hca_list,\r
- (DAPL_LLIST_ENTRY *)\r
- &uhca[idx]->entry);\r
- dapl_os_unlock(&g_hca_lock);\r
- uhca[idx]->destroy = 2;\r
- } else {\r
- dapli_cq_event_cb(uhca[idx]);\r
- dapli_async_event_cb(uhca[idx]);\r
- }\r
- }\r
- }\r
-\r
- dapl_dbg_log(DAPL_DBG_TYPE_UTIL, " ib_thread(%d) EXIT\n",\r
- dapl_os_getpid());\r
- g_ib_thread_state = IB_THREAD_EXIT;\r
- dapl_os_unlock(&g_hca_lock);\r
-}\r
-#else // _WIN64 || WIN32\r
-\r
-/* work thread for uAT, uCM, CQ, and async events */\r
-void dapli_thread(void *arg)\r
-{\r
- struct pollfd ufds[__FD_SETSIZE];\r
- struct _ib_hca_transport *uhca[__FD_SETSIZE] = { NULL };\r
- struct _ib_hca_transport *hca;\r
- int ret, idx, fds;\r
- char rbuf[2];\r
-\r
- dapl_dbg_log(DAPL_DBG_TYPE_THREAD,\r
- " ib_thread(%d,0x%x): ENTER: pipe %d \n",\r
- dapl_os_getpid(), g_ib_thread, g_ib_pipe[0]);\r
-\r
- /* Poll across pipe, CM, AT never changes */\r
- dapl_os_lock(&g_hca_lock);\r
- g_ib_thread_state = IB_THREAD_RUN;\r
-\r
- ufds[0].fd = g_ib_pipe[0]; /* pipe */\r
- ufds[0].events = POLLIN;\r
-\r
- while (g_ib_thread_state == IB_THREAD_RUN) {\r
-\r
- /* build ufds after pipe and uCMA events */\r
- ufds[0].revents = 0;\r
- idx = 0;\r
-\r
- /* Walk HCA list and setup async and CQ events */\r
- if (!dapl_llist_is_empty(&g_hca_list))\r
- hca = dapl_llist_peek_head(&g_hca_list);\r
- else\r
- hca = NULL;\r
-\r
- while (hca) {\r
-\r
- /* uASYNC events */\r
- ufds[++idx].fd = hca->ib_ctx->async_fd;\r
- ufds[idx].events = POLLIN;\r
- ufds[idx].revents = 0;\r
- uhca[idx] = hca;\r
-\r
- /* CQ events are non-direct with CNO's */\r
- ufds[++idx].fd = hca->ib_cq->fd;\r
- ufds[idx].events = POLLIN;\r
- ufds[idx].revents = 0;\r
- uhca[idx] = hca;\r
-\r
- dapl_dbg_log(DAPL_DBG_TYPE_THREAD,\r
- " ib_thread(%d) poll_fd: hca[%d]=%p,"\r
- " async=%d pipe=%d \n",\r
- dapl_os_getpid(), hca, ufds[idx - 1].fd,\r
- ufds[0].fd);\r
-\r
- hca = dapl_llist_next_entry(&g_hca_list,\r
- (DAPL_LLIST_ENTRY *)\r
- &hca->entry);\r
- }\r
-\r
- /* unlock, and setup poll */\r
- fds = idx + 1;\r
- dapl_os_unlock(&g_hca_lock);\r
- ret = poll(ufds, fds, -1);\r
- if (ret <= 0) {\r
- dapl_dbg_log(DAPL_DBG_TYPE_THREAD,\r
- " ib_thread(%d): ERR %s poll\n",\r
- dapl_os_getpid(), strerror(errno));\r
- dapl_os_lock(&g_hca_lock);\r
- continue;\r
- }\r
-\r
- dapl_dbg_log(DAPL_DBG_TYPE_THREAD,\r
- " ib_thread(%d) poll_event: "\r
- " async=0x%x pipe=0x%x \n",\r
- dapl_os_getpid(), ufds[idx].revents,\r
- ufds[0].revents);\r
-\r
- /* check and process CQ and ASYNC events, per device */\r
- for (idx = 1; idx < fds; idx++) {\r
- if (ufds[idx].revents == POLLIN) {\r
- dapli_cq_event_cb(uhca[idx]);\r
- dapli_async_event_cb(uhca[idx]);\r
- }\r
- }\r
-\r
- /* check and process user events, PIPE */\r
- if (ufds[0].revents == POLLIN) {\r
- if (read(g_ib_pipe[0], rbuf, 2) == -1)\r
- dapl_log(DAPL_DBG_TYPE_THREAD,\r
- " cr_thread: pipe rd err= %s\n",\r
- strerror(errno));\r
-\r
- /* cleanup any device on list marked for destroy */\r
- for (idx = 1; idx < fds; idx++) {\r
- if (uhca[idx] && uhca[idx]->destroy == 1) {\r
- dapl_os_lock(&g_hca_lock);\r
- dapl_llist_remove_entry(\r
- &g_hca_list,\r
- (DAPL_LLIST_ENTRY*)\r
- &uhca[idx]->entry);\r
- dapl_os_unlock(&g_hca_lock);\r
- uhca[idx]->destroy = 2;\r
- }\r
- }\r
- }\r
- dapl_os_lock(&g_hca_lock);\r
- }\r
-\r
- dapl_dbg_log(DAPL_DBG_TYPE_THREAD, " ib_thread(%d) EXIT\n",\r
- dapl_os_getpid());\r
- g_ib_thread_state = IB_THREAD_EXIT;\r
- dapl_os_unlock(&g_hca_lock);\r
-}\r
-#endif\r
+/*
+ * This Software is licensed under one of the following licenses:
+ *
+ * 1) under the terms of the "Common Public License 1.0" a copy of which is
+ * available from the Open Source Initiative, see
+ * http://www.opensource.org/licenses/cpl.php.
+ *
+ * 2) under the terms of the "The BSD License" a copy of which is
+ * available from the Open Source Initiative, see
+ * http://www.opensource.org/licenses/bsd-license.php.
+ *
+ * 3) under the terms of the "GNU General Public License (GPL) Version 2" a
+ * copy of which is available from the Open Source Initiative, see
+ * http://www.opensource.org/licenses/gpl-license.php.
+ *
+ * Licensee has the right to choose one of the above licenses.
+ *
+ * Redistributions of source code must retain the above copyright
+ * notice and one of the license notices.
+ *
+ * Redistributions in binary form must reproduce both the above copyright
+ * notice, one of the license notices in the documentation
+ * and/or other materials provided with the distribution.
+ */
+
+/***************************************************************************
+ *
+ * Module: uDAPL
+ *
+ * Filename: dapl_ib_util.c
+ *
+ * Author: Arlin Davis
+ *
+ * Created: 3/10/2005
+ *
+ * Description:
+ *
+ * The uDAPL openib provider - init, open, close, utilities
+ *
+ ****************************************************************************
+ * Source Control System Information
+ *
+ * $Id: $
+ *
+ * Copyright (c) 2005 Intel Corporation. All rights reserved.
+ *
+ **************************************************************************/
+#ifdef RCSID
+static const char rcsid[] = "$Id: $";
+#endif
+
+#include "openib_osd.h"
+#include "dapl.h"
+#include "dapl_adapter_util.h"
+#include "dapl_ib_util.h"
+#include "dapl_osd.h"
+
+#include <stdlib.h>
+
+ib_thread_state_t g_ib_thread_state = 0;
+DAPL_OS_THREAD g_ib_thread;
+DAPL_OS_LOCK g_hca_lock;
+struct dapl_llist_entry *g_hca_list;
+
+void dapli_thread(void *arg);
+DAT_RETURN dapli_ib_thread_init(void);
+void dapli_ib_thread_destroy(void);
+
+#if defined(_WIN64) || defined(_WIN32)
+#include "..\..\..\..\..\etc\user\comp_channel.cpp"
+#include <rdma\winverbs.h>
+
+static COMP_SET ufds;
+
+static int dapls_os_init(void)
+{
+ return CompSetInit(&ufds);
+}
+
+static void dapls_os_release(void)
+{
+ CompSetCleanup(&ufds);
+}
+
+static int dapls_config_verbs(struct ibv_context *verbs)
+{
+ verbs->channel.Milliseconds = 0;
+ return 0;
+}
+
+static int dapls_config_comp_channel(struct ibv_comp_channel *channel)
+{
+ channel->comp_channel.Milliseconds = 0;
+ return 0;
+}
+
+static int dapls_thread_signal(void)
+{
+ CompSetCancel(&ufds);
+ return 0;
+}
+#else // _WIN64 || WIN32
+int g_ib_pipe[2];
+
+static int dapls_os_init(void)
+{
+ /* create pipe for waking up work thread */
+ return pipe(g_ib_pipe);
+}
+
+static void dapls_os_release(void)
+{
+ /* close pipe? */
+}
+
+static int dapls_config_fd(int fd)
+{
+ int opts;
+
+ opts = fcntl(fd, F_GETFL);
+ if (opts < 0 || fcntl(fd, F_SETFL, opts | O_NONBLOCK) < 0) {
+ dapl_log(DAPL_DBG_TYPE_ERR,
+ " dapls_config_fd: fcntl on fd %d ERR %d %s\n",
+ fd, opts, strerror(errno));
+ return errno;
+ }
+
+ return 0;
+}
+
+static int dapls_config_verbs(struct ibv_context *verbs)
+{
+ return dapls_config_fd(verbs->async_fd);
+}
+
+static int dapls_config_comp_channel(struct ibv_comp_channel *channel)
+{
+ return dapls_config_fd(channel->fd);
+}
+
+static int dapls_thread_signal(void)
+{
+ return write(g_ib_pipe[1], "w", sizeof "w");
+}
+#endif
+
+
+static int32_t create_cr_pipe(IN DAPL_HCA * hca_ptr)
+{
+ DAPL_SOCKET listen_socket;
+ struct sockaddr_in addr;
+ socklen_t addrlen = sizeof(addr);
+ int ret;
+
+ listen_socket = socket(AF_INET, SOCK_STREAM, IPPROTO_TCP);
+ if (listen_socket == DAPL_INVALID_SOCKET)
+ return 1;
+
+ memset(&addr, 0, sizeof addr);
+ addr.sin_family = AF_INET;
+ addr.sin_addr.s_addr = htonl(0x7f000001);
+ ret = bind(listen_socket, (struct sockaddr *)&addr, sizeof addr);
+ if (ret)
+ goto err1;
+
+ ret = getsockname(listen_socket, (struct sockaddr *)&addr, &addrlen);
+ if (ret)
+ goto err1;
+
+ ret = listen(listen_socket, 0);
+ if (ret)
+ goto err1;
+
+ hca_ptr->ib_trans.scm[1] = socket(AF_INET, SOCK_STREAM, IPPROTO_TCP);
+ if (hca_ptr->ib_trans.scm[1] == DAPL_INVALID_SOCKET)
+ goto err1;
+
+ ret = connect(hca_ptr->ib_trans.scm[1],
+ (struct sockaddr *)&addr, sizeof(addr));
+ if (ret)
+ goto err2;
+
+ hca_ptr->ib_trans.scm[0] = accept(listen_socket, NULL, NULL);
+ if (hca_ptr->ib_trans.scm[0] == DAPL_INVALID_SOCKET)
+ goto err2;
+
+ closesocket(listen_socket);
+ return 0;
+
+ err2:
+ closesocket(hca_ptr->ib_trans.scm[1]);
+ err1:
+ closesocket(listen_socket);
+ return 1;
+}
+
+static void destroy_cr_pipe(IN DAPL_HCA * hca_ptr)
+{
+ closesocket(hca_ptr->ib_trans.scm[0]);
+ closesocket(hca_ptr->ib_trans.scm[1]);
+}
+
+
+/*
+ * dapls_ib_init, dapls_ib_release
+ *
+ * Initialize Verb related items for device open
+ *
+ * Input:
+ * none
+ *
+ * Output:
+ * none
+ *
+ * Returns:
+ * 0 success, -1 error
+ *
+ */
+int32_t dapls_ib_init(void)
+{
+ /* initialize hca_list */
+ dapl_os_lock_init(&g_hca_lock);
+ dapl_llist_init_head(&g_hca_list);
+
+ if (dapls_os_init())
+ return 1;
+
+ return 0;
+}
+
+int32_t dapls_ib_release(void)
+{
+ dapli_ib_thread_destroy();
+ dapls_os_release();
+ return 0;
+}
+
+/*
+ * dapls_ib_open_hca
+ *
+ * Open HCA
+ *
+ * Input:
+ * *hca_name pointer to provider device name
+ * *ib_hca_handle_p pointer to provide HCA handle
+ *
+ * Output:
+ * none
+ *
+ * Return:
+ * DAT_SUCCESS
+ * dapl_convert_errno
+ *
+ */
+DAT_RETURN dapls_ib_open_hca(IN IB_HCA_NAME hca_name, IN DAPL_HCA * hca_ptr)
+{
+ struct ibv_device **dev_list;
+ struct ibv_port_attr port_attr;
+ int i;
+ DAT_RETURN dat_status;
+
+ dapl_dbg_log(DAPL_DBG_TYPE_UTIL,
+ " open_hca: %s - %p\n", hca_name, hca_ptr);
+
+ /* get the IP address of the device */
+ dat_status = getlocalipaddr((DAT_SOCK_ADDR *) &hca_ptr->hca_address,
+ sizeof(DAT_SOCK_ADDR6));
+ if (dat_status != DAT_SUCCESS)
+ return dat_status;
+
+#ifdef DAPL_DBG
+ /* DBG: unused port, set process id, lower 16 bits of pid */
+ ((struct sockaddr_in *)&hca_ptr->hca_address)->sin_port =
+ htons((uint16_t)dapl_os_getpid());
+#endif
+ /* Get list of all IB devices, find match, open */
+ dev_list = ibv_get_device_list(NULL);
+ if (!dev_list) {
+ dapl_dbg_log(DAPL_DBG_TYPE_ERR,
+ " open_hca: ibv_get_device_list() failed\n",
+ hca_name);
+ return DAT_INTERNAL_ERROR;
+ }
+
+ for (i = 0; dev_list[i]; ++i) {
+ hca_ptr->ib_trans.ib_dev = dev_list[i];
+ if (!strcmp(ibv_get_device_name(hca_ptr->ib_trans.ib_dev),
+ hca_name))
+ goto found;
+ }
+
+ dapl_log(DAPL_DBG_TYPE_ERR,
+ " open_hca: device %s not found\n", hca_name);
+ goto err;
+
+found:
+ dapl_dbg_log(DAPL_DBG_TYPE_UTIL, " open_hca: Found dev %s %016llx\n",
+ ibv_get_device_name(hca_ptr->ib_trans.ib_dev),
+ (unsigned long long)
+ ntohll(ibv_get_device_guid(hca_ptr->ib_trans.ib_dev)));
+
+ hca_ptr->ib_hca_handle = ibv_open_device(hca_ptr->ib_trans.ib_dev);
+ if (!hca_ptr->ib_hca_handle) {
+ dapl_log(DAPL_DBG_TYPE_ERR,
+ " open_hca: dev open failed for %s, err=%s\n",
+ ibv_get_device_name(hca_ptr->ib_trans.ib_dev),
+ strerror(errno));
+ goto err;
+ }
+ hca_ptr->ib_trans.ib_ctx = hca_ptr->ib_hca_handle;
+ dapls_config_verbs(hca_ptr->ib_hca_handle);
+
+ /* get lid for this hca-port, network order */
+ if (ibv_query_port(hca_ptr->ib_hca_handle,
+ (uint8_t) hca_ptr->port_num, &port_attr)) {
+ dapl_log(DAPL_DBG_TYPE_ERR,
+ " open_hca: get lid ERR for %s, err=%s\n",
+ ibv_get_device_name(hca_ptr->ib_trans.ib_dev),
+ strerror(errno));
+ goto err;
+ } else {
+ hca_ptr->ib_trans.lid = htons(port_attr.lid);
+ }
+
+ /* get gid for this hca-port, network order */
+ if (ibv_query_gid(hca_ptr->ib_hca_handle,
+ (uint8_t) hca_ptr->port_num,
+ 0, &hca_ptr->ib_trans.gid)) {
+ dapl_log(DAPL_DBG_TYPE_ERR,
+ " open_hca: query GID ERR for %s, err=%s\n",
+ ibv_get_device_name(hca_ptr->ib_trans.ib_dev),
+ strerror(errno));
+ goto err;
+ }
+
+ /* set RC tunables via enviroment or default */
+ hca_ptr->ib_trans.max_inline_send =
+ dapl_os_get_env_val("DAPL_MAX_INLINE", INLINE_SEND_DEFAULT);
+ hca_ptr->ib_trans.ack_retry =
+ dapl_os_get_env_val("DAPL_ACK_RETRY", SCM_ACK_RETRY);
+ hca_ptr->ib_trans.ack_timer =
+ dapl_os_get_env_val("DAPL_ACK_TIMER", SCM_ACK_TIMER);
+ hca_ptr->ib_trans.rnr_retry =
+ dapl_os_get_env_val("DAPL_RNR_RETRY", SCM_RNR_RETRY);
+ hca_ptr->ib_trans.rnr_timer =
+ dapl_os_get_env_val("DAPL_RNR_TIMER", SCM_RNR_TIMER);
+ hca_ptr->ib_trans.global =
+ dapl_os_get_env_val("DAPL_GLOBAL_ROUTING", SCM_GLOBAL);
+ hca_ptr->ib_trans.hop_limit =
+ dapl_os_get_env_val("DAPL_HOP_LIMIT", SCM_HOP_LIMIT);
+ hca_ptr->ib_trans.tclass =
+ dapl_os_get_env_val("DAPL_TCLASS", SCM_TCLASS);
+ hca_ptr->ib_trans.mtu =
+ dapl_ib_mtu(dapl_os_get_env_val("DAPL_IB_MTU", SCM_IB_MTU));
+
+
+ /* EVD events without direct CQ channels, CNO support */
+ hca_ptr->ib_trans.ib_cq =
+ ibv_create_comp_channel(hca_ptr->ib_hca_handle);
+ if (hca_ptr->ib_trans.ib_cq == NULL) {
+ dapl_log(DAPL_DBG_TYPE_ERR,
+ " open_hca: ibv_create_comp_channel ERR %s\n",
+ strerror(errno));
+ goto bail;
+ }
+ dapls_config_comp_channel(hca_ptr->ib_trans.ib_cq);
+
+ dat_status = dapli_ib_thread_init();
+ if (dat_status != DAT_SUCCESS) {
+ dapl_log(DAPL_DBG_TYPE_ERR,
+ " open_hca: failed to init cq thread lock\n");
+ goto bail;
+ }
+ /*
+ * Put new hca_transport on list for async and CQ event processing
+ * Wakeup work thread to add to polling list
+ */
+ dapl_llist_init_entry((DAPL_LLIST_ENTRY *)&hca_ptr->ib_trans.entry);
+ dapl_os_lock(&g_hca_lock);
+ dapl_llist_add_tail(&g_hca_list,
+ (DAPL_LLIST_ENTRY *) &hca_ptr->ib_trans.entry,
+ &hca_ptr->ib_trans.entry);
+ if (dapls_thread_signal() == -1)
+ dapl_log(DAPL_DBG_TYPE_UTIL,
+ " open_hca: thread wakeup error = %s\n",
+ strerror(errno));
+ dapl_os_unlock(&g_hca_lock);
+
+ /* initialize cr_list lock */
+ dat_status = dapl_os_lock_init(&hca_ptr->ib_trans.lock);
+ if (dat_status != DAT_SUCCESS) {
+ dapl_log(DAPL_DBG_TYPE_ERR,
+ " open_hca: failed to init cr_list lock\n");
+ goto bail;
+ }
+
+ /* initialize CM list for listens on this HCA */
+ dapl_llist_init_head(&hca_ptr->ib_trans.list);
+
+ /* initialize pipe, user level wakeup on select */
+ if (create_cr_pipe(hca_ptr)) {
+ dapl_log(DAPL_DBG_TYPE_ERR,
+ " open_hca: failed to init cr pipe - %s\n",
+ strerror(errno));
+ goto bail;
+ }
+
+ /* create thread to process inbound connect request */
+ hca_ptr->ib_trans.cr_state = IB_THREAD_INIT;
+ dat_status = dapl_os_thread_create(cr_thread,
+ (void *)hca_ptr,
+ &hca_ptr->ib_trans.thread);
+ if (dat_status != DAT_SUCCESS) {
+ dapl_log(DAPL_DBG_TYPE_ERR,
+ " open_hca: failed to create thread\n");
+ goto bail;
+ }
+
+ /* wait for thread */
+ while (hca_ptr->ib_trans.cr_state != IB_THREAD_RUN) {
+ dapl_os_sleep_usec(1000);
+ }
+
+ dapl_dbg_log(DAPL_DBG_TYPE_UTIL,
+ " open_hca: devname %s, port %d, hostname_IP %s\n",
+ ibv_get_device_name(hca_ptr->ib_trans.ib_dev),
+ hca_ptr->port_num, inet_ntoa(((struct sockaddr_in *)
+ &hca_ptr->hca_address)->
+ sin_addr));
+ dapl_dbg_log(DAPL_DBG_TYPE_UTIL,
+ " open_hca: LID 0x%x GID Subnet 0x" F64x " ID 0x" F64x
+ "\n", ntohs(hca_ptr->ib_trans.lid), (unsigned long long)
+ htonll(hca_ptr->ib_trans.gid.global.subnet_prefix),
+ (unsigned long long)htonll(hca_ptr->ib_trans.gid.global.
+ interface_id));
+
+ ibv_free_device_list(dev_list);
+ return dat_status;
+
+ bail:
+ ibv_close_device(hca_ptr->ib_hca_handle);
+ hca_ptr->ib_hca_handle = IB_INVALID_HANDLE;
+ err:
+ ibv_free_device_list(dev_list);
+ return DAT_INTERNAL_ERROR;
+}
+
+/*
+ * dapls_ib_close_hca
+ *
+ * Open HCA
+ *
+ * Input:
+ * DAPL_HCA provide CA handle
+ *
+ * Output:
+ * none
+ *
+ * Return:
+ * DAT_SUCCESS
+ * dapl_convert_errno
+ *
+ */
+DAT_RETURN dapls_ib_close_hca(IN DAPL_HCA * hca_ptr)
+{
+ dapl_dbg_log(DAPL_DBG_TYPE_UTIL, " close_hca: %p\n", hca_ptr);
+
+ if (hca_ptr->ib_hca_handle != IB_INVALID_HANDLE) {
+ if (ibv_close_device(hca_ptr->ib_hca_handle))
+ return (dapl_convert_errno(errno, "ib_close_device"));
+ hca_ptr->ib_hca_handle = IB_INVALID_HANDLE;
+ }
+
+ dapl_os_lock(&g_hca_lock);
+ if (g_ib_thread_state != IB_THREAD_RUN) {
+ dapl_os_unlock(&g_hca_lock);
+ return (DAT_SUCCESS);
+ }
+ dapl_os_unlock(&g_hca_lock);
+
+ /* destroy cr_thread and lock */
+ hca_ptr->ib_trans.cr_state = IB_THREAD_CANCEL;
+ send(hca_ptr->ib_trans.scm[1], "w", sizeof "w", 0);
+ while (hca_ptr->ib_trans.cr_state != IB_THREAD_EXIT) {
+ dapl_dbg_log(DAPL_DBG_TYPE_UTIL,
+ " close_hca: waiting for cr_thread\n");
+ send(hca_ptr->ib_trans.scm[1], "w", sizeof "w", 0);
+ dapl_os_sleep_usec(1000);
+ }
+ dapl_os_lock_destroy(&hca_ptr->ib_trans.lock);
+ destroy_cr_pipe(hca_ptr); /* no longer need pipe */
+
+ /*
+ * Remove hca from async event processing list
+ * Wakeup work thread to remove from polling list
+ */
+ hca_ptr->ib_trans.destroy = 1;
+ if (dapls_thread_signal() == -1)
+ dapl_log(DAPL_DBG_TYPE_UTIL,
+ " destroy: thread wakeup error = %s\n",
+ strerror(errno));
+
+ /* wait for thread to remove HCA references */
+ while (hca_ptr->ib_trans.destroy != 2) {
+ if (dapls_thread_signal() == -1)
+ dapl_log(DAPL_DBG_TYPE_UTIL,
+ " destroy: thread wakeup error = %s\n",
+ strerror(errno));
+ dapl_os_sleep_usec(1000);
+ }
+
+ return (DAT_SUCCESS);
+}
+
+DAT_RETURN dapli_ib_thread_init(void)
+{
+ DAT_RETURN dat_status;
+
+ dapl_os_lock(&g_hca_lock);
+ if (g_ib_thread_state != IB_THREAD_INIT) {
+ dapl_os_unlock(&g_hca_lock);
+ return DAT_SUCCESS;
+ }
+
+ g_ib_thread_state = IB_THREAD_CREATE;
+ dapl_os_unlock(&g_hca_lock);
+
+ /* create thread to process inbound connect request */
+ dat_status = dapl_os_thread_create(dapli_thread, NULL, &g_ib_thread);
+ if (dat_status != DAT_SUCCESS)
+ return (dapl_convert_errno(errno,
+ "create_thread ERR:"
+ " check resource limits"));
+
+ /* wait for thread to start */
+ dapl_os_lock(&g_hca_lock);
+ while (g_ib_thread_state != IB_THREAD_RUN) {
+ dapl_dbg_log(DAPL_DBG_TYPE_UTIL,
+ " ib_thread_init: waiting for ib_thread\n");
+ dapl_os_unlock(&g_hca_lock);
+ dapl_os_sleep_usec(1000);
+ dapl_os_lock(&g_hca_lock);
+ }
+ dapl_os_unlock(&g_hca_lock);
+
+ return DAT_SUCCESS;
+}
+
+void dapli_ib_thread_destroy(void)
+{
+ dapl_dbg_log(DAPL_DBG_TYPE_UTIL,
+ " ib_thread_destroy(%d)\n", dapl_os_getpid());
+ /*
+ * wait for async thread to terminate.
+ * pthread_join would be the correct method
+ * but some applications have some issues
+ */
+
+ /* destroy ib_thread, wait for termination, if not already */
+ dapl_os_lock(&g_hca_lock);
+ if (g_ib_thread_state != IB_THREAD_RUN)
+ goto bail;
+
+ g_ib_thread_state = IB_THREAD_CANCEL;
+ while (g_ib_thread_state != IB_THREAD_EXIT) {
+ dapl_dbg_log(DAPL_DBG_TYPE_UTIL,
+ " ib_thread_destroy: waiting for ib_thread\n");
+ if (dapls_thread_signal() == -1)
+ dapl_log(DAPL_DBG_TYPE_UTIL,
+ " destroy: thread wakeup error = %s\n",
+ strerror(errno));
+ dapl_os_unlock(&g_hca_lock);
+ dapl_os_sleep_usec(2000);
+ dapl_os_lock(&g_hca_lock);
+ }
+bail:
+ dapl_os_unlock(&g_hca_lock);
+
+ dapl_dbg_log(DAPL_DBG_TYPE_UTIL,
+ " ib_thread_destroy(%d) exit\n", dapl_os_getpid());
+}
+
+
+#if defined(_WIN64) || defined(_WIN32)
+/* work thread for uAT, uCM, CQ, and async events */
+void dapli_thread(void *arg)
+{
+ struct _ib_hca_transport *hca;
+ struct _ib_hca_transport *uhca[8];
+ int ret, idx, cnt;
+
+ dapl_dbg_log(DAPL_DBG_TYPE_UTIL, " ib_thread(%d,0x%x): ENTER: \n",
+ dapl_os_getpid(), g_ib_thread);
+
+ dapl_os_lock(&g_hca_lock);
+ for (g_ib_thread_state = IB_THREAD_RUN;
+ g_ib_thread_state == IB_THREAD_RUN;
+ dapl_os_lock(&g_hca_lock)) {
+
+ CompSetZero(&ufds);
+ idx = 0;
+ hca = dapl_llist_is_empty(&g_hca_list) ? NULL :
+ dapl_llist_peek_head(&g_hca_list);
+
+ while (hca) {
+ CompSetAdd(&hca->ib_ctx->channel, &ufds);
+ CompSetAdd(&hca->ib_cq->comp_channel, &ufds);
+ uhca[idx++] = hca;
+ hca = dapl_llist_next_entry(&g_hca_list,
+ (DAPL_LLIST_ENTRY *)
+ &hca->entry);
+ }
+ cnt = idx;
+
+ dapl_os_unlock(&g_hca_lock);
+ ret = CompSetPoll(&ufds, INFINITE);
+
+ dapl_dbg_log(DAPL_DBG_TYPE_UTIL,
+ " ib_thread(%d) poll_event 0x%x\n",
+ dapl_os_getpid(), ret);
+
+
+ /* check and process ASYNC events, per device */
+ for (idx = 0; idx < cnt; idx++) {
+ if (uhca[idx]->destroy == 1) {
+ dapl_os_lock(&g_hca_lock);
+ dapl_llist_remove_entry(&g_hca_list,
+ (DAPL_LLIST_ENTRY *)
+ &uhca[idx]->entry);
+ dapl_os_unlock(&g_hca_lock);
+ uhca[idx]->destroy = 2;
+ } else {
+ dapli_cq_event_cb(uhca[idx]);
+ dapli_async_event_cb(uhca[idx]);
+ }
+ }
+ }
+
+ dapl_dbg_log(DAPL_DBG_TYPE_UTIL, " ib_thread(%d) EXIT\n",
+ dapl_os_getpid());
+ g_ib_thread_state = IB_THREAD_EXIT;
+ dapl_os_unlock(&g_hca_lock);
+}
+#else // _WIN64 || WIN32
+
+/* work thread for uAT, uCM, CQ, and async events */
+void dapli_thread(void *arg)
+{
+ struct pollfd ufds[__FD_SETSIZE];
+ struct _ib_hca_transport *uhca[__FD_SETSIZE] = { NULL };
+ struct _ib_hca_transport *hca;
+ int ret, idx, fds;
+ char rbuf[2];
+
+ dapl_dbg_log(DAPL_DBG_TYPE_THREAD,
+ " ib_thread(%d,0x%x): ENTER: pipe %d \n",
+ dapl_os_getpid(), g_ib_thread, g_ib_pipe[0]);
+
+ /* Poll across pipe, CM, AT never changes */
+ dapl_os_lock(&g_hca_lock);
+ g_ib_thread_state = IB_THREAD_RUN;
+
+ ufds[0].fd = g_ib_pipe[0]; /* pipe */
+ ufds[0].events = POLLIN;
+
+ while (g_ib_thread_state == IB_THREAD_RUN) {
+
+ /* build ufds after pipe and uCMA events */
+ ufds[0].revents = 0;
+ idx = 0;
+
+ /* Walk HCA list and setup async and CQ events */
+ if (!dapl_llist_is_empty(&g_hca_list))
+ hca = dapl_llist_peek_head(&g_hca_list);
+ else
+ hca = NULL;
+
+ while (hca) {
+
+ /* uASYNC events */
+ ufds[++idx].fd = hca->ib_ctx->async_fd;
+ ufds[idx].events = POLLIN;
+ ufds[idx].revents = 0;
+ uhca[idx] = hca;
+
+ /* CQ events are non-direct with CNO's */
+ ufds[++idx].fd = hca->ib_cq->fd;
+ ufds[idx].events = POLLIN;
+ ufds[idx].revents = 0;
+ uhca[idx] = hca;
+
+ dapl_dbg_log(DAPL_DBG_TYPE_THREAD,
+ " ib_thread(%d) poll_fd: hca[%d]=%p,"
+ " async=%d pipe=%d \n",
+ dapl_os_getpid(), hca, ufds[idx - 1].fd,
+ ufds[0].fd);
+
+ hca = dapl_llist_next_entry(&g_hca_list,
+ (DAPL_LLIST_ENTRY *)
+ &hca->entry);
+ }
+
+ /* unlock, and setup poll */
+ fds = idx + 1;
+ dapl_os_unlock(&g_hca_lock);
+ ret = poll(ufds, fds, -1);
+ if (ret <= 0) {
+ dapl_dbg_log(DAPL_DBG_TYPE_THREAD,
+ " ib_thread(%d): ERR %s poll\n",
+ dapl_os_getpid(), strerror(errno));
+ dapl_os_lock(&g_hca_lock);
+ continue;
+ }
+
+ dapl_dbg_log(DAPL_DBG_TYPE_THREAD,
+ " ib_thread(%d) poll_event: "
+ " async=0x%x pipe=0x%x \n",
+ dapl_os_getpid(), ufds[idx].revents,
+ ufds[0].revents);
+
+ /* check and process CQ and ASYNC events, per device */
+ for (idx = 1; idx < fds; idx++) {
+ if (ufds[idx].revents == POLLIN) {
+ dapli_cq_event_cb(uhca[idx]);
+ dapli_async_event_cb(uhca[idx]);
+ }
+ }
+
+ /* check and process user events, PIPE */
+ if (ufds[0].revents == POLLIN) {
+ if (read(g_ib_pipe[0], rbuf, 2) == -1)
+ dapl_log(DAPL_DBG_TYPE_THREAD,
+ " cr_thread: pipe rd err= %s\n",
+ strerror(errno));
+
+ /* cleanup any device on list marked for destroy */
+ for (idx = 1; idx < fds; idx++) {
+ if (uhca[idx] && uhca[idx]->destroy == 1) {
+ dapl_os_lock(&g_hca_lock);
+ dapl_llist_remove_entry(
+ &g_hca_list,
+ (DAPL_LLIST_ENTRY*)
+ &uhca[idx]->entry);
+ dapl_os_unlock(&g_hca_lock);
+ uhca[idx]->destroy = 2;
+ }
+ }
+ }
+ dapl_os_lock(&g_hca_lock);
+ }
+
+ dapl_dbg_log(DAPL_DBG_TYPE_THREAD, " ib_thread(%d) EXIT\n",
+ dapl_os_getpid());
+ g_ib_thread_state = IB_THREAD_EXIT;
+ dapl_os_unlock(&g_hca_lock);
+}
+#endif
-/*\r
- * Copyright (c) 2002-2003, Network Appliance, Inc. All rights reserved.\r
- *\r
- * This Software is licensed under one of the following licenses:\r
- *\r
- * 1) under the terms of the "Common Public License 1.0" a copy of which is\r
- * available from the Open Source Initiative, see\r
- * http://www.opensource.org/licenses/cpl.php.\r
- *\r
- * 2) under the terms of the "The BSD License" a copy of which is\r
- * available from the Open Source Initiative, see\r
- * http://www.opensource.org/licenses/bsd-license.php.\r
- *\r
- * 3) under the terms of the "GNU General Public License (GPL) Version 2" a\r
- * copy of which is available from the Open Source Initiative, see\r
- * http://www.opensource.org/licenses/gpl-license.php.\r
- *\r
- * Licensee has the right to choose one of the above licenses.\r
- *\r
- * Redistributions of source code must retain the above copyright\r
- * notice and one of the license notices.\r
- *\r
- * Redistributions in binary form must reproduce both the above copyright\r
- * notice, one of the license notices in the documentation\r
- * and/or other materials provided with the distribution.\r
- */\r
-\r
-/**********************************************************************\r
- * \r
- * MODULE: dapl_evd_wait.c\r
- *\r
- * PURPOSE: EVENT management\r
- *\r
- * Description: Interfaces in this file are completely defined in \r
- * the uDAPL 1.1 API specification\r
- *\r
- * $Id:$\r
- **********************************************************************/\r
-\r
-#include "dapl.h"\r
-#include "dapl_evd_util.h"\r
-#include "dapl_ring_buffer_util.h"\r
-#include "dapl_adapter_util.h"\r
-\r
-/*\r
- * dapl_evd_wait\r
- *\r
- * UDAPL Requirements Version xxx, \r
- *\r
- * Wait, up to specified timeout, for notification event on EVD.\r
- * Then return first available event.\r
- *\r
- * Input:\r
- * evd_handle\r
- * timeout\r
- *\r
- * Output:\r
- * event\r
- *\r
- * Returns:\r
- * DAT_SUCCESS\r
- * DAT_INVALID_PARAMETER\r
- * DAT_INVALID_STATE\r
- */\r
-\r
-DAT_RETURN DAT_API dapl_evd_wait(IN DAT_EVD_HANDLE evd_handle,\r
- IN DAT_TIMEOUT time_out,\r
- IN DAT_COUNT threshold,\r
- OUT DAT_EVENT * event, OUT DAT_COUNT * nmore)\r
-{\r
- DAPL_EVD *evd_ptr;\r
- DAT_RETURN dat_status;\r
- DAT_EVENT *local_event;\r
- DAT_BOOLEAN notify_requested = DAT_FALSE;\r
- DAT_BOOLEAN waitable;\r
- DAPL_EVD_STATE evd_state;\r
-\r
- dapl_dbg_log(DAPL_DBG_TYPE_API,\r
- "dapl_evd_wait (%p, %d, %d, %p, %p)\n",\r
- evd_handle, time_out, threshold, event, nmore);\r
-\r
- evd_ptr = (DAPL_EVD *) evd_handle;\r
- dat_status = DAT_SUCCESS;\r
-\r
- if (DAPL_BAD_HANDLE(evd_ptr, DAPL_MAGIC_EVD)) {\r
- /*\r
- * We return directly rather than bailing because\r
- * bailing attempts to update the evd, and we don't have\r
- * one.\r
- */\r
- dat_status = DAT_ERROR(DAT_INVALID_HANDLE, 0);\r
- goto bail;\r
- }\r
- if (!event) {\r
- dat_status = DAT_ERROR(DAT_INVALID_PARAMETER, DAT_INVALID_ARG4);\r
- goto bail;\r
- }\r
- if (!nmore) {\r
- dat_status = DAT_ERROR(DAT_INVALID_PARAMETER, DAT_INVALID_ARG5);\r
- goto bail;\r
- }\r
- if (threshold <= 0 ||\r
- (threshold > 1\r
- && evd_ptr->completion_type != DAPL_EVD_STATE_THRESHOLD)\r
- || threshold > evd_ptr->qlen) {\r
- dat_status = DAT_ERROR(DAT_INVALID_PARAMETER, DAT_INVALID_ARG3);\r
- goto bail;\r
- }\r
- if (evd_ptr->catastrophic_overflow) {\r
- dat_status = DAT_ERROR(DAT_INVALID_STATE, 0);\r
- goto bail;\r
- }\r
- DAPL_CNTR(evd_ptr, DCNT_EVD_WAIT);\r
-\r
- dapl_dbg_log(DAPL_DBG_TYPE_EVD,\r
- "dapl_evd_wait: EVD %p, CQ %p\n",\r
- evd_ptr, (void *)evd_ptr->ib_cq_handle);\r
-\r
- /*\r
- * Make sure there are no other waiters and the evd is active.\r
- * Currently this means only the OPEN state is allowed.\r
- * Do this atomically. We need to take a lock to synchronize\r
- * with dapl_evd_dequeue(), but the atomic transition allows\r
- * non-locking synchronization with dapl_evd_query() and\r
- * dapl_evd_{query,enable,disable,{set,clear}_unwaitable}.\r
- */\r
-\r
- dapl_os_lock(&evd_ptr->header.lock);\r
- waitable = evd_ptr->evd_waitable;\r
-\r
- dapl_os_assert(sizeof(DAT_COUNT) == sizeof(DAPL_EVD_STATE));\r
- evd_state = evd_ptr->evd_state;\r
- if (evd_ptr->evd_state == DAPL_EVD_STATE_OPEN)\r
- evd_ptr->evd_state = DAPL_EVD_STATE_WAITED;\r
-\r
- if (evd_state != DAPL_EVD_STATE_OPEN) {\r
- /* Bogus state, bail out */\r
- dat_status = DAT_ERROR(DAT_INVALID_STATE, 0);\r
- goto bail;\r
- }\r
-\r
- if (!waitable) {\r
- /* This EVD is not waitable, reset the state and bail */\r
- if (evd_ptr->evd_state == DAPL_EVD_STATE_WAITED)\r
- evd_ptr->evd_state = evd_state;\r
-\r
- dat_status =\r
- DAT_ERROR(DAT_INVALID_STATE,\r
- DAT_INVALID_STATE_EVD_UNWAITABLE);\r
- goto bail;\r
- }\r
-\r
- /*\r
- * We now own the EVD, even though we don't have the lock anymore,\r
- * because we're in the WAITED state.\r
- */\r
-\r
- evd_ptr->threshold = threshold;\r
-\r
- for (;;) {\r
- /*\r
- * Ideally we'd just check the number of entries on the CQ, but\r
- * we don't have a way to do that. Because we have to set *nmore\r
- * at some point in this routine, we'll need to do this copy\r
- * sometime even if threshold == 1.\r
- *\r
- * For connection evd or async evd, the function checks and\r
- * return right away if the ib_cq_handle associate with these evd\r
- * equal to IB_INVALID_HANDLE\r
- */\r
- dapl_os_unlock(&evd_ptr->header.lock);\r
- dapls_evd_copy_cq(evd_ptr);\r
- dapl_os_lock(&evd_ptr->header.lock);\r
-\r
- if (dapls_rbuf_count(&evd_ptr->pending_event_queue) >=\r
- threshold) {\r
- break;\r
- }\r
-\r
- /*\r
- * Do not enable the completion notification if this evd is not \r
- * a DTO_EVD or RMR_BIND_EVD\r
- */\r
- if ((!notify_requested) &&\r
- (evd_ptr->evd_flags & (DAT_EVD_DTO_FLAG | DAT_EVD_RMR_BIND_FLAG))) {\r
- dat_status =\r
- dapls_ib_completion_notify(evd_ptr->header.\r
- owner_ia->hca_ptr->\r
- ib_hca_handle, evd_ptr,\r
- (evd_ptr->\r
- completion_type ==\r
- DAPL_EVD_STATE_SOLICITED_WAIT)\r
- ? IB_NOTIFY_ON_SOLIC_COMP\r
- :\r
- IB_NOTIFY_ON_NEXT_COMP);\r
-\r
- DAPL_CNTR(evd_ptr, DCNT_EVD_WAIT_NOTIFY);\r
- /* FIXME report error */\r
- dapl_os_assert(dat_status == DAT_SUCCESS);\r
-\r
- notify_requested = DAT_TRUE;\r
-\r
- /* Try again. */\r
- continue;\r
- }\r
-\r
- /*\r
- * Unused by poster; it has no way to tell how many\r
- * items are on the queue without copying them over to the\r
- * EVD queue, and we're the only ones allowed to dequeue\r
- * from the CQ for synchronization/locking reasons.\r
- */\r
- evd_ptr->threshold = threshold;\r
-\r
- DAPL_CNTR(evd_ptr, DCNT_EVD_WAIT_BLOCKED);\r
- dapl_os_unlock(&evd_ptr->header.lock);\r
-\r
- if ((!evd_ptr->cno_ptr) && \r
- (evd_ptr->evd_flags & (DAT_EVD_DTO_FLAG | DAT_EVD_RMR_BIND_FLAG))) {\r
- dat_status = dapls_evd_dto_wait(evd_ptr, time_out);\r
- } else {\r
- dat_status = dapl_os_wait_object_wait(&evd_ptr->wait_object, time_out);\r
- }\r
-\r
- dapl_os_lock(&evd_ptr->header.lock);\r
-\r
- /*\r
- * FIXME: if the thread loops around and waits again\r
- * the time_out value needs to be updated.\r
- */\r
-\r
- notify_requested = DAT_FALSE; /* We've used it up. */\r
-\r
- /* See if we were awakened by evd_set_unwaitable */\r
- if (!evd_ptr->evd_waitable) {\r
- dat_status = DAT_ERROR(DAT_INVALID_STATE, 0);\r
- }\r
-\r
- if (dat_status != DAT_SUCCESS) {\r
- /*\r
- * If the status is DAT_TIMEOUT, we'll break out of the\r
- * loop, *not* dequeue an event (because dat_status\r
- * != DAT_SUCCESS), set *nmore (as we should for timeout)\r
- * and return DAT_TIMEOUT.\r
- */\r
- break;\r
- }\r
- }\r
-\r
- evd_ptr->evd_state = DAPL_EVD_STATE_OPEN;\r
-\r
- if (dat_status == DAT_SUCCESS) {\r
- local_event = dapls_rbuf_remove(&evd_ptr->pending_event_queue);\r
- *event = *local_event;\r
- dapls_rbuf_add(&evd_ptr->free_event_queue, local_event);\r
- }\r
-\r
- /*\r
- * Valid if dat_status == DAT_SUCCESS || dat_status == DAT_TIMEOUT\r
- * Undefined otherwise, so ok to set it.\r
- */\r
- *nmore = dapls_rbuf_count(&evd_ptr->pending_event_queue);\r
-\r
- bail:\r
- dapl_os_unlock(&evd_ptr->header.lock);\r
- if (dat_status) {\r
- dapl_dbg_log(DAPL_DBG_TYPE_RTN,\r
- "dapl_evd_wait () returns 0x%x\n", dat_status);\r
- }\r
- return dat_status;\r
-}\r
+/*
+ * Copyright (c) 2002-2003, Network Appliance, Inc. All rights reserved.
+ *
+ * This Software is licensed under one of the following licenses:
+ *
+ * 1) under the terms of the "Common Public License 1.0" a copy of which is
+ * available from the Open Source Initiative, see
+ * http://www.opensource.org/licenses/cpl.php.
+ *
+ * 2) under the terms of the "The BSD License" a copy of which is
+ * available from the Open Source Initiative, see
+ * http://www.opensource.org/licenses/bsd-license.php.
+ *
+ * 3) under the terms of the "GNU General Public License (GPL) Version 2" a
+ * copy of which is available from the Open Source Initiative, see
+ * http://www.opensource.org/licenses/gpl-license.php.
+ *
+ * Licensee has the right to choose one of the above licenses.
+ *
+ * Redistributions of source code must retain the above copyright
+ * notice and one of the license notices.
+ *
+ * Redistributions in binary form must reproduce both the above copyright
+ * notice, one of the license notices in the documentation
+ * and/or other materials provided with the distribution.
+ */
+
+/**********************************************************************
+ *
+ * MODULE: dapl_evd_wait.c
+ *
+ * PURPOSE: EVENT management
+ *
+ * Description: Interfaces in this file are completely defined in
+ * the uDAPL 1.1 API specification
+ *
+ * $Id:$
+ **********************************************************************/
+
+#include "dapl.h"
+#include "dapl_evd_util.h"
+#include "dapl_ring_buffer_util.h"
+#include "dapl_adapter_util.h"
+
+/*
+ * dapl_evd_wait
+ *
+ * UDAPL Requirements Version xxx,
+ *
+ * Wait, up to specified timeout, for notification event on EVD.
+ * Then return first available event.
+ *
+ * Input:
+ * evd_handle
+ * timeout
+ *
+ * Output:
+ * event
+ *
+ * Returns:
+ * DAT_SUCCESS
+ * DAT_INVALID_PARAMETER
+ * DAT_INVALID_STATE
+ */
+
+DAT_RETURN DAT_API dapl_evd_wait(IN DAT_EVD_HANDLE evd_handle,
+ IN DAT_TIMEOUT time_out,
+ IN DAT_COUNT threshold,
+ OUT DAT_EVENT * event, OUT DAT_COUNT * nmore)
+{
+ DAPL_EVD *evd_ptr;
+ DAT_RETURN dat_status;
+ DAT_EVENT *local_event;
+ DAT_BOOLEAN notify_requested = DAT_FALSE;
+ DAT_BOOLEAN waitable;
+ DAPL_EVD_STATE evd_state;
+
+ dapl_dbg_log(DAPL_DBG_TYPE_API,
+ "dapl_evd_wait (%p, %d, %d, %p, %p)\n",
+ evd_handle, time_out, threshold, event, nmore);
+
+ evd_ptr = (DAPL_EVD *) evd_handle;
+ dat_status = DAT_SUCCESS;
+
+ if (DAPL_BAD_HANDLE(evd_ptr, DAPL_MAGIC_EVD)) {
+ /*
+ * We return directly rather than bailing because
+ * bailing attempts to update the evd, and we don't have
+ * one.
+ */
+ dat_status = DAT_ERROR(DAT_INVALID_HANDLE, 0);
+ goto bail;
+ }
+ if (!event) {
+ dat_status = DAT_ERROR(DAT_INVALID_PARAMETER, DAT_INVALID_ARG4);
+ goto bail;
+ }
+ if (!nmore) {
+ dat_status = DAT_ERROR(DAT_INVALID_PARAMETER, DAT_INVALID_ARG5);
+ goto bail;
+ }
+ if (threshold <= 0 ||
+ (threshold > 1
+ && evd_ptr->completion_type != DAPL_EVD_STATE_THRESHOLD)
+ || threshold > evd_ptr->qlen) {
+ dat_status = DAT_ERROR(DAT_INVALID_PARAMETER, DAT_INVALID_ARG3);
+ goto bail;
+ }
+ if (evd_ptr->catastrophic_overflow) {
+ dat_status = DAT_ERROR(DAT_INVALID_STATE, 0);
+ goto bail;
+ }
+ DAPL_CNTR(evd_ptr, DCNT_EVD_WAIT);
+
+ dapl_dbg_log(DAPL_DBG_TYPE_EVD,
+ "dapl_evd_wait: EVD %p, CQ %p\n",
+ evd_ptr, (void *)evd_ptr->ib_cq_handle);
+
+ /*
+ * Make sure there are no other waiters and the evd is active.
+ * Currently this means only the OPEN state is allowed.
+ * Do this atomically. We need to take a lock to synchronize
+ * with dapl_evd_dequeue(), but the atomic transition allows
+ * non-locking synchronization with dapl_evd_query() and
+ * dapl_evd_{query,enable,disable,{set,clear}_unwaitable}.
+ */
+
+ dapl_os_lock(&evd_ptr->header.lock);
+ waitable = evd_ptr->evd_waitable;
+
+ dapl_os_assert(sizeof(DAT_COUNT) == sizeof(DAPL_EVD_STATE));
+ evd_state = evd_ptr->evd_state;
+ if (evd_ptr->evd_state == DAPL_EVD_STATE_OPEN)
+ evd_ptr->evd_state = DAPL_EVD_STATE_WAITED;
+
+ if (evd_state != DAPL_EVD_STATE_OPEN) {
+ /* Bogus state, bail out */
+ dat_status = DAT_ERROR(DAT_INVALID_STATE, 0);
+ goto bail;
+ }
+
+ if (!waitable) {
+ /* This EVD is not waitable, reset the state and bail */
+ if (evd_ptr->evd_state == DAPL_EVD_STATE_WAITED)
+ evd_ptr->evd_state = evd_state;
+
+ dat_status =
+ DAT_ERROR(DAT_INVALID_STATE,
+ DAT_INVALID_STATE_EVD_UNWAITABLE);
+ goto bail;
+ }
+
+ /*
+ * We now own the EVD, even though we don't have the lock anymore,
+ * because we're in the WAITED state.
+ */
+
+ evd_ptr->threshold = threshold;
+
+ for (;;) {
+ /*
+ * Ideally we'd just check the number of entries on the CQ, but
+ * we don't have a way to do that. Because we have to set *nmore
+ * at some point in this routine, we'll need to do this copy
+ * sometime even if threshold == 1.
+ *
+ * For connection evd or async evd, the function checks and
+ * return right away if the ib_cq_handle associate with these evd
+ * equal to IB_INVALID_HANDLE
+ */
+ dapl_os_unlock(&evd_ptr->header.lock);
+ dapls_evd_copy_cq(evd_ptr);
+ dapl_os_lock(&evd_ptr->header.lock);
+
+ if (dapls_rbuf_count(&evd_ptr->pending_event_queue) >=
+ threshold) {
+ break;
+ }
+
+ /*
+ * Do not enable the completion notification if this evd is not
+ * a DTO_EVD or RMR_BIND_EVD
+ */
+ if ((!notify_requested) &&
+ (evd_ptr->evd_flags & (DAT_EVD_DTO_FLAG | DAT_EVD_RMR_BIND_FLAG))) {
+ dat_status =
+ dapls_ib_completion_notify(evd_ptr->header.
+ owner_ia->hca_ptr->
+ ib_hca_handle, evd_ptr,
+ (evd_ptr->
+ completion_type ==
+ DAPL_EVD_STATE_SOLICITED_WAIT)
+ ? IB_NOTIFY_ON_SOLIC_COMP
+ :
+ IB_NOTIFY_ON_NEXT_COMP);
+
+ DAPL_CNTR(evd_ptr, DCNT_EVD_WAIT_NOTIFY);
+ /* FIXME report error */
+ dapl_os_assert(dat_status == DAT_SUCCESS);
+
+ notify_requested = DAT_TRUE;
+
+ /* Try again. */
+ continue;
+ }
+
+ /*
+ * Unused by poster; it has no way to tell how many
+ * items are on the queue without copying them over to the
+ * EVD queue, and we're the only ones allowed to dequeue
+ * from the CQ for synchronization/locking reasons.
+ */
+ evd_ptr->threshold = threshold;
+
+ DAPL_CNTR(evd_ptr, DCNT_EVD_WAIT_BLOCKED);
+ dapl_os_unlock(&evd_ptr->header.lock);
+
+ if ((!evd_ptr->cno_ptr) &&
+ (evd_ptr->evd_flags & (DAT_EVD_DTO_FLAG | DAT_EVD_RMR_BIND_FLAG))) {
+ dat_status = dapls_evd_dto_wait(evd_ptr, time_out);
+ } else {
+ dat_status = dapl_os_wait_object_wait(&evd_ptr->wait_object, time_out);
+ }
+
+ dapl_os_lock(&evd_ptr->header.lock);
+
+ /*
+ * FIXME: if the thread loops around and waits again
+ * the time_out value needs to be updated.
+ */
+
+ notify_requested = DAT_FALSE; /* We've used it up. */
+
+ /* See if we were awakened by evd_set_unwaitable */
+ if (!evd_ptr->evd_waitable) {
+ dat_status = DAT_ERROR(DAT_INVALID_STATE, 0);
+ }
+
+ if (dat_status != DAT_SUCCESS) {
+ /*
+ * If the status is DAT_TIMEOUT, we'll break out of the
+ * loop, *not* dequeue an event (because dat_status
+ * != DAT_SUCCESS), set *nmore (as we should for timeout)
+ * and return DAT_TIMEOUT.
+ */
+ break;
+ }
+ }
+
+ evd_ptr->evd_state = DAPL_EVD_STATE_OPEN;
+
+ if (dat_status == DAT_SUCCESS) {
+ local_event = dapls_rbuf_remove(&evd_ptr->pending_event_queue);
+ *event = *local_event;
+ dapls_rbuf_add(&evd_ptr->free_event_queue, local_event);
+ }
+
+ /*
+ * Valid if dat_status == DAT_SUCCESS || dat_status == DAT_TIMEOUT
+ * Undefined otherwise, so ok to set it.
+ */
+ *nmore = dapls_rbuf_count(&evd_ptr->pending_event_queue);
+
+ bail:
+ dapl_os_unlock(&evd_ptr->header.lock);
+ if (dat_status) {
+ dapl_dbg_log(DAPL_DBG_TYPE_RTN,
+ "dapl_evd_wait () returns 0x%x\n", dat_status);
+ }
+ return dat_status;
+}
-/*\r
- * Copyright (c) 2007 Intel Corporation. All rights reserved.\r
- * \r
- * This Software is licensed under one of the following licenses:\r
- * \r
- * 1) under the terms of the "Common Public License 1.0" a copy of which is\r
- * in the file LICENSE.txt in the root directory. The license is also\r
- * available from the Open Source Initiative, see \r
- * http://www.opensource.org/licenses/cpl.php.\r
- * \r
- * 2) under the terms of the "The BSD License" a copy of which is in the file\r
- * LICENSE2.txt in the root directory. The license is also available from\r
- * the Open Source Initiative, see\r
- * http://www.opensource.org/licenses/bsd-license.php.\r
- * \r
- * 3) under the terms of the "GNU General Public License (GPL) Version 2" a\r
- * copy of which is in the file LICENSE3.txt in the root directory. The\r
- * license is also available from the Open Source Initiative, see\r
- * http://www.opensource.org/licenses/gpl-license.php.\r
- * \r
- * Licensee has the right to choose one of the above licenses.\r
- * \r
- * Redistributions of source code must retain the above copyright\r
- * notice and one of the license notices.\r
- * \r
- * Redistributions in binary form must reproduce both the above copyright\r
- * notice, one of the license notices in the documentation\r
- * and/or other materials provided with the distribution.\r
- */\r
-\r
-/**********************************************************************\r
- *\r
- * HEADER: dat_ib_extensions.h\r
- *\r
- * PURPOSE: extensions to the DAT API for IB transport specific services \r
- * NOTE: Prototyped IB extension support in openib-cma 1.2 provider.\r
- * Applications MUST recompile with new dat.h definitions\r
- * and include this file.\r
- *\r
- * Description: Header file for "uDAPL: User Direct Access Programming\r
- * Library, Version: 2.0"\r
- *\r
- * Mapping rules:\r
- * All global symbols are prepended with "DAT_" or "dat_"\r
- * All DAT objects have an 'api' tag which, such as 'ep' or 'lmr'\r
- * The method table is in the provider definition structure.\r
- *\r
- *\r
- **********************************************************************/\r
-#ifndef _DAT_IB_EXTENSIONS_H_\r
-#define _DAT_IB_EXTENSIONS_H_\r
-\r
-/* \r
- * Provider specific attribute strings for extension support \r
- * returned with dat_ia_query() and \r
- * DAT_PROVIDER_ATTR_MASK == DAT_PROVIDER_FIELD_PROVIDER_SPECIFIC_ATTR\r
- *\r
- * DAT_NAMED_ATTR name == extended operations and version, \r
- * version_value = version number of extension API\r
- */\r
-\r
-/* 2.0.1 - Initial IB extension support, atomic and immed data\r
- * dat_ib_post_fetch_and_add()\r
- * dat_ib_post_cmp_and_swap()\r
- * dat_ib_post_rdma_write_immed()\r
- * \r
- * 2.0.2 - Add UD support, post send and remote_ah via connect events \r
- * dat_ib_post_send_ud()\r
- *\r
- * 2.0.3 - Add query/print counter support for IA, EP, and EVD's \r
- * dat_query_counters(), dat_print_counters()\r
- *\r
- * 2.0.4 - Add DAT_IB_UD_CONNECTION_REJECT_EVENT extended UD event\r
- * 2.0.5 - Add DAT_IB_UD extended UD connection error events\r
- *\r
- */\r
-#define DAT_IB_EXTENSION_VERSION 205 /* 2.0.5 */\r
-#define DAT_ATTR_COUNTERS "DAT_COUNTERS"\r
-#define DAT_IB_ATTR_FETCH_AND_ADD "DAT_IB_FETCH_AND_ADD"\r
-#define DAT_IB_ATTR_CMP_AND_SWAP "DAT_IB_CMP_AND_SWAP"\r
-#define DAT_IB_ATTR_IMMED_DATA "DAT_IB_IMMED_DATA"\r
-#define DAT_IB_ATTR_UD "DAT_IB_UD"\r
-\r
-/* \r
- * Definition for extended EVENT numbers, DAT_IB_EXTENSION_BASE_RANGE\r
- * is used by these extensions as a starting point for extended event numbers \r
- *\r
- * DAT_IB_DTO_EVENT - All extended data transfers - req/recv evd\r
- * DAT_IB_AH_EVENT - address handle resolution - connect evd\r
- */\r
-typedef enum dat_ib_event_number\r
-{\r
- DAT_IB_DTO_EVENT = DAT_IB_EXTENSION_RANGE_BASE,\r
- DAT_IB_UD_CONNECTION_REQUEST_EVENT,\r
- DAT_IB_UD_CONNECTION_EVENT_ESTABLISHED,\r
- DAT_IB_UD_CONNECTION_REJECT_EVENT,\r
- DAT_IB_UD_CONNECTION_ERROR_EVENT\r
-\r
-} DAT_IB_EVENT_NUMBER;\r
-\r
-/* \r
- * Extension operations \r
- */\r
-typedef enum dat_ib_op\r
-{\r
- DAT_IB_FETCH_AND_ADD_OP,\r
- DAT_IB_CMP_AND_SWAP_OP,\r
- DAT_IB_RDMA_WRITE_IMMED_OP,\r
- DAT_IB_UD_SEND_OP,\r
- DAT_QUERY_COUNTERS_OP,\r
- DAT_PRINT_COUNTERS_OP\r
- \r
-} DAT_IB_OP;\r
-\r
-/*\r
- * The DAT_IB_EXT_TYPE enum specifies the type of extension operation that just\r
- * completed. All IB extended completion types both, DTO and NON-DTO, are \r
- * reported in the extended operation type with the single DAT_IB_DTO_EVENT type. \r
- * The specific extended DTO operation is reported with a DAT_IB_DTOS type in the \r
- * operation field of the base DAT_EVENT structure. All other extended events are \r
- * identified by unique DAT_IB_EVENT_NUMBER types.\r
- */\r
-typedef enum dat_ib_ext_type\r
-{\r
- DAT_IB_FETCH_AND_ADD, // 0\r
- DAT_IB_CMP_AND_SWAP, // 1\r
- DAT_IB_RDMA_WRITE_IMMED, // 2\r
- DAT_IB_RDMA_WRITE_IMMED_DATA, // 3\r
- DAT_IB_RECV_IMMED_DATA, // 4\r
- DAT_IB_UD_CONNECT_REQUEST, // 5\r
- DAT_IB_UD_REMOTE_AH, // 6\r
- DAT_IB_UD_PASSIVE_REMOTE_AH, // 7\r
- DAT_IB_UD_SEND, // 8\r
- DAT_IB_UD_RECV, // 9\r
- DAT_IB_UD_CONNECT_REJECT, // 10\r
- DAT_IB_UD_CONNECT_ERROR, // 11\r
-\r
-} DAT_IB_EXT_TYPE;\r
-\r
-/* \r
- * Extension event status\r
- */\r
-typedef enum dat_ib_status\r
-{\r
- DAT_OP_SUCCESS = DAT_SUCCESS,\r
- DAT_IB_OP_ERR,\r
-\r
-} DAT_IB_STATUS;\r
-\r
-\r
-/* \r
- * Definitions for additional extension type RETURN codes above\r
- * standard DAT types. Included with standard DAT_TYPE_STATUS \r
- * bits using a DAT_EXTENSION BASE as a starting point.\r
- */\r
-typedef enum dat_ib_return\r
-{\r
- DAT_IB_ERR = DAT_EXTENSION_BASE,\r
-\r
-} DAT_IB_RETURN;\r
-\r
-/* \r
- * Definition for extended IB DTO operations, DAT_DTO_EXTENSION_BASE\r
- * is used by DAT extensions as a starting point of extension DTOs \r
- */\r
-typedef enum dat_ib_dtos\r
-{\r
- DAT_IB_DTO_RDMA_WRITE_IMMED = DAT_DTO_EXTENSION_BASE,\r
- DAT_IB_DTO_RECV_IMMED,\r
- DAT_IB_DTO_FETCH_ADD,\r
- DAT_IB_DTO_CMP_SWAP,\r
- DAT_IB_DTO_RECV_MSG_IMMED,\r
- DAT_IB_DTO_SEND_UD,\r
- DAT_IB_DTO_RECV_UD,\r
- DAT_IB_DTO_RECV_UD_IMMED, \r
-\r
-} DAT_IB_DTOS;\r
-\r
-/* \r
- * Definitions for additional extension handle types beyond \r
- * standard DAT handle. New Bit definitions MUST start at \r
- * DAT_HANDLE_TYPE_EXTENSION_BASE\r
- */\r
-typedef enum dat_ib_handle_type\r
-{\r
- DAT_IB_HANDLE_TYPE_EXT = DAT_HANDLE_TYPE_EXTENSION_BASE,\r
-\r
-} DAT_IB_HANDLE_TYPE;\r
-\r
-/*\r
- * The DAT_IB_EVD_EXTENSION_FLAGS enum specifies the EVD extension flags that\r
- * do not map directly to existing DAT_EVD_FLAGS. This new EVD flag has been \r
- * added to identify an extended EVD that does not fit the existing stream \r
- * types.\r
- */\r
-typedef enum dat_ib_evd_extension_flags\r
-{\r
- DAT_IB_EVD_EXTENSION_FLAG = DAT_EVD_EXTENSION_BASE,\r
-\r
-} DAT_IB_EVD_EXTENSION_FLAGS;\r
-\r
-/* \r
- * Definition for memory privilege extension flags.\r
- * New privileges required for new atomic DTO type extensions.\r
- * New Bit definitions MUST start at DAT_MEM_PRIV_EXTENSION\r
- */\r
-typedef enum dat_ib_mem_priv_flags\r
-{\r
- DAT_IB_MEM_PRIV_REMOTE_ATOMIC = DAT_MEM_PRIV_EXTENSION_BASE,\r
- \r
-} DAT_IB_MEM_PRIV_FLAGS;\r
-\r
-/* \r
- * Definition for IB address handle, unreliable datagram.\r
- */\r
-typedef struct dat_ib_addr_handle\r
-{\r
- struct ibv_ah *ah;\r
- DAT_UINT32 qpn;\r
- DAT_SOCK_ADDR6 ia_addr;\r
- \r
-} DAT_IB_ADDR_HANDLE;\r
-\r
-/* \r
- * Definitions for extended event data:\r
- * When dat_event->event_number >= DAT_IB_EXTENSION_BASE_RANGE\r
- * then dat_event->extension_data == DAT_IB_EXT_EVENT_DATA type\r
- * and ((DAT_IB_EXT_EVENT_DATA*)dat_event->extension_data)->type\r
- * specifies extension data values. \r
- * NOTE: DAT_IB_EXT_EVENT_DATA cannot exceed 64 bytes as defined by \r
- * "DAT_UINT64 extension_data[8]" in DAT_EVENT (dat.h)\r
- */\r
-typedef struct dat_ib_immed_data \r
-{\r
- DAT_UINT32 data;\r
-\r
-} DAT_IB_IMMED_DATA;\r
-\r
-/* \r
- * Definitions for extended event data:\r
- * When dat_event->event_number >= DAT_IB_EXTENSION_BASE_RANGE\r
- * then dat_event->extension_data == DAT_EXTENSION_EVENT_DATA type\r
- * and ((DAT_EXTENSION_EVENT_DATA*)dat_event->extension_data)->type\r
- * specifies extension data values. \r
- * NOTE: DAT_EXTENSION_EVENT_DATA cannot exceed 64 bytes as defined by \r
- * "DAT_UINT64 extension_data[8]" in DAT_EVENT (dat.h)\r
- *\r
- * Provide UD address handles via extended connect establishment. \r
- * ia_addr provided with extended conn events for reference.\r
- */\r
-typedef struct dat_ib_extension_event_data\r
-{\r
- DAT_IB_EXT_TYPE type;\r
- DAT_IB_STATUS status;\r
- union {\r
- DAT_IB_IMMED_DATA immed;\r
- } val;\r
- DAT_IB_ADDR_HANDLE remote_ah; \r
-\r
-} DAT_IB_EXTENSION_EVENT_DATA;\r
-\r
-/* \r
- * Definitions for additional extension handle types beyond \r
- * standard DAT handle. New Bit definitions MUST start at \r
- * DAT_HANDLE_TYPE_EXTENSION_BASE\r
- */\r
-typedef enum dat_ib_service_type\r
-{\r
- DAT_IB_SERVICE_TYPE_UD = DAT_SERVICE_TYPE_EXTENSION_BASE,\r
-\r
-} DAT_IB_SERVICE_TYPE;\r
-\r
-/*\r
- * Definitions for 64-bit IA Counters\r
- */\r
-typedef enum dat_ia_counters\r
-{\r
- DCNT_IA_PZ_CREATE,\r
- DCNT_IA_PZ_FREE,\r
- DCNT_IA_LMR_CREATE,\r
- DCNT_IA_LMR_FREE,\r
- DCNT_IA_RMR_CREATE,\r
- DCNT_IA_RMR_FREE,\r
- DCNT_IA_PSP_CREATE,\r
- DCNT_IA_PSP_CREATE_ANY,\r
- DCNT_IA_PSP_FREE,\r
- DCNT_IA_RSP_CREATE,\r
- DCNT_IA_RSP_FREE,\r
- DCNT_IA_EVD_CREATE,\r
- DCNT_IA_EVD_FREE,\r
- DCNT_IA_EP_CREATE,\r
- DCNT_IA_EP_FREE,\r
- DCNT_IA_SRQ_CREATE,\r
- DCNT_IA_SRQ_FREE,\r
- DCNT_IA_SP_CR,\r
- DCNT_IA_SP_CR_ACCEPTED,\r
- DCNT_IA_SP_CR_REJECTED,\r
- DCNT_IA_MEM_ALLOC,\r
- DCNT_IA_MEM_ALLOC_DATA,\r
- DCNT_IA_MEM_FREE,\r
- DCNT_IA_ASYNC_ERROR,\r
- DCNT_IA_ASYNC_QP_ERROR,\r
- DCNT_IA_ASYNC_CQ_ERROR,\r
- DCNT_IA_ALL_COUNTERS, /* MUST be last */\r
-\r
-} DAT_IA_COUNTERS;\r
-\r
-/*\r
- * Definitions for 64-bit EP Counters\r
- */\r
-typedef enum dat_ep_counters\r
-{\r
- DCNT_EP_CONNECT,\r
- DCNT_EP_DISCONNECT,\r
- DCNT_EP_POST_SEND,\r
- DCNT_EP_POST_SEND_DATA,\r
- DCNT_EP_POST_SEND_UD,\r
- DCNT_EP_POST_SEND_UD_DATA,\r
- DCNT_EP_POST_RECV,\r
- DCNT_EP_POST_RECV_DATA,\r
- DCNT_EP_POST_WRITE,\r
- DCNT_EP_POST_WRITE_DATA,\r
- DCNT_EP_POST_WRITE_IMM,\r
- DCNT_EP_POST_WRITE_IMM_DATA,\r
- DCNT_EP_POST_READ,\r
- DCNT_EP_POST_READ_DATA,\r
- DCNT_EP_POST_CMP_SWAP,\r
- DCNT_EP_POST_FETCH_ADD,\r
- DCNT_EP_RECV,\r
- DCNT_EP_RECV_DATA,\r
- DCNT_EP_RECV_UD,\r
- DCNT_EP_RECV_UD_DATA,\r
- DCNT_EP_RECV_IMM,\r
- DCNT_EP_RECV_IMM_DATA,\r
- DCNT_EP_RECV_RDMA_IMM,\r
- DCNT_EP_RECV_RDMA_IMM_DATA,\r
- DCNT_EP_ALL_COUNTERS, /* MUST be last */\r
-\r
-} DAT_EP_COUNTERS;\r
-\r
-/*\r
- * Definitions for 64-bit EVD Counters\r
- */\r
-typedef enum dat_evd_counters\r
-{\r
- DCNT_EVD_WAIT,\r
- DCNT_EVD_WAIT_BLOCKED,\r
- DCNT_EVD_WAIT_NOTIFY,\r
- DCNT_EVD_DEQUEUE,\r
- DCNT_EVD_DEQUEUE_FOUND,\r
- DCNT_EVD_DEQUEUE_NOT_FOUND,\r
- DCNT_EVD_DEQUEUE_POLL,\r
- DCNT_EVD_DEQUEUE_POLL_FOUND,\r
- DCNT_EVD_CONN_CALLBACK,\r
- DCNT_EVD_DTO_CALLBACK,\r
- DCNT_EVD_ALL_COUNTERS, /* MUST be last */\r
-\r
-} DAT_EVD_COUNTERS;\r
-\r
-/* Extended RETURN and EVENT STATUS string helper functions */\r
-\r
-/* DAT_EXT_RETURN error to string */\r
-static __inline__ DAT_RETURN DAT_API\r
-dat_strerror_extension (\r
- IN DAT_IB_RETURN value,\r
- OUT const char **message )\r
-{\r
- switch( DAT_GET_TYPE(value) ) {\r
- case DAT_IB_ERR:\r
- *message = "DAT_IB_ERR";\r
- return DAT_SUCCESS;\r
- default:\r
- /* standard DAT return type */\r
- return(dat_strerror(value, message, NULL));\r
- }\r
-}\r
-\r
-/* DAT_EXT_STATUS error to string */\r
-static __inline__ DAT_RETURN DAT_API\r
-dat_strerror_ext_status (\r
- IN DAT_IB_STATUS value,\r
- OUT const char **message )\r
-{\r
- switch(value) {\r
- case 0:\r
- *message = " ";\r
- return DAT_SUCCESS;\r
- case DAT_IB_OP_ERR:\r
- *message = "DAT_IB_OP_ERR";\r
- return DAT_SUCCESS;\r
- default:\r
- *message = "unknown extension status";\r
- return DAT_INVALID_PARAMETER;\r
- }\r
-}\r
-\r
-/* \r
- * Extended IB transport specific APIs\r
- * redirection via DAT extension function\r
- */\r
-\r
-/*\r
- * This asynchronous call is modeled after the InfiniBand atomic \r
- * Fetch and Add operation. The add_value is added to the 64 bit \r
- * value stored at the remote memory location specified in remote_iov\r
- * and the result is stored in the local_iov. \r
- */\r
-#define dat_ib_post_fetch_and_add(ep, add_val, lbuf, cookie, rbuf, flgs) \\r
- dat_extension_op( ep, \\r
- DAT_IB_FETCH_AND_ADD_OP, \\r
- (add_val), \\r
- (lbuf), \\r
- (cookie), \\r
- (rbuf), \\r
- (flgs))\r
- \r
-/*\r
- * This asynchronous call is modeled after the InfiniBand atomic \r
- * Compare and Swap operation. The cmp_value is compared to the 64 bit \r
- * value stored at the remote memory location specified in remote_iov. \r
- * If the two values are equal, the 64 bit swap_value is stored in \r
- * the remote memory location. In all cases, the original 64 bit \r
- * value stored in the remote memory location is copied to the local_iov.\r
- */\r
-#define dat_ib_post_cmp_and_swap(ep, cmp_val, swap_val, lbuf, cookie, rbuf, flgs) \\r
- dat_extension_op( ep, \\r
- DAT_IB_CMP_AND_SWAP_OP, \\r
- (cmp_val), \\r
- (swap_val), \\r
- (lbuf), \\r
- (cookie), \\r
- (rbuf), \\r
- (flgs))\r
-\r
-/* \r
- * RDMA Write with IMMEDIATE:\r
- *\r
- * This asynchronous call is modeled after the InfiniBand rdma write with \r
- * immediate data operation. Event completion for the request completes as an \r
- * DAT_EXTENSION with extension type set to DAT_DTO_EXTENSION_IMMED_DATA.\r
- * Event completion on the remote endpoint completes as receive DTO operation\r
- * type of DAT_EXTENSION with operation set to DAT_DTO_EXTENSION_IMMED_DATA.\r
- * The immediate data will be provided in the extented DTO event data structure.\r
- *\r
- * Note to Consumers: the immediate data will consume a receive\r
- * buffer at the Data Sink. \r
- *\r
- * Other extension flags:\r
- * n/a\r
- */\r
-#define dat_ib_post_rdma_write_immed(ep, size, lbuf, cookie, rbuf, idata, flgs) \\r
- dat_extension_op( ep, \\r
- DAT_IB_RDMA_WRITE_IMMED_OP, \\r
- (size), \\r
- (lbuf), \\r
- (cookie), \\r
- (rbuf), \\r
- (idata), \\r
- (flgs))\r
-\r
-/* \r
- * Unreliable datagram: msg send \r
- *\r
- * This asynchronous call is modeled after the InfiniBand UD message send\r
- * Event completion for the request completes as an \r
- * DAT_EXTENSION with extension type set to DAT_DTO_EXTENSION_UD_SEND.\r
- * Event completion on the remote endpoint completes as receive DTO operation\r
- * type of DAT_EXTENSION with operation set to DAT_DTO_EXTENSION_UD_RECV.\r
- *\r
- * Other extension flags:\r
- * n/a\r
- */\r
-#define dat_ib_post_send_ud(ep, segments, lbuf, ah_ptr, cookie, flgs) \\r
- dat_extension_op( ep, \\r
- DAT_IB_UD_SEND_OP, \\r
- (segments), \\r
- (lbuf), \\r
- (ah_ptr), \\r
- (cookie), \\r
- (flgs))\r
-\r
-\r
-/* \r
- * Query counter(s): \r
- * Provide IA, EP, or EVD and call will return appropriate counters\r
- * DAT_HANDLE dat_handle, enum cntr, *DAT_UINT64 p_cntrs_out, int reset\r
- *\r
- * use _ALL_COUNTERS to query all\r
- */\r
-#define dat_query_counters(dat_handle, cntr, p_cntrs_out, reset) \\r
- dat_extension_op( dat_handle, \\r
- DAT_QUERY_COUNTERS_OP, \\r
- (cntr), \\r
- (p_cntrs_out), \\r
- (reset))\r
-/* \r
- * Print counter(s): \r
- * Provide IA, EP, or EVD and call will print appropriate counters\r
- * DAT_HANDLE dat_handle, int cntr, int reset\r
- * \r
- * use _ALL_COUNTERS to print all\r
- */\r
-#define dat_print_counters(dat_handle, cntr, reset) \\r
- dat_extension_op( dat_handle, \\r
- DAT_PRINT_COUNTERS_OP, \\r
- (cntr), \\r
- (reset))\r
-\r
-#endif /* _DAT_IB_EXTENSIONS_H_ */\r
-\r
+/*
+ * Copyright (c) 2007 Intel Corporation. All rights reserved.
+ *
+ * This Software is licensed under one of the following licenses:
+ *
+ * 1) under the terms of the "Common Public License 1.0" a copy of which is
+ * in the file LICENSE.txt in the root directory. The license is also
+ * available from the Open Source Initiative, see
+ * http://www.opensource.org/licenses/cpl.php.
+ *
+ * 2) under the terms of the "The BSD License" a copy of which is in the file
+ * LICENSE2.txt in the root directory. The license is also available from
+ * the Open Source Initiative, see
+ * http://www.opensource.org/licenses/bsd-license.php.
+ *
+ * 3) under the terms of the "GNU General Public License (GPL) Version 2" a
+ * copy of which is in the file LICENSE3.txt in the root directory. The
+ * license is also available from the Open Source Initiative, see
+ * http://www.opensource.org/licenses/gpl-license.php.
+ *
+ * Licensee has the right to choose one of the above licenses.
+ *
+ * Redistributions of source code must retain the above copyright
+ * notice and one of the license notices.
+ *
+ * Redistributions in binary form must reproduce both the above copyright
+ * notice, one of the license notices in the documentation
+ * and/or other materials provided with the distribution.
+ */
+
+/**********************************************************************
+ *
+ * HEADER: dat_ib_extensions.h
+ *
+ * PURPOSE: extensions to the DAT API for IB transport specific services
+ * NOTE: Prototyped IB extension support in openib-cma 1.2 provider.
+ * Applications MUST recompile with new dat.h definitions
+ * and include this file.
+ *
+ * Description: Header file for "uDAPL: User Direct Access Programming
+ * Library, Version: 2.0"
+ *
+ * Mapping rules:
+ * All global symbols are prepended with "DAT_" or "dat_"
+ * All DAT objects have an 'api' tag which, such as 'ep' or 'lmr'
+ * The method table is in the provider definition structure.
+ *
+ *
+ **********************************************************************/
+#ifndef _DAT_IB_EXTENSIONS_H_
+#define _DAT_IB_EXTENSIONS_H_
+
+/*
+ * Provider specific attribute strings for extension support
+ * returned with dat_ia_query() and
+ * DAT_PROVIDER_ATTR_MASK == DAT_PROVIDER_FIELD_PROVIDER_SPECIFIC_ATTR
+ *
+ * DAT_NAMED_ATTR name == extended operations and version,
+ * version_value = version number of extension API
+ */
+
+/* 2.0.1 - Initial IB extension support, atomic and immed data
+ * dat_ib_post_fetch_and_add()
+ * dat_ib_post_cmp_and_swap()
+ * dat_ib_post_rdma_write_immed()
+ *
+ * 2.0.2 - Add UD support, post send and remote_ah via connect events
+ * dat_ib_post_send_ud()
+ *
+ * 2.0.3 - Add query/print counter support for IA, EP, and EVD's
+ * dat_query_counters(), dat_print_counters()
+ *
+ * 2.0.4 - Add DAT_IB_UD_CONNECTION_REJECT_EVENT extended UD event
+ * 2.0.5 - Add DAT_IB_UD extended UD connection error events
+ *
+ */
+#define DAT_IB_EXTENSION_VERSION 205 /* 2.0.5 */
+#define DAT_ATTR_COUNTERS "DAT_COUNTERS"
+#define DAT_IB_ATTR_FETCH_AND_ADD "DAT_IB_FETCH_AND_ADD"
+#define DAT_IB_ATTR_CMP_AND_SWAP "DAT_IB_CMP_AND_SWAP"
+#define DAT_IB_ATTR_IMMED_DATA "DAT_IB_IMMED_DATA"
+#define DAT_IB_ATTR_UD "DAT_IB_UD"
+
+/*
+ * Definition for extended EVENT numbers, DAT_IB_EXTENSION_BASE_RANGE
+ * is used by these extensions as a starting point for extended event numbers
+ *
+ * DAT_IB_DTO_EVENT - All extended data transfers - req/recv evd
+ * DAT_IB_AH_EVENT - address handle resolution - connect evd
+ */
+typedef enum dat_ib_event_number
+{
+ DAT_IB_DTO_EVENT = DAT_IB_EXTENSION_RANGE_BASE,
+ DAT_IB_UD_CONNECTION_REQUEST_EVENT,
+ DAT_IB_UD_CONNECTION_EVENT_ESTABLISHED,
+ DAT_IB_UD_CONNECTION_REJECT_EVENT,
+ DAT_IB_UD_CONNECTION_ERROR_EVENT
+
+} DAT_IB_EVENT_NUMBER;
+
+/*
+ * Extension operations
+ */
+typedef enum dat_ib_op
+{
+ DAT_IB_FETCH_AND_ADD_OP,
+ DAT_IB_CMP_AND_SWAP_OP,
+ DAT_IB_RDMA_WRITE_IMMED_OP,
+ DAT_IB_UD_SEND_OP,
+ DAT_QUERY_COUNTERS_OP,
+ DAT_PRINT_COUNTERS_OP
+
+} DAT_IB_OP;
+
+/*
+ * The DAT_IB_EXT_TYPE enum specifies the type of extension operation that just
+ * completed. All IB extended completion types both, DTO and NON-DTO, are
+ * reported in the extended operation type with the single DAT_IB_DTO_EVENT type.
+ * The specific extended DTO operation is reported with a DAT_IB_DTOS type in the
+ * operation field of the base DAT_EVENT structure. All other extended events are
+ * identified by unique DAT_IB_EVENT_NUMBER types.
+ */
+typedef enum dat_ib_ext_type
+{
+ DAT_IB_FETCH_AND_ADD, // 0
+ DAT_IB_CMP_AND_SWAP, // 1
+ DAT_IB_RDMA_WRITE_IMMED, // 2
+ DAT_IB_RDMA_WRITE_IMMED_DATA, // 3
+ DAT_IB_RECV_IMMED_DATA, // 4
+ DAT_IB_UD_CONNECT_REQUEST, // 5
+ DAT_IB_UD_REMOTE_AH, // 6
+ DAT_IB_UD_PASSIVE_REMOTE_AH, // 7
+ DAT_IB_UD_SEND, // 8
+ DAT_IB_UD_RECV, // 9
+ DAT_IB_UD_CONNECT_REJECT, // 10
+ DAT_IB_UD_CONNECT_ERROR, // 11
+
+} DAT_IB_EXT_TYPE;
+
+/*
+ * Extension event status
+ */
+typedef enum dat_ib_status
+{
+ DAT_OP_SUCCESS = DAT_SUCCESS,
+ DAT_IB_OP_ERR,
+
+} DAT_IB_STATUS;
+
+
+/*
+ * Definitions for additional extension type RETURN codes above
+ * standard DAT types. Included with standard DAT_TYPE_STATUS
+ * bits using a DAT_EXTENSION BASE as a starting point.
+ */
+typedef enum dat_ib_return
+{
+ DAT_IB_ERR = DAT_EXTENSION_BASE,
+
+} DAT_IB_RETURN;
+
+/*
+ * Definition for extended IB DTO operations, DAT_DTO_EXTENSION_BASE
+ * is used by DAT extensions as a starting point of extension DTOs
+ */
+typedef enum dat_ib_dtos
+{
+ DAT_IB_DTO_RDMA_WRITE_IMMED = DAT_DTO_EXTENSION_BASE,
+ DAT_IB_DTO_RECV_IMMED,
+ DAT_IB_DTO_FETCH_ADD,
+ DAT_IB_DTO_CMP_SWAP,
+ DAT_IB_DTO_RECV_MSG_IMMED,
+ DAT_IB_DTO_SEND_UD,
+ DAT_IB_DTO_RECV_UD,
+ DAT_IB_DTO_RECV_UD_IMMED,
+
+} DAT_IB_DTOS;
+
+/*
+ * Definitions for additional extension handle types beyond
+ * standard DAT handle. New Bit definitions MUST start at
+ * DAT_HANDLE_TYPE_EXTENSION_BASE
+ */
+typedef enum dat_ib_handle_type
+{
+ DAT_IB_HANDLE_TYPE_EXT = DAT_HANDLE_TYPE_EXTENSION_BASE,
+
+} DAT_IB_HANDLE_TYPE;
+
+/*
+ * The DAT_IB_EVD_EXTENSION_FLAGS enum specifies the EVD extension flags that
+ * do not map directly to existing DAT_EVD_FLAGS. This new EVD flag has been
+ * added to identify an extended EVD that does not fit the existing stream
+ * types.
+ */
+typedef enum dat_ib_evd_extension_flags
+{
+ DAT_IB_EVD_EXTENSION_FLAG = DAT_EVD_EXTENSION_BASE,
+
+} DAT_IB_EVD_EXTENSION_FLAGS;
+
+/*
+ * Definition for memory privilege extension flags.
+ * New privileges required for new atomic DTO type extensions.
+ * New Bit definitions MUST start at DAT_MEM_PRIV_EXTENSION
+ */
+typedef enum dat_ib_mem_priv_flags
+{
+ DAT_IB_MEM_PRIV_REMOTE_ATOMIC = DAT_MEM_PRIV_EXTENSION_BASE,
+
+} DAT_IB_MEM_PRIV_FLAGS;
+
+/*
+ * Definition for IB address handle, unreliable datagram.
+ */
+typedef struct dat_ib_addr_handle
+{
+ struct ibv_ah *ah;
+ DAT_UINT32 qpn;
+ DAT_SOCK_ADDR6 ia_addr;
+
+} DAT_IB_ADDR_HANDLE;
+
+/*
+ * Definitions for extended event data:
+ * When dat_event->event_number >= DAT_IB_EXTENSION_BASE_RANGE
+ * then dat_event->extension_data == DAT_IB_EXT_EVENT_DATA type
+ * and ((DAT_IB_EXT_EVENT_DATA*)dat_event->extension_data)->type
+ * specifies extension data values.
+ * NOTE: DAT_IB_EXT_EVENT_DATA cannot exceed 64 bytes as defined by
+ * "DAT_UINT64 extension_data[8]" in DAT_EVENT (dat.h)
+ */
+typedef struct dat_ib_immed_data
+{
+ DAT_UINT32 data;
+
+} DAT_IB_IMMED_DATA;
+
+/*
+ * Definitions for extended event data:
+ * When dat_event->event_number >= DAT_IB_EXTENSION_BASE_RANGE
+ * then dat_event->extension_data == DAT_EXTENSION_EVENT_DATA type
+ * and ((DAT_EXTENSION_EVENT_DATA*)dat_event->extension_data)->type
+ * specifies extension data values.
+ * NOTE: DAT_EXTENSION_EVENT_DATA cannot exceed 64 bytes as defined by
+ * "DAT_UINT64 extension_data[8]" in DAT_EVENT (dat.h)
+ *
+ * Provide UD address handles via extended connect establishment.
+ * ia_addr provided with extended conn events for reference.
+ */
+typedef struct dat_ib_extension_event_data
+{
+ DAT_IB_EXT_TYPE type;
+ DAT_IB_STATUS status;
+ union {
+ DAT_IB_IMMED_DATA immed;
+ } val;
+ DAT_IB_ADDR_HANDLE remote_ah;
+
+} DAT_IB_EXTENSION_EVENT_DATA;
+
+/*
+ * Definitions for additional extension handle types beyond
+ * standard DAT handle. New Bit definitions MUST start at
+ * DAT_HANDLE_TYPE_EXTENSION_BASE
+ */
+typedef enum dat_ib_service_type
+{
+ DAT_IB_SERVICE_TYPE_UD = DAT_SERVICE_TYPE_EXTENSION_BASE,
+
+} DAT_IB_SERVICE_TYPE;
+
+/*
+ * Definitions for 64-bit IA Counters
+ */
+typedef enum dat_ia_counters
+{
+ DCNT_IA_PZ_CREATE,
+ DCNT_IA_PZ_FREE,
+ DCNT_IA_LMR_CREATE,
+ DCNT_IA_LMR_FREE,
+ DCNT_IA_RMR_CREATE,
+ DCNT_IA_RMR_FREE,
+ DCNT_IA_PSP_CREATE,
+ DCNT_IA_PSP_CREATE_ANY,
+ DCNT_IA_PSP_FREE,
+ DCNT_IA_RSP_CREATE,
+ DCNT_IA_RSP_FREE,
+ DCNT_IA_EVD_CREATE,
+ DCNT_IA_EVD_FREE,
+ DCNT_IA_EP_CREATE,
+ DCNT_IA_EP_FREE,
+ DCNT_IA_SRQ_CREATE,
+ DCNT_IA_SRQ_FREE,
+ DCNT_IA_SP_CR,
+ DCNT_IA_SP_CR_ACCEPTED,
+ DCNT_IA_SP_CR_REJECTED,
+ DCNT_IA_MEM_ALLOC,
+ DCNT_IA_MEM_ALLOC_DATA,
+ DCNT_IA_MEM_FREE,
+ DCNT_IA_ASYNC_ERROR,
+ DCNT_IA_ASYNC_QP_ERROR,
+ DCNT_IA_ASYNC_CQ_ERROR,
+ DCNT_IA_ALL_COUNTERS, /* MUST be last */
+
+} DAT_IA_COUNTERS;
+
+/*
+ * Definitions for 64-bit EP Counters
+ */
+typedef enum dat_ep_counters
+{
+ DCNT_EP_CONNECT,
+ DCNT_EP_DISCONNECT,
+ DCNT_EP_POST_SEND,
+ DCNT_EP_POST_SEND_DATA,
+ DCNT_EP_POST_SEND_UD,
+ DCNT_EP_POST_SEND_UD_DATA,
+ DCNT_EP_POST_RECV,
+ DCNT_EP_POST_RECV_DATA,
+ DCNT_EP_POST_WRITE,
+ DCNT_EP_POST_WRITE_DATA,
+ DCNT_EP_POST_WRITE_IMM,
+ DCNT_EP_POST_WRITE_IMM_DATA,
+ DCNT_EP_POST_READ,
+ DCNT_EP_POST_READ_DATA,
+ DCNT_EP_POST_CMP_SWAP,
+ DCNT_EP_POST_FETCH_ADD,
+ DCNT_EP_RECV,
+ DCNT_EP_RECV_DATA,
+ DCNT_EP_RECV_UD,
+ DCNT_EP_RECV_UD_DATA,
+ DCNT_EP_RECV_IMM,
+ DCNT_EP_RECV_IMM_DATA,
+ DCNT_EP_RECV_RDMA_IMM,
+ DCNT_EP_RECV_RDMA_IMM_DATA,
+ DCNT_EP_ALL_COUNTERS, /* MUST be last */
+
+} DAT_EP_COUNTERS;
+
+/*
+ * Definitions for 64-bit EVD Counters
+ */
+typedef enum dat_evd_counters
+{
+ DCNT_EVD_WAIT,
+ DCNT_EVD_WAIT_BLOCKED,
+ DCNT_EVD_WAIT_NOTIFY,
+ DCNT_EVD_DEQUEUE,
+ DCNT_EVD_DEQUEUE_FOUND,
+ DCNT_EVD_DEQUEUE_NOT_FOUND,
+ DCNT_EVD_DEQUEUE_POLL,
+ DCNT_EVD_DEQUEUE_POLL_FOUND,
+ DCNT_EVD_CONN_CALLBACK,
+ DCNT_EVD_DTO_CALLBACK,
+ DCNT_EVD_ALL_COUNTERS, /* MUST be last */
+
+} DAT_EVD_COUNTERS;
+
+/* Extended RETURN and EVENT STATUS string helper functions */
+
+/* DAT_EXT_RETURN error to string */
+static __inline__ DAT_RETURN DAT_API
+dat_strerror_extension (
+ IN DAT_IB_RETURN value,
+ OUT const char **message )
+{
+ switch( DAT_GET_TYPE(value) ) {
+ case DAT_IB_ERR:
+ *message = "DAT_IB_ERR";
+ return DAT_SUCCESS;
+ default:
+ /* standard DAT return type */
+ return(dat_strerror(value, message, NULL));
+ }
+}
+
+/* DAT_EXT_STATUS error to string */
+static __inline__ DAT_RETURN DAT_API
+dat_strerror_ext_status (
+ IN DAT_IB_STATUS value,
+ OUT const char **message )
+{
+ switch(value) {
+ case 0:
+ *message = " ";
+ return DAT_SUCCESS;
+ case DAT_IB_OP_ERR:
+ *message = "DAT_IB_OP_ERR";
+ return DAT_SUCCESS;
+ default:
+ *message = "unknown extension status";
+ return DAT_INVALID_PARAMETER;
+ }
+}
+
+/*
+ * Extended IB transport specific APIs
+ * redirection via DAT extension function
+ */
+
+/*
+ * This asynchronous call is modeled after the InfiniBand atomic
+ * Fetch and Add operation. The add_value is added to the 64 bit
+ * value stored at the remote memory location specified in remote_iov
+ * and the result is stored in the local_iov.
+ */
+#define dat_ib_post_fetch_and_add(ep, add_val, lbuf, cookie, rbuf, flgs) \
+ dat_extension_op( ep, \
+ DAT_IB_FETCH_AND_ADD_OP, \
+ (add_val), \
+ (lbuf), \
+ (cookie), \
+ (rbuf), \
+ (flgs))
+
+/*
+ * This asynchronous call is modeled after the InfiniBand atomic
+ * Compare and Swap operation. The cmp_value is compared to the 64 bit
+ * value stored at the remote memory location specified in remote_iov.
+ * If the two values are equal, the 64 bit swap_value is stored in
+ * the remote memory location. In all cases, the original 64 bit
+ * value stored in the remote memory location is copied to the local_iov.
+ */
+#define dat_ib_post_cmp_and_swap(ep, cmp_val, swap_val, lbuf, cookie, rbuf, flgs) \
+ dat_extension_op( ep, \
+ DAT_IB_CMP_AND_SWAP_OP, \
+ (cmp_val), \
+ (swap_val), \
+ (lbuf), \
+ (cookie), \
+ (rbuf), \
+ (flgs))
+
+/*
+ * RDMA Write with IMMEDIATE:
+ *
+ * This asynchronous call is modeled after the InfiniBand rdma write with
+ * immediate data operation. Event completion for the request completes as an
+ * DAT_EXTENSION with extension type set to DAT_DTO_EXTENSION_IMMED_DATA.
+ * Event completion on the remote endpoint completes as receive DTO operation
+ * type of DAT_EXTENSION with operation set to DAT_DTO_EXTENSION_IMMED_DATA.
+ * The immediate data will be provided in the extented DTO event data structure.
+ *
+ * Note to Consumers: the immediate data will consume a receive
+ * buffer at the Data Sink.
+ *
+ * Other extension flags:
+ * n/a
+ */
+#define dat_ib_post_rdma_write_immed(ep, size, lbuf, cookie, rbuf, idata, flgs) \
+ dat_extension_op( ep, \
+ DAT_IB_RDMA_WRITE_IMMED_OP, \
+ (size), \
+ (lbuf), \
+ (cookie), \
+ (rbuf), \
+ (idata), \
+ (flgs))
+
+/*
+ * Unreliable datagram: msg send
+ *
+ * This asynchronous call is modeled after the InfiniBand UD message send
+ * Event completion for the request completes as an
+ * DAT_EXTENSION with extension type set to DAT_DTO_EXTENSION_UD_SEND.
+ * Event completion on the remote endpoint completes as receive DTO operation
+ * type of DAT_EXTENSION with operation set to DAT_DTO_EXTENSION_UD_RECV.
+ *
+ * Other extension flags:
+ * n/a
+ */
+#define dat_ib_post_send_ud(ep, segments, lbuf, ah_ptr, cookie, flgs) \
+ dat_extension_op( ep, \
+ DAT_IB_UD_SEND_OP, \
+ (segments), \
+ (lbuf), \
+ (ah_ptr), \
+ (cookie), \
+ (flgs))
+
+
+/*
+ * Query counter(s):
+ * Provide IA, EP, or EVD and call will return appropriate counters
+ * DAT_HANDLE dat_handle, enum cntr, *DAT_UINT64 p_cntrs_out, int reset
+ *
+ * use _ALL_COUNTERS to query all
+ */
+#define dat_query_counters(dat_handle, cntr, p_cntrs_out, reset) \
+ dat_extension_op( dat_handle, \
+ DAT_QUERY_COUNTERS_OP, \
+ (cntr), \
+ (p_cntrs_out), \
+ (reset))
+/*
+ * Print counter(s):
+ * Provide IA, EP, or EVD and call will print appropriate counters
+ * DAT_HANDLE dat_handle, int cntr, int reset
+ *
+ * use _ALL_COUNTERS to print all
+ */
+#define dat_print_counters(dat_handle, cntr, reset) \
+ dat_extension_op( dat_handle, \
+ DAT_PRINT_COUNTERS_OP, \
+ (cntr), \
+ (reset))
+
+#endif /* _DAT_IB_EXTENSIONS_H_ */
+
-/*\r
- * Copyright (c) 2002-2005, Network Appliance, Inc. All rights reserved.\r
- *\r
- * This Software is licensed under one of the following licenses:\r
- *\r
- * 1) under the terms of the "Common Public License 1.0" a copy of which is\r
- * in the file LICENSE.txt in the root directory. The license is also\r
- * available from the Open Source Initiative, see\r
- * http://www.opensource.org/licenses/cpl.php.\r
- *\r
- * 2) under the terms of the "The BSD License" a copy of which is in the file\r
- * LICENSE2.txt in the root directory. The license is also available from\r
- * the Open Source Initiative, see\r
- * http://www.opensource.org/licenses/bsd-license.php.\r
- *\r
- * 3) under the terms of the "GNU General Public License (GPL) Version 2" a \r
- * copy of which is in the file LICENSE3.txt in the root directory. The \r
- * license is also available from the Open Source Initiative, see\r
- * http://www.opensource.org/licenses/gpl-license.php.\r
- *\r
- * Licensee has the right to choose one of the above licenses.\r
- *\r
- * Redistributions of source code must retain the above copyright\r
- * notice and one of the license notices.\r
- *\r
- * Redistributions in binary form must reproduce both the above copyright\r
- * notice, one of the license notices in the documentation\r
- * and/or other materials provided with the distribution.\r
- */\r
-\r
-/**********************************************************************\r
- *\r
- * MODULE: dat_sr_parser.c\r
- *\r
- * PURPOSE: static registry parser\r
- *\r
- * $Id: udat_sr_parser.c,v 1.6 2005/03/24 05:58:36 jlentini Exp $\r
- **********************************************************************/\r
-\r
-#include <dat2/udat.h>\r
-#include "udat_sr_parser.h"\r
-#include "dat_sr.h"\r
-\r
-/*********************************************************************\r
- * *\r
- * Constants *\r
- * *\r
- *********************************************************************/\r
-\r
-#define DAT_SR_CONF_ENV "DAT_OVERRIDE"\r
-#if defined(_WIN32) || defined(_WIN64)\r
-#define DAT_SR_CONF_DEFAULT "C:\\DAT\\dat.conf"\r
-#else\r
-#define DAT_SR_CONF_DEFAULT "/etc/dat.conf"\r
-#endif\r
-\r
-#define DAT_SR_TOKEN_THREADSAFE "threadsafe"\r
-#define DAT_SR_TOKEN_NONTHREADSAFE "nonthreadsafe"\r
-#define DAT_SR_TOKEN_DEFAULT "default"\r
-#define DAT_SR_TOKEN_NONDEFAULT "nondefault"\r
-\r
-#define DAT_SR_CHAR_NEWLINE '\n'\r
-#define DAT_SR_CHAR_COMMENT '#'\r
-#define DAT_SR_CHAR_QUOTE '"'\r
-#define DAT_SR_CHAR_BACKSLASH '\\'\r
-\r
-/*********************************************************************\r
- * *\r
- * Enumerations *\r
- * *\r
- *********************************************************************/\r
-\r
-typedef enum {\r
- DAT_SR_TOKEN_STRING, /* text field (both quoted or unquoted) */\r
- DAT_SR_TOKEN_EOR, /* end of record (newline) */\r
- DAT_SR_TOKEN_EOF /* end of file */\r
-} DAT_SR_TOKEN_TYPE;\r
-\r
-typedef enum {\r
- DAT_SR_API_UDAT,\r
- DAT_SR_API_KDAT\r
-} DAT_SR_API_TYPE;\r
-\r
-/*********************************************************************\r
- * *\r
- * Structures *\r
- * *\r
- *********************************************************************/\r
-\r
-typedef struct {\r
- DAT_SR_TOKEN_TYPE type;\r
- char *value; /* valid if type is DAT_SR_TOKEN_STRING */\r
- DAT_OS_SIZE value_len;\r
-} DAT_SR_TOKEN;\r
-\r
-typedef struct DAT_SR_STACK_NODE {\r
- DAT_SR_TOKEN token;\r
- struct DAT_SR_STACK_NODE *next;\r
-} DAT_SR_STACK_NODE;\r
-\r
-typedef struct {\r
- DAT_UINT32 major;\r
- DAT_UINT32 minor;\r
-} DAT_SR_VERSION;\r
-\r
-typedef struct {\r
- char *id;\r
- DAT_SR_VERSION version;\r
-} DAT_SR_PROVIDER_VERSION;\r
-\r
-typedef struct {\r
- DAT_SR_API_TYPE type;\r
- DAT_SR_VERSION version;\r
-} DAT_SR_API_VERSION;\r
-\r
-typedef struct {\r
- char *ia_name;\r
- DAT_SR_API_VERSION api_version;\r
- DAT_BOOLEAN is_thread_safe;\r
- DAT_BOOLEAN is_default;\r
- char *lib_path;\r
- DAT_SR_PROVIDER_VERSION provider_version;\r
- char *ia_params;\r
- char *platform_params;\r
-} DAT_SR_CONF_ENTRY;\r
-\r
-/*********************************************************************\r
- * *\r
- * Internal Function Declarations *\r
- * *\r
- *********************************************************************/\r
-\r
-static DAT_RETURN dat_sr_load_entry(DAT_SR_CONF_ENTRY * entry);\r
-\r
-static DAT_BOOLEAN dat_sr_is_valid_entry(DAT_SR_CONF_ENTRY * entry);\r
-\r
-static char *dat_sr_type_to_str(DAT_SR_TOKEN_TYPE type);\r
-\r
-static DAT_RETURN dat_sr_parse_eof(DAT_OS_FILE * file);\r
-\r
-static DAT_RETURN dat_sr_parse_entry(DAT_OS_FILE * file);\r
-\r
-static DAT_RETURN\r
-dat_sr_parse_ia_name(DAT_OS_FILE * file, DAT_SR_CONF_ENTRY * entry);\r
-\r
-static DAT_RETURN\r
-dat_sr_parse_api(DAT_OS_FILE * file, DAT_SR_CONF_ENTRY * entry);\r
-\r
-static DAT_RETURN\r
-dat_sr_parse_thread_safety(DAT_OS_FILE * file, DAT_SR_CONF_ENTRY * entry);\r
-\r
-static DAT_RETURN\r
-dat_sr_parse_default(DAT_OS_FILE * file, DAT_SR_CONF_ENTRY * entry);\r
-\r
-static DAT_RETURN\r
-dat_sr_parse_lib_path(DAT_OS_FILE * file, DAT_SR_CONF_ENTRY * entry);\r
-\r
-static DAT_RETURN\r
-dat_sr_parse_provider_version(DAT_OS_FILE * file, DAT_SR_CONF_ENTRY * entry);\r
-\r
-static DAT_RETURN\r
-dat_sr_parse_ia_params(DAT_OS_FILE * file, DAT_SR_CONF_ENTRY * entry);\r
-\r
-static DAT_RETURN\r
-dat_sr_parse_platform_params(DAT_OS_FILE * file, DAT_SR_CONF_ENTRY * entry);\r
-\r
-static DAT_RETURN\r
-dat_sr_parse_eoe(DAT_OS_FILE * file, DAT_SR_CONF_ENTRY * entry);\r
-\r
-static DAT_RETURN\r
-dat_sr_convert_api(char *str, DAT_SR_API_VERSION * api_version);\r
-\r
-static DAT_RETURN\r
-dat_sr_convert_thread_safety(char *str, DAT_BOOLEAN * is_thread_safe);\r
-\r
-static DAT_RETURN dat_sr_convert_default(char *str, DAT_BOOLEAN * is_default);\r
-\r
-static DAT_RETURN\r
-dat_sr_convert_provider_version(char *str,\r
- DAT_SR_PROVIDER_VERSION * provider_version);\r
-\r
-static DAT_RETURN dat_sr_get_token(DAT_OS_FILE * file, DAT_SR_TOKEN * token);\r
-\r
-static DAT_RETURN dat_sr_put_token(DAT_OS_FILE * file, DAT_SR_TOKEN * token);\r
-\r
-static DAT_RETURN dat_sr_read_token(DAT_OS_FILE * file, DAT_SR_TOKEN * token);\r
-\r
-static DAT_RETURN\r
-dat_sr_read_str(DAT_OS_FILE * file,\r
- DAT_SR_TOKEN * token, DAT_OS_SIZE token_len);\r
-\r
-static DAT_RETURN\r
-dat_sr_read_quoted_str(DAT_OS_FILE * file,\r
- DAT_SR_TOKEN * token,\r
- DAT_OS_SIZE token_len, DAT_COUNT num_escape_seq);\r
-\r
-static void dat_sr_read_comment(DAT_OS_FILE * file);\r
-\r
-/*********************************************************************\r
- * *\r
- * Global Variables *\r
- * *\r
- *********************************************************************/\r
-\r
-static DAT_SR_STACK_NODE *g_token_stack = NULL;\r
-\r
-/*********************************************************************\r
- * *\r
- * External Function Definitions *\r
- * *\r
- *********************************************************************/\r
-\r
-/***********************************************************************\r
- * Function: dat_sr_load\r
- ***********************************************************************/\r
-\r
-DAT_RETURN dat_sr_load(void)\r
-{\r
- char *sr_path;\r
- DAT_OS_FILE *sr_file;\r
-\r
- sr_path = dat_os_getenv(DAT_SR_CONF_ENV);\r
-\r
- /* environment override */\r
- if ((sr_path != NULL) && ((sr_file = dat_os_fopen(sr_path)) == NULL)) {\r
- dat_os_dbg_print(DAT_OS_DBG_TYPE_ERROR,\r
- "DAT Registry: DAT_OVERRIDE, "\r
- "bad filename - %s, aborting\n", sr_path);\r
- goto bail;\r
- }\r
-\r
- if (sr_path == NULL) {\r
-\r
-#ifdef DAT_CONF\r
- sr_path = DAT_CONF;\r
-#else\r
- sr_path = DAT_SR_CONF_DEFAULT;\r
-#endif\r
- sr_file = dat_os_fopen(sr_path);\r
- if (sr_file == NULL) {\r
-#ifdef DAT_CONF\r
- dat_os_dbg_print(DAT_OS_DBG_TYPE_SR,\r
- "DAT Registry: sysconfdir, "\r
- "bad filename - %s, retry default at %s\n",\r
- sr_path, DAT_SR_CONF_DEFAULT);\r
- /* try default after sysconfdir fails */\r
- sr_path = DAT_SR_CONF_DEFAULT;\r
- sr_file = dat_os_fopen(sr_path);\r
- if (sr_file == NULL) {\r
-#endif\r
- dat_os_dbg_print(DAT_OS_DBG_TYPE_ERROR,\r
- "DAT Registry: default, "\r
- "bad filename - %s, aborting\n",\r
- sr_path);\r
- goto bail;\r
-#ifdef DAT_CONF\r
- }\r
-#endif\r
- }\r
- }\r
-\r
- dat_os_dbg_print(DAT_OS_DBG_TYPE_GENERIC,\r
- "DAT Registry: using config file %s\n", sr_path);\r
-\r
- for (;;) {\r
- if (DAT_SUCCESS == dat_sr_parse_eof(sr_file)) {\r
- break;\r
- } else if (DAT_SUCCESS == dat_sr_parse_entry(sr_file)) {\r
- continue;\r
- } else {\r
- dat_os_dbg_print(DAT_OS_DBG_TYPE_ERROR,\r
- "DAT Registry: ERROR parsing - %s\n",\r
- sr_path);\r
- goto cleanup;\r
- }\r
- }\r
-\r
- if (0 != dat_os_fclose(sr_file)) {\r
- dat_os_dbg_print(DAT_OS_DBG_TYPE_ERROR,\r
- "DAT Registry: ERROR closing - %s\n", sr_path);\r
- goto bail;\r
- }\r
-\r
- return DAT_SUCCESS;\r
-\r
- cleanup:\r
- dat_os_fclose(sr_file);\r
- bail:\r
- return DAT_INTERNAL_ERROR;\r
-\r
-}\r
-\r
-/*********************************************************************\r
- * *\r
- * Internal Function Definitions *\r
- * *\r
- *********************************************************************/\r
-\r
-/***********************************************************************\r
- * Function: dat_sr_is_valid_entry\r
- ***********************************************************************/\r
-\r
-DAT_BOOLEAN dat_sr_is_valid_entry(DAT_SR_CONF_ENTRY * entry)\r
-{\r
- if ((DAT_SR_API_UDAT == entry->api_version.type) && (entry->is_default)) {\r
- return DAT_TRUE;\r
- } else {\r
- return DAT_FALSE;\r
- }\r
-}\r
-\r
-/***********************************************************************\r
- * Function: dat_sr_load_entry\r
- ***********************************************************************/\r
-\r
-DAT_RETURN dat_sr_load_entry(DAT_SR_CONF_ENTRY * conf_entry)\r
-{\r
- DAT_SR_ENTRY entry;\r
-\r
- if (DAT_NAME_MAX_LENGTH < (strlen(conf_entry->ia_name) + 1)) {\r
- dat_os_dbg_print(DAT_OS_DBG_TYPE_SR,\r
- "DAT Registry: ia name %s is longer than "\r
- "DAT_NAME_MAX_LENGTH (%i)\n",\r
- conf_entry->ia_name, DAT_NAME_MAX_LENGTH);\r
-\r
- return DAT_INSUFFICIENT_RESOURCES;\r
- }\r
-\r
- dat_os_strncpy(entry.info.ia_name, conf_entry->ia_name,\r
- DAT_NAME_MAX_LENGTH);\r
- entry.info.dapl_version_major = conf_entry->api_version.version.major;\r
- entry.info.dapl_version_minor = conf_entry->api_version.version.minor;\r
- entry.info.is_thread_safe = conf_entry->is_thread_safe;\r
- entry.lib_path = conf_entry->lib_path;\r
- entry.ia_params = conf_entry->ia_params;\r
- entry.lib_handle = NULL;\r
- entry.ref_count = 0;\r
-\r
- dat_os_dbg_print(DAT_OS_DBG_TYPE_SR,\r
- "DAT Registry: loading provider for %s\n",\r
- conf_entry->ia_name);\r
-\r
- return dat_sr_insert(&entry.info, &entry);\r
-}\r
-\r
-/***********************************************************************\r
- * Function: dat_sr_type_to_str\r
- ***********************************************************************/\r
-\r
-char *dat_sr_type_to_str(DAT_SR_TOKEN_TYPE type)\r
-{\r
- static char *str_array[] = { "string", "eor", "eof" };\r
-\r
- if ((type < 0) || (2 < type)) {\r
- return "error: invalid token type";\r
- }\r
-\r
- return str_array[type];\r
-}\r
-\r
-/***********************************************************************\r
- * Function: dat_sr_parse_eof\r
- ***********************************************************************/\r
-\r
-DAT_RETURN dat_sr_parse_eof(DAT_OS_FILE * file)\r
-{\r
- DAT_SR_TOKEN token;\r
-\r
- if (DAT_SUCCESS != dat_sr_get_token(file, &token)) {\r
- return DAT_INTERNAL_ERROR;\r
- }\r
-\r
- if (DAT_SR_TOKEN_EOF == token.type) {\r
- return DAT_SUCCESS;\r
- } else {\r
- dat_sr_put_token(file, &token);\r
- return DAT_INTERNAL_ERROR;\r
- }\r
-}\r
-\r
-/***********************************************************************\r
- * Function: dat_sr_parse_ia_name\r
- ***********************************************************************/\r
-\r
-DAT_RETURN dat_sr_parse_entry(DAT_OS_FILE * file)\r
-{\r
- DAT_SR_CONF_ENTRY entry;\r
- DAT_RETURN status;\r
-\r
- dat_os_memset(&entry, 0, sizeof(DAT_SR_CONF_ENTRY));\r
-\r
- if ((DAT_SUCCESS == dat_sr_parse_ia_name(file, &entry)) &&\r
- (DAT_SUCCESS == dat_sr_parse_api(file, &entry)) &&\r
- (DAT_SUCCESS == dat_sr_parse_thread_safety(file, &entry)) &&\r
- (DAT_SUCCESS == dat_sr_parse_default(file, &entry)) &&\r
- (DAT_SUCCESS == dat_sr_parse_lib_path(file, &entry)) &&\r
- (DAT_SUCCESS == dat_sr_parse_provider_version(file, &entry)) &&\r
- (DAT_SUCCESS == dat_sr_parse_ia_params(file, &entry)) &&\r
- (DAT_SUCCESS == dat_sr_parse_platform_params(file, &entry)) &&\r
- (DAT_SUCCESS == dat_sr_parse_eoe(file, &entry))) {\r
- dat_os_dbg_print(DAT_OS_DBG_TYPE_SR,\r
- "\n"\r
- "DAT Registry: entry \n"\r
- " ia_name %s\n"\r
- " api_version\n"\r
- " type 0x%X\n"\r
- " major.minor %d.%d\n"\r
- " is_thread_safe %d\n"\r
- " is_default %d\n"\r
- " lib_path %s\n"\r
- " provider_version\n"\r
- " id %s\n"\r
- " major.minor %d.%d\n"\r
- " ia_params %s\n"\r
- "\n",\r
- entry.ia_name,\r
- entry.api_version.type,\r
- entry.api_version.version.major,\r
- entry.api_version.version.minor,\r
- entry.is_thread_safe,\r
- entry.is_default,\r
- entry.lib_path,\r
- entry.provider_version.id,\r
- entry.provider_version.version.major,\r
- entry.provider_version.version.minor,\r
- entry.ia_params);\r
-\r
- if (DAT_TRUE == dat_sr_is_valid_entry(&entry)) {\r
- /*\r
- * The static registry configuration file may have multiple\r
- * entries with the same IA name. The first entry will be\r
- * installed in the static registry causing subsequent attempts\r
- * to register the same IA name to fail. Therefore the return code\r
- * from dat_sr_load_entry() is ignored.\r
- */\r
- (void)dat_sr_load_entry(&entry);\r
- }\r
-\r
- status = DAT_SUCCESS;\r
- } else { /* resync */\r
-\r
- DAT_SR_TOKEN token;\r
-\r
- /*\r
- * The static registry format is specified in the DAT specification.\r
- * While the registry file's contents may change between revisions of\r
- * the specification, there is no way to determine the specification\r
- * version to which the configuration file conforms. If an entry is\r
- * found that does not match the expected format, the entry is discarded\r
- * and the parsing of the file continues. There is no way to determine if\r
- * the entry was an error or an entry confirming to an alternate version\r
- * of specification.\r
- */\r
-\r
- for (;;) {\r
- if (DAT_SUCCESS != dat_sr_get_token(file, &token)) {\r
- status = DAT_INTERNAL_ERROR;\r
- break;\r
- }\r
-\r
- if (DAT_SR_TOKEN_STRING != token.type) {\r
- status = DAT_SUCCESS;\r
- break;\r
- } else {\r
- dat_os_free(token.value,\r
- (sizeof(char) *\r
- dat_os_strlen(token.value)) + 1);\r
- continue;\r
- }\r
- }\r
- }\r
-\r
- /* free resources */\r
- if (NULL != entry.ia_name) {\r
- dat_os_free(entry.ia_name,\r
- sizeof(char) * (dat_os_strlen(entry.ia_name) + 1));\r
- }\r
- if (NULL != entry.lib_path) {\r
- dat_os_free(entry.lib_path,\r
- sizeof(char) * (dat_os_strlen(entry.lib_path) + 1));\r
- }\r
-\r
- if (NULL != entry.provider_version.id) {\r
- dat_os_free(entry.provider_version.id,\r
- sizeof(char) *\r
- (dat_os_strlen(entry.provider_version.id) + 1));\r
- }\r
-\r
- if (NULL != entry.ia_params) {\r
- dat_os_free(entry.ia_params,\r
- sizeof(char) * (dat_os_strlen(entry.ia_params) +\r
- 1));\r
- }\r
-\r
- return status;\r
-}\r
-\r
-/***********************************************************************\r
- * Function: dat_sr_parse_ia_name\r
- ***********************************************************************/\r
-\r
-DAT_RETURN dat_sr_parse_ia_name(DAT_OS_FILE * file, DAT_SR_CONF_ENTRY * entry)\r
-{\r
- DAT_SR_TOKEN token;\r
-\r
- if (DAT_SUCCESS != dat_sr_get_token(file, &token))\r
- goto bail;\r
-\r
- if (DAT_SR_TOKEN_STRING != token.type) {\r
- dat_sr_put_token(file, &token);\r
- goto bail;\r
- }\r
- entry->ia_name = token.value;\r
- return DAT_SUCCESS;\r
-\r
- bail:\r
- return DAT_INTERNAL_ERROR;\r
-}\r
-\r
-/***********************************************************************\r
- * Function: dat_sr_parse_ia_name\r
- ***********************************************************************/\r
-\r
-DAT_RETURN dat_sr_parse_api(DAT_OS_FILE * file, DAT_SR_CONF_ENTRY * entry)\r
-{\r
- DAT_SR_TOKEN token;\r
-\r
- if (DAT_SUCCESS != dat_sr_get_token(file, &token))\r
- goto bail;\r
-\r
- if (DAT_SR_TOKEN_STRING != token.type)\r
- goto cleanup;\r
-\r
- if (DAT_SUCCESS != dat_sr_convert_api(token.value, &entry->api_version))\r
- goto cleanup;\r
-\r
- dat_os_free(token.value,\r
- (sizeof(char) * dat_os_strlen(token.value)) + 1);\r
- return DAT_SUCCESS;\r
-\r
- cleanup:\r
- dat_sr_put_token(file, &token);\r
- bail:\r
- dat_os_dbg_print(DAT_OS_DBG_TYPE_ERROR,\r
- " ERR: corrupt dat.conf entry field:"\r
- " api_ver, file offset=%ld\n", ftell(file));\r
- return DAT_INTERNAL_ERROR;\r
-}\r
-\r
-/***********************************************************************\r
- * Function: dat_sr_parse_thread_safety\r
- ***********************************************************************/\r
-\r
-static DAT_RETURN\r
-dat_sr_parse_thread_safety(DAT_OS_FILE * file, DAT_SR_CONF_ENTRY * entry)\r
-{\r
- DAT_SR_TOKEN token;\r
-\r
- if (DAT_SUCCESS != dat_sr_get_token(file, &token))\r
- goto bail;\r
-\r
- if (DAT_SR_TOKEN_STRING != token.type)\r
- goto cleanup;\r
-\r
- if (DAT_SUCCESS !=\r
- dat_sr_convert_thread_safety(token.value, &entry->is_thread_safe))\r
- goto cleanup;\r
-\r
- dat_os_free(token.value,\r
- (sizeof(char) * dat_os_strlen(token.value)) + 1);\r
- return DAT_SUCCESS;\r
-\r
- cleanup:\r
- dat_sr_put_token(file, &token);\r
- bail:\r
- dat_os_dbg_print(DAT_OS_DBG_TYPE_ERROR,\r
- " ERR: corrupt dat.conf entry field:"\r
- " thread_safety, file offset=%ld\n", ftell(file));\r
- return DAT_INTERNAL_ERROR;\r
-}\r
-\r
-/***********************************************************************\r
- * Function: dat_sr_parse_default\r
- ***********************************************************************/\r
-\r
-DAT_RETURN dat_sr_parse_default(DAT_OS_FILE * file, DAT_SR_CONF_ENTRY * entry)\r
-{\r
- DAT_SR_TOKEN token;\r
-\r
- if (DAT_SUCCESS != dat_sr_get_token(file, &token))\r
- goto bail;\r
-\r
- if (DAT_SR_TOKEN_STRING != token.type)\r
- goto cleanup;\r
-\r
- if (DAT_SUCCESS !=\r
- dat_sr_convert_default(token.value, &entry->is_default))\r
- goto cleanup;\r
-\r
- dat_os_free(token.value,\r
- (sizeof(char) * dat_os_strlen(token.value)) + 1);\r
- return DAT_SUCCESS;\r
-\r
- cleanup:\r
- dat_sr_put_token(file, &token);\r
- bail:\r
- dat_os_dbg_print(DAT_OS_DBG_TYPE_ERROR,\r
- " ERR: corrupt dat.conf entry field:"\r
- " default section, file offset=%ld\n", ftell(file));\r
- return DAT_INTERNAL_ERROR;\r
-}\r
-\r
-/***********************************************************************\r
- * Function: dat_sr_parse_lib_path\r
- ***********************************************************************/\r
-\r
-DAT_RETURN dat_sr_parse_lib_path(DAT_OS_FILE * file, DAT_SR_CONF_ENTRY * entry)\r
-{\r
- DAT_SR_TOKEN token;\r
-\r
- if (DAT_SUCCESS != dat_sr_get_token(file, &token))\r
- goto bail;\r
-\r
- if (DAT_SR_TOKEN_STRING != token.type) {\r
- dat_sr_put_token(file, &token);\r
- goto bail;\r
- }\r
- entry->lib_path = token.value;\r
- return DAT_SUCCESS;\r
-\r
- bail:\r
- dat_os_dbg_print(DAT_OS_DBG_TYPE_ERROR,\r
- " ERR: corrupt dat.conf entry field:"\r
- " lib_path, file offset=%ld\n", ftell(file));\r
- return DAT_INTERNAL_ERROR;\r
-}\r
-\r
-/***********************************************************************\r
- * Function: dat_sr_parse_provider_version\r
- ***********************************************************************/\r
-\r
-DAT_RETURN\r
-dat_sr_parse_provider_version(DAT_OS_FILE * file, DAT_SR_CONF_ENTRY * entry)\r
-{\r
- DAT_SR_TOKEN token;\r
-\r
- if (DAT_SUCCESS != dat_sr_get_token(file, &token))\r
- goto bail;\r
-\r
- if (DAT_SR_TOKEN_STRING != token.type)\r
- goto cleanup;\r
-\r
- if (DAT_SUCCESS !=\r
- dat_sr_convert_provider_version(token.value,\r
- &entry->provider_version))\r
- goto cleanup;\r
-\r
- dat_os_free(token.value,\r
- (sizeof(char) * dat_os_strlen(token.value)) + 1);\r
- return DAT_SUCCESS;\r
-\r
- cleanup:\r
- dat_sr_put_token(file, &token);\r
- bail:\r
- dat_os_dbg_print(DAT_OS_DBG_TYPE_ERROR,\r
- " ERR: corrupt dat.conf entry field:"\r
- " provider_ver, file offset=%ld\n", ftell(file));\r
- return DAT_INTERNAL_ERROR;\r
-}\r
-\r
-/***********************************************************************\r
- * Function: dat_sr_parse_ia_params\r
- ***********************************************************************/\r
-\r
-DAT_RETURN dat_sr_parse_ia_params(DAT_OS_FILE * file, DAT_SR_CONF_ENTRY * entry)\r
-{\r
- DAT_SR_TOKEN token;\r
-\r
- if (DAT_SUCCESS != dat_sr_get_token(file, &token))\r
- goto bail;\r
-\r
- if (DAT_SR_TOKEN_STRING != token.type) {\r
- dat_sr_put_token(file, &token);\r
- goto bail;\r
- }\r
-\r
- entry->ia_params = token.value;\r
- return DAT_SUCCESS;\r
-\r
- bail:\r
- dat_os_dbg_print(DAT_OS_DBG_TYPE_ERROR,\r
- " ERR: corrupt dat.conf entry field:"\r
- " ia_params, file offset=%ld\n", ftell(file));\r
- return DAT_INTERNAL_ERROR;\r
-}\r
-\r
-/***********************************************************************\r
- * Function: dat_sr_parse_platform_params\r
- ***********************************************************************/\r
-\r
-DAT_RETURN\r
-dat_sr_parse_platform_params(DAT_OS_FILE * file, DAT_SR_CONF_ENTRY * entry)\r
-{\r
- DAT_SR_TOKEN token;\r
-\r
- if (DAT_SUCCESS != dat_sr_get_token(file, &token))\r
- goto bail;\r
-\r
- if (DAT_SR_TOKEN_STRING != token.type) {\r
- dat_sr_put_token(file, &token);\r
- goto bail;\r
- }\r
-\r
- entry->platform_params = token.value;\r
- return DAT_SUCCESS;\r
-\r
- bail:\r
- dat_os_dbg_print(DAT_OS_DBG_TYPE_ERROR,\r
- " ERR: corrupt dat.conf entry field:"\r
- " platform_params, file offset=%ld\n", ftell(file));\r
- return DAT_INTERNAL_ERROR;\r
-}\r
-\r
-/***********************************************************************\r
- * Function: dat_sr_parse_eoe\r
- ***********************************************************************/\r
-\r
-DAT_RETURN dat_sr_parse_eoe(DAT_OS_FILE * file, DAT_SR_CONF_ENTRY * entry)\r
-{\r
- DAT_SR_TOKEN token;\r
-\r
- if (DAT_SUCCESS != dat_sr_get_token(file, &token))\r
- goto bail;\r
-\r
- if ((DAT_SR_TOKEN_EOF != token.type) &&\r
- (DAT_SR_TOKEN_EOR != token.type)) {\r
- dat_sr_put_token(file, &token);\r
- goto bail;\r
- }\r
-\r
- return DAT_SUCCESS;\r
-\r
- bail:\r
- dat_os_dbg_print(DAT_OS_DBG_TYPE_ERROR,\r
- " ERR: corrupt dat.conf entry field:"\r
- " EOR, EOF, file offset=%ld\n", ftell(file));\r
- return DAT_INTERNAL_ERROR;\r
-}\r
-\r
-/***********************************************************************\r
- * Function: dat_sr_convert_api\r
- ***********************************************************************/\r
-\r
-DAT_RETURN dat_sr_convert_api(char *str, DAT_SR_API_VERSION * api_version)\r
-{\r
- int i;\r
- int minor_i;\r
-\r
- if (dat_os_strlen(str) <= 0)\r
- return DAT_INTERNAL_ERROR;\r
-\r
- if ('u' == str[0]) {\r
- api_version->type = DAT_SR_API_UDAT;\r
- } else if ('k' == str[0]) {\r
- api_version->type = DAT_SR_API_KDAT;\r
- } else {\r
- return DAT_INTERNAL_ERROR;\r
- }\r
-\r
- for (i = 1 /* move past initial [u|k] */ ; '\0' != str[i]; i++) {\r
- if ('.' == str[i]) {\r
- break;\r
- } else if (DAT_TRUE != dat_os_isdigit(str[i])) {\r
- return DAT_INTERNAL_ERROR;\r
- }\r
- }\r
-\r
- api_version->version.major =\r
- (DAT_UINT32) dat_os_strtol(str + 1, NULL, 10);\r
-\r
- /* move past '.' */\r
- minor_i = ++i;\r
-\r
- for (; '\0' != str[i]; i++) {\r
- if (DAT_TRUE != dat_os_isdigit(str[i])) {\r
- return DAT_INTERNAL_ERROR;\r
- }\r
- }\r
-\r
- api_version->version.minor =\r
- (DAT_UINT32) dat_os_strtol(str + minor_i, NULL, 10);\r
-\r
- if ('\0' != str[i]) {\r
- return DAT_INTERNAL_ERROR;\r
- }\r
-\r
- return DAT_SUCCESS;\r
-}\r
-\r
-/***********************************************************************\r
- * Function: dat_sr_convert_thread_safety\r
- ***********************************************************************/\r
-\r
-static DAT_RETURN\r
-dat_sr_convert_thread_safety(char *str, DAT_BOOLEAN * is_thread_safe)\r
-{\r
- if (!dat_os_strncmp(str,\r
- DAT_SR_TOKEN_THREADSAFE,\r
- dat_os_strlen(DAT_SR_TOKEN_THREADSAFE))) {\r
- *is_thread_safe = DAT_TRUE;\r
- return DAT_SUCCESS;\r
- } else if (!dat_os_strncmp(str,\r
- DAT_SR_TOKEN_NONTHREADSAFE,\r
- dat_os_strlen(DAT_SR_TOKEN_NONTHREADSAFE))) {\r
- *is_thread_safe = DAT_FALSE;\r
- return DAT_SUCCESS;\r
- } else {\r
- return DAT_INTERNAL_ERROR;\r
- }\r
-}\r
-\r
-/***********************************************************************\r
- * Function: dat_sr_convert_default\r
- ***********************************************************************/\r
-\r
-static DAT_RETURN dat_sr_convert_default(char *str, DAT_BOOLEAN * is_default)\r
-{\r
- if (!dat_os_strncmp(str,\r
- DAT_SR_TOKEN_DEFAULT,\r
- dat_os_strlen(DAT_SR_TOKEN_DEFAULT))) {\r
- *is_default = DAT_TRUE;\r
- return DAT_SUCCESS;\r
- } else if (!dat_os_strncmp(str,\r
- DAT_SR_TOKEN_NONDEFAULT,\r
- dat_os_strlen(DAT_SR_TOKEN_NONDEFAULT))) {\r
- *is_default = DAT_FALSE;\r
- return DAT_SUCCESS;\r
- } else {\r
- return DAT_INTERNAL_ERROR;\r
- }\r
-}\r
-\r
-/***********************************************************************\r
- * Function: dat_sr_convert_provider_version\r
- ***********************************************************************/\r
-\r
-DAT_RETURN\r
-dat_sr_convert_provider_version(char *str,\r
- DAT_SR_PROVIDER_VERSION * provider_version)\r
-{\r
- DAT_RETURN status;\r
- int i;\r
- int decimal_i;\r
-\r
- if ((dat_os_strlen(str) <= 0) || (NULL != provider_version->id))\r
- return DAT_INTERNAL_ERROR;\r
-\r
- status = DAT_SUCCESS;\r
-\r
- for (i = 0; '\0' != str[i]; i++) {\r
- if ('.' == str[i]) {\r
- break;\r
- }\r
- }\r
-\r
- /* if no id value was found */\r
- if (0 == i) {\r
- status = DAT_INTERNAL_ERROR;\r
- goto exit;\r
- }\r
-\r
- if (NULL ==\r
- (provider_version->id = dat_os_alloc(sizeof(char) * (i + 1)))) {\r
- status =\r
- DAT_ERROR(DAT_INSUFFICIENT_RESOURCES, DAT_RESOURCE_MEMORY);\r
- goto exit;\r
- }\r
-\r
- dat_os_strncpy(provider_version->id, str, i);\r
- provider_version->id[i] = '\0';\r
-\r
- /* move past '.' */\r
- decimal_i = ++i;\r
-\r
- for (; '\0' != str[i]; i++) {\r
- if ('.' == str[i]) {\r
- break;\r
- } else if (DAT_TRUE != dat_os_isdigit(str[i])) {\r
- status = DAT_INTERNAL_ERROR;\r
- goto exit;\r
- }\r
- }\r
-\r
- /* if no version value was found */\r
- if (decimal_i == i) {\r
- status = DAT_INTERNAL_ERROR;\r
- goto exit;\r
- }\r
-\r
- provider_version->version.major = (DAT_UINT32)\r
- dat_os_strtol(str + decimal_i, NULL, 10);\r
-\r
- /* move past '.' */\r
- decimal_i = ++i;\r
-\r
- for (; '\0' != str[i]; i++) {\r
- if (DAT_TRUE != dat_os_isdigit(str[i])) {\r
- status = DAT_INTERNAL_ERROR;\r
- goto exit;\r
- }\r
- }\r
-\r
- /* if no version value was found */\r
- if (decimal_i == i) {\r
- status = DAT_INTERNAL_ERROR;\r
- goto exit;\r
- }\r
-\r
- provider_version->version.minor = (DAT_UINT32)\r
- dat_os_strtol(str + decimal_i, NULL, 10);\r
-\r
- if ('\0' != str[i]) {\r
- status = DAT_INTERNAL_ERROR;\r
- goto exit;\r
- }\r
-\r
- exit:\r
- if (DAT_SUCCESS != status) {\r
- if (NULL != provider_version->id) {\r
- dat_os_free(provider_version->id,\r
- sizeof(char) *\r
- (dat_os_strlen(provider_version->id) + 1));\r
- provider_version->id = NULL;\r
- }\r
- }\r
-\r
- return status;\r
-}\r
-\r
-/***********************************************************************\r
- * Function: dat_sr_get_token\r
- ***********************************************************************/\r
-\r
-DAT_RETURN dat_sr_get_token(DAT_OS_FILE * file, DAT_SR_TOKEN * token)\r
-{\r
- if (NULL == g_token_stack) {\r
- return dat_sr_read_token(file, token);\r
- } else {\r
- DAT_SR_STACK_NODE *top;\r
-\r
- top = g_token_stack;\r
-\r
- *token = top->token;\r
- g_token_stack = top->next;\r
-\r
- dat_os_free(top, sizeof(DAT_SR_STACK_NODE));\r
-\r
- return DAT_SUCCESS;\r
- }\r
-}\r
-\r
-/***********************************************************************\r
- * Function: dat_sr_put_token\r
- ***********************************************************************/\r
-\r
-DAT_RETURN dat_sr_put_token(DAT_OS_FILE * file, DAT_SR_TOKEN * token)\r
-{\r
- DAT_SR_STACK_NODE *top;\r
-\r
- if (NULL == (top = dat_os_alloc(sizeof(DAT_SR_STACK_NODE)))) {\r
- return DAT_INSUFFICIENT_RESOURCES | DAT_RESOURCE_MEMORY;\r
- }\r
-\r
- top->token = *token;\r
- top->next = g_token_stack;\r
- g_token_stack = top;\r
-\r
- return DAT_SUCCESS;\r
-}\r
-\r
-/***********************************************************************\r
- * Function: dat_sr_read_token\r
- ***********************************************************************/\r
-\r
-DAT_RETURN dat_sr_read_token(DAT_OS_FILE * file, DAT_SR_TOKEN * token)\r
-{\r
- DAT_OS_FILE_POS pos;\r
- DAT_OS_SIZE token_len;\r
- DAT_COUNT num_escape_seq;\r
- DAT_BOOLEAN is_quoted_str;\r
- DAT_BOOLEAN is_prev_char_backslash;\r
-\r
- /*\r
- * The DAT standard does not specify a maximum size for quoted strings.\r
- * Therefore the tokenizer must be able to read in a token of arbitrary\r
- * size. Instead of allocating a fixed length buffer, the tokenizer first\r
- * scans the input a single character at a time looking for the begining\r
- * and end of the token. Once the these positions are found, the entire\r
- * token is read into memory. By using this algorithm,the implementation\r
- * does not place an arbitrary maximum on the token size.\r
- */\r
-\r
- token_len = 0;\r
- num_escape_seq = 0;\r
- is_quoted_str = DAT_FALSE;\r
- is_prev_char_backslash = DAT_FALSE;\r
-\r
- for (;;) {\r
- DAT_OS_FILE_POS cur_pos;\r
- int c;\r
-\r
- /* if looking for start of the token */\r
- if (0 == token_len) {\r
- if (DAT_SUCCESS != dat_os_fgetpos(file, &cur_pos)) {\r
- return DAT_INTERNAL_ERROR;\r
- }\r
- }\r
-\r
- c = dat_os_fgetc(file);\r
-\r
- /* if looking for start of the token */\r
- if (0 == token_len) {\r
- if (EOF == c) {\r
- token->type = DAT_SR_TOKEN_EOF;\r
- token->value = NULL;\r
- token->value_len = 0;\r
- goto success;\r
- } else if (DAT_SR_CHAR_NEWLINE == c) {\r
- token->type = DAT_SR_TOKEN_EOR;\r
- token->value = NULL;\r
- token->value_len = 0;\r
- goto success;\r
- } else if (dat_os_isblank(c)) {\r
- continue;\r
- } else if (DAT_SR_CHAR_COMMENT == c) {\r
- dat_sr_read_comment(file);\r
- continue;\r
- } else {\r
- if (DAT_SR_CHAR_QUOTE == c) {\r
- is_quoted_str = DAT_TRUE;\r
- }\r
-\r
- pos = cur_pos;\r
- token_len++;\r
- }\r
- } else { /* looking for the end of the token */\r
-\r
- if (EOF == c) {\r
- break;\r
- } else if (DAT_SR_CHAR_NEWLINE == c) {\r
- /* put back the newline */\r
- dat_os_fputc(file, c);\r
- break;\r
- } else if (!is_quoted_str && dat_os_isblank(c)) {\r
- break;\r
- } else {\r
- token_len++;\r
-\r
- if ((DAT_SR_CHAR_QUOTE == c)\r
- && !is_prev_char_backslash) {\r
- break;\r
- } else if ((DAT_SR_CHAR_BACKSLASH == c)\r
- && !is_prev_char_backslash) {\r
- is_prev_char_backslash = DAT_TRUE;\r
- num_escape_seq++;\r
- } else {\r
- is_prev_char_backslash = DAT_FALSE;\r
- }\r
- }\r
- }\r
- }\r
-\r
- /* the token was a string */\r
- if (DAT_SUCCESS != dat_os_fsetpos(file, &pos)) {\r
- return DAT_INTERNAL_ERROR;\r
- }\r
-\r
- if (is_quoted_str) {\r
- if (DAT_SUCCESS != dat_sr_read_quoted_str(file,\r
- token,\r
- token_len,\r
- num_escape_seq)) {\r
- return DAT_INTERNAL_ERROR;\r
- }\r
- } else {\r
- if (DAT_SUCCESS != dat_sr_read_str(file, token, token_len)) {\r
- return DAT_INTERNAL_ERROR;\r
- }\r
- }\r
-\r
- success:\r
- dat_os_dbg_print(DAT_OS_DBG_TYPE_SR,\r
- "\n"\r
- "DAT Registry: token\n"\r
- " type %s\n"\r
- " value <%s>\n"\r
- "\n",\r
- dat_sr_type_to_str(token->type),\r
- ((DAT_SR_TOKEN_STRING ==\r
- token->type) ? token->value : ""));\r
-\r
- return DAT_SUCCESS;\r
-}\r
-\r
-/***********************************************************************\r
- * Function: dat_sr_read_str\r
- ***********************************************************************/\r
-\r
-DAT_RETURN\r
-dat_sr_read_str(DAT_OS_FILE * file, DAT_SR_TOKEN * token, DAT_OS_SIZE token_len)\r
-{\r
- token->type = DAT_SR_TOKEN_STRING;\r
- token->value_len = sizeof(char) * (token_len + 1); /* +1 for null termination */\r
- if (NULL == (token->value = dat_os_alloc(token->value_len))) {\r
- return DAT_INSUFFICIENT_RESOURCES | DAT_RESOURCE_MEMORY;\r
- }\r
-\r
- if (token_len != dat_os_fread(file, token->value, token_len)) {\r
- dat_os_free(token->value, token->value_len);\r
- token->value = NULL;\r
-\r
- return DAT_INTERNAL_ERROR;\r
- }\r
-\r
- token->value[token->value_len - 1] = '\0';\r
-\r
- return DAT_SUCCESS;\r
-}\r
-\r
-/***********************************************************************\r
- * Function: dat_sr_read_quoted_str\r
- ***********************************************************************/\r
-\r
-DAT_RETURN\r
-dat_sr_read_quoted_str(DAT_OS_FILE * file,\r
- DAT_SR_TOKEN * token,\r
- DAT_OS_SIZE token_len, DAT_COUNT num_escape_seq)\r
-{\r
- DAT_OS_SIZE str_len;\r
- DAT_OS_SIZE i;\r
- DAT_OS_SIZE j;\r
- int c;\r
- DAT_RETURN status;\r
- DAT_BOOLEAN is_prev_char_backslash;\r
-\r
- str_len = token_len - 2; /* minus 2 " characters */\r
- is_prev_char_backslash = DAT_FALSE;\r
- status = DAT_SUCCESS;\r
-\r
- token->type = DAT_SR_TOKEN_STRING;\r
- /* +1 for null termination */\r
- token->value_len = sizeof(char) * (str_len - num_escape_seq + 1);\r
-\r
- if (NULL == (token->value = dat_os_alloc(token->value_len))) {\r
- status = DAT_INSUFFICIENT_RESOURCES | DAT_RESOURCE_MEMORY;\r
- goto exit;\r
- }\r
-\r
- /* throw away " */\r
- if (DAT_SR_CHAR_QUOTE != dat_os_fgetc(file)) {\r
- status = DAT_INTERNAL_ERROR;\r
- goto exit;\r
- }\r
-\r
- for (i = 0, j = 0; i < str_len; i++) {\r
- c = dat_os_fgetc(file);\r
-\r
- if (EOF == c) {\r
- status = DAT_INTERNAL_ERROR;\r
- goto exit;\r
- } else if ((DAT_SR_CHAR_BACKSLASH == c)\r
- && !is_prev_char_backslash) {\r
- is_prev_char_backslash = DAT_TRUE;\r
- } else {\r
- token->value[j] = (char)c;\r
- j++;\r
-\r
- is_prev_char_backslash = DAT_FALSE;\r
- }\r
- }\r
-\r
- /* throw away " */\r
- if (DAT_SR_CHAR_QUOTE != dat_os_fgetc(file)) {\r
- status = DAT_INTERNAL_ERROR;\r
- goto exit;\r
- }\r
-\r
- token->value[token->value_len - 1] = '\0';\r
-\r
- exit:\r
- if (DAT_SUCCESS != status) {\r
- if (NULL != token->value) {\r
- dat_os_free(token->value, token->value_len);\r
- token->value = NULL;\r
- }\r
- }\r
-\r
- return status;\r
-}\r
-\r
-/***********************************************************************\r
- * Function: dat_sr_read_comment\r
- ***********************************************************************/\r
-\r
-void dat_sr_read_comment(DAT_OS_FILE * file)\r
-{\r
- int c;\r
-\r
- /* read up to an EOR or EOF to move past the comment */\r
- do {\r
- c = dat_os_fgetc(file);\r
- } while ((DAT_SR_CHAR_NEWLINE != c) && (EOF != c));\r
-\r
- /* put back the newline */\r
- dat_os_ungetc(file, c);\r
-}\r
+/*
+ * Copyright (c) 2002-2005, Network Appliance, Inc. All rights reserved.
+ *
+ * This Software is licensed under one of the following licenses:
+ *
+ * 1) under the terms of the "Common Public License 1.0" a copy of which is
+ * in the file LICENSE.txt in the root directory. The license is also
+ * available from the Open Source Initiative, see
+ * http://www.opensource.org/licenses/cpl.php.
+ *
+ * 2) under the terms of the "The BSD License" a copy of which is in the file
+ * LICENSE2.txt in the root directory. The license is also available from
+ * the Open Source Initiative, see
+ * http://www.opensource.org/licenses/bsd-license.php.
+ *
+ * 3) under the terms of the "GNU General Public License (GPL) Version 2" a
+ * copy of which is in the file LICENSE3.txt in the root directory. The
+ * license is also available from the Open Source Initiative, see
+ * http://www.opensource.org/licenses/gpl-license.php.
+ *
+ * Licensee has the right to choose one of the above licenses.
+ *
+ * Redistributions of source code must retain the above copyright
+ * notice and one of the license notices.
+ *
+ * Redistributions in binary form must reproduce both the above copyright
+ * notice, one of the license notices in the documentation
+ * and/or other materials provided with the distribution.
+ */
+
+/**********************************************************************
+ *
+ * MODULE: dat_sr_parser.c
+ *
+ * PURPOSE: static registry parser
+ *
+ * $Id: udat_sr_parser.c,v 1.6 2005/03/24 05:58:36 jlentini Exp $
+ **********************************************************************/
+
+#include <dat2/udat.h>
+#include "udat_sr_parser.h"
+#include "dat_sr.h"
+
+/*********************************************************************
+ * *
+ * Constants *
+ * *
+ *********************************************************************/
+
+#define DAT_SR_CONF_ENV "DAT_OVERRIDE"
+#if defined(_WIN32) || defined(_WIN64)
+#define DAT_SR_CONF_DEFAULT "C:\\DAT\\dat.conf"
+#else
+#define DAT_SR_CONF_DEFAULT "/etc/dat.conf"
+#endif
+
+#define DAT_SR_TOKEN_THREADSAFE "threadsafe"
+#define DAT_SR_TOKEN_NONTHREADSAFE "nonthreadsafe"
+#define DAT_SR_TOKEN_DEFAULT "default"
+#define DAT_SR_TOKEN_NONDEFAULT "nondefault"
+
+#define DAT_SR_CHAR_NEWLINE '\n'
+#define DAT_SR_CHAR_COMMENT '#'
+#define DAT_SR_CHAR_QUOTE '"'
+#define DAT_SR_CHAR_BACKSLASH '\\'
+
+/*********************************************************************
+ * *
+ * Enumerations *
+ * *
+ *********************************************************************/
+
+typedef enum {
+ DAT_SR_TOKEN_STRING, /* text field (both quoted or unquoted) */
+ DAT_SR_TOKEN_EOR, /* end of record (newline) */
+ DAT_SR_TOKEN_EOF /* end of file */
+} DAT_SR_TOKEN_TYPE;
+
+typedef enum {
+ DAT_SR_API_UDAT,
+ DAT_SR_API_KDAT
+} DAT_SR_API_TYPE;
+
+/*********************************************************************
+ * *
+ * Structures *
+ * *
+ *********************************************************************/
+
+typedef struct {
+ DAT_SR_TOKEN_TYPE type;
+ char *value; /* valid if type is DAT_SR_TOKEN_STRING */
+ DAT_OS_SIZE value_len;
+} DAT_SR_TOKEN;
+
+typedef struct DAT_SR_STACK_NODE {
+ DAT_SR_TOKEN token;
+ struct DAT_SR_STACK_NODE *next;
+} DAT_SR_STACK_NODE;
+
+typedef struct {
+ DAT_UINT32 major;
+ DAT_UINT32 minor;
+} DAT_SR_VERSION;
+
+typedef struct {
+ char *id;
+ DAT_SR_VERSION version;
+} DAT_SR_PROVIDER_VERSION;
+
+typedef struct {
+ DAT_SR_API_TYPE type;
+ DAT_SR_VERSION version;
+} DAT_SR_API_VERSION;
+
+typedef struct {
+ char *ia_name;
+ DAT_SR_API_VERSION api_version;
+ DAT_BOOLEAN is_thread_safe;
+ DAT_BOOLEAN is_default;
+ char *lib_path;
+ DAT_SR_PROVIDER_VERSION provider_version;
+ char *ia_params;
+ char *platform_params;
+} DAT_SR_CONF_ENTRY;
+
+/*********************************************************************
+ * *
+ * Internal Function Declarations *
+ * *
+ *********************************************************************/
+
+static DAT_RETURN dat_sr_load_entry(DAT_SR_CONF_ENTRY * entry);
+
+static DAT_BOOLEAN dat_sr_is_valid_entry(DAT_SR_CONF_ENTRY * entry);
+
+static char *dat_sr_type_to_str(DAT_SR_TOKEN_TYPE type);
+
+static DAT_RETURN dat_sr_parse_eof(DAT_OS_FILE * file);
+
+static DAT_RETURN dat_sr_parse_entry(DAT_OS_FILE * file);
+
+static DAT_RETURN
+dat_sr_parse_ia_name(DAT_OS_FILE * file, DAT_SR_CONF_ENTRY * entry);
+
+static DAT_RETURN
+dat_sr_parse_api(DAT_OS_FILE * file, DAT_SR_CONF_ENTRY * entry);
+
+static DAT_RETURN
+dat_sr_parse_thread_safety(DAT_OS_FILE * file, DAT_SR_CONF_ENTRY * entry);
+
+static DAT_RETURN
+dat_sr_parse_default(DAT_OS_FILE * file, DAT_SR_CONF_ENTRY * entry);
+
+static DAT_RETURN
+dat_sr_parse_lib_path(DAT_OS_FILE * file, DAT_SR_CONF_ENTRY * entry);
+
+static DAT_RETURN
+dat_sr_parse_provider_version(DAT_OS_FILE * file, DAT_SR_CONF_ENTRY * entry);
+
+static DAT_RETURN
+dat_sr_parse_ia_params(DAT_OS_FILE * file, DAT_SR_CONF_ENTRY * entry);
+
+static DAT_RETURN
+dat_sr_parse_platform_params(DAT_OS_FILE * file, DAT_SR_CONF_ENTRY * entry);
+
+static DAT_RETURN
+dat_sr_parse_eoe(DAT_OS_FILE * file, DAT_SR_CONF_ENTRY * entry);
+
+static DAT_RETURN
+dat_sr_convert_api(char *str, DAT_SR_API_VERSION * api_version);
+
+static DAT_RETURN
+dat_sr_convert_thread_safety(char *str, DAT_BOOLEAN * is_thread_safe);
+
+static DAT_RETURN dat_sr_convert_default(char *str, DAT_BOOLEAN * is_default);
+
+static DAT_RETURN
+dat_sr_convert_provider_version(char *str,
+ DAT_SR_PROVIDER_VERSION * provider_version);
+
+static DAT_RETURN dat_sr_get_token(DAT_OS_FILE * file, DAT_SR_TOKEN * token);
+
+static DAT_RETURN dat_sr_put_token(DAT_OS_FILE * file, DAT_SR_TOKEN * token);
+
+static DAT_RETURN dat_sr_read_token(DAT_OS_FILE * file, DAT_SR_TOKEN * token);
+
+static DAT_RETURN
+dat_sr_read_str(DAT_OS_FILE * file,
+ DAT_SR_TOKEN * token, DAT_OS_SIZE token_len);
+
+static DAT_RETURN
+dat_sr_read_quoted_str(DAT_OS_FILE * file,
+ DAT_SR_TOKEN * token,
+ DAT_OS_SIZE token_len, DAT_COUNT num_escape_seq);
+
+static void dat_sr_read_comment(DAT_OS_FILE * file);
+
+/*********************************************************************
+ * *
+ * Global Variables *
+ * *
+ *********************************************************************/
+
+static DAT_SR_STACK_NODE *g_token_stack = NULL;
+
+/*********************************************************************
+ * *
+ * External Function Definitions *
+ * *
+ *********************************************************************/
+
+/***********************************************************************
+ * Function: dat_sr_load
+ ***********************************************************************/
+
+DAT_RETURN dat_sr_load(void)
+{
+ char *sr_path;
+ DAT_OS_FILE *sr_file;
+
+ sr_path = dat_os_getenv(DAT_SR_CONF_ENV);
+
+ /* environment override */
+ if ((sr_path != NULL) && ((sr_file = dat_os_fopen(sr_path)) == NULL)) {
+ dat_os_dbg_print(DAT_OS_DBG_TYPE_ERROR,
+ "DAT Registry: DAT_OVERRIDE, "
+ "bad filename - %s, aborting\n", sr_path);
+ goto bail;
+ }
+
+ if (sr_path == NULL) {
+
+#ifdef DAT_CONF
+ sr_path = DAT_CONF;
+#else
+ sr_path = DAT_SR_CONF_DEFAULT;
+#endif
+ sr_file = dat_os_fopen(sr_path);
+ if (sr_file == NULL) {
+#ifdef DAT_CONF
+ dat_os_dbg_print(DAT_OS_DBG_TYPE_SR,
+ "DAT Registry: sysconfdir, "
+ "bad filename - %s, retry default at %s\n",
+ sr_path, DAT_SR_CONF_DEFAULT);
+ /* try default after sysconfdir fails */
+ sr_path = DAT_SR_CONF_DEFAULT;
+ sr_file = dat_os_fopen(sr_path);
+ if (sr_file == NULL) {
+#endif
+ dat_os_dbg_print(DAT_OS_DBG_TYPE_ERROR,
+ "DAT Registry: default, "
+ "bad filename - %s, aborting\n",
+ sr_path);
+ goto bail;
+#ifdef DAT_CONF
+ }
+#endif
+ }
+ }
+
+ dat_os_dbg_print(DAT_OS_DBG_TYPE_GENERIC,
+ "DAT Registry: using config file %s\n", sr_path);
+
+ for (;;) {
+ if (DAT_SUCCESS == dat_sr_parse_eof(sr_file)) {
+ break;
+ } else if (DAT_SUCCESS == dat_sr_parse_entry(sr_file)) {
+ continue;
+ } else {
+ dat_os_dbg_print(DAT_OS_DBG_TYPE_ERROR,
+ "DAT Registry: ERROR parsing - %s\n",
+ sr_path);
+ goto cleanup;
+ }
+ }
+
+ if (0 != dat_os_fclose(sr_file)) {
+ dat_os_dbg_print(DAT_OS_DBG_TYPE_ERROR,
+ "DAT Registry: ERROR closing - %s\n", sr_path);
+ goto bail;
+ }
+
+ return DAT_SUCCESS;
+
+ cleanup:
+ dat_os_fclose(sr_file);
+ bail:
+ return DAT_INTERNAL_ERROR;
+
+}
+
+/*********************************************************************
+ * *
+ * Internal Function Definitions *
+ * *
+ *********************************************************************/
+
+/***********************************************************************
+ * Function: dat_sr_is_valid_entry
+ ***********************************************************************/
+
+DAT_BOOLEAN dat_sr_is_valid_entry(DAT_SR_CONF_ENTRY * entry)
+{
+ if ((DAT_SR_API_UDAT == entry->api_version.type) && (entry->is_default)) {
+ return DAT_TRUE;
+ } else {
+ return DAT_FALSE;
+ }
+}
+
+/***********************************************************************
+ * Function: dat_sr_load_entry
+ ***********************************************************************/
+
+DAT_RETURN dat_sr_load_entry(DAT_SR_CONF_ENTRY * conf_entry)
+{
+ DAT_SR_ENTRY entry;
+
+ if (DAT_NAME_MAX_LENGTH < (strlen(conf_entry->ia_name) + 1)) {
+ dat_os_dbg_print(DAT_OS_DBG_TYPE_SR,
+ "DAT Registry: ia name %s is longer than "
+ "DAT_NAME_MAX_LENGTH (%i)\n",
+ conf_entry->ia_name, DAT_NAME_MAX_LENGTH);
+
+ return DAT_INSUFFICIENT_RESOURCES;
+ }
+
+ dat_os_strncpy(entry.info.ia_name, conf_entry->ia_name,
+ DAT_NAME_MAX_LENGTH);
+ entry.info.dapl_version_major = conf_entry->api_version.version.major;
+ entry.info.dapl_version_minor = conf_entry->api_version.version.minor;
+ entry.info.is_thread_safe = conf_entry->is_thread_safe;
+ entry.lib_path = conf_entry->lib_path;
+ entry.ia_params = conf_entry->ia_params;
+ entry.lib_handle = NULL;
+ entry.ref_count = 0;
+
+ dat_os_dbg_print(DAT_OS_DBG_TYPE_SR,
+ "DAT Registry: loading provider for %s\n",
+ conf_entry->ia_name);
+
+ return dat_sr_insert(&entry.info, &entry);
+}
+
+/***********************************************************************
+ * Function: dat_sr_type_to_str
+ ***********************************************************************/
+
+char *dat_sr_type_to_str(DAT_SR_TOKEN_TYPE type)
+{
+ static char *str_array[] = { "string", "eor", "eof" };
+
+ if ((type < 0) || (2 < type)) {
+ return "error: invalid token type";
+ }
+
+ return str_array[type];
+}
+
+/***********************************************************************
+ * Function: dat_sr_parse_eof
+ ***********************************************************************/
+
+DAT_RETURN dat_sr_parse_eof(DAT_OS_FILE * file)
+{
+ DAT_SR_TOKEN token;
+
+ if (DAT_SUCCESS != dat_sr_get_token(file, &token)) {
+ return DAT_INTERNAL_ERROR;
+ }
+
+ if (DAT_SR_TOKEN_EOF == token.type) {
+ return DAT_SUCCESS;
+ } else {
+ dat_sr_put_token(file, &token);
+ return DAT_INTERNAL_ERROR;
+ }
+}
+
+/***********************************************************************
+ * Function: dat_sr_parse_ia_name
+ ***********************************************************************/
+
+DAT_RETURN dat_sr_parse_entry(DAT_OS_FILE * file)
+{
+ DAT_SR_CONF_ENTRY entry;
+ DAT_RETURN status;
+
+ dat_os_memset(&entry, 0, sizeof(DAT_SR_CONF_ENTRY));
+
+ if ((DAT_SUCCESS == dat_sr_parse_ia_name(file, &entry)) &&
+ (DAT_SUCCESS == dat_sr_parse_api(file, &entry)) &&
+ (DAT_SUCCESS == dat_sr_parse_thread_safety(file, &entry)) &&
+ (DAT_SUCCESS == dat_sr_parse_default(file, &entry)) &&
+ (DAT_SUCCESS == dat_sr_parse_lib_path(file, &entry)) &&
+ (DAT_SUCCESS == dat_sr_parse_provider_version(file, &entry)) &&
+ (DAT_SUCCESS == dat_sr_parse_ia_params(file, &entry)) &&
+ (DAT_SUCCESS == dat_sr_parse_platform_params(file, &entry)) &&
+ (DAT_SUCCESS == dat_sr_parse_eoe(file, &entry))) {
+ dat_os_dbg_print(DAT_OS_DBG_TYPE_SR,
+ "\n"
+ "DAT Registry: entry \n"
+ " ia_name %s\n"
+ " api_version\n"
+ " type 0x%X\n"
+ " major.minor %d.%d\n"
+ " is_thread_safe %d\n"
+ " is_default %d\n"
+ " lib_path %s\n"
+ " provider_version\n"
+ " id %s\n"
+ " major.minor %d.%d\n"
+ " ia_params %s\n"
+ "\n",
+ entry.ia_name,
+ entry.api_version.type,
+ entry.api_version.version.major,
+ entry.api_version.version.minor,
+ entry.is_thread_safe,
+ entry.is_default,
+ entry.lib_path,
+ entry.provider_version.id,
+ entry.provider_version.version.major,
+ entry.provider_version.version.minor,
+ entry.ia_params);
+
+ if (DAT_TRUE == dat_sr_is_valid_entry(&entry)) {
+ /*
+ * The static registry configuration file may have multiple
+ * entries with the same IA name. The first entry will be
+ * installed in the static registry causing subsequent attempts
+ * to register the same IA name to fail. Therefore the return code
+ * from dat_sr_load_entry() is ignored.
+ */
+ (void)dat_sr_load_entry(&entry);
+ }
+
+ status = DAT_SUCCESS;
+ } else { /* resync */
+
+ DAT_SR_TOKEN token;
+
+ /*
+ * The static registry format is specified in the DAT specification.
+ * While the registry file's contents may change between revisions of
+ * the specification, there is no way to determine the specification
+ * version to which the configuration file conforms. If an entry is
+ * found that does not match the expected format, the entry is discarded
+ * and the parsing of the file continues. There is no way to determine if
+ * the entry was an error or an entry confirming to an alternate version
+ * of specification.
+ */
+
+ for (;;) {
+ if (DAT_SUCCESS != dat_sr_get_token(file, &token)) {
+ status = DAT_INTERNAL_ERROR;
+ break;
+ }
+
+ if (DAT_SR_TOKEN_STRING != token.type) {
+ status = DAT_SUCCESS;
+ break;
+ } else {
+ dat_os_free(token.value,
+ (sizeof(char) *
+ dat_os_strlen(token.value)) + 1);
+ continue;
+ }
+ }
+ }
+
+ /* free resources */
+ if (NULL != entry.ia_name) {
+ dat_os_free(entry.ia_name,
+ sizeof(char) * (dat_os_strlen(entry.ia_name) + 1));
+ }
+ if (NULL != entry.lib_path) {
+ dat_os_free(entry.lib_path,
+ sizeof(char) * (dat_os_strlen(entry.lib_path) + 1));
+ }
+
+ if (NULL != entry.provider_version.id) {
+ dat_os_free(entry.provider_version.id,
+ sizeof(char) *
+ (dat_os_strlen(entry.provider_version.id) + 1));
+ }
+
+ if (NULL != entry.ia_params) {
+ dat_os_free(entry.ia_params,
+ sizeof(char) * (dat_os_strlen(entry.ia_params) +
+ 1));
+ }
+
+ return status;
+}
+
+/***********************************************************************
+ * Function: dat_sr_parse_ia_name
+ ***********************************************************************/
+
+DAT_RETURN dat_sr_parse_ia_name(DAT_OS_FILE * file, DAT_SR_CONF_ENTRY * entry)
+{
+ DAT_SR_TOKEN token;
+
+ if (DAT_SUCCESS != dat_sr_get_token(file, &token))
+ goto bail;
+
+ if (DAT_SR_TOKEN_STRING != token.type) {
+ dat_sr_put_token(file, &token);
+ goto bail;
+ }
+ entry->ia_name = token.value;
+ return DAT_SUCCESS;
+
+ bail:
+ return DAT_INTERNAL_ERROR;
+}
+
+/***********************************************************************
+ * Function: dat_sr_parse_ia_name
+ ***********************************************************************/
+
+DAT_RETURN dat_sr_parse_api(DAT_OS_FILE * file, DAT_SR_CONF_ENTRY * entry)
+{
+ DAT_SR_TOKEN token;
+
+ if (DAT_SUCCESS != dat_sr_get_token(file, &token))
+ goto bail;
+
+ if (DAT_SR_TOKEN_STRING != token.type)
+ goto cleanup;
+
+ if (DAT_SUCCESS != dat_sr_convert_api(token.value, &entry->api_version))
+ goto cleanup;
+
+ dat_os_free(token.value,
+ (sizeof(char) * dat_os_strlen(token.value)) + 1);
+ return DAT_SUCCESS;
+
+ cleanup:
+ dat_sr_put_token(file, &token);
+ bail:
+ dat_os_dbg_print(DAT_OS_DBG_TYPE_ERROR,
+ " ERR: corrupt dat.conf entry field:"
+ " api_ver, file offset=%ld\n", ftell(file));
+ return DAT_INTERNAL_ERROR;
+}
+
+/***********************************************************************
+ * Function: dat_sr_parse_thread_safety
+ ***********************************************************************/
+
+static DAT_RETURN
+dat_sr_parse_thread_safety(DAT_OS_FILE * file, DAT_SR_CONF_ENTRY * entry)
+{
+ DAT_SR_TOKEN token;
+
+ if (DAT_SUCCESS != dat_sr_get_token(file, &token))
+ goto bail;
+
+ if (DAT_SR_TOKEN_STRING != token.type)
+ goto cleanup;
+
+ if (DAT_SUCCESS !=
+ dat_sr_convert_thread_safety(token.value, &entry->is_thread_safe))
+ goto cleanup;
+
+ dat_os_free(token.value,
+ (sizeof(char) * dat_os_strlen(token.value)) + 1);
+ return DAT_SUCCESS;
+
+ cleanup:
+ dat_sr_put_token(file, &token);
+ bail:
+ dat_os_dbg_print(DAT_OS_DBG_TYPE_ERROR,
+ " ERR: corrupt dat.conf entry field:"
+ " thread_safety, file offset=%ld\n", ftell(file));
+ return DAT_INTERNAL_ERROR;
+}
+
+/***********************************************************************
+ * Function: dat_sr_parse_default
+ ***********************************************************************/
+
+DAT_RETURN dat_sr_parse_default(DAT_OS_FILE * file, DAT_SR_CONF_ENTRY * entry)
+{
+ DAT_SR_TOKEN token;
+
+ if (DAT_SUCCESS != dat_sr_get_token(file, &token))
+ goto bail;
+
+ if (DAT_SR_TOKEN_STRING != token.type)
+ goto cleanup;
+
+ if (DAT_SUCCESS !=
+ dat_sr_convert_default(token.value, &entry->is_default))
+ goto cleanup;
+
+ dat_os_free(token.value,
+ (sizeof(char) * dat_os_strlen(token.value)) + 1);
+ return DAT_SUCCESS;
+
+ cleanup:
+ dat_sr_put_token(file, &token);
+ bail:
+ dat_os_dbg_print(DAT_OS_DBG_TYPE_ERROR,
+ " ERR: corrupt dat.conf entry field:"
+ " default section, file offset=%ld\n", ftell(file));
+ return DAT_INTERNAL_ERROR;
+}
+
+/***********************************************************************
+ * Function: dat_sr_parse_lib_path
+ ***********************************************************************/
+
+DAT_RETURN dat_sr_parse_lib_path(DAT_OS_FILE * file, DAT_SR_CONF_ENTRY * entry)
+{
+ DAT_SR_TOKEN token;
+
+ if (DAT_SUCCESS != dat_sr_get_token(file, &token))
+ goto bail;
+
+ if (DAT_SR_TOKEN_STRING != token.type) {
+ dat_sr_put_token(file, &token);
+ goto bail;
+ }
+ entry->lib_path = token.value;
+ return DAT_SUCCESS;
+
+ bail:
+ dat_os_dbg_print(DAT_OS_DBG_TYPE_ERROR,
+ " ERR: corrupt dat.conf entry field:"
+ " lib_path, file offset=%ld\n", ftell(file));
+ return DAT_INTERNAL_ERROR;
+}
+
+/***********************************************************************
+ * Function: dat_sr_parse_provider_version
+ ***********************************************************************/
+
+DAT_RETURN
+dat_sr_parse_provider_version(DAT_OS_FILE * file, DAT_SR_CONF_ENTRY * entry)
+{
+ DAT_SR_TOKEN token;
+
+ if (DAT_SUCCESS != dat_sr_get_token(file, &token))
+ goto bail;
+
+ if (DAT_SR_TOKEN_STRING != token.type)
+ goto cleanup;
+
+ if (DAT_SUCCESS !=
+ dat_sr_convert_provider_version(token.value,
+ &entry->provider_version))
+ goto cleanup;
+
+ dat_os_free(token.value,
+ (sizeof(char) * dat_os_strlen(token.value)) + 1);
+ return DAT_SUCCESS;
+
+ cleanup:
+ dat_sr_put_token(file, &token);
+ bail:
+ dat_os_dbg_print(DAT_OS_DBG_TYPE_ERROR,
+ " ERR: corrupt dat.conf entry field:"
+ " provider_ver, file offset=%ld\n", ftell(file));
+ return DAT_INTERNAL_ERROR;
+}
+
+/***********************************************************************
+ * Function: dat_sr_parse_ia_params
+ ***********************************************************************/
+
+DAT_RETURN dat_sr_parse_ia_params(DAT_OS_FILE * file, DAT_SR_CONF_ENTRY * entry)
+{
+ DAT_SR_TOKEN token;
+
+ if (DAT_SUCCESS != dat_sr_get_token(file, &token))
+ goto bail;
+
+ if (DAT_SR_TOKEN_STRING != token.type) {
+ dat_sr_put_token(file, &token);
+ goto bail;
+ }
+
+ entry->ia_params = token.value;
+ return DAT_SUCCESS;
+
+ bail:
+ dat_os_dbg_print(DAT_OS_DBG_TYPE_ERROR,
+ " ERR: corrupt dat.conf entry field:"
+ " ia_params, file offset=%ld\n", ftell(file));
+ return DAT_INTERNAL_ERROR;
+}
+
+/***********************************************************************
+ * Function: dat_sr_parse_platform_params
+ ***********************************************************************/
+
+DAT_RETURN
+dat_sr_parse_platform_params(DAT_OS_FILE * file, DAT_SR_CONF_ENTRY * entry)
+{
+ DAT_SR_TOKEN token;
+
+ if (DAT_SUCCESS != dat_sr_get_token(file, &token))
+ goto bail;
+
+ if (DAT_SR_TOKEN_STRING != token.type) {
+ dat_sr_put_token(file, &token);
+ goto bail;
+ }
+
+ entry->platform_params = token.value;
+ return DAT_SUCCESS;
+
+ bail:
+ dat_os_dbg_print(DAT_OS_DBG_TYPE_ERROR,
+ " ERR: corrupt dat.conf entry field:"
+ " platform_params, file offset=%ld\n", ftell(file));
+ return DAT_INTERNAL_ERROR;
+}
+
+/***********************************************************************
+ * Function: dat_sr_parse_eoe
+ ***********************************************************************/
+
+DAT_RETURN dat_sr_parse_eoe(DAT_OS_FILE * file, DAT_SR_CONF_ENTRY * entry)
+{
+ DAT_SR_TOKEN token;
+
+ if (DAT_SUCCESS != dat_sr_get_token(file, &token))
+ goto bail;
+
+ if ((DAT_SR_TOKEN_EOF != token.type) &&
+ (DAT_SR_TOKEN_EOR != token.type)) {
+ dat_sr_put_token(file, &token);
+ goto bail;
+ }
+
+ return DAT_SUCCESS;
+
+ bail:
+ dat_os_dbg_print(DAT_OS_DBG_TYPE_ERROR,
+ " ERR: corrupt dat.conf entry field:"
+ " EOR, EOF, file offset=%ld\n", ftell(file));
+ return DAT_INTERNAL_ERROR;
+}
+
+/***********************************************************************
+ * Function: dat_sr_convert_api
+ ***********************************************************************/
+
+DAT_RETURN dat_sr_convert_api(char *str, DAT_SR_API_VERSION * api_version)
+{
+ int i;
+ int minor_i;
+
+ if (dat_os_strlen(str) <= 0)
+ return DAT_INTERNAL_ERROR;
+
+ if ('u' == str[0]) {
+ api_version->type = DAT_SR_API_UDAT;
+ } else if ('k' == str[0]) {
+ api_version->type = DAT_SR_API_KDAT;
+ } else {
+ return DAT_INTERNAL_ERROR;
+ }
+
+ for (i = 1 /* move past initial [u|k] */ ; '\0' != str[i]; i++) {
+ if ('.' == str[i]) {
+ break;
+ } else if (DAT_TRUE != dat_os_isdigit(str[i])) {
+ return DAT_INTERNAL_ERROR;
+ }
+ }
+
+ api_version->version.major =
+ (DAT_UINT32) dat_os_strtol(str + 1, NULL, 10);
+
+ /* move past '.' */
+ minor_i = ++i;
+
+ for (; '\0' != str[i]; i++) {
+ if (DAT_TRUE != dat_os_isdigit(str[i])) {
+ return DAT_INTERNAL_ERROR;
+ }
+ }
+
+ api_version->version.minor =
+ (DAT_UINT32) dat_os_strtol(str + minor_i, NULL, 10);
+
+ if ('\0' != str[i]) {
+ return DAT_INTERNAL_ERROR;
+ }
+
+ return DAT_SUCCESS;
+}
+
+/***********************************************************************
+ * Function: dat_sr_convert_thread_safety
+ ***********************************************************************/
+
+static DAT_RETURN
+dat_sr_convert_thread_safety(char *str, DAT_BOOLEAN * is_thread_safe)
+{
+ if (!dat_os_strncmp(str,
+ DAT_SR_TOKEN_THREADSAFE,
+ dat_os_strlen(DAT_SR_TOKEN_THREADSAFE))) {
+ *is_thread_safe = DAT_TRUE;
+ return DAT_SUCCESS;
+ } else if (!dat_os_strncmp(str,
+ DAT_SR_TOKEN_NONTHREADSAFE,
+ dat_os_strlen(DAT_SR_TOKEN_NONTHREADSAFE))) {
+ *is_thread_safe = DAT_FALSE;
+ return DAT_SUCCESS;
+ } else {
+ return DAT_INTERNAL_ERROR;
+ }
+}
+
+/***********************************************************************
+ * Function: dat_sr_convert_default
+ ***********************************************************************/
+
+static DAT_RETURN dat_sr_convert_default(char *str, DAT_BOOLEAN * is_default)
+{
+ if (!dat_os_strncmp(str,
+ DAT_SR_TOKEN_DEFAULT,
+ dat_os_strlen(DAT_SR_TOKEN_DEFAULT))) {
+ *is_default = DAT_TRUE;
+ return DAT_SUCCESS;
+ } else if (!dat_os_strncmp(str,
+ DAT_SR_TOKEN_NONDEFAULT,
+ dat_os_strlen(DAT_SR_TOKEN_NONDEFAULT))) {
+ *is_default = DAT_FALSE;
+ return DAT_SUCCESS;
+ } else {
+ return DAT_INTERNAL_ERROR;
+ }
+}
+
+/***********************************************************************
+ * Function: dat_sr_convert_provider_version
+ ***********************************************************************/
+
+DAT_RETURN
+dat_sr_convert_provider_version(char *str,
+ DAT_SR_PROVIDER_VERSION * provider_version)
+{
+ DAT_RETURN status;
+ int i;
+ int decimal_i;
+
+ if ((dat_os_strlen(str) <= 0) || (NULL != provider_version->id))
+ return DAT_INTERNAL_ERROR;
+
+ status = DAT_SUCCESS;
+
+ for (i = 0; '\0' != str[i]; i++) {
+ if ('.' == str[i]) {
+ break;
+ }
+ }
+
+ /* if no id value was found */
+ if (0 == i) {
+ status = DAT_INTERNAL_ERROR;
+ goto exit;
+ }
+
+ if (NULL ==
+ (provider_version->id = dat_os_alloc(sizeof(char) * (i + 1)))) {
+ status =
+ DAT_ERROR(DAT_INSUFFICIENT_RESOURCES, DAT_RESOURCE_MEMORY);
+ goto exit;
+ }
+
+ dat_os_strncpy(provider_version->id, str, i);
+ provider_version->id[i] = '\0';
+
+ /* move past '.' */
+ decimal_i = ++i;
+
+ for (; '\0' != str[i]; i++) {
+ if ('.' == str[i]) {
+ break;
+ } else if (DAT_TRUE != dat_os_isdigit(str[i])) {
+ status = DAT_INTERNAL_ERROR;
+ goto exit;
+ }
+ }
+
+ /* if no version value was found */
+ if (decimal_i == i) {
+ status = DAT_INTERNAL_ERROR;
+ goto exit;
+ }
+
+ provider_version->version.major = (DAT_UINT32)
+ dat_os_strtol(str + decimal_i, NULL, 10);
+
+ /* move past '.' */
+ decimal_i = ++i;
+
+ for (; '\0' != str[i]; i++) {
+ if (DAT_TRUE != dat_os_isdigit(str[i])) {
+ status = DAT_INTERNAL_ERROR;
+ goto exit;
+ }
+ }
+
+ /* if no version value was found */
+ if (decimal_i == i) {
+ status = DAT_INTERNAL_ERROR;
+ goto exit;
+ }
+
+ provider_version->version.minor = (DAT_UINT32)
+ dat_os_strtol(str + decimal_i, NULL, 10);
+
+ if ('\0' != str[i]) {
+ status = DAT_INTERNAL_ERROR;
+ goto exit;
+ }
+
+ exit:
+ if (DAT_SUCCESS != status) {
+ if (NULL != provider_version->id) {
+ dat_os_free(provider_version->id,
+ sizeof(char) *
+ (dat_os_strlen(provider_version->id) + 1));
+ provider_version->id = NULL;
+ }
+ }
+
+ return status;
+}
+
+/***********************************************************************
+ * Function: dat_sr_get_token
+ ***********************************************************************/
+
+DAT_RETURN dat_sr_get_token(DAT_OS_FILE * file, DAT_SR_TOKEN * token)
+{
+ if (NULL == g_token_stack) {
+ return dat_sr_read_token(file, token);
+ } else {
+ DAT_SR_STACK_NODE *top;
+
+ top = g_token_stack;
+
+ *token = top->token;
+ g_token_stack = top->next;
+
+ dat_os_free(top, sizeof(DAT_SR_STACK_NODE));
+
+ return DAT_SUCCESS;
+ }
+}
+
+/***********************************************************************
+ * Function: dat_sr_put_token
+ ***********************************************************************/
+
+DAT_RETURN dat_sr_put_token(DAT_OS_FILE * file, DAT_SR_TOKEN * token)
+{
+ DAT_SR_STACK_NODE *top;
+
+ if (NULL == (top = dat_os_alloc(sizeof(DAT_SR_STACK_NODE)))) {
+ return DAT_INSUFFICIENT_RESOURCES | DAT_RESOURCE_MEMORY;
+ }
+
+ top->token = *token;
+ top->next = g_token_stack;
+ g_token_stack = top;
+
+ return DAT_SUCCESS;
+}
+
+/***********************************************************************
+ * Function: dat_sr_read_token
+ ***********************************************************************/
+
+DAT_RETURN dat_sr_read_token(DAT_OS_FILE * file, DAT_SR_TOKEN * token)
+{
+ DAT_OS_FILE_POS pos;
+ DAT_OS_SIZE token_len;
+ DAT_COUNT num_escape_seq;
+ DAT_BOOLEAN is_quoted_str;
+ DAT_BOOLEAN is_prev_char_backslash;
+
+ /*
+ * The DAT standard does not specify a maximum size for quoted strings.
+ * Therefore the tokenizer must be able to read in a token of arbitrary
+ * size. Instead of allocating a fixed length buffer, the tokenizer first
+ * scans the input a single character at a time looking for the begining
+ * and end of the token. Once the these positions are found, the entire
+ * token is read into memory. By using this algorithm,the implementation
+ * does not place an arbitrary maximum on the token size.
+ */
+
+ token_len = 0;
+ num_escape_seq = 0;
+ is_quoted_str = DAT_FALSE;
+ is_prev_char_backslash = DAT_FALSE;
+
+ for (;;) {
+ DAT_OS_FILE_POS cur_pos;
+ int c;
+
+ /* if looking for start of the token */
+ if (0 == token_len) {
+ if (DAT_SUCCESS != dat_os_fgetpos(file, &cur_pos)) {
+ return DAT_INTERNAL_ERROR;
+ }
+ }
+
+ c = dat_os_fgetc(file);
+
+ /* if looking for start of the token */
+ if (0 == token_len) {
+ if (EOF == c) {
+ token->type = DAT_SR_TOKEN_EOF;
+ token->value = NULL;
+ token->value_len = 0;
+ goto success;
+ } else if (DAT_SR_CHAR_NEWLINE == c) {
+ token->type = DAT_SR_TOKEN_EOR;
+ token->value = NULL;
+ token->value_len = 0;
+ goto success;
+ } else if (dat_os_isblank(c)) {
+ continue;
+ } else if (DAT_SR_CHAR_COMMENT == c) {
+ dat_sr_read_comment(file);
+ continue;
+ } else {
+ if (DAT_SR_CHAR_QUOTE == c) {
+ is_quoted_str = DAT_TRUE;
+ }
+
+ pos = cur_pos;
+ token_len++;
+ }
+ } else { /* looking for the end of the token */
+
+ if (EOF == c) {
+ break;
+ } else if (DAT_SR_CHAR_NEWLINE == c) {
+ /* put back the newline */
+ dat_os_fputc(file, c);
+ break;
+ } else if (!is_quoted_str && dat_os_isblank(c)) {
+ break;
+ } else {
+ token_len++;
+
+ if ((DAT_SR_CHAR_QUOTE == c)
+ && !is_prev_char_backslash) {
+ break;
+ } else if ((DAT_SR_CHAR_BACKSLASH == c)
+ && !is_prev_char_backslash) {
+ is_prev_char_backslash = DAT_TRUE;
+ num_escape_seq++;
+ } else {
+ is_prev_char_backslash = DAT_FALSE;
+ }
+ }
+ }
+ }
+
+ /* the token was a string */
+ if (DAT_SUCCESS != dat_os_fsetpos(file, &pos)) {
+ return DAT_INTERNAL_ERROR;
+ }
+
+ if (is_quoted_str) {
+ if (DAT_SUCCESS != dat_sr_read_quoted_str(file,
+ token,
+ token_len,
+ num_escape_seq)) {
+ return DAT_INTERNAL_ERROR;
+ }
+ } else {
+ if (DAT_SUCCESS != dat_sr_read_str(file, token, token_len)) {
+ return DAT_INTERNAL_ERROR;
+ }
+ }
+
+ success:
+ dat_os_dbg_print(DAT_OS_DBG_TYPE_SR,
+ "\n"
+ "DAT Registry: token\n"
+ " type %s\n"
+ " value <%s>\n"
+ "\n",
+ dat_sr_type_to_str(token->type),
+ ((DAT_SR_TOKEN_STRING ==
+ token->type) ? token->value : ""));
+
+ return DAT_SUCCESS;
+}
+
+/***********************************************************************
+ * Function: dat_sr_read_str
+ ***********************************************************************/
+
+DAT_RETURN
+dat_sr_read_str(DAT_OS_FILE * file, DAT_SR_TOKEN * token, DAT_OS_SIZE token_len)
+{
+ token->type = DAT_SR_TOKEN_STRING;
+ token->value_len = sizeof(char) * (token_len + 1); /* +1 for null termination */
+ if (NULL == (token->value = dat_os_alloc(token->value_len))) {
+ return DAT_INSUFFICIENT_RESOURCES | DAT_RESOURCE_MEMORY;
+ }
+
+ if (token_len != dat_os_fread(file, token->value, token_len)) {
+ dat_os_free(token->value, token->value_len);
+ token->value = NULL;
+
+ return DAT_INTERNAL_ERROR;
+ }
+
+ token->value[token->value_len - 1] = '\0';
+
+ return DAT_SUCCESS;
+}
+
+/***********************************************************************
+ * Function: dat_sr_read_quoted_str
+ ***********************************************************************/
+
+DAT_RETURN
+dat_sr_read_quoted_str(DAT_OS_FILE * file,
+ DAT_SR_TOKEN * token,
+ DAT_OS_SIZE token_len, DAT_COUNT num_escape_seq)
+{
+ DAT_OS_SIZE str_len;
+ DAT_OS_SIZE i;
+ DAT_OS_SIZE j;
+ int c;
+ DAT_RETURN status;
+ DAT_BOOLEAN is_prev_char_backslash;
+
+ str_len = token_len - 2; /* minus 2 " characters */
+ is_prev_char_backslash = DAT_FALSE;
+ status = DAT_SUCCESS;
+
+ token->type = DAT_SR_TOKEN_STRING;
+ /* +1 for null termination */
+ token->value_len = sizeof(char) * (str_len - num_escape_seq + 1);
+
+ if (NULL == (token->value = dat_os_alloc(token->value_len))) {
+ status = DAT_INSUFFICIENT_RESOURCES | DAT_RESOURCE_MEMORY;
+ goto exit;
+ }
+
+ /* throw away " */
+ if (DAT_SR_CHAR_QUOTE != dat_os_fgetc(file)) {
+ status = DAT_INTERNAL_ERROR;
+ goto exit;
+ }
+
+ for (i = 0, j = 0; i < str_len; i++) {
+ c = dat_os_fgetc(file);
+
+ if (EOF == c) {
+ status = DAT_INTERNAL_ERROR;
+ goto exit;
+ } else if ((DAT_SR_CHAR_BACKSLASH == c)
+ && !is_prev_char_backslash) {
+ is_prev_char_backslash = DAT_TRUE;
+ } else {
+ token->value[j] = (char)c;
+ j++;
+
+ is_prev_char_backslash = DAT_FALSE;
+ }
+ }
+
+ /* throw away " */
+ if (DAT_SR_CHAR_QUOTE != dat_os_fgetc(file)) {
+ status = DAT_INTERNAL_ERROR;
+ goto exit;
+ }
+
+ token->value[token->value_len - 1] = '\0';
+
+ exit:
+ if (DAT_SUCCESS != status) {
+ if (NULL != token->value) {
+ dat_os_free(token->value, token->value_len);
+ token->value = NULL;
+ }
+ }
+
+ return status;
+}
+
+/***********************************************************************
+ * Function: dat_sr_read_comment
+ ***********************************************************************/
+
+void dat_sr_read_comment(DAT_OS_FILE * file)
+{
+ int c;
+
+ /* read up to an EOR or EOF to move past the comment */
+ do {
+ c = dat_os_fgetc(file);
+ } while ((DAT_SR_CHAR_NEWLINE != c) && (EOF != c));
+
+ /* put back the newline */
+ dat_os_ungetc(file, c);
+}
-/*\r
- * Copyright (c) 2005-2008 Intel Corporation. All rights reserved.\r
- *\r
- * This software is available to you under a choice of one of two\r
- * licenses. You may choose to be licensed under the terms of the GNU\r
- * General Public License (GPL) Version 2, available from the file\r
- * COPYING in the main directory of this source tree, or the\r
- * OpenIB.org BSD license below:\r
- *\r
- * Redistribution and use in source and binary forms, with or\r
- * without modification, are permitted provided that the following\r
- * conditions are met:\r
- *\r
- * - Redistributions of source code must retain the above\r
- * copyright notice, this list of conditions and the following\r
- * disclaimer.\r
- *\r
- * - Redistributions in binary form must reproduce the above\r
- * copyright notice, this list of conditions and the following\r
- * disclaimer in the documentation and/or other materials\r
- * provided with the distribution.\r
- *\r
- * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,\r
- * EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF\r
- * MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND\r
- * NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS\r
- * BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN\r
- * ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN\r
- * CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE\r
- * SOFTWARE.\r
- *\r
- * $Id: $\r
- */\r
-#include <stdio.h>\r
-#include <stdlib.h>\r
-#include <string.h>\r
-\r
-#ifdef DAPL_PROVIDER\r
-#undef DAPL_PROVIDER\r
-#endif\r
-\r
-#if defined(_WIN32) || defined(_WIN64)\r
-\r
-#include <windows.h>\r
-#include <winsock2.h>\r
-#include <ws2tcpip.h>\r
-#include <io.h>\r
-#include <process.h>\r
-#include <complib/cl_types.h>\r
-#include "..\..\..\..\etc\user\getopt.c"\r
-\r
-#define getpid() ((int)GetCurrentProcessId())\r
-#define F64x "%I64x"\r
-\r
-#ifdef DBG\r
-#define DAPL_PROVIDER "ibnic0v2d"\r
-#else\r
-#define DAPL_PROVIDER "ibnic0v2"\r
-#endif\r
-\r
-#define ntohll _byteswap_uint64\r
-#define htonll _byteswap_uint64\r
-\r
-#else // _WIN32 || _WIN64\r
-\r
-#include <endian.h>\r
-#include <byteswap.h>\r
-#include <netdb.h>\r
-#include <sys/types.h>\r
-#include <sys/socket.h>\r
-#include <sys/time.h>\r
-#include <netinet/in.h>\r
-#include <netinet/tcp.h>\r
-#include <arpa/inet.h>\r
-#include <sys/mman.h>\r
-#include <getopt.h>\r
-#include <inttypes.h>\r
-#include <unistd.h>\r
-#include <stdlib.h>\r
-\r
-#define DAPL_PROVIDER "ofa-v2-ib0"\r
-\r
-#define F64x "%"PRIx64""\r
-\r
-#if __BYTE_ORDER == __BIG_ENDIAN\r
-#define htonll(x) (x)\r
-#define ntohll(x) (x)\r
-#elif __BYTE_ORDER == __LITTLE_ENDIAN\r
-#define htonll(x) bswap_64(x)\r
-#define ntohll(x) bswap_64(x)\r
-#endif\r
-\r
-#endif // _WIN32 || _WIN64\r
-\r
-/* Debug: 1 == connect & close only, otherwise full-meal deal */\r
-#define CONNECT_ONLY 0\r
-\r
-#define MAX_POLLING_CNT 50000\r
-#define MAX_RDMA_RD 4\r
-#define MAX_PROCS 1000\r
-\r
-/* Header files needed for DAT/uDAPL */\r
-#include "dat2/udat.h"\r
-\r
-/* definitions */\r
-#define SERVER_CONN_QUAL 45248\r
-#define DTO_TIMEOUT (1000*1000*5)\r
-#define CNO_TIMEOUT (1000*1000*1)\r
-#define DTO_FLUSH_TIMEOUT (1000*1000*2)\r
-#define CONN_TIMEOUT (1000*1000*100)\r
-#define SERVER_TIMEOUT DAT_TIMEOUT_INFINITE\r
-#define RDMA_BUFFER_SIZE (64)\r
-\r
-/* Global DAT vars */\r
-static DAT_IA_HANDLE h_ia = DAT_HANDLE_NULL;\r
-static DAT_PZ_HANDLE h_pz = DAT_HANDLE_NULL;\r
-static DAT_EP_HANDLE h_ep = DAT_HANDLE_NULL;\r
-static DAT_PSP_HANDLE h_psp = DAT_HANDLE_NULL;\r
-static DAT_CR_HANDLE h_cr = DAT_HANDLE_NULL;\r
-\r
-static DAT_EVD_HANDLE h_async_evd = DAT_HANDLE_NULL;\r
-static DAT_EVD_HANDLE h_dto_req_evd = DAT_HANDLE_NULL;\r
-static DAT_EVD_HANDLE h_dto_rcv_evd = DAT_HANDLE_NULL;\r
-static DAT_EVD_HANDLE h_cr_evd = DAT_HANDLE_NULL;\r
-static DAT_EVD_HANDLE h_conn_evd = DAT_HANDLE_NULL;\r
-static DAT_CNO_HANDLE h_dto_cno = DAT_HANDLE_NULL;\r
-\r
-/* RDMA buffers */\r
-static DAT_LMR_HANDLE h_lmr_send = DAT_HANDLE_NULL;\r
-static DAT_LMR_HANDLE h_lmr_recv = DAT_HANDLE_NULL;\r
-static DAT_LMR_CONTEXT lmr_context_send;\r
-static DAT_LMR_CONTEXT lmr_context_recv;\r
-static DAT_RMR_CONTEXT rmr_context_send;\r
-static DAT_RMR_CONTEXT rmr_context_recv;\r
-static DAT_VLEN registered_size_send;\r
-static DAT_VLEN registered_size_recv;\r
-static DAT_VADDR registered_addr_send;\r
-static DAT_VADDR registered_addr_recv;\r
-\r
-/* Initial msg receive buf, RMR exchange, and Rdma-write notification */\r
-#define MSG_BUF_COUNT 3\r
-#define MSG_IOV_COUNT 2\r
-static DAT_RMR_TRIPLET rmr_recv_msg[MSG_BUF_COUNT];\r
-static DAT_LMR_HANDLE h_lmr_recv_msg = DAT_HANDLE_NULL;\r
-static DAT_LMR_CONTEXT lmr_context_recv_msg;\r
-static DAT_RMR_CONTEXT rmr_context_recv_msg;\r
-static DAT_VLEN registered_size_recv_msg;\r
-static DAT_VADDR registered_addr_recv_msg;\r
-\r
-/* message send buffer */\r
-static DAT_RMR_TRIPLET rmr_send_msg;\r
-static DAT_LMR_HANDLE h_lmr_send_msg = DAT_HANDLE_NULL;\r
-static DAT_LMR_CONTEXT lmr_context_send_msg;\r
-static DAT_RMR_CONTEXT rmr_context_send_msg;\r
-static DAT_VLEN registered_size_send_msg;\r
-static DAT_VADDR registered_addr_send_msg;\r
-static DAT_EP_ATTR ep_attr;\r
-char hostname[256] = { 0 };\r
-char provider[64] = DAPL_PROVIDER;\r
-char addr_str[INET_ADDRSTRLEN];\r
-\r
-/* rdma pointers */\r
-char *rbuf = NULL;\r
-char *sbuf = NULL;\r
-int status;\r
-\r
-/* timers */\r
-double start, stop, total_us, total_sec;\r
-\r
-struct dt_time {\r
- double total;\r
- double open;\r
- double reg;\r
- double unreg;\r
- double pzc;\r
- double pzf;\r
- double evdc;\r
- double evdf;\r
- double cnoc;\r
- double cnof;\r
- double epc;\r
- double epf;\r
- double rdma_wr;\r
- double rdma_rd[MAX_RDMA_RD];\r
- double rdma_rd_total;\r
- double rtt;\r
- double close;\r
- double conn;\r
-};\r
-\r
-struct dt_time ts;\r
-\r
-/* defaults */\r
-static int failed = 0;\r
-static int performance_times = 0;\r
-static int connected = 0;\r
-static int burst = 10;\r
-static int server = 1;\r
-static int verbose = 0;\r
-static int polling = 0;\r
-static int poll_count = 0;\r
-static int rdma_wr_poll_count = 0;\r
-static int conn_poll_count = 0;\r
-static int rdma_rd_poll_count[MAX_RDMA_RD] = { 0 };\r
-static int delay = 0;\r
-static int buf_len = RDMA_BUFFER_SIZE;\r
-static int use_cno = 0;\r
-static int recv_msg_index = 0;\r
-static int burst_msg_posted = 0;\r
-static int burst_msg_index = 0;\r
-static int ucm = 0;\r
-static DAT_SOCK_ADDR6 remote;\r
-\r
-/* forward prototypes */\r
-const char *DT_RetToStr(DAT_RETURN ret_value);\r
-const char *DT_EventToStr(DAT_EVENT_NUMBER event_code);\r
-void print_usage(void);\r
-double get_time(void);\r
-void init_data(void);\r
-\r
-DAT_RETURN send_msg(void *data,\r
- DAT_COUNT size,\r
- DAT_LMR_CONTEXT context,\r
- DAT_DTO_COOKIE cookie, DAT_COMPLETION_FLAGS flags);\r
-\r
-DAT_RETURN connect_ep(char *hostname, DAT_CONN_QUAL conn_id);\r
-void disconnect_ep(void);\r
-DAT_RETURN register_rdma_memory(void);\r
-DAT_RETURN unregister_rdma_memory(void);\r
-DAT_RETURN create_events(void);\r
-DAT_RETURN destroy_events(void);\r
-DAT_RETURN do_rdma_write_with_msg(void);\r
-DAT_RETURN do_rdma_read_with_msg(void);\r
-DAT_RETURN do_ping_pong_msg(void);\r
-\r
-#define LOGPRINTF if (verbose) printf\r
-\r
-void flush_evds(void)\r
-{\r
- DAT_EVENT event;\r
-\r
- /* Flush async error queue */\r
- printf("%d ERR: Checking ASYNC EVD...\n", getpid());\r
- while (dat_evd_dequeue(h_async_evd, &event) == DAT_SUCCESS) {\r
- printf(" ASYNC EVD ENTRY: handle=%p reason=%d\n",\r
- event.event_data.asynch_error_event_data.dat_handle,\r
- event.event_data.asynch_error_event_data.reason);\r
- }\r
- /* Flush receive queue */\r
- printf("%d ERR: Checking RECEIVE EVD...\n", getpid());\r
- while (dat_evd_dequeue(h_dto_rcv_evd, &event) == DAT_SUCCESS) {\r
- printf(" RCV EVD ENTRY: op=%d stat=%d ln=%d ck="F64x"\n",\r
- event.event_data.dto_completion_event_data.operation,\r
- event.event_data.dto_completion_event_data.status,\r
- event.event_data.dto_completion_event_data.transfered_length,\r
- event.event_data.dto_completion_event_data.user_cookie.as_64);\r
- }\r
- /* Flush request queue */\r
- printf("%d ERR: Checking REQUEST EVD...\n", getpid());\r
- while (dat_evd_dequeue(h_dto_req_evd, &event) == DAT_SUCCESS) {\r
- printf(" REQ EVD ENTRY: op=%d stat=%d ln=%d ck="F64x"\n",\r
- event.event_data.dto_completion_event_data.operation,\r
- event.event_data.dto_completion_event_data.status,\r
- event.event_data.dto_completion_event_data.transfered_length,\r
- event.event_data.dto_completion_event_data.user_cookie.as_64);\r
- }\r
-}\r
-\r
-\r
-static inline DAT_RETURN\r
-collect_event(DAT_EVD_HANDLE dto_evd,\r
- DAT_EVENT *event,\r
- DAT_TIMEOUT timeout,\r
- int *counter)\r
-{\r
- DAT_EVD_HANDLE evd = DAT_HANDLE_NULL;\r
- DAT_COUNT nmore;\r
- DAT_RETURN ret = DAT_SUCCESS;\r
-\r
- if (use_cno) {\r
-retry:\r
- /* CNO wait could return EVD's in any order and\r
- * may drop some EVD notification's if already\r
- * triggered. Once woken, simply dequeue the \r
- * Evd the caller wants to collect and return.\r
- * If notification without EVD, retry.\r
- */\r
- ret = dat_cno_wait(h_dto_cno, CNO_TIMEOUT, &evd);\r
- if (dat_evd_dequeue(dto_evd, event) != DAT_SUCCESS) {\r
- if (ret == DAT_SUCCESS)\r
- printf(" WARNING: CNO notification:"\r
- " without EVD?\n");\r
- goto retry;\r
- }\r
- ret = DAT_SUCCESS; /* cno timed out, but EVD dequeued */\r
- \r
- } else if (!polling) {\r
-\r
- /* use wait to dequeue */\r
- ret = dat_evd_wait(dto_evd, timeout, 1, event, &nmore);\r
- if (ret != DAT_SUCCESS)\r
- fprintf(stderr,\r
- "Error waiting on h_dto_evd %p: %s\n",\r
- dto_evd, DT_RetToStr(ret));\r
- \r
- } else {\r
- while (dat_evd_dequeue(dto_evd, event) == DAT_QUEUE_EMPTY)\r
- if (counter)\r
- (*counter)++;\r
- }\r
- return (ret);\r
-}\r
-\r
-static void print_ia_address(struct sockaddr *sa)\r
-{\r
- char str[INET6_ADDRSTRLEN] = {" ??? "};\r
-\r
- switch(sa->sa_family) {\r
- case AF_INET:\r
- inet_ntop(AF_INET, &((struct sockaddr_in *)sa)->sin_addr, str, INET6_ADDRSTRLEN);\r
- printf("%d Local Address AF_INET - %s port %d\n", getpid(), str, SERVER_CONN_QUAL);\r
- break;\r
- case AF_INET6:\r
- inet_ntop(AF_INET6, &((struct sockaddr_in6 *)sa)->sin6_addr, str, INET6_ADDRSTRLEN);\r
- printf("%d Local Address AF_INET6 - %s flowinfo(QPN)=0x%x, port(LID)=0x%x\n",\r
- getpid(), str, \r
- ntohl(((struct sockaddr_in6 *)sa)->sin6_flowinfo),\r
- ntohs(((struct sockaddr_in6 *)sa)->sin6_port));\r
- break;\r
- default:\r
- printf("%d Local Address UNKOWN FAMILY - port %d\n", getpid(), SERVER_CONN_QUAL);\r
- }\r
-}\r
-\r
-int main(int argc, char **argv)\r
-{\r
- int i, c;\r
- DAT_RETURN ret;\r
- DAT_EP_PARAM ep_param;\r
- DAT_IA_ATTR ia_attr;\r
-\r
- /* parse arguments */\r
- while ((c = getopt(argc, argv, "tscvpq:l:b:d:B:h:P:")) != -1) {\r
- switch (c) {\r
- case 't':\r
- performance_times = 1;\r
- fflush(stdout);\r
- break;\r
- case 's':\r
- server = 1;\r
- fflush(stdout);\r
- break;\r
- case 'c':\r
- use_cno = 1;\r
- printf("%d Creating CNO for DTO EVD's\n", getpid());\r
- fflush(stdout);\r
- break;\r
- case 'v':\r
- verbose = 1;\r
- printf("%d Verbose\n", getpid());\r
- fflush(stdout);\r
- break;\r
- case 'p':\r
- polling = 1;\r
- printf("%d Polling\n", getpid());\r
- fflush(stdout);\r
- break;\r
- case 'q':\r
- /* map UCM qpn into AF_INET6 sin6_flowinfo */\r
- remote.sin6_family = AF_INET6;\r
- remote.sin6_flowinfo = htonl(strtol(optarg,NULL,0));\r
- ucm = 1;\r
- server = 0;\r
- break;\r
- case 'l':\r
- /* map UCM lid into AF_INET6 sin6_port */\r
- remote.sin6_family = AF_INET6;\r
- remote.sin6_port = htons(strtol(optarg,NULL,0));\r
- ucm = 1;\r
- server = 0;\r
- break;\r
- case 'B':\r
- burst = atoi(optarg);\r
- break;\r
- case 'd':\r
- delay = atoi(optarg);\r
- break;\r
- case 'b':\r
- buf_len = atoi(optarg);\r
- break;\r
- case 'h':\r
- server = 0;\r
- strcpy(hostname, optarg);\r
- break;\r
- case 'P':\r
- strcpy(provider, optarg);\r
- break;\r
- default:\r
- print_usage();\r
- exit(-12);\r
- }\r
- }\r
-\r
-#if defined(_WIN32) || defined(_WIN64)\r
- {\r
- WSADATA wsaData;\r
-\r
- i = WSAStartup(MAKEWORD(2, 2), &wsaData);\r
- if (i != 0) {\r
- printf("%s WSAStartup(2.2) failed? (0x%x)\n", argv[0],\r
- i);\r
- fflush(stdout);\r
- exit(1);\r
- }\r
- }\r
-#endif\r
-\r
- if (!server) {\r
- printf("%d Running as client - %s\n", getpid(), provider);\r
- } else {\r
- printf("%d Running as server - %s\n", getpid(), provider);\r
- }\r
- fflush(stdout);\r
-\r
- /* allocate send and receive buffers */\r
- if (((rbuf = malloc(buf_len * (burst+1))) == NULL) ||\r
- ((sbuf = malloc(buf_len * (burst+1))) == NULL)) {\r
- perror("malloc");\r
- exit(1);\r
- }\r
- memset(&ts, 0, sizeof(struct dt_time));\r
- LOGPRINTF("%d Allocated RDMA buffers (r:%p,s:%p) len %d \n",\r
- getpid(), rbuf, sbuf, buf_len);\r
-\r
- /* dat_ia_open, dat_pz_create */\r
- h_async_evd = DAT_HANDLE_NULL;\r
- start = get_time();\r
- ret = dat_ia_open(provider, 8, &h_async_evd, &h_ia);\r
- stop = get_time();\r
- ts.open += ((stop - start) * 1.0e6);\r
- if (ret != DAT_SUCCESS) {\r
- fprintf(stderr, "%d: Error Adaptor open: %s\n",\r
- getpid(), DT_RetToStr(ret));\r
- exit(1);\r
- } else\r
- LOGPRINTF("%d Opened Interface Adaptor\n", getpid());\r
-\r
- ret = dat_ia_query(h_ia, 0, DAT_IA_FIELD_ALL, &ia_attr, 0, 0);\r
- if (ret != DAT_SUCCESS) {\r
- fprintf(stderr, "%d: Error Adaptor query: %s\n",\r
- getpid(), DT_RetToStr(ret));\r
- exit(1);\r
- }\r
- print_ia_address(ia_attr.ia_address_ptr);\r
-\r
- /* Create Protection Zone */\r
- start = get_time();\r
- LOGPRINTF("%d Create Protection Zone\n", getpid());\r
- ret = dat_pz_create(h_ia, &h_pz);\r
- stop = get_time();\r
- ts.pzc += ((stop - start) * 1.0e6);\r
- if (ret != DAT_SUCCESS) {\r
- fprintf(stderr, "%d Error creating Protection Zone: %s\n",\r
- getpid(), DT_RetToStr(ret));\r
- exit(1);\r
- } else\r
- LOGPRINTF("%d Created Protection Zone\n", getpid());\r
-\r
- /* Register memory */\r
- LOGPRINTF("%d Register RDMA memory\n", getpid());\r
- ret = register_rdma_memory();\r
- if (ret != DAT_SUCCESS) {\r
- fprintf(stderr, "%d Error registering RDMA memory: %s\n",\r
- getpid(), DT_RetToStr(ret));\r
- goto cleanup;\r
- } else\r
- LOGPRINTF("%d Register RDMA memory done\n", getpid());\r
-\r
- LOGPRINTF("%d Create events\n", getpid());\r
- ret = create_events();\r
- if (ret != DAT_SUCCESS) {\r
- fprintf(stderr, "%d Error creating events: %s\n",\r
- getpid(), DT_RetToStr(ret));\r
- goto cleanup;\r
- } else {\r
- LOGPRINTF("%d Create events done\n", getpid());\r
- }\r
-\r
- /* create EP */\r
- memset(&ep_attr, 0, sizeof(ep_attr));\r
- ep_attr.service_type = DAT_SERVICE_TYPE_RC;\r
- ep_attr.max_rdma_size = 0x10000;\r
- ep_attr.qos = 0;\r
- ep_attr.recv_completion_flags = 0;\r
- ep_attr.max_recv_dtos = MSG_BUF_COUNT + (burst * 3);\r
- ep_attr.max_request_dtos = MSG_BUF_COUNT + (burst * 3) + MAX_RDMA_RD;\r
- ep_attr.max_recv_iov = MSG_IOV_COUNT;\r
- ep_attr.max_request_iov = MSG_IOV_COUNT;\r
- ep_attr.max_rdma_read_in = MAX_RDMA_RD;\r
- ep_attr.max_rdma_read_out = MAX_RDMA_RD;\r
- ep_attr.request_completion_flags = DAT_COMPLETION_DEFAULT_FLAG;\r
- ep_attr.ep_transport_specific_count = 0;\r
- ep_attr.ep_transport_specific = NULL;\r
- ep_attr.ep_provider_specific_count = 0;\r
- ep_attr.ep_provider_specific = NULL;\r
-\r
- start = get_time();\r
- ret = dat_ep_create(h_ia, h_pz, h_dto_rcv_evd,\r
- h_dto_req_evd, h_conn_evd, &ep_attr, &h_ep);\r
- stop = get_time();\r
- ts.epc += ((stop - start) * 1.0e6);\r
- ts.total += ts.epc;\r
- if (ret != DAT_SUCCESS) {\r
- fprintf(stderr, "%d Error dat_ep_create: %s\n",\r
- getpid(), DT_RetToStr(ret));\r
- goto cleanup;\r
- } else\r
- LOGPRINTF("%d EP created %p \n", getpid(), h_ep);\r
-\r
- /*\r
- * register message buffers, establish connection, and\r
- * exchange DMA RMR information info via messages\r
- */\r
- ret = connect_ep(hostname, SERVER_CONN_QUAL);\r
- if (ret != DAT_SUCCESS) {\r
- fprintf(stderr, "%d Error connect_ep: %s\n",\r
- getpid(), DT_RetToStr(ret));\r
- goto cleanup;\r
- } else\r
- LOGPRINTF("%d connect_ep complete\n", getpid());\r
-\r
- /* Query EP for local and remote address information, print */\r
- ret = dat_ep_query(h_ep, DAT_EP_FIELD_ALL, &ep_param);\r
- if (ret != DAT_SUCCESS) {\r
- fprintf(stderr, "%d Error dat_ep_query: %s\n",\r
- getpid(), DT_RetToStr(ret));\r
- goto cleanup;\r
- } else\r
- LOGPRINTF("%d EP queried %p \n", getpid(), h_ep);\r
-#if defined(_WIN32)\r
- printf("\n%d Query EP: LOCAL addr %s port %lld\n", getpid(),\r
- inet_ntoa(((struct sockaddr_in *)\r
- ep_param.local_ia_address_ptr)->sin_addr),\r
- (ep_param.local_port_qual));\r
-#else\r
- inet_ntop(AF_INET,\r
- &((struct sockaddr_in *)ep_param.local_ia_address_ptr)->\r
- sin_addr, addr_str, sizeof(addr_str));\r
- printf("\n%d Query EP: LOCAL addr %s port " F64x "\n", getpid(),\r
- addr_str, (ep_param.local_port_qual));\r
-#endif\r
-#if defined(_WIN32)\r
- printf("%d Query EP: REMOTE addr %s port %lld\n", getpid(),\r
- inet_ntoa(((struct sockaddr_in *)\r
- ep_param.local_ia_address_ptr)->sin_addr),\r
- (ep_param.remote_port_qual));\r
-#else\r
- inet_ntop(AF_INET,\r
- &((struct sockaddr_in *)ep_param.remote_ia_address_ptr)->\r
- sin_addr, addr_str, sizeof(addr_str));\r
- printf("%d Query EP: REMOTE addr %s port " F64x "\n", getpid(),\r
- addr_str, (ep_param.remote_port_qual));\r
-#endif\r
- fflush(stdout);\r
-\r
-#if CONNECT_ONLY\r
-#if defined(_WIN32) || defined(_WIN64)\r
- Sleep(1 * 1000);\r
-#else\r
- sleep(1);\r
-#endif\r
- goto cleanup;\r
-#endif\r
-\r
- /*********** RDMA write data *************/\r
- ret = do_rdma_write_with_msg();\r
- if (ret != DAT_SUCCESS) {\r
- fprintf(stderr, "%d Error do_rdma_write_with_msg: %s\n",\r
- getpid(), DT_RetToStr(ret));\r
- goto cleanup;\r
- } else\r
- LOGPRINTF("%d do_rdma_write_with_msg complete\n", getpid());\r
-\r
- /*********** RDMA read data *************/\r
- ret = do_rdma_read_with_msg();\r
- if (ret != DAT_SUCCESS) {\r
- fprintf(stderr, "%d Error do_rdma_read_with_msg: %s\n",\r
- getpid(), DT_RetToStr(ret));\r
- goto cleanup;\r
- } else\r
- LOGPRINTF("%d do_rdma_read_with_msg complete\n", getpid());\r
-\r
- /*********** PING PING messages ************/\r
- ret = do_ping_pong_msg();\r
- if (ret != DAT_SUCCESS) {\r
- fprintf(stderr, "%d Error do_ping_pong_msg: %s\n",\r
- getpid(), DT_RetToStr(ret));\r
- goto cleanup;\r
- } else {\r
- LOGPRINTF("%d do_ping_pong_msg complete\n", getpid());\r
- goto complete;\r
- }\r
-\r
-cleanup:\r
- flush_evds();\r
- failed++;\r
-complete:\r
-\r
- /* disconnect and free EP resources */\r
- if (h_ep != DAT_HANDLE_NULL) {\r
- /* unregister message buffers and tear down connection */\r
- LOGPRINTF("%d Disconnect and Free EP %p \n", getpid(), h_ep);\r
- disconnect_ep();\r
-\r
- /* free EP */\r
- LOGPRINTF("%d Free EP %p \n", getpid(), h_ep);\r
- start = get_time();\r
- ret = dat_ep_free(h_ep);\r
- stop = get_time();\r
- ts.epf += ((stop - start) * 1.0e6);\r
- ts.total += ts.epf;\r
- if (ret != DAT_SUCCESS) {\r
- fprintf(stderr, "%d Error freeing EP: %s\n",\r
- getpid(), DT_RetToStr(ret));\r
- } else {\r
- LOGPRINTF("%d Freed EP\n", getpid());\r
- h_ep = DAT_HANDLE_NULL;\r
- }\r
- }\r
-\r
- /* free EVDs */\r
- LOGPRINTF("%d destroy events\n", getpid());\r
- ret = destroy_events();\r
- if (ret != DAT_SUCCESS)\r
- fprintf(stderr, "%d Error destroy_events: %s\n",\r
- getpid(), DT_RetToStr(ret));\r
- else\r
- LOGPRINTF("%d destroy events done\n", getpid());\r
-\r
- ret = unregister_rdma_memory();\r
- LOGPRINTF("%d unregister_rdma_memory \n", getpid());\r
- if (ret != DAT_SUCCESS)\r
- fprintf(stderr, "%d Error unregister_rdma_memory: %s\n",\r
- getpid(), DT_RetToStr(ret));\r
- else\r
- LOGPRINTF("%d unregister_rdma_memory done\n", getpid());\r
-\r
- /* Free protection domain */\r
- LOGPRINTF("%d Freeing pz\n", getpid());\r
- start = get_time();\r
- ret = dat_pz_free(h_pz);\r
- stop = get_time();\r
- ts.pzf += ((stop - start) * 1.0e6);\r
- if (ret != DAT_SUCCESS) {\r
- fprintf(stderr, "%d Error freeing PZ: %s\n",\r
- getpid(), DT_RetToStr(ret));\r
- } else {\r
- LOGPRINTF("%d Freed pz\n", getpid());\r
- h_pz = NULL;\r
- }\r
-\r
- /* close the device */\r
- LOGPRINTF("%d Closing Interface Adaptor\n", getpid());\r
- start = get_time();\r
- ret = dat_ia_close(h_ia, DAT_CLOSE_ABRUPT_FLAG);\r
- stop = get_time();\r
- ts.close += ((stop - start) * 1.0e6);\r
- if (ret != DAT_SUCCESS) {\r
- fprintf(stderr, "%d: Error Adaptor close: %s\n",\r
- getpid(), DT_RetToStr(ret));\r
- } else\r
- LOGPRINTF("%d Closed Interface Adaptor\n", getpid());\r
-\r
- /* free rdma buffers */\r
- free(rbuf);\r
- free(sbuf);\r
-\r
- printf("\n%d: DAPL Test Complete. %s\n\n",\r
- getpid(), failed ? "FAILED" : "PASSED");\r
-\r
- fflush(stderr);\r
- fflush(stdout);\r
-\r
- if (!performance_times)\r
- exit(0);\r
-\r
- printf("\n%d: DAPL Test Complete.\n\n", getpid());\r
- printf("%d: Message RTT: Total=%10.2lf usec, %d bursts, itime=%10.2lf"\r
- " usec, pc=%d\n",\r
- getpid(), ts.rtt, burst, ts.rtt / burst, poll_count);\r
- printf("%d: RDMA write: Total=%10.2lf usec, %d bursts, itime=%10.2lf"\r
- " usec, pc=%d\n",\r
- getpid(), ts.rdma_wr, burst,\r
- ts.rdma_wr / burst, rdma_wr_poll_count);\r
- for (i = 0; i < MAX_RDMA_RD; i++) {\r
- printf("%d: RDMA read: Total=%10.2lf usec, %d bursts, "\r
- "itime=%10.2lf usec, pc=%d\n",\r
- getpid(), ts.rdma_rd_total, MAX_RDMA_RD,\r
- ts.rdma_rd[i], rdma_rd_poll_count[i]);\r
- }\r
- printf("%d: open: %10.2lf usec\n", getpid(), ts.open);\r
- printf("%d: close: %10.2lf usec\n", getpid(), ts.close);\r
- printf("%d: PZ create: %10.2lf usec\n", getpid(), ts.pzc);\r
- printf("%d: PZ free: %10.2lf usec\n", getpid(), ts.pzf);\r
- printf("%d: LMR create:%10.2lf usec\n", getpid(), ts.reg);\r
- printf("%d: LMR free: %10.2lf usec\n", getpid(), ts.unreg);\r
- printf("%d: EVD create:%10.2lf usec\n", getpid(), ts.evdc);\r
- printf("%d: EVD free: %10.2lf usec\n", getpid(), ts.evdf);\r
- if (use_cno) {\r
- printf("%d: CNO create: %10.2lf usec\n", getpid(), ts.cnoc);\r
- printf("%d: CNO free: %10.2lf usec\n", getpid(), ts.cnof);\r
- }\r
- printf("%d: EP create: %10.2lf usec\n", getpid(), ts.epc);\r
- printf("%d: EP free: %10.2lf usec\n", getpid(), ts.epf);\r
- if (!server)\r
- printf("%d: connect: %10.2lf usec, poll_cnt=%d\n", \r
- getpid(), ts.conn, conn_poll_count);\r
- printf("%d: TOTAL: %10.2lf usec\n", getpid(), ts.total);\r
-\r
-#if defined(_WIN32) || defined(_WIN64)\r
- WSACleanup();\r
-#endif\r
- return (0);\r
-}\r
-\r
-double get_time(void)\r
-{\r
- struct timeval tp;\r
-\r
- gettimeofday(&tp, NULL);\r
- return ((double)tp.tv_sec + (double)tp.tv_usec * 1e-6);\r
-}\r
-\r
-void init_data(void)\r
-{\r
- memset(rbuf, 'a', buf_len);\r
- memset(sbuf, 'b', buf_len);\r
-}\r
-\r
-DAT_RETURN\r
-send_msg(void *data,\r
- DAT_COUNT size,\r
- DAT_LMR_CONTEXT context,\r
- DAT_DTO_COOKIE cookie, DAT_COMPLETION_FLAGS flags)\r
-{\r
- DAT_LMR_TRIPLET iov;\r
- DAT_EVENT event;\r
- DAT_RETURN ret;\r
-\r
- iov.lmr_context = context;\r
-#if defined(_WIN32)\r
- iov.virtual_address = (DAT_VADDR) data;\r
-#else\r
- iov.virtual_address = (DAT_VADDR) (unsigned long)data;\r
-#endif\r
- iov.segment_length = size;\r
-\r
- LOGPRINTF("%d calling post_send\n", getpid());\r
- cookie.as_64 = 0xaaaa;\r
- ret = dat_ep_post_send(h_ep, 1, &iov, cookie, flags);\r
-\r
- if (ret != DAT_SUCCESS) {\r
- fprintf(stderr, "%d: ERROR: dat_ep_post_send() %s\n",\r
- getpid(), DT_RetToStr(ret));\r
- return ret;\r
- }\r
-\r
- if (!(flags & DAT_COMPLETION_SUPPRESS_FLAG)) {\r
- \r
- if (collect_event(h_dto_req_evd, \r
- &event, \r
- DTO_TIMEOUT, \r
- &poll_count) != DAT_SUCCESS)\r
- return (DAT_ABORT);\r
-\r
- /* validate event number, len, cookie, and status */\r
- if (event.event_number != DAT_DTO_COMPLETION_EVENT) {\r
- fprintf(stderr, "%d: ERROR: DTO event number %s\n",\r
- getpid(), \r
- DT_EventToStr(event.event_number));\r
- return (DAT_ABORT);\r
- }\r
-\r
- if ((event.event_data.dto_completion_event_data.\r
- transfered_length != size)\r
- || (event.event_data.dto_completion_event_data.user_cookie.\r
- as_64 != 0xaaaa)) {\r
- fprintf(stderr,\r
- "%d: ERROR: DTO len %d or cookie " F64x " \n",\r
- getpid(),\r
- event.event_data.dto_completion_event_data.\r
- transfered_length,\r
- event.event_data.dto_completion_event_data.\r
- user_cookie.as_64);\r
- return (DAT_ABORT);\r
-\r
- }\r
- if (event.event_data.dto_completion_event_data.status !=\r
- DAT_SUCCESS) {\r
- fprintf(stderr, "%d: ERROR: DTO event status %s\n",\r
- getpid(), DT_RetToStr(ret));\r
- return (DAT_ABORT);\r
- }\r
- }\r
-\r
- return DAT_SUCCESS;\r
-}\r
-\r
-DAT_RETURN connect_ep(char *hostname, DAT_CONN_QUAL conn_id)\r
-{\r
- DAT_IA_ADDRESS_PTR remote_addr = (DAT_IA_ADDRESS_PTR)&remote;\r
- DAT_RETURN ret;\r
- DAT_REGION_DESCRIPTION region;\r
- DAT_EVENT event;\r
- DAT_COUNT nmore;\r
- DAT_LMR_TRIPLET l_iov;\r
- DAT_RMR_TRIPLET r_iov;\r
- DAT_DTO_COOKIE cookie;\r
- int i;\r
- unsigned char *buf;\r
- DAT_CR_PARAM cr_param = { 0 };\r
- unsigned char pdata[48] = { 0 };\r
-\r
- /* Register send message buffer */\r
- LOGPRINTF("%d Registering send Message Buffer %p, len %d\n",\r
- getpid(), &rmr_send_msg, (int)sizeof(DAT_RMR_TRIPLET));\r
- region.for_va = &rmr_send_msg;\r
- ret = dat_lmr_create(h_ia,\r
- DAT_MEM_TYPE_VIRTUAL,\r
- region,\r
- sizeof(DAT_RMR_TRIPLET),\r
- h_pz,\r
- DAT_MEM_PRIV_LOCAL_WRITE_FLAG,\r
- DAT_VA_TYPE_VA,\r
- &h_lmr_send_msg,\r
- &lmr_context_send_msg,\r
- &rmr_context_send_msg,\r
- ®istered_size_send_msg,\r
- ®istered_addr_send_msg);\r
-\r
- if (ret != DAT_SUCCESS) {\r
- fprintf(stderr, "%d Error registering send msg buffer: %s\n",\r
- getpid(), DT_RetToStr(ret));\r
- return (ret);\r
- } else\r
- LOGPRINTF("%d Registered send Message Buffer %p \n",\r
- getpid(), region.for_va);\r
-\r
- /* Register Receive buffers */\r
- LOGPRINTF("%d Registering Receive Message Buffer %p\n",\r
- getpid(), rmr_recv_msg);\r
- region.for_va = rmr_recv_msg;\r
- ret = dat_lmr_create(h_ia,\r
- DAT_MEM_TYPE_VIRTUAL,\r
- region,\r
- sizeof(DAT_RMR_TRIPLET) * MSG_BUF_COUNT,\r
- h_pz,\r
- DAT_MEM_PRIV_LOCAL_WRITE_FLAG,\r
- DAT_VA_TYPE_VA,\r
- &h_lmr_recv_msg,\r
- &lmr_context_recv_msg,\r
- &rmr_context_recv_msg,\r
- ®istered_size_recv_msg,\r
- ®istered_addr_recv_msg);\r
- if (ret != DAT_SUCCESS) {\r
- fprintf(stderr, "%d Error registering recv msg buffer: %s\n",\r
- getpid(), DT_RetToStr(ret));\r
- return (ret);\r
- } else\r
- LOGPRINTF("%d Registered Receive Message Buffer %p\n",\r
- getpid(), region.for_va);\r
-\r
- for (i = 0; i < MSG_BUF_COUNT; i++) {\r
- cookie.as_64 = i;\r
- l_iov.lmr_context = lmr_context_recv_msg;\r
-#if defined(_WIN32)\r
- l_iov.virtual_address = (DAT_VADDR) & rmr_recv_msg[i];\r
-#else\r
- l_iov.virtual_address =\r
- (DAT_VADDR) (unsigned long)&rmr_recv_msg[i];\r
-#endif\r
- l_iov.segment_length = sizeof(DAT_RMR_TRIPLET);\r
-\r
- LOGPRINTF("%d Posting Receive Message Buffer %p\n",\r
- getpid(), &rmr_recv_msg[i]);\r
- ret = dat_ep_post_recv(h_ep,\r
- 1,\r
- &l_iov,\r
- cookie, DAT_COMPLETION_DEFAULT_FLAG);\r
-\r
- if (ret != DAT_SUCCESS) {\r
- fprintf(stderr,\r
- "%d Error registering recv msg buffer: %s\n",\r
- getpid(), DT_RetToStr(ret));\r
- return (ret);\r
- } else\r
- LOGPRINTF("%d Registered Receive Message Buffer %p\n",\r
- getpid(), region.for_va);\r
-\r
- }\r
-\r
- /* setup receive rdma buffer to initial string to be overwritten */\r
- strcpy((char *)rbuf, "blah, blah, blah\n");\r
-\r
- /* clear event structure */\r
- memset(&event, 0, sizeof(DAT_EVENT));\r
-\r
- if (server) { /* SERVER */\r
-\r
- /* create the service point for server listen */\r
- LOGPRINTF("%d Creating service point for listen\n", getpid());\r
- ret = dat_psp_create(h_ia,\r
- conn_id,\r
- h_cr_evd, DAT_PSP_CONSUMER_FLAG, &h_psp);\r
- if (ret != DAT_SUCCESS) {\r
- fprintf(stderr, "%d Error dat_psp_create: %s\n",\r
- getpid(), DT_RetToStr(ret));\r
- return (ret);\r
- } else\r
- LOGPRINTF("%d dat_psp_created for server listen\n",\r
- getpid());\r
-\r
- printf("%d Server waiting for connect request on port " F64x\r
- "\n", getpid(), conn_id);\r
-\r
- ret = dat_evd_wait(h_cr_evd, SERVER_TIMEOUT, 1, &event, &nmore);\r
- if (ret != DAT_SUCCESS) {\r
- fprintf(stderr, "%d Error dat_evd_wait: %s\n",\r
- getpid(), DT_RetToStr(ret));\r
- return (ret);\r
- } else\r
- LOGPRINTF("%d dat_evd_wait for cr_evd completed\n",\r
- getpid());\r
-\r
- if (event.event_number != DAT_CONNECTION_REQUEST_EVENT) {\r
- fprintf(stderr, "%d Error unexpected cr event : %s\n",\r
- getpid(), \r
- DT_EventToStr(event.event_number));\r
- return (DAT_ABORT);\r
- }\r
- if ((event.event_data.cr_arrival_event_data.conn_qual !=\r
- SERVER_CONN_QUAL)\r
- || (event.event_data.cr_arrival_event_data.sp_handle.\r
- psp_handle != h_psp)) {\r
- fprintf(stderr, "%d Error wrong cr event data : %s\n",\r
- getpid(), \r
- DT_EventToStr(event.event_number));\r
- return (DAT_ABORT);\r
- }\r
-\r
- /* use to test rdma_cma timeout logic */\r
-#if defined(_WIN32) || defined(_WIN64)\r
- if (delay)\r
- Sleep(delay * 1000);\r
-#else\r
- if (delay)\r
- sleep(delay);\r
-#endif\r
-\r
- /* accept connect request from client */\r
- h_cr = event.event_data.cr_arrival_event_data.cr_handle;\r
- LOGPRINTF("%d Accepting connect request from client\n",\r
- getpid());\r
-\r
- /* private data - check and send it back */\r
- dat_cr_query(h_cr, DAT_CSP_FIELD_ALL, &cr_param);\r
-\r
- buf = (unsigned char *)cr_param.private_data;\r
- LOGPRINTF("%d CONN REQUEST Private Data %p[0]=%d [47]=%d\n",\r
- getpid(), buf, buf[0], buf[47]);\r
- for (i = 0; i < 48; i++) {\r
- if (buf[i] != i + 1) {\r
- fprintf(stderr, "%d Error with CONNECT REQUEST"\r
- " private data: %p[%d]=%d s/be %d\n",\r
- getpid(), buf, i, buf[i], i + 1);\r
- dat_cr_reject(h_cr, 0, NULL);\r
- return (DAT_ABORT);\r
- }\r
- buf[i]++; /* change for trip back */\r
- }\r
-\r
-#ifdef TEST_REJECT_WITH_PRIVATE_DATA\r
- printf("%d REJECT request with 48 bytes of private data\n",\r
- getpid());\r
- ret = dat_cr_reject(h_cr, 48, cr_param.private_data);\r
- printf("\n%d: DAPL Test Complete. %s\n\n",\r
- getpid(), ret ? "FAILED" : "PASSED");\r
- exit(0);\r
-#endif\r
-\r
- ret = dat_cr_accept(h_cr, h_ep, 48, cr_param.private_data);\r
-\r
- if (ret != DAT_SUCCESS) {\r
- fprintf(stderr, "%d Error dat_cr_accept: %s\n",\r
- getpid(), DT_RetToStr(ret));\r
- return (ret);\r
- } else\r
- LOGPRINTF("%d dat_cr_accept completed\n", getpid());\r
- } else { /* CLIENT */\r
- struct addrinfo *target;\r
- int rval;\r
-\r
- if (ucm)\r
- goto no_resolution;\r
-\r
-#if defined(_WIN32) || defined(_WIN64)\r
- if ((rval = getaddrinfo(hostname, "ftp", NULL, &target)) != 0) {\r
- printf("\n remote name resolution failed! %s\n",\r
- gai_strerror(rval));\r
- exit(1);\r
- }\r
- rval = ((struct sockaddr_in *)target->ai_addr)->sin_addr.s_addr;\r
-#else\r
- if (getaddrinfo(hostname, NULL, NULL, &target) != 0) {\r
- perror("\n remote name resolution failed!");\r
- exit(1);\r
- }\r
- rval = ((struct sockaddr_in *)target->ai_addr)->sin_addr.s_addr;\r
-#endif\r
- printf("%d Server Name: %s \n", getpid(), hostname);\r
- printf("%d Server Net Address: %d.%d.%d.%d port " F64x "\n",\r
- getpid(), (rval >> 0) & 0xff, (rval >> 8) & 0xff,\r
- (rval >> 16) & 0xff, (rval >> 24) & 0xff, conn_id);\r
-\r
- remote_addr = (DAT_IA_ADDRESS_PTR)target->ai_addr; /* IP */\r
-no_resolution:\r
- for (i = 0; i < 48; i++) /* simple pattern in private data */\r
- pdata[i] = i + 1;\r
-\r
- LOGPRINTF("%d Connecting to server\n", getpid());\r
- start = get_time();\r
- ret = dat_ep_connect(h_ep,\r
- remote_addr,\r
- conn_id,\r
- CONN_TIMEOUT,\r
- 48,\r
- (DAT_PVOID) pdata,\r
- 0, DAT_CONNECT_DEFAULT_FLAG);\r
- if (ret != DAT_SUCCESS) {\r
- fprintf(stderr, "%d Error dat_ep_connect: %s\n",\r
- getpid(), DT_RetToStr(ret));\r
- return (ret);\r
- } else\r
- LOGPRINTF("%d dat_ep_connect completed\n", getpid());\r
-\r
- if (!ucm)\r
- freeaddrinfo(target);\r
- }\r
-\r
- printf("%d Waiting for connect response\n", getpid());\r
-\r
- if (polling) \r
- while (DAT_GET_TYPE(dat_evd_dequeue(h_conn_evd, &event)) == \r
- DAT_QUEUE_EMPTY)\r
- conn_poll_count++;\r
- else \r
- ret = dat_evd_wait(h_conn_evd, DAT_TIMEOUT_INFINITE, \r
- 1, &event, &nmore);\r
-\r
- if (!server) {\r
- stop = get_time();\r
- ts.conn += ((stop - start) * 1.0e6);\r
- }\r
-\r
-#ifdef TEST_REJECT_WITH_PRIVATE_DATA\r
- if (event.event_number != DAT_CONNECTION_EVENT_PEER_REJECTED) {\r
- fprintf(stderr, "%d expected conn reject event : %s\n",\r
- getpid(), DT_EventToStr(event.event_number));\r
- return (DAT_ABORT);\r
- }\r
- /* get the reject private data and validate */\r
- buf = (unsigned char *)event.event_data.connect_event_data.private_data;\r
- printf("%d Received REJECT with private data %p[0]=%d [47]=%d\n",\r
- getpid(), buf, buf[0], buf[47]);\r
- for (i = 0; i < 48; i++) {\r
- if (buf[i] != i + 2) {\r
- fprintf(stderr, "%d client: Error with REJECT event"\r
- " private data: %p[%d]=%d s/be %d\n",\r
- getpid(), buf, i, buf[i], i + 2);\r
- dat_ep_disconnect(h_ep, DAT_CLOSE_ABRUPT_FLAG);\r
- return (DAT_ABORT);\r
- }\r
- }\r
- printf("\n%d: DAPL Test Complete. PASSED\n\n", getpid());\r
- exit(0);\r
-#endif\r
-\r
- if (event.event_number != DAT_CONNECTION_EVENT_ESTABLISHED) {\r
- fprintf(stderr, "%d Error unexpected conn event : 0x%x %s\n",\r
- getpid(), event.event_number,\r
- DT_EventToStr(event.event_number));\r
- return (DAT_ABORT);\r
- }\r
-\r
- /* check private data back from server */\r
- if (!server) {\r
- buf =\r
- (unsigned char *)event.event_data.connect_event_data.\r
- private_data;\r
- LOGPRINTF("%d CONN Private Data %p[0]=%d [47]=%d\n", getpid(),\r
- buf, buf[0], buf[47]);\r
- for (i = 0; i < 48; i++) {\r
- if (buf[i] != i + 2) {\r
- fprintf(stderr, "%d Error with CONNECT event"\r
- " private data: %p[%d]=%d s/be %d\n",\r
- getpid(), buf, i, buf[i], i + 2);\r
- dat_ep_disconnect(h_ep, DAT_CLOSE_ABRUPT_FLAG);\r
- LOGPRINTF\r
- ("%d waiting for disconnect event...\n",\r
- getpid());\r
- dat_evd_wait(h_conn_evd, DAT_TIMEOUT_INFINITE,\r
- 1, &event, &nmore);\r
- return (DAT_ABORT);\r
- }\r
- }\r
- }\r
-\r
- printf("\n%d CONNECTED!\n\n", getpid());\r
- connected = 1;\r
-\r
-#if CONNECT_ONLY\r
- return 0;\r
-#endif\r
-\r
- /*\r
- * Setup our remote memory and tell the other side about it\r
- */\r
- rmr_send_msg.virtual_address = htonll((DAT_VADDR) (uintptr_t) rbuf);\r
- rmr_send_msg.segment_length = htonl(RDMA_BUFFER_SIZE);\r
- rmr_send_msg.rmr_context = htonl(rmr_context_recv);\r
-\r
- printf("%d Send RMR msg to remote: r_key_ctx=0x%x,va=%p,len=0x%x\n",\r
- getpid(), rmr_context_recv, rbuf, RDMA_BUFFER_SIZE);\r
-\r
- ret = send_msg(&rmr_send_msg,\r
- sizeof(DAT_RMR_TRIPLET),\r
- lmr_context_send_msg,\r
- cookie, DAT_COMPLETION_SUPPRESS_FLAG);\r
-\r
- if (ret != DAT_SUCCESS) {\r
- fprintf(stderr, "%d Error send_msg: %s\n",\r
- getpid(), DT_RetToStr(ret));\r
- return (ret);\r
- } else\r
- LOGPRINTF("%d send_msg completed\n", getpid());\r
-\r
- /*\r
- * Wait for remote RMR information for RDMA\r
- */\r
- if (collect_event(h_dto_rcv_evd, \r
- &event, \r
- DTO_TIMEOUT, \r
- &poll_count) != DAT_SUCCESS)\r
- return (DAT_ABORT);\r
- \r
- printf("%d remote RMR data arrived!\n", getpid());\r
-\r
- if (event.event_number != DAT_DTO_COMPLETION_EVENT) {\r
- fprintf(stderr, "%d Error unexpected DTO event : %s\n",\r
- getpid(), DT_EventToStr(event.event_number));\r
- return (DAT_ABORT);\r
- }\r
- if ((event.event_data.dto_completion_event_data.transfered_length !=\r
- sizeof(DAT_RMR_TRIPLET)) ||\r
- (event.event_data.dto_completion_event_data.user_cookie.as_64 !=\r
- recv_msg_index)) {\r
- fprintf(stderr,\r
- "ERR recv event: len=%d cookie=" F64x\r
- " expected %d/%d\n",\r
- (int)event.event_data.dto_completion_event_data.\r
- transfered_length,\r
- event.event_data.dto_completion_event_data.user_cookie.\r
- as_64, (int)sizeof(DAT_RMR_TRIPLET), recv_msg_index);\r
- return (DAT_ABORT);\r
- }\r
-\r
- /* swap received RMR msg: network order to host order */\r
- r_iov = rmr_recv_msg[recv_msg_index];\r
- rmr_recv_msg[recv_msg_index].rmr_context = ntohl(r_iov.rmr_context);\r
- rmr_recv_msg[recv_msg_index].virtual_address =\r
- ntohll(r_iov.virtual_address);\r
- rmr_recv_msg[recv_msg_index].segment_length =\r
- ntohl(r_iov.segment_length);\r
-\r
- printf("%d Received RMR from remote: "\r
- "r_iov: r_key_ctx=%x,va=" F64x ",len=0x%x\n",\r
- getpid(), rmr_recv_msg[recv_msg_index].rmr_context,\r
- rmr_recv_msg[recv_msg_index].virtual_address,\r
- rmr_recv_msg[recv_msg_index].segment_length);\r
-\r
- recv_msg_index++;\r
-\r
- return (DAT_SUCCESS);\r
-}\r
-\r
-void disconnect_ep(void)\r
-{\r
- DAT_RETURN ret;\r
- DAT_EVENT event;\r
- DAT_COUNT nmore;\r
-\r
- if (connected) {\r
-\r
- /* \r
- * Only the client needs to call disconnect. The server _should_ be able\r
- * to just wait on the EVD associated with connection events for a\r
- * disconnect request and then exit.\r
- */\r
- if (!server) {\r
- LOGPRINTF("%d dat_ep_disconnect\n", getpid());\r
- ret = dat_ep_disconnect(h_ep, DAT_CLOSE_DEFAULT);\r
- if (ret != DAT_SUCCESS) {\r
- fprintf(stderr,\r
- "%d Error dat_ep_disconnect: %s\n",\r
- getpid(), DT_RetToStr(ret));\r
- } else {\r
- LOGPRINTF("%d dat_ep_disconnect completed\n",\r
- getpid());\r
- }\r
- } else {\r
- LOGPRINTF("%d Server waiting for disconnect...\n",\r
- getpid());\r
- }\r
-\r
- ret =\r
- dat_evd_wait(h_conn_evd, DAT_TIMEOUT_INFINITE, 1, &event,\r
- &nmore);\r
- if (ret != DAT_SUCCESS) {\r
- fprintf(stderr, "%d Error dat_evd_wait: %s\n",\r
- getpid(), DT_RetToStr(ret));\r
- } else {\r
- LOGPRINTF("%d dat_evd_wait for h_conn_evd completed\n",\r
- getpid());\r
- }\r
- }\r
-\r
- /* destroy service point */\r
- if ((server) && (h_psp != DAT_HANDLE_NULL)) {\r
- ret = dat_psp_free(h_psp);\r
- if (ret != DAT_SUCCESS) {\r
- fprintf(stderr, "%d Error dat_psp_free: %s\n",\r
- getpid(), DT_RetToStr(ret));\r
- } else {\r
- LOGPRINTF("%d dat_psp_free completed\n", getpid());\r
- }\r
- }\r
-\r
- /* Unregister Send message Buffer */\r
- if (h_lmr_send_msg != DAT_HANDLE_NULL) {\r
- LOGPRINTF("%d Unregister send message h_lmr %p \n", getpid(),\r
- h_lmr_send_msg);\r
- ret = dat_lmr_free(h_lmr_send_msg);\r
- if (ret != DAT_SUCCESS) {\r
- fprintf(stderr,\r
- "%d Error deregistering send msg mr: %s\n",\r
- getpid(), DT_RetToStr(ret));\r
- } else {\r
- LOGPRINTF("%d Unregistered send message Buffer\n",\r
- getpid());\r
- h_lmr_send_msg = NULL;\r
- }\r
- }\r
-\r
- /* Unregister recv message Buffer */\r
- if (h_lmr_recv_msg != DAT_HANDLE_NULL) {\r
- LOGPRINTF("%d Unregister recv message h_lmr %p \n", getpid(),\r
- h_lmr_recv_msg);\r
- ret = dat_lmr_free(h_lmr_recv_msg);\r
- if (ret != DAT_SUCCESS) {\r
- fprintf(stderr,\r
- "%d Error deregistering recv msg mr: %s\n",\r
- getpid(), DT_RetToStr(ret));\r
- } else {\r
- LOGPRINTF("%d Unregistered recv message Buffer\n",\r
- getpid());\r
- h_lmr_recv_msg = NULL;\r
- }\r
- }\r
- return;\r
-}\r
-\r
-DAT_RETURN do_rdma_write_with_msg(void)\r
-{\r
- DAT_EVENT event;\r
- DAT_LMR_TRIPLET l_iov[MSG_IOV_COUNT];\r
- DAT_RMR_TRIPLET r_iov;\r
- DAT_DTO_COOKIE cookie;\r
- DAT_RETURN ret;\r
- int i;\r
-\r
- printf("\n %d RDMA WRITE DATA with SEND MSG\n\n", getpid());\r
-\r
- cookie.as_64 = 0x5555;\r
-\r
- if (recv_msg_index >= MSG_BUF_COUNT)\r
- return (DAT_ABORT);\r
-\r
- /* get RMR information from previously received message */\r
- r_iov = rmr_recv_msg[recv_msg_index - 1];\r
-\r
- if (server)\r
- strcpy((char *)sbuf, "server RDMA write data...");\r
- else\r
- strcpy((char *)sbuf, "client RDMA write data...");\r
-\r
- for (i = 0; i < MSG_IOV_COUNT; i++) {\r
- l_iov[i].lmr_context = lmr_context_send;\r
- l_iov[i].segment_length = buf_len / MSG_IOV_COUNT;\r
- l_iov[i].virtual_address = (DAT_VADDR) (uintptr_t)\r
- (&sbuf[l_iov[i].segment_length * i]);\r
-\r
- LOGPRINTF("%d rdma_write iov[%d] buf=%p,len=%d\n",\r
- getpid(), i, &sbuf[l_iov[i].segment_length * i],\r
- l_iov[i].segment_length);\r
- }\r
-\r
- start = get_time();\r
- for (i = 0; i < burst; i++) {\r
- cookie.as_64 = 0x9999;\r
- ret = dat_ep_post_rdma_write(h_ep, // ep_handle\r
- MSG_IOV_COUNT, // num_segments\r
- l_iov, // LMR\r
- cookie, // user_cookie\r
- &r_iov, // RMR\r
- DAT_COMPLETION_SUPPRESS_FLAG);\r
- if (ret != DAT_SUCCESS) {\r
- fprintf(stderr,\r
- "%d: ERROR: dat_ep_post_rdma_write() %s\n",\r
- getpid(), DT_RetToStr(ret));\r
- return (DAT_ABORT);\r
- }\r
- LOGPRINTF("%d rdma_write # %d completed\n", getpid(), i + 1);\r
- }\r
-\r
- /*\r
- * Send RMR information a 2nd time to indicate completion\r
- * NOTE: already swapped to network order in connect_ep\r
- */\r
- printf("%d Sending RDMA WRITE completion message\n", getpid());\r
-\r
- ret = send_msg(&rmr_send_msg,\r
- sizeof(DAT_RMR_TRIPLET),\r
- lmr_context_send_msg,\r
- cookie, DAT_COMPLETION_SUPPRESS_FLAG);\r
-\r
- if (ret != DAT_SUCCESS) {\r
- fprintf(stderr, "%d Error send_msg: %s\n",\r
- getpid(), DT_RetToStr(ret));\r
- return (ret);\r
- } else {\r
- LOGPRINTF("%d send_msg completed\n", getpid());\r
- }\r
-\r
- /* inbound recv event, send completion's suppressed */\r
- if (collect_event(h_dto_rcv_evd, \r
- &event, \r
- DTO_TIMEOUT, \r
- &rdma_wr_poll_count) != DAT_SUCCESS)\r
- return (DAT_ABORT);\r
- \r
- stop = get_time();\r
- ts.rdma_wr = ((stop - start) * 1.0e6);\r
-\r
- /* validate event number and status */\r
- printf("%d inbound rdma_write; send message arrived!\n", getpid());\r
- if (event.event_number != DAT_DTO_COMPLETION_EVENT) {\r
- fprintf(stderr, "%d Error unexpected DTO event : %s\n",\r
- getpid(), DT_EventToStr(event.event_number));\r
- return (DAT_ABORT);\r
- }\r
-\r
- if ((event.event_data.dto_completion_event_data.transfered_length !=\r
- sizeof(DAT_RMR_TRIPLET))\r
- || (event.event_data.dto_completion_event_data.user_cookie.as_64 !=\r
- recv_msg_index)) {\r
- fprintf(stderr,\r
- "unexpected event data for receive: len=%d cookie=" F64x\r
- " exp %d/%d\n",\r
- (int)event.event_data.dto_completion_event_data.\r
- transfered_length,\r
- event.event_data.dto_completion_event_data.user_cookie.\r
- as_64, (int)sizeof(DAT_RMR_TRIPLET), recv_msg_index);\r
-\r
- return (DAT_ABORT);\r
- }\r
-\r
- /* swap received RMR msg: network order to host order */\r
- r_iov = rmr_recv_msg[recv_msg_index];\r
- rmr_recv_msg[recv_msg_index].virtual_address =\r
- ntohll(rmr_recv_msg[recv_msg_index].virtual_address);\r
- rmr_recv_msg[recv_msg_index].segment_length =\r
- ntohl(rmr_recv_msg[recv_msg_index].segment_length);\r
- rmr_recv_msg[recv_msg_index].rmr_context =\r
- ntohl(rmr_recv_msg[recv_msg_index].rmr_context);\r
-\r
- printf("%d Received RMR from remote: "\r
- "r_iov: r_key_ctx=%x,va=" F64x ",len=0x%x\n",\r
- getpid(), rmr_recv_msg[recv_msg_index].rmr_context,\r
- rmr_recv_msg[recv_msg_index].virtual_address,\r
- rmr_recv_msg[recv_msg_index].segment_length);\r
-\r
- LOGPRINTF("%d inbound rdma_write; send msg event SUCCESS!!\n",\r
- getpid());\r
-\r
- printf("%d %s RDMA write buffer contains: %s\n",\r
- getpid(), server ? "SERVER:" : "CLIENT:", rbuf);\r
-\r
- recv_msg_index++;\r
-\r
- return (DAT_SUCCESS);\r
-}\r
-\r
-DAT_RETURN do_rdma_read_with_msg(void)\r
-{\r
- DAT_EVENT event;\r
- DAT_LMR_TRIPLET l_iov;\r
- DAT_RMR_TRIPLET r_iov;\r
- DAT_DTO_COOKIE cookie;\r
- DAT_RETURN ret;\r
- int i;\r
-\r
- printf("\n %d RDMA READ DATA with SEND MSG\n\n", getpid());\r
-\r
- if (recv_msg_index >= MSG_BUF_COUNT)\r
- return (DAT_ABORT);\r
-\r
- /* get RMR information from previously received message */\r
- r_iov = rmr_recv_msg[recv_msg_index - 1];\r
-\r
- /* setup rdma read buffer to initial string to be overwritten */\r
- strcpy((char *)sbuf, "blah, blah, blah\n");\r
-\r
- if (server)\r
- strcpy((char *)rbuf, "server RDMA read data...");\r
- else\r
- strcpy((char *)rbuf, "client RDMA read data...");\r
-\r
- l_iov.lmr_context = lmr_context_send;\r
- l_iov.virtual_address = (DAT_VADDR) (uintptr_t) sbuf;\r
- l_iov.segment_length = buf_len;\r
-\r
- for (i = 0; i < MAX_RDMA_RD; i++) {\r
- cookie.as_64 = 0x9999;\r
- start = get_time();\r
- ret = dat_ep_post_rdma_read(h_ep, // ep_handle\r
- 1, // num_segments\r
- &l_iov, // LMR\r
- cookie, // user_cookie\r
- &r_iov, // RMR\r
- DAT_COMPLETION_DEFAULT_FLAG);\r
- if (ret != DAT_SUCCESS) {\r
- fprintf(stderr,\r
- "%d: ERROR: dat_ep_post_rdma_read() %s\n",\r
- getpid(), DT_RetToStr(ret));\r
- return (DAT_ABORT);\r
- }\r
-\r
- /* RDMA read completion event */\r
- if (collect_event(h_dto_req_evd, \r
- &event, \r
- DTO_TIMEOUT, \r
- &rdma_rd_poll_count[i]) != DAT_SUCCESS)\r
- return (DAT_ABORT);\r
-\r
- /* validate event number, len, cookie, and status */\r
- if (event.event_number != DAT_DTO_COMPLETION_EVENT) {\r
- fprintf(stderr, "%d: ERROR: DTO event number %s\n",\r
- getpid(), DT_EventToStr(event.event_number));\r
- return (DAT_ABORT);\r
- }\r
- if ((event.event_data.dto_completion_event_data.\r
- transfered_length != buf_len)\r
- || (event.event_data.dto_completion_event_data.user_cookie.\r
- as_64 != 0x9999)) {\r
- fprintf(stderr,\r
- "%d: ERROR: DTO len %d or cookie " F64x "\n",\r
- getpid(),\r
- event.event_data.dto_completion_event_data.\r
- transfered_length,\r
- event.event_data.dto_completion_event_data.\r
- user_cookie.as_64);\r
- return (DAT_ABORT);\r
- }\r
- if (event.event_data.dto_completion_event_data.status !=\r
- DAT_SUCCESS) {\r
- fprintf(stderr, "%d: ERROR: DTO event status %s\n",\r
- getpid(), DT_RetToStr(ret));\r
- return (DAT_ABORT);\r
- }\r
- stop = get_time();\r
- ts.rdma_rd[i] = ((stop - start) * 1.0e6);\r
- ts.rdma_rd_total += ts.rdma_rd[i];\r
-\r
- LOGPRINTF("%d rdma_read # %d completed\n", getpid(), i + 1);\r
- }\r
-\r
- /*\r
- * Send RMR information a 3rd time to indicate completion\r
- * NOTE: already swapped to network order in connect_ep\r
- */\r
- printf("%d Sending RDMA read completion message\n", getpid());\r
-\r
- /* give remote chance to process read completes */\r
- if (use_cno) {\r
-#if defined(_WIN32) || defined(_WIN64)\r
- Sleep(1000);\r
-#else\r
- sleep(1);\r
-#endif\r
- }\r
-\r
- ret = send_msg(&rmr_send_msg,\r
- sizeof(DAT_RMR_TRIPLET),\r
- lmr_context_send_msg,\r
- cookie, DAT_COMPLETION_SUPPRESS_FLAG);\r
-\r
- if (ret != DAT_SUCCESS) {\r
- fprintf(stderr, "%d Error send_msg: %s\n",\r
- getpid(), DT_RetToStr(ret));\r
- return (ret);\r
- } else {\r
- LOGPRINTF("%d send_msg completed\n", getpid());\r
- }\r
-\r
- printf("%d Waiting for inbound message....\n", getpid());\r
-\r
- if (collect_event(h_dto_rcv_evd, \r
- &event, \r
- DTO_TIMEOUT, \r
- &poll_count) != DAT_SUCCESS)\r
- return (DAT_ABORT);\r
-\r
- /* validate event number and status */\r
- printf("%d inbound rdma_read; send message arrived!\n", getpid());\r
- if (event.event_number != DAT_DTO_COMPLETION_EVENT) {\r
- fprintf(stderr, "%d Error unexpected DTO event : %s\n",\r
- getpid(), DT_EventToStr(event.event_number));\r
- return (DAT_ABORT);\r
- }\r
-\r
- if ((event.event_data.dto_completion_event_data.transfered_length !=\r
- sizeof(DAT_RMR_TRIPLET))\r
- || (event.event_data.dto_completion_event_data.user_cookie.as_64 !=\r
- recv_msg_index)) {\r
-\r
- fprintf(stderr,\r
- "unexpected event data for receive: len=%d cookie=" F64x\r
- " exp %d/%d\n",\r
- (int)event.event_data.dto_completion_event_data.\r
- transfered_length,\r
- event.event_data.dto_completion_event_data.user_cookie.\r
- as_64, (int)sizeof(DAT_RMR_TRIPLET), recv_msg_index);\r
-\r
- return (DAT_ABORT);\r
- }\r
-\r
- /* swap received RMR msg: network order to host order */\r
- r_iov = rmr_recv_msg[recv_msg_index];\r
- rmr_recv_msg[recv_msg_index].virtual_address =\r
- ntohll(rmr_recv_msg[recv_msg_index].virtual_address);\r
- rmr_recv_msg[recv_msg_index].segment_length =\r
- ntohl(rmr_recv_msg[recv_msg_index].segment_length);\r
- rmr_recv_msg[recv_msg_index].rmr_context =\r
- ntohl(rmr_recv_msg[recv_msg_index].rmr_context);\r
-\r
- printf("%d Received RMR from remote: "\r
- "r_iov: r_key_ctx=%x,va=" F64x ",len=0x%x\n",\r
- getpid(), rmr_recv_msg[recv_msg_index].rmr_context,\r
- rmr_recv_msg[recv_msg_index].virtual_address,\r
- rmr_recv_msg[recv_msg_index].segment_length);\r
-\r
- LOGPRINTF("%d inbound rdma_write; send msg event SUCCESS!!\n",\r
- getpid());\r
-\r
- printf("%d %s RCV RDMA read buffer contains: %s\n",\r
- getpid(), server ? "SERVER:" : "CLIENT:", sbuf);\r
-\r
- recv_msg_index++;\r
-\r
- return (DAT_SUCCESS);\r
-}\r
-\r
-DAT_RETURN do_ping_pong_msg()\r
-{\r
- DAT_EVENT event;\r
- DAT_DTO_COOKIE cookie;\r
- DAT_LMR_TRIPLET l_iov;\r
- DAT_RETURN ret;\r
- int i;\r
- char *snd_buf;\r
- char *rcv_buf;\r
-\r
- printf("\n %d PING DATA with SEND MSG\n\n", getpid());\r
-\r
- snd_buf = sbuf;\r
- rcv_buf = rbuf;\r
-\r
- /* pre-post all buffers */\r
- for (i = 0; i < burst; i++) {\r
- burst_msg_posted++;\r
- cookie.as_64 = i;\r
- l_iov.lmr_context = lmr_context_recv;\r
- l_iov.virtual_address = (DAT_VADDR) (uintptr_t) rcv_buf;\r
- l_iov.segment_length = buf_len;\r
-\r
- LOGPRINTF("%d Pre-posting Receive Message Buffers %p\n",\r
- getpid(), rcv_buf);\r
-\r
- ret = dat_ep_post_recv(h_ep,\r
- 1,\r
- &l_iov,\r
- cookie, DAT_COMPLETION_DEFAULT_FLAG);\r
-\r
- if (ret != DAT_SUCCESS) {\r
- fprintf(stderr,\r
- "%d Error posting recv msg buffer: %s\n",\r
- getpid(), DT_RetToStr(ret));\r
- return (ret);\r
- } else {\r
- LOGPRINTF("%d Posted Receive Message Buffer %p\n",\r
- getpid(), rcv_buf);\r
- }\r
-\r
- /* next buffer */\r
- rcv_buf += buf_len;\r
- }\r
-#if defined(_WIN32) || defined(_WIN64)\r
- Sleep(1000);\r
-#else\r
- sleep(1);\r
-#endif\r
-\r
- /* Initialize recv_buf and index to beginning */\r
- rcv_buf = rbuf;\r
- burst_msg_index = 0;\r
-\r
- /* client ping 0x55, server pong 0xAA in first byte */\r
- start = get_time();\r
- for (i = 0; i < burst; i++) {\r
- /* walk the send and recv buffers */\r
- if (!server) {\r
- *snd_buf = 0x55;\r
-\r
- LOGPRINTF("%d %s SND buffer %p contains: 0x%x len=%d\n",\r
- getpid(), server ? "SERVER:" : "CLIENT:",\r
- snd_buf, *snd_buf, buf_len);\r
-\r
- ret = send_msg(snd_buf,\r
- buf_len,\r
- lmr_context_send,\r
- cookie, DAT_COMPLETION_SUPPRESS_FLAG);\r
-\r
- if (ret != DAT_SUCCESS) {\r
- fprintf(stderr, "%d Error send_msg: %s\n",\r
- getpid(), DT_RetToStr(ret));\r
- return (ret);\r
- } else {\r
- LOGPRINTF("%d send_msg completed\n", getpid());\r
- }\r
- }\r
-\r
- /* recv message, send completions suppressed */\r
- if (collect_event(h_dto_rcv_evd, \r
- &event, \r
- DTO_TIMEOUT, \r
- &poll_count) != DAT_SUCCESS)\r
- return (DAT_ABORT);\r
-\r
- \r
- /* start timer after first message arrives on server */\r
- if (i == 0) {\r
- start = get_time();\r
- }\r
- /* validate event number and status */\r
- LOGPRINTF("%d inbound message; message arrived!\n", getpid());\r
- if (event.event_number != DAT_DTO_COMPLETION_EVENT) {\r
- fprintf(stderr, "%d Error unexpected DTO event : %s\n",\r
- getpid(), DT_EventToStr(event.event_number));\r
- return (DAT_ABORT);\r
- }\r
- if ((event.event_data.dto_completion_event_data.\r
- transfered_length != buf_len)\r
- || (event.event_data.dto_completion_event_data.user_cookie.\r
- as_64 != burst_msg_index)) {\r
- fprintf(stderr,\r
- "ERR: recv event: len=%d cookie=" F64x\r
- " exp %d/%d\n",\r
- (int)event.event_data.dto_completion_event_data.\r
- transfered_length,\r
- event.event_data.dto_completion_event_data.\r
- user_cookie.as_64, (int)buf_len,\r
- (int)burst_msg_index);\r
-\r
- return (DAT_ABORT);\r
- }\r
-\r
- LOGPRINTF("%d %s RCV buffer %p contains: 0x%x len=%d\n",\r
- getpid(), server ? "SERVER:" : "CLIENT:",\r
- rcv_buf, *rcv_buf, buf_len);\r
-\r
- burst_msg_index++;\r
-\r
- /* If server, change data and send it back to client */\r
- if (server) {\r
- *snd_buf = 0xaa;\r
-\r
- LOGPRINTF("%d %s SND buffer %p contains: 0x%x len=%d\n",\r
- getpid(), server ? "SERVER:" : "CLIENT:",\r
- snd_buf, *snd_buf, buf_len);\r
-\r
- ret = send_msg(snd_buf,\r
- buf_len,\r
- lmr_context_send,\r
- cookie, DAT_COMPLETION_SUPPRESS_FLAG);\r
-\r
- if (ret != DAT_SUCCESS) {\r
- fprintf(stderr, "%d Error send_msg: %s\n",\r
- getpid(), DT_RetToStr(ret));\r
- return (ret);\r
- } else {\r
- LOGPRINTF("%d send_msg completed\n", getpid());\r
- }\r
- }\r
-\r
- /* next buffers */\r
- rcv_buf += buf_len;\r
- snd_buf += buf_len;\r
- }\r
- stop = get_time();\r
- ts.rtt = ((stop - start) * 1.0e6);\r
-\r
- return (DAT_SUCCESS);\r
-}\r
-\r
-/* Register RDMA Receive buffer */\r
-DAT_RETURN register_rdma_memory(void)\r
-{\r
- DAT_RETURN ret;\r
- DAT_REGION_DESCRIPTION region;\r
-\r
- region.for_va = rbuf;\r
- start = get_time();\r
- ret = dat_lmr_create(h_ia,\r
- DAT_MEM_TYPE_VIRTUAL,\r
- region,\r
- buf_len * (burst+1),\r
- h_pz,\r
- DAT_MEM_PRIV_ALL_FLAG,\r
- DAT_VA_TYPE_VA,\r
- &h_lmr_recv,\r
- &lmr_context_recv,\r
- &rmr_context_recv,\r
- ®istered_size_recv, ®istered_addr_recv);\r
- stop = get_time();\r
- ts.reg += ((stop - start) * 1.0e6);\r
- ts.total += ts.reg;\r
-\r
- if (ret != DAT_SUCCESS) {\r
- fprintf(stderr,\r
- "%d Error registering Receive RDMA buffer: %s\n",\r
- getpid(), DT_RetToStr(ret));\r
- return (ret);\r
- } else {\r
- LOGPRINTF("%d Registered Receive RDMA Buffer %p\n",\r
- getpid(), region.for_va);\r
- }\r
-\r
- /* Register RDMA Send buffer */\r
- region.for_va = sbuf;\r
- ret = dat_lmr_create(h_ia,\r
- DAT_MEM_TYPE_VIRTUAL,\r
- region,\r
- buf_len * (burst + 1),\r
- h_pz,\r
- DAT_MEM_PRIV_ALL_FLAG,\r
- DAT_VA_TYPE_VA,\r
- &h_lmr_send,\r
- &lmr_context_send,\r
- &rmr_context_send,\r
- ®istered_size_send, ®istered_addr_send);\r
- if (ret != DAT_SUCCESS) {\r
- fprintf(stderr, "%d Error registering send RDMA buffer: %s\n",\r
- getpid(), DT_RetToStr(ret));\r
- return (ret);\r
- } else {\r
- LOGPRINTF("%d Registered Send RDMA Buffer %p\n",\r
- getpid(), region.for_va);\r
- }\r
-\r
- return DAT_SUCCESS;\r
-}\r
-\r
-/*\r
- * Unregister RDMA memory\r
- */\r
-DAT_RETURN unregister_rdma_memory(void)\r
-{\r
- DAT_RETURN ret;\r
-\r
- /* Unregister Recv Buffer */\r
- if (h_lmr_recv != DAT_HANDLE_NULL) {\r
- LOGPRINTF("%d Unregister h_lmr %p \n", getpid(), h_lmr_recv);\r
- start = get_time();\r
- ret = dat_lmr_free(h_lmr_recv);\r
- stop = get_time();\r
- ts.unreg += ((stop - start) * 1.0e6);\r
- ts.total += ts.unreg;\r
- if (ret != DAT_SUCCESS) {\r
- fprintf(stderr, "%d Error deregistering recv mr: %s\n",\r
- getpid(), DT_RetToStr(ret));\r
- return (ret);\r
- } else {\r
- LOGPRINTF("%d Unregistered Recv Buffer\n", getpid());\r
- h_lmr_recv = NULL;\r
- }\r
- }\r
-\r
- /* Unregister Send Buffer */\r
- if (h_lmr_send != DAT_HANDLE_NULL) {\r
- LOGPRINTF("%d Unregister h_lmr %p \n", getpid(), h_lmr_send);\r
- ret = dat_lmr_free(h_lmr_send);\r
- if (ret != DAT_SUCCESS) {\r
- fprintf(stderr, "%d Error deregistering send mr: %s\n",\r
- getpid(), DT_RetToStr(ret));\r
- return (ret);\r
- } else {\r
- LOGPRINTF("%d Unregistered send Buffer\n", getpid());\r
- h_lmr_send = NULL;\r
- }\r
- }\r
- return DAT_SUCCESS;\r
-}\r
-\r
- /*\r
- * Create CNO, CR, CONN, and DTO events\r
- */\r
-DAT_RETURN create_events(void)\r
-{\r
- DAT_RETURN ret;\r
- DAT_EVD_PARAM param;\r
-\r
- /* create CNO */\r
- if (use_cno) {\r
- start = get_time();\r
-#if defined(_WIN32) || defined(_WIN64)\r
- {\r
- DAT_OS_WAIT_PROXY_AGENT pa = { NULL, NULL };\r
- ret = dat_cno_create(h_ia, pa, &h_dto_cno);\r
- }\r
-#else\r
- ret =\r
- dat_cno_create(h_ia, DAT_OS_WAIT_PROXY_AGENT_NULL,\r
- &h_dto_cno);\r
-#endif\r
- stop = get_time();\r
- ts.cnoc += ((stop - start) * 1.0e6);\r
- ts.total += ts.cnoc;\r
- if (ret != DAT_SUCCESS) {\r
- fprintf(stderr, "%d Error dat_cno_create: %s\n",\r
- getpid(), DT_RetToStr(ret));\r
- return (ret);\r
- } else {\r
- LOGPRINTF("%d cr_evd created, %p\n", getpid(),\r
- h_dto_cno);\r
- }\r
- }\r
-\r
- /* create cr EVD */\r
- start = get_time();\r
- ret =\r
- dat_evd_create(h_ia, 10, DAT_HANDLE_NULL, DAT_EVD_CR_FLAG,\r
- &h_cr_evd);\r
- stop = get_time();\r
- ts.evdc += ((stop - start) * 1.0e6);\r
- ts.total += ts.evdc;\r
- if (ret != DAT_SUCCESS) {\r
- fprintf(stderr, "%d Error dat_evd_create: %s\n",\r
- getpid(), DT_RetToStr(ret));\r
- return (ret);\r
- } else {\r
- LOGPRINTF("%d cr_evd created %p\n", getpid(), h_cr_evd);\r
- }\r
-\r
- /* create conn EVD */\r
- ret = dat_evd_create(h_ia,\r
- 10,\r
- DAT_HANDLE_NULL,\r
- DAT_EVD_CONNECTION_FLAG, &h_conn_evd);\r
- if (ret != DAT_SUCCESS) {\r
- fprintf(stderr, "%d Error dat_evd_create: %s\n",\r
- getpid(), DT_RetToStr(ret));\r
- return (ret);\r
- } else {\r
- LOGPRINTF("%d con_evd created %p\n", getpid(), h_conn_evd);\r
- }\r
-\r
- /* create dto SND EVD, with CNO if use_cno was set */\r
- ret = dat_evd_create(h_ia,\r
- (MSG_BUF_COUNT + MAX_RDMA_RD + burst) * 2,\r
- h_dto_cno, DAT_EVD_DTO_FLAG, &h_dto_req_evd);\r
- if (ret != DAT_SUCCESS) {\r
- fprintf(stderr, "%d Error dat_evd_create REQ: %s\n",\r
- getpid(), DT_RetToStr(ret));\r
- return (ret);\r
- } else {\r
- LOGPRINTF("%d dto_req_evd created %p\n", getpid(),\r
- h_dto_req_evd);\r
- }\r
-\r
- /* create dto RCV EVD, with CNO if use_cno was set */\r
- ret = dat_evd_create(h_ia,\r
- MSG_BUF_COUNT + burst,\r
- h_dto_cno, DAT_EVD_DTO_FLAG, &h_dto_rcv_evd);\r
- if (ret != DAT_SUCCESS) {\r
- fprintf(stderr, "%d Error dat_evd_create RCV: %s\n",\r
- getpid(), DT_RetToStr(ret));\r
- return (ret);\r
- } else {\r
- LOGPRINTF("%d dto_rcv_evd created %p\n", getpid(),\r
- h_dto_rcv_evd);\r
- }\r
-\r
- /* query DTO req EVD and check size */\r
- ret = dat_evd_query(h_dto_req_evd, DAT_EVD_FIELD_EVD_QLEN, ¶m);\r
- if (ret != DAT_SUCCESS) {\r
- fprintf(stderr, "%d Error dat_evd_query request evd: %s\n",\r
- getpid(), DT_RetToStr(ret));\r
- return (ret);\r
- } else if (param.evd_qlen < (MSG_BUF_COUNT + MAX_RDMA_RD + burst) * 2) {\r
- fprintf(stderr, "%d Error dat_evd qsize too small: %d < %d\n",\r
- getpid(), param.evd_qlen,\r
- (MSG_BUF_COUNT + MAX_RDMA_RD + burst) * 2);\r
- return (ret);\r
- }\r
-\r
- LOGPRINTF("%d dto_req_evd QLEN - requested %d and actual %d\n",\r
- getpid(), (MSG_BUF_COUNT + MAX_RDMA_RD + burst) * 2,\r
- param.evd_qlen);\r
-\r
- return DAT_SUCCESS;\r
-}\r
-\r
-/*\r
- * Destroy CR, CONN, CNO, and DTO events\r
- */\r
-\r
-DAT_RETURN destroy_events(void)\r
-{\r
- DAT_RETURN ret;\r
-\r
- /* free cr EVD */\r
- if (h_cr_evd != DAT_HANDLE_NULL) {\r
- LOGPRINTF("%d Free cr EVD %p \n", getpid(), h_cr_evd);\r
- ret = dat_evd_free(h_cr_evd);\r
- if (ret != DAT_SUCCESS) {\r
- fprintf(stderr, "%d Error freeing cr EVD: %s\n",\r
- getpid(), DT_RetToStr(ret));\r
- return (ret);\r
- } else {\r
- LOGPRINTF("%d Freed cr EVD\n", getpid());\r
- h_cr_evd = DAT_HANDLE_NULL;\r
- }\r
- }\r
-\r
- /* free conn EVD */\r
- if (h_conn_evd != DAT_HANDLE_NULL) {\r
- LOGPRINTF("%d Free conn EVD %p \n", getpid(), h_conn_evd);\r
- ret = dat_evd_free(h_conn_evd);\r
- if (ret != DAT_SUCCESS) {\r
- fprintf(stderr, "%d Error freeing conn EVD: %s\n",\r
- getpid(), DT_RetToStr(ret));\r
- return (ret);\r
- } else {\r
- LOGPRINTF("%d Freed conn EVD\n", getpid());\r
- h_conn_evd = DAT_HANDLE_NULL;\r
- }\r
- }\r
-\r
- /* free RCV dto EVD */\r
- if (h_dto_rcv_evd != DAT_HANDLE_NULL) {\r
- LOGPRINTF("%d Free RCV dto EVD %p \n", getpid(), h_dto_rcv_evd);\r
- start = get_time();\r
- ret = dat_evd_free(h_dto_rcv_evd);\r
- stop = get_time();\r
- ts.evdf += ((stop - start) * 1.0e6);\r
- ts.total += ts.evdf;\r
- if (ret != DAT_SUCCESS) {\r
- fprintf(stderr, "%d Error freeing dto EVD: %s\n",\r
- getpid(), DT_RetToStr(ret));\r
- return (ret);\r
- } else {\r
- LOGPRINTF("%d Freed dto EVD\n", getpid());\r
- h_dto_rcv_evd = DAT_HANDLE_NULL;\r
- }\r
- }\r
-\r
- /* free REQ dto EVD */\r
- if (h_dto_req_evd != DAT_HANDLE_NULL) {\r
- LOGPRINTF("%d Free REQ dto EVD %p \n", getpid(), h_dto_req_evd);\r
- ret = dat_evd_free(h_dto_req_evd);\r
- if (ret != DAT_SUCCESS) {\r
- fprintf(stderr, "%d Error freeing dto EVD: %s\n",\r
- getpid(), DT_RetToStr(ret));\r
- return (ret);\r
- } else {\r
- LOGPRINTF("%d Freed dto EVD\n", getpid());\r
- h_dto_req_evd = DAT_HANDLE_NULL;\r
- }\r
- }\r
-\r
- /* free CNO */\r
- if (h_dto_cno != DAT_HANDLE_NULL) {\r
- LOGPRINTF("%d Free dto CNO %p \n", getpid(), h_dto_cno);\r
- start = get_time();\r
- ret = dat_cno_free(h_dto_cno);\r
- stop = get_time();\r
- ts.cnof += ((stop - start) * 1.0e6);\r
- ts.total += ts.cnof;\r
- if (ret != DAT_SUCCESS) {\r
- fprintf(stderr, "%d Error freeing dto CNO: %s\n",\r
- getpid(), DT_RetToStr(ret));\r
- return (ret);\r
- } else {\r
- LOGPRINTF("%d Freed dto CNO\n", getpid());\r
- h_dto_cno = DAT_HANDLE_NULL;\r
- }\r
- }\r
- return DAT_SUCCESS;\r
-}\r
-\r
-/*\r
- * Map DAT_RETURN values to readable strings,\r
- * but don't assume the values are zero-based or contiguous.\r
- */\r
-char errmsg[512] = { 0 };\r
-const char *DT_RetToStr(DAT_RETURN ret_value)\r
-{\r
- const char *major_msg, *minor_msg;\r
-\r
- dat_strerror(ret_value, &major_msg, &minor_msg);\r
-\r
- strcpy(errmsg, major_msg);\r
- strcat(errmsg, " ");\r
- strcat(errmsg, minor_msg);\r
-\r
- return errmsg;\r
-}\r
-\r
-/*\r
- * Map DAT_EVENT_CODE values to readable strings\r
- */\r
-const char *DT_EventToStr(DAT_EVENT_NUMBER event_code)\r
-{\r
- unsigned int i;\r
- static struct {\r
- const char *name;\r
- DAT_RETURN value;\r
- } dat_events[] = {\r
-# define DATxx(x) { # x, x }\r
- DATxx(DAT_DTO_COMPLETION_EVENT),\r
- DATxx(DAT_RMR_BIND_COMPLETION_EVENT),\r
- DATxx(DAT_CONNECTION_REQUEST_EVENT),\r
- DATxx(DAT_CONNECTION_EVENT_ESTABLISHED),\r
- DATxx(DAT_CONNECTION_EVENT_PEER_REJECTED),\r
- DATxx(DAT_CONNECTION_EVENT_NON_PEER_REJECTED),\r
- DATxx(DAT_CONNECTION_EVENT_ACCEPT_COMPLETION_ERROR),\r
- DATxx(DAT_CONNECTION_EVENT_DISCONNECTED),\r
- DATxx(DAT_CONNECTION_EVENT_BROKEN),\r
- DATxx(DAT_CONNECTION_EVENT_TIMED_OUT),\r
- DATxx(DAT_CONNECTION_EVENT_UNREACHABLE),\r
- DATxx(DAT_ASYNC_ERROR_EVD_OVERFLOW),\r
- DATxx(DAT_ASYNC_ERROR_IA_CATASTROPHIC),\r
- DATxx(DAT_ASYNC_ERROR_EP_BROKEN),\r
- DATxx(DAT_ASYNC_ERROR_TIMED_OUT),\r
- DATxx(DAT_ASYNC_ERROR_PROVIDER_INTERNAL_ERROR),\r
- DATxx(DAT_SOFTWARE_EVENT)\r
-# undef DATxx\r
- };\r
-# define NUM_EVENTS (sizeof(dat_events)/sizeof(dat_events[0]))\r
-\r
- for (i = 0; i < NUM_EVENTS; i++) {\r
- if (dat_events[i].value == event_code) {\r
- return (dat_events[i].name);\r
- }\r
- }\r
-\r
- return ("Invalid_DAT_EVENT_NUMBER");\r
-}\r
-\r
-void print_usage(void)\r
-{\r
- printf("\n DAPL USAGE \n\n");\r
- printf("s: server\n");\r
- printf("t: performance times\n");\r
- printf("c: use cno\n");\r
- printf("v: verbose\n");\r
- printf("p: polling\n");\r
- printf("d: delay before accept\n");\r
- printf("b: buf length to allocate\n");\r
- printf("B: burst count, rdma and msgs \n");\r
- printf("h: hostname/address of server, specified on client\n");\r
- printf("P: provider name (default = OpenIB-cma)\n");\r
- printf("l: server lid (required ucm provider)\n");\r
- printf("q: server qpn (required ucm provider)\n");\r
- printf("\n");\r
-}\r
-\r
+/*
+ * Copyright (c) 2005-2008 Intel Corporation. All rights reserved.
+ *
+ * This software is available to you under a choice of one of two
+ * licenses. You may choose to be licensed under the terms of the GNU
+ * General Public License (GPL) Version 2, available from the file
+ * COPYING in the main directory of this source tree, or the
+ * OpenIB.org BSD license below:
+ *
+ * Redistribution and use in source and binary forms, with or
+ * without modification, are permitted provided that the following
+ * conditions are met:
+ *
+ * - Redistributions of source code must retain the above
+ * copyright notice, this list of conditions and the following
+ * disclaimer.
+ *
+ * - Redistributions in binary form must reproduce the above
+ * copyright notice, this list of conditions and the following
+ * disclaimer in the documentation and/or other materials
+ * provided with the distribution.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
+ * EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
+ * MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
+ * NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS
+ * BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN
+ * ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN
+ * CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+ * SOFTWARE.
+ *
+ * $Id: $
+ */
+#include <stdio.h>
+#include <stdlib.h>
+#include <string.h>
+
+#ifdef DAPL_PROVIDER
+#undef DAPL_PROVIDER
+#endif
+
+#if defined(_WIN32) || defined(_WIN64)
+
+#include <windows.h>
+#include <winsock2.h>
+#include <ws2tcpip.h>
+#include <io.h>
+#include <process.h>
+#include <complib/cl_types.h>
+#include "..\..\..\..\etc\user\getopt.c"
+
+#define getpid() ((int)GetCurrentProcessId())
+#define F64x "%I64x"
+
+#ifdef DBG
+#define DAPL_PROVIDER "ibnic0v2d"
+#else
+#define DAPL_PROVIDER "ibnic0v2"
+#endif
+
+#define ntohll _byteswap_uint64
+#define htonll _byteswap_uint64
+
+#else // _WIN32 || _WIN64
+
+#include <endian.h>
+#include <byteswap.h>
+#include <netdb.h>
+#include <sys/types.h>
+#include <sys/socket.h>
+#include <sys/time.h>
+#include <netinet/in.h>
+#include <netinet/tcp.h>
+#include <arpa/inet.h>
+#include <sys/mman.h>
+#include <getopt.h>
+#include <inttypes.h>
+#include <unistd.h>
+#include <stdlib.h>
+
+#define DAPL_PROVIDER "ofa-v2-ib0"
+
+#define F64x "%"PRIx64""
+
+#if __BYTE_ORDER == __BIG_ENDIAN
+#define htonll(x) (x)
+#define ntohll(x) (x)
+#elif __BYTE_ORDER == __LITTLE_ENDIAN
+#define htonll(x) bswap_64(x)
+#define ntohll(x) bswap_64(x)
+#endif
+
+#endif // _WIN32 || _WIN64
+
+/* Debug: 1 == connect & close only, otherwise full-meal deal */
+#define CONNECT_ONLY 0
+
+#define MAX_POLLING_CNT 50000
+#define MAX_RDMA_RD 4
+#define MAX_PROCS 1000
+
+/* Header files needed for DAT/uDAPL */
+#include "dat2/udat.h"
+
+/* definitions */
+#define SERVER_CONN_QUAL 45248
+#define DTO_TIMEOUT (1000*1000*5)
+#define CNO_TIMEOUT (1000*1000*1)
+#define DTO_FLUSH_TIMEOUT (1000*1000*2)
+#define CONN_TIMEOUT (1000*1000*100)
+#define SERVER_TIMEOUT DAT_TIMEOUT_INFINITE
+#define RDMA_BUFFER_SIZE (64)
+
+/* Global DAT vars */
+static DAT_IA_HANDLE h_ia = DAT_HANDLE_NULL;
+static DAT_PZ_HANDLE h_pz = DAT_HANDLE_NULL;
+static DAT_EP_HANDLE h_ep = DAT_HANDLE_NULL;
+static DAT_PSP_HANDLE h_psp = DAT_HANDLE_NULL;
+static DAT_CR_HANDLE h_cr = DAT_HANDLE_NULL;
+
+static DAT_EVD_HANDLE h_async_evd = DAT_HANDLE_NULL;
+static DAT_EVD_HANDLE h_dto_req_evd = DAT_HANDLE_NULL;
+static DAT_EVD_HANDLE h_dto_rcv_evd = DAT_HANDLE_NULL;
+static DAT_EVD_HANDLE h_cr_evd = DAT_HANDLE_NULL;
+static DAT_EVD_HANDLE h_conn_evd = DAT_HANDLE_NULL;
+static DAT_CNO_HANDLE h_dto_cno = DAT_HANDLE_NULL;
+
+/* RDMA buffers */
+static DAT_LMR_HANDLE h_lmr_send = DAT_HANDLE_NULL;
+static DAT_LMR_HANDLE h_lmr_recv = DAT_HANDLE_NULL;
+static DAT_LMR_CONTEXT lmr_context_send;
+static DAT_LMR_CONTEXT lmr_context_recv;
+static DAT_RMR_CONTEXT rmr_context_send;
+static DAT_RMR_CONTEXT rmr_context_recv;
+static DAT_VLEN registered_size_send;
+static DAT_VLEN registered_size_recv;
+static DAT_VADDR registered_addr_send;
+static DAT_VADDR registered_addr_recv;
+
+/* Initial msg receive buf, RMR exchange, and Rdma-write notification */
+#define MSG_BUF_COUNT 3
+#define MSG_IOV_COUNT 2
+static DAT_RMR_TRIPLET rmr_recv_msg[MSG_BUF_COUNT];
+static DAT_LMR_HANDLE h_lmr_recv_msg = DAT_HANDLE_NULL;
+static DAT_LMR_CONTEXT lmr_context_recv_msg;
+static DAT_RMR_CONTEXT rmr_context_recv_msg;
+static DAT_VLEN registered_size_recv_msg;
+static DAT_VADDR registered_addr_recv_msg;
+
+/* message send buffer */
+static DAT_RMR_TRIPLET rmr_send_msg;
+static DAT_LMR_HANDLE h_lmr_send_msg = DAT_HANDLE_NULL;
+static DAT_LMR_CONTEXT lmr_context_send_msg;
+static DAT_RMR_CONTEXT rmr_context_send_msg;
+static DAT_VLEN registered_size_send_msg;
+static DAT_VADDR registered_addr_send_msg;
+static DAT_EP_ATTR ep_attr;
+char hostname[256] = { 0 };
+char provider[64] = DAPL_PROVIDER;
+char addr_str[INET_ADDRSTRLEN];
+
+/* rdma pointers */
+char *rbuf = NULL;
+char *sbuf = NULL;
+int status;
+
+/* timers */
+double start, stop, total_us, total_sec;
+
+struct dt_time {
+ double total;
+ double open;
+ double reg;
+ double unreg;
+ double pzc;
+ double pzf;
+ double evdc;
+ double evdf;
+ double cnoc;
+ double cnof;
+ double epc;
+ double epf;
+ double rdma_wr;
+ double rdma_rd[MAX_RDMA_RD];
+ double rdma_rd_total;
+ double rtt;
+ double close;
+ double conn;
+};
+
+struct dt_time ts;
+
+/* defaults */
+static int failed = 0;
+static int performance_times = 0;
+static int connected = 0;
+static int burst = 10;
+static int server = 1;
+static int verbose = 0;
+static int polling = 0;
+static int poll_count = 0;
+static int rdma_wr_poll_count = 0;
+static int conn_poll_count = 0;
+static int rdma_rd_poll_count[MAX_RDMA_RD] = { 0 };
+static int delay = 0;
+static int buf_len = RDMA_BUFFER_SIZE;
+static int use_cno = 0;
+static int recv_msg_index = 0;
+static int burst_msg_posted = 0;
+static int burst_msg_index = 0;
+static int ucm = 0;
+static DAT_SOCK_ADDR6 remote;
+
+/* forward prototypes */
+const char *DT_RetToStr(DAT_RETURN ret_value);
+const char *DT_EventToStr(DAT_EVENT_NUMBER event_code);
+void print_usage(void);
+double get_time(void);
+void init_data(void);
+
+DAT_RETURN send_msg(void *data,
+ DAT_COUNT size,
+ DAT_LMR_CONTEXT context,
+ DAT_DTO_COOKIE cookie, DAT_COMPLETION_FLAGS flags);
+
+DAT_RETURN connect_ep(char *hostname, DAT_CONN_QUAL conn_id);
+void disconnect_ep(void);
+DAT_RETURN register_rdma_memory(void);
+DAT_RETURN unregister_rdma_memory(void);
+DAT_RETURN create_events(void);
+DAT_RETURN destroy_events(void);
+DAT_RETURN do_rdma_write_with_msg(void);
+DAT_RETURN do_rdma_read_with_msg(void);
+DAT_RETURN do_ping_pong_msg(void);
+
+#define LOGPRINTF if (verbose) printf
+
+void flush_evds(void)
+{
+ DAT_EVENT event;
+
+ /* Flush async error queue */
+ printf("%d ERR: Checking ASYNC EVD...\n", getpid());
+ while (dat_evd_dequeue(h_async_evd, &event) == DAT_SUCCESS) {
+ printf(" ASYNC EVD ENTRY: handle=%p reason=%d\n",
+ event.event_data.asynch_error_event_data.dat_handle,
+ event.event_data.asynch_error_event_data.reason);
+ }
+ /* Flush receive queue */
+ printf("%d ERR: Checking RECEIVE EVD...\n", getpid());
+ while (dat_evd_dequeue(h_dto_rcv_evd, &event) == DAT_SUCCESS) {
+ printf(" RCV EVD ENTRY: op=%d stat=%d ln=%d ck="F64x"\n",
+ event.event_data.dto_completion_event_data.operation,
+ event.event_data.dto_completion_event_data.status,
+ event.event_data.dto_completion_event_data.transfered_length,
+ event.event_data.dto_completion_event_data.user_cookie.as_64);
+ }
+ /* Flush request queue */
+ printf("%d ERR: Checking REQUEST EVD...\n", getpid());
+ while (dat_evd_dequeue(h_dto_req_evd, &event) == DAT_SUCCESS) {
+ printf(" REQ EVD ENTRY: op=%d stat=%d ln=%d ck="F64x"\n",
+ event.event_data.dto_completion_event_data.operation,
+ event.event_data.dto_completion_event_data.status,
+ event.event_data.dto_completion_event_data.transfered_length,
+ event.event_data.dto_completion_event_data.user_cookie.as_64);
+ }
+}
+
+
+static inline DAT_RETURN
+collect_event(DAT_EVD_HANDLE dto_evd,
+ DAT_EVENT *event,
+ DAT_TIMEOUT timeout,
+ int *counter)
+{
+ DAT_EVD_HANDLE evd = DAT_HANDLE_NULL;
+ DAT_COUNT nmore;
+ DAT_RETURN ret = DAT_SUCCESS;
+
+ if (use_cno) {
+retry:
+ /* CNO wait could return EVD's in any order and
+ * may drop some EVD notification's if already
+ * triggered. Once woken, simply dequeue the
+ * Evd the caller wants to collect and return.
+ * If notification without EVD, retry.
+ */
+ ret = dat_cno_wait(h_dto_cno, CNO_TIMEOUT, &evd);
+ if (dat_evd_dequeue(dto_evd, event) != DAT_SUCCESS) {
+ if (ret == DAT_SUCCESS)
+ printf(" WARNING: CNO notification:"
+ " without EVD?\n");
+ goto retry;
+ }
+ ret = DAT_SUCCESS; /* cno timed out, but EVD dequeued */
+
+ } else if (!polling) {
+
+ /* use wait to dequeue */
+ ret = dat_evd_wait(dto_evd, timeout, 1, event, &nmore);
+ if (ret != DAT_SUCCESS)
+ fprintf(stderr,
+ "Error waiting on h_dto_evd %p: %s\n",
+ dto_evd, DT_RetToStr(ret));
+
+ } else {
+ while (dat_evd_dequeue(dto_evd, event) == DAT_QUEUE_EMPTY)
+ if (counter)
+ (*counter)++;
+ }
+ return (ret);
+}
+
+static void print_ia_address(struct sockaddr *sa)
+{
+ char str[INET6_ADDRSTRLEN] = {" ??? "};
+
+ switch(sa->sa_family) {
+ case AF_INET:
+ inet_ntop(AF_INET, &((struct sockaddr_in *)sa)->sin_addr, str, INET6_ADDRSTRLEN);
+ printf("%d Local Address AF_INET - %s port %d\n", getpid(), str, SERVER_CONN_QUAL);
+ break;
+ case AF_INET6:
+ inet_ntop(AF_INET6, &((struct sockaddr_in6 *)sa)->sin6_addr, str, INET6_ADDRSTRLEN);
+ printf("%d Local Address AF_INET6 - %s flowinfo(QPN)=0x%x, port(LID)=0x%x\n",
+ getpid(), str,
+ ntohl(((struct sockaddr_in6 *)sa)->sin6_flowinfo),
+ ntohs(((struct sockaddr_in6 *)sa)->sin6_port));
+ break;
+ default:
+ printf("%d Local Address UNKOWN FAMILY - port %d\n", getpid(), SERVER_CONN_QUAL);
+ }
+}
+
+int main(int argc, char **argv)
+{
+ int i, c;
+ DAT_RETURN ret;
+ DAT_EP_PARAM ep_param;
+ DAT_IA_ATTR ia_attr;
+
+ /* parse arguments */
+ while ((c = getopt(argc, argv, "tscvpq:l:b:d:B:h:P:")) != -1) {
+ switch (c) {
+ case 't':
+ performance_times = 1;
+ fflush(stdout);
+ break;
+ case 's':
+ server = 1;
+ fflush(stdout);
+ break;
+ case 'c':
+ use_cno = 1;
+ printf("%d Creating CNO for DTO EVD's\n", getpid());
+ fflush(stdout);
+ break;
+ case 'v':
+ verbose = 1;
+ printf("%d Verbose\n", getpid());
+ fflush(stdout);
+ break;
+ case 'p':
+ polling = 1;
+ printf("%d Polling\n", getpid());
+ fflush(stdout);
+ break;
+ case 'q':
+ /* map UCM qpn into AF_INET6 sin6_flowinfo */
+ remote.sin6_family = AF_INET6;
+ remote.sin6_flowinfo = htonl(strtol(optarg,NULL,0));
+ ucm = 1;
+ server = 0;
+ break;
+ case 'l':
+ /* map UCM lid into AF_INET6 sin6_port */
+ remote.sin6_family = AF_INET6;
+ remote.sin6_port = htons(strtol(optarg,NULL,0));
+ ucm = 1;
+ server = 0;
+ break;
+ case 'B':
+ burst = atoi(optarg);
+ break;
+ case 'd':
+ delay = atoi(optarg);
+ break;
+ case 'b':
+ buf_len = atoi(optarg);
+ break;
+ case 'h':
+ server = 0;
+ strcpy(hostname, optarg);
+ break;
+ case 'P':
+ strcpy(provider, optarg);
+ break;
+ default:
+ print_usage();
+ exit(-12);
+ }
+ }
+
+#if defined(_WIN32) || defined(_WIN64)
+ {
+ WSADATA wsaData;
+
+ i = WSAStartup(MAKEWORD(2, 2), &wsaData);
+ if (i != 0) {
+ printf("%s WSAStartup(2.2) failed? (0x%x)\n", argv[0],
+ i);
+ fflush(stdout);
+ exit(1);
+ }
+ }
+#endif
+
+ if (!server) {
+ printf("%d Running as client - %s\n", getpid(), provider);
+ } else {
+ printf("%d Running as server - %s\n", getpid(), provider);
+ }
+ fflush(stdout);
+
+ /* allocate send and receive buffers */
+ if (((rbuf = malloc(buf_len * (burst+1))) == NULL) ||
+ ((sbuf = malloc(buf_len * (burst+1))) == NULL)) {
+ perror("malloc");
+ exit(1);
+ }
+ memset(&ts, 0, sizeof(struct dt_time));
+ LOGPRINTF("%d Allocated RDMA buffers (r:%p,s:%p) len %d \n",
+ getpid(), rbuf, sbuf, buf_len);
+
+ /* dat_ia_open, dat_pz_create */
+ h_async_evd = DAT_HANDLE_NULL;
+ start = get_time();
+ ret = dat_ia_open(provider, 8, &h_async_evd, &h_ia);
+ stop = get_time();
+ ts.open += ((stop - start) * 1.0e6);
+ if (ret != DAT_SUCCESS) {
+ fprintf(stderr, "%d: Error Adaptor open: %s\n",
+ getpid(), DT_RetToStr(ret));
+ exit(1);
+ } else
+ LOGPRINTF("%d Opened Interface Adaptor\n", getpid());
+
+ ret = dat_ia_query(h_ia, 0, DAT_IA_FIELD_ALL, &ia_attr, 0, 0);
+ if (ret != DAT_SUCCESS) {
+ fprintf(stderr, "%d: Error Adaptor query: %s\n",
+ getpid(), DT_RetToStr(ret));
+ exit(1);
+ }
+ print_ia_address(ia_attr.ia_address_ptr);
+
+ /* Create Protection Zone */
+ start = get_time();
+ LOGPRINTF("%d Create Protection Zone\n", getpid());
+ ret = dat_pz_create(h_ia, &h_pz);
+ stop = get_time();
+ ts.pzc += ((stop - start) * 1.0e6);
+ if (ret != DAT_SUCCESS) {
+ fprintf(stderr, "%d Error creating Protection Zone: %s\n",
+ getpid(), DT_RetToStr(ret));
+ exit(1);
+ } else
+ LOGPRINTF("%d Created Protection Zone\n", getpid());
+
+ /* Register memory */
+ LOGPRINTF("%d Register RDMA memory\n", getpid());
+ ret = register_rdma_memory();
+ if (ret != DAT_SUCCESS) {
+ fprintf(stderr, "%d Error registering RDMA memory: %s\n",
+ getpid(), DT_RetToStr(ret));
+ goto cleanup;
+ } else
+ LOGPRINTF("%d Register RDMA memory done\n", getpid());
+
+ LOGPRINTF("%d Create events\n", getpid());
+ ret = create_events();
+ if (ret != DAT_SUCCESS) {
+ fprintf(stderr, "%d Error creating events: %s\n",
+ getpid(), DT_RetToStr(ret));
+ goto cleanup;
+ } else {
+ LOGPRINTF("%d Create events done\n", getpid());
+ }
+
+ /* create EP */
+ memset(&ep_attr, 0, sizeof(ep_attr));
+ ep_attr.service_type = DAT_SERVICE_TYPE_RC;
+ ep_attr.max_rdma_size = 0x10000;
+ ep_attr.qos = 0;
+ ep_attr.recv_completion_flags = 0;
+ ep_attr.max_recv_dtos = MSG_BUF_COUNT + (burst * 3);
+ ep_attr.max_request_dtos = MSG_BUF_COUNT + (burst * 3) + MAX_RDMA_RD;
+ ep_attr.max_recv_iov = MSG_IOV_COUNT;
+ ep_attr.max_request_iov = MSG_IOV_COUNT;
+ ep_attr.max_rdma_read_in = MAX_RDMA_RD;
+ ep_attr.max_rdma_read_out = MAX_RDMA_RD;
+ ep_attr.request_completion_flags = DAT_COMPLETION_DEFAULT_FLAG;
+ ep_attr.ep_transport_specific_count = 0;
+ ep_attr.ep_transport_specific = NULL;
+ ep_attr.ep_provider_specific_count = 0;
+ ep_attr.ep_provider_specific = NULL;
+
+ start = get_time();
+ ret = dat_ep_create(h_ia, h_pz, h_dto_rcv_evd,
+ h_dto_req_evd, h_conn_evd, &ep_attr, &h_ep);
+ stop = get_time();
+ ts.epc += ((stop - start) * 1.0e6);
+ ts.total += ts.epc;
+ if (ret != DAT_SUCCESS) {
+ fprintf(stderr, "%d Error dat_ep_create: %s\n",
+ getpid(), DT_RetToStr(ret));
+ goto cleanup;
+ } else
+ LOGPRINTF("%d EP created %p \n", getpid(), h_ep);
+
+ /*
+ * register message buffers, establish connection, and
+ * exchange DMA RMR information info via messages
+ */
+ ret = connect_ep(hostname, SERVER_CONN_QUAL);
+ if (ret != DAT_SUCCESS) {
+ fprintf(stderr, "%d Error connect_ep: %s\n",
+ getpid(), DT_RetToStr(ret));
+ goto cleanup;
+ } else
+ LOGPRINTF("%d connect_ep complete\n", getpid());
+
+ /* Query EP for local and remote address information, print */
+ ret = dat_ep_query(h_ep, DAT_EP_FIELD_ALL, &ep_param);
+ if (ret != DAT_SUCCESS) {
+ fprintf(stderr, "%d Error dat_ep_query: %s\n",
+ getpid(), DT_RetToStr(ret));
+ goto cleanup;
+ } else
+ LOGPRINTF("%d EP queried %p \n", getpid(), h_ep);
+#if defined(_WIN32)
+ printf("\n%d Query EP: LOCAL addr %s port %lld\n", getpid(),
+ inet_ntoa(((struct sockaddr_in *)
+ ep_param.local_ia_address_ptr)->sin_addr),
+ (ep_param.local_port_qual));
+#else
+ inet_ntop(AF_INET,
+ &((struct sockaddr_in *)ep_param.local_ia_address_ptr)->
+ sin_addr, addr_str, sizeof(addr_str));
+ printf("\n%d Query EP: LOCAL addr %s port " F64x "\n", getpid(),
+ addr_str, (ep_param.local_port_qual));
+#endif
+#if defined(_WIN32)
+ printf("%d Query EP: REMOTE addr %s port %lld\n", getpid(),
+ inet_ntoa(((struct sockaddr_in *)
+ ep_param.local_ia_address_ptr)->sin_addr),
+ (ep_param.remote_port_qual));
+#else
+ inet_ntop(AF_INET,
+ &((struct sockaddr_in *)ep_param.remote_ia_address_ptr)->
+ sin_addr, addr_str, sizeof(addr_str));
+ printf("%d Query EP: REMOTE addr %s port " F64x "\n", getpid(),
+ addr_str, (ep_param.remote_port_qual));
+#endif
+ fflush(stdout);
+
+#if CONNECT_ONLY
+#if defined(_WIN32) || defined(_WIN64)
+ Sleep(1 * 1000);
+#else
+ sleep(1);
+#endif
+ goto cleanup;
+#endif
+
+ /*********** RDMA write data *************/
+ ret = do_rdma_write_with_msg();
+ if (ret != DAT_SUCCESS) {
+ fprintf(stderr, "%d Error do_rdma_write_with_msg: %s\n",
+ getpid(), DT_RetToStr(ret));
+ goto cleanup;
+ } else
+ LOGPRINTF("%d do_rdma_write_with_msg complete\n", getpid());
+
+ /*********** RDMA read data *************/
+ ret = do_rdma_read_with_msg();
+ if (ret != DAT_SUCCESS) {
+ fprintf(stderr, "%d Error do_rdma_read_with_msg: %s\n",
+ getpid(), DT_RetToStr(ret));
+ goto cleanup;
+ } else
+ LOGPRINTF("%d do_rdma_read_with_msg complete\n", getpid());
+
+ /*********** PING PING messages ************/
+ ret = do_ping_pong_msg();
+ if (ret != DAT_SUCCESS) {
+ fprintf(stderr, "%d Error do_ping_pong_msg: %s\n",
+ getpid(), DT_RetToStr(ret));
+ goto cleanup;
+ } else {
+ LOGPRINTF("%d do_ping_pong_msg complete\n", getpid());
+ goto complete;
+ }
+
+cleanup:
+ flush_evds();
+ failed++;
+complete:
+
+ /* disconnect and free EP resources */
+ if (h_ep != DAT_HANDLE_NULL) {
+ /* unregister message buffers and tear down connection */
+ LOGPRINTF("%d Disconnect and Free EP %p \n", getpid(), h_ep);
+ disconnect_ep();
+
+ /* free EP */
+ LOGPRINTF("%d Free EP %p \n", getpid(), h_ep);
+ start = get_time();
+ ret = dat_ep_free(h_ep);
+ stop = get_time();
+ ts.epf += ((stop - start) * 1.0e6);
+ ts.total += ts.epf;
+ if (ret != DAT_SUCCESS) {
+ fprintf(stderr, "%d Error freeing EP: %s\n",
+ getpid(), DT_RetToStr(ret));
+ } else {
+ LOGPRINTF("%d Freed EP\n", getpid());
+ h_ep = DAT_HANDLE_NULL;
+ }
+ }
+
+ /* free EVDs */
+ LOGPRINTF("%d destroy events\n", getpid());
+ ret = destroy_events();
+ if (ret != DAT_SUCCESS)
+ fprintf(stderr, "%d Error destroy_events: %s\n",
+ getpid(), DT_RetToStr(ret));
+ else
+ LOGPRINTF("%d destroy events done\n", getpid());
+
+ ret = unregister_rdma_memory();
+ LOGPRINTF("%d unregister_rdma_memory \n", getpid());
+ if (ret != DAT_SUCCESS)
+ fprintf(stderr, "%d Error unregister_rdma_memory: %s\n",
+ getpid(), DT_RetToStr(ret));
+ else
+ LOGPRINTF("%d unregister_rdma_memory done\n", getpid());
+
+ /* Free protection domain */
+ LOGPRINTF("%d Freeing pz\n", getpid());
+ start = get_time();
+ ret = dat_pz_free(h_pz);
+ stop = get_time();
+ ts.pzf += ((stop - start) * 1.0e6);
+ if (ret != DAT_SUCCESS) {
+ fprintf(stderr, "%d Error freeing PZ: %s\n",
+ getpid(), DT_RetToStr(ret));
+ } else {
+ LOGPRINTF("%d Freed pz\n", getpid());
+ h_pz = NULL;
+ }
+
+ /* close the device */
+ LOGPRINTF("%d Closing Interface Adaptor\n", getpid());
+ start = get_time();
+ ret = dat_ia_close(h_ia, DAT_CLOSE_ABRUPT_FLAG);
+ stop = get_time();
+ ts.close += ((stop - start) * 1.0e6);
+ if (ret != DAT_SUCCESS) {
+ fprintf(stderr, "%d: Error Adaptor close: %s\n",
+ getpid(), DT_RetToStr(ret));
+ } else
+ LOGPRINTF("%d Closed Interface Adaptor\n", getpid());
+
+ /* free rdma buffers */
+ free(rbuf);
+ free(sbuf);
+
+ printf("\n%d: DAPL Test Complete. %s\n\n",
+ getpid(), failed ? "FAILED" : "PASSED");
+
+ fflush(stderr);
+ fflush(stdout);
+
+ if (!performance_times)
+ exit(0);
+
+ printf("\n%d: DAPL Test Complete.\n\n", getpid());
+ printf("%d: Message RTT: Total=%10.2lf usec, %d bursts, itime=%10.2lf"
+ " usec, pc=%d\n",
+ getpid(), ts.rtt, burst, ts.rtt / burst, poll_count);
+ printf("%d: RDMA write: Total=%10.2lf usec, %d bursts, itime=%10.2lf"
+ " usec, pc=%d\n",
+ getpid(), ts.rdma_wr, burst,
+ ts.rdma_wr / burst, rdma_wr_poll_count);
+ for (i = 0; i < MAX_RDMA_RD; i++) {
+ printf("%d: RDMA read: Total=%10.2lf usec, %d bursts, "
+ "itime=%10.2lf usec, pc=%d\n",
+ getpid(), ts.rdma_rd_total, MAX_RDMA_RD,
+ ts.rdma_rd[i], rdma_rd_poll_count[i]);
+ }
+ printf("%d: open: %10.2lf usec\n", getpid(), ts.open);
+ printf("%d: close: %10.2lf usec\n", getpid(), ts.close);
+ printf("%d: PZ create: %10.2lf usec\n", getpid(), ts.pzc);
+ printf("%d: PZ free: %10.2lf usec\n", getpid(), ts.pzf);
+ printf("%d: LMR create:%10.2lf usec\n", getpid(), ts.reg);
+ printf("%d: LMR free: %10.2lf usec\n", getpid(), ts.unreg);
+ printf("%d: EVD create:%10.2lf usec\n", getpid(), ts.evdc);
+ printf("%d: EVD free: %10.2lf usec\n", getpid(), ts.evdf);
+ if (use_cno) {
+ printf("%d: CNO create: %10.2lf usec\n", getpid(), ts.cnoc);
+ printf("%d: CNO free: %10.2lf usec\n", getpid(), ts.cnof);
+ }
+ printf("%d: EP create: %10.2lf usec\n", getpid(), ts.epc);
+ printf("%d: EP free: %10.2lf usec\n", getpid(), ts.epf);
+ if (!server)
+ printf("%d: connect: %10.2lf usec, poll_cnt=%d\n",
+ getpid(), ts.conn, conn_poll_count);
+ printf("%d: TOTAL: %10.2lf usec\n", getpid(), ts.total);
+
+#if defined(_WIN32) || defined(_WIN64)
+ WSACleanup();
+#endif
+ return (0);
+}
+
+double get_time(void)
+{
+ struct timeval tp;
+
+ gettimeofday(&tp, NULL);
+ return ((double)tp.tv_sec + (double)tp.tv_usec * 1e-6);
+}
+
+void init_data(void)
+{
+ memset(rbuf, 'a', buf_len);
+ memset(sbuf, 'b', buf_len);
+}
+
+DAT_RETURN
+send_msg(void *data,
+ DAT_COUNT size,
+ DAT_LMR_CONTEXT context,
+ DAT_DTO_COOKIE cookie, DAT_COMPLETION_FLAGS flags)
+{
+ DAT_LMR_TRIPLET iov;
+ DAT_EVENT event;
+ DAT_RETURN ret;
+
+ iov.lmr_context = context;
+#if defined(_WIN32)
+ iov.virtual_address = (DAT_VADDR) data;
+#else
+ iov.virtual_address = (DAT_VADDR) (unsigned long)data;
+#endif
+ iov.segment_length = size;
+
+ LOGPRINTF("%d calling post_send\n", getpid());
+ cookie.as_64 = 0xaaaa;
+ ret = dat_ep_post_send(h_ep, 1, &iov, cookie, flags);
+
+ if (ret != DAT_SUCCESS) {
+ fprintf(stderr, "%d: ERROR: dat_ep_post_send() %s\n",
+ getpid(), DT_RetToStr(ret));
+ return ret;
+ }
+
+ if (!(flags & DAT_COMPLETION_SUPPRESS_FLAG)) {
+
+ if (collect_event(h_dto_req_evd,
+ &event,
+ DTO_TIMEOUT,
+ &poll_count) != DAT_SUCCESS)
+ return (DAT_ABORT);
+
+ /* validate event number, len, cookie, and status */
+ if (event.event_number != DAT_DTO_COMPLETION_EVENT) {
+ fprintf(stderr, "%d: ERROR: DTO event number %s\n",
+ getpid(),
+ DT_EventToStr(event.event_number));
+ return (DAT_ABORT);
+ }
+
+ if ((event.event_data.dto_completion_event_data.
+ transfered_length != size)
+ || (event.event_data.dto_completion_event_data.user_cookie.
+ as_64 != 0xaaaa)) {
+ fprintf(stderr,
+ "%d: ERROR: DTO len %d or cookie " F64x " \n",
+ getpid(),
+ event.event_data.dto_completion_event_data.
+ transfered_length,
+ event.event_data.dto_completion_event_data.
+ user_cookie.as_64);
+ return (DAT_ABORT);
+
+ }
+ if (event.event_data.dto_completion_event_data.status !=
+ DAT_SUCCESS) {
+ fprintf(stderr, "%d: ERROR: DTO event status %s\n",
+ getpid(), DT_RetToStr(ret));
+ return (DAT_ABORT);
+ }
+ }
+
+ return DAT_SUCCESS;
+}
+
+DAT_RETURN connect_ep(char *hostname, DAT_CONN_QUAL conn_id)
+{
+ DAT_IA_ADDRESS_PTR remote_addr = (DAT_IA_ADDRESS_PTR)&remote;
+ DAT_RETURN ret;
+ DAT_REGION_DESCRIPTION region;
+ DAT_EVENT event;
+ DAT_COUNT nmore;
+ DAT_LMR_TRIPLET l_iov;
+ DAT_RMR_TRIPLET r_iov;
+ DAT_DTO_COOKIE cookie;
+ int i;
+ unsigned char *buf;
+ DAT_CR_PARAM cr_param = { 0 };
+ unsigned char pdata[48] = { 0 };
+
+ /* Register send message buffer */
+ LOGPRINTF("%d Registering send Message Buffer %p, len %d\n",
+ getpid(), &rmr_send_msg, (int)sizeof(DAT_RMR_TRIPLET));
+ region.for_va = &rmr_send_msg;
+ ret = dat_lmr_create(h_ia,
+ DAT_MEM_TYPE_VIRTUAL,
+ region,
+ sizeof(DAT_RMR_TRIPLET),
+ h_pz,
+ DAT_MEM_PRIV_LOCAL_WRITE_FLAG,
+ DAT_VA_TYPE_VA,
+ &h_lmr_send_msg,
+ &lmr_context_send_msg,
+ &rmr_context_send_msg,
+ ®istered_size_send_msg,
+ ®istered_addr_send_msg);
+
+ if (ret != DAT_SUCCESS) {
+ fprintf(stderr, "%d Error registering send msg buffer: %s\n",
+ getpid(), DT_RetToStr(ret));
+ return (ret);
+ } else
+ LOGPRINTF("%d Registered send Message Buffer %p \n",
+ getpid(), region.for_va);
+
+ /* Register Receive buffers */
+ LOGPRINTF("%d Registering Receive Message Buffer %p\n",
+ getpid(), rmr_recv_msg);
+ region.for_va = rmr_recv_msg;
+ ret = dat_lmr_create(h_ia,
+ DAT_MEM_TYPE_VIRTUAL,
+ region,
+ sizeof(DAT_RMR_TRIPLET) * MSG_BUF_COUNT,
+ h_pz,
+ DAT_MEM_PRIV_LOCAL_WRITE_FLAG,
+ DAT_VA_TYPE_VA,
+ &h_lmr_recv_msg,
+ &lmr_context_recv_msg,
+ &rmr_context_recv_msg,
+ ®istered_size_recv_msg,
+ ®istered_addr_recv_msg);
+ if (ret != DAT_SUCCESS) {
+ fprintf(stderr, "%d Error registering recv msg buffer: %s\n",
+ getpid(), DT_RetToStr(ret));
+ return (ret);
+ } else
+ LOGPRINTF("%d Registered Receive Message Buffer %p\n",
+ getpid(), region.for_va);
+
+ for (i = 0; i < MSG_BUF_COUNT; i++) {
+ cookie.as_64 = i;
+ l_iov.lmr_context = lmr_context_recv_msg;
+#if defined(_WIN32)
+ l_iov.virtual_address = (DAT_VADDR) & rmr_recv_msg[i];
+#else
+ l_iov.virtual_address =
+ (DAT_VADDR) (unsigned long)&rmr_recv_msg[i];
+#endif
+ l_iov.segment_length = sizeof(DAT_RMR_TRIPLET);
+
+ LOGPRINTF("%d Posting Receive Message Buffer %p\n",
+ getpid(), &rmr_recv_msg[i]);
+ ret = dat_ep_post_recv(h_ep,
+ 1,
+ &l_iov,
+ cookie, DAT_COMPLETION_DEFAULT_FLAG);
+
+ if (ret != DAT_SUCCESS) {
+ fprintf(stderr,
+ "%d Error registering recv msg buffer: %s\n",
+ getpid(), DT_RetToStr(ret));
+ return (ret);
+ } else
+ LOGPRINTF("%d Registered Receive Message Buffer %p\n",
+ getpid(), region.for_va);
+
+ }
+
+ /* setup receive rdma buffer to initial string to be overwritten */
+ strcpy((char *)rbuf, "blah, blah, blah\n");
+
+ /* clear event structure */
+ memset(&event, 0, sizeof(DAT_EVENT));
+
+ if (server) { /* SERVER */
+
+ /* create the service point for server listen */
+ LOGPRINTF("%d Creating service point for listen\n", getpid());
+ ret = dat_psp_create(h_ia,
+ conn_id,
+ h_cr_evd, DAT_PSP_CONSUMER_FLAG, &h_psp);
+ if (ret != DAT_SUCCESS) {
+ fprintf(stderr, "%d Error dat_psp_create: %s\n",
+ getpid(), DT_RetToStr(ret));
+ return (ret);
+ } else
+ LOGPRINTF("%d dat_psp_created for server listen\n",
+ getpid());
+
+ printf("%d Server waiting for connect request on port " F64x
+ "\n", getpid(), conn_id);
+
+ ret = dat_evd_wait(h_cr_evd, SERVER_TIMEOUT, 1, &event, &nmore);
+ if (ret != DAT_SUCCESS) {
+ fprintf(stderr, "%d Error dat_evd_wait: %s\n",
+ getpid(), DT_RetToStr(ret));
+ return (ret);
+ } else
+ LOGPRINTF("%d dat_evd_wait for cr_evd completed\n",
+ getpid());
+
+ if (event.event_number != DAT_CONNECTION_REQUEST_EVENT) {
+ fprintf(stderr, "%d Error unexpected cr event : %s\n",
+ getpid(),
+ DT_EventToStr(event.event_number));
+ return (DAT_ABORT);
+ }
+ if ((event.event_data.cr_arrival_event_data.conn_qual !=
+ SERVER_CONN_QUAL)
+ || (event.event_data.cr_arrival_event_data.sp_handle.
+ psp_handle != h_psp)) {
+ fprintf(stderr, "%d Error wrong cr event data : %s\n",
+ getpid(),
+ DT_EventToStr(event.event_number));
+ return (DAT_ABORT);
+ }
+
+ /* use to test rdma_cma timeout logic */
+#if defined(_WIN32) || defined(_WIN64)
+ if (delay)
+ Sleep(delay * 1000);
+#else
+ if (delay)
+ sleep(delay);
+#endif
+
+ /* accept connect request from client */
+ h_cr = event.event_data.cr_arrival_event_data.cr_handle;
+ LOGPRINTF("%d Accepting connect request from client\n",
+ getpid());
+
+ /* private data - check and send it back */
+ dat_cr_query(h_cr, DAT_CSP_FIELD_ALL, &cr_param);
+
+ buf = (unsigned char *)cr_param.private_data;
+ LOGPRINTF("%d CONN REQUEST Private Data %p[0]=%d [47]=%d\n",
+ getpid(), buf, buf[0], buf[47]);
+ for (i = 0; i < 48; i++) {
+ if (buf[i] != i + 1) {
+ fprintf(stderr, "%d Error with CONNECT REQUEST"
+ " private data: %p[%d]=%d s/be %d\n",
+ getpid(), buf, i, buf[i], i + 1);
+ dat_cr_reject(h_cr, 0, NULL);
+ return (DAT_ABORT);
+ }
+ buf[i]++; /* change for trip back */
+ }
+
+#ifdef TEST_REJECT_WITH_PRIVATE_DATA
+ printf("%d REJECT request with 48 bytes of private data\n",
+ getpid());
+ ret = dat_cr_reject(h_cr, 48, cr_param.private_data);
+ printf("\n%d: DAPL Test Complete. %s\n\n",
+ getpid(), ret ? "FAILED" : "PASSED");
+ exit(0);
+#endif
+
+ ret = dat_cr_accept(h_cr, h_ep, 48, cr_param.private_data);
+
+ if (ret != DAT_SUCCESS) {
+ fprintf(stderr, "%d Error dat_cr_accept: %s\n",
+ getpid(), DT_RetToStr(ret));
+ return (ret);
+ } else
+ LOGPRINTF("%d dat_cr_accept completed\n", getpid());
+ } else { /* CLIENT */
+ struct addrinfo *target;
+ int rval;
+
+ if (ucm)
+ goto no_resolution;
+
+#if defined(_WIN32) || defined(_WIN64)
+ if ((rval = getaddrinfo(hostname, "ftp", NULL, &target)) != 0) {
+ printf("\n remote name resolution failed! %s\n",
+ gai_strerror(rval));
+ exit(1);
+ }
+ rval = ((struct sockaddr_in *)target->ai_addr)->sin_addr.s_addr;
+#else
+ if (getaddrinfo(hostname, NULL, NULL, &target) != 0) {
+ perror("\n remote name resolution failed!");
+ exit(1);
+ }
+ rval = ((struct sockaddr_in *)target->ai_addr)->sin_addr.s_addr;
+#endif
+ printf("%d Server Name: %s \n", getpid(), hostname);
+ printf("%d Server Net Address: %d.%d.%d.%d port " F64x "\n",
+ getpid(), (rval >> 0) & 0xff, (rval >> 8) & 0xff,
+ (rval >> 16) & 0xff, (rval >> 24) & 0xff, conn_id);
+
+ remote_addr = (DAT_IA_ADDRESS_PTR)target->ai_addr; /* IP */
+no_resolution:
+ for (i = 0; i < 48; i++) /* simple pattern in private data */
+ pdata[i] = i + 1;
+
+ LOGPRINTF("%d Connecting to server\n", getpid());
+ start = get_time();
+ ret = dat_ep_connect(h_ep,
+ remote_addr,
+ conn_id,
+ CONN_TIMEOUT,
+ 48,
+ (DAT_PVOID) pdata,
+ 0, DAT_CONNECT_DEFAULT_FLAG);
+ if (ret != DAT_SUCCESS) {
+ fprintf(stderr, "%d Error dat_ep_connect: %s\n",
+ getpid(), DT_RetToStr(ret));
+ return (ret);
+ } else
+ LOGPRINTF("%d dat_ep_connect completed\n", getpid());
+
+ if (!ucm)
+ freeaddrinfo(target);
+ }
+
+ printf("%d Waiting for connect response\n", getpid());
+
+ if (polling)
+ while (DAT_GET_TYPE(dat_evd_dequeue(h_conn_evd, &event)) ==
+ DAT_QUEUE_EMPTY)
+ conn_poll_count++;
+ else
+ ret = dat_evd_wait(h_conn_evd, DAT_TIMEOUT_INFINITE,
+ 1, &event, &nmore);
+
+ if (!server) {
+ stop = get_time();
+ ts.conn += ((stop - start) * 1.0e6);
+ }
+
+#ifdef TEST_REJECT_WITH_PRIVATE_DATA
+ if (event.event_number != DAT_CONNECTION_EVENT_PEER_REJECTED) {
+ fprintf(stderr, "%d expected conn reject event : %s\n",
+ getpid(), DT_EventToStr(event.event_number));
+ return (DAT_ABORT);
+ }
+ /* get the reject private data and validate */
+ buf = (unsigned char *)event.event_data.connect_event_data.private_data;
+ printf("%d Received REJECT with private data %p[0]=%d [47]=%d\n",
+ getpid(), buf, buf[0], buf[47]);
+ for (i = 0; i < 48; i++) {
+ if (buf[i] != i + 2) {
+ fprintf(stderr, "%d client: Error with REJECT event"
+ " private data: %p[%d]=%d s/be %d\n",
+ getpid(), buf, i, buf[i], i + 2);
+ dat_ep_disconnect(h_ep, DAT_CLOSE_ABRUPT_FLAG);
+ return (DAT_ABORT);
+ }
+ }
+ printf("\n%d: DAPL Test Complete. PASSED\n\n", getpid());
+ exit(0);
+#endif
+
+ if (event.event_number != DAT_CONNECTION_EVENT_ESTABLISHED) {
+ fprintf(stderr, "%d Error unexpected conn event : 0x%x %s\n",
+ getpid(), event.event_number,
+ DT_EventToStr(event.event_number));
+ return (DAT_ABORT);
+ }
+
+ /* check private data back from server */
+ if (!server) {
+ buf =
+ (unsigned char *)event.event_data.connect_event_data.
+ private_data;
+ LOGPRINTF("%d CONN Private Data %p[0]=%d [47]=%d\n", getpid(),
+ buf, buf[0], buf[47]);
+ for (i = 0; i < 48; i++) {
+ if (buf[i] != i + 2) {
+ fprintf(stderr, "%d Error with CONNECT event"
+ " private data: %p[%d]=%d s/be %d\n",
+ getpid(), buf, i, buf[i], i + 2);
+ dat_ep_disconnect(h_ep, DAT_CLOSE_ABRUPT_FLAG);
+ LOGPRINTF
+ ("%d waiting for disconnect event...\n",
+ getpid());
+ dat_evd_wait(h_conn_evd, DAT_TIMEOUT_INFINITE,
+ 1, &event, &nmore);
+ return (DAT_ABORT);
+ }
+ }
+ }
+
+ printf("\n%d CONNECTED!\n\n", getpid());
+ connected = 1;
+
+#if CONNECT_ONLY
+ return 0;
+#endif
+
+ /*
+ * Setup our remote memory and tell the other side about it
+ */
+ rmr_send_msg.virtual_address = htonll((DAT_VADDR) (uintptr_t) rbuf);
+ rmr_send_msg.segment_length = htonl(RDMA_BUFFER_SIZE);
+ rmr_send_msg.rmr_context = htonl(rmr_context_recv);
+
+ printf("%d Send RMR msg to remote: r_key_ctx=0x%x,va=%p,len=0x%x\n",
+ getpid(), rmr_context_recv, rbuf, RDMA_BUFFER_SIZE);
+
+ ret = send_msg(&rmr_send_msg,
+ sizeof(DAT_RMR_TRIPLET),
+ lmr_context_send_msg,
+ cookie, DAT_COMPLETION_SUPPRESS_FLAG);
+
+ if (ret != DAT_SUCCESS) {
+ fprintf(stderr, "%d Error send_msg: %s\n",
+ getpid(), DT_RetToStr(ret));
+ return (ret);
+ } else
+ LOGPRINTF("%d send_msg completed\n", getpid());
+
+ /*
+ * Wait for remote RMR information for RDMA
+ */
+ if (collect_event(h_dto_rcv_evd,
+ &event,
+ DTO_TIMEOUT,
+ &poll_count) != DAT_SUCCESS)
+ return (DAT_ABORT);
+
+ printf("%d remote RMR data arrived!\n", getpid());
+
+ if (event.event_number != DAT_DTO_COMPLETION_EVENT) {
+ fprintf(stderr, "%d Error unexpected DTO event : %s\n",
+ getpid(), DT_EventToStr(event.event_number));
+ return (DAT_ABORT);
+ }
+ if ((event.event_data.dto_completion_event_data.transfered_length !=
+ sizeof(DAT_RMR_TRIPLET)) ||
+ (event.event_data.dto_completion_event_data.user_cookie.as_64 !=
+ recv_msg_index)) {
+ fprintf(stderr,
+ "ERR recv event: len=%d cookie=" F64x
+ " expected %d/%d\n",
+ (int)event.event_data.dto_completion_event_data.
+ transfered_length,
+ event.event_data.dto_completion_event_data.user_cookie.
+ as_64, (int)sizeof(DAT_RMR_TRIPLET), recv_msg_index);
+ return (DAT_ABORT);
+ }
+
+ /* swap received RMR msg: network order to host order */
+ r_iov = rmr_recv_msg[recv_msg_index];
+ rmr_recv_msg[recv_msg_index].rmr_context = ntohl(r_iov.rmr_context);
+ rmr_recv_msg[recv_msg_index].virtual_address =
+ ntohll(r_iov.virtual_address);
+ rmr_recv_msg[recv_msg_index].segment_length =
+ ntohl(r_iov.segment_length);
+
+ printf("%d Received RMR from remote: "
+ "r_iov: r_key_ctx=%x,va=" F64x ",len=0x%x\n",
+ getpid(), rmr_recv_msg[recv_msg_index].rmr_context,
+ rmr_recv_msg[recv_msg_index].virtual_address,
+ rmr_recv_msg[recv_msg_index].segment_length);
+
+ recv_msg_index++;
+
+ return (DAT_SUCCESS);
+}
+
+void disconnect_ep(void)
+{
+ DAT_RETURN ret;
+ DAT_EVENT event;
+ DAT_COUNT nmore;
+
+ if (connected) {
+
+ /*
+ * Only the client needs to call disconnect. The server _should_ be able
+ * to just wait on the EVD associated with connection events for a
+ * disconnect request and then exit.
+ */
+ if (!server) {
+ LOGPRINTF("%d dat_ep_disconnect\n", getpid());
+ ret = dat_ep_disconnect(h_ep, DAT_CLOSE_DEFAULT);
+ if (ret != DAT_SUCCESS) {
+ fprintf(stderr,
+ "%d Error dat_ep_disconnect: %s\n",
+ getpid(), DT_RetToStr(ret));
+ } else {
+ LOGPRINTF("%d dat_ep_disconnect completed\n",
+ getpid());
+ }
+ } else {
+ LOGPRINTF("%d Server waiting for disconnect...\n",
+ getpid());
+ }
+
+ ret =
+ dat_evd_wait(h_conn_evd, DAT_TIMEOUT_INFINITE, 1, &event,
+ &nmore);
+ if (ret != DAT_SUCCESS) {
+ fprintf(stderr, "%d Error dat_evd_wait: %s\n",
+ getpid(), DT_RetToStr(ret));
+ } else {
+ LOGPRINTF("%d dat_evd_wait for h_conn_evd completed\n",
+ getpid());
+ }
+ }
+
+ /* destroy service point */
+ if ((server) && (h_psp != DAT_HANDLE_NULL)) {
+ ret = dat_psp_free(h_psp);
+ if (ret != DAT_SUCCESS) {
+ fprintf(stderr, "%d Error dat_psp_free: %s\n",
+ getpid(), DT_RetToStr(ret));
+ } else {
+ LOGPRINTF("%d dat_psp_free completed\n", getpid());
+ }
+ }
+
+ /* Unregister Send message Buffer */
+ if (h_lmr_send_msg != DAT_HANDLE_NULL) {
+ LOGPRINTF("%d Unregister send message h_lmr %p \n", getpid(),
+ h_lmr_send_msg);
+ ret = dat_lmr_free(h_lmr_send_msg);
+ if (ret != DAT_SUCCESS) {
+ fprintf(stderr,
+ "%d Error deregistering send msg mr: %s\n",
+ getpid(), DT_RetToStr(ret));
+ } else {
+ LOGPRINTF("%d Unregistered send message Buffer\n",
+ getpid());
+ h_lmr_send_msg = NULL;
+ }
+ }
+
+ /* Unregister recv message Buffer */
+ if (h_lmr_recv_msg != DAT_HANDLE_NULL) {
+ LOGPRINTF("%d Unregister recv message h_lmr %p \n", getpid(),
+ h_lmr_recv_msg);
+ ret = dat_lmr_free(h_lmr_recv_msg);
+ if (ret != DAT_SUCCESS) {
+ fprintf(stderr,
+ "%d Error deregistering recv msg mr: %s\n",
+ getpid(), DT_RetToStr(ret));
+ } else {
+ LOGPRINTF("%d Unregistered recv message Buffer\n",
+ getpid());
+ h_lmr_recv_msg = NULL;
+ }
+ }
+ return;
+}
+
+DAT_RETURN do_rdma_write_with_msg(void)
+{
+ DAT_EVENT event;
+ DAT_LMR_TRIPLET l_iov[MSG_IOV_COUNT];
+ DAT_RMR_TRIPLET r_iov;
+ DAT_DTO_COOKIE cookie;
+ DAT_RETURN ret;
+ int i;
+
+ printf("\n %d RDMA WRITE DATA with SEND MSG\n\n", getpid());
+
+ cookie.as_64 = 0x5555;
+
+ if (recv_msg_index >= MSG_BUF_COUNT)
+ return (DAT_ABORT);
+
+ /* get RMR information from previously received message */
+ r_iov = rmr_recv_msg[recv_msg_index - 1];
+
+ if (server)
+ strcpy((char *)sbuf, "server RDMA write data...");
+ else
+ strcpy((char *)sbuf, "client RDMA write data...");
+
+ for (i = 0; i < MSG_IOV_COUNT; i++) {
+ l_iov[i].lmr_context = lmr_context_send;
+ l_iov[i].segment_length = buf_len / MSG_IOV_COUNT;
+ l_iov[i].virtual_address = (DAT_VADDR) (uintptr_t)
+ (&sbuf[l_iov[i].segment_length * i]);
+
+ LOGPRINTF("%d rdma_write iov[%d] buf=%p,len=%d\n",
+ getpid(), i, &sbuf[l_iov[i].segment_length * i],
+ l_iov[i].segment_length);
+ }
+
+ start = get_time();
+ for (i = 0; i < burst; i++) {
+ cookie.as_64 = 0x9999;
+ ret = dat_ep_post_rdma_write(h_ep, // ep_handle
+ MSG_IOV_COUNT, // num_segments
+ l_iov, // LMR
+ cookie, // user_cookie
+ &r_iov, // RMR
+ DAT_COMPLETION_SUPPRESS_FLAG);
+ if (ret != DAT_SUCCESS) {
+ fprintf(stderr,
+ "%d: ERROR: dat_ep_post_rdma_write() %s\n",
+ getpid(), DT_RetToStr(ret));
+ return (DAT_ABORT);
+ }
+ LOGPRINTF("%d rdma_write # %d completed\n", getpid(), i + 1);
+ }
+
+ /*
+ * Send RMR information a 2nd time to indicate completion
+ * NOTE: already swapped to network order in connect_ep
+ */
+ printf("%d Sending RDMA WRITE completion message\n", getpid());
+
+ ret = send_msg(&rmr_send_msg,
+ sizeof(DAT_RMR_TRIPLET),
+ lmr_context_send_msg,
+ cookie, DAT_COMPLETION_SUPPRESS_FLAG);
+
+ if (ret != DAT_SUCCESS) {
+ fprintf(stderr, "%d Error send_msg: %s\n",
+ getpid(), DT_RetToStr(ret));
+ return (ret);
+ } else {
+ LOGPRINTF("%d send_msg completed\n", getpid());
+ }
+
+ /* inbound recv event, send completion's suppressed */
+ if (collect_event(h_dto_rcv_evd,
+ &event,
+ DTO_TIMEOUT,
+ &rdma_wr_poll_count) != DAT_SUCCESS)
+ return (DAT_ABORT);
+
+ stop = get_time();
+ ts.rdma_wr = ((stop - start) * 1.0e6);
+
+ /* validate event number and status */
+ printf("%d inbound rdma_write; send message arrived!\n", getpid());
+ if (event.event_number != DAT_DTO_COMPLETION_EVENT) {
+ fprintf(stderr, "%d Error unexpected DTO event : %s\n",
+ getpid(), DT_EventToStr(event.event_number));
+ return (DAT_ABORT);
+ }
+
+ if ((event.event_data.dto_completion_event_data.transfered_length !=
+ sizeof(DAT_RMR_TRIPLET))
+ || (event.event_data.dto_completion_event_data.user_cookie.as_64 !=
+ recv_msg_index)) {
+ fprintf(stderr,
+ "unexpected event data for receive: len=%d cookie=" F64x
+ " exp %d/%d\n",
+ (int)event.event_data.dto_completion_event_data.
+ transfered_length,
+ event.event_data.dto_completion_event_data.user_cookie.
+ as_64, (int)sizeof(DAT_RMR_TRIPLET), recv_msg_index);
+
+ return (DAT_ABORT);
+ }
+
+ /* swap received RMR msg: network order to host order */
+ r_iov = rmr_recv_msg[recv_msg_index];
+ rmr_recv_msg[recv_msg_index].virtual_address =
+ ntohll(rmr_recv_msg[recv_msg_index].virtual_address);
+ rmr_recv_msg[recv_msg_index].segment_length =
+ ntohl(rmr_recv_msg[recv_msg_index].segment_length);
+ rmr_recv_msg[recv_msg_index].rmr_context =
+ ntohl(rmr_recv_msg[recv_msg_index].rmr_context);
+
+ printf("%d Received RMR from remote: "
+ "r_iov: r_key_ctx=%x,va=" F64x ",len=0x%x\n",
+ getpid(), rmr_recv_msg[recv_msg_index].rmr_context,
+ rmr_recv_msg[recv_msg_index].virtual_address,
+ rmr_recv_msg[recv_msg_index].segment_length);
+
+ LOGPRINTF("%d inbound rdma_write; send msg event SUCCESS!!\n",
+ getpid());
+
+ printf("%d %s RDMA write buffer contains: %s\n",
+ getpid(), server ? "SERVER:" : "CLIENT:", rbuf);
+
+ recv_msg_index++;
+
+ return (DAT_SUCCESS);
+}
+
+DAT_RETURN do_rdma_read_with_msg(void)
+{
+ DAT_EVENT event;
+ DAT_LMR_TRIPLET l_iov;
+ DAT_RMR_TRIPLET r_iov;
+ DAT_DTO_COOKIE cookie;
+ DAT_RETURN ret;
+ int i;
+
+ printf("\n %d RDMA READ DATA with SEND MSG\n\n", getpid());
+
+ if (recv_msg_index >= MSG_BUF_COUNT)
+ return (DAT_ABORT);
+
+ /* get RMR information from previously received message */
+ r_iov = rmr_recv_msg[recv_msg_index - 1];
+
+ /* setup rdma read buffer to initial string to be overwritten */
+ strcpy((char *)sbuf, "blah, blah, blah\n");
+
+ if (server)
+ strcpy((char *)rbuf, "server RDMA read data...");
+ else
+ strcpy((char *)rbuf, "client RDMA read data...");
+
+ l_iov.lmr_context = lmr_context_send;
+ l_iov.virtual_address = (DAT_VADDR) (uintptr_t) sbuf;
+ l_iov.segment_length = buf_len;
+
+ for (i = 0; i < MAX_RDMA_RD; i++) {
+ cookie.as_64 = 0x9999;
+ start = get_time();
+ ret = dat_ep_post_rdma_read(h_ep, // ep_handle
+ 1, // num_segments
+ &l_iov, // LMR
+ cookie, // user_cookie
+ &r_iov, // RMR
+ DAT_COMPLETION_DEFAULT_FLAG);
+ if (ret != DAT_SUCCESS) {
+ fprintf(stderr,
+ "%d: ERROR: dat_ep_post_rdma_read() %s\n",
+ getpid(), DT_RetToStr(ret));
+ return (DAT_ABORT);
+ }
+
+ /* RDMA read completion event */
+ if (collect_event(h_dto_req_evd,
+ &event,
+ DTO_TIMEOUT,
+ &rdma_rd_poll_count[i]) != DAT_SUCCESS)
+ return (DAT_ABORT);
+
+ /* validate event number, len, cookie, and status */
+ if (event.event_number != DAT_DTO_COMPLETION_EVENT) {
+ fprintf(stderr, "%d: ERROR: DTO event number %s\n",
+ getpid(), DT_EventToStr(event.event_number));
+ return (DAT_ABORT);
+ }
+ if ((event.event_data.dto_completion_event_data.
+ transfered_length != buf_len)
+ || (event.event_data.dto_completion_event_data.user_cookie.
+ as_64 != 0x9999)) {
+ fprintf(stderr,
+ "%d: ERROR: DTO len %d or cookie " F64x "\n",
+ getpid(),
+ event.event_data.dto_completion_event_data.
+ transfered_length,
+ event.event_data.dto_completion_event_data.
+ user_cookie.as_64);
+ return (DAT_ABORT);
+ }
+ if (event.event_data.dto_completion_event_data.status !=
+ DAT_SUCCESS) {
+ fprintf(stderr, "%d: ERROR: DTO event status %s\n",
+ getpid(), DT_RetToStr(ret));
+ return (DAT_ABORT);
+ }
+ stop = get_time();
+ ts.rdma_rd[i] = ((stop - start) * 1.0e6);
+ ts.rdma_rd_total += ts.rdma_rd[i];
+
+ LOGPRINTF("%d rdma_read # %d completed\n", getpid(), i + 1);
+ }
+
+ /*
+ * Send RMR information a 3rd time to indicate completion
+ * NOTE: already swapped to network order in connect_ep
+ */
+ printf("%d Sending RDMA read completion message\n", getpid());
+
+ /* give remote chance to process read completes */
+ if (use_cno) {
+#if defined(_WIN32) || defined(_WIN64)
+ Sleep(1000);
+#else
+ sleep(1);
+#endif
+ }
+
+ ret = send_msg(&rmr_send_msg,
+ sizeof(DAT_RMR_TRIPLET),
+ lmr_context_send_msg,
+ cookie, DAT_COMPLETION_SUPPRESS_FLAG);
+
+ if (ret != DAT_SUCCESS) {
+ fprintf(stderr, "%d Error send_msg: %s\n",
+ getpid(), DT_RetToStr(ret));
+ return (ret);
+ } else {
+ LOGPRINTF("%d send_msg completed\n", getpid());
+ }
+
+ printf("%d Waiting for inbound message....\n", getpid());
+
+ if (collect_event(h_dto_rcv_evd,
+ &event,
+ DTO_TIMEOUT,
+ &poll_count) != DAT_SUCCESS)
+ return (DAT_ABORT);
+
+ /* validate event number and status */
+ printf("%d inbound rdma_read; send message arrived!\n", getpid());
+ if (event.event_number != DAT_DTO_COMPLETION_EVENT) {
+ fprintf(stderr, "%d Error unexpected DTO event : %s\n",
+ getpid(), DT_EventToStr(event.event_number));
+ return (DAT_ABORT);
+ }
+
+ if ((event.event_data.dto_completion_event_data.transfered_length !=
+ sizeof(DAT_RMR_TRIPLET))
+ || (event.event_data.dto_completion_event_data.user_cookie.as_64 !=
+ recv_msg_index)) {
+
+ fprintf(stderr,
+ "unexpected event data for receive: len=%d cookie=" F64x
+ " exp %d/%d\n",
+ (int)event.event_data.dto_completion_event_data.
+ transfered_length,
+ event.event_data.dto_completion_event_data.user_cookie.
+ as_64, (int)sizeof(DAT_RMR_TRIPLET), recv_msg_index);
+
+ return (DAT_ABORT);
+ }
+
+ /* swap received RMR msg: network order to host order */
+ r_iov = rmr_recv_msg[recv_msg_index];
+ rmr_recv_msg[recv_msg_index].virtual_address =
+ ntohll(rmr_recv_msg[recv_msg_index].virtual_address);
+ rmr_recv_msg[recv_msg_index].segment_length =
+ ntohl(rmr_recv_msg[recv_msg_index].segment_length);
+ rmr_recv_msg[recv_msg_index].rmr_context =
+ ntohl(rmr_recv_msg[recv_msg_index].rmr_context);
+
+ printf("%d Received RMR from remote: "
+ "r_iov: r_key_ctx=%x,va=" F64x ",len=0x%x\n",
+ getpid(), rmr_recv_msg[recv_msg_index].rmr_context,
+ rmr_recv_msg[recv_msg_index].virtual_address,
+ rmr_recv_msg[recv_msg_index].segment_length);
+
+ LOGPRINTF("%d inbound rdma_write; send msg event SUCCESS!!\n",
+ getpid());
+
+ printf("%d %s RCV RDMA read buffer contains: %s\n",
+ getpid(), server ? "SERVER:" : "CLIENT:", sbuf);
+
+ recv_msg_index++;
+
+ return (DAT_SUCCESS);
+}
+
+DAT_RETURN do_ping_pong_msg()
+{
+ DAT_EVENT event;
+ DAT_DTO_COOKIE cookie;
+ DAT_LMR_TRIPLET l_iov;
+ DAT_RETURN ret;
+ int i;
+ char *snd_buf;
+ char *rcv_buf;
+
+ printf("\n %d PING DATA with SEND MSG\n\n", getpid());
+
+ snd_buf = sbuf;
+ rcv_buf = rbuf;
+
+ /* pre-post all buffers */
+ for (i = 0; i < burst; i++) {
+ burst_msg_posted++;
+ cookie.as_64 = i;
+ l_iov.lmr_context = lmr_context_recv;
+ l_iov.virtual_address = (DAT_VADDR) (uintptr_t) rcv_buf;
+ l_iov.segment_length = buf_len;
+
+ LOGPRINTF("%d Pre-posting Receive Message Buffers %p\n",
+ getpid(), rcv_buf);
+
+ ret = dat_ep_post_recv(h_ep,
+ 1,
+ &l_iov,
+ cookie, DAT_COMPLETION_DEFAULT_FLAG);
+
+ if (ret != DAT_SUCCESS) {
+ fprintf(stderr,
+ "%d Error posting recv msg buffer: %s\n",
+ getpid(), DT_RetToStr(ret));
+ return (ret);
+ } else {
+ LOGPRINTF("%d Posted Receive Message Buffer %p\n",
+ getpid(), rcv_buf);
+ }
+
+ /* next buffer */
+ rcv_buf += buf_len;
+ }
+#if defined(_WIN32) || defined(_WIN64)
+ Sleep(1000);
+#else
+ sleep(1);
+#endif
+
+ /* Initialize recv_buf and index to beginning */
+ rcv_buf = rbuf;
+ burst_msg_index = 0;
+
+ /* client ping 0x55, server pong 0xAA in first byte */
+ start = get_time();
+ for (i = 0; i < burst; i++) {
+ /* walk the send and recv buffers */
+ if (!server) {
+ *snd_buf = 0x55;
+
+ LOGPRINTF("%d %s SND buffer %p contains: 0x%x len=%d\n",
+ getpid(), server ? "SERVER:" : "CLIENT:",
+ snd_buf, *snd_buf, buf_len);
+
+ ret = send_msg(snd_buf,
+ buf_len,
+ lmr_context_send,
+ cookie, DAT_COMPLETION_SUPPRESS_FLAG);
+
+ if (ret != DAT_SUCCESS) {
+ fprintf(stderr, "%d Error send_msg: %s\n",
+ getpid(), DT_RetToStr(ret));
+ return (ret);
+ } else {
+ LOGPRINTF("%d send_msg completed\n", getpid());
+ }
+ }
+
+ /* recv message, send completions suppressed */
+ if (collect_event(h_dto_rcv_evd,
+ &event,
+ DTO_TIMEOUT,
+ &poll_count) != DAT_SUCCESS)
+ return (DAT_ABORT);
+
+
+ /* start timer after first message arrives on server */
+ if (i == 0) {
+ start = get_time();
+ }
+ /* validate event number and status */
+ LOGPRINTF("%d inbound message; message arrived!\n", getpid());
+ if (event.event_number != DAT_DTO_COMPLETION_EVENT) {
+ fprintf(stderr, "%d Error unexpected DTO event : %s\n",
+ getpid(), DT_EventToStr(event.event_number));
+ return (DAT_ABORT);
+ }
+ if ((event.event_data.dto_completion_event_data.
+ transfered_length != buf_len)
+ || (event.event_data.dto_completion_event_data.user_cookie.
+ as_64 != burst_msg_index)) {
+ fprintf(stderr,
+ "ERR: recv event: len=%d cookie=" F64x
+ " exp %d/%d\n",
+ (int)event.event_data.dto_completion_event_data.
+ transfered_length,
+ event.event_data.dto_completion_event_data.
+ user_cookie.as_64, (int)buf_len,
+ (int)burst_msg_index);
+
+ return (DAT_ABORT);
+ }
+
+ LOGPRINTF("%d %s RCV buffer %p contains: 0x%x len=%d\n",
+ getpid(), server ? "SERVER:" : "CLIENT:",
+ rcv_buf, *rcv_buf, buf_len);
+
+ burst_msg_index++;
+
+ /* If server, change data and send it back to client */
+ if (server) {
+ *snd_buf = 0xaa;
+
+ LOGPRINTF("%d %s SND buffer %p contains: 0x%x len=%d\n",
+ getpid(), server ? "SERVER:" : "CLIENT:",
+ snd_buf, *snd_buf, buf_len);
+
+ ret = send_msg(snd_buf,
+ buf_len,
+ lmr_context_send,
+ cookie, DAT_COMPLETION_SUPPRESS_FLAG);
+
+ if (ret != DAT_SUCCESS) {
+ fprintf(stderr, "%d Error send_msg: %s\n",
+ getpid(), DT_RetToStr(ret));
+ return (ret);
+ } else {
+ LOGPRINTF("%d send_msg completed\n", getpid());
+ }
+ }
+
+ /* next buffers */
+ rcv_buf += buf_len;
+ snd_buf += buf_len;
+ }
+ stop = get_time();
+ ts.rtt = ((stop - start) * 1.0e6);
+
+ return (DAT_SUCCESS);
+}
+
+/* Register RDMA Receive buffer */
+DAT_RETURN register_rdma_memory(void)
+{
+ DAT_RETURN ret;
+ DAT_REGION_DESCRIPTION region;
+
+ region.for_va = rbuf;
+ start = get_time();
+ ret = dat_lmr_create(h_ia,
+ DAT_MEM_TYPE_VIRTUAL,
+ region,
+ buf_len * (burst+1),
+ h_pz,
+ DAT_MEM_PRIV_ALL_FLAG,
+ DAT_VA_TYPE_VA,
+ &h_lmr_recv,
+ &lmr_context_recv,
+ &rmr_context_recv,
+ ®istered_size_recv, ®istered_addr_recv);
+ stop = get_time();
+ ts.reg += ((stop - start) * 1.0e6);
+ ts.total += ts.reg;
+
+ if (ret != DAT_SUCCESS) {
+ fprintf(stderr,
+ "%d Error registering Receive RDMA buffer: %s\n",
+ getpid(), DT_RetToStr(ret));
+ return (ret);
+ } else {
+ LOGPRINTF("%d Registered Receive RDMA Buffer %p\n",
+ getpid(), region.for_va);
+ }
+
+ /* Register RDMA Send buffer */
+ region.for_va = sbuf;
+ ret = dat_lmr_create(h_ia,
+ DAT_MEM_TYPE_VIRTUAL,
+ region,
+ buf_len * (burst + 1),
+ h_pz,
+ DAT_MEM_PRIV_ALL_FLAG,
+ DAT_VA_TYPE_VA,
+ &h_lmr_send,
+ &lmr_context_send,
+ &rmr_context_send,
+ ®istered_size_send, ®istered_addr_send);
+ if (ret != DAT_SUCCESS) {
+ fprintf(stderr, "%d Error registering send RDMA buffer: %s\n",
+ getpid(), DT_RetToStr(ret));
+ return (ret);
+ } else {
+ LOGPRINTF("%d Registered Send RDMA Buffer %p\n",
+ getpid(), region.for_va);
+ }
+
+ return DAT_SUCCESS;
+}
+
+/*
+ * Unregister RDMA memory
+ */
+DAT_RETURN unregister_rdma_memory(void)
+{
+ DAT_RETURN ret;
+
+ /* Unregister Recv Buffer */
+ if (h_lmr_recv != DAT_HANDLE_NULL) {
+ LOGPRINTF("%d Unregister h_lmr %p \n", getpid(), h_lmr_recv);
+ start = get_time();
+ ret = dat_lmr_free(h_lmr_recv);
+ stop = get_time();
+ ts.unreg += ((stop - start) * 1.0e6);
+ ts.total += ts.unreg;
+ if (ret != DAT_SUCCESS) {
+ fprintf(stderr, "%d Error deregistering recv mr: %s\n",
+ getpid(), DT_RetToStr(ret));
+ return (ret);
+ } else {
+ LOGPRINTF("%d Unregistered Recv Buffer\n", getpid());
+ h_lmr_recv = NULL;
+ }
+ }
+
+ /* Unregister Send Buffer */
+ if (h_lmr_send != DAT_HANDLE_NULL) {
+ LOGPRINTF("%d Unregister h_lmr %p \n", getpid(), h_lmr_send);
+ ret = dat_lmr_free(h_lmr_send);
+ if (ret != DAT_SUCCESS) {
+ fprintf(stderr, "%d Error deregistering send mr: %s\n",
+ getpid(), DT_RetToStr(ret));
+ return (ret);
+ } else {
+ LOGPRINTF("%d Unregistered send Buffer\n", getpid());
+ h_lmr_send = NULL;
+ }
+ }
+ return DAT_SUCCESS;
+}
+
+ /*
+ * Create CNO, CR, CONN, and DTO events
+ */
+DAT_RETURN create_events(void)
+{
+ DAT_RETURN ret;
+ DAT_EVD_PARAM param;
+
+ /* create CNO */
+ if (use_cno) {
+ start = get_time();
+#if defined(_WIN32) || defined(_WIN64)
+ {
+ DAT_OS_WAIT_PROXY_AGENT pa = { NULL, NULL };
+ ret = dat_cno_create(h_ia, pa, &h_dto_cno);
+ }
+#else
+ ret =
+ dat_cno_create(h_ia, DAT_OS_WAIT_PROXY_AGENT_NULL,
+ &h_dto_cno);
+#endif
+ stop = get_time();
+ ts.cnoc += ((stop - start) * 1.0e6);
+ ts.total += ts.cnoc;
+ if (ret != DAT_SUCCESS) {
+ fprintf(stderr, "%d Error dat_cno_create: %s\n",
+ getpid(), DT_RetToStr(ret));
+ return (ret);
+ } else {
+ LOGPRINTF("%d cr_evd created, %p\n", getpid(),
+ h_dto_cno);
+ }
+ }
+
+ /* create cr EVD */
+ start = get_time();
+ ret =
+ dat_evd_create(h_ia, 10, DAT_HANDLE_NULL, DAT_EVD_CR_FLAG,
+ &h_cr_evd);
+ stop = get_time();
+ ts.evdc += ((stop - start) * 1.0e6);
+ ts.total += ts.evdc;
+ if (ret != DAT_SUCCESS) {
+ fprintf(stderr, "%d Error dat_evd_create: %s\n",
+ getpid(), DT_RetToStr(ret));
+ return (ret);
+ } else {
+ LOGPRINTF("%d cr_evd created %p\n", getpid(), h_cr_evd);
+ }
+
+ /* create conn EVD */
+ ret = dat_evd_create(h_ia,
+ 10,
+ DAT_HANDLE_NULL,
+ DAT_EVD_CONNECTION_FLAG, &h_conn_evd);
+ if (ret != DAT_SUCCESS) {
+ fprintf(stderr, "%d Error dat_evd_create: %s\n",
+ getpid(), DT_RetToStr(ret));
+ return (ret);
+ } else {
+ LOGPRINTF("%d con_evd created %p\n", getpid(), h_conn_evd);
+ }
+
+ /* create dto SND EVD, with CNO if use_cno was set */
+ ret = dat_evd_create(h_ia,
+ (MSG_BUF_COUNT + MAX_RDMA_RD + burst) * 2,
+ h_dto_cno, DAT_EVD_DTO_FLAG, &h_dto_req_evd);
+ if (ret != DAT_SUCCESS) {
+ fprintf(stderr, "%d Error dat_evd_create REQ: %s\n",
+ getpid(), DT_RetToStr(ret));
+ return (ret);
+ } else {
+ LOGPRINTF("%d dto_req_evd created %p\n", getpid(),
+ h_dto_req_evd);
+ }
+
+ /* create dto RCV EVD, with CNO if use_cno was set */
+ ret = dat_evd_create(h_ia,
+ MSG_BUF_COUNT + burst,
+ h_dto_cno, DAT_EVD_DTO_FLAG, &h_dto_rcv_evd);
+ if (ret != DAT_SUCCESS) {
+ fprintf(stderr, "%d Error dat_evd_create RCV: %s\n",
+ getpid(), DT_RetToStr(ret));
+ return (ret);
+ } else {
+ LOGPRINTF("%d dto_rcv_evd created %p\n", getpid(),
+ h_dto_rcv_evd);
+ }
+
+ /* query DTO req EVD and check size */
+ ret = dat_evd_query(h_dto_req_evd, DAT_EVD_FIELD_EVD_QLEN, ¶m);
+ if (ret != DAT_SUCCESS) {
+ fprintf(stderr, "%d Error dat_evd_query request evd: %s\n",
+ getpid(), DT_RetToStr(ret));
+ return (ret);
+ } else if (param.evd_qlen < (MSG_BUF_COUNT + MAX_RDMA_RD + burst) * 2) {
+ fprintf(stderr, "%d Error dat_evd qsize too small: %d < %d\n",
+ getpid(), param.evd_qlen,
+ (MSG_BUF_COUNT + MAX_RDMA_RD + burst) * 2);
+ return (ret);
+ }
+
+ LOGPRINTF("%d dto_req_evd QLEN - requested %d and actual %d\n",
+ getpid(), (MSG_BUF_COUNT + MAX_RDMA_RD + burst) * 2,
+ param.evd_qlen);
+
+ return DAT_SUCCESS;
+}
+
+/*
+ * Destroy CR, CONN, CNO, and DTO events
+ */
+
+DAT_RETURN destroy_events(void)
+{
+ DAT_RETURN ret;
+
+ /* free cr EVD */
+ if (h_cr_evd != DAT_HANDLE_NULL) {
+ LOGPRINTF("%d Free cr EVD %p \n", getpid(), h_cr_evd);
+ ret = dat_evd_free(h_cr_evd);
+ if (ret != DAT_SUCCESS) {
+ fprintf(stderr, "%d Error freeing cr EVD: %s\n",
+ getpid(), DT_RetToStr(ret));
+ return (ret);
+ } else {
+ LOGPRINTF("%d Freed cr EVD\n", getpid());
+ h_cr_evd = DAT_HANDLE_NULL;
+ }
+ }
+
+ /* free conn EVD */
+ if (h_conn_evd != DAT_HANDLE_NULL) {
+ LOGPRINTF("%d Free conn EVD %p \n", getpid(), h_conn_evd);
+ ret = dat_evd_free(h_conn_evd);
+ if (ret != DAT_SUCCESS) {
+ fprintf(stderr, "%d Error freeing conn EVD: %s\n",
+ getpid(), DT_RetToStr(ret));
+ return (ret);
+ } else {
+ LOGPRINTF("%d Freed conn EVD\n", getpid());
+ h_conn_evd = DAT_HANDLE_NULL;
+ }
+ }
+
+ /* free RCV dto EVD */
+ if (h_dto_rcv_evd != DAT_HANDLE_NULL) {
+ LOGPRINTF("%d Free RCV dto EVD %p \n", getpid(), h_dto_rcv_evd);
+ start = get_time();
+ ret = dat_evd_free(h_dto_rcv_evd);
+ stop = get_time();
+ ts.evdf += ((stop - start) * 1.0e6);
+ ts.total += ts.evdf;
+ if (ret != DAT_SUCCESS) {
+ fprintf(stderr, "%d Error freeing dto EVD: %s\n",
+ getpid(), DT_RetToStr(ret));
+ return (ret);
+ } else {
+ LOGPRINTF("%d Freed dto EVD\n", getpid());
+ h_dto_rcv_evd = DAT_HANDLE_NULL;
+ }
+ }
+
+ /* free REQ dto EVD */
+ if (h_dto_req_evd != DAT_HANDLE_NULL) {
+ LOGPRINTF("%d Free REQ dto EVD %p \n", getpid(), h_dto_req_evd);
+ ret = dat_evd_free(h_dto_req_evd);
+ if (ret != DAT_SUCCESS) {
+ fprintf(stderr, "%d Error freeing dto EVD: %s\n",
+ getpid(), DT_RetToStr(ret));
+ return (ret);
+ } else {
+ LOGPRINTF("%d Freed dto EVD\n", getpid());
+ h_dto_req_evd = DAT_HANDLE_NULL;
+ }
+ }
+
+ /* free CNO */
+ if (h_dto_cno != DAT_HANDLE_NULL) {
+ LOGPRINTF("%d Free dto CNO %p \n", getpid(), h_dto_cno);
+ start = get_time();
+ ret = dat_cno_free(h_dto_cno);
+ stop = get_time();
+ ts.cnof += ((stop - start) * 1.0e6);
+ ts.total += ts.cnof;
+ if (ret != DAT_SUCCESS) {
+ fprintf(stderr, "%d Error freeing dto CNO: %s\n",
+ getpid(), DT_RetToStr(ret));
+ return (ret);
+ } else {
+ LOGPRINTF("%d Freed dto CNO\n", getpid());
+ h_dto_cno = DAT_HANDLE_NULL;
+ }
+ }
+ return DAT_SUCCESS;
+}
+
+/*
+ * Map DAT_RETURN values to readable strings,
+ * but don't assume the values are zero-based or contiguous.
+ */
+char errmsg[512] = { 0 };
+const char *DT_RetToStr(DAT_RETURN ret_value)
+{
+ const char *major_msg, *minor_msg;
+
+ dat_strerror(ret_value, &major_msg, &minor_msg);
+
+ strcpy(errmsg, major_msg);
+ strcat(errmsg, " ");
+ strcat(errmsg, minor_msg);
+
+ return errmsg;
+}
+
+/*
+ * Map DAT_EVENT_CODE values to readable strings
+ */
+const char *DT_EventToStr(DAT_EVENT_NUMBER event_code)
+{
+ unsigned int i;
+ static struct {
+ const char *name;
+ DAT_RETURN value;
+ } dat_events[] = {
+# define DATxx(x) { # x, x }
+ DATxx(DAT_DTO_COMPLETION_EVENT),
+ DATxx(DAT_RMR_BIND_COMPLETION_EVENT),
+ DATxx(DAT_CONNECTION_REQUEST_EVENT),
+ DATxx(DAT_CONNECTION_EVENT_ESTABLISHED),
+ DATxx(DAT_CONNECTION_EVENT_PEER_REJECTED),
+ DATxx(DAT_CONNECTION_EVENT_NON_PEER_REJECTED),
+ DATxx(DAT_CONNECTION_EVENT_ACCEPT_COMPLETION_ERROR),
+ DATxx(DAT_CONNECTION_EVENT_DISCONNECTED),
+ DATxx(DAT_CONNECTION_EVENT_BROKEN),
+ DATxx(DAT_CONNECTION_EVENT_TIMED_OUT),
+ DATxx(DAT_CONNECTION_EVENT_UNREACHABLE),
+ DATxx(DAT_ASYNC_ERROR_EVD_OVERFLOW),
+ DATxx(DAT_ASYNC_ERROR_IA_CATASTROPHIC),
+ DATxx(DAT_ASYNC_ERROR_EP_BROKEN),
+ DATxx(DAT_ASYNC_ERROR_TIMED_OUT),
+ DATxx(DAT_ASYNC_ERROR_PROVIDER_INTERNAL_ERROR),
+ DATxx(DAT_SOFTWARE_EVENT)
+# undef DATxx
+ };
+# define NUM_EVENTS (sizeof(dat_events)/sizeof(dat_events[0]))
+
+ for (i = 0; i < NUM_EVENTS; i++) {
+ if (dat_events[i].value == event_code) {
+ return (dat_events[i].name);
+ }
+ }
+
+ return ("Invalid_DAT_EVENT_NUMBER");
+}
+
+void print_usage(void)
+{
+ printf("\n DAPL USAGE \n\n");
+ printf("s: server\n");
+ printf("t: performance times\n");
+ printf("c: use cno\n");
+ printf("v: verbose\n");
+ printf("p: polling\n");
+ printf("d: delay before accept\n");
+ printf("b: buf length to allocate\n");
+ printf("B: burst count, rdma and msgs \n");
+ printf("h: hostname/address of server, specified on client\n");
+ printf("P: provider name (default = OpenIB-cma)\n");
+ printf("l: server lid (required ucm provider)\n");
+ printf("q: server qpn (required ucm provider)\n");
+ printf("\n");
+}
+
-/*\r
- * Copyright (c) 2009 Intel Corporation. All rights reserved.\r
- *\r
- * This software is available to you under a choice of one of two\r
- * licenses. You may choose to be licensed under the terms of the GNU\r
- * General Public License (GPL) Version 2, available from the file\r
- * COPYING in the main directory of this source tree, or the\r
- * OpenIB.org BSD license below:\r
- *\r
- * Redistribution and use in source and binary forms, with or\r
- * without modification, are permitted provided that the following\r
- * conditions are met:\r
- *\r
- * - Redistributions of source code must retain the above\r
- * copyright notice, this list of conditions and the following\r
- * disclaimer.\r
- *\r
- * - Redistributions in binary form must reproduce the above\r
- * copyright notice, this list of conditions and the following\r
- * disclaimer in the documentation and/or other materials\r
- * provided with the distribution.\r
- *\r
- * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,\r
- * EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF\r
- * MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND\r
- * NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS\r
- * BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN\r
- * ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN\r
- * CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE\r
- * SOFTWARE.\r
- *\r
- * $Id: $\r
- */\r
-#include <stdio.h>\r
-#include <stdlib.h>\r
-#include <string.h>\r
-\r
-#ifdef DAPL_PROVIDER\r
-#undef DAPL_PROVIDER\r
-#endif\r
-\r
-#if defined(_WIN32) || defined(_WIN64)\r
-\r
-#include <windows.h>\r
-#include <winsock2.h>\r
-#include <ws2tcpip.h>\r
-#include <io.h>\r
-#include <process.h>\r
-#include <complib/cl_types.h>\r
-#include "..\..\..\..\etc\user\getopt.c"\r
-\r
-#define getpid() ((int)GetCurrentProcessId())\r
-#define F64x "%I64x"\r
-#define F64d "%I64d"\r
-\r
-#ifdef DBG\r
-#define DAPL_PROVIDER "ibnic0v2d"\r
-#else\r
-#define DAPL_PROVIDER "ibnic0v2"\r
-#endif\r
-\r
-#define ntohll _byteswap_uint64\r
-#define htonll _byteswap_uint64\r
-\r
-#else // _WIN32 || _WIN64\r
-\r
-#include <endian.h>\r
-#include <byteswap.h>\r
-#include <netdb.h>\r
-#include <sys/types.h>\r
-#include <sys/socket.h>\r
-#include <sys/time.h>\r
-#include <netinet/in.h>\r
-#include <netinet/tcp.h>\r
-#include <arpa/inet.h>\r
-#include <sys/mman.h>\r
-#include <getopt.h>\r
-#include <inttypes.h>\r
-#include <unistd.h>\r
-#include <stdlib.h>\r
-\r
-#define DAPL_PROVIDER "ofa-v2-ib0"\r
-\r
-#define F64x "%"PRIx64""\r
-#define F64d "%"PRId64""\r
-\r
-\r
-#if __BYTE_ORDER == __BIG_ENDIAN\r
-#define htonll(x) (x)\r
-#define ntohll(x) (x)\r
-#elif __BYTE_ORDER == __LITTLE_ENDIAN\r
-#define htonll(x) bswap_64(x)\r
-#define ntohll(x) bswap_64(x)\r
-#endif\r
-\r
-#endif // _WIN32 || _WIN64\r
-\r
-#define MAX_POLLING_CNT 50000\r
-\r
-/* Header files needed for DAT/uDAPL */\r
-#include "dat2/udat.h"\r
-#include "dat2/dat_ib_extensions.h"\r
-\r
-/* definitions */\r
-#define SERVER_CONN_QUAL 45248\r
-#define CONN_TIMEOUT (1000*1000*100)\r
-#define CR_TIMEOUT DAT_TIMEOUT_INFINITE\r
-\r
-/* Global DAT vars */\r
-static DAT_IA_HANDLE h_ia = DAT_HANDLE_NULL;\r
-static DAT_PZ_HANDLE h_pz = DAT_HANDLE_NULL;\r
-static DAT_EP_HANDLE *h_ep;\r
-static DAT_PSP_HANDLE *h_psp;\r
-static DAT_CR_HANDLE h_cr = DAT_HANDLE_NULL;\r
-\r
-static DAT_EVD_HANDLE h_async_evd = DAT_HANDLE_NULL;\r
-static DAT_EVD_HANDLE h_dto_req_evd = DAT_HANDLE_NULL;\r
-static DAT_EVD_HANDLE h_dto_rcv_evd = DAT_HANDLE_NULL;\r
-static DAT_EVD_HANDLE h_cr_evd = DAT_HANDLE_NULL;\r
-static DAT_EVD_HANDLE h_conn_evd = DAT_HANDLE_NULL;\r
-\r
-static DAT_EP_ATTR ep_attr;\r
-char hostname[256] = { 0 };\r
-char provider[64] = DAPL_PROVIDER;\r
-char addr_str[INET_ADDRSTRLEN];\r
-\r
-int status;\r
-\r
-/* timers */\r
-double start, stop, total_us, total_sec;\r
-\r
-struct dt_time {\r
- double total;\r
- double open;\r
- double reg;\r
- double unreg;\r
- double pzc;\r
- double pzf;\r
- double evdc;\r
- double evdf;\r
- double cnoc;\r
- double cnof;\r
- double epc;\r
- double epf;\r
- double rtt;\r
- double close;\r
- double conn;\r
-};\r
-\r
-struct dt_time ts;\r
-\r
-/* defaults */\r
-static int connected = 0;\r
-static int multi_listens = 0;\r
-static int ud_test = 0;\r
-static int server = 1;\r
-static int waiting = 0;\r
-static int verbose = 0;\r
-static int cr_poll_count = 0;\r
-static int conn_poll_count = 0;\r
-static int delay = 0;\r
-static int connections = 1000;\r
-static int burst = 100;\r
-static int port_id = SERVER_CONN_QUAL;\r
-static int ucm = 0;\r
-static DAT_SOCK_ADDR6 remote;\r
-\r
-/* forward prototypes */\r
-const char *DT_RetToString(DAT_RETURN ret_value);\r
-const char *DT_EventToSTr(DAT_EVENT_NUMBER event_code);\r
-void print_usage(void);\r
-double get_time(void);\r
-DAT_RETURN conn_client(void);\r
-DAT_RETURN conn_server(void);\r
-DAT_RETURN disconnect_eps(void);\r
-DAT_RETURN create_events(void);\r
-DAT_RETURN destroy_events(void);\r
-\r
-#define LOGPRINTF if (verbose) printf\r
-\r
-void flush_evds(void)\r
-{\r
- DAT_EVENT event;\r
-\r
- /* Flush async error queue */\r
- printf(" ERR: Checking ASYNC EVD...\n");\r
- while (dat_evd_dequeue(h_async_evd, &event) == DAT_SUCCESS) {\r
- printf(" ASYNC EVD ENTRY: handle=%p reason=%d\n",\r
- event.event_data.asynch_error_event_data.dat_handle,\r
- event.event_data.asynch_error_event_data.reason);\r
- }\r
-}\r
-\r
-static void print_ia_address(struct sockaddr *sa)\r
-{\r
- char str[INET6_ADDRSTRLEN] = {" ??? "};\r
-\r
- switch(sa->sa_family) {\r
- case AF_INET:\r
- inet_ntop(AF_INET, &((struct sockaddr_in *)sa)->sin_addr, str, INET6_ADDRSTRLEN);\r
- printf("%d Local Address AF_INET - %s port %d\n", getpid(), str, SERVER_CONN_QUAL);\r
- break;\r
- case AF_INET6:\r
- inet_ntop(AF_INET6, &((struct sockaddr_in6 *)sa)->sin6_addr, str, INET6_ADDRSTRLEN);\r
- printf("%d Local Address AF_INET6 - %s flowinfo(QPN)=0x%x, port(LID)=0x%x\n",\r
- getpid(), str,\r
- ntohl(((struct sockaddr_in6 *)sa)->sin6_flowinfo),\r
- ntohs(((struct sockaddr_in6 *)sa)->sin6_port));\r
- break;\r
- default:\r
- printf("%d Local Address UNKOWN FAMILY - port %d\n", getpid(), SERVER_CONN_QUAL);\r
- }\r
-}\r
-\r
-int main(int argc, char **argv)\r
-{\r
- int i, c, len;\r
- DAT_RETURN ret;\r
- DAT_IA_ATTR ia_attr;\r
- \r
- /* parse arguments */\r
- while ((c = getopt(argc, argv, "smwvub:c:d:h:P:p:q:l:")) != -1) {\r
- switch (c) {\r
- case 's':\r
- server = 1;\r
- break;\r
- case 'm':\r
- multi_listens = 1;\r
- break;\r
- case 'w':\r
- waiting = 1;\r
- break;\r
- case 'u':\r
- ud_test = 1;\r
- break;\r
- case 'c':\r
- connections = atoi(optarg);\r
- break;\r
- case 'p':\r
- port_id = atoi(optarg);\r
- break;\r
- case 'v':\r
- verbose = 1;\r
- fflush(stdout);\r
- break;\r
- case 'd':\r
- delay = atoi(optarg);\r
- break;\r
- case 'b':\r
- burst = atoi(optarg);\r
- break;\r
- case 'h':\r
- server = 0;\r
- strcpy(hostname, optarg);\r
- break;\r
- case 'P':\r
- strcpy(provider, optarg);\r
- break;\r
- case 'q':\r
- /* map UCM qpn into AF_INET6 sin6_flowinfo */\r
- remote.sin6_family = AF_INET6;\r
- remote.sin6_flowinfo = htonl(strtol(optarg,NULL,0));\r
- ucm = 1;\r
- server = 0;\r
- break;\r
- case 'l':\r
- /* map UCM lid into AF_INET6 sin6_port */\r
- remote.sin6_family = AF_INET6;\r
- remote.sin6_port = htons(strtol(optarg,NULL,0));\r
- ucm = 1;\r
- server = 0;\r
- break;\r
- default:\r
- print_usage();\r
- exit(-12);\r
- }\r
- }\r
-\r
-#if defined(_WIN32) || defined(_WIN64)\r
- {\r
- WSADATA wsaData;\r
-\r
- i = WSAStartup(MAKEWORD(2, 2), &wsaData);\r
- if (i != 0) {\r
- printf("%s WSAStartup(2.2) failed? (0x%x)\n", argv[0],\r
- i);\r
- fflush(stdout);\r
- exit(1);\r
- }\r
- }\r
-#endif\r
-\r
- if (!server) {\r
- printf(" Running client on %s with %d %s connections\n", \r
- provider, connections, ud_test ? "UD" : "RC");\r
- } else {\r
- printf(" Running server on %s with %d %s connections\n", \r
- provider, connections, ud_test ? "UD" : "RC");\r
- }\r
- fflush(stdout);\r
- \r
- if (burst > connections)\r
- burst = connections;\r
-\r
- \r
- /* allocate EP handles for all connections */\r
- h_ep = (DAT_EP_HANDLE*)malloc(connections * sizeof(DAT_EP_HANDLE));\r
- if (h_ep == NULL) { \r
- perror("malloc ep");\r
- exit(1);\r
- }\r
- memset(h_ep, 0, (burst * sizeof(DAT_PSP_HANDLE)));\r
- \r
- /* allocate PSP handles, check for multi-listens */\r
- if (multi_listens)\r
- len = burst * sizeof(DAT_PSP_HANDLE);\r
- else\r
- len = sizeof(DAT_PSP_HANDLE);\r
-\r
- h_psp = (DAT_PSP_HANDLE*)malloc(len);\r
- if (h_psp == NULL) { \r
- perror("malloc psp");\r
- exit(1);\r
- }\r
- memset(h_psp, 0, len);\r
- memset(&ts, 0, sizeof(struct dt_time));\r
-\r
- /* dat_ia_open, dat_pz_create */\r
- h_async_evd = DAT_HANDLE_NULL;\r
- start = get_time();\r
- ret = dat_ia_open(provider, 8, &h_async_evd, &h_ia);\r
- stop = get_time();\r
- ts.open += ((stop - start) * 1.0e6);\r
- if (ret != DAT_SUCCESS) {\r
- fprintf(stderr, " Error Adaptor open: %s\n",\r
- DT_RetToString(ret));\r
- exit(1);\r
- } else\r
- LOGPRINTF(" Opened Interface Adaptor\n");\r
-\r
- /* query for UCM addressing */\r
- ret = dat_ia_query(h_ia, 0, DAT_IA_FIELD_ALL, &ia_attr, 0, 0);\r
- if (ret != DAT_SUCCESS) {\r
- fprintf(stderr, "%d: Error Adaptor query: %s\n",\r
- getpid(), DT_RetToString(ret));\r
- exit(1);\r
- }\r
- print_ia_address(ia_attr.ia_address_ptr);\r
-\r
- /* Create Protection Zone */\r
- start = get_time();\r
- LOGPRINTF(" Create Protection Zone\n");\r
- ret = dat_pz_create(h_ia, &h_pz);\r
- stop = get_time();\r
- ts.pzc += ((stop - start) * 1.0e6);\r
- if (ret != DAT_SUCCESS) {\r
- fprintf(stderr, " Error creating Protection Zone: %s\n",\r
- DT_RetToString(ret));\r
- exit(1);\r
- } else\r
- LOGPRINTF(" Created Protection Zone\n");\r
-\r
- LOGPRINTF(" Create events\n");\r
- ret = create_events();\r
- if (ret != DAT_SUCCESS) {\r
- fprintf(stderr, " Error creating events: %s\n",\r
- DT_RetToString(ret));\r
- goto cleanup;\r
- } else {\r
- LOGPRINTF(" Create events done\n");\r
- }\r
-\r
- /* create EP */\r
- memset(&ep_attr, 0, sizeof(ep_attr));\r
- if (ud_test) {\r
- ep_attr.service_type = DAT_IB_SERVICE_TYPE_UD;\r
- ep_attr.max_message_size = 2048;\r
- } else {\r
- ep_attr.service_type = DAT_SERVICE_TYPE_RC;\r
- ep_attr.max_rdma_size = 0x10000;\r
- ep_attr.max_rdma_read_in = 4;\r
- ep_attr.max_rdma_read_out = 4;\r
- }\r
- ep_attr.max_recv_dtos = 1;\r
- ep_attr.max_request_dtos = 1;\r
- ep_attr.max_recv_iov = 1;\r
- ep_attr.max_request_iov = 1;\r
- ep_attr.request_completion_flags = DAT_COMPLETION_DEFAULT_FLAG;\r
- \r
- start = get_time();\r
- for (i = 0; i < connections; i++) {\r
- ret = dat_ep_create(h_ia, h_pz, h_dto_rcv_evd,\r
- h_dto_req_evd, h_conn_evd, \r
- &ep_attr, &h_ep[i]);\r
- }\r
- stop = get_time();\r
- ts.epc += ((stop - start) * 1.0e6);\r
- ts.total += ts.epc;\r
- if (ret != DAT_SUCCESS) {\r
- fprintf(stderr, " Error dat_ep_create: %s\n",\r
- DT_RetToString(ret));\r
- goto cleanup;\r
- } else\r
- LOGPRINTF(" EP created %p \n", h_ep[i]);\r
-\r
- /* create the service point for server listen */\r
- if (server) {\r
- LOGPRINTF(" Creating server service point(s)\n");\r
- for (i = 0; i < burst; i++) {\r
- ret = dat_psp_create(h_ia,\r
- port_id+i,\r
- h_cr_evd, \r
- DAT_PSP_CONSUMER_FLAG, \r
- &h_psp[i]);\r
-\r
- if (ret != DAT_SUCCESS) {\r
- fprintf(stderr, " ERR psp_create: %s\n",\r
- DT_RetToString(ret));\r
- goto cleanup;\r
- } else\r
- LOGPRINTF(" psp_created for listen\n");\r
-\r
- printf(" Server ready on port %d\n", \r
- port_id+i);\r
-\r
- if (!multi_listens)\r
- break;\r
- }\r
- }\r
- \r
- /* Connect all */\r
- if (server)\r
- ret = conn_server();\r
- else \r
- ret = conn_client();\r
- \r
- if (ret != DAT_SUCCESS) {\r
- fprintf(stderr, " Error %s: %s\n",\r
- server ? "server()" : "client()",\r
- DT_RetToString(ret));\r
- goto cleanup;\r
- } else\r
- LOGPRINTF(" connect_ep complete\n");\r
-\r
- connected = 1;\r
- goto complete;\r
-\r
-cleanup:\r
- flush_evds();\r
- goto bail;\r
-complete:\r
-\r
- /* disconnect and free EP resources */\r
- if (h_ep[0]) {\r
- /* unregister message buffers and tear down connection */\r
- LOGPRINTF(" Disconnect EPs\n");\r
- ret = disconnect_eps();\r
- if (ret != DAT_SUCCESS) {\r
- fprintf(stderr, " Error disconnect_eps: %s\n",\r
- DT_RetToString(ret));\r
- goto bail;\r
- } else {\r
- LOGPRINTF(" disconnect_eps complete\n");\r
- }\r
- }\r
-\r
- /* destroy server service point(s) */\r
- if ((server) && (h_psp[0] != DAT_HANDLE_NULL)) {\r
- for (i = 0; i < burst; i++) {\r
- ret = dat_psp_free(h_psp[i]);\r
- if (ret != DAT_SUCCESS) {\r
- fprintf(stderr, " Error dat_psp_free: %s\n",\r
- DT_RetToString(ret));\r
- goto bail;\r
- } else {\r
- LOGPRINTF(" psp_free[%d] complete\n",i);\r
- }\r
- if (!multi_listens)\r
- break;\r
- }\r
- }\r
-\r
- /* free EVDs */\r
- LOGPRINTF(" destroy events\n");\r
- ret = destroy_events();\r
- if (ret != DAT_SUCCESS) {\r
- fprintf(stderr, " Error destroy_events: %s\n",\r
- DT_RetToString(ret));\r
- goto bail;\r
- } else\r
- LOGPRINTF(" destroy events done\n");\r
-\r
-\r
- /* Free protection domain */\r
- LOGPRINTF(" Freeing pz\n");\r
- start = get_time();\r
- ret = dat_pz_free(h_pz);\r
- stop = get_time();\r
- ts.pzf += ((stop - start) * 1.0e6);\r
- if (ret != DAT_SUCCESS) {\r
- fprintf(stderr, " Error freeing PZ: %s\n",\r
- DT_RetToString(ret));\r
- goto bail;\r
- } else {\r
- LOGPRINTF(" Freed pz\n");\r
- h_pz = NULL;\r
- }\r
-\r
- /* close the device */\r
- LOGPRINTF(" Closing Interface Adaptor\n");\r
- start = get_time();\r
- ret = dat_ia_close(h_ia, DAT_CLOSE_ABRUPT_FLAG);\r
- stop = get_time();\r
- ts.close += ((stop - start) * 1.0e6);\r
- if (ret != DAT_SUCCESS) {\r
- fprintf(stderr, " Error Adaptor close: %s\n",\r
- DT_RetToString(ret));\r
- goto bail;\r
- } else\r
- LOGPRINTF(" Closed Interface Adaptor\n");\r
-\r
- printf(" DAPL Connection Test Complete.\n");\r
- printf(" open: %10.2lf usec\n", ts.open);\r
- printf(" close: %10.2lf usec\n", ts.close);\r
- printf(" PZ create: %10.2lf usec\n", ts.pzc);\r
- printf(" PZ free: %10.2lf usec\n", ts.pzf);\r
- printf(" LMR create:%10.2lf usec\n", ts.reg);\r
- printf(" LMR free: %10.2lf usec\n", ts.unreg);\r
- printf(" EVD create:%10.2lf usec\n", ts.evdc);\r
- printf(" EVD free: %10.2lf usec\n", ts.evdf);\r
- printf(" EP create: %10.2lf usec avg\n", ts.epc/connections);\r
- printf(" EP free: %10.2lf usec avg\n", ts.epf/connections);\r
- if (!server) {\r
- printf(" Connections: %8.2lf usec, CPS %7.2lf "\r
- "Total %4.2lf secs, poll_cnt=%u, Num=%d\n", \r
- (double)(ts.conn/connections), \r
- (double)(1/(ts.conn/1000000/connections)), \r
- (double)(ts.conn/1000000), \r
- conn_poll_count, connections);\r
- }\r
- printf(" TOTAL: %4.2lf sec\n", ts.total/1000000);\r
- fflush(stderr); fflush(stdout);\r
-bail:\r
- free(h_ep);\r
- free(h_psp);\r
-\r
-#if defined(_WIN32) || defined(_WIN64)\r
- WSACleanup();\r
-#endif\r
- return (0);\r
-}\r
-\r
-double get_time(void)\r
-{\r
- struct timeval tp;\r
-\r
- gettimeofday(&tp, NULL);\r
- return ((double)tp.tv_sec + (double)tp.tv_usec * 1e-6);\r
-}\r
-\r
-DAT_RETURN conn_server()\r
-{\r
- DAT_RETURN ret;\r
- DAT_EVENT event;\r
- DAT_COUNT nmore;\r
- int i,bi;\r
- unsigned char *buf;\r
- DAT_CR_ARRIVAL_EVENT_DATA *cr_event =\r
- &event.event_data.cr_arrival_event_data;\r
- DAT_CR_PARAM cr_param = { 0 };\r
- \r
- printf(" Accepting...\n");\r
- for (i = 0; i < connections; i++) {\r
- \r
- /* poll for CR's */\r
- if (!waiting) {\r
- cr_poll_count = 0;\r
- while (DAT_GET_TYPE(dat_evd_dequeue(h_cr_evd, &event)) \r
- == DAT_QUEUE_EMPTY)\r
- cr_poll_count++;\r
- } else {\r
- ret = dat_evd_wait(h_cr_evd, CR_TIMEOUT, \r
- 1, &event, &nmore);\r
- if (ret != DAT_SUCCESS) {\r
- fprintf(stderr,\r
- " ERR: CR dat_evd_wait() %s\n",\r
- DT_RetToString(ret));\r
- return ret;\r
- }\r
- }\r
- \r
- if ((event.event_number != DAT_CONNECTION_REQUEST_EVENT) &&\r
- (ud_test && event.event_number != \r
- DAT_IB_UD_CONNECTION_REQUEST_EVENT)) {\r
- fprintf(stderr, " Error unexpected CR event : %s\n",\r
- DT_EventToSTr(event.event_number));\r
- return (DAT_ABORT);\r
- }\r
-\r
- \r
- /* use to test rdma_cma timeout logic */\r
-#if defined(_WIN32) || defined(_WIN64)\r
- if (delay) {\r
- printf(" Accept delayed by %d seconds...\n", delay);\r
- Sleep(delay * 1000);\r
- }\r
-#else\r
- if (delay) {\r
- printf(" Accept delayed by %d seconds...\n", delay);\r
- sleep(delay);\r
- }\r
-#endif\r
- /* accept connect request from client */\r
- h_cr = cr_event->cr_handle;\r
- LOGPRINTF(" Accepting connect request from client\n");\r
-\r
- /* private data - check and send it back */\r
- dat_cr_query(h_cr, DAT_CSP_FIELD_ALL, &cr_param);\r
-\r
- buf = (unsigned char *)cr_param.private_data;\r
- LOGPRINTF(" CONN REQUEST Private Data %p[0]=%d [47]=%d\n",\r
- buf, buf[0], buf[47]);\r
- \r
- for (bi = 0; bi < 48; bi++) {\r
- if (buf[bi] != bi + 1) {\r
- fprintf(stderr, " ERR on CONNECT REQUEST"\r
- " private data: %p[%d]=%d s/be %d\n",\r
- buf, bi, buf[bi], bi + 1);\r
- dat_cr_reject(h_cr, 0, NULL);\r
- return (DAT_ABORT);\r
- }\r
- buf[bi]++; /* change for trip back */\r
- }\r
-\r
-#ifdef TEST_REJECT_WITH_PRIVATE_DATA\r
- printf(" REJECT request with 48 bytes of private data\n");\r
- ret = dat_cr_reject(h_cr, 48, cr_param.private_data);\r
- printf("\n DAPL Test Complete. %s\n\n",\r
- ret ? "FAILED" : "PASSED");\r
- exit(0);\r
-#endif\r
- ret = dat_cr_accept(h_cr, h_ep[i], 48, \r
- cr_param.private_data);\r
-\r
- if (ret != DAT_SUCCESS) {\r
- fprintf(stderr, " ERR dat_cr_accept: %s\n",\r
- DT_RetToString(ret));\r
- return (ret);\r
- } else\r
- LOGPRINTF(" accept[%d] complete\n", i);\r
- \r
- event.event_number = 0;\r
- }\r
-\r
- /* process the RTU, ESTABLISHMENT event */\r
- printf(" Completing...\n");\r
- for (i=0;i<connections;i++) {\r
- \r
- /* process completions */\r
- if (!waiting) {\r
- conn_poll_count = 0;\r
- while (DAT_GET_TYPE(dat_evd_dequeue(h_conn_evd, \r
- &event)) == DAT_QUEUE_EMPTY)\r
- conn_poll_count++;\r
- } else {\r
- ret = dat_evd_wait(h_conn_evd, CONN_TIMEOUT, \r
- 1, &event, &nmore);\r
- if (ret != DAT_SUCCESS) {\r
- fprintf(stderr,\r
- " ERR: CONN evd_wait() %s\n",\r
- DT_RetToString(ret));\r
- return ret;\r
- }\r
- }\r
- if ((event.event_number != DAT_CONNECTION_EVENT_ESTABLISHED) &&\r
- (ud_test && event.event_number != \r
- DAT_IB_UD_CONNECTION_EVENT_ESTABLISHED)) {\r
-\r
- fprintf(stderr, " Error unexpected CR EST "\r
- "event : 0x%x %s\n",\r
- event.event_number,\r
- DT_EventToSTr(event.event_number));\r
- return (DAT_ABORT);\r
- }\r
- event.event_number = 0;\r
- LOGPRINTF(" CONN_EST[%d] complete\n", i);\r
- }\r
-\r
- printf("\n ALL %d CONNECTED on Server!\n\n", connections);\r
- return DAT_SUCCESS;\r
-}\r
-\r
- \r
-DAT_RETURN conn_client() \r
-{\r
- DAT_IA_ADDRESS_PTR raddr = (DAT_IA_ADDRESS_PTR)&remote;\r
- DAT_RETURN ret;\r
- DAT_EVENT event;\r
- DAT_COUNT nmore;\r
- DAT_CONN_QUAL conn_id;\r
- DAT_CONNECTION_EVENT_DATA *conn_event =\r
- &event.event_data.connect_event_data;\r
- int i,ii,bi;\r
- unsigned char *buf;\r
- unsigned char pdata[48] = { 0 };\r
- struct addrinfo *target;\r
- int rval;\r
-\r
- if (ucm)\r
- goto no_resolution;\r
-\r
-#if defined(_WIN32) || defined(_WIN64)\r
- if ((rval = getaddrinfo(hostname, "ftp", NULL, &target)) != 0) {\r
- printf("\n remote name resolution failed! %s\n",\r
- gai_strerror(rval));\r
- exit(1);\r
- }\r
- rval = ((struct sockaddr_in *)target->ai_addr)->sin_addr.s_addr;\r
-#else\r
- if (getaddrinfo(hostname, NULL, NULL, &target) != 0) {\r
- perror("\n remote name resolution failed!");\r
- exit(1);\r
- }\r
- rval = ((struct sockaddr_in *)target->ai_addr)->sin_addr.s_addr;\r
-#endif\r
- printf(" Connecting to Server: %s \n", hostname);\r
- printf(" Address: %d.%d.%d.%d port %d\n",\r
- (rval >> 0) & 0xff, (rval >> 8) & 0xff,\r
- (rval >> 16) & 0xff, (rval >> 24) & 0xff, \r
- port_id);\r
-\r
- raddr = (DAT_IA_ADDRESS_PTR)target->ai_addr;\r
- \r
-no_resolution:\r
-\r
- for (i = 0; i < 48; i++) /* simple pattern in private data */\r
- pdata[i] = i + 1;\r
-\r
- printf(" Connecting...\n");\r
- start = get_time();\r
- for (i = 0; i < connections; i += burst) {\r
- for (ii = 0; ii < burst; ii++) { /* conn_reqs */\r
- if (multi_listens)\r
- conn_id = port_id + ii;\r
- else\r
- conn_id = port_id;\r
-\r
- ret = dat_ep_connect(h_ep[i+ii], raddr, \r
- conn_id, CONN_TIMEOUT,\r
- 48, (DAT_PVOID) pdata, 0, \r
- DAT_CONNECT_DEFAULT_FLAG);\r
- if (ret != DAT_SUCCESS) {\r
- fprintf(stderr, " ERR dat_ep_connect: %s\n",\r
- DT_RetToString(ret));\r
- return (ret);\r
- } else\r
- LOGPRINTF(" dat_ep_connect[%d] complete\n", \r
- i+ii);\r
-\r
- } \r
- for (ii = 0; ii < burst; ii++) { /* conn_events */\r
- if (!waiting) {\r
- conn_poll_count = 0;\r
- while (DAT_GET_TYPE(dat_evd_dequeue(\r
- h_conn_evd, &event)) == \r
- DAT_QUEUE_EMPTY)\r
- conn_poll_count++;\r
- } else {\r
- ret = dat_evd_wait(h_conn_evd, CONN_TIMEOUT, \r
- 1, &event, &nmore);\r
-\r
- if (ret != DAT_SUCCESS) {\r
- fprintf(stderr,\r
- " ERR: CONN evd_wait() %s\n",\r
- DT_RetToString(ret));\r
- return ret;\r
- }\r
- }\r
-\r
-#ifdef TEST_REJECT_WITH_PRIVATE_DATA\r
- if (event.event_number != \r
- DAT_CONNECTION_EVENT_PEER_REJECTED) {\r
- fprintf(stderr, " expected conn reject "\r
- "event : %s\n",\r
- DT_EventToSTr(event.event_number));\r
- return (DAT_ABORT);\r
- }\r
-\r
- /* get the reject private data and validate */\r
- buf = (unsigned char *)conn_event->private_data;\r
- printf(" Recv REJ with pdata %p[0]=%d [47]=%d\n",\r
- buf, buf[0], buf[47]);\r
- for (bi = 0; bi < 48; bi++) {\r
- if (buf[bi] != idx + 2) {\r
- fprintf(stderr, " client: Error"\r
- " with REJECT event private"\r
- " data: %p[%d]=%d s/be %d\n",\r
- buf, bi, \r
- buf[bi], bi + 2);\r
- dat_ep_disconnect(h_ep[i+ii], 0);\r
- return (DAT_ABORT);\r
- }\r
- }\r
- printf("\n Rej Test Done. PASSED\n\n");\r
- exit(0);\r
-#endif\r
- if ((event.event_number != \r
- DAT_CONNECTION_EVENT_ESTABLISHED) &&\r
- (ud_test && event.event_number != \r
- DAT_IB_UD_CONNECTION_EVENT_ESTABLISHED)) {\r
- fprintf(stderr, " Error unexpected conn "\r
- "event : 0x%x %s\n",\r
- event.event_number,\r
- DT_EventToSTr(event.event_number));\r
- return (DAT_ABORT);\r
- }\r
-\r
- /* check private data back from server */\r
- buf = (unsigned char *)conn_event->private_data;\r
-\r
- LOGPRINTF(" CONN[%d] Private Data "\r
- "%p[0]=%d [47]=%d\n",\r
- i+ii, buf, buf[0], buf[47]);\r
-\r
- for (bi = 0; bi < 48; bi++) {\r
- if (buf[bi] != bi + 2) {\r
- DAT_COUNT nmore;\r
- fprintf(stderr, " ERR CONN event"\r
- " pdata: %p[%d]=%d s/be %d\n",\r
- buf, bi, buf[bi], \r
- bi + 2);\r
- dat_ep_disconnect(h_ep[i+ii], \r
- DAT_CLOSE_ABRUPT_FLAG);\r
- LOGPRINTF(" waiting for disc...\n");\r
- dat_evd_wait(h_conn_evd, \r
- DAT_TIMEOUT_INFINITE,\r
- 1, &event, &nmore); \r
- return (DAT_ABORT);\r
- }\r
- }\r
- event.event_number = 0;\r
- } \r
- }\r
-\r
- stop = get_time();\r
- ts.conn += ((stop - start) * 1.0e6);\r
-\r
- if (!ucm)\r
- freeaddrinfo(target);\r
-\r
- printf("\n ALL %d CONNECTED on Client!\n\n", connections);\r
-\r
- return (DAT_SUCCESS);\r
-}\r
-\r
-/* validate disconnected EP's and free them */\r
-DAT_RETURN disconnect_eps(void)\r
-{\r
- DAT_RETURN ret;\r
- DAT_EVENT event;\r
- DAT_COUNT nmore;\r
- int i,ii;\r
- DAT_CONNECTION_EVENT_DATA *conn_event =\r
- &event.event_data.connect_event_data;\r
-\r
- if (!connected)\r
- return DAT_SUCCESS;\r
-\r
- /* UD, no connection to disconnect, just free EP's */\r
- if (ud_test) {\r
- for (i = 0; i < connections; i++) {\r
- ret = dat_ep_free(h_ep[i]);\r
- if (ret != DAT_SUCCESS) {\r
- fprintf(stderr, \r
- " ERR free EP[%d] %p: %s\n",\r
- i, h_ep[i], DT_RetToString(ret));\r
- } else {\r
- LOGPRINTF(" Freed EP[%d] %p\n", \r
- i, h_ep[i]);\r
- h_ep[i] = DAT_HANDLE_NULL;\r
- }\r
- }\r
- stop = get_time();\r
- ts.epf += ((stop - start) * 1.0e6);\r
- ts.total += ts.epf;\r
- return DAT_SUCCESS;\r
- }\r
- \r
- /* \r
- * Only the client needs to call disconnect. The server _should_ be able\r
- * to just wait on the EVD associated with connection events for a\r
- * disconnect request and then exit.\r
- */\r
- if (!server) {\r
- start = get_time();\r
- for (i = 0; i < connections; i++) {\r
- LOGPRINTF(" dat_ep_disconnect\n");\r
- ret = dat_ep_disconnect(h_ep[i], \r
- DAT_CLOSE_DEFAULT);\r
- if (ret != DAT_SUCCESS) {\r
- fprintf(stderr,\r
- " Error disconnect: %s\n",\r
- DT_RetToString(ret));\r
- return ret;\r
- } else {\r
- LOGPRINTF(" disconnect completed\n");\r
- }\r
- }\r
- } else {\r
- LOGPRINTF(" Server waiting for disconnect...\n");\r
- }\r
-\r
- LOGPRINTF(" Wait for Disc event, free EPs as completed\n");\r
- start = get_time();\r
- for (i = 0; i < connections; i++) {\r
- event.event_number = 0;\r
- conn_event->ep_handle = NULL;\r
- ret = dat_evd_wait(h_conn_evd, DAT_TIMEOUT_INFINITE, \r
- 1, &event, &nmore);\r
- if (ret != DAT_SUCCESS) {\r
- fprintf(stderr, " Error dat_evd_wait: %s\n",\r
- DT_RetToString(ret));\r
- return ret;\r
- } else {\r
- LOGPRINTF(" disc event[%d] complete,"\r
- " check for valid EP...\n", i);\r
- }\r
-\r
- /* check for valid EP in creation list */\r
- for (ii = 0; ii < connections; ii++) {\r
- if (h_ep[ii] == conn_event->ep_handle) {\r
- LOGPRINTF(" valid EP[%d] %p !\n", \r
- ii, h_ep[ii]);\r
- ret = dat_ep_free(h_ep[ii]);\r
- if (ret != DAT_SUCCESS) {\r
- fprintf(stderr, \r
- " ERR free EP[%d] %p: %s\n",\r
- i, h_ep[ii], \r
- DT_RetToString(ret));\r
- } else {\r
- LOGPRINTF(" Freed EP[%d] %p\n", \r
- i, h_ep[ii]);\r
- h_ep[ii] = DAT_HANDLE_NULL;\r
- }\r
- break;\r
- } else {\r
- continue;\r
- }\r
- }\r
- if (ii == connections) {\r
- LOGPRINTF(" %s: invalid EP[%d] %p via DISC event!\n", \r
- server ? "Server" : "Client", \r
- i, conn_event->ep_handle);\r
- return DAT_INVALID_HANDLE;\r
- }\r
- }\r
- /* free EPs */\r
- stop = get_time();\r
- ts.epf += ((stop - start) * 1.0e6);\r
- ts.total += ts.epf;\r
- return DAT_SUCCESS;\r
-}\r
-\r
-\r
- /*\r
- * Create CR, CONN, and DTO events\r
- */\r
-DAT_RETURN create_events(void)\r
-{\r
- DAT_RETURN ret;\r
-\r
- /* create cr EVD */\r
- start = get_time();\r
- ret = dat_evd_create(h_ia, connections, DAT_HANDLE_NULL, \r
- DAT_EVD_CR_FLAG, &h_cr_evd);\r
- stop = get_time();\r
- ts.evdc += ((stop - start) * 1.0e6);\r
- ts.total += ts.evdc;\r
- if (ret != DAT_SUCCESS) {\r
- fprintf(stderr, " Error dat_evd_create: %s\n",\r
- DT_RetToString(ret));\r
- return (ret);\r
- } else {\r
- LOGPRINTF(" cr_evd created %p\n", h_cr_evd);\r
- }\r
-\r
- /* create conn EVD */\r
- ret = dat_evd_create(h_ia,\r
- connections*2,\r
- DAT_HANDLE_NULL,\r
- DAT_EVD_CONNECTION_FLAG, &h_conn_evd);\r
- if (ret != DAT_SUCCESS) {\r
- fprintf(stderr, " Error dat_evd_create: %s\n",\r
- DT_RetToString(ret));\r
- return (ret);\r
- } else {\r
- LOGPRINTF(" con_evd created %p\n", h_conn_evd);\r
- }\r
-\r
- /* create dto SND EVD */\r
- ret = dat_evd_create(h_ia, 1, NULL, \r
- DAT_EVD_DTO_FLAG, &h_dto_req_evd);\r
- if (ret != DAT_SUCCESS) {\r
- fprintf(stderr, " Error dat_evd_create REQ: %s\n",\r
- DT_RetToString(ret));\r
- return (ret);\r
- } else {\r
- LOGPRINTF(" dto_req_evd created %p\n", \r
- h_dto_req_evd);\r
- }\r
-\r
- /* create dto RCV EVD */\r
- ret = dat_evd_create(h_ia, 1, NULL,\r
- DAT_EVD_DTO_FLAG, &h_dto_rcv_evd);\r
- if (ret != DAT_SUCCESS) {\r
- fprintf(stderr, " Error dat_evd_create RCV: %s\n",\r
- DT_RetToString(ret));\r
- return (ret);\r
- } else {\r
- LOGPRINTF(" dto_rcv_evd created %p\n", \r
- h_dto_rcv_evd);\r
- }\r
- return DAT_SUCCESS;\r
-}\r
-\r
-/*\r
- * Destroy CR, CONN, CNO, and DTO events\r
- */\r
-\r
-DAT_RETURN destroy_events(void)\r
-{\r
- DAT_RETURN ret;\r
-\r
- /* free cr EVD */\r
- if (h_cr_evd != DAT_HANDLE_NULL) {\r
- LOGPRINTF(" Free cr EVD %p \n", h_cr_evd);\r
- ret = dat_evd_free(h_cr_evd);\r
- if (ret != DAT_SUCCESS) {\r
- fprintf(stderr, " Error freeing cr EVD: %s\n",\r
- DT_RetToString(ret));\r
- return (ret);\r
- } else {\r
- LOGPRINTF(" Freed cr EVD\n");\r
- h_cr_evd = DAT_HANDLE_NULL;\r
- }\r
- }\r
-\r
- /* free conn EVD */\r
- if (h_conn_evd != DAT_HANDLE_NULL) {\r
- LOGPRINTF(" Free conn EVD %p\n", h_conn_evd);\r
- ret = dat_evd_free(h_conn_evd);\r
- if (ret != DAT_SUCCESS) {\r
- fprintf(stderr, " Error freeing conn EVD: %s\n",\r
- DT_RetToString(ret));\r
- return (ret);\r
- } else {\r
- LOGPRINTF(" Freed conn EVD\n");\r
- h_conn_evd = DAT_HANDLE_NULL;\r
- }\r
- }\r
-\r
- /* free RCV dto EVD */\r
- if (h_dto_rcv_evd != DAT_HANDLE_NULL) {\r
- LOGPRINTF(" Free RCV dto EVD %p\n", h_dto_rcv_evd);\r
- start = get_time();\r
- ret = dat_evd_free(h_dto_rcv_evd);\r
- stop = get_time();\r
- ts.evdf += ((stop - start) * 1.0e6);\r
- ts.total += ts.evdf;\r
- if (ret != DAT_SUCCESS) {\r
- fprintf(stderr, " Error freeing dto EVD: %s\n",\r
- DT_RetToString(ret));\r
- return (ret);\r
- } else {\r
- LOGPRINTF(" Freed dto EVD\n");\r
- h_dto_rcv_evd = DAT_HANDLE_NULL;\r
- }\r
- }\r
-\r
- /* free REQ dto EVD */\r
- if (h_dto_req_evd != DAT_HANDLE_NULL) {\r
- LOGPRINTF(" Free REQ dto EVD %p\n", h_dto_req_evd);\r
- ret = dat_evd_free(h_dto_req_evd);\r
- if (ret != DAT_SUCCESS) {\r
- fprintf(stderr, " Error freeing dto EVD: %s\n",\r
- DT_RetToString(ret));\r
- return (ret);\r
- } else {\r
- LOGPRINTF(" Freed dto EVD\n");\r
- h_dto_req_evd = DAT_HANDLE_NULL;\r
- }\r
- }\r
-\r
- return DAT_SUCCESS;\r
-}\r
-\r
-/*\r
- * Map DAT_RETURN values to readable strings,\r
- * but don't assume the values are zero-based or contiguous.\r
- */\r
-char errmsg[512] = { 0 };\r
-const char *DT_RetToString(DAT_RETURN ret_value)\r
-{\r
- const char *major_msg, *minor_msg;\r
-\r
- dat_strerror(ret_value, &major_msg, &minor_msg);\r
-\r
- strcpy(errmsg, major_msg);\r
- strcat(errmsg, " ");\r
- strcat(errmsg, minor_msg);\r
-\r
- return errmsg;\r
-}\r
-\r
-/*\r
- * Map DAT_EVENT_CODE values to readable strings\r
- */\r
-const char *DT_EventToSTr(DAT_EVENT_NUMBER event_code)\r
-{\r
- unsigned int i;\r
- static struct {\r
- const char *name;\r
- DAT_RETURN value;\r
- } dat_events[] = {\r
-# define DATxx(x) { # x, x }\r
- DATxx(DAT_DTO_COMPLETION_EVENT),\r
- DATxx(DAT_RMR_BIND_COMPLETION_EVENT),\r
- DATxx(DAT_CONNECTION_REQUEST_EVENT),\r
- DATxx(DAT_CONNECTION_EVENT_ESTABLISHED),\r
- DATxx(DAT_CONNECTION_EVENT_PEER_REJECTED),\r
- DATxx(DAT_CONNECTION_EVENT_NON_PEER_REJECTED),\r
- DATxx(DAT_CONNECTION_EVENT_ACCEPT_COMPLETION_ERROR),\r
- DATxx(DAT_CONNECTION_EVENT_DISCONNECTED),\r
- DATxx(DAT_CONNECTION_EVENT_BROKEN),\r
- DATxx(DAT_CONNECTION_EVENT_TIMED_OUT),\r
- DATxx(DAT_CONNECTION_EVENT_UNREACHABLE),\r
- DATxx(DAT_ASYNC_ERROR_EVD_OVERFLOW),\r
- DATxx(DAT_ASYNC_ERROR_IA_CATASTROPHIC),\r
- DATxx(DAT_ASYNC_ERROR_EP_BROKEN),\r
- DATxx(DAT_ASYNC_ERROR_TIMED_OUT),\r
- DATxx(DAT_ASYNC_ERROR_PROVIDER_INTERNAL_ERROR),\r
- DATxx(DAT_SOFTWARE_EVENT)\r
-# undef DATxx\r
- };\r
-# define NUM_EVENTS (sizeof(dat_events)/sizeof(dat_events[0]))\r
-\r
- for (i = 0; i < NUM_EVENTS; i++) {\r
- if (dat_events[i].value == event_code) {\r
- return (dat_events[i].name);\r
- }\r
- }\r
-\r
- return ("Invalid_DAT_EVENT_NUMBER");\r
-}\r
-\r
-void print_usage(void)\r
-{\r
- printf("\n DAPL USAGE \n\n");\r
- printf("s: server\n");\r
- printf("c: connections (default = 1000)\n");\r
- printf("v: verbose\n");\r
- printf("w: wait on event (default, polling)\n");\r
- printf("d: delay before accept\n");\r
- printf("h: hostname/address of server, specified on client\n");\r
- printf("P: provider name (default = OpenIB-v2-ib0)\n");\r
- printf("\n");\r
-}\r
-\r
+/*
+ * Copyright (c) 2009 Intel Corporation. All rights reserved.
+ *
+ * This software is available to you under a choice of one of two
+ * licenses. You may choose to be licensed under the terms of the GNU
+ * General Public License (GPL) Version 2, available from the file
+ * COPYING in the main directory of this source tree, or the
+ * OpenIB.org BSD license below:
+ *
+ * Redistribution and use in source and binary forms, with or
+ * without modification, are permitted provided that the following
+ * conditions are met:
+ *
+ * - Redistributions of source code must retain the above
+ * copyright notice, this list of conditions and the following
+ * disclaimer.
+ *
+ * - Redistributions in binary form must reproduce the above
+ * copyright notice, this list of conditions and the following
+ * disclaimer in the documentation and/or other materials
+ * provided with the distribution.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
+ * EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
+ * MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
+ * NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS
+ * BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN
+ * ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN
+ * CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+ * SOFTWARE.
+ *
+ * $Id: $
+ */
+#include <stdio.h>
+#include <stdlib.h>
+#include <string.h>
+
+#ifdef DAPL_PROVIDER
+#undef DAPL_PROVIDER
+#endif
+
+#if defined(_WIN32) || defined(_WIN64)
+
+#include <windows.h>
+#include <winsock2.h>
+#include <ws2tcpip.h>
+#include <io.h>
+#include <process.h>
+#include <complib/cl_types.h>
+#include "..\..\..\..\etc\user\getopt.c"
+
+#define getpid() ((int)GetCurrentProcessId())
+#define F64x "%I64x"
+#define F64d "%I64d"
+
+#ifdef DBG
+#define DAPL_PROVIDER "ibnic0v2d"
+#else
+#define DAPL_PROVIDER "ibnic0v2"
+#endif
+
+#define ntohll _byteswap_uint64
+#define htonll _byteswap_uint64
+
+#else // _WIN32 || _WIN64
+
+#include <endian.h>
+#include <byteswap.h>
+#include <netdb.h>
+#include <sys/types.h>
+#include <sys/socket.h>
+#include <sys/time.h>
+#include <netinet/in.h>
+#include <netinet/tcp.h>
+#include <arpa/inet.h>
+#include <sys/mman.h>
+#include <getopt.h>
+#include <inttypes.h>
+#include <unistd.h>
+#include <stdlib.h>
+
+#define DAPL_PROVIDER "ofa-v2-ib0"
+
+#define F64x "%"PRIx64""
+#define F64d "%"PRId64""
+
+
+#if __BYTE_ORDER == __BIG_ENDIAN
+#define htonll(x) (x)
+#define ntohll(x) (x)
+#elif __BYTE_ORDER == __LITTLE_ENDIAN
+#define htonll(x) bswap_64(x)
+#define ntohll(x) bswap_64(x)
+#endif
+
+#endif // _WIN32 || _WIN64
+
+#define MAX_POLLING_CNT 50000
+
+/* Header files needed for DAT/uDAPL */
+#include "dat2/udat.h"
+#include "dat2/dat_ib_extensions.h"
+
+/* definitions */
+#define SERVER_CONN_QUAL 45248
+#define CONN_TIMEOUT (1000*1000*100)
+#define CR_TIMEOUT DAT_TIMEOUT_INFINITE
+
+/* Global DAT vars */
+static DAT_IA_HANDLE h_ia = DAT_HANDLE_NULL;
+static DAT_PZ_HANDLE h_pz = DAT_HANDLE_NULL;
+static DAT_EP_HANDLE *h_ep;
+static DAT_PSP_HANDLE *h_psp;
+static DAT_CR_HANDLE h_cr = DAT_HANDLE_NULL;
+
+static DAT_EVD_HANDLE h_async_evd = DAT_HANDLE_NULL;
+static DAT_EVD_HANDLE h_dto_req_evd = DAT_HANDLE_NULL;
+static DAT_EVD_HANDLE h_dto_rcv_evd = DAT_HANDLE_NULL;
+static DAT_EVD_HANDLE h_cr_evd = DAT_HANDLE_NULL;
+static DAT_EVD_HANDLE h_conn_evd = DAT_HANDLE_NULL;
+
+static DAT_EP_ATTR ep_attr;
+char hostname[256] = { 0 };
+char provider[64] = DAPL_PROVIDER;
+char addr_str[INET_ADDRSTRLEN];
+
+int status;
+
+/* timers */
+double start, stop, total_us, total_sec;
+
+struct dt_time {
+ double total;
+ double open;
+ double reg;
+ double unreg;
+ double pzc;
+ double pzf;
+ double evdc;
+ double evdf;
+ double cnoc;
+ double cnof;
+ double epc;
+ double epf;
+ double rtt;
+ double close;
+ double conn;
+};
+
+struct dt_time ts;
+
+/* defaults */
+static int connected = 0;
+static int multi_listens = 0;
+static int ud_test = 0;
+static int server = 1;
+static int waiting = 0;
+static int verbose = 0;
+static int cr_poll_count = 0;
+static int conn_poll_count = 0;
+static int delay = 0;
+static int connections = 1000;
+static int burst = 100;
+static int port_id = SERVER_CONN_QUAL;
+static int ucm = 0;
+static DAT_SOCK_ADDR6 remote;
+
+/* forward prototypes */
+const char *DT_RetToString(DAT_RETURN ret_value);
+const char *DT_EventToSTr(DAT_EVENT_NUMBER event_code);
+void print_usage(void);
+double get_time(void);
+DAT_RETURN conn_client(void);
+DAT_RETURN conn_server(void);
+DAT_RETURN disconnect_eps(void);
+DAT_RETURN create_events(void);
+DAT_RETURN destroy_events(void);
+
+#define LOGPRINTF if (verbose) printf
+
+void flush_evds(void)
+{
+ DAT_EVENT event;
+
+ /* Flush async error queue */
+ printf(" ERR: Checking ASYNC EVD...\n");
+ while (dat_evd_dequeue(h_async_evd, &event) == DAT_SUCCESS) {
+ printf(" ASYNC EVD ENTRY: handle=%p reason=%d\n",
+ event.event_data.asynch_error_event_data.dat_handle,
+ event.event_data.asynch_error_event_data.reason);
+ }
+}
+
+static void print_ia_address(struct sockaddr *sa)
+{
+ char str[INET6_ADDRSTRLEN] = {" ??? "};
+
+ switch(sa->sa_family) {
+ case AF_INET:
+ inet_ntop(AF_INET, &((struct sockaddr_in *)sa)->sin_addr, str, INET6_ADDRSTRLEN);
+ printf("%d Local Address AF_INET - %s port %d\n", getpid(), str, SERVER_CONN_QUAL);
+ break;
+ case AF_INET6:
+ inet_ntop(AF_INET6, &((struct sockaddr_in6 *)sa)->sin6_addr, str, INET6_ADDRSTRLEN);
+ printf("%d Local Address AF_INET6 - %s flowinfo(QPN)=0x%x, port(LID)=0x%x\n",
+ getpid(), str,
+ ntohl(((struct sockaddr_in6 *)sa)->sin6_flowinfo),
+ ntohs(((struct sockaddr_in6 *)sa)->sin6_port));
+ break;
+ default:
+ printf("%d Local Address UNKOWN FAMILY - port %d\n", getpid(), SERVER_CONN_QUAL);
+ }
+}
+
+int main(int argc, char **argv)
+{
+ int i, c, len;
+ DAT_RETURN ret;
+ DAT_IA_ATTR ia_attr;
+
+ /* parse arguments */
+ while ((c = getopt(argc, argv, "smwvub:c:d:h:P:p:q:l:")) != -1) {
+ switch (c) {
+ case 's':
+ server = 1;
+ break;
+ case 'm':
+ multi_listens = 1;
+ break;
+ case 'w':
+ waiting = 1;
+ break;
+ case 'u':
+ ud_test = 1;
+ break;
+ case 'c':
+ connections = atoi(optarg);
+ break;
+ case 'p':
+ port_id = atoi(optarg);
+ break;
+ case 'v':
+ verbose = 1;
+ fflush(stdout);
+ break;
+ case 'd':
+ delay = atoi(optarg);
+ break;
+ case 'b':
+ burst = atoi(optarg);
+ break;
+ case 'h':
+ server = 0;
+ strcpy(hostname, optarg);
+ break;
+ case 'P':
+ strcpy(provider, optarg);
+ break;
+ case 'q':
+ /* map UCM qpn into AF_INET6 sin6_flowinfo */
+ remote.sin6_family = AF_INET6;
+ remote.sin6_flowinfo = htonl(strtol(optarg,NULL,0));
+ ucm = 1;
+ server = 0;
+ break;
+ case 'l':
+ /* map UCM lid into AF_INET6 sin6_port */
+ remote.sin6_family = AF_INET6;
+ remote.sin6_port = htons(strtol(optarg,NULL,0));
+ ucm = 1;
+ server = 0;
+ break;
+ default:
+ print_usage();
+ exit(-12);
+ }
+ }
+
+#if defined(_WIN32) || defined(_WIN64)
+ {
+ WSADATA wsaData;
+
+ i = WSAStartup(MAKEWORD(2, 2), &wsaData);
+ if (i != 0) {
+ printf("%s WSAStartup(2.2) failed? (0x%x)\n", argv[0],
+ i);
+ fflush(stdout);
+ exit(1);
+ }
+ }
+#endif
+
+ if (!server) {
+ printf(" Running client on %s with %d %s connections\n",
+ provider, connections, ud_test ? "UD" : "RC");
+ } else {
+ printf(" Running server on %s with %d %s connections\n",
+ provider, connections, ud_test ? "UD" : "RC");
+ }
+ fflush(stdout);
+
+ if (burst > connections)
+ burst = connections;
+
+
+ /* allocate EP handles for all connections */
+ h_ep = (DAT_EP_HANDLE*)malloc(connections * sizeof(DAT_EP_HANDLE));
+ if (h_ep == NULL) {
+ perror("malloc ep");
+ exit(1);
+ }
+ memset(h_ep, 0, (burst * sizeof(DAT_PSP_HANDLE)));
+
+ /* allocate PSP handles, check for multi-listens */
+ if (multi_listens)
+ len = burst * sizeof(DAT_PSP_HANDLE);
+ else
+ len = sizeof(DAT_PSP_HANDLE);
+
+ h_psp = (DAT_PSP_HANDLE*)malloc(len);
+ if (h_psp == NULL) {
+ perror("malloc psp");
+ exit(1);
+ }
+ memset(h_psp, 0, len);
+ memset(&ts, 0, sizeof(struct dt_time));
+
+ /* dat_ia_open, dat_pz_create */
+ h_async_evd = DAT_HANDLE_NULL;
+ start = get_time();
+ ret = dat_ia_open(provider, 8, &h_async_evd, &h_ia);
+ stop = get_time();
+ ts.open += ((stop - start) * 1.0e6);
+ if (ret != DAT_SUCCESS) {
+ fprintf(stderr, " Error Adaptor open: %s\n",
+ DT_RetToString(ret));
+ exit(1);
+ } else
+ LOGPRINTF(" Opened Interface Adaptor\n");
+
+ /* query for UCM addressing */
+ ret = dat_ia_query(h_ia, 0, DAT_IA_FIELD_ALL, &ia_attr, 0, 0);
+ if (ret != DAT_SUCCESS) {
+ fprintf(stderr, "%d: Error Adaptor query: %s\n",
+ getpid(), DT_RetToString(ret));
+ exit(1);
+ }
+ print_ia_address(ia_attr.ia_address_ptr);
+
+ /* Create Protection Zone */
+ start = get_time();
+ LOGPRINTF(" Create Protection Zone\n");
+ ret = dat_pz_create(h_ia, &h_pz);
+ stop = get_time();
+ ts.pzc += ((stop - start) * 1.0e6);
+ if (ret != DAT_SUCCESS) {
+ fprintf(stderr, " Error creating Protection Zone: %s\n",
+ DT_RetToString(ret));
+ exit(1);
+ } else
+ LOGPRINTF(" Created Protection Zone\n");
+
+ LOGPRINTF(" Create events\n");
+ ret = create_events();
+ if (ret != DAT_SUCCESS) {
+ fprintf(stderr, " Error creating events: %s\n",
+ DT_RetToString(ret));
+ goto cleanup;
+ } else {
+ LOGPRINTF(" Create events done\n");
+ }
+
+ /* create EP */
+ memset(&ep_attr, 0, sizeof(ep_attr));
+ if (ud_test) {
+ ep_attr.service_type = DAT_IB_SERVICE_TYPE_UD;
+ ep_attr.max_message_size = 2048;
+ } else {
+ ep_attr.service_type = DAT_SERVICE_TYPE_RC;
+ ep_attr.max_rdma_size = 0x10000;
+ ep_attr.max_rdma_read_in = 4;
+ ep_attr.max_rdma_read_out = 4;
+ }
+ ep_attr.max_recv_dtos = 1;
+ ep_attr.max_request_dtos = 1;
+ ep_attr.max_recv_iov = 1;
+ ep_attr.max_request_iov = 1;
+ ep_attr.request_completion_flags = DAT_COMPLETION_DEFAULT_FLAG;
+
+ start = get_time();
+ for (i = 0; i < connections; i++) {
+ ret = dat_ep_create(h_ia, h_pz, h_dto_rcv_evd,
+ h_dto_req_evd, h_conn_evd,
+ &ep_attr, &h_ep[i]);
+ }
+ stop = get_time();
+ ts.epc += ((stop - start) * 1.0e6);
+ ts.total += ts.epc;
+ if (ret != DAT_SUCCESS) {
+ fprintf(stderr, " Error dat_ep_create: %s\n",
+ DT_RetToString(ret));
+ goto cleanup;
+ } else
+ LOGPRINTF(" EP created %p \n", h_ep[i]);
+
+ /* create the service point for server listen */
+ if (server) {
+ LOGPRINTF(" Creating server service point(s)\n");
+ for (i = 0; i < burst; i++) {
+ ret = dat_psp_create(h_ia,
+ port_id+i,
+ h_cr_evd,
+ DAT_PSP_CONSUMER_FLAG,
+ &h_psp[i]);
+
+ if (ret != DAT_SUCCESS) {
+ fprintf(stderr, " ERR psp_create: %s\n",
+ DT_RetToString(ret));
+ goto cleanup;
+ } else
+ LOGPRINTF(" psp_created for listen\n");
+
+ printf(" Server ready on port %d\n",
+ port_id+i);
+
+ if (!multi_listens)
+ break;
+ }
+ }
+
+ /* Connect all */
+ if (server)
+ ret = conn_server();
+ else
+ ret = conn_client();
+
+ if (ret != DAT_SUCCESS) {
+ fprintf(stderr, " Error %s: %s\n",
+ server ? "server()" : "client()",
+ DT_RetToString(ret));
+ goto cleanup;
+ } else
+ LOGPRINTF(" connect_ep complete\n");
+
+ connected = 1;
+ goto complete;
+
+cleanup:
+ flush_evds();
+ goto bail;
+complete:
+
+ /* disconnect and free EP resources */
+ if (h_ep[0]) {
+ /* unregister message buffers and tear down connection */
+ LOGPRINTF(" Disconnect EPs\n");
+ ret = disconnect_eps();
+ if (ret != DAT_SUCCESS) {
+ fprintf(stderr, " Error disconnect_eps: %s\n",
+ DT_RetToString(ret));
+ goto bail;
+ } else {
+ LOGPRINTF(" disconnect_eps complete\n");
+ }
+ }
+
+ /* destroy server service point(s) */
+ if ((server) && (h_psp[0] != DAT_HANDLE_NULL)) {
+ for (i = 0; i < burst; i++) {
+ ret = dat_psp_free(h_psp[i]);
+ if (ret != DAT_SUCCESS) {
+ fprintf(stderr, " Error dat_psp_free: %s\n",
+ DT_RetToString(ret));
+ goto bail;
+ } else {
+ LOGPRINTF(" psp_free[%d] complete\n",i);
+ }
+ if (!multi_listens)
+ break;
+ }
+ }
+
+ /* free EVDs */
+ LOGPRINTF(" destroy events\n");
+ ret = destroy_events();
+ if (ret != DAT_SUCCESS) {
+ fprintf(stderr, " Error destroy_events: %s\n",
+ DT_RetToString(ret));
+ goto bail;
+ } else
+ LOGPRINTF(" destroy events done\n");
+
+
+ /* Free protection domain */
+ LOGPRINTF(" Freeing pz\n");
+ start = get_time();
+ ret = dat_pz_free(h_pz);
+ stop = get_time();
+ ts.pzf += ((stop - start) * 1.0e6);
+ if (ret != DAT_SUCCESS) {
+ fprintf(stderr, " Error freeing PZ: %s\n",
+ DT_RetToString(ret));
+ goto bail;
+ } else {
+ LOGPRINTF(" Freed pz\n");
+ h_pz = NULL;
+ }
+
+ /* close the device */
+ LOGPRINTF(" Closing Interface Adaptor\n");
+ start = get_time();
+ ret = dat_ia_close(h_ia, DAT_CLOSE_ABRUPT_FLAG);
+ stop = get_time();
+ ts.close += ((stop - start) * 1.0e6);
+ if (ret != DAT_SUCCESS) {
+ fprintf(stderr, " Error Adaptor close: %s\n",
+ DT_RetToString(ret));
+ goto bail;
+ } else
+ LOGPRINTF(" Closed Interface Adaptor\n");
+
+ printf(" DAPL Connection Test Complete.\n");
+ printf(" open: %10.2lf usec\n", ts.open);
+ printf(" close: %10.2lf usec\n", ts.close);
+ printf(" PZ create: %10.2lf usec\n", ts.pzc);
+ printf(" PZ free: %10.2lf usec\n", ts.pzf);
+ printf(" LMR create:%10.2lf usec\n", ts.reg);
+ printf(" LMR free: %10.2lf usec\n", ts.unreg);
+ printf(" EVD create:%10.2lf usec\n", ts.evdc);
+ printf(" EVD free: %10.2lf usec\n", ts.evdf);
+ printf(" EP create: %10.2lf usec avg\n", ts.epc/connections);
+ printf(" EP free: %10.2lf usec avg\n", ts.epf/connections);
+ if (!server) {
+ printf(" Connections: %8.2lf usec, CPS %7.2lf "
+ "Total %4.2lf secs, poll_cnt=%u, Num=%d\n",
+ (double)(ts.conn/connections),
+ (double)(1/(ts.conn/1000000/connections)),
+ (double)(ts.conn/1000000),
+ conn_poll_count, connections);
+ }
+ printf(" TOTAL: %4.2lf sec\n", ts.total/1000000);
+ fflush(stderr); fflush(stdout);
+bail:
+ free(h_ep);
+ free(h_psp);
+
+#if defined(_WIN32) || defined(_WIN64)
+ WSACleanup();
+#endif
+ return (0);
+}
+
+double get_time(void)
+{
+ struct timeval tp;
+
+ gettimeofday(&tp, NULL);
+ return ((double)tp.tv_sec + (double)tp.tv_usec * 1e-6);
+}
+
+DAT_RETURN conn_server()
+{
+ DAT_RETURN ret;
+ DAT_EVENT event;
+ DAT_COUNT nmore;
+ int i,bi;
+ unsigned char *buf;
+ DAT_CR_ARRIVAL_EVENT_DATA *cr_event =
+ &event.event_data.cr_arrival_event_data;
+ DAT_CR_PARAM cr_param = { 0 };
+
+ printf(" Accepting...\n");
+ for (i = 0; i < connections; i++) {
+
+ /* poll for CR's */
+ if (!waiting) {
+ cr_poll_count = 0;
+ while (DAT_GET_TYPE(dat_evd_dequeue(h_cr_evd, &event))
+ == DAT_QUEUE_EMPTY)
+ cr_poll_count++;
+ } else {
+ ret = dat_evd_wait(h_cr_evd, CR_TIMEOUT,
+ 1, &event, &nmore);
+ if (ret != DAT_SUCCESS) {
+ fprintf(stderr,
+ " ERR: CR dat_evd_wait() %s\n",
+ DT_RetToString(ret));
+ return ret;
+ }
+ }
+
+ if ((event.event_number != DAT_CONNECTION_REQUEST_EVENT) &&
+ (ud_test && event.event_number !=
+ DAT_IB_UD_CONNECTION_REQUEST_EVENT)) {
+ fprintf(stderr, " Error unexpected CR event : %s\n",
+ DT_EventToSTr(event.event_number));
+ return (DAT_ABORT);
+ }
+
+
+ /* use to test rdma_cma timeout logic */
+#if defined(_WIN32) || defined(_WIN64)
+ if (delay) {
+ printf(" Accept delayed by %d seconds...\n", delay);
+ Sleep(delay * 1000);
+ }
+#else
+ if (delay) {
+ printf(" Accept delayed by %d seconds...\n", delay);
+ sleep(delay);
+ }
+#endif
+ /* accept connect request from client */
+ h_cr = cr_event->cr_handle;
+ LOGPRINTF(" Accepting connect request from client\n");
+
+ /* private data - check and send it back */
+ dat_cr_query(h_cr, DAT_CSP_FIELD_ALL, &cr_param);
+
+ buf = (unsigned char *)cr_param.private_data;
+ LOGPRINTF(" CONN REQUEST Private Data %p[0]=%d [47]=%d\n",
+ buf, buf[0], buf[47]);
+
+ for (bi = 0; bi < 48; bi++) {
+ if (buf[bi] != bi + 1) {
+ fprintf(stderr, " ERR on CONNECT REQUEST"
+ " private data: %p[%d]=%d s/be %d\n",
+ buf, bi, buf[bi], bi + 1);
+ dat_cr_reject(h_cr, 0, NULL);
+ return (DAT_ABORT);
+ }
+ buf[bi]++; /* change for trip back */
+ }
+
+#ifdef TEST_REJECT_WITH_PRIVATE_DATA
+ printf(" REJECT request with 48 bytes of private data\n");
+ ret = dat_cr_reject(h_cr, 48, cr_param.private_data);
+ printf("\n DAPL Test Complete. %s\n\n",
+ ret ? "FAILED" : "PASSED");
+ exit(0);
+#endif
+ ret = dat_cr_accept(h_cr, h_ep[i], 48,
+ cr_param.private_data);
+
+ if (ret != DAT_SUCCESS) {
+ fprintf(stderr, " ERR dat_cr_accept: %s\n",
+ DT_RetToString(ret));
+ return (ret);
+ } else
+ LOGPRINTF(" accept[%d] complete\n", i);
+
+ event.event_number = 0;
+ }
+
+ /* process the RTU, ESTABLISHMENT event */
+ printf(" Completing...\n");
+ for (i=0;i<connections;i++) {
+
+ /* process completions */
+ if (!waiting) {
+ conn_poll_count = 0;
+ while (DAT_GET_TYPE(dat_evd_dequeue(h_conn_evd,
+ &event)) == DAT_QUEUE_EMPTY)
+ conn_poll_count++;
+ } else {
+ ret = dat_evd_wait(h_conn_evd, CONN_TIMEOUT,
+ 1, &event, &nmore);
+ if (ret != DAT_SUCCESS) {
+ fprintf(stderr,
+ " ERR: CONN evd_wait() %s\n",
+ DT_RetToString(ret));
+ return ret;
+ }
+ }
+ if ((event.event_number != DAT_CONNECTION_EVENT_ESTABLISHED) &&
+ (ud_test && event.event_number !=
+ DAT_IB_UD_CONNECTION_EVENT_ESTABLISHED)) {
+
+ fprintf(stderr, " Error unexpected CR EST "
+ "event : 0x%x %s\n",
+ event.event_number,
+ DT_EventToSTr(event.event_number));
+ return (DAT_ABORT);
+ }
+ event.event_number = 0;
+ LOGPRINTF(" CONN_EST[%d] complete\n", i);
+ }
+
+ printf("\n ALL %d CONNECTED on Server!\n\n", connections);
+ return DAT_SUCCESS;
+}
+
+
+DAT_RETURN conn_client()
+{
+ DAT_IA_ADDRESS_PTR raddr = (DAT_IA_ADDRESS_PTR)&remote;
+ DAT_RETURN ret;
+ DAT_EVENT event;
+ DAT_COUNT nmore;
+ DAT_CONN_QUAL conn_id;
+ DAT_CONNECTION_EVENT_DATA *conn_event =
+ &event.event_data.connect_event_data;
+ int i,ii,bi;
+ unsigned char *buf;
+ unsigned char pdata[48] = { 0 };
+ struct addrinfo *target;
+ int rval;
+
+ if (ucm)
+ goto no_resolution;
+
+#if defined(_WIN32) || defined(_WIN64)
+ if ((rval = getaddrinfo(hostname, "ftp", NULL, &target)) != 0) {
+ printf("\n remote name resolution failed! %s\n",
+ gai_strerror(rval));
+ exit(1);
+ }
+ rval = ((struct sockaddr_in *)target->ai_addr)->sin_addr.s_addr;
+#else
+ if (getaddrinfo(hostname, NULL, NULL, &target) != 0) {
+ perror("\n remote name resolution failed!");
+ exit(1);
+ }
+ rval = ((struct sockaddr_in *)target->ai_addr)->sin_addr.s_addr;
+#endif
+ printf(" Connecting to Server: %s \n", hostname);
+ printf(" Address: %d.%d.%d.%d port %d\n",
+ (rval >> 0) & 0xff, (rval >> 8) & 0xff,
+ (rval >> 16) & 0xff, (rval >> 24) & 0xff,
+ port_id);
+
+ raddr = (DAT_IA_ADDRESS_PTR)target->ai_addr;
+
+no_resolution:
+
+ for (i = 0; i < 48; i++) /* simple pattern in private data */
+ pdata[i] = i + 1;
+
+ printf(" Connecting...\n");
+ start = get_time();
+ for (i = 0; i < connections; i += burst) {
+ for (ii = 0; ii < burst; ii++) { /* conn_reqs */
+ if (multi_listens)
+ conn_id = port_id + ii;
+ else
+ conn_id = port_id;
+
+ ret = dat_ep_connect(h_ep[i+ii], raddr,
+ conn_id, CONN_TIMEOUT,
+ 48, (DAT_PVOID) pdata, 0,
+ DAT_CONNECT_DEFAULT_FLAG);
+ if (ret != DAT_SUCCESS) {
+ fprintf(stderr, " ERR dat_ep_connect: %s\n",
+ DT_RetToString(ret));
+ return (ret);
+ } else
+ LOGPRINTF(" dat_ep_connect[%d] complete\n",
+ i+ii);
+
+ }
+ for (ii = 0; ii < burst; ii++) { /* conn_events */
+ if (!waiting) {
+ conn_poll_count = 0;
+ while (DAT_GET_TYPE(dat_evd_dequeue(
+ h_conn_evd, &event)) ==
+ DAT_QUEUE_EMPTY)
+ conn_poll_count++;
+ } else {
+ ret = dat_evd_wait(h_conn_evd, CONN_TIMEOUT,
+ 1, &event, &nmore);
+
+ if (ret != DAT_SUCCESS) {
+ fprintf(stderr,
+ " ERR: CONN evd_wait() %s\n",
+ DT_RetToString(ret));
+ return ret;
+ }
+ }
+
+#ifdef TEST_REJECT_WITH_PRIVATE_DATA
+ if (event.event_number !=
+ DAT_CONNECTION_EVENT_PEER_REJECTED) {
+ fprintf(stderr, " expected conn reject "
+ "event : %s\n",
+ DT_EventToSTr(event.event_number));
+ return (DAT_ABORT);
+ }
+
+ /* get the reject private data and validate */
+ buf = (unsigned char *)conn_event->private_data;
+ printf(" Recv REJ with pdata %p[0]=%d [47]=%d\n",
+ buf, buf[0], buf[47]);
+ for (bi = 0; bi < 48; bi++) {
+ if (buf[bi] != idx + 2) {
+ fprintf(stderr, " client: Error"
+ " with REJECT event private"
+ " data: %p[%d]=%d s/be %d\n",
+ buf, bi,
+ buf[bi], bi + 2);
+ dat_ep_disconnect(h_ep[i+ii], 0);
+ return (DAT_ABORT);
+ }
+ }
+ printf("\n Rej Test Done. PASSED\n\n");
+ exit(0);
+#endif
+ if ((event.event_number !=
+ DAT_CONNECTION_EVENT_ESTABLISHED) &&
+ (ud_test && event.event_number !=
+ DAT_IB_UD_CONNECTION_EVENT_ESTABLISHED)) {
+ fprintf(stderr, " Error unexpected conn "
+ "event : 0x%x %s\n",
+ event.event_number,
+ DT_EventToSTr(event.event_number));
+ return (DAT_ABORT);
+ }
+
+ /* check private data back from server */
+ buf = (unsigned char *)conn_event->private_data;
+
+ LOGPRINTF(" CONN[%d] Private Data "
+ "%p[0]=%d [47]=%d\n",
+ i+ii, buf, buf[0], buf[47]);
+
+ for (bi = 0; bi < 48; bi++) {
+ if (buf[bi] != bi + 2) {
+ DAT_COUNT nmore;
+ fprintf(stderr, " ERR CONN event"
+ " pdata: %p[%d]=%d s/be %d\n",
+ buf, bi, buf[bi],
+ bi + 2);
+ dat_ep_disconnect(h_ep[i+ii],
+ DAT_CLOSE_ABRUPT_FLAG);
+ LOGPRINTF(" waiting for disc...\n");
+ dat_evd_wait(h_conn_evd,
+ DAT_TIMEOUT_INFINITE,
+ 1, &event, &nmore);
+ return (DAT_ABORT);
+ }
+ }
+ event.event_number = 0;
+ }
+ }
+
+ stop = get_time();
+ ts.conn += ((stop - start) * 1.0e6);
+
+ if (!ucm)
+ freeaddrinfo(target);
+
+ printf("\n ALL %d CONNECTED on Client!\n\n", connections);
+
+ return (DAT_SUCCESS);
+}
+
+/* validate disconnected EP's and free them */
+DAT_RETURN disconnect_eps(void)
+{
+ DAT_RETURN ret;
+ DAT_EVENT event;
+ DAT_COUNT nmore;
+ int i,ii;
+ DAT_CONNECTION_EVENT_DATA *conn_event =
+ &event.event_data.connect_event_data;
+
+ if (!connected)
+ return DAT_SUCCESS;
+
+ /* UD, no connection to disconnect, just free EP's */
+ if (ud_test) {
+ for (i = 0; i < connections; i++) {
+ ret = dat_ep_free(h_ep[i]);
+ if (ret != DAT_SUCCESS) {
+ fprintf(stderr,
+ " ERR free EP[%d] %p: %s\n",
+ i, h_ep[i], DT_RetToString(ret));
+ } else {
+ LOGPRINTF(" Freed EP[%d] %p\n",
+ i, h_ep[i]);
+ h_ep[i] = DAT_HANDLE_NULL;
+ }
+ }
+ stop = get_time();
+ ts.epf += ((stop - start) * 1.0e6);
+ ts.total += ts.epf;
+ return DAT_SUCCESS;
+ }
+
+ /*
+ * Only the client needs to call disconnect. The server _should_ be able
+ * to just wait on the EVD associated with connection events for a
+ * disconnect request and then exit.
+ */
+ if (!server) {
+ start = get_time();
+ for (i = 0; i < connections; i++) {
+ LOGPRINTF(" dat_ep_disconnect\n");
+ ret = dat_ep_disconnect(h_ep[i],
+ DAT_CLOSE_DEFAULT);
+ if (ret != DAT_SUCCESS) {
+ fprintf(stderr,
+ " Error disconnect: %s\n",
+ DT_RetToString(ret));
+ return ret;
+ } else {
+ LOGPRINTF(" disconnect completed\n");
+ }
+ }
+ } else {
+ LOGPRINTF(" Server waiting for disconnect...\n");
+ }
+
+ LOGPRINTF(" Wait for Disc event, free EPs as completed\n");
+ start = get_time();
+ for (i = 0; i < connections; i++) {
+ event.event_number = 0;
+ conn_event->ep_handle = NULL;
+ ret = dat_evd_wait(h_conn_evd, DAT_TIMEOUT_INFINITE,
+ 1, &event, &nmore);
+ if (ret != DAT_SUCCESS) {
+ fprintf(stderr, " Error dat_evd_wait: %s\n",
+ DT_RetToString(ret));
+ return ret;
+ } else {
+ LOGPRINTF(" disc event[%d] complete,"
+ " check for valid EP...\n", i);
+ }
+
+ /* check for valid EP in creation list */
+ for (ii = 0; ii < connections; ii++) {
+ if (h_ep[ii] == conn_event->ep_handle) {
+ LOGPRINTF(" valid EP[%d] %p !\n",
+ ii, h_ep[ii]);
+ ret = dat_ep_free(h_ep[ii]);
+ if (ret != DAT_SUCCESS) {
+ fprintf(stderr,
+ " ERR free EP[%d] %p: %s\n",
+ i, h_ep[ii],
+ DT_RetToString(ret));
+ } else {
+ LOGPRINTF(" Freed EP[%d] %p\n",
+ i, h_ep[ii]);
+ h_ep[ii] = DAT_HANDLE_NULL;
+ }
+ break;
+ } else {
+ continue;
+ }
+ }
+ if (ii == connections) {
+ LOGPRINTF(" %s: invalid EP[%d] %p via DISC event!\n",
+ server ? "Server" : "Client",
+ i, conn_event->ep_handle);
+ return DAT_INVALID_HANDLE;
+ }
+ }
+ /* free EPs */
+ stop = get_time();
+ ts.epf += ((stop - start) * 1.0e6);
+ ts.total += ts.epf;
+ return DAT_SUCCESS;
+}
+
+
+ /*
+ * Create CR, CONN, and DTO events
+ */
+DAT_RETURN create_events(void)
+{
+ DAT_RETURN ret;
+
+ /* create cr EVD */
+ start = get_time();
+ ret = dat_evd_create(h_ia, connections, DAT_HANDLE_NULL,
+ DAT_EVD_CR_FLAG, &h_cr_evd);
+ stop = get_time();
+ ts.evdc += ((stop - start) * 1.0e6);
+ ts.total += ts.evdc;
+ if (ret != DAT_SUCCESS) {
+ fprintf(stderr, " Error dat_evd_create: %s\n",
+ DT_RetToString(ret));
+ return (ret);
+ } else {
+ LOGPRINTF(" cr_evd created %p\n", h_cr_evd);
+ }
+
+ /* create conn EVD */
+ ret = dat_evd_create(h_ia,
+ connections*2,
+ DAT_HANDLE_NULL,
+ DAT_EVD_CONNECTION_FLAG, &h_conn_evd);
+ if (ret != DAT_SUCCESS) {
+ fprintf(stderr, " Error dat_evd_create: %s\n",
+ DT_RetToString(ret));
+ return (ret);
+ } else {
+ LOGPRINTF(" con_evd created %p\n", h_conn_evd);
+ }
+
+ /* create dto SND EVD */
+ ret = dat_evd_create(h_ia, 1, NULL,
+ DAT_EVD_DTO_FLAG, &h_dto_req_evd);
+ if (ret != DAT_SUCCESS) {
+ fprintf(stderr, " Error dat_evd_create REQ: %s\n",
+ DT_RetToString(ret));
+ return (ret);
+ } else {
+ LOGPRINTF(" dto_req_evd created %p\n",
+ h_dto_req_evd);
+ }
+
+ /* create dto RCV EVD */
+ ret = dat_evd_create(h_ia, 1, NULL,
+ DAT_EVD_DTO_FLAG, &h_dto_rcv_evd);
+ if (ret != DAT_SUCCESS) {
+ fprintf(stderr, " Error dat_evd_create RCV: %s\n",
+ DT_RetToString(ret));
+ return (ret);
+ } else {
+ LOGPRINTF(" dto_rcv_evd created %p\n",
+ h_dto_rcv_evd);
+ }
+ return DAT_SUCCESS;
+}
+
+/*
+ * Destroy CR, CONN, CNO, and DTO events
+ */
+
+DAT_RETURN destroy_events(void)
+{
+ DAT_RETURN ret;
+
+ /* free cr EVD */
+ if (h_cr_evd != DAT_HANDLE_NULL) {
+ LOGPRINTF(" Free cr EVD %p \n", h_cr_evd);
+ ret = dat_evd_free(h_cr_evd);
+ if (ret != DAT_SUCCESS) {
+ fprintf(stderr, " Error freeing cr EVD: %s\n",
+ DT_RetToString(ret));
+ return (ret);
+ } else {
+ LOGPRINTF(" Freed cr EVD\n");
+ h_cr_evd = DAT_HANDLE_NULL;
+ }
+ }
+
+ /* free conn EVD */
+ if (h_conn_evd != DAT_HANDLE_NULL) {
+ LOGPRINTF(" Free conn EVD %p\n", h_conn_evd);
+ ret = dat_evd_free(h_conn_evd);
+ if (ret != DAT_SUCCESS) {
+ fprintf(stderr, " Error freeing conn EVD: %s\n",
+ DT_RetToString(ret));
+ return (ret);
+ } else {
+ LOGPRINTF(" Freed conn EVD\n");
+ h_conn_evd = DAT_HANDLE_NULL;
+ }
+ }
+
+ /* free RCV dto EVD */
+ if (h_dto_rcv_evd != DAT_HANDLE_NULL) {
+ LOGPRINTF(" Free RCV dto EVD %p\n", h_dto_rcv_evd);
+ start = get_time();
+ ret = dat_evd_free(h_dto_rcv_evd);
+ stop = get_time();
+ ts.evdf += ((stop - start) * 1.0e6);
+ ts.total += ts.evdf;
+ if (ret != DAT_SUCCESS) {
+ fprintf(stderr, " Error freeing dto EVD: %s\n",
+ DT_RetToString(ret));
+ return (ret);
+ } else {
+ LOGPRINTF(" Freed dto EVD\n");
+ h_dto_rcv_evd = DAT_HANDLE_NULL;
+ }
+ }
+
+ /* free REQ dto EVD */
+ if (h_dto_req_evd != DAT_HANDLE_NULL) {
+ LOGPRINTF(" Free REQ dto EVD %p\n", h_dto_req_evd);
+ ret = dat_evd_free(h_dto_req_evd);
+ if (ret != DAT_SUCCESS) {
+ fprintf(stderr, " Error freeing dto EVD: %s\n",
+ DT_RetToString(ret));
+ return (ret);
+ } else {
+ LOGPRINTF(" Freed dto EVD\n");
+ h_dto_req_evd = DAT_HANDLE_NULL;
+ }
+ }
+
+ return DAT_SUCCESS;
+}
+
+/*
+ * Map DAT_RETURN values to readable strings,
+ * but don't assume the values are zero-based or contiguous.
+ */
+char errmsg[512] = { 0 };
+const char *DT_RetToString(DAT_RETURN ret_value)
+{
+ const char *major_msg, *minor_msg;
+
+ dat_strerror(ret_value, &major_msg, &minor_msg);
+
+ strcpy(errmsg, major_msg);
+ strcat(errmsg, " ");
+ strcat(errmsg, minor_msg);
+
+ return errmsg;
+}
+
+/*
+ * Map DAT_EVENT_CODE values to readable strings
+ */
+const char *DT_EventToSTr(DAT_EVENT_NUMBER event_code)
+{
+ unsigned int i;
+ static struct {
+ const char *name;
+ DAT_RETURN value;
+ } dat_events[] = {
+# define DATxx(x) { # x, x }
+ DATxx(DAT_DTO_COMPLETION_EVENT),
+ DATxx(DAT_RMR_BIND_COMPLETION_EVENT),
+ DATxx(DAT_CONNECTION_REQUEST_EVENT),
+ DATxx(DAT_CONNECTION_EVENT_ESTABLISHED),
+ DATxx(DAT_CONNECTION_EVENT_PEER_REJECTED),
+ DATxx(DAT_CONNECTION_EVENT_NON_PEER_REJECTED),
+ DATxx(DAT_CONNECTION_EVENT_ACCEPT_COMPLETION_ERROR),
+ DATxx(DAT_CONNECTION_EVENT_DISCONNECTED),
+ DATxx(DAT_CONNECTION_EVENT_BROKEN),
+ DATxx(DAT_CONNECTION_EVENT_TIMED_OUT),
+ DATxx(DAT_CONNECTION_EVENT_UNREACHABLE),
+ DATxx(DAT_ASYNC_ERROR_EVD_OVERFLOW),
+ DATxx(DAT_ASYNC_ERROR_IA_CATASTROPHIC),
+ DATxx(DAT_ASYNC_ERROR_EP_BROKEN),
+ DATxx(DAT_ASYNC_ERROR_TIMED_OUT),
+ DATxx(DAT_ASYNC_ERROR_PROVIDER_INTERNAL_ERROR),
+ DATxx(DAT_SOFTWARE_EVENT)
+# undef DATxx
+ };
+# define NUM_EVENTS (sizeof(dat_events)/sizeof(dat_events[0]))
+
+ for (i = 0; i < NUM_EVENTS; i++) {
+ if (dat_events[i].value == event_code) {
+ return (dat_events[i].name);
+ }
+ }
+
+ return ("Invalid_DAT_EVENT_NUMBER");
+}
+
+void print_usage(void)
+{
+ printf("\n DAPL USAGE \n\n");
+ printf("s: server\n");
+ printf("c: connections (default = 1000)\n");
+ printf("v: verbose\n");
+ printf("w: wait on event (default, polling)\n");
+ printf("d: delay before accept\n");
+ printf("h: hostname/address of server, specified on client\n");
+ printf("P: provider name (default = OpenIB-v2-ib0)\n");
+ printf("\n");
+}
+
-/*\r
- * Copyright (c) 2007-2008 Intel Corporation. All rights reserved.\r
- *\r
- * This software is available to you under a choice of one of two\r
- * licenses. You may choose to be licensed under the terms of the GNU\r
- * General Public License (GPL) Version 2, available from the file\r
- * COPYING in the main directory of this source tree, or the\r
- * OpenIB.org BSD license below:\r
- *\r
- * Redistribution and use in source and binary forms, with or\r
- * without modification, are permitted provided that the following\r
- * conditions are met:\r
- *\r
- * - Redistributions of source code must retain the above\r
- * copyright notice, this list of conditions and the following\r
- * disclaimer.\r
- *\r
- * - Redistributions in binary form must reproduce the above\r
- * copyright notice, this list of conditions and the following\r
- * disclaimer in the documentation and/or other materials\r
- * provided with the distribution.\r
- *\r
- * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,\r
- * EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF\r
- * MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND\r
- * NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS\r
- * BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN\r
- * ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN\r
- * CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE\r
- * SOFTWARE.\r
- *\r
- * $Id: $\r
- */\r
-#include <stdio.h>\r
-#include <stdlib.h>\r
-\r
-#if defined(_WIN32) || defined(_WIN64)\r
-#include <windows.h>\r
-#include <winsock2.h>\r
-#include <ws2tcpip.h>\r
-#include <io.h>\r
-#include <process.h>\r
-#include <complib/cl_types.h>\r
-#include "..\..\..\..\etc\user\getopt.c"\r
-#define __BYTE_ORDER __LITTLE_ENDIAN\r
-\r
-#define getpid() ((int)GetCurrentProcessId())\r
-#define F64x "%I64x"\r
-#define F64u "%I64u"\r
-#define DAPL_PROVIDER "ibnic0v2"\r
-#else\r
-#include <netdb.h>\r
-#include <sys/types.h>\r
-#include <sys/socket.h>\r
-#include <sys/time.h>\r
-#include <netinet/in.h>\r
-#include <netinet/tcp.h>\r
-#include <arpa/inet.h>\r
-#include <inttypes.h>\r
-#include <string.h>\r
-#include <unistd.h>\r
-\r
-#define DAPL_PROVIDER "ofa-v2-ib0"\r
-#define F64x "%"PRIx64""\r
-#define F64u "%"PRIu64""\r
-\r
-#endif\r
-\r
-#include "dat2/udat.h"\r
-#include "dat2/dat_ib_extensions.h"\r
-\r
-int disconnect_ep(void);\r
-\r
-#define _OK(status, str) \\r
-{\\r
- const char *maj_msg, *min_msg;\\r
- if (status != DAT_SUCCESS) {\\r
- dat_strerror(status, &maj_msg, &min_msg);\\r
- fprintf(stderr, str " returned %s : %s\n", maj_msg, min_msg);\\r
- dat_ia_close(ia, DAT_CLOSE_DEFAULT);\\r
- exit(1);\\r
- } else if (verbose) {\\r
- printf("dtestx: %s success\n",str);\\r
- }\\r
-}\r
-\r
-#define _OK2(status, str)\\r
-{\\r
- const char *maj_msg, *min_msg;\\r
- if (status != DAT_SUCCESS) {\\r
- dat_strerror(status, &maj_msg, &min_msg);\\r
- fprintf(stderr, str " returned %s : %s\n", maj_msg, min_msg);\\r
- dat_ia_close(ia, DAT_CLOSE_DEFAULT);\\r
- exit(1);\\r
- } else if (verbose) {\\r
- printf("dtestx: %s\n",str);\\r
- }\\r
-}\r
-\r
-/* byte swap helpers from Complib */\r
-#if __BYTE_ORDER == __LITTLE_ENDIAN\r
-#define ntoh16(x) (uint16_t)( \\r
- (((uint16_t)(x) & 0x00FF) << 8) | \\r
- (((uint16_t)(x) & 0xFF00) >> 8))\r
-#define hton16(x) ntoh16(x)\r
-#define ntoh32(x) (uint32_t)( \\r
- (((uint32_t)(x) & 0x000000FF) << 24)| \\r
- (((uint32_t)(x) & 0x0000FF00) << 8) | \\r
- (((uint32_t)(x) & 0x00FF0000) >> 8) | \\r
- (((uint32_t)(x) & 0xFF000000) >> 24))\r
-#define hton32(x) ntoh32(x)\r
-#define ntoh64(x) (uint64_t)( \\r
- (((uint64_t)x & 0x00000000000000FFULL) << 56) | \\r
- (((uint64_t)x & 0x000000000000FF00ULL) << 40) | \\r
- (((uint64_t)x & 0x0000000000FF0000ULL) << 24) | \\r
- (((uint64_t)x & 0x00000000FF000000ULL) << 8 ) | \\r
- (((uint64_t)x & 0x000000FF00000000ULL) >> 8 ) | \\r
- (((uint64_t)x & 0x0000FF0000000000ULL) >> 24) | \\r
- (((uint64_t)x & 0x00FF000000000000ULL) >> 40) | \\r
- (((uint64_t)x & 0xFF00000000000000ULL) >> 56))\r
-#define hton64(x) ntoh64(x)\r
-#elif __BYTE_ORDER == __BIG_ENDIAN\r
-#define hton16(x) (x)\r
-#define ntoh16(x) (x)\r
-#define hton32(x) (x)\r
-#define ntoh32(x) (x)\r
-#define hton64(x) (x)\r
-#define ntoh64(x) (x)\r
-#endif /* __BYTE_ORDER == __BIG_ENDIAN */\r
-\r
-#define MIN(a, b) ((a < b) ? (a) : (b))\r
-#define MAX(a, b) ((a > b) ? (a) : (b))\r
-\r
-#define DTO_TIMEOUT (1000*1000*5)\r
-#define CONN_TIMEOUT (1000*1000*30)\r
-#define SERVER_TIMEOUT (DAT_TIMEOUT_INFINITE)\r
-#define CLIENT_ID 31111\r
-#define SERVER_ID 31112\r
-#define BUF_SIZE 256\r
-#define BUF_SIZE_ATOMIC 8\r
-#define REG_MEM_COUNT 10\r
-#define SND_RDMA_BUF_INDEX 0\r
-#define RCV_RDMA_BUF_INDEX 1\r
-#define SEND_BUF_INDEX 2\r
-#define RECV_BUF_INDEX 3\r
-#define MAX_EP_COUNT 8\r
-\r
-DAT_VADDR *atomic_buf;\r
-DAT_LMR_HANDLE lmr_atomic;\r
-DAT_LMR_CONTEXT lmr_atomic_context;\r
-DAT_RMR_CONTEXT rmr_atomic_context;\r
-DAT_VLEN reg_atomic_size;\r
-DAT_VADDR reg_atomic_addr;\r
-DAT_LMR_HANDLE lmr[REG_MEM_COUNT * MAX_EP_COUNT];\r
-DAT_LMR_CONTEXT lmr_context[REG_MEM_COUNT * MAX_EP_COUNT];\r
-DAT_RMR_TRIPLET rmr[REG_MEM_COUNT * MAX_EP_COUNT];\r
-DAT_RMR_CONTEXT rmr_context[REG_MEM_COUNT * MAX_EP_COUNT];\r
-DAT_VLEN reg_size[REG_MEM_COUNT * MAX_EP_COUNT];\r
-DAT_VADDR reg_addr[REG_MEM_COUNT * MAX_EP_COUNT];\r
-DAT_RMR_TRIPLET *buf[REG_MEM_COUNT * MAX_EP_COUNT];\r
-DAT_EP_HANDLE ep[MAX_EP_COUNT];\r
-DAT_EVD_HANDLE async_evd = DAT_HANDLE_NULL;\r
-DAT_IA_HANDLE ia = DAT_HANDLE_NULL;\r
-DAT_PZ_HANDLE pz = DAT_HANDLE_NULL;\r
-DAT_EVD_HANDLE cr_evd = DAT_HANDLE_NULL;\r
-DAT_EVD_HANDLE con_evd = DAT_HANDLE_NULL;\r
-DAT_EVD_HANDLE dto_evd = DAT_HANDLE_NULL;\r
-DAT_PSP_HANDLE psp = DAT_HANDLE_NULL;\r
-int server = 1;\r
-int remote_host = 0;\r
-int ud_test = 0;\r
-int multi_eps = 0;\r
-int buf_size = BUF_SIZE;\r
-int msg_size = sizeof(DAT_RMR_TRIPLET);\r
-char provider[64] = DAPL_PROVIDER;\r
-char hostname[256] = { 0 };\r
-DAT_IB_ADDR_HANDLE remote_ah[MAX_EP_COUNT];\r
-int eps = 1;\r
-int verbose = 0;\r
-int counters = 0;\r
-int counters_ok = 0;\r
-static int ucm = 0;\r
-static DAT_SOCK_ADDR6 remote;\r
-\r
-#define LOGPRINTF if (verbose) printf\r
-\r
-void print_usage(void)\r
-{\r
- printf("\n dtestx usage \n\n");\r
- printf("v: verbose\n");\r
- printf("p: print counters\n");\r
- printf("u unreliable datagram test\n");\r
- printf("U: unreliable datagram test, UD endpoint count\n");\r
- printf("m unreliable datagram test, multiple Server endpoints\n");\r
- printf("b: buf length to allocate\n");\r
- printf("h: hostname/address of Server, client and UDP server\n");\r
- printf("c: Client\n");\r
- printf("s: Server, default\n");\r
- printf("P: provider name (default = ofa-v2-ib0)\n");\r
- printf("\n");\r
-}\r
-\r
-#if defined(_WIN32) || defined(_WIN64)\r
-static void sleep(int secs)\r
-{\r
- Sleep(secs * 1000);\r
-}\r
-\r
-#define _WSACleanup() WSACleanup()\r
-#else\r
-#define _WSACleanup()\r
-#endif\r
-\r
-static void print_ia_address(struct sockaddr *sa)\r
-{\r
- char str[INET6_ADDRSTRLEN] = {" ??? "};\r
-\r
- switch(sa->sa_family) {\r
- case AF_INET:\r
- inet_ntop(AF_INET, &((struct sockaddr_in *)sa)->sin_addr, str, INET6_ADDRSTRLEN);\r
- printf("%d Local Address AF_INET - %s port %d\n", getpid(), str, SERVER_ID);\r
- break;\r
- case AF_INET6:\r
- inet_ntop(AF_INET6, &((struct sockaddr_in6 *)sa)->sin6_addr, str, INET6_ADDRSTRLEN);\r
- printf("%d Local Address AF_INET6 - %s flowinfo(QPN)=0x%x, port(LID)=0x%x\n",\r
- getpid(), str,\r
- ntohl(((struct sockaddr_in6 *)sa)->sin6_flowinfo),\r
- ntohs(((struct sockaddr_in6 *)sa)->sin6_port));\r
- break;\r
- default:\r
- printf("%d Local Address UNKOWN FAMILY - port %d\n", getpid(), SERVER_ID);\r
- }\r
-}\r
-\r
-void\r
-send_msg(void *data,\r
- DAT_COUNT size,\r
- DAT_LMR_CONTEXT context,\r
- DAT_DTO_COOKIE cookie, DAT_COMPLETION_FLAGS flags)\r
-{\r
- DAT_LMR_TRIPLET iov;\r
- DAT_EVENT event;\r
- DAT_COUNT nmore;\r
- DAT_RETURN status;\r
- int i, ep_idx = 0, ah_idx = 0;\r
- DAT_DTO_COMPLETION_EVENT_DATA *dto_event =\r
- &event.event_data.dto_completion_event_data;\r
-\r
- iov.lmr_context = context;\r
- iov.virtual_address = (DAT_VADDR) (uintptr_t) data;\r
- iov.segment_length = (DAT_VLEN) size;\r
-\r
- for (i = 0; i < eps; i++) {\r
- if (ud_test) {\r
- /* \r
- * Client and Server: ep[0] and ah[0] on single \r
- * and ep[i] on multiple (-m) endpoint options. \r
- */\r
- if (multi_eps) {\r
- ep_idx = i;\r
- ah_idx = server ? 0 : i;\r
- }\r
- printf("%s sending on ep=%p to remote_ah: ah=%p"\r
- " qpn=0x%x addr=%s\n",\r
- server ? "Server" : "Client", ep[ep_idx],\r
- remote_ah[ah_idx].ah,\r
- remote_ah[ah_idx].qpn,\r
- inet_ntoa(((struct sockaddr_in *)\r
- &remote_ah[ah_idx].ia_addr)->\r
- sin_addr));\r
-\r
- /* client expects all data in on first EP */\r
- status = dat_ib_post_send_ud(ep[ep_idx],\r
- 1,\r
- &iov,\r
- &remote_ah[ah_idx],\r
- cookie, flags);\r
-\r
- } else {\r
- status = dat_ep_post_send(ep[0], 1, &iov,\r
- cookie, flags);\r
- }\r
- _OK(status, "dat_ep_post_send");\r
-\r
- if (!(flags & DAT_COMPLETION_SUPPRESS_FLAG)) {\r
- status = dat_evd_wait(dto_evd, DTO_TIMEOUT,\r
- 1, &event, &nmore);\r
- _OK(status, "dat_evd_wait after dat_ep_post_send");\r
-\r
- if (event.event_number != DAT_DTO_COMPLETION_EVENT &&\r
- ud_test && event.event_number != DAT_IB_DTO_EVENT) {\r
- printf("unexpected event waiting post_send "\r
- "completion - 0x%x\n",\r
- event.event_number);\r
- exit(1);\r
- }\r
- _OK(dto_event->status, "event status for post_send");\r
- }\r
- }\r
-}\r
-\r
-/* RC - Server only, UD - Server and Client, one per EP */\r
-void process_cr(int idx)\r
-{\r
- DAT_EVENT event;\r
- DAT_COUNT nmore;\r
- DAT_RETURN status;\r
- int pdata;\r
- DAT_CR_HANDLE cr = DAT_HANDLE_NULL;\r
- DAT_CONN_QUAL exp_qual = server ? SERVER_ID : CLIENT_ID;\r
- DAT_CR_PARAM cr_param;\r
- DAT_CR_ARRIVAL_EVENT_DATA *cr_event =\r
- &event.event_data.cr_arrival_event_data;\r
-\r
- LOGPRINTF("%s waiting for connect[%d] request\n",\r
- server ? "Server" : "Client", idx);\r
-\r
- status = dat_evd_wait(cr_evd, SERVER_TIMEOUT, 1, &event, &nmore);\r
- _OK(status, "CR dat_evd_wait");\r
-\r
- if (event.event_number != DAT_CONNECTION_REQUEST_EVENT &&\r
- (ud_test && event.event_number !=\r
- DAT_IB_UD_CONNECTION_REQUEST_EVENT)) {\r
- printf("unexpected event,!conn req: 0x%x\n",\r
- event.event_number);\r
- exit(1);\r
- }\r
-\r
- if ((cr_event->conn_qual != exp_qual) ||\r
- (cr_event->sp_handle.psp_handle != psp)) {\r
- printf("wrong cr event data\n");\r
- exit(1);\r
- }\r
-\r
- cr = cr_event->cr_handle;\r
- status = dat_cr_query(cr, DAT_CSP_FIELD_ALL, &cr_param);\r
- _OK(status, "dat_cr_query");\r
-\r
- /* use private data to select EP */\r
- pdata = ntoh32(*((int *)cr_param.private_data));\r
-\r
- LOGPRINTF("%s recvd pdata=0x%x, send pdata=0x%x\n",\r
- server ? "Server" : "Client", pdata,\r
- *(int *)cr_param.private_data);\r
-\r
- status = dat_cr_accept(cr, ep[pdata], 4, cr_param.private_data);\r
- _OK(status, "dat_cr_accept");\r
-\r
- printf("%s accepted CR on EP[%d]=%p\n",\r
- server ? "Server" : "Client", pdata, ep[pdata]);\r
-}\r
-\r
-/* RC - Client and Server: 1, UD - Client: 1 per EP, Server: 2 per EP's */\r
-void process_conn(int idx)\r
-{\r
- DAT_EVENT event;\r
- DAT_COUNT nmore;\r
- DAT_RETURN status;\r
- int pdata, exp_event;\r
- DAT_IB_EXTENSION_EVENT_DATA *ext_event = (DAT_IB_EXTENSION_EVENT_DATA *)\r
- & event.event_extension_data[0];\r
- DAT_CONNECTION_EVENT_DATA *conn_event =\r
- &event.event_data.connect_event_data;\r
-\r
- LOGPRINTF("%s waiting for connect[%d] establishment\n",\r
- server ? "Server" : "Client", idx);\r
-\r
- status = dat_evd_wait(con_evd, CONN_TIMEOUT, 1, &event, &nmore);\r
- _OK(status, "CONN dat_evd_wait");\r
-\r
- LOGPRINTF("%s got connect[%d] event 0x%x, pdata %p sz=%d\n",\r
- server ? "Server" : "Client", idx,\r
- event.event_number, conn_event->private_data, \r
- conn_event->private_data_size);\r
-\r
- if (ud_test) \r
- exp_event = DAT_IB_UD_CONNECTION_EVENT_ESTABLISHED;\r
- else\r
- exp_event = DAT_CONNECTION_EVENT_ESTABLISHED;\r
-\r
- /* Waiting on CR's or CONN_EST */\r
- if (event.event_number != exp_event || \r
- (ud_test && event.event_number !=\r
- DAT_IB_UD_CONNECTION_EVENT_ESTABLISHED)) {\r
- printf("unexpected event, !conn established: 0x%x\n",\r
- event.event_number);\r
- exit(1);\r
- }\r
-\r
- /* RC or PASSIVE CONN_EST we are done */\r
- if (!ud_test)\r
- return;\r
-\r
- /* store each remote_ah according to remote EP index */\r
- pdata = ntoh32(*((int *)conn_event->private_data));\r
- LOGPRINTF(" Client got private data=0x%x\n", pdata);\r
-\r
- /* UD, get AH for sends. \r
- * NOTE: bi-directional AH resolution results in a CONN_EST\r
- * for both outbound connect and inbound CR.\r
- * Use Active CONN_EST which includes server's CR\r
- * pdata for remote_ah idx to send on and ignore PASSIVE CONN_EST.\r
- *\r
- * DAT_IB_UD_PASSIVE_REMOTE_AH == passive side CONN_EST\r
- * DAT_IB_UD_REMOTE_AH == active side CONN_EST\r
- */\r
- if (ext_event->type == DAT_IB_UD_REMOTE_AH) {\r
- remote_ah[pdata] = ext_event->remote_ah;\r
- printf("remote_ah[%d]: ah=%p, qpn=0x%x "\r
- "addr=%s\n",\r
- pdata, remote_ah[pdata].ah,\r
- remote_ah[pdata].qpn, inet_ntoa(((struct sockaddr_in *)\r
- &remote_ah[pdata].\r
- ia_addr)->sin_addr));\r
-\r
- } else if (ext_event->type != DAT_IB_UD_PASSIVE_REMOTE_AH) {\r
- printf("unexpected UD ext_event type: 0x%x\n", ext_event->type);\r
- exit(1);\r
- }\r
-}\r
-\r
-int connect_ep(char *hostname)\r
-{\r
- DAT_IA_ADDRESS_PTR remote_addr = (DAT_IA_ADDRESS_PTR)&remote;\r
- DAT_EP_ATTR ep_attr;\r
- DAT_IA_ATTR ia_attr;\r
- DAT_RETURN status;\r
- DAT_REGION_DESCRIPTION region;\r
- DAT_EVENT event;\r
- DAT_COUNT nmore;\r
- DAT_LMR_TRIPLET iov;\r
- DAT_RMR_TRIPLET *r_iov;\r
- DAT_DTO_COOKIE cookie;\r
- DAT_CONN_QUAL conn_qual;\r
- DAT_BOOLEAN in, out;\r
- int i, ii, pdata, ctx;\r
- DAT_PROVIDER_ATTR prov_attrs;\r
- DAT_DTO_COMPLETION_EVENT_DATA *dto_event =\r
- &event.event_data.dto_completion_event_data;\r
-\r
- status = dat_ia_open(provider, 8, &async_evd, &ia);\r
- _OK(status, "dat_ia_open");\r
-\r
- memset(&prov_attrs, 0, sizeof(prov_attrs));\r
- status = dat_ia_query(ia, NULL, \r
- DAT_IA_FIELD_ALL, &ia_attr,\r
- DAT_PROVIDER_FIELD_ALL, &prov_attrs);\r
- _OK(status, "dat_ia_query");\r
-\r
- print_ia_address(ia_attr.ia_address_ptr);\r
-\r
- if (ucm && ud_test) {\r
- printf("%d UD test over UCM provider not supported\n",\r
- getpid());\r
- exit(1);\r
- }\r
-\r
- /* Print provider specific attributes */\r
- for (i = 0; i < prov_attrs.num_provider_specific_attr; i++) {\r
- LOGPRINTF(" Provider Specific Attribute[%d] %s=%s\n",\r
- i, prov_attrs.provider_specific_attr[i].name,\r
- prov_attrs.provider_specific_attr[i].value);\r
-\r
- /* check for counter support */\r
- status = strcmp(prov_attrs.provider_specific_attr[i].name,\r
- "DAT_COUNTERS");\r
- if (!status)\r
- counters_ok = 1;\r
- }\r
-\r
- /* make sure provider supports counters */\r
- if ((counters) && (!counters_ok)) {\r
- printf("Disable dat_query_counters:"\r
- " Provider not built with counters\n");\r
- counters = 0;\r
- }\r
-\r
- status = dat_pz_create(ia, &pz);\r
- _OK(status, "dat_pz_create");\r
-\r
- status = dat_evd_create(ia, eps * 2, DAT_HANDLE_NULL, DAT_EVD_CR_FLAG,\r
- &cr_evd);\r
- _OK(status, "dat_evd_create CR");\r
- status = dat_evd_create(ia, eps * 2, DAT_HANDLE_NULL,\r
- DAT_EVD_CONNECTION_FLAG, &con_evd);\r
- _OK(status, "dat_evd_create CR");\r
- status = dat_evd_create(ia, eps * 10, DAT_HANDLE_NULL, DAT_EVD_DTO_FLAG,\r
- &dto_evd);\r
- _OK(status, "dat_evd_create DTO");\r
-\r
- memset(&ep_attr, 0, sizeof(ep_attr));\r
- if (ud_test) {\r
- msg_size += 40;\r
- ep_attr.service_type = DAT_IB_SERVICE_TYPE_UD;\r
- ep_attr.max_message_size = buf_size;\r
- ep_attr.max_rdma_read_in = 0;\r
- ep_attr.max_rdma_read_out = 0;\r
- } else {\r
- ep_attr.service_type = DAT_SERVICE_TYPE_RC;\r
- ep_attr.max_rdma_size = 0x10000;\r
- ep_attr.max_rdma_read_in = 4;\r
- ep_attr.max_rdma_read_out = 4;\r
- }\r
- ep_attr.qos = 0;\r
- ep_attr.recv_completion_flags = 0;\r
- ep_attr.max_recv_dtos = eps * 10;\r
- ep_attr.max_request_dtos = eps * 10;\r
- ep_attr.max_recv_iov = 1;\r
- ep_attr.max_request_iov = 1;\r
- ep_attr.request_completion_flags = DAT_COMPLETION_DEFAULT_FLAG;\r
- ep_attr.ep_transport_specific_count = 0;\r
- ep_attr.ep_transport_specific = NULL;\r
- ep_attr.ep_provider_specific_count = 0;\r
- ep_attr.ep_provider_specific = NULL;\r
-\r
- for (i = 0; i < eps; i++) {\r
- status = dat_ep_create(ia, pz, dto_evd, dto_evd,\r
- con_evd, &ep_attr, &ep[i]);\r
- _OK(status, "dat_ep_create");\r
- LOGPRINTF(" create_ep[%d]=%p\n", i, ep[i]);\r
- }\r
-\r
- for (i = 0; i < REG_MEM_COUNT * eps; i++) {\r
- buf[i] = (DAT_RMR_TRIPLET *) malloc(buf_size);\r
- region.for_va = buf[i];\r
- status = dat_lmr_create(ia,\r
- DAT_MEM_TYPE_VIRTUAL,\r
- region,\r
- buf_size,\r
- pz,\r
- DAT_MEM_PRIV_ALL_FLAG |\r
- DAT_IB_MEM_PRIV_REMOTE_ATOMIC,\r
- DAT_VA_TYPE_VA,\r
- &lmr[i],\r
- &lmr_context[i],\r
- &rmr_context[i],\r
- ®_size[i], ®_addr[i]);\r
- _OK(status, "dat_lmr_create");\r
- }\r
-\r
- /* register atomic return buffer for original data */\r
- atomic_buf = (DAT_UINT64 *) malloc(BUF_SIZE_ATOMIC);\r
- region.for_va = atomic_buf;\r
- status = dat_lmr_create(ia,\r
- DAT_MEM_TYPE_VIRTUAL,\r
- region,\r
- BUF_SIZE_ATOMIC,\r
- pz,\r
- DAT_MEM_PRIV_ALL_FLAG |\r
- DAT_IB_MEM_PRIV_REMOTE_ATOMIC,\r
- DAT_VA_TYPE_VA,\r
- &lmr_atomic,\r
- &lmr_atomic_context,\r
- &rmr_atomic_context,\r
- ®_atomic_size, ®_atomic_addr);\r
- _OK(status, "dat_lmr_create atomic");\r
-\r
- for (ii = 0; ii < eps; ii++) {\r
- for (i = RECV_BUF_INDEX; i < REG_MEM_COUNT; i++) {\r
- int ep_idx = 0;\r
- cookie.as_64 = (ii * REG_MEM_COUNT) + i;\r
- iov.lmr_context = lmr_context[(ii * REG_MEM_COUNT) + i];\r
- iov.virtual_address =\r
- (DAT_VADDR) (uintptr_t) buf[(ii * REG_MEM_COUNT) +\r
- i];\r
- iov.segment_length = buf_size;\r
- LOGPRINTF(" post_recv (%p) on ep[%d]=%p\n",\r
- buf[(ii * REG_MEM_COUNT) + i], ii, ep[ii]);\r
- /* ep[0], unless testing Server and multi EP's */\r
- if (server && multi_eps) {\r
- ep_idx = ii;\r
- cookie.as_64 = i;\r
- }\r
- status = dat_ep_post_recv(ep[ep_idx],\r
- 1,\r
- &iov,\r
- cookie,\r
- DAT_COMPLETION_DEFAULT_FLAG);\r
- _OK(status, "dat_ep_post_recv");\r
- }\r
- }\r
- /* setup receive buffer to initial string to be overwritten */\r
- strcpy((char *)buf[RCV_RDMA_BUF_INDEX], "blah, blah, blah\n");\r
-\r
- /* ud can resolve_ah and connect both ways, same EP */\r
- if (server || (!server && ud_test)) {\r
- if (server) {\r
- conn_qual = SERVER_ID;\r
- strcpy((char *)buf[SND_RDMA_BUF_INDEX], "Server data");\r
- } else {\r
- conn_qual = CLIENT_ID;\r
- strcpy((char *)buf[SND_RDMA_BUF_INDEX], "Client data");\r
- }\r
- status = dat_psp_create(ia,\r
- conn_qual,\r
- cr_evd, DAT_PSP_CONSUMER_FLAG, &psp);\r
- _OK(status, "dat_psp_create");\r
-\r
- /* Server always waits for first CR from Client */\r
- if (server)\r
- process_cr(0);\r
-\r
- }\r
-\r
- /* ud can resolve_ah and connect both ways */\r
- if (!server || (server && ud_test)) {\r
- struct addrinfo *target;\r
-\r
- if (ucm)\r
- goto no_resolution;\r
-\r
- if (getaddrinfo(hostname, NULL, NULL, &target) != 0) {\r
- printf("Error getting remote address.\n");\r
- exit(1);\r
- }\r
-\r
- printf("Remote %s Name: %s \n",\r
- server ? "Client" : "Server", hostname);\r
- printf("Remote %s Net Address: %s\n",\r
- server ? "Client" : "Server",\r
- inet_ntoa(((struct sockaddr_in *)\r
- target->ai_addr)->sin_addr));\r
-\r
- strcpy((char *)buf[SND_RDMA_BUF_INDEX], "Client written data");\r
- \r
- remote_addr = (DAT_IA_ADDRESS_PTR)target->ai_addr; /* IP */\r
-no_resolution:\r
- \r
- /* one Client EP, multiple Server EPs, same conn_qual \r
- * use private data to select EP on Server \r
- */\r
- for (i = 0; i < eps; i++) {\r
- /* pdata selects Server EP, \r
- * support both muliple Server and single EP's \r
- */\r
- if (multi_eps)\r
- pdata = hton32(i);\r
- else\r
- pdata = 0; /* just use first EP */\r
-\r
- status = dat_ep_connect(ep[0],\r
- remote_addr,\r
- (server ? CLIENT_ID :\r
- SERVER_ID), CONN_TIMEOUT, 4,\r
- (DAT_PVOID) & pdata, 0,\r
- DAT_CONNECT_DEFAULT_FLAG);\r
- _OK(status, "dat_ep_connect");\r
- }\r
-\r
- if (!ucm)\r
- freeaddrinfo(target);\r
- }\r
-\r
- /* UD: process CR's starting with 2nd on server, 1st for client */\r
- if (ud_test) {\r
- for (i = (server ? 1 : 0); i < eps; i++)\r
- process_cr(i);\r
- }\r
-\r
- /* RC and UD: process CONN EST events */\r
- for (i = 0; i < eps; i++)\r
- process_conn(i);\r
-\r
- /* UD: CONN EST events for CONN's and CR's */\r
- if (ud_test) {\r
- for (i = 0; i < eps; i++)\r
- process_conn(i);\r
- }\r
-\r
- printf("Connected! %d endpoints\n", eps);\r
-\r
- /*\r
- * Setup our remote memory and tell the other side about it\r
- * Swap to network order.\r
- */\r
- r_iov = (DAT_RMR_TRIPLET *) buf[SEND_BUF_INDEX];\r
- r_iov->rmr_context = hton32(rmr_context[RCV_RDMA_BUF_INDEX]);\r
- r_iov->virtual_address =\r
- hton64((DAT_VADDR) (uintptr_t) buf[RCV_RDMA_BUF_INDEX]);\r
- r_iov->segment_length = hton32(buf_size);\r
-\r
- printf("Send RMR message: r_key_ctx=0x%x,va=" F64x ",len=0x%x\n",\r
- hton32(r_iov->rmr_context),\r
- hton64(r_iov->virtual_address), hton32(r_iov->segment_length));\r
-\r
- send_msg(buf[SEND_BUF_INDEX],\r
- sizeof(DAT_RMR_TRIPLET),\r
- lmr_context[SEND_BUF_INDEX],\r
- cookie, DAT_COMPLETION_SUPPRESS_FLAG);\r
-\r
- dat_ep_get_status(ep[0], NULL, &in, &out);\r
- printf("EP[0] status: posted buffers: Req=%d, Rcv=%d\n", in, out);\r
-\r
- /*\r
- * Wait for their RMR\r
- */\r
- for (i = 0, ctx = 0; i < eps; i++, ctx++) {\r
- /* expected cookie, recv buf idx in every mem pool */\r
- ctx = (ctx % REG_MEM_COUNT) ? ctx : ctx + RECV_BUF_INDEX;\r
- LOGPRINTF("Waiting for remote to send RMR data\n");\r
-\r
- status = dat_evd_wait(dto_evd, DTO_TIMEOUT, 1, &event, &nmore);\r
- _OK(status, "dat_evd_wait after dat_ep_post_send");\r
-\r
- if ((event.event_number != DAT_DTO_COMPLETION_EVENT) &&\r
- (ud_test && event.event_number != DAT_IB_DTO_EVENT)) {\r
- printf("unexpected event waiting for RMR context "\r
- "- 0x%x\n", event.event_number);\r
- exit(1);\r
- }\r
- _OK(dto_event->status, "event status for post_recv");\r
-\r
- /* careful when checking cookies:\r
- * Client - receiving multi messages on a single EP \r
- * Server - not receiving on multiple EP's\r
- */\r
- if (!server || (server && !multi_eps)) {\r
- if (dto_event->transfered_length != msg_size ||\r
- dto_event->user_cookie.as_64 != ctx) {\r
- printf("unexpected event data on recv: len=%d"\r
- " cookie=" F64x " expected %d/%d\n",\r
- (int)dto_event->transfered_length,\r
- dto_event->user_cookie.as_64,\r
- msg_size, ctx);\r
- exit(1);\r
- }\r
- /* Server - receiving one message each across many EP's */\r
- } else {\r
- if (dto_event->transfered_length != msg_size ||\r
- dto_event->user_cookie.as_64 != RECV_BUF_INDEX) {\r
- printf("unexpected event data on recv: len=%d"\r
- "cookie=" F64x " expected %d/%d\n",\r
- (int)dto_event->transfered_length,\r
- dto_event->user_cookie.as_64,\r
- msg_size, RECV_BUF_INDEX);\r
- exit(1);\r
- }\r
- }\r
-\r
- /* swap RMR,address info to host order */\r
- if (!server || (server && !multi_eps))\r
- r_iov = (DAT_RMR_TRIPLET *) buf[ctx];\r
- else\r
- r_iov =\r
- (DAT_RMR_TRIPLET *) buf[(i * REG_MEM_COUNT) +\r
- RECV_BUF_INDEX];\r
-\r
- if (ud_test)\r
- r_iov = (DAT_RMR_TRIPLET *) ((char *)r_iov + 40);\r
-\r
- r_iov->rmr_context = ntoh32(r_iov->rmr_context);\r
- r_iov->virtual_address = ntoh64(r_iov->virtual_address);\r
- r_iov->segment_length = ntoh32(r_iov->segment_length);\r
-\r
- printf("Recv RMR message: r_iov(%p):"\r
- " r_key_ctx=%x,va=" F64x ",len=0x%x on EP=%p\n",\r
- r_iov, r_iov->rmr_context,\r
- r_iov->virtual_address,\r
- r_iov->segment_length, dto_event->ep_handle);\r
- }\r
- return (0);\r
-}\r
-\r
-int disconnect_ep(void)\r
-{\r
- DAT_RETURN status;\r
- DAT_EVENT event;\r
- DAT_COUNT nmore;\r
- int i;\r
- \r
- if (counters) { /* examples of query and print */\r
- int ii;\r
- DAT_UINT64 ia_cntrs[DCNT_IA_ALL_COUNTERS];\r
-\r
- dat_query_counters(ia, DCNT_IA_ALL_COUNTERS, ia_cntrs, 0);\r
- printf(" IA Cntrs:");\r
- for (ii = 0; ii < DCNT_IA_ALL_COUNTERS; ii++)\r
- printf(" " F64u "", ia_cntrs[ii]);\r
- printf("\n");\r
- dat_print_counters(ia, DCNT_IA_ALL_COUNTERS, 0);\r
- }\r
- \r
- if (!ud_test) {\r
- status = dat_ep_disconnect(ep[0], DAT_CLOSE_DEFAULT);\r
- _OK2(status, "dat_ep_disconnect");\r
-\r
- status = dat_evd_wait(con_evd, DAT_TIMEOUT_INFINITE, 1,\r
- &event, &nmore);\r
- _OK(status, "dat_evd_wait");\r
- }\r
- if (psp) {\r
- status = dat_psp_free(psp);\r
- _OK2(status, "dat_psp_free");\r
- }\r
- for (i = 0; i < REG_MEM_COUNT * eps; i++) {\r
- status = dat_lmr_free(lmr[i]);\r
- _OK2(status, "dat_lmr_free");\r
- }\r
- if (lmr_atomic) {\r
- status = dat_lmr_free(lmr_atomic);\r
- _OK2(status, "dat_lmr_free_atomic");\r
- }\r
- for (i = 0; i < eps; i++) {\r
- if (counters) { /* examples of query and print */\r
- int ii;\r
- DAT_UINT64 ep_cntrs[DCNT_EP_ALL_COUNTERS];\r
-\r
- dat_query_counters(ep[i], DCNT_EP_ALL_COUNTERS,\r
- ep_cntrs, 0);\r
- printf(" EP[%d] Cntrs:", i);\r
- for (ii = 0; ii < DCNT_EP_ALL_COUNTERS; ii++)\r
- printf(" " F64u "", ep_cntrs[ii]);\r
- printf("\n");\r
- dat_print_counters(ep[i], DCNT_EP_ALL_COUNTERS, 0);\r
- }\r
- status = dat_ep_free(ep[i]);\r
- _OK2(status, "dat_ep_free");\r
- }\r
- if (counters) { /* examples of query and print */\r
- int ii;\r
- DAT_UINT64 evd_cntrs[DCNT_EVD_ALL_COUNTERS];\r
-\r
- dat_query_counters(dto_evd, DCNT_EVD_ALL_COUNTERS,\r
- evd_cntrs, 0);\r
- printf(" DTO_EVD Cntrs:");\r
- for (ii = 0; ii < DCNT_EVD_ALL_COUNTERS; ii++)\r
- printf(" " F64u "", evd_cntrs[ii]);\r
- printf("\n");\r
- dat_print_counters(dto_evd, DCNT_EVD_ALL_COUNTERS, 0);\r
-\r
- dat_query_counters(con_evd, DCNT_EVD_ALL_COUNTERS,\r
- evd_cntrs, 0);\r
- printf(" CONN_EVD Cntrs:");\r
- for (ii = 0; ii < DCNT_EVD_ALL_COUNTERS; ii++)\r
- printf(" " F64u "", evd_cntrs[ii]);\r
- printf("\n");\r
- dat_print_counters(con_evd, DCNT_EVD_ALL_COUNTERS, 0);\r
-\r
- dat_query_counters(cr_evd, DCNT_EVD_ALL_COUNTERS, evd_cntrs, 0);\r
- printf(" CR_EVD Cntrs:");\r
- for (ii = 0; ii < DCNT_EVD_ALL_COUNTERS; ii++)\r
- printf(" " F64u "", evd_cntrs[ii]);\r
- printf("\n");\r
- dat_print_counters(cr_evd, DCNT_EVD_ALL_COUNTERS, 0);\r
- }\r
- status = dat_evd_free(dto_evd);\r
- _OK2(status, "dat_evd_free DTO");\r
-\r
- status = dat_evd_free(con_evd);\r
- _OK2(status, "dat_evd_free CON");\r
-\r
- status = dat_evd_free(cr_evd);\r
- _OK2(status, "dat_evd_free CR");\r
-\r
- status = dat_pz_free(pz);\r
- _OK2(status, "dat_pz_free");\r
-\r
- status = dat_ia_close(ia, DAT_CLOSE_DEFAULT);\r
- _OK2(status, "dat_ia_close");\r
-\r
- return (0);\r
-}\r
-\r
-int do_immediate()\r
-{\r
- DAT_EVENT event;\r
- DAT_COUNT nmore;\r
- DAT_LMR_TRIPLET iov;\r
- DAT_RMR_TRIPLET r_iov;\r
- DAT_DTO_COOKIE cookie;\r
- DAT_RETURN status;\r
- DAT_UINT32 immed_data;\r
- DAT_UINT32 immed_data_recv = 0;\r
- DAT_DTO_COMPLETION_EVENT_DATA *dto_event =\r
- &event.event_data.dto_completion_event_data;\r
- DAT_IB_EXTENSION_EVENT_DATA *ext_event =\r
- (DAT_IB_EXTENSION_EVENT_DATA *) & event.event_extension_data[0];\r
-\r
- printf("\nDoing RDMA WRITE IMMEDIATE DATA\n");\r
-\r
- if (server) {\r
- immed_data = 0x1111;\r
- } else {\r
- immed_data = 0x7777;\r
- }\r
-\r
- cookie.as_64 = 0x5555;\r
-\r
- /* RMR info already swapped back to host order in connect_ep */\r
- r_iov = *buf[RECV_BUF_INDEX];\r
-\r
- iov.lmr_context = lmr_context[SND_RDMA_BUF_INDEX];\r
- iov.virtual_address = (DAT_VADDR) (uintptr_t) buf[SND_RDMA_BUF_INDEX];\r
- iov.segment_length = buf_size;\r
-\r
- cookie.as_64 = 0x9999;\r
-\r
- status = dat_ib_post_rdma_write_immed(ep[0], // ep_handle\r
- 1, // segments\r
- &iov, // LMR\r
- cookie, // user_cookie\r
- &r_iov, // RMR\r
- immed_data,\r
- DAT_COMPLETION_DEFAULT_FLAG);\r
- _OK(status, "dat_ib_post_rdma_write_immed");\r
-\r
- /*\r
- * Collect first event, write completion or inbound recv with immed\r
- */\r
- status = dat_evd_wait(dto_evd, DTO_TIMEOUT, 1, &event, &nmore);\r
- _OK(status, "dat_evd_wait after dat_ib_post_rdma_write");\r
- if (event.event_number != DAT_IB_DTO_EVENT) {\r
- printf("unexpected event #0x%x waiting for WR-IMMED #0x%x\n",\r
- event.event_number, DAT_IB_DTO_EVENT);\r
- exit(1);\r
- }\r
-\r
- if (nmore)\r
- printf("%s() nmore %d\n", __FUNCTION__, nmore);\r
- _OK(dto_event->status, "DTO event status");\r
- if (ext_event->type == DAT_IB_RDMA_WRITE_IMMED) {\r
- if ((dto_event->transfered_length != buf_size) ||\r
- (dto_event->user_cookie.as_64 != 0x9999)) {\r
- printf\r
- ("unexpected event data for rdma_write_immed: len=%d "\r
- "cookie=0x%x\n", (int)dto_event->transfered_length,\r
- (int)dto_event->user_cookie.as_64);\r
- exit(1);\r
- }\r
- } else if (ext_event->type == DAT_IB_RDMA_WRITE_IMMED_DATA) {\r
- if ((dto_event->transfered_length != buf_size) ||\r
- (dto_event->user_cookie.as_64 != RECV_BUF_INDEX + 1)) {\r
- printf\r
- ("unexpected event data of immediate write: len=%d "\r
- "cookie=" F64x " expected %d/%d\n",\r
- (int)dto_event->transfered_length,\r
- dto_event->user_cookie.as_64, (int)sizeof(int),\r
- RECV_BUF_INDEX + 1);\r
- exit(1);\r
- }\r
-\r
- /* get immediate data from event */\r
- immed_data_recv = ext_event->val.immed.data;\r
- } else {\r
- printf("unexpected extension type for event - 0x%x, 0x%x\n",\r
- event.event_number, ext_event->type);\r
- exit(1);\r
- }\r
-\r
- /*\r
- * Collect second event, write completion or inbound recv with immed\r
- */\r
- status = dat_evd_wait(dto_evd, DTO_TIMEOUT, 1, &event, &nmore);\r
- _OK(status, "dat_evd_wait after dat_ib_post_rdma_write");\r
- if (event.event_number != DAT_IB_DTO_EVENT) {\r
- printf("unexpected event # waiting for WR-IMMED - 0x%x\n",\r
- event.event_number);\r
- exit(1);\r
- }\r
-\r
- _OK(dto_event->status, "event status");\r
- if (ext_event->type == DAT_IB_RDMA_WRITE_IMMED) {\r
- if ((dto_event->transfered_length != buf_size) ||\r
- (dto_event->user_cookie.as_64 != 0x9999)) {\r
- printf\r
- ("unexpected event data for rdma_write_immed: len=%d "\r
- "cookie=0x%x\n", (int)dto_event->transfered_length,\r
- (int)dto_event->user_cookie.as_64);\r
- exit(1);\r
- }\r
- } else if (ext_event->type == DAT_IB_RDMA_WRITE_IMMED_DATA) {\r
- if ((dto_event->transfered_length != buf_size) ||\r
- (dto_event->user_cookie.as_64 != RECV_BUF_INDEX + 1)) {\r
- printf\r
- ("unexpected event data of immediate write: len=%d "\r
- "cookie=" F64x " expected %d/%d\n",\r
- (int)dto_event->transfered_length,\r
- dto_event->user_cookie.as_64, (int)sizeof(int),\r
- RECV_BUF_INDEX + 1);\r
- exit(1);\r
- }\r
-\r
- /* get immediate data from event */\r
- immed_data_recv = ext_event->val.immed.data;\r
- } else {\r
- printf("unexpected extension type for event - 0x%x, 0x%x\n",\r
- event.event_number, ext_event->type);\r
- exit(1);\r
- }\r
-\r
- if ((server) && (immed_data_recv != 0x7777)) {\r
- printf("ERROR: Server: unexpected imm_data_recv 0x%x/0x%x\n",\r
- 0x7777, immed_data_recv);\r
- exit(1);\r
- } else if ((!server) && (immed_data_recv != 0x1111)) {\r
- printf("ERROR: Client: unexpected imm_data_recv 0x%x/0x%x\n",\r
- 0x1111, immed_data_recv);\r
- exit(1);\r
- }\r
-\r
- if (server)\r
- printf("Server received immed_data=0x%x\n", immed_data_recv);\r
- else\r
- printf("Client received immed_data=0x%x\n", immed_data_recv);\r
-\r
- printf("rdma buffer %p contains: %s\n",\r
- buf[RCV_RDMA_BUF_INDEX], (char *)buf[RCV_RDMA_BUF_INDEX]);\r
-\r
- printf("\n RDMA_WRITE_WITH_IMMEDIATE_DATA test - PASSED\n");\r
- return (0);\r
-}\r
-\r
-int do_cmp_swap()\r
-{\r
- DAT_DTO_COOKIE cookie;\r
- DAT_RETURN status;\r
- DAT_EVENT event;\r
- DAT_COUNT nmore;\r
- DAT_LMR_TRIPLET l_iov;\r
- DAT_RMR_TRIPLET r_iov;\r
- volatile DAT_UINT64 *target = (DAT_UINT64 *) buf[RCV_RDMA_BUF_INDEX];\r
- DAT_DTO_COMPLETION_EVENT_DATA *dto_event =\r
- &event.event_data.dto_completion_event_data;\r
- DAT_IB_EXTENSION_EVENT_DATA *ext_event =\r
- (DAT_IB_EXTENSION_EVENT_DATA *) & event.event_extension_data[0];\r
-\r
- printf("\nDoing CMP and SWAP\n");\r
-\r
- /* RMR info already swapped back to host order in connect_ep */\r
- r_iov = *buf[RECV_BUF_INDEX];\r
-\r
- l_iov.lmr_context = lmr_atomic_context;\r
- l_iov.virtual_address = (DAT_UINT64) (uintptr_t) atomic_buf;\r
- l_iov.segment_length = BUF_SIZE_ATOMIC;\r
-\r
- cookie.as_64 = 3333;\r
-\r
- if (server) {\r
- *target = 0x12345;\r
- sleep(1);\r
- /* Server does not compare and should not swap */\r
- printf("dtx svr - starting cmp_swap\n");\r
- status = dat_ib_post_cmp_and_swap(ep[0],\r
- (DAT_UINT64) 0x654321,\r
- (DAT_UINT64) 0x6789A,\r
- &l_iov,\r
- cookie,\r
- &r_iov,\r
- DAT_COMPLETION_DEFAULT_FLAG);\r
- printf("dtx svr - done cmp_swap, chk status\n");\r
- } else {\r
- *target = 0x54321;\r
- sleep(1);\r
- printf("dtx cli - starting cmp_swap\n");\r
- /* Client does compare and should swap */\r
- status = dat_ib_post_cmp_and_swap(ep[0],\r
- (DAT_UINT64) 0x12345,\r
- (DAT_UINT64) 0x98765,\r
- &l_iov,\r
- cookie,\r
- &r_iov,\r
- DAT_COMPLETION_DEFAULT_FLAG);\r
- printf("dtx cli - done cmp_swap, chk status\n");\r
- }\r
- _OK(status, "dat_ib_post_cmp_and_swap");\r
- status = dat_evd_wait(dto_evd, DTO_TIMEOUT, 1, &event, &nmore);\r
- _OK(status, "dat_evd_wait for compare and swap");\r
- if (event.event_number != DAT_IB_DTO_EVENT) {\r
- printf("unexpected event after post_cmp_and_swap: 0x%x\n",\r
- event.event_number);\r
- exit(1);\r
- }\r
-\r
- _OK(dto_event->status, "event status for CMP and SWAP");\r
- if (ext_event->type != DAT_IB_CMP_AND_SWAP) {\r
- printf("unexpected event data of cmp_swap: type=%d cookie=%d "\r
- "original " F64x "\n",\r
- (int)ext_event->type,\r
- (int)dto_event->user_cookie.as_64, *atomic_buf);\r
- exit(1);\r
- }\r
-\r
- sleep(2); /* wait for other side to complete swap */\r
-\r
- if (server) {\r
- printf("Server got original data = " F64x ", expected "\r
- "0x54321\n", *atomic_buf);\r
- printf("Client final result (on Server) = " F64x ", expected "\r
- "0x98765\n", *target);\r
-\r
- if (*atomic_buf != 0x54321 || *target != 0x98765) {\r
- printf("ERROR: Server CMP_SWAP\n");\r
- exit(1);\r
- }\r
- } else {\r
- printf("Client got original data = " F64x ", expected "\r
- "0x12345\n", *atomic_buf);\r
- printf("Server final result (on Client) = 0x" F64x ", expected "\r
- "0x54321\n", *target);\r
-\r
- if (*atomic_buf != 0x12345 || *target != 0x54321) {\r
- printf("ERROR: Client CMP_SWAP\n");\r
- exit(1);\r
- }\r
- }\r
- printf("\n CMP_SWAP test - PASSED\n");\r
- return (0);\r
-}\r
-\r
-int do_fetch_add()\r
-{\r
- DAT_DTO_COOKIE cookie;\r
- DAT_RETURN status;\r
- DAT_EVENT event;\r
- DAT_COUNT nmore;\r
- DAT_LMR_TRIPLET l_iov;\r
- DAT_RMR_TRIPLET r_iov;\r
- volatile DAT_UINT64 *target = (DAT_UINT64 *) buf[RCV_RDMA_BUF_INDEX];\r
- DAT_DTO_COMPLETION_EVENT_DATA *dto_event =\r
- &event.event_data.dto_completion_event_data;\r
- DAT_IB_EXTENSION_EVENT_DATA *ext_event =\r
- (DAT_IB_EXTENSION_EVENT_DATA *) & event.event_extension_data[0];\r
-\r
- printf("\nDoing FETCH and ADD\n");\r
-\r
- /* RMR info already swapped back to host order in connect_ep */\r
- r_iov = *buf[RECV_BUF_INDEX];\r
-\r
- l_iov.lmr_context = lmr_atomic_context;\r
- l_iov.virtual_address = (DAT_UINT64) (uintptr_t) atomic_buf;\r
- l_iov.segment_length = BUF_SIZE_ATOMIC;\r
-\r
- cookie.as_64 = 0x7777;\r
- if (server) {\r
- /* Wait for Client to finish cmp_swap */\r
- while (*target != 0x98765)\r
- sleep(1);\r
- *target = 0x10;\r
- sleep(1);\r
- status = dat_ib_post_fetch_and_add(ep[0],\r
- (DAT_UINT64) 0x100,\r
- &l_iov,\r
- cookie,\r
- &r_iov,\r
- DAT_COMPLETION_DEFAULT_FLAG);\r
- } else {\r
- /* Wait for Server, no swap so nothing to check */\r
- *target = 0x100;\r
- sleep(1);\r
- status = dat_ib_post_fetch_and_add(ep[0],\r
- (DAT_UINT64) 0x10,\r
- &l_iov,\r
- cookie,\r
- &r_iov,\r
- DAT_COMPLETION_DEFAULT_FLAG);\r
- }\r
- _OK(status, "dat_ib_post_fetch_and_add");\r
- status = dat_evd_wait(dto_evd, DTO_TIMEOUT, 1, &event, &nmore);\r
- _OK(status, "dat_evd_wait for fetch and add");\r
- if (event.event_number != DAT_IB_DTO_EVENT) {\r
- printf("unexpected event after post_fetch_and_add: 0x%x\n",\r
- event.event_number);\r
- exit(1);\r
- }\r
-\r
- _OK(dto_event->status, "event status for FETCH and ADD");\r
- if (ext_event->type != DAT_IB_FETCH_AND_ADD) {\r
- printf("unexpected event data of fetch and add : type=%d "\r
- "cookie=%d original%d\n",\r
- (int)ext_event->type,\r
- (int)dto_event->user_cookie.as_64, (int)*atomic_buf);\r
- exit(1);\r
- }\r
-\r
- if (server) {\r
- printf("Client original data (on Server) = " F64x ", expected "\r
- "0x100\n", *atomic_buf);\r
- } else {\r
- printf("Server original data (on Client) = " F64x ", expected "\r
- "0x10\n", *atomic_buf);\r
- }\r
-\r
- sleep(1);\r
-\r
- if (server) {\r
- status = dat_ib_post_fetch_and_add(ep[0],\r
- (DAT_UINT64) 0x100,\r
- &l_iov,\r
- cookie,\r
- &r_iov,\r
- DAT_COMPLETION_DEFAULT_FLAG);\r
- } else {\r
- status = dat_ib_post_fetch_and_add(ep[0],\r
- (DAT_UINT64) 0x10,\r
- &l_iov,\r
- cookie,\r
- &r_iov,\r
- DAT_COMPLETION_DEFAULT_FLAG);\r
- }\r
-\r
- status = dat_evd_wait(dto_evd, DTO_TIMEOUT, 1, &event, &nmore);\r
- _OK(status, "dat_evd_wait for second fetch and add");\r
- if (event.event_number != DAT_IB_DTO_EVENT) {\r
- printf("unexpected event after second post_fetch_and_add: "\r
- "0x%x\n", event.event_number);\r
- exit(1);\r
- }\r
-\r
- _OK(dto_event->status, "event status for second FETCH and ADD");\r
- if (ext_event->type != DAT_IB_FETCH_AND_ADD) {\r
- printf("unexpected event data of second fetch and add : "\r
- "type=%d cookie=%d original%p\n",\r
- (int)ext_event->type,\r
- (int)dto_event->user_cookie.as_64, atomic_buf);\r
- exit(1);\r
- }\r
-\r
- sleep(1); /* wait for other side to complete fetch_add */\r
-\r
- if (server) {\r
- printf("Server got original data = " F64x ", expected "\r
- "0x200\n", *atomic_buf);\r
- printf("Client final result (on Server) = " F64x ", expected "\r
- "0x30\n", *target);\r
-\r
- if (*atomic_buf != 0x200 || *target != 0x30) {\r
- printf("ERROR: Server FETCH_ADD\n");\r
- exit(1);\r
- }\r
- } else {\r
- printf("Server side original data = " F64x ", expected "\r
- "0x20\n", *atomic_buf);\r
- printf("Server final result (on Client) = " F64x ", expected "\r
- "0x300\n", *target);\r
-\r
- if (*atomic_buf != 0x20 || *target != 0x300) {\r
- printf("ERROR: Server FETCH_ADD\n");\r
- exit(1);\r
- }\r
- }\r
- printf("\n FETCH_ADD test - PASSED\n");\r
- return (0);\r
-}\r
-\r
-int main(int argc, char **argv)\r
-{\r
- int rc;\r
-\r
- /* parse arguments */\r
- while ((rc = getopt(argc, argv, "csvumpU:h:b:P:q:l:")) != -1) {\r
- switch (rc) {\r
- case 'u':\r
- ud_test = 1;\r
- eps = MAX_EP_COUNT / 2;\r
- break;\r
- case 'm':\r
- multi_eps = 1;\r
- break;\r
- case 'c':\r
- server = 0;\r
- break;\r
- case 's':\r
- server = 1;\r
- break;\r
- case 'p':\r
- counters = 1;\r
- break;\r
- case 'h':\r
- remote_host = 1;\r
- strcpy(hostname, optarg);\r
- break;\r
- case 'b':\r
- buf_size = atoi(optarg);\r
- break;\r
- case 'U':\r
- ud_test = 1;\r
- eps = MIN(atoi(optarg), MAX_EP_COUNT);\r
- break;\r
- case 'P':\r
- strcpy(provider, optarg);\r
- break;\r
- case 'v':\r
- verbose = 1;\r
- break;\r
- case 'q':\r
- /* map UCM qpn into AF_INET6 sin6_flowinfo */\r
- remote.sin6_family = AF_INET6;\r
- remote.sin6_flowinfo = htonl(strtol(optarg,NULL,0));\r
- ucm = 1;\r
- server = 0;\r
- break;\r
- case 'l':\r
- /* map UCM lid into AF_INET6 sin6_port */\r
- remote.sin6_family = AF_INET6;\r
- remote.sin6_port = htons(strtol(optarg,NULL,0));\r
- ucm = 1;\r
- server = 0;\r
- break;\r
- default:\r
- print_usage();\r
- exit(-12);\r
- }\r
- }\r
-\r
-#if defined(_WIN32) || defined(_WIN64)\r
- {\r
- WSADATA wsaData;\r
- int i;\r
-\r
- i = WSAStartup(MAKEWORD(2, 2), &wsaData);\r
- if (i != 0) {\r
- printf("%s WSAStartup(2.2) fail? (0x%x)\n", argv[0], i);\r
- fflush(stdout);\r
- exit(1);\r
- }\r
- }\r
-#endif\r
- /* for non UD tests, -h is always client */\r
- if (remote_host && !ud_test)\r
- server = 0;\r
-\r
- if (!server) {\r
- printf("\nRunning as Client - %s %s %d endpoint(s)\n",\r
- provider, ud_test ? "UD test" : "", eps);\r
- } else {\r
- printf("\nRunning as Server - %s %s %d endpoint(s)\n",\r
- provider, ud_test ? "UD test" : "", eps);\r
- }\r
-\r
- /*\r
- * connect\r
- */\r
- if (connect_ep(hostname)) {\r
- _WSACleanup();\r
- exit(1);\r
- }\r
- if (ud_test)\r
- goto bail;\r
-\r
- if (do_immediate()) {\r
- _WSACleanup();\r
- exit(1);\r
- }\r
- if (do_cmp_swap()) {\r
- _WSACleanup();\r
- exit(1);\r
- }\r
- if (do_fetch_add()) {\r
- _WSACleanup();\r
- exit(1);\r
- }\r
- bail:\r
- rc = disconnect_ep();\r
- _WSACleanup();\r
-\r
- if (!rc)\r
- printf("\n IB extension test - %s test PASSED\n\n",\r
- ud_test ? "UD" : "immed/atomic");\r
- return rc;\r
-}\r
+/*
+ * Copyright (c) 2007-2008 Intel Corporation. All rights reserved.
+ *
+ * This software is available to you under a choice of one of two
+ * licenses. You may choose to be licensed under the terms of the GNU
+ * General Public License (GPL) Version 2, available from the file
+ * COPYING in the main directory of this source tree, or the
+ * OpenIB.org BSD license below:
+ *
+ * Redistribution and use in source and binary forms, with or
+ * without modification, are permitted provided that the following
+ * conditions are met:
+ *
+ * - Redistributions of source code must retain the above
+ * copyright notice, this list of conditions and the following
+ * disclaimer.
+ *
+ * - Redistributions in binary form must reproduce the above
+ * copyright notice, this list of conditions and the following
+ * disclaimer in the documentation and/or other materials
+ * provided with the distribution.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
+ * EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
+ * MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
+ * NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS
+ * BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN
+ * ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN
+ * CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+ * SOFTWARE.
+ *
+ * $Id: $
+ */
+#include <stdio.h>
+#include <stdlib.h>
+
+#if defined(_WIN32) || defined(_WIN64)
+#include <windows.h>
+#include <winsock2.h>
+#include <ws2tcpip.h>
+#include <io.h>
+#include <process.h>
+#include <complib/cl_types.h>
+#include "..\..\..\..\etc\user\getopt.c"
+#define __BYTE_ORDER __LITTLE_ENDIAN
+
+#define getpid() ((int)GetCurrentProcessId())
+#define F64x "%I64x"
+#define F64u "%I64u"
+#define DAPL_PROVIDER "ibnic0v2"
+#else
+#include <netdb.h>
+#include <sys/types.h>
+#include <sys/socket.h>
+#include <sys/time.h>
+#include <netinet/in.h>
+#include <netinet/tcp.h>
+#include <arpa/inet.h>
+#include <inttypes.h>
+#include <string.h>
+#include <unistd.h>
+
+#define DAPL_PROVIDER "ofa-v2-ib0"
+#define F64x "%"PRIx64""
+#define F64u "%"PRIu64""
+
+#endif
+
+#include "dat2/udat.h"
+#include "dat2/dat_ib_extensions.h"
+
+int disconnect_ep(void);
+
+#define _OK(status, str) \
+{\
+ const char *maj_msg, *min_msg;\
+ if (status != DAT_SUCCESS) {\
+ dat_strerror(status, &maj_msg, &min_msg);\
+ fprintf(stderr, str " returned %s : %s\n", maj_msg, min_msg);\
+ dat_ia_close(ia, DAT_CLOSE_DEFAULT);\
+ exit(1);\
+ } else if (verbose) {\
+ printf("dtestx: %s success\n",str);\
+ }\
+}
+
+#define _OK2(status, str)\
+{\
+ const char *maj_msg, *min_msg;\
+ if (status != DAT_SUCCESS) {\
+ dat_strerror(status, &maj_msg, &min_msg);\
+ fprintf(stderr, str " returned %s : %s\n", maj_msg, min_msg);\
+ dat_ia_close(ia, DAT_CLOSE_DEFAULT);\
+ exit(1);\
+ } else if (verbose) {\
+ printf("dtestx: %s\n",str);\
+ }\
+}
+
+/* byte swap helpers from Complib */
+#if __BYTE_ORDER == __LITTLE_ENDIAN
+#define ntoh16(x) (uint16_t)( \
+ (((uint16_t)(x) & 0x00FF) << 8) | \
+ (((uint16_t)(x) & 0xFF00) >> 8))
+#define hton16(x) ntoh16(x)
+#define ntoh32(x) (uint32_t)( \
+ (((uint32_t)(x) & 0x000000FF) << 24)| \
+ (((uint32_t)(x) & 0x0000FF00) << 8) | \
+ (((uint32_t)(x) & 0x00FF0000) >> 8) | \
+ (((uint32_t)(x) & 0xFF000000) >> 24))
+#define hton32(x) ntoh32(x)
+#define ntoh64(x) (uint64_t)( \
+ (((uint64_t)x & 0x00000000000000FFULL) << 56) | \
+ (((uint64_t)x & 0x000000000000FF00ULL) << 40) | \
+ (((uint64_t)x & 0x0000000000FF0000ULL) << 24) | \
+ (((uint64_t)x & 0x00000000FF000000ULL) << 8 ) | \
+ (((uint64_t)x & 0x000000FF00000000ULL) >> 8 ) | \
+ (((uint64_t)x & 0x0000FF0000000000ULL) >> 24) | \
+ (((uint64_t)x & 0x00FF000000000000ULL) >> 40) | \
+ (((uint64_t)x & 0xFF00000000000000ULL) >> 56))
+#define hton64(x) ntoh64(x)
+#elif __BYTE_ORDER == __BIG_ENDIAN
+#define hton16(x) (x)
+#define ntoh16(x) (x)
+#define hton32(x) (x)
+#define ntoh32(x) (x)
+#define hton64(x) (x)
+#define ntoh64(x) (x)
+#endif /* __BYTE_ORDER == __BIG_ENDIAN */
+
+#define MIN(a, b) ((a < b) ? (a) : (b))
+#define MAX(a, b) ((a > b) ? (a) : (b))
+
+#define DTO_TIMEOUT (1000*1000*5)
+#define CONN_TIMEOUT (1000*1000*30)
+#define SERVER_TIMEOUT (DAT_TIMEOUT_INFINITE)
+#define CLIENT_ID 31111
+#define SERVER_ID 31112
+#define BUF_SIZE 256
+#define BUF_SIZE_ATOMIC 8
+#define REG_MEM_COUNT 10
+#define SND_RDMA_BUF_INDEX 0
+#define RCV_RDMA_BUF_INDEX 1
+#define SEND_BUF_INDEX 2
+#define RECV_BUF_INDEX 3
+#define MAX_EP_COUNT 8
+
+DAT_VADDR *atomic_buf;
+DAT_LMR_HANDLE lmr_atomic;
+DAT_LMR_CONTEXT lmr_atomic_context;
+DAT_RMR_CONTEXT rmr_atomic_context;
+DAT_VLEN reg_atomic_size;
+DAT_VADDR reg_atomic_addr;
+DAT_LMR_HANDLE lmr[REG_MEM_COUNT * MAX_EP_COUNT];
+DAT_LMR_CONTEXT lmr_context[REG_MEM_COUNT * MAX_EP_COUNT];
+DAT_RMR_TRIPLET rmr[REG_MEM_COUNT * MAX_EP_COUNT];
+DAT_RMR_CONTEXT rmr_context[REG_MEM_COUNT * MAX_EP_COUNT];
+DAT_VLEN reg_size[REG_MEM_COUNT * MAX_EP_COUNT];
+DAT_VADDR reg_addr[REG_MEM_COUNT * MAX_EP_COUNT];
+DAT_RMR_TRIPLET *buf[REG_MEM_COUNT * MAX_EP_COUNT];
+DAT_EP_HANDLE ep[MAX_EP_COUNT];
+DAT_EVD_HANDLE async_evd = DAT_HANDLE_NULL;
+DAT_IA_HANDLE ia = DAT_HANDLE_NULL;
+DAT_PZ_HANDLE pz = DAT_HANDLE_NULL;
+DAT_EVD_HANDLE cr_evd = DAT_HANDLE_NULL;
+DAT_EVD_HANDLE con_evd = DAT_HANDLE_NULL;
+DAT_EVD_HANDLE dto_evd = DAT_HANDLE_NULL;
+DAT_PSP_HANDLE psp = DAT_HANDLE_NULL;
+int server = 1;
+int remote_host = 0;
+int ud_test = 0;
+int multi_eps = 0;
+int buf_size = BUF_SIZE;
+int msg_size = sizeof(DAT_RMR_TRIPLET);
+char provider[64] = DAPL_PROVIDER;
+char hostname[256] = { 0 };
+DAT_IB_ADDR_HANDLE remote_ah[MAX_EP_COUNT];
+int eps = 1;
+int verbose = 0;
+int counters = 0;
+int counters_ok = 0;
+static int ucm = 0;
+static DAT_SOCK_ADDR6 remote;
+
+#define LOGPRINTF if (verbose) printf
+
+void print_usage(void)
+{
+ printf("\n dtestx usage \n\n");
+ printf("v: verbose\n");
+ printf("p: print counters\n");
+ printf("u unreliable datagram test\n");
+ printf("U: unreliable datagram test, UD endpoint count\n");
+ printf("m unreliable datagram test, multiple Server endpoints\n");
+ printf("b: buf length to allocate\n");
+ printf("h: hostname/address of Server, client and UDP server\n");
+ printf("c: Client\n");
+ printf("s: Server, default\n");
+ printf("P: provider name (default = ofa-v2-ib0)\n");
+ printf("\n");
+}
+
+#if defined(_WIN32) || defined(_WIN64)
+static void sleep(int secs)
+{
+ Sleep(secs * 1000);
+}
+
+#define _WSACleanup() WSACleanup()
+#else
+#define _WSACleanup()
+#endif
+
+static void print_ia_address(struct sockaddr *sa)
+{
+ char str[INET6_ADDRSTRLEN] = {" ??? "};
+
+ switch(sa->sa_family) {
+ case AF_INET:
+ inet_ntop(AF_INET, &((struct sockaddr_in *)sa)->sin_addr, str, INET6_ADDRSTRLEN);
+ printf("%d Local Address AF_INET - %s port %d\n", getpid(), str, SERVER_ID);
+ break;
+ case AF_INET6:
+ inet_ntop(AF_INET6, &((struct sockaddr_in6 *)sa)->sin6_addr, str, INET6_ADDRSTRLEN);
+ printf("%d Local Address AF_INET6 - %s flowinfo(QPN)=0x%x, port(LID)=0x%x\n",
+ getpid(), str,
+ ntohl(((struct sockaddr_in6 *)sa)->sin6_flowinfo),
+ ntohs(((struct sockaddr_in6 *)sa)->sin6_port));
+ break;
+ default:
+ printf("%d Local Address UNKOWN FAMILY - port %d\n", getpid(), SERVER_ID);
+ }
+}
+
+void
+send_msg(void *data,
+ DAT_COUNT size,
+ DAT_LMR_CONTEXT context,
+ DAT_DTO_COOKIE cookie, DAT_COMPLETION_FLAGS flags)
+{
+ DAT_LMR_TRIPLET iov;
+ DAT_EVENT event;
+ DAT_COUNT nmore;
+ DAT_RETURN status;
+ int i, ep_idx = 0, ah_idx = 0;
+ DAT_DTO_COMPLETION_EVENT_DATA *dto_event =
+ &event.event_data.dto_completion_event_data;
+
+ iov.lmr_context = context;
+ iov.virtual_address = (DAT_VADDR) (uintptr_t) data;
+ iov.segment_length = (DAT_VLEN) size;
+
+ for (i = 0; i < eps; i++) {
+ if (ud_test) {
+ /*
+ * Client and Server: ep[0] and ah[0] on single
+ * and ep[i] on multiple (-m) endpoint options.
+ */
+ if (multi_eps) {
+ ep_idx = i;
+ ah_idx = server ? 0 : i;
+ }
+ printf("%s sending on ep=%p to remote_ah: ah=%p"
+ " qpn=0x%x addr=%s\n",
+ server ? "Server" : "Client", ep[ep_idx],
+ remote_ah[ah_idx].ah,
+ remote_ah[ah_idx].qpn,
+ inet_ntoa(((struct sockaddr_in *)
+ &remote_ah[ah_idx].ia_addr)->
+ sin_addr));
+
+ /* client expects all data in on first EP */
+ status = dat_ib_post_send_ud(ep[ep_idx],
+ 1,
+ &iov,
+ &remote_ah[ah_idx],
+ cookie, flags);
+
+ } else {
+ status = dat_ep_post_send(ep[0], 1, &iov,
+ cookie, flags);
+ }
+ _OK(status, "dat_ep_post_send");
+
+ if (!(flags & DAT_COMPLETION_SUPPRESS_FLAG)) {
+ status = dat_evd_wait(dto_evd, DTO_TIMEOUT,
+ 1, &event, &nmore);
+ _OK(status, "dat_evd_wait after dat_ep_post_send");
+
+ if (event.event_number != DAT_DTO_COMPLETION_EVENT &&
+ ud_test && event.event_number != DAT_IB_DTO_EVENT) {
+ printf("unexpected event waiting post_send "
+ "completion - 0x%x\n",
+ event.event_number);
+ exit(1);
+ }
+ _OK(dto_event->status, "event status for post_send");
+ }
+ }
+}
+
+/* RC - Server only, UD - Server and Client, one per EP */
+void process_cr(int idx)
+{
+ DAT_EVENT event;
+ DAT_COUNT nmore;
+ DAT_RETURN status;
+ int pdata;
+ DAT_CR_HANDLE cr = DAT_HANDLE_NULL;
+ DAT_CONN_QUAL exp_qual = server ? SERVER_ID : CLIENT_ID;
+ DAT_CR_PARAM cr_param;
+ DAT_CR_ARRIVAL_EVENT_DATA *cr_event =
+ &event.event_data.cr_arrival_event_data;
+
+ LOGPRINTF("%s waiting for connect[%d] request\n",
+ server ? "Server" : "Client", idx);
+
+ status = dat_evd_wait(cr_evd, SERVER_TIMEOUT, 1, &event, &nmore);
+ _OK(status, "CR dat_evd_wait");
+
+ if (event.event_number != DAT_CONNECTION_REQUEST_EVENT &&
+ (ud_test && event.event_number !=
+ DAT_IB_UD_CONNECTION_REQUEST_EVENT)) {
+ printf("unexpected event,!conn req: 0x%x\n",
+ event.event_number);
+ exit(1);
+ }
+
+ if ((cr_event->conn_qual != exp_qual) ||
+ (cr_event->sp_handle.psp_handle != psp)) {
+ printf("wrong cr event data\n");
+ exit(1);
+ }
+
+ cr = cr_event->cr_handle;
+ status = dat_cr_query(cr, DAT_CSP_FIELD_ALL, &cr_param);
+ _OK(status, "dat_cr_query");
+
+ /* use private data to select EP */
+ pdata = ntoh32(*((int *)cr_param.private_data));
+
+ LOGPRINTF("%s recvd pdata=0x%x, send pdata=0x%x\n",
+ server ? "Server" : "Client", pdata,
+ *(int *)cr_param.private_data);
+
+ status = dat_cr_accept(cr, ep[pdata], 4, cr_param.private_data);
+ _OK(status, "dat_cr_accept");
+
+ printf("%s accepted CR on EP[%d]=%p\n",
+ server ? "Server" : "Client", pdata, ep[pdata]);
+}
+
+/* RC - Client and Server: 1, UD - Client: 1 per EP, Server: 2 per EP's */
+void process_conn(int idx)
+{
+ DAT_EVENT event;
+ DAT_COUNT nmore;
+ DAT_RETURN status;
+ int pdata, exp_event;
+ DAT_IB_EXTENSION_EVENT_DATA *ext_event = (DAT_IB_EXTENSION_EVENT_DATA *)
+ & event.event_extension_data[0];
+ DAT_CONNECTION_EVENT_DATA *conn_event =
+ &event.event_data.connect_event_data;
+
+ LOGPRINTF("%s waiting for connect[%d] establishment\n",
+ server ? "Server" : "Client", idx);
+
+ status = dat_evd_wait(con_evd, CONN_TIMEOUT, 1, &event, &nmore);
+ _OK(status, "CONN dat_evd_wait");
+
+ LOGPRINTF("%s got connect[%d] event 0x%x, pdata %p sz=%d\n",
+ server ? "Server" : "Client", idx,
+ event.event_number, conn_event->private_data,
+ conn_event->private_data_size);
+
+ if (ud_test)
+ exp_event = DAT_IB_UD_CONNECTION_EVENT_ESTABLISHED;
+ else
+ exp_event = DAT_CONNECTION_EVENT_ESTABLISHED;
+
+ /* Waiting on CR's or CONN_EST */
+ if (event.event_number != exp_event ||
+ (ud_test && event.event_number !=
+ DAT_IB_UD_CONNECTION_EVENT_ESTABLISHED)) {
+ printf("unexpected event, !conn established: 0x%x\n",
+ event.event_number);
+ exit(1);
+ }
+
+ /* RC or PASSIVE CONN_EST we are done */
+ if (!ud_test)
+ return;
+
+ /* store each remote_ah according to remote EP index */
+ pdata = ntoh32(*((int *)conn_event->private_data));
+ LOGPRINTF(" Client got private data=0x%x\n", pdata);
+
+ /* UD, get AH for sends.
+ * NOTE: bi-directional AH resolution results in a CONN_EST
+ * for both outbound connect and inbound CR.
+ * Use Active CONN_EST which includes server's CR
+ * pdata for remote_ah idx to send on and ignore PASSIVE CONN_EST.
+ *
+ * DAT_IB_UD_PASSIVE_REMOTE_AH == passive side CONN_EST
+ * DAT_IB_UD_REMOTE_AH == active side CONN_EST
+ */
+ if (ext_event->type == DAT_IB_UD_REMOTE_AH) {
+ remote_ah[pdata] = ext_event->remote_ah;
+ printf("remote_ah[%d]: ah=%p, qpn=0x%x "
+ "addr=%s\n",
+ pdata, remote_ah[pdata].ah,
+ remote_ah[pdata].qpn, inet_ntoa(((struct sockaddr_in *)
+ &remote_ah[pdata].
+ ia_addr)->sin_addr));
+
+ } else if (ext_event->type != DAT_IB_UD_PASSIVE_REMOTE_AH) {
+ printf("unexpected UD ext_event type: 0x%x\n", ext_event->type);
+ exit(1);
+ }
+}
+
+int connect_ep(char *hostname)
+{
+ DAT_IA_ADDRESS_PTR remote_addr = (DAT_IA_ADDRESS_PTR)&remote;
+ DAT_EP_ATTR ep_attr;
+ DAT_IA_ATTR ia_attr;
+ DAT_RETURN status;
+ DAT_REGION_DESCRIPTION region;
+ DAT_EVENT event;
+ DAT_COUNT nmore;
+ DAT_LMR_TRIPLET iov;
+ DAT_RMR_TRIPLET *r_iov;
+ DAT_DTO_COOKIE cookie;
+ DAT_CONN_QUAL conn_qual;
+ DAT_BOOLEAN in, out;
+ int i, ii, pdata, ctx;
+ DAT_PROVIDER_ATTR prov_attrs;
+ DAT_DTO_COMPLETION_EVENT_DATA *dto_event =
+ &event.event_data.dto_completion_event_data;
+
+ status = dat_ia_open(provider, 8, &async_evd, &ia);
+ _OK(status, "dat_ia_open");
+
+ memset(&prov_attrs, 0, sizeof(prov_attrs));
+ status = dat_ia_query(ia, NULL,
+ DAT_IA_FIELD_ALL, &ia_attr,
+ DAT_PROVIDER_FIELD_ALL, &prov_attrs);
+ _OK(status, "dat_ia_query");
+
+ print_ia_address(ia_attr.ia_address_ptr);
+
+ if (ucm && ud_test) {
+ printf("%d UD test over UCM provider not supported\n",
+ getpid());
+ exit(1);
+ }
+
+ /* Print provider specific attributes */
+ for (i = 0; i < prov_attrs.num_provider_specific_attr; i++) {
+ LOGPRINTF(" Provider Specific Attribute[%d] %s=%s\n",
+ i, prov_attrs.provider_specific_attr[i].name,
+ prov_attrs.provider_specific_attr[i].value);
+
+ /* check for counter support */
+ status = strcmp(prov_attrs.provider_specific_attr[i].name,
+ "DAT_COUNTERS");
+ if (!status)
+ counters_ok = 1;
+ }
+
+ /* make sure provider supports counters */
+ if ((counters) && (!counters_ok)) {
+ printf("Disable dat_query_counters:"
+ " Provider not built with counters\n");
+ counters = 0;
+ }
+
+ status = dat_pz_create(ia, &pz);
+ _OK(status, "dat_pz_create");
+
+ status = dat_evd_create(ia, eps * 2, DAT_HANDLE_NULL, DAT_EVD_CR_FLAG,
+ &cr_evd);
+ _OK(status, "dat_evd_create CR");
+ status = dat_evd_create(ia, eps * 2, DAT_HANDLE_NULL,
+ DAT_EVD_CONNECTION_FLAG, &con_evd);
+ _OK(status, "dat_evd_create CR");
+ status = dat_evd_create(ia, eps * 10, DAT_HANDLE_NULL, DAT_EVD_DTO_FLAG,
+ &dto_evd);
+ _OK(status, "dat_evd_create DTO");
+
+ memset(&ep_attr, 0, sizeof(ep_attr));
+ if (ud_test) {
+ msg_size += 40;
+ ep_attr.service_type = DAT_IB_SERVICE_TYPE_UD;
+ ep_attr.max_message_size = buf_size;
+ ep_attr.max_rdma_read_in = 0;
+ ep_attr.max_rdma_read_out = 0;
+ } else {
+ ep_attr.service_type = DAT_SERVICE_TYPE_RC;
+ ep_attr.max_rdma_size = 0x10000;
+ ep_attr.max_rdma_read_in = 4;
+ ep_attr.max_rdma_read_out = 4;
+ }
+ ep_attr.qos = 0;
+ ep_attr.recv_completion_flags = 0;
+ ep_attr.max_recv_dtos = eps * 10;
+ ep_attr.max_request_dtos = eps * 10;
+ ep_attr.max_recv_iov = 1;
+ ep_attr.max_request_iov = 1;
+ ep_attr.request_completion_flags = DAT_COMPLETION_DEFAULT_FLAG;
+ ep_attr.ep_transport_specific_count = 0;
+ ep_attr.ep_transport_specific = NULL;
+ ep_attr.ep_provider_specific_count = 0;
+ ep_attr.ep_provider_specific = NULL;
+
+ for (i = 0; i < eps; i++) {
+ status = dat_ep_create(ia, pz, dto_evd, dto_evd,
+ con_evd, &ep_attr, &ep[i]);
+ _OK(status, "dat_ep_create");
+ LOGPRINTF(" create_ep[%d]=%p\n", i, ep[i]);
+ }
+
+ for (i = 0; i < REG_MEM_COUNT * eps; i++) {
+ buf[i] = (DAT_RMR_TRIPLET *) malloc(buf_size);
+ region.for_va = buf[i];
+ status = dat_lmr_create(ia,
+ DAT_MEM_TYPE_VIRTUAL,
+ region,
+ buf_size,
+ pz,
+ DAT_MEM_PRIV_ALL_FLAG |
+ DAT_IB_MEM_PRIV_REMOTE_ATOMIC,
+ DAT_VA_TYPE_VA,
+ &lmr[i],
+ &lmr_context[i],
+ &rmr_context[i],
+ ®_size[i], ®_addr[i]);
+ _OK(status, "dat_lmr_create");
+ }
+
+ /* register atomic return buffer for original data */
+ atomic_buf = (DAT_UINT64 *) malloc(BUF_SIZE_ATOMIC);
+ region.for_va = atomic_buf;
+ status = dat_lmr_create(ia,
+ DAT_MEM_TYPE_VIRTUAL,
+ region,
+ BUF_SIZE_ATOMIC,
+ pz,
+ DAT_MEM_PRIV_ALL_FLAG |
+ DAT_IB_MEM_PRIV_REMOTE_ATOMIC,
+ DAT_VA_TYPE_VA,
+ &lmr_atomic,
+ &lmr_atomic_context,
+ &rmr_atomic_context,
+ ®_atomic_size, ®_atomic_addr);
+ _OK(status, "dat_lmr_create atomic");
+
+ for (ii = 0; ii < eps; ii++) {
+ for (i = RECV_BUF_INDEX; i < REG_MEM_COUNT; i++) {
+ int ep_idx = 0;
+ cookie.as_64 = (ii * REG_MEM_COUNT) + i;
+ iov.lmr_context = lmr_context[(ii * REG_MEM_COUNT) + i];
+ iov.virtual_address =
+ (DAT_VADDR) (uintptr_t) buf[(ii * REG_MEM_COUNT) +
+ i];
+ iov.segment_length = buf_size;
+ LOGPRINTF(" post_recv (%p) on ep[%d]=%p\n",
+ buf[(ii * REG_MEM_COUNT) + i], ii, ep[ii]);
+ /* ep[0], unless testing Server and multi EP's */
+ if (server && multi_eps) {
+ ep_idx = ii;
+ cookie.as_64 = i;
+ }
+ status = dat_ep_post_recv(ep[ep_idx],
+ 1,
+ &iov,
+ cookie,
+ DAT_COMPLETION_DEFAULT_FLAG);
+ _OK(status, "dat_ep_post_recv");
+ }
+ }
+ /* setup receive buffer to initial string to be overwritten */
+ strcpy((char *)buf[RCV_RDMA_BUF_INDEX], "blah, blah, blah\n");
+
+ /* ud can resolve_ah and connect both ways, same EP */
+ if (server || (!server && ud_test)) {
+ if (server) {
+ conn_qual = SERVER_ID;
+ strcpy((char *)buf[SND_RDMA_BUF_INDEX], "Server data");
+ } else {
+ conn_qual = CLIENT_ID;
+ strcpy((char *)buf[SND_RDMA_BUF_INDEX], "Client data");
+ }
+ status = dat_psp_create(ia,
+ conn_qual,
+ cr_evd, DAT_PSP_CONSUMER_FLAG, &psp);
+ _OK(status, "dat_psp_create");
+
+ /* Server always waits for first CR from Client */
+ if (server)
+ process_cr(0);
+
+ }
+
+ /* ud can resolve_ah and connect both ways */
+ if (!server || (server && ud_test)) {
+ struct addrinfo *target;
+
+ if (ucm)
+ goto no_resolution;
+
+ if (getaddrinfo(hostname, NULL, NULL, &target) != 0) {
+ printf("Error getting remote address.\n");
+ exit(1);
+ }
+
+ printf("Remote %s Name: %s \n",
+ server ? "Client" : "Server", hostname);
+ printf("Remote %s Net Address: %s\n",
+ server ? "Client" : "Server",
+ inet_ntoa(((struct sockaddr_in *)
+ target->ai_addr)->sin_addr));
+
+ strcpy((char *)buf[SND_RDMA_BUF_INDEX], "Client written data");
+
+ remote_addr = (DAT_IA_ADDRESS_PTR)target->ai_addr; /* IP */
+no_resolution:
+
+ /* one Client EP, multiple Server EPs, same conn_qual
+ * use private data to select EP on Server
+ */
+ for (i = 0; i < eps; i++) {
+ /* pdata selects Server EP,
+ * support both muliple Server and single EP's
+ */
+ if (multi_eps)
+ pdata = hton32(i);
+ else
+ pdata = 0; /* just use first EP */
+
+ status = dat_ep_connect(ep[0],
+ remote_addr,
+ (server ? CLIENT_ID :
+ SERVER_ID), CONN_TIMEOUT, 4,
+ (DAT_PVOID) & pdata, 0,
+ DAT_CONNECT_DEFAULT_FLAG);
+ _OK(status, "dat_ep_connect");
+ }
+
+ if (!ucm)
+ freeaddrinfo(target);
+ }
+
+ /* UD: process CR's starting with 2nd on server, 1st for client */
+ if (ud_test) {
+ for (i = (server ? 1 : 0); i < eps; i++)
+ process_cr(i);
+ }
+
+ /* RC and UD: process CONN EST events */
+ for (i = 0; i < eps; i++)
+ process_conn(i);
+
+ /* UD: CONN EST events for CONN's and CR's */
+ if (ud_test) {
+ for (i = 0; i < eps; i++)
+ process_conn(i);
+ }
+
+ printf("Connected! %d endpoints\n", eps);
+
+ /*
+ * Setup our remote memory and tell the other side about it
+ * Swap to network order.
+ */
+ r_iov = (DAT_RMR_TRIPLET *) buf[SEND_BUF_INDEX];
+ r_iov->rmr_context = hton32(rmr_context[RCV_RDMA_BUF_INDEX]);
+ r_iov->virtual_address =
+ hton64((DAT_VADDR) (uintptr_t) buf[RCV_RDMA_BUF_INDEX]);
+ r_iov->segment_length = hton32(buf_size);
+
+ printf("Send RMR message: r_key_ctx=0x%x,va=" F64x ",len=0x%x\n",
+ hton32(r_iov->rmr_context),
+ hton64(r_iov->virtual_address), hton32(r_iov->segment_length));
+
+ send_msg(buf[SEND_BUF_INDEX],
+ sizeof(DAT_RMR_TRIPLET),
+ lmr_context[SEND_BUF_INDEX],
+ cookie, DAT_COMPLETION_SUPPRESS_FLAG);
+
+ dat_ep_get_status(ep[0], NULL, &in, &out);
+ printf("EP[0] status: posted buffers: Req=%d, Rcv=%d\n", in, out);
+
+ /*
+ * Wait for their RMR
+ */
+ for (i = 0, ctx = 0; i < eps; i++, ctx++) {
+ /* expected cookie, recv buf idx in every mem pool */
+ ctx = (ctx % REG_MEM_COUNT) ? ctx : ctx + RECV_BUF_INDEX;
+ LOGPRINTF("Waiting for remote to send RMR data\n");
+
+ status = dat_evd_wait(dto_evd, DTO_TIMEOUT, 1, &event, &nmore);
+ _OK(status, "dat_evd_wait after dat_ep_post_send");
+
+ if ((event.event_number != DAT_DTO_COMPLETION_EVENT) &&
+ (ud_test && event.event_number != DAT_IB_DTO_EVENT)) {
+ printf("unexpected event waiting for RMR context "
+ "- 0x%x\n", event.event_number);
+ exit(1);
+ }
+ _OK(dto_event->status, "event status for post_recv");
+
+ /* careful when checking cookies:
+ * Client - receiving multi messages on a single EP
+ * Server - not receiving on multiple EP's
+ */
+ if (!server || (server && !multi_eps)) {
+ if (dto_event->transfered_length != msg_size ||
+ dto_event->user_cookie.as_64 != ctx) {
+ printf("unexpected event data on recv: len=%d"
+ " cookie=" F64x " expected %d/%d\n",
+ (int)dto_event->transfered_length,
+ dto_event->user_cookie.as_64,
+ msg_size, ctx);
+ exit(1);
+ }
+ /* Server - receiving one message each across many EP's */
+ } else {
+ if (dto_event->transfered_length != msg_size ||
+ dto_event->user_cookie.as_64 != RECV_BUF_INDEX) {
+ printf("unexpected event data on recv: len=%d"
+ "cookie=" F64x " expected %d/%d\n",
+ (int)dto_event->transfered_length,
+ dto_event->user_cookie.as_64,
+ msg_size, RECV_BUF_INDEX);
+ exit(1);
+ }
+ }
+
+ /* swap RMR,address info to host order */
+ if (!server || (server && !multi_eps))
+ r_iov = (DAT_RMR_TRIPLET *) buf[ctx];
+ else
+ r_iov =
+ (DAT_RMR_TRIPLET *) buf[(i * REG_MEM_COUNT) +
+ RECV_BUF_INDEX];
+
+ if (ud_test)
+ r_iov = (DAT_RMR_TRIPLET *) ((char *)r_iov + 40);
+
+ r_iov->rmr_context = ntoh32(r_iov->rmr_context);
+ r_iov->virtual_address = ntoh64(r_iov->virtual_address);
+ r_iov->segment_length = ntoh32(r_iov->segment_length);
+
+ printf("Recv RMR message: r_iov(%p):"
+ " r_key_ctx=%x,va=" F64x ",len=0x%x on EP=%p\n",
+ r_iov, r_iov->rmr_context,
+ r_iov->virtual_address,
+ r_iov->segment_length, dto_event->ep_handle);
+ }
+ return (0);
+}
+
+int disconnect_ep(void)
+{
+ DAT_RETURN status;
+ DAT_EVENT event;
+ DAT_COUNT nmore;
+ int i;
+
+ if (counters) { /* examples of query and print */
+ int ii;
+ DAT_UINT64 ia_cntrs[DCNT_IA_ALL_COUNTERS];
+
+ dat_query_counters(ia, DCNT_IA_ALL_COUNTERS, ia_cntrs, 0);
+ printf(" IA Cntrs:");
+ for (ii = 0; ii < DCNT_IA_ALL_COUNTERS; ii++)
+ printf(" " F64u "", ia_cntrs[ii]);
+ printf("\n");
+ dat_print_counters(ia, DCNT_IA_ALL_COUNTERS, 0);
+ }
+
+ if (!ud_test) {
+ status = dat_ep_disconnect(ep[0], DAT_CLOSE_DEFAULT);
+ _OK2(status, "dat_ep_disconnect");
+
+ status = dat_evd_wait(con_evd, DAT_TIMEOUT_INFINITE, 1,
+ &event, &nmore);
+ _OK(status, "dat_evd_wait");
+ }
+ if (psp) {
+ status = dat_psp_free(psp);
+ _OK2(status, "dat_psp_free");
+ }
+ for (i = 0; i < REG_MEM_COUNT * eps; i++) {
+ status = dat_lmr_free(lmr[i]);
+ _OK2(status, "dat_lmr_free");
+ }
+ if (lmr_atomic) {
+ status = dat_lmr_free(lmr_atomic);
+ _OK2(status, "dat_lmr_free_atomic");
+ }
+ for (i = 0; i < eps; i++) {
+ if (counters) { /* examples of query and print */
+ int ii;
+ DAT_UINT64 ep_cntrs[DCNT_EP_ALL_COUNTERS];
+
+ dat_query_counters(ep[i], DCNT_EP_ALL_COUNTERS,
+ ep_cntrs, 0);
+ printf(" EP[%d] Cntrs:", i);
+ for (ii = 0; ii < DCNT_EP_ALL_COUNTERS; ii++)
+ printf(" " F64u "", ep_cntrs[ii]);
+ printf("\n");
+ dat_print_counters(ep[i], DCNT_EP_ALL_COUNTERS, 0);
+ }
+ status = dat_ep_free(ep[i]);
+ _OK2(status, "dat_ep_free");
+ }
+ if (counters) { /* examples of query and print */
+ int ii;
+ DAT_UINT64 evd_cntrs[DCNT_EVD_ALL_COUNTERS];
+
+ dat_query_counters(dto_evd, DCNT_EVD_ALL_COUNTERS,
+ evd_cntrs, 0);
+ printf(" DTO_EVD Cntrs:");
+ for (ii = 0; ii < DCNT_EVD_ALL_COUNTERS; ii++)
+ printf(" " F64u "", evd_cntrs[ii]);
+ printf("\n");
+ dat_print_counters(dto_evd, DCNT_EVD_ALL_COUNTERS, 0);
+
+ dat_query_counters(con_evd, DCNT_EVD_ALL_COUNTERS,
+ evd_cntrs, 0);
+ printf(" CONN_EVD Cntrs:");
+ for (ii = 0; ii < DCNT_EVD_ALL_COUNTERS; ii++)
+ printf(" " F64u "", evd_cntrs[ii]);
+ printf("\n");
+ dat_print_counters(con_evd, DCNT_EVD_ALL_COUNTERS, 0);
+
+ dat_query_counters(cr_evd, DCNT_EVD_ALL_COUNTERS, evd_cntrs, 0);
+ printf(" CR_EVD Cntrs:");
+ for (ii = 0; ii < DCNT_EVD_ALL_COUNTERS; ii++)
+ printf(" " F64u "", evd_cntrs[ii]);
+ printf("\n");
+ dat_print_counters(cr_evd, DCNT_EVD_ALL_COUNTERS, 0);
+ }
+ status = dat_evd_free(dto_evd);
+ _OK2(status, "dat_evd_free DTO");
+
+ status = dat_evd_free(con_evd);
+ _OK2(status, "dat_evd_free CON");
+
+ status = dat_evd_free(cr_evd);
+ _OK2(status, "dat_evd_free CR");
+
+ status = dat_pz_free(pz);
+ _OK2(status, "dat_pz_free");
+
+ status = dat_ia_close(ia, DAT_CLOSE_DEFAULT);
+ _OK2(status, "dat_ia_close");
+
+ return (0);
+}
+
+int do_immediate()
+{
+ DAT_EVENT event;
+ DAT_COUNT nmore;
+ DAT_LMR_TRIPLET iov;
+ DAT_RMR_TRIPLET r_iov;
+ DAT_DTO_COOKIE cookie;
+ DAT_RETURN status;
+ DAT_UINT32 immed_data;
+ DAT_UINT32 immed_data_recv = 0;
+ DAT_DTO_COMPLETION_EVENT_DATA *dto_event =
+ &event.event_data.dto_completion_event_data;
+ DAT_IB_EXTENSION_EVENT_DATA *ext_event =
+ (DAT_IB_EXTENSION_EVENT_DATA *) & event.event_extension_data[0];
+
+ printf("\nDoing RDMA WRITE IMMEDIATE DATA\n");
+
+ if (server) {
+ immed_data = 0x1111;
+ } else {
+ immed_data = 0x7777;
+ }
+
+ cookie.as_64 = 0x5555;
+
+ /* RMR info already swapped back to host order in connect_ep */
+ r_iov = *buf[RECV_BUF_INDEX];
+
+ iov.lmr_context = lmr_context[SND_RDMA_BUF_INDEX];
+ iov.virtual_address = (DAT_VADDR) (uintptr_t) buf[SND_RDMA_BUF_INDEX];
+ iov.segment_length = buf_size;
+
+ cookie.as_64 = 0x9999;
+
+ status = dat_ib_post_rdma_write_immed(ep[0], // ep_handle
+ 1, // segments
+ &iov, // LMR
+ cookie, // user_cookie
+ &r_iov, // RMR
+ immed_data,
+ DAT_COMPLETION_DEFAULT_FLAG);
+ _OK(status, "dat_ib_post_rdma_write_immed");
+
+ /*
+ * Collect first event, write completion or inbound recv with immed
+ */
+ status = dat_evd_wait(dto_evd, DTO_TIMEOUT, 1, &event, &nmore);
+ _OK(status, "dat_evd_wait after dat_ib_post_rdma_write");
+ if (event.event_number != DAT_IB_DTO_EVENT) {
+ printf("unexpected event #0x%x waiting for WR-IMMED #0x%x\n",
+ event.event_number, DAT_IB_DTO_EVENT);
+ exit(1);
+ }
+
+ if (nmore)
+ printf("%s() nmore %d\n", __FUNCTION__, nmore);
+ _OK(dto_event->status, "DTO event status");
+ if (ext_event->type == DAT_IB_RDMA_WRITE_IMMED) {
+ if ((dto_event->transfered_length != buf_size) ||
+ (dto_event->user_cookie.as_64 != 0x9999)) {
+ printf
+ ("unexpected event data for rdma_write_immed: len=%d "
+ "cookie=0x%x\n", (int)dto_event->transfered_length,
+ (int)dto_event->user_cookie.as_64);
+ exit(1);
+ }
+ } else if (ext_event->type == DAT_IB_RDMA_WRITE_IMMED_DATA) {
+ if ((dto_event->transfered_length != buf_size) ||
+ (dto_event->user_cookie.as_64 != RECV_BUF_INDEX + 1)) {
+ printf
+ ("unexpected event data of immediate write: len=%d "
+ "cookie=" F64x " expected %d/%d\n",
+ (int)dto_event->transfered_length,
+ dto_event->user_cookie.as_64, (int)sizeof(int),
+ RECV_BUF_INDEX + 1);
+ exit(1);
+ }
+
+ /* get immediate data from event */
+ immed_data_recv = ext_event->val.immed.data;
+ } else {
+ printf("unexpected extension type for event - 0x%x, 0x%x\n",
+ event.event_number, ext_event->type);
+ exit(1);
+ }
+
+ /*
+ * Collect second event, write completion or inbound recv with immed
+ */
+ status = dat_evd_wait(dto_evd, DTO_TIMEOUT, 1, &event, &nmore);
+ _OK(status, "dat_evd_wait after dat_ib_post_rdma_write");
+ if (event.event_number != DAT_IB_DTO_EVENT) {
+ printf("unexpected event # waiting for WR-IMMED - 0x%x\n",
+ event.event_number);
+ exit(1);
+ }
+
+ _OK(dto_event->status, "event status");
+ if (ext_event->type == DAT_IB_RDMA_WRITE_IMMED) {
+ if ((dto_event->transfered_length != buf_size) ||
+ (dto_event->user_cookie.as_64 != 0x9999)) {
+ printf
+ ("unexpected event data for rdma_write_immed: len=%d "
+ "cookie=0x%x\n", (int)dto_event->transfered_length,
+ (int)dto_event->user_cookie.as_64);
+ exit(1);
+ }
+ } else if (ext_event->type == DAT_IB_RDMA_WRITE_IMMED_DATA) {
+ if ((dto_event->transfered_length != buf_size) ||
+ (dto_event->user_cookie.as_64 != RECV_BUF_INDEX + 1)) {
+ printf
+ ("unexpected event data of immediate write: len=%d "
+ "cookie=" F64x " expected %d/%d\n",
+ (int)dto_event->transfered_length,
+ dto_event->user_cookie.as_64, (int)sizeof(int),
+ RECV_BUF_INDEX + 1);
+ exit(1);
+ }
+
+ /* get immediate data from event */
+ immed_data_recv = ext_event->val.immed.data;
+ } else {
+ printf("unexpected extension type for event - 0x%x, 0x%x\n",
+ event.event_number, ext_event->type);
+ exit(1);
+ }
+
+ if ((server) && (immed_data_recv != 0x7777)) {
+ printf("ERROR: Server: unexpected imm_data_recv 0x%x/0x%x\n",
+ 0x7777, immed_data_recv);
+ exit(1);
+ } else if ((!server) && (immed_data_recv != 0x1111)) {
+ printf("ERROR: Client: unexpected imm_data_recv 0x%x/0x%x\n",
+ 0x1111, immed_data_recv);
+ exit(1);
+ }
+
+ if (server)
+ printf("Server received immed_data=0x%x\n", immed_data_recv);
+ else
+ printf("Client received immed_data=0x%x\n", immed_data_recv);
+
+ printf("rdma buffer %p contains: %s\n",
+ buf[RCV_RDMA_BUF_INDEX], (char *)buf[RCV_RDMA_BUF_INDEX]);
+
+ printf("\n RDMA_WRITE_WITH_IMMEDIATE_DATA test - PASSED\n");
+ return (0);
+}
+
+int do_cmp_swap()
+{
+ DAT_DTO_COOKIE cookie;
+ DAT_RETURN status;
+ DAT_EVENT event;
+ DAT_COUNT nmore;
+ DAT_LMR_TRIPLET l_iov;
+ DAT_RMR_TRIPLET r_iov;
+ volatile DAT_UINT64 *target = (DAT_UINT64 *) buf[RCV_RDMA_BUF_INDEX];
+ DAT_DTO_COMPLETION_EVENT_DATA *dto_event =
+ &event.event_data.dto_completion_event_data;
+ DAT_IB_EXTENSION_EVENT_DATA *ext_event =
+ (DAT_IB_EXTENSION_EVENT_DATA *) & event.event_extension_data[0];
+
+ printf("\nDoing CMP and SWAP\n");
+
+ /* RMR info already swapped back to host order in connect_ep */
+ r_iov = *buf[RECV_BUF_INDEX];
+
+ l_iov.lmr_context = lmr_atomic_context;
+ l_iov.virtual_address = (DAT_UINT64) (uintptr_t) atomic_buf;
+ l_iov.segment_length = BUF_SIZE_ATOMIC;
+
+ cookie.as_64 = 3333;
+
+ if (server) {
+ *target = 0x12345;
+ sleep(1);
+ /* Server does not compare and should not swap */
+ printf("dtx svr - starting cmp_swap\n");
+ status = dat_ib_post_cmp_and_swap(ep[0],
+ (DAT_UINT64) 0x654321,
+ (DAT_UINT64) 0x6789A,
+ &l_iov,
+ cookie,
+ &r_iov,
+ DAT_COMPLETION_DEFAULT_FLAG);
+ printf("dtx svr - done cmp_swap, chk status\n");
+ } else {
+ *target = 0x54321;
+ sleep(1);
+ printf("dtx cli - starting cmp_swap\n");
+ /* Client does compare and should swap */
+ status = dat_ib_post_cmp_and_swap(ep[0],
+ (DAT_UINT64) 0x12345,
+ (DAT_UINT64) 0x98765,
+ &l_iov,
+ cookie,
+ &r_iov,
+ DAT_COMPLETION_DEFAULT_FLAG);
+ printf("dtx cli - done cmp_swap, chk status\n");
+ }
+ _OK(status, "dat_ib_post_cmp_and_swap");
+ status = dat_evd_wait(dto_evd, DTO_TIMEOUT, 1, &event, &nmore);
+ _OK(status, "dat_evd_wait for compare and swap");
+ if (event.event_number != DAT_IB_DTO_EVENT) {
+ printf("unexpected event after post_cmp_and_swap: 0x%x\n",
+ event.event_number);
+ exit(1);
+ }
+
+ _OK(dto_event->status, "event status for CMP and SWAP");
+ if (ext_event->type != DAT_IB_CMP_AND_SWAP) {
+ printf("unexpected event data of cmp_swap: type=%d cookie=%d "
+ "original " F64x "\n",
+ (int)ext_event->type,
+ (int)dto_event->user_cookie.as_64, *atomic_buf);
+ exit(1);
+ }
+
+ sleep(2); /* wait for other side to complete swap */
+
+ if (server) {
+ printf("Server got original data = " F64x ", expected "
+ "0x54321\n", *atomic_buf);
+ printf("Client final result (on Server) = " F64x ", expected "
+ "0x98765\n", *target);
+
+ if (*atomic_buf != 0x54321 || *target != 0x98765) {
+ printf("ERROR: Server CMP_SWAP\n");
+ exit(1);
+ }
+ } else {
+ printf("Client got original data = " F64x ", expected "
+ "0x12345\n", *atomic_buf);
+ printf("Server final result (on Client) = 0x" F64x ", expected "
+ "0x54321\n", *target);
+
+ if (*atomic_buf != 0x12345 || *target != 0x54321) {
+ printf("ERROR: Client CMP_SWAP\n");
+ exit(1);
+ }
+ }
+ printf("\n CMP_SWAP test - PASSED\n");
+ return (0);
+}
+
+int do_fetch_add()
+{
+ DAT_DTO_COOKIE cookie;
+ DAT_RETURN status;
+ DAT_EVENT event;
+ DAT_COUNT nmore;
+ DAT_LMR_TRIPLET l_iov;
+ DAT_RMR_TRIPLET r_iov;
+ volatile DAT_UINT64 *target = (DAT_UINT64 *) buf[RCV_RDMA_BUF_INDEX];
+ DAT_DTO_COMPLETION_EVENT_DATA *dto_event =
+ &event.event_data.dto_completion_event_data;
+ DAT_IB_EXTENSION_EVENT_DATA *ext_event =
+ (DAT_IB_EXTENSION_EVENT_DATA *) & event.event_extension_data[0];
+
+ printf("\nDoing FETCH and ADD\n");
+
+ /* RMR info already swapped back to host order in connect_ep */
+ r_iov = *buf[RECV_BUF_INDEX];
+
+ l_iov.lmr_context = lmr_atomic_context;
+ l_iov.virtual_address = (DAT_UINT64) (uintptr_t) atomic_buf;
+ l_iov.segment_length = BUF_SIZE_ATOMIC;
+
+ cookie.as_64 = 0x7777;
+ if (server) {
+ /* Wait for Client to finish cmp_swap */
+ while (*target != 0x98765)
+ sleep(1);
+ *target = 0x10;
+ sleep(1);
+ status = dat_ib_post_fetch_and_add(ep[0],
+ (DAT_UINT64) 0x100,
+ &l_iov,
+ cookie,
+ &r_iov,
+ DAT_COMPLETION_DEFAULT_FLAG);
+ } else {
+ /* Wait for Server, no swap so nothing to check */
+ *target = 0x100;
+ sleep(1);
+ status = dat_ib_post_fetch_and_add(ep[0],
+ (DAT_UINT64) 0x10,
+ &l_iov,
+ cookie,
+ &r_iov,
+ DAT_COMPLETION_DEFAULT_FLAG);
+ }
+ _OK(status, "dat_ib_post_fetch_and_add");
+ status = dat_evd_wait(dto_evd, DTO_TIMEOUT, 1, &event, &nmore);
+ _OK(status, "dat_evd_wait for fetch and add");
+ if (event.event_number != DAT_IB_DTO_EVENT) {
+ printf("unexpected event after post_fetch_and_add: 0x%x\n",
+ event.event_number);
+ exit(1);
+ }
+
+ _OK(dto_event->status, "event status for FETCH and ADD");
+ if (ext_event->type != DAT_IB_FETCH_AND_ADD) {
+ printf("unexpected event data of fetch and add : type=%d "
+ "cookie=%d original%d\n",
+ (int)ext_event->type,
+ (int)dto_event->user_cookie.as_64, (int)*atomic_buf);
+ exit(1);
+ }
+
+ if (server) {
+ printf("Client original data (on Server) = " F64x ", expected "
+ "0x100\n", *atomic_buf);
+ } else {
+ printf("Server original data (on Client) = " F64x ", expected "
+ "0x10\n", *atomic_buf);
+ }
+
+ sleep(1);
+
+ if (server) {
+ status = dat_ib_post_fetch_and_add(ep[0],
+ (DAT_UINT64) 0x100,
+ &l_iov,
+ cookie,
+ &r_iov,
+ DAT_COMPLETION_DEFAULT_FLAG);
+ } else {
+ status = dat_ib_post_fetch_and_add(ep[0],
+ (DAT_UINT64) 0x10,
+ &l_iov,
+ cookie,
+ &r_iov,
+ DAT_COMPLETION_DEFAULT_FLAG);
+ }
+
+ status = dat_evd_wait(dto_evd, DTO_TIMEOUT, 1, &event, &nmore);
+ _OK(status, "dat_evd_wait for second fetch and add");
+ if (event.event_number != DAT_IB_DTO_EVENT) {
+ printf("unexpected event after second post_fetch_and_add: "
+ "0x%x\n", event.event_number);
+ exit(1);
+ }
+
+ _OK(dto_event->status, "event status for second FETCH and ADD");
+ if (ext_event->type != DAT_IB_FETCH_AND_ADD) {
+ printf("unexpected event data of second fetch and add : "
+ "type=%d cookie=%d original%p\n",
+ (int)ext_event->type,
+ (int)dto_event->user_cookie.as_64, atomic_buf);
+ exit(1);
+ }
+
+ sleep(1); /* wait for other side to complete fetch_add */
+
+ if (server) {
+ printf("Server got original data = " F64x ", expected "
+ "0x200\n", *atomic_buf);
+ printf("Client final result (on Server) = " F64x ", expected "
+ "0x30\n", *target);
+
+ if (*atomic_buf != 0x200 || *target != 0x30) {
+ printf("ERROR: Server FETCH_ADD\n");
+ exit(1);
+ }
+ } else {
+ printf("Server side original data = " F64x ", expected "
+ "0x20\n", *atomic_buf);
+ printf("Server final result (on Client) = " F64x ", expected "
+ "0x300\n", *target);
+
+ if (*atomic_buf != 0x20 || *target != 0x300) {
+ printf("ERROR: Server FETCH_ADD\n");
+ exit(1);
+ }
+ }
+ printf("\n FETCH_ADD test - PASSED\n");
+ return (0);
+}
+
+int main(int argc, char **argv)
+{
+ int rc;
+
+ /* parse arguments */
+ while ((rc = getopt(argc, argv, "csvumpU:h:b:P:q:l:")) != -1) {
+ switch (rc) {
+ case 'u':
+ ud_test = 1;
+ eps = MAX_EP_COUNT / 2;
+ break;
+ case 'm':
+ multi_eps = 1;
+ break;
+ case 'c':
+ server = 0;
+ break;
+ case 's':
+ server = 1;
+ break;
+ case 'p':
+ counters = 1;
+ break;
+ case 'h':
+ remote_host = 1;
+ strcpy(hostname, optarg);
+ break;
+ case 'b':
+ buf_size = atoi(optarg);
+ break;
+ case 'U':
+ ud_test = 1;
+ eps = MIN(atoi(optarg), MAX_EP_COUNT);
+ break;
+ case 'P':
+ strcpy(provider, optarg);
+ break;
+ case 'v':
+ verbose = 1;
+ break;
+ case 'q':
+ /* map UCM qpn into AF_INET6 sin6_flowinfo */
+ remote.sin6_family = AF_INET6;
+ remote.sin6_flowinfo = htonl(strtol(optarg,NULL,0));
+ ucm = 1;
+ server = 0;
+ break;
+ case 'l':
+ /* map UCM lid into AF_INET6 sin6_port */
+ remote.sin6_family = AF_INET6;
+ remote.sin6_port = htons(strtol(optarg,NULL,0));
+ ucm = 1;
+ server = 0;
+ break;
+ default:
+ print_usage();
+ exit(-12);
+ }
+ }
+
+#if defined(_WIN32) || defined(_WIN64)
+ {
+ WSADATA wsaData;
+ int i;
+
+ i = WSAStartup(MAKEWORD(2, 2), &wsaData);
+ if (i != 0) {
+ printf("%s WSAStartup(2.2) fail? (0x%x)\n", argv[0], i);
+ fflush(stdout);
+ exit(1);
+ }
+ }
+#endif
+ /* for non UD tests, -h is always client */
+ if (remote_host && !ud_test)
+ server = 0;
+
+ if (!server) {
+ printf("\nRunning as Client - %s %s %d endpoint(s)\n",
+ provider, ud_test ? "UD test" : "", eps);
+ } else {
+ printf("\nRunning as Server - %s %s %d endpoint(s)\n",
+ provider, ud_test ? "UD test" : "", eps);
+ }
+
+ /*
+ * connect
+ */
+ if (connect_ep(hostname)) {
+ _WSACleanup();
+ exit(1);
+ }
+ if (ud_test)
+ goto bail;
+
+ if (do_immediate()) {
+ _WSACleanup();
+ exit(1);
+ }
+ if (do_cmp_swap()) {
+ _WSACleanup();
+ exit(1);
+ }
+ if (do_fetch_add()) {
+ _WSACleanup();
+ exit(1);
+ }
+ bail:
+ rc = disconnect_ep();
+ _WSACleanup();
+
+ if (!rc)
+ printf("\n IB extension test - %s test PASSED\n\n",
+ ud_test ? "UD" : "immed/atomic");
+ return rc;
+}
-!if $(FREEBUILD)
-TARGETNAME = dtest2
-!else
-TARGETNAME = dtest2d
-!endif
-
-TARGETPATH = ..\..\..\..\..\..\bin\user\obj$(BUILD_ALT_DIR)
-TARGETTYPE = PROGRAM
-UMTYPE = console
-USE_MSVCRT = 1
-
-SOURCES = \
- dtest.rc \
- dtest.c
-
-INCLUDES = ..\..\..\..\dat\include;..\..\..\..\..\..\inc;\
- ..\..\..\..\..\..\inc\user;\
- ..\..\..\..\..\..\inc\user\linux;
-
-RCOPTIONS=/I..\..\..\..\..\..\inc;
-
-# Set defines particular to the driver.
-USER_C_FLAGS = $(USER_C_FLAGS) /DDAT_EXTENSIONS /DFD_SETSIZE=1024
-
-!if $(FREEBUILD)
-DATLIB = dat2.lib
-!else
-DATLIB = dat2d.lib
-!endif
-
-TARGETLIBS = $(TARGETPATH)\*\$(DATLIB) $(SDK_LIB_PATH)\ws2_32.lib
-
-# XXX do this ASAP - MSC_WARNING_LEVEL= /W3
-MSC_WARNING_LEVEL = /W1
+!if $(FREEBUILD)\r
+TARGETNAME = dtest2\r
+!else\r
+TARGETNAME = dtest2d\r
+!endif\r
+\r
+TARGETPATH = ..\..\..\..\..\..\bin\user\obj$(BUILD_ALT_DIR)\r
+TARGETTYPE = PROGRAM\r
+UMTYPE = console\r
+USE_MSVCRT = 1\r
+\r
+SOURCES = \\r
+ dtest.rc \\r
+ dtest.c\r
+ \r
+INCLUDES = ..\..\..\..\dat\include;..\..\..\..\..\..\inc;\\r
+ ..\..\..\..\..\..\inc\user;\\r
+ ..\..\..\..\..\..\inc\user\linux;\r
+\r
+RCOPTIONS=/I..\..\..\..\..\..\inc;\r
+\r
+# Set defines particular to the driver.\r
+USER_C_FLAGS = $(USER_C_FLAGS) /DDAT_EXTENSIONS /DFD_SETSIZE=1024\r
+\r
+!if $(FREEBUILD)\r
+DATLIB = dat2.lib\r
+!else\r
+DATLIB = dat2d.lib\r
+!endif\r
+\r
+TARGETLIBS = $(TARGETPATH)\*\$(DATLIB) $(SDK_LIB_PATH)\ws2_32.lib\r
+\r
+# XXX do this ASAP - MSC_WARNING_LEVEL= /W3\r
+MSC_WARNING_LEVEL = /W1\r
-#include "..\..\..\..\..\..\etc\user\inet.c"
-#include "..\..\..\..\..\..\etc\user\gtod.c"
-#include "..\..\dtest.c"
-
+#include "..\..\..\..\..\..\etc\user\inet.c"\r
+#include "..\..\..\..\..\..\etc\user\gtod.c"\r
+#include "..\..\dtest.c"\r
+\r
-!if $(FREEBUILD)
-TARGETNAME = dtestcm
-!else
-TARGETNAME = dtestcmd
-!endif
-
-TARGETPATH = ..\..\..\..\..\..\bin\user\obj$(BUILD_ALT_DIR)
-TARGETTYPE = PROGRAM
-UMTYPE = console
-USE_MSVCRT = 1
-
-SOURCES = \
- dtestcm.rc \
- dtestcm.c
-
-INCLUDES = ..\..\..\..\dat\include;..\..\..\..\..\..\inc;\
- ..\..\..\..\..\..\inc\user;\
- ..\..\..\..\..\..\inc\user\linux;
-
-RCOPTIONS=/I..\..\..\..\..\..\inc;
-
-# Set defines particular to the driver.
-USER_C_FLAGS = $(USER_C_FLAGS) /DDAT_EXTENSIONS /DFD_SETSIZE=1024
-
-!if $(FREEBUILD)
-DATLIB = dat2.lib
-!else
-DATLIB = dat2d.lib
-!endif
-
-TARGETLIBS = $(TARGETPATH)\*\$(DATLIB) $(SDK_LIB_PATH)\ws2_32.lib
-
-# XXX do this ASAP - MSC_WARNING_LEVEL= /W3
-MSC_WARNING_LEVEL = /W1
+!if $(FREEBUILD)\r
+TARGETNAME = dtestcm\r
+!else\r
+TARGETNAME = dtestcmd\r
+!endif\r
+\r
+TARGETPATH = ..\..\..\..\..\..\bin\user\obj$(BUILD_ALT_DIR)\r
+TARGETTYPE = PROGRAM\r
+UMTYPE = console\r
+USE_MSVCRT = 1\r
+\r
+SOURCES = \\r
+ dtestcm.rc \\r
+ dtestcm.c\r
+ \r
+INCLUDES = ..\..\..\..\dat\include;..\..\..\..\..\..\inc;\\r
+ ..\..\..\..\..\..\inc\user;\\r
+ ..\..\..\..\..\..\inc\user\linux;\r
+\r
+RCOPTIONS=/I..\..\..\..\..\..\inc;\r
+\r
+# Set defines particular to the driver.\r
+USER_C_FLAGS = $(USER_C_FLAGS) /DDAT_EXTENSIONS /DFD_SETSIZE=1024\r
+\r
+!if $(FREEBUILD)\r
+DATLIB = dat2.lib\r
+!else\r
+DATLIB = dat2d.lib\r
+!endif\r
+\r
+TARGETLIBS = $(TARGETPATH)\*\$(DATLIB) $(SDK_LIB_PATH)\ws2_32.lib\r
+\r
+# XXX do this ASAP - MSC_WARNING_LEVEL= /W3\r
+MSC_WARNING_LEVEL = /W1\r
-#include "..\..\..\..\..\..\etc\user\inet.c"
-#include "..\..\dtestx.c"
+#include "..\..\..\..\..\..\etc\user\inet.c"\r
+#include "..\..\dtestx.c"\r
projects / ~shefty / rdma-win.git / commitdiff