--- /dev/null
+Sean Hefty <sean.hefty@intel.com>
--- /dev/null
+This software is available to you under a choice of one of two
+licenses. You may choose to be licensed under the terms of the the
+OpenIB.org BSD license or the GNU General Public License (GPL) Version
+2, both included below.
+
+Copyright (c) 2009 Intel Corporation. All rights reserved.
+
+==================================================================
+
+ OpenIB.org BSD license
+
+Redistribution and use in source and binary forms, with or without
+modification, are permitted provided that the following conditions
+are met:
+
+ * Redistributions of source code must retain the above copyright
+ notice, this list of conditions and the following disclaimer.
+
+ * Redistributions in binary form must reproduce the above
+ copyright notice, this list of conditions and the following
+ disclaimer in the documentation and/or other materials provided
+ with the distribution.
+
+THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+"AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS
+FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE
+COPYRIGHT OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT,
+INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING,
+BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES;
+LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER
+CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT
+LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN
+ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
+POSSIBILITY OF SUCH DAMAGE.
+
+==================================================================
+
+ GNU GENERAL PUBLIC LICENSE
+ Version 2, June 1991
+
+ Copyright (C) 1989, 1991 Free Software Foundation, Inc.
+ 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
+ Everyone is permitted to copy and distribute verbatim copies
+ of this license document, but changing it is not allowed.
+
+ Preamble
+
+ The licenses for most software are designed to take away your
+freedom to share and change it. By contrast, the GNU General Public
+License is intended to guarantee your freedom to share and change free
+software--to make sure the software is free for all its users. This
+General Public License applies to most of the Free Software
+Foundation's software and to any other program whose authors commit to
+using it. (Some other Free Software Foundation software is covered by
+the GNU Library General Public License instead.) You can apply it to
+your programs, too.
+
+ When we speak of free software, we are referring to freedom, not
+price. Our General Public Licenses are designed to make sure that you
+have the freedom to distribute copies of free software (and charge for
+this service if you wish), that you receive source code or can get it
+if you want it, that you can change the software or use pieces of it
+in new free programs; and that you know you can do these things.
+
+ To protect your rights, we need to make restrictions that forbid
+anyone to deny you these rights or to ask you to surrender the rights.
+These restrictions translate to certain responsibilities for you if you
+distribute copies of the software, or if you modify it.
+
+ For example, if you distribute copies of such a program, whether
+gratis or for a fee, you must give the recipients all the rights that
+you have. You must make sure that they, too, receive or can get the
+source code. And you must show them these terms so they know their
+rights.
+
+ We protect your rights with two steps: (1) copyright the software, and
+(2) offer you this license which gives you legal permission to copy,
+distribute and/or modify the software.
+
+ Also, for each author's protection and ours, we want to make certain
+that everyone understands that there is no warranty for this free
+software. If the software is modified by someone else and passed on, we
+want its recipients to know that what they have is not the original, so
+that any problems introduced by others will not reflect on the original
+authors' reputations.
+
+ Finally, any free program is threatened constantly by software
+patents. We wish to avoid the danger that redistributors of a free
+program will individually obtain patent licenses, in effect making the
+program proprietary. To prevent this, we have made it clear that any
+patent must be licensed for everyone's free use or not licensed at all.
+
+ The precise terms and conditions for copying, distribution and
+modification follow.
+
+ GNU GENERAL PUBLIC LICENSE
+ TERMS AND CONDITIONS FOR COPYING, DISTRIBUTION AND MODIFICATION
+
+ 0. This License applies to any program or other work which contains
+a notice placed by the copyright holder saying it may be distributed
+under the terms of this General Public License. The "Program", below,
+refers to any such program or work, and a "work based on the Program"
+means either the Program or any derivative work under copyright law:
+that is to say, a work containing the Program or a portion of it,
+either verbatim or with modifications and/or translated into another
+language. (Hereinafter, translation is included without limitation in
+the term "modification".) Each licensee is addressed as "you".
+
+Activities other than copying, distribution and modification are not
+covered by this License; they are outside its scope. The act of
+running the Program is not restricted, and the output from the Program
+is covered only if its contents constitute a work based on the
+Program (independent of having been made by running the Program).
+Whether that is true depends on what the Program does.
+
+ 1. You may copy and distribute verbatim copies of the Program's
+source code as you receive it, in any medium, provided that you
+conspicuously and appropriately publish on each copy an appropriate
+copyright notice and disclaimer of warranty; keep intact all the
+notices that refer to this License and to the absence of any warranty;
+and give any other recipients of the Program a copy of this License
+along with the Program.
+
+You may charge a fee for the physical act of transferring a copy, and
+you may at your option offer warranty protection in exchange for a fee.
+
+ 2. You may modify your copy or copies of the Program or any portion
+of it, thus forming a work based on the Program, and copy and
+distribute such modifications or work under the terms of Section 1
+above, provided that you also meet all of these conditions:
+
+ a) You must cause the modified files to carry prominent notices
+ stating that you changed the files and the date of any change.
+
+ b) You must cause any work that you distribute or publish, that in
+ whole or in part contains or is derived from the Program or any
+ part thereof, to be licensed as a whole at no charge to all third
+ parties under the terms of this License.
+
+ c) If the modified program normally reads commands interactively
+ when run, you must cause it, when started running for such
+ interactive use in the most ordinary way, to print or display an
+ announcement including an appropriate copyright notice and a
+ notice that there is no warranty (or else, saying that you provide
+ a warranty) and that users may redistribute the program under
+ these conditions, and telling the user how to view a copy of this
+ License. (Exception: if the Program itself is interactive but
+ does not normally print such an announcement, your work based on
+ the Program is not required to print an announcement.)
+
+These requirements apply to the modified work as a whole. If
+identifiable sections of that work are not derived from the Program,
+and can be reasonably considered independent and separate works in
+themselves, then this License, and its terms, do not apply to those
+sections when you distribute them as separate works. But when you
+distribute the same sections as part of a whole which is a work based
+on the Program, the distribution of the whole must be on the terms of
+this License, whose permissions for other licensees extend to the
+entire whole, and thus to each and every part regardless of who wrote it.
+
+Thus, it is not the intent of this section to claim rights or contest
+your rights to work written entirely by you; rather, the intent is to
+exercise the right to control the distribution of derivative or
+collective works based on the Program.
+
+In addition, mere aggregation of another work not based on the Program
+with the Program (or with a work based on the Program) on a volume of
+a storage or distribution medium does not bring the other work under
+the scope of this License.
+
+ 3. You may copy and distribute the Program (or a work based on it,
+under Section 2) in object code or executable form under the terms of
+Sections 1 and 2 above provided that you also do one of the following:
+
+ a) Accompany it with the complete corresponding machine-readable
+ source code, which must be distributed under the terms of Sections
+ 1 and 2 above on a medium customarily used for software interchange; or,
+
+ b) Accompany it with a written offer, valid for at least three
+ years, to give any third party, for a charge no more than your
+ cost of physically performing source distribution, a complete
+ machine-readable copy of the corresponding source code, to be
+ distributed under the terms of Sections 1 and 2 above on a medium
+ customarily used for software interchange; or,
+
+ c) Accompany it with the information you received as to the offer
+ to distribute corresponding source code. (This alternative is
+ allowed only for noncommercial distribution and only if you
+ received the program in object code or executable form with such
+ an offer, in accord with Subsection b above.)
+
+The source code for a work means the preferred form of the work for
+making modifications to it. For an executable work, complete source
+code means all the source code for all modules it contains, plus any
+associated interface definition files, plus the scripts used to
+control compilation and installation of the executable. However, as a
+special exception, the source code distributed need not include
+anything that is normally distributed (in either source or binary
+form) with the major components (compiler, kernel, and so on) of the
+operating system on which the executable runs, unless that component
+itself accompanies the executable.
+
+If distribution of executable or object code is made by offering
+access to copy from a designated place, then offering equivalent
+access to copy the source code from the same place counts as
+distribution of the source code, even though third parties are not
+compelled to copy the source along with the object code.
+
+ 4. You may not copy, modify, sublicense, or distribute the Program
+except as expressly provided under this License. Any attempt
+otherwise to copy, modify, sublicense or distribute the Program is
+void, and will automatically terminate your rights under this License.
+However, parties who have received copies, or rights, from you under
+this License will not have their licenses terminated so long as such
+parties remain in full compliance.
+
+ 5. You are not required to accept this License, since you have not
+signed it. However, nothing else grants you permission to modify or
+distribute the Program or its derivative works. These actions are
+prohibited by law if you do not accept this License. Therefore, by
+modifying or distributing the Program (or any work based on the
+Program), you indicate your acceptance of this License to do so, and
+all its terms and conditions for copying, distributing or modifying
+the Program or works based on it.
+
+ 6. Each time you redistribute the Program (or any work based on the
+Program), the recipient automatically receives a license from the
+original licensor to copy, distribute or modify the Program subject to
+these terms and conditions. You may not impose any further
+restrictions on the recipients' exercise of the rights granted herein.
+You are not responsible for enforcing compliance by third parties to
+this License.
+
+ 7. If, as a consequence of a court judgment or allegation of patent
+infringement or for any other reason (not limited to patent issues),
+conditions are imposed on you (whether by court order, agreement or
+otherwise) that contradict the conditions of this License, they do not
+excuse you from the conditions of this License. If you cannot
+distribute so as to satisfy simultaneously your obligations under this
+License and any other pertinent obligations, then as a consequence you
+may not distribute the Program at all. For example, if a patent
+license would not permit royalty-free redistribution of the Program by
+all those who receive copies directly or indirectly through you, then
+the only way you could satisfy both it and this License would be to
+refrain entirely from distribution of the Program.
+
+If any portion of this section is held invalid or unenforceable under
+any particular circumstance, the balance of the section is intended to
+apply and the section as a whole is intended to apply in other
+circumstances.
+
+It is not the purpose of this section to induce you to infringe any
+patents or other property right claims or to contest validity of any
+such claims; this section has the sole purpose of protecting the
+integrity of the free software distribution system, which is
+implemented by public license practices. Many people have made
+generous contributions to the wide range of software distributed
+through that system in reliance on consistent application of that
+system; it is up to the author/donor to decide if he or she is willing
+to distribute software through any other system and a licensee cannot
+impose that choice.
+
+This section is intended to make thoroughly clear what is believed to
+be a consequence of the rest of this License.
+
+ 8. If the distribution and/or use of the Program is restricted in
+certain countries either by patents or by copyrighted interfaces, the
+original copyright holder who places the Program under this License
+may add an explicit geographical distribution limitation excluding
+those countries, so that distribution is permitted only in or among
+countries not thus excluded. In such case, this License incorporates
+the limitation as if written in the body of this License.
+
+ 9. The Free Software Foundation may publish revised and/or new versions
+of the General Public License from time to time. Such new versions will
+be similar in spirit to the present version, but may differ in detail to
+address new problems or concerns.
+
+Each version is given a distinguishing version number. If the Program
+specifies a version number of this License which applies to it and "any
+later version", you have the option of following the terms and conditions
+either of that version or of any later version published by the Free
+Software Foundation. If the Program does not specify a version number of
+this License, you may choose any version ever published by the Free Software
+Foundation.
+
+ 10. If you wish to incorporate parts of the Program into other free
+programs whose distribution conditions are different, write to the author
+to ask for permission. For software which is copyrighted by the Free
+Software Foundation, write to the Free Software Foundation; we sometimes
+make exceptions for this. Our decision will be guided by the two goals
+of preserving the free status of all derivatives of our free software and
+of promoting the sharing and reuse of software generally.
+
+ NO WARRANTY
+
+ 11. BECAUSE THE PROGRAM IS LICENSED FREE OF CHARGE, THERE IS NO WARRANTY
+FOR THE PROGRAM, TO THE EXTENT PERMITTED BY APPLICABLE LAW. EXCEPT WHEN
+OTHERWISE STATED IN WRITING THE COPYRIGHT HOLDERS AND/OR OTHER PARTIES
+PROVIDE THE PROGRAM "AS IS" WITHOUT WARRANTY OF ANY KIND, EITHER EXPRESSED
+OR IMPLIED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF
+MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE. THE ENTIRE RISK AS
+TO THE QUALITY AND PERFORMANCE OF THE PROGRAM IS WITH YOU. SHOULD THE
+PROGRAM PROVE DEFECTIVE, YOU ASSUME THE COST OF ALL NECESSARY SERVICING,
+REPAIR OR CORRECTION.
+
+ 12. IN NO EVENT UNLESS REQUIRED BY APPLICABLE LAW OR AGREED TO IN WRITING
+WILL ANY COPYRIGHT HOLDER, OR ANY OTHER PARTY WHO MAY MODIFY AND/OR
+REDISTRIBUTE THE PROGRAM AS PERMITTED ABOVE, BE LIABLE TO YOU FOR DAMAGES,
+INCLUDING ANY GENERAL, SPECIAL, INCIDENTAL OR CONSEQUENTIAL DAMAGES ARISING
+OUT OF THE USE OR INABILITY TO USE THE PROGRAM (INCLUDING BUT NOT LIMITED
+TO LOSS OF DATA OR DATA BEING RENDERED INACCURATE OR LOSSES SUSTAINED BY
+YOU OR THIRD PARTIES OR A FAILURE OF THE PROGRAM TO OPERATE WITH ANY OTHER
+PROGRAMS), EVEN IF SUCH HOLDER OR OTHER PARTY HAS BEEN ADVISED OF THE
+POSSIBILITY OF SUCH DAMAGES.
+
+ END OF TERMS AND CONDITIONS
+
+ How to Apply These Terms to Your New Programs
+
+ If you develop a new program, and you want it to be of the greatest
+possible use to the public, the best way to achieve this is to make it
+free software which everyone can redistribute and change under these terms.
+
+ To do so, attach the following notices to the program. It is safest
+to attach them to the start of each source file to most effectively
+convey the exclusion of warranty; and each file should have at least
+the "copyright" line and a pointer to where the full notice is found.
+
+ <one line to give the program's name and a brief idea of what it does.>
+ Copyright (C) <year> <name of author>
+
+ This program is free software; you can redistribute it and/or modify
+ it under the terms of the GNU General Public License as published by
+ the Free Software Foundation; either version 2 of the License, or
+ (at your option) any later version.
+
+ This program is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ GNU General Public License for more details.
+
+ You should have received a copy of the GNU General Public License
+ along with this program; if not, write to the Free Software
+ Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
+
+
+Also add information on how to contact you by electronic and paper mail.
+
+If the program is interactive, make it output a short notice like this
+when it starts in an interactive mode:
+
+ Gnomovision version 69, Copyright (C) year name of author
+ Gnomovision comes with ABSOLUTELY NO WARRANTY; for details type `show w'.
+ This is free software, and you are welcome to redistribute it
+ under certain conditions; type `show c' for details.
+
+The hypothetical commands `show w' and `show c' should show the appropriate
+parts of the General Public License. Of course, the commands you use may
+be called something other than `show w' and `show c'; they could even be
+mouse-clicks or menu items--whatever suits your program.
+
+You should also get your employer (if you work as a programmer) or your
+school, if any, to sign a "copyright disclaimer" for the program, if
+necessary. Here is a sample; alter the names:
+
+ Yoyodyne, Inc., hereby disclaims all copyright interest in the program
+ `Gnomovision' (which makes passes at compilers) written by James Hacker.
+
+ <signature of Ty Coon>, 1 April 1989
+ Ty Coon, President of Vice
+
+This General Public License does not permit incorporating your program into
+proprietary programs. If your program is a subroutine library, you may
+consider it more useful to permit linking proprietary applications with the
+library. If this is what you want to do, use the GNU Library General
+Public License instead of this License.
--- /dev/null
+INCLUDES = -I$(srcdir)/include -I$(srcdir)/linux
+
+lib_LTLIBRARIES = linux/libibacm.la
+
+AM_CFLAGS = -g -Wall -D_GNU_SOURCE -march=i686
+
+linux_libibacm_la_CFLAGS = $(AM_CFLAGS)
+
+if HAVE_LD_VERSION_SCRIPT
+ libibacm_version_script = -Wl,--version-script=$(srcdir)/linux/libibacm.map
+else
+ libibacm_version_script =
+endif
+
+linux_libibacm_la_SOURCES = src/libacm.c linux/libacm_linux.c
+linux_libibacm_la_LDFLAGS = -version-info 0 -export-dynamic \
+ $(libibacm_version_script)
+linux_libibacm_la_DEPENDENCIES = $(srcdir)/linux/libibacm.map
+
+bin_PROGRAMS = svc/ib_acm util/ib_acme
+svc_ib_acm_SOURCES = src/acm.c
+util_ib_acme_SOURCES = src/acme.c linux/acme_linux.c
+util_ib_acme_LDADD = $(top_builddir)/linux/libibacm.la
+svc_ib_acm_CFLAGS = $(AM_CFLAGS)
+util_ib_acme_CFLAGS = $(AM_CFLAGS)
+
+libibacmincludedir = $(includedir)/infiniband
+
+libibacminclude_HEADERS = include/infiniband/ib_acm.h include/infiniband/acm.h
+
+man_MANS = \
+ man/ib_acm_resolve_name.3 \
+ man/ib_acm_resolve_ip.3 \
+ man/ib_acm_resolve_path.3 \
+ man/ib_acm_query_path.3 \
+ man/ib_acm_convert_to_path.3 \
+ man/ib_acme.1 \
+ man/ib_acm.1 \
+ man/ib_acm.7
+
+EXTRA_DIST = linux/libibacm.map libibacm.spec.in $(man_MANS)
+
+dist-hook: libibacm.spec
+ cp libibacm.spec $(distdir)
--- /dev/null
+This README is for userspace IB CM assistant library.
+
+
+Building
+========
+To make this directory, run:
+./autogen.sh && ./configure && make && make install
+
+Typically the autogen and configure steps only need be done the first
+time unless configure.in or Makefile.am changes.
+
+Libraries are installed by default at /usr/local/lib.
--- /dev/null
+# InfiniBand Communication Management Assistant for clusters address file\r
+#\r
+# Use ib_acme utility with -G option to automatically generate a sample\r
+# acm_addr.cfg file for the current system.\r
+#\r
+# Entry format is:\r
+# address device port pkey\r
+#\r
+# The address may be one of the following:\r
+# host_name - ascii character string, up to 31 characters\r
+# address - IPv4 or IPv6 formatted address\r
+#\r
+# device name - struct ibv_device name\r
+# port number - valid port number on device (numbering starts at 1)\r
+# pkey - partition key in hex (can specify 'default' for pkey 0xFFFF)\r
+#\r
+# Up to 4 addresses can be associated with a given <device, port, pkey> tuple\r
+#\r
+# Samples:\r
+# node31 ibv_device0 1 default\r
+# node31-1 ibv_device0 1 0x00FF\r
+# node31-2 ibv_device0 2 0x00FF\r
+# 192.168.0.1 ibv_device0 1 0xFFFF\r
+# 192.168.0.2 ibv_device0 2 default
\ No newline at end of file
--- /dev/null
+Assistant for InfiniBand Communication Management (IB ACM)\r
+\r
+Note: The IB ACM should be considered experimental.\r
+\r
+\r
+Overview\r
+--------\r
+The IB ACM package implements and provides a framework for experimental name, address, and route resolution services over InfiniBand. It is intended to address connection setup scalability issues running MPI applications on large clusters. The IB ACM provides information needed to establish a connection, but does not implement the CM protocol. Long term, the IB ACM may support multiple resolution mechanisms.\r
+\r
+The IB ACM is focused on being scalable and efficient. The current implementation limits network traffic, SA interactions, and centralized services. As a trade-off, it is not expected to support all cluster routing configurations. However, it is anticipated that additional functionality, such as path record caching, can be incorporated into the IB ACM to support a wider range of configurations.\r
+\r
+The IB ACM package is comprised of three components: the ib_acm service, a libibacm library, and a test/configuration utility - ib_acme. All are userspace components and are available for Linux and Windows. Additional details are given below.\r
+\r
+\r
+Quick Start Guide\r
+-----------------\r
+1. Prerequisites: libibverbs and libibumad must be installed.\r
+ The IB stack should be running with IPoIB configured\r
+2. Install the IB ACM package\r
+ This installs libibacm, ib_acm, and ib_acme.\r
+3. Run ib_acme \96A \96O\r
+ This will generate IB ACM address and options configuration files.\r
+ (acm_addr.cfg and acm_opts.cfg)\r
+4. Run ib_acm and leave running\r
+5. Optionally, run ib_acme \96s <source_ip> -d <dest_ip> \96v\r
+ This will verify that the ib_acm service is running.\r
+ It also verifies the path is usable on the given cluster.\r
+5. Install librdmacm.\r
+6. Define the following environment variable: RDMA_CM_USE_IB_ACM=1\r
+ The librdmacm will automatically use the ib_acm service.\r
+ On failures, the librdmacm will fall back to normal resolution.\r
+\r
+\r
+Details\r
+-------\r
+libibacm:\r
+The libibacm is an end-user library with simple interfaces for communicating with the ib_acm service. The libibacm implements the ib_acm client protocol. Although the interfaces to the libibacm are considered experimental, it\92s expected that existing calls will be supported going forward.\r
+\r
+For simplicity, all calls operate synchronously and are serialized. Possible future changes to the libibacm would be to process calls in parallel and add asynchronous interfaces.\r
+\r
+\r
+ib_acme:\r
+The ib_acme program serves a dual role. It acts as a utility to test ib_acm operation and help verify if the ib_acm is usable for a given cluster configuration. Additionally, it automatically generates ib_acm configuration files to assist with or eliminate manual setup.\r
+\r
+\r
+acm configuration files:\r
+The ib_acm service relies on two configuration files. The acm_addr.cfg file contains name and address mappings for each IB <device, port, pkey> endpoint. Although the names in the acm_addr.cfg file can be anything, ib_acme maps the host name and IP addresses to the IB endpoints.\r
+\r
+The acm_opts.cfg file provides a set of configurable options for the ib_acm service, such as timeout, number of retries, logging level, etc. ib_acme generates the acm_opts.cfg file using static information. A future enhancement would adjust options based on the current system and cluster size. \r
+\r
+\r
+ib_acm:\r
+The ib_acm service is responsible for resolving names and addresses to InfiniBand path information and caching such data. It is currently implemented as an executable application, but is a conceptual service or daemon that should execute with administrative privileges.\r
+\r
+The ib_acm implements a client interface over TCP sockets, which is abstracted by the libibacm library. One or more back-end protocols are used by the ib_acm service to satisfy user requests. Although the ib_acm supports standard SA path record queries on the back-end, it provides an experimental resolution protocol in hope of achieving greater scalability. \r
+\r
+Conceptually, the ib_acm service implements an ARP like protocol and uses IB multicast records to construct path record data. It makes the assumption that a unicast path between two endpoints is realizable if those endpoints can communicate over a multicast group with similar properties (rate, mtu, etc.)\r
+\r
+Specifically, all IB endpoints join a number of multicast groups. Multicast groups differ based on rates, mtu, sl, etc., and are prioritized. All participating endpoints must be able to communicate on the lowest priority multicast group. The ib_acm assigns one or more names/addresses to each IB endpoint using the acm_addr.cfg file. Clients provide source and destination names or addresses as input to the service, and receive as output path record data.\r
+\r
+The service maps a client's source name/address to a local IB endpoint. If the destination name/address is not cached locally, it sends a multicast request out on the lowest priority multicast group on the local endpoint. The request carries a list of multicast groups that the sender can use. The recipient of the request selects the highest priority multicast group that it can use as well and returns that information directly to the sender. The request data is cached by all endpoints that receive the multicast request message. The source endpoint also caches the response and uses the multicast group that was selected to construct path record data, which is returned to the client.\r
+\r
+The current implementation of the IB ACM has several additional restrictions. The ib_acm is limited in its handling of dynamic changes; the ib_acm must be stopped and restarted if a cluster is reconfigured. Cached data does not timed out and is only updated if a new resolution request is received from a different QPN than a cached request. Support for IPv6 has not been verified. The number of addresses that can be assigned to a single endpoint is limited to 4, and the number of multicast groups that an endpoint can support is limited to 2.\r
--- /dev/null
+# InfiniBand Multicast Communication Manager for clusters configuration file\r
+#\r
+# Use ib_acme utility with -O option to automatically generate a sample\r
+# acm_opts.cfg file for the current system.\r
+#\r
+# Entry format is:\r
+# name value\r
+\r
+# log_file:\r
+# Specifies the location of the ACM service output. The log file is used to\r
+# assist with ACM service debugging and troubleshooting. The log_file can\r
+# be set to 'stdout', 'stderr', or the base name of a file. If a file name\r
+# is specified, the actual name formed by appending a process ID and '.log'\r
+# extension to the end of the specified file name.\r
+# Examples:\r
+# log_file stdout\r
+# log_file stderr\r
+# log_file /tmp/acm_\r
+\r
+log_file stdout\r
+\r
+# log_level:\r
+# Indicates the amount of detailed data written to the log file. Log levels\r
+# should be one of the following values:\r
+# 0 - basic configuration & errors\r
+# 1 - verbose configuation & errors\r
+# 2 - verbose operation\r
+\r
+log_level 0\r
+\r
+# server_port:\r
+# TCP port number that the server listens on.\r
+# If this value is changed, then a corresponding change is required for\r
+# client applications.\r
+\r
+server_port 6125\r
+\r
+# timeout:\r
+# Additional time, in milliseconds, that the ACM service will wait for a\r
+# response from a remote ACM service or the IB SA. The actual request\r
+# timeout is this value plus the subnet timeout.\r
+\r
+timeout 2000\r
+\r
+# retries:\r
+# Number of times that the ACM service will retry a request. This affects\r
+# both ACM multicast messages and and IB SA messages.\r
+\r
+retries 15\r
+\r
+# send_depth:\r
+# Specifies the maximum number of outstanding requests that can be in\r
+# progress simultaneously. A larger send depth allows for greater\r
+# parallelism, but increases system resource usage and subnet load.\r
+# If the number of pending requests is greater than the send_depth,\r
+# the additional requests will automatically be queued until some of\r
+# the previous requests complete.\r
+\r
+send_depth 8\r
+\r
+# recv_depth:\r
+# Specifies the number of buffers allocated and ready to receive remote\r
+# requests. A larger receive depth consumes more system resources, but\r
+# can avoid dropping requests due to insufficient receive buffers.\r
+\r
+recv_depth 1024\r
+\r
+# min_mtu:\r
+# Indicates the minimum MTU supported by the ACM service. The ACM service\r
+# negotiates to use the largest MTU available between both sides of a\r
+# connection. It is most efficient and recommended that min_mtu be set\r
+# to the largest MTU value supported by all nodes in a cluster.\r
+\r
+min_mtu 2048\r
+\r
+#min_rate:\r
+# Indicates the minimum link rate, in Gbps, supported by the ACM service.\r
+# The ACM service negotiates to use the highest rate available between both\r
+# sides of a connection. It is most efficient and recommended that the\r
+# min_rate be set to the largest rate supported by all nodes in a cluster.\r
+\r
+min_rate 10\r
--- /dev/null
+#! /bin/sh
+
+set -x
+test -d ./config || mkdir ./config
+aclocal -I config
+libtoolize --force --copy
+autoheader
+automake --foreign --add-missing --copy
+autoconf
--- /dev/null
+dnl Process this file with autoconf to produce a configure script.\r
+\r
+AC_PREREQ(2.57)\r
+AC_INIT(ib_acm, 0.0.1, general@lists.openfabrics.org)\r
+AC_CONFIG_SRCDIR([src/libacm.c])\r
+AC_CONFIG_AUX_DIR(config)\r
+AM_CONFIG_HEADER(config.h)\r
+AM_INIT_AUTOMAKE(libibacm, 0.0.1)\r
+\r
+AM_PROG_LIBTOOL\r
+\r
+AC_ARG_ENABLE(libcheck, [ --disable-libcheck do not test for presence of ib libraries],\r
+[ if test "$enableval" = "no"; then\r
+ disable_libcheck=yes\r
+ fi\r
+])\r
+\r
+dnl Checks for programs\r
+AC_PROG_CC\r
+\r
+dnl Checks for typedefs, structures, and compiler characteristics.\r
+AC_C_CONST\r
+AC_CHECK_SIZEOF(long)\r
+\r
+dnl Checks for libraries\r
+if test "$disable_libcheck" != "yes"; then\r
+AC_CHECK_LIB(ibverbs, ibv_get_device_list, [],\r
+ AC_MSG_ERROR([ibv_get_device_list() not found. ib_acm requires libibverbs.]))\r
+AC_CHECK_LIB(ibumad, umad_send, [],\r
+ AC_MSG_ERROR([umad_send() not found. ib_acm requires libibumad.]))\r
+fi\r
+\r
+dnl Checks for header files.\r
+AC_HEADER_STDC\r
+if test "$disable_libcheck" != "yes"; then\r
+AC_CHECK_HEADER(infiniband/verbs.h, [],\r
+ AC_MSG_ERROR([<infiniband/verbs.h> not found. Is libibverbs installed?]))\r
+AC_CHECK_HEADER(infiniband/umad.h, [],\r
+ AC_MSG_ERROR([<infiniband/umad.h> not found. Is libibumad installed?]))\r
+fi\r
+\r
+AC_CACHE_CHECK(whether ld accepts --version-script, ac_cv_version_script,\r
+ if test -n "`$LD --help < /dev/null 2>/dev/null | grep version-script`"; then\r
+ ac_cv_version_script=yes\r
+ else\r
+ ac_cv_version_script=no\r
+ fi)\r
+\r
+AM_CONDITIONAL(HAVE_LD_VERSION_SCRIPT, test "$ac_cv_version_script" = "yes")\r
+\r
+AC_CONFIG_FILES([Makefile libibacm.spec])\r
+AC_OUTPUT\r
--- /dev/null
+DIRS = windows
\ No newline at end of file
--- /dev/null
+/*\r
+ * Copyright (c) 2009 Intel Corporation. All rights reserved.\r
+ *\r
+ * This software is available to you under the OpenFabrics.org BSD license\r
+ * below:\r
+ *\r
+ * Redistribution and use in source and binary forms, with or\r
+ * without modification, are permitted provided that the following\r
+ * conditions are met:\r
+ *\r
+ * - Redistributions of source code must retain the above\r
+ * copyright notice, this list of conditions and the following\r
+ * disclaimer.\r
+ *\r
+ * - Redistributions in binary form must reproduce the above\r
+ * copyright notice, this list of conditions and the following\r
+ * disclaimer in the documentation and/or other materials\r
+ * provided with the distribution.\r
+ *\r
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,\r
+ * EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF\r
+ * MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AWV\r
+ * NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS\r
+ * BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN\r
+ * ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN\r
+ * CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE\r
+ * SOFTWARE.\r
+ */\r
+\r
+#if !defined(ACM_H)\r
+#define ACM_H\r
+\r
+#include <infiniband/ib_acm.h>\r
+\r
+#define ACM_VERSION 1\r
+\r
+#define ACM_OP_MASK 0x0F\r
+#define ACM_OP_RESOLVE 0x01\r
+#define ACM_OP_QUERY 0x02\r
+//#define ACM_OP_CM 0x03\r
+//#define ACM_OP_ACK_REQ 0x40 /* optional ack is required */\r
+#define ACM_OP_ACK 0x80\r
+\r
+#define ACM_STATUS_SUCCESS 0\r
+#define ACM_STATUS_ENOMEM 1\r
+#define ACM_STATUS_EINVAL 2\r
+#define ACM_STATUS_ENODATA 3\r
+#define ACM_STATUS_ENOTCONN 5\r
+#define ACM_STATUS_ETIMEDOUT 6\r
+#define ACM_STATUS_ESRCADDR 7\r
+#define ACM_STATUS_ESRCTYPE 8\r
+#define ACM_STATUS_EDESTADDR 9\r
+#define ACM_STATUS_EDESTTYPE 10\r
+\r
+struct acm_hdr\r
+{\r
+ uint8_t version;\r
+ uint8_t opcode;\r
+ uint8_t status;\r
+ uint8_t param;\r
+ uint8_t dest_type;\r
+ uint8_t src_type;\r
+ uint8_t reserved[2];\r
+ uint64_t tid;\r
+};\r
+\r
+#define ACM_EP_TYPE_NAME 0x01\r
+#define ACM_EP_TYPE_ADDRESS_IP 0x02\r
+#define ACM_EP_TYPE_ADDRESS_IP6 0x03\r
+#define ACM_EP_TYPE_DEVICE 0x10\r
+#define ACM_EP_TYPE_AV 0x20\r
+\r
+#define ACM_MAX_ADDRESS 32\r
+\r
+union acm_ep_addr\r
+{\r
+ uint8_t addr[ACM_MAX_ADDRESS];\r
+ uint8_t name[ACM_MAX_ADDRESS];\r
+ struct ib_acm_dev_addr dev;\r
+ struct ibv_ah_attr av;\r
+};\r
+\r
+struct acm_resolve_msg\r
+{\r
+ struct acm_hdr hdr;\r
+ union acm_ep_addr src;\r
+ union acm_ep_addr dest;\r
+ struct ib_acm_resolve_data data;\r
+};\r
+\r
+//struct acm_cm_param\r
+//{\r
+// uint32_t qpn;\r
+// uint8_t init_depth;\r
+// uint8_t resp_resources;\r
+// uint8_t retry_cnt;\r
+// uint8_t rnr_retry_cnt;\r
+// uint16_t src_port;\r
+// uint16_t dest_port;\r
+// uint8_t reserved[4];\r
+//};\r
+\r
+//struct acm_cm_msg\r
+//{\r
+// struct acm_hdr hdr;\r
+// union acm_ep_addr src;\r
+// union acm_ep_addr dest;\r
+// struct acm_cm_param param;\r
+//};\r
+\r
+#define ACM_QUERY_PATH_RECORD 0x01\r
+#define ACM_QUERY_SA 0x80\r
+\r
+#define ACM_EP_TYPE_LID 0x01\r
+#define ACM_EP_TYPE_GID 0x02\r
+\r
+union acm_query_data\r
+{\r
+ struct ib_path_record path;\r
+};\r
+\r
+struct acm_query_msg\r
+{\r
+ struct acm_hdr hdr;\r
+ union acm_query_data data;\r
+ uint8_t reserved[16];\r
+};\r
+\r
+#define ACM_MSG_DATA_SIZE 80\r
+\r
+struct acm_msg\r
+{\r
+ struct acm_hdr hdr;\r
+ uint8_t data[ACM_MSG_DATA_SIZE];\r
+};\r
+\r
+#endif /* ACM_H */\r
--- /dev/null
+/*\r
+ * Copyright (c) 2009 Intel Corporation. All rights reserved.\r
+ *\r
+ * This software is available to you under the OpenFabrics.org BSD license\r
+ * below:\r
+ *\r
+ * Redistribution and use in source and binary forms, with or\r
+ * without modification, are permitted provided that the following\r
+ * conditions are met:\r
+ *\r
+ * - Redistributions of source code must retain the above\r
+ * copyright notice, this list of conditions and the following\r
+ * disclaimer.\r
+ *\r
+ * - Redistributions in binary form must reproduce the above\r
+ * copyright notice, this list of conditions and the following\r
+ * disclaimer in the documentation and/or other materials\r
+ * provided with the distribution.\r
+ *\r
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,\r
+ * EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF\r
+ * MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AWV\r
+ * NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS\r
+ * BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN\r
+ * ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN\r
+ * CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE\r
+ * SOFTWARE.\r
+ */\r
+\r
+#if !defined(IB_ACM_H)\r
+#define IB_ACM_H\r
+\r
+#include <infiniband/verbs.h>\r
+\r
+#if defined(_WIN32)\r
+#define LIB_EXPORT __declspec(dllexport)\r
+#else\r
+#define LIB_EXPORT\r
+#endif\r
+\r
+#ifdef __cplusplus\r
+extern "C" {\r
+#endif\r
+\r
+struct ib_acm_dev_addr\r
+{\r
+ uint64_t guid;\r
+ uint16_t pkey_index;\r
+ uint8_t port_num;\r
+ uint8_t reserved[5];\r
+};\r
+\r
+struct ib_acm_resolve_data\r
+{\r
+ uint32_t reserved1;\r
+ uint8_t init_depth;\r
+ uint8_t resp_resources;\r
+ uint8_t packet_lifetime;\r
+ uint8_t mtu;\r
+ uint8_t reserved2[8];\r
+};\r
+\r
+/**\r
+ * ib_acm_resolve_name - Resolve path data between the specified names.\r
+ * Description:\r
+ * Discover path information, including identifying the local device,\r
+ * between the given the source and destination names.\r
+ * Notes:\r
+ * The source and destination names should match entries in acm_addr.cfg\r
+ * configuration files on their respective systems. Typically, the\r
+ * source and destination names will refer to system host names\r
+ * assigned to an Infiniband port.\r
+ */\r
+LIB_EXPORT\r
+int ib_acm_resolve_name(char *src, char *dest,\r
+ struct ib_acm_dev_addr *dev_addr, struct ibv_ah_attr *ah,\r
+ struct ib_acm_resolve_data *data);\r
+\r
+/**\r
+ * ib_acm_resolve_ip - Resolve path data between the specified addresses.\r
+ * Description:\r
+ * Discover path information, including identifying the local device,\r
+ * between the given the source and destination addresses.\r
+ * Notes:\r
+ * The source and destination addresses should match entries in acm_addr.cfg\r
+ * configuration files on their respective systems. Typically, the\r
+ * source and destination addresses will refer to IP addresses assigned\r
+ * to an IPoIB instance.\r
+ */\r
+LIB_EXPORT\r
+int ib_acm_resolve_ip(struct sockaddr *src, struct sockaddr *dest,\r
+ struct ib_acm_dev_addr *dev_addr, struct ibv_ah_attr *ah,\r
+ struct ib_acm_resolve_data *data);\r
+\r
+\r
+#define IB_PATH_RECORD_REVERSIBLE 0x80\r
+\r
+struct ib_path_record\r
+{\r
+ uint64_t service_id;\r
+ union ibv_gid dgid;\r
+ union ibv_gid sgid;\r
+ uint16_t dlid;\r
+ uint16_t slid;\r
+ uint32_t flowlabel_hoplimit; /* resv-31:28 flow label-27:8 hop limit-7:0*/\r
+ uint8_t tclass;\r
+ uint8_t reversible_numpath; /* reversible-7:7 num path-6:0 */\r
+ uint16_t pkey;\r
+ uint16_t qosclass_sl; /* qos class-15:4 sl-3:0 */\r
+ uint8_t mtu; /* mtu selector-7:6 mtu-5:0 */\r
+ uint8_t rate; /* rate selector-7:6 rate-5:0 */\r
+ uint8_t packetlifetime; /* lifetime selector-7:6 lifetime-5:0 */\r
+ uint8_t preference;\r
+ uint8_t reserved[6];\r
+};\r
+\r
+/**\r
+ * ib_acm_resolve_path - Resolve path data meeting specified restrictions\r
+ * Description:\r
+ * Discover path information using the provided path record to\r
+ * restrict the discovery.\r
+ * Notes:\r
+ * Uses the provided path record as input into an query for path\r
+ * information. If successful, fills in any missing information. The\r
+ * caller must provide at least the source and destination LIDs as input.\r
+ */\r
+LIB_EXPORT\r
+int ib_acm_resolve_path(struct ib_path_record *path);\r
+\r
+/**\r
+ * ib_acm_query_path - Resolve path data meeting specified restrictions\r
+ * Description:\r
+ * Queries the IB SA for a path record using the provided path record to\r
+ * restrict the query.\r
+ * Notes:\r
+ * Uses the provided path record as input into an SA query for path\r
+ * information. If successful, fills in any missing information. The\r
+ * caller must provide at least the source and destination LIDs as input.\r
+ * Use of this call always results in sending a query to the IB SA.\r
+ */\r
+LIB_EXPORT\r
+int ib_acm_query_path(struct ib_path_record *path);\r
+\r
+/**\r
+ * ib_acm_convert_to_path - Convert resolved path data to a path record\r
+ * Description:\r
+ * Converts path information returned from resolving a host name or address\r
+ * to the format of an IB path record.\r
+ */\r
+LIB_EXPORT\r
+int ib_acm_convert_to_path(struct ib_acm_dev_addr *dev_addr,\r
+ struct ibv_ah_attr *ah, struct ib_acm_resolve_data *data,\r
+ struct ib_path_record *path);\r
+\r
+#ifdef __cplusplus\r
+}\r
+#endif\r
+\r
+#endif /* IB_ACM_H */\r
--- /dev/null
+%define ver @VERSION@
+
+Name: libibacm
+Version: 0.0.1
+Release: 1%{?dist}
+Summary: InfiniBand Communication Manager Assistant
+
+Group: System Environment/Libraries
+License: GPLv2 or BSD
+Url: http://www.openfabrics.org/
+Source: http://www.openfabrics.org/downloads/rdmacm/%{name}-%{version}.tar.gz
+BuildRoot: %{_tmppath}/%{name}-%{version}-%{release}-root-%(%{__id_u} -n)
+
+BuildRequires: libibverbs-devel >= 1.1-1
+
+%description
+libibacm assists with establishing communication over Infiniband.
+
+%package devel
+Summary: Development files for the libibacm library
+Group: System Environment/Libraries
+Requires: %{name} = %{version}-%{release}
+
+%description devel
+Development files for the libibacm library.
+
+%package svc
+Summary: IB CM pre-connection service application
+Group: System Environment/Libraries
+Requires: %{name} = %{version}-%{release} %{_includedir}/infiniband/verbs.h
+
+%description svc
+Application daemon for IB CM pre-connection services.
+
+%prep
+%setup -q -n %{name}-%{ver}
+
+%build
+%configure
+make %{?_smp_mflags}
+
+%install
+rm -rf $RPM_BUILD_ROOT
+%makeinstall
+# remove unpackaged files from the buildroot
+rm -f $RPM_BUILD_ROOT%{_libdir}/*.la
+
+%clean
+rm -rf $RPM_BUILD_ROOT
+
+%post -p /sbin/ldconfig
+%postun -p /sbin/ldconfig
+
+%files
+%defattr(-,root,root,-)
+%{_libdir}/libibacm*.so.*
+%doc AUTHORS COPYING README
+
+%files devel
+%defattr(-,root,root)
+%{_libdir}/lib*.so
+%{_libdir}/*.a
+%{_includedir}/*
+%{_mandir}/man3/*
+%{_mandir}/man7/*
+
+%files svc
+%defattr(-,root,root,-)
+%{_bindir}/*
+%{_mandir}/man1/*
+
+%changelog
--- /dev/null
+/*\r
+ * Copyright (c) 2009 Intel Corporation. All rights reserved.\r
+ *\r
+ * This software is available to you under the OpenIB.org BSD license\r
+ * below:\r
+ *\r
+ * Redistribution and use in source and binary forms, with or\r
+ * without modification, are permitted provided that the following\r
+ * conditions are met:\r
+ *\r
+ * - Redistributions of source code must retain the above\r
+ * copyright notice, this list of conditions and the following\r
+ * disclaimer.\r
+ *\r
+ * - Redistributions in binary form must reproduce the above\r
+ * copyright notice, this list of conditions and the following\r
+ * disclaimer in the documentation and/or other materials\r
+ * provided with the distribution.\r
+ *\r
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,\r
+ * EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF\r
+ * MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AWV\r
+ * NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS\r
+ * BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN\r
+ * ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN\r
+ * CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE\r
+ * SOFTWARE.\r
+ */\r
+\r
+#include <stdio.h>\r
+#include <stdlib.h>\r
+#include <string.h>\r
+#include <unistd.h>\r
+#include <arpa/inet.h>\r
+#include <sys/ioctl.h>\r
+#include <net/if.h>\r
+#include <net/if_arp.h>\r
+#include <netinet/in.h>\r
+#include <sys/socket.h>\r
+#include <sys/types.h>\r
+\r
+#include <infiniband/verbs.h>\r
+#include <infiniband/ib_acm.h>\r
+\r
+extern struct ibv_context **verbs;\r
+extern int dev_cnt;\r
+\r
+\r
+static int\r
+get_pkey(struct ifreq *ifreq, uint16_t *pkey)\r
+{\r
+ char buf[128], *end;\r
+ FILE *f;\r
+ int ret;\r
+\r
+ sprintf(buf, "//sys//class//net//%s//pkey", ifreq->ifr_name);\r
+ f = fopen(buf, "r");\r
+ if (!f) {\r
+ printf("failed to open %s\n", buf);\r
+ return -1;\r
+ }\r
+\r
+ if (fgets(buf, sizeof buf, f)) {\r
+ *pkey = strtol(buf, &end, 16);\r
+ ret = 0;\r
+ } else {\r
+ printf("failed to read pkey\n");\r
+ ret = -1;\r
+ } \r
+\r
+ fclose(f);\r
+ return ret;\r
+}\r
+\r
+static int\r
+get_sgid(struct ifreq *ifr, union ibv_gid *sgid)\r
+{\r
+ char buf[128], *end;\r
+ FILE *f;\r
+ int i, p, ret;\r
+\r
+ sprintf(buf, "//sys//class//net//%s//address", ifr->ifr_name);\r
+ f = fopen(buf, "r");\r
+ if (!f) {\r
+ printf("failed to open %s\n", buf);\r
+ return -1;\r
+ }\r
+\r
+ if (fgets(buf, sizeof buf, f)) {\r
+ for (i = 0, p = 12; i < 16; i++, p += 3) {\r
+ buf[p + 2] = '\0';\r
+ sgid->raw[i] = (uint8_t) strtol(buf + p, &end, 16);\r
+ }\r
+ ret = 0;\r
+ } else {\r
+ printf("failed to read sgid\n");\r
+ ret = -1;\r
+ }\r
+\r
+ fclose(f);\r
+ return ret;\r
+}\r
+\r
+static int\r
+get_devaddr(int s, struct ifreq *ifr,\r
+ int *dev_index, uint8_t *port, uint16_t *pkey)\r
+{\r
+ struct ibv_device_attr dev_attr;\r
+ struct ibv_port_attr port_attr;\r
+ union ibv_gid sgid, gid;\r
+ int ret, i;\r
+\r
+ ret = get_sgid(ifr, &sgid);\r
+ if (ret) {\r
+ printf("unable to get sgid\n");\r
+ return ret;\r
+ }\r
+\r
+ ret = get_pkey(ifr, pkey);\r
+ if (ret) {\r
+ printf("unable to get pkey\n");\r
+ return ret;\r
+ }\r
+\r
+ for (*dev_index = 0; *dev_index < dev_cnt; (*dev_index)++) {\r
+ ret = ibv_query_device(verbs[*dev_index], &dev_attr);\r
+ if (ret)\r
+ continue;\r
+\r
+ for (*port = 1; *port <= dev_attr.phys_port_cnt; (*port)++) {\r
+ ret = ibv_query_port(verbs[*dev_index], *port, &port_attr);\r
+ if (ret)\r
+ continue;\r
+\r
+ for (i = 0; i < port_attr.gid_tbl_len; i++) {\r
+ ret = ibv_query_gid(verbs[*dev_index], *port, i, &gid);\r
+ if (ret || !gid.global.interface_id)\r
+ break;\r
+\r
+ if (!memcmp(sgid.raw, gid.raw, sizeof gid))\r
+ return 0;\r
+ }\r
+ }\r
+ }\r
+ return -1;\r
+}\r
+\r
+int gen_addr_ip(FILE *f)\r
+{\r
+ struct ifconf *ifc;\r
+ struct ifreq *ifr;\r
+ char ip[sizeof "ffff:ffff:ffff:ffff:ffff:ffff:ffff:ffff"];\r
+ int s, ret, dev_index, i, len;\r
+ uint16_t pkey;\r
+ uint8_t port;\r
+\r
+ s = socket(AF_INET6, SOCK_DGRAM, 0);\r
+ if (!s)\r
+ return -1;\r
+\r
+ len = sizeof(*ifc) + sizeof(*ifr) * 64;\r
+ ifc = malloc(len);\r
+ if (!ifc) {\r
+ ret = -1;\r
+ goto out1;\r
+ }\r
+\r
+ memset(ifc, 0, len);\r
+ ifc->ifc_len = len;\r
+ ifc->ifc_req = (struct ifreq *) (ifc + 1);\r
+\r
+ ret = ioctl(s, SIOCGIFCONF, ifc);\r
+ if (ret < 0) {\r
+ printf("ioctl ifconf error %d\n", ret);\r
+ goto out2;\r
+ }\r
+\r
+ ifr = ifc->ifc_req;\r
+ for (i = 0; i < ifc->ifc_len / sizeof(struct ifreq); i++) {\r
+ switch (ifr[i].ifr_addr.sa_family) {\r
+ case AF_INET:\r
+ inet_ntop(ifr[i].ifr_addr.sa_family,\r
+ &((struct sockaddr_in *) &ifr[i].ifr_addr)->sin_addr, ip, sizeof ip);\r
+ break;\r
+ case AF_INET6:\r
+ inet_ntop(ifr[i].ifr_addr.sa_family,\r
+ &((struct sockaddr_in6 *) &ifr[i].ifr_addr)->sin6_addr, ip, sizeof ip);\r
+ break;\r
+ default:\r
+ continue;\r
+ }\r
+\r
+ ret = ioctl(s, SIOCGIFHWADDR, &ifr[i]);\r
+ if (ret) {\r
+ printf("failed to get hw address %d\n", ret);\r
+ continue;\r
+ }\r
+\r
+ if (ifr[i].ifr_hwaddr.sa_family != ARPHRD_INFINIBAND)\r
+ continue;\r
+\r
+ ret = get_devaddr(s, &ifr[i], &dev_index, &port, &pkey);\r
+ if (ret)\r
+ continue;\r
+\r
+ printf("%s %s %d 0x%x\n", ip, verbs[dev_index]->device->name, port, pkey);\r
+ fprintf(f, "%s %s %d 0x%x\n", ip, verbs[dev_index]->device->name, port, pkey);\r
+ }\r
+ ret = 0;\r
+\r
+out2:\r
+ free(ifc);\r
+out1:\r
+ close(s);\r
+ return ret;\r
+}\r
--- /dev/null
+/*\r
+ * Copyright (c) 2009 Intel Corporation. All rights reserved.\r
+ *\r
+ * This software is available to you under the OpenIB.org BSD license\r
+ * below:\r
+ *\r
+ * Redistribution and use in source and binary forms, with or\r
+ * without modification, are permitted provided that the following\r
+ * conditions are met:\r
+ *\r
+ * - Redistributions of source code must retain the above\r
+ * copyright notice, this list of conditions and the following\r
+ * disclaimer.\r
+ *\r
+ * - Redistributions in binary form must reproduce the above\r
+ * copyright notice, this list of conditions and the following\r
+ * disclaimer in the documentation and/or other materials\r
+ * provided with the distribution.\r
+ *\r
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,\r
+ * EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF\r
+ * MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AWV\r
+ * NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS\r
+ * BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN\r
+ * ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN\r
+ * CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE\r
+ * SOFTWARE.\r
+ */\r
+\r
+#ifndef _DLIST_H_\r
+#define _DLIST_H_\r
+\r
+#ifdef __cplusplus\r
+extern "C" {\r
+#endif\r
+\r
+typedef struct _DLIST_ENTRY\r
+{\r
+ struct _DLIST_ENTRY *Next;\r
+ struct _DLIST_ENTRY *Prev;\r
+\r
+} DLIST_ENTRY;\r
+\r
+static void DListInit(DLIST_ENTRY *pHead)\r
+{\r
+ pHead->Next = pHead;\r
+ pHead->Prev = pHead;\r
+}\r
+\r
+static int DListEmpty(DLIST_ENTRY *pHead)\r
+{\r
+ return pHead->Next == pHead;\r
+}\r
+\r
+static void DListInsertAfter(DLIST_ENTRY *pNew, DLIST_ENTRY *pHead)\r
+{\r
+ pNew->Next = pHead->Next;\r
+ pNew->Prev = pHead;\r
+ pHead->Next->Prev = pNew;\r
+ pHead->Next = pNew;\r
+}\r
+\r
+static void DListInsertBefore(DLIST_ENTRY *pNew, DLIST_ENTRY *pHead)\r
+{\r
+ DListInsertAfter(pNew, pHead->Prev);\r
+}\r
+\r
+#define DListInsertHead DListInsertAfter\r
+#define DListInsertTail DListInsertBefore\r
+\r
+static void DListRemove(DLIST_ENTRY *pEntry)\r
+{\r
+ pEntry->Prev->Next = pEntry->Next;\r
+ pEntry->Next->Prev = pEntry->Prev;\r
+}\r
+\r
+#ifdef __cplusplus\r
+}\r
+#endif\r
+\r
+#endif // _DLIST_H_
\ No newline at end of file
--- /dev/null
+/*\r
+ * Copyright (c) 2009 Intel Corporation. All rights reserved.\r
+ *\r
+ * This software is available to you under the OpenIB.org BSD license\r
+ * below:\r
+ *\r
+ * Redistribution and use in source and binary forms, with or\r
+ * without modification, are permitted provided that the following\r
+ * conditions are met:\r
+ *\r
+ * - Redistributions of source code must retain the above\r
+ * copyright notice, this list of conditions and the following\r
+ * disclaimer.\r
+ *\r
+ * - Redistributions in binary form must reproduce the above\r
+ * copyright notice, this list of conditions and the following\r
+ * disclaimer in the documentation and/or other materials\r
+ * provided with the distribution.\r
+ *\r
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,\r
+ * EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF\r
+ * MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AWV\r
+ * NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS\r
+ * BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN\r
+ * ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN\r
+ * CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE\r
+ * SOFTWARE.\r
+ */\r
+\r
+#include <osd.h>\r
+\r
+lock_t lock;\r
+\r
+static void __attribute__((constructor)) lib_init(void)\r
+{\r
+ lock_init(&lock);\r
+}\r
--- /dev/null
+IBACM_1.0 {
+ global:
+ ib_acm_resolve_ip;
+ ib_acm_resolve_name;
+ ib_acm_resolve_path;
+ ib_acm_query_path;
+ ib_acm_convert_to_path;
+ local: *;
+};
--- /dev/null
+/*\r
+ * Copyright (c) 2009 Intel Corporation. All rights reserved.\r
+ *\r
+ * This software is available to you under the OpenFabrics.org BSD license\r
+ * below:\r
+ *\r
+ * Redistribution and use in source and binary forms, with or\r
+ * without modification, are permitted provided that the following\r
+ * conditions are met:\r
+ *\r
+ * - Redistributions of source code must retain the above\r
+ * copyright notice, this list of conditions and the following\r
+ * disclaimer.\r
+ *\r
+ * - Redistributions in binary form must reproduce the above\r
+ * copyright notice, this list of conditions and the following\r
+ * disclaimer in the documentation and/or other materials\r
+ * provided with the distribution.\r
+ *\r
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,\r
+ * EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF\r
+ * MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AWV\r
+ * NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS\r
+ * BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN\r
+ * ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN\r
+ * CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE\r
+ * SOFTWARE.\r
+ */\r
+\r
+#if !defined(OSD_H)\r
+#define OSD_H\r
+\r
+#include <stdlib.h>\r
+#include <string.h>\r
+#include <stdio.h>\r
+#include <unistd.h>\r
+#include <errno.h>\r
+#include <byteswap.h>\r
+#include <pthread.h>\r
+#include <sys/socket.h>\r
+#include <sys/types.h>\r
+#include <malloc.h>\r
+#include <arpa/inet.h>\r
+#include <sys/time.h>\r
+#include <netinet/in.h>\r
+\r
+#define LIB_DESTRUCTOR __attribute__((destructor))\r
+#define CDECL_FUNC\r
+\r
+#define container_of(ptr, type, field) \\r
+ ((type *) ((void *) ptr - offsetof(type, field)))\r
+\r
+#define min(a, b) (a < b ? a : b)\r
+#define max(a, b) (a > b ? a : b)\r
+\r
+#if __BYTE_ORDER == __LITTLE_ENDIAN\r
+#define htonll(x) bswap_64(x)\r
+#else\r
+#define htonll(x) (x)\r
+#endif\r
+#define ntohll(x) htonll(x)\r
+\r
+typedef struct { volatile int val; } atomic_t;\r
+#define atomic_inc(v) (__sync_fetch_and_add(&(v)->val, 1) + 1)\r
+#define atomic_dec(v) (__sync_fetch_and_sub(&(v)->val, 1) - 1)\r
+#define atomic_get(v) ((v)->val)\r
+#define atomic_set(v, s) ((v)->val = s)\r
+\r
+#define stricmp strcasecmp\r
+#define strnicmp strncasecmp\r
+\r
+typedef struct { pthread_cond_t cond; pthread_mutex_t mutex; } event_t;\r
+static inline void event_init(event_t *e)\r
+{\r
+ pthread_cond_init(&e->cond, NULL);\r
+ pthread_mutex_init(&e->mutex, NULL);\r
+}\r
+#define event_signal(e) pthread_cond_signal(&(e)->cond)\r
+static inline int event_wait(event_t *e, int timeout) \r
+{\r
+ struct timeval curtime;\r
+ struct timespec wait;\r
+ int ret;\r
+\r
+ gettimeofday(&curtime, NULL);\r
+ wait.tv_sec = curtime.tv_sec + ((unsigned) timeout) / 1000;\r
+ wait.tv_nsec = (curtime.tv_usec + (((unsigned) timeout) % 1000) * 1000) * 1000;\r
+ pthread_mutex_lock(&e->mutex);\r
+ ret = pthread_cond_timedwait(&e->cond, &e->mutex, &wait);\r
+ pthread_mutex_unlock(&e->mutex);\r
+ return ret;\r
+}\r
+\r
+#define lock_t pthread_mutex_t\r
+#define lock_init(x) pthread_mutex_init(x, NULL)\r
+#define lock_acquire pthread_mutex_lock\r
+#define lock_release pthread_mutex_unlock\r
+\r
+#define osd_init() 0\r
+#define osd_close()\r
+\r
+#define SOCKET int\r
+#define SOCKET_ERROR -1\r
+#define INVALID_SOCKET -1\r
+#define socket_errno() errno\r
+#define closesocket close\r
+\r
+static inline uint64_t time_stamp_us(void)\r
+{\r
+ struct timeval curtime;\r
+ timerclear(&curtime);\r
+ gettimeofday(&curtime, NULL);\r
+ return (uint64_t) curtime.tv_sec * 1000000 + (uint64_t) curtime.tv_usec;\r
+}\r
+\r
+#define time_stamp_ms() (time_stamp_us() / 1000)\r
+\r
+static inline int beginthread(void (*func)(void *), void *arg)\r
+{\r
+ pthread_t thread;\r
+ return pthread_create(&thread, NULL, (void *(*)(void*)) func, arg);\r
+}\r
+\r
+#endif /* OSD_H */\r
--- /dev/null
+.TH "ib_acm" 1 "2009-09-09" "ib_acm" "ib_acm" ib_acm
+.SH NAME
+ib_acm \- address and route resolution services for InfiniBand.
+.SH SYNOPSIS
+.sp
+.nf
+\fIib_acm\fR
+.fi
+.SH "DESCRIPTION"
+ib_acm provides address and route (path) resolution services
+over Infiniband. It resolves names and IP addresses and returns path
+information needed to communicate with a remote node.
+.SH "NOTES"
+The ib_acm application should actively run with administrative privileges on
+all nodes in a cluster. This version of the ib_acm is for evaluation purposes
+only and is intended to assist with MPI application scaling. See the
+acm_opts.cfg file included with the installation for details on available
+configuration options.
+.SH "SEE ALSO"
+ib_acm(7) ib_acme(1)
--- /dev/null
+.TH "IB_ACM" 7 "2009-09-09" "IB_ACM" "IB ACM User Guide" IB_ACM\r
+.SH NAME\r
+ib_acm \- InfiniBand communication management assistant\r
+.SH SYNOPSIS\r
+.B "#include <infiniband/ib_acm.h>"\r
+.SH "DESCRIPTION"\r
+Used to resolve remote endpoint information before establishing communications\r
+over InfiniBand.\r
+.SH "NOTES"\r
+Th IB ACM provides scalable address and route resolution services over\r
+InfiniBand. It resolves system network names and IP addresses to InfiniBand\r
+path record data using efficient mechanisms, including caching of data.\r
+.P\r
+The IB ACM provides information needed to establish a connection, but does\r
+not implement the communication management protocol. It provides services\r
+similar to getaddrinfo, rdma_resolve_addr, and rdma_resolve_route using\r
+IB multicast.\r
+The IB ACM does not require IPoIB or use standard naming services, such as\r
+DNS, and limits network communication, especially with the IB SA.\r
+Because it uses multicast traffic and limits SA interaction, it may\r
+not support all cluster configurations. The ib_acme utility assists in\r
+verifying what options of the ib_acm service may be usable.\r
+.P\r
+Client interactions with the ib_acm service are done over sockets through\r
+a standard TCP connection. The libibacm abstracts this interaction with\r
+a simple set of interfaces.\r
+.SH "RETURN CODES"\r
+.IP "== 0"\r
+success\r
+.IP "!= 0"\r
+error\r
+.SH "SEE ALSO"\r
+ib_acm_resolve_ip(3), ib_acm_resolve_name(3), ib_acm_resolve_path(3),\r
+ib_acm_query_path(3), ib_acm_convert_to_path(3), ib_acme(1), ib_acm(1)\r
--- /dev/null
+.TH "IB_ACM_CONVERT_TO_PATH" 3 "2009-09-09" "libibacm" "Libibacm Programmer's Manual" libibacm
+.SH NAME
+ib_acm_convert_to_path \- Convert resolved path data to a path record.
+.SH SYNOPSIS
+.B "#include <infiniband/ib_acm.h>"
+.P
+.B "int" ib_acm_convert_to_path
+.BI "(struct ib_acm_dev_addr *" dev_addr ","\r
+.BI "struct ibv_ah_attr *" ah ","\r
+.BI "struct ib_acm_resolve_data *" data ","\r
+.BI "struct ib_path_record *" path ");"\r
+.SH ARGUMENTS
+.IP "dev_addr" 12
+Local IB device address information.
+.IP "ah" 12
+Address handle attributes to send to the destination.
+.IP "data" 12
+Additional route and endpoint information.
+.SH "DESCRIPTION"
+Converts path information obtained from ib_acm_resolve_ip or ib_acm_resolve_name\r
+to the format of an IB path record.\r
+.SH "SEE ALSO"
+ib_acm_resolve_name(3), ib_acm_resolve_ip(3), ib_acm_resolve_path(3),
+ib_acm_query_path(3), ib_acm(7)
+
--- /dev/null
+.TH "IB_ACM_QUERY_PATH" 3 "2009-09-09" "libibacm" "Libibacm Programmer's Manual" libibacm
+.SH NAME
+ib_acm_query_path \- Resolve path data meeting specified restrictions.
+.SH SYNOPSIS
+.B "#include <infiniband/ib_acm.h>"
+.P
+.B "int" ib_acm_resolve_name
+.BI "(struct ib_path_record *" path ");"\r
+.SH ARGUMENTS
+.IP "path" 12
+On input, provides path information used to restrict path resolution. On
+output, contains the full path record.
+.SH "DESCRIPTION"
+Queries the IB SA for a path record using the provided path record to\r
+restrict the query.
+.SH "NOTES"
+Uses the provided path record as input into an SA query for path\r
+information. If successful, fills in any missing information. The\r
+caller must provide at least the source and destination LIDs as input.\r
+Use of this call always results in sending a query to the IB SA.\r
+.SH "SEE ALSO"
+ib_acm_resolve_name(3), ib_acm_resolve_ip(3), ib_acm_resolve_path(3),
+ib_acm_convert_to_path(3), ib_acm(7)
+
--- /dev/null
+.TH "IB_ACM_RESOLVE_IP" 3 "2009-09-09" "libibacm" "Libibacm Programmer's Manual" libibacm
+.SH NAME
+ib_acm_resolve_ip \- Resolve path data between the specified addresses.
+.SH SYNOPSIS
+.B "#include <infiniband/ib_acm.h>"
+.P
+.B "int" ib_acm_resolve_ip
+.BI "(struct sockaddr *" src ","\r
+.BI "struct sockaddr *" dest ","\r
+.BI "struct ib_acm_dev_addr *" dev_addr ","\r
+.BI "struct ibv_ah_attr *" ah ","\r
+.BI "struct ib_acm_resolve_data *" data ");"\r
+.SH ARGUMENTS
+.IP "src" 12
+Source address.
+.IP "dest" 12
+Destination address.
+.IP "dev_addr" 12
+On success, local IB device address information. This includes the
+local device GUID, port number, and pkey index to use for communication.
+.IP "ah" 12
+On success, address handle attributes to send to the destination. The
+address handle attributes may be used to configure a connected QP or
+create an address handle for datagram traffic.
+.IP "data" 12
+On success, additional route and endpoint information. This includes
+the MTU, packet lifetime, and maximum initiator depth and responder
+resources information usable between the end points.
+.SH "DESCRIPTION"
+Discover path information, including identifying the local device,\r
+between the given the source and destination addresses.
+.SH "NOTES"
+The source and destination addresses should match entries in acm_addr.cfg\r
+configuration files on their respective systems. Typically, the\r
+source and destination addresses will refer to IP addresses assigned\r
+to an IPoIB instance.\r
+.SH "SEE ALSO"
+ib_acm_resolve_name(3), ib_acm_resolve_path(3), ib_acm_query_path(3),
+ib_acm_convert_to_path(3), ib_acm(7)
+
--- /dev/null
+.TH "IB_ACM_RESOLVE_NAME" 3 "2009-09-09" "libibacm" "Libibacm Programmer's Manual" libibacm
+.SH NAME
+ib_acm_resolve_name \- Resolve path data between the specified names.
+.SH SYNOPSIS
+.B "#include <infiniband/ib_acm.h>"
+.P
+.B "int" ib_acm_resolve_name
+.BI "(char *" src ","\r
+.BI "char *" dest ","\r
+.BI "struct ib_acm_dev_addr *" dev_addr ","\r
+.BI "struct ibv_ah_attr *" ah ","\r
+.BI "struct ib_acm_resolve_data *" data ");"\r
+.SH ARGUMENTS
+.IP "src" 12
+Source system network name or host name.
+.IP "dest" 12
+Destination system network name or host name.
+.IP "dev_addr" 12
+On success, local IB device address information. This includes the
+local device GUID, port number, and pkey index to use for communication.
+.IP "ah" 12
+On success, address handle attributes to send to the destination. The
+address handle attributes may be used to configure a connected QP or
+create an address handle for datagram traffic.
+.IP "data" 12
+On success, additional route and endpoint information. This includes
+the MTU, packet lifetime, and maximum initiator depth and responder
+resources information usable between the end points.
+.SH "DESCRIPTION"
+Discover path information, including identifying the local device,\r
+between the given the source and destination names.
+.SH "NOTES"
+The source and destination names should match entries in acm_addr.cfg\r
+configuration files on their respective systems. Typically, the\r
+source and destination names will refer to system host names\r
+assigned to an Infiniband port.\r
+.SH "SEE ALSO"
+ib_acm_resolve_ip(3), ib_acm_resolve_path(3), ib_acm_query_path(3),
+ib_acm_convert_to_path(3), ib_acm(7)
+
--- /dev/null
+.TH "IB_ACM_RESOLVE_PATH" 3 "2009-09-09" "libibacm" "Libibacm Programmer's Manual" libibacm
+.SH NAME
+ib_acm_resolve_path \- Resolve path data meeting specified restrictions.
+.SH SYNOPSIS
+.B "#include <infiniband/ib_acm.h>"
+.P
+.B "int" ib_acm_resolve_name
+.BI "(struct ib_path_record *" path ");"\r
+.SH ARGUMENTS
+.IP "path" 12
+On input, provides path information used to restrict path resolution. On
+output, contains the full path record.
+.SH "DESCRIPTION"
+Discover path information using the provided path record to\r
+restrict the discovery.\r
+.SH "NOTES"
+Uses the provided path record as input into an query for path\r
+information. If successful, fills in any missing information. The\r
+caller must provide at least the source and destination LIDs as input.\r
+The returned path record may come from cached data.\r
+.SH "SEE ALSO"
+ib_acm_resolve_name(3), ib_acm_resolve_ip(3), ib_acm_query_path(3),
+ib_acm_convert_to_path(3), ib_acm(7)
+
--- /dev/null
+.TH "ib_acme" 7 "2009-09-09" "ib_acme" "ib_acme" ib_acme\r
+.SH NAME\r
+ib_acme \- test and configuration utility for the IB ACM\r
+.SH SYNOPSIS\r
+.sp\r
+.nf\r
+\fIib_acm\fR [-f addr_format] -s src_addr -d dest_addr [-v]\r
+.fi\r
+.nf\r
+\fIib_acm\fR [-G] [-O]\r
+.fi\r
+.SH "DESCRIPTION"\r
+ib_acm provides address and route (path) resolution services\r
+over Infiniband. It resolves names and IP addresses and returns path\r
+information needed to communicate with a remote node.\r
+.SH "OPTIONS"\r
+.TP\r
+\-f addr_format\r
+Specifies the format of the src_addr and dest_addr parameters. Valid\r
+address formats are: 'i', 'n', and 'p', which indicate that the src_addr\r
+and dest_addr parameters are ip addresses, system network names, or LIDs,\r
+respectively. If the -f option is omitted, ip addressing is assumed.\r
+.TP\r
+\-s src_addr\r
+Specifies the local source address of the path to resolve. The source\r
+address can be an IP address, system network name, or LID, as indicated by\r
+the addr_format option.\r
+.TP\r
+\-s dest_addr\r
+Specifies the destination address of the path to resolve. The destination\r
+address can be an IP address, system network name, or LID, as indicated by\r
+the addr_format option.\r
+.TP\r
+\-v\r
+Indicates that the resolved path information should be verified with the\r
+active IB SA. Use of the -v option provides a sanity check that\r
+resolved path information is usable given the current cluster configuration.\r
+.TP\r
+\-A\r
+With this option, the ib_acme utility automatically generates the address\r
+configuration file acm_addr.cfg. The generated file is\r
+constructed using the system host name and any IP addresses that are\r
+assigned to IPoIB device instances.\r
+.TP\r
+\-O\r
+With this option, the ib_acme utility automatically generates the option\r
+configuration file acm_opts.cfg. The generated file is currently generated\r
+using static information.\r
+.SH "NOTES"\r
+The ib_acme utility performs two main functions. With the -A and -O options,\r
+it automatically generates address or options configuration files. The\r
+generated files are text based and may be edited. These options are intended\r
+to provide a simple way to configure address and option information on all\r
+nodes on a cluster.\r
+.P\r
+The other function of the ib_acme utility is to test the ib_acm service,\r
+including helping to verify that the service is usable given the current\r
+cluster configuration. The ib_acme utility can resolve IP addresses,\r
+network names, or IB LIDs into a path record. It can then compare that\r
+path record against one obtained by the SA. When used to test the\r
+ib_acm service, the ib_acme utility has the side effect of loading the\r
+ib_acm caches.\r
+.SH "SEE ALSO"\r
+ib_acm(7) ib_acm(1)\r
--- /dev/null
+/*\r
+ * Copyright (c) 2009 Intel Corporation. All rights reserved.\r
+ *\r
+ * This software is available to you under the OpenIB.org BSD license\r
+ * below:\r
+ *\r
+ * Redistribution and use in source and binary forms, with or\r
+ * without modification, are permitted provided that the following\r
+ * conditions are met:\r
+ *\r
+ * - Redistributions of source code must retain the above\r
+ * copyright notice, this list of conditions and the following\r
+ * disclaimer.\r
+ *\r
+ * - Redistributions in binary form must reproduce the above\r
+ * copyright notice, this list of conditions and the following\r
+ * disclaimer in the documentation and/or other materials\r
+ * provided with the distribution.\r
+ *\r
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,\r
+ * EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF\r
+ * MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AWV\r
+ * NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS\r
+ * BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN\r
+ * ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN\r
+ * CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE\r
+ * SOFTWARE.\r
+ */\r
+\r
+#include <stdio.h>\r
+#include <stdarg.h>\r
+#include <string.h>\r
+#include <osd.h>\r
+#include <arpa/inet.h>\r
+#include <infiniband/acm.h>\r
+#include <infiniband/umad.h>\r
+#include <dlist.h>\r
+#include <search.h>\r
+#include "acm_mad.h"\r
+\r
+#define MAX_EP_ADDR 4\r
+#define MAX_EP_MC 2\r
+\r
+struct acm_dest\r
+{\r
+ uint8_t address[ACM_MAX_ADDRESS]; /* keep first */\r
+ struct ibv_ah *ah;\r
+ struct ibv_ah_attr av;\r
+ union ibv_gid mgid;\r
+ DLIST_ENTRY req_queue;\r
+ uint32_t remote_qpn;\r
+ uint8_t init_depth;\r
+ uint8_t resp_resources;\r
+ uint8_t mtu;\r
+ uint8_t packet_lifetime;\r
+};\r
+\r
+struct acm_port\r
+{\r
+ struct acm_device *dev;\r
+ DLIST_ENTRY ep_list;\r
+ int mad_portid;\r
+ int mad_agentid;\r
+ struct acm_dest sa_dest;\r
+ enum ibv_port_state state;\r
+ enum ibv_mtu mtu;\r
+ enum ibv_rate rate;\r
+ int subnet_timeout;\r
+ int gid_cnt;\r
+ uint16_t pkey_cnt;\r
+ uint16_t lid;\r
+ uint8_t lmc;\r
+ uint8_t port_num;\r
+};\r
+\r
+struct acm_device\r
+{\r
+ struct ibv_context *verbs;\r
+ struct ibv_comp_channel *channel;\r
+ struct ibv_pd *pd;\r
+ uint64_t guid;\r
+ DLIST_ENTRY entry;\r
+ uint8_t active;\r
+ uint8_t init_depth;\r
+ uint8_t resp_resources;\r
+ int port_cnt;\r
+ struct acm_port port[0];\r
+};\r
+\r
+struct acm_ep\r
+{\r
+ struct acm_port *port;\r
+ struct ibv_cq *cq;\r
+ struct ibv_qp *qp;\r
+ struct ibv_mr *mr;\r
+ uint8_t *recv_bufs;\r
+ DLIST_ENTRY entry;\r
+ union acm_ep_addr addr[MAX_EP_ADDR];\r
+ uint8_t addr_type[MAX_EP_ADDR];\r
+ void *dest_map[ACM_ADDRESS_RESERVED - 1];\r
+ struct acm_dest mc_dest[MAX_EP_MC];\r
+ int mc_cnt;\r
+ uint16_t pkey_index;\r
+ uint16_t pkey;\r
+ lock_t lock;\r
+ int available_sends;\r
+ DLIST_ENTRY pending_queue;\r
+ DLIST_ENTRY active_queue;\r
+ DLIST_ENTRY wait_queue;\r
+};\r
+\r
+struct acm_send_msg\r
+{\r
+ DLIST_ENTRY entry;\r
+ struct acm_ep *ep;\r
+ struct ibv_mr *mr;\r
+ struct ibv_send_wr wr;\r
+ struct ibv_sge sge;\r
+ uint64_t expires;\r
+ int tries;\r
+ uint8_t data[ACM_SEND_SIZE];\r
+};\r
+\r
+struct acm_client\r
+{\r
+ lock_t lock; /* acquire ep lock first */\r
+ SOCKET sock;\r
+ int index;\r
+ atomic_t refcnt;\r
+};\r
+\r
+struct acm_request\r
+{\r
+ struct acm_client *client;\r
+ DLIST_ENTRY entry;\r
+ struct acm_msg msg;\r
+};\r
+\r
+static DLIST_ENTRY dev_list;\r
+\r
+static atomic_t tid;\r
+static DLIST_ENTRY timeout_list;\r
+static event_t timeout_event;\r
+static atomic_t wait_cnt;\r
+\r
+static SOCKET listen_socket;\r
+static struct acm_client client[FD_SETSIZE - 1];\r
+\r
+static FILE *flog;\r
+static lock_t log_lock;\r
+\r
+static char log_file[128] = "stdout";\r
+static int log_level = 0;\r
+static short server_port = 6125;\r
+static int timeout = 2000;\r
+static int retries = 15;\r
+static int send_depth = 64;\r
+static int recv_depth = 1024;\r
+static uint8_t min_mtu = IBV_MTU_2048;\r
+static uint8_t min_rate = IBV_RATE_10_GBPS;\r
+\r
+#define acm_log(level, format, ...) \\r
+ acm_write(level, "%s: "format, __func__, ## __VA_ARGS__)\r
+\r
+static void acm_write(int level, const char *format, ...)\r
+{\r
+ va_list args;\r
+\r
+ if (level > log_level)\r
+ return;\r
+\r
+ va_start(args, format);\r
+ lock_acquire(&log_lock);\r
+ vfprintf(flog, format, args);\r
+ lock_release(&log_lock);\r
+ va_end(args);\r
+}\r
+\r
+static void acm_log_ep_addr(int level, const char *msg,\r
+ union acm_ep_addr *addr, uint8_t ep_type)\r
+{\r
+ char ip_addr[ACM_MAX_ADDRESS];\r
+\r
+ if (level > log_level)\r
+ return;\r
+\r
+ lock_acquire(&log_lock);\r
+ fprintf(flog, msg);\r
+ switch (ep_type) {\r
+ case ACM_EP_TYPE_NAME:\r
+ fprintf(flog, "%s\n", addr->name);\r
+ break;\r
+ case ACM_EP_TYPE_ADDRESS_IP:\r
+ inet_ntop(AF_INET, addr->addr, ip_addr, ACM_MAX_ADDRESS);\r
+ fprintf(flog, "%s\n", ip_addr);\r
+ break;\r
+ case ACM_EP_TYPE_ADDRESS_IP6:\r
+ inet_ntop(AF_INET6, addr->addr, ip_addr, ACM_MAX_ADDRESS);\r
+ fprintf(flog, "%s\n", ip_addr);\r
+ break;\r
+ case ACM_EP_TYPE_DEVICE:\r
+ fprintf(flog, "device guid 0x%llx, pkey index %d, port %d\n",\r
+ addr->dev.guid, addr->dev.pkey_index, addr->dev.port_num);\r
+ break;\r
+ case ACM_EP_TYPE_AV:\r
+ fprintf(flog, "endpoint specified using address vector\n");\r
+ break;\r
+ default:\r
+ fprintf(flog, "unknown endpoint address 0x%x\n", ep_type);\r
+ }\r
+ lock_release(&log_lock);\r
+}\r
+\r
+static void *zalloc(size_t size)\r
+{\r
+ void *buf;\r
+\r
+ buf = malloc(size);\r
+ if (buf)\r
+ memset(buf, 0, size);\r
+ return buf;\r
+}\r
+\r
+static struct acm_send_msg *\r
+acm_alloc_send(struct acm_ep *ep, struct acm_dest *dest, size_t size)\r
+{\r
+ struct acm_send_msg *msg;\r
+\r
+ msg = (struct acm_send_msg *) zalloc(sizeof *msg);\r
+ if (!msg) {\r
+ acm_log(0, "ERROR - unable to allocate send buffer\n");\r
+ return NULL;\r
+ }\r
+\r
+ msg->ep = ep;\r
+ msg->mr = ibv_reg_mr(ep->port->dev->pd, msg->data, size, 0);\r
+ if (!msg->mr) {\r
+ acm_log(0, "ERROR - failed to register send buffer\n");\r
+ goto err;\r
+ }\r
+\r
+ msg->wr.next = NULL;\r
+ msg->wr.sg_list = &msg->sge;\r
+ msg->wr.num_sge = 1;\r
+ msg->wr.opcode = IBV_WR_SEND;\r
+ msg->wr.send_flags = IBV_SEND_SIGNALED;\r
+ msg->wr.wr_id = (uintptr_t) msg;\r
+\r
+ msg->wr.wr.ud.ah = dest->ah;\r
+ msg->wr.wr.ud.remote_qpn = dest->remote_qpn;\r
+ msg->wr.wr.ud.remote_qkey = ACM_QKEY;\r
+\r
+ msg->sge.length = size;\r
+ msg->sge.lkey = msg->mr->lkey;\r
+ msg->sge.addr = (uintptr_t) msg->data;\r
+ return msg;\r
+err:\r
+ free(msg);\r
+ return NULL;\r
+}\r
+\r
+static void acm_free_send(struct acm_send_msg *msg)\r
+{\r
+ ibv_dereg_mr(msg->mr);\r
+ free(msg);\r
+}\r
+\r
+static void acm_post_send(struct acm_send_msg *msg)\r
+{\r
+ struct acm_ep *ep = msg->ep;\r
+ struct ibv_send_wr *bad_wr;\r
+\r
+ if (ep->available_sends) {\r
+ acm_log(2, "posting send to QP\n");\r
+ ep->available_sends--;\r
+ DListInsertTail(&msg->entry, &ep->active_queue);\r
+ ibv_post_send(ep->qp, &msg->wr, &bad_wr);\r
+ } else {\r
+ acm_log(2, "no sends available, queuing message\n");\r
+ DListInsertTail(&msg->entry, &ep->pending_queue);\r
+ }\r
+}\r
+\r
+static void acm_post_recv(struct acm_ep *ep, uint64_t address)\r
+{\r
+ struct ibv_recv_wr wr, *bad_wr;\r
+ struct ibv_sge sge;\r
+\r
+ wr.next = NULL;\r
+ wr.sg_list = &sge;\r
+ wr.num_sge = 1;\r
+ wr.wr_id = address;\r
+\r
+ sge.length = ACM_RECV_SIZE;\r
+ sge.lkey = ep->mr->lkey;\r
+ sge.addr = address;\r
+\r
+ ibv_post_recv(ep->qp, &wr, &bad_wr);\r
+}\r
+\r
+static void acm_send_available(struct acm_ep *ep)\r
+{\r
+ struct acm_send_msg *msg;\r
+ struct ibv_send_wr *bad_wr;\r
+ DLIST_ENTRY *entry;\r
+\r
+ if (DListEmpty(&ep->pending_queue)) {\r
+ ep->available_sends++;\r
+ } else {\r
+ acm_log(2, "posting queued send message\n");\r
+ entry = ep->pending_queue.Next;\r
+ DListRemove(entry);\r
+ msg = container_of(entry, struct acm_send_msg, entry);\r
+ DListInsertTail(&msg->entry, &ep->active_queue);\r
+ ibv_post_send(ep->qp, &msg->wr, &bad_wr);\r
+ }\r
+}\r
+\r
+static void acm_complete_send(struct acm_send_msg *msg)\r
+{\r
+ struct acm_ep *ep = msg->ep;\r
+\r
+ lock_acquire(&ep->lock);\r
+ DListRemove(&msg->entry);\r
+ if (msg->tries) {\r
+ acm_log(2, "waiting for response\n");\r
+ msg->expires = time_stamp_ms() + ep->port->subnet_timeout + timeout;\r
+ DListInsertTail(&msg->entry, &ep->wait_queue);\r
+ if (atomic_inc(&wait_cnt) == 1)\r
+ event_signal(&timeout_event);\r
+ } else {\r
+ acm_log(2, "freeing\n");\r
+ acm_send_available(ep);\r
+ acm_free_send(msg);\r
+ }\r
+ lock_release(&ep->lock);\r
+}\r
+\r
+static struct acm_send_msg *acm_get_request(struct acm_ep *ep, uint64_t tid, int *free)\r
+{\r
+ struct acm_send_msg *msg, *req = NULL;\r
+ struct acm_mad *mad;\r
+ DLIST_ENTRY *entry, *next;\r
+\r
+ acm_log(2, "\n");\r
+ lock_acquire(&ep->lock);\r
+ for (entry = ep->wait_queue.Next; entry != &ep->wait_queue; entry = next) {\r
+ next = entry->Next;\r
+ msg = container_of(entry, struct acm_send_msg, entry);\r
+ mad = (struct acm_mad *) msg->data;\r
+ if (mad->tid == tid) {\r
+ acm_log(2, "match found in wait queue\n");\r
+ req = msg;\r
+ DListRemove(entry);\r
+ (void) atomic_dec(&wait_cnt);\r
+ acm_send_available(ep);\r
+ *free = 1;\r
+ goto unlock;\r
+ }\r
+ }\r
+\r
+ for (entry = ep->active_queue.Next; entry != &ep->active_queue; entry = entry->Next) {\r
+ msg = container_of(entry, struct acm_send_msg, entry);\r
+ mad = (struct acm_mad *) msg->data;\r
+ if (mad->tid == tid && msg->tries) {\r
+ acm_log(2, "match found in active queue\n");\r
+ req = msg;\r
+ req->tries = 0;\r
+ *free = 0;\r
+ break;\r
+ }\r
+ }\r
+unlock:\r
+ lock_release(&ep->lock);\r
+ return req;\r
+}\r
+\r
+static uint8_t acm_gid_index(struct acm_port *port, union ibv_gid *gid)\r
+{\r
+ union ibv_gid cmp_gid;\r
+ uint8_t i;\r
+\r
+ for (i = 0; i < port->gid_cnt; i++) {\r
+ ibv_query_gid(port->dev->verbs, port->port_num, i, &cmp_gid);\r
+ if (!memcmp(&cmp_gid, gid, sizeof cmp_gid))\r
+ break;\r
+ }\r
+ return i;\r
+}\r
+\r
+static int acm_mc_index(struct acm_ep *ep, union ibv_gid *gid)\r
+{\r
+ int i;\r
+\r
+ for (i = 0; i < ep->mc_cnt; i++) {\r
+ if (!memcmp(&ep->mc_dest[i].address, gid, sizeof(*gid)))\r
+ return i;\r
+ }\r
+ return -1;\r
+}\r
+\r
+static void\r
+acm_init_mc_av(struct acm_port *port, struct ib_mc_member_rec *mc_rec,\r
+ struct ibv_ah_attr *av)\r
+{\r
+ uint32_t sl_flow_hop;\r
+\r
+ sl_flow_hop = ntohl(mc_rec->sl_flow_hop);\r
+\r
+ av->dlid = ntohs(mc_rec->mlid);\r
+ av->sl = (uint8_t) (sl_flow_hop >> 28);\r
+ av->src_path_bits = port->sa_dest.av.src_path_bits;\r
+ av->static_rate = mc_rec->rate & 0x3F;\r
+ av->port_num = port->port_num;\r
+\r
+ av->is_global = 1;\r
+ av->grh.dgid = mc_rec->mgid;\r
+ av->grh.flow_label = (sl_flow_hop >> 8) & 0xFFFFF;\r
+ av->grh.sgid_index = acm_gid_index(port, &mc_rec->port_gid);\r
+ av->grh.hop_limit = (uint8_t) sl_flow_hop;\r
+ av->grh.traffic_class = mc_rec->tclass;\r
+}\r
+\r
+static void acm_process_join_resp(struct acm_ep *ep, struct ib_user_mad *umad)\r
+{\r
+ struct acm_dest *dest;\r
+ struct ib_mc_member_rec *mc_rec;\r
+ struct ib_sa_mad *mad;\r
+ int index, ret;\r
+\r
+ mad = (struct ib_sa_mad *) umad->data;\r
+ acm_log(1, "response status: 0x%x, mad status: 0x%x\n",\r
+ umad->status, mad->status);\r
+ if (umad->status) {\r
+ acm_log(0, "ERROR - send join failed 0x%x\n", umad->status);\r
+ return;\r
+ }\r
+ if (mad->status) {\r
+ acm_log(0, "ERROR - join response status 0x%x\n", mad->status);\r
+ return;\r
+ }\r
+\r
+ mc_rec = (struct ib_mc_member_rec *) mad->data;\r
+ lock_acquire(&ep->lock);\r
+ index = acm_mc_index(ep, &mc_rec->mgid);\r
+ if (index >= 0) {\r
+ dest = &ep->mc_dest[index];\r
+ dest->remote_qpn = IB_MC_QPN;\r
+ dest->mgid = mc_rec->mgid;\r
+ acm_init_mc_av(ep->port, mc_rec, &dest->av);\r
+ dest->mtu = mc_rec->mtu & 0x3F;\r
+ dest->packet_lifetime = mc_rec->packet_lifetime & 0x3F;\r
+ dest->ah = ibv_create_ah(ep->port->dev->pd, &dest->av);\r
+ ret = ibv_attach_mcast(ep->qp, &mc_rec->mgid, mc_rec->mlid);\r
+ if (ret) {\r
+ acm_log(0, "ERROR - unable to attach QP to multicast group\n");\r
+ }\r
+ acm_log(1, "join successful\n");\r
+ } else {\r
+ acm_log(0, "ERROR - MGID in join response not found\n");\r
+ }\r
+ lock_release(&ep->lock);\r
+}\r
+\r
+static int acm_compare_dest(const void *dest1, const void *dest2)\r
+{\r
+ return memcmp(dest1, dest2, ACM_MAX_ADDRESS);\r
+}\r
+\r
+static int acm_addr_index(struct acm_ep *ep, uint8_t *addr, uint8_t addr_type)\r
+{\r
+ int i;\r
+\r
+ for (i = 0; i < MAX_EP_ADDR; i++) {\r
+ if (ep->addr_type[i] != addr_type)\r
+ continue;\r
+\r
+ if ((addr_type == ACM_ADDRESS_NAME &&\r
+ !strnicmp((char *) ep->addr[i].name, (char *) addr, ACM_MAX_ADDRESS)) ||\r
+ !memcmp(ep->addr[i].addr, addr, ACM_MAX_ADDRESS))\r
+ return i;\r
+ }\r
+ return -1;\r
+}\r
+\r
+/*\r
+ * Multicast groups are ordered lowest to highest preference.\r
+ */\r
+static int\r
+acm_record_av(struct acm_dest *dest, struct acm_ep *ep,\r
+ struct ibv_wc *wc, struct acm_resolve_rec *rec)\r
+{\r
+ int i, index;\r
+\r
+ acm_log(2, "\n");\r
+ for (i = min(rec->gid_cnt, ACM_MAX_GID_COUNT) - 1; i >= 0; i--) {\r
+ index = acm_mc_index(ep, &rec->gid[i]);\r
+ if (index >= 0) {\r
+ acm_log(2, "selecting MC group at index %d\n", index);\r
+ dest->av = ep->mc_dest[index].av;\r
+ dest->av.dlid = wc->slid;\r
+ dest->av.src_path_bits = wc->dlid_path_bits;\r
+ dest->av.grh.dgid = ((struct ibv_grh *) (uintptr_t) wc->wr_id)->sgid;\r
+ \r
+ dest->mgid = ep->mc_dest[index].mgid;\r
+ dest->mtu = ep->mc_dest[index].mtu;\r
+ dest->packet_lifetime = ep->mc_dest[index].packet_lifetime;\r
+ return ACM_STATUS_SUCCESS;\r
+ }\r
+ }\r
+\r
+ return ACM_STATUS_ENODATA;\r
+}\r
+\r
+/* \r
+ * Record the source of a resolve request. Use the source QPN to see if\r
+ * the remote service has relocated and we need to update our cache.\r
+ */\r
+static struct acm_dest *\r
+acm_record_src(struct acm_ep *ep, struct ibv_wc *wc, struct acm_resolve_rec *rec)\r
+{\r
+ struct acm_dest *dest, **tdest;\r
+ int ret;\r
+\r
+ acm_log(2, "\n");\r
+ lock_acquire(&ep->lock);\r
+ tdest = tfind(rec->src, &ep->dest_map[rec->src_type - 1], acm_compare_dest);\r
+ if (!tdest) {\r
+ acm_log(2, "creating new dest\n");\r
+ dest = zalloc(sizeof *dest);\r
+ if (!dest) {\r
+ acm_log(0, "ERROR - unable to allocate dest\n");\r
+ goto unlock;\r
+ }\r
+\r
+ memcpy(dest->address, rec->src, ACM_MAX_ADDRESS);\r
+ DListInit(&dest->req_queue);\r
+ tsearch(dest, &ep->dest_map[rec->src_type - 1], acm_compare_dest);\r
+ } else {\r
+ dest = *tdest;\r
+ }\r
+\r
+ if (dest->ah) {\r
+ if (dest->remote_qpn == wc->src_qp)\r
+ goto unlock;\r
+\r
+ ibv_destroy_ah(dest->ah); // TODO: ah could be in use\r
+ dest->ah = NULL;\r
+ }\r
+\r
+ acm_log(2, "creating address handle\n");\r
+ ret = acm_record_av(dest, ep, wc, rec);\r
+ if (ret) {\r
+ acm_log(0, "ERROR - failed to record av\n");\r
+ goto err;\r
+ }\r
+\r
+ dest->ah = ibv_create_ah(ep->port->dev->pd, &dest->av);\r
+ if (!dest->ah) {\r
+ acm_log(0, "ERROR - failed to create ah\n");\r
+ goto err;\r
+ }\r
+\r
+ dest->remote_qpn = wc->src_qp;\r
+ dest->init_depth = rec->init_depth;\r
+ dest->resp_resources = rec->resp_resources;\r
+\r
+unlock:\r
+ lock_release(&ep->lock);\r
+ return dest;\r
+\r
+err:\r
+ if (!tdest) {\r
+ tdelete(dest->address, &ep->dest_map[rec->src_type - 1], acm_compare_dest);\r
+ free(dest);\r
+ }\r
+ lock_release(&ep->lock);\r
+ return NULL;\r
+}\r
+\r
+static void acm_init_resp_mad(struct acm_mad *resp, struct acm_mad *req)\r
+{\r
+ resp->base_version = req->base_version;\r
+ resp->mgmt_class = req->mgmt_class;\r
+ resp->class_version = req->class_version;\r
+ resp->method = req->method | IB_METHOD_RESP;\r
+ resp->status = ACM_STATUS_SUCCESS;\r
+ resp->control = req->control;\r
+ resp->tid = req->tid;\r
+}\r
+\r
+static int acm_validate_resolve_req(struct acm_mad *mad)\r
+{\r
+ struct acm_resolve_rec *rec;\r
+\r
+ if (mad->method != IB_METHOD_GET) {\r
+ acm_log(0, "ERROR - invalid method 0x%x\n", mad->method);\r
+ return ACM_STATUS_EINVAL;\r
+ }\r
+\r
+ rec = (struct acm_resolve_rec *) mad->data;\r
+ if (!rec->src_type || rec->src_type >= ACM_ADDRESS_RESERVED) {\r
+ acm_log(0, "ERROR - unknown src type 0x%x\n", rec->src_type);\r
+ return ACM_STATUS_EINVAL;\r
+ }\r
+\r
+ return 0;\r
+}\r
+\r
+static void\r
+acm_process_resolve_req(struct acm_ep *ep, struct ibv_wc *wc, struct acm_mad *mad)\r
+{\r
+ struct acm_resolve_rec *rec, *resp_rec;\r
+ struct acm_dest *dest;\r
+ struct acm_send_msg *msg;\r
+ struct acm_mad *resp_mad;\r
+\r
+ acm_log(2, "\n");\r
+ if (acm_validate_resolve_req(mad)) {\r
+ acm_log(0, "ERROR - invalid request\n");\r
+ return;\r
+ }\r
+\r
+ rec = (struct acm_resolve_rec *) mad->data;\r
+ dest = acm_record_src(ep, wc, rec);\r
+ if (!dest) {\r
+ acm_log(0, "ERROR - failed to record source\n");\r
+ return;\r
+ }\r
+\r
+ if (acm_addr_index(ep, rec->dest, rec->dest_type) < 0) {\r
+ acm_log(2, "no matching address - discarding\n");\r
+ return;\r
+ }\r
+\r
+ msg = acm_alloc_send(ep, dest, sizeof (*resp_mad));\r
+ if (!msg) {\r
+ acm_log(0, "ERROR - failed to allocate message\n");\r
+ return;\r
+ }\r
+\r
+ resp_mad = (struct acm_mad *) msg->data;\r
+ resp_rec = (struct acm_resolve_rec *) resp_mad->data;\r
+ acm_init_resp_mad(resp_mad, mad);\r
+ resp_rec->dest_type = rec->src_type;\r
+ resp_rec->dest_length = rec->src_length;\r
+ resp_rec->src_type = rec->dest_type;\r
+ resp_rec->src_length = rec->dest_length;\r
+ resp_rec->gid_cnt = 1;\r
+ resp_rec->resp_resources = ep->port->dev->resp_resources;\r
+ resp_rec->init_depth = ep->port->dev->init_depth;\r
+ memcpy(resp_rec->dest, rec->src, ACM_MAX_ADDRESS);\r
+ memcpy(resp_rec->src, rec->dest, ACM_MAX_ADDRESS);\r
+ memcpy(resp_rec->gid, dest->mgid.raw, sizeof(union ibv_gid));\r
+\r
+ acm_log(2, "sending resolve response\n");\r
+ lock_acquire(&ep->lock);\r
+ acm_post_send(msg);\r
+ lock_release(&ep->lock);\r
+}\r
+\r
+static int\r
+acm_client_resolve_resp(struct acm_ep *ep, struct acm_client *client,\r
+ struct acm_resolve_msg *msg, struct acm_dest *dest, uint8_t status)\r
+{\r
+ int ret;\r
+\r
+ acm_log(1, "status 0x%x\n", status);\r
+ lock_acquire(&client->lock);\r
+ if (client->sock == INVALID_SOCKET) {\r
+ acm_log(0, "ERROR - connection lost\n");\r
+ ret = ACM_STATUS_ENOTCONN;\r
+ goto release;\r
+ }\r
+\r
+ msg->hdr.opcode |= ACM_OP_ACK;\r
+ msg->hdr.status = status;\r
+ msg->hdr.param = 0;\r
+\r
+ if (!status) {\r
+ msg->hdr.src_type = ACM_EP_TYPE_DEVICE;\r
+ msg->src.dev.guid = ep->port->dev->guid;\r
+ msg->src.dev.pkey_index = ep->pkey_index;\r
+ msg->src.dev.port_num = ep->port->port_num;\r
+\r
+ if (dest) {\r
+ acm_log(2, "destination found\n");\r
+ msg->hdr.dest_type = ACM_EP_TYPE_AV;\r
+ msg->dest.av = dest->av;\r
+ msg->data.init_depth = min(ep->port->dev->init_depth, dest->resp_resources);\r
+ msg->data.resp_resources = min(ep->port->dev->resp_resources, dest->init_depth);\r
+ msg->data.packet_lifetime = dest->packet_lifetime;\r
+ msg->data.mtu = dest->mtu;\r
+ }\r
+ }\r
+\r
+ ret = send(client->sock, (char *) msg, sizeof *msg, 0);\r
+ if (ret != sizeof(*msg))\r
+ acm_log(0, "failed to send response\n");\r
+ else\r
+ ret = 0;\r
+\r
+release:\r
+ lock_release(&client->lock);\r
+ (void) atomic_dec(&client->refcnt);\r
+ return ret;\r
+}\r
+\r
+static struct acm_dest *\r
+acm_record_dest(struct acm_ep *ep, struct ibv_wc *wc,\r
+ struct acm_resolve_rec *req_rec, struct acm_resolve_rec *resp_rec)\r
+{\r
+ struct acm_dest *dest, **tdest;\r
+ int ret;\r
+\r
+ acm_log(2, "\n");\r
+ lock_acquire(&ep->lock);\r
+ tdest = tfind(req_rec->dest, &ep->dest_map[req_rec->dest_type - 1], acm_compare_dest);\r
+ if (!tdest) {\r
+ dest = NULL;\r
+ goto unlock;\r
+ }\r
+\r
+ dest = *tdest;\r
+ if (dest->ah)\r
+ goto unlock;\r
+\r
+ acm_log(2, "creating address handle\n");\r
+ ret = acm_record_av(dest, ep, wc, resp_rec);\r
+ if (ret) {\r
+ acm_log(0, "ERROR - failed to record av\n");\r
+ goto unlock;\r
+ }\r
+\r
+ dest->ah = ibv_create_ah(ep->port->dev->pd, &dest->av);\r
+ if (!dest->ah) {\r
+ acm_log(0, "ERROR - failed to create ah\n");\r
+ goto unlock;\r
+ }\r
+\r
+ dest->remote_qpn = wc->src_qp;\r
+ dest->init_depth = resp_rec->init_depth;\r
+ dest->resp_resources = resp_rec->resp_resources;\r
+\r
+unlock:\r
+ lock_release(&ep->lock);\r
+ return dest;\r
+}\r
+\r
+static void\r
+acm_process_resolve_resp(struct acm_ep *ep, struct ibv_wc *wc,\r
+ struct acm_send_msg *msg, struct acm_mad *mad)\r
+{\r
+ struct acm_resolve_rec *req_rec, *resp_rec;\r
+ struct acm_dest *dest;\r
+ struct acm_request *client_req;\r
+ DLIST_ENTRY *entry;\r
+ uint8_t status;\r
+\r
+ status = acm_class_status(mad->status);\r
+ acm_log(2, "resp status 0x%x\n", status);\r
+ req_rec = (struct acm_resolve_rec *) ((struct acm_mad *) msg->data)->data;\r
+ resp_rec = (struct acm_resolve_rec *) mad->data;\r
+\r
+ dest = acm_record_dest(ep, wc, req_rec, resp_rec);\r
+ if (!dest) {\r
+ acm_log(0, "ERROR - cannot record dest\n");\r
+ return;\r
+ }\r
+\r
+ if (!status && !dest->ah)\r
+ status = ACM_STATUS_EINVAL;\r
+\r
+ lock_acquire(&ep->lock);\r
+ while (!DListEmpty(&dest->req_queue)) {\r
+ entry = dest->req_queue.Next;\r
+ DListRemove(entry);\r
+ client_req = container_of(entry, struct acm_request, entry);\r
+ lock_release(&ep->lock);\r
+ acm_log(2, "completing queued client request\n");\r
+ acm_client_resolve_resp(ep, client_req->client,\r
+ (struct acm_resolve_msg *) &client_req->msg, dest, status);\r
+ lock_acquire(&ep->lock);\r
+ }\r
+ if (status) {\r
+ acm_log(0, "resp failed 0x%x\n", status);\r
+ tdelete(dest->address, &ep->dest_map[req_rec->dest_type - 1], acm_compare_dest);\r
+ }\r
+ lock_release(&ep->lock);\r
+}\r
+\r
+static int acm_validate_recv(struct acm_mad *mad)\r
+{\r
+ if (mad->base_version != 1 || mad->class_version != 1) {\r
+ acm_log(0, "ERROR - invalid version %d %d\n",\r
+ mad->base_version, mad->class_version);\r
+ return ACM_STATUS_EINVAL;\r
+ }\r
+ \r
+ if (mad->mgmt_class != ACM_MGMT_CLASS) {\r
+ acm_log(0, "ERROR - invalid mgmt class 0x%x\n", mad->mgmt_class);\r
+ return ACM_STATUS_EINVAL;\r
+ }\r
+\r
+ if (mad->control != ACM_CTRL_RESOLVE) {\r
+ acm_log(0, "ERROR - invalid control 0x%x\n", mad->control);\r
+ return ACM_STATUS_EINVAL;\r
+ }\r
+\r
+ return 0;\r
+}\r
+\r
+static void acm_process_recv(struct acm_ep *ep, struct ibv_wc *wc)\r
+{\r
+ struct acm_mad *mad;\r
+ struct acm_send_msg *req;\r
+ int free;\r
+\r
+ acm_log(2, "\n");\r
+ mad = (struct acm_mad *) (uintptr_t) (wc->wr_id + sizeof(struct ibv_grh));\r
+ if (acm_validate_recv(mad)) {\r
+ acm_log(0, "ERROR - discarding message\n");\r
+ goto out;\r
+ }\r
+\r
+ if (mad->method & IB_METHOD_RESP) {\r
+ acm_log(2, "received response\n");\r
+ req = acm_get_request(ep, mad->tid, &free);\r
+ if (!req) {\r
+ acm_log(0, "response did not match active request\n");\r
+ goto out;\r
+ }\r
+ acm_log(2, "found matching request\n");\r
+ acm_process_resolve_resp(ep, wc, req, mad);\r
+ if (free)\r
+ acm_free_send(req);\r
+ } else {\r
+ acm_log(2, "unsolicited request\n");\r
+ acm_process_resolve_req(ep, wc, mad);\r
+ free = 0;\r
+ }\r
+\r
+out:\r
+ acm_post_recv(ep, wc->wr_id);\r
+}\r
+\r
+static void acm_process_comp(struct acm_ep *ep, struct ibv_wc *wc)\r
+{\r
+ if (wc->status) {\r
+ acm_log(0, "ERROR - work completion error\n"\r
+ "\topcode %d, completion status %d\n",\r
+ wc->opcode, wc->status);\r
+ return;\r
+ }\r
+\r
+ if (wc->opcode & IBV_WC_RECV)\r
+ acm_process_recv(ep, wc);\r
+ else\r
+ acm_complete_send((struct acm_send_msg *) (uintptr_t) wc->wr_id);\r
+}\r
+\r
+static void CDECL_FUNC acm_comp_handler(void *context)\r
+{\r
+ struct acm_device *dev = (struct acm_device *) context;\r
+ struct acm_ep *ep;\r
+ struct ibv_cq *cq;\r
+ struct ibv_wc wc;\r
+ int cnt;\r
+\r
+ acm_log(1, "started\n");\r
+ while (1) {\r
+ ibv_get_cq_event(dev->channel, &cq, (void *) &ep);\r
+\r
+ cnt = 0;\r
+ while (ibv_poll_cq(cq, 1, &wc) > 0) {\r
+ cnt++;\r
+ acm_process_comp(ep, &wc);\r
+ }\r
+\r
+ ibv_req_notify_cq(cq, 0);\r
+ while (ibv_poll_cq(cq, 1, &wc) > 0) {\r
+ cnt++;\r
+ acm_process_comp(ep, &wc);\r
+ }\r
+\r
+ ibv_ack_cq_events(cq, cnt);\r
+ }\r
+}\r
+\r
+static void acm_format_mgid(union ibv_gid *mgid, uint16_t pkey, uint8_t tos,\r
+ uint8_t rate, uint8_t mtu)\r
+{\r
+ mgid->raw[0] = 0xFF;\r
+ mgid->raw[1] = 0x10 | 0x05;\r
+ mgid->raw[2] = 0x40;\r
+ mgid->raw[3] = 0x01;\r
+ mgid->raw[4] = (uint8_t) (pkey >> 8);\r
+ mgid->raw[5] = (uint8_t) pkey;\r
+ mgid->raw[6] = tos;\r
+ mgid->raw[7] = rate;\r
+ mgid->raw[8] = mtu;\r
+ mgid->raw[9] = 0;\r
+ mgid->raw[10] = 0;\r
+ mgid->raw[11] = 0;\r
+ mgid->raw[12] = 0;\r
+ mgid->raw[13] = 0;\r
+ mgid->raw[14] = 0;\r
+ mgid->raw[15] = 0;\r
+}\r
+\r
+static void acm_init_path_query(struct ib_sa_mad *mad, struct ib_path_record *path)\r
+{\r
+ uint32_t fl_hop;\r
+ uint16_t qos_sl;\r
+\r
+ acm_log(2, "\n");\r
+ mad->base_version = 1;\r
+ mad->mgmt_class = IB_MGMT_CLASS_SA;\r
+ mad->class_version = 2;\r
+ mad->method = IB_METHOD_GET;\r
+ mad->tid = (uint64_t) atomic_inc(&tid);\r
+ mad->attr_id = IB_SA_ATTR_PATH_REC;\r
+\r
+ memcpy(mad->data, path, sizeof(*path));\r
+ if (path->service_id)\r
+ mad->comp_mask |= IB_COMP_MASK_PR_SERVICE_ID;\r
+ if (path->dgid.global.interface_id || path->dgid.global.subnet_prefix)\r
+ mad->comp_mask |= IB_COMP_MASK_PR_DGID;\r
+ if (path->sgid.global.interface_id || path->sgid.global.subnet_prefix)\r
+ mad->comp_mask |= IB_COMP_MASK_PR_SGID;\r
+ if (path->dlid)\r
+ mad->comp_mask |= IB_COMP_MASK_PR_DLID;\r
+ if (path->slid)\r
+ mad->comp_mask |= IB_COMP_MASK_PR_SLID;\r
+\r
+ fl_hop = ntohl(path->flowlabel_hoplimit);\r
+ if (fl_hop >> 8)\r
+ mad->comp_mask |= IB_COMP_MASK_PR_FLOW_LABEL;\r
+ if (fl_hop & 0xFF)\r
+ mad->comp_mask |= IB_COMP_MASK_PR_HOP_LIMIT;\r
+\r
+ if (path->tclass)\r
+ mad->comp_mask |= IB_COMP_MASK_PR_TCLASS;\r
+ if (path->reversible_numpath & 0x80)\r
+ mad->comp_mask |= IB_COMP_MASK_PR_REVERSIBLE;\r
+ if (path->pkey)\r
+ mad->comp_mask |= IB_COMP_MASK_PR_PKEY;\r
+\r
+ qos_sl = ntohs(path->qosclass_sl);\r
+ if (qos_sl >> 4)\r
+ mad->comp_mask |= IB_COMP_MASK_PR_QOS_CLASS;\r
+ if (qos_sl & 0xF)\r
+ mad->comp_mask |= IB_COMP_MASK_PR_SL;\r
+\r
+ if (path->mtu & 0xC0)\r
+ mad->comp_mask |= IB_COMP_MASK_PR_MTU_SELECTOR;\r
+ if (path->mtu & 0x3F)\r
+ mad->comp_mask |= IB_COMP_MASK_PR_MTU;\r
+ if (path->rate & 0xC0)\r
+ mad->comp_mask |= IB_COMP_MASK_PR_RATE_SELECTOR;\r
+ if (path->rate & 0x3F)\r
+ mad->comp_mask |= IB_COMP_MASK_PR_RATE;\r
+ if (path->packetlifetime & 0xC0)\r
+ mad->comp_mask |= IB_COMP_MASK_PR_PACKET_LIFETIME_SELECTOR;\r
+ if (path->packetlifetime & 0x3F)\r
+ mad->comp_mask |= IB_COMP_MASK_PR_PACKET_LIFETIME;\r
+}\r
+\r
+static void acm_init_join(struct ib_sa_mad *mad, union ibv_gid *port_gid,\r
+ uint16_t pkey, uint8_t tos, uint8_t tclass, uint8_t sl, uint8_t rate, uint8_t mtu)\r
+{\r
+ struct ib_mc_member_rec *mc_rec;\r
+\r
+ acm_log(2, "\n");\r
+ mad->base_version = 1;\r
+ mad->mgmt_class = IB_MGMT_CLASS_SA;\r
+ mad->class_version = 2;\r
+ mad->method = IB_METHOD_SET;\r
+ mad->tid = (uint64_t) atomic_inc(&tid);\r
+ mad->attr_id = IB_SA_ATTR_MC_MEMBER_REC;\r
+ mad->comp_mask =\r
+ IB_COMP_MASK_MC_MGID | IB_COMP_MASK_MC_PORT_GID |\r
+ IB_COMP_MASK_MC_QKEY | IB_COMP_MASK_MC_MTU_SEL| IB_COMP_MASK_MC_MTU |\r
+ IB_COMP_MASK_MC_TCLASS | IB_COMP_MASK_MC_PKEY | IB_COMP_MASK_MC_RATE_SEL |\r
+ IB_COMP_MASK_MC_RATE | IB_COMP_MASK_MC_SL | IB_COMP_MASK_MC_FLOW |\r
+ IB_COMP_MASK_MC_SCOPE | IB_COMP_MASK_MC_JOIN_STATE;\r
+\r
+ mc_rec = (struct ib_mc_member_rec *) mad->data;\r
+ acm_format_mgid(&mc_rec->mgid, pkey, tos, rate, mtu);\r
+ mc_rec->port_gid = *port_gid;\r
+ mc_rec->qkey = ACM_QKEY;\r
+ mc_rec->mtu = 0x80 | mtu;\r
+ mc_rec->tclass = tclass;\r
+ mc_rec->pkey = htons(pkey);\r
+ mc_rec->rate = 0x80 | rate;\r
+ mc_rec->sl_flow_hop = htonl(((uint32_t) sl) << 28);\r
+ mc_rec->scope_state = 0x51;\r
+}\r
+\r
+static void acm_join_group(struct acm_ep *ep, union ibv_gid *port_gid,\r
+ uint8_t tos, uint8_t tclass, uint8_t sl, uint8_t rate, uint8_t mtu)\r
+{\r
+ struct acm_port *port;\r
+ struct ib_sa_mad *mad;\r
+ struct ib_user_mad *umad;\r
+ struct ib_mc_member_rec *mc_rec;\r
+ int ret, len;\r
+\r
+ acm_log(2, "\n");\r
+ len = sizeof(*umad) + sizeof(*mad);\r
+ umad = (struct ib_user_mad *) zalloc(len);\r
+ if (!umad) {\r
+ acm_log(0, "ERROR - unable to allocate MAD for join\n");\r
+ return;\r
+ }\r
+\r
+ port = ep->port;\r
+ umad->addr.qpn = htonl(port->sa_dest.remote_qpn);\r
+ umad->addr.qkey = htonl(ACM_QKEY);\r
+ umad->addr.pkey_index = ep->pkey_index;\r
+ umad->addr.lid = htons(port->sa_dest.av.dlid);\r
+ umad->addr.sl = port->sa_dest.av.sl;\r
+ umad->addr.path_bits = port->sa_dest.av.src_path_bits;\r
+\r
+ acm_log(0, "%s %d pkey 0x%x, sl 0x%x, rate 0x%x, mtu 0x%x\n",\r
+ ep->port->dev->verbs->device->name, ep->port->port_num,\r
+ ep->pkey, sl, rate, mtu);\r
+ mad = (struct ib_sa_mad *) umad->data;\r
+ acm_init_join(mad, port_gid, ep->pkey, tos, tclass, sl, rate, mtu);\r
+ mc_rec = (struct ib_mc_member_rec *) mad->data;\r
+ memcpy(&ep->mc_dest[ep->mc_cnt++], &mc_rec->mgid, sizeof(mc_rec->mgid));\r
+\r
+ ret = umad_send(port->mad_portid, port->mad_agentid, (void *) umad,\r
+ sizeof(*mad), timeout, retries);\r
+ if (ret) {\r
+ acm_log(0, "ERROR - failed to send multicast join request %d\n", ret);\r
+ goto out;\r
+ }\r
+\r
+ acm_log(1, "waiting for response from SA to join request\n");\r
+ ret = umad_recv(port->mad_portid, (void *) umad, &len, -1);\r
+ if (ret < 0) {\r
+ acm_log(0, "ERROR - recv error for multicast join response %d\n", ret);\r
+ goto out;\r
+ }\r
+\r
+ acm_process_join_resp(ep, umad);\r
+out:\r
+ free(umad);\r
+}\r
+\r
+static void acm_port_join(void *context)\r
+{\r
+ struct acm_device *dev;\r
+ struct acm_port *port = (struct acm_port *) context;\r
+ struct acm_ep *ep;\r
+ union ibv_gid port_gid;\r
+ DLIST_ENTRY *ep_entry;\r
+ int ret;\r
+\r
+ dev = port->dev;\r
+ acm_log(1, "device %s port %d\n", dev->verbs->device->name,\r
+ port->port_num);\r
+\r
+ ret = ibv_query_gid(dev->verbs, port->port_num, 0, &port_gid);\r
+ if (ret) {\r
+ acm_log(0, "ERROR - ibv_query_gid %d device %s port %d\n",\r
+ ret, dev->verbs->device->name, port->port_num);\r
+ return;\r
+ }\r
+\r
+ for (ep_entry = port->ep_list.Next; ep_entry != &port->ep_list;\r
+ ep_entry = ep_entry->Next) {\r
+\r
+ ep = container_of(ep_entry, struct acm_ep, entry);\r
+ acm_join_group(ep, &port_gid, 0, 0, 0, min_rate, min_mtu);\r
+ if (port->rate != min_rate || port->mtu != min_mtu)\r
+ acm_join_group(ep, &port_gid, 0, 0, 0, port->rate, port->mtu);\r
+ }\r
+ acm_log(1, "joins for device %s port %d complete\n", dev->verbs->device->name,\r
+ port->port_num);\r
+}\r
+\r
+static void acm_join_groups(void)\r
+{\r
+ struct acm_device *dev;\r
+ struct acm_port *port;\r
+ DLIST_ENTRY *dev_entry;\r
+ int i;\r
+\r
+ acm_log(1, "initiating multicast joins for all ports\n");\r
+ for (dev_entry = dev_list.Next; dev_entry != &dev_list;\r
+ dev_entry = dev_entry->Next) {\r
+\r
+ dev = container_of(dev_entry, struct acm_device, entry);\r
+\r
+ for (i = 0; i < dev->port_cnt; i++) {\r
+ port = &dev->port[i];\r
+ if (port->state != IBV_PORT_ACTIVE)\r
+ continue;\r
+\r
+ acm_log(1, "starting join for device %s, port %d\n",\r
+ dev->verbs->device->name, port->port_num);\r
+ // TODO: handle dynamic changes\r
+ //beginthread(acm_port_join, port);\r
+ acm_port_join(port);\r
+ }\r
+ }\r
+}\r
+\r
+static void acm_process_timeouts(void)\r
+{\r
+ DLIST_ENTRY *entry;\r
+ struct acm_send_msg *msg;\r
+ struct acm_mad *mad;\r
+ struct acm_resolve_rec *rec;\r
+ struct acm_dest *dest, **tdest;\r
+ struct acm_request *req;\r
+ struct acm_ep *ep;\r
+ \r
+ while (!DListEmpty(&timeout_list)) {\r
+ entry = timeout_list.Next;\r
+ DListRemove(entry);\r
+\r
+ msg = container_of(entry, struct acm_send_msg, entry);\r
+ mad = (struct acm_mad *) msg->data;\r
+\r
+ rec = (struct acm_resolve_rec *) mad->data;\r
+ ep = msg->ep;\r
+\r
+ acm_log_ep_addr(0, "acm_process_timeouts: dest ",\r
+ (union acm_ep_addr *) &rec->dest, rec->dest_type);\r
+ lock_acquire(&ep->lock);\r
+ tdest = tfind(rec->dest, &ep->dest_map[rec->dest_type - 1], acm_compare_dest);\r
+ if (!tdest) {\r
+ acm_log(0, "destination already removed\n");\r
+ lock_release(&ep->lock);\r
+ continue;\r
+ } else {\r
+ dest = *tdest;\r
+ }\r
+\r
+ acm_log(2, "failing pending client requests\n");\r
+ while (!DListEmpty(&dest->req_queue)) {\r
+ entry = dest->req_queue.Next;\r
+ DListRemove(entry);\r
+ \r
+ req = container_of(entry, struct acm_request, entry);\r
+ lock_release(&ep->lock);\r
+ acm_client_resolve_resp(ep, req->client,\r
+ (struct acm_resolve_msg *) &req->msg, dest,\r
+ ACM_STATUS_ETIMEDOUT);\r
+ lock_acquire(&ep->lock);\r
+ }\r
+\r
+ acm_log(2, "resolve timed out, releasing destination\n");\r
+ tdelete(dest->address, &ep->dest_map[rec->dest_type - 1], acm_compare_dest);\r
+ lock_release(&ep->lock);\r
+ }\r
+}\r
+\r
+static void acm_process_wait_queue(struct acm_ep *ep, uint64_t *next_expire)\r
+{\r
+ struct acm_send_msg *msg;\r
+ DLIST_ENTRY *entry, *next;\r
+ struct ibv_send_wr *bad_wr;\r
+\r
+ for (entry = ep->wait_queue.Next; entry != &ep->wait_queue; entry = next) {\r
+ next = entry->Next;\r
+ msg = container_of(entry, struct acm_send_msg, entry);\r
+ if (msg->expires < time_stamp_ms()) {\r
+ DListRemove(entry);\r
+ (void) atomic_dec(&wait_cnt);\r
+ if (--msg->tries) {\r
+ acm_log(2, "retrying request\n");\r
+ DListInsertTail(&msg->entry, &ep->active_queue);\r
+ ibv_post_send(ep->qp, &msg->wr, &bad_wr);\r
+ } else {\r
+ acm_log(0, "failing request\n");\r
+ acm_send_available(ep);\r
+ DListInsertTail(&msg->entry, &timeout_list);\r
+ }\r
+ } else {\r
+ *next_expire = min(*next_expire, msg->expires);\r
+ break;\r
+ }\r
+ }\r
+}\r
+\r
+static void CDECL_FUNC acm_retry_handler(void *context)\r
+{\r
+ struct acm_device *dev;\r
+ struct acm_port *port;\r
+ struct acm_ep *ep;\r
+ DLIST_ENTRY *dev_entry, *ep_entry;\r
+ uint64_t next_expire;\r
+ int i, wait;\r
+\r
+ acm_log(0, "started\n");\r
+ while (1) {\r
+ while (!atomic_get(&wait_cnt))\r
+ event_wait(&timeout_event, -1);\r
+\r
+ next_expire = -1;\r
+ for (dev_entry = dev_list.Next; dev_entry != &dev_list;\r
+ dev_entry = dev_entry->Next) {\r
+\r
+ dev = container_of(dev_entry, struct acm_device, entry);\r
+\r
+ for (i = 0; i < dev->port_cnt; i++) {\r
+ port = &dev->port[i];\r
+\r
+ for (ep_entry = port->ep_list.Next;\r
+ ep_entry != &port->ep_list;\r
+ ep_entry = ep_entry->Next) {\r
+\r
+ ep = container_of(ep_entry, struct acm_ep, entry);\r
+ lock_acquire(&ep->lock);\r
+ if (!DListEmpty(&ep->wait_queue))\r
+ acm_process_wait_queue(ep, &next_expire);\r
+ lock_release(&ep->lock);\r
+ }\r
+ }\r
+ }\r
+\r
+ acm_process_timeouts();\r
+ wait = (int) (next_expire - time_stamp_ms());\r
+ if (wait > 0 && atomic_get(&wait_cnt))\r
+ event_wait(&timeout_event, wait);\r
+ }\r
+}\r
+\r
+static void acm_init_server(void)\r
+{\r
+ int i;\r
+\r
+ for (i = 0; i < FD_SETSIZE - 1; i++) {\r
+ lock_init(&client[i].lock);\r
+ client[i].index = i;\r
+ client[i].sock = INVALID_SOCKET;\r
+ }\r
+}\r
+\r
+static int acm_listen(void)\r
+{\r
+ struct sockaddr_in addr;\r
+ int ret;\r
+\r
+ acm_log(2, "\n");\r
+ listen_socket = socket(AF_INET, SOCK_STREAM, IPPROTO_TCP);\r
+ if (listen_socket == INVALID_SOCKET) {\r
+ acm_log(0, "ERROR - unable to allocate listen socket\n");\r
+ return socket_errno();\r
+ }\r
+\r
+ memset(&addr, 0, sizeof addr);\r
+ addr.sin_family = AF_INET;\r
+ addr.sin_port = htons(server_port);\r
+ ret = bind(listen_socket, (struct sockaddr *) &addr, sizeof addr);\r
+ if (ret == SOCKET_ERROR) {\r
+ acm_log(0, "ERROR - unable to bind listen socket\n");\r
+ return socket_errno();\r
+ }\r
+ \r
+ ret = listen(listen_socket, 0);\r
+ if (ret == SOCKET_ERROR) {\r
+ acm_log(0, "ERROR - unable to start listen\n");\r
+ return socket_errno();\r
+ }\r
+\r
+ acm_log(2, "listen active\n");\r
+ return 0;\r
+}\r
+\r
+static void acm_release_client(struct acm_client *client)\r
+{\r
+ lock_acquire(&client->lock);\r
+ shutdown(client->sock, SHUT_RDWR);\r
+ closesocket(client->sock);\r
+ client->sock = INVALID_SOCKET;\r
+ lock_release(&client->lock);\r
+ (void) atomic_dec(&client->refcnt);\r
+}\r
+\r
+static void acm_svr_accept(void)\r
+{\r
+ SOCKET s;\r
+ int i;\r
+\r
+ acm_log(2, "\n");\r
+ s = accept(listen_socket, NULL, NULL);\r
+ if (s == INVALID_SOCKET) {\r
+ acm_log(0, "ERROR - failed to accept connection\n");\r
+ return;\r
+ }\r
+\r
+ for (i = 0; i < FD_SETSIZE - 1; i++) {\r
+ if (!atomic_get(&client[i].refcnt))\r
+ break;\r
+ }\r
+\r
+ if (i == FD_SETSIZE - 1) {\r
+ acm_log(0, "all connections busy - rejecting\n");\r
+ closesocket(s);\r
+ return;\r
+ }\r
+\r
+ client[i].sock = s;\r
+ atomic_set(&client[i].refcnt, 1);\r
+ acm_log(2, "assigned client id %d\n", i);\r
+}\r
+\r
+static uint8_t acm_get_addr_type(uint8_t ep_type)\r
+{\r
+ if (ep_type >= ACM_ADDRESS_RESERVED) {\r
+ acm_log(0, "ERROR - invalid ep type %d\n", ep_type);\r
+ return ACM_ADDRESS_INVALID;\r
+ }\r
+ return ep_type;\r
+}\r
+\r
+static int\r
+acm_client_query_resp(struct acm_ep *ep, struct acm_client *client,\r
+ struct acm_query_msg *msg, uint8_t status)\r
+{\r
+ int ret;\r
+\r
+ acm_log(1, "status 0x%x\n", status);\r
+ lock_acquire(&client->lock);\r
+ if (client->sock == INVALID_SOCKET) {\r
+ acm_log(0, "ERROR - connection lost\n");\r
+ ret = ACM_STATUS_ENOTCONN;\r
+ goto release;\r
+ }\r
+\r
+ msg->hdr.opcode |= ACM_OP_ACK;\r
+ msg->hdr.status = status;\r
+\r
+ ret = send(client->sock, (char *) msg, sizeof *msg, 0);\r
+ if (ret != sizeof(*msg))\r
+ acm_log(0, "failed to send response\n");\r
+ else\r
+ ret = 0;\r
+\r
+release:\r
+ lock_release(&client->lock);\r
+ (void) atomic_dec(&client->refcnt);\r
+ return ret;\r
+}\r
+\r
+static struct acm_ep *\r
+acm_get_ep_by_path(struct ib_path_record *path)\r
+{\r
+ struct acm_device *dev;\r
+ struct acm_port *port;\r
+ struct acm_ep *ep;\r
+ DLIST_ENTRY *dev_entry, *ep_entry;\r
+ int i;\r
+\r
+ for (dev_entry = dev_list.Next; dev_entry != &dev_list;\r
+ dev_entry = dev_entry->Next) {\r
+\r
+ dev = container_of(dev_entry, struct acm_device, entry);\r
+ for (i = 0; i < dev->port_cnt; i++) {\r
+ port = &dev->port[i];\r
+\r
+ // requires slid\r
+ if (port->lid != ntohs(path->slid))\r
+ continue;\r
+\r
+ for (ep_entry = port->ep_list.Next; ep_entry != &port->ep_list;\r
+ ep_entry = ep_entry->Next) {\r
+\r
+ // ignores pkey\r
+ ep = container_of(ep_entry, struct acm_ep, entry);\r
+ return ep;\r
+ }\r
+ }\r
+ }\r
+\r
+ acm_log(0, "could not find endpoint\n");\r
+ return NULL;\r
+}\r
+\r
+// TODO: process send/recv asynchronously\r
+static uint8_t acm_query_sa(struct acm_ep *ep, uint8_t query, union acm_query_data *data)\r
+{\r
+ struct acm_port *port;\r
+ struct ib_sa_mad *mad;\r
+ struct ib_user_mad *umad;\r
+ int ret, len;\r
+ size_t size;\r
+\r
+ acm_log(2, "\n");\r
+ len = sizeof(*umad) + sizeof(*mad);\r
+ umad = (struct ib_user_mad *) zalloc(len);\r
+ if (!umad) {\r
+ acm_log(0, "ERROR - unable to allocate MAD\n");\r
+ return ACM_STATUS_ENOMEM;\r
+ }\r
+\r
+ port = ep->port;\r
+ umad->addr.qpn = htonl(port->sa_dest.remote_qpn);\r
+ umad->addr.qkey = htonl(ACM_QKEY);\r
+ umad->addr.pkey_index = ep->pkey_index;\r
+ umad->addr.lid = htons(port->sa_dest.av.dlid);\r
+ umad->addr.sl = port->sa_dest.av.sl;\r
+ umad->addr.path_bits = port->sa_dest.av.src_path_bits;\r
+\r
+ mad = (struct ib_sa_mad *) umad->data;\r
+ switch (query) {\r
+ case ACM_QUERY_PATH_RECORD:\r
+ acm_init_path_query(mad, &data->path);\r
+ size = sizeof(data->path);\r
+ break;\r
+ default:\r
+ acm_log(0, "ERROR - unknown attribute id\n");\r
+ ret = ACM_STATUS_EINVAL;\r
+ goto out;\r
+ }\r
+\r
+ ret = umad_send(port->mad_portid, port->mad_agentid, (void *) umad,\r
+ sizeof(*mad), timeout, retries);\r
+ if (ret) {\r
+ acm_log(0, "ERROR - umad_send %d\n", ret);\r
+ goto out;\r
+ }\r
+\r
+ acm_log(2, "waiting to receive SA response\n");\r
+ ret = umad_recv(port->mad_portid, (void *) umad, &len, -1);\r
+ if (ret < 0) {\r
+ acm_log(0, "ERROR - umad_recv %d\n", ret);\r
+ goto out;\r
+ }\r
+\r
+ memcpy(data, mad->data, size);\r
+ ret = umad->status ? umad->status : mad->status;\r
+ if (ret) {\r
+ acm_log(0, "SA query response error: 0x%x\n", ret);\r
+ ret = ((uint8_t) ret) ? ret : -1;\r
+ }\r
+out:\r
+ free(umad);\r
+ return (uint8_t) ret;\r
+}\r
+\r
+static int\r
+acm_svr_query(struct acm_client *client, struct acm_query_msg *msg)\r
+{\r
+ struct acm_ep *ep;\r
+ uint8_t status;\r
+\r
+ acm_log(2, "processing client query\n");\r
+ ep = acm_get_ep_by_path(&msg->data.path);\r
+ if (!ep) {\r
+ acm_log(0, "could not find local end point\n");\r
+ status = ACM_STATUS_ESRCADDR;\r
+ goto resp;\r
+ }\r
+\r
+ (void) atomic_inc(&client->refcnt);\r
+ lock_acquire(&ep->lock);\r
+ status = acm_query_sa(ep, msg->hdr.param & ~ACM_QUERY_SA, &msg->data);\r
+ lock_release(&ep->lock);\r
+\r
+resp:\r
+ return acm_client_query_resp(ep, client, msg, status);\r
+}\r
+\r
+static uint8_t\r
+acm_send_resolve(struct acm_ep *ep, union acm_ep_addr *src, uint8_t src_type,\r
+ struct acm_dest *dest, uint8_t dest_type)\r
+{\r
+ struct acm_send_msg *msg;\r
+ struct acm_mad *mad;\r
+ struct acm_resolve_rec *rec;\r
+ int i;\r
+\r
+ acm_log(2, "\n");\r
+ if (!ep->mc_dest[0].ah) {\r
+ acm_log(0, "ERROR - multicast group not ready\n");\r
+ return ACM_STATUS_ENOTCONN;\r
+ }\r
+\r
+ msg = acm_alloc_send(ep, &ep->mc_dest[0], sizeof(struct acm_mad));\r
+ if (!msg) {\r
+ acm_log(0, "ERROR - cannot allocate send msg\n");\r
+ return ACM_STATUS_ENOMEM;\r
+ }\r
+\r
+ msg->tries = retries + 1;\r
+ mad = (struct acm_mad *) msg->data;\r
+ mad->base_version = 1;\r
+ mad->mgmt_class = ACM_MGMT_CLASS;\r
+ mad->class_version = 1;\r
+ mad->method = IB_METHOD_GET;\r
+ mad->control = ACM_CTRL_RESOLVE;\r
+ mad->tid = (uint64_t) atomic_inc(&tid);\r
+\r
+ rec = (struct acm_resolve_rec *) mad->data;\r
+ rec->src_type = src_type;\r
+ rec->src_length = ACM_MAX_ADDRESS;\r
+ memcpy(rec->src, src->addr, ACM_MAX_ADDRESS);\r
+ rec->dest_type = dest_type;\r
+ rec->dest_length = ACM_MAX_ADDRESS;\r
+ memcpy(rec->dest, dest->address, ACM_MAX_ADDRESS);\r
+ rec->resp_resources = ep->port->dev->resp_resources;\r
+ rec->init_depth = ep->port->dev->init_depth;\r
+\r
+ rec->gid_cnt = (uint8_t) ep->mc_cnt;\r
+ for (i = 0; i < ep->mc_cnt; i++)\r
+ memcpy(&rec->gid[i], ep->mc_dest[i].address, 16);\r
+ \r
+ acm_post_send(msg);\r
+ return 0;\r
+}\r
+\r
+static struct acm_ep *\r
+acm_get_ep_by_addr(union acm_ep_addr *addr, uint8_t src_type)\r
+{\r
+ struct acm_device *dev;\r
+ struct acm_port *port;\r
+ struct acm_ep *ep;\r
+ DLIST_ENTRY *dev_entry, *ep_entry;\r
+ int i;\r
+\r
+ acm_log_ep_addr(2, "acm_get_ep_by_addr: ", addr, src_type);\r
+ for (dev_entry = dev_list.Next; dev_entry != &dev_list;\r
+ dev_entry = dev_entry->Next) {\r
+\r
+ dev = container_of(dev_entry, struct acm_device, entry);\r
+ for (i = 0; i < dev->port_cnt; i++) {\r
+ port = &dev->port[i];\r
+\r
+ for (ep_entry = port->ep_list.Next; ep_entry != &port->ep_list;\r
+ ep_entry = ep_entry->Next) {\r
+\r
+ ep = container_of(ep_entry, struct acm_ep, entry);\r
+ if (acm_addr_index(ep, addr->addr, src_type) >= 0)\r
+ return ep;\r
+ }\r
+ }\r
+ }\r
+\r
+ acm_log_ep_addr(0, "acm_get_ep_by_addr: could not find ", addr, src_type);\r
+ return NULL;\r
+}\r
+\r
+static int\r
+acm_svr_resolve(struct acm_client *client, struct acm_resolve_msg *msg)\r
+{\r
+ struct acm_ep *ep;\r
+ struct acm_dest *dest, **tdest;\r
+ struct acm_request *req;\r
+ uint8_t dest_type, src_type;\r
+ uint8_t status;\r
+\r
+ acm_log_ep_addr(2, "acm_svr_resolve: source ", &msg->src, msg->hdr.src_type);\r
+ ep = acm_get_ep_by_addr(&msg->src, msg->hdr.src_type);\r
+ if (!ep) {\r
+ acm_log(0, "unknown local end point\n");\r
+ status = ACM_STATUS_ESRCADDR;\r
+ goto resp;\r
+ }\r
+\r
+ dest_type = acm_get_addr_type(msg->hdr.dest_type);\r
+ if (dest_type == ACM_ADDRESS_INVALID) {\r
+ acm_log(0, "ERROR - unknown destination type\n");\r
+ status = ACM_STATUS_EDESTTYPE;\r
+ goto resp;\r
+ }\r
+\r
+ acm_log_ep_addr(2, "acm_svr_resolve: dest ", &msg->dest, msg->hdr.dest_type);\r
+ (void) atomic_inc(&client->refcnt);\r
+ lock_acquire(&ep->lock);\r
+ tdest = tfind(msg->dest.addr, &ep->dest_map[dest_type - 1], acm_compare_dest);\r
+ dest = tdest ? *tdest : NULL;\r
+ if (dest && dest->ah) {\r
+ acm_log(2, "request satisfied from local cache\n");\r
+ status = ACM_STATUS_SUCCESS;\r
+ goto release;\r
+ }\r
+\r
+ req = zalloc(sizeof *req);\r
+ if (!req) {\r
+ acm_log(0, "ERROR - unable to allocate memory to queue client request\n");\r
+ status = ACM_STATUS_ENOMEM;\r
+ goto release;\r
+ }\r
+\r
+ if (!dest) {\r
+ acm_log(2, "adding new destination\n");\r
+ dest = zalloc(sizeof *dest);\r
+ if (!dest) {\r
+ acm_log(0, "ERROR - unable to allocate destination in client request\n");\r
+ status = ACM_STATUS_ENOMEM;\r
+ goto free_req;\r
+ }\r
+\r
+ memcpy(dest->address, msg->dest.addr, ACM_MAX_ADDRESS);\r
+ src_type = acm_get_addr_type(msg->hdr.src_type);\r
+ acm_log(2, "sending resolve msg to dest\n");\r
+ status = acm_send_resolve(ep, &msg->src, src_type, dest, dest_type);\r
+ if (status) {\r
+ acm_log(0, "ERROR - failure sending resolve request 0x%x\n", status);\r
+ goto free_dest;\r
+ }\r
+\r
+ DListInit(&dest->req_queue);\r
+ tsearch(dest, &ep->dest_map[dest_type - 1], acm_compare_dest);\r
+ }\r
+\r
+ acm_log(2, "queuing client request\n");\r
+ req->client = client;\r
+ memcpy(&req->msg, msg, sizeof(req->msg));\r
+ DListInsertTail(&req->entry, &dest->req_queue);\r
+ lock_release(&ep->lock);\r
+ return 0;\r
+\r
+free_dest:\r
+ free(dest);\r
+ dest = NULL;\r
+free_req:\r
+ free(req);\r
+release:\r
+ lock_release(&ep->lock);\r
+resp:\r
+ return acm_client_resolve_resp(ep, client, msg, dest, status);\r
+}\r
+\r
+static void acm_svr_receive(struct acm_client *client)\r
+{\r
+ struct acm_msg msg;\r
+ int ret;\r
+\r
+ acm_log(2, "\n");\r
+ ret = recv(client->sock, (char *) &msg, sizeof msg, 0);\r
+ if (ret != sizeof msg) {\r
+ acm_log(2, "client disconnected\n");\r
+ ret = ACM_STATUS_ENOTCONN;\r
+ goto out;\r
+ }\r
+\r
+ if (msg.hdr.version != ACM_VERSION) {\r
+ acm_log(0, "ERROR - unsupported version %d\n", msg.hdr.version);\r
+ goto out;\r
+ }\r
+\r
+ switch (msg.hdr.opcode & ACM_OP_MASK) {\r
+ case ACM_OP_RESOLVE:\r
+ ret = acm_svr_resolve(client, (struct acm_resolve_msg *) &msg);\r
+ break;\r
+ case ACM_OP_QUERY:\r
+ ret = acm_svr_query(client, (struct acm_query_msg *) &msg);\r
+ break;\r
+ default:\r
+ acm_log(0, "ERROR - unknown opcode 0x%x\n", msg.hdr.opcode);\r
+ ret = -1;\r
+ break;\r
+ }\r
+\r
+out:\r
+ if (ret)\r
+ acm_release_client(client);\r
+}\r
+\r
+static void acm_server(void)\r
+{\r
+ fd_set readfds;\r
+ int i, n, ret;\r
+\r
+ acm_log(0, "started\n");\r
+ acm_init_server();\r
+ ret = acm_listen();\r
+ if (ret) {\r
+ acm_log(0, "ERROR - server listen failed\n");\r
+ return;\r
+ }\r
+\r
+ while (1) {\r
+ n = (int) listen_socket;\r
+ FD_ZERO(&readfds);\r
+ FD_SET(listen_socket, &readfds);\r
+\r
+ for (i = 0; i < FD_SETSIZE - 1; i++) {\r
+ if (client[i].sock != INVALID_SOCKET) {\r
+ FD_SET(client[i].sock, &readfds);\r
+ n = max(n, (int) client[i].sock);\r
+ }\r
+ }\r
+\r
+ ret = select(n + 1, &readfds, NULL, NULL, NULL);\r
+ if (ret == SOCKET_ERROR) {\r
+ acm_log(0, "ERROR - server select error\n");\r
+ continue;\r
+ }\r
+\r
+ if (FD_ISSET(listen_socket, &readfds))\r
+ acm_svr_accept();\r
+\r
+ for (i = 0; i < FD_SETSIZE - 1; i++) {\r
+ if (client[i].sock != INVALID_SOCKET &&\r
+ FD_ISSET(client[i].sock, &readfds)) {\r
+ acm_log(2, "receiving from client %d\n", i);\r
+ acm_svr_receive(&client[i]);\r
+ }\r
+ }\r
+ }\r
+}\r
+\r
+static enum ibv_rate acm_get_rate(uint8_t width, uint8_t speed)\r
+{\r
+ switch (width) {\r
+ case 1:\r
+ switch (speed) {\r
+ case 1: return IBV_RATE_2_5_GBPS;\r
+ case 2: return IBV_RATE_5_GBPS;\r
+ case 4: return IBV_RATE_10_GBPS;\r
+ default: return IBV_RATE_MAX;\r
+ }\r
+ case 2:\r
+ switch (speed) {\r
+ case 1: return IBV_RATE_10_GBPS;\r
+ case 2: return IBV_RATE_20_GBPS;\r
+ case 4: return IBV_RATE_40_GBPS;\r
+ default: return IBV_RATE_MAX;\r
+ }\r
+ case 4:\r
+ switch (speed) {\r
+ case 1: return IBV_RATE_20_GBPS;\r
+ case 2: return IBV_RATE_40_GBPS;\r
+ case 4: return IBV_RATE_80_GBPS;\r
+ default: return IBV_RATE_MAX;\r
+ }\r
+ case 8:\r
+ switch (speed) {\r
+ case 1: return IBV_RATE_30_GBPS;\r
+ case 2: return IBV_RATE_60_GBPS;\r
+ case 4: return IBV_RATE_120_GBPS;\r
+ default: return IBV_RATE_MAX;\r
+ }\r
+ default:\r
+ acm_log(0, "ERROR - unknown link width 0x%x\n", width);\r
+ return IBV_RATE_MAX;\r
+ }\r
+}\r
+\r
+static enum ibv_mtu acm_convert_mtu(int mtu)\r
+{\r
+ switch (mtu) {\r
+ case 256: return IBV_MTU_256;\r
+ case 512: return IBV_MTU_512;\r
+ case 1024: return IBV_MTU_1024;\r
+ case 2048: return IBV_MTU_2048;\r
+ case 4096: return IBV_MTU_4096;\r
+ default: return IBV_MTU_2048;\r
+ }\r
+}\r
+\r
+static enum ibv_rate acm_convert_rate(int rate)\r
+{\r
+ switch (rate) {\r
+ case 2: return IBV_RATE_2_5_GBPS;\r
+ case 5: return IBV_RATE_5_GBPS;\r
+ case 10: return IBV_RATE_10_GBPS;\r
+ case 20: return IBV_RATE_20_GBPS;\r
+ case 30: return IBV_RATE_30_GBPS;\r
+ case 40: return IBV_RATE_40_GBPS;\r
+ case 60: return IBV_RATE_60_GBPS;\r
+ case 80: return IBV_RATE_80_GBPS;\r
+ case 120: return IBV_RATE_120_GBPS;\r
+ default: return IBV_RATE_10_GBPS;\r
+ }\r
+}\r
+\r
+static int acm_post_recvs(struct acm_ep *ep)\r
+{\r
+ int i, size;\r
+\r
+ size = recv_depth * ACM_RECV_SIZE;\r
+ ep->recv_bufs = malloc(size);\r
+ if (!ep->recv_bufs) {\r
+ acm_log(0, "ERROR - unable to allocate receive buffer\n");\r
+ return ACM_STATUS_ENOMEM;\r
+ }\r
+\r
+ ep->mr = ibv_reg_mr(ep->port->dev->pd, ep->recv_bufs, size,\r
+ IBV_ACCESS_LOCAL_WRITE);\r
+ if (!ep->mr) {\r
+ acm_log(0, "ERROR - unable to register receive buffer\n");\r
+ goto err;\r
+ }\r
+\r
+ for (i = 0; i < recv_depth; i++) {\r
+ acm_post_recv(ep, (uintptr_t) (ep->recv_bufs + ACM_RECV_SIZE * i));\r
+ }\r
+ return 0;\r
+\r
+err:\r
+ free(ep->recv_bufs);\r
+ return -1;\r
+}\r
+\r
+static int acm_assign_ep_names(struct acm_ep *ep)\r
+{\r
+ char *dev_name;\r
+ FILE *f;\r
+ char s[120];\r
+ char dev[32], addr[32], pkey_str[8];\r
+ uint16_t pkey;\r
+ uint8_t type;\r
+ int port, index = 0;\r
+ struct in6_addr ip_addr;\r
+\r
+ dev_name = ep->port->dev->verbs->device->name;\r
+ acm_log(1, "device %s, port %d, pkey 0x%x\n",\r
+ dev_name, ep->port->port_num, ep->pkey);\r
+\r
+ if (!(f = fopen("acm_addr.cfg", "r"))) {\r
+ acm_log(0, "ERROR - unable to open acm_addr.cfg file\n");\r
+ return ACM_STATUS_ENODATA;\r
+ }\r
+\r
+ while (fgets(s, sizeof s, f)) {\r
+ if (s[0] == '#')\r
+ continue;\r
+\r
+ if (sscanf(s, "%32s%32s%d%8s", addr, dev, &port, pkey_str) != 4)\r
+ continue;\r
+\r
+ acm_log(2, "%s", s);\r
+ if (inet_pton(AF_INET, addr, &ip_addr) > 0)\r
+ type = ACM_ADDRESS_IP;\r
+ else if (inet_pton(AF_INET6, addr, &ip_addr) > 0)\r
+ type = ACM_ADDRESS_IP6;\r
+ else\r
+ type = ACM_ADDRESS_NAME;\r
+\r
+ if (stricmp(pkey_str, "default")) {\r
+ if (sscanf(pkey_str, "%hx", &pkey) != 1) {\r
+ acm_log(0, "ERROR - bad pkey format %s\n", pkey_str);\r
+ continue;\r
+ }\r
+ } else {\r
+ pkey = 0xFFFF;\r
+ }\r
+\r
+ if (!stricmp(dev_name, dev) && (ep->port->port_num == (uint8_t) port) &&\r
+ (ep->pkey == pkey)) {\r
+\r
+ ep->addr_type[index] = type;\r
+ acm_log(1, "assigning %s\n", addr);\r
+ if (type == ACM_ADDRESS_IP)\r
+ memcpy(ep->addr[index].addr, &ip_addr, 4);\r
+ else if (type == ACM_ADDRESS_IP6)\r
+ memcpy(ep->addr[index].addr, &ip_addr, sizeof ip_addr);\r
+ else\r
+ strncpy((char *) ep->addr[index].addr, addr, ACM_MAX_ADDRESS);\r
+\r
+ if (++index == MAX_EP_ADDR) {\r
+ acm_log(1, "maximum number of names assigned to EP\n");\r
+ break;\r
+ }\r
+ }\r
+ }\r
+\r
+ fclose(f);\r
+ return !index;\r
+}\r
+\r
+static int acm_activate_ep(struct acm_port *port, struct acm_ep *ep, uint16_t pkey_index)\r
+{\r
+ struct ibv_qp_init_attr init_attr;\r
+ struct ibv_qp_attr attr;\r
+ int ret;\r
+\r
+ acm_log(1, "\n");\r
+ ep->port = port;\r
+ ep->pkey_index = pkey_index;\r
+ ep->available_sends = send_depth;\r
+ DListInit(&ep->pending_queue);\r
+ DListInit(&ep->active_queue);\r
+ DListInit(&ep->wait_queue);\r
+ lock_init(&ep->lock);\r
+\r
+ ret = ibv_query_pkey(port->dev->verbs, port->port_num, pkey_index, &ep->pkey);\r
+ if (ret)\r
+ return ACM_STATUS_EINVAL;\r
+\r
+ ret = acm_assign_ep_names(ep);\r
+ if (ret) {\r
+ acm_log(0, "ERROR - unable to assign EP name\n");\r
+ return ret;\r
+ }\r
+\r
+ ep->cq = ibv_create_cq(port->dev->verbs, send_depth + recv_depth, ep,\r
+ port->dev->channel, 0);\r
+ if (!ep->cq) {\r
+ acm_log(0, "ERROR - failed to create CQ\n");\r
+ return -1;\r
+ }\r
+\r
+ ret = ibv_req_notify_cq(ep->cq, 0);\r
+ if (ret) {\r
+ acm_log(0, "ERROR - failed to arm CQ\n");\r
+ goto err1;\r
+ }\r
+\r
+ memset(&init_attr, 0, sizeof init_attr);\r
+ init_attr.cap.max_send_wr = send_depth;\r
+ init_attr.cap.max_recv_wr = recv_depth;\r
+ init_attr.cap.max_send_sge = 1;\r
+ init_attr.cap.max_recv_sge = 1;\r
+ init_attr.qp_context = ep;\r
+ init_attr.sq_sig_all = 1;\r
+ init_attr.qp_type = IBV_QPT_UD;\r
+ init_attr.send_cq = ep->cq;\r
+ init_attr.recv_cq = ep->cq;\r
+ ep->qp = ibv_create_qp(ep->port->dev->pd, &init_attr);\r
+ if (!ep->qp) {\r
+ acm_log(0, "ERROR - failed to create QP\n");\r
+ goto err1;\r
+ }\r
+\r
+ attr.qp_state = IBV_QPS_INIT;\r
+ attr.port_num = port->port_num;\r
+ attr.pkey_index = pkey_index;\r
+ attr.qkey = ACM_QKEY;\r
+ ret = ibv_modify_qp(ep->qp, &attr, IBV_QP_STATE | IBV_QP_PKEY_INDEX |\r
+ IBV_QP_PORT | IBV_QP_QKEY);\r
+ if (ret) {\r
+ acm_log(0, "ERROR - failed to modify QP to init\n");\r
+ goto err2;\r
+ }\r
+\r
+ attr.qp_state = IBV_QPS_RTR;\r
+ ret = ibv_modify_qp(ep->qp, &attr, IBV_QP_STATE);\r
+ if (ret) {\r
+ acm_log(0, "ERROR - failed to modify QP to rtr\n");\r
+ goto err2;\r
+ }\r
+\r
+ attr.qp_state = IBV_QPS_RTS;\r
+ attr.sq_psn = 0;\r
+ ret = ibv_modify_qp(ep->qp, &attr, IBV_QP_STATE | IBV_QP_SQ_PSN);\r
+ if (ret) {\r
+ acm_log(0, "ERROR - failed to modify QP to rts\n");\r
+ goto err2;\r
+ }\r
+\r
+ ret = acm_post_recvs(ep);\r
+ if (ret)\r
+ goto err2;\r
+\r
+ return 0;\r
+\r
+err2:\r
+ ibv_destroy_qp(ep->qp);\r
+err1:\r
+ ibv_destroy_cq(ep->cq);\r
+ return -1;\r
+}\r
+\r
+static void acm_activate_port(struct acm_port *port)\r
+{\r
+ struct acm_ep *ep;\r
+ int i, ret;\r
+\r
+ acm_log(1, "%s %d\n", port->dev->verbs->device->name,\r
+ port->port_num);\r
+\r
+ for (i = 0; i < port->pkey_cnt; i++) {\r
+ ep = zalloc(sizeof *ep);\r
+ if (!ep)\r
+ break;\r
+\r
+ ret = acm_activate_ep(port, ep, (uint16_t) i);\r
+ if (!ret) {\r
+ DListInsertHead(&ep->entry, &port->ep_list);\r
+ } else {\r
+ acm_log(0, "ERROR - failed to activate EP\n");\r
+ free(ep);\r
+ }\r
+ }\r
+\r
+ if (DListEmpty(&port->ep_list))\r
+ goto err1;\r
+\r
+ port->mad_portid = umad_open_port(port->dev->verbs->device->name, port->port_num);\r
+ if (port->mad_portid < 0) {\r
+ acm_log(0, "ERROR - unable to open MAD port\n");\r
+ goto err2;\r
+ }\r
+\r
+ port->mad_agentid = umad_register(port->mad_portid,\r
+ IB_MGMT_CLASS_SA, 1, 1, NULL);\r
+ if (port->mad_agentid < 0) {\r
+ acm_log(0, "ERROR - unable to register MAD client\n");\r
+ goto err3;\r
+ }\r
+\r
+ return;\r
+\r
+err3:\r
+ umad_close_port(port->mad_portid);\r
+err2:\r
+ /* TODO: cleanup ep list */\r
+err1:\r
+ port->state = IBV_PORT_NOP;\r
+ port->dev->active--;\r
+}\r
+\r
+static int acm_activate_dev(struct acm_device *dev)\r
+{\r
+ int i;\r
+\r
+ acm_log(1, "%s\n", dev->verbs->device->name);\r
+ dev->pd = ibv_alloc_pd(dev->verbs);\r
+ if (!dev->pd)\r
+ return ACM_STATUS_ENOMEM;\r
+\r
+ dev->channel = ibv_create_comp_channel(dev->verbs);\r
+ if (!dev->channel) {\r
+ acm_log(0, "ERROR - unable to create comp channel\n");\r
+ goto err1;\r
+ }\r
+\r
+ for (i = 0; i < dev->port_cnt; i++) {\r
+ acm_log(2, "checking port %d\n", dev->port[i].port_num);\r
+ if (dev->port[i].state == IBV_PORT_ACTIVE)\r
+ acm_activate_port(&dev->port[i]);\r
+ }\r
+\r
+ if (!dev->active)\r
+ goto err2;\r
+\r
+ acm_log(1, "starting completion thread\n");\r
+ beginthread(acm_comp_handler, dev);\r
+ return 0;\r
+\r
+err2:\r
+ ibv_destroy_comp_channel(dev->channel);\r
+err1:\r
+ ibv_dealloc_pd(dev->pd);\r
+ return -1;\r
+}\r
+\r
+static void acm_init_port(struct acm_port *port)\r
+{\r
+ struct ibv_port_attr attr;\r
+ union ibv_gid gid;\r
+ uint16_t pkey;\r
+ int ret;\r
+\r
+ acm_log(1, "%s %d\n", port->dev->verbs->device->name, port->port_num);\r
+ DListInit(&port->ep_list);\r
+ ret = ibv_query_port(port->dev->verbs, port->port_num, &attr);\r
+ if (ret)\r
+ return;\r
+\r
+ port->state = attr.state;\r
+ port->mtu = attr.active_mtu;\r
+ port->rate = acm_get_rate(attr.active_width, attr.active_speed);\r
+ port->subnet_timeout = 1 << (attr.subnet_timeout - 8);\r
+ for (;; port->gid_cnt++) {\r
+ ret = ibv_query_gid(port->dev->verbs, port->port_num, port->gid_cnt, &gid);\r
+ if (ret || !gid.global.interface_id)\r
+ break;\r
+ }\r
+\r
+ for (;; port->pkey_cnt++) {\r
+ ret = ibv_query_pkey(port->dev->verbs, port->port_num, port->pkey_cnt, &pkey);\r
+ if (ret || !pkey)\r
+ break;\r
+ }\r
+ port->lid = attr.lid;\r
+ port->lmc = attr.lmc;\r
+\r
+ port->sa_dest.av.dlid = attr.sm_lid;\r
+ port->sa_dest.av.sl = attr.sm_sl;\r
+ port->sa_dest.av.port_num = port->port_num;\r
+ port->sa_dest.remote_qpn = 1;\r
+\r
+ if (port->state == IBV_PORT_ACTIVE)\r
+ port->dev->active++;\r
+}\r
+\r
+static void acm_open_dev(struct ibv_device *ibdev)\r
+{\r
+ struct acm_device *dev;\r
+ struct ibv_device_attr attr;\r
+ struct ibv_context *verbs;\r
+ size_t size;\r
+ int i, ret;\r
+\r
+ acm_log(1, "%s\n", ibdev->name);\r
+ verbs = ibv_open_device(ibdev);\r
+ if (verbs == NULL) {\r
+ acm_log(0, "ERROR - opening device %s\n", ibdev->name);\r
+ return;\r
+ }\r
+\r
+ ret = ibv_query_device(verbs, &attr);\r
+ if (ret) {\r
+ acm_log(0, "ERROR - ibv_query_device (%s) %d\n", ret, ibdev->name);\r
+ goto err1;\r
+ }\r
+\r
+ size = sizeof(*dev) + sizeof(struct acm_port) * attr.phys_port_cnt;\r
+ dev = (struct acm_device *) zalloc(size);\r
+ if (!dev)\r
+ goto err1;\r
+\r
+ dev->verbs = verbs;\r
+ dev->guid = ibv_get_device_guid(ibdev);\r
+ dev->port_cnt = attr.phys_port_cnt;\r
+ dev->init_depth = (uint8_t) attr.max_qp_init_rd_atom;\r
+ dev->resp_resources = (uint8_t) attr.max_qp_rd_atom;\r
+\r
+ for (i = 0; i < dev->port_cnt; i++) {\r
+ dev->port[i].dev = dev;\r
+ dev->port[i].port_num = i + 1;\r
+ acm_init_port(&dev->port[i]);\r
+ }\r
+\r
+ if (!dev->active || acm_activate_dev(dev))\r
+ goto err2;\r
+\r
+ acm_log(1, "%s now active\n", ibdev->name);\r
+ DListInsertHead(&dev->entry, &dev_list);\r
+ return;\r
+\r
+err2:\r
+ free(dev);\r
+err1:\r
+ ibv_close_device(verbs);\r
+}\r
+\r
+static void acm_set_options(void)\r
+{\r
+ FILE *f;\r
+ char s[120];\r
+ char opt[32], value[32];\r
+\r
+ if (!(f = fopen("acm_opts.cfg", "r")))\r
+ return;\r
+\r
+ while (fgets(s, sizeof s, f)) {\r
+ if (s[0] == '#')\r
+ continue;\r
+\r
+ if (sscanf(s, "%32s%32s", opt, value) != 2)\r
+ continue;\r
+\r
+ if (!stricmp("log_file", opt))\r
+ strcpy(log_file, value);\r
+ else if (!stricmp("log_level", opt))\r
+ log_level = atoi(value);\r
+ else if (!stricmp("server_port", opt))\r
+ server_port = (short) atoi(value);\r
+ else if (!stricmp("timeout", opt))\r
+ timeout = atoi(value);\r
+ else if (!stricmp("retries", opt))\r
+ retries = atoi(value);\r
+ else if (!stricmp("send_depth", opt))\r
+ send_depth = atoi(value);\r
+ else if (!stricmp("recv_depth", opt))\r
+ recv_depth = atoi(value);\r
+ else if (!stricmp("min_mtu", opt))\r
+ min_mtu = acm_convert_mtu(atoi(value));\r
+ else if (!stricmp("min_rate", opt))\r
+ min_rate = acm_convert_rate(atoi(value));\r
+ }\r
+\r
+ fclose(f);\r
+}\r
+\r
+static void acm_log_options(void)\r
+{\r
+ acm_log(0, "log level %d\n", log_level);\r
+ acm_log(0, "server_port %d\n", server_port);\r
+ acm_log(0, "timeout %d ms\n", timeout);\r
+ acm_log(0, "retries %d\n", retries);\r
+ acm_log(0, "send depth %d\n", send_depth);\r
+ acm_log(0, "receive depth %d\n", recv_depth);\r
+ acm_log(0, "minimum mtu %d\n", min_mtu);\r
+ acm_log(0, "minimum rate %d\n", min_rate);\r
+}\r
+\r
+static FILE *acm_open_log(void)\r
+{\r
+ FILE *f;\r
+ int n;\r
+\r
+ if (!stricmp(log_file, "stdout"))\r
+ return stdout;\r
+\r
+ if (!stricmp(log_file, "stderr"))\r
+ return stderr;\r
+\r
+ n = strlen(log_file);\r
+ sprintf(&log_file[n], "%5u.log", getpid());\r
+ if (!(f = fopen(log_file, "w")))\r
+ f = stdout;\r
+\r
+ return f;\r
+}\r
+\r
+int CDECL_FUNC main(int argc, char **argv)\r
+{\r
+ struct ibv_device **ibdev;\r
+ int dev_cnt;\r
+ int i;\r
+\r
+ if (osd_init())\r
+ return -1;\r
+\r
+ acm_set_options();\r
+\r
+ lock_init(&log_lock);\r
+ flog = acm_open_log();\r
+\r
+ acm_log(0, "Assistant to the InfiniBand Communication Manager\n");\r
+ acm_log_options();\r
+\r
+ DListInit(&dev_list);\r
+ DListInit(&timeout_list);\r
+ event_init(&timeout_event);\r
+\r
+ umad_init();\r
+ ibdev = ibv_get_device_list(&dev_cnt);\r
+ if (!ibdev) {\r
+ acm_log(0, "ERROR - unable to get device list\n");\r
+ return -1;\r
+ }\r
+\r
+ acm_log(1, "opening devices\n");\r
+ for (i = 0; i < dev_cnt; i++)\r
+ acm_open_dev(ibdev[i]);\r
+\r
+ ibv_free_device_list(ibdev);\r
+\r
+ acm_log(1, "initiating multicast joins\n");\r
+ acm_join_groups();\r
+ acm_log(1, "multicast joins done\n");\r
+ acm_log(1, "starting timeout/retry thread\n");\r
+ beginthread(acm_retry_handler, NULL);\r
+ acm_log(1, "starting server\n");\r
+ acm_server();\r
+\r
+ acm_log(0, "shutting down\n");\r
+ fclose(flog);\r
+ return 0;\r
+}\r
--- /dev/null
+/*\r
+ * Copyright (c) 2009 Intel Corporation. All rights reserved.\r
+ *\r
+ * This software is available to you under the OpenFabrics.org BSD license\r
+ * below:\r
+ *\r
+ * Redistribution and use in source and binary forms, with or\r
+ * without modification, are permitted provided that the following\r
+ * conditions are met:\r
+ *\r
+ * - Redistributions of source code must retain the above\r
+ * copyright notice, this list of conditions and the following\r
+ * disclaimer.\r
+ *\r
+ * - Redistributions in binary form must reproduce the above\r
+ * copyright notice, this list of conditions and the following\r
+ * disclaimer in the documentation and/or other materials\r
+ * provided with the distribution.\r
+ *\r
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,\r
+ * EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF\r
+ * MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AWV\r
+ * NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS\r
+ * BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN\r
+ * ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN\r
+ * CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE\r
+ * SOFTWARE.\r
+ */\r
+\r
+#if !defined(ACM_MAD_H)\r
+#define ACM_MAD_H\r
+\r
+#include <infiniband/verbs.h>\r
+#include <infiniband/acm.h>\r
+#include <netinet/in.h>\r
+\r
+#define ACM_SEND_SIZE 256\r
+#define ACM_RECV_SIZE (ACM_SEND_SIZE + sizeof(struct ibv_grh))\r
+\r
+#define IB_METHOD_GET 0x01\r
+#define IB_METHOD_SET 0x02\r
+#define IB_METHOD_SEND 0x03\r
+#define IB_METHOD_GET_TABLE 0x12\r
+#define IB_METHOD_DELETE 0x15\r
+#define IB_METHOD_RESP 0x80\r
+\r
+#define ACM_MGMT_CLASS 0x2C\r
+\r
+#define ACM_CTRL_ACK htons(0x8000)\r
+#define ACM_CTRL_RESOLVE htons(0x0001)\r
+#define ACM_CTRL_CM htons(0x0002)\r
+\r
+struct acm_mad\r
+{\r
+ uint8_t base_version;\r
+ uint8_t mgmt_class;\r
+ uint8_t class_version;\r
+ uint8_t method;\r
+ uint16_t status;\r
+ uint16_t control;\r
+ uint64_t tid;\r
+\r
+ uint8_t data[240];\r
+};\r
+\r
+#define acm_class_status(status) ((uint8_t) (ntohs(status) >> 8))\r
+\r
+#define ACM_QKEY 0x80010000\r
+\r
+#define ACM_ADDRESS_INVALID 0x00\r
+#define ACM_ADDRESS_NAME 0x01\r
+#define ACM_ADDRESS_IP 0x02\r
+#define ACM_ADDRESS_IP6 0x03\r
+#define ACM_ADDRESS_RESERVED 0x04 /* start of reserved range */\r
+\r
+#define ACM_MAX_GID_COUNT 10\r
+\r
+struct acm_resolve_rec\r
+{\r
+ uint8_t dest_type;\r
+ uint8_t dest_length;\r
+ uint8_t src_type;\r
+ uint8_t src_length;\r
+ uint8_t gid_cnt;\r
+ uint8_t resp_resources;\r
+ uint8_t init_depth;\r
+ uint8_t reserved;\r
+ uint8_t dest[ACM_MAX_ADDRESS];\r
+ uint8_t src[ACM_MAX_ADDRESS];\r
+ union ibv_gid gid[ACM_MAX_GID_COUNT];\r
+};\r
+\r
+#define IB_MGMT_CLASS_SA 0x03\r
+\r
+struct ib_sa_mad\r
+{\r
+ uint8_t base_version;\r
+ uint8_t mgmt_class;\r
+ uint8_t class_version;\r
+ uint8_t method;\r
+ uint16_t status;\r
+ uint16_t reserved1;\r
+ uint64_t tid;\r
+ uint16_t attr_id;\r
+ uint16_t reserved2;\r
+ uint32_t attr_mod;\r
+\r
+ uint8_t rmpp_version;\r
+ uint8_t rmpp_type;\r
+ uint8_t rmpp_flags;\r
+ uint8_t rmpp_status;\r
+ uint32_t seg_num;\r
+ uint32_t paylen_newwin;\r
+\r
+ uint32_t sm_key[2];\r
+ uint16_t attr_offset;\r
+ uint16_t reserved3;\r
+ uint64_t comp_mask;\r
+\r
+ uint8_t data[200];\r
+};\r
+\r
+#define IB_SA_ATTR_PATH_REC htons(0x0035)\r
+\r
+#define IB_COMP_MASK_PR_SERVICE_ID (htonll(1 << 0) | \\r
+ htonll(1 << 1))\r
+#define IB_COMP_MASK_PR_DGID htonll(1 << 2)\r
+#define IB_COMP_MASK_PR_SGID htonll(1 << 3)\r
+#define IB_COMP_MASK_PR_DLID htonll(1 << 4)\r
+#define IB_COMP_MASK_PR_SLID htonll(1 << 5)\r
+#define IB_COMP_MASK_PR_RAW_TRAFFIC htonll(1 << 6)\r
+/* RESERVED htonll(1 << 7) */\r
+#define IB_COMP_MASK_PR_FLOW_LABEL htonll(1 << 8)\r
+#define IB_COMP_MASK_PR_HOP_LIMIT htonll(1 << 9)\r
+#define IB_COMP_MASK_PR_TCLASS htonll(1 << 10)\r
+#define IB_COMP_MASK_PR_REVERSIBLE htonll(1 << 11)\r
+#define IB_COMP_MASK_PR_NUM_PATH htonll(1 << 12)\r
+#define IB_COMP_MASK_PR_PKEY htonll(1 << 13)\r
+#define IB_COMP_MASK_PR_QOS_CLASS htonll(1 << 14)\r
+#define IB_COMP_MASK_PR_SL htonll(1 << 15)\r
+#define IB_COMP_MASK_PR_MTU_SELECTOR htonll(1 << 16)\r
+#define IB_COMP_MASK_PR_MTU htonll(1 << 17)\r
+#define IB_COMP_MASK_PR_RATE_SELECTOR htonll(1 << 18)\r
+#define IB_COMP_MASK_PR_RATE htonll(1 << 19)\r
+#define IB_COMP_MASK_PR_PACKET_LIFETIME_SELECTOR htonll(1 << 20)\r
+#define IB_COMP_MASK_PR_PACKET_LIFETIME htonll(1 << 21)\r
+#define IB_COMP_MASK_PR_PREFERENCE htonll(1 << 22)\r
+/* RESERVED htonll(1 << 23) */\r
+\r
+#define IB_MC_QPN 0xffffff\r
+#define IB_SA_ATTR_MC_MEMBER_REC htons(0x0038)\r
+\r
+#define IB_COMP_MASK_MC_MGID htonll(1 << 0)\r
+#define IB_COMP_MASK_MC_PORT_GID htonll(1 << 1)\r
+#define IB_COMP_MASK_MC_QKEY htonll(1 << 2)\r
+#define IB_COMP_MASK_MC_MLID htonll(1 << 3)\r
+#define IB_COMP_MASK_MC_MTU_SEL htonll(1 << 4)\r
+#define IB_COMP_MASK_MC_MTU htonll(1 << 5)\r
+#define IB_COMP_MASK_MC_TCLASS htonll(1 << 6)\r
+#define IB_COMP_MASK_MC_PKEY htonll(1 << 7)\r
+#define IB_COMP_MASK_MC_RATE_SEL htonll(1 << 8)\r
+#define IB_COMP_MASK_MC_RATE htonll(1 << 9)\r
+#define IB_COMP_MASK_MC_PACKET_LIFETIME_SEL htonll(1 << 10)\r
+#define IB_COMP_MASK_MC_PACKET_LIFETIME htonll(1 << 11)\r
+#define IB_COMP_MASK_MC_SL htonll(1 << 12)\r
+#define IB_COMP_MASK_MC_FLOW htonll(1 << 13)\r
+#define IB_COMP_MASK_MC_HOP htonll(1 << 14)\r
+#define IB_COMP_MASK_MC_SCOPE htonll(1 << 15)\r
+#define IB_COMP_MASK_MC_JOIN_STATE htonll(1 << 16)\r
+#define IB_COMP_MASK_MC_PROXY_JOIN htonll(1 << 17)\r
+\r
+struct ib_mc_member_rec\r
+{\r
+ union ibv_gid mgid;\r
+ union ibv_gid port_gid;\r
+ uint32_t qkey;\r
+ uint16_t mlid;\r
+ uint8_t mtu;\r
+ uint8_t tclass;\r
+ uint16_t pkey;\r
+ uint8_t rate;\r
+ uint8_t packet_lifetime;\r
+ uint32_t sl_flow_hop;\r
+ uint8_t scope_state;\r
+ uint8_t proxy_join;\r
+ uint8_t reserved[2];\r
+ uint8_t pad[4];\r
+};\r
+\r
+#endif /* ACM_MAD_H */\r
--- /dev/null
+/*\r
+ * Copyright (c) 2009 Intel Corporation. All rights reserved.\r
+ *\r
+ * This software is available to you under the OpenIB.org BSD license\r
+ * below:\r
+ *\r
+ * Redistribution and use in source and binary forms, with or\r
+ * without modification, are permitted provided that the following\r
+ * conditions are met:\r
+ *\r
+ * - Redistributions of source code must retain the above\r
+ * copyright notice, this list of conditions and the following\r
+ * disclaimer.\r
+ *\r
+ * - Redistributions in binary form must reproduce the above\r
+ * copyright notice, this list of conditions and the following\r
+ * disclaimer in the documentation and/or other materials\r
+ * provided with the distribution.\r
+ *\r
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,\r
+ * EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF\r
+ * MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AWV\r
+ * NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS\r
+ * BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN\r
+ * ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN\r
+ * CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE\r
+ * SOFTWARE.\r
+ */\r
+\r
+#include <stdio.h>\r
+#include <stdlib.h>\r
+#include <string.h>\r
+#include <getopt.h>\r
+#include <netdb.h>\r
+#include <arpa/inet.h>\r
+\r
+#include <osd.h>\r
+#include <infiniband/verbs.h>\r
+#include <infiniband/ib_acm.h>\r
+\r
+static char *dest_addr;\r
+static char *src_addr;\r
+static char addr_type = 'i';\r
+static int verify;\r
+static int make_addr;\r
+static int make_opts;\r
+\r
+struct ibv_context **verbs;\r
+int dev_cnt;\r
+\r
+extern int gen_addr_ip(FILE *f);\r
+\r
+\r
+static void show_usage(char *program)\r
+{\r
+ printf("usage 1: %s\n", program);\r
+ printf(" [-f addr_format] - i(p), n(ame), or l(id)\n");\r
+ printf(" default: 'i'\n");\r
+ printf(" -s src_addr - format defined by -f option\n");\r
+ printf(" -d dest_addr - format defined by -f option\n");\r
+ printf(" [-v] - verify ACM response against SA query response\n");\r
+ printf("usage 2: %s\n", program);\r
+ printf(" -A - generate local acm_addr.cfg configuration file\n");\r
+ printf(" -O - generate local acm_ops.cfg options file\n");\r
+}\r
+\r
+static void gen_opts_temp(FILE *f)\r
+{\r
+ fprintf(f, "# InfiniBand Multicast Communication Manager for clusters configuration file\n");\r
+ fprintf(f, "#\n");\r
+ fprintf(f, "# Use ib_acme utility with -O option to automatically generate a sample\n");\r
+ fprintf(f, "# acm_opts.cfg file for the current system.\n");\r
+ fprintf(f, "#\n");\r
+ fprintf(f, "# Entry format is:\n");\r
+ fprintf(f, "# name value\n");\r
+ fprintf(f, "\n");\r
+ fprintf(f, "# log_file:\n");\r
+ fprintf(f, "# Specifies the location of the ACM service output. The log file is used to\n");\r
+ fprintf(f, "# assist with ACM service debugging and troubleshooting. The log_file can\n");\r
+ fprintf(f, "# be set to 'stdout', 'stderr', or the base name of a file. If a file name\n");\r
+ fprintf(f, "# is specified, the actual name formed by appending a process ID and '.log'\n");\r
+ fprintf(f, "# extension to the end of the specified file name.\n");\r
+ fprintf(f, "# Examples:\n");\r
+ fprintf(f, "# log_file stdout\n");\r
+ fprintf(f, "# log_file stderr\n");\r
+ fprintf(f, "# log_file /tmp/acm_\n");\r
+ fprintf(f, "\n");\r
+ fprintf(f, "log_file stdout\n");\r
+ fprintf(f, "\n");\r
+ fprintf(f, "# log_level:\n");\r
+ fprintf(f, "# Indicates the amount of detailed data written to the log file. Log levels\n");\r
+ fprintf(f, "# should be one of the following values:\n");\r
+ fprintf(f, "# 0 - basic configuration & errors\n");\r
+ fprintf(f, "# 1 - verbose configuation & errors\n");\r
+ fprintf(f, "# 2 - verbose operation\n");\r
+ fprintf(f, "\n");\r
+ fprintf(f, "log_level 0\n");\r
+ fprintf(f, "\n");\r
+ fprintf(f, "# server_port:\n");\r
+ fprintf(f, "# TCP port number that the server listens on.\n");\r
+ fprintf(f, "# If this value is changed, then a corresponding change is required for\n");\r
+ fprintf(f, "# client applications.\n");\r
+ fprintf(f, "\n");\r
+ fprintf(f, "server_port 6125\n");\r
+ fprintf(f, "\n");\r
+ fprintf(f, "# timeout:\n");\r
+ fprintf(f, "# Additional time, in milliseconds, that the ACM service will wait for a\n");\r
+ fprintf(f, "# response from a remote ACM service or the IB SA. The actual request\n");\r
+ fprintf(f, "# timeout is this value plus the subnet timeout.\n");\r
+ fprintf(f, "\n");\r
+ fprintf(f, "timeout 2000\n");\r
+ fprintf(f, "\n");\r
+ fprintf(f, "# retries:\n");\r
+ fprintf(f, "# Number of times that the ACM service will retry a request. This affects\n");\r
+ fprintf(f, "# both ACM multicast messages and and IB SA messages.\n");\r
+ fprintf(f, "\n");\r
+ fprintf(f, "retries 15\n");\r
+ fprintf(f, "\n");\r
+ fprintf(f, "# send_depth:\n");\r
+ fprintf(f, "# Specifies the maximum number of outstanding requests that can be in\n");\r
+ fprintf(f, "# progress simultaneously. A larger send depth allows for greater\n");\r
+ fprintf(f, "# parallelism, but increases system resource usage and subnet load.\n");\r
+ fprintf(f, "# If the number of pending requests is greater than the send_depth,\n");\r
+ fprintf(f, "# the additional requests will automatically be queued until some of\n");\r
+ fprintf(f, "# the previous requests complete.\n");\r
+ fprintf(f, "\n");\r
+ fprintf(f, "send_depth 8\n");\r
+ fprintf(f, "\n");\r
+ fprintf(f, "# recv_depth:\n");\r
+ fprintf(f, "# Specifies the number of buffers allocated and ready to receive remote\n");\r
+ fprintf(f, "# requests. A larger receive depth consumes more system resources, but\n");\r
+ fprintf(f, "# can avoid dropping requests due to insufficient receive buffers.\n");\r
+ fprintf(f, "\n");\r
+ fprintf(f, "recv_depth 1024\n");\r
+ fprintf(f, "\n");\r
+ fprintf(f, "# min_mtu:\n");\r
+ fprintf(f, "# Indicates the minimum MTU supported by the ACM service. The ACM service\n");\r
+ fprintf(f, "# negotiates to use the largest MTU available between both sides of a\n");\r
+ fprintf(f, "# connection. It is most efficient and recommended that min_mtu be set\n");\r
+ fprintf(f, "# to the largest MTU value supported by all nodes in a cluster.\n");\r
+ fprintf(f, "\n");\r
+ fprintf(f, "min_mtu 2048\n");\r
+ fprintf(f, "\n");\r
+ fprintf(f, "#min_rate:\n");\r
+ fprintf(f, "# Indicates the minimum link rate, in Gbps, supported by the ACM service.\n");\r
+ fprintf(f, "# The ACM service negotiates to use the highest rate available between both\n");\r
+ fprintf(f, "# sides of a connection. It is most efficient and recommended that the\n");\r
+ fprintf(f, "# min_rate be set to the largest rate supported by all nodes in a cluster.\n");\r
+ fprintf(f, "\n");\r
+ fprintf(f, "min_rate 10\n");\r
+ fprintf(f, "\n");\r
+}\r
+\r
+static int gen_opts(void)\r
+{\r
+ FILE *f;\r
+\r
+ printf("Generating acm_opts.cfg\n");\r
+ if (!(f = fopen("acm_opts.cfg", "w"))) {\r
+ printf("Failed to open option configuration file\n");\r
+ return -1;\r
+ }\r
+\r
+ gen_opts_temp(f);\r
+ fclose(f);\r
+ return 0;\r
+}\r
+\r
+static void gen_addr_temp(FILE *f)\r
+{\r
+ fprintf(f, "# InfiniBand Communication Management Assistant for clusters address file\n");\r
+ fprintf(f, "#\n");\r
+ fprintf(f, "# Use ib_acme utility with -G option to automatically generate a sample\n");\r
+ fprintf(f, "# acm_addr.cfg file for the current system.\n");\r
+ fprintf(f, "#\n");\r
+ fprintf(f, "# Entry format is:\n");\r
+ fprintf(f, "# address device port pkey\n");\r
+ fprintf(f, "#\n");\r
+ fprintf(f, "# The address may be one of the following:\n");\r
+ fprintf(f, "# host_name - ascii character string, up to 31 characters\n");\r
+ fprintf(f, "# address - IPv4 or IPv6 formatted address\n");\r
+ fprintf(f, "#\n");\r
+ fprintf(f, "# device name - struct ibv_device name\n");\r
+ fprintf(f, "# port number - valid port number on device (numbering starts at 1)\n");\r
+ fprintf(f, "# pkey - partition key in hex (can specify 'default' for pkey 0xFFFF)\n");\r
+ fprintf(f, "#\n");\r
+ fprintf(f, "# Up to 4 addresses can be associated with a given <device, port, pkey> tuple\n");\r
+ fprintf(f, "#\n");\r
+ fprintf(f, "# Samples:\n");\r
+ fprintf(f, "# node31 ibv_device0 1 default\n");\r
+ fprintf(f, "# node31-1 ibv_device0 1 0x00FF\n");\r
+ fprintf(f, "# node31-2 ibv_device0 2 0x00FF\n");\r
+ fprintf(f, "# 192.168.0.1 ibv_device0 1 0xFFFF\n");\r
+ fprintf(f, "# 192.168.0.2 ibv_device0 2 default\n");\r
+}\r
+\r
+static int open_verbs(void)\r
+{\r
+ struct ibv_device **dev_array;\r
+ int i, ret;\r
+\r
+ dev_array = ibv_get_device_list(&dev_cnt);\r
+ if (!dev_array) {\r
+ printf("ibv_get_device_list - no devices present?\n");\r
+ return -1;\r
+ }\r
+\r
+ verbs = malloc(sizeof(struct ibv_context *) * dev_cnt);\r
+ if (!verbs) {\r
+ ret = -1;\r
+ goto err1;\r
+ }\r
+\r
+ for (i = 0; i < dev_cnt; i++) {\r
+ verbs[i] = ibv_open_device(dev_array[i]);\r
+ if (!verbs) {\r
+ printf("ibv_open_device - failed to open device\n");\r
+ ret = -1;\r
+ goto err2;\r
+ }\r
+ }\r
+\r
+ ibv_free_device_list(dev_array);\r
+ return 0;\r
+\r
+err2:\r
+ while (i--)\r
+ ibv_close_device(verbs[i]);\r
+ free(verbs);\r
+err1:\r
+ ibv_free_device_list(dev_array);\r
+ return ret;\r
+}\r
+\r
+static void close_verbs(void)\r
+{\r
+ int i;\r
+\r
+ for (i = 0; i < dev_cnt; i++)\r
+ ibv_close_device(verbs[i]);\r
+ free(verbs);\r
+}\r
+\r
+static int gen_addr_names(FILE *f)\r
+{\r
+ struct ibv_device_attr dev_attr;\r
+ struct ibv_port_attr port_attr;\r
+ int i, index, ret, found_active;\r
+ char host_name[256];\r
+ uint8_t p;\r
+\r
+ ret = gethostname(host_name, sizeof host_name);\r
+ if (ret) {\r
+ printf("gethostname error: %d\n", ret);\r
+ return ret;\r
+ }\r
+ strtok(host_name, ".");\r
+\r
+ found_active = 0;\r
+ index = 1;\r
+ for (i = 0; i < dev_cnt; i++) {\r
+ ret = ibv_query_device(verbs[i], &dev_attr);\r
+ if (ret)\r
+ break;\r
+\r
+ for (p = 1; p <= dev_attr.phys_port_cnt; p++) {\r
+ if (!found_active) {\r
+ ret = ibv_query_port(verbs[i], p, &port_attr);\r
+ if (!ret && port_attr.state == IBV_PORT_ACTIVE) {\r
+ printf("%s %s %d default\n",\r
+ host_name, verbs[i]->device->name, p);\r
+ fprintf(f, "%s %s %d default\n",\r
+ host_name, verbs[i]->device->name, p);\r
+ found_active = 1;\r
+ }\r
+ }\r
+\r
+ printf("%s-%d %s %d default\n",\r
+ host_name, index, verbs[i]->device->name, p);\r
+ fprintf(f, "%s-%d %s %d default\n",\r
+ host_name, index++, verbs[i]->device->name, p);\r
+ }\r
+ }\r
+\r
+ return ret;\r
+}\r
+\r
+static int gen_addr(void)\r
+{\r
+ FILE *f;\r
+ int ret;\r
+\r
+ printf("Generating acm_addr.cfg\n");\r
+ if (!(f = fopen("acm_addr.cfg", "w"))) {\r
+ printf("Failed to open address configuration file\n");\r
+ return -1;\r
+ }\r
+\r
+ ret = open_verbs();\r
+ if (ret) {\r
+ goto out1;\r
+ }\r
+\r
+ gen_addr_temp(f);\r
+ ret = gen_addr_names(f);\r
+ if (ret) {\r
+ printf("Failed to auto generate host names in config file\n");\r
+ goto out2;\r
+ }\r
+\r
+ ret = gen_addr_ip(f);\r
+ if (ret) {\r
+ printf("Failed to auto generate IP addresses in config file\n");\r
+ goto out2;\r
+ }\r
+\r
+out2:\r
+ close_verbs();\r
+out1:\r
+ fclose(f);\r
+ return ret;\r
+}\r
+\r
+static void show_path(struct ib_path_record *path)\r
+{\r
+ char gid[sizeof "ffff:ffff:ffff:ffff:ffff:ffff:ffff:ffff"];\r
+ uint32_t fl_hop;\r
+\r
+ printf("Path information\n");\r
+ inet_ntop(AF_INET6, path->dgid.raw, gid, sizeof gid);\r
+ printf(" dgid: %s\n", gid);\r
+ inet_ntop(AF_INET6, path->sgid.raw, gid, sizeof gid);\r
+ printf(" sgid: %s\n", gid);\r
+ printf(" dlid: 0x%x\n", ntohs(path->dlid));\r
+ printf(" slid: 0x%x\n", ntohs(path->slid));\r
+ fl_hop = ntohl(path->flowlabel_hoplimit);\r
+ printf(" flow label: 0x%x\n", fl_hop >> 8);\r
+ printf(" hop limit: %d\n", (uint8_t) fl_hop);\r
+ printf(" tclass: %d\n", path->tclass);\r
+ printf(" reverisible: %d\n", path->reversible_numpath >> 7);\r
+ printf(" pkey: 0x%x\n", ntohs(path->pkey));\r
+ printf(" sl: %d\n", ntohs(path->qosclass_sl) & 0xF);\r
+ printf(" mtu: %d\n", path->mtu & 0x1F);\r
+ printf(" rate: %d\n", path->rate & 0x1F);\r
+ printf(" packet lifetime: %d\n", path->packetlifetime & 0x1F);\r
+}\r
+\r
+static int resolve_ip(struct ib_path_record *path)\r
+{\r
+ struct ib_acm_dev_addr dev_addr;\r
+ struct ibv_ah_attr ah;\r
+ struct ib_acm_resolve_data data;\r
+ struct sockaddr_in src, dest;\r
+ int ret;\r
+\r
+ src.sin_family = AF_INET;\r
+ ret = inet_pton(AF_INET, src_addr, &src.sin_addr);\r
+ if (ret <= 0) {\r
+ printf("inet_pton error on source address (%s): 0x%x\n", src_addr, ret);\r
+ return ret;\r
+ }\r
+\r
+ dest.sin_family = AF_INET;\r
+ ret = inet_pton(AF_INET, dest_addr, &dest.sin_addr);\r
+ if (ret <= 0) {\r
+ printf("inet_pton error on destination address (%s): 0x%x\n", dest_addr, ret);\r
+ return ret;\r
+ }\r
+\r
+ ret = ib_acm_resolve_ip((struct sockaddr *) &src, (struct sockaddr *) &dest,\r
+ &dev_addr, &ah, &data);\r
+ if (ret) {\r
+ printf("ib_acm_resolve_ip failed: 0x%x\n", ret);\r
+ return ret;\r
+ }\r
+\r
+ ret = ib_acm_convert_to_path(&dev_addr, &ah, &data, path);\r
+ if (ret)\r
+ printf("ib_acm_convert_to_path failed: 0x%x\n", ret);\r
+\r
+ return ret;\r
+}\r
+\r
+static int resolve_name(struct ib_path_record *path)\r
+{\r
+ struct ib_acm_dev_addr dev_addr;\r
+ struct ibv_ah_attr ah;\r
+ struct ib_acm_resolve_data data;\r
+ int ret;\r
+\r
+ ret = ib_acm_resolve_name(src_addr, dest_addr, &dev_addr, &ah, &data);\r
+ if (ret) {\r
+ printf("ib_acm_resolve_name failed: 0x%x\n", ret);\r
+ return ret;\r
+ }\r
+\r
+ ret = ib_acm_convert_to_path(&dev_addr, &ah, &data, path);\r
+ if (ret)\r
+ printf("ib_acm_convert_to_path failed: 0x%x\n", ret);\r
+\r
+ return ret;\r
+}\r
+\r
+static int resolve_lid(struct ib_path_record *path)\r
+{\r
+ int ret;\r
+\r
+ path->slid = htons((uint16_t) atoi(src_addr));\r
+ path->dlid = htons((uint16_t) atoi(dest_addr));\r
+ path->reversible_numpath = IB_PATH_RECORD_REVERSIBLE | 1;\r
+\r
+ ret = ib_acm_resolve_path(path);\r
+ if (ret)\r
+ printf("ib_acm_resolve_path failed: 0x%x\n", ret);\r
+\r
+ return ret;\r
+}\r
+\r
+static int verify_resolve(struct ib_path_record *path)\r
+{\r
+ int ret;\r
+\r
+ ret = ib_acm_query_path(path);\r
+ if (ret)\r
+ printf("SA verification: failed 0x%x\n", ret);\r
+ else\r
+ printf("SA verification: success\n");\r
+\r
+ return ret;\r
+}\r
+\r
+static int resolve(char *program)\r
+{\r
+ struct ib_path_record path;\r
+ int ret;\r
+\r
+ switch (addr_type) {\r
+ case 'i':\r
+ ret = resolve_ip(&path);\r
+ break;\r
+ case 'n':\r
+ ret = resolve_name(&path);\r
+ break;\r
+ case 'l':\r
+ memset(&path, 0, sizeof path);\r
+ ret = resolve_lid(&path);\r
+ break;\r
+ default:\r
+ show_usage(program);\r
+ exit(1);\r
+ }\r
+\r
+ if (!ret)\r
+ show_path(&path);\r
+\r
+ if (verify)\r
+ ret = verify_resolve(&path);\r
+\r
+ return ret;\r
+}\r
+\r
+int main(int argc, char **argv)\r
+{\r
+ int op, ret;\r
+\r
+ ret = osd_init();\r
+ if (ret)\r
+ goto out;\r
+\r
+ while ((op = getopt(argc, argv, "f:s:d:vAO")) != -1) {\r
+ switch (op) {\r
+ case 'f':\r
+ addr_type = optarg[0];\r
+ break;\r
+ case 's':\r
+ src_addr = optarg;\r
+ break;\r
+ case 'd':\r
+ dest_addr = optarg;\r
+ break;\r
+ case 'v':\r
+ verify = 1;\r
+ break;\r
+ case 'A':\r
+ make_addr = 1;\r
+ break;\r
+ case 'O':\r
+ make_opts = 1;\r
+ break;\r
+ default:\r
+ show_usage(argv[0]);\r
+ exit(1);\r
+ }\r
+ }\r
+\r
+ if ((src_addr && !dest_addr) || (dest_addr && !src_addr) ||\r
+ (!src_addr && !dest_addr && !make_addr && !make_opts)) {\r
+ show_usage(argv[0]);\r
+ exit(1);\r
+ }\r
+\r
+ if (src_addr)\r
+ ret = resolve(argv[0]);\r
+\r
+ if (!ret && make_addr)\r
+ ret = gen_addr();\r
+\r
+ if (!ret && make_opts)\r
+ ret = gen_opts();\r
+\r
+out:\r
+ printf("return status 0x%x\n", ret);\r
+ return ret;\r
+}\r
--- /dev/null
+/*\r
+ * Copyright (c) 2009 Intel Corporation. All rights reserved.\r
+ *\r
+ * This software is available to you under the OpenIB.org BSD license\r
+ * below:\r
+ *\r
+ * Redistribution and use in source and binary forms, with or\r
+ * without modification, are permitted provided that the following\r
+ * conditions are met:\r
+ *\r
+ * - Redistributions of source code must retain the above\r
+ * copyright notice, this list of conditions and the following\r
+ * disclaimer.\r
+ *\r
+ * - Redistributions in binary form must reproduce the above\r
+ * copyright notice, this list of conditions and the following\r
+ * disclaimer in the documentation and/or other materials\r
+ * provided with the distribution.\r
+ *\r
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,\r
+ * EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF\r
+ * MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AWV\r
+ * NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS\r
+ * BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN\r
+ * ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN\r
+ * CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE\r
+ * SOFTWARE.\r
+ */\r
+\r
+#include <osd.h>\r
+#include <infiniband/ib_acm.h>\r
+#include <infiniband/acm.h>\r
+#include <stdio.h>\r
+\r
+extern lock_t lock;\r
+static SOCKET sock = INVALID_SOCKET;\r
+static short server_port = 6125;\r
+static int ready;\r
+\r
+static int acm_init(void)\r
+{\r
+ struct sockaddr_in addr;\r
+ int ret;\r
+\r
+ ret = osd_init();\r
+ if (ret)\r
+ return ret;\r
+\r
+ sock = socket(AF_INET, SOCK_STREAM, IPPROTO_TCP);\r
+ if (sock == INVALID_SOCKET) {\r
+ ret = socket_errno();\r
+ goto err1;\r
+ }\r
+\r
+ memset(&addr, 0, sizeof addr);\r
+ addr.sin_family = AF_INET;\r
+ addr.sin_addr.s_addr = htonl(INADDR_LOOPBACK);\r
+ addr.sin_port = htons(server_port);\r
+ ret = connect(sock, (struct sockaddr *) &addr, sizeof(addr));\r
+ if (ret)\r
+ goto err2;\r
+\r
+ ready = 1;\r
+ return 0;\r
+\r
+err2:\r
+ closesocket(sock);\r
+ sock = INVALID_SOCKET;\r
+err1:\r
+ osd_close();\r
+ return ret;\r
+}\r
+\r
+void LIB_DESTRUCTOR acm_cleanup(void)\r
+{\r
+ if (sock != INVALID_SOCKET) {\r
+ shutdown(sock, SHUT_RDWR);\r
+ closesocket(sock);\r
+ }\r
+}\r
+\r
+static int acm_resolve(uint8_t *src, uint8_t *dest, uint8_t type,\r
+ struct ib_acm_dev_addr *dev_addr, struct ibv_ah_attr *ah,\r
+ struct ib_acm_resolve_data *data)\r
+{\r
+ struct acm_resolve_msg msg;\r
+ int ret;\r
+\r
+ lock_acquire(&lock);\r
+ if (!ready && (ret = acm_init()))\r
+ goto out;\r
+\r
+ memset(&msg, 0, sizeof msg);\r
+ msg.hdr.version = ACM_VERSION;\r
+ msg.hdr.opcode = ACM_OP_RESOLVE;\r
+ msg.hdr.dest_type = type;\r
+ msg.hdr.src_type = type;\r
+\r
+ switch (type) {\r
+ case ACM_EP_TYPE_NAME:\r
+ strncpy((char *) msg.src.name, (char *) src, ACM_MAX_ADDRESS);\r
+ strncpy((char *) msg.dest.name, (char *) dest, ACM_MAX_ADDRESS);\r
+ break;\r
+ case ACM_EP_TYPE_ADDRESS_IP:\r
+ memcpy(msg.src.addr, &((struct sockaddr_in *) src)->sin_addr, 4);\r
+ memcpy(msg.dest.addr, &((struct sockaddr_in *) dest)->sin_addr, 4);\r
+ break;\r
+ case ACM_EP_TYPE_ADDRESS_IP6:\r
+ memcpy(msg.src.addr, &((struct sockaddr_in6 *) src)->sin6_addr, 16);\r
+ memcpy(msg.dest.addr, &((struct sockaddr_in *) dest)->sin_addr, 16);\r
+ break;\r
+ case ACM_EP_TYPE_AV:\r
+ memcpy(&msg.src.av, src, sizeof(msg.src.av));\r
+ memcpy(&msg.dest.av, dest, sizeof(msg.dest.av));\r
+ break;\r
+ default:\r
+ ret = -1;\r
+ goto out;\r
+ }\r
+ \r
+ ret = send(sock, (char *) &msg, sizeof msg, 0);\r
+ if (ret != sizeof msg)\r
+ goto out;\r
+\r
+ ret = recv(sock, (char *) &msg, sizeof msg, 0);\r
+ if (ret != sizeof msg)\r
+ goto out;\r
+\r
+ memcpy(dev_addr, &msg.src.dev, sizeof(*dev_addr));\r
+ *ah = msg.dest.av;\r
+ memcpy(data, &msg.data, sizeof(*data));\r
+ ret = 0;\r
+\r
+out:\r
+ lock_release(&lock);\r
+ return ret;\r
+}\r
+\r
+LIB_EXPORT\r
+int ib_acm_resolve_name(char *src, char *dest,\r
+ struct ib_acm_dev_addr *dev_addr, struct ibv_ah_attr *ah,\r
+ struct ib_acm_resolve_data *data)\r
+{\r
+ return acm_resolve((uint8_t *) src, (uint8_t *) dest,\r
+ ACM_EP_TYPE_NAME, dev_addr, ah, data);\r
+}\r
+\r
+LIB_EXPORT\r
+int ib_acm_resolve_ip(struct sockaddr *src, struct sockaddr *dest,\r
+ struct ib_acm_dev_addr *dev_addr, struct ibv_ah_attr *ah,\r
+ struct ib_acm_resolve_data *data)\r
+{\r
+ if (((struct sockaddr *) dest)->sa_family == AF_INET) {\r
+ return acm_resolve((uint8_t *) src, (uint8_t *) dest,\r
+ ACM_EP_TYPE_ADDRESS_IP, dev_addr, ah, data);\r
+ } else {\r
+ return acm_resolve((uint8_t *) src, (uint8_t *) dest,\r
+ ACM_EP_TYPE_ADDRESS_IP6, dev_addr, ah, data);\r
+ }\r
+}\r
+\r
+static int acm_query_path(struct ib_path_record *path, uint8_t query_sa)\r
+{\r
+ struct acm_query_msg msg;\r
+ int ret;\r
+\r
+ lock_acquire(&lock);\r
+ if (!ready && (ret = acm_init()))\r
+ goto out;\r
+\r
+ memset(&msg, 0, sizeof msg);\r
+ msg.hdr.version = ACM_VERSION;\r
+ msg.hdr.opcode = ACM_OP_QUERY;\r
+ msg.hdr.param = ACM_QUERY_PATH_RECORD | query_sa;\r
+\r
+ if (path->dgid.global.interface_id || path->dgid.global.subnet_prefix) {\r
+ msg.hdr.dest_type = ACM_EP_TYPE_GID;\r
+ } else if (path->dlid) {\r
+ msg.hdr.dest_type = ACM_EP_TYPE_LID;\r
+ } else {\r
+ ret = -1;\r
+ goto out;\r
+ }\r
+\r
+ if (path->sgid.global.interface_id || path->sgid.global.subnet_prefix) {\r
+ msg.hdr.src_type = ACM_EP_TYPE_GID;\r
+ } else if (path->slid) {\r
+ msg.hdr.src_type = ACM_EP_TYPE_LID;\r
+ } else {\r
+ ret = -1;\r
+ goto out;\r
+ }\r
+\r
+ msg.data.path = *path;\r
+ \r
+ ret = send(sock, (char *) &msg, sizeof msg, 0);\r
+ if (ret != sizeof msg)\r
+ goto out;\r
+\r
+ ret = recv(sock, (char *) &msg, sizeof msg, 0);\r
+ if (ret != sizeof msg)\r
+ goto out;\r
+\r
+ *path = msg.data.path;\r
+ ret = msg.hdr.status;\r
+\r
+out:\r
+ lock_release(&lock);\r
+ return ret;\r
+}\r
+\r
+LIB_EXPORT\r
+int ib_acm_query_path(struct ib_path_record *path)\r
+{\r
+ return acm_query_path(path, ACM_QUERY_SA);\r
+}\r
+\r
+LIB_EXPORT\r
+int ib_acm_resolve_path(struct ib_path_record *path)\r
+{\r
+ return acm_query_path(path, 0);\r
+}\r
+\r
+static struct ibv_context *acm_open_device(uint64_t guid)\r
+{\r
+ struct ibv_device **dev_array;\r
+ struct ibv_context *verbs = NULL;\r
+ int i, cnt;\r
+\r
+ dev_array = ibv_get_device_list(&cnt);\r
+ if (!dev_array)\r
+ return NULL;\r
+\r
+ for (i = 0; i < cnt; i++) {\r
+ if (guid == ibv_get_device_guid(dev_array[i])) {\r
+ verbs = ibv_open_device(dev_array[i]);\r
+ break;\r
+ }\r
+ }\r
+\r
+ ibv_free_device_list(dev_array);\r
+ return verbs;\r
+}\r
+\r
+LIB_EXPORT\r
+int ib_acm_convert_to_path(struct ib_acm_dev_addr *dev_addr,\r
+ struct ibv_ah_attr *ah, struct ib_acm_resolve_data *data,\r
+ struct ib_path_record *path)\r
+{\r
+ struct ibv_context *verbs;\r
+ struct ibv_port_attr attr;\r
+ int ret;\r
+\r
+ verbs = acm_open_device(dev_addr->guid);\r
+ if (!verbs)\r
+ return -1;\r
+\r
+ if (ah->is_global) {\r
+ path->dgid = ah->grh.dgid;\r
+ ret = ibv_query_gid(verbs, dev_addr->port_num, ah->grh.sgid_index, &path->sgid);\r
+ if (ret)\r
+ goto out;\r
+\r
+ path->flowlabel_hoplimit =\r
+ htonl(ah->grh.flow_label << 8 | (uint32_t) ah->grh.hop_limit);\r
+ path->tclass = ah->grh.traffic_class;\r
+ }\r
+\r
+ path->dlid = htons(ah->dlid);\r
+ ret = ibv_query_port(verbs, dev_addr->port_num, &attr);\r
+ if (ret)\r
+ goto out;\r
+\r
+ path->slid = htons(attr.lid | ah->src_path_bits);\r
+ path->reversible_numpath = IB_PATH_RECORD_REVERSIBLE | 1;\r
+ ret = ibv_query_pkey(verbs, dev_addr->port_num, dev_addr->pkey_index, &path->pkey);\r
+ if (ret)\r
+ goto out;\r
+\r
+ path->pkey = htons(path->pkey);\r
+ path->qosclass_sl = htons((uint16_t) ah->sl);\r
+ path->mtu = (2 << 6) | data->mtu;\r
+ path->rate = (2 << 6) | ah->static_rate;\r
+ path->packetlifetime = (2 << 6) | data->packet_lifetime;\r
+\r
+out:\r
+ ibv_close_device(verbs);\r
+ return ret;\r
+}\r
--- /dev/null
+/*\r
+ * Copyright (c) 2009 Intel Corporation. All rights reserved.\r
+ *\r
+ * This software is available to you under the OpenIB.org BSD license\r
+ * below:\r
+ *\r
+ * Redistribution and use in source and binary forms, with or\r
+ * without modification, are permitted provided that the following\r
+ * conditions are met:\r
+ *\r
+ * - Redistributions of source code must retain the above\r
+ * copyright notice, this list of conditions and the following\r
+ * disclaimer.\r
+ *\r
+ * - Redistributions in binary form must reproduce the above\r
+ * copyright notice, this list of conditions and the following\r
+ * disclaimer in the documentation and/or other materials\r
+ * provided with the distribution.\r
+ *\r
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,\r
+ * EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF\r
+ * MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AWV\r
+ * NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS\r
+ * BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN\r
+ * ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN\r
+ * CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE\r
+ * SOFTWARE.\r
+ */\r
+\r
+#include "..\..\..\..\etc\user\search.c"\r
+#include "..\..\..\..\etc\user\inet.c"\r
+#include "..\src\acm.c"\r
+\r
--- /dev/null
+TARGETNAME = ib_acme\r
+TARGETPATH = ..\..\..\..\bin\user\obj$(BUILD_ALT_DIR)\r
+TARGETTYPE = PROGRAM\r
+\r
+UMTYPE = console\r
+UMENTRY = main\r
+\r
+USE_MSVCRT = 1\r
+USE_STL = 1\r
+USE_NATIVE_EH = 1\r
+USE_IOSTREAM = 1\r
+\r
+SOURCES = ..\acme_windows.c\r
+ \r
+INCLUDES = ..;..\..\include;..\..\..\..\inc;..\..\..\..\inc\user;\\r
+ ..\..\..\libibverbs\include;..\..\..\librdmacm\include;\\r
+ ..\..\..\..\inc\user\linux;\r
+\r
+TARGETLIBS = \\r
+ $(SDK_LIB_PATH)\kernel32.lib \\r
+ $(SDK_LIB_PATH)\advapi32.lib \\r
+ $(SDK_LIB_PATH)\user32.lib \\r
+ $(SDK_LIB_PATH)\ole32.lib \\r
+ $(SDK_LIB_PATH)\ws2_32.lib \\r
+!if $(FREEBUILD)\r
+ $(TARGETPATH)\*\libibverbs.lib \\r
+ $(TARGETPATH)\*\librdmacm.lib \\r
+ $(TARGETPATH)\*\libibacm.lib \\r
+ $(TARGETPATH)\*\winverbs.lib\r
+!else\r
+ $(TARGETPATH)\*\libibverbsd.lib \\r
+ $(TARGETPATH)\*\librdmacmd.lib \\r
+ $(TARGETPATH)\*\libibacmd.lib \\r
+ $(TARGETPATH)\*\winverbsd.lib\r
+!endif\r
+\r
--- /dev/null
+#\r
+# DO NOT EDIT THIS FILE!!! Edit .\sources. if you want to add a new source\r
+# file to this component. This file merely indirects to the real make file\r
+# that is shared by all the driver components of the OpenIB Windows project.\r
+#\r
+\r
+!INCLUDE ..\..\..\..\inc\openib.def\r
--- /dev/null
+/*\r
+ * Copyright (c) 2009 Intel Corporation. All rights reserved.\r
+ *\r
+ * This software is available to you under the OpenIB.org BSD license\r
+ * below:\r
+ *\r
+ * Redistribution and use in source and binary forms, with or\r
+ * without modification, are permitted provided that the following\r
+ * conditions are met:\r
+ *\r
+ * - Redistributions of source code must retain the above\r
+ * copyright notice, this list of conditions and the following\r
+ * disclaimer.\r
+ *\r
+ * - Redistributions in binary form must reproduce the above\r
+ * copyright notice, this list of conditions and the following\r
+ * disclaimer in the documentation and/or other materials\r
+ * provided with the distribution.\r
+ *\r
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,\r
+ * EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF\r
+ * MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AWV\r
+ * NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS\r
+ * BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN\r
+ * ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN\r
+ * CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE\r
+ * SOFTWARE.\r
+ */\r
+\r
+#include "..\..\..\..\etc\user\getopt.c"\r
+#include "..\src\acme.c"\r
+#include "..\..\..\..\etc\user\inet.c"\r
+#include <rdma/winverbs.h>\r
+\r
+extern struct ibv_context **verbs;\r
+extern int dev_cnt;\r
+\r
+int gen_addr_ip(FILE *f)\r
+{\r
+ WV_DEVICE_ADDRESS devaddr;\r
+ IWVProvider *prov;\r
+ HRESULT hr;\r
+ struct addrinfo *res, hint, *ai;\r
+ char ip[sizeof "ffff:ffff:ffff:ffff:ffff:ffff:ffff:ffff"];\r
+ int i;\r
+\r
+ hr = WvGetObject(&IID_IWVProvider, (LPVOID *) &prov);\r
+ if (FAILED(hr))\r
+ return hr;\r
+\r
+ memset(&hint, 0, sizeof hint);\r
+ hint.ai_protocol = IPPROTO_TCP;\r
+\r
+ hr = getaddrinfo("..localmachine", NULL, &hint, &res);\r
+ if (hr) {\r
+ printf("getaddrinfo error %d\n", hr);\r
+ goto release;\r
+ }\r
+\r
+ for (ai = res; ai; ai = ai->ai_next) {\r
+ switch (ai->ai_family) {\r
+ case AF_INET:\r
+ inet_ntop(ai->ai_family,\r
+ &((struct sockaddr_in *) ai->ai_addr)->sin_addr, ip, sizeof ip);\r
+ break;\r
+ case AF_INET6:\r
+ inet_ntop(ai->ai_family,\r
+ &((struct sockaddr_in6 *) ai->ai_addr)->sin6_addr, ip, sizeof ip);\r
+ break;\r
+ default:\r
+ continue;\r
+ }\r
+\r
+ hr = prov->lpVtbl->TranslateAddress(prov, ai->ai_addr, &devaddr);\r
+ if (FAILED(hr))\r
+ continue;\r
+\r
+ for (i = 0; i < dev_cnt; i++) {\r
+ if (devaddr.DeviceGuid == ibv_get_device_guid(verbs[i]->device)) {\r
+ printf("%s %s %d 0x%x\n", ip, verbs[i]->device->name,\r
+ devaddr.PortNumber, ntohs(devaddr.Pkey));\r
+ fprintf(f, "%s %s %d 0x%x\n", ip, verbs[i]->device->name,\r
+ devaddr.PortNumber, ntohs(devaddr.Pkey));\r
+ }\r
+ }\r
+ }\r
+\r
+ hr = 0;\r
+ freeaddrinfo(res);\r
+release:\r
+ prov->lpVtbl->Release(prov);\r
+ return hr;\r
+}\r
--- /dev/null
+DIRS = service library acme\r
--- /dev/null
+/*\r
+ * Copyright (c) 2009 Intel Corporation. All rights reserved.\r
+ *\r
+ * This software is available to you under the OpenIB.org BSD license\r
+ * below:\r
+ *\r
+ * Redistribution and use in source and binary forms, with or\r
+ * without modification, are permitted provided that the following\r
+ * conditions are met:\r
+ *\r
+ * - Redistributions of source code must retain the above\r
+ * copyright notice, this list of conditions and the following\r
+ * disclaimer.\r
+ *\r
+ * - Redistributions in binary form must reproduce the above\r
+ * copyright notice, this list of conditions and the following\r
+ * disclaimer in the documentation and/or other materials\r
+ * provided with the distribution.\r
+ *\r
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,\r
+ * EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF\r
+ * MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AWV\r
+ * NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS\r
+ * BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN\r
+ * ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN\r
+ * CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE\r
+ * SOFTWARE.\r
+ */\r
+\r
+#include "..\src\libacm.c"\r
+\r
+extern void LIB_DESTRUCTOR acm_cleanup();\r
+lock_t lock;\r
+\r
+BOOL WINAPI DllMain(HINSTANCE hInstance, DWORD dwReason, LPVOID lpReserved)\r
+{\r
+ UNREFERENCED_PARAMETER(hInstance);\r
+ UNREFERENCED_PARAMETER(lpReserved);\r
+\r
+ switch (dwReason) {\r
+ case DLL_PROCESS_ATTACH:\r
+ lock_init(&lock);\r
+ break;\r
+ case DLL_PROCESS_DETACH:\r
+ acm_cleanup();\r
+ break;\r
+ default:\r
+ break;\r
+ }\r
+\r
+ return TRUE;\r
+}\r
--- /dev/null
+!if $(FREEBUILD)\r
+TARGETNAME = libibacm\r
+!else\r
+TARGETNAME = libibacmd\r
+!endif\r
+\r
+TARGETPATH = ..\..\..\..\bin\user\obj$(BUILD_ALT_DIR)\r
+TARGETTYPE = DYNLINK\r
+\r
+!if $(_NT_TOOLS_VERSION) == 0x700\r
+DLLDEF = $O\acm_exports.def\r
+!else\r
+DLLDEF = $(OBJ_PATH)\$O\libacm_exports.def\r
+!endif\r
+\r
+DLLENTRY = DllMain\r
+USE_MSVCRT = 1\r
+\r
+SOURCES = ..\libacm_windows.c\r
+\r
+INCLUDES = ..;..\..\include;..\..\..\..\inc;..\..\..\..\inc\user;\\r
+ ..\..\..\libibverbs\include;\r
+ \r
+TARGETLIBS = \\r
+ $(SDK_LIB_PATH)\kernel32.lib \\r
+ $(SDK_LIB_PATH)\ws2_32.lib \\r
+ $(SDK_LIB_PATH)\iphlpapi.lib \\r
+!if $(FREEBUILD)\r
+ $(TARGETPATH)\*\libibverbs.lib\r
+!else\r
+ $(TARGETPATH)\*\libibverbsd.lib\r
+!endif\r
--- /dev/null
+/*\r
+ * Copyright (c) 2008 Intel Corporation. All rights reserved.\r
+ *\r
+ * This software is available to you under the OpenIB.org BSD license\r
+ * below:\r
+ *\r
+ * Redistribution and use in source and binary forms, with or\r
+ * without modification, are permitted provided that the following\r
+ * conditions are met:\r
+ *\r
+ * - Redistributions of source code must retain the above\r
+ * copyright notice, this list of conditions and the following\r
+ * disclaimer.\r
+ *\r
+ * - Redistributions in binary form must reproduce the above\r
+ * copyright notice, this list of conditions and the following\r
+ * disclaimer in the documentation and/or other materials\r
+ * provided with the distribution.\r
+ *\r
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,\r
+ * EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF\r
+ * MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND\r
+ * NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS\r
+ * BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN\r
+ * ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN\r
+ * CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE\r
+ * SOFTWARE.\r
+ */\r
+\r
+\r
+#include <oib_ver.h>\r
+\r
+#define VER_FILETYPE VFT_DLL\r
+#define VER_FILESUBTYPE VFT2_UNKNOWN\r
+\r
+#ifdef _DEBUG_\r
+#define VER_FILEDESCRIPTION_STR "libibacm (Debug)"\r
+#define VER_INTERNALNAME_STR "libibacmd.dll"\r
+#define VER_ORIGINALFILENAME_STR "libibacmd.dll"\r
+#else\r
+#define VER_FILEDESCRIPTION_STR "libibacm"\r
+#define VER_INTERNALNAME_STR "libibacm.dll"\r
+#define VER_ORIGINALFILENAME_STR "libibacm.dll"\r
+#endif\r
+\r
+#include <common.ver>\r
--- /dev/null
+/*\r
+ * Copyright (c) 2008 Intel Corporation. All rights reserved.\r
+ *\r
+ * This software is available to you under the OpenIB.org BSD license\r
+ * below:\r
+ *\r
+ * Redistribution and use in source and binary forms, with or\r
+ * without modification, are permitted provided that the following\r
+ * conditions are met:\r
+ *\r
+ * - Redistributions of source code must retain the above\r
+ * copyright notice, this list of conditions and the following\r
+ * disclaimer.\r
+ *\r
+ * - Redistributions in binary form must reproduce the above\r
+ * copyright notice, this list of conditions and the following\r
+ * disclaimer in the documentation and/or other materials\r
+ * provided with the distribution.\r
+ *\r
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,\r
+ * EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF\r
+ * MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AWV\r
+ * NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS\r
+ * BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN\r
+ * ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN\r
+ * CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE\r
+ * SOFTWARE.\r
+ */\r
+\r
+LIBRARY LIBIBACM.DLL\r
+\r
+EXPORTS\r
+ DllCanUnloadNow PRIVATE\r
+ DllGetClassObject PRIVATE\r
--- /dev/null
+#if DBG\r
+LIBRARY libibacmd.dll\r
+#else\r
+LIBRARY libibacm.dll\r
+#endif\r
+\r
+#ifndef _WIN64\r
+EXPORTS\r
+ib_acm_resolve_name\r
+ib_acm_resolve_ip\r
+#endif\r
--- /dev/null
+#\r
+# DO NOT EDIT THIS FILE!!! Edit .\sources. if you want to add a new source\r
+# file to this component. This file merely indirects to the real make file\r
+# that is shared by all the driver components of the OpenIB Windows project.\r
+#\r
+\r
+!INCLUDE ..\..\..\..\inc\openib.def\r
--- /dev/null
+/*\r
+ * Copyright (c) 2009 Intel Corporation. All rights reserved.\r
+ *\r
+ * This software is available to you under the OpenFabrics.org BSD license\r
+ * below:\r
+ *\r
+ * Redistribution and use in source and binary forms, with or\r
+ * without modification, are permitted provided that the following\r
+ * conditions are met:\r
+ *\r
+ * - Redistributions of source code must retain the above\r
+ * copyright notice, this list of conditions and the following\r
+ * disclaimer.\r
+ *\r
+ * - Redistributions in binary form must reproduce the above\r
+ * copyright notice, this list of conditions and the following\r
+ * disclaimer in the documentation and/or other materials\r
+ * provided with the distribution.\r
+ *\r
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,\r
+ * EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF\r
+ * MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AWV\r
+ * NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS\r
+ * BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN\r
+ * ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN\r
+ * CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE\r
+ * SOFTWARE.\r
+ */\r
+\r
+#if !defined(MC_CM_OSD_H)\r
+#define MC_CM_OSD_H\r
+\r
+#include <windows.h>\r
+#include <process.h>\r
+#include <winsock2.h>\r
+\r
+#define __func__ __FUNCTION__\r
+#define LIB_DESTRUCTOR\r
+#define CDECL_FUNC __cdecl\r
+\r
+typedef struct { volatile LONG val; } atomic_t;\r
+#define atomic_inc(v) InterlockedIncrement(&(v)->val)\r
+#define atomic_dec(v) InterlockedDecrement(&(v)->val)\r
+#define atomic_get(v) ((v)->val)\r
+#define atomic_set(v, s) ((v)->val = s)\r
+\r
+#define event_t HANDLE\r
+#define event_init(e) *(e) = CreateEvent(NULL, FALSE, FALSE, NULL)\r
+#define event_signal(e) SetEvent(*(e))\r
+#define event_wait(e, t) WaitForSingleObject(*(e), t) \r
+\r
+#define lock_t CRITICAL_SECTION\r
+#define lock_init InitializeCriticalSection\r
+#define lock_acquire EnterCriticalSection\r
+#define lock_release LeaveCriticalSection\r
+\r
+static __inline int osd_init()\r
+{\r
+ WSADATA wsadata;\r
+ return WSAStartup(MAKEWORD(2, 2), &wsadata);\r
+}\r
+\r
+static __inline void osd_close()\r
+{\r
+ WSACleanup();\r
+}\r
+\r
+#define stricmp _stricmp\r
+#define strnicmp _strnicmp\r
+\r
+#define socket_errno WSAGetLastError\r
+#define SHUT_RDWR SD_BOTH\r
+\r
+static __inline UINT64 time_stamp_us(void)\r
+{\r
+ LARGE_INTEGER cnt, freq;\r
+ QueryPerformanceFrequency(&freq);\r
+ QueryPerformanceCounter(&cnt);\r
+ return (UINT64) cnt.QuadPart / freq.QuadPart * 1000000;\r
+}\r
+\r
+#define time_stamp_ms() (time_stamp_us() * 1000)\r
+\r
+#define getpid() ((int) GetCurrentProcessId())\r
+#define beginthread(func, arg) (int) _beginthread(func, 0, arg)\r
+#define container_of CONTAINING_RECORD\r
+\r
+#endif /* MC_CM_OSD_H */\r
--- /dev/null
+TARGETNAME = ib_acm\r
+TARGETPATH = ..\..\..\..\bin\user\obj$(BUILD_ALT_DIR)\r
+TARGETTYPE = PROGRAM\r
+\r
+UMTYPE = console\r
+UMENTRY = main\r
+\r
+USE_MSVCRT = 1\r
+USE_STL = 1\r
+USE_NATIVE_EH = 1\r
+USE_IOSTREAM = 1\r
+\r
+SOURCES = ..\acm_windows.c\r
+\r
+INCLUDES = ..;..\..\include;..\..\..\..\inc;..\..\..\..\inc\user;\\r
+ ..\..\..\..\inc\iba;..\..\..\libibverbs\include;\\r
+ ..\..\..\libibumad\include;..\..\..\..\inc\user\linux\r
+\r
+TARGETLIBS = \\r
+ $(SDK_LIB_PATH)\kernel32.lib \\r
+ $(SDK_LIB_PATH)\ws2_32.lib \\r
+ $(SDK_LIB_PATH)\iphlpapi.lib \\r
+!if $(FREEBUILD)\r
+ $(TARGETPATH)\*\libibverbs.lib \\r
+ $(TARGETPATH)\*\libibumad.lib \\r
+ $(TARGETPATH)\*\complib.lib\r
+!else\r
+ $(TARGETPATH)\*\libibverbsd.lib \\r
+ $(TARGETPATH)\*\libibumadd.lib \\r
+ $(TARGETPATH)\*\complibd.lib\r
+!endif\r
--- /dev/null
+#\r
+# DO NOT EDIT THIS FILE!!! Edit .\sources. if you want to add a new source\r
+# file to this component. This file merely indirects to the real make file\r
+# that is shared by all the driver components of the OpenIB Windows project.\r
+#\r
+\r
+!INCLUDE ..\..\..\..\inc\openib.def\r
projects / ~shefty / ibacm.git / commitdiff