--- /dev/null
+Bottom: af83bf966d67892a75ec2686b099da14c206b9f4
+Top: 43c15e3c62fe2a8ce56efcf9fa62e4a5203bc355
+Author: Hal Rosenstock <hal@mellanox.com>
+Date: 2013-06-12 21:37:37 +0300
+
+From 375c5116236e3ab40e12dc4cc8dbd86986ac7384 Mon Sep 17 00:00:00 2001
+Subject: [PATCH 01/15 acm] acm_notes.txt: Change DOS formatting to unix
+ formatting
+
+Signed-off-by: Hal Rosenstock <hal@mellanox.com>
+
+
+---
+
+diff --git a/acm_notes.txt b/acm_notes.txt
+index ed519ab..99eaf9a 100644
+--- a/acm_notes.txt
++++ b/acm_notes.txt
+@@ -1,130 +1,130 @@
+-Assistant for InfiniBand Communication Management (IB ACM)\r
+-\r
+-Note: The IB ACM should be considered experimental.\r
+-\r
+-\r
+-Overview\r
+---------\r
+-The IB ACM package implements and provides a framework for experimental name,\r
+-address, and route resolution services over InfiniBand. It is intended to\r
+-address connection setup scalability issues running MPI applications on\r
+-large clusters. The IB ACM provides information needed to establish a\r
+-connection, but does not implement the CM protocol.\r
+-\r
+-The librdmacm can invoke IB ACM services when built using the --with-ib_acm\r
+-option. The IB ACM services tie in under the rdma_resolve_addr,\r
+-rdma_resolve_route, and rdma_getaddrinfo routines. For maximum benefit,\r
+-the rdma_getaddrinfo routine should be used, however existing applications\r
+-should still see significant connection scaling benefits using the calls\r
+-available in librdmacm 1.0.11 and previous releases.\r
+-\r
+-The IB ACM is focused on being scalable and efficient. The current\r
+-implementation limits network traffic, SA interactions, and centralized\r
+-services. ACM supports multiple resolution protocols in order to handle\r
+-different fabric topologies.\r
+-\r
+-This release is limited in its handling of dynamic changes.\r
+-\r
+-The IB ACM package is comprised of two components: the ib_acm service\r
+-and a test/configuration utility - ib_acme. Both are userspace components\r
+-and are available for Linux and Windows. Additional details are given below.\r
+-\r
+-\r
+-Quick Start Guide\r
+------------------\r
+-1. Prerequisites: libibverbs and libibumad must be installed.\r
+- The IB stack should be running with IPoIB configured.\r
+- These steps assume that the user has administrative privileges.\r
+-2. Install the IB ACM package\r
+- This installs ib_acm, and ib_acme.\r
+-3. Run ib_acme -A -O\r
+- This will generate IB ACM address and options configuration files.\r
+- (acm_addr.cfg and acm_opts.cfg)\r
+-4. Run ib_acm and leave running.\r
+- ib_acm will eventually be converted to a service/daemon, but for now\r
+- is a userspace application. Because ib_acm uses the libibumad\r
+- interfaces, it should be run with administrative privileges.\r
+-5. Optionally, run ib_acme -s <source_ip> -d <dest_ip> -v\r
+- This will verify that the ib_acm service is running.\r
+-5. Install librdmacm.\r
+- The librdmacm will automatically use the ib_acm service.\r
+- On failures, the librdmacm will fall back to normal resolution.\r
+-\r
+-\r
+-Details\r
+--------\r
+-ib_acme:\r
+-The ib_acme program serves a dual role. It acts as a utility to test\r
+-ib_acm operation and help verify if the ib_acm service and selected\r
+-protocol is usable for a given cluster configuration. Additionally,\r
+-it automatically generates ib_acm configuration files to assist with\r
+-or eliminate manual setup.\r
+-\r
+-\r
+-acm configuration files:\r
+-The ib_acm service relies on two configuration files.\r
+-\r
+-The acm_addr.cfg file contains name and address mappings for each IB\r
+-<device, port, pkey> endpoint. Although the names in the acm_addr.cfg\r
+-file can be anything, ib_acme maps the host name and IP addresses to\r
+-the IB endpoints.\r
+-\r
+-The acm_opts.cfg file provides a set of configurable options for the\r
+-ib_acm service, such as timeout, number of retries, logging level, etc.\r
+-ib_acme generates the acm_opts.cfg file using static information. A\r
+-future enhancement would adjust options based on the current system\r
+-and cluster size. \r
+-\r
+-\r
+-ib_acm:\r
+-The ib_acm service is responsible for resolving names and addresses to\r
+-InfiniBand path information and caching such data. It is currently\r
+-implemented as an executable application, but is a conceptual service\r
+-or daemon that should execute with administrative privileges.\r
+-\r
+-The ib_acm implements a client interface over TCP sockets, which is\r
+-abstracted by the librdmacm library. One or more back-end protocols are\r
+-used by the ib_acm service to satisfy user requests. Although the\r
+-ib_acm supports standard SA path record queries on the back-end, it\r
+-provides an experimental multicast resolution protocol in hope of\r
+-achieving greater scalability. The latter is not usable on all fabric\r
+-topologies, specifically ones that may not have reversible paths.\r
+-Users should use the ib_acme utility to verify that multicast protocol\r
+-is usable before running other applications.\r
+-\r
+-Conceptually, the ib_acm service implements an ARP like protocol and either\r
+-uses IB multicast records to construct path record data or queries the\r
+-SA directly, depending on the selected route protocol. By default, the\r
+-ib_acm services uses and caches SA path record queries.\r
+-\r
+-Specifically, all IB endpoints join a number of multicast groups.\r
+-Multicast groups differ based on rates, mtu, sl, etc., and are prioritized.\r
+-All participating endpoints must be able to communicate on the lowest\r
+-priority multicast group. The ib_acm assigns one or more names/addresses\r
+-to each IB endpoint using the acm_addr.cfg file. Clients provide source\r
+-and destination names or addresses as input to the service, and receive\r
+-as output path record data.\r
+-\r
+-The service maps a client's source name/address to a local IB endpoint.\r
+-If a client does not provide a source address, then the ib_acm service\r
+-will select one based on the destination and local routing tables. If the\r
+-destination name/address is not cached locally, it sends a multicast\r
+-request out on the lowest priority multicast group on the local endpoint.\r
+-The request carries a list of multicast groups that the sender can use.\r
+-The recipient of the request selects the highest priority multicast group\r
+-that it can use as well and returns that information directly to the sender.\r
+-The request data is cached by all endpoints that receive the multicast\r
+-request message. The source endpoint also caches the response and uses\r
+-the multicast group that was selected to construct or obtain path record\r
+-data, which is returned to the client.\r
+-\r
+-The current implementation of the IB ACM has several additional restrictions:\r
+-- The ib_acm is limited in its handling of dynamic changes;\r
+- the ib_acm must be stopped and restarted if a cluster is reconfigured.\r
+-- Cached data does not timed out and is only updated if a new resolution\r
+- request is received from a different QPN than a cached request.\r
+-- Support for IPv6 has not been verified.\r
+-- The number of addresses that can be assigned to a single endpoint is\r
+- limited to 4.\r
+-- The number of multicast groups that an endpoint can support is limited to 2.\r
+-\r
++Assistant for InfiniBand Communication Management (IB ACM)
++
++Note: The IB ACM should be considered experimental.
++
++
++Overview
++--------
++The IB ACM package implements and provides a framework for experimental name,
++address, and route resolution services over InfiniBand. It is intended to
++address connection setup scalability issues running MPI applications on
++large clusters. The IB ACM provides information needed to establish a
++connection, but does not implement the CM protocol.
++
++The librdmacm can invoke IB ACM services when built using the --with-ib_acm
++option. The IB ACM services tie in under the rdma_resolve_addr,
++rdma_resolve_route, and rdma_getaddrinfo routines. For maximum benefit,
++the rdma_getaddrinfo routine should be used, however existing applications
++should still see significant connection scaling benefits using the calls
++available in librdmacm 1.0.11 and previous releases.
++
++The IB ACM is focused on being scalable and efficient. The current
++implementation limits network traffic, SA interactions, and centralized
++services. ACM supports multiple resolution protocols in order to handle
++different fabric topologies.
++
++This release is limited in its handling of dynamic changes.
++
++The IB ACM package is comprised of two components: the ib_acm service
++and a test/configuration utility - ib_acme. Both are userspace components
++and are available for Linux and Windows. Additional details are given below.
++
++
++Quick Start Guide
++-----------------
++1. Prerequisites: libibverbs and libibumad must be installed.
++ The IB stack should be running with IPoIB configured.
++ These steps assume that the user has administrative privileges.
++2. Install the IB ACM package
++ This installs ib_acm, and ib_acme.
++3. Run ib_acme -A -O
++ This will generate IB ACM address and options configuration files.
++ (acm_addr.cfg and acm_opts.cfg)
++4. Run ib_acm and leave running.
++ ib_acm will eventually be converted to a service/daemon, but for now
++ is a userspace application. Because ib_acm uses the libibumad
++ interfaces, it should be run with administrative privileges.
++5. Optionally, run ib_acme -s <source_ip> -d <dest_ip> -v
++ This will verify that the ib_acm service is running.
++5. Install librdmacm.
++ The librdmacm will automatically use the ib_acm service.
++ On failures, the librdmacm will fall back to normal resolution.
++
++
++Details
++-------
++ib_acme:
++The ib_acme program serves a dual role. It acts as a utility to test
++ib_acm operation and help verify if the ib_acm service and selected
++protocol is usable for a given cluster configuration. Additionally,
++it automatically generates ib_acm configuration files to assist with
++or eliminate manual setup.
++
++
++acm configuration files:
++The ib_acm service relies on two configuration files.
++
++The acm_addr.cfg file contains name and address mappings for each IB
++<device, port, pkey> endpoint. Although the names in the acm_addr.cfg
++file can be anything, ib_acme maps the host name and IP addresses to
++the IB endpoints.
++
++The acm_opts.cfg file provides a set of configurable options for the
++ib_acm service, such as timeout, number of retries, logging level, etc.
++ib_acme generates the acm_opts.cfg file using static information. A
++future enhancement would adjust options based on the current system
++and cluster size.
++
++
++ib_acm:
++The ib_acm service is responsible for resolving names and addresses to
++InfiniBand path information and caching such data. It is currently
++implemented as an executable application, but is a conceptual service
++or daemon that should execute with administrative privileges.
++
++The ib_acm implements a client interface over TCP sockets, which is
++abstracted by the librdmacm library. One or more back-end protocols are
++used by the ib_acm service to satisfy user requests. Although the
++ib_acm supports standard SA path record queries on the back-end, it
++provides an experimental multicast resolution protocol in hope of
++achieving greater scalability. The latter is not usable on all fabric
++topologies, specifically ones that may not have reversible paths.
++Users should use the ib_acme utility to verify that multicast protocol
++is usable before running other applications.
++
++Conceptually, the ib_acm service implements an ARP like protocol and either
++uses IB multicast records to construct path record data or queries the
++SA directly, depending on the selected route protocol. By default, the
++ib_acm services uses and caches SA path record queries.
++
++Specifically, all IB endpoints join a number of multicast groups.
++Multicast groups differ based on rates, mtu, sl, etc., and are prioritized.
++All participating endpoints must be able to communicate on the lowest
++priority multicast group. The ib_acm assigns one or more names/addresses
++to each IB endpoint using the acm_addr.cfg file. Clients provide source
++and destination names or addresses as input to the service, and receive
++as output path record data.
++
++The service maps a client's source name/address to a local IB endpoint.
++If a client does not provide a source address, then the ib_acm service
++will select one based on the destination and local routing tables. If the
++destination name/address is not cached locally, it sends a multicast
++request out on the lowest priority multicast group on the local endpoint.
++The request carries a list of multicast groups that the sender can use.
++The recipient of the request selects the highest priority multicast group
++that it can use as well and returns that information directly to the sender.
++The request data is cached by all endpoints that receive the multicast
++request message. The source endpoint also caches the response and uses
++the multicast group that was selected to construct or obtain path record
++data, which is returned to the client.
++
++The current implementation of the IB ACM has several additional restrictions:
++- The ib_acm is limited in its handling of dynamic changes;
++ the ib_acm must be stopped and restarted if a cluster is reconfigured.
++- Cached data does not timed out and is only updated if a new resolution
++ request is received from a different QPN than a cached request.
++- Support for IPv6 has not been verified.
++- The number of addresses that can be assigned to a single endpoint is
++ limited to 4.
++- The number of multicast groups that an endpoint can support is limited to 2.
++
projects / ~shefty / ibacm.git / commitdiff