--- /dev/null
+.TH OPENSM 8 "October 22, 2009" "OpenIB" "OpenIB Management"\r
+\r
+.SH NAME\r
+opensm \- InfiniBand subnet manager and administration (SM/SA)\r
+\r
+.SH SYNOPSIS\r
+.B opensm\r
+[\-\-version]]\r
+[\-F | \-\-config <file_name>]\r
+[\-c(reate-config) <file_name>]\r
+[\-g(uid) <GUID in hex>]\r
+[\-l(mc) <LMC>]\r
+[\-p(riority) <PRIORITY>]\r
+[\-smkey <SM_Key>]\r
+[\-\-sm_sl <SL number>]\r
+[\-r(eassign_lids)]\r
+[\-R <engine name(s)> | \-\-routing_engine <engine name(s)>]\r
+[\-\-do_mesh_analysis]\r
+[\-\-lash_start_vl <vl number>]\r
+[\-A | \-\-ucast_cache]\r
+[\-z | \-\-connect_roots]\r
+[\-M <file name> | \-\-lid_matrix_file <file name>]\r
+[\-U <file name> | \-\-lfts_file <file name>]\r
+[\-S | \-\-sadb_file <file name>]\r
+[\-a | \-\-root_guid_file <path to file>]\r
+[\-u | \-\-cn_guid_file <path to file>]\r
+[\-G | \-\-io_guid_file <path to file>]\r
+[\-H | \-\-max_reverse_hops <max reverse hops allowed>]\r
+[\-X | \-\-guid_routing_order_file <path to file>]\r
+[\-m | \-\-ids_guid_file <path to file>]\r
+[\-o(nce)]\r
+[\-s(weep) <interval>]\r
+[\-t(imeout) <milliseconds>]\r
+[\-\-retries <number>]\r
+[\-maxsmps <number>]\r
+[\-console [off | local | socket | loopback]]\r
+[\-console-port <port>]\r
+[\-i(gnore-guids) <equalize-ignore-guids-file>]\r
+[\-w | \-\-hop_weights_file <path to file>]\r
+[\-f <log file path> | \-\-log_file <log file path> ]\r
+[\-L | \-\-log_limit <size in MB>] [\-e(rase_log_file)]\r
+[\-P(config) <partition config file> ]\r
+[\-N | \-\-no_part_enforce]\r
+[\-Q | \-\-qos [\-Y | \-\-qos_policy_file <file name>]]\r
+[\-y | \-\-stay_on_fatal]\r
+[\-B | \-\-daemon]\r
+[\-I | \-\-inactive]\r
+[\-\-perfmgr]\r
+[\-\-perfmgr_sweep_time_s <seconds>]\r
+[\-\-prefix_routes_file <path>]\r
+[\-\-consolidate_ipv6_snm_req]\r
+[\-v(erbose)] [\-V] [\-D <flags>] [\-d(ebug) <number>]\r
+[\-h(elp)] [\-?]\r
+\r
+.SH DESCRIPTION\r
+.PP\r
+opensm is an InfiniBand compliant Subnet Manager and Administration,\r
+and runs on top of OpenIB.\r
+\r
+opensm provides an implementation of an InfiniBand Subnet Manager and\r
+Administration. Such a software entity is required to run for in order\r
+to initialize the InfiniBand hardware (at least one per each\r
+InfiniBand subnet).\r
+\r
+opensm also now contains an experimental version of a performance\r
+manager as well.\r
+\r
+opensm defaults were designed to meet the common case usage on clusters with up to a few hundred nodes. Thus, in this default mode, opensm will scan the IB\r
+fabric, initialize it, and sweep occasionally for changes.\r
+\r
+opensm attaches to a specific IB port on the local machine and configures only\r
+the fabric connected to it. (If the local machine has other IB ports,\r
+opensm will ignore the fabrics connected to those other ports). If no port is\r
+specified, it will select the first "best" available port.\r
+\r
+opensm can present the available ports and prompt for a port number to\r
+attach to.\r
+\r
+By default, the run is logged to two files: /var/log/messages and /var/log/opensm.log.\r
+The first file will register only general major events, whereas the second\r
+will include details of reported errors. All errors reported in this second\r
+file should be treated as indicators of IB fabric health issues.\r
+(Note that when a fatal and non-recoverable error occurs, opensm will exit.)\r
+Both log files should include the message "SUBNET UP" if opensm was able to\r
+setup the subnet correctly.\r
+\r
+.SH OPTIONS\r
+\r
+.PP\r
+.TP\r
+\fB\-\-version\fR\r
+Prints OpenSM version and exits.\r
+.TP\r
+\fB\-F\fR, \fB\-\-config\fR <config file>\r
+The name of the OpenSM config file. When not specified\r
+\fB\% @OPENSM_CONFIG_DIR@/@OPENSM_CONFIG_FILE@\fP will be used (if exists).\r
+.TP\r
+\fB\-c\fR, \fB\-\-create-config\fR <file name>\r
+OpenSM will dump its configuration to the specified file and exit.\r
+This is a way to generate OpenSM configuration file template.\r
+.TP\r
+\fB\-g\fR, \fB\-\-guid\fR <GUID in hex>\r
+This option specifies the local port GUID value\r
+with which OpenSM should bind. OpenSM may be\r
+bound to 1 port at a time.\r
+If GUID given is 0, OpenSM displays a list\r
+of possible port GUIDs and waits for user input.\r
+Without -g, OpenSM tries to use the default port.\r
+.TP\r
+\fB\-l\fR, \fB\-\-lmc\fR <LMC value>\r
+This option specifies the subnet's LMC value.\r
+The number of LIDs assigned to each port is 2^LMC.\r
+The LMC value must be in the range 0-7.\r
+LMC values > 0 allow multiple paths between ports.\r
+LMC values > 0 should only be used if the subnet\r
+topology actually provides multiple paths between\r
+ports, i.e. multiple interconnects between switches.\r
+Without -l, OpenSM defaults to LMC = 0, which allows\r
+one path between any two ports.\r
+.TP\r
+\fB\-p\fR, \fB\-\-priority\fR <Priority value>\r
+This option specifies the SM\'s PRIORITY.\r
+This will effect the handover cases, where master\r
+is chosen by priority and GUID. Range goes from 0\r
+(default and lowest priority) to 15 (highest).\r
+.TP\r
+\fB\-smkey\fR <SM_Key value>\r
+This option specifies the SM\'s SM_Key (64 bits).\r
+This will effect SM authentication.\r
+Note that OpenSM version 3.2.1 and below used the default value '1'\r
+in a host byte order, it is fixed now but you may need this option to\r
+interoperate with old OpenSM running on a little endian machine.\r
+.TP\r
+\fB\-\-sm_sl\fR <SL number>\r
+This option sets the SL to use for communication with the SM/SA.\r
+Defaults to 0.\r
+.TP\r
+\fB\-r\fR, \fB\-\-reassign_lids\fR\r
+This option causes OpenSM to reassign LIDs to all\r
+end nodes. Specifying -r on a running subnet\r
+may disrupt subnet traffic.\r
+Without -r, OpenSM attempts to preserve existing\r
+LID assignments resolving multiple use of same LID.\r
+.TP\r
+\fB\-R\fR, \fB\-\-routing_engine\fR <Routing engine names>\r
+This option chooses routing engine(s) to use instead of Min Hop\r
+algorithm (default). Multiple routing engines can be specified\r
+separated by commas so that specific ordering of routing algorithms\r
+will be tried if earlier routing engines fail.\r
+Supported engines: minhop, updn, file, ftree, lash, dor\r
+.TP\r
+\fB\-\-do_mesh_analysis\fR\r
+This option enables additional analysis for the lash routing engine to\r
+precondition switch port assignments in regular cartesian meshes which\r
+may reduce the number of SLs required to give a deadlock free routing.\r
+.TP\r
+\fB\-\-lash_start_vl\fR <vl number>\r
+This option sets the starting VL to use for the lash routing algorithm.\r
+Defaults to 0.\r
+.TP\r
+\fB\-A\fR, \fB\-\-ucast_cache\fR\r
+This option enables unicast routing cache and prevents routing\r
+recalculation (which is a heavy task in a large cluster) when\r
+there was no topology change detected during the heavy sweep, or\r
+when the topology change does not require new routing calculation,\r
+e.g. when one or more CAs/RTRs/leaf switches going down, or one or\r
+more of these nodes coming back after being down.\r
+A very common case that is handled by the unicast routing cache\r
+is host reboot, which otherwise would cause two full routing\r
+recalculations: one when the host goes down, and the other when\r
+the host comes back online.\r
+.TP\r
+\fB\-z\fR, \fB\-\-connect_roots\fR\r
+This option enforces routing engines (up/down and\r
+fat-tree) to make connectivity between root switches and in\r
+this way to be fully IBA complaint. In many cases this can\r
+violate "pure" deadlock free algorithm, so use it carefully.\r
+.TP\r
+\fB\-M\fR, \fB\-\-lid_matrix_file\fR <file name>\r
+This option specifies the name of the lid matrix dump file\r
+from where switch lid matrices (min hops tables will be\r
+loaded.\r
+.TP\r
+\fB\-U\fR, \fB\-\-lfts_file\fR <file name>\r
+This option specifies the name of the LFTs file\r
+from where switch forwarding tables will be loaded.\r
+.TP\r
+\fB\-S\fR, \fB\-\-sadb_file\fR <file name>\r
+This option specifies the name of the SA DB dump file\r
+from where SA database will be loaded.\r
+.TP\r
+\fB\-a\fR, \fB\-\-root_guid_file\fR <file name>\r
+Set the root nodes for the Up/Down or Fat-Tree routing\r
+algorithm to the guids provided in the given file (one to a line).\r
+.TP\r
+\fB\-u\fR, \fB\-\-cn_guid_file\fR <file name>\r
+Set the compute nodes for the Fat-Tree routing algorithm\r
+to the guids provided in the given file (one to a line).\r
+.TP\r
+\fB\-G\fR, \fB\-\-io_guid_file\fR <file name>\r
+Set the I/O nodes for the Fat-Tree routing algorithm\r
+to the guids provided in the given file (one to a line).\r
+I/O nodes are non-CN nodes allowed to use up to max_reverse_hops switches\r
+the wrong way around to improve connectivity.\r
+.TP\r
+\fB\-H\fR, \fB\-\-max_reverse_hops\fR <file name>\r
+Set the maximum number of reverse hops an I/O node is allowed\r
+to make. A reverse hop is the use of a switch the wrong way around.\r
+.TP\r
+\fB\-m\fR, \fB\-\-ids_guid_file\fR <file name>\r
+Name of the map file with set of the IDs which will be used\r
+by Up/Down routing algorithm instead of node GUIDs\r
+(format: <guid> <id> per line).\r
+.TP\r
+\fB\-X\fR, \fB\-\-guid_routing_order_file\fR <file name>\r
+Set the order port guids will be routed for the MinHop\r
+and Up/Down routing algorithms to the guids provided in the\r
+given file (one to a line).\r
+.TP\r
+\fB\-o\fR, \fB\-\-once\fR\r
+This option causes OpenSM to configure the subnet\r
+once, then exit. Ports remain in the ACTIVE state.\r
+.TP\r
+\fB\-s\fR, \fB\-\-sweep\fR <interval value>\r
+This option specifies the number of seconds between\r
+subnet sweeps. Specifying -s 0 disables sweeping.\r
+Without -s, OpenSM defaults to a sweep interval of\r
+10 seconds.\r
+.TP\r
+\fB\-t\fR, \fB\-\-timeout\fR <value>\r
+This option specifies the time in milliseconds\r
+used for transaction timeouts.\r
+Specifying -t 0 disables timeouts.\r
+Without -t, OpenSM defaults to a timeout value of\r
+200 milliseconds.\r
+.TP\r
+\fB\-\-retries\fR <number>\r
+This option specifies the number of retries used\r
+for transactions.\r
+Without --retries, OpenSM defaults to 3 retries\r
+for transactions.\r
+.TP\r
+\fB\-maxsmps\fR <number>\r
+This option specifies the number of VL15 SMP MADs\r
+allowed on the wire at any one time.\r
+Specifying -maxsmps 0 allows unlimited outstanding\r
+SMPs.\r
+Without -maxsmps, OpenSM defaults to a maximum of\r
+4 outstanding SMPs.\r
+.TP\r
+\fB\-console [off | local | socket | loopback]\fR\r
+This option brings up the OpenSM console (default off).\r
+Note that the socket and loopback options will only be available\r
+if OpenSM was built with --enable-console-socket.\r
+.TP\r
+\fB\-console-port\fR <port>\r
+Specify an alternate telnet port for the socket console (default 10000).\r
+Note that this option only appears if OpenSM was built with\r
+--enable-console-socket.\r
+.TP\r
+\fB\-i\fR, \fB\-ignore-guids\fR <equalize-ignore-guids-file>\r
+This option provides the means to define a set of ports\r
+(by node guid and port number) that will be ignored by the link load\r
+equalization algorithm.\r
+.TP\r
+\fB\-w\fR, \fB\-\-hop_weights_file\fR <path to file>\r
+This option provides weighting factors per port representing a hop cost in\r
+computing the lid matrix. The file consists of lines containing a switch port\r
+GUID (specified as a 64 bit hex number, with leading 0x), output port number,\r
+and weighting factor. Any port not listed in the file defaults to a weighting\r
+factor of 1. Lines starting with # are comments. Weights affect only the\r
+output route from the port, so many useful configurations will require weights\r
+to be specified in pairs.\r
+.TP\r
+\fB\-x\fR, \fB\-\-honor_guid2lid\fR\r
+This option forces OpenSM to honor the guid2lid file,\r
+when it comes out of Standby state, if such file exists\r
+under OSM_CACHE_DIR, and is valid.\r
+By default, this is FALSE.\r
+.TP\r
+\fB\-f\fR, \fB\-\-log_file\fR <file name>\r
+This option defines the log to be the given file.\r
+By default, the log goes to /var/log/opensm.log.\r
+For the log to go to standard output use -f stdout.\r
+.TP\r
+\fB\-L\fR, \fB\-\-log_limit\fR <size in MB>\r
+This option defines maximal log file size in MB. When\r
+specified the log file will be truncated upon reaching\r
+this limit.\r
+.TP\r
+\fB\-e\fR, \fB\-\-erase_log_file\fR\r
+This option will cause deletion of the log file\r
+(if it previously exists). By default, the log file\r
+is accumulative.\r
+.TP\r
+\fB\-P\fR, \fB\-\-Pconfig\fR <partition config file>\r
+This option defines the optional partition configuration file.\r
+The default name is \fB\%@OPENSM_CONFIG_DIR@/@PARTITION_CONFIG_FILE@\fP.\r
+.TP\r
+\fB\-\-prefix_routes_file\fR <file name>\r
+Prefix routes control how the SA responds to path record queries for\r
+off-subnet DGIDs. By default, the SA fails such queries. The\r
+.B PREFIX ROUTES\r
+section below describes the format of the configuration file.\r
+The default path is \fB\%@OPENSM_CONFIG_DIR@/prefix\-routes.conf\fP.\r
+.TP\r
+\fB\-Q\fR, \fB\-\-qos\fR\r
+This option enables QoS setup. It is disabled by default.\r
+.TP\r
+\fB\-Y\fR, \fB\-\-qos_policy_file\fR <file name>\r
+This option defines the optional QoS policy file. The default\r
+name is \fB\%@OPENSM_CONFIG_DIR@/@QOS_POLICY_FILE@\fP. See\r
+QoS_management_in_OpenSM.txt in opensm doc for more information on\r
+configuring QoS policy via this file.\r
+.TP\r
+\fB\-N\fR, \fB\-\-no_part_enforce\fR\r
+This option disables partition enforcement on switch external ports.\r
+.TP\r
+\fB\-y\fR, \fB\-\-stay_on_fatal\fR\r
+This option will cause SM not to exit on fatal initialization\r
+issues: if SM discovers duplicated guids or a 12x link with\r
+lane reversal badly configured.\r
+By default, the SM will exit on these errors.\r
+.TP\r
+\fB\-B\fR, \fB\-\-daemon\fR\r
+Run in daemon mode - OpenSM will run in the background.\r
+.TP\r
+\fB\-I\fR, \fB\-\-inactive\fR\r
+Start SM in inactive rather than init SM state. This\r
+option can be used in conjunction with the perfmgr so as to\r
+run a standalone performance manager without SM/SA. However,\r
+this is NOT currently implemented in the performance manager.\r
+.TP\r
+\fB\-perfmgr\fR\r
+Enable the perfmgr. Only takes effect if --enable-perfmgr was specified at\r
+configure time. See performance-manager-HOWTO.txt in opensm doc for\r
+more information on running perfmgr.\r
+.TP\r
+\fB\-perfmgr_sweep_time_s\fR <seconds>\r
+Specify the sweep time for the performance manager in seconds\r
+(default is 180 seconds). Only takes\r
+effect if --enable-perfmgr was specified at configure time.\r
+.TP\r
+.BI --consolidate_ipv6_snm_req\r
+Use shared MLID for IPv6 Solicited Node Multicast groups per MGID scope\r
+and P_Key.\r
+.TP\r
+\fB\-v\fR, \fB\-\-verbose\fR\r
+This option increases the log verbosity level.\r
+The -v option may be specified multiple times\r
+to further increase the verbosity level.\r
+See the -D option for more information about\r
+log verbosity.\r
+.TP\r
+\fB\-V\fR\r
+This option sets the maximum verbosity level and\r
+forces log flushing.\r
+The -V option is equivalent to \'-D 0xFF -d 2\'.\r
+See the -D option for more information about\r
+log verbosity.\r
+.TP\r
+\fB\-D\fR <value>\r
+This option sets the log verbosity level.\r
+A flags field must follow the -D option.\r
+A bit set/clear in the flags enables/disables a\r
+specific log level as follows:\r
+\r
+ BIT LOG LEVEL ENABLED\r
+ ---- -----------------\r
+ 0x01 - ERROR (error messages)\r
+ 0x02 - INFO (basic messages, low volume)\r
+ 0x04 - VERBOSE (interesting stuff, moderate volume)\r
+ 0x08 - DEBUG (diagnostic, high volume)\r
+ 0x10 - FUNCS (function entry/exit, very high volume)\r
+ 0x20 - FRAMES (dumps all SMP and GMP frames)\r
+ 0x40 - ROUTING (dump FDB routing information)\r
+ 0x80 - currently unused.\r
+\r
+Without -D, OpenSM defaults to ERROR + INFO (0x3).\r
+Specifying -D 0 disables all messages.\r
+Specifying -D 0xFF enables all messages (see -V).\r
+High verbosity levels may require increasing\r
+the transaction timeout with the -t option.\r
+.TP\r
+\fB\-d\fR, \fB\-\-debug\fR <value>\r
+This option specifies a debug option.\r
+These options are not normally needed.\r
+The number following -d selects the debug\r
+option to enable as follows:\r
+\r
+ OPT Description\r
+ --- -----------------\r
+ -d0 - Ignore other SM nodes\r
+ -d1 - Force single threaded dispatching\r
+ -d2 - Force log flushing after each log message\r
+ -d3 - Disable multicast support\r
+.TP\r
+\fB\-h\fR, \fB\-\-help\fR\r
+Display this usage info then exit.\r
+.TP\r
+\fB\-?\fR\r
+Display this usage info then exit.\r
+\r
+.SH ENVIRONMENT VARIABLES\r
+.PP\r
+The following environment variables control opensm behavior:\r
+\r
+OSM_TMP_DIR - controls the directory in which the temporary files generated by\r
+opensm are created. These files are: opensm-subnet.lst, opensm.fdbs, and\r
+opensm.mcfdbs. By default, this directory is /var/log.\r
+\r
+OSM_CACHE_DIR - opensm stores certain data to the disk such that subsequent\r
+runs are consistent. The default directory used is /var/cache/opensm.\r
+The following file is included in it:\r
+\r
+ guid2lid - stores the LID range assigned to each GUID\r
+\r
+.SH NOTES\r
+.PP\r
+When opensm receives a HUP signal, it starts a new heavy sweep as if a trap was received or a topology change was found.\r
+.PP\r
+Also, SIGUSR1 can be used to trigger a reopen of /var/log/opensm.log for\r
+logrotate purposes.\r
+\r
+.SH PARTITION CONFIGURATION\r
+.PP\r
+The default name of OpenSM partitions configuration file is\r
+\fB\%@OPENSM_CONFIG_DIR@/@PARTITION_CONFIG_FILE@\fP. The default may be changed\r
+by using the --Pconfig (-P) option with OpenSM.\r
+\r
+The default partition will be created by OpenSM unconditionally even\r
+when partition configuration file does not exist or cannot be accessed.\r
+\r
+The default partition has P_Key value 0x7fff. OpenSM\'s port will always\r
+have full membership in default partition. All other end ports will have\r
+full membership if the partition configuration file is not found or cannot\r
+be accessed, or limited membership if the file exists and can be accessed\r
+but there is no rule for the Default partition.\r
+\r
+Effectively, this amounts to the same as if one of the following rules\r
+below appear in the partition configuration file.\r
+\r
+In the case of no rule for the Default partition:\r
+\r
+Default=0x7fff : ALL=limited, SELF=full ;\r
+\r
+In the case of no partition configuration file or file cannot be accessed:\r
+\r
+Default=0x7fff : ALL=full ;\r
+\r
+\r
+File Format\r
+\r
+Comments:\r
+\r
+Line content followed after \'#\' character is comment and ignored by\r
+parser.\r
+\r
+General file format:\r
+\r
+<Partition Definition>:<PortGUIDs list> ;\r
+\r
+Partition Definition:\r
+\r
+[PartitionName][=PKey][,flag[=value]][,defmember=full|limited]\r
+\r
+ PartitionName - string, will be used with logging. When omitted\r
+ empty string will be used.\r
+ PKey - P_Key value for this partition. Only low 15 bits will\r
+ be used. When omitted will be autogenerated.\r
+ flag - used to indicate IPoIB capability of this partition.\r
+ defmember=full|limited - specifies default membership for port guid\r
+ list. Default is limited.\r
+\r
+Currently recognized flags are:\r
+\r
+ ipoib - indicates that this partition may be used for IPoIB, as\r
+ result IPoIB capable MC group will be created.\r
+ rate=<val> - specifies rate for this IPoIB MC group\r
+ (default is 3 (10GBps))\r
+ mtu=<val> - specifies MTU for this IPoIB MC group\r
+ (default is 4 (2048))\r
+ sl=<val> - specifies SL for this IPoIB MC group\r
+ (default is 0)\r
+ scope=<val> - specifies scope for this IPoIB MC group\r
+ (default is 2 (link local)). Multiple scope settings\r
+ are permitted for a partition.\r
+\r
+Note that values for rate, mtu, and scope should be specified as\r
+defined in the IBTA specification (for example, mtu=4 for 2048).\r
+\r
+PortGUIDs list:\r
+\r
+ PortGUID - GUID of partition member EndPort. Hexadecimal\r
+ numbers should start from 0x, decimal numbers\r
+ are accepted too.\r
+ full or limited - indicates full or limited membership for this\r
+ port. When omitted (or unrecognized) limited\r
+ membership is assumed.\r
+\r
+There are two useful keywords for PortGUID definition:\r
+\r
+ - 'ALL' means all end ports in this subnet.\r
+ - 'ALL_CAS' means all Channel Adapter end ports in this subnet.\r
+ - 'ALL_SWITCHES' means all Switch end ports in this subnet.\r
+ - 'ALL_ROUTERS' means all Router end ports in this subnet.\r
+ - 'SELF' means subnet manager's port.\r
+\r
+Empty list means no ports in this partition.\r
+\r
+Notes:\r
+\r
+White space is permitted between delimiters ('=', ',',':',';').\r
+\r
+The line can be wrapped after ':' followed after Partition Definition and\r
+between.\r
+\r
+PartitionName does not need to be unique, PKey does need to be unique.\r
+If PKey is repeated then those partition configurations will be merged\r
+and first PartitionName will be used (see also next note).\r
+\r
+It is possible to split partition configuration in more than one\r
+definition, but then PKey should be explicitly specified (otherwise\r
+different PKey values will be generated for those definitions).\r
+\r
+Examples:\r
+\r
+ Default=0x7fff : ALL, SELF=full ;\r
+ Default=0x7fff : ALL, ALL_SWITCHES=full, SELF=full ;\r
+\r
+ NewPartition , ipoib : 0x123456=full, 0x3456789034=limi, 0x2134af2306 ;\r
+\r
+ YetAnotherOne = 0x300 : SELF=full ;\r
+ YetAnotherOne = 0x300 : ALL=limited ;\r
+\r
+ ShareIO = 0x80 , defmember=full : 0x123451, 0x123452;\r
+ # 0x123453, 0x123454 will be limited\r
+ ShareIO = 0x80 : 0x123453, 0x123454, 0x123455=full;\r
+ # 0x123456, 0x123457 will be limited\r
+ ShareIO = 0x80 : defmember=limited : 0x123456, 0x123457, 0x123458=full;\r
+ ShareIO = 0x80 , defmember=full : 0x123459, 0x12345a;\r
+ ShareIO = 0x80 , defmember=full : 0x12345b, 0x12345c=limited, 0x12345d;\r
+\r
+\r
+Note:\r
+\r
+The following rule is equivalent to how OpenSM used to run prior to the\r
+partition manager:\r
+\r
+ Default=0x7fff,ipoib:ALL=full;\r
+\r
+.SH QOS CONFIGURATION\r
+.PP\r
+There are a set of QoS related low-level configuration parameters.\r
+All these parameter names are prefixed by "qos_" string. Here is a full\r
+list of these parameters:\r
+\r
+ qos_max_vls - The maximum number of VLs that will be on the subnet\r
+ qos_high_limit - The limit of High Priority component of VL\r
+ Arbitration table (IBA 7.6.9)\r
+ qos_vlarb_low - Low priority VL Arbitration table (IBA 7.6.9)\r
+ template\r
+ qos_vlarb_high - High priority VL Arbitration table (IBA 7.6.9)\r
+ template\r
+ Both VL arbitration templates are pairs of\r
+ VL and weight\r
+ qos_sl2vl - SL2VL Mapping table (IBA 7.6.6) template. It is\r
+ a list of VLs corresponding to SLs 0-15 (Note\r
+ that VL15 used here means drop this SL)\r
+\r
+Typical default values (hard-coded in OpenSM initialization) are:\r
+\r
+ qos_max_vls 15\r
+ qos_high_limit 0\r
+ qos_vlarb_low 0:0,1:4,2:4,3:4,4:4,5:4,6:4,7:4,8:4,9:4,10:4,11:4,12:4,13:4,14:4\r
+ qos_vlarb_high 0:4,1:0,2:0,3:0,4:0,5:0,6:0,7:0,8:0,9:0,10:0,11:0,12:0,13:0,14:0\r
+ qos_sl2vl 0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,7\r
+\r
+The syntax is compatible with rest of OpenSM configuration options and\r
+values may be stored in OpenSM config file (cached options file).\r
+\r
+In addition to the above, we may define separate QoS configuration\r
+parameters sets for various target types. As targets, we currently support\r
+CAs, routers, switch external ports, and switch's enhanced port 0. The\r
+names of such specialized parameters are prefixed by "qos_<type>_"\r
+string. Here is a full list of the currently supported sets:\r
+\r
+ qos_ca_ - QoS configuration parameters set for CAs.\r
+ qos_rtr_ - parameters set for routers.\r
+ qos_sw0_ - parameters set for switches' port 0.\r
+ qos_swe_ - parameters set for switches' external ports.\r
+\r
+Examples:\r
+ qos_sw0_max_vls=2\r
+ qos_ca_sl2vl=0,1,2,3,5,5,5,12,12,0,\r
+ qos_swe_high_limit=0\r
+\r
+.SH PREFIX ROUTES\r
+.PP\r
+Prefix routes control how the SA responds to path record queries for\r
+off-subnet DGIDs. By default, the SA fails such queries.\r
+Note that IBA does not specify how the SA should obtain off-subnet path\r
+record information.\r
+The prefix routes configuration is meant as a stop-gap until the\r
+specification is completed.\r
+.PP\r
+Each line in the configuration file is a 64-bit prefix followed by a\r
+64-bit GUID, separated by white space.\r
+The GUID specifies the router port on the local subnet that will\r
+handle the prefix.\r
+Blank lines are ignored, as is anything between a \fB#\fP character\r
+and the end of the line.\r
+The prefix and GUID are both in hex, the leading 0x is optional.\r
+Either, or both, can be wild-carded by specifying an\r
+asterisk instead of an explicit prefix or GUID.\r
+.PP\r
+When responding to a path record query for an off-subnet DGID,\r
+opensm searches for the first prefix match in the configuration file.\r
+Therefore, the order of the lines in the configuration file is important:\r
+a wild-carded prefix at the beginning of the configuration file renders\r
+all subsequent lines useless.\r
+If there is no match, then opensm fails the query.\r
+It is legal to repeat prefixes in the configuration file,\r
+opensm will return the path to the first available matching router.\r
+A configuration file with a single line where both prefix and GUID\r
+are wild-carded means that a path record query specifying any\r
+off-subnet DGID should return a path to the first available router.\r
+This configuration yields the same behavior formerly achieved by\r
+compiling opensm with -DROUTER_EXP which has been obsoleted.\r
+\r
+.SH ROUTING\r
+.PP\r
+OpenSM now offers five routing engines:\r
+\r
+1. Min Hop Algorithm - based on the minimum hops to each node where the\r
+path length is optimized.\r
+\r
+2. UPDN Unicast routing algorithm - also based on the minimum hops to each\r
+node, but it is constrained to ranking rules. This algorithm should be chosen\r
+if the subnet is not a pure Fat Tree, and deadlock may occur due to a\r
+loop in the subnet.\r
+\r
+3. Fat Tree Unicast routing algorithm - this algorithm optimizes routing\r
+for congestion-free "shift" communication pattern.\r
+It should be chosen if a subnet is a symmetrical or almost symmetrical\r
+fat-tree of various types, not just K-ary-N-Trees: non-constant K, not\r
+fully staffed, any Constant Bisectional Bandwidth (CBB) ratio.\r
+Similar to UPDN, Fat Tree routing is constrained to ranking rules.\r
+\r
+4. LASH unicast routing algorithm - uses Infiniband virtual layers\r
+(SL) to provide deadlock-free shortest-path routing while also\r
+distributing the paths between layers. LASH is an alternative\r
+deadlock-free topology-agnostic routing algorithm to the non-minimal\r
+UPDN algorithm avoiding the use of a potentially congested root node.\r
+\r
+5. DOR Unicast routing algorithm - based on the Min Hop algorithm, but\r
+avoids port equalization except for redundant links between the same\r
+two switches. This provides deadlock free routes for hypercubes when\r
+the fabric is cabled as a hypercube and for meshes when cabled as a\r
+mesh (see details below).\r
+\r
+OpenSM also supports a file method which\r
+can load routes from a table. See \'Modular Routing Engine\' for more\r
+information on this.\r
+\r
+The basic routing algorithm is comprised of two stages:\r
+\r
+1. MinHop matrix calculation\r
+ How many hops are required to get from each port to each LID ?\r
+ The algorithm to fill these tables is different if you run standard\r
+(min hop) or Up/Down.\r
+ For standard routing, a "relaxation" algorithm is used to propagate\r
+min hop from every destination LID through neighbor switches\r
+ For Up/Down routing, a BFS from every target is used. The BFS tracks link\r
+direction (up or down) and avoid steps that will perform up after a down\r
+step was used.\r
+\r
+2. Once MinHop matrices exist, each switch is visited and for each target LID a\r
+decision is made as to what port should be used to get to that LID.\r
+ This step is common to standard and Up/Down routing. Each port has a\r
+counter counting the number of target LIDs going through it.\r
+ When there are multiple alternative ports with same MinHop to a LID,\r
+the one with less previously assigned ports is selected.\r
+ If LMC > 0, more checks are added: Within each group of LIDs assigned to\r
+same target port,\r
+ a. use only ports which have same MinHop\r
+ b. first prefer the ones that go to different systemImageGuid (then\r
+the previous LID of the same LMC group)\r
+ c. if none - prefer those which go through another NodeGuid\r
+ d. fall back to the number of paths method (if all go to same node).\r
+\r
+Effect of Topology Changes\r
+\r
+OpenSM will preserve existing routing in any case where there is no change in\r
+the fabric switches unless the -r (--reassign_lids) option is specified.\r
+\r
+-r\r
+.br\r
+--reassign_lids\r
+ This option causes OpenSM to reassign LIDs to all\r
+ end nodes. Specifying -r on a running subnet\r
+ may disrupt subnet traffic.\r
+ Without -r, OpenSM attempts to preserve existing\r
+ LID assignments resolving multiple use of same LID.\r
+\r
+If a link is added or removed, OpenSM does not recalculate\r
+the routes that do not have to change. A route has to change\r
+if the port is no longer UP or no longer the MinHop. When routing changes\r
+are performed, the same algorithm for balancing the routes is invoked.\r
+\r
+In the case of using the file based routing, any topology changes are\r
+currently ignored The 'file' routing engine just loads the LFTs from the file\r
+specified, with no reaction to real topology. Obviously, this will not be able\r
+to recheck LIDs (by GUID) for disconnected nodes, and LFTs for non-existent\r
+switches will be skipped. Multicast is not affected by 'file' routing engine\r
+(this uses min hop tables).\r
+\r
+\r
+Min Hop Algorithm\r
+\r
+The Min Hop algorithm is invoked by default if no routing algorithm is\r
+specified. It can also be invoked by specifying '-R minhop'.\r
+\r
+The Min Hop algorithm is divided into two stages: computation of\r
+min-hop tables on every switch and LFT output port assignment. Link\r
+subscription is also equalized with the ability to override based on\r
+port GUID. The latter is supplied by:\r
+\r
+-i <equalize-ignore-guids-file>\r
+.br\r
+-ignore-guids <equalize-ignore-guids-file>\r
+ This option provides the means to define a set of ports\r
+ (by guid) that will be ignored by the link load\r
+ equalization algorithm. Note that only endports (CA,\r
+ switch port 0, and router ports) and not switch external\r
+ ports are supported.\r
+\r
+LMC awareness routes based on (remote) system or switch basis.\r
+\r
+\r
+Purpose of UPDN Algorithm\r
+\r
+The UPDN algorithm is designed to prevent deadlocks from occurring in loops\r
+of the subnet. A loop-deadlock is a situation in which it is no longer\r
+possible to send data between any two hosts connected through the loop. As\r
+such, the UPDN routing algorithm should be used if the subnet is not a pure\r
+Fat Tree, and one of its loops may experience a deadlock (due, for example,\r
+to high pressure).\r
+\r
+The UPDN algorithm is based on the following main stages:\r
+\r
+1. Auto-detect root nodes - based on the CA hop length from any switch in\r
+the subnet, a statistical histogram is built for each switch (hop num vs\r
+number of occurrences). If the histogram reflects a specific column (higher\r
+than others) for a certain node, then it is marked as a root node. Since\r
+the algorithm is statistical, it may not find any root nodes. The list of\r
+the root nodes found by this auto-detect stage is used by the ranking\r
+process stage.\r
+\r
+ Note 1: The user can override the node list manually.\r
+ Note 2: If this stage cannot find any root nodes, and the user did\r
+ not specify a guid list file, OpenSM defaults back to the\r
+ Min Hop routing algorithm.\r
+\r
+2. Ranking process - All root switch nodes (found in stage 1) are assigned\r
+a rank of 0. Using the BFS algorithm, the rest of the switch nodes in the\r
+subnet are ranked incrementally. This ranking aids in the process of enforcing\r
+rules that ensure loop-free paths.\r
+\r
+3. Min Hop Table setting - after ranking is done, a BFS algorithm is run from\r
+each (CA or switch) node in the subnet. During the BFS process, the FDB table\r
+of each switch node traversed by BFS is updated, in reference to the starting\r
+node, based on the ranking rules and guid values.\r
+\r
+At the end of the process, the updated FDB tables ensure loop-free paths\r
+through the subnet.\r
+\r
+Note: Up/Down routing does not allow LID routing communication between\r
+switches that are located inside spine "switch systems".\r
+The reason is that there is no way to allow a LID route between them\r
+that does not break the Up/Down rule.\r
+One ramification of this is that you cannot run SM on switches other\r
+than the leaf switches of the fabric.\r
+\r
+\r
+UPDN Algorithm Usage\r
+\r
+Activation through OpenSM\r
+\r
+Use '-R updn' option (instead of old '-u') to activate the UPDN algorithm.\r
+Use '-a <root_guid_file>' for adding an UPDN guid file that contains the\r
+root nodes for ranking.\r
+If the `-a' option is not used, OpenSM uses its auto-detect root nodes\r
+algorithm.\r
+\r
+Notes on the guid list file:\r
+\r
+1. A valid guid file specifies one guid in each line. Lines with an invalid\r
+format will be discarded.\r
+.br\r
+2. The user should specify the root switch guids. However, it is also\r
+possible to specify CA guids; OpenSM will use the guid of the switch (if\r
+it exists) that connects the CA to the subnet as a root node.\r
+\r
+\r
+Fat-tree Routing Algorithm\r
+\r
+The fat-tree algorithm optimizes routing for "shift" communication pattern.\r
+It should be chosen if a subnet is a symmetrical or almost symmetrical\r
+fat-tree of various types.\r
+It supports not just K-ary-N-Trees, by handling for non-constant K,\r
+cases where not all leafs (CAs) are present, any CBB ratio.\r
+As in UPDN, fat-tree also prevents credit-loop-deadlocks.\r
+\r
+If the root guid file is not provided ('-a' or '--root_guid_file' options),\r
+the topology has to be pure fat-tree that complies with the following rules:\r
+ - Tree rank should be between two and eight (inclusively)\r
+ - Switches of the same rank should have the same number\r
+ of UP-going port groups*, unless they are root switches,\r
+ in which case the shouldn't have UP-going ports at all.\r
+ - Switches of the same rank should have the same number\r
+ of DOWN-going port groups, unless they are leaf switches.\r
+ - Switches of the same rank should have the same number\r
+ of ports in each UP-going port group.\r
+ - Switches of the same rank should have the same number\r
+ of ports in each DOWN-going port group.\r
+ - All the CAs have to be at the same tree level (rank).\r
+\r
+If the root guid file is provided, the topology doesn't have to be pure\r
+fat-tree, and it should only comply with the following rules:\r
+ - Tree rank should be between two and eight (inclusively)\r
+ - All the Compute Nodes** have to be at the same tree level (rank).\r
+ Note that non-compute node CAs are allowed here to be at different\r
+ tree ranks.\r
+\r
+* ports that are connected to the same remote switch are referenced as\r
+\'port group\'.\r
+\r
+** list of compute nodes (CNs) can be specified by \'-u\' or \'--cn_guid_file\'\r
+OpenSM options.\r
+\r
+Topologies that do not comply cause a fallback to min hop routing.\r
+Note that this can also occur on link failures which cause the topology\r
+to no longer be "pure" fat-tree.\r
+\r
+Note that although fat-tree algorithm supports trees with non-integer CBB\r
+ratio, the routing will not be as balanced as in case of integer CBB ratio.\r
+In addition to this, although the algorithm allows leaf switches to have any\r
+number of CAs, the closer the tree is to be fully populated, the more\r
+effective the "shift" communication pattern will be.\r
+In general, even if the root list is provided, the closer the topology to a\r
+pure and symmetrical fat-tree, the more optimal the routing will be.\r
+\r
+The algorithm also dumps compute node ordering file (opensm-ftree-ca-order.dump)\r
+in the same directory where the OpenSM log resides. This ordering file provides\r
+the CN order that may be used to create efficient communication pattern, that\r
+will match the routing tables.\r
+\r
+Routing between non-CN nodes\r
+\r
+The use of the cn_guid_file option allows non-CN nodes to be located on different levels in the fat tree.\r
+In such case, it is not guaranteed that the Fat Tree algorithm will route between two non-CN nodes.\r
+To solve this problem, a list of non-CN nodes can be specified by \'-G\' or \'--io_guid_file\' option.\r
+Theses nodes will be allowed to use switches the wrong way round a specific number of times (specified by \'-H\' or \'--max_reverse_hops\'.\r
+With the proper max_reverse_hops and io_guid_file values, you can ensure full connectivity in the Fat Tree.\r
+\r
+Please note that using max_reverse_hops creates routes that use the switch in a counter-stream way.\r
+This option should never be used to connect nodes with high bandwidth traffic between them ! It should only be used\r
+to allow connectivity for HA purposes or similar.\r
+Also having routes the other way around can in theory cause credit loops.\r
+\r
+Use these options with extreme care !\r
+\r
+Activation through OpenSM\r
+\r
+Use '-R ftree' option to activate the fat-tree algorithm.\r
+Use '-a <root_guid_file>' to provide root nodes for ranking. If the `-a' option\r
+is not used, routing algorithm will detect roots automatically.\r
+Use '-u <root_cn_file>' to provide the list of compute nodes. If the `-u' option\r
+is not used, all the CAs are considered as compute nodes.\r
+\r
+Note: LMC > 0 is not supported by fat-tree routing. If this is\r
+specified, the default routing algorithm is invoked instead.\r
+\r
+\r
+LASH Routing Algorithm\r
+\r
+LASH is an acronym for LAyered SHortest Path Routing. It is a\r
+deterministic shortest path routing algorithm that enables topology\r
+agnostic deadlock-free routing within communication networks.\r
+\r
+When computing the routing function, LASH analyzes the network\r
+topology for the shortest-path routes between all pairs of sources /\r
+destinations and groups these paths into virtual layers in such a way\r
+as to avoid deadlock.\r
+\r
+Note LASH analyzes routes and ensures deadlock freedom between switch\r
+pairs. The link from HCA between and switch does not need virtual\r
+layers as deadlock will not arise between switch and HCA.\r
+\r
+In more detail, the algorithm works as follows:\r
+\r
+1) LASH determines the shortest-path between all pairs of source /\r
+destination switches. Note, LASH ensures the same SL is used for all\r
+SRC/DST - DST/SRC pairs and there is no guarantee that the return\r
+path for a given DST/SRC will be the reverse of the route SRC/DST.\r
+\r
+2) LASH then begins an SL assignment process where a route is assigned\r
+to a layer (SL) if the addition of that route does not cause deadlock\r
+within that layer. This is achieved by maintaining and analysing a\r
+channel dependency graph for each layer. Once the potential addition\r
+of a path could lead to deadlock, LASH opens a new layer and continues\r
+the process.\r
+\r
+3) Once this stage has been completed, it is highly likely that the\r
+first layers processed will contain more paths than the latter ones.\r
+To better balance the use of layers, LASH moves paths from one layer\r
+to another so that the number of paths in each layer averages out.\r
+\r
+Note, the implementation of LASH in opensm attempts to use as few layers\r
+as possible. This number can be less than the number of actual layers\r
+available.\r
+\r
+In general LASH is a very flexible algorithm. It can, for example,\r
+reduce to Dimension Order Routing in certain topologies, it is topology\r
+agnostic and fares well in the face of faults.\r
+\r
+It has been shown that for both regular and irregular topologies, LASH\r
+outperforms Up/Down. The reason for this is that LASH distributes the\r
+traffic more evenly through a network, avoiding the bottleneck issues\r
+related to a root node and always routes shortest-path.\r
+\r
+The algorithm was developed by Simula Research Laboratory.\r
+\r
+\r
+Use '-R lash -Q ' option to activate the LASH algorithm.\r
+\r
+Note: QoS support has to be turned on in order that SL/VL mappings are\r
+used.\r
+\r
+Note: LMC > 0 is not supported by the LASH routing. If this is\r
+specified, the default routing algorithm is invoked instead.\r
+\r
+For open regular cartesian meshes the DOR algorithm is the ideal\r
+routing algorithm. For toroidal meshes on the other hand there\r
+are routing loops that can cause deadlocks. LASH can be used to\r
+route these cases. The performance of LASH can be improved by\r
+preconditioning the mesh in cases where there are multiple links\r
+connecting switches and also in cases where the switches are not\r
+cabled consistently. An option exists for LASH to do this. To\r
+invoke this use '-R lash -Q --do_mesh_analysis'. This will\r
+add an additional phase that analyses the mesh to try to determine\r
+the dimension and size of a mesh. If it determines that the mesh\r
+looks like an open or closed cartesian mesh it reorders the ports\r
+in dimension order before the rest of the LASH algorithm runs.\r
+\r
+DOR Routing Algorithm\r
+\r
+The Dimension Order Routing algorithm is based on the Min Hop\r
+algorithm and so uses shortest paths. Instead of spreading traffic\r
+out across different paths with the same shortest distance, it chooses\r
+among the available shortest paths based on an ordering of dimensions.\r
+Each port must be consistently cabled to represent a hypercube\r
+dimension or a mesh dimension. Paths are grown from a destination\r
+back to a source using the lowest dimension (port) of available paths\r
+at each step. This provides the ordering necessary to avoid deadlock.\r
+When there are multiple links between any two switches, they still\r
+represent only one dimension and traffic is balanced across them\r
+unless port equalization is turned off. In the case of hypercubes,\r
+the same port must be used throughout the fabric to represent the\r
+hypercube dimension and match on both ends of the cable. In the case\r
+of meshes, the dimension should consistently use the same pair of\r
+ports, one port on one end of the cable, and the other port on the\r
+other end, continuing along the mesh dimension.\r
+\r
+Use '-R dor' option to activate the DOR algorithm.\r
+\r
+\r
+Routing References\r
+\r
+To learn more about deadlock-free routing, see the article\r
+"Deadlock Free Message Routing in Multiprocessor Interconnection Networks"\r
+by William J Dally and Charles L Seitz (1985).\r
+\r
+To learn more about the up/down algorithm, see the article\r
+"Effective Strategy to Compute Forwarding Tables for InfiniBand Networks"\r
+by Jose Carlos Sancho, Antonio Robles, and Jose Duato at the\r
+Universidad Politecnica de Valencia.\r
+\r
+To learn more about LASH and the flexibility behind it, the requirement\r
+for layers, performance comparisons to other algorithms, see the\r
+following articles:\r
+\r
+"Layered Routing in Irregular Networks", Lysne et al, IEEE\r
+Transactions on Parallel and Distributed Systems, VOL.16, No12,\r
+December 2005.\r
+\r
+"Routing for the ASI Fabric Manager", Solheim et al. IEEE\r
+Communications Magazine, Vol.44, No.7, July 2006.\r
+\r
+"Layered Shortest Path (LASH) Routing in Irregular System Area\r
+Networks", Skeie et al. IEEE Computer Society Communication\r
+Architecture for Clusters 2002.\r
+\r
+\r
+Modular Routine Engine\r
+\r
+Modular routing engine structure allows for the ease of\r
+"plugging" new routing modules.\r
+\r
+Currently, only unicast callbacks are supported. Multicast\r
+can be added later.\r
+\r
+One existing routing module is up-down "updn", which may be\r
+activated with '-R updn' option (instead of old '-u').\r
+\r
+General usage is:\r
+$ opensm -R 'module-name'\r
+\r
+There is also a trivial routing module which is able\r
+to load LFT tables from a file.\r
+\r
+Main features:\r
+\r
+ - this will load switch LFTs and/or LID matrices (min hops tables)\r
+ - this will load switch LFTs according to the path entries introduced\r
+ in the file\r
+ - no additional checks will be performed (such as "is port connected",\r
+ etc.)\r
+ - in case when fabric LIDs were changed this will try to reconstruct\r
+ LFTs correctly if endport GUIDs are represented in the file\r
+ (in order to disable this, GUIDs may be removed from the file\r
+ or zeroed)\r
+\r
+The file format is compatible with output of 'ibroute' util and for\r
+whole fabric can be generated with dump_lfts.sh script.\r
+\r
+To activate file based routing module, use:\r
+\r
+ opensm -R file -U /path/to/lfts_file\r
+\r
+If the lfts_file is not found or is in error, the default routing\r
+algorithm is utilized.\r
+\r
+The ability to dump switch lid matrices (aka min hops tables) to file and\r
+later to load these is also supported.\r
+\r
+The usage is similar to unicast forwarding tables loading from a lfts\r
+file (introduced by 'file' routing engine), but new lid matrix file\r
+name should be specified by -M or --lid_matrix_file option. For example:\r
+\r
+ opensm -R file -M ./opensm-lid-matrix.dump\r
+\r
+The dump file is named \'opensm-lid-matrix.dump\' and will be generated\r
+in standard opensm dump directory (/var/log by default) when\r
+OSM_LOG_ROUTING logging flag is set.\r
+\r
+When routing engine 'file' is activated, but the lfts file is not specified\r
+or not cannot be open default lid matrix algorithm will be used.\r
+\r
+There is also a switch forwarding tables dumper which generates\r
+a file compatible with dump_lfts.sh output. This file can be used\r
+as input for forwarding tables loading by 'file' routing engine.\r
+Both or one of options -U and -M can be specified together with \'-R file\'.\r
+\r
+.SH FILES\r
+.TP\r
+.B @OPENSM_CONFIG_DIR@/@OPENSM_CONFIG_FILE@\r
+default OpenSM config file.\r
+\r
+.TP\r
+.B @OPENSM_CONFIG_DIR@/@NODENAMEMAPFILE@\r
+default node name map file. See ibnetdiscover for more information on format.\r
+\r
+.TP\r
+.B @OPENSM_CONFIG_DIR@/@PARTITION_CONFIG_FILE@\r
+default partition config file\r
+\r
+.TP\r
+.B @OPENSM_CONFIG_DIR@/@QOS_POLICY_FILE@\r
+default QOS policy config file\r
+\r
+.TP\r
+.B @OPENSM_CONFIG_DIR@/@PREFIX_ROUTES_FILE@\r
+default prefix routes file.\r
+\r
+.SH AUTHORS\r
+.TP\r
+Hal Rosenstock\r
+.RI < hal.rosenstock@gmail.com >\r
+.TP\r
+Sasha Khapyorsky\r
+.RI < sashak@voltaire.com >\r
+.TP\r
+Eitan Zahavi\r
+.RI < eitan@mellanox.co.il >\r
+.TP\r
+Yevgeny Kliteynik\r
+.RI < kliteyn@mellanox.co.il >\r
+.TP\r
+Thomas Sodring\r
+.RI < tsodring@simula.no >\r
+.TP\r
+Ira Weiny\r
+.RI < weiny2@llnl.gov >\r
+++ /dev/null
-.TH OPENSM 8 "October 22, 2009" "OpenIB" "OpenIB Management"\r
-\r
-.SH NAME\r
-opensm \- InfiniBand subnet manager and administration (SM/SA)\r
-\r
-.SH SYNOPSIS\r
-.B opensm\r
-[\-\-version]]\r
-[\-F | \-\-config <file_name>]\r
-[\-c(reate-config) <file_name>]\r
-[\-g(uid) <GUID in hex>]\r
-[\-l(mc) <LMC>]\r
-[\-p(riority) <PRIORITY>]\r
-[\-smkey <SM_Key>]\r
-[\-\-sm_sl <SL number>]\r
-[\-r(eassign_lids)]\r
-[\-R <engine name(s)> | \-\-routing_engine <engine name(s)>]\r
-[\-\-do_mesh_analysis]\r
-[\-\-lash_start_vl <vl number>]\r
-[\-A | \-\-ucast_cache]\r
-[\-z | \-\-connect_roots]\r
-[\-M <file name> | \-\-lid_matrix_file <file name>]\r
-[\-U <file name> | \-\-lfts_file <file name>]\r
-[\-S | \-\-sadb_file <file name>]\r
-[\-a | \-\-root_guid_file <path to file>]\r
-[\-u | \-\-cn_guid_file <path to file>]\r
-[\-G | \-\-io_guid_file <path to file>]\r
-[\-H | \-\-max_reverse_hops <max reverse hops allowed>]\r
-[\-X | \-\-guid_routing_order_file <path to file>]\r
-[\-m | \-\-ids_guid_file <path to file>]\r
-[\-o(nce)]\r
-[\-s(weep) <interval>]\r
-[\-t(imeout) <milliseconds>]\r
-[\-\-retries <number>]\r
-[\-maxsmps <number>]\r
-[\-console [off | local | socket | loopback]]\r
-[\-console-port <port>]\r
-[\-i(gnore-guids) <equalize-ignore-guids-file>]\r
-[\-w | \-\-hop_weights_file <path to file>]\r
-[\-f <log file path> | \-\-log_file <log file path> ]\r
-[\-L | \-\-log_limit <size in MB>] [\-e(rase_log_file)]\r
-[\-P(config) <partition config file> ]\r
-[\-N | \-\-no_part_enforce]\r
-[\-Q | \-\-qos [\-Y | \-\-qos_policy_file <file name>]]\r
-[\-y | \-\-stay_on_fatal]\r
-[\-B | \-\-daemon]\r
-[\-I | \-\-inactive]\r
-[\-\-perfmgr]\r
-[\-\-perfmgr_sweep_time_s <seconds>]\r
-[\-\-prefix_routes_file <path>]\r
-[\-\-consolidate_ipv6_snm_req]\r
-[\-v(erbose)] [\-V] [\-D <flags>] [\-d(ebug) <number>]\r
-[\-h(elp)] [\-?]\r
-\r
-.SH DESCRIPTION\r
-.PP\r
-opensm is an InfiniBand compliant Subnet Manager and Administration,\r
-and runs on top of OpenIB.\r
-\r
-opensm provides an implementation of an InfiniBand Subnet Manager and\r
-Administration. Such a software entity is required to run for in order\r
-to initialize the InfiniBand hardware (at least one per each\r
-InfiniBand subnet).\r
-\r
-opensm also now contains an experimental version of a performance\r
-manager as well.\r
-\r
-opensm defaults were designed to meet the common case usage on clusters with up to a few hundred nodes. Thus, in this default mode, opensm will scan the IB\r
-fabric, initialize it, and sweep occasionally for changes.\r
-\r
-opensm attaches to a specific IB port on the local machine and configures only\r
-the fabric connected to it. (If the local machine has other IB ports,\r
-opensm will ignore the fabrics connected to those other ports). If no port is\r
-specified, it will select the first "best" available port.\r
-\r
-opensm can present the available ports and prompt for a port number to\r
-attach to.\r
-\r
-By default, the run is logged to two files: /var/log/messages and /var/log/opensm.log.\r
-The first file will register only general major events, whereas the second\r
-will include details of reported errors. All errors reported in this second\r
-file should be treated as indicators of IB fabric health issues.\r
-(Note that when a fatal and non-recoverable error occurs, opensm will exit.)\r
-Both log files should include the message "SUBNET UP" if opensm was able to\r
-setup the subnet correctly.\r
-\r
-.SH OPTIONS\r
-\r
-.PP\r
-.TP\r
-\fB\-\-version\fR\r
-Prints OpenSM version and exits.\r
-.TP\r
-\fB\-F\fR, \fB\-\-config\fR <config file>\r
-The name of the OpenSM config file. When not specified\r
-\fB\% @OPENSM_CONFIG_DIR@/@OPENSM_CONFIG_FILE@\fP will be used (if exists).\r
-.TP\r
-\fB\-c\fR, \fB\-\-create-config\fR <file name>\r
-OpenSM will dump its configuration to the specified file and exit.\r
-This is a way to generate OpenSM configuration file template.\r
-.TP\r
-\fB\-g\fR, \fB\-\-guid\fR <GUID in hex>\r
-This option specifies the local port GUID value\r
-with which OpenSM should bind. OpenSM may be\r
-bound to 1 port at a time.\r
-If GUID given is 0, OpenSM displays a list\r
-of possible port GUIDs and waits for user input.\r
-Without -g, OpenSM tries to use the default port.\r
-.TP\r
-\fB\-l\fR, \fB\-\-lmc\fR <LMC value>\r
-This option specifies the subnet's LMC value.\r
-The number of LIDs assigned to each port is 2^LMC.\r
-The LMC value must be in the range 0-7.\r
-LMC values > 0 allow multiple paths between ports.\r
-LMC values > 0 should only be used if the subnet\r
-topology actually provides multiple paths between\r
-ports, i.e. multiple interconnects between switches.\r
-Without -l, OpenSM defaults to LMC = 0, which allows\r
-one path between any two ports.\r
-.TP\r
-\fB\-p\fR, \fB\-\-priority\fR <Priority value>\r
-This option specifies the SM\'s PRIORITY.\r
-This will effect the handover cases, where master\r
-is chosen by priority and GUID. Range goes from 0\r
-(default and lowest priority) to 15 (highest).\r
-.TP\r
-\fB\-smkey\fR <SM_Key value>\r
-This option specifies the SM\'s SM_Key (64 bits).\r
-This will effect SM authentication.\r
-Note that OpenSM version 3.2.1 and below used the default value '1'\r
-in a host byte order, it is fixed now but you may need this option to\r
-interoperate with old OpenSM running on a little endian machine.\r
-.TP\r
-\fB\-\-sm_sl\fR <SL number>\r
-This option sets the SL to use for communication with the SM/SA.\r
-Defaults to 0.\r
-.TP\r
-\fB\-r\fR, \fB\-\-reassign_lids\fR\r
-This option causes OpenSM to reassign LIDs to all\r
-end nodes. Specifying -r on a running subnet\r
-may disrupt subnet traffic.\r
-Without -r, OpenSM attempts to preserve existing\r
-LID assignments resolving multiple use of same LID.\r
-.TP\r
-\fB\-R\fR, \fB\-\-routing_engine\fR <Routing engine names>\r
-This option chooses routing engine(s) to use instead of Min Hop\r
-algorithm (default). Multiple routing engines can be specified\r
-separated by commas so that specific ordering of routing algorithms\r
-will be tried if earlier routing engines fail.\r
-Supported engines: minhop, updn, file, ftree, lash, dor\r
-.TP\r
-\fB\-\-do_mesh_analysis\fR\r
-This option enables additional analysis for the lash routing engine to\r
-precondition switch port assignments in regular cartesian meshes which\r
-may reduce the number of SLs required to give a deadlock free routing.\r
-.TP\r
-\fB\-\-lash_start_vl\fR <vl number>\r
-This option sets the starting VL to use for the lash routing algorithm.\r
-Defaults to 0.\r
-.TP\r
-\fB\-A\fR, \fB\-\-ucast_cache\fR\r
-This option enables unicast routing cache and prevents routing\r
-recalculation (which is a heavy task in a large cluster) when\r
-there was no topology change detected during the heavy sweep, or\r
-when the topology change does not require new routing calculation,\r
-e.g. when one or more CAs/RTRs/leaf switches going down, or one or\r
-more of these nodes coming back after being down.\r
-A very common case that is handled by the unicast routing cache\r
-is host reboot, which otherwise would cause two full routing\r
-recalculations: one when the host goes down, and the other when\r
-the host comes back online.\r
-.TP\r
-\fB\-z\fR, \fB\-\-connect_roots\fR\r
-This option enforces a routing engine (currently up/down\r
-only) to make connectivity between root switches and in\r
-this way to be fully IBA complaint. In many cases this can\r
-violate "pure" deadlock free algorithm, so use it carefully.\r
-.TP\r
-\fB\-M\fR, \fB\-\-lid_matrix_file\fR <file name>\r
-This option specifies the name of the lid matrix dump file\r
-from where switch lid matrices (min hops tables will be\r
-loaded.\r
-.TP\r
-\fB\-U\fR, \fB\-\-lfts_file\fR <file name>\r
-This option specifies the name of the LFTs file\r
-from where switch forwarding tables will be loaded.\r
-.TP\r
-\fB\-S\fR, \fB\-\-sadb_file\fR <file name>\r
-This option specifies the name of the SA DB dump file\r
-from where SA database will be loaded.\r
-.TP\r
-\fB\-a\fR, \fB\-\-root_guid_file\fR <file name>\r
-Set the root nodes for the Up/Down or Fat-Tree routing\r
-algorithm to the guids provided in the given file (one to a line).\r
-.TP\r
-\fB\-u\fR, \fB\-\-cn_guid_file\fR <file name>\r
-Set the compute nodes for the Fat-Tree routing algorithm\r
-to the guids provided in the given file (one to a line).\r
-.TP\r
-\fB\-G\fR, \fB\-\-io_guid_file\fR <file name>\r
-Set the I/O nodes for the Fat-Tree routing algorithm\r
-to the guids provided in the given file (one to a line).\r
-I/O nodes are non-CN nodes allowed to use up to max_reverse_hops switches\r
-the wrong way around to improve connectivity.\r
-.TP\r
-\fB\-H\fR, \fB\-\-max_reverse_hops\fR <file name>\r
-Set the maximum number of reverse hops an I/O node is allowed\r
-to make. A reverse hop is the use of a switch the wrong way around.\r
-.TP\r
-\fB\-m\fR, \fB\-\-ids_guid_file\fR <file name>\r
-Name of the map file with set of the IDs which will be used\r
-by Up/Down routing algorithm instead of node GUIDs\r
-(format: <guid> <id> per line).\r
-.TP\r
-\fB\-X\fR, \fB\-\-guid_routing_order_file\fR <file name>\r
-Set the order port guids will be routed for the MinHop\r
-and Up/Down routing algorithms to the guids provided in the\r
-given file (one to a line).\r
-.TP\r
-\fB\-o\fR, \fB\-\-once\fR\r
-This option causes OpenSM to configure the subnet\r
-once, then exit. Ports remain in the ACTIVE state.\r
-.TP\r
-\fB\-s\fR, \fB\-\-sweep\fR <interval value>\r
-This option specifies the number of seconds between\r
-subnet sweeps. Specifying -s 0 disables sweeping.\r
-Without -s, OpenSM defaults to a sweep interval of\r
-10 seconds.\r
-.TP\r
-\fB\-t\fR, \fB\-\-timeout\fR <value>\r
-This option specifies the time in milliseconds\r
-used for transaction timeouts.\r
-Specifying -t 0 disables timeouts.\r
-Without -t, OpenSM defaults to a timeout value of\r
-200 milliseconds.\r
-.TP\r
-\fB\-\-retries\fR <number>\r
-This option specifies the number of retries used\r
-for transactions.\r
-Without --retries, OpenSM defaults to 3 retries\r
-for transactions.\r
-.TP\r
-\fB\-maxsmps\fR <number>\r
-This option specifies the number of VL15 SMP MADs\r
-allowed on the wire at any one time.\r
-Specifying -maxsmps 0 allows unlimited outstanding\r
-SMPs.\r
-Without -maxsmps, OpenSM defaults to a maximum of\r
-4 outstanding SMPs.\r
-.TP\r
-\fB\-console [off | local | socket | loopback]\fR\r
-This option brings up the OpenSM console (default off).\r
-Note that the socket and loopback options will only be available\r
-if OpenSM was built with --enable-console-socket.\r
-.TP\r
-\fB\-console-port\fR <port>\r
-Specify an alternate telnet port for the socket console (default 10000).\r
-Note that this option only appears if OpenSM was built with\r
---enable-console-socket.\r
-.TP\r
-\fB\-i\fR, \fB\-ignore-guids\fR <equalize-ignore-guids-file>\r
-This option provides the means to define a set of ports\r
-(by node guid and port number) that will be ignored by the link load\r
-equalization algorithm.\r
-.TP\r
-\fB\-w\fR, \fB\-\-hop_weights_file\fR <path to file>\r
-This option provides weighting factors per port representing a hop cost in\r
-computing the lid matrix. The file consists of lines containing a switch port\r
-GUID (specified as a 64 bit hex number, with leading 0x), output port number,\r
-and weighting factor. Any port not listed in the file defaults to a weighting\r
-factor of 1. Lines starting with # are comments. Weights affect only the\r
-output route from the port, so many useful configurations will require weights\r
-to be specified in pairs.\r
-.TP\r
-\fB\-x\fR, \fB\-\-honor_guid2lid\fR\r
-This option forces OpenSM to honor the guid2lid file,\r
-when it comes out of Standby state, if such file exists\r
-under OSM_CACHE_DIR, and is valid.\r
-By default, this is FALSE.\r
-.TP\r
-\fB\-f\fR, \fB\-\-log_file\fR <file name>\r
-This option defines the log to be the given file.\r
-By default, the log goes to /var/log/opensm.log.\r
-For the log to go to standard output use -f stdout.\r
-.TP\r
-\fB\-L\fR, \fB\-\-log_limit\fR <size in MB>\r
-This option defines maximal log file size in MB. When\r
-specified the log file will be truncated upon reaching\r
-this limit.\r
-.TP\r
-\fB\-e\fR, \fB\-\-erase_log_file\fR\r
-This option will cause deletion of the log file\r
-(if it previously exists). By default, the log file\r
-is accumulative.\r
-.TP\r
-\fB\-P\fR, \fB\-\-Pconfig\fR <partition config file>\r
-This option defines the optional partition configuration file.\r
-The default name is \fB\%@OPENSM_CONFIG_DIR@/@PARTITION_CONFIG_FILE@\fP.\r
-.TP\r
-\fB\-\-prefix_routes_file\fR <file name>\r
-Prefix routes control how the SA responds to path record queries for\r
-off-subnet DGIDs. By default, the SA fails such queries. The\r
-.B PREFIX ROUTES\r
-section below describes the format of the configuration file.\r
-The default path is \fB\%@OPENSM_CONFIG_DIR@/prefix\-routes.conf\fP.\r
-.TP\r
-\fB\-Q\fR, \fB\-\-qos\fR\r
-This option enables QoS setup. It is disabled by default.\r
-.TP\r
-\fB\-Y\fR, \fB\-\-qos_policy_file\fR <file name>\r
-This option defines the optional QoS policy file. The default\r
-name is \fB\%@OPENSM_CONFIG_DIR@/@QOS_POLICY_FILE@\fP. See\r
-QoS_management_in_OpenSM.txt in opensm doc for more information on\r
-configuring QoS policy via this file.\r
-.TP\r
-\fB\-N\fR, \fB\-\-no_part_enforce\fR\r
-This option disables partition enforcement on switch external ports.\r
-.TP\r
-\fB\-y\fR, \fB\-\-stay_on_fatal\fR\r
-This option will cause SM not to exit on fatal initialization\r
-issues: if SM discovers duplicated guids or a 12x link with\r
-lane reversal badly configured.\r
-By default, the SM will exit on these errors.\r
-.TP\r
-\fB\-B\fR, \fB\-\-daemon\fR\r
-Run in daemon mode - OpenSM will run in the background.\r
-.TP\r
-\fB\-I\fR, \fB\-\-inactive\fR\r
-Start SM in inactive rather than init SM state. This\r
-option can be used in conjunction with the perfmgr so as to\r
-run a standalone performance manager without SM/SA. However,\r
-this is NOT currently implemented in the performance manager.\r
-.TP\r
-\fB\-perfmgr\fR\r
-Enable the perfmgr. Only takes effect if --enable-perfmgr was specified at\r
-configure time. See performance-manager-HOWTO.txt in opensm doc for\r
-more information on running perfmgr.\r
-.TP\r
-\fB\-perfmgr_sweep_time_s\fR <seconds>\r
-Specify the sweep time for the performance manager in seconds\r
-(default is 180 seconds). Only takes\r
-effect if --enable-perfmgr was specified at configure time.\r
-.TP\r
-.BI --consolidate_ipv6_snm_req\r
-Consolidate IPv6 Solicited Node Multicast group join requests into one\r
-multicast group per MGID PKey.\r
-.TP\r
-\fB\-v\fR, \fB\-\-verbose\fR\r
-This option increases the log verbosity level.\r
-The -v option may be specified multiple times\r
-to further increase the verbosity level.\r
-See the -D option for more information about\r
-log verbosity.\r
-.TP\r
-\fB\-V\fR\r
-This option sets the maximum verbosity level and\r
-forces log flushing.\r
-The -V option is equivalent to \'-D 0xFF -d 2\'.\r
-See the -D option for more information about\r
-log verbosity.\r
-.TP\r
-\fB\-D\fR <value>\r
-This option sets the log verbosity level.\r
-A flags field must follow the -D option.\r
-A bit set/clear in the flags enables/disables a\r
-specific log level as follows:\r
-\r
- BIT LOG LEVEL ENABLED\r
- ---- -----------------\r
- 0x01 - ERROR (error messages)\r
- 0x02 - INFO (basic messages, low volume)\r
- 0x04 - VERBOSE (interesting stuff, moderate volume)\r
- 0x08 - DEBUG (diagnostic, high volume)\r
- 0x10 - FUNCS (function entry/exit, very high volume)\r
- 0x20 - FRAMES (dumps all SMP and GMP frames)\r
- 0x40 - ROUTING (dump FDB routing information)\r
- 0x80 - currently unused.\r
-\r
-Without -D, OpenSM defaults to ERROR + INFO (0x3).\r
-Specifying -D 0 disables all messages.\r
-Specifying -D 0xFF enables all messages (see -V).\r
-High verbosity levels may require increasing\r
-the transaction timeout with the -t option.\r
-.TP\r
-\fB\-d\fR, \fB\-\-debug\fR <value>\r
-This option specifies a debug option.\r
-These options are not normally needed.\r
-The number following -d selects the debug\r
-option to enable as follows:\r
-\r
- OPT Description\r
- --- -----------------\r
- -d0 - Ignore other SM nodes\r
- -d1 - Force single threaded dispatching\r
- -d2 - Force log flushing after each log message\r
- -d3 - Disable multicast support\r
-.TP\r
-\fB\-h\fR, \fB\-\-help\fR\r
-Display this usage info then exit.\r
-.TP\r
-\fB\-?\fR\r
-Display this usage info then exit.\r
-\r
-.SH ENVIRONMENT VARIABLES\r
-.PP\r
-The following environment variables control opensm behavior:\r
-\r
-OSM_TMP_DIR - controls the directory in which the temporary files generated by\r
-opensm are created. These files are: opensm-subnet.lst, opensm.fdbs, and\r
-opensm.mcfdbs. By default, this directory is /var/log.\r
-\r
-OSM_CACHE_DIR - opensm stores certain data to the disk such that subsequent\r
-runs are consistent. The default directory used is /var/cache/opensm.\r
-The following file is included in it:\r
-\r
- guid2lid - stores the LID range assigned to each GUID\r
-\r
-.SH NOTES\r
-.PP\r
-When opensm receives a HUP signal, it starts a new heavy sweep as if a trap was received or a topology change was found.\r
-.PP\r
-Also, SIGUSR1 can be used to trigger a reopen of /var/log/opensm.log for\r
-logrotate purposes.\r
-\r
-.SH PARTITION CONFIGURATION\r
-.PP\r
-The default name of OpenSM partitions configuration file is\r
-\fB\%@OPENSM_CONFIG_DIR@/@PARTITION_CONFIG_FILE@\fP. The default may be changed\r
-by using the --Pconfig (-P) option with OpenSM.\r
-\r
-The default partition will be created by OpenSM unconditionally even\r
-when partition configuration file does not exist or cannot be accessed.\r
-\r
-The default partition has P_Key value 0x7fff. OpenSM\'s port will always\r
-have full membership in default partition. All other end ports will have\r
-full membership if the partition configuration file is not found or cannot\r
-be accessed, or limited membership if the file exists and can be accessed\r
-but there is no rule for the Default partition.\r
-\r
-Effectively, this amounts to the same as if one of the following rules\r
-below appear in the partition configuration file.\r
-\r
-In the case of no rule for the Default partition:\r
-\r
-Default=0x7fff : ALL=limited, SELF=full ;\r
-\r
-In the case of no partition configuration file or file cannot be accessed:\r
-\r
-Default=0x7fff : ALL=full ;\r
-\r
-\r
-File Format\r
-\r
-Comments:\r
-\r
-Line content followed after \'#\' character is comment and ignored by\r
-parser.\r
-\r
-General file format:\r
-\r
-<Partition Definition>:<PortGUIDs list> ;\r
-\r
-Partition Definition:\r
-\r
-[PartitionName][=PKey][,flag[=value]][,defmember=full|limited]\r
-\r
- PartitionName - string, will be used with logging. When omitted\r
- empty string will be used.\r
- PKey - P_Key value for this partition. Only low 15 bits will\r
- be used. When omitted will be autogenerated.\r
- flag - used to indicate IPoIB capability of this partition.\r
- defmember=full|limited - specifies default membership for port guid\r
- list. Default is limited.\r
-\r
-Currently recognized flags are:\r
-\r
- ipoib - indicates that this partition may be used for IPoIB, as\r
- result IPoIB capable MC group will be created.\r
- rate=<val> - specifies rate for this IPoIB MC group\r
- (default is 3 (10GBps))\r
- mtu=<val> - specifies MTU for this IPoIB MC group\r
- (default is 4 (2048))\r
- sl=<val> - specifies SL for this IPoIB MC group\r
- (default is 0)\r
- scope=<val> - specifies scope for this IPoIB MC group\r
- (default is 2 (link local)). Multiple scope settings\r
- are permitted for a partition.\r
-\r
-Note that values for rate, mtu, and scope should be specified as\r
-defined in the IBTA specification (for example, mtu=4 for 2048).\r
-\r
-PortGUIDs list:\r
-\r
- PortGUID - GUID of partition member EndPort. Hexadecimal\r
- numbers should start from 0x, decimal numbers\r
- are accepted too.\r
- full or limited - indicates full or limited membership for this\r
- port. When omitted (or unrecognized) limited\r
- membership is assumed.\r
-\r
-There are two useful keywords for PortGUID definition:\r
-\r
- - 'ALL' means all end ports in this subnet.\r
- - 'ALL_CAS' means all Channel Adapter end ports in this subnet.\r
- - 'ALL_SWITCHES' means all Switch end ports in this subnet.\r
- - 'ALL_ROUTERS' means all Router end ports in this subnet.\r
- - 'SELF' means subnet manager's port.\r
-\r
-Empty list means no ports in this partition.\r
-\r
-Notes:\r
-\r
-White space is permitted between delimiters ('=', ',',':',';').\r
-\r
-The line can be wrapped after ':' followed after Partition Definition and\r
-between.\r
-\r
-PartitionName does not need to be unique, PKey does need to be unique.\r
-If PKey is repeated then those partition configurations will be merged\r
-and first PartitionName will be used (see also next note).\r
-\r
-It is possible to split partition configuration in more than one\r
-definition, but then PKey should be explicitly specified (otherwise\r
-different PKey values will be generated for those definitions).\r
-\r
-Examples:\r
-\r
- Default=0x7fff : ALL, SELF=full ;\r
- Default=0x7fff : ALL, ALL_SWITCHES=full, SELF=full ;\r
-\r
- NewPartition , ipoib : 0x123456=full, 0x3456789034=limi, 0x2134af2306 ;\r
-\r
- YetAnotherOne = 0x300 : SELF=full ;\r
- YetAnotherOne = 0x300 : ALL=limited ;\r
-\r
- ShareIO = 0x80 , defmember=full : 0x123451, 0x123452;\r
- # 0x123453, 0x123454 will be limited\r
- ShareIO = 0x80 : 0x123453, 0x123454, 0x123455=full;\r
- # 0x123456, 0x123457 will be limited\r
- ShareIO = 0x80 : defmember=limited : 0x123456, 0x123457, 0x123458=full;\r
- ShareIO = 0x80 , defmember=full : 0x123459, 0x12345a;\r
- ShareIO = 0x80 , defmember=full : 0x12345b, 0x12345c=limited, 0x12345d;\r
-\r
-\r
-Note:\r
-\r
-The following rule is equivalent to how OpenSM used to run prior to the\r
-partition manager:\r
-\r
- Default=0x7fff,ipoib:ALL=full;\r
-\r
-.SH QOS CONFIGURATION\r
-.PP\r
-There are a set of QoS related low-level configuration parameters.\r
-All these parameter names are prefixed by "qos_" string. Here is a full\r
-list of these parameters:\r
-\r
- qos_max_vls - The maximum number of VLs that will be on the subnet\r
- qos_high_limit - The limit of High Priority component of VL\r
- Arbitration table (IBA 7.6.9)\r
- qos_vlarb_low - Low priority VL Arbitration table (IBA 7.6.9)\r
- template\r
- qos_vlarb_high - High priority VL Arbitration table (IBA 7.6.9)\r
- template\r
- Both VL arbitration templates are pairs of\r
- VL and weight\r
- qos_sl2vl - SL2VL Mapping table (IBA 7.6.6) template. It is\r
- a list of VLs corresponding to SLs 0-15 (Note\r
- that VL15 used here means drop this SL)\r
-\r
-Typical default values (hard-coded in OpenSM initialization) are:\r
-\r
- qos_max_vls 15\r
- qos_high_limit 0\r
- qos_vlarb_low 0:0,1:4,2:4,3:4,4:4,5:4,6:4,7:4,8:4,9:4,10:4,11:4,12:4,13:4,14:4\r
- qos_vlarb_high 0:4,1:0,2:0,3:0,4:0,5:0,6:0,7:0,8:0,9:0,10:0,11:0,12:0,13:0,14:0\r
- qos_sl2vl 0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,7\r
-\r
-The syntax is compatible with rest of OpenSM configuration options and\r
-values may be stored in OpenSM config file (cached options file).\r
-\r
-In addition to the above, we may define separate QoS configuration\r
-parameters sets for various target types. As targets, we currently support\r
-CAs, routers, switch external ports, and switch's enhanced port 0. The\r
-names of such specialized parameters are prefixed by "qos_<type>_"\r
-string. Here is a full list of the currently supported sets:\r
-\r
- qos_ca_ - QoS configuration parameters set for CAs.\r
- qos_rtr_ - parameters set for routers.\r
- qos_sw0_ - parameters set for switches' port 0.\r
- qos_swe_ - parameters set for switches' external ports.\r
-\r
-Examples:\r
- qos_sw0_max_vls=2\r
- qos_ca_sl2vl=0,1,2,3,5,5,5,12,12,0,\r
- qos_swe_high_limit=0\r
-\r
-.SH PREFIX ROUTES\r
-.PP\r
-Prefix routes control how the SA responds to path record queries for\r
-off-subnet DGIDs. By default, the SA fails such queries.\r
-Note that IBA does not specify how the SA should obtain off-subnet path\r
-record information.\r
-The prefix routes configuration is meant as a stop-gap until the\r
-specification is completed.\r
-.PP\r
-Each line in the configuration file is a 64-bit prefix followed by a\r
-64-bit GUID, separated by white space.\r
-The GUID specifies the router port on the local subnet that will\r
-handle the prefix.\r
-Blank lines are ignored, as is anything between a \fB#\fP character\r
-and the end of the line.\r
-The prefix and GUID are both in hex, the leading 0x is optional.\r
-Either, or both, can be wild-carded by specifying an\r
-asterisk instead of an explicit prefix or GUID.\r
-.PP\r
-When responding to a path record query for an off-subnet DGID,\r
-opensm searches for the first prefix match in the configuration file.\r
-Therefore, the order of the lines in the configuration file is important:\r
-a wild-carded prefix at the beginning of the configuration file renders\r
-all subsequent lines useless.\r
-If there is no match, then opensm fails the query.\r
-It is legal to repeat prefixes in the configuration file,\r
-opensm will return the path to the first available matching router.\r
-A configuration file with a single line where both prefix and GUID\r
-are wild-carded means that a path record query specifying any\r
-off-subnet DGID should return a path to the first available router.\r
-This configuration yields the same behavior formerly achieved by\r
-compiling opensm with -DROUTER_EXP which has been obsoleted.\r
-\r
-.SH ROUTING\r
-.PP\r
-OpenSM now offers five routing engines:\r
-\r
-1. Min Hop Algorithm - based on the minimum hops to each node where the\r
-path length is optimized.\r
-\r
-2. UPDN Unicast routing algorithm - also based on the minimum hops to each\r
-node, but it is constrained to ranking rules. This algorithm should be chosen\r
-if the subnet is not a pure Fat Tree, and deadlock may occur due to a\r
-loop in the subnet.\r
-\r
-3. Fat Tree Unicast routing algorithm - this algorithm optimizes routing\r
-for congestion-free "shift" communication pattern.\r
-It should be chosen if a subnet is a symmetrical or almost symmetrical\r
-fat-tree of various types, not just K-ary-N-Trees: non-constant K, not\r
-fully staffed, any Constant Bisectional Bandwidth (CBB) ratio.\r
-Similar to UPDN, Fat Tree routing is constrained to ranking rules.\r
-\r
-4. LASH unicast routing algorithm - uses Infiniband virtual layers\r
-(SL) to provide deadlock-free shortest-path routing while also\r
-distributing the paths between layers. LASH is an alternative\r
-deadlock-free topology-agnostic routing algorithm to the non-minimal\r
-UPDN algorithm avoiding the use of a potentially congested root node.\r
-\r
-5. DOR Unicast routing algorithm - based on the Min Hop algorithm, but\r
-avoids port equalization except for redundant links between the same\r
-two switches. This provides deadlock free routes for hypercubes when\r
-the fabric is cabled as a hypercube and for meshes when cabled as a\r
-mesh (see details below).\r
-\r
-OpenSM also supports a file method which\r
-can load routes from a table. See \'Modular Routing Engine\' for more\r
-information on this.\r
-\r
-The basic routing algorithm is comprised of two stages:\r
-\r
-1. MinHop matrix calculation\r
- How many hops are required to get from each port to each LID ?\r
- The algorithm to fill these tables is different if you run standard\r
-(min hop) or Up/Down.\r
- For standard routing, a "relaxation" algorithm is used to propagate\r
-min hop from every destination LID through neighbor switches\r
- For Up/Down routing, a BFS from every target is used. The BFS tracks link\r
-direction (up or down) and avoid steps that will perform up after a down\r
-step was used.\r
-\r
-2. Once MinHop matrices exist, each switch is visited and for each target LID a\r
-decision is made as to what port should be used to get to that LID.\r
- This step is common to standard and Up/Down routing. Each port has a\r
-counter counting the number of target LIDs going through it.\r
- When there are multiple alternative ports with same MinHop to a LID,\r
-the one with less previously assigned ports is selected.\r
- If LMC > 0, more checks are added: Within each group of LIDs assigned to\r
-same target port,\r
- a. use only ports which have same MinHop\r
- b. first prefer the ones that go to different systemImageGuid (then\r
-the previous LID of the same LMC group)\r
- c. if none - prefer those which go through another NodeGuid\r
- d. fall back to the number of paths method (if all go to same node).\r
-\r
-Effect of Topology Changes\r
-\r
-OpenSM will preserve existing routing in any case where there is no change in\r
-the fabric switches unless the -r (--reassign_lids) option is specified.\r
-\r
--r\r
-.br\r
---reassign_lids\r
- This option causes OpenSM to reassign LIDs to all\r
- end nodes. Specifying -r on a running subnet\r
- may disrupt subnet traffic.\r
- Without -r, OpenSM attempts to preserve existing\r
- LID assignments resolving multiple use of same LID.\r
-\r
-If a link is added or removed, OpenSM does not recalculate\r
-the routes that do not have to change. A route has to change\r
-if the port is no longer UP or no longer the MinHop. When routing changes\r
-are performed, the same algorithm for balancing the routes is invoked.\r
-\r
-In the case of using the file based routing, any topology changes are\r
-currently ignored The 'file' routing engine just loads the LFTs from the file\r
-specified, with no reaction to real topology. Obviously, this will not be able\r
-to recheck LIDs (by GUID) for disconnected nodes, and LFTs for non-existent\r
-switches will be skipped. Multicast is not affected by 'file' routing engine\r
-(this uses min hop tables).\r
-\r
-\r
-Min Hop Algorithm\r
-\r
-The Min Hop algorithm is invoked by default if no routing algorithm is\r
-specified. It can also be invoked by specifying '-R minhop'.\r
-\r
-The Min Hop algorithm is divided into two stages: computation of\r
-min-hop tables on every switch and LFT output port assignment. Link\r
-subscription is also equalized with the ability to override based on\r
-port GUID. The latter is supplied by:\r
-\r
--i <equalize-ignore-guids-file>\r
-.br\r
--ignore-guids <equalize-ignore-guids-file>\r
- This option provides the means to define a set of ports\r
- (by guid) that will be ignored by the link load\r
- equalization algorithm. Note that only endports (CA,\r
- switch port 0, and router ports) and not switch external\r
- ports are supported.\r
-\r
-LMC awareness routes based on (remote) system or switch basis.\r
-\r
-\r
-Purpose of UPDN Algorithm\r
-\r
-The UPDN algorithm is designed to prevent deadlocks from occurring in loops\r
-of the subnet. A loop-deadlock is a situation in which it is no longer\r
-possible to send data between any two hosts connected through the loop. As\r
-such, the UPDN routing algorithm should be used if the subnet is not a pure\r
-Fat Tree, and one of its loops may experience a deadlock (due, for example,\r
-to high pressure).\r
-\r
-The UPDN algorithm is based on the following main stages:\r
-\r
-1. Auto-detect root nodes - based on the CA hop length from any switch in\r
-the subnet, a statistical histogram is built for each switch (hop num vs\r
-number of occurrences). If the histogram reflects a specific column (higher\r
-than others) for a certain node, then it is marked as a root node. Since\r
-the algorithm is statistical, it may not find any root nodes. The list of\r
-the root nodes found by this auto-detect stage is used by the ranking\r
-process stage.\r
-\r
- Note 1: The user can override the node list manually.\r
- Note 2: If this stage cannot find any root nodes, and the user did\r
- not specify a guid list file, OpenSM defaults back to the\r
- Min Hop routing algorithm.\r
-\r
-2. Ranking process - All root switch nodes (found in stage 1) are assigned\r
-a rank of 0. Using the BFS algorithm, the rest of the switch nodes in the\r
-subnet are ranked incrementally. This ranking aids in the process of enforcing\r
-rules that ensure loop-free paths.\r
-\r
-3. Min Hop Table setting - after ranking is done, a BFS algorithm is run from\r
-each (CA or switch) node in the subnet. During the BFS process, the FDB table\r
-of each switch node traversed by BFS is updated, in reference to the starting\r
-node, based on the ranking rules and guid values.\r
-\r
-At the end of the process, the updated FDB tables ensure loop-free paths\r
-through the subnet.\r
-\r
-Note: Up/Down routing does not allow LID routing communication between\r
-switches that are located inside spine "switch systems".\r
-The reason is that there is no way to allow a LID route between them\r
-that does not break the Up/Down rule.\r
-One ramification of this is that you cannot run SM on switches other\r
-than the leaf switches of the fabric.\r
-\r
-\r
-UPDN Algorithm Usage\r
-\r
-Activation through OpenSM\r
-\r
-Use '-R updn' option (instead of old '-u') to activate the UPDN algorithm.\r
-Use '-a <root_guid_file>' for adding an UPDN guid file that contains the\r
-root nodes for ranking.\r
-If the `-a' option is not used, OpenSM uses its auto-detect root nodes\r
-algorithm.\r
-\r
-Notes on the guid list file:\r
-\r
-1. A valid guid file specifies one guid in each line. Lines with an invalid\r
-format will be discarded.\r
-.br\r
-2. The user should specify the root switch guids. However, it is also\r
-possible to specify CA guids; OpenSM will use the guid of the switch (if\r
-it exists) that connects the CA to the subnet as a root node.\r
-\r
-\r
-Fat-tree Routing Algorithm\r
-\r
-The fat-tree algorithm optimizes routing for "shift" communication pattern.\r
-It should be chosen if a subnet is a symmetrical or almost symmetrical\r
-fat-tree of various types.\r
-It supports not just K-ary-N-Trees, by handling for non-constant K,\r
-cases where not all leafs (CAs) are present, any CBB ratio.\r
-As in UPDN, fat-tree also prevents credit-loop-deadlocks.\r
-\r
-If the root guid file is not provided ('-a' or '--root_guid_file' options),\r
-the topology has to be pure fat-tree that complies with the following rules:\r
- - Tree rank should be between two and eight (inclusively)\r
- - Switches of the same rank should have the same number\r
- of UP-going port groups*, unless they are root switches,\r
- in which case the shouldn't have UP-going ports at all.\r
- - Switches of the same rank should have the same number\r
- of DOWN-going port groups, unless they are leaf switches.\r
- - Switches of the same rank should have the same number\r
- of ports in each UP-going port group.\r
- - Switches of the same rank should have the same number\r
- of ports in each DOWN-going port group.\r
- - All the CAs have to be at the same tree level (rank).\r
-\r
-If the root guid file is provided, the topology doesn't have to be pure\r
-fat-tree, and it should only comply with the following rules:\r
- - Tree rank should be between two and eight (inclusively)\r
- - All the Compute Nodes** have to be at the same tree level (rank).\r
- Note that non-compute node CAs are allowed here to be at different\r
- tree ranks.\r
-\r
-* ports that are connected to the same remote switch are referenced as\r
-\'port group\'.\r
-\r
-** list of compute nodes (CNs) can be specified by \'-u\' or \'--cn_guid_file\'\r
-OpenSM options.\r
-\r
-Topologies that do not comply cause a fallback to min hop routing.\r
-Note that this can also occur on link failures which cause the topology\r
-to no longer be "pure" fat-tree.\r
-\r
-Note that although fat-tree algorithm supports trees with non-integer CBB\r
-ratio, the routing will not be as balanced as in case of integer CBB ratio.\r
-In addition to this, although the algorithm allows leaf switches to have any\r
-number of CAs, the closer the tree is to be fully populated, the more\r
-effective the "shift" communication pattern will be.\r
-In general, even if the root list is provided, the closer the topology to a\r
-pure and symmetrical fat-tree, the more optimal the routing will be.\r
-\r
-The algorithm also dumps compute node ordering file (opensm-ftree-ca-order.dump)\r
-in the same directory where the OpenSM log resides. This ordering file provides\r
-the CN order that may be used to create efficient communication pattern, that\r
-will match the routing tables.\r
-\r
-Routing between non-CN nodes\r
-\r
-The use of the cn_guid_file option allows non-CN nodes to be located on different levels in the fat tree.\r
-In such case, it is not guaranteed that the Fat Tree algorithm will route between two non-CN nodes.\r
-To solve this problem, a list of non-CN nodes can be specified by \'-G\' or \'--io_guid_file\' option.\r
-Theses nodes will be allowed to use switches the wrong way round a specific number of times (specified by \'-H\' or \'--max_reverse_hops\'.\r
-With the proper max_reverse_hops and io_guid_file values, you can ensure full connectivity in the Fat Tree.\r
-\r
-Please note that using max_reverse_hops creates routes that use the switch in a counter-stream way.\r
-This option should never be used to connect nodes with high bandwidth traffic between them ! It should only be used\r
-to allow connectivity for HA purposes or similar.\r
-Also having routes the other way around can in theory cause credit loops.\r
-\r
-Use these options with extreme care !\r
-\r
-Activation through OpenSM\r
-\r
-Use '-R ftree' option to activate the fat-tree algorithm.\r
-Use '-a <root_guid_file>' to provide root nodes for ranking. If the `-a' option\r
-is not used, routing algorithm will detect roots automatically.\r
-Use '-u <root_cn_file>' to provide the list of compute nodes. If the `-u' option\r
-is not used, all the CAs are considered as compute nodes.\r
-\r
-Note: LMC > 0 is not supported by fat-tree routing. If this is\r
-specified, the default routing algorithm is invoked instead.\r
-\r
-\r
-LASH Routing Algorithm\r
-\r
-LASH is an acronym for LAyered SHortest Path Routing. It is a\r
-deterministic shortest path routing algorithm that enables topology\r
-agnostic deadlock-free routing within communication networks.\r
-\r
-When computing the routing function, LASH analyzes the network\r
-topology for the shortest-path routes between all pairs of sources /\r
-destinations and groups these paths into virtual layers in such a way\r
-as to avoid deadlock.\r
-\r
-Note LASH analyzes routes and ensures deadlock freedom between switch\r
-pairs. The link from HCA between and switch does not need virtual\r
-layers as deadlock will not arise between switch and HCA.\r
-\r
-In more detail, the algorithm works as follows:\r
-\r
-1) LASH determines the shortest-path between all pairs of source /\r
-destination switches. Note, LASH ensures the same SL is used for all\r
-SRC/DST - DST/SRC pairs and there is no guarantee that the return\r
-path for a given DST/SRC will be the reverse of the route SRC/DST.\r
-\r
-2) LASH then begins an SL assignment process where a route is assigned\r
-to a layer (SL) if the addition of that route does not cause deadlock\r
-within that layer. This is achieved by maintaining and analysing a\r
-channel dependency graph for each layer. Once the potential addition\r
-of a path could lead to deadlock, LASH opens a new layer and continues\r
-the process.\r
-\r
-3) Once this stage has been completed, it is highly likely that the\r
-first layers processed will contain more paths than the latter ones.\r
-To better balance the use of layers, LASH moves paths from one layer\r
-to another so that the number of paths in each layer averages out.\r
-\r
-Note, the implementation of LASH in opensm attempts to use as few layers\r
-as possible. This number can be less than the number of actual layers\r
-available.\r
-\r
-In general LASH is a very flexible algorithm. It can, for example,\r
-reduce to Dimension Order Routing in certain topologies, it is topology\r
-agnostic and fares well in the face of faults.\r
-\r
-It has been shown that for both regular and irregular topologies, LASH\r
-outperforms Up/Down. The reason for this is that LASH distributes the\r
-traffic more evenly through a network, avoiding the bottleneck issues\r
-related to a root node and always routes shortest-path.\r
-\r
-The algorithm was developed by Simula Research Laboratory.\r
-\r
-\r
-Use '-R lash -Q ' option to activate the LASH algorithm.\r
-\r
-Note: QoS support has to be turned on in order that SL/VL mappings are\r
-used.\r
-\r
-Note: LMC > 0 is not supported by the LASH routing. If this is\r
-specified, the default routing algorithm is invoked instead.\r
-\r
-For open regular cartesian meshes the DOR algorithm is the ideal\r
-routing algorithm. For toroidal meshes on the other hand there\r
-are routing loops that can cause deadlocks. LASH can be used to\r
-route these cases. The performance of LASH can be improved by\r
-preconditioning the mesh in cases where there are multiple links\r
-connecting switches and also in cases where the switches are not\r
-cabled consistently. An option exists for LASH to do this. To\r
-invoke this use '-R lash -Q --do_mesh_analysis'. This will\r
-add an additional phase that analyses the mesh to try to determine\r
-the dimension and size of a mesh. If it determines that the mesh\r
-looks like an open or closed cartesian mesh it reorders the ports\r
-in dimension order before the rest of the LASH algorithm runs.\r
-\r
-DOR Routing Algorithm\r
-\r
-The Dimension Order Routing algorithm is based on the Min Hop\r
-algorithm and so uses shortest paths. Instead of spreading traffic\r
-out across different paths with the same shortest distance, it chooses\r
-among the available shortest paths based on an ordering of dimensions.\r
-Each port must be consistently cabled to represent a hypercube\r
-dimension or a mesh dimension. Paths are grown from a destination\r
-back to a source using the lowest dimension (port) of available paths\r
-at each step. This provides the ordering necessary to avoid deadlock.\r
-When there are multiple links between any two switches, they still\r
-represent only one dimension and traffic is balanced across them\r
-unless port equalization is turned off. In the case of hypercubes,\r
-the same port must be used throughout the fabric to represent the\r
-hypercube dimension and match on both ends of the cable. In the case\r
-of meshes, the dimension should consistently use the same pair of\r
-ports, one port on one end of the cable, and the other port on the\r
-other end, continuing along the mesh dimension.\r
-\r
-Use '-R dor' option to activate the DOR algorithm.\r
-\r
-\r
-Routing References\r
-\r
-To learn more about deadlock-free routing, see the article\r
-"Deadlock Free Message Routing in Multiprocessor Interconnection Networks"\r
-by William J Dally and Charles L Seitz (1985).\r
-\r
-To learn more about the up/down algorithm, see the article\r
-"Effective Strategy to Compute Forwarding Tables for InfiniBand Networks"\r
-by Jose Carlos Sancho, Antonio Robles, and Jose Duato at the\r
-Universidad Politecnica de Valencia.\r
-\r
-To learn more about LASH and the flexibility behind it, the requirement\r
-for layers, performance comparisons to other algorithms, see the\r
-following articles:\r
-\r
-"Layered Routing in Irregular Networks", Lysne et al, IEEE\r
-Transactions on Parallel and Distributed Systems, VOL.16, No12,\r
-December 2005.\r
-\r
-"Routing for the ASI Fabric Manager", Solheim et al. IEEE\r
-Communications Magazine, Vol.44, No.7, July 2006.\r
-\r
-"Layered Shortest Path (LASH) Routing in Irregular System Area\r
-Networks", Skeie et al. IEEE Computer Society Communication\r
-Architecture for Clusters 2002.\r
-\r
-\r
-Modular Routine Engine\r
-\r
-Modular routing engine structure allows for the ease of\r
-"plugging" new routing modules.\r
-\r
-Currently, only unicast callbacks are supported. Multicast\r
-can be added later.\r
-\r
-One existing routing module is up-down "updn", which may be\r
-activated with '-R updn' option (instead of old '-u').\r
-\r
-General usage is:\r
-$ opensm -R 'module-name'\r
-\r
-There is also a trivial routing module which is able\r
-to load LFT tables from a file.\r
-\r
-Main features:\r
-\r
- - this will load switch LFTs and/or LID matrices (min hops tables)\r
- - this will load switch LFTs according to the path entries introduced\r
- in the file\r
- - no additional checks will be performed (such as "is port connected",\r
- etc.)\r
- - in case when fabric LIDs were changed this will try to reconstruct\r
- LFTs correctly if endport GUIDs are represented in the file\r
- (in order to disable this, GUIDs may be removed from the file\r
- or zeroed)\r
-\r
-The file format is compatible with output of 'ibroute' util and for\r
-whole fabric can be generated with dump_lfts.sh script.\r
-\r
-To activate file based routing module, use:\r
-\r
- opensm -R file -U /path/to/lfts_file\r
-\r
-If the lfts_file is not found or is in error, the default routing\r
-algorithm is utilized.\r
-\r
-The ability to dump switch lid matrices (aka min hops tables) to file and\r
-later to load these is also supported.\r
-\r
-The usage is similar to unicast forwarding tables loading from a lfts\r
-file (introduced by 'file' routing engine), but new lid matrix file\r
-name should be specified by -M or --lid_matrix_file option. For example:\r
-\r
- opensm -R file -M ./opensm-lid-matrix.dump\r
-\r
-The dump file is named \'opensm-lid-matrix.dump\' and will be generated\r
-in standard opensm dump directory (/var/log by default) when\r
-OSM_LOG_ROUTING logging flag is set.\r
-\r
-When routing engine 'file' is activated, but the lfts file is not specified\r
-or not cannot be open default lid matrix algorithm will be used.\r
-\r
-There is also a switch forwarding tables dumper which generates\r
-a file compatible with dump_lfts.sh output. This file can be used\r
-as input for forwarding tables loading by 'file' routing engine.\r
-Both or one of options -U and -M can be specified together with \'-R file\'.\r
-\r
-.SH FILES\r
-.TP\r
-.B @OPENSM_CONFIG_DIR@/@OPENSM_CONFIG_FILE@\r
-default OpenSM config file.\r
-\r
-.TP\r
-.B @OPENSM_CONFIG_DIR@/@NODENAMEMAPFILE@\r
-default node name map file. See ibnetdiscover for more information on format.\r
-\r
-.TP\r
-.B @OPENSM_CONFIG_DIR@/@PARTITION_CONFIG_FILE@\r
-default partition config file\r
-\r
-.TP\r
-.B @OPENSM_CONFIG_DIR@/@QOS_POLICY_FILE@\r
-default QOS policy config file\r
-\r
-.TP\r
-.B @OPENSM_CONFIG_DIR@/@PREFIX_ROUTES_FILE@\r
-default prefix routes file.\r
-\r
-.SH AUTHORS\r
-.TP\r
-Hal Rosenstock\r
-.RI < hal.rosenstock@gmail.com >\r
-.TP\r
-Sasha Khapyorsky\r
-.RI < sashak@voltaire.com >\r
-.TP\r
-Eitan Zahavi\r
-.RI < eitan@mellanox.co.il >\r
-.TP\r
-Yevgeny Kliteynik\r
-.RI < kliteyn@mellanox.co.il >\r
-.TP\r
-Thomas Sodring\r
-.RI < tsodring@simula.no >\r
-.TP\r
-Ira Weiny\r
-.RI < weiny2@llnl.gov >\r
projects / ~shefty / rdma-win.git / commitdiff