Difference between revisions of "Basic Commands"

From OpenFabrics Alliance Wiki
Jump to: navigation, search
(→‎sminfo)
(→‎ib_tracert)
 
(9 intermediate revisions by one user not shown)
Line 16: Line 16:
 
   -p Displays IB port information.
 
   -p Displays IB port information.
 
   -v Displays all IB device information.
 
   -v Displays all IB device information.
 +
 +
Example output:
 +
 +
  [markus]# ibstat
 +
  CA 'mlx4_0'
 +
        CA type: MT26428
 +
        Number of ports: 1
 +
        Firmware version: 2.9.1000
 +
        Hardware version: b0
 +
        Node GUID: 0x0002c903004af586
 +
        System image GUID: 0x0002c903004af589
 +
        Port 1:
 +
                State: Active
 +
                Physical state: LinkUp
 +
                Rate: 40
 +
                Base lid: 250
 +
                LMC: 0
 +
                SM lid: 250
 +
                Capability mask: 0x0251086a
 +
                Port GUID: 0x0002c903004af587
 +
                Link layer: InfiniBand
  
 
=== ibv_devinfo ===
 
=== ibv_devinfo ===
Line 26: Line 47:
 
   -l, --list            only list names of RDMA devices
 
   -l, --list            only list names of RDMA devices
 
   -v, --verbose        print all available
 
   -v, --verbose        print all available
 +
 +
Example output:
 +
 +
  [markus]# ibv_devinfo
 +
  hca_id: mlx4_0
 +
        transport: InfiniBand (0)
 +
        fw_ver: 2.9.1000
 +
        node_guid: 0002:c903:004a:f586
 +
        sys_image_guid: 0002:c903:004a:f589
 +
        vendor_id: 0x02c9
 +
        vendor_part_id: 26428
 +
        hw_ver: 0xB0
 +
        board_id: MT_0D90110009
 +
        phys_port_cnt: 1
 +
                port: 1
 +
                        state: PORT_ACTIVE (4)
 +
                        max_mtu: 4096 (5)
 +
                        active_mtu: 4096 (5)
 +
                        sm_lid: 250
 +
                        port_lid: 250
 +
                        port_lmc: 0x00
 +
                        link_layer: InfiniBand
  
 
=== sminfo ===
 
=== sminfo ===
Line 33: Line 76:
 
  This command indicates where the running SM is.
 
  This command indicates where the running SM is.
 
   
 
   
 +
Example output:
 +
 
  sminfo: sm lid 250 sm guid 0x2c903004af587, activity count 1828732680 priority 15 state 3 SMINFO_MASTER
 
  sminfo: sm lid 250 sm guid 0x2c903004af587, activity count 1828732680 priority 15 state 3 SMINFO_MASTER
  
Line 40: Line 85:
 
   
 
   
 
  ibping uses vendor mads to validate connectivity between IB nodes.  
 
  ibping uses vendor mads to validate connectivity between IB nodes.  
 +
The destination is specified by lid.
 
  On exit, (IP) ping like output is show. ibping is run as client/server. Default is to run as client.  
 
  On exit, (IP) ping like output is show. ibping is run as client/server. Default is to run as client.  
 
  Note also that a default ping server is implemented within the kernel.
 
  Note also that a default ping server is implemented within the kernel.
−
 
+
 
  Flags:
 
  Flags:
 
  -c            stop after count packets
 
  -c            stop after count packets
Line 49: Line 95:
 
  -S, --Server  start in server mode (do not return)
 
  -S, --Server  start in server mode (do not return)
  
−
=== ib_tracert ===
+
Server:  (server has a lid of 250)
−
{{{
+
−
Uses SMP to trace path from Source GID/LID to Destinatino GID/LID
+
−
ibtracert uses SMPs to trace the path from a source GID/LID to a destination GID/LID. Each hop along the path is displayed until the destination is reached or a hop does not respond. By using the -m option, multicast path tracing can be performed between source and destination nodes.
+
  
−
Flags:
+
  [markus]# ibping -S
−
-n, --no_info                  simple format; don't show additional information
+
−
-m                              show the multicast trace of the specified mlid
+
−
--node-name-map <node-name-map> Specify a node name map. The node name map file maps GUIDs to more user friendly names.
+
−
}}}
+
  
−
=== perfquery ===
+
Client:
−
{{{
+
−
perfquery uses PerfMgt GMPs to obtain the PortCounters (basic performance and error counters), PortExtendedCounters, PortXmitDataSL, PortRcvDataSL, PortRcvErrorDetails, PortXmitDiscardDetails, PortExtendedSpeedsCounters, or PortSamplesControl from the PMA at the node/port specified. Optionally shows aggregated counters for all ports of node.  Also, optionally, reset after read, or only reset counters.
+
  
−
Note: In PortCounters, PortCountersExtended, PortXmitDataSL, and PortRcvDataSL, components that represent Data (e.g. PortXmitData and PortRcvData) indicate octets divided by 4 rather than just octets.
+
  [mu0007 ~]# ibping 250
 +
  Pong from mu-master.lanl.gov.(none) (Lid 250): time 0.196 ms
 +
  Pong from mu-master.lanl.gov.(none) (Lid 250): time 0.102 ms
 +
  Pong from mu-master.lanl.gov.(none) (Lid 250): time 0.141 ms
  
−
Note: Inputting a port of 255 indicates an operation be performed on all ports.
+
=== ib_tracert ===
  
−
Common Flags:
+
Traces the path from Source GID/LID to Destinatino GID/LID
−
# Debugging Flags
+
Each hop along the path is displayed until the destination is reached or a hop does not respond.  
−
-d    : raise the IB debugging level.
+
By using the -m option, multicast path tracing can be performed between source and destination nodes.
−
      : May be used several times (-ddd or -d -d -d).
+
   
−
-e    : show send and receive errors (timeouts and others)
+
  Flags:
−
-h    : show the usage message
+
  -n, --no_info                   simple format; don't show additional information
−
-v    : increase the application verbosity level.
+
-m                              show the multicast trace of the specified mlid
−
      : May be used several times (-vv or -v -v -v)
+
--node-name-map <node-name-map> Specify a node name map. The node name map file maps GUIDs to more user friendly names.
−
-V    : show the version info.
+
−
 
+
−
# Addressing flags
+
−
-G    : use GUID address argument. In most cases, it is the Port GUID.
+
−
      : Example: "0x08f1040023"
+
−
-s <smlid>:use ’smlid’ as the target lid for SM/SA queries.
+
−
 
+
−
# Other common flags:
+
−
-C <ca_name>: use the specified ca_name.
+
−
-P <ca_port>: use the specified ca_port.
+
−
-t <timeout_ms>: override the default timeout for the solicited mads.
+
−
 
+
−
Multiple CA/Multiple Port Support
+
−
When no IB device or port is specified, the port to use is selected by the following criteria:
+
−
  1. the first port that is ACTIVE.
+
−
  2. if not found, the first port that is UP (physical link up).
+
−
If a port and/or CA name is specified, the user request is attempted to be fulfilled, and will fail if it is not possible.
+
−
 
+
−
}}}
+
−
 
+
−
=== ibdiagnet ===
+
−
{{{
+
−
ibdiagnet scans the fabric using directed route packets and extracts all the available information regarding its connectivity and devices.  It then produces the following files in the output directory (which are defined by the -o option (see below)).
+
−
 
+
−
Common Flag Options:
+
−
-c <count>      : Min number of packets to be sent across each link (default = 10)
+
−
-v              : Enable verbose mode
+
−
-r              : Provides a report of the fabric qualities
+
−
-u              : Credit loop check based on UpDown rules
+
−
-t <topo-file>  : Specifies the topology file name
+
−
-s <sys-name>  : Specifies the local system name. Meaningful only if a topology file is specified
+
−
-i <dev-index> : Specifies the index of the device of the port used to connect to the IB fabric (in case of multiple devices on the local system)
+
−
-p <port-num>  : Specifies the local device’s port num used to connect to the IB fabric
+
−
-o <out-dir>    : Specifies the directory where the output files will be placed (default = /tmp)
+
−
-lw <1x|4x|12x> : Specifies the expected link width
+
−
-ls <2.5|5|10> : Specifies the expected link speed
+
−
-pm            : Dump all the fabric links, pm Counters into ibdiagnet.pm
+
−
-pc            : Reset all the fabric links pmCounters
+
−
-P <PM=<Trash>> : If any of the provided pm is greater then its provided value, print it to screen
+
−
-skip <skip-option(s)> : Skip the executions of the selected checks. Skip options (one or more can be specified) : dup_guids zero_guids pm logical_state part ipoib all
+
−
-wt <file-name>>:  Write out the discovered topology into the given file. This flag is useful if you later want to check
+
−
                    for changes from the current state of the fabric. A directory named ibdiag_ibnl is also created by this
+
−
                    option, and holds the IBNL files required to load this topology. To use these files you will need to set
+
−
                    the environment variable named IBDM_IBNL_PATH to that directory. The directory is located in /tmp or in
+
−
                    the output directory provided by the -o flag.
+
−
-load_db <file-name>> : Load subnet data from the given .db file, and skip subnet discovery stage. Note: Some of the checks require actual subnet discovery, and therefore would not run when load_db is specified. These checks are: Duplicated/zero guids, link state, SMs status.
+
−
-vlr            : VL based Routing - provide this option to create the paths SL file (psl), SL2VL file (slvl) and use them in credit loops check (if -r option is provided).
+
−
                   NOTE: Since number of paths is N^2 extracting the PSL file may take some time.
+
−
-h|--help      : Prints the help page information
+
−
-V|--version    : Prints the version of the tool
+
−
--vars          : Prints the tool’s environment variables and their values
+
−
}}}
+
−
 
+
−
=== ibnetdiscover ===
+
−
{{{
+
−
ibnetdiscover performs IB subnet discovery and outputs a human readable topology file. GUIDs, node types, and port numbers are displayed as well as port LIDs and NodeDescriptions. All nodes (and links) are displayed (full topology). Optionally, this utility can be used to list the current connected nodes by nodetype. The output is printed to standard output unless a topology file is specified.
+
−
 
+
−
Common Options:
+
−
-l, --list        : List of connected nodes
+
−
-g, --grouping    : Show grouping. Grouping correlates IB nodes by different vendor specific schemes. It may also show the switch external ports correspondence.
+
−
-H, --Hca_list    : List of connected CAs
+
−
-S, --Switch_list : List of connected switches
+
−
-R, --Router_list : List of connected routers
+
−
-s, --show        : Show progress information during discovery.
+
−
--node-name-map <node-name-map>: Specify a node name map. The node name map file maps GUIDs to more user friendly names.
+
−
-p, --ports      : Obtain a ports report which is a list of connected ports with relevant information (like LID, portnum, GUID, width, speed, and NodeDescription).
+
−
-m, --max_hops    : Report max hops discovered.
+
−
}}}
+
−
 
+
−
 
+
−
 
+
−
=== ibportstate ===
+
−
{{{
+
−
ibportstate can be used for looking at the current configuration for a port on the fabric as well as reset a particular port on the fabric.  This is handy for some of the leaf switches that don't have any management ability for the switch and you can reset a port on a switch and well as reset the width and speed for that port
+
−
 
+
−
Usage: ibportstate [options] <dest dr_path|lid|guid> <portnum> [<op>]
+
−
 
+
−
Supported ops: enable, disable, reset, speed, espeed, fdr10,
+
−
width, query, down, arm, active, vls, mtu, lid, smlid, lmc,
+
−
mkey, mkeylease, mkeyprot
+
−
 
+
−
Options:
+
−
  --config, -z <config>  use config file, default: /etc/infiniband-diags/ibdiag.conf
+
−
  --Ca, -C <ca>          Ca name to use
+
−
  --Port, -P <port>      Ca port number to use
+
−
  --Direct, -D            use Direct address argument
+
−
  --Lid, -L              use LID address argument
+
−
  --Guid, -G              use GUID address argument
+
−
  --timeout, -t <ms>      timeout in ms
+
−
  --sm_port, -s <lid>    SM port lid
+
−
  --show_keys, -K        display security keys in output
+
−
  --m_key, -y <key>      M_Key to use in request
+
−
  --errors, -e            show send and receive errors
+
−
  --verbose, -v          increase verbosity level
+
−
  --debug, -d            raise debug level
+
−
  --help, -h              help message
+
−
  --version, -V          show version
+
  
−
Examples:
+
Example output:
−
  ibportstate 3 1 disable # by lid
+
−
  ibportstate -G 0x2C9000100D051 1 enable # by guid
+
−
  ibportstate -D 0 1 # (query) by direct route
+
−
  ibportstate 3 1 reset # by lid
+
−
  ibportstate 3 1 speed 1 # by lid
+
−
  ibportstate 3 1 width 1 # by lid
+
−
  ibportstate -D 0 1 lid 0x1234 arm # by direct route
+
  
−
}}}
+
  [root@mu-master markus]# ibtracert 250 199
 +
  From ca {0x0002c903004af586} portnum 1 lid 250-250 "mu-master"
 +
  [1] -> switch port {0x0002c902004484c0}[36] lid 427-427 "muib91"
 +
  [7] -> switch port {0x0002c90200449680}[2] lid 2-2 "muibcore1 Line 32"
 +
  [19] -> switch port {0x0002c902004486f0}[32] lid 449-449 "muibcore1 Spine 01"
 +
  [3] -> switch port {0x0002c90200421750}[19] lid 65-65 "muibcore1 Line 03"
 +
  [1] -> switch port {0x0002c902004492c0}[5] lid 433-433 "muib1"
 +
  [14] -> ca port {0x002590ffff167f8d}[1] lid 199-199 "mu0007"
 +
  To ca {0x002590ffff167f8c} portnum 1 lid 199-199 "mu0007"

Latest revision as of 15:12, 12 November 2014

IB Basic Commands

ibstat

 Displays operational information about one or more InfiniBand network devices. This command displays InfiniBand operational
 information pertaining to a specified Host Channel Adapter Device (HCAD). If an HCAD device name is not entered, status for 
 all available HCADs are displayed. Select a flag to narrow down your search results. You can display specific categories of 
 information, including Node, Port, Interface, and Debug information. You can also choose to display all of the information 
 categories.

 Flags:
 -d	Displays current debug setting.
 -h	Displays ibstat command usage.
 -i	Displays network interface information.
 -n	Displays IB node information.
 -p	Displays IB port information.
 -v	Displays all IB device information.

Example output: 

 [markus]# ibstat
 CA 'mlx4_0'
       CA type: MT26428
       Number of ports: 1
       Firmware version: 2.9.1000
       Hardware version: b0
       Node GUID: 0x0002c903004af586
       System image GUID: 0x0002c903004af589
       Port 1:
               State: Active
               Physical state: LinkUp
               Rate: 40
               Base lid: 250
               LMC: 0
               SM lid: 250
               Capability mask: 0x0251086a
               Port GUID: 0x0002c903004af587
               Link layer: InfiniBand

ibv_devinfo

 Print information about RDMA devices available for use from user space.

 Flags:
 -d, --ib-dev=DEVICE   use IB device DEVICE (default first device found)
 -i, --ib-port=PORT    query port PORT (default all ports)
 -l, --list            only list names of RDMA devices
 -v, --verbose         print all available

Example output: 

 [markus]# ibv_devinfo 
 hca_id:	mlx4_0
       transport:			InfiniBand (0)
       fw_ver:				2.9.1000
       node_guid:			0002:c903:004a:f586
       sys_image_guid:			0002:c903:004a:f589
       vendor_id:			0x02c9
       vendor_part_id:			26428
       hw_ver:				0xB0
       board_id:			MT_0D90110009
       phys_port_cnt:			1
               port:	1
                       state:			PORT_ACTIVE (4)
                       max_mtu:		4096 (5)
                       active_mtu:		4096 (5)
                       sm_lid:			250
                       port_lid:		250
                       port_lmc:		0x00
                       link_layer:		InfiniBand

sminfo

Query InfiniBand SMInfo attribute.
Optionally set and display the output of a sminfo query in human readable format. 
This command indicates where the running SM is.

Example output: 

sminfo: sm lid 250 sm guid 0x2c903004af587, activity count 1828732680 priority 15 state 3 SMINFO_MASTER

ping hostname-ib0

ibping

ibping uses vendor mads to validate connectivity between IB nodes. 
The destination is specified by lid.
On exit, (IP) ping like output is show. ibping is run as client/server. Default is to run as client. 
Note also that a default ping server is implemented within the kernel.

Flags:
-c            stop after count packets
-f, --flood   flood destination: send packets back to back without delay
-o, --oui     use specified OUI number to multiplex vendor mads
-S, --Server  start in server mode (do not return)

Server: (server has a lid of 250) 

 [markus]# ibping -S

Client: 

 [mu0007 ~]# ibping 250
 Pong from mu-master.lanl.gov.(none) (Lid 250): time 0.196 ms
 Pong from mu-master.lanl.gov.(none) (Lid 250): time 0.102 ms
 Pong from mu-master.lanl.gov.(none) (Lid 250): time 0.141 ms

ib_tracert

Traces the path from Source GID/LID to Destinatino GID/LID
Each hop along the path is displayed until the destination is reached or a hop does not respond. 
By using the -m option, multicast path tracing can be performed between source and destination nodes.

Flags:
-n, --no_info                   simple format; don't show additional information
-m                              show the multicast trace of the specified mlid
--node-name-map <node-name-map> Specify a node name map. The node name map file maps GUIDs to more user friendly names.

Example output: 

 [root@mu-master markus]# ibtracert 250 199
 From ca {0x0002c903004af586} portnum 1 lid 250-250 "mu-master"
 [1] -> switch port {0x0002c902004484c0}[36] lid 427-427 "muib91"
 [7] -> switch port {0x0002c90200449680}[2] lid 2-2 "muibcore1 Line 32"
 [19] -> switch port {0x0002c902004486f0}[32] lid 449-449 "muibcore1 Spine 01"
 [3] -> switch port {0x0002c90200421750}[19] lid 65-65 "muibcore1 Line 03"
 [1] -> switch port {0x0002c902004492c0}[5] lid 433-433 "muib1"
 [14] -> ca port {0x002590ffff167f8d}[1] lid 199-199 "mu0007"
 To ca {0x002590ffff167f8c} portnum 1 lid 199-199 "mu0007"
Retrieved from "https://www.openfabrics.org/mediawiki/index.php?title=Basic_Commands&oldid=98"