From: Tziporet Koren Date: Mon, 12 Mar 2007 12:13:37 +0000 (+0200) Subject: adding VNIC docs files X-Git-Url: https://openfabrics.org/gitweb/?a=commitdiff_plain;h=8c8af26d83b820d8a822448bfd03b99d544a2cd8;p=~ardavis%2Fofed_docs%2F.git adding VNIC docs files --- diff --git a/VNIC.README b/VNIC.README new file mode 100755 index 0000000..65d88b3 --- /dev/null +++ b/VNIC.README @@ -0,0 +1,134 @@ +This file describes the use of the VNIC ULP service on an OFED stack +and covers the following points: + +A) Creating VNIC interfaces +B) Discovering VEx IOCs present on the fabric using ibvexdm +C) Starting the VNIC driver and the VNIC interfaces +D) Assigning IP addresses etc for the VNIC interfaces +E) Information about the VNIC interfaces +F) Deleting a specific VNIC interface + +A) Creating VNIC interfaces + +The VNIC interfaces can be created with the help of +a configuration file which must be placed at /etc/infiniband/qlogic_vnic.cfg. + +Please take a look at qlogic_vnic.cfg.sample file (available as part of the +documentation) to see how VNIC configuration files are written. You can use +this configuration file as the basis for creating a VNIC configuration file by +copying it to /etc/infiniband/qlogic_vnic.cfg. Of course you will have to +replace the IOCGUID, IOCSTRING values etc in the sample configuration file +with those of the VEx IOCs present on your fabric. + +[As back ward compatibility for the QLogic Infiniserv stack, if the +/etc/infiniband/qlogic_vnic.cfg file is not present, the VNIC service will +look for the /etc/sysconfig/ics_inic.cfg configuration file] + +B) Discovering VEx IOCs present on the fabric using ibvexdm + +For writing the configuration file, you will need information +about the VEx IOCs present on the fabric like their IOCGUID, +IOCSTRING etc. The ibvexdm tool should be used to get this +information. + +When ibvexdm is executed without any options, it displays +detailed information about all the VEx IOCs present on the fabric: + +# ibvexdm +IO Unit Info: + port LID: 0003 + port GID: fe8000000000000000066a0258000001 + change ID: 0009 + max controllers: 0x03 + + + controller[ 1] + GUID: 00066a0130000001 + vendor ID: 00066a + device ID: 000030 + IO class : 2000 + ID: Chassis 0x00066A00010003F2, Slot 1, IOC 1 + service entries: 2 + service[ 0]: 1000066a00000001 / InfiniNIC.InfiniConSys.Control:01 + service[ 1]: 1000066a00000101 / InfiniNIC.InfiniConSys.Data:01 + + controller[ 2] + GUID: 00066a0230000001 + vendor ID: 00066a + device ID: 000030 + IO class : 2000 + ID: Chassis 0x00066A00010003F2, Slot 1, IOC 2 + service entries: 2 + service[ 0]: 1000066a00000002 / InfiniNIC.InfiniConSys.Control:02 + service[ 1]: 1000066a00000102 / InfiniNIC.InfiniConSys.Data:02 + + controller[ 3] + GUID: 00066a0330000001 + vendor ID: 00066a + device ID: 000030 + IO class : 2000 + ID: Chassis 0x00066A00010003F2, Slot 1, IOC 3 + service entries: 2 + service[ 0]: 1000066a00000003 / InfiniNIC.InfiniConSys.Control:03 + service[ 1]: 1000066a00000103 / InfiniNIC.InfiniConSys.Data:03 + + +When ibvexdm is run with -e option, it reports the IOCGUID information +and with -s option it reports the IOCSTRING information for the VEx IOCs +present on the fabric. + +# ibvexdm -e +ioc_guid=00066a0130000001,dgid=fe8000000000000000066a0258000001,pkey=ffff +ioc_guid=00066a0230000001,dgid=fe8000000000000000066a0258000001,pkey=ffff +ioc_guid=00066a0330000001,dgid=fe8000000000000000066a0258000001,pkey=ffff + +#ibvexdm -s +"Chassis 0x00066A00010003F2, Slot 1, IOC 1" +"Chassis 0x00066A00010003F2, Slot 1, IOC 2" +"Chassis 0x00066A00010003F2, Slot 1, IOC 3" + +#ibvexdm -es +ioc_guid=00066a0130000001,dgid=fe8000000000000000066a0258000001,pkey=ffff,"Chassis 0x00066A00010003F2, Slot 1, IOC 1" +ioc_guid=00066a0230000001,dgid=fe8000000000000000066a0258000001,pkey=ffff,"Chassis 0x00066A00010003F2, Slot 1, IOC 2" +ioc_guid=00066a0330000001,dgid=fe8000000000000000066a0258000001,pkey=ffff,"Chassis 0x00066A00010003F2, Slot 1, IOC 3" + +C) Starting the VNIC driver and the VNIC interfaces + +Once you have created a configuration file, you can start the VNIC driver +and create the VNIC interfaces specified in the configuration file with: + +#/sbin/service ql_vnic start + +You can stop the VNIC driver and bring down the VNIC interfaces with + +#/sbin/service ql_vnic stop + +To restart the VNIC driver, you can use + +#/sbin/service ql_vnic restart + +D) Assigning IP addresses etc for the VNIC interfaces + +This can be done with ifconfig or by setting up the ifcfg-XXX (ifcfg-veth0 for +an interface named veth0 etc) network files for the corresponding VNIC interfaces. + +E) Information about the VNIC interfaces + +Information about the created VNIC interfaces can be obtained from +/sys/class/infiniband_vnic/interfaces/. A directory is created +for each interface under this directory. + +The directory for each interface contains information about the interface +and the primary and secondary connections. + +F) Deleting a specific VNIC interface + +VNIC interfaces can be deleted by writing the name of the interface to +the /sys/class/infiniband_vnic/interfaces/delete_vnic file. + +For example to delete interface veth0 + +echo -n veth0 > /sys/class/infiniband_vnic/interfaces/delete_vnic + + + diff --git a/qlogic_vnic.cfg.sample b/qlogic_vnic.cfg.sample new file mode 100755 index 0000000..c74b154 --- /dev/null +++ b/qlogic_vnic.cfg.sample @@ -0,0 +1,102 @@ +# QLogic VNIC configuration file +# +# This file documents and describes the use of the +# VNIC configuration file qlogic_vnic.cfg. This file +# should reside in /etc/infiniband/ql_vnic.cfg +# +# +# Knowing how to fill the configuration file +############################################### +# +# For filling the configuration file you need to know +# some information about your VEx device. This information +# can be obtained with the help of the ibvexdm tool. +# ibvexdm -e will give IOCGUID information about the VEx IOCs +# whereas ibvexdm -s will give IOCSTRING information. +# Refer to QLogic_VNIC.README available as part of the documentation +# for more information about the ibvexdm tool. +# +# +# General structure of the configuration file +############################################### +# +# All lines beginning with a # are treated as comments. +# +# A simple configuration file consists of CREATE commands +# for each VNIC interface to be created. +# +# A simple CREATE command looks like this: +# +# {CREATE; NAME="eioc1"; IOCGUID=0x66A013000010C; +# RX_CSUM=TRUE; +# HEARTBEAT=100; } +# +#Where +# +#NAME - The device name for the interface +# +#IOCGUID - The GUID of the IOC to use. All of the IOC GUIDS detected on +# the fabric can be found with "ibvexdm -e". This can be used instead +# of IOCSTRING. It can not be used at the same time as IOCSTRING. +# +#IOCSTRING - The IOC Profile ID String of the IOC to use. All of the IOC +# Profile ID Strings detected on the fabric can be found with +# "ibvexdm -s". This can be used instead of IOCGUID. +# It can not be used at the same time as IOCGUID. +# +#INSTANCE - Defaults to 0. Range 0-255. If a host will connect to the +# same IOC more than once, each connection must be assigned a unique +# number. +# +#RX_CSUM - defaults to TRUE. When true, indicates that the receive checksum +# should be done by the VEx +# +#HEARTBEAT - defaults to 100. Specifies the time in 1/100'ths of a second +# between heartbeats +# +#PORT - Specification for local HCA port. First port is 1. +# +#HCA - Optional HCA specification for use with PORT specification. First HCA is 1. +# +# Example of IOCGUID based configuration: +# +# {CREATE; NAME="eioc1"; IOCGUID=0x66A013000010C; +# RX_CSUM=TRUE; +# HEARTBEAT=100; } +# +# Example of IOCSTRING based configuration: +# +# {CREATE; NAME="eioc1"; IOCSTRING="Chassis 0x00066A0050000018, Slot 2, IOC 1"; +# RX_CSUM=TRUE; +# HEARTBEAT=100; } +# +# +#Failover configuration: +######################### +# +# It is possible to create a VNIC interface with failover configuration +# by using the PRIMARY and SECONDARY commands. The IOC specified in +# the PRIMARY command will be used as the primary IOC for this interface +# and the IOC specified in the SECONDARY command will be used as the +# fail-over backup in case the connection with the primary IOC fails +# for some reason. +# +# PRIMARY and SECONDARY commands are written in the following way: +# +# PRIMARY={IOCGUID=...; INSTANCE=... } - IOCGUID may be substituted with +# IOCSTRING. IOCGUID, IOCSTRING, and INSTANCE must be values that are +# unique to the primary interface +# +# SECONDARY={IOCGUID=...; INSTANCE=... } - bring up a secondary connection for +# fail-over. IOCGUID may be substituted with IOCSTRING. IOCGUID, IOCSTRING, +# and INSTANCE values to be used for the secondary connection +# +# +#Example failover configuration: +# +# {CREATE; NAME="eioc2"; +# PRIMARY = {IOCGUID=0x66A0130000105; INSTANCE=0; PORT=1; } +# SECONDARY = {IOCGUID=0x66A013000010C; INSTANCE=0; PORT=2; } +# } +# +