-------------------------------------------------------------------------------
ib-bonding is a High Availability solution for IPoIB interfaces. It is based
on the Linux Ethernet Bonding Driver and was adopted to work with IPoIB.
+However, the support for for IPoIB interfaces is only for the active-backup
+mode, other modes should not be used.
ib-bonding package contains a bonding driver and a utility called ib-bond to
manage and control the driver operation.
-----------------------------------------------------------------
* In the master (bond) interface script add the line:
TYPE=Bonding
+MTU=<according to the slave's MTU>
Exmaple: for bond0 (master) the file is named /etc/sysconfig/network-scripts/ifcfg-bond0
with the following text in the file:
BOOTPROTO=none
USERCTL=no
TYPE=Bonding
+MTU=65520
+
+Note: 65520 is a valid mtu value only if all IPoIB slaves operate in connected
+mode and are configured with the same value. For IPoIB slaves that work in
+datagram modee, use MTU=2044. If you don't set correct mtu or don't set mtu at
+all (and letting it to be set to the default value), performance of the
+interface might decrease.
* In the slave (ib) interface script put the following lines:
SLAVE=yes
MASTER=<bond name>
TYPE=InfiniBand
+PRIMARY=<yes|no>
Example: the script for ib0 (slave) would be named /etc/sysconfig/network-scripts/ifcfg-ib0
with the following text in the file:
SLAVE=yes
BOOTPROTO=none
TYPE=InfiniBand
+PRIMARY=yes
+
+Note: If the slave interface is not primary then the line PRIMARY= is not
+required and can be omitted.
After the configuration is saved, restart the network service by running:
/etc/init.d/network restart
Follow the instructions in 3.1.1 (Writing network scripts under Redhat-AS4)
with the following changes:
* In the bondX (master) script - the line TYPE=Bonding is not needed.
+* In the bondX (master) script - you may add to the configuration more options
+with the following line
+BONDING_OPTS=" primary=ib0 updelay=0 downdelay=0"
* in the ibX (slave) script - the line TYPE=InfiniBand necessary when using
bonding over devices configured with partitions ( p_key)
Example:
BONDING_MODULE_OPTS="mode=active-backup miimon=<value>"
BONDING_SLAVE0=slave0
BONDING_SLAVE1=slave1
+MTU=<according to the slave's MTU>
Exmaple: for bond0 (master) the file is named /etc/sysconfig/network/ifcfg-bond0
with the following text in the file:
REMOTE_IPADDR=""
STARTMODE="onboot"
BONDING_MASTER="yes"
-BONDING_MODULE_OPTS="mode=active-backup miimon=100"
+BONDING_MODULE_OPTS="mode=active-backup miimon=100 primary=ib0 updelay=0 downdelay=0"
BONDING_SLAVE0=ib0
BONDING_SLAVE1=ib1
+MTU=65520
+
+Note: 65520 is a valid mtu value only if all IPoIB slaves operate in connected
+mode and are configured with the same value. For IPoIB slaves that work in
+datagram modee, use MTU=2044. If you don't set correct mtu or don't set mtu at
+all (and letting it to be set to the default value), performance of the
+interface might decrease.
+
+Note: primary, downdelay and updelay is an optional bonding interface
+configuration. You may choose to use them, change them or delete them from the
+configuration script (by editing the line that starts with BONDING_OPTS)
* The slave (ib) interace script should look like this:
It is not possible to have a mix of Ethernt slaves and IPoIB slaves under the
same bonding master. It is possible however that a bonding master of Ethernet
slaves and a bonding master of IPoIB slaves will co-exist in one machne.
-To configure Ethernet slaves under a bonding master use the same instructions
-as for IPoIB slaves (according to the OS) with one exception. When working
-under Redhat-AS4 do the following when configuring a bonding master with
-Ethernet slaves
+To configure Ethernet slaves under a bonding master use the following
+instructios (depending on the OS)
+
+* Under Redhat-AS4
+
+Use the same instructions as for IPoIB slaves with the following exceptions
- In the master configuration file add the line
SLAVEDEV=1
- In the slave configuration file leave the line
TYPE=InfiniBand
+- For Ethernet, it is possible to set parameters of the bonding module in /etc/modprobe.conf
+with the following line for example
+options bonding miimon=100 mode=1 primary=eth0
+Note that alias names for the bonding module (such as bond0) may not work.
+
+* Under Redhat-AS5
+
+No special instructions are required.
+
+* Under SLES10
-When using both type of bonding under SLES-10, it is neccessary to update the
+When using both type of bonding under, it is neccessary to update the
MANDATORY_DEVICES environment variable in /etc/sysconfig/network/config with the names
of the InfiniBand devices ( ib0, ib1, etc. ). Otherwise, bonding devices will be created
before InfiniBand devices at boot time.
+Note: If there is more than one Ethernet NIC installed then there might be a
+race for the interface name eth0, eth1 etc. This may lead to unexpected
+relation between logical and physical devices which may lead to wrong bonding
+configuration. This issue may be solved by binding a logical device name (e.g.
+eth0) to a physical (hardware) device by specifying the MAC address in the
+ethN configuration file.