Creating additional Xen virtual network bridges

Configuring additional bridges between virtual machines and physical networks in Xen increases virtual network performance. An expert gives the necessary commands and scripts in this tip.

Xen's networking capabilities are still under construction, but Xen bridges are ready to go. The virtual network bridge mode really works without problems, and there are many configurations options available to users. In this article, you'll read how to create additional Xen bridges, which come in quite handy when your physical server has more than one network interface and you want to bind virtual machines to a given network board.

Networking with Xen
Let's give a quick overview of the way networking is organized in Xen. On the privileged domain, you'll see a xenbr0 device by default. Connected to this virtual bridge, you'll see the vif interface with a name that looks like vifx.y. In this name, x is the numerical representation of the domain, and y is the representation of the interface on the bridge. Within the virtual machines themselves, virtual Ethernet interfaces are used. These virtual Ethernet interfaces are connected to one of the vif interfaces. For example, eth0 in the virtual machine with id 1 is connected to vif1.0 in the privileged domain. A useful command for finding out the configuration of a Xen bridge is brctl show, which shows the bridge, some configuration settings and all interfaces connected to it.

Here, the brctl show command gives an overview of the current bridge configuration:

lin:~ # brctl show 
bridge name     bridge id               STP enabled     interfaces 
xenbr0              8000.feffffffffff       no                   vif0.0 

Configuring the Xen bridge
First, take a look at the commands and scripts that are used to configure the virtual network bridge in Xen. The first script that is used is /etc/xen/xend-config.sxp. This script has some generic settings for the virtual machine, including the following two lines:
(network-script network-bridge) (vif-script vif-bridge)

The first line makes sure that the network-bridge script is executed, which sets up the virtual network bridge. This script uses the brctl and ip commands to set up the bridge. When it starts, the following steps are executed:

  1. The physical interface eth0 is renamed to peth0.
  2. virtual interface eth0 is created.
  3. The Media Access Control (MAC) address and configuration associated to peth0 is copied to eth0.
  4. The Address Resolution Protocol (ARP) protocol is disabled for peth0, which actually disabled functionality on the interface completely.
  5. The virtual bridge xenbr0 is created.
  6. The interfaces peth0 and vif0.0 are connected to the bridge.

After setting up the bridge in this way, the network-bridge script adds other necessary interfaces. For instance, if the dom0 needs a second interface to be added to the bridge, the following would happen:

ip address add dev veth1
ip link set veth1 up
ip link set vif0.1 up 
brctl addif xenbr0 vif0.1

In this command sequence, the interface veth1 also plays a role. This interface is not much of a concern when managing the bridge; however, for the bridge to function well, it needs this device internally.

Once the network bridge is up, the vif-bridge script comes into sight. This script is responsible for creating the vifx.y interfaces for unprivileged domains. This script will first add the vifx.y interface to the bridge and then disable ARP on this interface, which makes sure that the interface is used internally for the bridge.

Working with more than one Xen bridge
One bridge works fine for a server that has only one Ethernet interface. If a server has more than one interface, it may be useful to separate traffic between interfaces by creating additional bridges. In this way, one could connect xenbr0 to peth0 and xenbr1 to peth1 and give a virtual machine in say dom1 exclusive access to xenbr1. Everything necessary for this is present in the network-bridge script. For example, the following command would create a second bridge that is connected to the eth1 network board:

lin:/etc/xen/scripts # ./network-bridge netdev=eth1 bridge=xenbr1 start

While useful from the command line, the /etc/xen/xend-config.sxp script will require some tuning in order to create bridges automatically during boot. By default, this script calls the network-bridge script; however, this script can only be called once. To avoid this problem, you need to create a network-wrapper script, which can be configured to call the network-bridge script twice. Use the following steps to configure this:

  1. Create a script /etc/xen/scripts/network-wrapper with the following contents:
    /etc/xen/scripts/network-bridge netdev=eth0 bridge=xenbr0 start /etc/xen/scripts/network-bridge netdev=eth1 bridge=xenbr1 start
  2. Tune the /etc/xen/xend-config.sxp script so that it calls this network wrapper script, by adding the following:
    # (network-script network-bridge) (network-script network-wrapper_ (vif-script vif-bridge)
  3. Make sure that in the configuration file for each of the unprivileged domains, you indicate what network bridge to use. This would make the vif lines look like the following example line:
    vif=[ 'bridge=xenbr1', 'mac=00:16:3e:07:d2:0e', ]

Modern servers normally have several physical network boards. To benefit from this in a Xen environment, it's a good idea to configure additional network bridges and give virtual machines exclusive access, creating better performance on a virtual network.

About the author: Sander van Vugt is an author and independent technical trainer, specializing in Linux since 1994. Vugt is also a technical consultant for high availability (HA) clustering and performance optimization, as well as an expert on SUSE Linux Enterprise Desktop 10 (SLED 10) administration.

Dig Deeper on Server consolidation and improved resource utilization