Bridging on OpenVZ

I need to create a bridge from an interface inside container, which bridge to my physical LAN. I followed the tutorial / guides on http://wiki.openvz.org/Virtual_Ethernet_device, but it doesn’t seem to make bridging works correctly. So I made some changes:

1. Install bridge-utils

   yum install bridge-utils
   

2. I am using centos. I need to make bridge enabled at startup. Copy this init.d bridge start/stop script. It is based on existing /etc/init.d/network

   #! /bin/bash
   #
   # bridge       Bring up/down bridging
   #
   # chkconfig: 2345 11 89
   # description: Activates/Deactivates network bridges configured to \
   #              start at boot time.
   #
   # (c) Sayid Munawar 
   #
   ### BEGIN INIT INFO
   # Provides: $network
   ### END INIT INFO
   
   # Source function library.
   . /etc/init.d/functions
   
   # Check that networking is up.
   [ "${NETWORKING}" = "no" ] && exit 0
   
   # if the brctl utility isn't around we can't function.
   [ -x /usr/sbin/brctl ] || ( echo "brctl is missing! perhaps need 'yum install bridge-utils' ?" && exit 1 )
   
   CWD=`pwd`
   cd /etc/sysconfig/network-scripts
   
   . ./network-functions
   
   # find all the interfaces besides loopback.
   # ignore aliases, alternative configurations, and editor backup files
   bridges=$(ls brcfg* | \
               LANG=C sed -e "$__sed_discard_ignored_files" \
                          -e '/brcfg-[A-Za-z0-9\._-]\+$/ { s/^brcfg-//g;s/[0-9]/ &/}' | \
               LANG=C sort -k 1,1 -k 2n | \
               LANG=C sed 's/ //')
   
   # params: name bridges
   function brup {
     eval $(LANG=C fgrep "INTERFACES=" brcfg-$1)
     eval $(LANG=C fgrep "IPADDR=" brcfg-$1)
     eval $(LANG=C fgrep "NETMASK=" brcfg-$1)
     ifaces=`echo $INTERFACES | sed 's/[ ;,]/ /g'`
     brctl addbr $1
     for int in "$ifaces"; do (./ifdown $int 1>/dev/null 2>&1;); brctl addif $1 $int; /sbin/ifconfig $int 0; done
     ifconfig $1 $IPADDR netmask $NETMASK
   }
   
   function brdown {
     ifconfig $1 down
     brctl delbr $1
   }
   
   rc=0
   # See how we were called.
   case "$1" in
     start)
           rc=0
           sysctl -e -p /etc/sysctl.conf >/dev/null 2>&1
   
           # bring up all other interfaces configured to come up at boot time
           for i in $bridges; do
                   unset DEVICE INTERFACES
                   eval $(LANG=C fgrep "DEVICE=" brcfg-$i)
                   eval $(LANG=C fgrep "INTERFACES=" brcfg-$i)
   
                   if [ -z "$DEVICE" ] ; then DEVICE="$i"; fi
   
                   if LANG=C egrep -L "^ONBOOT=['\"]?[Nn][Oo]['\"]?" brcfg-$i > /dev/null ; then
                           # this loads the module, to preserve ordering
                           is_available $i
                           continue
                   fi
                   # If we're in confirmation mode, get user confirmation.
                   if [ -f /var/run/confirm ]; then
                           confirm $i
                           test $? = 1 && continue
                   fi
                   action $"Bringing up bridge $i: " brup $i $INTERFACES
                   rc=$((rc+$?))
           done
   
           # Run this again to catch any interface-specific actions
           sysctl -e -p /etc/sysctl.conf >/dev/null 2>&1
   
           touch /var/lock/subsys/bridge
   
           [ -n "${NETWORKDELAY}" ] && /bin/sleep ${NETWORKDELAY}
           ;;
     stop)
           for i in $bridges; do
                   (. brcfg-$i
                   if [ -z "$DEVICE" ] ; then DEVICE="$i"; fi
   
                   action $"Shutting down bridge $i: " brdown $i
                   rc=$((rc+$?))
                   )
           done
   
           rm -f /var/lock/subsys/bridge
           ;;
     status)
           echo $"Configured bridges:"
           brctl show
           ;;
     restart|reload)
           cd "$CWD"
           $0 stop
           $0 start
           ;;
     *)
           echo $"Usage: $0 {start|stop|restart|reload|status}"
           exit 1
   esac
   
   exit $rc
   

   You will need a configuration called brcfg-* in /etc/sysconfig/network-scripts/. For example:

   [root@alengka ~]# cat /etc/sysconfig/network-scripts/brcfg-br0
   DEVICE=br0
   ONBOOT=yes
   IPADDR=192.168.1.82
   NETMASK=255.255.255.0
   INTERFACES="eth1"
   [root@alengka ~]#
   

   note that INTERFACES is the interface you “want” to bridge to internal CT / VPS. Normally, it need at least 2 interfaces to be a real bridge, but in this situation, the openvz script called /usr/sbin/vznetaddbr will do the rest (add CT’s interface to the bridge).

3. Add veth on CT:

   [root@alengka ~]# vzctl set 110 --netif_add eth2,,veth110,,br0 --save
   Saved parameters for CT 110
   

   note that bridge interface (br0) need to be the same as configured in /etc/sysconfig/network-scripts/brcfg-br0

4. edit /etc/vz/vznet.conf to be:

   #!/bin/bash
   EXTERNAL_SCRIPT="/usr/sbin/vznetaddbr"
   

5. modify /usr/sbin/vznetaddbr. I don’t know, the original script didn’t work for me, i need to reorder the way the bridge interface is created. Mine is:

   #!/bin/bash
   #
   # Add virtual network interfaces (veth's) in a container to a bridge on CT0
   
   CONFIGFILE=/etc/vz/conf/$VEID.conf
   . $CONFIGFILE
   
   NETIFLIST=$(echo $NETIF | sed 's/;/\n/g')
   
   if [ ! -n "$NETIFLIST" ]; then
      echo "According to $CONFIGFILE, CT$VEID has no veth interface configured."
      exit 1
   fi
   
   IFACES=$(echo $NETIFLIST | sed 's/;/\n/g')
   for tmp in $IFACES; do
       CTIFNAME=
       CTBRIDGE=
       VZHOSTIF=
   
       NETIF_OPTIONS=$(echo $tmp | sed 's/,/\n/g')
       for str in $NETIF_OPTIONS; do
           # getting 'ifname' parameter value
           if [[ "$str" =~ ^ifname= ]]; then
               # remove the parameter name from the string (along with '=')
   	    CTIFNAME=${str#*=}
           fi
           if [[ "$str" =~ ^bridge= ]]; then
               # remove the parameter name from the string (along with '=')
   	    CTBRIDGE=${str#*=}
           fi
           # getting 'host_ifname' parameter value
           if [[ "$str" =~ ^host_ifname= ]]; then
               # remove the parameter name from the string (along with '=')
   	    VZHOSTIF=${str#*=}
           fi
       done
   
       if [ "$VZHOSTIF" != "$3" ]; then
   	continue
       fi
       if [ ! -n "$CTBRIDGE" ]; then
   	CTBRIDGE=vmbr0
       fi
   
       echo "Adding interface $VZHOSTIF to bridge $CTBRIDGE on CT0 for CT$VEID"
       /usr/sbin/brctl addif $CTBRIDGE $VZHOSTIF
       /sbin/ifconfig $VZHOSTIF 0
       echo 1 > /proc/sys/net/ipv4/conf/$VZHOSTIF/proxy_arp
       echo 1 > /proc/sys/net/ipv4/conf/$VZHOSTIF/forwarding
   
       break
   done
   
   exit 0
   

6. Start the bridge:

   [root@alengka ~]# /etc/init.d/bridge start
   Bringing up bridge br0:                                    [  OK  ]
   [root@alengka ~]#
   

   Note: I didn’t know why, my SSH session freezes about 30 s when I’m doing this, maybe i missed something in my bridge script

7. Start the CT

   [root@alengka ~]# vzctl start 110
   Starting container ...
   Container is mounted
   Adding IP address(es): aaa.bb.cc.dd 192.168.12.110
   Setting CPU limit: 400
   Setting CPU units: 1000
   Setting CPUs: 1
   Configure meminfo: 512000
   Set hostname: somehost.jogjacamp.com
   File resolv.conf was modified
   Setting quota ugidlimit: 10000
   Configure veth devices: veth110
   Adding interface veth110 to bridge br0 on CT0 for CT110
   Container start in progress...
   [root@alengka ~]#
   

8. Test it!
   On CT:

   [root@alengka ~]# vzctl enter 110
   entered into CT 110
   mesg: error: tty device is not owned by group `tty'
   root@amarta [/]# nmblookup mataram
   querying mataram on 192.168.1.255
   192.168.1.1 mataram<00>
   root@amarta [/]#
   

   Voila ! my VPS can now act as SAMBA Server

   Whoops .. dont’t forget to enable it on startup

   chkconfig --add bridge
   

   November 19th, 2009 in Diary | tags: bridge, linux, openvz, vps