KVM

Configuring networking on KVM to work with individually routed IPs (or a small subnet of routed IPs) where the routed IPs aren't related to the primary IP of the host involves creating a virtual bridge, enabling some firewall rules and manually creating some routes on both the host and guest.

Virtual bridge configuration

Virtual bridge definition is as follows. The IP address used can be anything private since it's only used internally for routing.

<network>
  <name>routed</name>
  <forward mode='route'/>
  <bridge name='virbr1' dev='eth0' delay='0' />
  <ip address='192.168.123.1' netmask='255.255.255.255'>
  </ip>
</network>

Save the above as net-routed.xml and then create/start the network.

# virsh net-define net-routed.xml
# virsh net-start routed
# virsh net-autostart routed

Startup hooks

Define the IP address(es) to be routed in /etc/libvirt/hooks/routed-ips

ROUTED_GW="192.168.123.1"
ROUTED_DEV="virbr1"
ROUTED_IPS="92.48.112.177 92.48.112.178 92.48.112.179"

This qemu/libvirt script uses the above file and should be created as /etc/libvirt/hooks/qemu (don't forget to set the permissions as +x). The additions to manage the iptables rules were added by me, the original script only added the routes.

#!/bin/sh
# Found at http://blog.gadi.cc/single-ip-routing-in-libvirt/
# Add individual IPs for our routed network to the routing table
#
# Since no hook exists for net-start, the best we can do is check if
# all the IPs are added everytime a VM is launched, without re-adding.
# When a net-destroy occurs, the routes will be automatically removed.
. `dirname $0`/routed-ips
if [ "$2" == "start" ]; then
   for IP in $ROUTED_IPS ; do
       if [ "`ip route list | grep $IP`" == "" ] ; then
           ip route add $IP via $ROUTED_GW dev $ROUTED_DEV
       fi
       # Remove the old firewall rules if present
       iptables -D FORWARD -d $IP -o virbr1 -j ACCEPT 
       iptables -D FORWARD -s $IP -i virbr1 -j ACCEPT 
       # Add them back in before
       iptables -I FORWARD -d $IP -o virbr1 -j ACCEPT 
       iptables -I FORWARD -s $IP -i virbr1 -j ACCEPT 
   done
fi
exit 0

The script is run like this during the startup phase of virtual machines.

# /etc/libvirt/hooks/qemu guest_name start begin -

Guest kickstart

Guest kickstart config should contain the following sections. The post-install script creates default routing via the virtual bridge internal IP.

network --device eth0 --bootproto static --ip=92.48.112.178 --netmask=255.255.255.255 --nameserver=213.229.103.79

%post

cat > /etc/sysconfig/network-script/route-eth0 <<EOF
192.168.123.1 dev eth0
default via 192.168.123.1 dev eth0
EOF