Lukav's Weblog

Draft

This is a simple how to make your openwrt work with more than one WAN. I’ve used these steps to make my ASUS WL-500G Premium work with 2 WANs where in my case I route specific IP ranges through the second interface. However the instruction can be used to setup load balancing although not so perfect. I’ve actually installed X-Wrt to begin with so I use webif2 interface to setup some of the things. I’ll try to describe those steps without webif2 using ssh, but I must stress that those would be my assumptions and if something doesn’t work you should refer to openwrt and xwrt pages and forums.

As a first step make sure you setup your primary WAN interface. Then we are going setup the second interface. Choose one of the LAN ports of your router and separate this port in a new vlan. In my case I’ve setup my 1-st port as vlan2.

webif2: Network/VLAN, click on “Add New VLAN” then deselect eNet1 from VLAN 0 row and select it on the VLAN 2 row, check Internal also on that row. Click Save and then Apply.

ssh: Execute

nvram set vlan0ports=2 3 4 5*

to remove the port from vlan2 and

nvram set vlan2ports=1 5

to add it in the new vlan, I guess you should also execute:

nvram set vlan2hwname=et0

Now we need to setup IP address and bring up the second interface.

webif2: you can use the System/NVRAM page to setup the NVRAM variables described below: ssh:

nvram set wan2_ifname=vlan2
nvram set wan2_proto=static
nvram set wan2_ipaddr=xyz.xyz.xyz.xyz
nvram set wan2_netmask=xyz.xyz.xyz.xyz
nvram set wan2_gateway=xyz.xyz.xyz.xyz
nvram set ifup_interfaces="lan wan wan2 wifi"

The last line would make sure wan2 interface is broth up on restart. Let’s try

ifup wan2

now. If

ifconfig

show you vlan2 with the ip address you should be in business. Please note that you can use dhcp instead of static proto, but I haven’t tested this.
You can do

nvram commit

so your changes are saved.

Ok. Now we must make sure that requests on a certain interface are replied via the same interface. Detailed information can be found at http://lartc.org/howto/lartc.rpdb.multiple-links.html
Check if the file /etc/iproute2/rt_tables exists and if not create it. Then put/add lines for the 2 interfaces in it:

mkdir /etc/iproute2
echo "201 WAN1" >> /etc/iproute2/rt_tables
echo "202 WAN2" >> /etc/iproute2/rt_tables

I’ve setup a little script that will add all the necessary rules in the ip tables as described in lartc howto. It would also remove the extra default route added for the second interface by the S40network script. So just save the following as S45routing and put it in /etc/init.d so it gets executed after on boot. Make sure it is executable:

S45routing

cat << EOF > /etc/init.d/S45routing
#!/bin/sh

. /etc/functions.sh
WAN1="$(nvram get wan_ifname)"
WAN1_IP="$(nvram get wan_ipaddr)"
WAN1_NETMASK="$(nvram get wan_netmask)"
WAN1_GETEWAY="$(nvram get wan_gateway)"

eval $(ipcalc.sh "$WAN1_IP" "$WAN1_NETMASK")
WAN1_NETWORK=$NETWORK
WAN1_PREFIX=$PREFIX

WAN2="$(nvram get wan2_ifname)"
WAN2_IP="$(nvram get wan2_ipaddr)"
WAN2_NETMASK="$(nvram get wan2_netmask)"
WAN2_GETEWAY="$(nvram get wan2_gateway)"

eval $(ipcalc.sh "$WAN2_IP" "$WAN2_NETMASK")
WAN2_NETWORK=$NETWORK
WAN2_PREFIX=$PREFIX

LAN="$(nvram get lan_ifname)"
LAN_IP="$(nvram get lan_ipaddr)"
LAN_NETMASK="$(nvram get lan_netmask)"

eval $(ipcalc.sh "$LAN_IP" "$LAN_NETMASK")
LAN_NETWORK=$NETWORK
LAN_PREFIX=$PREFIX

route del default gw $WAN2_GETEWAY

ip route flush table WAN1
ip route flush table WAN2
ip route del $WAN1_NETWORK/$WAN1_PREFIX dev $WAN1 src $WAN1_IP
ip route del $WAN2_NETWORK/$WAN2_PREFIX dev $WAN2 src $WAN2_IP
ip rule del from $WAN1_NETWORK/$WAN1_PREFIX table WAN1
ip rule del from $WAN2_NETWORK/$WAN2_PREFIX table WAN2

ip rule add from $WAN1_NETWORK/$WAN1_PREFIX table WAN1 prio 201
ip rule add from $WAN2_NETWORK/$WAN2_PREFIX table WAN2 prio 202

ip route add $WAN1_NETWORK/$WAN1_PREFIX dev $WAN1 src $WAN1_IP
ip route add $WAN2_NETWORK/$WAN2_PREFIX dev $WAN2 src $WAN2_IP

ip route add table WAN1 $WAN1_NETWORK/$WAN1_PREFIX dev $WAN1 src $WAN1_IP
ip route add table WAN1 $LAN_NETWORK/$LAN_PREFIX dev $LAN
ip route add table WAN1 127.0.0.0/8 dev lo
ip route add table WAN1 $WAN2_NETWORK/$WAN2_PREFIX dev $WAN2
ip route add table WAN1 default via $WAN1_GETEWAY dev $WAN1

ip route add table WAN2 $WAN2_NETWORK/$WAN2_PREFIX dev $WAN2 src $WAN2_IP
ip route add table WAN2 $LAN_NETWORK/$LAN_PREFIX dev $LAN
ip route add table WAN2 127.0.0.0/8 dev lo
ip route add table WAN2 $WAN1_NETWORK/$WAN1_PREFIX dev $WAN1
ip route add table WAN2 default via $WAN2_GETEWAY dev $WAN2

ip rule del table BG_ROUTES prio 220
ip rule add table BG_ROUTES prio 220

EOF
chmod +x /etc/init.d/S45routing

Execute the file or restart your router. At this point you should have 2 working WANs where they both reply on ping from external network. Now there are 2 approaches from here:

1: Setup load balancing by specifying multiple default routes. I’ll quote the LARTC on this:

quote: ”

The second question is how to balance traffic going out over the two providers. This is actually not hard if you already have set up split access as above.

Instead of choosing one of the two providers as your default route, you now set up the default route to be a multipath route. In the default kernel this will balance routes over the two providers. It is done as follows (once more building on the example in the section on split-access):

  ip route add default scope global nexthop via $P1 dev $IF1 weight 1
     nexthop via $P2 dev $IF2 weight 1

This will balance the routes over both providers. The weight parameters can be tweaked to favor one provider over the other.

Note that balancing will not be perfect, as it is route based, and routes are cached. This means that routes to often-used sites will always be over the same provider.

”

2: route specific range of ip via the second interface as me:

I’ll layout how I do it and you can tweak it as you like:

First add one more line in /etc/iproute2/rt_tables

echo "220 BG_ROUTES" >> /etc/iproute2/rt_tables

Then I have 2 files in /etc/route directory:

mkdir /etc/route

touch /etc/route/bg_routes.txt

– those are the ip ranges that goes via the second interface

touch /etc/route/exc_bg_routes.txt

– those are the ranges that I want to be sure would go via the first. I do this becuase I get bg_routes.txt from a third party periodically.

Then I use those 2 scripts to populate the tables:

clean_bg_routes.sh

cat << EOF > /etc/route/clean_bg_routes.sh
#!/bin/sh
ip route flush table BG_ROUTES
EOF
chmod +x /etc/route/clean_bg_routes.sh

update_bg_routes.sh (2007.04.01 – this script has been updated with the contribution from robert at irrelevant dot com)

cat << EOF > /etc/route/update_bg_routes.sh
#!/bin/sh

WAN1="$(nvram get wan_ifname)"
WAN1_IP="$(nvram get wan_ipaddr)"
WAN1_NETMASK="$(nvram get wan_netmask)"
WAN1_GETEWAY="$(nvram get wan_gateway)"

WAN2="$(nvram get wan2_ifname)"
WAN2_IP="$(nvram get wan2_ipaddr)"
WAN2_NETMASK="$(nvram get wan2_netmask)"
WAN2_GETEWAY="$(nvram get wan2_gateway)"

ip route flush table BG_ROUTES

awk '{
       if (match($1,/^[0-9]/) && $1 != "192.168.0.0/16" && $1 != "172.16.0.0/12" && $1 != "10.0.0.0/8" )
         print "ip route add table BG_ROUTES "$1" via '$WAN2_GETEWAY' dev '$WAN2'"
     }' /etc/route/bg_routes.txt | ash

awk '{ if (match($1,/^[0-9]/))
         print "ip route add table BG_ROUTES "$1" via '$WAN1_GETEWAY' dev '$WAN1'"
     }' /etc/route/exc_bg_routes.txt | ash
EOF
chmod +x /etc/route/update_bg_routes.sh

I have added /etc/route/update_bg_routes.sh as a last line to the file S45routing so it gets executed at boot.

echo "/etc/route/update_bg_routes.sh" >> /etc/init.d/S45routing

Ok the last thing we need to do is to modify /etc/init.d/S35firewall so the firewall rules apply to both interfaces….

Here is my modified version: (It was been reported that this version is old so it would be better to make the changes by hand in your current version. In the beggining of the script add the lines that sets the variables WAN2 and WAN2DEV. Then follow the script line by line and whenever you see a rule for WAN, dublicate it for WAN2. Skip the rule MINIUPNPD)

last update of this file was:12.04.2007

#!/bin/sh 

## Please make changes in /etc/firewall.user

. /etc/functions.sh
WAN="$(nvram get wan_ifname)"
WAN2="$(nvram get wan2_ifname)"
WANDEV="$(nvram get wan_device)"
WAN2DEV="$(nvram get wan2_device)"
LAN="$(nvram get lan_ifname)"

## CLEAR TABLES
for T in filter nat; do
  iptables -t $T -F
  iptables -t $T -X
done

iptables -N input_rule
iptables -N input_wan
iptables -N output_rule
iptables -N forwarding_rule
iptables -N forwarding_wan

iptables -t nat -N NEW
iptables -t nat -N prerouting_wan
iptables -t nat -N prerouting_rule
iptables -t nat -N postrouting_rule

iptables -N LAN_ACCEPT
[ -z "$WAN" ] || iptables -A LAN_ACCEPT -i "$WAN" -j RETURN
[ -z "$WAN2" ] || iptables -A LAN_ACCEPT -i "$WAN2" -j RETURN
[ -z "$WANDEV" -o "$WANDEV" = "$WAN" ] || iptables -A LAN_ACCEPT -i "$WANDEV" -j RETURN
[ -z "$WAN2DEV" -o "$WAN2DEV" = "$WAN2" ] || iptables -A LAN_ACCEPT -i "$WAN2DEV" -j RETURN
iptables -A LAN_ACCEPT -j ACCEPT

### INPUT
###  (connections with the router as destination)

  # base case
  iptables -P INPUT DROP
  iptables -A INPUT -m state --state INVALID -j DROP
  iptables -A INPUT -m state --state RELATED,ESTABLISHED -j ACCEPT
  iptables -A INPUT -p tcp --tcp-flags SYN SYN --tcp-option ! 2 -j  DROP

  #
  # insert accept rule or to jump to new accept-check table here
  #
  iptables -A INPUT -j input_rule
  iptables -A INPUT -i $WAN -j input_wan
  iptables -A INPUT -i $WAN2 -j input_wan

  # allow
  iptables -A INPUT -j LAN_ACCEPT       # allow from lan/wifi interfaces
  iptables -A INPUT -p icmp     -j ACCEPT       # allow ICMP
  iptables -A INPUT -p gre      -j ACCEPT       # allow GRE

  # reject (what to do with anything not allowed earlier)
  iptables -A INPUT -p tcp -j REJECT --reject-with tcp-reset
  iptables -A INPUT -j REJECT --reject-with icmp-port-unreachable

### OUTPUT
### (connections with the router as source)

  # base case
  iptables -P OUTPUT DROP
  iptables -A OUTPUT -m state --state INVALID -j DROP
  iptables -A OUTPUT -m state --state RELATED,ESTABLISHED -j ACCEPT

  #
  # insert accept rule or to jump to new accept-check table here
  #
  iptables -A OUTPUT -j output_rule

  # allow
  iptables -A OUTPUT -j ACCEPT          #allow everything out

  # reject (what to do with anything not allowed earlier)
  iptables -A OUTPUT -p tcp -j REJECT --reject-with tcp-reset
  iptables -A OUTPUT -j REJECT --reject-with icmp-port-unreachable

### FORWARDING
### (connections routed through the router)

  # base case
  iptables -P FORWARD DROP
  iptables -A FORWARD -m state --state INVALID -j DROP
  iptables -A FORWARD -p tcp --tcp-flags SYN,RST SYN -j TCPMSS --clamp-mss-to-pmtu
  iptables -A FORWARD -m state --state RELATED,ESTABLISHED -j ACCEPT

  #
  # insert accept rule or to jump to new accept-check table here
  #
  iptables -A FORWARD -j forwarding_rule
  iptables -A FORWARD -i $WAN -j forwarding_wan
  iptables -A FORWARD -i $WAN2 -j forwarding_wan

  # allow
  iptables -A FORWARD -i br0 -o br0 -j ACCEPT
  iptables -A FORWARD -i $LAN -o $WAN -j ACCEPT
  iptables -A FORWARD -i $LAN -o $WAN2 -j ACCEPT

  # reject (what to do with anything not allowed earlier)
  # uses the default -P DROP

### MASQ
  iptables -t nat -A PREROUTING -m state --state NEW -p tcp -j NEW
  iptables -t nat -A PREROUTING -j prerouting_rule
  iptables -t nat -A PREROUTING -i $WAN -j prerouting_wan
  iptables -t nat -A PREROUTING -i $WAN2 -j prerouting_wan

  iptables -t nat -A POSTROUTING -j postrouting_rule
  iptables -t nat -A POSTROUTING -o $WAN -j MASQUERADE
  iptables -t nat -A POSTROUTING -o $WAN2 -j MASQUERADE

  iptables -t nat -A NEW -m limit --limit 50 --limit-burst 100 -j RETURN && 
        iptables -t nat -A NEW -j DROP

## USER RULES
[ -f /etc/firewall.user ] && . /etc/firewall.user
[ -e /etc/config/firewall ] && {
        awk -f /usr/lib/common.awk -f /usr/lib/firewall.awk /etc/config/firewall | ash
}

So, that’s about it. I may have missed something or some parts may be inaccurate or unclear, but I’ll update this as feedback from you dear readers comes in.