Core Router

Core Router

The core router for the project is a Netgear R6220 running OpenWrt 19.07.1.  It is available used for around $20.

OpenWrt

OpenWrt is an opensource replacement operating system for many consumer routers.  It brings many features to the hardware that aren't usually available with the manufacturer's firmware.  The installation instructions are fairly simple and listed on the OpenWrt web site.  Not all consumer routers will run OpenWrt, many are too memory constraint and many have hardware that requires proprietary drivers that can't or haven't been reverse engineered.  You may have to dig around to find the right combination of OpenWrt and hardware.

Why OpenWrt?

To make the cluster work well and to mimic what one might find in the working world there are features needed namely VLAN tagging and dynamic routing.    The virtual networks in OpenNebula are commonly implemented using VLAN tagging, so it's useful if the external router also supports VLANs.    The dynamic routing is needed because some of the Kubernetes load balancers require upstream routers with dynamic routing protocols.  Also, in my particular setup, I need the ability to advertise the cluster address ranges to the rest of my network.   I could have used one of the many Mikrotik Routerboards I have in my kits, but I wanted to keep this example running off of used hardware consumers were likely to be able to find easily.

Local Changes

Switch Vlan Configuration

A little explanation of the VLAN configuration on this router is needed. There is a simple layer 2 switch in the router.   All of the layer3 forwarding is done in the CPU.  The CPU only has one "ethernet port" so what the designer has done is created a logical "router on a stick".    In the base config, VLAN 1 is typically attached to all the LAN ports and VLAN 2 is attached to the WAN port.  In reality, all the ports are the same at the layer 2 switch level.  In our case, I "disconnected " all the LAN ports from VLAN 1 and attached them to VLAN 200 which is functioning as our native VLAN in our cluster.  We then are attaching VLANs 201,202,203 as tagged VLANs on the LAN ports.  Notice that I had to check the "Enable VLAN functionality" checkbox to be able to create the configuration I wanted.

Adding Interfaces

To add the Layer 3 (IP) interfaces I first clicked the "Add VLAN" button on the Switch screen.  I added a VLAN(I'm using VLAN 204 as an example).  Since 204 is not my LAN native VLAN I marked it as tagged all the way across the LAN ports.   I then clicked "Save & Apply".   Then when I went to the Network->Interfaces screen, I clicked "Add new Interface".  I gave the new Interface the name LAN204, selected "Static address" as the Protocol, and then pulled down to "-- custom --" on the interface pulldown and entered eth0.204 in that field.  Next, I entered the address 192.168.204.1 and selected netmask 255.255.255.0 with the netmask pulldown. ( I often match the VLAN ids and the 3rd octet of the IPv4 address if I'm using /24s just to simplify things.)   I didn't enter anything else.  The DNS and default gateway was configured on the WAN interfaces and should be left blank on all of the LAN interface configurations.  

DNS configuration

For simplicity and ease, I'm using the built-in DNS for the DNS server for the cluster.  Under OpenWRT, the user interface doesn't explicitly mention DNS.  It's actually listed under the Network->Hostnames menu.  It's actually using dnsmasq under the covers which makes it easy to configure both forward and reverse DNS.

Dynamic Routing configuration

root@cr1:~# vtysh
Hello, this is Quagga (version 1.1.1).
Copyright 1996-2005 Kunihiro Ishiguro, et al.
cr1.cluster.sysnetinc.com# show running-config
Building configuration...
Current configuration:
!
log file /tmp/test.log
!
debug ospf6 lsa unknown
!
password zebra
!
interface 6in4-henet
!
interface br-LAN200
!
interface br-LAN201
!
interface br-lan
!
interface eth0
!
interface eth0.1
!
interface eth0.2
 ip ospf area 0.0.0.0
!
interface eth0.200
!
interface eth0.201
!
interface eth0.202
!
interface eth0.203
!
interface eth0.204
!
interface lo
!
interface sit0
!
interface wlan0
!
interface wlan1
!
router ospf
 ospf router-id 192.168.198.254
 redistribute connected
 network 192.168.198.0/24 area 0.0.0.0
 area 0.0.0.0 range 192.168.198.0/24
!
access-list vty permit 127.0.0.0/8
access-list vty deny any
!
ip forwarding
ipv6 forwarding
!
line vty
 access-class vty
!
end

This is the current Quagga/Zebra configuration for the Core router.   There is pretty decent documentation on the Quagga web site.   This configuration will become more complicated as we add Kubernetes load balancers later in this project.  The additional packages added to the OpenWRT router for the opkg repository are:

• quagga
• quagga-bgpd
• quagga-isisd
• quagga-libospf
• quagga-libzebra
• quagga-ospf6d
• quagga-ospfd
• quagga-ripd
• quagga-ripngd
• quagga-vtysh
• quagga-watch
• quaggaquagga-zebra