Core Router
I'm using a Ubiquiti Networks Edgerouter POE-5-Port Router as the core router for this project. They are currently selling used on Amazon for $88. It checks every box I need for this project.
- BGP - To talk to the Metallb Load balancer
- 802.1Q tagging - Allows the logically separation the management of the ProxMox nodes and MicroK8s VMs
- OSPF - Share the cluster routes with the rest of the office/house
- Built in switch - allows for low latency line rate communications between the Kubernetes nodes.
The EdgeRouter POE-5 is an elegant little design for a router. Eth0 and eth1 are both connected to the CPU, presumably to go to two different ISPs for load balancing and redundancy. Eth2, eth3, and eth4 are connected to a switch chip, which in turn is connected to the CPU. The upside of the switch chip is that it allows for high-speed, low-latency traffic between those ports. The downside is that all three devices share the 1 gig connection to the CPU (Conveniently, I have 3 nodes in the cluster so this fits well ;-) ). In this scenario, my uplink to the rest of the office is limited to 1G anyway, and my connection to the outside world is well below that in actuality, so the downside doesn't affect anything in this case.
Configuration
interfaces {
ethernet eth0 {
address 192.168.198.254/24
duplex auto
ip {
ospf {
dead-interval 40
hello-interval 10
priority 1
retransmit-interval 5
transmit-delay 1
}
}
speed auto
}
ethernet eth1 {
address dhcp
duplex auto
poe {
output off
}
speed auto
}
ethernet eth2 {
duplex auto
poe {
output off
}
speed auto
}
ethernet eth3 {
duplex auto
poe {
output off
}
speed auto
}
ethernet eth4 {
duplex auto
poe {
output off
}
speed auto
}
loopback lo {
}
switch switch0 {
address 192.168.200.1/24
mtu 7000
switch-port {
interface eth2 {
}
interface eth3 {
}
interface eth4 {
}
vlan-aware disable
}
vif 201 {
address 192.168.201.1/24
description "VLAN 201"
ip {
}
}
}
}
protocols {
bgp 64500 {
maximum-paths {
ibgp 8
}
neighbor 192.168.201.101 {
remote-as 64500
}
neighbor 192.168.201.102 {
remote-as 64500
}
neighbor 192.168.201.103 {
remote-as 64500
}
parameters {
router-id 192.168.201.1
}
}
ospf {
area 0.0.0.0 {
area-type {
normal
}
network 172.31.255.4/32
network 192.168.198.0/24
}
parameters {
abr-type cisco
router-id 172.31.255.4
}
redistribute {
bgp {
metric-type 2
}
connected {
metric-type 2
}
}
}
}
service {
dns {
forwarding {
cache-size 10000
except-interface eth0
name-server 192.168.198.1
}
}
}
system {
name-server 192.168.198.1
ntp {
server 192.168.198.1 {
}
}
time-zone UTC
}
Synopsis of Configuration
- interfaces
- ethernet eth0
- Transit network to rest of office
- OSPF enabled
- ethernet eth1
- unused in this configuration
- ethernet eth2
- configured as part of switch
- ethernet eth3
- configured as part of switch
- ethernet eth4
- configured as part of switch
- switch switch0
- address 192.168.200.1/24 - management network for proxmox nodes
- vif 201
- address 192.168.201.1/24 - access network for Kubernetes VMs
- protocols
- bgp 64500
- bgp configuration for Metallb Load balancer
- ospf
- ospf configuration for sharing routes with and learning routes from the rest of the network
- redistribute
- bgp
- share bgp routes with ospf neighbors
- connected
- share physically connected routes with ospf neighbors
- service
- dns
- proxy dns server for both proxmox cluster and Kubernetes cluster
- system
- name-server
- define nameserver for both router and proxy requests
- ntp
- define ntp server for routers
- time-zone
- leave time zone as UTC -- habit from real jobs where systems and network gear are all over the world
No comments:
Post a Comment