3 ms·
This is all in my basement but I tried to build it like I would a rack in a datacenter the stack is: 1: EdgeRouter4 connected to ISP. This operates as a typic
by 0xEFF 4y ago
This is all in my basement but I tried to build it like I would a rack in a datacenter the stack is:
1: EdgeRouter4 connected to ISP. This operates as a typical router, nothing special other than some static routes to item 2.
2: Pair of VyOS routers running in VMs. I think of them as top of rack L3 switches in a datacenter. Their primary purpose is for ECMP [1].
They also use VRRP so the can be upgraded without downtime.
3: Three node caddy cluster I mentioned. Running plain old Debian. anycast-healthchecker configures the anycast IP on each VM's loopback interface. Bird advertises the address to the two VyOS routers via BGP.
4: Caddy binds to each address.
All of the hosts in this network are configured with a default route to the VRRP address on the VyOS VM's. That's pretty much it.
I've recently started using Calico in k8s on some separate VM's, and they work the exact same way, advertising Services of type LoadBalancer to the VyOS routers which does ECMP across the nodes. I learned quite a bit configuring anycast-healthchecker with Caddy, then comparing it to how Calico works.
[1]: https://codecave.cc/multipath-routing-in-linux-part-2.html https://codecave.cc/multipath-routing-in-linux-part-2.html
- jabart 4y agoGot it, ECMP was the missing link to that and makes sense. Thanks!