3 ms·
I've started building a kubernetes cluster (Talos Linux) across town with wireguard between various houses. ZFS boxes for persistent volumes (democratic-csi) i
by benlivengood 7mo ago
I've started building a kubernetes cluster (Talos Linux) across town with wireguard between various houses. ZFS boxes for persistent volumes (democratic-csi) in each "zone" with cross-site snapshot replication and Gateway (Traefik) running at each site behind the ISP. CrunchyPGO allows separate StorageClasses to easily split the leader/followers up.
- nickorlow 7mo agoHave had issues w/ doing k8s over residential wan once I had enough hosts in my cluster (though they were halfway across the US from each other, and not town)
- benlivengood 7mo agoSo far everything is under 15ms apart, but it is a small number of nodes so far. Did you mostly have trouble with etcd?
- nickorlow 7mo agoYeah, etcd was the main culprit, but latency was 150-300ms in my case. At 3 nodes, it was relatively stable (had an issue every week or so that lasted < 5 min), but at 4 the camel's back broke.