3 ms·

I do a lot of similar stuff in my home network with a lot less equipment. I have a trunk switch that sits behind my router, Ethernet running into each of my roo

I do a lot of similar stuff in my home network with a lot less equipment. I have a trunk switch that sits behind my router, Ethernet running into each of my rooms (came with the house), and small point switches that sit in front of each patch. I have 3 APs in my house. I have a UPS which connects to our router, trunk switch, and a single AP that uses PoE which is good enough if we need Wifi when the power goes down. To much shock and horror, I just bought discount Cat6 cabling and didn't bother buying a roll and crimping and making my cabling look nice (but I did test every point in my network exhaustively to make sure I don't have high amounts of latency or jitter anywhere.) The rest of the services that OP runs I run on a combination of a 12 year old SBC and a Raspberry Pi. Modern computers (read < 15 years old) when used as server appliances are very capable and can packet switch very quickly, even while running a kernel and a usermode. Doubt me? Run your own iperf tests and check where things are bottlenecking. My IP cameras are all Raspberry Pis that use the camera module and stream using RTSP back to the SBC. Instead of managing VLANs (I do offer a guest VLAN but my partner and friends found it a pain so it doesn't get much use shrug), I run Zerotier through the house. My IP cameras sit on a single Zerotier network. All other services I run internally use Zerotier and if I do accept external traffic, I expose a port on a VPS (like OP) and have some nftables rules which NAT the traffic through Zerotier. I've thought about getting a failover ISP (our area has 2), but our primary ISP offers us fantastic peering (I've exhaustively tested the bandwidth and jitter to the VPS I proxy traffic through) and good enough uptime for me to not bother. Like a lot of other hobbies, people can get very consumeristic. This is what a consumeristic version of homelabbing looks like. This shouldn't stop you or anyone else from building a humble homelab that can do almost all of what OPs home network does. The key is to test where your network has bottlenecks. I've iperf tested my network exhaustively and frequently look at kernel counters and other things to make sure that my network is performing up to spec, but that's a lot less fun than buying lots of cool networking gear. Much like you don't need an F-150 to get around day-to-day, maybe even just a bike :)

adrian_b 3y ago
> I do a lot of similar stuff in my home network with a lot less equipment. This is also my case. For Internet access, I use a router which has 4 2.5 GbE ports for the internal LAN, which was made several years ago with an Intel NUC and 4 USB Ethernet dongles, but which could be done today with a small and cheap fanless router using an "Alder Lake N" Intel N100 or N200 CPU. There are also such small routers with more 2.5 GbE ports for the internal LAN, e.g. 6 or 8. That router implements a lot of network services, including DNS servers for my domain and my own e-mail server. I also have 5 servers that are interconnected by a 10 GbE subnetwork, but that is done without external switches. Each server has a dual-port 10 Gb/s NIC and they are interconnected directly, in a ring. The servers are normally powered off. Whenever needed, I power on the first of them through Wake-on-Ethernet. Through it, I can power on, also using Wake-on-Ethernet, as many of the others as needed.
- Karrot_Kream 3y ago
  Huh the ring topology is a really cool idea. With the right routing rules it could open up some ports on my trunk switch, though I'd need to upgrade my NICs. Great idea for whenever I decide to retire the SBC though!
  - adrian_b 3y ago
    When all the servers of the ring are powered on, the packets are distributed on both links, so the throughput of the network doubles. When a part of the servers are powered off, the topology of the subnetwork becomes linear and the throughput becomes limited to 10 Gb/s, but with less active servers the traffic is also less. Greater performance can be obtained by using two Ethernet cards with dual 10 Gb/s ports in each computer. With 4 ports per computer total interconnection can be achieved up to 5 nodes and above that each node can be connected to two intersecting rings, which is a topology very frequently used in HPC. An alternative is to use a single card with dual 25 Gb/s ports, per computer. Such cards are not much more expensive than the 10 Gb/s NICs, but the 25 Gb/s switches are very expensive, so a ring topology without switches is even more attractive.

adrian_b 3y ago

> I do a lot of similar stuff in my home network with a lot less equipment. This is also my case. For Internet access, I use a router which has 4 2.5 GbE ports for the internal LAN, which was made several years ago with an Intel NUC and 4 USB Ethernet dongles, but which could be done today with a small and cheap fanless router using an "Alder Lake N" Intel N100 or N200 CPU. There are also such small routers with more 2.5 GbE ports for the internal LAN, e.g. 6 or 8. That router implements a lot of network services, including DNS servers for my domain and my own e-mail server. I also have 5 servers that are interconnected by a 10 GbE subnetwork, but that is done without external switches. Each server has a dual-port 10 Gb/s NIC and they are interconnected directly, in a ring. The servers are normally powered off. Whenever needed, I power on the first of them through Wake-on-Ethernet. Through it, I can power on, also using Wake-on-Ethernet, as many of the others as needed.

Karrot_Kream 3y ago
Huh the ring topology is a really cool idea. With the right routing rules it could open up some ports on my trunk switch, though I'd need to upgrade my NICs. Great idea for whenever I decide to retire the SBC though!
- adrian_b 3y ago
  When all the servers of the ring are powered on, the packets are distributed on both links, so the throughput of the network doubles. When a part of the servers are powered off, the topology of the subnetwork becomes linear and the throughput becomes limited to 10 Gb/s, but with less active servers the traffic is also less. Greater performance can be obtained by using two Ethernet cards with dual 10 Gb/s ports in each computer. With 4 ports per computer total interconnection can be achieved up to 5 nodes and above that each node can be connected to two intersecting rings, which is a topology very frequently used in HPC. An alternative is to use a single card with dual 25 Gb/s ports, per computer. Such cards are not much more expensive than the 10 Gb/s NICs, but the 25 Gb/s switches are very expensive, so a ring topology without switches is even more attractive.