7 ms·

Neat guide but I'd much rather see a guide on how to set up a cost-effective 10Gbps WAN to LAN throughput router/gateway with NAT. Those of us building our own

Neat guide but I'd much rather see a guide on how to set up a cost-effective 10Gbps WAN to LAN throughput router/gateway with NAT. Those of us building our own routers (I've done so since about 2006 when I got 1Gbps fibre installed for the first time) usually have a quite large cross-section with people on bleeding edge internet connections, and with (pro-)consumer 10Gbps internet and networking becoming more and more common a router capable of such is also more relevant. I see very little information about what's the minimal/cost effective hardware requirements capable of routing with NAT at wirespeed for these applications. Most of the content I read is either geared towards slower speeds (e.g. 1Gbps) or using overly beefy machines which are both power hungry and most likely overkill for the purpose. Something that would be interesting to see as well is high speed NAT routing performance comparison between Linux, NetBSD, OpenBSD and FreeBSD using Intel or other well supported 10Gbps nics.

thedougd 5y ago
I've found a few open source NAT implementations on DPDK, but several appears to be reference/demo implementations. This project looks interesting: https://vigor-nf.github.io/ https://vigor-nf.github.io/ DPDK network HBAs have been out for several years now and 25/50/100 is replacing 10 in the datacenter. Maybe there's some deals available on ebay? Quick glance, an Intel X520 is available for ~90 USD.
- ay 5y ago
  VPP (http://fd.io http://fd.io) NAT plugins are used in various production deployments. Various NAT-related perf tests: https://docs.fd.io/csit/master/report/vpp_performance_tests/packet_throughput_graphs/nat44.html https://docs.fd.io/csit/master/report/vpp_performance_tests/... Disclaimer: I am involved with the project
q3k 5y ago
0. Accept that for NAT on home gateway you really don't need wirespeed at minimal MTU, as the majority of your data transfers will be over TCP at max MSS/MTU. 1. Get a Xeon D box (like a Supermicro X10SDV-TLN4F, TDP 45W). 2. Install any Linux distribution on it and just use iptables/nftables. It's good enough. 3. You now have a normal Linux box with normal userspace networking, feel free to also run some more services on it without having to deal with DPDK getting in your way. DPDK is nice and all if you absolutely need to scream through tiny packets at multi-dozen-gigabit speeds, but you almost certainly don't need it at home for Netflix and seeding Ubuntu ISOs.
- dragontamer 5y ago
  > 0. Accept that for NAT on home gateway you really don't need wirespeed at minimal MTU, as the majority of your data transfers will be over TCP at max MSS/MTU. This can't be true for video gamers? I'd assume most video game protocols are tiny UDP packets. But maybe video game protocols don't need "wirespeed", since they're actually quite efficient in terms of bytes transferred. But you want to minimize latency.
  - q3k 5y ago
    I don't think any games rely on more than a few dozen megabits of Internet bandwidth at a few hundred packets per second. And that netfilter/iptables NAT on a Xeon is not gonna add more latency than the equivalent netfilter/iptables NAT that runs on a typical ARM/MIPS home gateway/router.
  - Hikikomori 5y ago
    It's not needed for games, they use a few megabits at most. We use the wirespeed term in the context of networking devices when discussing their limitations. If a device is wirespeed it's capable of routing or switching on all of its interfaces using the smallest packet size, for a 10Gbit interface that is 14.8m packets per second in one direction.
deleted 5y ago
[deleted]
Hikikomori 5y ago
Any old gaming PC with 10G NICs from ebay will do the trick. Install any Linux distro, do some basic config and you're done. Wire speed with the smallest packet size isn't required, you'll probably get at least a few million pps which should be enough. Or ubiquity ER-8-XG.
wmf 5y ago
Vanilla Linux/BSD isn't bad at 10G these days but if you want more performance out of less hardware, TNSR is a productized DPDK/VPP router: https://www.tnsr.com/ https://www.tnsr.com/

thedougd 5y ago

I've found a few open source NAT implementations on DPDK, but several appears to be reference/demo implementations. This project looks interesting: https://vigor-nf.github.io/ https://vigor-nf.github.io/ DPDK network HBAs have been out for several years now and 25/50/100 is replacing 10 in the datacenter. Maybe there's some deals available on ebay? Quick glance, an Intel X520 is available for ~90 USD.

ay 5y ago
VPP (http://fd.io http://fd.io) NAT plugins are used in various production deployments. Various NAT-related perf tests: https://docs.fd.io/csit/master/report/vpp_performance_tests/packet_throughput_graphs/nat44.html https://docs.fd.io/csit/master/report/vpp_performance_tests/... Disclaimer: I am involved with the project

q3k 5y ago

0. Accept that for NAT on home gateway you really don't need wirespeed at minimal MTU, as the majority of your data transfers will be over TCP at max MSS/MTU. 1. Get a Xeon D box (like a Supermicro X10SDV-TLN4F, TDP 45W). 2. Install any Linux distribution on it and just use iptables/nftables. It's good enough. 3. You now have a normal Linux box with normal userspace networking, feel free to also run some more services on it without having to deal with DPDK getting in your way. DPDK is nice and all if you absolutely need to scream through tiny packets at multi-dozen-gigabit speeds, but you almost certainly don't need it at home for Netflix and seeding Ubuntu ISOs.

dragontamer 5y ago
> 0. Accept that for NAT on home gateway you really don't need wirespeed at minimal MTU, as the majority of your data transfers will be over TCP at max MSS/MTU. This can't be true for video gamers? I'd assume most video game protocols are tiny UDP packets. But maybe video game protocols don't need "wirespeed", since they're actually quite efficient in terms of bytes transferred. But you want to minimize latency.
- q3k 5y ago
  I don't think any games rely on more than a few dozen megabits of Internet bandwidth at a few hundred packets per second. And that netfilter/iptables NAT on a Xeon is not gonna add more latency than the equivalent netfilter/iptables NAT that runs on a typical ARM/MIPS home gateway/router.
- Hikikomori 5y ago
  It's not needed for games, they use a few megabits at most. We use the wirespeed term in the context of networking devices when discussing their limitations. If a device is wirespeed it's capable of routing or switching on all of its interfaces using the smallest packet size, for a 10Gbit interface that is 14.8m packets per second in one direction.

deleted 5y ago

[deleted]

Hikikomori 5y ago

Any old gaming PC with 10G NICs from ebay will do the trick. Install any Linux distro, do some basic config and you're done. Wire speed with the smallest packet size isn't required, you'll probably get at least a few million pps which should be enough. Or ubiquity ER-8-XG.

wmf 5y ago

Vanilla Linux/BSD isn't bad at 10G these days but if you want more performance out of less hardware, TNSR is a productized DPDK/VPP router: https://www.tnsr.com/ https://www.tnsr.com/