4 ms·

One downside to the Raspberry Pi (CM)4 is that it doesn't have hardware crypto. So it'd make a poor VPN machine. But otherwise I think it'd be fine. Some people

One downside to the Raspberry Pi (CM)4 is that it doesn't have hardware crypto. So it'd make a poor VPN machine. But otherwise I think it'd be fine. Some people are happy just using a regular Pi 4 connected to a VLAN-capable switch or with a USB adapter for the second NIC. I recently bought a GL.iNet Brume. It's a nice tiny ARM machine I use as a wired-only router. /proc/cpuinfo confirms it supports hardware crypto... Features : fp asimd evtstrm aes pmull sha1 sha2 crc32 cpuid ...although I haven't set up a VPN on it yet. Downsides: the WAN and LAN0/LAN1 ports are all the same switch with only one network interface, so it'd be impossible to achieve >500Mbps full-duplex. (It's never come up for me; my Internet connection's too slow.) It's also a little pricy for what it is. I might have tried the FriendlyELEC NanoPI R4S instead, which actually has two separate NICs. But if I'd ordered that I'd still be waiting for it... One thing I don't care about: hardware NAT/routing offload. I think that's often unsupported by OpenWRT, and it's incompatible with SQM.

squarefoot 5y ago
Not sure if it supports all the features you need, anyway you might find interesting the PCEngines APU2 boards. https://pcengines.ch/apu2.htm https://pcengines.ch/apu2.htm
- scottlamb 5y ago
  Those are neat looking machines. I saw them and iirc quickly ruled them out because they're not available now.
- cycomanic 5y ago
  Anyone has experiences with the complete build from https://teklager.se/en/products/routers/ https://teklager.se/en/products/routers/ They look very nice (although unavailable till January)
stragies 5y ago
Concerning HW NAT/Offload support: the DSA support is (iiuc) supposed to be the framework, through which the offload functionalities will be implemented. There is a pull-request for DSA even for architectures as far back as ath79. I haven't read into the specifics yet, but it sounds like nft-tables configuration will be translated to a eBPF program, that will then hook into network flows, and tell the switching HW/NPE over some side channel which accelerations to use and shortcuts to take. But do I really want to trust a "Black Box Network Processing Engine" bolted onto my CPU, that can route packets entirely outside the realm of control of the main CPU? I always wonder about intentional or unintentional "in-band debug features", when people mention, that they run their home-lan on the same switch as the WAN-Uplink. I mean, is there really a HW vendor, that we trust to get the switching logic 100% correct/secure/performant?
- scottlamb 5y ago
  My understanding from the kernel docs [1] is that DSA is just for switching, not routing. Do you have a citation that suggests otherwise? My GL.iNet Brume supports DSA on its switch with OpenWRT 21.02, as does my older Netgear WNDR3800. I'm not sure there's any hardware/software combination out there that supports offloading NAT/routing with proper QoS. So I'm puzzled when people talk about the importance of offloading or suggest using machines with very low-power CPUs. The Brume is Dual-Core ARM Cortex-A53 @1.0GHz, which gets the job done for me, but I wouldn't want to go any slower than that. [1] https://www.kernel.org/doc/Documentation/networking/dsa/dsa.txt https://www.kernel.org/doc/Documentation/networking/dsa/dsa....
  - stragies 5y ago
    The link you posted mentions "setting up offloads", and unifying the functionality of DSA and switchdev. To me, DSA (as a "convenient" means to access/name/treat individual switch ports) is a prerequisite to enable usage of whatever Switch-Chip-Level-Offloads the switch chip supports. Without DSA, vendors had to use nasty hacks to make use of e.g. HWNAT, which is why there never was mainline support for these acceleration features. HWNAT is a routing function, and many/most? switches of the last decade support that. But, OK, true, most of them will probably not be able to HWNAT "with proper QoS". But newer devices are not "just" a switch-chip, they contain a "Network processing engine", which is not a "fixed-function" device, but rather a "programmable" accelerator, that you could perhaps even "teach" how to do PPP, Encrption/VPN wrapping, and/or "offloading NAT/routing with proper QoS". And now that DSA and eBPF are in mainline, using eBPF to generate a program/set_of_rules for the Ethernet acceleration device from your nftable-rules seems like the natural next step for me. Or where do you see my reasoning go off the rails?
    - scottlamb 5y ago
      To paraphrase: you're saying that DSA doesn't support offloads, and most HW NATs don't do QoS, but newer hardware is capable of it, and you expect mainline Linux to make full use of it in the future with DSA. No, I don't know of any problem in your reasoning. I'll look forward to that happening! I wasn't aware newer hardware was so flexible.

squarefoot 5y ago

Not sure if it supports all the features you need, anyway you might find interesting the PCEngines APU2 boards. https://pcengines.ch/apu2.htm https://pcengines.ch/apu2.htm

scottlamb 5y ago
Those are neat looking machines. I saw them and iirc quickly ruled them out because they're not available now.
cycomanic 5y ago
Anyone has experiences with the complete build from https://teklager.se/en/products/routers/ https://teklager.se/en/products/routers/ They look very nice (although unavailable till January)

stragies 5y ago

Concerning HW NAT/Offload support: the DSA support is (iiuc) supposed to be the framework, through which the offload functionalities will be implemented. There is a pull-request for DSA even for architectures as far back as ath79. I haven't read into the specifics yet, but it sounds like nft-tables configuration will be translated to a eBPF program, that will then hook into network flows, and tell the switching HW/NPE over some side channel which accelerations to use and shortcuts to take. But do I really want to trust a "Black Box Network Processing Engine" bolted onto my CPU, that can route packets entirely outside the realm of control of the main CPU? I always wonder about intentional or unintentional "in-band debug features", when people mention, that they run their home-lan on the same switch as the WAN-Uplink. I mean, is there really a HW vendor, that we trust to get the switching logic 100% correct/secure/performant?

scottlamb 5y ago
My understanding from the kernel docs [1] is that DSA is just for switching, not routing. Do you have a citation that suggests otherwise? My GL.iNet Brume supports DSA on its switch with OpenWRT 21.02, as does my older Netgear WNDR3800. I'm not sure there's any hardware/software combination out there that supports offloading NAT/routing with proper QoS. So I'm puzzled when people talk about the importance of offloading or suggest using machines with very low-power CPUs. The Brume is Dual-Core ARM Cortex-A53 @1.0GHz, which gets the job done for me, but I wouldn't want to go any slower than that. [1] https://www.kernel.org/doc/Documentation/networking/dsa/dsa.txt https://www.kernel.org/doc/Documentation/networking/dsa/dsa....
- stragies 5y ago
  The link you posted mentions "setting up offloads", and unifying the functionality of DSA and switchdev. To me, DSA (as a "convenient" means to access/name/treat individual switch ports) is a prerequisite to enable usage of whatever Switch-Chip-Level-Offloads the switch chip supports. Without DSA, vendors had to use nasty hacks to make use of e.g. HWNAT, which is why there never was mainline support for these acceleration features. HWNAT is a routing function, and many/most? switches of the last decade support that. But, OK, true, most of them will probably not be able to HWNAT "with proper QoS". But newer devices are not "just" a switch-chip, they contain a "Network processing engine", which is not a "fixed-function" device, but rather a "programmable" accelerator, that you could perhaps even "teach" how to do PPP, Encrption/VPN wrapping, and/or "offloading NAT/routing with proper QoS". And now that DSA and eBPF are in mainline, using eBPF to generate a program/set_of_rules for the Ethernet acceleration device from your nftable-rules seems like the natural next step for me. Or where do you see my reasoning go off the rails?
  - scottlamb 5y ago
    To paraphrase: you're saying that DSA doesn't support offloads, and most HW NATs don't do QoS, but newer hardware is capable of it, and you expect mainline Linux to make full use of it in the future with DSA. No, I don't know of any problem in your reasoning. I'll look forward to that happening! I wasn't aware newer hardware was so flexible.