5 ms·

Concerning HW NAT/Offload support: the DSA support is (iiuc) supposed to be the framework, through which the offload functionalities will be implemented. There

Concerning HW NAT/Offload support: the DSA support is (iiuc) supposed to be the framework, through which the offload functionalities will be implemented. There is a pull-request for DSA even for architectures as far back as ath79. I haven't read into the specifics yet, but it sounds like nft-tables configuration will be translated to a eBPF program, that will then hook into network flows, and tell the switching HW/NPE over some side channel which accelerations to use and shortcuts to take. But do I really want to trust a "Black Box Network Processing Engine" bolted onto my CPU, that can route packets entirely outside the realm of control of the main CPU? I always wonder about intentional or unintentional "in-band debug features", when people mention, that they run their home-lan on the same switch as the WAN-Uplink. I mean, is there really a HW vendor, that we trust to get the switching logic 100% correct/secure/performant?

scottlamb 5y ago
My understanding from the kernel docs [1] is that DSA is just for switching, not routing. Do you have a citation that suggests otherwise? My GL.iNet Brume supports DSA on its switch with OpenWRT 21.02, as does my older Netgear WNDR3800. I'm not sure there's any hardware/software combination out there that supports offloading NAT/routing with proper QoS. So I'm puzzled when people talk about the importance of offloading or suggest using machines with very low-power CPUs. The Brume is Dual-Core ARM Cortex-A53 @1.0GHz, which gets the job done for me, but I wouldn't want to go any slower than that. [1] https://www.kernel.org/doc/Documentation/networking/dsa/dsa.txt https://www.kernel.org/doc/Documentation/networking/dsa/dsa....
- stragies 5y ago
  The link you posted mentions "setting up offloads", and unifying the functionality of DSA and switchdev. To me, DSA (as a "convenient" means to access/name/treat individual switch ports) is a prerequisite to enable usage of whatever Switch-Chip-Level-Offloads the switch chip supports. Without DSA, vendors had to use nasty hacks to make use of e.g. HWNAT, which is why there never was mainline support for these acceleration features. HWNAT is a routing function, and many/most? switches of the last decade support that. But, OK, true, most of them will probably not be able to HWNAT "with proper QoS". But newer devices are not "just" a switch-chip, they contain a "Network processing engine", which is not a "fixed-function" device, but rather a "programmable" accelerator, that you could perhaps even "teach" how to do PPP, Encrption/VPN wrapping, and/or "offloading NAT/routing with proper QoS". And now that DSA and eBPF are in mainline, using eBPF to generate a program/set_of_rules for the Ethernet acceleration device from your nftable-rules seems like the natural next step for me. Or where do you see my reasoning go off the rails?
  - scottlamb 5y ago
    To paraphrase: you're saying that DSA doesn't support offloads, and most HW NATs don't do QoS, but newer hardware is capable of it, and you expect mainline Linux to make full use of it in the future with DSA. No, I don't know of any problem in your reasoning. I'll look forward to that happening! I wasn't aware newer hardware was so flexible.
    - stragies 5y ago
      Yes, nearly: DSA currently only supports a few basic offloads, and all?/most of those only work on some of the supported DSA targets. I think, MT76xx devices are the most "advanced" in that respect right now. But nothing "advanced" like even pppoe-offloading (yet). I just hope, "flexible" doesn't come with a large "Difficult-to-audit-caveat" side-dish. NPEs "governing" traffic could potentially substantially reduce the trust the user can have towards the platform.

scottlamb 5y ago

My understanding from the kernel docs [1] is that DSA is just for switching, not routing. Do you have a citation that suggests otherwise? My GL.iNet Brume supports DSA on its switch with OpenWRT 21.02, as does my older Netgear WNDR3800. I'm not sure there's any hardware/software combination out there that supports offloading NAT/routing with proper QoS. So I'm puzzled when people talk about the importance of offloading or suggest using machines with very low-power CPUs. The Brume is Dual-Core ARM Cortex-A53 @1.0GHz, which gets the job done for me, but I wouldn't want to go any slower than that. [1] https://www.kernel.org/doc/Documentation/networking/dsa/dsa.txt https://www.kernel.org/doc/Documentation/networking/dsa/dsa....

stragies 5y ago
The link you posted mentions "setting up offloads", and unifying the functionality of DSA and switchdev. To me, DSA (as a "convenient" means to access/name/treat individual switch ports) is a prerequisite to enable usage of whatever Switch-Chip-Level-Offloads the switch chip supports. Without DSA, vendors had to use nasty hacks to make use of e.g. HWNAT, which is why there never was mainline support for these acceleration features. HWNAT is a routing function, and many/most? switches of the last decade support that. But, OK, true, most of them will probably not be able to HWNAT "with proper QoS". But newer devices are not "just" a switch-chip, they contain a "Network processing engine", which is not a "fixed-function" device, but rather a "programmable" accelerator, that you could perhaps even "teach" how to do PPP, Encrption/VPN wrapping, and/or "offloading NAT/routing with proper QoS". And now that DSA and eBPF are in mainline, using eBPF to generate a program/set_of_rules for the Ethernet acceleration device from your nftable-rules seems like the natural next step for me. Or where do you see my reasoning go off the rails?
- scottlamb 5y ago
  To paraphrase: you're saying that DSA doesn't support offloads, and most HW NATs don't do QoS, but newer hardware is capable of it, and you expect mainline Linux to make full use of it in the future with DSA. No, I don't know of any problem in your reasoning. I'll look forward to that happening! I wasn't aware newer hardware was so flexible.
  - stragies 5y ago
    Yes, nearly: DSA currently only supports a few basic offloads, and all?/most of those only work on some of the supported DSA targets. I think, MT76xx devices are the most "advanced" in that respect right now. But nothing "advanced" like even pppoe-offloading (yet). I just hope, "flexible" doesn't come with a large "Difficult-to-audit-caveat" side-dish. NPEs "governing" traffic could potentially substantially reduce the trust the user can have towards the platform.