3 ms·

I wonder if these large cache sizes make it possible to operate functionally RAM-less servers? When optimized for size, it'd probably be possible to fit some mi

by crote 2y ago

I wonder if these large cache sizes make it possible to operate functionally RAM-less servers? When optimized for size, it'd probably be possible to fit some microservices in an MB of 8 of RAM. You can fit a decent bit of per-connection client state in a few KB, which can take up the rest of the cache. Throw in the X3D cache for 96MB per CCD and you can actually do some pretty serious stuff. With the right preprocessing, it should be possible to essentially stream from NIC straight to CPU, process some stuff, and output it straight to the NIC again without ever touching RAM - although I doubt current DMA hardware allows this. It'd require quite a bit of re-engineering of the on-wire protocol to remove the need for any nontrivially-sized buffers, but I reckon it isn't impossible.

yusyusyus 2y ago
> With the right preprocessing, it should be possible to essentially stream from NIC straight to CPU, process some stuff, and output it straight to the NIC again without ever touching RAM this already exists [0] [0] https://www.intel.com/content/dam/www/public/us/en/documents/technology-briefs/data-direct-i-o-technology-brief.pdf https://www.intel.com/content/dam/www/public/us/en/documents...
- crote 2y ago
  I wasn't aware of that, thanks for sharing!
timschmidt 2y ago
Lots of older or low end wireless routers have only 16 or 32mb RAM. A full linux stack fits in that. I do a full HTTPS stack on ESP32-C3 in ~500k. Lots of limitations, of course. Probably the biggest difficulty is that most modern systems simply refuse to boot without RAM installed, so you can't choose any platform without BIOS / EFI modifications. Definitely a worthwhile idea though.
- crote 2y ago
  The difference is that those low end routers offload most or even all of the packet processing to dedicated networking hardware: it never even touches the CPU. The whole Linux stack is only used to host the management interface, configure the switching hardware, and deal with high-level stuff like DNS and DHCP. That's also why VPN performance is so horribly bad: there's no hardware acceleration for encryption, so it needs to be done on the CPU, which is not sized to deal with that kinds of traffic.
  - timschmidt 2y ago
    Offloading the packet processing doesn't really save any RAM at all. The limitations you're discussing are related to running on a low power ~700mhz MIPS or ARM core with very low performance. As you said, using a VPN or adding additional routing rules requires the packets to pass through the CPU, which demonstrates that all the software and buffers for doing the routing on the CPU are present within the 16 or 32Mb ram. It's just attached to a very low performance CPU core. It's perfectly possible to load the same OpenWRT release on a modern x86 and run in the same amount of ram with much higher performance. That's how I do my routing on a 1gbit/s synchronous fiber connection and I can do line-speed wireguard just fine. Distributions like OpenWRT require so little RAM because they select very conservative options when compiling everything, are very selective about what they include, build against minimal C libraries like musl libc, make use of highly integrated system utilities like busybox, and strip things like debugging symbols from the resulting binaries.
wmf 2y ago
Intel definitely has cache injection for DMA; I'm not sure about AMD. With DPDK you should be able to process packets completely in the L3.
foota 2y ago
This is theoretically true, but I think you'd have to get a lot of support from AMD and be responsible for implementing most of a traditional computer yourself. I have a feeling it's be roughly as feasible to do it all with fully custom hardware. Otoh if you just mean try to write software to minimize the need to hit memory, that's totally reasonable -- and what you have to do if you want the best performance.
tremon 2y ago
But in that case, wouldn't it be more efficient to put a processing core on/near the NIC, and do simple responses without even bothering the main CPU? I mean, if your "process some stuff" doesn't need data from RAM, it doesn't need to involve the main CPU either.
- crote 2y ago
  Pretty much, yeah! But the only NICs with compute that I'm aware of use quite expensive FPGAs. Being able to use off-the-shelf hardware would be far cheaper, far easier to develop for, and far more flexible.

by crote 2y ago

yusyusyus 2y ago

> With the right preprocessing, it should be possible to essentially stream from NIC straight to CPU, process some stuff, and output it straight to the NIC again without ever touching RAM this already exists [0] [0] https://www.intel.com/content/dam/www/public/us/en/documents/technology-briefs/data-direct-i-o-technology-brief.pdf https://www.intel.com/content/dam/www/public/us/en/documents...

crote 2y ago
I wasn't aware of that, thanks for sharing!

timschmidt 2y ago

Lots of older or low end wireless routers have only 16 or 32mb RAM. A full linux stack fits in that. I do a full HTTPS stack on ESP32-C3 in ~500k. Lots of limitations, of course. Probably the biggest difficulty is that most modern systems simply refuse to boot without RAM installed, so you can't choose any platform without BIOS / EFI modifications. Definitely a worthwhile idea though.

crote 2y ago
The difference is that those low end routers offload most or even all of the packet processing to dedicated networking hardware: it never even touches the CPU. The whole Linux stack is only used to host the management interface, configure the switching hardware, and deal with high-level stuff like DNS and DHCP. That's also why VPN performance is so horribly bad: there's no hardware acceleration for encryption, so it needs to be done on the CPU, which is not sized to deal with that kinds of traffic.
- timschmidt 2y ago
  Offloading the packet processing doesn't really save any RAM at all. The limitations you're discussing are related to running on a low power ~700mhz MIPS or ARM core with very low performance. As you said, using a VPN or adding additional routing rules requires the packets to pass through the CPU, which demonstrates that all the software and buffers for doing the routing on the CPU are present within the 16 or 32Mb ram. It's just attached to a very low performance CPU core. It's perfectly possible to load the same OpenWRT release on a modern x86 and run in the same amount of ram with much higher performance. That's how I do my routing on a 1gbit/s synchronous fiber connection and I can do line-speed wireguard just fine. Distributions like OpenWRT require so little RAM because they select very conservative options when compiling everything, are very selective about what they include, build against minimal C libraries like musl libc, make use of highly integrated system utilities like busybox, and strip things like debugging symbols from the resulting binaries.

wmf 2y ago

Intel definitely has cache injection for DMA; I'm not sure about AMD. With DPDK you should be able to process packets completely in the L3.

foota 2y ago

This is theoretically true, but I think you'd have to get a lot of support from AMD and be responsible for implementing most of a traditional computer yourself. I have a feeling it's be roughly as feasible to do it all with fully custom hardware. Otoh if you just mean try to write software to minimize the need to hit memory, that's totally reasonable -- and what you have to do if you want the best performance.

tremon 2y ago

But in that case, wouldn't it be more efficient to put a processing core on/near the NIC, and do simple responses without even bothering the main CPU? I mean, if your "process some stuff" doesn't need data from RAM, it doesn't need to involve the main CPU either.

crote 2y ago
Pretty much, yeah! But the only NICs with compute that I'm aware of use quite expensive FPGAs. Being able to use off-the-shelf hardware would be far cheaper, far easier to develop for, and far more flexible.