2 ms·
This is amazing work from the Netflix team. I'm looking forward to 1.6 Tb/s in 4 years. It is interesting that this work is happening on FreeBSD, and potentia
by pclmulqdq 4y ago
This is amazing work from the Netflix team. I'm looking forward to 1.6 Tb/s in 4 years.
It is interesting that this work is happening on FreeBSD, and potentially with diverging implementations than Linux. Linux programs seem to be moving towards userspace getting more power, with things like io_uring and increasing use of frameworks like DPDK/SPDK. This work is all about getting userspace out of the way, with things like async sendfile and kernel TLS. That's pretty neat!
- the8472 4y agokTLS has been added to linux too including offload. It also has p2p-dma, so in principle you can shovel the file directly from NVMe to the NIC and have the NIC encrypt it, so it'll never touch the CPU or main memory. But that only works on specific hardware.
- deleted 4y ago[deleted]
- robocat 4y agoMemory is the cache for popular content. You couldn’t serve fast enough directly from NVMe. “~200GB/sec of memory bandwidth is needed to serve 800Gb/s” and “16x Intel Gen4 x4 14TB NVME”. So each NVMe drive would need to serve 12.5GB/s which is more than the 8GB/s limit for PCIe 4.0 x4. Also popular content would need to be on every drive, drastically lowering the total content stored. Also see drewg’s comment on this for a different reason: https://news.ycombinator.com/item?id=32523509 https://news.ycombinator.com/item?id=32523509
- pclmulqdq 4y agoWith HBM2 sapphire rapids chips, I assume you can actually get there. There is probably an insane price premium for them, though, so I wouldn't hold my breath.
- mgerdts 4y agoPCIe Gen 5 drives look poised for wide availability next year and NVIDIA has been demoing CX7 [1] which is also PCIe Gen 5. Intel already has some Gen 5 chips and AMD looks like they will follow soon [2]. Surely there will be other bumps, but I bet they pull it off in way less than 4 years. 1. https://www.servethehome.com/nvidia-connectx-7-shown-at-isc-2022-supermicro/ https://www.servethehome.com/nvidia-connectx-7-shown-at-isc-... 2. https://wccftech.com/amd-epyc-7004-genoa-32-zen-4-core-cpu-sample-spotted-more-cache-4-6-ghz-clocks/ https://wccftech.com/amd-epyc-7004-genoa-32-zen-4-core-cpu-s...