3 ms·
The RSS is Linux is not particularly useful (for netflix scale workloads) because it does not integrate the RSS hashing across the entire stack. So all you ge
by drewg123 10y ago
The RSS is Linux is not particularly useful (for netflix scale workloads) because it does not integrate the RSS hashing across the entire stack. So all you get is connection sharding. With real RSS, as done in Windows and FreeBSD where the kernel has intimate knowledge of the hash key and algorithm, you can use RSS to split the TCP hash table up and make it per-CPU. By using multiple accept sockets for per-CPU workers, you can effectively keep everything for a single connection on a CPU, and run almost everything with no cross-CPU contention. You can't move the connection around at will between CPUs, but you don't care to, because no connection is special (in a netflix workload), it just one of tens of thousands.
Adrian Chadd did most of the FreeBSD RSS work, and gave a good talk about it at BAFUG: https://www.youtube.com/watch?v=7CvIztTz-RQ https://www.youtube.com/watch?v=7CvIztTz-RQ
The RSS in Linux was just used for load spreading (the last I checked, I haven't used Linux much since I left Google 1.5 years ago). If this has improved, I'd love to hear about it.
Linux RFS depends on the packets being dispatched to the correct CPU for the connection by the interrupt handler running wherever the packet happened to land. This has cache & memory locality implications, especially on NUMA.
Linux aRFS lets the NIC do the steering. Unfortunately, each connection requires an interaction with the NIC to poke it into the steering table, and most NICs can't steer 100,000 connections.
So, to sum up, Linux has a lot of cool tech for steering individual connections and support for that varies greatly by NIC. Windows and FreeBSD use standard RSS to predictably steer an unlimited number of connections. For a large CDN server, the latter is more useful. However, for low-latency / high bandwidth applications, I can see the advantage to aRFS.