2 ms·
Given that server processors have up to 64 cores now, it sounds like the NIC (even 200 Gbps) is squarely the bottleneck.
by wmf 6y ago
Given that server processors have up to 64 cores now, it sounds like the NIC (even 200 Gbps) is squarely the bottleneck.
- jiggawatts 6y agoNot for any single user or connection. High capacity doesn't equal low latency, and scalability to many users doesn't necessarily help one user get better performance! Very often, you'll see n-tier applications where for some reason (typically load-balancers), the requests are muxed into a single TCP stream. In the past, this improved efficiency by eliminating the per-connection overhead. Now, with high core counts and high bandwidths, some parallelism is absolutely required to even begin to approach the performance ceiling. If the application is naturally single-threaded, such as some ETL jobs, these single-stream bottlenecks are very difficult to overcome. In the field, I very often see 8-core VMs with a "suspicious" utilisation graph flatlining at 12.5% because of issues like this. It boils my blood when people say that this is perfectly fine, because clearly the server has "adequate capacity". In reality, there's a problem, and the server is at 100% of its capacity an the other 7 cores are just heating the data centre air.