2 ms·
Not for any single user or connection. High capacity doesn't equal low latency, and scalability to many users doesn't necessarily help one user get better perfo
by jiggawatts 6y ago
Not for any single user or connection. High capacity doesn't equal low latency, and scalability to many users doesn't necessarily help one user get better performance!
Very often, you'll see n-tier applications where for some reason (typically load-balancers), the requests are muxed into a single TCP stream. In the past, this improved efficiency by eliminating the per-connection overhead.
Now, with high core counts and high bandwidths, some parallelism is absolutely required to even begin to approach the performance ceiling. If the application is naturally single-threaded, such as some ETL jobs, these single-stream bottlenecks are very difficult to overcome.
In the field, I very often see 8-core VMs with a "suspicious" utilisation graph flatlining at 12.5% because of issues like this. It boils my blood when people say that this is perfectly fine, because clearly the server has "adequate capacity". In reality, there's a problem, and the server is at 100% of its capacity an the other 7 cores are just heating the data centre air.