3 ms·

It wouldn't matter if the links were 10,000 terabits! Because of the way TCP works, it has a bounded speed for small chatty transactions that is determined prim

It wouldn't matter if the links were 10,000 terabits! Because of the way TCP works, it has a bounded speed for small chatty transactions that is determined primarily by the latency, not the throughput. If you look at a network throughput graph from a packet capture, it looks like a sawtooth pattern. This is called slow start, and its a key feature of TCP and all similar protocols. So if a server A wants to talk to a server B, it sends 8 packets, waits for a response, then sends 16 packets, waits, 24 packets, waits, and so on until a response is dropped. It then resets to 8 packets. There are lots of variations on this algorithm, such as using a "cubic" curve instead of a linear curve, but the end result is pretty much the same. Even on an infinite bandwidth link, sending a small blob of JSON -- say 200 kilobytes -- will take pretty much the same time as it would on a 1 Gbps link! As a side effect of this, anything that reduces latency can have a dramatic effect on effective bandwidth. I've seen some applications triple in speed simply because I enabled "Accelerated Networking" in Azure and used a Proximity Placement Group.

10000truths 4y ago
The slow start behavior you describe is not inherent to TCP proper, but rather, a detail of the congestion control algorithm in use by the endpoints' TCP stacks. Most such algorithms will have some kind of AIMD feedback loop to achieve some balance of fairness and efficiency. But for applications where you have control over the endpoints and the network in between them, you can minimize slow start by setting a high initcwnd/initrwnd and using a less aggressive window shrinking mechanism.
dilyevsky 4y ago
No one is forcing anyone to use slowstart. You can also disable things like Nagles to improve latency on small packets. > Even on an infinite bandwidth link, sending a small blob of JSON -- say 200 kilobytes -- will take pretty much the same time as it would on a 1 Gbps link! Technically that’s incorrect- that will take rtt+200kb/rate assuming your window is over 200kb. So depending on how large rtt is bw component may or may not be significant

10000truths 4y ago

The slow start behavior you describe is not inherent to TCP proper, but rather, a detail of the congestion control algorithm in use by the endpoints' TCP stacks. Most such algorithms will have some kind of AIMD feedback loop to achieve some balance of fairness and efficiency. But for applications where you have control over the endpoints and the network in between them, you can minimize slow start by setting a high initcwnd/initrwnd and using a less aggressive window shrinking mechanism.

dilyevsky 4y ago

No one is forcing anyone to use slowstart. You can also disable things like Nagles to improve latency on small packets. > Even on an infinite bandwidth link, sending a small blob of JSON -- say 200 kilobytes -- will take pretty much the same time as it would on a 1 Gbps link! Technically that’s incorrect- that will take rtt+200kb/rate assuming your window is over 200kb. So depending on how large rtt is bw component may or may not be significant