4 ms·
They don't need the generality of full TCP for their cluster, so they're using a tweaked, incompatible subset. One that's been optimized for better performance
by lallysingh 2y ago
They don't need the generality of full TCP for their cluster, so they're using a tweaked, incompatible subset. One that's been optimized for better performance on cheaper hardware than you can get with TCP h/w offload. In the offload case you're still paying the latency, wire protocol overhead, and efficiency costs of full TCP.
(Disclaimer: I work at Tesla, not related to this group, opinions on public info only)
- throw0101d 2y ago> One that's been optimized for better performance on cheaper hardware than you can get with TCP h/w offload. How many ≥10 Gbps chipsets that you'd find in a typical server do not have offload nowadays? Further, once you're in the ≥50 Gbps card range you can often get ROCE, which helps with things like latency.
- lallysingh 2y agoAnd you're still paying the other performance and efficiency costs of TCP. ROCE also isn't a magic bullet. Every system has a cost and tradeoffs. Just because someone took an unusual path doesn't mean that they were wrong. And the larger and more specialized their use case is, the less likely that a generic solution is the best match.
- chipdart 2y ago> And you're still paying the other performance and efficiency costs of TCP. ROCE also isn't a magic bullet. Tesla's own charts show ROCE achieving also one-way write latencies in the single-digit microsecond range. If that doesn't qualify as magic bullet, what does that say about TTPoE?