5 ms·

> corporations don't see any "immediate shareholder value", so they sit around happy as pigs in shit with the status quo. This is ridiculous. Hyperscalars see

> corporations don't see any "immediate shareholder value", so they sit around happy as pigs in shit with the status quo. This is ridiculous. Hyperscalars see an immediate ROI from efficiency/reliability improvements and actively invest in TCP alternatives all of the time. It's just really hard. Networking companies see an ability to differentiate their products from their peers and work on this kind of thing as well. I did a 3 second google for "QUIC acceleration Mellanox" and got a hit on Nvidia's blog right away. You just can't trivially replace something with an investment totally 50 years of clock time and thousands of years of engineer time. It will either take a long time or a massive shift in needs/technology. FWIW, I wouldn't be surprised if the high-performance RDMA networks being put together for AI workloads were the thing that grew into the "next" thing.

oconnor663 4y ago
> 50 years of clock time and thousands of years of engineer time It's not just the size of the investment, it's that it's the protocol everyone uses to talk to other people's machines, and you can't upgrade or replace other people's machines.
- kebman 4y ago
  Yes you can. Just offer a better product, and people will buy it instead of the old or bad product. Better yet, make the new product backwards compatible, and fewer people will have qualms about forking out for it. Better yet, do an aggressive takeover, like Microsoft did, and just force the entire industry to adopt your stuff...
  - scantydolt 4y ago
    Great! When do you think you'll have it done?
    - kebman 4y ago
      Done? What do you mean "done"? Consulting hours are much better on projects that cannot ever be finished!1
  - atoav 4y ago
    You mean like IPv6?
    - sneak 4y ago
      I think QUIC/http2 is a much better example. Google made that happen almost unilaterally via their Chrome dominance.
      - withinboredom 4y ago
        I mean, this is how new features come about, for the most part (look at ajax, from Microsoft's IE dominance). The consortium allows anyone to contribute, not just the dominate browser, but the dominate browser will always be able to experiment with new web features without having to discuss it with anyone.
- alexgartrell 4y ago
  In this case we're talking about within the Datacenter, and you could conceivably update every network device and system to talk the new thing if you wanted. This is more achievable at a hyperscalar, where there tends to be < 3 distinct protocols, proxies, etc. TCP gives you three things: 1. Reasonable performance - This is hard but not impossible to replicate 2. Reliability - This is very hard to replicate because networking edge cases are very hard to isolate 3. Fairness - this one is roughly impossible, because the "fairness" is an artifact of the experimentation and tweaking of Congestion Control Algorithms. To elaborate on fairness, dynamic traffic control of all flows within a DC while maintaining high utilization is roughly impossible. You can get really close to this by picking your battles wisely (i.e. solid demand control for data warehouse workloads), but you'll always end up counting on individual flows to react appropriately to loss. They need to back off enough to make room for others without tanking their own throughput. The people who design and implement these algorithms are definitely geniuses, but even they rely on TONS of empirical evidence to narrow parameters to what's appropriate. Of the Kernel Networking people I've worked with, Lawrence Brakmo had the most sophisticated network testing harness I've seen. Even then, you don't really know if it works (and can't finish tuning it) until you run it in production. Running novel congestion control algorithms in production at a sufficient scale to figure out whether or not they're working appropriately is a great way to kill your network, so we end up conducting the equivalent of CCA drug testing to roll it out slowly and safely. The end result of all of this is that it's really hard to solve the "arbitrary connections sharing arbitrary network topologies with high utilization" problem quickly enough for it ever to look like a breakthrough rather than just steady progress. It's also worth noting that it's usually easiest to prove performance, so you'll see a lot of excitement about performance benchmarks from people who don't yet know what they're about to learn about networking. We were very much in this camp at Facebook when we were all-in on memcache-over-udp, and we later abandoned it completely.
  - lamontcg 4y ago
    After having lived through Amazon's early (pre-2003ish) UDP-based networking I got a laugh around 2006-ish or so reading about how facebook was into UDP. I assume there are people who worked there who still have the scars.
samgaw 4y ago
> FWIW, I wouldn't be surprised if the high-performance RDMA networks being put together for AI workloads were the thing that grew into the "next" thing. Maybe we were just early in giving (HFT) customers RDMA back in ~2007[1][2] but I don't see it entering the mainstream anytime soon. And after a relatively short 20 years of adoption, the "next" thing for hyperscalers is not going to be the next thing for everyone else. [1] https://downloads.openfabrics.org/Media/IB_LowLatencyForum_2007/IB_2007_03_Wombat%20Financial%20Software-DoNotUse.pdf https://downloads.openfabrics.org/Media/IB_LowLatencyForum_2... [2] https://www.thetradenews.com/wombat-and-voltaire-break-million-messages-per-second-barrier-with-wombat-data-fabric/ https://www.thetradenews.com/wombat-and-voltaire-break-milli...
- pclmulqdq 4y ago
  HFT networks are also a lot smaller than hyperscaler datacenters, and designed with more cross-sectional bandwidth. A good chunk of the traffic (trading-related messages) also tends to not use congestion control. In large web company datacenters, RDMA and RoCE have had a much "rockier" path forward.

oconnor663 4y ago

> 50 years of clock time and thousands of years of engineer time It's not just the size of the investment, it's that it's the protocol everyone uses to talk to other people's machines, and you can't upgrade or replace other people's machines.

kebman 4y ago
Yes you can. Just offer a better product, and people will buy it instead of the old or bad product. Better yet, make the new product backwards compatible, and fewer people will have qualms about forking out for it. Better yet, do an aggressive takeover, like Microsoft did, and just force the entire industry to adopt your stuff...
- scantydolt 4y ago
  Great! When do you think you'll have it done?
  - kebman 4y ago
    Done? What do you mean "done"? Consulting hours are much better on projects that cannot ever be finished!1
- atoav 4y ago
  You mean like IPv6?
  - sneak 4y ago
    I think QUIC/http2 is a much better example. Google made that happen almost unilaterally via their Chrome dominance.
    - withinboredom 4y ago
      I mean, this is how new features come about, for the most part (look at ajax, from Microsoft's IE dominance). The consortium allows anyone to contribute, not just the dominate browser, but the dominate browser will always be able to experiment with new web features without having to discuss it with anyone.
alexgartrell 4y ago
In this case we're talking about within the Datacenter, and you could conceivably update every network device and system to talk the new thing if you wanted. This is more achievable at a hyperscalar, where there tends to be < 3 distinct protocols, proxies, etc. TCP gives you three things: 1. Reasonable performance - This is hard but not impossible to replicate 2. Reliability - This is very hard to replicate because networking edge cases are very hard to isolate 3. Fairness - this one is roughly impossible, because the "fairness" is an artifact of the experimentation and tweaking of Congestion Control Algorithms. To elaborate on fairness, dynamic traffic control of all flows within a DC while maintaining high utilization is roughly impossible. You can get really close to this by picking your battles wisely (i.e. solid demand control for data warehouse workloads), but you'll always end up counting on individual flows to react appropriately to loss. They need to back off enough to make room for others without tanking their own throughput. The people who design and implement these algorithms are definitely geniuses, but even they rely on TONS of empirical evidence to narrow parameters to what's appropriate. Of the Kernel Networking people I've worked with, Lawrence Brakmo had the most sophisticated network testing harness I've seen. Even then, you don't really know if it works (and can't finish tuning it) until you run it in production. Running novel congestion control algorithms in production at a sufficient scale to figure out whether or not they're working appropriately is a great way to kill your network, so we end up conducting the equivalent of CCA drug testing to roll it out slowly and safely. The end result of all of this is that it's really hard to solve the "arbitrary connections sharing arbitrary network topologies with high utilization" problem quickly enough for it ever to look like a breakthrough rather than just steady progress. It's also worth noting that it's usually easiest to prove performance, so you'll see a lot of excitement about performance benchmarks from people who don't yet know what they're about to learn about networking. We were very much in this camp at Facebook when we were all-in on memcache-over-udp, and we later abandoned it completely.
- lamontcg 4y ago
  After having lived through Amazon's early (pre-2003ish) UDP-based networking I got a laugh around 2006-ish or so reading about how facebook was into UDP. I assume there are people who worked there who still have the scars.

samgaw 4y ago

> FWIW, I wouldn't be surprised if the high-performance RDMA networks being put together for AI workloads were the thing that grew into the "next" thing. Maybe we were just early in giving (HFT) customers RDMA back in ~2007[1][2] but I don't see it entering the mainstream anytime soon. And after a relatively short 20 years of adoption, the "next" thing for hyperscalers is not going to be the next thing for everyone else. [1] https://downloads.openfabrics.org/Media/IB_LowLatencyForum_2007/IB_2007_03_Wombat%20Financial%20Software-DoNotUse.pdf https://downloads.openfabrics.org/Media/IB_LowLatencyForum_2... [2] https://www.thetradenews.com/wombat-and-voltaire-break-million-messages-per-second-barrier-with-wombat-data-fabric/ https://www.thetradenews.com/wombat-and-voltaire-break-milli...

pclmulqdq 4y ago
HFT networks are also a lot smaller than hyperscaler datacenters, and designed with more cross-sectional bandwidth. A good chunk of the traffic (trading-related messages) also tends to not use congestion control. In large web company datacenters, RDMA and RoCE have had a much "rockier" path forward.