5 ms·

Scoping this to users, it wouldn't change much at all. For data transfers you would also need storage, memory and CPUs that can handle it. For streaming it also

Scoping this to users, it wouldn't change much at all. For data transfers you would also need storage, memory and CPUs that can handle it. For streaming it also doesn't change much since even a 4K HDR stream would work on a legacy VDSL2 system. Same goes for remote compute. For non-users (datacenters, companies, cluster systems etc) that might mostly help with distributed storage, but again, you'd also need all the other things, because distributing anything like that also means every piece of the puzzle needs to be able to handle it. Within a datacenter or a public cloud, even top speeds of 400Gb/s (which is less than half of the thought experiment) is at such a high tier that it isn't useful for individual applications or nodes. Something that would actually make an impact after you get to around 2Gbps would be lower latency, lower jitter, net neutrality (including getting a full connection with no administrative limits), and more peered routes. More bandwidth doesn't really do much, especially if you can't use that bandwidth anyway if the amount of connections, the latency and the computing equipment doesn't scale with it. When you have low enough latency combined with high enough bandwidth you can start to actually get new apps and also develop novel use cases (imagine a CPU-to-CPU interconnect that can span continents). But the speed of light (or some more exact latency-per-distance quantity) prevents that. Beyond the likes of BitTorrent and Gnutella we're not likely to see network-based ideas that are currently impossible due to limits on the average speed. Perhaps the real problem right now is the lack of universal availability of reasonable connectivity.

fouronnes3 2y ago
If you think of the distribution of network speed across the population, improving the lowest quantiles has the most effect.
ndriscoll 2y ago
I think there would be diminishing returns after a couple dozen Gb/s, but my almost 10 year old computer can easily handle over 10k web requests/second (including database queries that do several joins), or ~40k serving cached pages from nginx. Phoronix benchmarked the 9950X at over 200k requests/second with nginx, so I'd expect you could host something like reddit (sans video) at home with sufficient uplink if you were willing to take the legal risk and you had a new computer or two.
- avidiax 2y ago
  There are diminishing returns after 1GBps. I previously had 10Gbps symmetrical fiber[1], and it was simply impossible to saturate without running a speed test against another customer. Servers were generally not fast enough to make use of that speed. XBOX downloads would sometimes peak near 1Gbps, but not sustain. The main issue, I suppose, is that disk drives are not fast enough, or at least, not fast enough relative to their size. You can have 500MBps (40Gbps) hard drives, but your cloud provider has 50-100 customers at least accessing that drive, so your share of the straw is fairly small. More pedestrian uses can't possibly benefit from such bandwidth either. 4k Blu-rays max out at 128Mbps. I suppose we could have 3D 8k120 streams taking 4Gbps (128*4*4*2). Maybe you can just go uncompressed, so 382Gbps (2*7680*4320*120*48), but that seems like it probably causes more trouble than compressing/decompressing, since it will be rather hard to buffer, and small hiccups will lose huge amounts of data. In short, I think it wouldn't be substantially different than having a good 1Gbps symmetrical internet. It might allow Stadia like experiences to be really good, but those still have latency issues. [1] https://www.init7.net/en/internet/fiber7/ https://www.init7.net/en/internet/fiber7/
  - ndriscoll 2y ago
    You shouldn't actually need that much disk bandwidth to run something like a reddit though. 192 GB of RAM (which gaming motherboards can support) costs under $1k and should be enough to keep several weeks worth of threads warm. On the other hand, a large thread might need to transfer ~100 kB, so at 10k requests/second, that's ~8 Gb/s. I can't find stats on how many page views/second they get, but I imagine it's more than 10k/s and less than 100k/s, so presumably somewhere between 10Gb/s and 80Gb/s you have more than you need for that use case.
aftbit 2y ago
Latency and peering are huge. I'm amazed at how poor internet latency constantly is. Assuming fiber optics have a speed of light around 0.6c, one would expect that Cincinnati to Chicago (~450 km) is around 2.5 light-milliseconds one way, but actual pings take me at least 10ms, usually more like 20. Worse, my home ISP has pretty bad peerings, so for example, traffic from my home to a data center about thirty minutes drive away (well under 50km as crow flies) ends up going via Ashburn and Washington, which results in RTT in excess of 30ms. There is so much more room for improvement in latency. Even pinging my friend's house (under 10km away) on the SAME ISP (with a total of 4 traceroute hops, including start and end) takes 3 to 4ms. At 0.6c, even assuming we're both wired to the core of downtown, only 0.4ms of our RTT is actually spent on speed of light. The other ~3ms is spent on packet switching and protocol conversions. Admittedly, all of these latencies are sufficient for most things that I want to do on the internet, and most users don't start seeing issues for most tasks until they're up in the 100ms+ range. That said, I think there's still more to gain here than from straight up throughput improvements.
- oneplane 2y ago
  The latency is also where the bottleneck of really novel mass-market network applications sits, most of the time. Just looking at examples like: https://gist.github.com/jboner/2841832 https://gist.github.com/jboner/2841832 for a network to be usable as a somewhat fast SSD, you'd need a maximum latency of 0.15ms. And at that point all we have gained is a network that can be somewhat used like a 10 year old SSD. To then get to a really relevant latency we have to get to memory reference speeds, 100ns at most. That's 0.0001ms or something like that (might be off by one order of magnitude). But with a 0.6c limit, we're not going to get there. Like the other comment, having everyone get a much better minimum speed (say 500Mbps) and low base latency (say 1ms at most to the first IX you can get to) would make a huge difference. But to reference yet another comment, you'd probably just get 4K video ads and larger frameworks with less optimisation since the bandwidth takes care of it anyway.
- h_tbob 2y ago
  Haha this made me laugh. If our ancestors could see us commenting… “it took 4 milliseconds… should be faster”. So funny, I don’t understand how you even notice the difference win 4ms vs .4!

fouronnes3 2y ago

If you think of the distribution of network speed across the population, improving the lowest quantiles has the most effect.

ndriscoll 2y ago

I think there would be diminishing returns after a couple dozen Gb/s, but my almost 10 year old computer can easily handle over 10k web requests/second (including database queries that do several joins), or ~40k serving cached pages from nginx. Phoronix benchmarked the 9950X at over 200k requests/second with nginx, so I'd expect you could host something like reddit (sans video) at home with sufficient uplink if you were willing to take the legal risk and you had a new computer or two.

avidiax 2y ago
There are diminishing returns after 1GBps. I previously had 10Gbps symmetrical fiber[1], and it was simply impossible to saturate without running a speed test against another customer. Servers were generally not fast enough to make use of that speed. XBOX downloads would sometimes peak near 1Gbps, but not sustain. The main issue, I suppose, is that disk drives are not fast enough, or at least, not fast enough relative to their size. You can have 500MBps (40Gbps) hard drives, but your cloud provider has 50-100 customers at least accessing that drive, so your share of the straw is fairly small. More pedestrian uses can't possibly benefit from such bandwidth either. 4k Blu-rays max out at 128Mbps. I suppose we could have 3D 8k120 streams taking 4Gbps (128*4*4*2). Maybe you can just go uncompressed, so 382Gbps (2*7680*4320*120*48), but that seems like it probably causes more trouble than compressing/decompressing, since it will be rather hard to buffer, and small hiccups will lose huge amounts of data. In short, I think it wouldn't be substantially different than having a good 1Gbps symmetrical internet. It might allow Stadia like experiences to be really good, but those still have latency issues. [1] https://www.init7.net/en/internet/fiber7/ https://www.init7.net/en/internet/fiber7/
- ndriscoll 2y ago
  You shouldn't actually need that much disk bandwidth to run something like a reddit though. 192 GB of RAM (which gaming motherboards can support) costs under $1k and should be enough to keep several weeks worth of threads warm. On the other hand, a large thread might need to transfer ~100 kB, so at 10k requests/second, that's ~8 Gb/s. I can't find stats on how many page views/second they get, but I imagine it's more than 10k/s and less than 100k/s, so presumably somewhere between 10Gb/s and 80Gb/s you have more than you need for that use case.

aftbit 2y ago

Latency and peering are huge. I'm amazed at how poor internet latency constantly is. Assuming fiber optics have a speed of light around 0.6c, one would expect that Cincinnati to Chicago (~450 km) is around 2.5 light-milliseconds one way, but actual pings take me at least 10ms, usually more like 20. Worse, my home ISP has pretty bad peerings, so for example, traffic from my home to a data center about thirty minutes drive away (well under 50km as crow flies) ends up going via Ashburn and Washington, which results in RTT in excess of 30ms. There is so much more room for improvement in latency. Even pinging my friend's house (under 10km away) on the SAME ISP (with a total of 4 traceroute hops, including start and end) takes 3 to 4ms. At 0.6c, even assuming we're both wired to the core of downtown, only 0.4ms of our RTT is actually spent on speed of light. The other ~3ms is spent on packet switching and protocol conversions. Admittedly, all of these latencies are sufficient for most things that I want to do on the internet, and most users don't start seeing issues for most tasks until they're up in the 100ms+ range. That said, I think there's still more to gain here than from straight up throughput improvements.

oneplane 2y ago
The latency is also where the bottleneck of really novel mass-market network applications sits, most of the time. Just looking at examples like: https://gist.github.com/jboner/2841832 https://gist.github.com/jboner/2841832 for a network to be usable as a somewhat fast SSD, you'd need a maximum latency of 0.15ms. And at that point all we have gained is a network that can be somewhat used like a 10 year old SSD. To then get to a really relevant latency we have to get to memory reference speeds, 100ns at most. That's 0.0001ms or something like that (might be off by one order of magnitude). But with a 0.6c limit, we're not going to get there. Like the other comment, having everyone get a much better minimum speed (say 500Mbps) and low base latency (say 1ms at most to the first IX you can get to) would make a huge difference. But to reference yet another comment, you'd probably just get 4K video ads and larger frameworks with less optimisation since the bandwidth takes care of it anyway.
h_tbob 2y ago
Haha this made me laugh. If our ancestors could see us commenting… “it took 4 milliseconds… should be faster”. So funny, I don’t understand how you even notice the difference win 4ms vs .4!