4 ms·
> there is no reason to think that there is fundamental limit in how many processor you could connect Square-cube law. As the number of processors grow how do
by blamestross 5y ago
> there is no reason to think that there is fundamental limit in how many processor you could connect
Square-cube law. As the number of processors grow how do you hook them together? If you want to minimize latency you end up with a solid packed sphere. O(cube root) latency. In that situation, heat dissipation only scales with the surface area of that sphere. So there is a limit to the scale. If you choose a topology that permits cooling, then you accept ever increasing inter-processor latency which eventually bottlenecks your computation.
Computers are not magic. They exist in a physical reality and have to deal with thermodynamics and the speed of light. Just because we feel like we have a lot of them doesn't mean we have enough for "magical super AI" to work.