Yes, you are. Most of hard problems in computer science are that of highly-branched connectivity (both in terms of the number of connections that are mode from

Yes, you are. Most of hard problems in computer science are that of highly-branched connectivity (both in terms of the number of connections that are mode from the compute node and in terms of the branching required in the algorithms). E.G. 1 - branched) if it takes you 100ms to send 1Gb of data over network, but you don't have to wait for it's result, then latency does not matter at all. E.G. 2 - number of connections to the node) if you have a matrix value that depends on a full row in a matrix, this matrix value somehow needs to be computationally connected to a full row in a matrix. Cluster or GPU, these physical connections need to be made either through Ethernet or by reading values from RAM. Again, the latency does not matter as long as you can pipeline these. So frankly, no. It's not the micrometers on the silicon that matters, it's just that our hardware had been scaled so much that a 50 year old compute cluster can be literally scaled down to a single die. It's _not_ more efficient, though.