> But can a 10-node-iphone-cluster really match an A100 for pure flops?! Only if the data is local, small, easily synchronizable, doesn't require fp32 precisio

> But can a 10-node-iphone-cluster really match an A100 for pure flops?! Only if the data is local, small, easily synchronizable, doesn't require fp32 precision and a dozen other caveats. Can a 10 iphone cluster match an A100 in calculating digits of pi? Yes. Can it do backpropagation in large machine learning models? The gradients won't even fit.