The fundamental problem is that neural nets scale too well. A model trained on a billion dollar GPUs is n times better than a model trained on a $1 million GPU,

The fundamental problem is that neural nets scale too well. A model trained on a billion dollar GPUs is n times better than a model trained on a $1 million GPU, not to speak of consumer hardware. Unless some efficient form of distributed learning appears, open source will not stand a chance, Except for (niche) applications where smaller brains don't matter. Imagine all these crypto computers would've been used to train models!