Neural nets only need linearly more data for optimal performance: https://www.lesswrong.com/posts/midXmMb2Xg37F2Kgn/new-scaling-laws-for-large-language-models h

Neural nets only need linearly more data for optimal performance: https://www.lesswrong.com/posts/midXmMb2Xg37F2Kgn/new-scaling-laws-for-large-language-models https://www.lesswrong.com/posts/midXmMb2Xg37F2Kgn/new-scalin... At the same time, fine-tuning sample efficiency increases with scale, so at some point you can possibly one-shot learn state and get rid of exponential searches, solving NP-Hard problems with heuristics. Sounds like a free lunch to me. At least if you can afford a net large enough.