3 ms·
Here's a good example from 2018: "Deep learning (does not) scale... We can't just scale up AlexNet and get respectively better results - we have to fiddle with
by nl 6y ago
Here's a good example from 2018:
"Deep learning (does not) scale... We can't just scale up AlexNet and get respectively better results - we have to fiddle with specific architectures, and effectively additional compute does not buy much without order of magnitude more data samples, which are in practice only available in simulated game environments."
https://blog.piekniewski.info/2018/05/28/ai-winter-is-well-on-its-way/ https://blog.piekniewski.info/2018/05/28/ai-winter-is-well-o...
But Gary Marcus is the gold standard. Check out his 2018 take-down attempt, which is clearly mostly wrong:
https://arxiv.org/ftp/arxiv/papers/1801/1801.00631.pdf https://arxiv.org/ftp/arxiv/papers/1801/1801.00631.pdf