3 ms·
One point that I've not seen mentioned yet, is that the neural revolution somewhat aggravates economic inequality. What recent progress basically has shown is t
by hacker42 10y ago
One point that I've not seen mentioned yet, is that the neural revolution somewhat aggravates economic inequality. What recent progress basically has shown is that deep learning works better with more layers and more resources. Geoffrey Hinton has also recently conjectured that there exist a pretty much unexplored regime of applying gradient descent to huge models with strong regularization trained on relatively small (but still big) data. This inequality is alleviated to some extent by the fact that the machine learning community fully embraces online education and open science, but still, you need 50-150 GPUs to play human-level Go and having several grad students that explore a wide variety of complex and huge models is key for progress. I can only see this aspect getting worse in the years to come.