2 ms·
The key insight for both evolution and contemporary AI is that hill-climbing (either completely random a la evolution, or guided locally like in back-propagatio
by eru 2y ago
The key insight for both evolution and contemporary AI is that hill-climbing (either completely random a la evolution, or guided locally like in back-propagation) can work really well, if you have enough dimensions to play with.
Hill climbing obviously gets stuck easily in 2 or 3 dimensions. So our intuition ain't reliable.
- carlmr 2y ago>Hill climbing obviously gets stuck easily in 2 or 3 dimensions. So our intuition ain't reliable. Also our intuition about overfitting from lower-dimensional representations seems to be less of an issue at high dimensions.