42 ms·
The story your telling was the mainstream story from when "deep learning" first arose, using Boltzmann machines for pretraining, etc. But then the field explod
by jeremysalwen 4y ago
The story your telling was the mainstream story from when "deep learning" first arose, using Boltzmann machines for pretraining, etc. But then the field exploded, more compute power was directed at the problem, and people discovered that yes, in fact basic MLPs with SGD work well with enough compute.
- sdenton4 4y agoBut the point is that there were super important algorithm advanced that allowed training bigger models. It wasn't all hardware... A big part of it was getting more eyes on the problems (via hype over imagenet results circa 2012) to figure out how to solve them.