3 ms·
One year effect? What exactly are they speaking about? The only reason deep learning exists is because by now we've finally learned how to build GPUs fast enou
by heavenlyblue 8y ago
One year effect? What exactly are they speaking about?
The only reason deep learning exists is because by now we've finally learned how to build GPUs fast enough to run the algorithm invented in 1983.
And let's be honest - most of the current, state-of-the-art algorithms only work today because they've got access to scaled up massive databases of data. You don't really need to be as smart here any more.
- yetihehe 8y ago> One year effect? What exactly are they speaking about? From article: > What may take a cluster to compute one year takes a consumer machine the next. So, one year after introducing something you don't need cluster for computing, consumer machine will be sufficient. From first comment: > Is that not partly because the hardware is ever improving? [...] Certainly improving the model will help, who hasn't marvelled at how better his code ran when he fixed that On^2 hot spot. But I can't help but think improving hardware plays a role too. > Am I off base here? Now my comment was about original commenter being off base. Hardware improvement plays a role, but not during one year. 1000x speedup in a year is due to algorithms.
- twtw 8y agoI don't mean to reduce the work of whichever researchers published in 1983 (are you thinking of Rumelhart et al 1986?), but gradient descent and reverse mode automatic differentiation are way older than 1983. Adaptive filters were being "trained" in the 50s using stochastic gradient descent.