3 ms·

I love looking at the results and seeing the annual improvement and breakout strategies. Deep learning has really come into its own during the last eight years.

by jbarrow 12y ago

I love looking at the results and seeing the annual improvement and breakout strategies. Deep learning has really come into its own during the last eight years. People interested in learning about some of the techniques employed (CNNs and deep networks) are encouraged to take Geoff Hinton's Coursers course [1], which does a really good job of covering modern techniques from one of their pioneers. [1] https://www.coursera.org/course/neuralnets https://www.coursera.org/course/neuralnets

rjtavares 12y ago
How much are results improving? What's the general long term trend?
- dave_sullivan 12y ago
  The trend is depreciating returns with current architectures (after truly breakthrough results in the first place). It's a good time for companies to update their ML pipelines to use deep learning. Performance does seem to scale with hardware though, so as hardware improves (as well as the software), these models will see some more gains without any real technology change. But I should also point out deep learning doesn't work for everything. Random forests or SVMs work about as well on certain (I would argue "more linear") datasets. Deep learning is for when RFs and SVMs aren't giving the results you need (or at all). It doesn't negate all things that came before, nor should it. One thing to consider: As far as I can tell, there's an order of magnitude more people working on these types of problems with an order of magnitude more money being thrown at it and an order of magnitude increase in "collaborative efficiency" due to Internet and open source/research. I think breakthroughs are thought of best as a probability distribution, but I think the probability of further breakthrough successes has increased considerably in just the last few years. There are reasons for optimism and pessimism, as usual.
- m_ke 12y ago
  15% in 2012, 11% last year and 6% this year. We might also see late submissions beat the this years top result in the next few months.
theschwa 12y ago
Is there any place to view Hinton's lectures? I've been waiting for that Coursera course to open again since I barely missed out on the first one, but I haven't seen it run since.
- Houshalter 12y ago
  All of the lectures are there, you just need to join the course to see them.

by jbarrow 12y ago

rjtavares 12y ago

How much are results improving? What's the general long term trend?

dave_sullivan 12y ago
The trend is depreciating returns with current architectures (after truly breakthrough results in the first place). It's a good time for companies to update their ML pipelines to use deep learning. Performance does seem to scale with hardware though, so as hardware improves (as well as the software), these models will see some more gains without any real technology change. But I should also point out deep learning doesn't work for everything. Random forests or SVMs work about as well on certain (I would argue "more linear") datasets. Deep learning is for when RFs and SVMs aren't giving the results you need (or at all). It doesn't negate all things that came before, nor should it. One thing to consider: As far as I can tell, there's an order of magnitude more people working on these types of problems with an order of magnitude more money being thrown at it and an order of magnitude increase in "collaborative efficiency" due to Internet and open source/research. I think breakthroughs are thought of best as a probability distribution, but I think the probability of further breakthrough successes has increased considerably in just the last few years. There are reasons for optimism and pessimism, as usual.
m_ke 12y ago
15% in 2012, 11% last year and 6% this year. We might also see late submissions beat the this years top result in the next few months.

theschwa 12y ago

Is there any place to view Hinton's lectures? I've been waiting for that Coursera course to open again since I barely missed out on the first one, but I haven't seen it run since.

Houshalter 12y ago
All of the lectures are there, you just need to join the course to see them.