5 ms·

I didn't know there was a deep learning gold rush. Maybe this explains the crazy number of stars on my libdeep library on Github, while there being no comments

by motters 12y ago

I didn't know there was a deep learning gold rush. Maybe this explains the crazy number of stars on my libdeep library on Github, while there being no comments or issues raised. Deep learning is not any sort of magic bullet. It may be marginally better than other machine learning methods in specific contexts, but I'm not convinced that there are going to be any deep learning tycoons or deep learning entrepreneurs (were there any SVM tycoons?). But I suppose as a buzz term "deep learning" is better than the meaningless "big data". Just replace the latter with the former in the marketing literature.

bluishgreen 12y ago
About libdeep: Would be very useful if you had the man page inserted into the README. This way I can make a decision as to if I need this library without having to download/install.
- motters 12y ago
  Yes. There is lots of room for improvement and more tests. libdeep isn't something which I've been actively developing recently.
robert_tweed 12y ago
I've been noticing a slight trend among the same group of people that are likely to use the big data buzzword without really understanding it, to also use deep learning in the same way, to simply mean "gain business insights from [obviously big] data". As far as I know the only gold rush is around marketing surrounding this and related buzzwords, i.e., it's the latest thing that your business absolutely must be doing to keep up with your competitors. That particular usage of course has as little to do with actual deep learning as the misappropriation of big data has to do with anything. Fortunately this trend has been a bit slower to take off, presumably because whereas big data is a fairly nebulous concept, it's much easier to correct someone when they talk about deep learning quite wrongly.
- agibsonccc 12y ago
  That's the exact market I'm cashing in on. But rather than just be someone in the space who talks about it, I made it[1]. I think at the end of the day, the stuff talked about in the media may or may not have some merit (otherwise why would google or these other companies put resources in to it?) Rather than read the blogspam, read the papers instead though. Try to understand the merits of what's going on and apply it to your use case. [1]: http://deeplearning4j.org/ http://deeplearning4j.org/
  - beaucronin 12y ago
    Blogspam? I demand satisfaction!
    - agibsonccc 12y ago
      Let me clarify: Techcrunch articles talking about papers being published. Blogspam to me is just something that talks about the stuff at such a high level, there's no meat in it. Within that subset I'm talking about academic concepts where you can learn a lot more if you just read the papers.
      - agibsonccc 12y ago
        Before I get bombarded here, I think it's best to clarify that the word blogspam is a term thrown around a lot around here. When the press talks about an academic paper, they tend not to add much value in terms of actually explaining what's going on. I think people drawing their own conclusions from the original works is the better way to go. Obviously not everyone will agree with me here, but so be it.
      - deleted 12y ago
        [deleted]
joelthelion 12y ago
The fact that features can be learned means we could see the emergence of machine learning methods that could be used by anyone with a technical background, not just ML experts. That would be a big deal IMHO.
- motters 12y ago
  That has already been the case for a long time. There have been many neural net libraries which don't require the developer to understand the details of the algorithms.
mtrimpe 12y ago
After watching 'Recent Developments in Deep Learning' by Geoff Hinton [1] a while ago I did come away believing that there does seem to be quite a bit of untapped potential in deep neural nets. Do you feel that talk is overly optimistic/misleading? Or just that you don't think there's that much money to be made with it? [1] https://www.youtube.com/watch?v=vShMxxqtDDs https://www.youtube.com/watch?v=vShMxxqtDDs
- motters 12y ago
  I've also seen that talk. In many cases I think the advantages which may be had from deep learning are likely to be marginal - a few percent improvement over the next best algorithm. Multi-layer neural nets already existed for a couple of decades, and deep learning is really just a refinement on that, making the training process faster and more robust to overfitting.
  - bluishgreen 12y ago
    Do you know of any other system that gets ~14% error rate on the IMAGENET dataset? The next best system as recent as 2012 was doing ~26% error. That is a pretty big gap, especially considering IMAGENET is a million image data set. (btw did not see the talk)
    - visarga 12y ago
      > especially considering IMAGENET is a million image data set ... and that human error rate is also > 0. Sometimes machine learning bests the humans.
convexing 12y ago
Broadly speaking, the people who have significant, nontrivial expertise in the field is limited (to a first approximation) to - students of Hinton/LeCun/Bengio/Ng, and/or - members of Google/Facebook/Baidu's deep learning group. An actual deep learning expert can have unrivalled access to infrastructure, excellent colleagues, data, and compensation, at any of Google/Facebook/Baidu etc. Given this backdrop, it's less surprising that a significant number of the current crop of deep learning companies are unimpressive technologically - those with actual expertise have much better opportunities.
dj-wonk 12y ago
After seeing the term "gold rush" get thrown around, I compared a 19th century gold rush to a (supposed) deep learning / data-based gold rush in terms of economic terms (e.g. rivalry and excludability): http://djwonk.tumblr.com/post/87513512919/economics-of-a-supposed-deep-learning-gold-rush http://djwonk.tumblr.com/post/87513512919/economics-of-a-sup... UPDATE: My commentary may have conflated "data analytics" with "deep learning". I probably should disentangle them.
Houshalter 12y ago
It's not marginally better, it's significantly better in some areas. There were massive improvements on some benchmarks like speech recognition and image classification. It's not another machine learning algorithm, it's a feature learning algorithm. You can feed raw pixels into an SVM, but you aren't taking advantage of the structure of the data at all.

by motters 12y ago

bluishgreen 12y ago

About libdeep: Would be very useful if you had the man page inserted into the README. This way I can make a decision as to if I need this library without having to download/install.

motters 12y ago
Yes. There is lots of room for improvement and more tests. libdeep isn't something which I've been actively developing recently.

robert_tweed 12y ago

I've been noticing a slight trend among the same group of people that are likely to use the big data buzzword without really understanding it, to also use deep learning in the same way, to simply mean "gain business insights from [obviously big] data". As far as I know the only gold rush is around marketing surrounding this and related buzzwords, i.e., it's the latest thing that your business absolutely must be doing to keep up with your competitors. That particular usage of course has as little to do with actual deep learning as the misappropriation of big data has to do with anything. Fortunately this trend has been a bit slower to take off, presumably because whereas big data is a fairly nebulous concept, it's much easier to correct someone when they talk about deep learning quite wrongly.

agibsonccc 12y ago
That's the exact market I'm cashing in on. But rather than just be someone in the space who talks about it, I made it[1]. I think at the end of the day, the stuff talked about in the media may or may not have some merit (otherwise why would google or these other companies put resources in to it?) Rather than read the blogspam, read the papers instead though. Try to understand the merits of what's going on and apply it to your use case. [1]: http://deeplearning4j.org/ http://deeplearning4j.org/
- beaucronin 12y ago
  Blogspam? I demand satisfaction!
  - agibsonccc 12y ago
    Let me clarify: Techcrunch articles talking about papers being published. Blogspam to me is just something that talks about the stuff at such a high level, there's no meat in it. Within that subset I'm talking about academic concepts where you can learn a lot more if you just read the papers.
    - agibsonccc 12y ago
      Before I get bombarded here, I think it's best to clarify that the word blogspam is a term thrown around a lot around here. When the press talks about an academic paper, they tend not to add much value in terms of actually explaining what's going on. I think people drawing their own conclusions from the original works is the better way to go. Obviously not everyone will agree with me here, but so be it.
    - deleted 12y ago
      [deleted]

joelthelion 12y ago

The fact that features can be learned means we could see the emergence of machine learning methods that could be used by anyone with a technical background, not just ML experts. That would be a big deal IMHO.

motters 12y ago
That has already been the case for a long time. There have been many neural net libraries which don't require the developer to understand the details of the algorithms.

mtrimpe 12y ago

After watching 'Recent Developments in Deep Learning' by Geoff Hinton [1] a while ago I did come away believing that there does seem to be quite a bit of untapped potential in deep neural nets. Do you feel that talk is overly optimistic/misleading? Or just that you don't think there's that much money to be made with it? [1] https://www.youtube.com/watch?v=vShMxxqtDDs https://www.youtube.com/watch?v=vShMxxqtDDs

motters 12y ago
I've also seen that talk. In many cases I think the advantages which may be had from deep learning are likely to be marginal - a few percent improvement over the next best algorithm. Multi-layer neural nets already existed for a couple of decades, and deep learning is really just a refinement on that, making the training process faster and more robust to overfitting.
- bluishgreen 12y ago
  Do you know of any other system that gets ~14% error rate on the IMAGENET dataset? The next best system as recent as 2012 was doing ~26% error. That is a pretty big gap, especially considering IMAGENET is a million image data set. (btw did not see the talk)
  - visarga 12y ago
    > especially considering IMAGENET is a million image data set ... and that human error rate is also > 0. Sometimes machine learning bests the humans.

convexing 12y ago

Broadly speaking, the people who have significant, nontrivial expertise in the field is limited (to a first approximation) to - students of Hinton/LeCun/Bengio/Ng, and/or - members of Google/Facebook/Baidu's deep learning group. An actual deep learning expert can have unrivalled access to infrastructure, excellent colleagues, data, and compensation, at any of Google/Facebook/Baidu etc. Given this backdrop, it's less surprising that a significant number of the current crop of deep learning companies are unimpressive technologically - those with actual expertise have much better opportunities.

dj-wonk 12y ago

After seeing the term "gold rush" get thrown around, I compared a 19th century gold rush to a (supposed) deep learning / data-based gold rush in terms of economic terms (e.g. rivalry and excludability): http://djwonk.tumblr.com/post/87513512919/economics-of-a-supposed-deep-learning-gold-rush http://djwonk.tumblr.com/post/87513512919/economics-of-a-sup... UPDATE: My commentary may have conflated "data analytics" with "deep learning". I probably should disentangle them.

Houshalter 12y ago

It's not marginally better, it's significantly better in some areas. There were massive improvements on some benchmarks like speech recognition and image classification. It's not another machine learning algorithm, it's a feature learning algorithm. You can feed raw pixels into an SVM, but you aren't taking advantage of the structure of the data at all.