5 ms·

Any bets on when the current deep learning bubble is going to burst? It’s shocking to me how much technical people buy into this, how “this time it’s different

Any bets on when the current deep learning bubble is going to burst? It’s shocking to me how much technical people buy into this, how “this time it’s different” and AI isn’t “over-promising and substantially under-delivering” this time. Really odd to watch it come round again, when the reality is we’re more likely to see some near incremental progresses, partly fueled by more compute and algorithmic advances. Partly by a lot of PR.

brucephillips 9y ago
The big tech companies are demonstrably using deep learning to solve previously unsolvable problems. It's a significant advance. What's yet to be seen is if startups can profit from this advance, since it depends on massive data and compute.
- comstock 9y ago
  AlphaGo is interesting. But what big new problems have been solved? (rather than incrementally improved).
  - brucephillips 9y ago
    https://m.cacm.acm.org/magazines/2017/6/217734-deep-learning-takes-on-translation/fulltext https://m.cacm.acm.org/magazines/2017/6/217734-deep-learning...
    - comstock 9y ago
      Looks like great incremental progess. Have you seen the state of Japanese<->English translation? It’s almost completely useless. I really don’t see this as a huge win for deep learning, anything else?
      - brucephillips 9y ago
        Whether progress is incremental is an ill defined question. I don't consider "super human translation" to be incremental. The key point here is that deep learning has produced significant results. I'm not sure why you care to argue semantics.
        comstock 9y ago
        Well, I’m interested in understanding how valuable deep learning is and if lives up to the hype. Better translation of European languages (which wasn’t a totally unsolved problem anyway) doesn’t seem to be something that really lives up to the hype. Particularly as the article cited doesn’t seem to back up its statements very well. So... anything else?
        brucephillips 9y ago
        If super human translation doesn't impress you, what will?
        comstock 9y ago
        The article doesn’t make that statement. The article doesn’t provide data to support any statements (it’s a pop science piece). The original blog: https://research.googleblog.com/2016/09/a-neural-network-for-machine.html?m=1 https://research.googleblog.com/2016/09/a-neural-network-for... Is better suggests deep learning resulted in maybe 10% improvement. Isn’t as good as human in all cases.
        brucephillips 9y ago
        Ah I misread "as good" as "better". It's still an epsilon difference, though. And the article lists other applications that have had "step wise" improvements, which is the opposite of incremental of course. Also, you didn't answer the question.
        deleted 9y ago
        [deleted]
  - joe_the_user 9y ago
    The big results as far as I understand: - Image recognition - Winning the games computers hadn't won already - Incremental progress on translation. Plus translation that doesn't need as many domain experts - Self-driving cars (with related automation applications) Of image there, image recognition stands out as the big leap and the rest are relatively incremental. One of the things with the other applications is that they provide a recipe format that's more systematic than previous approaches. A lot of vision approaches pre-deep-learning were very hit-or-miss. Deep learning has a lot of black art involved in effective training and a lot of time investment but my impression it is more reliable than what came before. Any other examples welcome
    - sanxiyn 9y ago
      You forgot speech recognition.
  - eksemplar 9y ago
    This isn't related to alpha go as such, but we can now predict which citizens are going to need help raising their children by having a machine cross reference their case history with public records. It's not legal yet, but it will be, because it will potentially save lives (and money).
  - mfukar 9y ago
    Well, "big new" sounds like a destructive qualification. Why can't it be "big" and "old"? https://arxiv.org/pdf/1712.01208.pdf https://arxiv.org/pdf/1712.01208.pdf
eanzenberg 9y ago
Depends who you ask. If you talk to people knowledgeable about deep learning and its applicability they’ll say we’re in the productivity regime. If you’re asking people who aren’t knowledgeable then they will display their hype.
marshray 9y ago
I think we're just used to computers advancing noticeably on a regular basis: "Is this year's iPhone better enough to justify an upgrade?" Also, we judge the difficulty of things by our own experience. It took us ~1 billion years to get to the point where we could communicate abstract ideas and play chess. These were once believed to be the challenging problems in AI. It turned out that chess is easy we're just relatively bad at it.
- comstock 9y ago
  I find it somewhat understandable from non-tech people. I’m more surprised at now much people within the tech world but the hype.
- yters 9y ago
  Chess is easy when you have the hardware to effectively brute force it. Once someone develops an algorithm that requires an order of moves comparable to a human, and significantly outperforms a human, then AI will be interesting.
eksemplar 9y ago
It won't. It's going to change the way we do public sector. Not so much because the tech is revolutionary but because the upper echelons of society are sold on it and are actually putting it to use. Technically you could do a lot of the decision making it'll be doing with human made models and a lot of data, but the machine is cheaper and it's backed by consulting agencies. RPA was the first indication. It's basically screenscraping and small bots, stuff that's been around for a long time, I mean, it's basically what people use to bot in video games. Yet it's become a multimillion dollar industry over the course of a few years because it caught the right drift. Like RPA, machine learning isn't just hype. It actually does some things with data really well, and when you couple that with the fact that ministers want this tech, well, that's all you need.

brucephillips 9y ago

The big tech companies are demonstrably using deep learning to solve previously unsolvable problems. It's a significant advance. What's yet to be seen is if startups can profit from this advance, since it depends on massive data and compute.

comstock 9y ago
AlphaGo is interesting. But what big new problems have been solved? (rather than incrementally improved).
- brucephillips 9y ago
  https://m.cacm.acm.org/magazines/2017/6/217734-deep-learning-takes-on-translation/fulltext https://m.cacm.acm.org/magazines/2017/6/217734-deep-learning...
  - comstock 9y ago
    Looks like great incremental progess. Have you seen the state of Japanese<->English translation? It’s almost completely useless. I really don’t see this as a huge win for deep learning, anything else?
    - brucephillips 9y ago
      Whether progress is incremental is an ill defined question. I don't consider "super human translation" to be incremental. The key point here is that deep learning has produced significant results. I'm not sure why you care to argue semantics.
      - comstock 9y ago
        Well, I’m interested in understanding how valuable deep learning is and if lives up to the hype. Better translation of European languages (which wasn’t a totally unsolved problem anyway) doesn’t seem to be something that really lives up to the hype. Particularly as the article cited doesn’t seem to back up its statements very well. So... anything else?
        brucephillips 9y ago
        If super human translation doesn't impress you, what will?
        comstock 9y ago
        The article doesn’t make that statement. The article doesn’t provide data to support any statements (it’s a pop science piece). The original blog: https://research.googleblog.com/2016/09/a-neural-network-for-machine.html?m=1 https://research.googleblog.com/2016/09/a-neural-network-for... Is better suggests deep learning resulted in maybe 10% improvement. Isn’t as good as human in all cases.
        brucephillips 9y ago
        Ah I misread "as good" as "better". It's still an epsilon difference, though. And the article lists other applications that have had "step wise" improvements, which is the opposite of incremental of course. Also, you didn't answer the question.
        deleted 9y ago
        [deleted]
- joe_the_user 9y ago
  The big results as far as I understand: - Image recognition - Winning the games computers hadn't won already - Incremental progress on translation. Plus translation that doesn't need as many domain experts - Self-driving cars (with related automation applications) Of image there, image recognition stands out as the big leap and the rest are relatively incremental. One of the things with the other applications is that they provide a recipe format that's more systematic than previous approaches. A lot of vision approaches pre-deep-learning were very hit-or-miss. Deep learning has a lot of black art involved in effective training and a lot of time investment but my impression it is more reliable than what came before. Any other examples welcome
  - sanxiyn 9y ago
    You forgot speech recognition.
- eksemplar 9y ago
  This isn't related to alpha go as such, but we can now predict which citizens are going to need help raising their children by having a machine cross reference their case history with public records. It's not legal yet, but it will be, because it will potentially save lives (and money).
- mfukar 9y ago
  Well, "big new" sounds like a destructive qualification. Why can't it be "big" and "old"? https://arxiv.org/pdf/1712.01208.pdf https://arxiv.org/pdf/1712.01208.pdf

eanzenberg 9y ago

Depends who you ask. If you talk to people knowledgeable about deep learning and its applicability they’ll say we’re in the productivity regime. If you’re asking people who aren’t knowledgeable then they will display their hype.

marshray 9y ago

I think we're just used to computers advancing noticeably on a regular basis: "Is this year's iPhone better enough to justify an upgrade?" Also, we judge the difficulty of things by our own experience. It took us ~1 billion years to get to the point where we could communicate abstract ideas and play chess. These were once believed to be the challenging problems in AI. It turned out that chess is easy we're just relatively bad at it.

comstock 9y ago
I find it somewhat understandable from non-tech people. I’m more surprised at now much people within the tech world but the hype.
yters 9y ago
Chess is easy when you have the hardware to effectively brute force it. Once someone develops an algorithm that requires an order of moves comparable to a human, and significantly outperforms a human, then AI will be interesting.

eksemplar 9y ago

It won't. It's going to change the way we do public sector. Not so much because the tech is revolutionary but because the upper echelons of society are sold on it and are actually putting it to use. Technically you could do a lot of the decision making it'll be doing with human made models and a lot of data, but the machine is cheaper and it's backed by consulting agencies. RPA was the first indication. It's basically screenscraping and small bots, stuff that's been around for a long time, I mean, it's basically what people use to bot in video games. Yet it's become a multimillion dollar industry over the course of a few years because it caught the right drift. Like RPA, machine learning isn't just hype. It actually does some things with data really well, and when you couple that with the fact that ministers want this tech, well, that's all you need.