8 ms·

They miss for the same reasons they have missed throughout the decades: they overpromise and underdeliver. That was the reason for the past two AI "winters" and

They miss for the same reasons they have missed throughout the decades: they overpromise and underdeliver. That was the reason for the past two AI "winters" and most likely will be the same reason for the upcoming AI winter. What we have successfully accomplished this time around is big data analytics that have used machine learning technology to derive insights and patterns from data that traditional data analysis approaches have not been able to achieve. And dramatic improvements in computer vision and natural language processing among a few other areas. Those things will stay. But I'm looking forward to the next wave of AI, which should be in the mid- to late-2030s if the current cycles hold.

slver 5y ago
> That was the reason for the past two AI "winters" and most likely will be the same reason for the upcoming AI winter. If you follow the papers coming out, I'd say we're very far from any winter. Public perception may ebb and flow, but nothing can stop us now, research has gathered that critical mass and we're going nuclear on AI right as we speak.
- lumost 5y ago
  I wouldn't be that bullish on the current crop of research. While each technique comes out with some impressive wins. Many of these wins are on toy tasks, or the gains are incremental on tasks well below the commercial utility threshold. Even large models like GPT-3 can't tame language and are steadily pricing themselves to be un-competitive with people.
  - solidasparagus 5y ago
    Deep learning is pretty consistently putting out groundbreaking work in new domains. In the current boom, we have seen massive breakthroughs in: - image classification - image understanding (object detection and segmentation) - text to speech - speech to text - natural language modeling - style transfer - intelligent agents (go/Dota/etc) - protein folding And many other less openly documented breakthroughs that are actively delivering business value. New successful research (impactful stuff, not small improvements in SOTA) is happening at a pace that suggests even if we enter a bit of a winter due to overpromised technologies, the core technology of differentiable programming will continue to break into new domains and revolutionize them.
    - lumost 5y ago
      It’s not as clear cut on the commercial impact front. While reading the literature can sometimes give the impression that NNs have delivered game changing performance. In many cases they are delivering 1-10% gains over prior techniques. Alternatively in fields like speech to text the most impressive neural results are commercially impractical due to the cost of inference. It cost roughly 30 million dollars to train alpha star which rated in the top .3% of Starcraft players. Presuming that the technology reasonably transferred to other fields ( lack of prepplayed pro games would be a problem ) there are few reasons to believe one could gain 30 million dollars of value from the result.
      - solidasparagus 5y ago
        I've seen internal work at large companies like Amazon and my sense was that the financial impact was very meaningful. Deep Learning is in many ways a new economy of scale and a 5% reduction in say warehouse space or reduced fraud at Amazon scale is a LOT of money in comparison to the researchers and compute that went into building that model. Not to mention the impact of completely new technologies like voice assistants, automated high-quality image modification, automated computer vision systems (ones that have produced real value like automated industrial maintenance notification systems). > In many cases they are delivering 1-10% gains over prior techniques. As a general statement, that is false. At least among the companies that are good at DL. As for alphastar, I somewhat agree - it's very early and that tech hasn't left the lab yet AFAICT. But if you could build an agent that was human average at some common human task (e.g. driving) that is easily worth hundreds of millions of dollars if not billions. It takes time to go from the lab to commercial usage. Older breakthroughs like CV and transcription/TTS has made that transition successfully. More recent breakthroughs like like RL agents, NLP, style transfer are still making that transition (at varying paces). And there continue to be new research breakthroughs like protein folding that are still many years from making their way into industry, but the continuous nature of these breakthroughs bode very well for the future of DL.
ChefboyOG 5y ago
An "AI winter" like we had before isn't plausible now. In the past, it was the failure of AI research to deliver business impact. The current ML boom is fundamentally different in that it already has become a standard part of the stack at major companies: - Recommendation engines (Google, YouTube, Facebook, etc.) - Fraud detection (Stripe, basically all banks) - ETA prediction (Uber, every delivery app you use) - Speech-to-text (Siri, Google Assistant, Alexa) - Sequence-to-sequence translation (used in everything from language translation to medicinal chemistry) - The entire field of NLP, which is now powering basically any popular app you use that analyzes text content or has some kind of filtering functionality (i.e. toxicity filtering on social platforms). And that's a very cursory scan. You can go much, much deeper. That isn't to say that there isn't plenty of snake oil out there, just that ML (and by extension, AI research) is generating billions in revenue for businesses today. There's not going to be a slow down in ML research for a while, as a result.
- slowmovintarget 5y ago
  Partly this is because we've given up trying to get computers to "understand" and have focused on making them useful with sophisticated software. That is, the work is no longer about artificial intelligence, but about trained, new-style, expert systems.
  - jsharf 5y ago
    This is just redefining what counts as "real intelligence" to raise the bar. Once we have computers that can think like humans, there'll be people saying 'oh, it's not real "understanding", it's just generating speech that matches things it's heard in the past with some added style', not realizing that the same also applies to human writers. As long as the bar keeps rising and we've found business applications, there will be no AI winter.
    - slowmovintarget 5y ago
      It isn't redefining anything. There's a classification in AI research called "strong AI" that includes Artificial General Intelligence and machine consciousness. Current uses of ML are instances of so-called "weak AI" which is focused on problem-solving and utility. No redefinition, and using proper distinctions takes nothing away from the accomplishments made in the field. We don't even have anything close to a scientific understanding of awareness or consciousness. Being able to create machines actually possessing awareness seems fairly far off. Being able to create machines that can simulate the qualities of a conscious being, doesn't seem so far off. I suspect when we get there the data will point to there being a qualitative difference between real consciousness and simulated. Commercial interests, and likely government bureaucracy will have a vested interest drowning out such ideas, though. The bar hasn't moved. We've just begun aiming at different targets. That we succeed in hitting the more modest ones only makes sense.
foobarian 5y ago
On one hand, maybe the name is just unfortunate. The minute you mention AI the expectation is set to deliver machine sentience. On the other hand, maybe it is a good thing to set the goalposts high. We may not reach the target, but still end up with worthwhile results; see: current ML use cases in industry.
- dfilppi 5y ago
  Well "intelligence" does imply reasoning, which has never been delivered.
gmadsen 5y ago
I think this isn't really giving credit. Standard deep learning is not really slowing down. It is improving exponentially still. Just yesterday I was looking into new deep learning methods for solving PDEs. That is a huge area to explore and will change many things dramatically

slver 5y ago

> That was the reason for the past two AI "winters" and most likely will be the same reason for the upcoming AI winter. If you follow the papers coming out, I'd say we're very far from any winter. Public perception may ebb and flow, but nothing can stop us now, research has gathered that critical mass and we're going nuclear on AI right as we speak.

lumost 5y ago
I wouldn't be that bullish on the current crop of research. While each technique comes out with some impressive wins. Many of these wins are on toy tasks, or the gains are incremental on tasks well below the commercial utility threshold. Even large models like GPT-3 can't tame language and are steadily pricing themselves to be un-competitive with people.
- solidasparagus 5y ago
  Deep learning is pretty consistently putting out groundbreaking work in new domains. In the current boom, we have seen massive breakthroughs in: - image classification - image understanding (object detection and segmentation) - text to speech - speech to text - natural language modeling - style transfer - intelligent agents (go/Dota/etc) - protein folding And many other less openly documented breakthroughs that are actively delivering business value. New successful research (impactful stuff, not small improvements in SOTA) is happening at a pace that suggests even if we enter a bit of a winter due to overpromised technologies, the core technology of differentiable programming will continue to break into new domains and revolutionize them.
  - lumost 5y ago
    It’s not as clear cut on the commercial impact front. While reading the literature can sometimes give the impression that NNs have delivered game changing performance. In many cases they are delivering 1-10% gains over prior techniques. Alternatively in fields like speech to text the most impressive neural results are commercially impractical due to the cost of inference. It cost roughly 30 million dollars to train alpha star which rated in the top .3% of Starcraft players. Presuming that the technology reasonably transferred to other fields ( lack of prepplayed pro games would be a problem ) there are few reasons to believe one could gain 30 million dollars of value from the result.
    - solidasparagus 5y ago
      I've seen internal work at large companies like Amazon and my sense was that the financial impact was very meaningful. Deep Learning is in many ways a new economy of scale and a 5% reduction in say warehouse space or reduced fraud at Amazon scale is a LOT of money in comparison to the researchers and compute that went into building that model. Not to mention the impact of completely new technologies like voice assistants, automated high-quality image modification, automated computer vision systems (ones that have produced real value like automated industrial maintenance notification systems). > In many cases they are delivering 1-10% gains over prior techniques. As a general statement, that is false. At least among the companies that are good at DL. As for alphastar, I somewhat agree - it's very early and that tech hasn't left the lab yet AFAICT. But if you could build an agent that was human average at some common human task (e.g. driving) that is easily worth hundreds of millions of dollars if not billions. It takes time to go from the lab to commercial usage. Older breakthroughs like CV and transcription/TTS has made that transition successfully. More recent breakthroughs like like RL agents, NLP, style transfer are still making that transition (at varying paces). And there continue to be new research breakthroughs like protein folding that are still many years from making their way into industry, but the continuous nature of these breakthroughs bode very well for the future of DL.

ChefboyOG 5y ago

An "AI winter" like we had before isn't plausible now. In the past, it was the failure of AI research to deliver business impact. The current ML boom is fundamentally different in that it already has become a standard part of the stack at major companies: - Recommendation engines (Google, YouTube, Facebook, etc.) - Fraud detection (Stripe, basically all banks) - ETA prediction (Uber, every delivery app you use) - Speech-to-text (Siri, Google Assistant, Alexa) - Sequence-to-sequence translation (used in everything from language translation to medicinal chemistry) - The entire field of NLP, which is now powering basically any popular app you use that analyzes text content or has some kind of filtering functionality (i.e. toxicity filtering on social platforms). And that's a very cursory scan. You can go much, much deeper. That isn't to say that there isn't plenty of snake oil out there, just that ML (and by extension, AI research) is generating billions in revenue for businesses today. There's not going to be a slow down in ML research for a while, as a result.

slowmovintarget 5y ago
Partly this is because we've given up trying to get computers to "understand" and have focused on making them useful with sophisticated software. That is, the work is no longer about artificial intelligence, but about trained, new-style, expert systems.
- jsharf 5y ago
  This is just redefining what counts as "real intelligence" to raise the bar. Once we have computers that can think like humans, there'll be people saying 'oh, it's not real "understanding", it's just generating speech that matches things it's heard in the past with some added style', not realizing that the same also applies to human writers. As long as the bar keeps rising and we've found business applications, there will be no AI winter.
  - slowmovintarget 5y ago
    It isn't redefining anything. There's a classification in AI research called "strong AI" that includes Artificial General Intelligence and machine consciousness. Current uses of ML are instances of so-called "weak AI" which is focused on problem-solving and utility. No redefinition, and using proper distinctions takes nothing away from the accomplishments made in the field. We don't even have anything close to a scientific understanding of awareness or consciousness. Being able to create machines actually possessing awareness seems fairly far off. Being able to create machines that can simulate the qualities of a conscious being, doesn't seem so far off. I suspect when we get there the data will point to there being a qualitative difference between real consciousness and simulated. Commercial interests, and likely government bureaucracy will have a vested interest drowning out such ideas, though. The bar hasn't moved. We've just begun aiming at different targets. That we succeed in hitting the more modest ones only makes sense.

foobarian 5y ago

On one hand, maybe the name is just unfortunate. The minute you mention AI the expectation is set to deliver machine sentience. On the other hand, maybe it is a good thing to set the goalposts high. We may not reach the target, but still end up with worthwhile results; see: current ML use cases in industry.

dfilppi 5y ago
Well "intelligence" does imply reasoning, which has never been delivered.

gmadsen 5y ago

I think this isn't really giving credit. Standard deep learning is not really slowing down. It is improving exponentially still. Just yesterday I was looking into new deep learning methods for solving PDEs. That is a huge area to explore and will change many things dramatically