3 ms·

I think the conclusion of the article really says it all. it is unwise to assume that these techniques will suffice for AI in general. It may be an effecti

I think the conclusion of the article really says it all. it is unwise to assume that these techniques will suffice for AI in general. It may be an effective short-term research strategy to focus on the immediate challenges that seem to be surmountable, but focusing on the surmountable may not get us to what is most necessary: a firm foundation for reliably integrating statistics and machine learning with reasoning, knowledge, common sense and human values. We tried that for ~50 years. It seemed to make progress for a little while, then stopped dead for decades. Now we're trying something else, and it's progressing incredibly fast. Maybe huge models with expensive training aren't the right approach to AI, but it's certainly the best candidate and continues to be incredibly fruitful. A few cherry-picked bad examples from an early prototype aren't a good reason to jump off the rocket ship.

visarga 5y ago
What the critics forget is that previous models were even worse than foundation models, much worse. The fact we're only now having a discussion about their intelligence is proof they closed the gap considerably. Here are some authoritative opinions on the topic from Yoav Golberg > so, don't believe the hype, and yes, we are still very very far from understanding language, and the new models have many shortcomings, but they are also very strong. much stronger than what we had before. going back is not the solution. just keep pushing. > well, all true, but at least they work. like, previous-gen models also didn't understand meaning, and also considered only form. they were just much worse at this. so much worse that no one could ever imagine that they capture any kind of meaning whatsoever. they didn't work. > oh, but we didn't have text-generators that associate muslims with terrorism! sure, we didn't have any text generators that could hold even a remotely coherent several sentence open domain text. Source: https://twitter.com/yoavgo/status/1432122812991430659 https://twitter.com/yoavgo/status/1432122812991430659 So they are simultaneously much better and not good enough.
- Jensson 5y ago
  > What the critics forget is that previous models were even worse than foundation models, much worse. The fact we're only now having a discussion about their intelligence is proof they closed the gap considerably. People have been arguing that the latest in AI research has been evidence that the singularity is just around the corner for decades, I don't see the current discussions as any different from before. Its just the same old, AI solved a new problem, AI optimists thinks this means human intelligence is now solved and AI will soon rule everything. Reality is that AI just solved that problem and other than that problem we are no closer to human intelligence. The problem AI solved the past decade was pattern recognition. That is useful for labelling images, detecting faces, evaluating positions in games like Go etc. But it is very apparent that just pattern recognition isn't enough to reach human intelligence. We still need a fundamental new breakthrough to reach it, and such breakthroughs can take centuries if they happen at all.
  - PeterisP 5y ago
    I believe that the big takeaway from GPT-3 experimental results is not about a potential path to general human-level intelligence, but rather about the observation that so many of the "intelligence-related" practical tasks that we wanted to solve (and could not before) don't really need human intelligence or anything close to it; that apparently pattern-recognition-on-steroids gets us close enough to automate tasks for which we mistakenly thought that actual intelligence is necessary. If a stochastic parrot can do a complex task, it does not tell much about the intelligence of that system, but it does tell a lot about the properties of that task.

visarga 5y ago

What the critics forget is that previous models were even worse than foundation models, much worse. The fact we're only now having a discussion about their intelligence is proof they closed the gap considerably. Here are some authoritative opinions on the topic from Yoav Golberg > so, don't believe the hype, and yes, we are still very very far from understanding language, and the new models have many shortcomings, but they are also very strong. much stronger than what we had before. going back is not the solution. just keep pushing. > well, all true, but at least they work. like, previous-gen models also didn't understand meaning, and also considered only form. they were just much worse at this. so much worse that no one could ever imagine that they capture any kind of meaning whatsoever. they didn't work. > oh, but we didn't have text-generators that associate muslims with terrorism! sure, we didn't have any text generators that could hold even a remotely coherent several sentence open domain text. Source: https://twitter.com/yoavgo/status/1432122812991430659 https://twitter.com/yoavgo/status/1432122812991430659 So they are simultaneously much better and not good enough.

Jensson 5y ago
> What the critics forget is that previous models were even worse than foundation models, much worse. The fact we're only now having a discussion about their intelligence is proof they closed the gap considerably. People have been arguing that the latest in AI research has been evidence that the singularity is just around the corner for decades, I don't see the current discussions as any different from before. Its just the same old, AI solved a new problem, AI optimists thinks this means human intelligence is now solved and AI will soon rule everything. Reality is that AI just solved that problem and other than that problem we are no closer to human intelligence. The problem AI solved the past decade was pattern recognition. That is useful for labelling images, detecting faces, evaluating positions in games like Go etc. But it is very apparent that just pattern recognition isn't enough to reach human intelligence. We still need a fundamental new breakthrough to reach it, and such breakthroughs can take centuries if they happen at all.
- PeterisP 5y ago
  I believe that the big takeaway from GPT-3 experimental results is not about a potential path to general human-level intelligence, but rather about the observation that so many of the "intelligence-related" practical tasks that we wanted to solve (and could not before) don't really need human intelligence or anything close to it; that apparently pattern-recognition-on-steroids gets us close enough to automate tasks for which we mistakenly thought that actual intelligence is necessary. If a stochastic parrot can do a complex task, it does not tell much about the intelligence of that system, but it does tell a lot about the properties of that task.