3 ms·

The problem is that current generative AI is not actually intelligent. Yann LeCunn had a great tweet on this: Sometimes, the obvious must be studied so it can

The problem is that current generative AI is not actually intelligent. Yann LeCunn had a great tweet on this: Sometimes, the obvious must be studied so it can be asserted with full confidence: - LLMs can not answer questions whose answers are not in their training set in some form, - they can not solve problems they haven't been trained on, - they can not acquire new skills our knowledge without lots of human help, - they can not invent new things. Now, LLMs are merely a subset of AI techniques. Merely scaling up LLMs will not lead systems with these capabilities. link https://x.com/ylecun/status/1823313599252533594?ref_src=twsrc%5Egoogle%7Ctwcamp%5Eserp%7Ctwgr%5Etweet https://x.com/ylecun/status/1823313599252533594?ref_src=twsr... To focus on this: - LLMs can not answer questions whose answers are not in their training set in some form, - they can not solve problems they haven't been trained on Given that we are close to maximum in the size of the training set, this means they are not going to improve without some completely unknown at the moment technical breakthrough. Going from "not intelligent" to "intelligent" is a massive shift.

jonahx 2y ago
"in some form" is doing a lot of work there. The problem is that, by the standards of most human beings, they are in fact doing what we informally call "inference" or "creating new things". That this is being accomplished by something that is "technically a fancy autocomplete" doesn't seem to matter practically... it's still doing all this useful and surprising stuff.
- intended 2y ago
  It is carrying lots of weight. At this moment that’s the most sophisticated we can be in talking about LLMs. I will say that the utility of these tools is not being denied. It’s just the struggle to explain the varied experiences. I can get only as far as analogy, not precise definitions. For me, LLMs are like the invention of microwave ovens. Very useful. They aren’t like the discovery of fire.
stevenhuang 2y ago
Yea, LeCunn is kind of a meme. You're doing yourself and readers a disservice when you quote him without mentioning his conflict of interest. His research is in analytical approaches to ML hence his bitterness against current LLM techniques and skepticism towards Sutton's Bitter Lesson.

jonahx 2y ago

"in some form" is doing a lot of work there. The problem is that, by the standards of most human beings, they are in fact doing what we informally call "inference" or "creating new things". That this is being accomplished by something that is "technically a fancy autocomplete" doesn't seem to matter practically... it's still doing all this useful and surprising stuff.

intended 2y ago
It is carrying lots of weight. At this moment that’s the most sophisticated we can be in talking about LLMs. I will say that the utility of these tools is not being denied. It’s just the struggle to explain the varied experiences. I can get only as far as analogy, not precise definitions. For me, LLMs are like the invention of microwave ovens. Very useful. They aren’t like the discovery of fire.

stevenhuang 2y ago

Yea, LeCunn is kind of a meme. You're doing yourself and readers a disservice when you quote him without mentioning his conflict of interest. His research is in analytical approaches to ML hence his bitterness against current LLM techniques and skepticism towards Sutton's Bitter Lesson.