4 ms·

The Arc stuff just felt intuitively wrong as soon as I heard it. I don't find any of Chollet's critiques of LLMs to be convincing. It's almost as if he's being

The Arc stuff just felt intuitively wrong as soon as I heard it. I don't find any of Chollet's critiques of LLMs to be convincing. It's almost as if he's being overly negative about them to make a point or something to push back against all the unbridled optimism. The problem is, the optimism really seems to be justified, and the rate of improvement of LLMs in the past 12 months has been nothing short of astonishing. So it's not at all surprising to me to see Arc already being mostly solved using existing models, just with different prompting techniques and some tool usage. At some point, the naysayers about LLMs are going to have to confront the problem that, if they are right about LLMs not really thinking/understanding/being sentient, then a very large percentage of people living today are also not thinking/understanding/sentient!

HarHarVeryFunny 2y ago
Actually the solution being discussed here is the one that Chollet mentioned in his interview with Dwarkesh, and only bolsters his case. The LLM isn't doing the reasoning here, it's just pattern matching the before/after diff and generating thousands of Python programs. The actual reasoning is done by an agentic like loop wrapped around the LLM, as described in the linked blog.
- awwaiid 2y ago
  When you peer into the soul of the machine it delicately resolves to `while(1){...}`. All Hail The REPL.
Smaug123 2y ago
> a very large percentage of people living today are also not thinking/understanding/sentient This isn't that big a bullet to bite (https://www.lesswrong.com/posts/4AHXDwcGab5PhKhHT/humans-who-are-not-concentrating-are-not-general https://www.lesswrong.com/posts/4AHXDwcGab5PhKhHT/humans-who... comes from well before ChatGPT's launch), and I myself am inclined to bite it. System 1 alone does not a general intelligence make, although the article is extremely interesting in asking the question "is System 1 plus Python enough for a general intelligence?". But it's not a very relevant philosophical point, because Chollet's position is consistent with humans being obsoleted and/or driven extinct whether or not the LLMs are "general intelligences". His position is that training LLMs results in an ever-larger number of learned algorithms and no ability to construct new algorithms. This is consistent with the possibility that, after some threshold of size and training, the LLM has learned every algorithm it needs to supplant humans in (say) 99.9% of cases. (It would definitely be going out with a whimper rather than a bang, on that hypothesis, to be out-competed by something that _really is_ just a gigantic lookup table!)
threeseed 2y ago
a) 50% result is not solving the problem. Especially when the implementation is brute forcing the problem and is against the spirit of ARC. b) He is not being overly negative of LLMs. In fact he believes they will play a role in any AGI system. c) OpenAI CTO has publicly said that ChatGPT 5 will not be significantly better than existing models. So the rate of improvements you believe in simply doesn't match reality.
- janalsncm 2y ago
  For the record, a lot of problems might turn out to be like this, where we figure out a brute force approach that stands in for human creativity.
- deleted 2y ago
  [deleted]
- hackerlight 2y ago
  Skeptical about (c), source please. She did say they don't have anything much better than GPT-4o currently, but GPT-5 likely only started training recently.
traject_ 2y ago
> It's almost as if he's being overly negative about them to make a point or something to push back against all the unbridled optimism. I don't think it is like that but rather Chollet wants to see stronger neuroplasticity in these models. I think there is a divide between the effectiveness of existing AI models versus their ability to be autonomous, robust and consistently learn from unanticipated problems. My guess is Chollet wants to see something more similar to biological organisms especially mammals or birds in their level of autonomous nature. I think people underestimate the degree of novel problems birds and mammals alone face in just simply navigating their environment and it is the comparison here that LLMs, for now at least, seem lacking. So when he says LLMs are not sentient, he's asking to consider the novel problems animals let alone humans have to face in navigating their environment. This is especially apparent in young children but declines as we age and gain experience/lose a sense of novelty.
- infgeoax 2y ago
  Agree. When I first saw ARC, my reaction was this could possibly be the kind of problem that gives us evolutionary pressure.
adroniser 2y ago
I don't see how the point about the typical human is relevant. Either you can reason or you can't, the ARC test is supposed to be an objective way to measure this. Clearly a vanilla LLM currently cannot do this, and somehow an expert crafting a super-specific prompt is supposed to be impressive.
- eigenvalue 2y ago
  The point is that if you have some test of whether an AI is intelligent that the vast majority of living humans would fail or do worse on than gpt4-o (let alone future LLMs) then it’s not a very persuasive argument.
TacticalCoder 2y ago
> I don't find any of Chollet's critiques of LLMs to be convincing. It's almost as if he's being overly negative about them to make a point or something to push back against all the unbridled optimism. Chollet published his paper On the measure of intelligence in 2019. In Internet time that is a lifetime before the LLM hype started.
- refulgentis 2y ago
  Einstein, infamously, couldn't really make much progress with quantum physics, even though he invented the precursors (ex. Brownian motion). Your world model is hard to update.
  - imperfect_light 2y ago
    A bit of a stretch given that Chollet is a researcher in deep learning and transformers and his criticism is that memorization (training LLMs on lots and lots of problems) doesn't equate to AGI.
    - refulgentis 2y ago
      > A bit of a stretch Is that true? C.f. what we're discussing He's actively encouraging using LLMs to solve his benchmark, called ARC AGI. 8 hours ago, from Chollet, re: TFA "The best solution to fight combinatorial explosion is to leverage intuition over the structure of program space, provided by a deep learning model. For instance, you can use a LLM to sample a program..." Source: https://x.com/fchollet/status/1802801425514410275 https://x.com/fchollet/status/1802801425514410275
      - imperfect_light 2y ago
        The stretch was in reference to comparing Chollet to Einstein. Chollet clearly understands LLMs (and transformers and deep learning), he simply doesn't believe they are sufficient for AGI.
imtringued 2y ago
Yeah I agree. We have reached the end of LLMs. LLMs are infallible and require no further improvement. Anyone who points out shortcomings of current architectures and training approaches should be ignored as a naysayer. Anyone who proposes a solution to perceived flaws is a crank trying to fix something that was never broken. Everyone knows humans are incapable of internal monologues or visualization and vocalisation. Humans don't actually move their lips to speak to produce a sound that can be interpreted by a speaker of the same language, they produce universally understood tokens encoding objective reality and the fact that they use the local language is merely a habit that is hard to break out of.
- mrtranscendence 2y ago
  Sometimes, when I'm undertaking the arduous work of assigning probabilities to everything I could possibly say next in a conversation, I wish that I weren't merely a stochastic autoregressive next-token generator. Them's the breaks, though.
biophysboy 2y ago
I don't think he's as critical as you say. He just views LLMs as the product of intelligence rather than intelligence itself. LLM fans will say this is a false distinction, I guess. His definition of intelligence is interesting: something that can quickly achieve tasks with few priors or experience. I also think the idea of using human "Core Knowledge" priors is a clever way to make a test.
lassoiat 2y ago
I am a chatGPT fan boy and have been quite impressed by 4o but I will really be impressed when it stops inventing aspects of python libraries that don't exists and instead just tells me it doesn't exist. It literally just did this for me 15 minutes ago. You can't talk about AGI when it is this easy to push it over the edge into something it doesn't know. Paper references have got better the last 12 months but just this week it made up both a book and paper for me that do not exist. The authors exist and they did not write what it said they did. It is very interesting if you ask "do you understand your responses?" sometimes it will say yes and sometimes it will so no not like a human understands. We should forget about AGI until it can at least say it doesn't know something. It is hardly a sign of intelligence in humans to make up answers to questions you don't know.
- motoxpro 2y ago
  Every time you’re wrong and you disagree with someone who is right you are inventing things that don’t exist. Unless you’re saying you have never held on to a wrong opinion that was at some point proven to be wrong?
imperfect_light 2y ago
Did you listen to what Chollet said? How much of LLM improvements are due to enlarging the training sets to cover more problems and how much is due to any emergent properties?
Lockal 2y ago
That's a big jump in generalization that bruteforcing 4 colors in 9x9 grids with 8000 programs has anything near to what sentient human can do. Back in the days similar generalization was used for Deep Blue chess computer. Computer won in 1997, but the AGI abyss is still as big.

HarHarVeryFunny 2y ago

Actually the solution being discussed here is the one that Chollet mentioned in his interview with Dwarkesh, and only bolsters his case. The LLM isn't doing the reasoning here, it's just pattern matching the before/after diff and generating thousands of Python programs. The actual reasoning is done by an agentic like loop wrapped around the LLM, as described in the linked blog.

awwaiid 2y ago
When you peer into the soul of the machine it delicately resolves to `while(1){...}`. All Hail The REPL.

Smaug123 2y ago

> a very large percentage of people living today are also not thinking/understanding/sentient This isn't that big a bullet to bite (https://www.lesswrong.com/posts/4AHXDwcGab5PhKhHT/humans-who-are-not-concentrating-are-not-general https://www.lesswrong.com/posts/4AHXDwcGab5PhKhHT/humans-who... comes from well before ChatGPT's launch), and I myself am inclined to bite it. System 1 alone does not a general intelligence make, although the article is extremely interesting in asking the question "is System 1 plus Python enough for a general intelligence?". But it's not a very relevant philosophical point, because Chollet's position is consistent with humans being obsoleted and/or driven extinct whether or not the LLMs are "general intelligences". His position is that training LLMs results in an ever-larger number of learned algorithms and no ability to construct new algorithms. This is consistent with the possibility that, after some threshold of size and training, the LLM has learned every algorithm it needs to supplant humans in (say) 99.9% of cases. (It would definitely be going out with a whimper rather than a bang, on that hypothesis, to be out-competed by something that _really is_ just a gigantic lookup table!)

threeseed 2y ago

a) 50% result is not solving the problem. Especially when the implementation is brute forcing the problem and is against the spirit of ARC. b) He is not being overly negative of LLMs. In fact he believes they will play a role in any AGI system. c) OpenAI CTO has publicly said that ChatGPT 5 will not be significantly better than existing models. So the rate of improvements you believe in simply doesn't match reality.

janalsncm 2y ago
For the record, a lot of problems might turn out to be like this, where we figure out a brute force approach that stands in for human creativity.
deleted 2y ago
[deleted]
hackerlight 2y ago
Skeptical about (c), source please. She did say they don't have anything much better than GPT-4o currently, but GPT-5 likely only started training recently.

traject_ 2y ago

> It's almost as if he's being overly negative about them to make a point or something to push back against all the unbridled optimism. I don't think it is like that but rather Chollet wants to see stronger neuroplasticity in these models. I think there is a divide between the effectiveness of existing AI models versus their ability to be autonomous, robust and consistently learn from unanticipated problems. My guess is Chollet wants to see something more similar to biological organisms especially mammals or birds in their level of autonomous nature. I think people underestimate the degree of novel problems birds and mammals alone face in just simply navigating their environment and it is the comparison here that LLMs, for now at least, seem lacking. So when he says LLMs are not sentient, he's asking to consider the novel problems animals let alone humans have to face in navigating their environment. This is especially apparent in young children but declines as we age and gain experience/lose a sense of novelty.

infgeoax 2y ago
Agree. When I first saw ARC, my reaction was this could possibly be the kind of problem that gives us evolutionary pressure.

adroniser 2y ago

I don't see how the point about the typical human is relevant. Either you can reason or you can't, the ARC test is supposed to be an objective way to measure this. Clearly a vanilla LLM currently cannot do this, and somehow an expert crafting a super-specific prompt is supposed to be impressive.

eigenvalue 2y ago
The point is that if you have some test of whether an AI is intelligent that the vast majority of living humans would fail or do worse on than gpt4-o (let alone future LLMs) then it’s not a very persuasive argument.

TacticalCoder 2y ago

> I don't find any of Chollet's critiques of LLMs to be convincing. It's almost as if he's being overly negative about them to make a point or something to push back against all the unbridled optimism. Chollet published his paper On the measure of intelligence in 2019. In Internet time that is a lifetime before the LLM hype started.

refulgentis 2y ago
Einstein, infamously, couldn't really make much progress with quantum physics, even though he invented the precursors (ex. Brownian motion). Your world model is hard to update.
- imperfect_light 2y ago
  A bit of a stretch given that Chollet is a researcher in deep learning and transformers and his criticism is that memorization (training LLMs on lots and lots of problems) doesn't equate to AGI.
  - refulgentis 2y ago
    > A bit of a stretch Is that true? C.f. what we're discussing He's actively encouraging using LLMs to solve his benchmark, called ARC AGI. 8 hours ago, from Chollet, re: TFA "The best solution to fight combinatorial explosion is to leverage intuition over the structure of program space, provided by a deep learning model. For instance, you can use a LLM to sample a program..." Source: https://x.com/fchollet/status/1802801425514410275 https://x.com/fchollet/status/1802801425514410275
    - imperfect_light 2y ago
      The stretch was in reference to comparing Chollet to Einstein. Chollet clearly understands LLMs (and transformers and deep learning), he simply doesn't believe they are sufficient for AGI.

imtringued 2y ago

Yeah I agree. We have reached the end of LLMs. LLMs are infallible and require no further improvement. Anyone who points out shortcomings of current architectures and training approaches should be ignored as a naysayer. Anyone who proposes a solution to perceived flaws is a crank trying to fix something that was never broken. Everyone knows humans are incapable of internal monologues or visualization and vocalisation. Humans don't actually move their lips to speak to produce a sound that can be interpreted by a speaker of the same language, they produce universally understood tokens encoding objective reality and the fact that they use the local language is merely a habit that is hard to break out of.

mrtranscendence 2y ago
Sometimes, when I'm undertaking the arduous work of assigning probabilities to everything I could possibly say next in a conversation, I wish that I weren't merely a stochastic autoregressive next-token generator. Them's the breaks, though.

biophysboy 2y ago

I don't think he's as critical as you say. He just views LLMs as the product of intelligence rather than intelligence itself. LLM fans will say this is a false distinction, I guess. His definition of intelligence is interesting: something that can quickly achieve tasks with few priors or experience. I also think the idea of using human "Core Knowledge" priors is a clever way to make a test.

lassoiat 2y ago

I am a chatGPT fan boy and have been quite impressed by 4o but I will really be impressed when it stops inventing aspects of python libraries that don't exists and instead just tells me it doesn't exist. It literally just did this for me 15 minutes ago. You can't talk about AGI when it is this easy to push it over the edge into something it doesn't know. Paper references have got better the last 12 months but just this week it made up both a book and paper for me that do not exist. The authors exist and they did not write what it said they did. It is very interesting if you ask "do you understand your responses?" sometimes it will say yes and sometimes it will so no not like a human understands. We should forget about AGI until it can at least say it doesn't know something. It is hardly a sign of intelligence in humans to make up answers to questions you don't know.

motoxpro 2y ago
Every time you’re wrong and you disagree with someone who is right you are inventing things that don’t exist. Unless you’re saying you have never held on to a wrong opinion that was at some point proven to be wrong?

imperfect_light 2y ago

Did you listen to what Chollet said? How much of LLM improvements are due to enlarging the training sets to cover more problems and how much is due to any emergent properties?

Lockal 2y ago

That's a big jump in generalization that bruteforcing 4 colors in 9x9 grids with 8000 programs has anything near to what sentient human can do. Back in the days similar generalization was used for Deep Blue chess computer. Computer won in 1997, but the AGI abyss is still as big.