6 ms·

>As a quick aside, I am not going to entertain the notion that LLMs are intelligent, for any value of “intelligent.” They are robots. Programs. Fancy robots and

by woeirua 8mo ago

>As a quick aside, I am not going to entertain the notion that LLMs are intelligent, for any value of “intelligent.” They are robots. Programs. Fancy robots and big complicated programs, to be sure — but computer programs, nonetheless. The rest of this essay will treat them as such. If you are already of the belief that the human mind can be reduced to token regurgitation, you can stop reading here. I’m not interested in philosophical thought experiments. I can't imagine why someone would want to openly advertise that they're so closed minded. Everything after this paragraph is just anti-LLM ranting.

acjohnson55 8mo ago
It was actually much less anti LLM than I was expecting from the beginning. But I agree that it is self limiting to not bother to consider the ways that LLM inference and human thinking might be similar (or not). To me, they seem do a pretty reasonable emulation of single- threaded thinking.
- Zardoz84 8mo ago
  They are not similar. A LLM is a complex statistical machine. A brain is a highly complex neural network. A brain, is more similar the perceptron of some AMD CPUs that to a LLM.
  - PaulDavisThe1st 8mo ago
    I would recommend investigating how contemporary LLMs actually work. Possibly start with something like: https://transformer-circuits.pub/2025/attribution-graphs/biology.html https://transformer-circuits.pub/2025/attribution-graphs/bio...
hodgehog11 8mo ago
I disagree that the majority of it is anti-LLM ranting, there are several subtle points here that are grounded in realism. You should read on past the first bit if you're judging mainly from the initial (admittedly naive) first few paragraphs.
- woeirua 8mo ago
  I read the rest of it. It was intellectually lazy.
  - measurablefunc 8mo ago
    It's more intellectually lazy to think boolean logic at a sufficient scale crosses some event horizon wherein its execution on mechanical gadgets called computers somehow adds up to intelligence beyond human understanding.
    - hodgehog11 8mo ago
      It is intellectually lazy to proclaim something to be impossible in the absence of evidence or proof. In the case of the statement made here, it is provably true that Boolean logic at sufficient scale can replicate "intelligence" of any arbitrary degree. It is also easy to show that this can be perceived as an "event horizon" since the measurements of model quality that humans typically like to use are so nonlinear that they are virtually step function-like.
      - measurablefunc 8mo ago
        Doesn't seem like you have proof of anything but it does appear that you have something that is very much like religious faith in an unforeseeable inevitability. Which is fine as far as religion is concerned but it's better to not pretend it's anything other than blind faith. But if you really do have concrete proof of something then you'll have to spell it out better & explain how exactly it adds up to intelligence of such magnitude & scope that no one can make sense of it.
        hodgehog11 8mo ago
        > "religious faith in an unforeseeable inevitability" For reference, I work in academia, and my job is to find theoretical limitations of neural nets. If there was so much of a modicum of evidence to support the argument that "intelligence" cannot arise from sufficiently large systems, my colleagues and I would be utterly delighted and would be all over it. Here are a couple of standard elements without getting into details: 1. Any "intelligent" agent can be modelled as a random map from environmental input to actions. 2. Any random map can be suitably well-approximated by a generative transformer. This is the universal approximation theorem. Universal approximation does not mean that models of a given class can be trained using data to achieve an arbitrary level of accuracy, however... 3. The neural scaling laws (first empirical, now more theoretically established under NTK-type assumptions), as a refinement of the double descent curve, assert that a neural network class can get arbitrarily close to an "entropy level" given sufficient scale. This theoretical level is so much smaller than any performance metric that humans can reach. Whether "sufficiently large" is outside of the range that is physically possible is a much longer discussion, but bets are that human levels are not out of reach (I don't like this, to be clear). 4. The nonlinearity of accuracy metrics comes from the fact that they are constructed from the intersection of a large number of weakly independent events. Think the CDF of a Beta random variable with parameters tending to infinity. Look, I understand the scepticism, but from where I am, reality isn't leaning that way at the moment. I can't afford to think it isn't possible. I don't think you should either.
wolrah 8mo ago
> I can't imagine why someone would want to openly advertise that they're so closed minded. I would say the exact same about you, rejecting an absolutely accurate and factual statement like that as closed minded strikes me as the same as the people who insist that medical science is closed minded about crystals and magnets. I can't imagine why someone would want to openly advertise they think LLMs are actual intelligence, unless they were in a position to benefit financially from the LLM hype train of course.
- woeirua 8mo ago
  Cool, so clearly articulate the goal posts. What do LLMs have to do to convince you that they are intelligent? If the answer is there is no amount of evidence that can change your mind, then you're not arguing in good faith.
  - rmunn 8mo ago
    Maybe, I don't know, not be based on a statistical model? Come on. If you are actually entertaining the idea that LLMs can possibly be intelligent, you don't know how they work. But to take your silly question seriously for a minute, maybe I might consider LLMs to be capable of intelligence if they were able to learn, if they were able to solve problems that they weren't explicitly trained for. For example, have an LLM read a bunch of books about the strategy of Go, then actually apply that knowledge to beat an experienced Go player who was deliberately playing unconventional, poor strategies like opening in the center. Since pretty much nobody opens their Go game in the center (the corners are far superior), the LLM's training data is NOT going to have a lot of Go openings where one player plays mostly in the center. At which point you'll see that the LLM isn't actually intelligent, because an intelligent being would have understood the concepts in the book that you should mostly play in the corners at first in order to build territory with the smallest number of moves. But when faced with unconventional moves that aren't found anywhere on the Internet, the LLM would just crash and burn. That would be a good test of intelligence. Learning by reading books, and then being able to apply that knowledge to new situations where you can't just regurgitate the training material.
Ygg2 8mo ago
> I can't imagine why someone would want to openly advertise that they're so closed minded. Because humans often anthropomorphize completely inert things? E.g. a coffee machine or a bomb disposal robot. So far whatever behavior LLMs have shown is basically fueled by Sci-Fi stories of how a robot should behave under such and such.
Cloudef 8mo ago
What's wrong about the statement? The black box algorithm might have been generated by machine learning, but it's still a computer program in the end.
- km3r 8mo ago
  Because it's so entirely reductive and misunderstanding of where the technology has progressed. Hello world is s computer program. So it Microsoft Windows. New levels of "intelligence" unlock with greater complexity of a program. Like look at our brains. We know decently well how a single neuron works. We can simulate a single one with "just a computer program". But clearly with enough layers some form of complexity can emerge, and at some level that complexity becomes intelligence.
  - andsoitis 8mo ago
    > with enough layers some form of complexity can emerge, and at some level that complexity becomes intelligence. It isn’t a given that complexity begets intelligence.
    - PaulDavisThe1st 8mo ago
      and it isn't a given that it doesn't, so maybe a little openness towards the possibility is warranted?
      - andsoitis 8mo ago
        I’m open, but the comment I responded to asserted: “complexity becomes intelligence”, as if it is a fact. And it isn’t proven.
        DiogenesKynikos 8mo ago
        We have LLMs, which are obviously intelligent. How is it not proven?
palmotea 8mo ago
> I can't imagine why someone would want to openly advertise that they're so closed minded. It's not being closed-minded. It's not wanting to get sea-lioned to death by obnoxious people.
- PaulDavisThe1st 8mo ago
  [WARNING: seriously off-topic comment, I was triggered] Here's what sea-lioned means to me: I say something. You accuse me of sea-lioning. I have two choices: attempt to refute the sea-lioning, which becomes sea-lioning, or allowing your accusation to stand unchallenged, which appears to most people as a confirmation of some kind that I was sea-lioning. It is a nuclear weapon launched at discussion. It isn't that it doesn't describe a phenomena that actually happens in the world. However, it is a response/accusation to which there is never any way to respond to that doesn't confirm the accusation, whether it was true or not. It is also absolutely rooted in what appears to me to be a generational distinction: it seems that a bunch of younger people consider it to be a right to speak "in public" (i.e in any kind of online context where people who do not know you can read what you wrote) and expect to avoid a certain kind of response. Should that response arise? Various things will be said about the responder, including "sea-lioning". My experience is that people who were online in the 80s and 90s find this expectation somewhere between humorous and ridiculous, and that people who went online somewhere after about 2005 do not. Technologically, it seems to reflect a desire among many younger people for "private-public spaces". In the absence of any such actual systems really existing (at least from their POV), they believe they ought to be able to use very non-private public spaces (facebook, insta, and everything else under the rubric of "social media") as they wish to, rather than as the systems were designed. They are communicating with their friends and the fact that their conversations are visible is not significant. Thus, when a random stranger responds to their not-private-public remarks ... sea-lioning. We used to have more systems that were sort-of-private-public spaces - mailing lists being the most obvious. I sympathize with a generation that clearly wants more of these sorts of spaces to communicate with friends, but I am not sympathetic to their insistence that corporate creations that are not just very-much-non-private-public spaces but also essentially revenue generators should work the way they want them to.

by woeirua 8mo ago

acjohnson55 8mo ago

It was actually much less anti LLM than I was expecting from the beginning. But I agree that it is self limiting to not bother to consider the ways that LLM inference and human thinking might be similar (or not). To me, they seem do a pretty reasonable emulation of single- threaded thinking.

Zardoz84 8mo ago
They are not similar. A LLM is a complex statistical machine. A brain is a highly complex neural network. A brain, is more similar the perceptron of some AMD CPUs that to a LLM.
- PaulDavisThe1st 8mo ago
  I would recommend investigating how contemporary LLMs actually work. Possibly start with something like: https://transformer-circuits.pub/2025/attribution-graphs/biology.html https://transformer-circuits.pub/2025/attribution-graphs/bio...

hodgehog11 8mo ago

I disagree that the majority of it is anti-LLM ranting, there are several subtle points here that are grounded in realism. You should read on past the first bit if you're judging mainly from the initial (admittedly naive) first few paragraphs.

woeirua 8mo ago
I read the rest of it. It was intellectually lazy.
- measurablefunc 8mo ago
  It's more intellectually lazy to think boolean logic at a sufficient scale crosses some event horizon wherein its execution on mechanical gadgets called computers somehow adds up to intelligence beyond human understanding.
  - hodgehog11 8mo ago
    It is intellectually lazy to proclaim something to be impossible in the absence of evidence or proof. In the case of the statement made here, it is provably true that Boolean logic at sufficient scale can replicate "intelligence" of any arbitrary degree. It is also easy to show that this can be perceived as an "event horizon" since the measurements of model quality that humans typically like to use are so nonlinear that they are virtually step function-like.
    - measurablefunc 8mo ago
      Doesn't seem like you have proof of anything but it does appear that you have something that is very much like religious faith in an unforeseeable inevitability. Which is fine as far as religion is concerned but it's better to not pretend it's anything other than blind faith. But if you really do have concrete proof of something then you'll have to spell it out better & explain how exactly it adds up to intelligence of such magnitude & scope that no one can make sense of it.
      - hodgehog11 8mo ago
        > "religious faith in an unforeseeable inevitability" For reference, I work in academia, and my job is to find theoretical limitations of neural nets. If there was so much of a modicum of evidence to support the argument that "intelligence" cannot arise from sufficiently large systems, my colleagues and I would be utterly delighted and would be all over it. Here are a couple of standard elements without getting into details: 1. Any "intelligent" agent can be modelled as a random map from environmental input to actions. 2. Any random map can be suitably well-approximated by a generative transformer. This is the universal approximation theorem. Universal approximation does not mean that models of a given class can be trained using data to achieve an arbitrary level of accuracy, however... 3. The neural scaling laws (first empirical, now more theoretically established under NTK-type assumptions), as a refinement of the double descent curve, assert that a neural network class can get arbitrarily close to an "entropy level" given sufficient scale. This theoretical level is so much smaller than any performance metric that humans can reach. Whether "sufficiently large" is outside of the range that is physically possible is a much longer discussion, but bets are that human levels are not out of reach (I don't like this, to be clear). 4. The nonlinearity of accuracy metrics comes from the fact that they are constructed from the intersection of a large number of weakly independent events. Think the CDF of a Beta random variable with parameters tending to infinity. Look, I understand the scepticism, but from where I am, reality isn't leaning that way at the moment. I can't afford to think it isn't possible. I don't think you should either.

wolrah 8mo ago

> I can't imagine why someone would want to openly advertise that they're so closed minded. I would say the exact same about you, rejecting an absolutely accurate and factual statement like that as closed minded strikes me as the same as the people who insist that medical science is closed minded about crystals and magnets. I can't imagine why someone would want to openly advertise they think LLMs are actual intelligence, unless they were in a position to benefit financially from the LLM hype train of course.

woeirua 8mo ago
Cool, so clearly articulate the goal posts. What do LLMs have to do to convince you that they are intelligent? If the answer is there is no amount of evidence that can change your mind, then you're not arguing in good faith.
- rmunn 8mo ago
  Maybe, I don't know, not be based on a statistical model? Come on. If you are actually entertaining the idea that LLMs can possibly be intelligent, you don't know how they work. But to take your silly question seriously for a minute, maybe I might consider LLMs to be capable of intelligence if they were able to learn, if they were able to solve problems that they weren't explicitly trained for. For example, have an LLM read a bunch of books about the strategy of Go, then actually apply that knowledge to beat an experienced Go player who was deliberately playing unconventional, poor strategies like opening in the center. Since pretty much nobody opens their Go game in the center (the corners are far superior), the LLM's training data is NOT going to have a lot of Go openings where one player plays mostly in the center. At which point you'll see that the LLM isn't actually intelligent, because an intelligent being would have understood the concepts in the book that you should mostly play in the corners at first in order to build territory with the smallest number of moves. But when faced with unconventional moves that aren't found anywhere on the Internet, the LLM would just crash and burn. That would be a good test of intelligence. Learning by reading books, and then being able to apply that knowledge to new situations where you can't just regurgitate the training material.

Ygg2 8mo ago

> I can't imagine why someone would want to openly advertise that they're so closed minded. Because humans often anthropomorphize completely inert things? E.g. a coffee machine or a bomb disposal robot. So far whatever behavior LLMs have shown is basically fueled by Sci-Fi stories of how a robot should behave under such and such.

Cloudef 8mo ago

What's wrong about the statement? The black box algorithm might have been generated by machine learning, but it's still a computer program in the end.

km3r 8mo ago
Because it's so entirely reductive and misunderstanding of where the technology has progressed. Hello world is s computer program. So it Microsoft Windows. New levels of "intelligence" unlock with greater complexity of a program. Like look at our brains. We know decently well how a single neuron works. We can simulate a single one with "just a computer program". But clearly with enough layers some form of complexity can emerge, and at some level that complexity becomes intelligence.
- andsoitis 8mo ago
  > with enough layers some form of complexity can emerge, and at some level that complexity becomes intelligence. It isn’t a given that complexity begets intelligence.
  - PaulDavisThe1st 8mo ago
    and it isn't a given that it doesn't, so maybe a little openness towards the possibility is warranted?
    - andsoitis 8mo ago
      I’m open, but the comment I responded to asserted: “complexity becomes intelligence”, as if it is a fact. And it isn’t proven.
      - DiogenesKynikos 8mo ago
        We have LLMs, which are obviously intelligent. How is it not proven?

palmotea 8mo ago

> I can't imagine why someone would want to openly advertise that they're so closed minded. It's not being closed-minded. It's not wanting to get sea-lioned to death by obnoxious people.

PaulDavisThe1st 8mo ago
[WARNING: seriously off-topic comment, I was triggered] Here's what sea-lioned means to me: I say something. You accuse me of sea-lioning. I have two choices: attempt to refute the sea-lioning, which becomes sea-lioning, or allowing your accusation to stand unchallenged, which appears to most people as a confirmation of some kind that I was sea-lioning. It is a nuclear weapon launched at discussion. It isn't that it doesn't describe a phenomena that actually happens in the world. However, it is a response/accusation to which there is never any way to respond to that doesn't confirm the accusation, whether it was true or not. It is also absolutely rooted in what appears to me to be a generational distinction: it seems that a bunch of younger people consider it to be a right to speak "in public" (i.e in any kind of online context where people who do not know you can read what you wrote) and expect to avoid a certain kind of response. Should that response arise? Various things will be said about the responder, including "sea-lioning". My experience is that people who were online in the 80s and 90s find this expectation somewhere between humorous and ridiculous, and that people who went online somewhere after about 2005 do not. Technologically, it seems to reflect a desire among many younger people for "private-public spaces". In the absence of any such actual systems really existing (at least from their POV), they believe they ought to be able to use very non-private public spaces (facebook, insta, and everything else under the rubric of "social media") as they wish to, rather than as the systems were designed. They are communicating with their friends and the fact that their conversations are visible is not significant. Thus, when a random stranger responds to their not-private-public remarks ... sea-lioning. We used to have more systems that were sort-of-private-public spaces - mailing lists being the most obvious. I sympathize with a generation that clearly wants more of these sorts of spaces to communicate with friends, but I am not sympathetic to their insistence that corporate creations that are not just very-much-non-private-public spaces but also essentially revenue generators should work the way they want them to.