5 ms·

I currently operate under the assumption that humans are at most as powerful as Turing Machines. And from what I understand these models internally are modeling

by mooreat 4mo ago

I currently operate under the assumption that humans are at most as powerful as Turing Machines. And from what I understand these models internally are modeling increasingly harder and larger DFAs, so they're at least as powerful as regular languages. Assuming humans are more powerful than regular languages I could maybe agree that these methods may not eventually yield entirely human like intelligence, but just better and better approximations. The vibe I get though is that we aren't more powerful than regular languages, cause human beings feel computationally bounded. So I could see given enough "human signal" these things could learn to imitate us precisely.

davebren 4mo ago
Well yeah there is likely an equivalence between computability and epistemology, but I'm not sure it matters when comparing LLM intelligence to human intelligence. There is clearly a missing link that prevents the LLM from reaching beyond its training data the way humans do.
- virgildotcodes 4mo ago
  If you look at the life efforts and accomplishments of the ~100 billion humans who have ever lived, how many lifetimes would you discount as having "non-human intelligence" based on the lack of "novel" contributions to frontier of our species' scientific understanding according to the same high bar you apply to LLMs? Do you pass that bar yourself?
  - davebren 4mo ago
    Ordinary humans do novel things all the time. Where do you think LLMs got all the training data that their responses come from?
    - virgildotcodes 4mo ago
      You're not quite addressing the question. More and more of the training data is now synthetic. To be very specific - what novel things did the majority of the ~8 bil humans on Earth do say, yesterday, that you wouldn't otherwise dismiss as non-intelligent rehashing of the same tired patterns they always inhabit were those same actions attributed to LLMs? What I'm getting at is that I think you're falling into the trap of thinking of the rare geniuses of human history, and furthermore their rare moments of accomplishment (relative to the long span of their lifetimes filled mostly without these accomplishments) when you think of "human intelligence", which is of course far overstating what actual human intelligence is.
      - davebren 4mo ago
        Synthetic training data is carefully crafted by humans. The rare geniuses of human history use a different magnitude and configuration of the same kind of human intelligence that posted a dad joke on a site that got scraped into the training set and repeated, convincing people that it is intelligent like humans. > that you wouldn't otherwise dismiss as non-intelligent rehashing of the same tired patterns they always inhabit were those same actions attributed to LLMs? Regardless of whether something's been done before people still come up with them on their own without directly copying or amalgamating several copies. Pretty much every skilled profession includes figuring things out on the fly through the use of general reasoning that doesn't involve pattern matching against millions of examples.
        virgildotcodes 4mo ago
        > Synthetic training data is carefully crafted by humans. Much, if not the majority of synthetic data is AI generated. Human experts then evaluate samples of the data, but nothing like the entire corpus which can be trillions of tokens of generated material. See here where Qwen team discusses synthesizing trillions of tokens for their pre training dataset - https://arxiv.org/html/2505.09388v1 https://arxiv.org/html/2505.09388v1 > The rare geniuses of human history use a different magnitude and configuration of the same kind of human intelligence I agree. What I don’t see any strong evidence for is that this intelligence is unique to humans. Nor do I see how it could ever be anything other than recombinations of existing data with random mutation. Where else would the building blocks for each invention come from, divine insight? We build on the shoulders of giants etc etc Worth noting, as a sidebar, that we’re having this discussion on a post mentioning a novel breakthrough made by AI over a topic that many brilliant human mathematicians including Erdos himself failed to do. > Regardless of whether something's been done before people still come up with them on their own without directly copying or amalgamating several copies. I’m not even saying it in the “there’s nothing new under the sun” sense. If you follow an average person’s day from beginning to end. Let’s say in Bangkok or NYC or Paris, at which part of the day are they not simply repeating a variation of something they’ve done many times before, or seen others around them do before, or read about others doing before, or heard about others doing before, watched others do before on TV etc etc What you have left, how is it distinguishable, without reasoning backwards from the desired conclusion of human exceptionalism, from turning up the temperature on an LLM query? How many data points does a human parse when they attempt to stand up as a toddler? Sight, sound, sensation from every limb and body part, inner ear, internal thought processes at the time conscious and unconscious related to the moment and attempting to interpret it in relation to all that it’s experienced to this point, including all prior attempts and whatever retained associated data, a hard to even comprehend stream of data, coming in continuously over however many minutes, hours, etc of attempts. The stream of data the brain is processing from both external and internal sources from birth is incredibly rich, and if we attempted to represent the full depth of it it would far outweigh the size of any corpus models are being trained on now. I think what may be genuinely missing from AI is the type of data that doesn’t translate completely into text. The audio and images/video we feed in are a totally incomplete slice of the POV of say even a single average human through their lifetime, and bereft of all the associated data a human has access to in the moment (sensory etc). I think this tends more towards the world models that Yann Lecun et al are promoting as the key to more capable AI.
- killerstorm 4mo ago
  The act of discovery is usually associated with "abductive reasoning", i.e. finding a novel pattern in data. Usually people point out that humans are more sample efficient: they might notice a novel pattern in a handful of samples, whereas training NN might require take millions. However a claim that LLMs fundamentally cannot do abductive reasoning at all is not warranted - we don't see a clear cut, it just looks like the way LLMs do it is less efficient.

by mooreat 4mo ago

davebren 4mo ago

Well yeah there is likely an equivalence between computability and epistemology, but I'm not sure it matters when comparing LLM intelligence to human intelligence. There is clearly a missing link that prevents the LLM from reaching beyond its training data the way humans do.

virgildotcodes 4mo ago
If you look at the life efforts and accomplishments of the ~100 billion humans who have ever lived, how many lifetimes would you discount as having "non-human intelligence" based on the lack of "novel" contributions to frontier of our species' scientific understanding according to the same high bar you apply to LLMs? Do you pass that bar yourself?
- davebren 4mo ago
  Ordinary humans do novel things all the time. Where do you think LLMs got all the training data that their responses come from?
  - virgildotcodes 4mo ago
    You're not quite addressing the question. More and more of the training data is now synthetic. To be very specific - what novel things did the majority of the ~8 bil humans on Earth do say, yesterday, that you wouldn't otherwise dismiss as non-intelligent rehashing of the same tired patterns they always inhabit were those same actions attributed to LLMs? What I'm getting at is that I think you're falling into the trap of thinking of the rare geniuses of human history, and furthermore their rare moments of accomplishment (relative to the long span of their lifetimes filled mostly without these accomplishments) when you think of "human intelligence", which is of course far overstating what actual human intelligence is.
    - davebren 4mo ago
      Synthetic training data is carefully crafted by humans. The rare geniuses of human history use a different magnitude and configuration of the same kind of human intelligence that posted a dad joke on a site that got scraped into the training set and repeated, convincing people that it is intelligent like humans. > that you wouldn't otherwise dismiss as non-intelligent rehashing of the same tired patterns they always inhabit were those same actions attributed to LLMs? Regardless of whether something's been done before people still come up with them on their own without directly copying or amalgamating several copies. Pretty much every skilled profession includes figuring things out on the fly through the use of general reasoning that doesn't involve pattern matching against millions of examples.
      - virgildotcodes 4mo ago
        > Synthetic training data is carefully crafted by humans. Much, if not the majority of synthetic data is AI generated. Human experts then evaluate samples of the data, but nothing like the entire corpus which can be trillions of tokens of generated material. See here where Qwen team discusses synthesizing trillions of tokens for their pre training dataset - https://arxiv.org/html/2505.09388v1 https://arxiv.org/html/2505.09388v1 > The rare geniuses of human history use a different magnitude and configuration of the same kind of human intelligence I agree. What I don’t see any strong evidence for is that this intelligence is unique to humans. Nor do I see how it could ever be anything other than recombinations of existing data with random mutation. Where else would the building blocks for each invention come from, divine insight? We build on the shoulders of giants etc etc Worth noting, as a sidebar, that we’re having this discussion on a post mentioning a novel breakthrough made by AI over a topic that many brilliant human mathematicians including Erdos himself failed to do. > Regardless of whether something's been done before people still come up with them on their own without directly copying or amalgamating several copies. I’m not even saying it in the “there’s nothing new under the sun” sense. If you follow an average person’s day from beginning to end. Let’s say in Bangkok or NYC or Paris, at which part of the day are they not simply repeating a variation of something they’ve done many times before, or seen others around them do before, or read about others doing before, or heard about others doing before, watched others do before on TV etc etc What you have left, how is it distinguishable, without reasoning backwards from the desired conclusion of human exceptionalism, from turning up the temperature on an LLM query? How many data points does a human parse when they attempt to stand up as a toddler? Sight, sound, sensation from every limb and body part, inner ear, internal thought processes at the time conscious and unconscious related to the moment and attempting to interpret it in relation to all that it’s experienced to this point, including all prior attempts and whatever retained associated data, a hard to even comprehend stream of data, coming in continuously over however many minutes, hours, etc of attempts. The stream of data the brain is processing from both external and internal sources from birth is incredibly rich, and if we attempted to represent the full depth of it it would far outweigh the size of any corpus models are being trained on now. I think what may be genuinely missing from AI is the type of data that doesn’t translate completely into text. The audio and images/video we feed in are a totally incomplete slice of the POV of say even a single average human through their lifetime, and bereft of all the associated data a human has access to in the moment (sensory etc). I think this tends more towards the world models that Yann Lecun et al are promoting as the key to more capable AI.
killerstorm 4mo ago
The act of discovery is usually associated with "abductive reasoning", i.e. finding a novel pattern in data. Usually people point out that humans are more sample efficient: they might notice a novel pattern in a handful of samples, whereas training NN might require take millions. However a claim that LLMs fundamentally cannot do abductive reasoning at all is not warranted - we don't see a clear cut, it just looks like the way LLMs do it is less efficient.