3 ms·

"If a machine has to learn to understand humans to complete text, then that is what it has to do." But the machine doesn't have to understand humans to do that

by krupan 5mo ago

"If a machine has to learn to understand humans to complete text, then that is what it has to do." But the machine doesn't have to understand humans to do that. It gets trained on a whole bunch of sentences and then it is able to complete text. You could maybe claim that it "understands" the text but even that's a stretch.

hn_acc1 5mo ago
It can't even natively understand how many letters there are in words - how will it understand the meaning?
- hackinthebochs 5mo ago
  I wish people would do even the most basic amount of research into LLMs before opining about what they can or cannot do. There are very principled reasons why LLMs do not know how many letters are in words, and it says nothing about their facility for understanding meaning. Tokens are the most basic input unit of an LLM. But tokens don't generally correspond to words or letters, rather sub-word sequences. So Strawberry might be broken up into two tokens 'straw' and 'berry'. It has trouble distinguishing features that are "sub-token" like specific letter sequences because it doesn't see letter sequences but just the token as a single atomic unit. 'Straw' and 'r' are two tokens but an LLM is entirely blind to the fact that 'straw' has one 'r' in it. As an analogy, I might ask you to identify the relative activations of each of the three cone types on your retina as I present some solid color image to your eyes. But of course you can't do this, you simply do not have cognitive access to that information. Individual color experiences are your basic vision tokens. The widespread mistake people keep making is assuming the development of intelligence in LLMs should follow the same trajectory that human intelligence takes as it develops into adult levels of intelligence. Thus deficiency in some capacity that we take for granted in humans is an indictment on LLM intelligence. But this is specious. LLMs are entirely alien; their developmental paths do not and should not look anything like ours. Your intuition from human intelligence just works against understanding the potential for intelligence in LLMs.
  - krapp 5mo ago
    >The widespread mistake people keep making is assuming the development of intelligence in LLMs should follow the same trajectory that human intelligence takes as it develops into adult levels of intelligence. To be fair, almost everyone who claims LLMs are conscious tends to claim that they are conscious in exactly the way that humans are, to the point of stating that human brains are also just complex next-token prediction machines with a random seed. It's basically religious arguments on both sides.
    - Lerc 5mo ago
      Really? That has not been my experience. I have seen people say "you're a next token prediction machine" but only in a similar way one might say "you're a cup of old lard". Not actually meaning it literally. I have seen people interpret the request to show that they are not next token prediction machines to be a claim that they are, but this is almost always an argument to show certainty is difficult in this area. People like Hinton have declared that they believe them to be conscious, but clealy indicate that they do not mean just like us.
      - dpark 5mo ago
        Eh, I’ve seen it. I’m not entirely sure it’s entirely wrong either. Humans are certainly more than just next token predictors but it’s not clear that our typical language behavior is significantly different. We call it “stream of consciousness” when we just spew words out without thinking and that seems to be the default operating mode.
        krapp 5mo ago
        Given the fact that large language models are trained on human language, it shouldn't be surprising that the text they output resembles human language. That is what they're designed to do after all. But similarity in output doesn't necessarily map to similarity in process. And it seem obvious to me that language behavior does differ significantly between humans and LLMs based on the frequency and nature of failure states. LLMs routinely hallucinate, or get "AI strokes" or get obsessed about not talking about goblins, etc. This isn't typical language behavior for humans unless they have severe neurological or psychological impairment. People tend not to "spew words out without thinking" and certainly not all the time by default - we call that glossolalia and (outside of some fringe Christian sects) it's considered a "bug" not a "feature" of the human brain. Human language by default always has intent behind it, even if that intent isn't readily apparent to the speaker. People can recite by rote memory, but that isn't blind token prediction, it's the neurological equivalent of muscle memory. People can have conversations then forget about them because their attention was focused elsewhere, but that doesn't indicate that they were simply "spewing words out without thinking" at the time.
- dpark 5mo ago
  This is kind of a like assuming someone with bad spelling is stupid. Counting letters in a word seems to have little to do with understanding the word. Young kids can’t spell or count well at all but no one says that means they can’t understand.
- lern_too_spel 5mo ago
  You can't natively understand how many of your photoreceptors cells are activated by the period at the end of this sentence. How could you possibly understand the sentence's meaning?
- Jtarii 5mo ago
  This is like saying because humans can't multiply 23472 by 1836736 in less than 5 nanoseconds that they can't possibly understand anything about maths.
tsukikage 5mo ago
Your “and then” is doing a lot of work there. The steps between may or may not include some form of “learn to understand humans”, but you can’t just hide them behind “and then” if what we are doing is claiming some particular thing is not in the list. Through training on human text, we are building implicitly in the weights a statistical model of what humans might write in response when presented with arbitrary pieces of text. It turns out that we can make these incredibly accurate. If building an accurate internal model of something then using it to predict that thing’s behaviour is different to gaining understanding of that thing, we will need to pin down exactly what “understanding” means, or we are forever doomed to talk at cross purposes.
- krupan 5mo ago
  My "and then" simply implies order of operations. When it's fully "trained" then (and only then) can it generate text. And I will reassert that even if it "understands" the text it was trained on, that is not the same as understanding humans. I mean really, we ARE humans and we barely understand humans. The thing LLMs model and "predict" is simply, what words in what order are statistically common given these input words in this order. You can write (non-ai) software to model and predict things using the laws of physics. I'd wager it would do a better job than any LLM at predicting where a rocket will go through space. Does that mean the program is conscious and "understands" physics? No

by krupan 5mo ago

hn_acc1 5mo ago

It can't even natively understand how many letters there are in words - how will it understand the meaning?

hackinthebochs 5mo ago
I wish people would do even the most basic amount of research into LLMs before opining about what they can or cannot do. There are very principled reasons why LLMs do not know how many letters are in words, and it says nothing about their facility for understanding meaning. Tokens are the most basic input unit of an LLM. But tokens don't generally correspond to words or letters, rather sub-word sequences. So Strawberry might be broken up into two tokens 'straw' and 'berry'. It has trouble distinguishing features that are "sub-token" like specific letter sequences because it doesn't see letter sequences but just the token as a single atomic unit. 'Straw' and 'r' are two tokens but an LLM is entirely blind to the fact that 'straw' has one 'r' in it. As an analogy, I might ask you to identify the relative activations of each of the three cone types on your retina as I present some solid color image to your eyes. But of course you can't do this, you simply do not have cognitive access to that information. Individual color experiences are your basic vision tokens. The widespread mistake people keep making is assuming the development of intelligence in LLMs should follow the same trajectory that human intelligence takes as it develops into adult levels of intelligence. Thus deficiency in some capacity that we take for granted in humans is an indictment on LLM intelligence. But this is specious. LLMs are entirely alien; their developmental paths do not and should not look anything like ours. Your intuition from human intelligence just works against understanding the potential for intelligence in LLMs.
- krapp 5mo ago
  >The widespread mistake people keep making is assuming the development of intelligence in LLMs should follow the same trajectory that human intelligence takes as it develops into adult levels of intelligence. To be fair, almost everyone who claims LLMs are conscious tends to claim that they are conscious in exactly the way that humans are, to the point of stating that human brains are also just complex next-token prediction machines with a random seed. It's basically religious arguments on both sides.
  - Lerc 5mo ago
    Really? That has not been my experience. I have seen people say "you're a next token prediction machine" but only in a similar way one might say "you're a cup of old lard". Not actually meaning it literally. I have seen people interpret the request to show that they are not next token prediction machines to be a claim that they are, but this is almost always an argument to show certainty is difficult in this area. People like Hinton have declared that they believe them to be conscious, but clealy indicate that they do not mean just like us.
    - dpark 5mo ago
      Eh, I’ve seen it. I’m not entirely sure it’s entirely wrong either. Humans are certainly more than just next token predictors but it’s not clear that our typical language behavior is significantly different. We call it “stream of consciousness” when we just spew words out without thinking and that seems to be the default operating mode.
      - krapp 5mo ago
        Given the fact that large language models are trained on human language, it shouldn't be surprising that the text they output resembles human language. That is what they're designed to do after all. But similarity in output doesn't necessarily map to similarity in process. And it seem obvious to me that language behavior does differ significantly between humans and LLMs based on the frequency and nature of failure states. LLMs routinely hallucinate, or get "AI strokes" or get obsessed about not talking about goblins, etc. This isn't typical language behavior for humans unless they have severe neurological or psychological impairment. People tend not to "spew words out without thinking" and certainly not all the time by default - we call that glossolalia and (outside of some fringe Christian sects) it's considered a "bug" not a "feature" of the human brain. Human language by default always has intent behind it, even if that intent isn't readily apparent to the speaker. People can recite by rote memory, but that isn't blind token prediction, it's the neurological equivalent of muscle memory. People can have conversations then forget about them because their attention was focused elsewhere, but that doesn't indicate that they were simply "spewing words out without thinking" at the time.
dpark 5mo ago
This is kind of a like assuming someone with bad spelling is stupid. Counting letters in a word seems to have little to do with understanding the word. Young kids can’t spell or count well at all but no one says that means they can’t understand.
lern_too_spel 5mo ago
You can't natively understand how many of your photoreceptors cells are activated by the period at the end of this sentence. How could you possibly understand the sentence's meaning?
Jtarii 5mo ago
This is like saying because humans can't multiply 23472 by 1836736 in less than 5 nanoseconds that they can't possibly understand anything about maths.

tsukikage 5mo ago

Your “and then” is doing a lot of work there. The steps between may or may not include some form of “learn to understand humans”, but you can’t just hide them behind “and then” if what we are doing is claiming some particular thing is not in the list. Through training on human text, we are building implicitly in the weights a statistical model of what humans might write in response when presented with arbitrary pieces of text. It turns out that we can make these incredibly accurate. If building an accurate internal model of something then using it to predict that thing’s behaviour is different to gaining understanding of that thing, we will need to pin down exactly what “understanding” means, or we are forever doomed to talk at cross purposes.

krupan 5mo ago
My "and then" simply implies order of operations. When it's fully "trained" then (and only then) can it generate text. And I will reassert that even if it "understands" the text it was trained on, that is not the same as understanding humans. I mean really, we ARE humans and we barely understand humans. The thing LLMs model and "predict" is simply, what words in what order are statistically common given these input words in this order. You can write (non-ai) software to model and predict things using the laws of physics. I'd wager it would do a better job than any LLM at predicting where a rocket will go through space. Does that mean the program is conscious and "understands" physics? No