3 ms·

> just like you wouldn't judge a blind person's intelligence for not being able to tell if an image is red or blue. I would judge a blind person's intelligence

> just like you wouldn't judge a blind person's intelligence for not being able to tell if an image is red or blue. I would judge a blind person's intelligence if they couldn't remember the last sentence they spoke when specifically asked. Or if they couldn't identify how many people were speaking in a simple audio dialogue. This absolutely says something about their intelligence or reasoning capability. You have this comment: > LLMs don't actually "see" individual input characters, they see tokens, which are subwords. This alone is an indictment of their "reasoning" capability. People are saying these models understand theoretical physics but can't do what a 5 year old can do in the medium of text. It means that these are very much memorization/interpolation devices. Anything approximating reasoning is stepping through interpolation of tokens (and not even symbols) in the text. It means they're a runaway energy minimization algorithm chained to a set of tokens in their attention window, without the ability to reflect upon how any of those words relate to each other outside of syntax and ordering.

kgeist 2y ago
>This alone is an indictment of their "reasoning" capability. I'm not sure why it says anything about their reasoning capability. Some people are blind and can't see anything. Some people are short-sighted and can't see objects which are too far away. Some people have dyslexia. Does it say anything about their reasoning capability? LLMs "perceive" the world through tokens just like blind people perceive the world through touch or sound. Blind people can't discuss color just like LLMs can't count letters. I'm not saying LLM's can actually reason, but I think a different way to perceive the world says nothing about your reasoning capability. Did humans acquire reasoning capabilities only after the invention of the alphabet? A language isn't even required to have an alphabet, see Chinese. The question "how many letters in word X" doesn't make any sense in Chinese. There are character-level LLMs which can see every individual letter, but they're apparently less efficient to train.
- abernard1 2y ago
  The reason it is an indictment of their reasoning capability is that—no matter how much energy is spent trying to say they are not—these are really stochastic parrots: they do not understand the symbols they are operating with. They operate below that level. The fact they can't operate on full symbols reliably but require sub-symbols via tokens is concrete proof of that. They may add heuristics or build more CoT sub-chains to get around some of these trickier issues later, but this is the state of affairs right now. All efforts so far require exponential increases in training size to receive logarithmic increases (at best) in accuracy. And now with o1, it requires exponential compute at inference to scale with that sub-logarithmic accuracy. People have a short memory these days, but around GPT-3, the majority of people on HN and tech "luminary" founders were saying that these would actually have exponential output and diverge. They were wrong. These models are quickly converging to a training set because they are and always were a curve fit. And even there, they are notoriously unreliable for use cases without a human in the loop, because of the intrinsic amount of information entropy that can be packed into the size of these models. But there is nothing mysterious about them.
twobitshifter 2y ago
Would an LLM using character tokens perform better (ignoring performance)?

kgeist 2y ago

>This alone is an indictment of their "reasoning" capability. I'm not sure why it says anything about their reasoning capability. Some people are blind and can't see anything. Some people are short-sighted and can't see objects which are too far away. Some people have dyslexia. Does it say anything about their reasoning capability? LLMs "perceive" the world through tokens just like blind people perceive the world through touch or sound. Blind people can't discuss color just like LLMs can't count letters. I'm not saying LLM's can actually reason, but I think a different way to perceive the world says nothing about your reasoning capability. Did humans acquire reasoning capabilities only after the invention of the alphabet? A language isn't even required to have an alphabet, see Chinese. The question "how many letters in word X" doesn't make any sense in Chinese. There are character-level LLMs which can see every individual letter, but they're apparently less efficient to train.

abernard1 2y ago
The reason it is an indictment of their reasoning capability is that—no matter how much energy is spent trying to say they are not—these are really stochastic parrots: they do not understand the symbols they are operating with. They operate below that level. The fact they can't operate on full symbols reliably but require sub-symbols via tokens is concrete proof of that. They may add heuristics or build more CoT sub-chains to get around some of these trickier issues later, but this is the state of affairs right now. All efforts so far require exponential increases in training size to receive logarithmic increases (at best) in accuracy. And now with o1, it requires exponential compute at inference to scale with that sub-logarithmic accuracy. People have a short memory these days, but around GPT-3, the majority of people on HN and tech "luminary" founders were saying that these would actually have exponential output and diverge. They were wrong. These models are quickly converging to a training set because they are and always were a curve fit. And even there, they are notoriously unreliable for use cases without a human in the loop, because of the intrinsic amount of information entropy that can be packed into the size of these models. But there is nothing mysterious about them.

twobitshifter 2y ago

Would an LLM using character tokens perform better (ignoring performance)?