4 ms·

>This alone is an indictment of their "reasoning" capability. I'm not sure why it says anything about their reasoning capability. Some people are blind and can

by kgeist 2y ago

>This alone is an indictment of their "reasoning" capability. I'm not sure why it says anything about their reasoning capability. Some people are blind and can't see anything. Some people are short-sighted and can't see objects which are too far away. Some people have dyslexia. Does it say anything about their reasoning capability? LLMs "perceive" the world through tokens just like blind people perceive the world through touch or sound. Blind people can't discuss color just like LLMs can't count letters. I'm not saying LLM's can actually reason, but I think a different way to perceive the world says nothing about your reasoning capability. Did humans acquire reasoning capabilities only after the invention of the alphabet? A language isn't even required to have an alphabet, see Chinese. The question "how many letters in word X" doesn't make any sense in Chinese. There are character-level LLMs which can see every individual letter, but they're apparently less efficient to train.

abernard1 2y ago
The reason it is an indictment of their reasoning capability is that—no matter how much energy is spent trying to say they are not—these are really stochastic parrots: they do not understand the symbols they are operating with. They operate below that level. The fact they can't operate on full symbols reliably but require sub-symbols via tokens is concrete proof of that. They may add heuristics or build more CoT sub-chains to get around some of these trickier issues later, but this is the state of affairs right now. All efforts so far require exponential increases in training size to receive logarithmic increases (at best) in accuracy. And now with o1, it requires exponential compute at inference to scale with that sub-logarithmic accuracy. People have a short memory these days, but around GPT-3, the majority of people on HN and tech "luminary" founders were saying that these would actually have exponential output and diverge. They were wrong. These models are quickly converging to a training set because they are and always were a curve fit. And even there, they are notoriously unreliable for use cases without a human in the loop, because of the intrinsic amount of information entropy that can be packed into the size of these models. But there is nothing mysterious about them.

by kgeist 2y ago

abernard1 2y ago

The reason it is an indictment of their reasoning capability is that—no matter how much energy is spent trying to say they are not—these are really stochastic parrots: they do not understand the symbols they are operating with. They operate below that level. The fact they can't operate on full symbols reliably but require sub-symbols via tokens is concrete proof of that. They may add heuristics or build more CoT sub-chains to get around some of these trickier issues later, but this is the state of affairs right now. All efforts so far require exponential increases in training size to receive logarithmic increases (at best) in accuracy. And now with o1, it requires exponential compute at inference to scale with that sub-logarithmic accuracy. People have a short memory these days, but around GPT-3, the majority of people on HN and tech "luminary" founders were saying that these would actually have exponential output and diverge. They were wrong. These models are quickly converging to a training set because they are and always were a curve fit. And even there, they are notoriously unreliable for use cases without a human in the loop, because of the intrinsic amount of information entropy that can be packed into the size of these models. But there is nothing mysterious about them.