3 ms·
> "LLMs by their very nature cannot perform this task" The issue is that it's not LLMs that can't perform a given task, but that computers already can. Countin
by fragmede 2y ago
> "LLMs by their very nature cannot perform this task"
The issue is that it's not LLMs that can't perform a given task, but that computers already can. Counting the number of Rs in strawberry or comparing 9.11 to 9.7 is trivial for a regular computer program, but hard for an LLM due to the tokenization process. Where LLMs are a pile of matrixes and some math and some look up tables, it's easy to see that as the essential nature of LLMs, which is to say theres no thinking or reasoning happening because it's just a pile of math happening and it's just glorified auto-complete. Artificial things look a lot like the thing they resemble, but they also are artificial, and as such, are markedly different from the thing they resemble. is the very nature of an LLMs being a pile of math mean that it can not perform said task if given more math and more compute and more data? given enough compute, can we change that nature?
I make no prognostication as to whether or not AGI will come from transformers, and this is getting very philosophical, but I see it as irrelevant because I don't believe that AGI is the right measure.