4 ms·
FWIW I think most of the "tokenization problems" are in fact reasoning problems being falsely blamed on a minor technical thing when the issue is much more prof
by aithrowawaycomm 2y ago
FWIW I think most of the "tokenization problems" are in fact reasoning problems being falsely blamed on a minor technical thing when the issue is much more profound.
E.g. I still see people claiming that LLMs are bad at basic counting because of tokenization, but the same LLM counts perfectly well if you use chain-of-thought prompting. So it can't be explained by tokenization! The problem is reasoning: the LLM needs a human to tell it that a counting problem can be accurately solved if they go step-by-step. Without this assistance the LLM is likely to simply guess.
- ipsum2 2y agoThe more obvious alternative is that CoT is making up for the deficiencies in tokenization, which I believe is the case.
- aithrowawaycomm 2y agoI think the more obvious explanation has to do with computational complexity: counting is an O(n) problem, but transformer LLMs can’t solve O(n) problems unless you use CoT prompting: https://arxiv.org/abs/2310.07923 https://arxiv.org/abs/2310.07923
- ipsum2 2y agoWhat you're saying is an explanation what I said, but I agree with you ;)
- aithrowawaycomm 2y agoNo, it's a rebuttal of what you said: CoT is not making up for a deficiency in tokenization, it's making up for a deficiency in transformers themselves. These complexity results have nothing to do with tokenization, or even LLMs, it is about the complexity class of problems that can be solved by transformers.
- ipsum2 2y agoThere's a really obvious way to test whether the strawberry issue is tokenization - replace each letter with a number, then ask chatGPT to count the number of 3s. Count the number of 3s, only output a single number: 6 5 3 2 8 7 1 3 3 9. ChatGPT: 3.
- MacsHeadroom 2y agoThis paper does not support your position any more than it supports the position that the problem is tokenization. This paper posits that if the authors intuition was true then they would find certain empirical results. ie. "If A then B." Then they test and find the empirical results. But this does not imply that their intuition was correct, just as "If A then B" does not imply "If B then A." If the empirical results were due to tokenization absolutely nothing about this paper would change.
- Der_Einzige 2y agoI’m the one who will fight you including with peer reviewed papers indicating that it is in fact due to tokenization. I’m too tired but will edit this for later, so take this as my bookmark to remind me to respond.
- aithrowawaycomm 2y agoI am aware of errors in computations that can be fixed by better tokenization (e.g. long addition works better tokenizing right-left rather than L-R). But I am talking about counting, and talking about counting words, not characters. I don’t think tokenization explains why LLMs tend to fail at this without CoT prompting. I really think the answer is computational complexity: counting is simply too hard for transformers unless you use CoT. https://arxiv.org/abs/2310.07923 https://arxiv.org/abs/2310.07923
- cma 2y agoWords vs characters is a similar problem, since tokens can be less one word, multiple words, or multiple words and a partial word, or words with non-word punctuation like a sentence ending period.
- Jensson 2y agoWe know there are narrow solutions to these problems, that was never the argument that the specific narrow task is impossible to solve. The discussion is about general intelligence, the model isn't able to do a task that it can do simply because it chooses the wrong strategy, that is a problem of lack of generalization and not a problem of tokenization. Being able to choose the right strategy is core to general intelligence, altering input data to make it easier for the model to find the right solution to specific questions does not help it become more general, you just shift what narrow problems it is good at.
- azeirah 2y agoI strongly believe that the problem isn't that tokenization isn't the underlying problem, it's that, let's say bit-by-bit tokenization is too expensive to run at the scales things are currently being ran at (openai, claude etc)
- TZubiri 2y agoFWIW I think most of the "tokenization problems" List of actual tokenizarion limitations 1- strawberry 2- rhyming and metrics 3- whitespace (as displayed in the article)
- meroes 2y agoAt a certain level they are identical problems. My strongest piece of evidence is that I get paid as an RLHF'er to find ANY case of error, including "tokenization". You know how many errors an LLM gets in the simplest grid puzzles, with CoT, with specialized models that don't try to "one-shot" problems, with multiple models, etc? My assumption is that these large companies wouldn't pay hundreds of thousands of RLHF'ers through dozens of third party companies livable wages if tokenization errors were just that.
- 1propionyl 2y ago> hundreds of thousands of RLHF'ers through dozens of third party companies Out of curiosity, what are these companies? And where do they operate. I'm always interested in these sorts of "hidden" industries. See also: outsourced Facebook content moderation in Kenya.
- meroes 2y agoScale AI is a big one who owns companies who do this as well, such as Outlierai. There are many other AI trainer job companies though. A lot of it is gig work but the pay is more than the vast majority of gig jobs.
- csomar 2y agoIt can count words in a paragraph though. So I do think it's tokenization.
- PittleyDunkin 2y agoI feel like we can set our qualifying standards higher than counting.