7 ms·

This is a common myth, which I've written about before. https://hn.algolia.com/?dateRange=all&page=0&prefix=true&query=author%3Asillysaurusx%20common%20myth&sor

This is a common myth, which I've written about before. https://hn.algolia.com/?dateRange=all&page=0&prefix=true&query=author%3Asillysaurusx%20common%20myth&sort=byDate&type=comment https://hn.algolia.com/?dateRange=all&page=0&prefix=true&que... The closest anyone's come to proving that byte-level tokenization is better is the ByT5 paper https://arxiv.org/abs/2105.13626 https://arxiv.org/abs/2105.13626 But they only showed evidence for improvement on specific tasks, not general performance, which is an important distinction. And their own benchmarks show that the improvements tend to be marginal: https://i.imgur.com/6Cw0APS.png https://i.imgur.com/6Cw0APS.png One view of the situation is that byte-level access (or "digit-level" in this case) gives a way to accelerate training, and to achieve higher performance with fewer parameters. The model doesn't need to spend as much effort on learning the decompression algorithm (tokenization). But once learned, the tokenization doesn't seem to hinder a model from achieving higher performance, the same way that JPG compression doesn't hinder us from achieving an image that looks very good to humans. It's a bit like arguing an artist would be better if they only operated on raw bitmaps, or that our eyes would be better if our visual cortex didn't do any signal compression. Maybe, but the fact that our eyes do it is pretty strong evidence that compression isn't harmful.

sbierwagen 4y ago
I'm not sure how this is germane? I'm talking about specific tasks: saying if 10000 or 10050 is larger. GPT is demonstrably bad at that. The ByT5 paper doesn't mention arithmetic tasks or show benchmark results for the specific task I mention. Your linked comment says: >This is a common myth but in practice no one (as far as I know) has shown that byte level predictions result in superior overall performance. Stating if BPE or character tokenization is better for everything is a much broader claim, one I didn't make! One could easily imagine a toolformer that calls out to calc.exe for anything involving numbers which would get much better numeric performance while still using BPEs.
- sillysaurusx 4y ago
  > GPT is bad at math because BPE input compression obfuscates individual digits. https://bbot.org/etc/gpt-math.png https://bbot.org/etc/gpt-math.png You'd be bad at math too if every number was scrambled. This is the myth I was referring to. BPE compression may slow down training, but it doesn't follow that slower training is the reason for being bad at math. If you trained GPT specifically on arithmetic tasks, you'd get superior performance to GPT-3, regardless of which tokenization scheme you'd use. But you'd destroy most of its knowledge about everything not-arithmetic.
  - f_devd 4y ago
    > If you trained GPT specifically on arithmetic tasks Sure but you'd have a lot of overlapping tokens with BPE, which doesn't help with convergence. GP is claiming it's specifically worse at arithmetic because of BPE which is true.
  - sbierwagen 4y ago
    >BPE compression may slow down training, but it doesn't follow that slower training is the reason for being bad at math. It's not so much that it slows down training, is that it completely destroys the relationship between digits and results. Every number is assigned a random token ID, so GPT-3 had to memorize every operation separately. It couldn't generalize at all, which is why it got worse at larger numbers, which showed up less often in the training set-- no examples to remember. You can try the tokenizer online here: https://platform.openai.com/tokenizer https://platform.openai.com/tokenizer It assigns the input text `10 11 12 13 14 15 16` token IDs `940, 1367, 1105, 1511, 1478, 1315, 1467`. How is it supposed to figure out incrementing numbers from that? Well, it can't, so it memorizes them. "Neural nets want to work"! I used the past tense above, because while writing this comment I asked ChatGPT Mar 14 Version a bunch of manydigit addition and substraction questions and it got them all right. Then I asked it if one of those large numbers contained an 8 and it... hallucinated a completely wrong answer, oops: https://bbot.org/etc/gpt-math2.png https://bbot.org/etc/gpt-math2.png It's also still spotty at multiplication: "The product of 82368 and 33333 is 2745504384." Well, you got the first five digits right...
    - sillysaurusx 4y ago
      Again, repeating the myth that BPE is the reason for GPT being bad at math doesn't make it true. (Nor does labeling it a myth make it false.) It hasn't been demonstrated that this is the reason. And it's not merely a pedantic point – this is an interesting look into why models behave the way they do. To get us both on the same page, let's restate the arguments. - Your position is that in a hypothetical world where a GPT-3 was trained without BPE, yet somehow still managed to maintain the same context window as BPE (i.e. around three to five pages of text), it would be superior at math, because it can see the individual digits. - My position is that it can still see the same context window, and the context window is all that matters. Regardless of whether you tokenize it using OpenAI BPE or LLaMA BPE or digit level, model performance isn't tied to it. What matters is parameter count, training time, and dataset. If you were to pit the two models against each other, given the same dataset and same amount of training, the BPE model would likely perform significantly better at math. The larger the model, the more it can do, and the better the performance at any individual task. And the BPE model would perform better because the compression allows it to learn at a faster rate -- i.e. the same amount of training computation spent on the BPE'd model gives you much more impact than computation spent on the byte level model. In other words, the larger context window is the whole point. You get 2048 tokens. If you make it 2048 bytes, then you get a very limited view of the input text, which harms overall performance. But if you train it on specifically math problems, then of course it'll achieve superior performance, and not simply because it has an unscrambled view of the digits. As I said before, this is like saying that JPG compression is harmful to our understanding of an image. But it's clearly not. The JPG may be composed of wavelets, but what matters is the overall signal -- and that signal is what GPT is being trained on (the understanding of language), not the individual characters. This also explains why tokenization doesn't seem to affect performance across languages that much. Russian tokenizes into many more characters than English, yet GPT can still achieve pretty good performance on it. This isn't because the characters are scrambled, but rather because it's seen many more examples of English than Russian.
- stormfather 4y ago
  Are you referring to how BPE is permutation invariant? (ignoring positional encoding)
- forgotusername6 4y ago
  Complete tangent here. The current usage of the word germane apparently comes from a line in Hamlet. Try asking chatGPT to quote the line and explain it to you. When I tried it failed utterly, not in the explanation but in returning to the actual quote.
nl 4y ago
http://nlg.csie.ntu.edu.tw/~cjchen/papers/eacl2023.pdf http://nlg.csie.ntu.edu.tw/~cjchen/papers/eacl2023.pdf (which was pointed to downthread) indicates tokenizing and converting to scientific notation gives a 5% increase in performance on a "comparing numbers" benchmark.
PeterisP 4y ago
The general tokenization screws up performance on some very specific niche tasks, which are not really used in training or evaluation, but are used a 'toy examples' by people who demo or test the system, resulting in stupid outcomes that are then (IMHO unfairly) treated as informative of the model's general performance. I.e. "how many letters are in a word" or "which is the fifth letter of this word" type tasks is screwed up because the model has no information about the length of each token other than what it can infer from seeing some words in similar challenges in children learning materials; so it knows the length of some tokens (i.e. those used in simple words in similar materials) but not all of them. "does 123912879212392 contain the digit 7" can fail because the model has limited information that the token "79" in the middle is somehow related to the token "7". If you ask whether "87" is contained in that number, then it has to deduce that from seeing tokens "128" and "79", both of which have no direct relation to the digits in them. "is 50100 greater than 50200" becomes a tricky challenge because due to its tokenization it tries to compare "501"/"00" with "50"/"200" while knowing that "501" is greater than "50"; just as comparing "10000" with "100"/"50". We use Arabic numerals because their positional encoding makes arithmetic easier, but language models receive the same data without positional encoding, they get given something that's more like an extreme version of Roman numerals. I mean, all of it can be learned if given sufficient data, however, we shouldn't be surprised if some tasks seem more difficult than they should be if we have explicitly crippled the representation we use to communicate these tasks to the system. And it doesn't neccesarily mean that we should change the tokenization (because these tasks are toy challenges not relevant to most real functioning), we just shouldn't use these types of letter/digit challenges to judge the quality of models.
mirker 4y ago
I’d argue that tokenization is more analogous to having a losslessly encoded image where you don’t have access to the decoder. Humans don’t perform well on that one.

sbierwagen 4y ago

I'm not sure how this is germane? I'm talking about specific tasks: saying if 10000 or 10050 is larger. GPT is demonstrably bad at that. The ByT5 paper doesn't mention arithmetic tasks or show benchmark results for the specific task I mention. Your linked comment says: >This is a common myth but in practice no one (as far as I know) has shown that byte level predictions result in superior overall performance. Stating if BPE or character tokenization is better for everything is a much broader claim, one I didn't make! One could easily imagine a toolformer that calls out to calc.exe for anything involving numbers which would get much better numeric performance while still using BPEs.

sillysaurusx 4y ago
> GPT is bad at math because BPE input compression obfuscates individual digits. https://bbot.org/etc/gpt-math.png https://bbot.org/etc/gpt-math.png You'd be bad at math too if every number was scrambled. This is the myth I was referring to. BPE compression may slow down training, but it doesn't follow that slower training is the reason for being bad at math. If you trained GPT specifically on arithmetic tasks, you'd get superior performance to GPT-3, regardless of which tokenization scheme you'd use. But you'd destroy most of its knowledge about everything not-arithmetic.
- f_devd 4y ago
  > If you trained GPT specifically on arithmetic tasks Sure but you'd have a lot of overlapping tokens with BPE, which doesn't help with convergence. GP is claiming it's specifically worse at arithmetic because of BPE which is true.
- sbierwagen 4y ago
  >BPE compression may slow down training, but it doesn't follow that slower training is the reason for being bad at math. It's not so much that it slows down training, is that it completely destroys the relationship between digits and results. Every number is assigned a random token ID, so GPT-3 had to memorize every operation separately. It couldn't generalize at all, which is why it got worse at larger numbers, which showed up less often in the training set-- no examples to remember. You can try the tokenizer online here: https://platform.openai.com/tokenizer https://platform.openai.com/tokenizer It assigns the input text `10 11 12 13 14 15 16` token IDs `940, 1367, 1105, 1511, 1478, 1315, 1467`. How is it supposed to figure out incrementing numbers from that? Well, it can't, so it memorizes them. "Neural nets want to work"! I used the past tense above, because while writing this comment I asked ChatGPT Mar 14 Version a bunch of manydigit addition and substraction questions and it got them all right. Then I asked it if one of those large numbers contained an 8 and it... hallucinated a completely wrong answer, oops: https://bbot.org/etc/gpt-math2.png https://bbot.org/etc/gpt-math2.png It's also still spotty at multiplication: "The product of 82368 and 33333 is 2745504384." Well, you got the first five digits right...
  - sillysaurusx 4y ago
    Again, repeating the myth that BPE is the reason for GPT being bad at math doesn't make it true. (Nor does labeling it a myth make it false.) It hasn't been demonstrated that this is the reason. And it's not merely a pedantic point – this is an interesting look into why models behave the way they do. To get us both on the same page, let's restate the arguments. - Your position is that in a hypothetical world where a GPT-3 was trained without BPE, yet somehow still managed to maintain the same context window as BPE (i.e. around three to five pages of text), it would be superior at math, because it can see the individual digits. - My position is that it can still see the same context window, and the context window is all that matters. Regardless of whether you tokenize it using OpenAI BPE or LLaMA BPE or digit level, model performance isn't tied to it. What matters is parameter count, training time, and dataset. If you were to pit the two models against each other, given the same dataset and same amount of training, the BPE model would likely perform significantly better at math. The larger the model, the more it can do, and the better the performance at any individual task. And the BPE model would perform better because the compression allows it to learn at a faster rate -- i.e. the same amount of training computation spent on the BPE'd model gives you much more impact than computation spent on the byte level model. In other words, the larger context window is the whole point. You get 2048 tokens. If you make it 2048 bytes, then you get a very limited view of the input text, which harms overall performance. But if you train it on specifically math problems, then of course it'll achieve superior performance, and not simply because it has an unscrambled view of the digits. As I said before, this is like saying that JPG compression is harmful to our understanding of an image. But it's clearly not. The JPG may be composed of wavelets, but what matters is the overall signal -- and that signal is what GPT is being trained on (the understanding of language), not the individual characters. This also explains why tokenization doesn't seem to affect performance across languages that much. Russian tokenizes into many more characters than English, yet GPT can still achieve pretty good performance on it. This isn't because the characters are scrambled, but rather because it's seen many more examples of English than Russian.
stormfather 4y ago
Are you referring to how BPE is permutation invariant? (ignoring positional encoding)
forgotusername6 4y ago
Complete tangent here. The current usage of the word germane apparently comes from a line in Hamlet. Try asking chatGPT to quote the line and explain it to you. When I tried it failed utterly, not in the explanation but in returning to the actual quote.

nl 4y ago

http://nlg.csie.ntu.edu.tw/~cjchen/papers/eacl2023.pdf http://nlg.csie.ntu.edu.tw/~cjchen/papers/eacl2023.pdf (which was pointed to downthread) indicates tokenizing and converting to scientific notation gives a 5% increase in performance on a "comparing numbers" benchmark.

PeterisP 4y ago

The general tokenization screws up performance on some very specific niche tasks, which are not really used in training or evaluation, but are used a 'toy examples' by people who demo or test the system, resulting in stupid outcomes that are then (IMHO unfairly) treated as informative of the model's general performance. I.e. "how many letters are in a word" or "which is the fifth letter of this word" type tasks is screwed up because the model has no information about the length of each token other than what it can infer from seeing some words in similar challenges in children learning materials; so it knows the length of some tokens (i.e. those used in simple words in similar materials) but not all of them. "does 123912879212392 contain the digit 7" can fail because the model has limited information that the token "79" in the middle is somehow related to the token "7". If you ask whether "87" is contained in that number, then it has to deduce that from seeing tokens "128" and "79", both of which have no direct relation to the digits in them. "is 50100 greater than 50200" becomes a tricky challenge because due to its tokenization it tries to compare "501"/"00" with "50"/"200" while knowing that "501" is greater than "50"; just as comparing "10000" with "100"/"50". We use Arabic numerals because their positional encoding makes arithmetic easier, but language models receive the same data without positional encoding, they get given something that's more like an extreme version of Roman numerals. I mean, all of it can be learned if given sufficient data, however, we shouldn't be surprised if some tasks seem more difficult than they should be if we have explicitly crippled the representation we use to communicate these tasks to the system. And it doesn't neccesarily mean that we should change the tokenization (because these tasks are toy challenges not relevant to most real functioning), we just shouldn't use these types of letter/digit challenges to judge the quality of models.

mirker 4y ago

I’d argue that tokenization is more analogous to having a losslessly encoded image where you don’t have access to the decoder. Humans don’t perform well on that one.