6 ms·

>>Is 10000 bigger than 10050? >>Yes, 10000 is bigger than 10050. >But even the mighty ChatGPT often can't do simple math GPT is bad at math because BPE input

>>Is 10000 bigger than 10050? >>Yes, 10000 is bigger than 10050. >But even the mighty ChatGPT often can't do simple math GPT is bad at math because BPE input compression obfuscates individual digits. https://bbot.org/etc/gpt-math.png https://bbot.org/etc/gpt-math.png You'd be bad at math too if every number was scrambled. The graph is from page 22 of the GPT-3 paper from 2020. https://arxiv.org/abs/2005.14165 https://arxiv.org/abs/2005.14165 Even with 175 billion parameters it can't reliably do four digit addition. An example from 4 days ago of ChatGPT being as bad as you'd expect at string reversal: https://news.ycombinator.com/item?id=35297183 https://news.ycombinator.com/item?id=35297183 (Although, I just tested ChatGPT Mar 14 Version against the above question after doing a bunch of math prompting and it got it right...)

cma 4y ago
Llama individually tokenized digits, how much did it fix the issue?
- nl 4y ago
  The most relevant reported benchmark results are on https://arxiv.org/abs/2103.03874 https://arxiv.org/abs/2103.03874 They compare it against PaLM and Minerva, both of which are fine tuned on math datasets. It outperforms both at roughly the same parameter count (eg Llama 65B vs Palm and Minerva 62B) but unclear how much of this is due to encoding vs the many other differences. It is useful to see that the performance increase is clearly not due to fine turning though.
  - cma 4y ago
    Where do I look, the paper was from 2021 before llama but did they update it (or you mean the llama paper mentions that benchmark)?
    - nl 4y ago
      I posted the wrong link sorry! https://arxiv.org/pdf/2302.13971.pdf https://arxiv.org/pdf/2302.13971.pdf (the Llama paper), table 7.
sillysaurusx 4y ago
This is a common myth, which I've written about before. https://hn.algolia.com/?dateRange=all&page=0&prefix=true&query=author%3Asillysaurusx%20common%20myth&sort=byDate&type=comment https://hn.algolia.com/?dateRange=all&page=0&prefix=true&que... The closest anyone's come to proving that byte-level tokenization is better is the ByT5 paper https://arxiv.org/abs/2105.13626 https://arxiv.org/abs/2105.13626 But they only showed evidence for improvement on specific tasks, not general performance, which is an important distinction. And their own benchmarks show that the improvements tend to be marginal: https://i.imgur.com/6Cw0APS.png https://i.imgur.com/6Cw0APS.png One view of the situation is that byte-level access (or "digit-level" in this case) gives a way to accelerate training, and to achieve higher performance with fewer parameters. The model doesn't need to spend as much effort on learning the decompression algorithm (tokenization). But once learned, the tokenization doesn't seem to hinder a model from achieving higher performance, the same way that JPG compression doesn't hinder us from achieving an image that looks very good to humans. It's a bit like arguing an artist would be better if they only operated on raw bitmaps, or that our eyes would be better if our visual cortex didn't do any signal compression. Maybe, but the fact that our eyes do it is pretty strong evidence that compression isn't harmful.
- sbierwagen 4y ago
  I'm not sure how this is germane? I'm talking about specific tasks: saying if 10000 or 10050 is larger. GPT is demonstrably bad at that. The ByT5 paper doesn't mention arithmetic tasks or show benchmark results for the specific task I mention. Your linked comment says: >This is a common myth but in practice no one (as far as I know) has shown that byte level predictions result in superior overall performance. Stating if BPE or character tokenization is better for everything is a much broader claim, one I didn't make! One could easily imagine a toolformer that calls out to calc.exe for anything involving numbers which would get much better numeric performance while still using BPEs.
  - sillysaurusx 4y ago
    > GPT is bad at math because BPE input compression obfuscates individual digits. https://bbot.org/etc/gpt-math.png https://bbot.org/etc/gpt-math.png You'd be bad at math too if every number was scrambled. This is the myth I was referring to. BPE compression may slow down training, but it doesn't follow that slower training is the reason for being bad at math. If you trained GPT specifically on arithmetic tasks, you'd get superior performance to GPT-3, regardless of which tokenization scheme you'd use. But you'd destroy most of its knowledge about everything not-arithmetic.
spyder 4y ago
There was a paper where they have found that converting the numbers to scientific notation ( like 1.5e-7) has improved these transformer-based language models at math, if I remember correctly. (with a quick search I could not find the link to it now)
- nl 4y ago
  Interesting. It might be http://nlg.csie.ntu.edu.tw/~cjchen/papers/eacl2023.pdf http://nlg.csie.ntu.edu.tw/~cjchen/papers/eacl2023.pdf They show an increase from 65% to 70% on their "comparing numbers" benchmark.
  - spyder 4y ago
    That's also an interesting one, but I think I found it, but it's a little different from what I remembered because it's not just the scientific notation that helps but more so the addition of positional tokens: https://arxiv.org/abs/2102.13019 https://arxiv.org/abs/2102.13019 The essence of it from the abstract: ". In particular, the model fails to learn addition of five-digit numbers when using subwords (e.g., “32”), and it struggles to learn with character-level representations (e.g., “3 2”). By introducing position tokens (e.g., “3 10e1 2”), the model learns to accurately add and subtract numbers up to 60 digits. We conclude that modern pretrained language models can easily learn arithmetic from very few examples, as long as we use the proper surface representation."
    - nl 4y ago
      That's a really interesting paper too. Thanks!
croddin 4y ago
I'm not sure if these models use the GPT tokenizer, but if you type a long string of numbers into https://platform.openai.com/tokenizer https://platform.openai.com/tokenizer, you can see the tokens that the LLM would see. What the LLMs get as input for math is significantly worse then having to do mental math with roman numerals, tokenizing makes sense for words but for numbers it seems the like LLMs would have to learn a lot more steps. I wonder if limiting number tokens to 2 digits per token, instead of the 1-3 (and even more for repeated digits) it currently is would improve models math.
- deleted 4y ago
  [deleted]
- deleted 4y ago
  [deleted]
MuffinFlavored 4y ago
as an LLM how do we train it to know when to call a “math” plugin/API? (feed its input, go get output)
- sebzim4500 4y ago
  Either finetune with a few thousand example, or just tell it in the prompt to call the API when it needs to do maths (in context learning).
- discordance 4y ago
  In the pipeline, first run a classification model to determine what class of question is being asked. Then engage the most appropriate plugins.
thequadehunter 4y ago
I've been wondering...does it even matter at this point when you can just pull out a calculator in langchain? Also side note...I went to high school with the author and hadn't heard about him for a while lol. That was a neat surprise.
- spaethnl 4y ago
  I suspect it does matter because the inability to understand arithmetic likely effects higher level learning. For example, If you want it to learn statistics, it likely needs a good understanding of arithmetic. All tasks that rely on it will perform worse than expected.
  - thequadehunter 4y ago
    Oh, I see. It's about a cap in baseline ability rather than whether it's technically possible to give it the extension?
- est 4y ago
  > does it even matter at this point when you can just pull out a calculator in langchain? That's alrithmetics. For math problems: Each alien chicken has 3 legs, each alien rabbit has 7 legs, for a total of 33 legs, how many alien chicken are there? Now try to calculator that!
  - thequadehunter 4y ago
    Apparently it can. https://writings.stephenwolfram.com/2023/03/chatgpt-gets-its-wolfram-superpowers/ https://writings.stephenwolfram.com/2023/03/chatgpt-gets-its...
  - sbierwagen 4y ago
    A couple people have used LLMs as frontends to theorem provers/SAT solvers, which would handle that fairly easily: https://twitter.com/AlbertQJiang/status/1584877475502301184 https://twitter.com/AlbertQJiang/status/1584877475502301184 https://twitter.com/ericzelikman/status/1605385835624378368 https://twitter.com/ericzelikman/status/1605385835624378368
jquery 4y ago
I just fed this to ChatGPT 4. Q: “Is 10000 bigger than 10050” A: “No, 10,000 is not bigger than 10,050. 10,050 is 50 units larger than 10,000”
- bob1029 4y ago
  The more parameters you have, the less error you will find on these prompts. This is the emergent frontier.
- sbierwagen 4y ago
  Tried GPT4ALL-finetuned LLaMA-7B: >>> Is 10000 bigger than 10050? >Yes, 10000 is greater in magnitude than or exceeds the value of 10050. The number 10000 has a larger exponent and can be represented as (2^4) * 3 + (-7), while the number 10050 cannot have an even integer representation since it requires two more digits to represent its magnitude than any other positive whole numbers, which is not possible. Well that's not right. Restarted with a new seed. >>> Is 10000 bigger than 10050? >Yes, because 10000 > 10050 >>> Is 10,000 bigger than 10,050? >No, they are equal. The difference between them is only one digit (i.e., the last number). Therefore, it's not possible to tell if either value is greater or lesser by just looking at their values without knowing more information about what those numbers represent and how they were obtained in the first place. Oooof. The LLaMA tokenizer didn't save it here.
  - jquery 4y ago
    Ok those are funny
  - rikimaru0345 4y ago
    > The difference between them is only one digit (i.e., the last number). Therefore, it's not possible to tell if either value is greater or lesser by just looking at their values without knowing more information about what those numbers represent and how they were obtained in the first place. That one is especially hilarious. But the part at the end "how they were obtained" is really strange. Where in its dataset would it possibly have learned such nonsense? Doesn't matter where numbers come from to compare them. It implies that it doesn't understand what numbers even are in general, and that giving it a calculator (that it can use perfectly) only masks a much deeper problem. I mean, I'm reading a ton of people say "its not just pattern recognition and token prediction, it has emergent properties!!!!" and from experimentation I believe it. But if the models can pick up language and its intricacies, and even do simple logic tasks, shouldn't it also be able to pick up on what numbers are and how they work? At least knowing that where a number came from doesn't matter when its just about comparing their value in a pure mathematical sense? What does that mean for concepts other than numbers? Do those models fake a LOT more than we already believe they do?

cma 4y ago

Llama individually tokenized digits, how much did it fix the issue?

nl 4y ago
The most relevant reported benchmark results are on https://arxiv.org/abs/2103.03874 https://arxiv.org/abs/2103.03874 They compare it against PaLM and Minerva, both of which are fine tuned on math datasets. It outperforms both at roughly the same parameter count (eg Llama 65B vs Palm and Minerva 62B) but unclear how much of this is due to encoding vs the many other differences. It is useful to see that the performance increase is clearly not due to fine turning though.
- cma 4y ago
  Where do I look, the paper was from 2021 before llama but did they update it (or you mean the llama paper mentions that benchmark)?
  - nl 4y ago
    I posted the wrong link sorry! https://arxiv.org/pdf/2302.13971.pdf https://arxiv.org/pdf/2302.13971.pdf (the Llama paper), table 7.

sillysaurusx 4y ago

This is a common myth, which I've written about before. https://hn.algolia.com/?dateRange=all&page=0&prefix=true&query=author%3Asillysaurusx%20common%20myth&sort=byDate&type=comment https://hn.algolia.com/?dateRange=all&page=0&prefix=true&que... The closest anyone's come to proving that byte-level tokenization is better is the ByT5 paper https://arxiv.org/abs/2105.13626 https://arxiv.org/abs/2105.13626 But they only showed evidence for improvement on specific tasks, not general performance, which is an important distinction. And their own benchmarks show that the improvements tend to be marginal: https://i.imgur.com/6Cw0APS.png https://i.imgur.com/6Cw0APS.png One view of the situation is that byte-level access (or "digit-level" in this case) gives a way to accelerate training, and to achieve higher performance with fewer parameters. The model doesn't need to spend as much effort on learning the decompression algorithm (tokenization). But once learned, the tokenization doesn't seem to hinder a model from achieving higher performance, the same way that JPG compression doesn't hinder us from achieving an image that looks very good to humans. It's a bit like arguing an artist would be better if they only operated on raw bitmaps, or that our eyes would be better if our visual cortex didn't do any signal compression. Maybe, but the fact that our eyes do it is pretty strong evidence that compression isn't harmful.

sbierwagen 4y ago
I'm not sure how this is germane? I'm talking about specific tasks: saying if 10000 or 10050 is larger. GPT is demonstrably bad at that. The ByT5 paper doesn't mention arithmetic tasks or show benchmark results for the specific task I mention. Your linked comment says: >This is a common myth but in practice no one (as far as I know) has shown that byte level predictions result in superior overall performance. Stating if BPE or character tokenization is better for everything is a much broader claim, one I didn't make! One could easily imagine a toolformer that calls out to calc.exe for anything involving numbers which would get much better numeric performance while still using BPEs.
- sillysaurusx 4y ago
  > GPT is bad at math because BPE input compression obfuscates individual digits. https://bbot.org/etc/gpt-math.png https://bbot.org/etc/gpt-math.png You'd be bad at math too if every number was scrambled. This is the myth I was referring to. BPE compression may slow down training, but it doesn't follow that slower training is the reason for being bad at math. If you trained GPT specifically on arithmetic tasks, you'd get superior performance to GPT-3, regardless of which tokenization scheme you'd use. But you'd destroy most of its knowledge about everything not-arithmetic.

spyder 4y ago

There was a paper where they have found that converting the numbers to scientific notation ( like 1.5e-7) has improved these transformer-based language models at math, if I remember correctly. (with a quick search I could not find the link to it now)

nl 4y ago
Interesting. It might be http://nlg.csie.ntu.edu.tw/~cjchen/papers/eacl2023.pdf http://nlg.csie.ntu.edu.tw/~cjchen/papers/eacl2023.pdf They show an increase from 65% to 70% on their "comparing numbers" benchmark.
- spyder 4y ago
  That's also an interesting one, but I think I found it, but it's a little different from what I remembered because it's not just the scientific notation that helps but more so the addition of positional tokens: https://arxiv.org/abs/2102.13019 https://arxiv.org/abs/2102.13019 The essence of it from the abstract: ". In particular, the model fails to learn addition of five-digit numbers when using subwords (e.g., “32”), and it struggles to learn with character-level representations (e.g., “3 2”). By introducing position tokens (e.g., “3 10e1 2”), the model learns to accurately add and subtract numbers up to 60 digits. We conclude that modern pretrained language models can easily learn arithmetic from very few examples, as long as we use the proper surface representation."
  - nl 4y ago
    That's a really interesting paper too. Thanks!

croddin 4y ago

I'm not sure if these models use the GPT tokenizer, but if you type a long string of numbers into https://platform.openai.com/tokenizer https://platform.openai.com/tokenizer, you can see the tokens that the LLM would see. What the LLMs get as input for math is significantly worse then having to do mental math with roman numerals, tokenizing makes sense for words but for numbers it seems the like LLMs would have to learn a lot more steps. I wonder if limiting number tokens to 2 digits per token, instead of the 1-3 (and even more for repeated digits) it currently is would improve models math.

deleted 4y ago
[deleted]
deleted 4y ago
[deleted]

MuffinFlavored 4y ago

as an LLM how do we train it to know when to call a “math” plugin/API? (feed its input, go get output)

sebzim4500 4y ago
Either finetune with a few thousand example, or just tell it in the prompt to call the API when it needs to do maths (in context learning).
discordance 4y ago
In the pipeline, first run a classification model to determine what class of question is being asked. Then engage the most appropriate plugins.

thequadehunter 4y ago

I've been wondering...does it even matter at this point when you can just pull out a calculator in langchain? Also side note...I went to high school with the author and hadn't heard about him for a while lol. That was a neat surprise.

spaethnl 4y ago
I suspect it does matter because the inability to understand arithmetic likely effects higher level learning. For example, If you want it to learn statistics, it likely needs a good understanding of arithmetic. All tasks that rely on it will perform worse than expected.
- thequadehunter 4y ago
  Oh, I see. It's about a cap in baseline ability rather than whether it's technically possible to give it the extension?
est 4y ago
> does it even matter at this point when you can just pull out a calculator in langchain? That's alrithmetics. For math problems: Each alien chicken has 3 legs, each alien rabbit has 7 legs, for a total of 33 legs, how many alien chicken are there? Now try to calculator that!
- thequadehunter 4y ago
  Apparently it can. https://writings.stephenwolfram.com/2023/03/chatgpt-gets-its-wolfram-superpowers/ https://writings.stephenwolfram.com/2023/03/chatgpt-gets-its...
- sbierwagen 4y ago
  A couple people have used LLMs as frontends to theorem provers/SAT solvers, which would handle that fairly easily: https://twitter.com/AlbertQJiang/status/1584877475502301184 https://twitter.com/AlbertQJiang/status/1584877475502301184 https://twitter.com/ericzelikman/status/1605385835624378368 https://twitter.com/ericzelikman/status/1605385835624378368

jquery 4y ago

I just fed this to ChatGPT 4. Q: “Is 10000 bigger than 10050” A: “No, 10,000 is not bigger than 10,050. 10,050 is 50 units larger than 10,000”

bob1029 4y ago
The more parameters you have, the less error you will find on these prompts. This is the emergent frontier.
sbierwagen 4y ago
Tried GPT4ALL-finetuned LLaMA-7B: >>> Is 10000 bigger than 10050? >Yes, 10000 is greater in magnitude than or exceeds the value of 10050. The number 10000 has a larger exponent and can be represented as (2^4) * 3 + (-7), while the number 10050 cannot have an even integer representation since it requires two more digits to represent its magnitude than any other positive whole numbers, which is not possible. Well that's not right. Restarted with a new seed. >>> Is 10000 bigger than 10050? >Yes, because 10000 > 10050 >>> Is 10,000 bigger than 10,050? >No, they are equal. The difference between them is only one digit (i.e., the last number). Therefore, it's not possible to tell if either value is greater or lesser by just looking at their values without knowing more information about what those numbers represent and how they were obtained in the first place. Oooof. The LLaMA tokenizer didn't save it here.
- jquery 4y ago
  Ok those are funny
- rikimaru0345 4y ago
  > The difference between them is only one digit (i.e., the last number). Therefore, it's not possible to tell if either value is greater or lesser by just looking at their values without knowing more information about what those numbers represent and how they were obtained in the first place. That one is especially hilarious. But the part at the end "how they were obtained" is really strange. Where in its dataset would it possibly have learned such nonsense? Doesn't matter where numbers come from to compare them. It implies that it doesn't understand what numbers even are in general, and that giving it a calculator (that it can use perfectly) only masks a much deeper problem. I mean, I'm reading a ton of people say "its not just pattern recognition and token prediction, it has emergent properties!!!!" and from experimentation I believe it. But if the models can pick up language and its intricacies, and even do simple logic tasks, shouldn't it also be able to pick up on what numbers are and how they work? At least knowing that where a number came from doesn't matter when its just about comparing their value in a pure mathematical sense? What does that mean for concepts other than numbers? Do those models fake a LOT more than we already believe they do?