5 ms·

It's possible the prompts used by the researchers to gauge arithmetic ability could be improved by changing to a more conversational style. I found these serie

It's possible the prompts used by the researchers to gauge arithmetic ability could be improved by changing to a more conversational style. I found these series of tweets almost unbelievable in how well GPT-3 seems to reason about function composition f(f(x)). "I wonder if the AI would be better at math if you told it to show it's work": https://twitter.com/kleptid/status/1284098635689611264?s=20 https://twitter.com/kleptid/status/1284098635689611264?s=20

YeGoblynQueenne 6y ago
The important thing to understand is that there is no obvious reason why a language model should be good at arithmetic, rather than reproducing results in its training set. OpenAI is claiming that it is, which is tantamount to invoking magick. They need to back up their very strong claim with very strong evidence. They haven't, so it's nothing more than an absurd claim that follows in a long line of absurd claims about AI, since the early days of the field.
- dddbbb 6y ago
  But the example the poster you are replying to gave shows that GPT-3 can take the square root of a user-defined function composed with itself. It clearly can perform arithmetic, and we don't need to trust OpenAI now that users can interact with the model.
  - YeGoblynQueenne 6y ago
    And yet it can't always correctly subtract two two-digit numbers. Does that sound like a system that can perform arithmetic? As I say in previous comments, no. It sounds much more like a system that can reproduce resutls it has seen in trainig, but has no general concept of arithmetic. This would also explain the square root example example easily. Also, the examples in the OP's linked tweet are very simple examples of square roots and function composition that are very likely to have been lifted verbatim from some textbook, or who knows what ...and that's the problem, because who knows what the model has flat memorised and what it's composing from smaller components. >> It clearly can perform arithmetic, and we don't need to trust OpenAI now that users can interact with the model. The paper I link above performed a systematic evaluation of GPT-3's arithmetic ability. Playing around with the OpenAI API and eyballing a few results is not going to give a clearer understaning of its abilities. In general, hitting a language model with a few queries is never going to give any clear understanding of its capabilities. Systematic evaluation is always necessary and the average user (or the non-average user) is not going to be able to do that.
    - loxali 6y ago
      I think it's pretty clear that there is something more than memorising examples from the training set going on. Look at this: https://www.johnfaben.com/blog/gpt-3-arithmetic https://www.johnfaben.com/blog/gpt-3-arithmetic In which GPT-3 answers the question "what is one hundred and five divided by three?" with "35.7". It also gave several other close-but-not-correct answers. It seems pretty unlikely these are all present in the training set, and surely can't all have been lifted verbatim. I agree systematic testing is probably more useful, but find it really hard to believe this is all happening without any sort of model of arithmetic.
- hackinthebochs 6y ago
  >The important thing to understand is that there is no obvious reason why a language model should be good at arithmetic If there is enough arithmetical structure in the training corpus, eventually the best way to predict the training corpus is just to learn arithmetic rather than memorize every instance of arithmetical structure. Transformers have been shown to be equivalent to graph neural networks, so in some sense they have the power to self-discover novel architectures in service to learning a data set. So it is quite reasonable that it could have learned generic rules of arithmetic.
  - YeGoblynQueenne 6y ago
    Sorry, but that doesn't sound very reasonable at all. I'm also not sure what you mean by "arithmetical structure" to be honest. In any case, I think you're applying an overly permissive criterion for learning "generic rules of arithmetic". It's clear from the paper linked above that GPT-3 is extremely limited in its ability to return correct results given arithmetic operations as input. The only task that it performs with 100% accuracy is two-digit addition. It cannot even perform two-digit subtraction with perfect accuracy and it's all downhill from there. Furthermore, like I say division is conspicuously absent from the set of tested tasks reported in the paper, as are any operations with five digits or more [EDIT: sorry, that's "operations with more than five digits"- my bad.]. Going again by my heuristic from an earlier comment, that researchers publish positive results and avoid publishing negative results, this tells us that GPT-3 can't perform any division at all with any accuracy and it can't perform any arithmetic operations with five digits with any accuracy [EDIT: again, that's "more than five digits". Apologies]. That is hardly the hallmark of a system that has "learned generic rules of arithmetic". It is far more likely that GPT-3 has learned to reproduce results that it has seen during training. Even more so since, as I say in my comment above, it is much better at operations that are likely to be found more often in a natural language corpus.
    - hackinthebochs 6y ago
      But why think performing arithmetic with 100% accuracy is required? Children learning arithmetic aren't perfectly accurate but they're certainly learning arithmetic. The fact that there is a digit cut off where the quality of its results drop off isn't all that surprising either. How much arithmetic can you do in your head? I'm likely to fail at some point with two digit addition without using a pencil or paper. Three digits I would be significantly worse. Your criteria for what counts as "learning arithmetic" doesn't seem to be based on anything substantive. The cliff for GPT-3's arithmetic ability is likely due to the fact that it can't do recursive/recurrent calculations. That is, it can't reprocess and refine a tentative answer to improve it. You can't do arbitrary arithmetic with a finite amount of substrate without this sort of recursion or recurrency. The fact that it can only do two digits with 100% accuracy could be a hardware or architecture limitation.

YeGoblynQueenne 6y ago

The important thing to understand is that there is no obvious reason why a language model should be good at arithmetic, rather than reproducing results in its training set. OpenAI is claiming that it is, which is tantamount to invoking magick. They need to back up their very strong claim with very strong evidence. They haven't, so it's nothing more than an absurd claim that follows in a long line of absurd claims about AI, since the early days of the field.

dddbbb 6y ago
But the example the poster you are replying to gave shows that GPT-3 can take the square root of a user-defined function composed with itself. It clearly can perform arithmetic, and we don't need to trust OpenAI now that users can interact with the model.
- YeGoblynQueenne 6y ago
  And yet it can't always correctly subtract two two-digit numbers. Does that sound like a system that can perform arithmetic? As I say in previous comments, no. It sounds much more like a system that can reproduce resutls it has seen in trainig, but has no general concept of arithmetic. This would also explain the square root example example easily. Also, the examples in the OP's linked tweet are very simple examples of square roots and function composition that are very likely to have been lifted verbatim from some textbook, or who knows what ...and that's the problem, because who knows what the model has flat memorised and what it's composing from smaller components. >> It clearly can perform arithmetic, and we don't need to trust OpenAI now that users can interact with the model. The paper I link above performed a systematic evaluation of GPT-3's arithmetic ability. Playing around with the OpenAI API and eyballing a few results is not going to give a clearer understaning of its abilities. In general, hitting a language model with a few queries is never going to give any clear understanding of its capabilities. Systematic evaluation is always necessary and the average user (or the non-average user) is not going to be able to do that.
  - loxali 6y ago
    I think it's pretty clear that there is something more than memorising examples from the training set going on. Look at this: https://www.johnfaben.com/blog/gpt-3-arithmetic https://www.johnfaben.com/blog/gpt-3-arithmetic In which GPT-3 answers the question "what is one hundred and five divided by three?" with "35.7". It also gave several other close-but-not-correct answers. It seems pretty unlikely these are all present in the training set, and surely can't all have been lifted verbatim. I agree systematic testing is probably more useful, but find it really hard to believe this is all happening without any sort of model of arithmetic.
hackinthebochs 6y ago
>The important thing to understand is that there is no obvious reason why a language model should be good at arithmetic If there is enough arithmetical structure in the training corpus, eventually the best way to predict the training corpus is just to learn arithmetic rather than memorize every instance of arithmetical structure. Transformers have been shown to be equivalent to graph neural networks, so in some sense they have the power to self-discover novel architectures in service to learning a data set. So it is quite reasonable that it could have learned generic rules of arithmetic.
- YeGoblynQueenne 6y ago
  Sorry, but that doesn't sound very reasonable at all. I'm also not sure what you mean by "arithmetical structure" to be honest. In any case, I think you're applying an overly permissive criterion for learning "generic rules of arithmetic". It's clear from the paper linked above that GPT-3 is extremely limited in its ability to return correct results given arithmetic operations as input. The only task that it performs with 100% accuracy is two-digit addition. It cannot even perform two-digit subtraction with perfect accuracy and it's all downhill from there. Furthermore, like I say division is conspicuously absent from the set of tested tasks reported in the paper, as are any operations with five digits or more [EDIT: sorry, that's "operations with more than five digits"- my bad.]. Going again by my heuristic from an earlier comment, that researchers publish positive results and avoid publishing negative results, this tells us that GPT-3 can't perform any division at all with any accuracy and it can't perform any arithmetic operations with five digits with any accuracy [EDIT: again, that's "more than five digits". Apologies]. That is hardly the hallmark of a system that has "learned generic rules of arithmetic". It is far more likely that GPT-3 has learned to reproduce results that it has seen during training. Even more so since, as I say in my comment above, it is much better at operations that are likely to be found more often in a natural language corpus.
  - hackinthebochs 6y ago
    But why think performing arithmetic with 100% accuracy is required? Children learning arithmetic aren't perfectly accurate but they're certainly learning arithmetic. The fact that there is a digit cut off where the quality of its results drop off isn't all that surprising either. How much arithmetic can you do in your head? I'm likely to fail at some point with two digit addition without using a pencil or paper. Three digits I would be significantly worse. Your criteria for what counts as "learning arithmetic" doesn't seem to be based on anything substantive. The cliff for GPT-3's arithmetic ability is likely due to the fact that it can't do recursive/recurrent calculations. That is, it can't reprocess and refine a tentative answer to improve it. You can't do arbitrary arithmetic with a finite amount of substrate without this sort of recursion or recurrency. The fact that it can only do two digits with 100% accuracy could be a hardware or architecture limitation.