2 ms·

>The important thing to understand is that there is no obvious reason why a language model should be good at arithmetic If there is enough arithmetical structu

>The important thing to understand is that there is no obvious reason why a language model should be good at arithmetic If there is enough arithmetical structure in the training corpus, eventually the best way to predict the training corpus is just to learn arithmetic rather than memorize every instance of arithmetical structure. Transformers have been shown to be equivalent to graph neural networks, so in some sense they have the power to self-discover novel architectures in service to learning a data set. So it is quite reasonable that it could have learned generic rules of arithmetic.

YeGoblynQueenne 6y ago
Sorry, but that doesn't sound very reasonable at all. I'm also not sure what you mean by "arithmetical structure" to be honest. In any case, I think you're applying an overly permissive criterion for learning "generic rules of arithmetic". It's clear from the paper linked above that GPT-3 is extremely limited in its ability to return correct results given arithmetic operations as input. The only task that it performs with 100% accuracy is two-digit addition. It cannot even perform two-digit subtraction with perfect accuracy and it's all downhill from there. Furthermore, like I say division is conspicuously absent from the set of tested tasks reported in the paper, as are any operations with five digits or more [EDIT: sorry, that's "operations with more than five digits"- my bad.]. Going again by my heuristic from an earlier comment, that researchers publish positive results and avoid publishing negative results, this tells us that GPT-3 can't perform any division at all with any accuracy and it can't perform any arithmetic operations with five digits with any accuracy [EDIT: again, that's "more than five digits". Apologies]. That is hardly the hallmark of a system that has "learned generic rules of arithmetic". It is far more likely that GPT-3 has learned to reproduce results that it has seen during training. Even more so since, as I say in my comment above, it is much better at operations that are likely to be found more often in a natural language corpus.
- hackinthebochs 6y ago
  But why think performing arithmetic with 100% accuracy is required? Children learning arithmetic aren't perfectly accurate but they're certainly learning arithmetic. The fact that there is a digit cut off where the quality of its results drop off isn't all that surprising either. How much arithmetic can you do in your head? I'm likely to fail at some point with two digit addition without using a pencil or paper. Three digits I would be significantly worse. Your criteria for what counts as "learning arithmetic" doesn't seem to be based on anything substantive. The cliff for GPT-3's arithmetic ability is likely due to the fact that it can't do recursive/recurrent calculations. That is, it can't reprocess and refine a tentative answer to improve it. You can't do arbitrary arithmetic with a finite amount of substrate without this sort of recursion or recurrency. The fact that it can only do two digits with 100% accuracy could be a hardware or architecture limitation.
  - YeGoblynQueenne 6y ago
    >> But why think performing arithmetic with 100% accuracy is required? Because otherwise, how do you know that your system has learned the "rules of arithmetic", as per your comment, and not something completely different? And like I say in my other comments, there's a very obvious alternative about what that something completely different could be: a representation of already seen results. Besides, GPT-3 is a piece of software, it's not a child or a grown up human, who can make mistakes because their memory fails or because they get overwhelmed by the complexity of executing a complex set of rules. If a piece of software implements a set of rules, it's usually able to execute them right every time, without failure, certainly so for relatively simple rules like arithmetic. Pocket calculators with tiny resources can do that and they can do it with very long sequences of numbers, so why would a huge language model, running on very expensive hardware, fail? >> The cliff for GPT-3's arithmetic ability is likely due to the fact that it can't do recursive/recurrent calculations. Well, yes, exactly that. If a system can't represent recursion then it can't represent arithmetic between arbitrary numbers. Hell, without recursion, a system can't even count to arbitrary numbers. So in what sense can GPT-3 be said to have "learned the rules of arithmetic"? Learned them, how, if it can't represent them? Actually, your observation about recursion is the first thing I'd have normally said, but it doesn't seem to be commonly understood that neural networks (and propositional, attribute-value, learners in general) can not represent recursion. Similarly, such systems can't represent non-ground values, that is they can't represent the concept of a variable. But that's a big part of why they can't build general theories. In terms of arithmetic, it means they can't represent the relation x + y = z because they can't represent x, y and z as universally quantified variables. The only remaining alternative is to represent every ground expression, like 1 + 1 = 2, 1 + 2 = 3, etc. But that's not the rules of arithmetic! That's only some instances of specific operations. That is why GPT-3 hasn't learned arithmetic and can't learn arithmetic, no matter how much data it is fed. It's just not possible to represent the rules of arithmetic in a propositional language. A first-oder language and the ability to define relations recursively are necessary. Edit: OK, sorry, my claim about a first order language being necessary is maybe hard to substantiate outside of Peano arithmetic. But, recursion and the ability to represent variables are absolutely necessary. See primitive recursive functions: https://en.wikipedia.org/wiki/Primitive_recursive_function https://en.wikipedia.org/wiki/Primitive_recursive_function.

YeGoblynQueenne 6y ago

Sorry, but that doesn't sound very reasonable at all. I'm also not sure what you mean by "arithmetical structure" to be honest. In any case, I think you're applying an overly permissive criterion for learning "generic rules of arithmetic". It's clear from the paper linked above that GPT-3 is extremely limited in its ability to return correct results given arithmetic operations as input. The only task that it performs with 100% accuracy is two-digit addition. It cannot even perform two-digit subtraction with perfect accuracy and it's all downhill from there. Furthermore, like I say division is conspicuously absent from the set of tested tasks reported in the paper, as are any operations with five digits or more [EDIT: sorry, that's "operations with more than five digits"- my bad.]. Going again by my heuristic from an earlier comment, that researchers publish positive results and avoid publishing negative results, this tells us that GPT-3 can't perform any division at all with any accuracy and it can't perform any arithmetic operations with five digits with any accuracy [EDIT: again, that's "more than five digits". Apologies]. That is hardly the hallmark of a system that has "learned generic rules of arithmetic". It is far more likely that GPT-3 has learned to reproduce results that it has seen during training. Even more so since, as I say in my comment above, it is much better at operations that are likely to be found more often in a natural language corpus.

hackinthebochs 6y ago
But why think performing arithmetic with 100% accuracy is required? Children learning arithmetic aren't perfectly accurate but they're certainly learning arithmetic. The fact that there is a digit cut off where the quality of its results drop off isn't all that surprising either. How much arithmetic can you do in your head? I'm likely to fail at some point with two digit addition without using a pencil or paper. Three digits I would be significantly worse. Your criteria for what counts as "learning arithmetic" doesn't seem to be based on anything substantive. The cliff for GPT-3's arithmetic ability is likely due to the fact that it can't do recursive/recurrent calculations. That is, it can't reprocess and refine a tentative answer to improve it. You can't do arbitrary arithmetic with a finite amount of substrate without this sort of recursion or recurrency. The fact that it can only do two digits with 100% accuracy could be a hardware or architecture limitation.
- YeGoblynQueenne 6y ago
  >> But why think performing arithmetic with 100% accuracy is required? Because otherwise, how do you know that your system has learned the "rules of arithmetic", as per your comment, and not something completely different? And like I say in my other comments, there's a very obvious alternative about what that something completely different could be: a representation of already seen results. Besides, GPT-3 is a piece of software, it's not a child or a grown up human, who can make mistakes because their memory fails or because they get overwhelmed by the complexity of executing a complex set of rules. If a piece of software implements a set of rules, it's usually able to execute them right every time, without failure, certainly so for relatively simple rules like arithmetic. Pocket calculators with tiny resources can do that and they can do it with very long sequences of numbers, so why would a huge language model, running on very expensive hardware, fail? >> The cliff for GPT-3's arithmetic ability is likely due to the fact that it can't do recursive/recurrent calculations. Well, yes, exactly that. If a system can't represent recursion then it can't represent arithmetic between arbitrary numbers. Hell, without recursion, a system can't even count to arbitrary numbers. So in what sense can GPT-3 be said to have "learned the rules of arithmetic"? Learned them, how, if it can't represent them? Actually, your observation about recursion is the first thing I'd have normally said, but it doesn't seem to be commonly understood that neural networks (and propositional, attribute-value, learners in general) can not represent recursion. Similarly, such systems can't represent non-ground values, that is they can't represent the concept of a variable. But that's a big part of why they can't build general theories. In terms of arithmetic, it means they can't represent the relation x + y = z because they can't represent x, y and z as universally quantified variables. The only remaining alternative is to represent every ground expression, like 1 + 1 = 2, 1 + 2 = 3, etc. But that's not the rules of arithmetic! That's only some instances of specific operations. That is why GPT-3 hasn't learned arithmetic and can't learn arithmetic, no matter how much data it is fed. It's just not possible to represent the rules of arithmetic in a propositional language. A first-oder language and the ability to define relations recursively are necessary. Edit: OK, sorry, my claim about a first order language being necessary is maybe hard to substantiate outside of Peano arithmetic. But, recursion and the ability to represent variables are absolutely necessary. See primitive recursive functions: https://en.wikipedia.org/wiki/Primitive_recursive_function https://en.wikipedia.org/wiki/Primitive_recursive_function.