7 ms·

Well, there is something kind of interesting here. 5 + 5 = _ Predict what's next. The most accurate way to do this is to understand how arithmetic actually w

Well, there is something kind of interesting here. 5 + 5 = _ Predict what's next. The most accurate way to do this is to understand how arithmetic actually works. Sure, if you've only seen a handful of examples (or this exact example) you may have memorized 10 comes next or you may know a number comes next, but not which number. If you've seen enormous amounts of this you may deduce the underlying rules in order to more accurately predict what comes next. There's evidence of this happening with gpt-3. The scaling hypothesis may be stronger than people suspect.

typon 6y ago
What's the evidence?
- fossuser 6y ago
  https://www.gwern.net/newsletter/2020/05#meta-learning https://www.gwern.net/newsletter/2020/05#meta-learning
  - typon 6y ago
    The site you linked asserts "meta-learning" is happening in GPT3 without any evidence of it
    - fossuser 6y ago
      Isn't the capability to answer arithmetic prompts without the exact equation being present in the training data evidence? I'm not an AI researcher so I only have a lay-person's opinion from reading about this kind of thing via things like that blog post (I also don't have OpenAI access to play with the API myself), but if that is an accurate statement it seems like pretty good evidence of figuring out the underlying mathematical rules. I think this could be a path towards scaling being more effective than people think. I sense though that you're just arguing from a position of motivated reasoning based on a pre-existing conclusion rather than actually trying to look at new things that may contradict what you already believe to be true. Repeated one sentence responses arguing that none of this is evidence just isn't worth time to respond to. In the end we'll see what ends up happening.
      - deleted 6y ago
        [deleted]
  - YeGoblynQueenne 6y ago
    Can I ask why you are not citing the GPT-3 paper itself? Why do we need to look somewhere else for evidence about anything to do with GPT-3 (or indeed any system) than the work of the people who have created it?
    - fossuser 6y ago
      That's a fair criticism, original paper would be better. I linked to the blog because that's where I first read about this and it's more immediately accessible. [Edit: Paper Link, https://arxiv.org/abs/2005.14165 https://arxiv.org/abs/2005.14165] In fact after reading these sections of the actual paper, it's hard to believe that you could have read it yourself and taken away the idea that it was memorization? Pages 22-23 in particular. Part of the reason I tend to link to good blog posts instead of academic papers is when you link to papers nobody reads them. Often people linking them haven't read them either (I've only read small parts).
      - YeGoblynQueenne 6y ago
        I think you're saying I didn't read the paper, because I didn't draw the same conclusions from it as you did after reading a blog post about it? Again there's a simpler explanation: I read the paper and I drew different conclusions than you or your primary source. I give a very long explanation of why I drew those conclusions, above. To clarify, I discussed the exact same subject a couple of weeks ago, I think, after reading the GPT-3 paper. I made my original comment in this thread without re-reading the paper, so I misremembered what was in it. Then I re-read the relevant section (3.9.1. Arithmetic) to refresh my memory and made the very long comment above. That's in the intereste of full disclosure and so you don't have any reason to assume I didn't read the paper, which you shouldn't anyway.
- jointpdf 6y ago
  This (GPT-3’s performance on arithmetic tasks) is covered in the original paper (https://arxiv.org/abs/2005.14165 https://arxiv.org/abs/2005.14165) Pages: 21-23, 63
  - YeGoblynQueenne 6y ago
    See my reply above. The evidence in the paper does not support the claim that GPT-3 has learned rules of arithmetic (also claimed in the paper, not just in the OP's comment). There is a much simpler explanation, i.e. memorisation.
    - jointpdf 6y ago
      Your reply is misleading, sorry. You didn’t offer any actual evidence of your own to support the memorization claim. You didn’t even do your own arithmetic problem correctly. Since your performance on this task was <100% accurate, I can only assume you do not know the rules of arithmetic. > How many "x + y" questions can be formulated where x and y are both single-digit numbers? The answer is 2^10, or 100. Less snarkily, if there’s (10^4)^2 = 100 million combinations of 4 digit addition problems, and GPT-3 is reaching 25.5% accuracy on those problems (vs. 0.4% in the 13B parameter model). For 3 digit problems, it’s even better: 1 million combinations and 80.4% accuracy. Clearly, there is more happening than simple memorization—the training set does not contain 800k 3-digit addition problems. Thus, it’s fair to say that the model has at least a partial grasp of how to perform arithmetic operations (but probably not fair to say that it has synthesized the entire system of arithmetic). Also, the paper does say that they scrubbed exact examples from the training set to avoid memorization, a fact you left out: > (pg. 23): ”To spot-check whether the model is simply memorizing specific arithmetic problems, we took the 3-digit arithmetic problems in our test set and searched for them in our training data in both the forms "<NUM1> + <NUM2> =" and "<NUM1> plus <NUM2>". Out of 2,000 addition problems we found only 17 matches (0.8%) and out of 2,000 subtraction problems we found only 2 matches (0.1%), suggesting that only a trivial fraction of the correct answers could have been memorized. In addition, inspection of incorrect answers reveals that the model often makes mistakes such as not carrying a “1”, suggesting it is actually attempting to perform the relevant computation rather than memorizing a table.”
YeGoblynQueenne 6y ago
How many "x + y" questions can be formulated where x and y are both single-digit numbers? The answer is 2^10, or 100. If x and y are two-digit numbers, the number is 10,000. These are tiny, tiny numbers that are trivial to memorise for a model the size of GPT-3 and there is no need for "enormous" amounts of examples, let alone to "understand how arithmetic works". In fact, it is absolutely the case that if a system needs to see "enormous" amounts of examples before it can "deduce the underlying rules" (of arithmetic) then GPT-3 can't do that, because there simply aren't enough examples of such operations (between one- and two-digit numbers). Indeed, GPT-3 completes "x + y =" and "x - y =" prompts with 80% accuracy or more when x and y are one- or two-digit numbers. It scores 20% or less when x and y are three- to five-digit numbers and its accuracy is similarly abysmal on multiplication (and results on division are not even reported for one-digit x and y). This is very much what we would expect to see from a model that has memorised some, common, examples of one- and two-digit addition and subtraction and has not seen enough examples of other operations to learn any consistent representation of those operations.
- fossuser 6y ago
  [Edit: The reply below is a better response from someone who knows more of the specifics than I do, link: https://news.ycombinator.com/item?id=24285024 https://news.ycombinator.com/item?id=24285024] While it could be memorization, their testing seem to imply something else is going on. Particularly because they excluded the exact examples they tested from the training data (to try and test if it was memorization). Since gpt-3 could solve some arithmetic even without those exact example test cases in the training data, it seems more likely to me that it's not simply memorization. It might be that it has some incomplete idea of the underlying rules, and therefore makes mistakes. A child that is learning addition will have a harder time with larger numbers and make more mistakes too. It'll be interesting to see if continuing to scale up the model improves things or not. > "In fact, it is absolutely the case that if a system needs to see "enormous" amounts of examples before it can "deduce the underlying rules" (of arithmetic) then GPT-3 can't do that, because there simply aren't enough examples of such operations (between one- and two-digit numbers)." I don't know if this is true. It depends on 'enormous' and it depends on how many examples are required to deduce how math works from trying to predict what's next. I don't think anyone actually knows the answer to this yet?

typon 6y ago

What's the evidence?

fossuser 6y ago
https://www.gwern.net/newsletter/2020/05#meta-learning https://www.gwern.net/newsletter/2020/05#meta-learning
- typon 6y ago
  The site you linked asserts "meta-learning" is happening in GPT3 without any evidence of it
  - fossuser 6y ago
    Isn't the capability to answer arithmetic prompts without the exact equation being present in the training data evidence? I'm not an AI researcher so I only have a lay-person's opinion from reading about this kind of thing via things like that blog post (I also don't have OpenAI access to play with the API myself), but if that is an accurate statement it seems like pretty good evidence of figuring out the underlying mathematical rules. I think this could be a path towards scaling being more effective than people think. I sense though that you're just arguing from a position of motivated reasoning based on a pre-existing conclusion rather than actually trying to look at new things that may contradict what you already believe to be true. Repeated one sentence responses arguing that none of this is evidence just isn't worth time to respond to. In the end we'll see what ends up happening.
    - deleted 6y ago
      [deleted]
- YeGoblynQueenne 6y ago
  Can I ask why you are not citing the GPT-3 paper itself? Why do we need to look somewhere else for evidence about anything to do with GPT-3 (or indeed any system) than the work of the people who have created it?
  - fossuser 6y ago
    That's a fair criticism, original paper would be better. I linked to the blog because that's where I first read about this and it's more immediately accessible. [Edit: Paper Link, https://arxiv.org/abs/2005.14165 https://arxiv.org/abs/2005.14165] In fact after reading these sections of the actual paper, it's hard to believe that you could have read it yourself and taken away the idea that it was memorization? Pages 22-23 in particular. Part of the reason I tend to link to good blog posts instead of academic papers is when you link to papers nobody reads them. Often people linking them haven't read them either (I've only read small parts).
    - YeGoblynQueenne 6y ago
      I think you're saying I didn't read the paper, because I didn't draw the same conclusions from it as you did after reading a blog post about it? Again there's a simpler explanation: I read the paper and I drew different conclusions than you or your primary source. I give a very long explanation of why I drew those conclusions, above. To clarify, I discussed the exact same subject a couple of weeks ago, I think, after reading the GPT-3 paper. I made my original comment in this thread without re-reading the paper, so I misremembered what was in it. Then I re-read the relevant section (3.9.1. Arithmetic) to refresh my memory and made the very long comment above. That's in the intereste of full disclosure and so you don't have any reason to assume I didn't read the paper, which you shouldn't anyway.
jointpdf 6y ago
This (GPT-3’s performance on arithmetic tasks) is covered in the original paper (https://arxiv.org/abs/2005.14165 https://arxiv.org/abs/2005.14165) Pages: 21-23, 63
- YeGoblynQueenne 6y ago
  See my reply above. The evidence in the paper does not support the claim that GPT-3 has learned rules of arithmetic (also claimed in the paper, not just in the OP's comment). There is a much simpler explanation, i.e. memorisation.
  - jointpdf 6y ago
    Your reply is misleading, sorry. You didn’t offer any actual evidence of your own to support the memorization claim. You didn’t even do your own arithmetic problem correctly. Since your performance on this task was <100% accurate, I can only assume you do not know the rules of arithmetic. > How many "x + y" questions can be formulated where x and y are both single-digit numbers? The answer is 2^10, or 100. Less snarkily, if there’s (10^4)^2 = 100 million combinations of 4 digit addition problems, and GPT-3 is reaching 25.5% accuracy on those problems (vs. 0.4% in the 13B parameter model). For 3 digit problems, it’s even better: 1 million combinations and 80.4% accuracy. Clearly, there is more happening than simple memorization—the training set does not contain 800k 3-digit addition problems. Thus, it’s fair to say that the model has at least a partial grasp of how to perform arithmetic operations (but probably not fair to say that it has synthesized the entire system of arithmetic). Also, the paper does say that they scrubbed exact examples from the training set to avoid memorization, a fact you left out: > (pg. 23): ”To spot-check whether the model is simply memorizing specific arithmetic problems, we took the 3-digit arithmetic problems in our test set and searched for them in our training data in both the forms "<NUM1> + <NUM2> =" and "<NUM1> plus <NUM2>". Out of 2,000 addition problems we found only 17 matches (0.8%) and out of 2,000 subtraction problems we found only 2 matches (0.1%), suggesting that only a trivial fraction of the correct answers could have been memorized. In addition, inspection of incorrect answers reveals that the model often makes mistakes such as not carrying a “1”, suggesting it is actually attempting to perform the relevant computation rather than memorizing a table.”

YeGoblynQueenne 6y ago

How many "x + y" questions can be formulated where x and y are both single-digit numbers? The answer is 2^10, or 100. If x and y are two-digit numbers, the number is 10,000. These are tiny, tiny numbers that are trivial to memorise for a model the size of GPT-3 and there is no need for "enormous" amounts of examples, let alone to "understand how arithmetic works". In fact, it is absolutely the case that if a system needs to see "enormous" amounts of examples before it can "deduce the underlying rules" (of arithmetic) then GPT-3 can't do that, because there simply aren't enough examples of such operations (between one- and two-digit numbers). Indeed, GPT-3 completes "x + y =" and "x - y =" prompts with 80% accuracy or more when x and y are one- or two-digit numbers. It scores 20% or less when x and y are three- to five-digit numbers and its accuracy is similarly abysmal on multiplication (and results on division are not even reported for one-digit x and y). This is very much what we would expect to see from a model that has memorised some, common, examples of one- and two-digit addition and subtraction and has not seen enough examples of other operations to learn any consistent representation of those operations.

fossuser 6y ago
[Edit: The reply below is a better response from someone who knows more of the specifics than I do, link: https://news.ycombinator.com/item?id=24285024 https://news.ycombinator.com/item?id=24285024] While it could be memorization, their testing seem to imply something else is going on. Particularly because they excluded the exact examples they tested from the training data (to try and test if it was memorization). Since gpt-3 could solve some arithmetic even without those exact example test cases in the training data, it seems more likely to me that it's not simply memorization. It might be that it has some incomplete idea of the underlying rules, and therefore makes mistakes. A child that is learning addition will have a harder time with larger numbers and make more mistakes too. It'll be interesting to see if continuing to scale up the model improves things or not. > "In fact, it is absolutely the case that if a system needs to see "enormous" amounts of examples before it can "deduce the underlying rules" (of arithmetic) then GPT-3 can't do that, because there simply aren't enough examples of such operations (between one- and two-digit numbers)." I don't know if this is true. It depends on 'enormous' and it depends on how many examples are required to deduce how math works from trying to predict what's next. I don't think anyone actually knows the answer to this yet?