3 ms·

Did you ignore everything else in my comment except that example? I don't think additional conversation will be productive. The point is success on small numb

Did you ignore everything else in my comment except that example? I don't think additional conversation will be productive. The point is success on small numbers and more mistakes on larger numbers would be what I would predict both if results were memorized and if it had deduced some incomplete model of how to do basic arithmetic. Not having the examples in the training data is a point in favor of understanding the underlying rules and a point against memorization. > "GPT-3 is not a child. GPT-3 is a language model" Yes - thanks for the obvious condescension.

YeGoblynQueenne 6y ago
I'm sorry if my comment came across as condensending. That was not my intention. I addressed the other points in your commment in my much longer comment above. To be honest, I assumed you were making an anthropomorphising comment, suggesting that GPT-3 learns like a child would learn. Sorry if that wasn't what you meant. I'm used to seeing comments like that (on HN in particular) so I guess I jumped to conclusions. Anyway I guess I took your comment about not reading linked papers personally and you took my comment about GPT-3 not being a child personally. And the other poster pounced on my comment as if they had something to win and I reacted to it angrily. This is not a very good conversation and I haven't done my due dillience to keep it civil. I'm sorry we couldn't have a very constructive conversation this time. Maybe next time.
- fossuser 6y ago
  No worries, on re-read I think I might have just interpreted things as more hostile than they were intended. I do suspect that you may be too quick to draw the memorization conclusion because you find the other one too unlikely. I think you're setting the evidence barrier too high/dismissing evidence in favor of the conclusion you already believe to be true while not holding the memorization hypothesis to the same bar (rather than recognizing that the evidence suggests maybe something interesting is going on). Either way, when things scale up we should be able to see what ends up happening.
  - YeGoblynQueenne 6y ago
    Thanks, I can sound a bit craggy sometimes. It's the internets. My motivation here is that GPT-3 is a language model and language models are designed to do one thing, and only that thing (calculate probabilities of next-tokens). If we observe some unexpected behaviour of a language model, the logical first step is to try and explain it on the basis of what we know a language model to be able to do. However, the authors of the GPT-3 paper didn't do that and immediately jumped to wishful thinking, about their model doing something it wasn't designed to do, as if by magick, on the sole basis that it was a larger model, trained on more data, than others. But, the more data and more parameters make GPT-3 a quantitatively different, not qualitatively different model and if it's now behaving in ways that language models are not designed to behave, this requires a very thorough explanation and a very strong justification. I didn't see anything like that in the paper. I've given some links above, in my reply to jointpdf from today, to papers where people have tested language models more thoroughly and found that despite similar claims (e.g. for BERT etc) a careful examination of a model's behaviour and training corpus reveals that it's doing what a language model is designed to do and nothing else. So, yes, I don't think the two explanations are equally likely: memorisation and magickal arithmetic. There is a very strong prior in favour of the former and very little evidence in support of the latter.

YeGoblynQueenne 6y ago

I'm sorry if my comment came across as condensending. That was not my intention. I addressed the other points in your commment in my much longer comment above. To be honest, I assumed you were making an anthropomorphising comment, suggesting that GPT-3 learns like a child would learn. Sorry if that wasn't what you meant. I'm used to seeing comments like that (on HN in particular) so I guess I jumped to conclusions. Anyway I guess I took your comment about not reading linked papers personally and you took my comment about GPT-3 not being a child personally. And the other poster pounced on my comment as if they had something to win and I reacted to it angrily. This is not a very good conversation and I haven't done my due dillience to keep it civil. I'm sorry we couldn't have a very constructive conversation this time. Maybe next time.

fossuser 6y ago
No worries, on re-read I think I might have just interpreted things as more hostile than they were intended. I do suspect that you may be too quick to draw the memorization conclusion because you find the other one too unlikely. I think you're setting the evidence barrier too high/dismissing evidence in favor of the conclusion you already believe to be true while not holding the memorization hypothesis to the same bar (rather than recognizing that the evidence suggests maybe something interesting is going on). Either way, when things scale up we should be able to see what ends up happening.
- YeGoblynQueenne 6y ago
  Thanks, I can sound a bit craggy sometimes. It's the internets. My motivation here is that GPT-3 is a language model and language models are designed to do one thing, and only that thing (calculate probabilities of next-tokens). If we observe some unexpected behaviour of a language model, the logical first step is to try and explain it on the basis of what we know a language model to be able to do. However, the authors of the GPT-3 paper didn't do that and immediately jumped to wishful thinking, about their model doing something it wasn't designed to do, as if by magick, on the sole basis that it was a larger model, trained on more data, than others. But, the more data and more parameters make GPT-3 a quantitatively different, not qualitatively different model and if it's now behaving in ways that language models are not designed to behave, this requires a very thorough explanation and a very strong justification. I didn't see anything like that in the paper. I've given some links above, in my reply to jointpdf from today, to papers where people have tested language models more thoroughly and found that despite similar claims (e.g. for BERT etc) a careful examination of a model's behaviour and training corpus reveals that it's doing what a language model is designed to do and nothing else. So, yes, I don't think the two explanations are equally likely: memorisation and magickal arithmetic. There is a very strong prior in favour of the former and very little evidence in support of the latter.