4 ms·

No worries, on re-read I think I might have just interpreted things as more hostile than they were intended. I do suspect that you may be too quick to draw the

No worries, on re-read I think I might have just interpreted things as more hostile than they were intended. I do suspect that you may be too quick to draw the memorization conclusion because you find the other one too unlikely. I think you're setting the evidence barrier too high/dismissing evidence in favor of the conclusion you already believe to be true while not holding the memorization hypothesis to the same bar (rather than recognizing that the evidence suggests maybe something interesting is going on). Either way, when things scale up we should be able to see what ends up happening.

YeGoblynQueenne 6y ago
Thanks, I can sound a bit craggy sometimes. It's the internets. My motivation here is that GPT-3 is a language model and language models are designed to do one thing, and only that thing (calculate probabilities of next-tokens). If we observe some unexpected behaviour of a language model, the logical first step is to try and explain it on the basis of what we know a language model to be able to do. However, the authors of the GPT-3 paper didn't do that and immediately jumped to wishful thinking, about their model doing something it wasn't designed to do, as if by magick, on the sole basis that it was a larger model, trained on more data, than others. But, the more data and more parameters make GPT-3 a quantitatively different, not qualitatively different model and if it's now behaving in ways that language models are not designed to behave, this requires a very thorough explanation and a very strong justification. I didn't see anything like that in the paper. I've given some links above, in my reply to jointpdf from today, to papers where people have tested language models more thoroughly and found that despite similar claims (e.g. for BERT etc) a careful examination of a model's behaviour and training corpus reveals that it's doing what a language model is designed to do and nothing else. So, yes, I don't think the two explanations are equally likely: memorisation and magickal arithmetic. There is a very strong prior in favour of the former and very little evidence in support of the latter.

YeGoblynQueenne 6y ago

Thanks, I can sound a bit craggy sometimes. It's the internets. My motivation here is that GPT-3 is a language model and language models are designed to do one thing, and only that thing (calculate probabilities of next-tokens). If we observe some unexpected behaviour of a language model, the logical first step is to try and explain it on the basis of what we know a language model to be able to do. However, the authors of the GPT-3 paper didn't do that and immediately jumped to wishful thinking, about their model doing something it wasn't designed to do, as if by magick, on the sole basis that it was a larger model, trained on more data, than others. But, the more data and more parameters make GPT-3 a quantitatively different, not qualitatively different model and if it's now behaving in ways that language models are not designed to behave, this requires a very thorough explanation and a very strong justification. I didn't see anything like that in the paper. I've given some links above, in my reply to jointpdf from today, to papers where people have tested language models more thoroughly and found that despite similar claims (e.g. for BERT etc) a careful examination of a model's behaviour and training corpus reveals that it's doing what a language model is designed to do and nothing else. So, yes, I don't think the two explanations are equally likely: memorisation and magickal arithmetic. There is a very strong prior in favour of the former and very little evidence in support of the latter.