> You think it’s just generating plausible random crap that happens to exist verbatim on the internet? > I mean… read the paper, 0.8% outputs were verbatim for

> You think it’s just generating plausible random crap that happens to exist verbatim on the internet? > I mean… read the paper, 0.8% outputs were verbatim for gpt-3.5. Look at the sorts of outputs they claim are in the training data. Also note that their appendix includes huge chunks of text but they do not claim the entire chunk was matched to existing data — only a tiny amount of it. The “bug” to me is something about losing its state and generating a random token. Now if that random token is “Afgh”, I’m not surprised it follows up with “Afghanistan” and a perfect list of countries in alphabetical order. I’m also not surprised that appears in training data, because it appears on thousands of webpages. So it’s not that there isn’t an overlap between the GPT gibberish and internet content, and therefore likely training data. It’s that it’s not especially unique. If it were — like reproducing a one off Reddit thread verbatim — I think that would be greater cause for concern.