4 ms·

> They just show it to the AI, it doesn't store it - just data about it. The same as you when you look at something, be it a painting or a chair. In general fo

> They just show it to the AI, it doesn't store it - just data about it. The same as you when you look at something, be it a painting or a chair. In general for machine learning, yes, but deep learning models with large numbers of parameters such as transformers and stable diffusion do actually memorise part of their training data verbatim. The problem is mainly duplicate data points. E.g. https://arstechnica.com/information-technology/2023/02/researchers-extract-training-images-from-stable-diffusion-but-its-difficult/ https://arstechnica.com/information-technology/2023/02/resea... https://arxiv.org/pdf/2202.07646.pdf https://arxiv.org/pdf/2202.07646.pdf Quantifying Memorization Across Neural Language Models

LawTalkingGuy 4y ago
Yeah, currently many models do seem to have copied a few pieces of their source material. I was translating code with one of the code- models at OpenAI and it spit out about thirty words of someone's code of conduct from Github, almost verbatim. I found the repo by searching what it had produced. That's when I changed my prompts to remove comments, but it stuck with me. In general though, I think the pigeon-hole principle is relevant. The model is fairly small compared to the training material and couldn't contain much actual full content or it wouldn't be able to contain anything about the rest. I've heard people talking about using the model to judge similarity of new training data to what it's already seen, mainly to train more efficiently, but also to avoid this.