To me the difference is this: https://news.ycombinator.com/item?id=27710287 https://news.ycombinator.com/item?id=27710287 It's possible for generation models t

To me the difference is this: https://news.ycombinator.com/item?id=27710287 https://news.ycombinator.com/item?id=27710287 It's possible for generation models to perfectly memorize and reproduce training data, at which point I view it as a sort of indexed slightly-lossy compression, but it's almost never happening with image generation because the models are too small to memorize billions of pictures, it can't produce copies. Stable diffusion 1.4 has been shrunk to around 4.3GB, and has around 900 million parameters. I don't know how big Copilot is, but a relatively recently released 20 billion parameter language model is over 40GB. ( https://huggingface.co/EleutherAI/gpt-neox-20b/tree/main https://huggingface.co/EleutherAI/gpt-neox-20b/tree/main ) GPT-3, according to OpenAI, is 175 billion parameters. It's possible there are some images in there you can pull out exactly as is from the training data, if they were to appear enough times, like I suspect the Mona Lisa could be almost identically reconstructed, but it would take a lot of random generation. I'm trying it now and most of the images are cropped, colors blown out, wrong number of hands or fingers, eyes are wrong, etc.