5 ms·

I don't understand what is so surprising here. The training of the model consists of adding noise to training samples and denoising the resulting random sample

• chickenpotpie 4y ago
  Research isn't always about finding surprising things. Some of the most important research ever conducted is proving things that seem intuitive, especially when it proves the opposite.
• sdenton4 4y ago
  An alternative framing if the article is that reproving of training examples is /rare/. It works for highly replicated images and, in larger models trained for longer periods of time, really unusual training examples. Both of these failure modes have pretty obvious mitigations: deduplicate the training data, and trim the outliers.
• yorwba 4y ago
  After adding noise to a training sample, there's still some information about the original contained in the noisy image, so the model only needs to fix it up a little with the help of the prompt. But when you start from 100% noise, all of the information needs to come from the model and the prompt. Seen from another angle, if you take a random value for each pixel, you're unlikely to generate anything resembling a picture, let alone any given training picture you're trying to reproduce. That there's an input that makes the model output a training sample doesn't necessarily mean that it's easy to find. But the paper shows that you can find several by guessing randomly.
• bfgoodrich 4y ago
  [dead]