3 ms·
> E.g. "How is *GPT taking in data and producing an output different than a human learning a skill and making prose/code/art?" It can go the other way too, lik
by cma 4y ago
> E.g. "How is *GPT taking in data and producing an output different than a human learning a skill and making prose/code/art?"
It can go the other way too, like proof that diffusion models memorize heavily repeated training examples being used as evidence they aren't creating stuff at all and just copy pasting, but artists can memorize whole works too, especially artists that copy style really well.
Many English classes will have you verbatim memorize poetry as well, to show how things were done with memorization in oral storytelling traditions.
- MacsHeadroom 4y ago> like proof that diffusion models memorize heavily repeated training examples being used as evidence they aren't creating stuff at all and just copy pasting This proof actually demonstrated the opposite. In that paper it was found that something like only 11 pictures from millions were recoverable. They were only recoverable because those handful of pictures were accidentally duplicated in the training data many multiples more than other pictures. Even pictures which were accidentally duplicated dozens of times were found to be unrecoverable by the trained model.