4 ms·
The text includes an accurate quote from Richard Feynman: Richard Feynman was reported to have said: "What I cannot create, I do not understand," How does tha
by new299 6y ago
The text includes an accurate quote from Richard Feynman:
Richard Feynman was reported to have said: "What I cannot create, I do not understand,"
How does that happen? Does the model actually encode a bunch of complete fragments of text?
- skybrian 6y agoYes, it can memorize short phrases similar to how it "remembers" words. It's trained on a web corpus that includes and emphasizes Wikipedia. The model is big enough to memorize some things, though not in such a way that they can reliably be retrieved, and it will make stuff up when it doesn't remember. So it's not Google but sometimes it's reminiscent. Here is a Q&A conversation where I found some things it "learned". https://tildes.net/~games/qmc/ai_dungeon_dragon_model_upgrade#comment-5dcc https://tildes.net/~games/qmc/ai_dungeon_dragon_model_upgrad...