2 ms·
Uh, obviously it can't recite the entirety of Dune, for example. But just as obviously, the Litany Against Fear is famous enough to likely exist verbatim thousa
by Sharlin 3y ago
Uh, obviously it can't recite the entirety of Dune, for example. But just as obviously, the Litany Against Fear is famous enough to likely exist verbatim thousands of times in its training corpus.
- jncfhnb 3y agoI don’t think this is obvious at all. Ignore the algorithm and temperature and all that for a bit and just have it show you the word probabilities. I wouldn’t bat an eye if the entirety of dune was in there.
- Sharlin 3y agoMaybe obvious was too strong a word, but I very much doubt that it contains a lossless representation of any copyrighted novel-length work. Just due to the pigeonhole principle if not anything else. The model is a vastly compressed approximation of the full training corpus.
- voxelghost 3y agoEven though vastly compressed, in terms of lossy vs. fidelity, it seems to be able to recite pretty much any bible verse you ask it for.
- jncfhnb 3y agoThe intersection space of 10^15 parameters is a lot of pigeon holes though.