3 ms·
I doubt that the content of books is in the training data. OpenAI did not list Z-Library as one of their sources (though that would perhaps make the model more
by leobg 4y ago
I doubt that the content of books is in the training data. OpenAI did not list Z-Library as one of their sources (though that would perhaps make the model more interesting than the alternative trained on Reddit and CommonCrawl). Even if a book’s content had been in the training data by chance (say, from a crawl of archive.org), the model would, I believe, not know where the book starts and where it ends. I don’t think it has that kind of attention span. Hence I think it comes down to aggregating human reviews that are in the training corpus. Which is cool for sure. But is not the same as having the model summarize the actual book. I find that distinction important, because there actually are algorithms that attempt to summarize text, and it would be easy to believe that GPT-3 would be capable of doing that out of the box, and much better, too. I have yet to see someone build this.
- freediver 4y ago> Even if a book’s content had been in the training data by chance (say, from a crawl of archive.org), the model would, I believe, not know where the book starts and where it ends. Yeah, you are probably right. LLM summarization capabilities of such probably comes from reading the actual book reviews on the web. > I have yet to see someone build this. It is the area I am interested in. Check this: https://twitter.com/vladquant/status/1596855728735916033 https://twitter.com/vladquant/status/1596855728735916033
- leobg 4y agoThis looks great. Is it available to try? Might be a stretch, but if you cracked long form summarization without factual hallucination Elon/Twitter might be interested in that, as it is very much in line with what they’re trying to do, I believe. BTW, nice to see you again. Remember your name from the last time we talked here (that’s been a while ago).