3 ms·
> What's the value of these The Pile (the 800GB dataset by Eluther AI) contains BookCorpus2, along with two much larger datasets of books (and a whole lot of n
by wongarsu 3y ago
> What's the value of these
The Pile (the 800GB dataset by Eluther AI) contains BookCorpus2, along with two much larger datasets of books (and a whole lot of not-book stuff). From their paper [0] the reasoning for the book datasets is that they are "are invaluable for long-range context modeling research
and coherent storytelling". The reasoning for including BookCorpus2 next to Books3 and Project Gutenberg boils down to "no significant overlap with the other datasets, and others use it".
In general books are a great source of extremely high quality long-form content. They are longer than most content found on the web, and are generally of high quality, having gone through many revision rounds between editor and author. Just that both of these aren't really true of BookCorpus. Even a dump of highly rated stories from fanfiction.org might be better.
0: https://arxiv.org/pdf/2101.00027.pdf https://arxiv.org/pdf/2101.00027.pdf
- Kaibeezy 3y ago[flagged]
- Kaibeezy 3y agoWhy is that list ^ not interesting? Illustrative?
- gs17 3y agoIt's not useful. You can ask ChatGPT again in a new session, and you'll get a different list. You can then ask it about them and find out it's making it up. For example, "Wuthering Heights" is on your list, when I ask it "Pride and Prejudice" is on it. In another session, I can ask it for opening lines, character lists, etc. from those works and then ask for a crossover story where the characters from each meet each other. The model isn't likely to regurgitate the original text in its entirety, but it does know them.
- jamilton 3y agoAdditionally, I believe a general norm has been developing to not post raw output of ChatGPT without commentary/editing, because it's low value. If anyone wanted similar output, they could just ask ChatGPT themselves. It also often gives false info, as this one demonstrates. You need to fact check it and demonstrate that before posting it if you're posting it as a source of information, IMO.
- deleted 3y ago[deleted]
- jazzyjackson 3y agothe bots are not capable of reflection, they do not know and cannot check what is in their training data
- NoMoreNicksLeft 3y agoAre these primarily fiction books, or a mix of fiction and non-fiction?
- banana_giraffe 3y agoThe books in the books3 collection aren't categorized. The source, however, currently is at a ratio of 2:1 nonfiction to fiction, and from what I've seen, whoever created the books3 archive simply attempted to gather all the EPubs they could, with their only criteria being availability.