3 ms·
There also exists a working direct download link which I can’t post because the Danish Rights Alliance is following me around DMCA’ing anything to that effect.
by sillysaurusx 3y ago
There also exists a working direct download link which I can’t post because the Danish Rights Alliance is following me around DMCA’ing anything to that effect. They already struck my Twitter once.
But, BitTorrent is probably the only way a dataset like this can survive now. It’s a damn shame since OpenAI are the ones making money, and researchers are simply trying to replicate their scientific efforts.
Original books3 announcement thread: https://x.com/theshawwn/status/1320282149329784833?s=61&t=jQbmCk1JqL7depzFWJNuPA https://x.com/theshawwn/status/1320282149329784833?s=61&t=jQ...
An article on books3 from Gizmodo, which I quite like: https://gizmodo.com/anti-piracy-group-takes-ai-training-dataset-books3-off-1850743763 https://gizmodo.com/anti-piracy-group-takes-ai-training-data...
By the way, you can use aria2c to download just the books3 part of that torrent. Use aria2c —-select-file=44 and pass the torrent url. Takes about 30 minutes. Plus most people probably don’t have 800GB free.
- marginalia_nu 3y ago> Plus most people probably don’t have 800GB free. They should make space. Data sets in general are hugely slept on among hobbyists. Even just what's publicly available, like Wikipedia's dumps; you can do so much fun stuff with them and a few days of compute time!