3 ms·
There are public data dumps of Reddit comments available all the way up to December 2022. And they're only roughly ~2TB all together. There's nothing stopping
by antisthenes 3y ago
There are public data dumps of Reddit comments available all the way up to December 2022. And they're only roughly ~2TB all together.
There's nothing stopping AI companies from just using those instead of paying Reddit $50 million to scrape all of them using the API. It would also be 10x-100x quicker to do that rather than hammer their API for the comments (the API sucks for mass data retrieval)
- mynameisvlad 3y agoSure, but companies doing that also wouldn’t be paying Reddit for that data. The point of shredding comments isn’t to hurt the companies scraping the data (although that might be a nice side effect). Ultimately it’s to hurt Reddit.
- realce 3y agoWhere would someone find these?
- dontupvoteme 3y agohttps://academictorrents.com/browse.php?search=reddit&sort_field=size&sort_dir=DESC https://academictorrents.com/browse.php?search=reddit&sort_f...
- realce 3y agomerci good fellow :)
- dontupvoteme 3y agolieto di aiutare l'amico