4 ms·

I'm sure I'm missing something, but: aren't there publicly available corpuses of all reddit posts up to a certain date? Why wouldn't researchers train with thes

I'm sure I'm missing something, but: aren't there publicly available corpuses of all reddit posts up to a certain date? Why wouldn't researchers train with these? Are they just not recent enough? Even if they aren't very recent, how big of a deal is that when it comes to training models that presumably use lots of other data sources as well?

dathinab 3y ago
> up to a certain date? > just not recent enough thats a very big deal for a lot of the content you find on reddit which users might want to get answers from from an AI.
sebzim4500 3y ago
There are massive up to date archives available https://tracker.archiveteam.org/reddit/ https://tracker.archiveteam.org/reddit/ I guess either OP doesn't know about them or he assumes that AI researchers won't think of using them.
mattstir 3y ago
Basically the only thing that comes to mind is that our collective views and knowledge about topics can change over time. If we had a hypothetical LLM in the 1930's, it would have had quite different views on things like mental health, civil rights, and so on. But like you said, the models would ideally be trained on many other data sources as well.
LittleShaman 3y ago
The author is aware of that, mentions it towards the end: > Most of Reddit’s current data has been scraped anyway, so the game is to protect Reddit’s data going forward. But yes, Pushshift archives of all posts and comments until the recent ban [1] are freely available for download [2] [1] https://old.reddit.com/r/pushshift/comments/135tdl2/a_response_from_pushshift_a_call_for/ https://old.reddit.com/r/pushshift/comments/135tdl2/a_respon... The ban was followed by allowing the parent non-profit of Pushshift (Network Contagion Research Institute) to use the API provided access is restricted to a use-case Reddit has care for: mod tools. Reddit hasnt replaced those with its own just yet. The rest of us are shut out. [2] https://academictorrents.com/userdetails.php?id=9863 https://academictorrents.com/userdetails.php?id=9863
rjh29 3y ago
Reddit's API change will not stop LLM training, it is far too easy to manually scrape the site. It only needs to be done once. The intention of the API change is to kill third-party app users who cannot be as effectively monetized as first-party app users.

dathinab 3y ago

> up to a certain date? > just not recent enough thats a very big deal for a lot of the content you find on reddit which users might want to get answers from from an AI.

sebzim4500 3y ago

There are massive up to date archives available https://tracker.archiveteam.org/reddit/ https://tracker.archiveteam.org/reddit/ I guess either OP doesn't know about them or he assumes that AI researchers won't think of using them.

mattstir 3y ago

Basically the only thing that comes to mind is that our collective views and knowledge about topics can change over time. If we had a hypothetical LLM in the 1930's, it would have had quite different views on things like mental health, civil rights, and so on. But like you said, the models would ideally be trained on many other data sources as well.

LittleShaman 3y ago

The author is aware of that, mentions it towards the end: > Most of Reddit’s current data has been scraped anyway, so the game is to protect Reddit’s data going forward. But yes, Pushshift archives of all posts and comments until the recent ban [1] are freely available for download [2] [1] https://old.reddit.com/r/pushshift/comments/135tdl2/a_response_from_pushshift_a_call_for/ https://old.reddit.com/r/pushshift/comments/135tdl2/a_respon... The ban was followed by allowing the parent non-profit of Pushshift (Network Contagion Research Institute) to use the API provided access is restricted to a use-case Reddit has care for: mod tools. Reddit hasnt replaced those with its own just yet. The rest of us are shut out. [2] https://academictorrents.com/userdetails.php?id=9863 https://academictorrents.com/userdetails.php?id=9863

rjh29 3y ago

Reddit's API change will not stop LLM training, it is far too easy to manually scrape the site. It only needs to be done once. The intention of the API change is to kill third-party app users who cannot be as effectively monetized as first-party app users.