3 ms·
Can someone explain what value new Reddit data has for LLMs? Presumably the entire 2005-2023 (up to March) dataset which is (was) freely available on Pushshift
by CamelCaseName 3y ago
Can someone explain what value new Reddit data has for LLMs?
Presumably the entire 2005-2023 (up to March) dataset which is (was) freely available on Pushshift / torrents is enough?
Also, not that Sam Altman is unbiased, but he did say that social media (Reddit, Twitter) were not super important for developing LLMs (But of course, that's exactly what he would say if they were useful!)
- zamadatix 3y agoDepends what you're training it on, I suppose. If you're just wanting to train something to generically talk then sure, a slightly outdated data set is fine. If you're wanting it to be able to talk about recent things then it'll, naturally, not be fine. It's not so much a problem right now, only being a couple of months out of date, but once that turns into years it'd become more apparent. But sure, there are plenty of other data sources as well. On top of the problem of not wanting to train your data on content you know is littered with bots.
- BeFlatXIII 3y agoI'm now envisioning a sci-fi future where the AIs are an embodiment of xkcd #606. Far better than humans at mundane tasks but with significantly outdated cultural knowledge.
- ralfn 3y agoThere are humans today that refer to Africa as the 'third world' even though it's a continent with countries more developed than some US states or European countries. Significant outdated cultural knowledge seems the norm for humans too.
- fragmede 3y agoAs someone in a US state with blackouts to our power grid, I'm wondering what more developed things you might be referring to and I might be missing out on.
- Genghis_Khan 3y agoAre they using "third world" in its original sense of non-alignment? https://en.wikipedia.org/wiki/Third_World https://en.wikipedia.org/wiki/Third_World
- lost_tourist 3y agoI didn't even know it was available on torrents.