4 ms·
How many terabytes of information is that roughly? I wonder what LLMs would look like if they weren't able to be trained on the collective community efforts of
by MuffinFlavored 3y ago
How many terabytes of information is that roughly?
I wonder what LLMs would look like if they weren't able to be trained on the collective community efforts of Reddit + StackOverflow exports
- CSMastermind 3y agoI mean one of the speculations about ChatGPT's political bias at least early on was that Reddit featured prominently in its training data.
- wongarsu 3y agoAbout 12 TB uncompressed json until the middle of 2022, with a dataset that grows 250GB+ per month. If you throw away all metadata you are left with between half and a quarter of that in high quality text.
- awestroke 3y ago> high quality That's a hot take
- fennecbutt 3y ago"Community efforts" lmao. Don't put so much weight in the noise humans make. Most of what we talk about is either parroting information produced by somebody else or opinions about information produced by somebody else that always converge to relatively common speaking points. Unique human content is pretty minimal. Everything is a meme.
- Lammy 3y agohttps://academictorrents.com/details/9c263fc85366c1ef8f5bb9da0203f4c8c8db75f4 https://academictorrents.com/details/9c263fc85366c1ef8f5bb9d... Reddit comments/submissions 2005-06 to 2023-12 — 2.52TB compressed