2 ms·
Everything in archives pre-2021 is still untainted. All major social media, Q&A, code repos, and archive.org are timestamped. It taints future collection of tra
by humanistbot 3y ago
Everything in archives pre-2021 is still untainted. All major social media, Q&A, code repos, and archive.org are timestamped. It taints future collection of training data, but not existing collection of training data.
- jsheard 3y agoWhat's the plan then, to coast on pre-2021 data forever? How much utility would todays LLMs have if they were trained on fossilized archives of the internet from 10, 15, 20 years ago?