4 ms·
AI firms will soon exhaust most of the internet's data
- bookofjoe 2y agohttps://archive.ph/AbsLm https://archive.ph/AbsLm
- h2odragon 2y agoOnce you've read everything, what's left? Start reading some again and see if it improves your understanding?
- viraptor 2y ago- Throwing out the bad parts, or updating the good parts with better responses. (There's a few training programming data sets which are not great) - Self improvement without new data, like STaR https://arxiv.org/abs/2203.14465 https://arxiv.org/abs/2203.14465 - architecture changes to support better generalisation - humans learn a lot without consuming as much content.
- Arnt 2y agoYou can have some written to spec. "Please write uncertain answers to every question in this textbook", say, if you want your AI to sound a little more humble or uncertain when it doesn't have a full answer. A hundred million will buy you a lot of writing.