4 ms·
Color me unsurprised that the New York Times is writing an article drumming up fear around synthetic data and training on the internet, and concludes that the s
by reissbaker 2y ago
Color me unsurprised that the New York Times is writing an article drumming up fear around synthetic data and training on the internet, and concludes that the solution is for "A.I. companies to pay for this data instead of scooping it up from the internet, ensuring both human origin and high quality . . . there’s no replacement for the real thing."
This reads like an extended sales pitch rather than an article. It's true that if you train the same model on nothing but a limited set of its own output for 30 epochs, you're gonna get a shitty model (although 30 epochs of finetuning will result in a pretty bad model on most datasets smaller than the original one). But paying the New York Times isn't going to change that: you can already mark articles pulled from its domain as being high-quality, human-sourced data, even if you don't pay them. If they win their lawsuits, they might be able to force model trainers to pay them for it, but that doesn't have anything to do with model collapse.
- jay_kyburz 2y agoOMG, this the startup idea I have been pitching for 10 years, and everybody laughs at me when I pitch it. Newspapers need to pivot away from "telling stories about things that happen" in articles and headlines, to organizations that gather information into some large interconnected database. They should collect and record testimony from witnesses, press releases and statements. Court recordings and findings. They should be able to take any "fact" and trace back to who claimed it, where, and when. Then, when you have a solid foundation, you can put a front end on it with commentators musing on various topics, or digging deeper looking for meaning, or cause and effects. Newspapers could be the organizations we go to when we want to know who said what and when, and what happened as a result. They should be neutral, and trustworthy. The role of newspapers should be to record history as it happens.
- surfingdino 2y agoIsn't it what Google's mission is? "Our mission is to organise the world’s information and make it universally accessible and useful" https://about.google/intl/ALL_uk/ https://about.google/intl/ALL_uk/
- skybrian 2y agoAI companies are already paying for data. For example, OpenAI has done deals with the Atlantic and Reddit. It seems likely to continue?
- reissbaker 2y agoOpenAI is paying, but the others aren't — the NYT seems annoyed by the latter in the article.