TLDR: we don't know the data some of these models (particularly GPT4) is trained in, and labor practices related to their moderation seem shady. Pretty mislead

TLDR: we don't know the data some of these models (particularly GPT4) is trained in, and labor practices related to their moderation seem shady. Pretty misleading title, this is more of an overview of bad things to be aware of wrt large commercial models like GPT4. It is mostly reasonable, though complaining about carbon emissions always seems overblown to me. It is also not super accurate in describing what GPT4 is. Quote: "The key word in that phrase is “pre-trained.” Using all kinds of digitized content scraped from the Internet, GPT employs deep-learning techniques to find patterns, including words that are likely to appear together, while also acquiring facts, absorbing grammar, and learning rudimentary logic. " Pre-trained is only important to note for downstream fine tuning / transfer learning (what GPT and similar things were used for back in 2018/2019), but is largely irrelevant now - for most users it is just "trained". And "deep learning techniques..." makes it sound way fancier than it is - GPT4 is still (presumably) a language model that is fundamentally trained to do autocomplete (followed by some training on human chat data). It may be nit-picky to criticize these technical details, but given how massive the impact of ChatGPT and GPT4 has been (if only in terms of making people aware of this sort of tech), it sure would be nice if coverage was more technically informed by this point.