No, typically these sorts of models would not be trained on reddit, they would be trained on historical data that insurance companies and lending companies have

No, typically these sorts of models would not be trained on reddit, they would be trained on historical data that insurance companies and lending companies have. For example, a famous case is where they used AI to help decide if people will reoffend [1]. The bias here was already embedded in prior historical data. E.g. there is a level of systemic racism or bias built-in to the entire prison system. While HN is obsessed with LLMs, AI regulation targets more than just those systems. And yes, AI and machine learning were pervasive and used in a variety of useful settings long before these toy chat systems came about. Insurance and lending companies already have complex algorithms and formula they use to determine risk scores. Adding components of these systems that are learned from data shouldn't change the overall oversight mechanisms required of them in my opinion. [1] https://www.technologyreview.com/2019/01/21/137783/algorithms-criminal-justice-ai/ https://www.technologyreview.com/2019/01/21/137783/algorithm...