4 ms·

It's unfortunate that these laws focus on AI. Why isn't it illegal to use traditional algorithms that have bias? Why aren't there regulations ensuring bias test

by tensor 2y ago

It's unfortunate that these laws focus on AI. Why isn't it illegal to use traditional algorithms that have bias? Why aren't there regulations ensuring bias testing when humans themselves make decisions? In all cases the bias originates from human behaviour. The one advantage of AI being used is that now that bias is surfaced. But it's always been there, which is why the machine learning algorithms learn it. It's just that no one typically looks at the data in aggregate prior to AI. In any case, these rules should not be scoped to AI in my opinion. They should include algorithms and also require bias testing for teams of humans making decisions. If anything, I'd trust an AI system that has been analyzed for bias over a human making a decision.]

carbocation 2y ago
The behavior should be the start and the end of the regulation. No need to specify AI, algorithms, etc, at all.
doe_eyes 2y ago
> In all cases the bias originates from human behaviour. The one advantage of AI being used is that now that bias is surfaced. I think that's an odd way to look at it. The bias originates from the training data, which was what - idle conversations on Reddit and other similar content pilfered on the internet? The bias is doing some harm in places like that, but we're now using that to make real decisions. That's... worse.
- walt_grata 2y ago
  I think the point is that bias happens in all human behavior, but with AI it's more visible than in the general population. Which I'd say is a positive, but I don't believe if it offsets the downsides.
  - doe_eyes 2y ago
    But that's not a given. We conduct ourselves differently at work and in private life. Further, workplace processes are often designed to minimize bias. And here, we have a system that is trained on content that's two rungs above 4chan, and then coerced not to say racist things via RLHF. I don't think it's comparable at all.
- tensor 2y ago
  No, typically these sorts of models would not be trained on reddit, they would be trained on historical data that insurance companies and lending companies have. For example, a famous case is where they used AI to help decide if people will reoffend [1]. The bias here was already embedded in prior historical data. E.g. there is a level of systemic racism or bias built-in to the entire prison system. While HN is obsessed with LLMs, AI regulation targets more than just those systems. And yes, AI and machine learning were pervasive and used in a variety of useful settings long before these toy chat systems came about. Insurance and lending companies already have complex algorithms and formula they use to determine risk scores. Adding components of these systems that are learned from data shouldn't change the overall oversight mechanisms required of them in my opinion. [1] https://www.technologyreview.com/2019/01/21/137783/algorithms-criminal-justice-ai/ https://www.technologyreview.com/2019/01/21/137783/algorithm...
  - doe_eyes 2y ago
    People are obsessed with LLMs because they can be applied to unstructured data and produce human-like outputs, so they are applicable to vastly more processes with vastly less oversight. The kinds of "AI" you're describing have been with us for decades, but LLMs and their ilk are a more consequential development. And if we're passing laws today, that's what they will be regulating for the most part.
    - tensor 2y ago
      Which is what I'm say is wrong. If the oversight on insurance and other companies is lacking, it needs to be improved across the board not just with lip service to the popular technology of the day. The bias inherent is the exist risk scoring systems is absolutely equally if not more consequential than LLMs. Most of these regulations can replace "AI" with "technology, mathematical model, or human processes" and they will be 1000x better laws.
ethbr1 2y ago
> Why isn't it illegal to use traditional algorithms that have bias? Why aren't there regulations ensuring bias testing when humans themselves make decisions? It is and there are, but rule-based systems and humans have traditionally been more limited. You'd better bet that health insurers continually get audited in terms of claim pay rates and correctness. The major difference is that AI systems can be pre-audited at minimal cost. In contrast, human auditing traditionally consumes billable processing time and so is done post-hoc on production results. Honestly, I'm excited about these types of regulations. I think systems with more transparency, testability, and accountability are going to be better for society as a whole. If it took "cost savings" + "scary AI" to get us there (digitization/automation and proactive outcome auditing respectively), that is what it is.

by tensor 2y ago

carbocation 2y ago

The behavior should be the start and the end of the regulation. No need to specify AI, algorithms, etc, at all.

doe_eyes 2y ago

> In all cases the bias originates from human behaviour. The one advantage of AI being used is that now that bias is surfaced. I think that's an odd way to look at it. The bias originates from the training data, which was what - idle conversations on Reddit and other similar content pilfered on the internet? The bias is doing some harm in places like that, but we're now using that to make real decisions. That's... worse.

walt_grata 2y ago
I think the point is that bias happens in all human behavior, but with AI it's more visible than in the general population. Which I'd say is a positive, but I don't believe if it offsets the downsides.
- doe_eyes 2y ago
  But that's not a given. We conduct ourselves differently at work and in private life. Further, workplace processes are often designed to minimize bias. And here, we have a system that is trained on content that's two rungs above 4chan, and then coerced not to say racist things via RLHF. I don't think it's comparable at all.
tensor 2y ago
No, typically these sorts of models would not be trained on reddit, they would be trained on historical data that insurance companies and lending companies have. For example, a famous case is where they used AI to help decide if people will reoffend [1]. The bias here was already embedded in prior historical data. E.g. there is a level of systemic racism or bias built-in to the entire prison system. While HN is obsessed with LLMs, AI regulation targets more than just those systems. And yes, AI and machine learning were pervasive and used in a variety of useful settings long before these toy chat systems came about. Insurance and lending companies already have complex algorithms and formula they use to determine risk scores. Adding components of these systems that are learned from data shouldn't change the overall oversight mechanisms required of them in my opinion. [1] https://www.technologyreview.com/2019/01/21/137783/algorithms-criminal-justice-ai/ https://www.technologyreview.com/2019/01/21/137783/algorithm...
- doe_eyes 2y ago
  People are obsessed with LLMs because they can be applied to unstructured data and produce human-like outputs, so they are applicable to vastly more processes with vastly less oversight. The kinds of "AI" you're describing have been with us for decades, but LLMs and their ilk are a more consequential development. And if we're passing laws today, that's what they will be regulating for the most part.
  - tensor 2y ago
    Which is what I'm say is wrong. If the oversight on insurance and other companies is lacking, it needs to be improved across the board not just with lip service to the popular technology of the day. The bias inherent is the exist risk scoring systems is absolutely equally if not more consequential than LLMs. Most of these regulations can replace "AI" with "technology, mathematical model, or human processes" and they will be 1000x better laws.

ethbr1 2y ago

> Why isn't it illegal to use traditional algorithms that have bias? Why aren't there regulations ensuring bias testing when humans themselves make decisions? It is and there are, but rule-based systems and humans have traditionally been more limited. You'd better bet that health insurers continually get audited in terms of claim pay rates and correctness. The major difference is that AI systems can be pre-audited at minimal cost. In contrast, human auditing traditionally consumes billable processing time and so is done post-hoc on production results. Honestly, I'm excited about these types of regulations. I think systems with more transparency, testability, and accountability are going to be better for society as a whole. If it took "cost savings" + "scary AI" to get us there (digitization/automation and proactive outcome auditing respectively), that is what it is.