4 ms·

> They train on actual data often generated by humans > and care only about making the most accurate predictions possible of labels often generated by human

> They train on actual data often generated by humans > and care only about making the most accurate predictions possible of labels often generated by humans. Bias can get into a classifier. It can get there through a biased model, but that's very unlikely. Much more likely is that it's trained on biased data. Which is _easy_ to do, even by accident.

Houshalter 10y ago
Sure, but even then they would be no worse than the humans they replace. But in most cases, the whole point of using machine learned models is to do better than humans. Like an insurance company using ML to predict how likely a customer is to get in an accident. They aren't going to train the models to mimic actuaries, they have plenty of actual data to train it on. And it is quite possible that males get into accidents much more often than females. But that doesn't mean the model is prejudiced or that it's wrong.
- vintermann 10y ago
  > Sure, but even then they would be no worse than the humans they replace. That depends on two things: 1. How possible it is to inspect and criticize the judgements of the model, and its basis for it 2. How possible it is to inspect and criticize the judgements of the humans, and their basis for it I would say that 2 is the bigger problem all in all. But 1 can potentially still become a big problem if models are trusted blindly.
  - Houshalter 10y ago
    We can't inspect human brains, they are black boxes. People are incredibly biased, but also mostly unaware of their biases. For instance, judges have been found to give much harsher sentences just before lunch, when they are hungry. Or attractive people get much shorter sentences. In job interviews attractive people do better. It also matters way more in tipping waitresses and in elections. But on top of that, humans almost always do worse than even the most simple statistical baselines. Simple linear regression on a few relevant variables beats human 'experts' 99% of the time. Humans shouldn't be allowed to make decisions at all, yet everyone seems to fear teh scary algorithms instead.
    - daveloyall 10y ago
      Funny you should mention that human brains are black boxes... The research is about extracting information from black boxes. I want an AI to steal my model. And run it. Forever.

Houshalter 10y ago

Sure, but even then they would be no worse than the humans they replace. But in most cases, the whole point of using machine learned models is to do better than humans. Like an insurance company using ML to predict how likely a customer is to get in an accident. They aren't going to train the models to mimic actuaries, they have plenty of actual data to train it on. And it is quite possible that males get into accidents much more often than females. But that doesn't mean the model is prejudiced or that it's wrong.

vintermann 10y ago
> Sure, but even then they would be no worse than the humans they replace. That depends on two things: 1. How possible it is to inspect and criticize the judgements of the model, and its basis for it 2. How possible it is to inspect and criticize the judgements of the humans, and their basis for it I would say that 2 is the bigger problem all in all. But 1 can potentially still become a big problem if models are trusted blindly.
- Houshalter 10y ago
  We can't inspect human brains, they are black boxes. People are incredibly biased, but also mostly unaware of their biases. For instance, judges have been found to give much harsher sentences just before lunch, when they are hungry. Or attractive people get much shorter sentences. In job interviews attractive people do better. It also matters way more in tipping waitresses and in elections. But on top of that, humans almost always do worse than even the most simple statistical baselines. Simple linear regression on a few relevant variables beats human 'experts' 99% of the time. Humans shouldn't be allowed to make decisions at all, yet everyone seems to fear teh scary algorithms instead.
  - daveloyall 10y ago
    Funny you should mention that human brains are black boxes... The research is about extracting information from black boxes. I want an AI to steal my model. And run it. Forever.