8 ms·

Aha - I think I see our miscommunication. When you say bias you mean statistical bias. Yes, machine learning is able to correct for that kind of bias - 538's p

by wrsh07 10y ago

Aha - I think I see our miscommunication. When you say bias you mean statistical bias. Yes, machine learning is able to correct for that kind of bias - 538's polls forecast is a good example of that. But you don't get to redefine racial bias to be something innocuous. Yes, black names are more likely to have arrest records, but that "fact" is super misleading [1]. Finally, you're talking past me. I'm not saying that statistics is broken. I'm saying that we should be especially mindful of the OPs point when they say this: > So what’s your data being fried in? These algorithms train on large collections that you know nothing about. Sites like Google operate on a scale hundreds of times bigger than anything in the humanities. Any irregularities in that training data end up infused into in the classifier. I think the OP author also has a related post about the kind of bias I'm talking about: http://idlewords.com/talks/sase_panel.htm http://idlewords.com/talks/sase_panel.htm [1]: http://www.huffingtonpost.com/kim-farbota/black-crime-rates-your-st_b_8078586.html http://www.huffingtonpost.com/kim-farbota/black-crime-rates-...

yummyfajitas 10y ago
Without getting into a dispute about the definition of "bias", I'm saying that algorithms can accurately measure reality even if input(x=white, all else equal) != input(x=black, all else equal). You are saying that algorithms are accurately measuring a reality you wish were different. I don't disagree with this. The right thing to do is to actually answer unpleasant moral questions like "if blacks are 4x more likely to be dangerous criminals, what should we do about it?" But I guess overloading the word "bias" is a nice substitute for clearly thinking things through.
- srean 10y ago
  Are you saying that it can form a good estimate of the conditional probability ? I can believe that if the sampling process preserves the conditional. Otherwise one would have to make assumptions about (or in other words, model) the corruption process. The bias compensation machinery then has to be deliberate, wont happen on its own. Some sampling processes do not modify the conditional. In those cases no special machinery would be required.
  - yummyfajitas 10y ago
    tOne approach is to directly model the corruption process. Being the model-based-Bayesian guy I am, this is something I like to do. But if your model is sufficiently expressive you don't need to explicitly build or model the corruption process. In the example in my linked blog post, test scores might be biased against blacks. But race is also redundantly encoded, so the algorithm has enough information to fix the bias completely by accident. Fundamentally what I'm saying here is that bias is a statistics problem and has a statistics solution. Insofar as your complaint is algorithms finding the wrong answer, the solution is better stats. And nothing whatsoever that I've said here would be remotely controversial if the topic were remote sensing.
    - srean 10y ago
      > But if your model is sufficiently expressive you don't need to explicitly build or model the corruption process This is the claim that I am having trouble with. Say I have two random variable X,Y with some joint distribution. If a corruption process can mess with the samples drawn from it, I cannot see how it could possibly recover either the joint or the conditional. Are you saying that the corruption is benign like missing at random or missing completely at random ? Then its much more believable.
      - yummyfajitas 10y ago
        So we both agree that if the bias is linear, and your model is linear, you capture it. Similarly if the model involves interaction (score x is_black), and you include linear interaction terms, you'll also capture it. Now the question arises; what if things are more complex? In real life they always are; both your biasing factor and the rest of the model. So we've cooked up all sorts of fun models like SVMs, random forests and neural networks to analyze such complicated models and find hidden features and relations that we didn't think of. Bias is one such feature. If I built an algorithm that learned to display different ads to mobile and desktop people (i.e., treat mobile "time on site" differently from desktop "time on site"), would you be surprised by this?
- eridius 10y ago
  The problem is you're modeling a biased reality. And accurately modeling a biased reality may in many cases accentuate the bias. Take for example the previously-mentioned case of using an algorithm to determine where to focus your policing efforts. If the data you have says that more arrests are done in a particular part of the city, then you'll want to put more police there, right? But areas where there are more police will tend to see more arrests. So the fact that you're putting more police in an area where you see more arrests is just going to make the bias more extreme, causing even more arrests there. This causes a feedback loop. So you may be accurately modeling reality, but you're modeling a pre-existing bias and making it worse. And who knows why that pre-existing bias was even there? The fact that there were more arrests there may not be because that area actually has more crime committed, it could be due to other factors, such as racial profiling by police, and in that case your algorithm is now accidentally racist because it's perpetuating racial profiling.
  - dragonwriter 10y ago
    The problems are really twofold: (1) Defining the proper goals, and (2) Measuring the right things (such as the real goals of interest rather than biased proxies.) With police deployments, you are assuming the solution (rather than letting your algorithm optimize it) by saying "I want to put more police where more arrests occur". What you really want is probably something more like (the exact goal may be different, of course) "I want to deploy police resources where it will most effectively reduce the incidence of crime, weighted by some assigned measure of severity." Then let your ML algorithm crunch the various measurable factors and produce an optimum deployment to do that. (But, then again with that goal -- and similar problems exist with many likely real goals -- you run into the other problem, which is measuring the incidence of crime -- measuring crime reports may be the obvious approach, but there's plenty of evidence that lots of factors can bias crime reports, including communities having bad experience with police being less likely to report crimes.)
    - wrsh07 10y ago
      Thank you. This is so much clearer than what I was saying. As you say, proper goals and measurement can fix a lot of these problems, and I don't think it's obvious that ml algorithms solve either of those

by wrsh07 10y ago

yummyfajitas 10y ago

Without getting into a dispute about the definition of "bias", I'm saying that algorithms can accurately measure reality even if input(x=white, all else equal) != input(x=black, all else equal). You are saying that algorithms are accurately measuring a reality you wish were different. I don't disagree with this. The right thing to do is to actually answer unpleasant moral questions like "if blacks are 4x more likely to be dangerous criminals, what should we do about it?" But I guess overloading the word "bias" is a nice substitute for clearly thinking things through.

srean 10y ago
Are you saying that it can form a good estimate of the conditional probability ? I can believe that if the sampling process preserves the conditional. Otherwise one would have to make assumptions about (or in other words, model) the corruption process. The bias compensation machinery then has to be deliberate, wont happen on its own. Some sampling processes do not modify the conditional. In those cases no special machinery would be required.
- yummyfajitas 10y ago
  tOne approach is to directly model the corruption process. Being the model-based-Bayesian guy I am, this is something I like to do. But if your model is sufficiently expressive you don't need to explicitly build or model the corruption process. In the example in my linked blog post, test scores might be biased against blacks. But race is also redundantly encoded, so the algorithm has enough information to fix the bias completely by accident. Fundamentally what I'm saying here is that bias is a statistics problem and has a statistics solution. Insofar as your complaint is algorithms finding the wrong answer, the solution is better stats. And nothing whatsoever that I've said here would be remotely controversial if the topic were remote sensing.
  - srean 10y ago
    > But if your model is sufficiently expressive you don't need to explicitly build or model the corruption process This is the claim that I am having trouble with. Say I have two random variable X,Y with some joint distribution. If a corruption process can mess with the samples drawn from it, I cannot see how it could possibly recover either the joint or the conditional. Are you saying that the corruption is benign like missing at random or missing completely at random ? Then its much more believable.
    - yummyfajitas 10y ago
      So we both agree that if the bias is linear, and your model is linear, you capture it. Similarly if the model involves interaction (score x is_black), and you include linear interaction terms, you'll also capture it. Now the question arises; what if things are more complex? In real life they always are; both your biasing factor and the rest of the model. So we've cooked up all sorts of fun models like SVMs, random forests and neural networks to analyze such complicated models and find hidden features and relations that we didn't think of. Bias is one such feature. If I built an algorithm that learned to display different ads to mobile and desktop people (i.e., treat mobile "time on site" differently from desktop "time on site"), would you be surprised by this?
eridius 10y ago
The problem is you're modeling a biased reality. And accurately modeling a biased reality may in many cases accentuate the bias. Take for example the previously-mentioned case of using an algorithm to determine where to focus your policing efforts. If the data you have says that more arrests are done in a particular part of the city, then you'll want to put more police there, right? But areas where there are more police will tend to see more arrests. So the fact that you're putting more police in an area where you see more arrests is just going to make the bias more extreme, causing even more arrests there. This causes a feedback loop. So you may be accurately modeling reality, but you're modeling a pre-existing bias and making it worse. And who knows why that pre-existing bias was even there? The fact that there were more arrests there may not be because that area actually has more crime committed, it could be due to other factors, such as racial profiling by police, and in that case your algorithm is now accidentally racist because it's perpetuating racial profiling.
- dragonwriter 10y ago
  The problems are really twofold: (1) Defining the proper goals, and (2) Measuring the right things (such as the real goals of interest rather than biased proxies.) With police deployments, you are assuming the solution (rather than letting your algorithm optimize it) by saying "I want to put more police where more arrests occur". What you really want is probably something more like (the exact goal may be different, of course) "I want to deploy police resources where it will most effectively reduce the incidence of crime, weighted by some assigned measure of severity." Then let your ML algorithm crunch the various measurable factors and produce an optimum deployment to do that. (But, then again with that goal -- and similar problems exist with many likely real goals -- you run into the other problem, which is measuring the incidence of crime -- measuring crime reports may be the obvious approach, but there's plenty of evidence that lots of factors can bias crime reports, including communities having bad experience with police being less likely to report crimes.)
  - wrsh07 10y ago
    Thank you. This is so much clearer than what I was saying. As you say, proper goals and measurement can fix a lot of these problems, and I don't think it's obvious that ml algorithms solve either of those