4 ms·

This proposal shows more concern about being politically correct than actually correct. A truly unbiased system would just make the best analysis possible given

This proposal shows more concern about being politically correct than actually correct. A truly unbiased system would just make the best analysis possible given the data available. What you propose is to make a biased system in order to give an advantage to one group of people rather than the other group. This type of political correctness fuels resentment.

aoki 10y ago
consider danah's example in which external studies show that blacks are convicted for drug crimes at a rate disproportionate to the actual crime incidence rate. the fact that the easily-available training data (court stats) is unrepresentative of the underlying process (crime incidence) is a problem that has to be corrected or predictive output will also over-represent them. the point is that this prior or correction is also part of the "data available" but you have to actually know something about the problem domain to know about it; you can't just grab a court data dump and pour it into a model and say, "welp, that was the data available to me, it's not biased."
cortesoft 10y ago
What they are saying is that, if you know your data is biased, you have the option of designing your system to compensate for that bias. Why do you seem to think that you have to just throw your hands up in the air and accept the bias? Wouldn't the 'best possible analysis given the data available' include compensating for known biases in the data? If I was designing a system for analyzing temperature data, for example, and I knew that every measurement was 10 degrees warmer than the actual temperature, shouldn't my analysis system compensate for that?
- ZoF 10y ago
  The issue here is how am I as a software developer supposed to combat that bias? Given the contrived arrest rate example above, do I multiply every black arrest by .7 to compensate for bias? How would a system built this way be useful in practice? Why are these parallels always drawn against racial divides as well? How are we compensating for the poor white kid missing out or the rich black kid getting benefits he might not need? It's such a messy problem but I don't think building bias into software is the solution. I prefer factually correct, biased systems to incorrect, still-biased-just-in-another-way systems.
  - GavinMcG 10y ago
    > The issue here is how am I as a software developer supposed to combat that bias? Yes. And that is exactly the question you should ask yourself in work you do. It's on all of us to unroot social ills. Even though it's thankless and difficult work.
    - ZoF 10y ago
      Amorphously defined and not at all actable. Doesn't sound like hacking to me. Thanks but I'll take my code without the side of social justice warrior.
      - NetTechM 10y ago
        Well said, I am not a judge, nor do I plan to pour over millions of criminal arrests to make judgment calls (which would negate the purpose of automation)... All we can work with is the data given.
shkkmo 10y ago
If the "data available" is biased, an unbiased system must compensate for the biases of the data it is using. Otherwise you are failing to be "actually correct".
- selectron 10y ago
  If the data is truly biased in that it doesn't reflect reality then sure, go ahead and correct for this bias. But make sure you are not introducing your own biases which are wrong. "But is it OK that we’re using extraordinarily biased data about previous arrests to predict future arrests and determine where police are stationed?" The truth is that men living in certain areas are much more likely to commit violent crime and therefore get arrested than the average person in the United States. This is the sad reality of the world we live in. It isn't wrong to deploy police resources where they are most needed. If the data shows that these areas are no longer hotspots of crime, then we can reroute resources elsewhere.
  - shkkmo 10y ago
    We want to know where crime rates are higher so we can target our limited enforcement budget to where it will have the greatest effect. The whole point of targeting enforcement at problem areas is that it introduces a bias to the rate at which committed crimes lead to arrests and convictions. This is a good thing. The problem is that using arrest/conviction rates to predict crime rates will be inaccurate if you don't correct for the biases that exist in the ways in which crimes lead to arrests and convictions.
deleted 10y ago
[deleted]
Joof 10y ago
If the data itself is bad and known to be bad, the results will reflect the bad data. In the case of policing, the arrest record is racially biased (the US releases data that shows this bias and it's well researched). If you train on arrest records, it won't predict actual crime because the results will be racially biased. Being unbiased is actually a very difficult problem. Especially when we are predicting things rooted in social / cultural trends.
- teacup50 10y ago
  There's a difference between something being "offensive" because it's incorrect, and being "incorrect" because it's offensive. Our job is to be correct. If we avoid offense in the process, great.
- deleted 10y ago
  [deleted]

aoki 10y ago

consider danah's example in which external studies show that blacks are convicted for drug crimes at a rate disproportionate to the actual crime incidence rate. the fact that the easily-available training data (court stats) is unrepresentative of the underlying process (crime incidence) is a problem that has to be corrected or predictive output will also over-represent them. the point is that this prior or correction is also part of the "data available" but you have to actually know something about the problem domain to know about it; you can't just grab a court data dump and pour it into a model and say, "welp, that was the data available to me, it's not biased."

cortesoft 10y ago

What they are saying is that, if you know your data is biased, you have the option of designing your system to compensate for that bias. Why do you seem to think that you have to just throw your hands up in the air and accept the bias? Wouldn't the 'best possible analysis given the data available' include compensating for known biases in the data? If I was designing a system for analyzing temperature data, for example, and I knew that every measurement was 10 degrees warmer than the actual temperature, shouldn't my analysis system compensate for that?

ZoF 10y ago
The issue here is how am I as a software developer supposed to combat that bias? Given the contrived arrest rate example above, do I multiply every black arrest by .7 to compensate for bias? How would a system built this way be useful in practice? Why are these parallels always drawn against racial divides as well? How are we compensating for the poor white kid missing out or the rich black kid getting benefits he might not need? It's such a messy problem but I don't think building bias into software is the solution. I prefer factually correct, biased systems to incorrect, still-biased-just-in-another-way systems.
- GavinMcG 10y ago
  > The issue here is how am I as a software developer supposed to combat that bias? Yes. And that is exactly the question you should ask yourself in work you do. It's on all of us to unroot social ills. Even though it's thankless and difficult work.
  - ZoF 10y ago
    Amorphously defined and not at all actable. Doesn't sound like hacking to me. Thanks but I'll take my code without the side of social justice warrior.
    - NetTechM 10y ago
      Well said, I am not a judge, nor do I plan to pour over millions of criminal arrests to make judgment calls (which would negate the purpose of automation)... All we can work with is the data given.

shkkmo 10y ago

If the "data available" is biased, an unbiased system must compensate for the biases of the data it is using. Otherwise you are failing to be "actually correct".

selectron 10y ago
If the data is truly biased in that it doesn't reflect reality then sure, go ahead and correct for this bias. But make sure you are not introducing your own biases which are wrong. "But is it OK that we’re using extraordinarily biased data about previous arrests to predict future arrests and determine where police are stationed?" The truth is that men living in certain areas are much more likely to commit violent crime and therefore get arrested than the average person in the United States. This is the sad reality of the world we live in. It isn't wrong to deploy police resources where they are most needed. If the data shows that these areas are no longer hotspots of crime, then we can reroute resources elsewhere.
- shkkmo 10y ago
  We want to know where crime rates are higher so we can target our limited enforcement budget to where it will have the greatest effect. The whole point of targeting enforcement at problem areas is that it introduces a bias to the rate at which committed crimes lead to arrests and convictions. This is a good thing. The problem is that using arrest/conviction rates to predict crime rates will be inaccurate if you don't correct for the biases that exist in the ways in which crimes lead to arrests and convictions.

deleted 10y ago

[deleted]

Joof 10y ago

If the data itself is bad and known to be bad, the results will reflect the bad data. In the case of policing, the arrest record is racially biased (the US releases data that shows this bias and it's well researched). If you train on arrest records, it won't predict actual crime because the results will be racially biased. Being unbiased is actually a very difficult problem. Especially when we are predicting things rooted in social / cultural trends.

teacup50 10y ago
There's a difference between something being "offensive" because it's incorrect, and being "incorrect" because it's offensive. Our job is to be correct. If we avoid offense in the process, great.
deleted 10y ago
[deleted]