4 ms·

It is impossible for an algorithm to "illegally" discriminate, as it's a machine devoid of the human concept of "intent"; It doesn't know what "race" even means

by fein 9y ago

It is impossible for an algorithm to "illegally" discriminate, as it's a machine devoid of the human concept of "intent"; It doesn't know what "race" even means. So this narrows to these basic outcomes if we assume any given algorithm isn't purpose built to always have a discriminatory output: 1. The input data is junk data, so the output is junk data. 2. The input data is fine and humans don't like the output for human reasons. The problem I'm hitting on here is that if we assume any sort of "discrimination" based on a data set is "bad", then we don't really care what the outcome of a given algo is, and we're just trying to massage data sets into an "acceptable" output. Seems like the opposite of good science to just throw away results that we don't like, because reasons.

CPLX 9y ago
This is nonsense. It's quite possible for an algorithm that doesn't know what race is to be biased despite ignoring race if the real world is full of racism, and that racism manifests in dependent variables that the algorithm bases its decision on. This concept isn't even a little novel. Google the phrase "previous servitude" for an example of just such a dependent variable in action.
- fein 9y ago
  In this case, you seem to be implying garbage in/ garbage out. My point is that when you get a good input dataset and there is still some sort of bias in the output, you have to start asking questions on why the results are biased towards one group by examining the variables pertaining to that group. The position you're starting with seems to be that the output should never be biased, which means you're not really approaching this scientifically and won't stop messing with input sets until you get the output you agree with; not the output which is necessarily true but you disagree with for personal reasons. You can't just claim "bad data" every time you get a result that doesn't jive with your politics.
  - CPLX 9y ago
    What I am saying is much simpler. You said that the machine can't discriminate because the machine doesn't know what race is. I'm just pointing out that the machine doesn't know what anything at all is. All it can do is work with what humans are telling it. If the real world is biased it can then itself be biased. There are many feedback loops in societies. The rich get richer and the poor get poorer and those subjected to discrimination often have objectively worse outcomes. An algorithm can very easily be susceptible to just recursively documenting actual bias that exists in the real world and giving it the veneer of objectivity. It's an actual hard problem that is the topic of this thread, hand waving isn't going to fix it.
    - fein 9y ago
      I think the problem is that you're looking at "bias" as a dirty word. > If the real world is biased it can then itself be biased. And if said algorithm corroborates that bias, then is reality somehow "bad" as a result, or do you just not like the outcome because it "feels bad"? There is a difference between truth and an idealized outcome that you desperately want to be true.
      - saas_co_de 9y ago
        None of that is relevant to what the law is.
      - deleted 9y ago
        [deleted]
      - CPLX 9y ago
        Imagine a world ruled by blue people, who control all the roads. Every time a green person tries to drive a blue person will throw rocks at their car, pour oil in front of the wheels, etc. Also for historical reasons all the green people live in certain neighborhoods. Now imagine creating bureau that would review driving and accident data and attempt to answer the question "How good at driving is this person?" The idea being that we are going to try to keep the worst drivers off the road. People scream hey wait a second that's not fair at all. So you take steps to address this whole system instead and solve the real problem. You try to crack down on all the rock throwing, and not prejudge someone without knowing the circumstances. Until one day an algorithm comes along to answer the question. But wait, we thought of that, we make it so the algorithm can't take into account what color the driver is. It can only rely on other things. See the problem yet?
saas_co_de 9y ago
> It is impossible for an algorithm to "illegally" discriminate An algorithm is a set of steps given to it by a human. If the human can discriminate then so can the algorithm. In the case of ML, let's say you are making lending decisions and your data set consists entirely of the race of the applicants. Obviously that ML algorithm would illegally discriminate. Now, extrapolate from that extreme case and say that you feed your ML algorithm a whole bunch of random data points about your potential borrowers. How do you prove that your ML algorithm is not making its decision based on data points that are proxies for race? If you don't have an answer to that question good luck with the swarm of lawyers who will be suing you into oblivion.

by fein 9y ago

CPLX 9y ago

This is nonsense. It's quite possible for an algorithm that doesn't know what race is to be biased despite ignoring race if the real world is full of racism, and that racism manifests in dependent variables that the algorithm bases its decision on. This concept isn't even a little novel. Google the phrase "previous servitude" for an example of just such a dependent variable in action.

fein 9y ago
In this case, you seem to be implying garbage in/ garbage out. My point is that when you get a good input dataset and there is still some sort of bias in the output, you have to start asking questions on why the results are biased towards one group by examining the variables pertaining to that group. The position you're starting with seems to be that the output should never be biased, which means you're not really approaching this scientifically and won't stop messing with input sets until you get the output you agree with; not the output which is necessarily true but you disagree with for personal reasons. You can't just claim "bad data" every time you get a result that doesn't jive with your politics.
- CPLX 9y ago
  What I am saying is much simpler. You said that the machine can't discriminate because the machine doesn't know what race is. I'm just pointing out that the machine doesn't know what anything at all is. All it can do is work with what humans are telling it. If the real world is biased it can then itself be biased. There are many feedback loops in societies. The rich get richer and the poor get poorer and those subjected to discrimination often have objectively worse outcomes. An algorithm can very easily be susceptible to just recursively documenting actual bias that exists in the real world and giving it the veneer of objectivity. It's an actual hard problem that is the topic of this thread, hand waving isn't going to fix it.
  - fein 9y ago
    I think the problem is that you're looking at "bias" as a dirty word. > If the real world is biased it can then itself be biased. And if said algorithm corroborates that bias, then is reality somehow "bad" as a result, or do you just not like the outcome because it "feels bad"? There is a difference between truth and an idealized outcome that you desperately want to be true.
    - saas_co_de 9y ago
      None of that is relevant to what the law is.
    - deleted 9y ago
      [deleted]
    - CPLX 9y ago
      Imagine a world ruled by blue people, who control all the roads. Every time a green person tries to drive a blue person will throw rocks at their car, pour oil in front of the wheels, etc. Also for historical reasons all the green people live in certain neighborhoods. Now imagine creating bureau that would review driving and accident data and attempt to answer the question "How good at driving is this person?" The idea being that we are going to try to keep the worst drivers off the road. People scream hey wait a second that's not fair at all. So you take steps to address this whole system instead and solve the real problem. You try to crack down on all the rock throwing, and not prejudge someone without knowing the circumstances. Until one day an algorithm comes along to answer the question. But wait, we thought of that, we make it so the algorithm can't take into account what color the driver is. It can only rely on other things. See the problem yet?

saas_co_de 9y ago

> It is impossible for an algorithm to "illegally" discriminate An algorithm is a set of steps given to it by a human. If the human can discriminate then so can the algorithm. In the case of ML, let's say you are making lending decisions and your data set consists entirely of the race of the applicants. Obviously that ML algorithm would illegally discriminate. Now, extrapolate from that extreme case and say that you feed your ML algorithm a whole bunch of random data points about your potential borrowers. How do you prove that your ML algorithm is not making its decision based on data points that are proxies for race? If you don't have an answer to that question good luck with the swarm of lawyers who will be suing you into oblivion.