4 ms·

I feel like the implications of what she was saying are a little vague, so I'm not sure what she meant here. I think the main point is that this is still early

• xyzzyz 9y ago
  I don't understand what you are proposing here. The way things are done right now is we define some specific features which can include things like age, height, eye color, skin color, default history, average time a person wakes up, whether they prefer pancakes or waffles, income, and pretty much anything else you come up with. Then, we obtain a training set containing instances of the specific values of the features paired with the output variable we want to predict. Then, we pick a model and train it using this data. After training, we can test the model on some new data and measure its accuracy, that is, how often it is wrong, what is the direction of the error (i.e. bias), if it's wrong, by how much (variance) etc. Some of the variables might turn out to be more predictive, and some variables probably quite a bit less. Some variables might turn out to be predictive even though one would guess they shouldn't. For example, people with brown eyes are more likely to be victims of crime than people with blue eyes. It might seem like a mystery, until you realize that black people overwhelmingly have brown eyes, and they are more likely to be a victim of crime than white people. Here, removing race as a variable will only make you and your model more confused. It is important to realize that removing predictive variable will not necessarily make your model less "biased" -- in fact, it is likely to make your model more often wrong than before, and therefore less useful. The difficulty here stems from the fact that often laws require you to make "non-discriminatory" decisions, and if you attempt to do that, for example by introducing counteracting bias to the bias you get from reality, your model will be less predictive and so less useful. Fairness in general is very complicated issue. Suppose our model tells us that blue agents are much more likely to default on loan, and this is in fact the underlying reality. The creditor, to maximize his profit, should therefore charge blue agents higher interest than red agents. If you are blue agents, this is obviously unfair for you -- it's not your fault some other agents who share similar characteristic default more often. However, if we have laws forbidding discrimination based on agent's color, the creditor must then charge both colors of agents equally, and since one is more likely to default, this interest rate would be higher than what red agents would pay if the anti-discriminatory laws weren't put in place. Isn't it unfair towards red agents then? After all, it's not your fault that some other agents who share some characteristic default more often...
  • zengid 9y ago
    >> Fairness in general is very complicated issue. That's what I'm getting at. >>Suppose our model tells us that blue agents are much more likely to default on loan, and this is in fact the underlying reality. That's a cute thought experiment. Unfortunately, it is assuming human behaviors are caused by skin color. I do not believe that is 'the underlying reality'.
    • xyzzyz 9y ago
      The experiment doesn't mention humans, nor talks about anything being "caused" by skin color -- agents don't have skins, for that matter, they have steel armors. Talking about humans here is counterproductive, because people tend to get quite emotional and irrational when talking about these topics, so it is good to focus on abstract examples. You might ask what is the reason the blue agents have higher default rate, and this might provoke an interesting discussion, but it does concern the creditor only as far as the interest rate they need to charge to make money is concerned. The creditor does not care if there is some "fair" or "unfair" reason why blue agents default more often, they just want to make money by focusing on what they know best, which is giving loans and collecting interest.
    • swsieber 9y ago
      No, it doesn't. It's assuming human behaviors correlate with skin color. Which it often does. Correlation != Causation. The real causes are more likely things that happen to be disproportionately manifest among certain demagraphics - in this case, blue agents: - Living in an area with a higher cost of living - Societal expectations (negative - expectations of negative behaviors) - A lower average salary - Culture? (family size, fiscal knowledge / habits / etc) - And probably a lot more. Yet on average, it might be the fact that if you picked a blue agent with a loan, that loan would be more likely to default, than a random loan from the other agents. Unfair? yes. Reality? yes.
      • zengid 9y ago
        I know correlation != causation, but we're using imprecise language where these mathematical distinctions are mired and muddled. People don't think like statisticians, they think like humans who label things and create biases based on these averages without looking at the individuals. If we build these intricate models for predictions that are statistically minded, are we improving our predictive ability or are we just regurgitating the same flawed biases that we ourselves hold? (When I say flawed, I mean unfair.) What if blue agents were the victims of institutional racism for hundreds of years? Obviously, their lack of opportunities will have hampered their abilities to earn high credit scores and obtain legitimate forms of income. What then? It's as if were building models that measure agents who exist in two separate contexts predicting which one belongs to the 'socially acceptable' context, aka, which one belongs to the class of agents who were the perpetrators of social injustice for centuries. Is that really what we want to predict? No, we're trying to predict how trustworthy an individual is. This might mean that our standard techniques need reassessment if we're going to build models that affect the lives of future generations.