3 ms·

> The fact is that machine learning takes biased inputs and produces unbiased outputs That is not necessarily true, because it assumes that the biased inputs h

by SamBam 10y ago

> The fact is that machine learning takes biased inputs and produces unbiased outputs That is not necessarily true, because it assumes that the biased inputs have no influence on the outputs. Suppose the training data is a bank that has historically given higher-interest loans to minorities. Those loans will default more often, and so the algorithm could "correctly" conclude that minorities default more often. Cause and effect are very hard for an algorithm to distinguish when it's the clustered input variables that are linked together for some opaque reason. The fact is, both statements in this example would be true: "higher interest is correlated with more defaults" and "minority borrowers are correlated with defaults." The algorithm doesn't know one from the other. Sure, more training data could help the algorithm distinguish between the two, but there may not be enough training data that shows the inverse -- low-interest loans given to minorities -- if the historical data is biased enough.

yummyfajitas 10y ago
Suppose the training data is a bank that has historically given higher-interest loans to minorities. Those loans will default more often, and so the algorithm could "correctly" conclude that minorities default more often. But if the relationship is actually `defaultProb = a x interest_rate + b`, then the algorithm will reflect that. I discuss this case explicitly in the blog post I linked to in the section "What if black people don't perform as well?", and explicitly give an example of a linear model NOT doing what you claim it will do. The fact is, both statements in this example would be true: "higher interest is correlated with more defaults" and "minority borrowers are correlated with defaults." The algorithm doesn't know one from the other. No, it wouldn't because you'd also have cases of white people with high interest who defaulted. The model using race as a predictive variable would fail to fit those data points. I explicitly discuss this case in my blog post. You are correct that in some cases you don't have enough training data. But in these cases, there is no particular reason for bias to have a particular sign. Again, as I discuss in the blog post I linked, why would Captain Kirk be biased against "black on left guy" vs "black on right guy"? Insufficient training data causes variance, not bias, and could just as easily result in bias in favor of blacks. (In fact, I link to a number of examples where it does.)
- ramblenode 10y ago
  > Insufficient training data causes variance, not bias Key point. If one were to compare the two high-interest groups, a small sample for one group would be reflected in a large confidence interval around its mean, a greater overlap in the CIs of both group means, and less confidence that a real difference exists.

by SamBam 10y ago

yummyfajitas 10y ago

Suppose the training data is a bank that has historically given higher-interest loans to minorities. Those loans will default more often, and so the algorithm could "correctly" conclude that minorities default more often. But if the relationship is actually `defaultProb = a x interest_rate + b`, then the algorithm will reflect that. I discuss this case explicitly in the blog post I linked to in the section "What if black people don't perform as well?", and explicitly give an example of a linear model NOT doing what you claim it will do. The fact is, both statements in this example would be true: "higher interest is correlated with more defaults" and "minority borrowers are correlated with defaults." The algorithm doesn't know one from the other. No, it wouldn't because you'd also have cases of white people with high interest who defaulted. The model using race as a predictive variable would fail to fit those data points. I explicitly discuss this case in my blog post. You are correct that in some cases you don't have enough training data. But in these cases, there is no particular reason for bias to have a particular sign. Again, as I discuss in the blog post I linked, why would Captain Kirk be biased against "black on left guy" vs "black on right guy"? Insufficient training data causes variance, not bias, and could just as easily result in bias in favor of blacks. (In fact, I link to a number of examples where it does.)

ramblenode 10y ago
> Insufficient training data causes variance, not bias Key point. If one were to compare the two high-interest groups, a small sample for one group would be reflected in a large confidence interval around its mean, a greater overlap in the CIs of both group means, and less confidence that a real difference exists.