4 ms·

I have a really hard time following this kind of reasoning, partly because the word bias has very different meanings in social science and statistics. E.g. let’

I have a really hard time following this kind of reasoning, partly because the word bias has very different meanings in social science and statistics. E.g. let’s say I have a statistical model that takes among other variables a person’s ethnicity and produces a credit score. If that model is statistically unbiased it would probably produce different credit scores depending on ethnicity. If we “fix” that by making the model insensitive to ethnicity, then it probably becomes statistically biased. Wouldn’t it be better to talk about “fairness” or something?

Spivak 5y ago
You’re viewing “fixing” from the wrong lens. There are infinite bits of information about a person that could potentially be used in a model to calculate a credit score. At all times in every model you’re choosing a subset of one’s facets and can build statistically unbiased models based on the data you make available. But when you talking actually fixing models like this you’re actually forced to correct the final result, not filter the data. Being blind to ethnicity doesn’t work because one’s ethnicity permeates (to different degrees, sure) every part of their lives. All the data is bad, everything is a statistically detectable proxy for ethnicity.
- jhgb 5y ago
  But surely there's some kind of refining your inputs where all the paths converge to one result, much like the refinement of interval partitions in Riemann's integral converges to one value (for a certain class of functions). The bits of information may be infinite but there's some structure in them. I'm not sure that just because the optimal result (the one that uses all information) is something you don't like, the result is bad. Best thing you can say is that you're actually not predicting an existing credit score, but rather synthesizing some other indicator that is not an existing credit score.
BoiledCabbage 5y ago
Agreed, overloading of terms often makes topics much harder to grok. Quick summary, is there are many meanings of the term bias in ML. First let's define which the author is talking about. Some options are: 1. The idea of ignoring relevant features, ala bias-variance trade off. 2. Data bias, where data is trained on inputs / behavior that society would deem biased. Ex. A company frequently discriminates against group X, an ML model is trained on data / decisions this company has made and as a result encodes the same biases. 3. Features are used which are a proxy for other items. Ex. (pulling this out of the air), a repair shop charges a variable rate for repair. It takes in a number of inputs. It's illegal to offer higher prices to women for the same repair as men, but instead an ML model uses as input the customer's first name which ends up being nothing more than a proxy for their gender and introduces that bias. 4. A group of people A have a much smaller portion of the training data size in an ML model than group B. The model makes decisions for offering some good (ex. a loan). Because it's training data is larger on people from group B than group A it always has more confidence in its predictions from group B. The model is set to be risk adverse, so by default it rejects the majority of people from group A even though most of them would have been valid loans. (Where A and B are different across some societal class like religion, race, ethnicity, gender...) The author is talking about the fourth case. While yes there is over/under representation in the input data, the algorithm chosen to build the model encodes its own form of bias. A simple example everyone should be familiar with, using Mean Squared Error vs. Mean Absolute Error (or MBE) produce models that handle rarely seen features or data points very differently. This choice of loss function introduces algorithmic bias into a model. That's essentially what underlies the concept. Yes there is data bias, but what is being argued is that the solution isn't "find perfectly unbiased data" (because in many cases that's impossible), it's understand how the choice of algorithm will produce models that respond to and represent differently those data biases.

Spivak 5y ago

You’re viewing “fixing” from the wrong lens. There are infinite bits of information about a person that could potentially be used in a model to calculate a credit score. At all times in every model you’re choosing a subset of one’s facets and can build statistically unbiased models based on the data you make available. But when you talking actually fixing models like this you’re actually forced to correct the final result, not filter the data. Being blind to ethnicity doesn’t work because one’s ethnicity permeates (to different degrees, sure) every part of their lives. All the data is bad, everything is a statistically detectable proxy for ethnicity.

jhgb 5y ago
But surely there's some kind of refining your inputs where all the paths converge to one result, much like the refinement of interval partitions in Riemann's integral converges to one value (for a certain class of functions). The bits of information may be infinite but there's some structure in them. I'm not sure that just because the optimal result (the one that uses all information) is something you don't like, the result is bad. Best thing you can say is that you're actually not predicting an existing credit score, but rather synthesizing some other indicator that is not an existing credit score.

BoiledCabbage 5y ago

Agreed, overloading of terms often makes topics much harder to grok. Quick summary, is there are many meanings of the term bias in ML. First let's define which the author is talking about. Some options are: 1. The idea of ignoring relevant features, ala bias-variance trade off. 2. Data bias, where data is trained on inputs / behavior that society would deem biased. Ex. A company frequently discriminates against group X, an ML model is trained on data / decisions this company has made and as a result encodes the same biases. 3. Features are used which are a proxy for other items. Ex. (pulling this out of the air), a repair shop charges a variable rate for repair. It takes in a number of inputs. It's illegal to offer higher prices to women for the same repair as men, but instead an ML model uses as input the customer's first name which ends up being nothing more than a proxy for their gender and introduces that bias. 4. A group of people A have a much smaller portion of the training data size in an ML model than group B. The model makes decisions for offering some good (ex. a loan). Because it's training data is larger on people from group B than group A it always has more confidence in its predictions from group B. The model is set to be risk adverse, so by default it rejects the majority of people from group A even though most of them would have been valid loans. (Where A and B are different across some societal class like religion, race, ethnicity, gender...) The author is talking about the fourth case. While yes there is over/under representation in the input data, the algorithm chosen to build the model encodes its own form of bias. A simple example everyone should be familiar with, using Mean Squared Error vs. Mean Absolute Error (or MBE) produce models that handle rarely seen features or data points very differently. This choice of loss function introduces algorithmic bias into a model. That's essentially what underlies the concept. Yes there is data bias, but what is being argued is that the solution isn't "find perfectly unbiased data" (because in many cases that's impossible), it's understand how the choice of algorithm will produce models that respond to and represent differently those data biases.