3 ms·

There seems to be big confusion on what is bias and fairness and this course doesn’t seem to help. Fairness is ensuring that peaple are not deliberately punishe

There seems to be big confusion on what is bias and fairness and this course doesn’t seem to help. Fairness is ensuring that peaple are not deliberately punished for things that they have no control over. Typically this translates to making sure that decisions made by governments and employers do not depend on race, color and gender. US constitution adds religion in to the mix and some EU countries may also want to include sexual orientation, disability, age, political orientation but the minimal thing required by US laws in most states are those 4 attributes. Every ML model can have different bias on different features. That’s what makes model a model. The fairness means you should make sure that your model is not biased towards above specific attributes and explicitly test for it. This is not some liberal propaganda by tech but required by law in many instances even if ML was not involved. This only applies to models used for certain classes of decision making, for example, loan eligibility or hiring or FaceID login. It doesn’t apply to things like breast cancer predictions or shopping recommendation system.

btown 8y ago
It's more than this, though. For instance, is the presence of the word "captured" (e.g. in a dataset of performance reviews) correlated with gender? This was certainly the case for Amazon: https://www.theguardian.com/technology/2018/oct/10/amazon-hiring-ai-gender-bias-recruiting-engine https://www.theguardian.com/technology/2018/oct/10/amazon-hi... . If so, even if you don't include gender as a feature itself, your outputs may end up being biased (in the technical sense) by gender. If you want to correct for this, then you might want to include gender in your analysis in a structured way in order to determine: are there better features that apply relatively equally to indicate a superstar of either gender? Personally, I don't know what the battle-tested approaches in the ML literature are to do this type of thing, so this course would be helpful to people like me! At the end of the day, it all boils down to this: you can't be "blind" to race, color, religion, and gender if information is leaked to your evaluator (human or otherwise) via a side channel. You might get away with this legally (especially if there's not intent to discriminate), but if you really want to do it right, you have to engineer systems that minimize those side channels.
- Pengy7 8y ago
  > If so, even if you don't include gender as a feature itself, your outputs may end up being biased (in the technical sense) by gender. Part of the problem is people using the same word to mean multiple things. For instance, "bias" has a precise mathematical definition in the context of statistics: https://en.wikipedia.org/wiki/Bias_(statistics) https://en.wikipedia.org/wiki/Bias_(statistics) . And this sentence makes no sense with that definition. In fact, with linear models it is mathematically impossible to make a "worse" model (in terms of mean squared error) by including more variables (like gender, age, race, etc...). > you can't be "blind" to race, color, religion, and gender Also I am not sure that this train of thought actually leads to where we want to go. A perfect model isn't necessarily blind to these features, a perfect model treats everyone as an individual.
  - yorwba 8y ago
    > In fact, with linear models it is mathematically impossible to make a "worse" model (in terms of mean squared error) by including more variables (like gender, age, race, etc...). That's only true if you mean the mean squared error on the training data, which is not usually a good indicator of model quality. Instead you should use the mean squared error on test data, which gets worse if you add non-predictive variables to the input. If there are non-predictive variables, the linear model with the lowest expected square error should assign exactly zero weight to them, equivalent to the situation where those variables don't exist. But training on a finite sample, that "exactly zero" outcome is extremely unlikely (as in, the probability is 0) if the non-predictive variables vary at all. That variance allows identifying individual data points, even though the relationship is completely random and doesn't help generalize to unseen data. In other words, the model overfits to noise.

btown 8y ago

It's more than this, though. For instance, is the presence of the word "captured" (e.g. in a dataset of performance reviews) correlated with gender? This was certainly the case for Amazon: https://www.theguardian.com/technology/2018/oct/10/amazon-hiring-ai-gender-bias-recruiting-engine https://www.theguardian.com/technology/2018/oct/10/amazon-hi... . If so, even if you don't include gender as a feature itself, your outputs may end up being biased (in the technical sense) by gender. If you want to correct for this, then you might want to include gender in your analysis in a structured way in order to determine: are there better features that apply relatively equally to indicate a superstar of either gender? Personally, I don't know what the battle-tested approaches in the ML literature are to do this type of thing, so this course would be helpful to people like me! At the end of the day, it all boils down to this: you can't be "blind" to race, color, religion, and gender if information is leaked to your evaluator (human or otherwise) via a side channel. You might get away with this legally (especially if there's not intent to discriminate), but if you really want to do it right, you have to engineer systems that minimize those side channels.

Pengy7 8y ago
> If so, even if you don't include gender as a feature itself, your outputs may end up being biased (in the technical sense) by gender. Part of the problem is people using the same word to mean multiple things. For instance, "bias" has a precise mathematical definition in the context of statistics: https://en.wikipedia.org/wiki/Bias_(statistics) https://en.wikipedia.org/wiki/Bias_(statistics) . And this sentence makes no sense with that definition. In fact, with linear models it is mathematically impossible to make a "worse" model (in terms of mean squared error) by including more variables (like gender, age, race, etc...). > you can't be "blind" to race, color, religion, and gender Also I am not sure that this train of thought actually leads to where we want to go. A perfect model isn't necessarily blind to these features, a perfect model treats everyone as an individual.
- yorwba 8y ago
  > In fact, with linear models it is mathematically impossible to make a "worse" model (in terms of mean squared error) by including more variables (like gender, age, race, etc...). That's only true if you mean the mean squared error on the training data, which is not usually a good indicator of model quality. Instead you should use the mean squared error on test data, which gets worse if you add non-predictive variables to the input. If there are non-predictive variables, the linear model with the lowest expected square error should assign exactly zero weight to them, equivalent to the situation where those variables don't exist. But training on a finite sample, that "exactly zero" outcome is extremely unlikely (as in, the probability is 0) if the non-predictive variables vary at all. That variance allows identifying individual data points, even though the relationship is completely random and doesn't help generalize to unseen data. In other words, the model overfits to noise.