5 ms·

Aequitas – An open source bias audit toolkit for machine learning

46 points by lainon 8y ago

to_bpr 8y ago
If the goal is equity of outcome above-all-else, ignoring for any differences derived from the data, then why are we bothered investing so much time, money and effort into this area?
- microcolonel 8y ago
  Because they want to control the contradiction. There is, however, some merit to this concept. There is a problem if you are training hidden variables which are dependent on a "race" or similar field in a record. Outside of that, it is arguably unethical to attempt to remove any bias (bias, of one sort or another, being the whole point of deep learning) which does not produce hidden variables based on a protected characteristic. E.g, in medical diagnostics, with low quality inference inputs, having a "race" field could save lives; in your resumé system, it could unfairly disadvantage (or advantage) somebody based on hidden variables you did not measure which just happen to correlate with race, also leading possibly to worse results.
- Macuyiko 8y ago
  See https://www.propublica.org/article/machine-bias-risk-assessments-in-criminal-sentencing https://www.propublica.org/article/machine-bias-risk-assessm... for a well-known example in this space. This article: https://www.nature.com/articles/d41586-018-05469-3 https://www.nature.com/articles/d41586-018-05469-3 also highlights the issues pretty well IMO. It's not just about letting the data speak. The data is gathered by someone, containing historical bias we might nowadays disagree with, models are chosen by people, using parameters set by people, using evaluations set by people. It's about making sure models are both predictive, usable, and fair.
  - Alex888 8y ago
    Predictive, usable and fair. Pick any two!
Macuyiko 8y ago
FairML https://github.com/adebayoj/fairml https://github.com/adebayoj/fairml and algofairness https://github.com/algofairness/BlackBoxAuditing https://github.com/algofairness/BlackBoxAuditing are some similar, earlier projects in the same space.
opwieurposiu 8y ago
I find the premise that different groups should expect the same percentage of interventions highly suspect. Imagine we have a program that distributes seeing eye dogs. This toolkit would discover that sighted persons have a 0% chance of getting a dog, while blind persons have a 50% chance. Oh the injustice!
- milk_wolf_honey 8y ago
  It looks like there are multiple audit methods available, for different contexts, of which this is only one.

46 points by lainon 8y ago

If the goal is equity of outcome above-all-else, ignoring for any differences derived from the data, then why are we bothered investing so much time, money and effort into this area?

microcolonel 8y ago
Because they want to control the contradiction. There is, however, some merit to this concept. There is a problem if you are training hidden variables which are dependent on a "race" or similar field in a record. Outside of that, it is arguably unethical to attempt to remove any bias (bias, of one sort or another, being the whole point of deep learning) which does not produce hidden variables based on a protected characteristic. E.g, in medical diagnostics, with low quality inference inputs, having a "race" field could save lives; in your resumé system, it could unfairly disadvantage (or advantage) somebody based on hidden variables you did not measure which just happen to correlate with race, also leading possibly to worse results.
Macuyiko 8y ago
See https://www.propublica.org/article/machine-bias-risk-assessments-in-criminal-sentencing https://www.propublica.org/article/machine-bias-risk-assessm... for a well-known example in this space. This article: https://www.nature.com/articles/d41586-018-05469-3 https://www.nature.com/articles/d41586-018-05469-3 also highlights the issues pretty well IMO. It's not just about letting the data speak. The data is gathered by someone, containing historical bias we might nowadays disagree with, models are chosen by people, using parameters set by people, using evaluations set by people. It's about making sure models are both predictive, usable, and fair.
- Alex888 8y ago
  Predictive, usable and fair. Pick any two!

Macuyiko 8y ago

FairML https://github.com/adebayoj/fairml https://github.com/adebayoj/fairml and algofairness https://github.com/algofairness/BlackBoxAuditing https://github.com/algofairness/BlackBoxAuditing are some similar, earlier projects in the same space.

opwieurposiu 8y ago

I find the premise that different groups should expect the same percentage of interventions highly suspect. Imagine we have a program that distributes seeing eye dogs. This toolkit would discover that sighted persons have a 0% chance of getting a dog, while blind persons have a 50% chance. Oh the injustice!

milk_wolf_honey 8y ago
It looks like there are multiple audit methods available, for different contexts, of which this is only one.