25 ms·

> The fact that our AIs are becoming biased is a bug. It should be fixed. In many cases, it's the data that is biased. In that case it's impossible to differen

> The fact that our AIs are becoming biased is a bug. It should be fixed. In many cases, it's the data that is biased. In that case it's impossible to differentiate between bias that the AI should learn and bias that the AI shouldn't learn. Let's assume we have a database of incidents where a person was found to have cannabis. This database has the following data items: a timestamp, the persons name, the persons ethnicity and the amount of cannabis that was found. Now assume further that black and white people have the same base rate of cannabis use (which according to the studies I found seems to be the case). The last thing we have to assume in this case is that this database was created by racist policemen who arrest more black people for cannabis consumption. An AI trained using this data would assume a higher base rate of cannabis consumption by black people. It's impossible for this AI to differentiate between correlations it should learn (for example that people who used cannabis multiple times and were found to have much of it are worth looking at) and (untrue) correlations that it shouldn't learn (that black people have a higher base rate). The correct solution here is to use a dataset that is not biased, but it's hard to tell whether a dataset is biased.

tlb 7y ago
The data can't tell you how the groups differ, since you can't tell the difference between criminal behavior and policing behavior. So you have to add some priors. The most progressive approach is to assume that there are no intrinsic differences between protected groups, and any difference in the data is the legacy of past discrimination. You can add such a prior by adding a term to the loss function that penalizes any difference between the way the groups are treated. The math isn't hard, only the political decision of what is protected and what isn't.
- salawat 7y ago
  So... If you don't like the data, change the character of the data? I get what you're getting at. If the data collection is biased, massage the actor on the data to compensate. Got it. ...but if you're uncomfortable with the consequences of unleashing a Machine Learned system trained on your data as part of your decision making process from the get go, why in the heck would you want to anyway? There is no benefit to doing so when you realize that the system only works as well and unbiased as the sum of the biases of the data architects, data populators, and data interpreters combined, and that extra loss factor you added is, in fact, another bias. An unacceptable on to the minds of many no matter how disagreeable one may personally find it. Better to just leave it in human hands to deal with on a case by case basis by that point. At least by doing that we stay ultimately in control of the overall decision making process. People being in the loop is not a bug. Trying to solve social/political problems with poorly thought out applications of inexplicable technological systems is.
- MaxBarraclough 7y ago
  > assume that there are no intrinsic differences between protected groups, and any difference in the data is the legacy of past discrimination. I don't see why we should assume that this would reflect reality. If a law has racist roots, i.e. if it is written to target a particular ethnicity, then we should expect that certain ethnicities really do break that law more than others.
- gnaritas 7y ago
  > The most progressive approach is to assume that there are no intrinsic differences between protected groups, and any difference in the data is the legacy of past discrimination. That's an irrational assumption and an ideological one at that. Your analysis would thus be ideologically biased.
MaxBarraclough 7y ago
> An AI trained using this data would assume a higher base rate of cannabis consumption by black people This is a very good point, but it's not enough to ensure your input data aren't reflecting existing biases in society. (I believe it's necessary, but not sufficient.) In my other comment I gave the example of maternity leave. A woman who becomes pregnant won't be as productive that year. It's no-one's 'fault' that this is the case, and it doesn't reflect anyone's bias. It's still important to ensure, when making hiring decisions, that applicants are not eliminated on the grounds of 'high pregnancy risk'.

tlb 7y ago

The data can't tell you how the groups differ, since you can't tell the difference between criminal behavior and policing behavior. So you have to add some priors. The most progressive approach is to assume that there are no intrinsic differences between protected groups, and any difference in the data is the legacy of past discrimination. You can add such a prior by adding a term to the loss function that penalizes any difference between the way the groups are treated. The math isn't hard, only the political decision of what is protected and what isn't.

salawat 7y ago
So... If you don't like the data, change the character of the data? I get what you're getting at. If the data collection is biased, massage the actor on the data to compensate. Got it. ...but if you're uncomfortable with the consequences of unleashing a Machine Learned system trained on your data as part of your decision making process from the get go, why in the heck would you want to anyway? There is no benefit to doing so when you realize that the system only works as well and unbiased as the sum of the biases of the data architects, data populators, and data interpreters combined, and that extra loss factor you added is, in fact, another bias. An unacceptable on to the minds of many no matter how disagreeable one may personally find it. Better to just leave it in human hands to deal with on a case by case basis by that point. At least by doing that we stay ultimately in control of the overall decision making process. People being in the loop is not a bug. Trying to solve social/political problems with poorly thought out applications of inexplicable technological systems is.
MaxBarraclough 7y ago
> assume that there are no intrinsic differences between protected groups, and any difference in the data is the legacy of past discrimination. I don't see why we should assume that this would reflect reality. If a law has racist roots, i.e. if it is written to target a particular ethnicity, then we should expect that certain ethnicities really do break that law more than others.
gnaritas 7y ago
> The most progressive approach is to assume that there are no intrinsic differences between protected groups, and any difference in the data is the legacy of past discrimination. That's an irrational assumption and an ideological one at that. Your analysis would thus be ideologically biased.

MaxBarraclough 7y ago

> An AI trained using this data would assume a higher base rate of cannabis consumption by black people This is a very good point, but it's not enough to ensure your input data aren't reflecting existing biases in society. (I believe it's necessary, but not sufficient.) In my other comment I gave the example of maternity leave. A woman who becomes pregnant won't be as productive that year. It's no-one's 'fault' that this is the case, and it doesn't reflect anyone's bias. It's still important to ensure, when making hiring decisions, that applicants are not eliminated on the grounds of 'high pregnancy risk'.