2 ms·

The criticisms of the Implicit Association Test don't apply to testing AI systems for racial bias. You ask a word2vec-based system about predominantly-black na

by rspeer 9y ago

The criticisms of the Implicit Association Test don't apply to testing AI systems for racial bias. You ask a word2vec-based system about predominantly-black names and it associates them with negative emotions, gives them negative sentiment scores, hides their comments with PerspectiveAPI, etc. Predominantly-white names are more positive. This is a problem. The effect is so blatant and so repeatable (it's not about something fuzzy like response times, it's the actual predictions that come out of the system) that you don't have to quibble about the predictive validity of giving the IAT to humans. You need to actively de-bias your data, because data encodes and propagates the biases of the past. However, I do not follow this article's argument about why Fair Use will provide automatically de-biased data, especially when it leads with the fact that Google News doesn't.

AstralStorm 9y ago
The problem is then debiasing data. That in itself requires strong AI to work reliably. Dumb methods like say PCA/ICA do not work on multidimensional datasets. (They become unworkably slow and often trade biases between dimensions.) If you debias manually or declare subspace manually you're actually cheating and potentially introduce your own biases.
- rspeer 9y ago
  Enough of this false equivalence. You can call what I do "introducing my own biases". I am "biasing" NLP algorithms toward treating people equally whenever possible, toward not discriminating based on their name or how they describe themselves. The default biases are really bad, and whatever my biases are, they're not as bad. I don't even know what you say I'm "cheating" at. That is a very weird accusation. You always make a choice of what algorithms and heuristics to deploy. Deploying a system like pre-trained word2vec or GloVe that believes everything it reads on the Web is a harmful choice. Not even waiting forever for strong AI will fix it. You wouldn't even let a child believe everything that they read, and a child has real intelligence. When you deploy harmful AI, you can't blame its effects on the code or the data, because the code and data have no moral agency. You wrote the code and it's your responsibility.

by rspeer 9y ago

AstralStorm 9y ago

The problem is then debiasing data. That in itself requires strong AI to work reliably. Dumb methods like say PCA/ICA do not work on multidimensional datasets. (They become unworkably slow and often trade biases between dimensions.) If you debias manually or declare subspace manually you're actually cheating and potentially introduce your own biases.

rspeer 9y ago
Enough of this false equivalence. You can call what I do "introducing my own biases". I am "biasing" NLP algorithms toward treating people equally whenever possible, toward not discriminating based on their name or how they describe themselves. The default biases are really bad, and whatever my biases are, they're not as bad. I don't even know what you say I'm "cheating" at. That is a very weird accusation. You always make a choice of what algorithms and heuristics to deploy. Deploying a system like pre-trained word2vec or GloVe that believes everything it reads on the Web is a harmful choice. Not even waiting forever for strong AI will fix it. You wouldn't even let a child believe everything that they read, and a child has real intelligence. When you deploy harmful AI, you can't blame its effects on the code or the data, because the code and data have no moral agency. You wrote the code and it's your responsibility.