3 ms·
Apparently, the creator of that algorithm literally and consciously added bias against females and non-caucasian to his program, which made it pretty much an op
by microtherion 8y ago
Apparently, the creator of that algorithm literally and consciously added bias against females and non-caucasian to his program, which made it pretty much an open and shut case.
The modern approach would be to take some NLP machine learning kit, which would do a wonderful job learning and reproducing the human assessors' biases, without any overt evidence (because the reasoning is always opaque), without ever being explicitly instructed to do so, and possibly even without the author of the program being aware of it.
- sologoub 8y agoThis is where the creator has a very specific duty of being careful about what features are fed into the learning algorithms - including names, gender and other protected category info is a huge no-no. The NLP part is complex because language and dialect differences can reveal a lot about the individual and her/his background. If you have to analyze non-standardized inputs, some personalized data is going to leak. The question is how much does it affect the outcome.
- bilbo0s 8y ago>(because the reasoning is always opaque)... So it would still be bad in, say, a modern courtroom. It's just not tenable to go in there with a defense of "well we don't really know why your honor."