5 ms·

This is a game of whack-a-mole. Given enough data, the ML system will just find other characteristics or groups of characteristics that act as proxies. It might

This is a game of whack-a-mole. Given enough data, the ML system will just find other characteristics or groups of characteristics that act as proxies. It might unfairly penalize candidates in ways that are impossible to detect by human evaluators. The promise of these systems is that if you give them a pile of raw data, they will detect subtle patterns that aid in assessing individuals (be they job applicants, ex-cons, whatever.) However, some of the patterns that they detect are our biases against classes of people. If the solution is to sanitize the data to such a degree that the ML system can no longer infer that someone is a woman or that someone is over 40, etc., then the training data is probably also useless for detecting the non-obvious patterns that we want the system to discover.

mieseratte 7y ago
That it is a “game of whack-a-mole” isn’t clear to me, and you are asserting so without any backing, which is why I asked if anyone has real-world examples. Citing an example where they don’t provide gender but then provide “went to an all-women college” is not an example of proving non-protected information which results in protected-class discrimination, it is an example of telling an ML system protected information in a roundabout way.
- michaelt 7y ago
  Age is a protected class, and it's extremely difficult to sanitise age from a resume. Let's say the candidate got a BS in EECS in 1987, they've had 6 jobs since graduating, and their first job was cost-optimising floppy disk drives. Do you feed the ML system this data, which is clearly correlated with age? Do you keep the number of jobs but delete the duration, responding to 6-jobs-in-3-years the same as 6-jobs-in-30-years? How do you sanitise the age-correlated data out of job descriptions naming old tech or defunct employers? Do you delete all but the last 5 years of their employment history, assigning no value to experience beyond that?

mieseratte 7y ago

That it is a “game of whack-a-mole” isn’t clear to me, and you are asserting so without any backing, which is why I asked if anyone has real-world examples. Citing an example where they don’t provide gender but then provide “went to an all-women college” is not an example of proving non-protected information which results in protected-class discrimination, it is an example of telling an ML system protected information in a roundabout way.

michaelt 7y ago
Age is a protected class, and it's extremely difficult to sanitise age from a resume. Let's say the candidate got a BS in EECS in 1987, they've had 6 jobs since graduating, and their first job was cost-optimising floppy disk drives. Do you feed the ML system this data, which is clearly correlated with age? Do you keep the number of jobs but delete the duration, responding to 6-jobs-in-3-years the same as 6-jobs-in-30-years? How do you sanitise the age-correlated data out of job descriptions naming old tech or defunct employers? Do you delete all but the last 5 years of their employment history, assigning no value to experience beyond that?