5 ms·

Hypothetical question. Suppose no-one ever noticed the lead in the Flint water supply. People put test scores into an AI and the AI emotionlessly concludes that

by xamuel 7y ago

Hypothetical question. Suppose no-one ever noticed the lead in the Flint water supply. People put test scores into an AI and the AI emotionlessly concludes that people from Flint perform worse than average. Should you hack the AI to force it to stop saying something like that, because that's obviously racist? Or should you begin to wonder why people from Flint perform worse, and what is causing that (the undiagnosed lead in the water), and ultimately how to correct the problem (fix the water)?

knzhou 7y ago
Incidentally, some actual number crunching [0] shows that the lead levels in Flint were never that high, being less than the average level in many states, and like everywhere else in America had been declining significantly over decades. What's more, in the era of leaded gasoline, almost all children had blood lead levels about 10x what was seen in Flint. 0: https://www.nytimes.com/2018/07/22/opinion/flint-lead-poisoning-water.html https://www.nytimes.com/2018/07/22/opinion/flint-lead-poison...
gbrown 7y ago
As always, it depends entirely on what you're doing with the (predictions/estimates/inferences). This is an entirely false dilemma.
- xamuel 7y ago
  When would you want to hack the AI? Not a rhetorical question. Reading through the comment thread I'm slowly shifting more towards being on the fence, as opposed to having a "don't hack the AI" position. But rather than hacking the AI, why not just get rid of the AI? What is the point of the AI in the first place if you're going to hack it to get the results you want anyway?
  - gbrown 7y ago
    > When would you want to hack the AI? This is not a meaningful phrase - it adds literally nothing to the conversation but confusion. To bend over backwards to give you a reasonable answer: when you're interested in conditional effects. Let's say you're interested in the risk of cancer associated with alcohol consumption. People who drink some are often found to have lower cancer rates than people who don't drink at all. Reasonable models adjust for wealth/income - estimating the risk of cancer association with alcohol CONDITIONING ON wealth/income changes the picture; the positive association is clear. Adjusting for confounding is "removing bias", changing the effect estimates. In a predictive context, using ensembles, NNs etc, the problems don't go away, they're just sometimes harder to detect (and they're dressed up in sexy marketing-speak like "AI"). Repeat after me three times: "AI is not a magic truth telling oracle" "AI is not a magic truth telling oracle" "AI is not a magic truth telling oracle"
    - xamuel 7y ago
      I meant in my example about undiagnosed lead in Flint. Once you notice that the AI detects people in Flint under-perform, why would you want to modify the AI to avoid the AI detecting that? Your cancer example doesn't answer the question, which is a pity because I mean it when I say it's not a rhetorical question. I honestly want to understand your point of view better. In the cancer example, we wouldn't go in and filter the training set to force the AI to think that rich people have higher cancer than they actually do, etc. But that's precisely what it seems like people want to happen with AI: they don't want us to condition on wealth/income, rather, they flat-out want us to filter the training set to force the AI to think group A and group B are equal when that's not what the data says.

by xamuel 7y ago

knzhou 7y ago

Incidentally, some actual number crunching [0] shows that the lead levels in Flint were never that high, being less than the average level in many states, and like everywhere else in America had been declining significantly over decades. What's more, in the era of leaded gasoline, almost all children had blood lead levels about 10x what was seen in Flint. 0: https://www.nytimes.com/2018/07/22/opinion/flint-lead-poisoning-water.html https://www.nytimes.com/2018/07/22/opinion/flint-lead-poison...

gbrown 7y ago

As always, it depends entirely on what you're doing with the (predictions/estimates/inferences). This is an entirely false dilemma.

xamuel 7y ago
When would you want to hack the AI? Not a rhetorical question. Reading through the comment thread I'm slowly shifting more towards being on the fence, as opposed to having a "don't hack the AI" position. But rather than hacking the AI, why not just get rid of the AI? What is the point of the AI in the first place if you're going to hack it to get the results you want anyway?
- gbrown 7y ago
  > When would you want to hack the AI? This is not a meaningful phrase - it adds literally nothing to the conversation but confusion. To bend over backwards to give you a reasonable answer: when you're interested in conditional effects. Let's say you're interested in the risk of cancer associated with alcohol consumption. People who drink some are often found to have lower cancer rates than people who don't drink at all. Reasonable models adjust for wealth/income - estimating the risk of cancer association with alcohol CONDITIONING ON wealth/income changes the picture; the positive association is clear. Adjusting for confounding is "removing bias", changing the effect estimates. In a predictive context, using ensembles, NNs etc, the problems don't go away, they're just sometimes harder to detect (and they're dressed up in sexy marketing-speak like "AI"). Repeat after me three times: "AI is not a magic truth telling oracle" "AI is not a magic truth telling oracle" "AI is not a magic truth telling oracle"
  - xamuel 7y ago
    I meant in my example about undiagnosed lead in Flint. Once you notice that the AI detects people in Flint under-perform, why would you want to modify the AI to avoid the AI detecting that? Your cancer example doesn't answer the question, which is a pity because I mean it when I say it's not a rhetorical question. I honestly want to understand your point of view better. In the cancer example, we wouldn't go in and filter the training set to force the AI to think that rich people have higher cancer than they actually do, etc. But that's precisely what it seems like people want to happen with AI: they don't want us to condition on wealth/income, rather, they flat-out want us to filter the training set to force the AI to think group A and group B are equal when that's not what the data says.