8 ms·

Ironic choice of examples, because lacking these tools, you've just identified the tool the police tend to fall back on. We can build algorithms to avoid racia

Ironic choice of examples, because lacking these tools, you've just identified the tool the police tend to fall back on. We can build algorithms to avoid racial bias, with awareness and effort. How confident are you we can train cops to do the same?

AnthonyMouse 8y ago
> We can build algorithms to avoid racial bias, with awareness and effort. How confident are you we can train cops to do the same? How confident are you that we can build algorithms to do that? You can't just flatten the probability distribution based on race when many of the legitimate factors are correlated with it, any more than you could do so for gender or age or nationality. Unless you want an absurd false positive rate against octogenarian women from Japan. But given that some racial (or gender or age) disparity is expected, how do you know if the amount of disparity in the algorithm's results is legitimate? It could be too high, or too low, and knowing which one would imply possession of an algorithm that gives the "true" amount, which was the original problem to begin with. The answer is to require high standards of proof and conclusive evidence, so that it's impossible to choose a random innocent black man off the street and convict him of anything just because you're a racist or some algorithm decided he fit an aggregate profile. But that's the opposite of the dragnet approach. You need evidence on a specific person, not statistical data from thousands that doesn't give you a better than 5% probability that it was any given one of them. And suppose the actual perpetrator wasn't carrying a phone at the time of the crime, so now you've got a list of "possible suspects" 100% of which are innocent.
- fixermark 8y ago
  I believe the article clarifies that this system (much like its sibling, DNA database mining) is used to narrow the search space so that traditional (and comparatively more expensive) sleuthing can find the evidence needed for a conviction. The dragnet is, in a sense, more ruling people out than ruling people in.
  - AnthonyMouse 8y ago
    > I believe the article clarifies that this system (much like its sibling, DNA database mining) is used to narrow the search space so that traditional (and comparatively more expensive) sleuthing can find the evidence needed for a conviction. Which is exactly the problem. The system spits out a list of 25 names, 20 of them have an alibi and only one of the remaining five had the capacity to commit the crime. So you've got your man, all you have to do is dig up some evidence. Unless the true perpetrator wasn't on the original list to begin with, or fabricated an alibi, in which case you're attempting to convict an innocent person. Which huge databases are ideal for doing because it's very easy to use selection bias to raise the apparent probability of random events. It's like the reverse pyramid scheme. You choose a stock and send brochures to a million people telling half it will go up and half it will go down. Then it goes up or down and next week you send brochures to the half million people you sent the accurate prediction to last week, telling half of them that a different stock will go up and the other half that it will go down. By the end of ten weeks you have about a thousand people who think you can predict the market when all you've really done is to start with a large pool. "99.9% accurate" in a city of two million means that 99.95% of the matches are false positives. You can't start with a list of names and then look for evidence it was them. You have to start with the evidence and see who it points to.
alistairSH 8y ago
We can build algorithms to avoid racial bias Can we? Those algorithms are built/trained by humans with biases - how can you be sure we can build/train AI to avoid those same biases?
- fixermark 8y ago
  It's an excellent open question. I'd argue that relative to training it out of humans, we have a glimmer of hope of building algorithms that avoid racial bias because you can always bust an algorithm open and "examine its entrails," and the same can't be done with human thinking. ... but it's not categorically, definitely better, just more likely to be so.

AnthonyMouse 8y ago

> We can build algorithms to avoid racial bias, with awareness and effort. How confident are you we can train cops to do the same? How confident are you that we can build algorithms to do that? You can't just flatten the probability distribution based on race when many of the legitimate factors are correlated with it, any more than you could do so for gender or age or nationality. Unless you want an absurd false positive rate against octogenarian women from Japan. But given that some racial (or gender or age) disparity is expected, how do you know if the amount of disparity in the algorithm's results is legitimate? It could be too high, or too low, and knowing which one would imply possession of an algorithm that gives the "true" amount, which was the original problem to begin with. The answer is to require high standards of proof and conclusive evidence, so that it's impossible to choose a random innocent black man off the street and convict him of anything just because you're a racist or some algorithm decided he fit an aggregate profile. But that's the opposite of the dragnet approach. You need evidence on a specific person, not statistical data from thousands that doesn't give you a better than 5% probability that it was any given one of them. And suppose the actual perpetrator wasn't carrying a phone at the time of the crime, so now you've got a list of "possible suspects" 100% of which are innocent.

fixermark 8y ago
I believe the article clarifies that this system (much like its sibling, DNA database mining) is used to narrow the search space so that traditional (and comparatively more expensive) sleuthing can find the evidence needed for a conviction. The dragnet is, in a sense, more ruling people out than ruling people in.
- AnthonyMouse 8y ago
  > I believe the article clarifies that this system (much like its sibling, DNA database mining) is used to narrow the search space so that traditional (and comparatively more expensive) sleuthing can find the evidence needed for a conviction. Which is exactly the problem. The system spits out a list of 25 names, 20 of them have an alibi and only one of the remaining five had the capacity to commit the crime. So you've got your man, all you have to do is dig up some evidence. Unless the true perpetrator wasn't on the original list to begin with, or fabricated an alibi, in which case you're attempting to convict an innocent person. Which huge databases are ideal for doing because it's very easy to use selection bias to raise the apparent probability of random events. It's like the reverse pyramid scheme. You choose a stock and send brochures to a million people telling half it will go up and half it will go down. Then it goes up or down and next week you send brochures to the half million people you sent the accurate prediction to last week, telling half of them that a different stock will go up and the other half that it will go down. By the end of ten weeks you have about a thousand people who think you can predict the market when all you've really done is to start with a large pool. "99.9% accurate" in a city of two million means that 99.95% of the matches are false positives. You can't start with a list of names and then look for evidence it was them. You have to start with the evidence and see who it points to.

alistairSH 8y ago

We can build algorithms to avoid racial bias Can we? Those algorithms are built/trained by humans with biases - how can you be sure we can build/train AI to avoid those same biases?

fixermark 8y ago
It's an excellent open question. I'd argue that relative to training it out of humans, we have a glimmer of hope of building algorithms that avoid racial bias because you can always bust an algorithm open and "examine its entrails," and the same can't be done with human thinking. ... but it's not categorically, definitely better, just more likely to be so.