4 ms·
> Risk scores are computed for every traffic stop, wouldn't you want to take that process away from the office and give it to a computer? What I want is this p
by vkou 7y ago
> Risk scores are computed for every traffic stop, wouldn't you want to take that process away from the office and give it to a computer?
What I want is this process to be cross-examinable. You can't cross-examine an algorithm in any meaningful way at a jury trial. You can't cross-examine a neural network at all.
You know all the bullshit about how police dogs are used to conduct illegal searches, and how you can't cross-examine a dog? It'll be like that, but much, much worse.
- aey 7y agoNo, you can challenge the validity of the risk assessment. Bayes is used in jury trials all the time. https://en.m.wikipedia.org/wiki/Bayesian_inference#Applications https://en.m.wikipedia.org/wiki/Bayesian_inference#Applicati... What we have today is institutional racism based on people’s gut reactions. An NN can be tested and shown that given all other factors it risks race, or any other protected class higher than it should.
- JohnFen 7y ago> What we have today is institutional racism based on people’s gut reactions. Institutional racism is not based on people's gut reactions. Institutional racism is based on flawed algorithms. Not computer algorithms in this case, but bureaucratic ones. I don't see how a flawed computer algorithm would be any easier to correct than a flawed bureaucratic algorithm.
- aey 7y agoBecause we can black box test the “risk assessment” that whoever sells to police departments. We can’t black box test bureaucracies.
- JohnFen 7y agoTesting the results of the algorithms doesn't address the problem of institutional racism, though. Any bureaucracy will simply use the results that the algorithm supplies in a way that maintains the behavior that the bureaucracy is already programmed to maintain. To address things like institutional racism, you have to change the institution. That's a human problem for which there is no technological solution (although technology can certainly be one of the tools used to create a larger change).
- vkou 7y agoWe can black box test bureaucracies just as poorly as we can software. For both, all we can do is measure inputs, and outcomes. Since we've not made much traction on blackbox testing bureaucracies, I'm going to go ahead and say that blackbox testing algorithms will be even worse. (For some dumb reason, people are inclined to trust algorithms.)