2 ms·

It has always been possible to get stuck in bureaucratic hell by some mistake at some level, and often difficult to extricate oneself. Before, each of those ca

It has always been possible to get stuck in bureaucratic hell by some mistake at some level, and often difficult to extricate oneself. Before, each of those cases probably needed a human directly involved. Humans make mistakes, but they tend to make them one at a time, and we have a cultural understanding that humans can make mistakes, and systems to recover from them. Now that we have algorithms involved, we can screw people over wholesale, and we haven't yet developed the cultural knowledge that algorithms can make mistakes as well (astonishing, considering how often computers do not do what we expect them to do).

wnissen 12y ago
Yes, the human error rate is probably higher than a computer, but the correction rate is higher, too. A computer will never, in a million years, realize its mistake, or be swayed by an appeal. It's harder to fix some classes of computer errors, too. The DMV system's facial recognition algorithm will have to add a list of documented false positives, which is an additional feature with a cost. Errors will only be fixed if they are severe and affect multiple people. A human can just fix the mistake in the paper file as a one-off and move on. No generalized solution necessary.
jerf 12y ago
"and we haven't yet developed the cultural knowledge that algorithms can make mistakes as well (astonishing, considering how often computers do not do what we expect them to do)." I feel like we've made some progress in the past ~20 years I've been able to witness that effect. Once upon a time, the output of a computer might as well have been divine writ. Nowadays, the person on the street is a bit more realistic. I think the problem with a bureaucracy now isn't that they consider the computer holy writ, it's that they hear "the computer is wrong!" more often than it actually is, so they grow their own rather cynical reaction to that claim... and, alas, that's a rational response, too. Especially bad when their system is telling them that you're a bad person in one way or another... of course you're not really six months behind in payment, you deadbeat, I totally believe the computer screwed up. The check's in the mail, too, right?

wnissen 12y ago

Yes, the human error rate is probably higher than a computer, but the correction rate is higher, too. A computer will never, in a million years, realize its mistake, or be swayed by an appeal. It's harder to fix some classes of computer errors, too. The DMV system's facial recognition algorithm will have to add a list of documented false positives, which is an additional feature with a cost. Errors will only be fixed if they are severe and affect multiple people. A human can just fix the mistake in the paper file as a one-off and move on. No generalized solution necessary.

jerf 12y ago

"and we haven't yet developed the cultural knowledge that algorithms can make mistakes as well (astonishing, considering how often computers do not do what we expect them to do)." I feel like we've made some progress in the past ~20 years I've been able to witness that effect. Once upon a time, the output of a computer might as well have been divine writ. Nowadays, the person on the street is a bit more realistic. I think the problem with a bureaucracy now isn't that they consider the computer holy writ, it's that they hear "the computer is wrong!" more often than it actually is, so they grow their own rather cynical reaction to that claim... and, alas, that's a rational response, too. Especially bad when their system is telling them that you're a bad person in one way or another... of course you're not really six months behind in payment, you deadbeat, I totally believe the computer screwed up. The check's in the mail, too, right?