3 ms·

This is simply bad data governance. Algorithms should not be able to make destructive decisions like dropping people from insurance or canceling a driver's lice

by exelius 12y ago

This is simply bad data governance. Algorithms should not be able to make destructive decisions like dropping people from insurance or canceling a driver's license on their own. A better system would use the algorithm to identify potential cases, then having a human perform a more stringent verification once you've narrowed down the set to a manageable number. Yes, this is slightly more expensive than using an algorithm alone. But it could actually end up being cheaper in the long run if law suits are involved. Sloppy data governance causes all kinds of problems that are hard to quantify numerically, and that's the root cause here - not bad algorithms. At the end of the day, every algorithm is going to produce imperfect data: they're based on simplified logical models of real world systems. Recognizing that algorithms are imperfect is a key part of knowing when and how to use them.

cag_ii 12y ago
Well said. Errors will happen. How they're monitored/dealt with is not an algorithm problem, and it's importance can't be underestimated. In this case "the algorithm" is the fly in the printer from the '85 movie "Brazil". Except these are real stories, it's absurd.
- dragonwriter 12y ago
  > How they're monitored/dealt with is not an algorithm problem Sure it is; what inputs go into producing the ultimate decision and how those inputs are processed is an algorithm problem. Particularly, whether and when output is sent to one or more humans requesting input, and how input from those human(s) is factored into the ultimate decision is an algorithm problem. The fact that the decision algorithms at issue often do not involve human assessment as part of the input set when they should is a defect in the algorithms' fitness for their intended purposes.
  - exelius 12y ago
    I think you're combining "algorithm" and "process" which is something programmers often do (because for a programmer, they are the same thing). Businesses tend to think of "algorithms" as black boxes where you feed a data set in one side and get a modified data set out the other. That data is then incorporated as part of a business process. If you're designing a process where you take the output of a program and delete all records flagged by the program, you can add a human validation step in pretty easily. Companies often have process engineers whose job it is to do exactly this: and when you're talking about data management, process and governance are very important.

by exelius 12y ago

cag_ii 12y ago

Well said. Errors will happen. How they're monitored/dealt with is not an algorithm problem, and it's importance can't be underestimated. In this case "the algorithm" is the fly in the printer from the '85 movie "Brazil". Except these are real stories, it's absurd.

dragonwriter 12y ago
> How they're monitored/dealt with is not an algorithm problem Sure it is; what inputs go into producing the ultimate decision and how those inputs are processed is an algorithm problem. Particularly, whether and when output is sent to one or more humans requesting input, and how input from those human(s) is factored into the ultimate decision is an algorithm problem. The fact that the decision algorithms at issue often do not involve human assessment as part of the input set when they should is a defect in the algorithms' fitness for their intended purposes.
- exelius 12y ago
  I think you're combining "algorithm" and "process" which is something programmers often do (because for a programmer, they are the same thing). Businesses tend to think of "algorithms" as black boxes where you feed a data set in one side and get a modified data set out the other. That data is then incorporated as part of a business process. If you're designing a process where you take the output of a program and delete all records flagged by the program, you can add a human validation step in pretty easily. Companies often have process engineers whose job it is to do exactly this: and when you're talking about data management, process and governance are very important.