4 ms·

This would be a very interesting blog post with probably hundreds of comments here on HN. Though if it fails more often “than a roll of a die,” doesn’t that hi

by ptr 8y ago

This would be a very interesting blog post with probably hundreds of comments here on HN. Though if it fails more often “than a roll of a die,” doesn’t that hint at some predictive power? Maybe I’m misunderstanding.

setr 8y ago
I read that as, if a die has 1:6 chance of success, ML has a 1:7 or worse chance
eksemplar 8y ago
Well maybe. There are some pretty big differences in how ML is traditionally applied and how it's used in the public sector where we focus much more on an individual level and have a much lower tolerance of errors. Right now the predictions we're getting of ML are so terrible they might as well be random. That's terrible, because we're not trying to determine if we can sell you X, we're trying to predict how likely you are to fall into X. Because the public sector doesn't really interact with the adult you, unless you're either damaged, have broken something or want to change part of society. Imagine we scored you as a likely alcoholic because people with a case history like yours indicated a high risk of alcoholism. So we start taking preemptive measures and the entire system grinds into gear, partly trying to prevent you from falling to the bottle but also preparing for if you do. You really wouldn't want to interact with a public system that's put the wrong label on you, trust me. So we have to be more than a little sure we get things right. You can probably use ML to predict general trends, but I don't have much data on it, because we use traditional analysts in the public sector. Not because they are great at it, but because they're there. We've been predicting general trends since before computer technology, and the people who work in those departments typically aren't very tech savy. It would be a tremendous cost of resources to make them use ML, or to replace them with data scientists. I guess in the future those analytics departments may increasingly rely on ML, but to be completely honest, I have no idea if ML or traditional analytics is better at predicting the outcome. The fact that we haven't started using ML, haven't hired a single data scientist or signed a single contractor on the area probably speaks for itself though, if it was truly valuable, I suspect we'd have it by now. By comparison, RPA didn't really have a great start in the public sector, but today it's absolutely everywhere because it turned out to be really valuable. It's not like we don't use ML at all though. We do, for stuff like sorting through four million case files looking for specific errors. Something that takes tens of workers months of work to do, but can be done in a few hours when you rent enough hardware in the cloud. We just don't use ML for any sort of analytics or prediction.
- mcguire 8y ago
  I would really love to hear more about this. "Imagine we scored you as a likely alcoholic because people with a case history like yours indicated a high risk of alcoholism. So we start taking preemptive measures and the entire system grinds into gear, partly trying to prevent you from falling to the bottle but also preparing for if you do. You really wouldn't want to interact with a public system that's put the wrong label on you, trust me." That's exactly the sort of specific example that is missing from the stories about ML doing things like encoding existing biases.
Spooky23 8y ago
We’re still learning how to apply it. In many cases, ML is something that can solve a problem, but it’s often not the obvious/best solution appropriate for the public sector use case vs a pre-existing rules engine, for example.

by ptr 8y ago

setr 8y ago

I read that as, if a die has 1:6 chance of success, ML has a 1:7 or worse chance

eksemplar 8y ago

Well maybe. There are some pretty big differences in how ML is traditionally applied and how it's used in the public sector where we focus much more on an individual level and have a much lower tolerance of errors. Right now the predictions we're getting of ML are so terrible they might as well be random. That's terrible, because we're not trying to determine if we can sell you X, we're trying to predict how likely you are to fall into X. Because the public sector doesn't really interact with the adult you, unless you're either damaged, have broken something or want to change part of society. Imagine we scored you as a likely alcoholic because people with a case history like yours indicated a high risk of alcoholism. So we start taking preemptive measures and the entire system grinds into gear, partly trying to prevent you from falling to the bottle but also preparing for if you do. You really wouldn't want to interact with a public system that's put the wrong label on you, trust me. So we have to be more than a little sure we get things right. You can probably use ML to predict general trends, but I don't have much data on it, because we use traditional analysts in the public sector. Not because they are great at it, but because they're there. We've been predicting general trends since before computer technology, and the people who work in those departments typically aren't very tech savy. It would be a tremendous cost of resources to make them use ML, or to replace them with data scientists. I guess in the future those analytics departments may increasingly rely on ML, but to be completely honest, I have no idea if ML or traditional analytics is better at predicting the outcome. The fact that we haven't started using ML, haven't hired a single data scientist or signed a single contractor on the area probably speaks for itself though, if it was truly valuable, I suspect we'd have it by now. By comparison, RPA didn't really have a great start in the public sector, but today it's absolutely everywhere because it turned out to be really valuable. It's not like we don't use ML at all though. We do, for stuff like sorting through four million case files looking for specific errors. Something that takes tens of workers months of work to do, but can be done in a few hours when you rent enough hardware in the cloud. We just don't use ML for any sort of analytics or prediction.

mcguire 8y ago
I would really love to hear more about this. "Imagine we scored you as a likely alcoholic because people with a case history like yours indicated a high risk of alcoholism. So we start taking preemptive measures and the entire system grinds into gear, partly trying to prevent you from falling to the bottle but also preparing for if you do. You really wouldn't want to interact with a public system that's put the wrong label on you, trust me." That's exactly the sort of specific example that is missing from the stories about ML doing things like encoding existing biases.

Spooky23 8y ago

We’re still learning how to apply it. In many cases, ML is something that can solve a problem, but it’s often not the obvious/best solution appropriate for the public sector use case vs a pre-existing rules engine, for example.