5 ms·

Government’s Use of Algorithm Serves Up False Fraud Charges

55 points by noizejoy 6y ago

popotamonga 6y ago
Ive been on the wrong end of an IRS automated and wrong 10k€ fine. Had to spend 3k in court to get it back. Automated justice was once thought to be the next great thing. I'm not so sure.
Hokusai 6y ago
> Michigan Integrated Data Automated System, or MiDAS They should have listened to King Midas cautionary tale. This kind of system will work better if it worked both ways. A system that notifies you when you are eligible for unemployment and when you are not eligible anymore. Instead it seems it only tries to identify fraud from incomplete data. I know that automatic taxing and other government automation is not implemented because there are opposed lobbying efforts. But, the current state is a waste of money and negatively affect USA citizens mental health. If the USA wants to continue being competitive in the world economy it needs to step down from its pedestal and work hard to improve its infrastructure, digital and physical.
dabeeeenster 6y ago
Reminds me a little of this story: https://www.bbc.co.uk/news/uk-england-50747143 https://www.bbc.co.uk/news/uk-england-50747143 It's pretty insane.
- goodcanadian 6y ago
  There is actually an update on that today: https://www.bbc.co.uk/news/uk-52905378 https://www.bbc.co.uk/news/uk-52905378
  - ChrisMarshallNY 6y ago
    Much as we may decry the “lawyer disease,” when we read stories like these, we have to conclude that the only recourse for some folks is a lawsuit. It’s not helped by corporate PR efforts that gin up fervor about “frivolous” lawsuits. Like so: https://www.vox.com/policy-and-politics/2016/12/16/13971482/mcdonalds-coffee-lawsuit-stella-liebeck https://www.vox.com/policy-and-politics/2016/12/16/13971482/... I’ve known a number of folks that have been involved in lawsuits with considerable merit, and not one received anything even close to addressing the damage they suffered; often after years of bruising court battles.
mjevans 6y ago
In a quick skim I see the real problem is mentioned several times through the article. The system was designed to "save costs" by removing humans. Not by feeding the humans a curated and highlighted set of cases to review for confirmation. A proper system would hold for human review as well as have physical mail confirmation sent to relevant addresses.
jariel 6y ago
Is there no judicial process? How can someone not take the gov. to court, show the judge the numbers, and the judge put a hold on Midas? And how is such massive fraud only highlighted on some random blog? This is also quite a good example of illustrating how cable news is so rigorously narrative-driven: their websites right now may have 10 links relating to their 'favorite local story they chose to make a national story' which plays into some kind of politics, instead of having references to more nuanced things like this. But without some 'politician they don't like to blame' it's hard to make into clickbait. I believe these more 'operational' realities of governance are far more impactful on our civilization than the more ideological things we argue about all day. A highly functional and efficient bureaucracy makes political arguments effete.
ChrisMarshallNY 6y ago
I really feel as if software quality is at the heart of a lot of this. I’m a bit despondent over the casual approach to quality exhibited by a lot of corporations, and the stakes are getting higher. I dread what will happen when the software controls even more critical infrastructure. The software is getting so complex that I can’t see how it can be effectively tested. Meanwhile, the entire IT industry is on a wholesale mission to remove experienced engineers, and replace them with outsourced talent, or people right out of bootcamps. In this case, the people affected were probably not the ones that would (or could) raise a lot of hell. Also, some managers probably got bonuses or raises, based on this. It’s funny. I remember, many years ago, a story about errors in supermarket pricing, and how pricing errors invariably favored the supermarket. It was determined that was because the supermarkets only considered it a problem if they had to pay. I suspect this is similar.
- OldHand2018 6y ago
  > I’m a bit despondent over the casual approach to quality exhibited by a lot of corporations, and the stakes are getting higher. I don't think that there is any evidence that the quality of the software was low. The problem was the design of the system and the willingness of the developers to implement it.
  - ChrisMarshallNY 6y ago
    Fair 'nuff, but I still have my feelings about it. > The problem was the design of the system and the willingness of the developers to implement it. I would consider the design of the system as a "quality" metric. Usability, accessibility, training and support are all what I consider to be important components of "quality." We have a big problem (not just in our industry) of "Not mah job, man."
    - OldHand2018 6y ago
      Using your definition, I'd definitely agree that there is a big problem.

55 points by noizejoy 6y ago

Ive been on the wrong end of an IRS automated and wrong 10k€ fine. Had to spend 3k in court to get it back. Automated justice was once thought to be the next great thing. I'm not so sure.

Hokusai 6y ago

> Michigan Integrated Data Automated System, or MiDAS They should have listened to King Midas cautionary tale. This kind of system will work better if it worked both ways. A system that notifies you when you are eligible for unemployment and when you are not eligible anymore. Instead it seems it only tries to identify fraud from incomplete data. I know that automatic taxing and other government automation is not implemented because there are opposed lobbying efforts. But, the current state is a waste of money and negatively affect USA citizens mental health. If the USA wants to continue being competitive in the world economy it needs to step down from its pedestal and work hard to improve its infrastructure, digital and physical.

dabeeeenster 6y ago

Reminds me a little of this story: https://www.bbc.co.uk/news/uk-england-50747143 https://www.bbc.co.uk/news/uk-england-50747143 It's pretty insane.

goodcanadian 6y ago
There is actually an update on that today: https://www.bbc.co.uk/news/uk-52905378 https://www.bbc.co.uk/news/uk-52905378
- ChrisMarshallNY 6y ago
  Much as we may decry the “lawyer disease,” when we read stories like these, we have to conclude that the only recourse for some folks is a lawsuit. It’s not helped by corporate PR efforts that gin up fervor about “frivolous” lawsuits. Like so: https://www.vox.com/policy-and-politics/2016/12/16/13971482/mcdonalds-coffee-lawsuit-stella-liebeck https://www.vox.com/policy-and-politics/2016/12/16/13971482/... I’ve known a number of folks that have been involved in lawsuits with considerable merit, and not one received anything even close to addressing the damage they suffered; often after years of bruising court battles.

mjevans 6y ago

In a quick skim I see the real problem is mentioned several times through the article. The system was designed to "save costs" by removing humans. Not by feeding the humans a curated and highlighted set of cases to review for confirmation. A proper system would hold for human review as well as have physical mail confirmation sent to relevant addresses.

jariel 6y ago

Is there no judicial process? How can someone not take the gov. to court, show the judge the numbers, and the judge put a hold on Midas? And how is such massive fraud only highlighted on some random blog? This is also quite a good example of illustrating how cable news is so rigorously narrative-driven: their websites right now may have 10 links relating to their 'favorite local story they chose to make a national story' which plays into some kind of politics, instead of having references to more nuanced things like this. But without some 'politician they don't like to blame' it's hard to make into clickbait. I believe these more 'operational' realities of governance are far more impactful on our civilization than the more ideological things we argue about all day. A highly functional and efficient bureaucracy makes political arguments effete.

ChrisMarshallNY 6y ago

I really feel as if software quality is at the heart of a lot of this. I’m a bit despondent over the casual approach to quality exhibited by a lot of corporations, and the stakes are getting higher. I dread what will happen when the software controls even more critical infrastructure. The software is getting so complex that I can’t see how it can be effectively tested. Meanwhile, the entire IT industry is on a wholesale mission to remove experienced engineers, and replace them with outsourced talent, or people right out of bootcamps. In this case, the people affected were probably not the ones that would (or could) raise a lot of hell. Also, some managers probably got bonuses or raises, based on this. It’s funny. I remember, many years ago, a story about errors in supermarket pricing, and how pricing errors invariably favored the supermarket. It was determined that was because the supermarkets only considered it a problem if they had to pay. I suspect this is similar.

OldHand2018 6y ago
> I’m a bit despondent over the casual approach to quality exhibited by a lot of corporations, and the stakes are getting higher. I don't think that there is any evidence that the quality of the software was low. The problem was the design of the system and the willingness of the developers to implement it.
- ChrisMarshallNY 6y ago
  Fair 'nuff, but I still have my feelings about it. > The problem was the design of the system and the willingness of the developers to implement it. I would consider the design of the system as a "quality" metric. Usability, accessibility, training and support are all what I consider to be important components of "quality." We have a big problem (not just in our industry) of "Not mah job, man."
  - OldHand2018 6y ago
    Using your definition, I'd definitely agree that there is a big problem.