4 ms·

The biggest issue isn't the software glitch system. It's the legal system that threatened innocent people with prison for theft unless they admit they are guilt

The biggest issue isn't the software glitch system. It's the legal system that threatened innocent people with prison for theft unless they admit they are guilty for crimes they didn't do and pay for damages they didn't do. This case was brought to public attention and repairs were attempted only because it's huge and involved hundreds or thousands of people. How many disparate cases there are, where people's lives are destroyed and innocents are rotting in jails, we have to ask?

PakG1 3y ago
There are not as many that involve jail, but there are a variety that involve ruined lives and even bankruptcy. A recent example is the Phoenix pay system that was used to pay Canadian federal government employees and contractors. https://en.wikipedia.org/wiki/Phoenix_pay_system https://en.wikipedia.org/wiki/Phoenix_pay_system And I agree that the problem wasn't the glitches. I personally think it was the corporate governance that failed, not the software development and debugging process. The legal system was complicit and enlarged the overall consequences, but the but for test tells me that it was the poor corporate governance that was at fault for a root cause.
usr1106 3y ago
Every skillful programmer knows that all software is crap. Judges, salesman, and managers don't understand that.
- jack_riminton 3y ago
  A big problem is they don't want to do the work to understand that, which is the exact outlook the PO had... "We need some software, ok let's get a big reputable company in to do it for us, we shouldn't get bogged down with all those horrible technical details"
  - zimpenfish 3y ago
    Not wanting to defend the PO but it wasn't really their decision - it was a PFI (private finance initiative) foisted upon them by the Tory government of the day as one of their recurring "STOP BENEFIT FRAUD!" lunacies.
    - jack_riminton 3y ago
      Wasn't it enacted under Blair?
      - zimpenfish 3y ago
        Started in 1994 under the Tories, rolled out in 1999 under Labour as the reduced system after DSS withdrew.
        jack_riminton 3y ago
        Thanks
- watwut 3y ago
  > Judges, salesman, and managers don't understand that. They do understand that. They do not care.
- izacus 3y ago
  And that skillful programmer will fight with all power to avoid any kind of minimum standard and liability for crap software, continuing the cycle and abuse.
  - the8472 3y ago
    It's always spectrum from THE SOFTWARE IS PROVIDED “AS IS” to high-assurance methods used in aerospace and similar safety-critical fields. The skillfull programmer may accept liability when you give him a verification team with a few PhDs, the ability to withhold signoffs, flexible deadlines etc. etc. Few are willing or required to pay for that. So they get a mystery box with a 90% chance of crap.
- gwd 3y ago
  > Judges... don't understand that. I think for the courts the issue is a bit more subtle. The question is, who's job is it to prove that the other person is wrong ("burden of proof")? Should it be the job of the prosecutor to prove that Intel's processor produces the right answer when an ADD instruction is executed? Or should it be the job of the defendant to show that Intel's processor doesn't produce the right answer? What about proving that the compiler produced binaries which faithfully represent the algorithm? What about Excel? In our normal life, if a computer is doing the wrong thing, we don't start by assuming a broken compiler; we start by assuming that the new, not-well-tested code is probably broken. It seems that in the UK before the 90's, the burden of proof was always on the prosecutor to prove almost everything about the system, which is kind of ridiculous. So they passed a law trying to fix it, but messed it up the other way, putting the entire burden of proof on the defendant, without giving them any real way to disprove it. (I mean, shouldn't "discovery" at least mean I can inspect the source code?) A more balanced law would say that widely-used software with extensive test suites can generally be assumed to be working properly; but that custom-purpose software needs at least some level of evidence that it's correct, and that defendants have a right to inspect any software that's used against them in court for defects.
- inglor_cz 3y ago
  This is a somewhat Sith-like dealing with absolutes. Laypeople generally understand that software may crap the bed in the sense of "the system is down, please wait, then try again". But few people have experienced subtle changes in stored data. A judge looking into his document cloud may be ready to see a "sorry, not available right now" notice, but doesn't expect that some sinister program is, in the background, silently editing texts of his judgments and pronouncing people guilty when he intended to free them etc. The problem with the Horizon scandal is in this sinister manipulation of data. It may also have been done by Fujitsu people themselves, in order to cover some tracks and tamper with evidence. This is a very untypical failure mode.
bsder 3y ago
Agreed. The bug is a footnote. The legal system failed these people horribly. And the people who pursued these cases with no direct evidence whatsoever should suffer jail time.
- kibwen 3y ago
  It's correct that this is ultimately a failure of the legal system. However, the role of software here must not be minimized. Software makes it easier than ever to diffuse responsibility and create opaque processes that leave the least powerful people at the bottom of the hierarchy holding the bag. By rigidly encoding flawed assumptions and executing them without question, software is the ultimate realizer of our Kafkaesque nightmares.
  - lupusreal 3y ago
    It's not entirely unlike Therac-25, including the deaths (albeit more indirectly caused in this case.). There was a certain element of operator error, but that doesn't excuse the faulty programming.
  - bsder 3y ago
    > However, the role of software here must not be minimized. No argument that the software bears fault, too. However, when you accuse someone of stealing money, you should have to prove that they stole the money. This isn't some invisible crime. There should need to be evidence that the stolen money went into their account, got spent to buy something, got pulled from the till on camera, got transferred to Bitcoin--something. The fact that all these people got convicted with no evidence that the money was ever in their possession is a gigantic legal problem.
jrumbut 3y ago
> The biggest issue isn't the software glitch system. I disagree, the software glitch was the problem here. We are supposed to be able to rely on computers to store and add numbers or report a system failure. This accounting software showed in black and white that some funds that the sub-postmasters were responsible for had gone missing. What else was the legal system supposed to do? The broken software was simulating crime perfectly.
- Arn_Thor 3y ago
  Absolutely wrong. Mistakes happen. Bugs, fat fingers, laziness, hangovers--whether by human or machine, errors occur. The legal system was supposed to uncover the facts. Because of the Post Office coverup, the judges were told " no, there are no bugs. No, nobody has remote access to these terminals. Yes, the only possible way these figures could turn up is through theft." This despite the fact that at least one Post Office inspector explicitly wrote in a report that there was no evidence of theft. The legal system failed to penetrate the veil of lies and find the truth. That's a legal systemic failure.
- rcxdude 3y ago
  It wasn't though. If the post office enforcers had taken even a cursory look at the transactions around the 'thefts', they would have noticed obvious errors. One of the bugs basically just duplicated a close-of-day transaction, sometimes many times. This would obviously have looked like an error, it would be a stupid way to commit fraud. It was obvious that the Post Office just preferred to extort money out of the postmasters as opposed to actually work out what was going on (as evidenced by the bonuses for successful payments or convictions)
- DharmaPolice 3y ago
  I think if a handful of people had been prosecuted then it would still be an outrage but understandable. But this was hundreds of cases. I think the legal system has some responsibility for not maybe thinking "Huh, what are the chances of so many previously law abiding people all committing the same crime in the same time period?".
- YukiElectronics 3y ago
ivanhoe 3y ago
Exactly, and by the number of errors among the death-row/lifelong convictions - which one would presume are the most sensitive and carefully reviewed ones - these numbers are HUGE, especially in the US.
scott_w 3y ago
It also doesn't help that the lawyers in the case fabricated evidence and covered up the issues with Horizon to secure the convictions. The people doing this need to spend some time in one of the overcrowded prison cells they sent the sub-postmasters to.

PakG1 3y ago

There are not as many that involve jail, but there are a variety that involve ruined lives and even bankruptcy. A recent example is the Phoenix pay system that was used to pay Canadian federal government employees and contractors. https://en.wikipedia.org/wiki/Phoenix_pay_system https://en.wikipedia.org/wiki/Phoenix_pay_system And I agree that the problem wasn't the glitches. I personally think it was the corporate governance that failed, not the software development and debugging process. The legal system was complicit and enlarged the overall consequences, but the but for test tells me that it was the poor corporate governance that was at fault for a root cause.

usr1106 3y ago

Every skillful programmer knows that all software is crap. Judges, salesman, and managers don't understand that.

jack_riminton 3y ago
A big problem is they don't want to do the work to understand that, which is the exact outlook the PO had... "We need some software, ok let's get a big reputable company in to do it for us, we shouldn't get bogged down with all those horrible technical details"
- zimpenfish 3y ago
  Not wanting to defend the PO but it wasn't really their decision - it was a PFI (private finance initiative) foisted upon them by the Tory government of the day as one of their recurring "STOP BENEFIT FRAUD!" lunacies.
  - jack_riminton 3y ago
    Wasn't it enacted under Blair?
    - zimpenfish 3y ago
      Started in 1994 under the Tories, rolled out in 1999 under Labour as the reduced system after DSS withdrew.
      - jack_riminton 3y ago
        Thanks
watwut 3y ago
> Judges, salesman, and managers don't understand that. They do understand that. They do not care.
izacus 3y ago
And that skillful programmer will fight with all power to avoid any kind of minimum standard and liability for crap software, continuing the cycle and abuse.
- the8472 3y ago
  It's always spectrum from THE SOFTWARE IS PROVIDED “AS IS” to high-assurance methods used in aerospace and similar safety-critical fields. The skillfull programmer may accept liability when you give him a verification team with a few PhDs, the ability to withhold signoffs, flexible deadlines etc. etc. Few are willing or required to pay for that. So they get a mystery box with a 90% chance of crap.
gwd 3y ago
> Judges... don't understand that. I think for the courts the issue is a bit more subtle. The question is, who's job is it to prove that the other person is wrong ("burden of proof")? Should it be the job of the prosecutor to prove that Intel's processor produces the right answer when an ADD instruction is executed? Or should it be the job of the defendant to show that Intel's processor doesn't produce the right answer? What about proving that the compiler produced binaries which faithfully represent the algorithm? What about Excel? In our normal life, if a computer is doing the wrong thing, we don't start by assuming a broken compiler; we start by assuming that the new, not-well-tested code is probably broken. It seems that in the UK before the 90's, the burden of proof was always on the prosecutor to prove almost everything about the system, which is kind of ridiculous. So they passed a law trying to fix it, but messed it up the other way, putting the entire burden of proof on the defendant, without giving them any real way to disprove it. (I mean, shouldn't "discovery" at least mean I can inspect the source code?) A more balanced law would say that widely-used software with extensive test suites can generally be assumed to be working properly; but that custom-purpose software needs at least some level of evidence that it's correct, and that defendants have a right to inspect any software that's used against them in court for defects.
inglor_cz 3y ago
This is a somewhat Sith-like dealing with absolutes. Laypeople generally understand that software may crap the bed in the sense of "the system is down, please wait, then try again". But few people have experienced subtle changes in stored data. A judge looking into his document cloud may be ready to see a "sorry, not available right now" notice, but doesn't expect that some sinister program is, in the background, silently editing texts of his judgments and pronouncing people guilty when he intended to free them etc. The problem with the Horizon scandal is in this sinister manipulation of data. It may also have been done by Fujitsu people themselves, in order to cover some tracks and tamper with evidence. This is a very untypical failure mode.

bsder 3y ago

Agreed. The bug is a footnote. The legal system failed these people horribly. And the people who pursued these cases with no direct evidence whatsoever should suffer jail time.

kibwen 3y ago
It's correct that this is ultimately a failure of the legal system. However, the role of software here must not be minimized. Software makes it easier than ever to diffuse responsibility and create opaque processes that leave the least powerful people at the bottom of the hierarchy holding the bag. By rigidly encoding flawed assumptions and executing them without question, software is the ultimate realizer of our Kafkaesque nightmares.
- lupusreal 3y ago
  It's not entirely unlike Therac-25, including the deaths (albeit more indirectly caused in this case.). There was a certain element of operator error, but that doesn't excuse the faulty programming.
- bsder 3y ago
  > However, the role of software here must not be minimized. No argument that the software bears fault, too. However, when you accuse someone of stealing money, you should have to prove that they stole the money. This isn't some invisible crime. There should need to be evidence that the stolen money went into their account, got spent to buy something, got pulled from the till on camera, got transferred to Bitcoin--something. The fact that all these people got convicted with no evidence that the money was ever in their possession is a gigantic legal problem.

jrumbut 3y ago

> The biggest issue isn't the software glitch system. I disagree, the software glitch was the problem here. We are supposed to be able to rely on computers to store and add numbers or report a system failure. This accounting software showed in black and white that some funds that the sub-postmasters were responsible for had gone missing. What else was the legal system supposed to do? The broken software was simulating crime perfectly.

Arn_Thor 3y ago
Absolutely wrong. Mistakes happen. Bugs, fat fingers, laziness, hangovers--whether by human or machine, errors occur. The legal system was supposed to uncover the facts. Because of the Post Office coverup, the judges were told " no, there are no bugs. No, nobody has remote access to these terminals. Yes, the only possible way these figures could turn up is through theft." This despite the fact that at least one Post Office inspector explicitly wrote in a report that there was no evidence of theft. The legal system failed to penetrate the veil of lies and find the truth. That's a legal systemic failure.
rcxdude 3y ago
It wasn't though. If the post office enforcers had taken even a cursory look at the transactions around the 'thefts', they would have noticed obvious errors. One of the bugs basically just duplicated a close-of-day transaction, sometimes many times. This would obviously have looked like an error, it would be a stupid way to commit fraud. It was obvious that the Post Office just preferred to extort money out of the postmasters as opposed to actually work out what was going on (as evidenced by the bonuses for successful payments or convictions)
DharmaPolice 3y ago
I think if a handful of people had been prosecuted then it would still be an outrage but understandable. But this was hundreds of cases. I think the legal system has some responsibility for not maybe thinking "Huh, what are the chances of so many previously law abiding people all committing the same crime in the same time period?".
YukiElectronics 3y ago

ivanhoe 3y ago

Exactly, and by the number of errors among the death-row/lifelong convictions - which one would presume are the most sensitive and carefully reviewed ones - these numbers are HUGE, especially in the US.

scott_w 3y ago

It also doesn't help that the lawyers in the case fabricated evidence and covered up the issues with Horizon to secure the convictions. The people doing this need to spend some time in one of the overcrowded prison cells they sent the sub-postmasters to.