6 ms·

I think what a lot of people who don't do software development don't understand is that the majority of software is developed like this: - Weeks, months, or ye

• robertlagrant 3y ago
  > But it was also enabled by a belief in the reliability and credibility of IT systems, which is profoundly undeserved and inaccurate I don't think this is a real thing. The individual system was vouched for by senior people. That's why it wasn't questioned any further.
  • Zhyl 3y ago
    Yes, because they believed in the infallibility of the software in the first instance. It was only subsequently that that believe changed to outright lying to cover their own backs.
    • robertlagrant 3y ago
      I really don't think you can assume that. Other humans will have told them that it was good.
      • Zhyl 3y ago
        I can assume that a wider and deeper understanding of how software works would have meant that the investigations would have demanded a more thorough accounting of how the software worked, evidence provided by the system that it's calculations were correct (an audit trail) and that these questions would be asked at every juncture whenever the issue was raised to a new body or party. As can be seen by following the story, this wasn't the case. The only evidence provided in criminal cases was essentially a witness statement saying that the system was working correctly and that was the end of it.
        • robertlagrant 3y ago
          > The only evidence provided in criminal cases was essentially a witness statement saying that the system was working correctly and that was the end of it. I'm not saying it wasn't. You're assuming a blind faith in all IT is the cause of this witness' belief. I'm disagreeing with you there.
          • Zhyl 3y ago
            I'm assuming an ignorance in IT which allows for a vote of confidence from a trusted party to go unquestioned and unsubstantiated. Not a blind faith in all IT.
            • robertlagrant 3y ago
              Fair enough. I might have misread. It was this that made me think that: > it was also enabled by a belief in the reliability and credibility of IT systems
        • mike_hearn 3y ago
          Note that they had audit trails of various kinds, but some of the bugs yielded incorrect logs.
          • Zhyl 3y ago
            I didn't know that and that's insidious
      • ndsipa_pomu 3y ago
        Yes, but even after getting lots of reports from sub postmasters complaining about bugs, they persisted in telling their victims that they were the only one reporting any issues. They were lying and knew they were lying as they could just force the sub postmaster to make good any "losses". They were most definitely acting maliciously.
        • robertlagrant 3y ago
          I'm not disagreeing with that. If they're lying then we're not in the "blind faith" discussion any more anyway.
  • UglyToad 3y ago
    The other complication was that a change in the law was made to assume computer systems were correct by default due to the problems with the existing law around breathalysers and speed cameras: "In 1997 the Law Commission published a paper which went into some detail about the use of mechanical and computer evidence in court. It seemed a little too fixated with the effective workings of speedometers, traffic lights and breathalysing devices called ‘Intoximeters.’ It concluded that the present law is ‘unsatisfactory’ because of the necessity for prosecutors to ‘prove that the computer is reliable.’"[0] The amended law changed the burden of proof from the prosecution proving the system functioned correctly to the defence proving it didn't, without access to the systems being used to prosecute them. [0]: The Great Post Office Scandal, Nick Wallis
    • robertlagrant 3y ago
      This is a tricky one, because you could just claim that the system wrote audit trails that show you did something due to a bug. How do you prove it either way?
      • UglyToad 3y ago
        I think any (financial crime) case built solely on computer evidence is too weak to be prosecuted, even if that means you end up accepting some non-zero amount of financial crime. "it is better a hundred guilty persons should escape than one innocent person should suffer" as someone once said. In the case of the sub-postmasters the Post Office, as far as I'm aware, never proved where these stolen sums supposedly went. The computer evidence was thought terminating and was the only thing (except false confessions under duress) used to secure these convictions, rather than proper investigative work.
        • VBprogrammer 3y ago
          Yeah, it seems crazy to me that they didn't have to prove where this money went. I suspect the majority of people caught stealing money in any other capacity are probably caught by the changes to their lifestyle being noticed before the financial irregularities are ever spotted.
  • denton-scratch 3y ago
    > The individual system was vouched for by senior people. Well, how the Post Office commissioned the Horizon system is something that the inquiry doesn't seem to have got to yet. From what I've seen of the inquiry (not much), Post Office managers seem to have been desperate to protect the reputation of the Horizon system, and still are. But the miscarriages of justice have now been exposed, and the costs of putting them right are now unavoidable; so the reputation of Horizon can't now be just a matter of subpostmasters. My guess is that defects in Horizon, and damage to Horizon's reputation, will turn out to affect much more than the wronged subpostmasters. I suspect that some very large commercial contracts will turn out to be affected, and paying compensation to the subpostmasters will turn out to be a tiny component of the costs of this catastrophe. The Post Office is already warning that it may not be able to pay the compensation, and that the government will have to step up. The Post Office is a weird mess of subsidiaries and holding companies, and has contracts with all sorts of 3rd parties. I anticipate that we are going to witness a slooow explosion of the Post Office, over like ten years. It's ludicrous, of course, that a private company should be able to be complainant, investigator and prosecutor as well as being sole custodian of the evidence (Horizon logs). It's also ludicrous that the Post Office's lawyers, Cartwright King, should describe requests for disclosure as a "fishing expedition". Defendants shouldn't have to request disclosure of evidence that might help the defence; that is an obligation that falls on the prosecution as a matter of statute law. I'm waiting for the inquiry to find out how the Horizon system was commissioned, and how it was tested; and who was in charge of commissioning.
  • asdefghyk 3y ago
    Re "...The individual system was vouched for by senior people......" The obvious mistake most likely made here is these senior people where not asked , " exactly how they formed this opinion." and what specific evidence they had. I would have said , if they could NOT produce the evidence , that verified their statement, their is no way they could make the statement as accurate about the system accuracy. The people must not have had experience in software development and testing.
    • garyfirestorm 3y ago
      ‘Something something…I swear…to best of my knowledge…that the system works correctly’
• insickness 3y ago
  Also that the post office outright lied about having remote access to users' systems and the ability to change numbers.
• helboi4 3y ago
  Fr I feel like they need some explanation in this show for laymen that the fundamental issue here is not "COMPUTERS SCARY", it's that literally no competent software development team works on the assumption that they are infallible, and if they do these are the consequences. Fallibility is fine, the problem is that Fujitsu and the Post Office were so scared that admitting fallibility was somehow a massive moral failing. It's not. You just be prepared for it and then none of this would have happened. As an SE this is what I was thinking while watching the show throughout. But, as a layman you probably would never understand this nuance.
  • ndsipa_pomu 3y ago
    I don't think it was fear that led to them lying about the fallibility, but hubris. They knew that they had sub postmasters signing that the records were accurate and so they could get them to pay up for the "missing" amounts or face going to prison. They knew the software had many bugs but decided to use that to bully their employees and steal money from them.
    • VBprogrammer 3y ago
      > They knew the software had many bugs but decided to use that to bully their employees and steal money from them. I don't really see how this argument fits. In my understanding the amounts in question were in the range thousands to tens of thousands. It seems implausible that those sums of money, while life changing for the individuals involved, would ever move the needle at all for any of the companies involved. In fact the only people who made money out of it are likely the lawyers.
      • ndsipa_pomu 3y ago
        I suspect it was more to do with bullying than the profit. Make an example of a few sub postmasters and the rest will be too scared to raise a fuss or else be sent to prison.
        • VBprogrammer 3y ago
• prof-dr-ir 3y ago
  > a belief in the reliability and credibility of IT systems I found it fascinating to learn that UK law explicitly does (but did not always) encode this presumption: https://emptycity.substack.com/p/computer-says-guilty-an-introduction https://emptycity.substack.com/p/computer-says-guilty-an-int... (This link is to the substack of David Allen Green, an excellent legal commentator in the UK.)
  • tialaramex 3y ago
    Unfortunately that post claims to be the first of a series, but the series doesn't (yet) seem to continue. It's easy to see both sides of the decision to presume or not presume that machines work - we should like somehow to explain that on the one hand it's unlikely that without his intent Jim's iPhone sent Gemma a series of messages threatening to rape her, and so we shouldn't need to drag Apple engineers into court to convict Jim - but then on other hand it's far from certain that this bank database which says I made 590 cash withdrawals in sixteen minutes is correct, more likely a bug duplicated a single withdrawal I made, and we should require the bank to produce evidence otherwise if they insist I made these withdrawals. I can imagine that smart lawyers motivated by some real world examples and assisted by independent software engineers could come up with a general formula for courts, something a bit larger than either the old rule or the new one, but still relatively pithy. Whether a government would ever make time to do that work is another question.
    • bazzargh 3y ago
      He does have a new article today about this scandal in Prospect magazine. https://www.prospectmagazine.co.uk/ideas/law/64464/post-office-scandal-mr-bates-law https://www.prospectmagazine.co.uk/ideas/law/64464/post-offi...
• deleted 3y ago
  [deleted]
• belter 3y ago
  This interpretation is excessively charitable regarding what really happen here... Fundamentally, the scandal is not merely a result of misplaced trust in an IT system. It exemplifies a series of injustices, overlooked for over 15 years by complacent politicians. Crucially, it illustrates the extent of deceit possible when bonuses for senior management and technical staff are at stake. The UK legal system's appalling quality are starkly highlighted once again. Hundreds of postmasters faced wrongful convictions and prosecutions. Individuals were imprisoned for years, without any concrete evidence of their guilt. A single miscarriage of justice is concerning, but when there are over 250, it raises serious questions about the competence of the judiciary. Astonishingly, more than 700 postmasters were prosecuted without any executive, manager, or statistician recognizing this glaring anomaly. The longevity of the issue is also troubling. Even low-level Fujitsu employees, aware of the system's flaws, remained silent. Despite widespread media coverage and a string of wrongful convictions, leading even to suicides, not a single person within a large team exhibited any kind of moral responsibility or something that looked like a conscience. Their silence, just makes you believe again in conspiracy theories, and raises concerns about the integrity of modern IT systems and AI that we rely on. The simple fact that somebody decided to attest to the absence on bugs on an IT system...An achievement unheard of in the history of software development, and the fact they decided to reiterate that under oath, at a court of law, should be enough for their names to be published and banished for ever, from any profession related to technology.
  • VBprogrammer 3y ago
    > Astonishingly, more than 700 postmasters were prosecuted without any executive, manager, or statistician recognizing this glaring anomaly. I was thinking about this last night. I knew there where about 1000 McDonald's in the UK, Googling it there are about 11,500 post offices. At some point you'd have thought someone would have noticed it was a bit excessive.
    • justincormack 3y ago
      Apparently they thought that their staff were all always dishonest and this new computer system was finally revealing this truth, or something like that.
• genman 3y ago
  I think that the main (human) error is to assign the same level of importance to very different computer systems and their deficiencies and defects. In some industries, like aviation (at least in EU) it appears to be more understood, but in general it doesn't.
• tennis_sort 3y ago
  I worked on government systems circa 2003 and it was a mess of outsourcing, delays and consultants. It was always a question of which corner to cut. I don't think anyone was trying to do the wrong thing but rather timelines and budgets were tight.
• drc500free 3y ago
  I've found "Programming Sucks"[1] is a good way to explain to non-programmers what a shit-show it is. > Imagine joining an engineering team. You’re excited and full of ideas, probably just out of school and a world of clean, beautiful designs, awe-inspiring in their aesthetic unity of purpose, economy, and strength. You start by meeting Mary, project leader for a bridge in a major metropolitan area. Mary introduces you to Fred, after you get through the fifteen security checks installed by Dave because Dave had his sweater stolen off his desk once and Never Again. Fred only works with wood, so you ask why he’s involved because this bridge is supposed to allow rush-hour traffic full of cars full of mortal humans to cross a 200-foot drop over rapids. Don’t worry, says Mary, Fred’s going to handle the walkways. What walkways? Well Fred made a good case for walkways and they’re going to add to the bridge’s appeal. Of course, they’ll have to be built without railings, because there’s a strict no railings rule enforced by Phil, who’s not an engineer. Nobody’s sure what Phil does, but it’s definitely full of synergy and has to do with upper management, whom none of the engineers want to deal with so they just let Phil do what he wants. Sara, meanwhile, has found several hemorrhaging-edge paving techniques, and worked them all into the bridge design, so you’ll have to build around each one as the bridge progresses, since each one means different underlying support and safety concerns. Tom and Harry have been working together for years, but have an ongoing feud over whether to use metric or imperial measurements, and it’s become a case of “whoever got to that part of the design first.” This has been such a headache for the people actually screwing things together, they’ve given up and just forced, hammered, or welded their way through the day with whatever parts were handy. Also, the bridge was designed as a suspension bridge, but nobody actually knew how to build a suspension bridge, so they got halfway through it and then just added extra support columns to keep the thing standing, but they left the suspension cables because they’re still sort of holding up parts of the bridge. Nobody knows which parts, but everybody’s pretty sure they’re important parts. After the introductions are made, you are invited to come up with some new ideas, but you don’t have any because you’re a propulsion engineer and don’t know anything about bridges. > Would you drive across this bridge? No. If it somehow got built, everybody involved would be executed. Yet some version of this dynamic wrote every single program you have ever used, banking software, websites, and a ubiquitously used program that was supposed to protect information on the internet but didn’t. > You can’t restart the internet. Trillions of dollars depend on a rickety cobweb of unofficial agreements and “good enough for now” code with comments like “TODO: FIX THIS IT’S A REALLY DANGEROUS HACK BUT I DON’T KNOW WHAT’S WRONG” that were written ten years ago. [1] https://www.stilldrinking.org/programming-sucks https://www.stilldrinking.org/programming-sucks