3 ms·

David Allen Green is a great commentator on law in the UK. During the Brexit period, he was a contributing editor to the Financial Times, with similarly insight

David Allen Green is a great commentator on law in the UK. During the Brexit period, he was a contributing editor to the Financial Times, with similarly insightful analyses of how Boris Johnson et al were misusing the government's legal powers. Anyway, David also posted this to his non-gated "The Law and Policy Blog" at [1]. [1] https://davidallengreen.com/2023/09/computer-says-guilty-an-introduction-to-the-evidential-presumption-that-computers-are-operating-correctly/ https://davidallengreen.com/2023/09/computer-says-guilty-an-... There are more comments there, including this one from me (written for his blog's legal/non-technical audience, not for the HN crowd!): "There has been a big shift in the practice of computer programming in the last 25 years. In the “old days”, software was delivered on floppy disks or tape, and pretty much had to be correct first time. Nowadays, everything is on the web, “agile” means the functionality gets delivered incrementally in 2-week sprints. It is regarded as ok to leave out the difficult cases or ship bugs, provided they don’t affect too many users too much of the time. In other words, the software is never really finished. With that in mind, it becomes simple to rebut the presumption of correctness: if the company’s tech organisation describes itself as “agile”, using “scrum” etc, its software – by design – contains bugs and omissions, both known and unknown. Fortunately, modern development practices means evidence for this will be in the dev tickets in GitHub, Azure DevOps or whatever other systems their dev teams use."

bostik 3y ago
Thank you, I have been looking for a resource like this, one that would come with an RSS feed. Now I have one less thing to hunt around for.
_dain_ 3y ago
I had a visceral negative reaction to the idea that computers should be presumed to be working correctly, but I was having trouble articulating in precise words what was wrong with it. I mean, obviously we get work done with use computers every day and they seem to work alright. A commenter at that link does a good job: >To treat the operation of an extensive software system as being a mechanical device is as gross a category error as treating an airliner as being merely a collection of nuts and bolts. In either case we can satisfy ourselves that the elementary parts are performing within specification without having gained any confidence that the complete assemblage will function properly under all conditions to which it will be reasonably exposed; indeed we won’t even be confident that we understand what it was exposed to. Even showing that it has passed a collection of tests isn’t adequate: one needs to see what unfixed issues are being registered against it and how they are being resolved (fixed? Swept under the carpet to keep the numbers below since arbitrary threshold?) The opposing side in case hinging upon accuracy of computer records should have the right to access the issue database, and absence of a well-run system or a large number of issues in it (or evidence that issues have been “scrubbed”) should be taken as a strong argument that the computer records are unreliable. >Much could be learned by the software industry from the culture of aviation safety, where even minor incidents are followed up by detailed investigations intentionally run in a non-judgemental way and the findings distilled into public reports and recommendations. Similarly, one of the best legal developments in recent years has been the increasing tendency to publish detailed judgements.

bostik 3y ago

Thank you, I have been looking for a resource like this, one that would come with an RSS feed. Now I have one less thing to hunt around for.

_dain_ 3y ago

I had a visceral negative reaction to the idea that computers should be presumed to be working correctly, but I was having trouble articulating in precise words what was wrong with it. I mean, obviously we get work done with use computers every day and they seem to work alright. A commenter at that link does a good job: >To treat the operation of an extensive software system as being a mechanical device is as gross a category error as treating an airliner as being merely a collection of nuts and bolts. In either case we can satisfy ourselves that the elementary parts are performing within specification without having gained any confidence that the complete assemblage will function properly under all conditions to which it will be reasonably exposed; indeed we won’t even be confident that we understand what it was exposed to. Even showing that it has passed a collection of tests isn’t adequate: one needs to see what unfixed issues are being registered against it and how they are being resolved (fixed? Swept under the carpet to keep the numbers below since arbitrary threshold?) The opposing side in case hinging upon accuracy of computer records should have the right to access the issue database, and absence of a well-run system or a large number of issues in it (or evidence that issues have been “scrubbed”) should be taken as a strong argument that the computer records are unreliable. >Much could be learned by the software industry from the culture of aviation safety, where even minor incidents are followed up by detailed investigations intentionally run in a non-judgemental way and the findings distilled into public reports and recommendations. Similarly, one of the best legal developments in recent years has been the increasing tendency to publish detailed judgements.