4 ms·

I was reading an interesting bit today (https://read.uolpress.co.uk/read/electronic-evidence-and-electronic-signatures/section/0f113452-78ad-4c82-b28f-08cff1877

I was reading an interesting bit today (https://read.uolpress.co.uk/read/electronic-evidence-and-electronic-signatures/section/0f113452-78ad-4c82-b28f-08cff1877036#book-part-005 https://read.uolpress.co.uk/read/electronic-evidence-and-ele...) about the "presumption of reliability" that can occur in cases involving "mechanical instruments" which is extended to include software. I think the idea that complex systems are assumed to always be correct is... dicey at the best of times and even more so when it's critical to a criminal case.

ndsipa_pomu 3y ago
For some computer functions it makes sense, such as audit logs and other basic operations e.g. timestamps. For complicated accounting software, there should definitely be some kind of proof of correctness such as being self consistent and having sufficient auditing and logging systems.
- graemep 3y ago
  It was actually intended to stop people challenging the reliability of things like speed cameras.
  - ndsipa_pomu 3y ago
    Yes, that's an interesting case as cameras are routinely calibrated, but the software is closed source which I think is a mistake. It should be possible to challenge whether a system is operating correctly if you have decent evidence against it e.g. GPS tracking. I do think that modern cars should be fitted with a black box device with the owner having ultimate control over access to it, so they could submit it for evidence to prevent miscarriages of justice.
    - graemep 3y ago
      I think "presumed to be operating normally for certain devices is certain oversight and testing requirements are met" is reasonable. Definitely ought to be open source. I wonder whether the black boxes insurance companies like to push would provide the evidence required?
    - acdha 3y ago
      > I do think that modern cars should be fitted with a black box device with the owner having ultimate control over access to it, so they could submit it for evidence to prevent miscarriages of justice I’m not sure this is workable: if the owner had control, there’d be a cottage industry of people offering to fake evidence against camera tickets. If the owner doesn’t, it’d get requested by police and insurance routinely and since most drivers regularly break local traffic laws that’d have a big negative impact on the owners, which makes me think it’s politically infeasible. Insurance companies might try to mandate that but I imagine they are very careful about changes which could shift customers.
      - ndsipa_pomu 3y ago
        I think those issues could be overcome by using encryption and checksums for the logs with the decryption key held by a third party that would read and release the data on request by the owner or in criminal cases where the owner was a victim. The more likely model is that the insurance companies own and provide the black boxes in return for reduced premiums from the owner. The current situation is that some drivers run their own dashcams and can choose to provide video evidence, but that could be open to abuse if someone very carefully edits the video and no-one spots it. I agree that the drivers shouldn't be forced to self-incriminate - the police should be using their own evidence which could well be from other drivers submitting their own driving data/video.
- toyg 3y ago
  This is actually one of the few applications of blockchain techs that makes sense - as a write-only store of transactions for auditing purposes, it's potentially very practical.
  - ndsipa_pomu 3y ago
    I agree, but wouldn't there be problems with the speed and cost in electricity. Also, would it need to be a single logging blockchain shared by everyone or I would imagine there could be issues with every computer system running its own blockchain - would that make it easier to fake?
    - menzoic 3y ago
      >wouldn't there be problems with the speed and cost in electricity proof of stake blockchains don't have those issues
    - parineum 3y ago
      > I agree, but wouldn't there be problems with the speed and cost in electricity It's either that or it's vulnerable to 50%+1 attacks.
denton-scratch 3y ago
> complex systems are assumed to always be correct ..absent evidence to the contrary. But in the Post Office case, the Post Office had all the evidence, and refused to disclose it. As far as I'm aware, failure to disclose evidence that might help the defence is perverting the course of justice. I don't know why no manager's been charged.
- raesene9 3y ago
  Yep that's a huge challenge. The other one is that individual defendant's are unlikely to be able to afford the expertise needed to analyze a large complex setup, even if the details of the system (e.g. source code) are made available.
- ndsipa_pomu 3y ago
  It's worse than that - there was evidence that Fujitsu could remotely affect the different branch computers with at least one person seeing it happen before their eyes and the Post Office absolutely denied that it was possible or had ever been possible.
beardyw 3y ago
Yes it's an interesting point. At first I was horrified that it is so, but then the alternative is to require proof that a system is reliable, which no one could provide. It would need to be somewhere between, talking about best endeavours or something. It sounds messy however you approach it.
- raesene9 3y ago
  It is very tricky. In some industries you see things like device certification and regular external testing requirements (e.g. weights and measures) but applying that to the very fast moving world of software would be .... tricky, to put it mildly.
  - doubloon 3y ago
    It would not be tricky it would just cost a little money and annoy someone who is incompetent because you would force competency on them. Open source projects by volunteers have better testing than systems like this.

ndsipa_pomu 3y ago

For some computer functions it makes sense, such as audit logs and other basic operations e.g. timestamps. For complicated accounting software, there should definitely be some kind of proof of correctness such as being self consistent and having sufficient auditing and logging systems.

graemep 3y ago
It was actually intended to stop people challenging the reliability of things like speed cameras.
- ndsipa_pomu 3y ago
  Yes, that's an interesting case as cameras are routinely calibrated, but the software is closed source which I think is a mistake. It should be possible to challenge whether a system is operating correctly if you have decent evidence against it e.g. GPS tracking. I do think that modern cars should be fitted with a black box device with the owner having ultimate control over access to it, so they could submit it for evidence to prevent miscarriages of justice.
  - graemep 3y ago
    I think "presumed to be operating normally for certain devices is certain oversight and testing requirements are met" is reasonable. Definitely ought to be open source. I wonder whether the black boxes insurance companies like to push would provide the evidence required?
  - acdha 3y ago
    > I do think that modern cars should be fitted with a black box device with the owner having ultimate control over access to it, so they could submit it for evidence to prevent miscarriages of justice I’m not sure this is workable: if the owner had control, there’d be a cottage industry of people offering to fake evidence against camera tickets. If the owner doesn’t, it’d get requested by police and insurance routinely and since most drivers regularly break local traffic laws that’d have a big negative impact on the owners, which makes me think it’s politically infeasible. Insurance companies might try to mandate that but I imagine they are very careful about changes which could shift customers.
    - ndsipa_pomu 3y ago
      I think those issues could be overcome by using encryption and checksums for the logs with the decryption key held by a third party that would read and release the data on request by the owner or in criminal cases where the owner was a victim. The more likely model is that the insurance companies own and provide the black boxes in return for reduced premiums from the owner. The current situation is that some drivers run their own dashcams and can choose to provide video evidence, but that could be open to abuse if someone very carefully edits the video and no-one spots it. I agree that the drivers shouldn't be forced to self-incriminate - the police should be using their own evidence which could well be from other drivers submitting their own driving data/video.
toyg 3y ago
This is actually one of the few applications of blockchain techs that makes sense - as a write-only store of transactions for auditing purposes, it's potentially very practical.
- ndsipa_pomu 3y ago
  I agree, but wouldn't there be problems with the speed and cost in electricity. Also, would it need to be a single logging blockchain shared by everyone or I would imagine there could be issues with every computer system running its own blockchain - would that make it easier to fake?
  - menzoic 3y ago
    >wouldn't there be problems with the speed and cost in electricity proof of stake blockchains don't have those issues
  - parineum 3y ago
    > I agree, but wouldn't there be problems with the speed and cost in electricity It's either that or it's vulnerable to 50%+1 attacks.

denton-scratch 3y ago

> complex systems are assumed to always be correct ..absent evidence to the contrary. But in the Post Office case, the Post Office had all the evidence, and refused to disclose it. As far as I'm aware, failure to disclose evidence that might help the defence is perverting the course of justice. I don't know why no manager's been charged.

raesene9 3y ago
Yep that's a huge challenge. The other one is that individual defendant's are unlikely to be able to afford the expertise needed to analyze a large complex setup, even if the details of the system (e.g. source code) are made available.
ndsipa_pomu 3y ago
It's worse than that - there was evidence that Fujitsu could remotely affect the different branch computers with at least one person seeing it happen before their eyes and the Post Office absolutely denied that it was possible or had ever been possible.

beardyw 3y ago

Yes it's an interesting point. At first I was horrified that it is so, but then the alternative is to require proof that a system is reliable, which no one could provide. It would need to be somewhere between, talking about best endeavours or something. It sounds messy however you approach it.

raesene9 3y ago
It is very tricky. In some industries you see things like device certification and regular external testing requirements (e.g. weights and measures) but applying that to the very fast moving world of software would be .... tricky, to put it mildly.
- doubloon 3y ago
  It would not be tricky it would just cost a little money and annoy someone who is incompetent because you would force competency on them. Open source projects by volunteers have better testing than systems like this.