4 ms·

The standard in U.S. criminal courts is "beyond a reasonable doubt." All I understood Moxie's original article to be doing was sowing that seed of "reasonable

by smeej 5y ago

The standard in U.S. criminal courts is "beyond a reasonable doubt." All I understood Moxie's original article to be doing was sowing that seed of "reasonable doubt." Is it now reasonable, based on Moxie's article, to doubt that information obtained by a Cellebrite device from a device running Signal is reliable? If I were a juror, I would probably think so. That doesn't at all mean someone couldn't be convicted on the strength of other evidence, but if the primary evidence the prosecution relied on was Cellebrited off a phone running Signal, I'd have some trouble trusting it enough to render a guilty verdict.

tptacek 5y ago
Multiple legal experts have chimed in, all suggesting that this is unlikely to impact cases. People on HN might assume that the existence of a vulnerability strongly implies that it will have been exploited in every case where it is relevant, but that's not how normal people think, and in this case, the normal people are closer to the truth than the nerds are.
- roywiggins 5y ago
  It seems to me that a malicious boobytrapped file did it is really just a subset of I was hacked, which is neither new defense nor (I assume) a particularly successful one.
- KMag 5y ago
  But what about the vulnerability as an indicator of the general quality and reliability of the tool? Casting doubt as to the ability of the tool to generally be accurate and specifically maintain proper chain of custody documentation would seem to be a reasonable legal defense tactic. I would expect a defense lawyer to say something like "The tool so confuses its input that it can mistake message data for its own internal instructions. How certain can we be that it has properly analyzed its inputs and maintained the necessary chain-of-custody metadata, and provided adequate protections against evidence tampering? If a police officer were unable to tell the difference between his or her own thoughts and things he or she had read, we would dismiss him or her as a reliable witness."
  - tptacek 5y ago
    If that argument was going to be meaningful in court, it would have applied just as well to EnCase. Never did, though.
  - mikro2nd 5y ago
    That argument has never had the lightest effect on the (ongoing) use of polygraph testing by law enforcement agencies, nor on the use of polygraph results in obtaining convictions by courts. And yet it is well and truly established that polygraph testing is snake oil.
    - tptacek 5y ago
      Polygraphs are not generally admissible in court. https://www.justice.gov/archives/jm/criminal-resource-manual-262-polygraphs-introduction-trial https://www.justice.gov/archives/jm/criminal-resource-manual... (just one example).
      - mikro2nd 5y ago
        I stand corrected. Are they still so widely used on employees by Three Letter Agencies?
  - salawat 5y ago
    >I would expect a defense lawyer to say something like "The tool so confuses its input that it can mistake message data for its own internal instructions. How certain can we be that it has properly analyzed its inputs and maintained the necessary chain-of-custody metadata, and provided adequate protections against evidence tampering? To be clear, all you accomplish with that statement as a defense attorney is that you didn't get a credible enough expert, as any Computer Scientist should point out that is the fundamental character of the Von Neumann computing machine architecture, the very model of computing that most computers are designed according to, and most programs are written to run against. They would then further expound that software development had developed methods to mitigate this problem, which minimize the llkelihood of such architecture quirks being exploited, and most certainly leading to a state of affairs where any such vulnerability could be identified via a source code audit. This would open the door for the defense to require the prosecution to produce source code for their tool to prove to the court whether the vulnerability exists or not. A good defense would then follow up by asking whether or not there was some way to detect whether there had been a successful exploitation on a device. "That's where things get tricky", the expert should reply, "because if arbitrary code can be run, given enough time, someone could cover their tracks successfully. It is plausible a mistake could be made in terms of the implementer of the exploit missing a timestamp, not properly serializing something, not cleaning out a log that could be then reconciled with something else, but the possibility of a completely clean alteration given enough time and resource was still on the table. The prosecution would then endeavor through chain of custody logs, affidavits, data on the device, possibly comparisons to other cases convince the jury this is all hogwash, and the defense is grasping at straws, and ultimately full of shit, without tipping the defenses hand that if this case is in question, other cases may be. Mind the brilliance in Moxie's actions is not that he'd get someone off the hook, but that he's now forced prosecutors into a position where if they want to rely on Cellebrite data as a lynchpin of their case, they have to open the door to public scrutiny of the implementation. Of course, this will just be mitigated by law enforcement ultimately engaging in parallel construction anyway. Or, Cellebrite updates/audits their software to mitigate the vulnerability, or re-implements it on a non-Von-Neumann computer. Again, not a lawyer, just read some stuff on how to think like one once.
wglb 5y ago
Reasonable doubt matters to a judge and/or a jury. It is quite unclear that such vulnerabilities will raise reasonable doubt. For this to happen, the defense would need to demonstrate that files were actually damaged or modified. There is other forensic software used for desktops and servers that is very widely used in court cases. There are significant and substantial vulnerabilities there: https://www.cvedetails.com/vulnerability-list/vendor_id-3015/product_id-5258/Guidance-Software-Encase.html https://www.cvedetails.com/vulnerability-list/vendor_id-3015... This type of issue is not new with signal.
XorNot 5y ago
If the primary evidence relied off of Cellebrite at all it should be doubted, period. The exploits involved weren't difficult to develop, and it is reasonable to assume are now rapidly spreading as independent implementations to nefarious groups, foreign intelligence etc. if they weren't already there.
nearbuy 5y ago
What type of evidence are you thinking of? If they found something like child sex abuse material, or chat logs with gang members about trafficking drugs, how would the defense attorney convince you Signal might have planted that?
- LorenPechtel 5y ago
  It depends on the nature of the material. What if the booby trap went and downloaded random files of 100k+ size from some dark web site? On the other hand, if it's chat with names in it, it's likely not from the booby trap.
  - nearbuy 5y ago
    You're still going to have a very hard time convincing the jury that Signal booby trapped your phone to download child porn at random from the dark web. Law enforcement doesn't confiscate and use Cellebrite on phones at random. If they have a warrant to search your phone, they already have some reason for suspicion. For example, maybe someone accused the defendant of molesting a child, but the police didn't have hard evidence. They use Cellebrite and find some child porn. The defense argues that Signal might have planted it there with a booby trapped file that downloads stuff at random from the dark web. Do you think the jury will buy that? At that point, why not just say you left your phone unlocked in public and a stranger probably used it to download the child porn?
ReptileMan 5y ago
There are other uses for data outside court admissible evidence with lower standards. Like probable cause for further warrants.

by smeej 5y ago

tptacek 5y ago

Multiple legal experts have chimed in, all suggesting that this is unlikely to impact cases. People on HN might assume that the existence of a vulnerability strongly implies that it will have been exploited in every case where it is relevant, but that's not how normal people think, and in this case, the normal people are closer to the truth than the nerds are.

roywiggins 5y ago
It seems to me that a malicious boobytrapped file did it is really just a subset of I was hacked, which is neither new defense nor (I assume) a particularly successful one.
KMag 5y ago
But what about the vulnerability as an indicator of the general quality and reliability of the tool? Casting doubt as to the ability of the tool to generally be accurate and specifically maintain proper chain of custody documentation would seem to be a reasonable legal defense tactic. I would expect a defense lawyer to say something like "The tool so confuses its input that it can mistake message data for its own internal instructions. How certain can we be that it has properly analyzed its inputs and maintained the necessary chain-of-custody metadata, and provided adequate protections against evidence tampering? If a police officer were unable to tell the difference between his or her own thoughts and things he or she had read, we would dismiss him or her as a reliable witness."
- tptacek 5y ago
  If that argument was going to be meaningful in court, it would have applied just as well to EnCase. Never did, though.
- mikro2nd 5y ago
  That argument has never had the lightest effect on the (ongoing) use of polygraph testing by law enforcement agencies, nor on the use of polygraph results in obtaining convictions by courts. And yet it is well and truly established that polygraph testing is snake oil.
  - tptacek 5y ago
    Polygraphs are not generally admissible in court. https://www.justice.gov/archives/jm/criminal-resource-manual-262-polygraphs-introduction-trial https://www.justice.gov/archives/jm/criminal-resource-manual... (just one example).
    - mikro2nd 5y ago
      I stand corrected. Are they still so widely used on employees by Three Letter Agencies?
- salawat 5y ago
  >I would expect a defense lawyer to say something like "The tool so confuses its input that it can mistake message data for its own internal instructions. How certain can we be that it has properly analyzed its inputs and maintained the necessary chain-of-custody metadata, and provided adequate protections against evidence tampering? To be clear, all you accomplish with that statement as a defense attorney is that you didn't get a credible enough expert, as any Computer Scientist should point out that is the fundamental character of the Von Neumann computing machine architecture, the very model of computing that most computers are designed according to, and most programs are written to run against. They would then further expound that software development had developed methods to mitigate this problem, which minimize the llkelihood of such architecture quirks being exploited, and most certainly leading to a state of affairs where any such vulnerability could be identified via a source code audit. This would open the door for the defense to require the prosecution to produce source code for their tool to prove to the court whether the vulnerability exists or not. A good defense would then follow up by asking whether or not there was some way to detect whether there had been a successful exploitation on a device. "That's where things get tricky", the expert should reply, "because if arbitrary code can be run, given enough time, someone could cover their tracks successfully. It is plausible a mistake could be made in terms of the implementer of the exploit missing a timestamp, not properly serializing something, not cleaning out a log that could be then reconciled with something else, but the possibility of a completely clean alteration given enough time and resource was still on the table. The prosecution would then endeavor through chain of custody logs, affidavits, data on the device, possibly comparisons to other cases convince the jury this is all hogwash, and the defense is grasping at straws, and ultimately full of shit, without tipping the defenses hand that if this case is in question, other cases may be. Mind the brilliance in Moxie's actions is not that he'd get someone off the hook, but that he's now forced prosecutors into a position where if they want to rely on Cellebrite data as a lynchpin of their case, they have to open the door to public scrutiny of the implementation. Of course, this will just be mitigated by law enforcement ultimately engaging in parallel construction anyway. Or, Cellebrite updates/audits their software to mitigate the vulnerability, or re-implements it on a non-Von-Neumann computer. Again, not a lawyer, just read some stuff on how to think like one once.

wglb 5y ago

Reasonable doubt matters to a judge and/or a jury. It is quite unclear that such vulnerabilities will raise reasonable doubt. For this to happen, the defense would need to demonstrate that files were actually damaged or modified. There is other forensic software used for desktops and servers that is very widely used in court cases. There are significant and substantial vulnerabilities there: https://www.cvedetails.com/vulnerability-list/vendor_id-3015/product_id-5258/Guidance-Software-Encase.html https://www.cvedetails.com/vulnerability-list/vendor_id-3015... This type of issue is not new with signal.

XorNot 5y ago

If the primary evidence relied off of Cellebrite at all it should be doubted, period. The exploits involved weren't difficult to develop, and it is reasonable to assume are now rapidly spreading as independent implementations to nefarious groups, foreign intelligence etc. if they weren't already there.

nearbuy 5y ago

What type of evidence are you thinking of? If they found something like child sex abuse material, or chat logs with gang members about trafficking drugs, how would the defense attorney convince you Signal might have planted that?

LorenPechtel 5y ago
It depends on the nature of the material. What if the booby trap went and downloaded random files of 100k+ size from some dark web site? On the other hand, if it's chat with names in it, it's likely not from the booby trap.
- nearbuy 5y ago
  You're still going to have a very hard time convincing the jury that Signal booby trapped your phone to download child porn at random from the dark web. Law enforcement doesn't confiscate and use Cellebrite on phones at random. If they have a warrant to search your phone, they already have some reason for suspicion. For example, maybe someone accused the defendant of molesting a child, but the police didn't have hard evidence. They use Cellebrite and find some child porn. The defense argues that Signal might have planted it there with a booby trapped file that downloads stuff at random from the dark web. Do you think the jury will buy that? At that point, why not just say you left your phone unlocked in public and a stranger probably used it to download the child porn?

ReptileMan 5y ago

There are other uses for data outside court admissible evidence with lower standards. Like probable cause for further warrants.