3 ms·
These companies are disgusting. They peddle black box "models," that essentially ride the good reputation of DNA as infallible (which it is most certainly not)
by rland 6y ago
These companies are disgusting. They peddle black box "models," that essentially ride the good reputation of DNA as infallible (which it is most certainly not) to get convictions on dubious or no evidence.
The way it works is that if there is a sample from a crime scene, they send it to these guys and they analyze it with their software to detect "statistical" DNA from the sample. These samples are the ones that are too crappy to actually make a definitive match -- they are a statistical match. So you say "I think Jim, Bob, and Alice were on scene," and it says "10% likelihood Jim DNA, 5% likelihood Bob DNA, 45% Alice DNA." Do you think it ever says "99% no DNA" in the sample?
It's basically Theranos, except instead of wasting $50 on a shitty blood test you get life in prison.
Ostensibly, it searches the entire DNA database for matches, and only returns a positive result if there's a positive match.
But it's a statistical model, using inputs that are crappy at best (because if it was an actual DNA match, they would send it off to in house forensics who would be able to do PCR...) and which includes inputs from circumstantial evidence as priors. Like we believe Alice was at the scene therefore if you find any statistical likelihood that this is Alice's DNA boost that.
They often run the model multiple times in a row, and use the result that the DA likes the most to enter into evidence. This is because the models return different results each time -- of course they'd say, iTs StAtiStIcaL, so they can do that...
And the source code is completely impenetrable. They argue that it's a "trade secret" that jeopardizes their ability to make future profits, so it cannot be open-sourced. These guys could have a model that just says "what percentage should the thing read, Señor D.A.?" The entire product is a sham. And because it's 170k LOC, no one has the time or the qualifications (Judges/Attorneys reading source code? Yeah right!) to review it, even if it were open source.
Pure quackery, and often times, decades-long sentences or life in prison for the defendant. These companies are pure filth worthy of the lowest revulsion. It's a wonder any convictions happen at all because of this stuff, but jurors have very inaccurate conceptions of forensic science, thanks to shit like CSI, Law and Order, etc. These companies happily play into that image and people really believe this stuff works.
- deleted 6y ago[deleted]
- jrm5100 6y agoIssues with source code access aside, your description is mostly wrong. These programs take a DNA profile as input- it's just that the DNA profile is mixed (i.e. from multiple people). It reporting no DNA would be nonsensical. Figuring out exactly how many people are in a mixture isn't quite nailed down statistically (last I knew of), but it's usually pretty clear for up to 4 or so people. Yes, you could run different models and get different probabilities. For example, the likelihood that the sample is a mixture of the suspect, the victim, and some unknown person vs victim and two unknown people compared to saying the victim isn't in the sample. However, the specification of those models is part of the trial process. And the output probabilities (at least when being used to determine guilt) are usually quite high, orders of magnitude higher than 90% or even 99.99%. My point is that the science behind these calculations is well developed- validation studies get published all the time. Whether or not the specific software has errors (or isn't coded exactly as modeled) is an entirely different matter, but it still isn't all that likely. All of these cases rely on expert witnesses anyway- it's not the prosecutor pressing some buttons and printing a report. There is far more concerning quackery that gets used in forensics- bite marks, hair matching, etc.