8 ms·

The Dangers of DNA Testing

185 points by seibelj 8y ago

anoncoward111 8y ago
And justice for all. Technology can be horrifically negative when in the wrong hands.
- yters 8y ago
  How IBM got it's start...
emodendroket 8y ago
Since Sessions ended an inquiry into obvious junk science being used in courtrooms (which itself seemed to be moving pretty slowly given the scale of the problem), my prediction is that nobody will do anything whatsoever about this finding.
- amelius 8y ago
  Every defending lawyer will be waving this news article over their head ...
  - draw_down 8y ago
    OK, but then what?
  - MBCook 8y ago
    Does it matter? If it’s accepted science then courts are loathe to just start throwing it out even when there is very good evidence it’s junk.
  - emodendroket 8y ago
    If you check out the article, the authors of the study inserted a disclaimer that will make this very difficult.
- mschuster91 8y ago
  > Since Sessions ended an inquiry into obvious junk science being used in courtrooms (I'm not from the US!) Is this the stuff like "arson detectives" working with completely unsubstantiated theories, often enough sending people to jail even if they were innocent?
  - plopz 8y ago
    I think its things like hair analysis, bite mark matching even fingerprint matching.
    - heyyyouu 8y ago
      Actually it's turning out that arson detectives are on the level of hair and bite mark analysts -- completely and utterly made up.
    - maxxxxx 8y ago
      If you have enough money to pay for experts to shoot this nonsense down you are fine but if you don't have the money you are in serious trouble. Some years ago I was part of a jury in a trial about assault with a deadly weapon . The prosecution had an expert witness who talked about lighting conditions during sunset. I do a lot of photography so I know about little about this topic myself. I thought to myself "WTF is this guy talking about? This is plain wrong". But as jury member you can speak up only during deliberation and the defendant was a poor guy whose lawyer didn't challenge this nonsense. Thank God the case was so flimsy that we voted "Not guilty" anyway but it's really easy to get convicted based on junk science. Unless you have the money to challenge that.
      - DaiPlusPlus 8y ago
        It's a travesty that juries don't have a voice in a trial. It benefits no-one.
        emodendroket 8y ago
        That's more like a lay judge system.
dnafan123 8y ago
Link to open access publication: https://www.fsigenetics.com/article/S1872-4973(18)30248-5/fulltext https://www.fsigenetics.com/article/S1872-4973(18)30248-5/fu... Link to more detailed article: https://www.forensicmag.com/news/2018/08/nist-publishes-landmark-mix13-dna-study https://www.forensicmag.com/news/2018/08/nist-publishes-land...
- deleted 8y ago
  [deleted]
- AlexCoventry 8y ago
  I can't understunderstand why the OP article didn't at least cite the paper. I guess the quote works as a kind of content-based addressing, though.
vedtopkar 8y ago
It's shocking to me that something so critical to the criminal justice system is not more tightly regulated nor monitored continuously for sensitivity/specificity.
- MBCook 8y ago
  Given courts have ruled drug dogs don’t need to be tested for accuracy or to make sure they’re not influenced by their handlers this isn’t very surprising. Seems like the more I learn about the system the more of a scam it seems. Speaking of which Serial season 3 just started which covers this kind of stuff, more in the small than a grand ‘Thing X is the problem we want solved that will make things much better’.
  - ptaipale 8y ago
    I don't quite follow. Why would drug dogs need to be tested for accuracy? Is the indication of a dog accepted somewhere as evidence? I have been assuming that dogs give false positives every now and then, and there's a search for some actual evidence based on that indication, which everyone knows can be a false positive.
    - vkou 8y ago
      When the dog's accuracy can't be tested, then a search based on the dog giving an indication of "There are drugs here" is no different then a search based on a magic 8-ball giving an indication of "There are drugs here". Every time this technique leads to a search that is used to secure a conviction, it erodes your rights, as a non-criminal, against arbitrary search and seizure.
      - ptaipale 8y ago
        There's a difference in the way people see this here in Europe, because here a "probable cause" is not needed for searching for drugs. Particularly at the border controls, everyone is subject to searches. (Same applies for e.g. DUI testing on the roads; the police has the right to take alcohol test of any driver, without anyone looking like they're driving badly.) Any arbitrary seizure, if only based on dog indication, is of course a horrible miscarriage of justice.
frgtpsswrdlame 8y ago
Every age has required a 'noble lie' in regards to catching criminals - that actually this time, we really can do it. Take this result, take the shakiness of forensic fire patterns, the shakiness of fingerprint matching, the shakiness of bite marks and confessions and more and it becomes apparent that our time isn't so different from those in the past.
cheeko1234 8y ago
If we can't trust witness testimonies, lie-detectors, and now DNA results, what are we supposed to use?
- Krasnol 8y ago
  Skilled detective work?
- blacksmith_tb 8y ago
  Pervasive surveillance, clearly. We didn't want to, but we had to, nothing else was reliable...
- mattnewton 8y ago
  The problem is when you use those in isolation from other data points. You are supposed to doubt each individual piece of evidence, but put enough of them together to overcome the doubt.
- MysticFear 8y ago
  Probably now or very soon AI tech can convincingly merge you into a video as well. Justice system is going to get very interesting to say the least in the future.
- romwell 8y ago
  First, please don't mix in lie detectors here. They are plain quack science, and an "interrogation technique" (read: intimidation method) at best. Second, we have always known witness testimonies to be unreliable. That's why perjury is a crime! We wouldn't need it to be a crime if nobody was giving false testimonies. And that's assuming people don't make mistakes, which they do. We still use testimonies. A court case, after all, is all about probabilities. Very often we just don't know the truth 100%. It's just that compounding evidence, ideally, would convince that it is very unlikely that a particular scenario didn't happen. E.g.: someone could have mistaken another person for you when they said they saw you hitting the old pawnbroker lady with an axe, BUT given that you were caught with an axe AND blood was dripping from it AND that blood matched the old lady's AND you wrote an article about the merits of butchering old ladies in the local newspaper - ALL those things could have been mere coincidences and mistakes, HOWEVER they establish that it's highly unlikely that you didn't commit the crime. Same with DNA evidence. We just need to re-adjust our expectations of how foolproof it is for it to work effectively in a court system. As long as we know that the labs are not 100% reliable, we'll be fine. And that's what the authors are trying to do here. The next step would be punishments for quack/pseudo science being used as "expert" testimony, as well as actual experts being reckless in their work (as is the case with the DNA labs in the article) when that work is used to accuse someone of a crime. The punishment doesn't need to be jail-time; even banning the person/lab from providing evidence again would be a good first step. But that's my view on how to solve this problem, and a subject of another discussion.
exhilaration 8y ago
I know this doesn't help the folks in prison without any resources, but during a trial, can't a defendant ask for any DNA test to be rerun by a private lab at their expense?
- dnbgfher 8y ago
  Even if that is an option, and the results come back favorable to the defendant, they still don't have a great situation. Now there will be two experts coming in and saying different things to the jury. The details that might explain those differences are likely beyond what can be adequately explained to a jury during a trial. But the defendant themselves paid one of the experts, so...
  - oldandtired 8y ago
    Decades ago, I served on a jury where we had an expert witness for defence and an expert witness for prosecution. We, the jury, made a decision to ignore both expert witnesses as they each had a different opinion, please note the word "opinion". I do not know with what regard, under US jurisdictions, the evidence provided by "expert" witnesses is treated, but at the time, we were told by the presiding judge to treat it as opinion by an "expert" and not as direct evidence.
bjt2n3904 8y ago
This is one of my favorite pet peeves: a smart person comes out wearing a lab coat[1] and holding a clipboard. They make a claim, and one of two things happen. Firstly, news reporters with little understanding of the subject matter try to explain it to the masses, and the results are disastrous. (Not to mention, the drive for academia to publish papers that prove the hypothesis.) Little needs to be said here. The second is that I frequently see my peers latch onto any PDF they can download that supports their position. "Look, look! The data shows I'm right! And this man with a PhD supports my conclusion! (Of course I haven't read the report, but that's not important. What is important is my pet issue!)" Prosecutors using sloppy lab work to get convictions and computer algorithms deciding bail bonds are just end results. We need to put the brakes on the empiricism train. It's useful, but only to a certain point. Reason eventually needs to take over. If your argument lives and dies by the next PDF or Machine Learning model, you need to find a new argument, or bolster your argument with reason. 1 - https://xkcd.com/699 https://xkcd.com/699 (Meta argument: I'm reading a report about a paper someone published, and latching onto the report because it supports my conclusion that people will believe anything a "scientist" says! I'm victim to my own problems!)
- 394549 8y ago
  > Firstly, news reporters with little understanding of the subject matter try to explain it to the masses, and the results are disastrous. Do you have a better alternative?
  - godelski 8y ago
    That's why people like Carl Sagan, Bill Nye, and Neil deGrasse Tyson exist. Science communicators. They are the better alternative.
    - heyyyouu 8y ago
      Who exactly do you think communicates what they say to the masses? Reporters. Without them it would never reach a wide audience.
spsrich 8y ago
"It is uncomfortable to read the study’s authors praising labs for their careful work when they get things right, but offering sophomoric excuses for them when they get things wrong." But not surprising.... they want to continue to work in the field, after all !
lucio 8y ago
Isn't clickbait? "The Dangers of testing DNA mixtures" should be the title
heyyyouu 8y ago
Some articles are so important that they don't belong behind paywalls. I believe this is one of them. The NYT's ombudsman should help develop a policy to determine when that should occur.
- Pinckney 8y ago
  Isn't that why we have scihub?
heyyyouu 8y ago
There are many, many examples of DNA testing being unreliable because of HOW it is done (for example: https://www.theatlantic.com/magazine/archive/2016/06/a-reasonable-doubt/480747/ https://www.theatlantic.com/magazine/archive/2016/06/a-reaso... ) and yet the general public still sees it as infallible. It's extremely important for these kinds of studies to get more publicity, for the courts to start to understand the limitations, and most importantly for the labs that do the tests to start to be held accountable. The standards from a scientific standpoint are truly deplorable.
YeGoblynQueenne 8y ago
>> The good news is that there are methods to reanalyze old DNA mixture data using computer programs that can help analysts correct errors, without any new lab testing. In fact, one lesson from the study is that while only seven of the 108 labs in the study properly excluded the innocent profile, one of them used such a program (TrueAllele by Cybergenetics). >> (...) >> In fact, we have shown that this is possible. Working with Cybergenetics analysts and Innocence Network organizations in four states, our Boise State University laboratory has re-examined a few select cases and already persuaded courts to overturn a conviction in New Mexico, two in Indiana and two in Montana. We have also helped identify a new suspect in a 23-year-old murder. This leaves a bad taste. Have I just read an advert for a proprietary piece of software, masquerading as an article?
sundvor 8y ago
The article gets the timeline of Jackson's case completely wrong: https://www.innocenceproject.org/cases/dwayne-jackson/ https://www.innocenceproject.org/cases/dwayne-jackson/ This more detailed account states the exoneration happened years after the plea bargain release; the eff up discovery did not cause him to be released. So the real impact on this poor man's life was thus significantly higher. Imagine trying to get anything worthwhile happen in life with a criminal conviction...

185 points by seibelj 8y ago

anoncoward111 8y ago

And justice for all. Technology can be horrifically negative when in the wrong hands.

yters 8y ago
How IBM got it's start...

emodendroket 8y ago

Since Sessions ended an inquiry into obvious junk science being used in courtrooms (which itself seemed to be moving pretty slowly given the scale of the problem), my prediction is that nobody will do anything whatsoever about this finding.

amelius 8y ago
Every defending lawyer will be waving this news article over their head ...
- draw_down 8y ago
  OK, but then what?
- MBCook 8y ago
  Does it matter? If it’s accepted science then courts are loathe to just start throwing it out even when there is very good evidence it’s junk.
- emodendroket 8y ago
  If you check out the article, the authors of the study inserted a disclaimer that will make this very difficult.
mschuster91 8y ago
> Since Sessions ended an inquiry into obvious junk science being used in courtrooms (I'm not from the US!) Is this the stuff like "arson detectives" working with completely unsubstantiated theories, often enough sending people to jail even if they were innocent?
- plopz 8y ago
  I think its things like hair analysis, bite mark matching even fingerprint matching.
  - heyyyouu 8y ago
    Actually it's turning out that arson detectives are on the level of hair and bite mark analysts -- completely and utterly made up.
  - maxxxxx 8y ago
    If you have enough money to pay for experts to shoot this nonsense down you are fine but if you don't have the money you are in serious trouble. Some years ago I was part of a jury in a trial about assault with a deadly weapon . The prosecution had an expert witness who talked about lighting conditions during sunset. I do a lot of photography so I know about little about this topic myself. I thought to myself "WTF is this guy talking about? This is plain wrong". But as jury member you can speak up only during deliberation and the defendant was a poor guy whose lawyer didn't challenge this nonsense. Thank God the case was so flimsy that we voted "Not guilty" anyway but it's really easy to get convicted based on junk science. Unless you have the money to challenge that.
    - DaiPlusPlus 8y ago
      It's a travesty that juries don't have a voice in a trial. It benefits no-one.
      - emodendroket 8y ago
        That's more like a lay judge system.

dnafan123 8y ago

Link to open access publication: https://www.fsigenetics.com/article/S1872-4973(18)30248-5/fulltext https://www.fsigenetics.com/article/S1872-4973(18)30248-5/fu... Link to more detailed article: https://www.forensicmag.com/news/2018/08/nist-publishes-landmark-mix13-dna-study https://www.forensicmag.com/news/2018/08/nist-publishes-land...

deleted 8y ago
[deleted]
AlexCoventry 8y ago
I can't understunderstand why the OP article didn't at least cite the paper. I guess the quote works as a kind of content-based addressing, though.

vedtopkar 8y ago

It's shocking to me that something so critical to the criminal justice system is not more tightly regulated nor monitored continuously for sensitivity/specificity.

MBCook 8y ago
Given courts have ruled drug dogs don’t need to be tested for accuracy or to make sure they’re not influenced by their handlers this isn’t very surprising. Seems like the more I learn about the system the more of a scam it seems. Speaking of which Serial season 3 just started which covers this kind of stuff, more in the small than a grand ‘Thing X is the problem we want solved that will make things much better’.
- ptaipale 8y ago
  I don't quite follow. Why would drug dogs need to be tested for accuracy? Is the indication of a dog accepted somewhere as evidence? I have been assuming that dogs give false positives every now and then, and there's a search for some actual evidence based on that indication, which everyone knows can be a false positive.
  - vkou 8y ago
    When the dog's accuracy can't be tested, then a search based on the dog giving an indication of "There are drugs here" is no different then a search based on a magic 8-ball giving an indication of "There are drugs here". Every time this technique leads to a search that is used to secure a conviction, it erodes your rights, as a non-criminal, against arbitrary search and seizure.
    - ptaipale 8y ago
      There's a difference in the way people see this here in Europe, because here a "probable cause" is not needed for searching for drugs. Particularly at the border controls, everyone is subject to searches. (Same applies for e.g. DUI testing on the roads; the police has the right to take alcohol test of any driver, without anyone looking like they're driving badly.) Any arbitrary seizure, if only based on dog indication, is of course a horrible miscarriage of justice.

frgtpsswrdlame 8y ago

Every age has required a 'noble lie' in regards to catching criminals - that actually this time, we really can do it. Take this result, take the shakiness of forensic fire patterns, the shakiness of fingerprint matching, the shakiness of bite marks and confessions and more and it becomes apparent that our time isn't so different from those in the past.

cheeko1234 8y ago

If we can't trust witness testimonies, lie-detectors, and now DNA results, what are we supposed to use?

Krasnol 8y ago
Skilled detective work?
blacksmith_tb 8y ago
Pervasive surveillance, clearly. We didn't want to, but we had to, nothing else was reliable...
mattnewton 8y ago
The problem is when you use those in isolation from other data points. You are supposed to doubt each individual piece of evidence, but put enough of them together to overcome the doubt.
MysticFear 8y ago
Probably now or very soon AI tech can convincingly merge you into a video as well. Justice system is going to get very interesting to say the least in the future.
romwell 8y ago
First, please don't mix in lie detectors here. They are plain quack science, and an "interrogation technique" (read: intimidation method) at best. Second, we have always known witness testimonies to be unreliable. That's why perjury is a crime! We wouldn't need it to be a crime if nobody was giving false testimonies. And that's assuming people don't make mistakes, which they do. We still use testimonies. A court case, after all, is all about probabilities. Very often we just don't know the truth 100%. It's just that compounding evidence, ideally, would convince that it is very unlikely that a particular scenario didn't happen. E.g.: someone could have mistaken another person for you when they said they saw you hitting the old pawnbroker lady with an axe, BUT given that you were caught with an axe AND blood was dripping from it AND that blood matched the old lady's AND you wrote an article about the merits of butchering old ladies in the local newspaper - ALL those things could have been mere coincidences and mistakes, HOWEVER they establish that it's highly unlikely that you didn't commit the crime. Same with DNA evidence. We just need to re-adjust our expectations of how foolproof it is for it to work effectively in a court system. As long as we know that the labs are not 100% reliable, we'll be fine. And that's what the authors are trying to do here. The next step would be punishments for quack/pseudo science being used as "expert" testimony, as well as actual experts being reckless in their work (as is the case with the DNA labs in the article) when that work is used to accuse someone of a crime. The punishment doesn't need to be jail-time; even banning the person/lab from providing evidence again would be a good first step. But that's my view on how to solve this problem, and a subject of another discussion.

exhilaration 8y ago

I know this doesn't help the folks in prison without any resources, but during a trial, can't a defendant ask for any DNA test to be rerun by a private lab at their expense?

dnbgfher 8y ago
Even if that is an option, and the results come back favorable to the defendant, they still don't have a great situation. Now there will be two experts coming in and saying different things to the jury. The details that might explain those differences are likely beyond what can be adequately explained to a jury during a trial. But the defendant themselves paid one of the experts, so...
- oldandtired 8y ago
  Decades ago, I served on a jury where we had an expert witness for defence and an expert witness for prosecution. We, the jury, made a decision to ignore both expert witnesses as they each had a different opinion, please note the word "opinion". I do not know with what regard, under US jurisdictions, the evidence provided by "expert" witnesses is treated, but at the time, we were told by the presiding judge to treat it as opinion by an "expert" and not as direct evidence.

bjt2n3904 8y ago

This is one of my favorite pet peeves: a smart person comes out wearing a lab coat[1] and holding a clipboard. They make a claim, and one of two things happen. Firstly, news reporters with little understanding of the subject matter try to explain it to the masses, and the results are disastrous. (Not to mention, the drive for academia to publish papers that prove the hypothesis.) Little needs to be said here. The second is that I frequently see my peers latch onto any PDF they can download that supports their position. "Look, look! The data shows I'm right! And this man with a PhD supports my conclusion! (Of course I haven't read the report, but that's not important. What is important is my pet issue!)" Prosecutors using sloppy lab work to get convictions and computer algorithms deciding bail bonds are just end results. We need to put the brakes on the empiricism train. It's useful, but only to a certain point. Reason eventually needs to take over. If your argument lives and dies by the next PDF or Machine Learning model, you need to find a new argument, or bolster your argument with reason. 1 - https://xkcd.com/699 https://xkcd.com/699 (Meta argument: I'm reading a report about a paper someone published, and latching onto the report because it supports my conclusion that people will believe anything a "scientist" says! I'm victim to my own problems!)

394549 8y ago
> Firstly, news reporters with little understanding of the subject matter try to explain it to the masses, and the results are disastrous. Do you have a better alternative?
- godelski 8y ago
  That's why people like Carl Sagan, Bill Nye, and Neil deGrasse Tyson exist. Science communicators. They are the better alternative.
  - heyyyouu 8y ago
    Who exactly do you think communicates what they say to the masses? Reporters. Without them it would never reach a wide audience.

spsrich 8y ago

"It is uncomfortable to read the study’s authors praising labs for their careful work when they get things right, but offering sophomoric excuses for them when they get things wrong." But not surprising.... they want to continue to work in the field, after all !

lucio 8y ago

Isn't clickbait? "The Dangers of testing DNA mixtures" should be the title

heyyyouu 8y ago

Some articles are so important that they don't belong behind paywalls. I believe this is one of them. The NYT's ombudsman should help develop a policy to determine when that should occur.

Pinckney 8y ago
Isn't that why we have scihub?

heyyyouu 8y ago

There are many, many examples of DNA testing being unreliable because of HOW it is done (for example: https://www.theatlantic.com/magazine/archive/2016/06/a-reasonable-doubt/480747/ https://www.theatlantic.com/magazine/archive/2016/06/a-reaso... ) and yet the general public still sees it as infallible. It's extremely important for these kinds of studies to get more publicity, for the courts to start to understand the limitations, and most importantly for the labs that do the tests to start to be held accountable. The standards from a scientific standpoint are truly deplorable.

YeGoblynQueenne 8y ago

>> The good news is that there are methods to reanalyze old DNA mixture data using computer programs that can help analysts correct errors, without any new lab testing. In fact, one lesson from the study is that while only seven of the 108 labs in the study properly excluded the innocent profile, one of them used such a program (TrueAllele by Cybergenetics). >> (...) >> In fact, we have shown that this is possible. Working with Cybergenetics analysts and Innocence Network organizations in four states, our Boise State University laboratory has re-examined a few select cases and already persuaded courts to overturn a conviction in New Mexico, two in Indiana and two in Montana. We have also helped identify a new suspect in a 23-year-old murder. This leaves a bad taste. Have I just read an advert for a proprietary piece of software, masquerading as an article?

sundvor 8y ago

The article gets the timeline of Jackson's case completely wrong: https://www.innocenceproject.org/cases/dwayne-jackson/ https://www.innocenceproject.org/cases/dwayne-jackson/ This more detailed account states the exoneration happened years after the plea bargain release; the eff up discovery did not cause him to be released. So the real impact on this poor man's life was thus significantly higher. Imagine trying to get anything worthwhile happen in life with a criminal conviction...