5 ms·

FBI notifies crime labs of errors used in DNA match calculations since 1999

102 points by uxhacker 11y ago

falcolas 11y ago
Just goes to show that humans can even mess up "perfect" evidence. How many convictions will get overturned by this, I wonder. A factor of 10 is pretty large, if they're talking about "1 in 180" levels of probability.
- throwawayaway 11y ago
  I wonder if innocent men are rotting away in jail that have given up themselves and been given up on. They know they are innocent but I doubt 'the system' will swoop down and fight for them.
  - smeyer 11y ago
    I'm not sure if you are being rhetorical, but in case you aren't, such people most definitely exist.
    - throwawayaway 11y ago
      Rhetorical, hard to prove definitely. I wonder will we see more posthumous exhonorations of people that got the death penalty out of this.
      - contingencies 11y ago
        Just last month there was an exhibition in Paris at the excellent Jeau de Paume gallery featuring a video exploring this phenomenon in the US by artist Taryn Simon, who actually tracked down numerous people who had been falsely imprisoned for her 2003 project The Innocents. More info at https://en.wikipedia.org/wiki/Taryn_Simon https://en.wikipedia.org/wiki/Taryn_Simon and https://books.google.com/books?id=RRoiDl9rJtoC&printsec=frontcover&dq=isbn:9781884167188&hl=en&sa=X&ei=qhZqVe_gD4_GogT12IDgBQ&ved=0CB4Q6AEwAA#v=onepage&q&f=false https://books.google.com/books?id=RRoiDl9rJtoC&printsec=fron...
        throwawayaway 11y ago
        Interesting! I suppose there's a perverse incentive for the system not to track the falsely imprisoned down. They have a right to sue for false imprisonment.
jmnicolas 11y ago
"The public puts so much faith in DNA testing that it makes it especially important to make those the best estimates possible" The public was made to believe DNA testing was absolutely bomb proof, there are even entire TV shows centered around this "fact" (CSI whatever ...)
- veb 11y ago
  What I found weird while reading this article was that they all said the results would be "nominal" and nothing will realistically change regardless of if the DNA tests were actually erroneous. Uhhhh what? If people were sentenced for a crime based on the fact that their DNA matched with the DNA found at the scene of the crime they supposedly committed, then surely there's a massive problem here? Just one case of this would mean an innocent person has been rotting away in prison for the last decade... and it won't just be one case! I'm confused as to why they're not thinking this is a big deal. You're completely correct when you say the public was made to believe DNA testing was completely foolproof. If you're in court, being prosecuted and they prosecution says that the DNA matches yours everyone instantly thinks you are in fact guilty. If you've got a jury they'd all think the same too. I thnk this is a really big deal, but I'm not in the US so I may be missing something (again). :-)
  - tbrake 11y ago
    > If people were sentenced for a crime based on the fact that their DNA matched with the DNA found at the scene of the crime they supposedly committed, then surely there's a massive problem here? I get the impression - though I don't have the data/numbers obviously - that in almost all of those cases the DNA evidence was being used as the cherry on top of the evidence sundae they had already compiled. That's why it won't realistically change much; either very few or none at all are convicted _only_ because their dna happened to be found at a crime scene.
    - deleted 11y ago
      [deleted]
    - bilbo0s 11y ago
veb 11y ago
I know in New Zealand, you've got the right to redo tests with whoever you want (i.e. private labs). So, if the prosecution has said that your DNA was found at the scene or something, you can then get that test done privately (yes it'll cost money) to see if it matches up. You can do this as many times, with as many labs as you need. Is this a normal thing in the US too? I'm assuming it is. If my assumption is correct, then wouldn't another lab doing the same test come to a different conclusion?
- hobs 11y ago
  Do you have to pay for the tests with your own money? If so, doesnt matter, rich people dont need the extra help,and poor people cant afford it.
  - rhizome 11y ago
    Inspired by Anatole France, "In its majestic equality, the law allows the rich and poor alike to hire a private DNA testing lab..."
  - veb 11y ago
    It's not that expensive. About $500 NZD for a basic test. Yes, expensive if you don't have $500 but there's many ways to go about it: ask for invoice, get legal aid to pay, do it through lawyer (who adds to their bill), hope you still have family who loves you or friends. Alternatively, you can make the court do it to rule out any problems. Most of the time the labs come up with the same result but the courts are more than happy to make sure it's about water tight.
bane 11y ago
It kind of makes me think/wonder that in criminal cases perhaps forensic biometrics work should be done by two or more labs (maybe using two different methods) to help cross-validate the results. My understanding is that most of this work happens in a single lab under one test, so contamination or procedural problems go unchecked.
- UnoriginalGuy 11y ago
  Maybe the same lab shouldn't even be testing both samples. Just send the sample from the victim to lab #A and same from the accused to lab #B and compare the digital results. The two samples shouldn't be sent in together, there is absolutely no benefit in that. Plus the lab shouldn't know what they're testing so there can be no accusations of bias/manipulation (e.g. "this person is definitely a child rapist, if you guys find this DNA matches this child rapist goes to jail, if it doesn't' (wink wink) then they go free"). The anonymous digital results can then be shipped to the defence so that they can double-check the degree of similarity with an expert of their own.
  - bane 11y ago
    I see where you're coming from, but I'd be concerned if one of the labs has a broken process somewhere (or contaminates the sample) that it would produce a false negative and a possible match would not be found -- thereby leaving a bad person out in the wild.
    - mattstreet 11y ago
      I'd rather a false negative than a false positive. Too many people already feel they can't call the cops as it is when they truly need help.
jimrandomh 11y ago
This article is about errors in a set of population priors used in software. That could affect the outcome of tests, but it also might not. However, given that the FBI was recently caught having faked an entire field of forensic science (hair analysis), there is little reason to extend them the benefit of doubt. The important question is: are these crime labs routinely processing control samples, and if so, are they misclassifying any of the control samples?
- kiba 11y ago
  There is a difference between purposefully faking an entire field of science and believing in pseudoscience and not being skeptical. The effect is the same, either way.
  - rasz_pl 11y ago
    >and believing in pseudoscience lie detectors are a big thing for FBI
    - meowface 11y ago
      Polygraphs? They know full well polygraphs are not "lie detectors". And a lot of the general population knows that too. They're still useful when recruiting candidates. Obviously their results are not admissible in criminal cases.

102 points by uxhacker 11y ago

Just goes to show that humans can even mess up "perfect" evidence. How many convictions will get overturned by this, I wonder. A factor of 10 is pretty large, if they're talking about "1 in 180" levels of probability.

throwawayaway 11y ago
I wonder if innocent men are rotting away in jail that have given up themselves and been given up on. They know they are innocent but I doubt 'the system' will swoop down and fight for them.
- smeyer 11y ago
  I'm not sure if you are being rhetorical, but in case you aren't, such people most definitely exist.
  - throwawayaway 11y ago
    Rhetorical, hard to prove definitely. I wonder will we see more posthumous exhonorations of people that got the death penalty out of this.
    - contingencies 11y ago
      Just last month there was an exhibition in Paris at the excellent Jeau de Paume gallery featuring a video exploring this phenomenon in the US by artist Taryn Simon, who actually tracked down numerous people who had been falsely imprisoned for her 2003 project The Innocents. More info at https://en.wikipedia.org/wiki/Taryn_Simon https://en.wikipedia.org/wiki/Taryn_Simon and https://books.google.com/books?id=RRoiDl9rJtoC&printsec=frontcover&dq=isbn:9781884167188&hl=en&sa=X&ei=qhZqVe_gD4_GogT12IDgBQ&ved=0CB4Q6AEwAA#v=onepage&q&f=false https://books.google.com/books?id=RRoiDl9rJtoC&printsec=fron...
      - throwawayaway 11y ago
        Interesting! I suppose there's a perverse incentive for the system not to track the falsely imprisoned down. They have a right to sue for false imprisonment.

jmnicolas 11y ago

"The public puts so much faith in DNA testing that it makes it especially important to make those the best estimates possible" The public was made to believe DNA testing was absolutely bomb proof, there are even entire TV shows centered around this "fact" (CSI whatever ...)

veb 11y ago
What I found weird while reading this article was that they all said the results would be "nominal" and nothing will realistically change regardless of if the DNA tests were actually erroneous. Uhhhh what? If people were sentenced for a crime based on the fact that their DNA matched with the DNA found at the scene of the crime they supposedly committed, then surely there's a massive problem here? Just one case of this would mean an innocent person has been rotting away in prison for the last decade... and it won't just be one case! I'm confused as to why they're not thinking this is a big deal. You're completely correct when you say the public was made to believe DNA testing was completely foolproof. If you're in court, being prosecuted and they prosecution says that the DNA matches yours everyone instantly thinks you are in fact guilty. If you've got a jury they'd all think the same too. I thnk this is a really big deal, but I'm not in the US so I may be missing something (again). :-)
- tbrake 11y ago
  > If people were sentenced for a crime based on the fact that their DNA matched with the DNA found at the scene of the crime they supposedly committed, then surely there's a massive problem here? I get the impression - though I don't have the data/numbers obviously - that in almost all of those cases the DNA evidence was being used as the cherry on top of the evidence sundae they had already compiled. That's why it won't realistically change much; either very few or none at all are convicted _only_ because their dna happened to be found at a crime scene.
  - deleted 11y ago
    [deleted]
  - bilbo0s 11y ago

veb 11y ago

I know in New Zealand, you've got the right to redo tests with whoever you want (i.e. private labs). So, if the prosecution has said that your DNA was found at the scene or something, you can then get that test done privately (yes it'll cost money) to see if it matches up. You can do this as many times, with as many labs as you need. Is this a normal thing in the US too? I'm assuming it is. If my assumption is correct, then wouldn't another lab doing the same test come to a different conclusion?

hobs 11y ago
Do you have to pay for the tests with your own money? If so, doesnt matter, rich people dont need the extra help,and poor people cant afford it.
- rhizome 11y ago
  Inspired by Anatole France, "In its majestic equality, the law allows the rich and poor alike to hire a private DNA testing lab..."
- veb 11y ago
  It's not that expensive. About $500 NZD for a basic test. Yes, expensive if you don't have $500 but there's many ways to go about it: ask for invoice, get legal aid to pay, do it through lawyer (who adds to their bill), hope you still have family who loves you or friends. Alternatively, you can make the court do it to rule out any problems. Most of the time the labs come up with the same result but the courts are more than happy to make sure it's about water tight.

bane 11y ago

It kind of makes me think/wonder that in criminal cases perhaps forensic biometrics work should be done by two or more labs (maybe using two different methods) to help cross-validate the results. My understanding is that most of this work happens in a single lab under one test, so contamination or procedural problems go unchecked.

UnoriginalGuy 11y ago
Maybe the same lab shouldn't even be testing both samples. Just send the sample from the victim to lab #A and same from the accused to lab #B and compare the digital results. The two samples shouldn't be sent in together, there is absolutely no benefit in that. Plus the lab shouldn't know what they're testing so there can be no accusations of bias/manipulation (e.g. "this person is definitely a child rapist, if you guys find this DNA matches this child rapist goes to jail, if it doesn't' (wink wink) then they go free"). The anonymous digital results can then be shipped to the defence so that they can double-check the degree of similarity with an expert of their own.
- bane 11y ago
  I see where you're coming from, but I'd be concerned if one of the labs has a broken process somewhere (or contaminates the sample) that it would produce a false negative and a possible match would not be found -- thereby leaving a bad person out in the wild.
  - mattstreet 11y ago
    I'd rather a false negative than a false positive. Too many people already feel they can't call the cops as it is when they truly need help.

jimrandomh 11y ago

This article is about errors in a set of population priors used in software. That could affect the outcome of tests, but it also might not. However, given that the FBI was recently caught having faked an entire field of forensic science (hair analysis), there is little reason to extend them the benefit of doubt. The important question is: are these crime labs routinely processing control samples, and if so, are they misclassifying any of the control samples?

kiba 11y ago
There is a difference between purposefully faking an entire field of science and believing in pseudoscience and not being skeptical. The effect is the same, either way.
- rasz_pl 11y ago
  >and believing in pseudoscience lie detectors are a big thing for FBI
  - meowface 11y ago
    Polygraphs? They know full well polygraphs are not "lie detectors". And a lot of the general population knows that too. They're still useful when recruiting candidates. Obviously their results are not admissible in criminal cases.