4 ms·

No, the procedures and theory behind, e.g., bite mark analysis or fire analysis, are total junk.

giantg2 5y ago
Do you have some links for this? My understanding is the interpretation part can be BS, but stuff like finding and identifying potential ignition or accelerant locations and types should be more settled.
- emodendroket 5y ago
  https://theintercept.com/2019/05/05/forensic-evidence-aafs-junk-science/ https://theintercept.com/2019/05/05/forensic-evidence-aafs-j... https://www.rollingstone.com/politics/politics-features/jeff-sessions-is-keeping-junk-science-in-americas-courts-194492/ https://www.rollingstone.com/politics/politics-features/jeff...
  - giantg2 5y ago
    That doesn't exactly support your claims for most of forensics. If you read/skim the NAS report these articles are based on, you will see that much of the issue is not with the underlying science, but rather in the "interpretation" part. Yes, the odontology seems to be unsupported. But even the fire analysis that you list as an example is said to have a scientific basis like chemistry, but that the interpretation part has issues.
    - tooltower 5y ago
      While my stance on forensics is not as negative as the parent comment, I don't see how you are drawing such a sharp distinction between the "science" and the "interpretation". They are not separate things. When a scientist proposes a new test, they include measurement techniques as well as a method of interpretation. If a medical test is used in a way that is frequently prone to misinterpretation, it would be fair to call that entire test useless.
      - hervature 5y ago
        I disagree. The canonical example here are p-values. Completely valid and accepted interpretation which will forever be abused. In the medical realm, we see the same things with false-positive and false-negative rates causing massive confusion. An even more recent example is COVID vaccine efficacy. I would take the bet that 90% of the general population could not give the accurate definition of what the definition of efficacy is.
        hellbannedguy 5y ago
        Just curious about the downvotes. 1. Was it he said statistics lie sometimes? 2. Or, questioned 90% efficacy of some Covid vaccines?
        hervature 5y ago
        I think you are getting downvoted because (hopefully) people recognize I didn't question their efficacy. Just that 90% of people know what that means.
      - giantg2 5y ago
        "If a medical test is used in a way that is frequently prone to misinterpretation, it would be fair to call that entire test useless." And yet there are many examples of this with malpractice, unnecessary deaths, etc. Lyme tests are notorious for false negatives depending on the lab you use. It's also a widely held belief that you want tests done in the middle of the week instead of on a Friday or weekend because some techs just want to get out of there and are more prone to mistakes. The reason I'm drawing a hard line is because of fact vs opinion. This same line gets drawn in court. They will ask experts if it's a fact or an opinion that they testified about (usually to attack a witness). Autopsies are full of facts, and then there are opinions on what those facts mean. If it's an opinion, it's possible to counter it with another professional opinion. The only sort of opinion they should be sharing is included, excluded, or undetermined. Hair is not a "match", but it could include or exclude someone and maybe mathematical probabilities.
    - emodendroket 5y ago
      It may very well be the case that a version of fire analysis that is scientifically rigorous can be done, but this would have nothing to do with how it is typically practiced.
      - colejohnson66 5y ago
        Which is what @giantg2 said: > The science is generally sound (procedures and theory). Usually it's the prosecutors that present the forensics in a way that misrepresents what the results actually mean. https://news.ycombinator.com/item?id=28265884 https://news.ycombinator.com/item?id=28265884
        emodendroket 5y ago
        That’s not the same claim. The practitioners who come up with the results are not operating in a scientifically sound or rigorous way. One documentary I watched about a wrongful murder conviction (a man was found to have burned down his house to kill his entire family) had a discussion of how the fire investigators would talk about “feeling the beast” and other nonsense. It is very flattering to describe this as in any way “evidence-based.”
    - shkkmo 5y ago
      If you are using forensics in a trial, the validity of the "interpretation" part is key to it's use. The reliability of these interpretations has been historically hugely overstated. Given this history it is absolutely reasonable to reject most forensic evidence unless there actual good science supporting the reliability of expert interpretations.
      - TheOtherHobbes 5y ago
        The usual example is DNA evidence. The two most obvious fallacies are actually called The Prosecutor Fallacy and The Defendant Fallacy because it's so easy for either side to mis-sell what the evidence really means. It's basically impossible for someone who isn't pretty handy with Bayes to work out the significance for themselves. The average jury member has no chance.

Do you have some links for this? My understanding is the interpretation part can be BS, but stuff like finding and identifying potential ignition or accelerant locations and types should be more settled.

emodendroket 5y ago
https://theintercept.com/2019/05/05/forensic-evidence-aafs-junk-science/ https://theintercept.com/2019/05/05/forensic-evidence-aafs-j... https://www.rollingstone.com/politics/politics-features/jeff-sessions-is-keeping-junk-science-in-americas-courts-194492/ https://www.rollingstone.com/politics/politics-features/jeff...
- giantg2 5y ago
  That doesn't exactly support your claims for most of forensics. If you read/skim the NAS report these articles are based on, you will see that much of the issue is not with the underlying science, but rather in the "interpretation" part. Yes, the odontology seems to be unsupported. But even the fire analysis that you list as an example is said to have a scientific basis like chemistry, but that the interpretation part has issues.
  - tooltower 5y ago
    While my stance on forensics is not as negative as the parent comment, I don't see how you are drawing such a sharp distinction between the "science" and the "interpretation". They are not separate things. When a scientist proposes a new test, they include measurement techniques as well as a method of interpretation. If a medical test is used in a way that is frequently prone to misinterpretation, it would be fair to call that entire test useless.
    - hervature 5y ago
      I disagree. The canonical example here are p-values. Completely valid and accepted interpretation which will forever be abused. In the medical realm, we see the same things with false-positive and false-negative rates causing massive confusion. An even more recent example is COVID vaccine efficacy. I would take the bet that 90% of the general population could not give the accurate definition of what the definition of efficacy is.
      - hellbannedguy 5y ago
        Just curious about the downvotes. 1. Was it he said statistics lie sometimes? 2. Or, questioned 90% efficacy of some Covid vaccines?
        hervature 5y ago
        I think you are getting downvoted because (hopefully) people recognize I didn't question their efficacy. Just that 90% of people know what that means.
    - giantg2 5y ago
      "If a medical test is used in a way that is frequently prone to misinterpretation, it would be fair to call that entire test useless." And yet there are many examples of this with malpractice, unnecessary deaths, etc. Lyme tests are notorious for false negatives depending on the lab you use. It's also a widely held belief that you want tests done in the middle of the week instead of on a Friday or weekend because some techs just want to get out of there and are more prone to mistakes. The reason I'm drawing a hard line is because of fact vs opinion. This same line gets drawn in court. They will ask experts if it's a fact or an opinion that they testified about (usually to attack a witness). Autopsies are full of facts, and then there are opinions on what those facts mean. If it's an opinion, it's possible to counter it with another professional opinion. The only sort of opinion they should be sharing is included, excluded, or undetermined. Hair is not a "match", but it could include or exclude someone and maybe mathematical probabilities.
  - emodendroket 5y ago
    It may very well be the case that a version of fire analysis that is scientifically rigorous can be done, but this would have nothing to do with how it is typically practiced.
    - colejohnson66 5y ago
      Which is what @giantg2 said: > The science is generally sound (procedures and theory). Usually it's the prosecutors that present the forensics in a way that misrepresents what the results actually mean. https://news.ycombinator.com/item?id=28265884 https://news.ycombinator.com/item?id=28265884
      - emodendroket 5y ago
        That’s not the same claim. The practitioners who come up with the results are not operating in a scientifically sound or rigorous way. One documentary I watched about a wrongful murder conviction (a man was found to have burned down his house to kill his entire family) had a discussion of how the fire investigators would talk about “feeling the beast” and other nonsense. It is very flattering to describe this as in any way “evidence-based.”
  - shkkmo 5y ago
    If you are using forensics in a trial, the validity of the "interpretation" part is key to it's use. The reliability of these interpretations has been historically hugely overstated. Given this history it is absolutely reasonable to reject most forensic evidence unless there actual good science supporting the reliability of expert interpretations.
    - TheOtherHobbes 5y ago
      The usual example is DNA evidence. The two most obvious fallacies are actually called The Prosecutor Fallacy and The Defendant Fallacy because it's so easy for either side to mis-sell what the evidence really means. It's basically impossible for someone who isn't pretty handy with Bayes to work out the significance for themselves. The average jury member has no chance.