6 ms·

> Forensic labs and companies are expert witnesses with black box processes and the incentive to protect the authority of their profession. I imagine they're a

> Forensic labs and companies are expert witnesses with black box processes and the incentive to protect the authority of their profession. I imagine they're also incentivised to favour a result indicating guilt.

tantalor 6y ago
Why? The risk to their reputation & livelihood for a false positive should temper that.
- JoeAltmaier 6y ago
  While there have been cases of technicians guilty of systemic fudging of data, that is a tiny exception to the vast majority of honest technicians with self-respect.
  - kingkawn 6y ago
    Your sentimentality is not enough for a good justice system.
    - JoeAltmaier 6y ago
      On the contrary, that's what the American justice system is entirely based upon. Fair judges, juries of peers and so on. Its the bedrock of justice. Engineers have some notion that it can all be replaced with science. In so far as science can certify the reliability of tests, that is good. But in the end one has to trust the humans between those tests and the courtroom.
      - tsimionescu 6y ago
        The whole point of this discussion is that this trust is being largely misplaced. A lot of forensic science in itself is essentially phrenology (tooth prints, hair analysis - the non DNA kind, even fingerprinting is of exaggerated value). A lot of labs systematically turn out biased results with generally no expectation of risk, either personally to researchers and definitely not to higher ups. Ultimately, forensic evidence should be seen as a signal, but not nearly as trustworthy as witness testimony (which in itself is not very trustworthy), despite what many believe. And this is important for the general public to understand, the people who will participate in juries. The mystique that forensic experts have can make juries give extremely wrong decisions (even bad aquitalls on lack of forensic evidence).
        JoeAltmaier 6y ago
        Yes there is a need for controls and independent certification.
        tsimionescu 6y ago
        There is also a need to recognize that a lot of this science just doesn't work (to the extent that it is often presented). The right way to phrase a lot of the conclusions of these experts, at the state of the art, would be 'in my expert opinion, it looks like it might be their DNA/fingerprint/hair/teeth', not 'in my expert opinion it is [...]'. Because even the state of the art is often about at that level - probably closer to 75% accuracy than 99.99% like it's often treated. Especially on real world, partial, corrupted samples. And to emphasize again, I'm talking about the state of the art without the biases being discussed. The biases only compound this problem significantly.
- chopin 6y ago
  Why should that come to light if the process is opaque? Most defendants don't have the resources to challenge that. And if it comes to light, acknowledge a one-time error and carry on.
- wtetzner 6y ago
  I seems like there are no repercussions for sending an innocent person to prison. I don't know if that's true, but it definitely seems like it.
  - jimz 6y ago
    Depends on what you mean by repercussions. Since there is at least informal and sometimes fairly open scorekeeping happening at prosecutor's offices there's some hurt pride if your conviction is overturned later. Everything else either gets taken care of by the taxpayer or prosecutors' immunity for their official acts so on a personal level, no, there won't be any actual repercussions.
- andi999 6y ago
  The prosecution might call in a different expert in the next trial if the current one is not 'good'.
  - refurb 6y ago
    Then they get hired by the defense.
    - MaxBarraclough 6y ago
      The defence can rarely afford their services.
      - jimz 6y ago
        Prosecutors aren't necessarily blessed with bigger budgets in a lot of places, they simply have a separately-funded (and very well funded at that) investigative body that is the police or sheriff's office doing the really expensive part that the defense, especially a public defender's office, will only have a skeleton crew to do. The problem is that defense may not have the resources or knowledge to use inadequacies of something like this as a defense in the first place, and may not be able to effectively cross or direct an expert because of the lack of specific expertise and the reactive nature of the job. It's not that the venn diagram of tech-literate and criminal-defense lawyers are entirely separate circles, but having been "that guy" in a public defender's office for even basic stuff like cell tower triangulation accuracy to finding proprietary surveillance video codecs to decode exculpatory evidence, you really need to start at square one while the prosecution have the whole police department's resources, expertise, and initiative at their disposal. You can afford the expert, it just won't do any good when you don't know what questions to ask that will actually be effective. And unsurprisingly those who do have experience in technical, specialized fields tend to get poached into the private sector or out of trial (really plea) practice all together, so the knowledge/bullshit gap will still exist and there's no real consistent way to bridge it.
      - andi999 6y ago
        Also the defense is many customers while the prosecution is bulk.
- AntiImperialist 6y ago
  Two reasons: * they come across as effective if they "solve" the case i.e. they find someone guilty * it's practically impossible to challenge their findings because they hide their bs behind "experts" and "science" and other terms used by bs vendors
- dathinab 6y ago
  Because they likely use metrics which highlight how many criminals where caught due to their system. On the other side a metric which says how many times no conclusive evidence was found with DNA testing sounds more like a metric of failure (but is not, if you don't have qualitative good enough evidence including "complete enough" DNA you should not make up results which simply don't exists, but then courts ruling without conclusive evidence isn't that rare and some would say even necessary to some degree. It's just that many believe DNA tests are somehow unfailable perfects ways to prove the presence of an specific person at a specific place, but they are often not).
- MaxBarraclough 6y ago
  The hands-off reputation-based approach isn't enough to regulate food and drugs, or aviation. I don't see why for-profit forensics companies should be trusted implicitly. Also related: we know that funding bias [0] is a real problem in science, despite that scientists' reputations should (ideally) motivate them to resist such biases. [0] https://en.wikipedia.org/wiki/Funding_bias https://en.wikipedia.org/wiki/Funding_bias
  - kortilla 6y ago
    No, the question is what is the incentive for lying and claiming guilty? Do prosecutors choose different labs?
    - geocar 6y ago
      If you knew someone was guilty, because of a preponderance of other evidence you have the utmost faith in, such as a professional law enforcement officers’ eyewitness account, and the DNA testing vendor you were evaluating said “not guilty”, what exactly would you expect to happen?
      - boomboomsubban 6y ago
        That the eyewitness account would be discarded, they make for terrible evidence anyway.
        tekromancr 6y ago
        I think it is REALLY hard to overstate how dangerously naive a view of the american justice system that is
        boomboomsubban 6y ago
        I don't understand what you mean. I would expect them to make the obviously correct decision, though I'm fully aware that they very often don't.
- linuxftw 6y ago
  Perhaps in a just society. Most places, the state and it's agents can do no wrong, there's almost no incentive to do the right thing.
- irjustin 6y ago
  A lot of DA and other prosecutors are measured by the number of convictions they get vs the number of cases they take to court. Their goals are always to get plea bargain because that is automatic guilt or if they take it to court, they'd better win. That's a lot of state resources just to lose. Overtime, you'll get people willing to bend the rules, turn a blind eye to 'weak evidence' or whatever. You're only as good as your measurement/incentivisation.
  - treeman79 6y ago
    Deal: Parole for a year. If not, full resources of the government to convict you, jail time in the decades. Evan a failure to convict can ruin you for life. Even innocent people with a good alibi are going to take the deal.
- macinjosh 6y ago
  Prosecutors want a 100% conviction rate. They are the ones who hire these companies. Generally speaking companies give customers what they are looking for.
- vkou 6y ago
  When a DNA lab issues a false positive, its CEO doesn't go to prison, nor do they get hit with a charge-back for all the tests they've done in the past year. Their reputation doesn't matter as much as you think. Police procurement isn't based on the reputation of the vendor. It's based on whether or not the vendor delivers the results they want. In fact, government procurement in general, isn't based on the reputation of the vendor. This is by design.
- dimitrios1 6y ago
  This is a long tail regulatory force, but unfortunately, does nothing to stop bad actors from jumping in and making a quick profit for as long as they can until it goes noticed. Society also no longer has tolerance for processes that are not fully auditable and transparent, especially when it comes to our already marred justice system.
- _jal 6y ago
  Livelihood is frequently the reason why biases creep in. Independent labs tend to do better. Here's a look at two different Texas crime labs. Compare and contrast, and see if you can spot the differences. https://gritsforbreakfast.blogspot.com/2021/02/beyond-aid-to-law-enforcement-crime-lab.html https://gritsforbreakfast.blogspot.com/2021/02/beyond-aid-to... It starts with one pledging to: aid law enforcement in the detection, suppression, and prosecution of crime whereas the other seeks to provide medical examiner and crime laboratory services of the highest quality in an unbiased manner with uncompromised integrity. and goes from there.
- acranox 6y ago
  Maybe for some, but look what happened in Massachusetts. https://en.wikipedia.org/wiki/Annie_Dookhan https://en.wikipedia.org/wiki/Annie_Dookhan
- avs733 6y ago
  The risk to their business of negatives whether or not they are false negatives, is likely higher. The resource asymmetry between a tax payer funded prosecution and a defendant funded defense is enormous.
anonAndOn 6y ago
> incentivised to favour a result indicating guilt Exhibit A: Annie Dookhan of the Massachusetts State Crime Lab who is suspected of falsifying thousands of drug tests [0]https://www.cbsnews.com/news/massachusetts-lab-tech-arrested-for-alleged-improper-handling-of-drug-tests/ https://www.cbsnews.com/news/massachusetts-lab-tech-arrested...
- HarryHirsch 6y ago
  Annie Dookhan did about 3 years of jail, which is risible, considering that her cooking of evidence affected > 30000 cases. No visible consequences for the supervisor, which is scandalous, considering that the had been carrying on for at least 4 years.
heavyset_go 6y ago
They fill a similar role to prosecutors that field drug tests serve to cops[1]: they aren't meant to be accurate, they exist to give the cops legitimacy as they proceed to do whatever it is they wanted to do to you. In the cops' case, they need legitimacy to search and arrest you, and in the prosecutors' case, they need legitimacy to charge and convict you. If field drug tests were actually accurate, they wouldn't be bought and used. Similarly, if an expert witness fails to give prosecutors the results and testimony that they want, they wouldn't be hired again. [1] "How a $2 Roadside Drug Test Sends Innocent People to Jail": https://www.nytimes.com/2016/07/10/magazine/how-a-2-roadside-drug-test-sends-innocent-people-to-jail.html https://www.nytimes.com/2016/07/10/magazine/how-a-2-roadside...
- jdsalaro 6y ago
  > They fill a similar role to prosecutors that field drug tests serve to cops[1]: they aren't meant to be accurate, they exist to give the the cops legitimacy as they proceed to do whatever it is they wanted to do to you. In the cops' case, they need legitimacy to search and arrest you, and in the prosecutors' case, they need legitimacy to charge and convict you. In case any of you are interested, or completely flabbergasted as I was, by the idea that law enforcement's purpose and raison d'etre could ever become as distorted and contorted as this comment mentions, you should definitely read more on Walter "Johnny D." Macmillan [1] or watch the movie based on his story: Just Mercy[2]. Absolutely mind-blowing stuff. [1] https://en.wikipedia.org/wiki/Just_Mercy https://en.wikipedia.org/wiki/Just_Mercy [2] https://en.wikipedia.org/wiki/Walter_McMillian https://en.wikipedia.org/wiki/Walter_McMillian
  - bluntfang 6y ago
    Wow, think about what it takes to be an elected official (Sheriff) and not only perjure, lie, and coerce witnesses, but be outed for all of those crimes, and then continue to run for election and continue your tenure for decades. How does someone live with themselves knowing they purposely put an innocent man on death row, and then CONTINUE YOUR CAREER, probably thinking yourself as successful? The guilt I feel when I introduce a software bug that effects my corporate customers can sometimes burn my motivation for WEEKS. Who are these people?!
neodymiumphish 6y ago
This absolutely depends on the agency/situation. My agency used to rate our agents based on the amount of cases that led to convictions. Years back, we recognized that disproving an allegation was equally worthwhile, and settings promotions/evaluations based on "proving or disproving" the allegation was much more objectively reasonable than focusing strictly on convictions. There is still a reference to case completion speed, value of recovered money/property, and conviction results (providing a thorough enough investigation that the suspect is convicted for X years, for example), but disproving an allegation or proving that a different person committed the crime is far more appreciable to the agency. We're fortunate in that we are a federal agency, instead of local/state level, where they can be significantly more busy with countless lesser offenses, while still running some extremely high-profile stuff, but I think it would be a huge boon to law enforcement if states mandated an approach focused on this objective metric that isn't strictly on "getting him".

tantalor 6y ago

Why? The risk to their reputation & livelihood for a false positive should temper that.

JoeAltmaier 6y ago
While there have been cases of technicians guilty of systemic fudging of data, that is a tiny exception to the vast majority of honest technicians with self-respect.
- kingkawn 6y ago
  Your sentimentality is not enough for a good justice system.
  - JoeAltmaier 6y ago
    On the contrary, that's what the American justice system is entirely based upon. Fair judges, juries of peers and so on. Its the bedrock of justice. Engineers have some notion that it can all be replaced with science. In so far as science can certify the reliability of tests, that is good. But in the end one has to trust the humans between those tests and the courtroom.
    - tsimionescu 6y ago
      The whole point of this discussion is that this trust is being largely misplaced. A lot of forensic science in itself is essentially phrenology (tooth prints, hair analysis - the non DNA kind, even fingerprinting is of exaggerated value). A lot of labs systematically turn out biased results with generally no expectation of risk, either personally to researchers and definitely not to higher ups. Ultimately, forensic evidence should be seen as a signal, but not nearly as trustworthy as witness testimony (which in itself is not very trustworthy), despite what many believe. And this is important for the general public to understand, the people who will participate in juries. The mystique that forensic experts have can make juries give extremely wrong decisions (even bad aquitalls on lack of forensic evidence).
      - JoeAltmaier 6y ago
        Yes there is a need for controls and independent certification.
        tsimionescu 6y ago
        There is also a need to recognize that a lot of this science just doesn't work (to the extent that it is often presented). The right way to phrase a lot of the conclusions of these experts, at the state of the art, would be 'in my expert opinion, it looks like it might be their DNA/fingerprint/hair/teeth', not 'in my expert opinion it is [...]'. Because even the state of the art is often about at that level - probably closer to 75% accuracy than 99.99% like it's often treated. Especially on real world, partial, corrupted samples. And to emphasize again, I'm talking about the state of the art without the biases being discussed. The biases only compound this problem significantly.
chopin 6y ago
Why should that come to light if the process is opaque? Most defendants don't have the resources to challenge that. And if it comes to light, acknowledge a one-time error and carry on.
wtetzner 6y ago
I seems like there are no repercussions for sending an innocent person to prison. I don't know if that's true, but it definitely seems like it.
- jimz 6y ago
  Depends on what you mean by repercussions. Since there is at least informal and sometimes fairly open scorekeeping happening at prosecutor's offices there's some hurt pride if your conviction is overturned later. Everything else either gets taken care of by the taxpayer or prosecutors' immunity for their official acts so on a personal level, no, there won't be any actual repercussions.
andi999 6y ago
The prosecution might call in a different expert in the next trial if the current one is not 'good'.
- refurb 6y ago
  Then they get hired by the defense.
  - MaxBarraclough 6y ago
    The defence can rarely afford their services.
    - jimz 6y ago
      Prosecutors aren't necessarily blessed with bigger budgets in a lot of places, they simply have a separately-funded (and very well funded at that) investigative body that is the police or sheriff's office doing the really expensive part that the defense, especially a public defender's office, will only have a skeleton crew to do. The problem is that defense may not have the resources or knowledge to use inadequacies of something like this as a defense in the first place, and may not be able to effectively cross or direct an expert because of the lack of specific expertise and the reactive nature of the job. It's not that the venn diagram of tech-literate and criminal-defense lawyers are entirely separate circles, but having been "that guy" in a public defender's office for even basic stuff like cell tower triangulation accuracy to finding proprietary surveillance video codecs to decode exculpatory evidence, you really need to start at square one while the prosecution have the whole police department's resources, expertise, and initiative at their disposal. You can afford the expert, it just won't do any good when you don't know what questions to ask that will actually be effective. And unsurprisingly those who do have experience in technical, specialized fields tend to get poached into the private sector or out of trial (really plea) practice all together, so the knowledge/bullshit gap will still exist and there's no real consistent way to bridge it.
    - andi999 6y ago
      Also the defense is many customers while the prosecution is bulk.
AntiImperialist 6y ago
Two reasons: * they come across as effective if they "solve" the case i.e. they find someone guilty * it's practically impossible to challenge their findings because they hide their bs behind "experts" and "science" and other terms used by bs vendors
dathinab 6y ago
Because they likely use metrics which highlight how many criminals where caught due to their system. On the other side a metric which says how many times no conclusive evidence was found with DNA testing sounds more like a metric of failure (but is not, if you don't have qualitative good enough evidence including "complete enough" DNA you should not make up results which simply don't exists, but then courts ruling without conclusive evidence isn't that rare and some would say even necessary to some degree. It's just that many believe DNA tests are somehow unfailable perfects ways to prove the presence of an specific person at a specific place, but they are often not).
MaxBarraclough 6y ago
The hands-off reputation-based approach isn't enough to regulate food and drugs, or aviation. I don't see why for-profit forensics companies should be trusted implicitly. Also related: we know that funding bias [0] is a real problem in science, despite that scientists' reputations should (ideally) motivate them to resist such biases. [0] https://en.wikipedia.org/wiki/Funding_bias https://en.wikipedia.org/wiki/Funding_bias
- kortilla 6y ago
  No, the question is what is the incentive for lying and claiming guilty? Do prosecutors choose different labs?
  - geocar 6y ago
    If you knew someone was guilty, because of a preponderance of other evidence you have the utmost faith in, such as a professional law enforcement officers’ eyewitness account, and the DNA testing vendor you were evaluating said “not guilty”, what exactly would you expect to happen?
    - boomboomsubban 6y ago
      That the eyewitness account would be discarded, they make for terrible evidence anyway.
      - tekromancr 6y ago
        I think it is REALLY hard to overstate how dangerously naive a view of the american justice system that is
        boomboomsubban 6y ago
        I don't understand what you mean. I would expect them to make the obviously correct decision, though I'm fully aware that they very often don't.
linuxftw 6y ago
Perhaps in a just society. Most places, the state and it's agents can do no wrong, there's almost no incentive to do the right thing.
irjustin 6y ago
A lot of DA and other prosecutors are measured by the number of convictions they get vs the number of cases they take to court. Their goals are always to get plea bargain because that is automatic guilt or if they take it to court, they'd better win. That's a lot of state resources just to lose. Overtime, you'll get people willing to bend the rules, turn a blind eye to 'weak evidence' or whatever. You're only as good as your measurement/incentivisation.
- treeman79 6y ago
  Deal: Parole for a year. If not, full resources of the government to convict you, jail time in the decades. Evan a failure to convict can ruin you for life. Even innocent people with a good alibi are going to take the deal.
macinjosh 6y ago
Prosecutors want a 100% conviction rate. They are the ones who hire these companies. Generally speaking companies give customers what they are looking for.
vkou 6y ago
When a DNA lab issues a false positive, its CEO doesn't go to prison, nor do they get hit with a charge-back for all the tests they've done in the past year. Their reputation doesn't matter as much as you think. Police procurement isn't based on the reputation of the vendor. It's based on whether or not the vendor delivers the results they want. In fact, government procurement in general, isn't based on the reputation of the vendor. This is by design.
dimitrios1 6y ago
This is a long tail regulatory force, but unfortunately, does nothing to stop bad actors from jumping in and making a quick profit for as long as they can until it goes noticed. Society also no longer has tolerance for processes that are not fully auditable and transparent, especially when it comes to our already marred justice system.
_jal 6y ago
Livelihood is frequently the reason why biases creep in. Independent labs tend to do better. Here's a look at two different Texas crime labs. Compare and contrast, and see if you can spot the differences. https://gritsforbreakfast.blogspot.com/2021/02/beyond-aid-to-law-enforcement-crime-lab.html https://gritsforbreakfast.blogspot.com/2021/02/beyond-aid-to... It starts with one pledging to: aid law enforcement in the detection, suppression, and prosecution of crime whereas the other seeks to provide medical examiner and crime laboratory services of the highest quality in an unbiased manner with uncompromised integrity. and goes from there.
acranox 6y ago
Maybe for some, but look what happened in Massachusetts. https://en.wikipedia.org/wiki/Annie_Dookhan https://en.wikipedia.org/wiki/Annie_Dookhan
avs733 6y ago
The risk to their business of negatives whether or not they are false negatives, is likely higher. The resource asymmetry between a tax payer funded prosecution and a defendant funded defense is enormous.

anonAndOn 6y ago

> incentivised to favour a result indicating guilt Exhibit A: Annie Dookhan of the Massachusetts State Crime Lab who is suspected of falsifying thousands of drug tests [0]https://www.cbsnews.com/news/massachusetts-lab-tech-arrested-for-alleged-improper-handling-of-drug-tests/ https://www.cbsnews.com/news/massachusetts-lab-tech-arrested...

HarryHirsch 6y ago
Annie Dookhan did about 3 years of jail, which is risible, considering that her cooking of evidence affected > 30000 cases. No visible consequences for the supervisor, which is scandalous, considering that the had been carrying on for at least 4 years.

heavyset_go 6y ago

They fill a similar role to prosecutors that field drug tests serve to cops[1]: they aren't meant to be accurate, they exist to give the cops legitimacy as they proceed to do whatever it is they wanted to do to you. In the cops' case, they need legitimacy to search and arrest you, and in the prosecutors' case, they need legitimacy to charge and convict you. If field drug tests were actually accurate, they wouldn't be bought and used. Similarly, if an expert witness fails to give prosecutors the results and testimony that they want, they wouldn't be hired again. [1] "How a $2 Roadside Drug Test Sends Innocent People to Jail": https://www.nytimes.com/2016/07/10/magazine/how-a-2-roadside-drug-test-sends-innocent-people-to-jail.html https://www.nytimes.com/2016/07/10/magazine/how-a-2-roadside...

jdsalaro 6y ago
> They fill a similar role to prosecutors that field drug tests serve to cops[1]: they aren't meant to be accurate, they exist to give the the cops legitimacy as they proceed to do whatever it is they wanted to do to you. In the cops' case, they need legitimacy to search and arrest you, and in the prosecutors' case, they need legitimacy to charge and convict you. In case any of you are interested, or completely flabbergasted as I was, by the idea that law enforcement's purpose and raison d'etre could ever become as distorted and contorted as this comment mentions, you should definitely read more on Walter "Johnny D." Macmillan [1] or watch the movie based on his story: Just Mercy[2]. Absolutely mind-blowing stuff. [1] https://en.wikipedia.org/wiki/Just_Mercy https://en.wikipedia.org/wiki/Just_Mercy [2] https://en.wikipedia.org/wiki/Walter_McMillian https://en.wikipedia.org/wiki/Walter_McMillian
- bluntfang 6y ago
  Wow, think about what it takes to be an elected official (Sheriff) and not only perjure, lie, and coerce witnesses, but be outed for all of those crimes, and then continue to run for election and continue your tenure for decades. How does someone live with themselves knowing they purposely put an innocent man on death row, and then CONTINUE YOUR CAREER, probably thinking yourself as successful? The guilt I feel when I introduce a software bug that effects my corporate customers can sometimes burn my motivation for WEEKS. Who are these people?!

neodymiumphish 6y ago

This absolutely depends on the agency/situation. My agency used to rate our agents based on the amount of cases that led to convictions. Years back, we recognized that disproving an allegation was equally worthwhile, and settings promotions/evaluations based on "proving or disproving" the allegation was much more objectively reasonable than focusing strictly on convictions. There is still a reference to case completion speed, value of recovered money/property, and conviction results (providing a thorough enough investigation that the suspect is convicted for X years, for example), but disproving an allegation or proving that a different person committed the crime is far more appreciable to the agency. We're fortunate in that we are a federal agency, instead of local/state level, where they can be significantly more busy with countless lesser offenses, while still running some extremely high-profile stuff, but I think it would be a huge boon to law enforcement if states mandated an approach focused on this objective metric that isn't strictly on "getting him".