6 ms·

This reminds me of the case of Lucia de B., a nurse once suspected of killing her patients. There never was solid evidence against her. She was a suspect becau

This reminds me of the case of Lucia de B., a nurse once suspected of killing her patients. There never was solid evidence against her. She was a suspect because nearly everytime someone died in the hospital she was on her shift. The initial chance of her being present when these supposed murders happened, and being innocent, was estimated at 1 in 7 billion. This number caused the police to focus their research completely on Lucia de B. Eventually she was convicted for the murders based not on hard evidence or witnesses, but on statistics. The chance of innocence was put on 1 in 342 million. The econometrist Aart de Vos was the first to notice that the initial Bayesian analysis was plain wrong. For example they had presumed the murderer had to be found amongst the nurses, other possibilities were neglected. They also hadn't corrected for combined P-values. He reduced the chance Lucia was innocent to 1 in a million. The court said they had abandoned the statistical "proof", but remained of the view that it couldn't be a coincidence. Possible murder cases were chosen when they were negative for Lucia, other possible murder cases were left out of the equation. This further reduced the chance to 1 in 50. Statisticians Richard Gill and Piet Groeneboom further reduced the chance to 1 in 9. In my opinion statistics alone can never be adequate for conviction. It can muddy research and lead to confirmation bias. And the difference between a Bayesian chance of 1 in 7 billion and 1 in 9 is so big as to doubt the initial use of statistics even further. http://en.wikipedia.org/wiki/Lucia_de_Berk#Statistical_arguments http://en.wikipedia.org/wiki/Lucia_de_Berk#Statistical_argum...

delinka 14y ago
The phrase "...beyond a reasonable doubt" comes to mind. I would hope a jury involved in such a case would be more reasonable about actual hard evidence, but I hear that juries are selected for emotionality and not rationality. I would hope that the judge in such a case would be more reasonable and demand some actual evidence. Such is the case in a fearful populace, and one that demands vengeance over justice. I appears as though something bad happened, so someone must pay and ... here's a magic forumla with missing variables that says you might have done it so ... GUILTY.
- SEMW 14y ago
  > I hear that juries are selected for emotionality and not rationality The case the parent was talking about took place in the Netherlands. I don't think they even use juries in the Netherlands (though I could be wrong about that). Even in the UK -- the jurisdiction the article is about -- juries are purely random. We don't do the American practice of lawyers from both sides being able to reject jurors for their own reasons (as opposed to an actual good reason like their knowing the defendant, and the judge makes the decision on that).
  - DouweM 14y ago
    Correct, no juries in the Netherlands.
    - lifeisstillgood 14y ago
      wow. Err, what may I ask do you use?
      - ordinary 14y ago
        Civil law systems by and large do not use trial by jury. The judge (or judges) pronounces verdict.
        lifeisstillgood 14y ago
        The Netherlands is it seems one of the few democracies that do not use juries for any trials. Civil or not. The UK certainly has judge only civil trials (commercial law is basically a way for two barristers and a judge to get paid for sitting and talking for two months) http://www.dutchnews.nl/news/archives/2008/04/judge_calls_for_jury_trials_in.php http://www.dutchnews.nl/news/archives/2008/04/judge_calls_fo...
        ordinary 14y ago
        Most democratic countries use civil law, actually, though I don't think that's particularly relevant. There's a much stronger correlation between language and system of law than there is between form of government and system of law. Note the similarity between [0] and [1], and the complete lack thereof between [1] and [2]. [0] https://en.wikipedia.org/wiki/File:Anglospeak%28800px%29.png https://en.wikipedia.org/wiki/File:Anglospeak%28800px%29.png [1] https://en.wikipedia.org/wiki/File:LegalSystemsOfTheWorldMap.png https://en.wikipedia.org/wiki/File:LegalSystemsOfTheWorldMap... [2] https://en.wikipedia.org/wiki/File:Democracy_Index_2011.png https://en.wikipedia.org/wiki/File:Democracy_Index_2011.png
DanBC 14y ago
> In my opinion statistics alone can never be adequate for conviction. This is in an English court, and it is a civil case. Thus, there is no "conviction". For a criminal case the standard is "beyond reasonable doubt". For a civil case the standard is "balance of probability". So, no-one is going to jail or getting a criminal record. But they might have to pay compensation to someone else for fire damage.
- deleted 14y ago
  [deleted]
Anderkent 14y ago
It's not the courts decision that's jarring, it's the argumentation. If the court argued from human unreliability in enumerating options, the judgement would be sensible. But simply arguing from a mistaken understanding of probability makes it look silly.
davorak 14y ago
I do not think I understand your argument. In the case as you describe it the statistics are done incorrectly and cause a potential false conviction. How is this different then an expert on autopsies incorrectly does the job which leads to a potential false conclusion. In both case an expert made a mistake or did their job poorly leading to a potentially unfortunate outcome. Why should one be banned and not the other?
- Anderkent 14y ago
  The argument is mainly that it's (intuitively) much more common to make a significant error in statistics than autopsy. Additionally I'd guess most times if you make a significant error during an autopsy, you are aware of it. Whereas it's much more difficult to know whether you made an error in your statistical analysis or not.
  - davorak 14y ago
    When I made that statement I was think about how there have been blood spatter experts, bit mark experts, bug forensics experts that have all been discovered to frauds and cast doubts on to large swaths of forensics techniques that were accepted in general(by the court and public not always other experts in the field). That was impressions from remember articles where expertise in the above three fields was tested independently and experts were found to lack predictive power. I would have to search for that articles again if I wanted to provide a reference however I am working from memory.
- DanBC 14y ago
  For a criminal trial, with a jury? People are hopeless at statistics. Even on HN you'll find people making simple mistakes. Someone confidant presenting a mathematical argument would be persuasive to a jury, even if their numbers are totally bogus. > In both case an expert made a mistake or did their job poorly leading to a potentially unfortunate outcome. Why should one be banned and not the other? People doing the job poorly should be banned. But the nurse case is interesting because poor statistics became self-fulfilling. "We have some deaths, and this nurse is present when they happen, and so they are all suspicious", becomes "all the deaths that happen when this nurse is working are suspicious, and the deaths that don't happen when this nurse is working are not suspicious". And that sloppy thinking gets turned into "probability is 1 in a gajillion that these happen normally by chance."
mikeash 14y ago
"In my opinion statistics alone can never be adequate for conviction." I'm not sure I understand what you suggest there, because, what else is there? Let's say a murder conviction is based on 37 different security recordings clearly showing the defendant murdering the victim. The recordings make it a certainty that the defendant is guilty. But what is "certainty"? It's an expression of probability, and that comes from statistics. Given the evidence, what are the odds that the defendant is not guilty? Extremely low. We have to come up with pretty crazy alternatives. The probability of the defendant being innocent in this case might be, let's say, 1 in 10^20. We don't need to actually carry out this calculation, because we don't need the precise probability. It's clear that the probability is extremely low, so we can take a shortcut and refrain from figuring out exactly how low. But that shortcut is still ultimately an exercise in statistics.
- scott_w 14y ago
  You've completely missed the point. Having something on a security recording can be classed as "hard evidence". The case in question was one of using circumstantial evidence alone to "statistically" decide whether she was guilty or not. The law differentiates between circumstantial evidence and hard evidence. People aren't supposed to be convicted on circumstantial evidence alone.
  - mikeash 14y ago
    What differentiates hard evidence from circumstantial evidence? It comes down to the probabilities, which in turn comes down to statistics.
    - damoncali 14y ago
      Assumptions. The devil is in the details. Throw too much math with too many assumptions at anyone but a panel of PhD's and you will get all kinds of misinterpretation, misunderstanding, and just plain wrongheadedness.
      - mikeash 14y ago
crazygringo 14y ago
Regardless, even if it were "1 in 7 billion"... that ignores the entire possibility that, suppose, someone was intentionally trying to frame her. In that case, her chance of being present would be 1 in 1. But it doesn't mean she did it.
hadronzoo 14y ago
> the initial Bayesian analysis was plain wrong The initial analysis used Fisher's Exact Test, which is a frequentist rather than a Bayesian method.
pseut 14y ago
I have an MS in statistics, teach graduate statistics to econ PhD students (essentially; it's an 'econometrics' class); do research on statistical methods, etc. I'm really sympathetic to your point. I would not want the guilt or innocence of myself, a family member, a friend, etc depend on the numeracy and statistical literacy of a jury, with the stats explained to them through lawyers (even my own!) As to the OP, the key sentence for me is: > ...the fire had been started by a discarded cigarette, ... the other two explanations were even more implausible Only TWO other possible explanations for the cause of a fire!?!?! Regardless of the word choice, I think the appeals court has a better intuitive understanding of some of the pitfalls of model-based Bayesian statistics than the OP (namely, if you accidentally put zero weight on parts of the prior, you will necessarily put zero weight on those parts of the posterior). edit: if I'm going to be listing credentials, I should point out that my econ phd is in econometrics; I don't want to give the impression that my MS in stats is a sufficient credential to teach econ phd students.
lucisferre 14y ago
Bad statistics shouldn't be considered adequate for anything. I have no problem with solid statistics being used assuming they have been properly scrutinized by experts before being used to pass judgement. So many convictions come based on simple circumstantial evidence without even considering the statistical basis for those circumstances. Many convictions are entirely emotional bias on the part of the jury. Why should allowing statistical analysis to be considered as evidence be considered worse than the status quo?

delinka 14y ago

The phrase "...beyond a reasonable doubt" comes to mind. I would hope a jury involved in such a case would be more reasonable about actual hard evidence, but I hear that juries are selected for emotionality and not rationality. I would hope that the judge in such a case would be more reasonable and demand some actual evidence. Such is the case in a fearful populace, and one that demands vengeance over justice. I appears as though something bad happened, so someone must pay and ... here's a magic forumla with missing variables that says you might have done it so ... GUILTY.

SEMW 14y ago
> I hear that juries are selected for emotionality and not rationality The case the parent was talking about took place in the Netherlands. I don't think they even use juries in the Netherlands (though I could be wrong about that). Even in the UK -- the jurisdiction the article is about -- juries are purely random. We don't do the American practice of lawyers from both sides being able to reject jurors for their own reasons (as opposed to an actual good reason like their knowing the defendant, and the judge makes the decision on that).
- DouweM 14y ago
  Correct, no juries in the Netherlands.
  - lifeisstillgood 14y ago
    wow. Err, what may I ask do you use?
    - ordinary 14y ago
      Civil law systems by and large do not use trial by jury. The judge (or judges) pronounces verdict.
      - lifeisstillgood 14y ago
        The Netherlands is it seems one of the few democracies that do not use juries for any trials. Civil or not. The UK certainly has judge only civil trials (commercial law is basically a way for two barristers and a judge to get paid for sitting and talking for two months) http://www.dutchnews.nl/news/archives/2008/04/judge_calls_for_jury_trials_in.php http://www.dutchnews.nl/news/archives/2008/04/judge_calls_fo...
        ordinary 14y ago
        Most democratic countries use civil law, actually, though I don't think that's particularly relevant. There's a much stronger correlation between language and system of law than there is between form of government and system of law. Note the similarity between [0] and [1], and the complete lack thereof between [1] and [2]. [0] https://en.wikipedia.org/wiki/File:Anglospeak%28800px%29.png https://en.wikipedia.org/wiki/File:Anglospeak%28800px%29.png [1] https://en.wikipedia.org/wiki/File:LegalSystemsOfTheWorldMap.png https://en.wikipedia.org/wiki/File:LegalSystemsOfTheWorldMap... [2] https://en.wikipedia.org/wiki/File:Democracy_Index_2011.png https://en.wikipedia.org/wiki/File:Democracy_Index_2011.png

DanBC 14y ago

> In my opinion statistics alone can never be adequate for conviction. This is in an English court, and it is a civil case. Thus, there is no "conviction". For a criminal case the standard is "beyond reasonable doubt". For a civil case the standard is "balance of probability". So, no-one is going to jail or getting a criminal record. But they might have to pay compensation to someone else for fire damage.

deleted 14y ago
[deleted]

Anderkent 14y ago

It's not the courts decision that's jarring, it's the argumentation. If the court argued from human unreliability in enumerating options, the judgement would be sensible. But simply arguing from a mistaken understanding of probability makes it look silly.

davorak 14y ago

I do not think I understand your argument. In the case as you describe it the statistics are done incorrectly and cause a potential false conviction. How is this different then an expert on autopsies incorrectly does the job which leads to a potential false conclusion. In both case an expert made a mistake or did their job poorly leading to a potentially unfortunate outcome. Why should one be banned and not the other?

Anderkent 14y ago
The argument is mainly that it's (intuitively) much more common to make a significant error in statistics than autopsy. Additionally I'd guess most times if you make a significant error during an autopsy, you are aware of it. Whereas it's much more difficult to know whether you made an error in your statistical analysis or not.
- davorak 14y ago
  When I made that statement I was think about how there have been blood spatter experts, bit mark experts, bug forensics experts that have all been discovered to frauds and cast doubts on to large swaths of forensics techniques that were accepted in general(by the court and public not always other experts in the field). That was impressions from remember articles where expertise in the above three fields was tested independently and experts were found to lack predictive power. I would have to search for that articles again if I wanted to provide a reference however I am working from memory.
DanBC 14y ago
For a criminal trial, with a jury? People are hopeless at statistics. Even on HN you'll find people making simple mistakes. Someone confidant presenting a mathematical argument would be persuasive to a jury, even if their numbers are totally bogus. > In both case an expert made a mistake or did their job poorly leading to a potentially unfortunate outcome. Why should one be banned and not the other? People doing the job poorly should be banned. But the nurse case is interesting because poor statistics became self-fulfilling. "We have some deaths, and this nurse is present when they happen, and so they are all suspicious", becomes "all the deaths that happen when this nurse is working are suspicious, and the deaths that don't happen when this nurse is working are not suspicious". And that sloppy thinking gets turned into "probability is 1 in a gajillion that these happen normally by chance."

mikeash 14y ago

"In my opinion statistics alone can never be adequate for conviction." I'm not sure I understand what you suggest there, because, what else is there? Let's say a murder conviction is based on 37 different security recordings clearly showing the defendant murdering the victim. The recordings make it a certainty that the defendant is guilty. But what is "certainty"? It's an expression of probability, and that comes from statistics. Given the evidence, what are the odds that the defendant is not guilty? Extremely low. We have to come up with pretty crazy alternatives. The probability of the defendant being innocent in this case might be, let's say, 1 in 10^20. We don't need to actually carry out this calculation, because we don't need the precise probability. It's clear that the probability is extremely low, so we can take a shortcut and refrain from figuring out exactly how low. But that shortcut is still ultimately an exercise in statistics.

scott_w 14y ago
You've completely missed the point. Having something on a security recording can be classed as "hard evidence". The case in question was one of using circumstantial evidence alone to "statistically" decide whether she was guilty or not. The law differentiates between circumstantial evidence and hard evidence. People aren't supposed to be convicted on circumstantial evidence alone.
- mikeash 14y ago
  What differentiates hard evidence from circumstantial evidence? It comes down to the probabilities, which in turn comes down to statistics.
  - damoncali 14y ago
    Assumptions. The devil is in the details. Throw too much math with too many assumptions at anyone but a panel of PhD's and you will get all kinds of misinterpretation, misunderstanding, and just plain wrongheadedness.
    - mikeash 14y ago

crazygringo 14y ago

Regardless, even if it were "1 in 7 billion"... that ignores the entire possibility that, suppose, someone was intentionally trying to frame her. In that case, her chance of being present would be 1 in 1. But it doesn't mean she did it.

hadronzoo 14y ago

> the initial Bayesian analysis was plain wrong The initial analysis used Fisher's Exact Test, which is a frequentist rather than a Bayesian method.

pseut 14y ago

I have an MS in statistics, teach graduate statistics to econ PhD students (essentially; it's an 'econometrics' class); do research on statistical methods, etc. I'm really sympathetic to your point. I would not want the guilt or innocence of myself, a family member, a friend, etc depend on the numeracy and statistical literacy of a jury, with the stats explained to them through lawyers (even my own!) As to the OP, the key sentence for me is: > ...the fire had been started by a discarded cigarette, ... the other two explanations were even more implausible Only TWO other possible explanations for the cause of a fire!?!?! Regardless of the word choice, I think the appeals court has a better intuitive understanding of some of the pitfalls of model-based Bayesian statistics than the OP (namely, if you accidentally put zero weight on parts of the prior, you will necessarily put zero weight on those parts of the posterior). edit: if I'm going to be listing credentials, I should point out that my econ phd is in econometrics; I don't want to give the impression that my MS in stats is a sufficient credential to teach econ phd students.

lucisferre 14y ago

Bad statistics shouldn't be considered adequate for anything. I have no problem with solid statistics being used assuming they have been properly scrutinized by experts before being used to pass judgement. So many convictions come based on simple circumstantial evidence without even considering the statistical basis for those circumstances. Many convictions are entirely emotional bias on the part of the jury. Why should allowing statistical analysis to be considered as evidence be considered worse than the status quo?