10 ms·
Court of Appeal bans Bayesian probability
- PaulHoule 14y agoIf they make Bayesian analysis illegal only criminals will use Bayesian analysis.
- smoyer 14y agoIf they make Bayesian analysis illegal, they've turned those using it into criminals (by definition). Still a great comment!
- fauigerzigerk 14y agoIf Bayesian analysis were flawed to the extent of being useless quackery, it wouldn't hurt if only criminals used it (assuming criminals are bad).
- misnome 14y agoAs a frequentist I fully support this endeavour.
- jasonlingx 14y agoWhy is this linkbait at the top of HN?
- smoyer 14y agoWhat's the chance that the legal system would suddenly become coherent if we replaced all lawyers (and therefore judges) with mathematicians?
- muuh-gnu 14y agoLwayers are not the problem. The law makers and the law making and decision making process is the problem.
- jacques_chester 14y agoIn common law countries there are three main tributaries of law: legislation, the common law, and equity. Depending on where you are, the latter two are sometimes "fused" into a single type of law. "Law makers" -- I presume you mean legislators -- only control legislation. This is often called "positive law", in that somebody has to take positive action to "make" it where it didn't exist before. By convention, the common law is considered to be "discovered". It exists independent of men and women, and the role of judges is to determine what it is. In practice there is a dynamic tension between the evolution of the common law as it finds and adapts to new problems, and the intervention of the legislature when it decides that the law should be something else. Legislation itself quickly accretes case law that decides what the given piece of legislation means.
- Mvandenbergh 14y agoZero.
- jheriko 14y agoit might become consistent but at the cost of being complete... or the other way around. ;)
- jheriko 14y agoer... not being complete XD
- mich41 14y agoMathematicians are people who define probability as the measure of some well-defined subset of some well-defined set of all events. The people who use probability theory to build approximate models of real world events are called statisticians (and those who forget about "approximate" - applied statisticians).
- jacques_chester 14y agoThe law is highly coherent. But it deals with the largest and messiest possible problem domain: everything humans do.
- danso 14y agoWell, "everything humans do" includes creating the law, so by that definition, the law is highly incoherent. And a close reading of the law reveals this. In fact, it is often by close readings of the law (and precedent and every other historical quirk) on which controversial and contested decisions are rendered
- jacques_chester 14y ago> Well, "everything humans do" includes creating the law, so by that definition, the law is highly incoherent. Which meaning of incoherent are we taking here? > And a close reading of the law reveals this. Do go on.
- rayiner 14y ago90% of the work of a jury trial is managing human reactions to things. There are things that judges need to know about statistics. E.g. That in modern DNA forensics, lab error totally dominates the probability of a coincidental match, limiting accuracy to 1-2%. On the other hand, they must be deeply cognizant of human responses. A mathematician, told that some test shows a person 1,000 times more likely than average to be the killer, will intuitively realize that the person is still unlikely to be the killer. A jury of ordinary people won't. We have judges precisely to mediate between these two domains.
- tehwalrus 14y ago"I teach the Bayesian approach to post-graduate students attending my 'Applied Bayesian Statistics' course at Cambridge, and so I must now tell them that the entire philosophy behind their course has been declared illegal in the Court of Appeal. I hope they don't mind." Classic. I was told (at same uni) that, in a case involving DNA evidence, I could ask to be dismissed from a Jury, citing training in Bayesian statistics (which was prohibited because reversing the scientific certainty to look at false positives, in cases with weak circumstantial evidence, essentially kills DNA evidence's usefulness. If someone was in the right city, you expect hundreds of DNA matches out of millions of people, but if you can track them to the right street at the right time, you would expect 0.001 matches, and thus the evidence points to them much more strongly.)
- jacques_chester 14y agoEverything you hear about law that was not conveyed to you by a lawyer, a law lecturer or a judge is probably horseshit. People love to think they know some obscure wrinkle, some cool loophole, some nifty curio about the law. But so many of the stories you hear are just stories. ps. a better way to get disqualified is to have studied law.
- tehwalrus 14y agoindeed, this was my point. I didn't even contemplate trying it when I was called for jury duty.
- Anderkent 14y agoessentially kills DNA evidence's usefulness Could you elaborate? What kind of (valid) use does DNA evidence has that is disproved by bayesian reasoning?
- gizmo 14y agoExample DNA useful: Cops find criminal's blood at crime scene. Cops track down murder suspect using security video. Find suspect. Take DNA sample. Sample matches DNA at crime scene. DNA proves guilt beyond reasonable doubt 1:1000000. Example DNA not useful: Cops find suspect's blood at crime scene. Cops peek in DNA database and find 3 people in the region with matching DNA. One of them sort of looks like the shadow in the security video. They arrest him, get him picked out of a lineup and say DNA matches blood at the crime scene in the trial. Cops say DNA evidence proves guilt with odds 1:1000000, but odds of innocence are the complete opposite and closer to, say, 1000:1.
- jheriko 14y agothis seems like a complete over simplification and naive analysis of the situation - and the argument smells like a straw man and appeal to authority, comparing it to sherlock holmes - who incidentally /is obviously wrong/. the idea that 'whatever remains' is quantifiable and finite in the real world is a stark contrast with reality - and actually the fact that a probability is precisely 0 or 1 is very significant to its power in drawing conclusions. so not only does forbidding probability not forbid sherlock holmes style 'logic' but his logic is broken anyway in the vast majority of real world cases. what the court says is common sense... its a shame it offends some academic view... oh, no, wait, it really isn't.
- Anderkent 14y agoand the argument smells like a straw man and appeal to authority, comparing it to sherlock holmes A humorous reference to literature is hardly an appeal to authority. who incidentally /is obviously wrong/ He's not /obviously wrong/, he's subtly wrong in not explicitly acknowledging that the enumeration of options, not evaluation of their probability, is usually the difficult part. He's obviously right if you assume all possibilities are enumerated. actually the fact that a probability is precisely 0 or 1 is very significant to its power in drawing conclusions 0 and 1 are not probabilities, and while certainty has power in drawing conclusions, it only really occurs in purely theoretical discussions. what the court says is common sense... Whether it's common sense or not does not make it valid. oh, no, wait, it really isn't Is that about it being common sense, or it offending academic views?
- jacques_chester 14y ago> this seems like a complete over simplification and naive analysis of the situation The key phrase in the linked article: > and so I must now tell them that the entire philosophy behind their course has been declared illegal in the Court of Appeal. I hope they don't mind. Is what reveals the misunderstanding. Judges don't "declare" things illegal, they rule on matters of what the law is. That the law requires judgements to be rendered in certain terms is not a statement of the legality of making Bayesian arguments. The law requires a decision. The conceptual basis of the common law is a promise that a court will always render a clear and specific set of rulings or orders, and that there will always be an explanation of those rulings or orders. Judges are not free to say to disputants that they are X% likely to win their claim. This is one of those places where Bayesian probability isn't a good fit. Fuzzy logic -- so passé these days -- at least has an understood mechanism for "defuzzifying" in a fashion which lawyers would find quite familiar. It also overlooks that judges have many other places to introduce flexibility and weighed judgement. For example, judges may assign blame in portions for some crimes or torts; they may reject, moderate or modify some claims in equity; they have leeway to combine multiple considerations and legislative constraints in handing down criminal sentences and so on. One thing that bugs me tremendously about outsiders looking in at law is the assumption that, since lawyers don't immediately and entirely embrace new idea X, they are fusty old fools who are an impediment to the good. It's an argument born of ignorance that lawyers are deliberately obtuse fools, or judges out-of-touch theoreticians. Lawyers and judges touch on more problem domains in more depth, with greater consequences, than pretty much every academic and every software developer.
- blauwbilgorgel 14y agoThis reminds me of the case of Lucia de B., a nurse once suspected of killing her patients. There never was solid evidence against her. She was a suspect because nearly everytime someone died in the hospital she was on her shift. The initial chance of her being present when these supposed murders happened, and being innocent, was estimated at 1 in 7 billion. This number caused the police to focus their research completely on Lucia de B. Eventually she was convicted for the murders based not on hard evidence or witnesses, but on statistics. The chance of innocence was put on 1 in 342 million. The econometrist Aart de Vos was the first to notice that the initial Bayesian analysis was plain wrong. For example they had presumed the murderer had to be found amongst the nurses, other possibilities were neglected. They also hadn't corrected for combined P-values. He reduced the chance Lucia was innocent to 1 in a million. The court said they had abandoned the statistical "proof", but remained of the view that it couldn't be a coincidence. Possible murder cases were chosen when they were negative for Lucia, other possible murder cases were left out of the equation. This further reduced the chance to 1 in 50. Statisticians Richard Gill and Piet Groeneboom further reduced the chance to 1 in 9. In my opinion statistics alone can never be adequate for conviction. It can muddy research and lead to confirmation bias. And the difference between a Bayesian chance of 1 in 7 billion and 1 in 9 is so big as to doubt the initial use of statistics even further. http://en.wikipedia.org/wiki/Lucia_de_Berk#Statistical_arguments http://en.wikipedia.org/wiki/Lucia_de_Berk#Statistical_argum...
- delinka 14y agoThe phrase "...beyond a reasonable doubt" comes to mind. I would hope a jury involved in such a case would be more reasonable about actual hard evidence, but I hear that juries are selected for emotionality and not rationality. I would hope that the judge in such a case would be more reasonable and demand some actual evidence. Such is the case in a fearful populace, and one that demands vengeance over justice. I appears as though something bad happened, so someone must pay and ... here's a magic forumla with missing variables that says you might have done it so ... GUILTY.
- SEMW 14y ago> I hear that juries are selected for emotionality and not rationality The case the parent was talking about took place in the Netherlands. I don't think they even use juries in the Netherlands (though I could be wrong about that). Even in the UK -- the jurisdiction the article is about -- juries are purely random. We don't do the American practice of lawyers from both sides being able to reject jurors for their own reasons (as opposed to an actual good reason like their knowing the defendant, and the judge makes the decision on that).
- DanBC 14y ago> * In upholding the first instance decision, the Court of Appeal reiterated the principle in cases where there are competing explanations for a particular loss that causation cannot be established only by a process of elimination such that the 'least unlikely' cause of a loss is identified. A claimant must demonstrate that the particular version of events that they rely upon is more likely to have happened than not, in order for the civil burden of proof to be satisfied.* I'm not sure what the problem is. "It could have been A, B, or C. It's really unlikely to have been A or B, and thus it must be C" is obviously flawed, because for all we know it could have been D, and even if it was C you need to show (on the balance of probabilities) that C is the cause. Not just that C is more likely than A or B.
- jacques_chester 14y agoIt's not the judge's role to independently nominate D. It's up to the lawyers for the plaintiff and the respondent to present facts and make legal arguments. The judge's role is to weigh those legal arguments, and in (most) civil trials to weigh the facts presented on the balance of probability, and to render a decision. If you require judges to run a Bayesian network over the entire universe for each case, the legal system will get a wee bit slower.
- Anderkent 14y agoIt's not the judge's role to independently nominate D. But that is exactly why the judge can not assume all possibilities were enumerated, and requires positive evidence for C, rather than negative evidence for A and B.
- jacques_chester 14y agoJudges, oddly enough, decide cases according to the rules of legal reasoning. They cannot and won't introduce their own evidence. A judge that did so would be rightly removed from the bench.
- Anderkent 14y ago
- jacques_chester 14y agoPlease amend the title. It is not correct. The Court of Appeals has not "banned" Bayesian probability. It does not have the power to do so.
- flexie 14y agoThis guy extrapolates wildly from the decision. This is a question about whether or not a cigarette bud started a fire, not a rejection of bayesian probability as such. This guy should stick to teaching statistics, not law.
- 4ad 14y agoDid you bother reading the post at all? The inference in the ruling is based on the rejection of Bayesian statistics.
- flexie 14y agoDid you bother reading the ruling: http://www.bailii.org/ew/cases/EWCA/Civ/2013/15.html http://www.bailii.org/ew/cases/EWCA/Civ/2013/15.html The ruling is not concerned with statistics as a mathematical discipline. The "balance of probabilities" is a legal term. Please read the court ruling, and then read the blogger's conclusion (summed up sarcastically in his last sentence) and tell me if you think he's right :-)
- alan_cx 14y ago'The "balance of probabilities" is a legal term.' So is "reasonable doubt", but I am yet to see a perfect definition of it. Indeed, in the UK a jury questioned its meaning, the judge got snottily arrogant over it and ditched the case, but still not proper definition, either from the critical judge or anywhere else in the legal system. Interesting to me that the judge was so unable to answer the question, that he ditched a whole trial. I am amused by legal arrogance that thinks every one else should fully understand all its weirdness and definitions. Especially having done some work in a solicitors office where I was the arrogant one getting irritated with most of the legal people not having even a basic understand of the computers on their desks...
- SEMW 14y ago> in the UK a jury questioned its meaning, the judge got snottily arrogant over it and ditched the case ... Interesting to me that the judge was so unable to answer the question, that he ditched a whole trial. If you're talking about Huhne & Pryce, that's just wrong. The jury was discharged because after 2 days of deliberations they told the judge that they couldn't come to a verdict (which 10 of them agreed on). http://en.wikipedia.org/wiki/Hung_jury http://en.wikipedia.org/wiki/Hung_jury . I don't know where you got the idea that the judge got defensive because he didn't know how to answer what reasonable doubt meant, but it's utter nonsense. It's a very common question for juries to ask. The appeal courts have debated several times what the best guidance is. The judge was following the current guidance, which is tell the jury to treat it as ordinary english words and refuse to give any more specific definition. If each judge gave their own favourite interpretation of what 'beyond reasonable doubt' means (e.g. p(false positive) < 5%, or whatever), then the standard of proof you'd be tried under would depend on the judge, which would be obviously unacceptable.
- SEMW 14y agoSee also: R v T (Footwear Mark Evidence) http://www.bailii.org/ew/cases/EWCA/Crim/2010/2439.html http://www.bailii.org/ew/cases/EWCA/Crim/2010/2439.html (which is redacted to hide the appellant's identity, giving the judgment a delightfully dystopian feel). There it was held that an expert witness shouldn't use Bayesian reasoning to calculate probabilities to tell a jury "outside the field of DNA (and possibly other areas where there is a firm statistical base)". I think that, by a 'firm statistical base', the court's getting at the sort of situations where the right prior is a widely agreed on (such as DNA, where the size of the DNA database is known), so there won't be much opportunity for different expert witnesses to disagree on probabilities due to having different priors. (N.B. IANAL) This is still nonsense, IMHO. I can understand the court not wanting juries to be overly swayed by a spuriously precise probability that might have been very different if a different expert had been chosen. But there's no justification for restricting the statistical methods that the expert uses to reach their conclusion, however it's expressed to the jury. Interestingly, that case shared one of the appeal judges from the case in TFA (Lord Justice Beatson, then Mr Justice Beatson).
- diminoten 14y agoI think the distinction being made here is the difference from using statistical methods for determining overall guilt versus using statistical methods for determining specific facts about the case. We use statistics to say that someone was or was not at the scene of a crime (DNA testing doesn't 100% guarantee that a persons' blood is actually a match), for example. We don't make the leap that since they were there, "odds are" they did it, however.
- SEMW 14y agoThe question in R v T that Bayes theorem was being applied to was a specific, factual one: whether a footprint could have been made by a particular shoe.
- Anderkent 14y agoThe quotations in this article are taken out of context and presented incomplete. For example, the article quotes The chances of something happening in the future may be expressed in terms of percentage. Epidemiological evidence may enable doctors to say that on average smokers increase their risk of lung cancer by X%. But you cannot properly say that there is a 25 per cent chance that something has happened: Hotson v East Berkshire Health Authority [1987] AC 750. Either it has or it has not. And the judgement continues: In deciding a question of past fact the court will, of course, give the answer which it believes is more likely to be (more probably) the right answer than the wrong answer, but it arrives at its conclusion by considering on an overall assessment of the evidence (i.e. on a preponderance of the evidence) whether the case for believing that the suggested event happened is more compelling than the case for not reaching that belief (which is not necessarily the same as believing positively that it did not happen). Which is exactly the bayesian approach of 'probability as state of knowledge'.
- bo1024 14y agoThe quotes are contradictory. The Bayesian approach agrees with the second and disagrees with the first. I suppose you could weasel in some sort of consistency about the wording ... something like "You cannot properly say there is a 25 percent chance that something has happened, but you can properly say you believe there is a 25 percent chance that something has happened." OK, fine, but what's the difference? You still assign a percent probability to the chance that something happened, and you still rule on the basis of those beliefs. Or do you? (It's not clear to me from those quotes.)
- lifeisstillgood 14y agoI am paraphrasing the two quotes: You can say there is a 25% increase in chance of dying if you smoke. You cannot say there is a 25% chance that the nurse murdered those people - she either did or did not. (what we lack is the knowledge of the past event) second para: On the preponderance of evidence, the court believes beyond a reasonable doubt that the nurse *did* kill those people. That event A *did* happen - we have used evidence to fill in the gaps of our knowledge. We are not assigning a probability to it, but we recognise that life is all probabilities, but thats what reasonable doubt it all about. I think that those two quotes (modified as I understand them) are consistent. They are also not entirely Bayesian but I would prefer to be in a court of law than a court of probability.
- ekianjo 14y agoAccidents, murders, fires, etc... are all unlikely events by nature. Judging whether one of them can have occurred based on probability does not prove anything. The author post is wrong is the assumption that "an unlikely event"="impossible". Probability does NOT ascertain certitude. There is always an expression of confidence which is not equal to 100%, and therefore it seems logical that a court does not take the probability of an event as a tangible proof of what did or did not happen. Probability != science.
- Millennium 14y agoYou know, I think I'm OK with this. Sherlock Holmes was a detective, not a judge. What was appropriate for his line of work isn't appropriate in the courtroom. Courts are not about what MIGHT happen (or have happened); they are about what DID (or did not) happen. Probability can be an excellent guide for further investigation into these things, but it should be left to the investigators. If they can't find anything harder than a set of odds, then judges have no place betting on them.
- Tichy 14y agoStill, the argument "either A has happened or it has not happened" is silly. What matters is the knowledge we base our decisions on (ultimately we don't even know the outside world really exists, we only have a set of measurements and infer the existence of an outside world from that), and sometimes that is only partially certain.
- alan_cx 14y agoVery good point. I wish I could vote that up a few times. One will have to do.
- chimeracoder 14y ago> If they can't find anything harder than a set of odds, then judges have no place betting on them. But this is how our entire legal system is set up. For criminal proceedings, we have to find the defendant 'guilty beyond reasonable doubt'. What is 'reasonable doubt' if not a (theoretically) quantifiable or parameterizeable representation of our belief that the defendant committed the criminal act? We don't explicitly quantify this threshold as, eg. a 95% chance, but that's still the exact same process we require a jury to undergo, even if we don't attach quantifiable numbers to the results. The jury can never return a probability of 1 (in statistical terms, 'almost certain'), or else the entire appeal system wouldn't exist.
- Torgo 14y agoI believe that the purpose of randomize juries could be said to exist because while there are parameters, no one person or entity knows all the parameters.
- mmcnickle 14y agoI not sure what the OP expects the court of appeal to do. They are tasked with determining if there is cause for appeal; in this specific case whether the original judge erred in law. They determined that he did not. They aren't saying whether the judgement was right or wrong, rather that the judgement was arrived at in a lawful manner.
- arcadeparade 14y agoElizier Yodokowsky is going to be pissed.
- drucken 14y agoThe whole basis of law is elimination of "epistemic uncertainty" and threshold of evidence for an event. If this is not possible, the case is not proved, therefore case dimissed. Sherlock Holme's statement is a statement of method, not statement of proof. In other words, the law is far greater than Bayesian probability or any method, including scientific, since the burden of proof is from the evidence with the primary legal threshold of "innocent until proven guilty".
- Dylan16807 14y agoHow many times in this comment page do people need to be reminded that the burden of proof in civil cases is only 51%, not "beyond reasonable doubt" like criminal cases.
- pfortuny 14y agoThe problem probably lies in the fact that you cannot prove something by reduction to the absurd based on a probability distribution. Because law requires positive proof (or it should) not just "this chain of events is so unlikely under other assumptions that our assumptions must be right". This is where Sherlock's quotation is misleading: he says "rule out the IMPOSSIBLE", not the improbable. The judge is right epistemologically. Bayesian statistics has only a terminological issue.
- zimbatm 14y agoIt is well known that 70% of all statistics are wrong.
- hawkw 14y agoAs a Bayesian rationalist, wut.
- Draco6slayer 14y agoThe opening Holmes quote is being used improperly. Holmes used that as opening, then he went on to find further evidence to support that conclusion. He did not rely on logic alone to determine the culprit, but rather went on to find hard evidence.
- rayiner 14y agoIf anyone finds this subject interesting: my evidence professor had a PhD in statistics, and he's written about the application of probability to the law of evidence extensively. See: http://papers.ssrn.com/sol3/cf_dev/AbsByAuth.cfm?per_id=51570 http://papers.ssrn.com/sol3/cf_dev/AbsByAuth.cfm?per_id=5157...
- Colman 14y agoFor anyone curious about how this would play out in the US, federal courts and most state courts use the Daubert standard. TLDR: experts (including statisticians) can testify if they're using fairly standard methods and there are no significant gaps in the evidence-->analysis-->testimony chain. http://en.wikipedia.org/wiki/Daubert_standard http://en.wikipedia.org/wiki/Daubert_standard http://en.wikipedia.org/wiki/Daubert_v._Merrell_Dow_Pharmaceuticals http://en.wikipedia.org/wiki/Daubert_v._Merrell_Dow_Pharmace...
- jeremyjh 14y agoPoker players understand that a lack of knowledge of past events is exactly the same as uncertainty about future events. For example, if the first round of hold em has dealt two cards to each of 9 players, there are 18 cards out and you only know the state of two of them. The odds that the flop will contain specific cards that help your hand are calculated against the total number of unknown cards, regardless if the unknowns are held by other players or still in the deck. For example, you get a pair of 3s. There are two more threes in the deck, and it is pretty likely that if the flop contains one of them that you will have the best hand at the table at that point. For the first card turned up at the flop, there is a 2/50 chance it will be a three, the next is 2/49, the next 2/48. It doesn't matter how many face-down cards have been dealt to your opponents. All the unknowns are still in the pool of possibilities, and it is no different when you assess the evidence you have of any other type of event that has already happen. Each bit of evidence means what means, and the unknowns contribute to the pool of uncertainty.