6 ms·
When Do Extraordinary Claims Give Extraordinary Evidence? (2007) [pdf]
- nicholaides 11y agoFor example, you would assign a very low prior probability to your friend telling you that she met her 5’2” tall second cousin last Tuesday at 8:47am at 11 feet northwest of the smaller statue in a certain square. Nevertheless, after she made such a claim to you, you would likely place a high probability that the event happened as claimed. On topics like this, most people are fairly reliable. This is a bad example. "you would assign a very low prior probability to your friend telling you __________". If it was a prediction, yes, that would be extraordinary, but it's not. Its the same as if someone told you they flipped a coin 100 times and got a certain sequence of heads and tails. The probability of that happening is 1/2^100, but it's only extraordinary if you predict it in advance.
- nickpsecurity 11y agoI've always found the "extraordinary claims require extraordinary evidence" meme ridiculous. It defies reason and the scientific method. Extraordinary claims simply need evidence they're true. That's it. Then, analysis and experiments should be conceived to determine why they contradict the "ordinary" model plus look for sources of error. If error or deceit are found, then the extraordinary claim is rejected. If not, the "ordinary" model is amended or rejected with data from new. That simple. The need for extraordinary evidence seems to just promote orthodoxy. Better to focus on data and methods.
- paulddraper 11y agoIn context, that expression refers to claims that contradict existing evidence. For example, if someone finds a way to create mass/energy, that would be extraordinary. All existing evidence suggests mass/energy is conserved. Even if no one has tried XYZ before, the evidence of XYZ creating energy has to be weighed against the evidence of thousands of (other) processes not doing that. It's possible someone made a mistake, or the results are being interpreted incorrectly. While there is an ideal, the reality is that "evidence" is of varying types and qualities, and that perfect evidence is not always feasible. In the US, criminal convictions ("extraordinary" claims) require proof beyond reasonable doubt. But civil convictions ("ordinary" claims) require only a preponderance of the evidence.
- andrewflnr 11y agoThat may be the ideal usage, but I'm not sure I've ever seen it used that way. If the phrase is about contradicting existing evidence, we should just say that instead.
- nickpsecurity 11y agoMy point exactly. "Extraordinary evidence" just seems made up. I only require more and/or better evidence to back up an extraordinary claim versus claims contradicting it.
- GFK_of_xmaspast 11y ago> more and/or better evidence So, in other words, evidence that is beyond the ordinary. If only there were a prefix that could be applied here.
- nickpsecurity 11y agoNo, ordinary evidence or models that fit the observed data better or make more accurate predictions. I leave out the extraordinary part except for when praising scientists for talented observations or methods.
- paulddraper 11y agoOkay. Well, here extraordinary is supposed to mean better/more then ordinary. But if it means talented or praiseworthy or something similar, then I whole-heatedly agree.
- gwern 11y agoSo this is why you are a card-carrying member of a parapsychology association, right? Because many of the psi experiments, like Bem's recent ones, are better run than most psychology studies (sometimes much better run, as in, preregistered, highly powered, fully automated, and with adversarial collaborators running the study), and if you really believe that priors have no role in science, then you can't reject the psi studies without rejecting the rest as well.
- nickpsecurity 11y agoIt's why I changed my mind on occasion about various theories and why I was ahead of status quo on others. The secret? Focus on evidence and methods instead of where herd is going and how smart sources appear. Most of my beliefs are pretty orthodox because evidence supports them. Your claims are mere trolling. Also, a perfect example of baseless beliefs posing as reason and without supporting evidence. Basically what you were accusing me of.
- gwern 11y ago> Your claims are mere trolling. Also, a perfect example of baseless beliefs posing as reason and without supporting evidence. Basically what you were accusing me of. No, it's not. Maybe you should look into psi research if you truly believe that you can believe something based solely on research standards. The problem with psi research is not that the methodology is bad, it's that it's so absurd that experiments better than 99% of all published psychology research still aren't enough proof. So the question is, if you do not believe in psi, why not? It passes the criteria you have already laid out.
- nickpsecurity 11y agoWell, if you were being serious, then all I can say is no comment on that as I'd have to look at recent findings, methods, and verifications. Looked into it a long time ago with the PEAR work being the most interesting. Needed duplication by independents that especially unlikely to work as a group to defraud people. Additionally, I proposed to people I discussed it with that, if psi phenomenon exists, skeptics' own negativity toward the experiments could prevent successful verifications. So, we'd have to have people who could run the experiment other skeptics constructed and approved of while being neutral or encouraging the subjects to create positive, mental state. Not sure what's happened in the field since or if anyone else proposed something like I did. Radin & PEAR was only thing I saw then that looked like real science with well-thought methodology. Given high percentage of prior frauds, that not being ruled out in a substantive way in what I read on the topic had me saying "really interesting but who knows" as a result. I'd like to see the experiments re-run by diverse parties w/ conservative views on subject and reliable track record. Also, while mitigating both potential fraud and impact of negativity on test subjects. My initial response to you came from you picking an extremely controversial field of research rather than examples of widely-held or supported beliefs countered with newer, better evidence that wouldn't be dismissed immediately by readers as psi topic would. I figured, like how those do with "conspiracy theory" labels, you were trying to get my comment dismissed entirely. Btw, people are apparently calling out for you to respond in this thread about the Bitcoin leak: https://news.ycombinator.com/item?id=10699846 https://news.ycombinator.com/item?id=10699846