3 ms·
There is a similarly interesting result in ET Jaynes "Probability The Logic of Science" (chapter 5). Where Jaynes demonstrates that increased evidence actually
by Homunculiheaded 11y ago
There is a similarly interesting result in ET Jaynes "Probability The Logic of Science" (chapter 5). Where Jaynes demonstrates that increased evidence actually decreases your belief in a hypothesis.
Jaynes gives an example of an experiment in which a psychic predicts cards, getting n out of m correct. In classical hypothesis testing H0 would be "the psychic got lucky" and H1 is "the psychic has mystic powers". Jaynes first attempts to use Bayesian reasoning to work backward to determine your prior belief in psychics. That is, how much evidence would it take to convince you, compare that to the raw likelihood and what you have is your prior beliefs quantified.
He then points out that he personally would never believe the subject was psychic, this is because there are not just H0 and H1, but H2 "the psychic is deceiving the experimenters", H3 "the experimenters are making an error" , H4 "the experiments are fudging the results", H5 ... Each of these has their own prior.
If your prior for believing in psychics is low enough and your prior for "the experimenters are fraud" is high enough, the more extreme the evidence the more you will be convinced that the experimenters are disreputable con artists, and subsequently the less you will believe the subject is psychic.
This is actually Jaynes' solution to a huge problem with Bayesian reasoning as a form of human reasoning: "If more data should override a bad prior, then why in the 'age of information' does nobody argee on anything!" This example shows, according to Jaynes, that while we can certainly have irrational priors we can still explain human reason in Bayesian terms and still get a situation where two people faced with plentiful information will arrive at contradictory conclusions.
- im2w1l 11y agoI don't like wording "arrive at contradictory conclusions", because it implies the creation of differences in beliefs. I'd rather frame it as "maintain contradictory preconceptions". And the cause of this I would say is a lack of information: Information capable of distinguishing between H1-H5
- im2w1l 11y agoOr... now I am not that sure anymore. What if Person A believe in H0:97%, H1: 2%, H2: 1%, and Person B believe in H0:97%, H1: 1%, H2: 2%. They have quite similar beliefs, at least by some measures. But if H0 is ruled out, then suddenly their beliefs will be very different.
- newman111 11y agoThis doesn't seem particularly illustrative without some context. or significant figures. I'm all for contrived examples when they are used to help understand, but this feels like an artificial comparison.
- marvy 11y agoHow does this contradict your earlier comment? I think it's still valid: Person A believes H1 (real psychic), Person B believes H2 (talented fraud). The question now is how to get them to agree. And the answer is: design a better experiment, such that fraud is not practical, and see what happens.
- jkldotio 11y ago"A Bayes Factor Meta-Analysis of Recent Extrasensory Perception Experiments"[0] is a good read in this area. It extends the analysis of psi with Bayes factors and underscores how the priors, and a causal mechanism, really matter. [0] http://deanradin.com/evidence/Rouder2013Bayes.pdf http://deanradin.com/evidence/Rouder2013Bayes.pdf
- Tosh108 11y agoInteresting. Derren Brown, English mentalist, describes in one of his books how he deliberately failes every now and then to make it more convincing there are pschycic abilities at play instead of a mechanic trick.
- themartorana 11y agoWell that's a convenient excuse. (Get it?)
- haruspex 11y agoH0 and H1 should always be phrased so that H0 is "not H1". In the above case that would be H1 "The psychic is not just lucky". Rejecting the data based on hypothesis testing is a misapplication of the technique. That doesn't mean that data is always right but it's just another hypothesis about the data itself, e.g. H0 "data is fine", H1 "data is not fine".