3 ms·

That chapter was very interesting... until I got to this: "Scientists can reach agreement quickly because we trust our experimental colleagues to have high sta

That chapter was very interesting... until I got to this: "Scientists can reach agreement quickly because we trust our experimental colleagues to have high standards of intellectual honesty and sharp perception to detect possible sources of error. And this belief is justiﬁed because, after all, hundreds of new experiments are reported every month, but only about once in a decade is an experiment reported that turns out later to have been wrong." Er.. what? http://www.plosmedicine.org/article/info%3Adoi%2F10.1371%2Fjournal.pmed.0020124 http://www.plosmedicine.org/article/info%3Adoi%2F10.1371%2Fj... "Why Most Published Research Findings Are False" (Ok, so Ioannidis' article was written a year later than the book, but that's a pretty nasty blow to the argument.)

mdpopescu 12y ago
A second problem... "As a simple, but numerically stronger example illustrating this, if we toss a coin 1000 times, then no matter what the result is, the speciﬁc observed sequence of heads and tails has a probability of only 2^−1000" No... if you toss a coin a thousand times, the probability of observing the exact sequence you just observed is 1. It will ALWAYS happen (that's what probability 1 means). Yes, if you had an independent specification for the sequence (like writing it down before tossing the coin, or converting the binary representation to ASCII and discovering it spells "Kilroy was here") then the probability would indeed be 2^-1000; but that would be a different case.
- dthunt 12y ago
  It's actually not a problem. You can come up with any number of hypotheses about coins. Some of them take the form, "This coin will produce <some specific output> in the next 1000 typical flips". That hypothesis and others with similar, more complex form, like the pair of hypotheses that predict flip 1001 after the same first 1000, GAIN CREDENCE when you perform 1000 flips that conform to them. Others of the similar form lose it. Other hypotheses of wildly different construction, like, that a coin is more or less fair, lose and gain credence according to whether or not they predict the observed result. The fact that you didn't write a hypothesis down before you did the test has very little to do with whether or not the data supports the hypothesis. Hindsight bias matters, but only as far as it corrupts your experience. The machine with the infinite library of coin-flip-hypotheses updates just fine. On a side note, coins are not fair, in general, and Jaynes actually goes into some detail about the process of cheating at coin flips.
- loup-vaillant 12y ago
  Obviously, Jaynes was talking about a reasonable prior probability you had before you flipped the coin. And he was talking about flipping the coin in a way that the head/tail outcome is completely unpredictable for you (50/50) probability, minus epsilon for weird stuff like landing on the edge. He later addresses the coin flip problem more rigorously. Besides, once you have observed the outcome, it's probability is not 1. You could have misremembered the sequence, or you could have missed a flip, or otherwise done your observation wrong. The probability of making at least one such error with 1000 coin flips in an informal setting is actually quite high.
loup-vaillant 12y ago
Jaynes was a physicist. If he was a biologist or a doctor, he would probably have written the paper you cite himself (medical research appears to be quite sensitive to crazy statistics).

mdpopescu 12y ago

A second problem... "As a simple, but numerically stronger example illustrating this, if we toss a coin 1000 times, then no matter what the result is, the speciﬁc observed sequence of heads and tails has a probability of only 2^−1000" No... if you toss a coin a thousand times, the probability of observing the exact sequence you just observed is 1. It will ALWAYS happen (that's what probability 1 means). Yes, if you had an independent specification for the sequence (like writing it down before tossing the coin, or converting the binary representation to ASCII and discovering it spells "Kilroy was here") then the probability would indeed be 2^-1000; but that would be a different case.

dthunt 12y ago
It's actually not a problem. You can come up with any number of hypotheses about coins. Some of them take the form, "This coin will produce <some specific output> in the next 1000 typical flips". That hypothesis and others with similar, more complex form, like the pair of hypotheses that predict flip 1001 after the same first 1000, GAIN CREDENCE when you perform 1000 flips that conform to them. Others of the similar form lose it. Other hypotheses of wildly different construction, like, that a coin is more or less fair, lose and gain credence according to whether or not they predict the observed result. The fact that you didn't write a hypothesis down before you did the test has very little to do with whether or not the data supports the hypothesis. Hindsight bias matters, but only as far as it corrupts your experience. The machine with the infinite library of coin-flip-hypotheses updates just fine. On a side note, coins are not fair, in general, and Jaynes actually goes into some detail about the process of cheating at coin flips.
loup-vaillant 12y ago
Obviously, Jaynes was talking about a reasonable prior probability you had before you flipped the coin. And he was talking about flipping the coin in a way that the head/tail outcome is completely unpredictable for you (50/50) probability, minus epsilon for weird stuff like landing on the edge. He later addresses the coin flip problem more rigorously. Besides, once you have observed the outcome, it's probability is not 1. You could have misremembered the sequence, or you could have missed a flip, or otherwise done your observation wrong. The probability of making at least one such error with 1000 coin flips in an informal setting is actually quite high.

loup-vaillant 12y ago

Jaynes was a physicist. If he was a biologist or a doctor, he would probably have written the paper you cite himself (medical research appears to be quite sensitive to crazy statistics).