7 ms·
A History of Bayes' Theorem
- T_S_ 15y agoIf you find the conflict around frequentist and bayesian methods entertaining check out Jaynes' Theory of Probability, which profiles lots of great arguments and personality profiles. Also you might learn a few things about maxent, statistical physics, and my favorite: the mind-projection fallacy.
- aaronblohowiak 15y agoI can't find the book with that title, do you have a link or ISBN? http://www.amazon.com/Probability-Theory-Science-T-Jaynes/dp/0521592712/ref=ntt_at_ep_dpt_1 http://www.amazon.com/Probability-Theory-Science-T-Jaynes/dp... is the closest match but it does not seem to match your description.
- T_S_ 15y agoThanks and sorry--I misrecollected. The correct title is Probability Theory: The Logic of Science.
- jcarden 15y agoI've got a digital copy around here somewhere. Send me a message if you want.
- revorad 15y agoNot the full versions but still good: http://bayes.wustl.edu/etj/prob/book.pdf http://bayes.wustl.edu/etj/prob/book.pdf http://www-biba.inrialpes.fr/Jaynes/prob.html http://www-biba.inrialpes.fr/Jaynes/prob.html
- capnrefsmmat 15y agoAt what level is the presentation of statistics in his book? I'm interested to learn more about the application of statistics and Bayesian methods (as a physics major I intend to deal with lots of data), but I haven't taken formal courses on statistics yet. (Of course, being a physics major, I don't mind having to hurt my brain a bit to get through it, so long as it's good.)
- snippyhollow 15y agoBasic conditional probability understanding is sufficient.
- Jach 15y agoI've recently started going through http://www.amazon.com/Data-Analysis-Bayesian-Devinderjit-Sivia/dp/0198568320/ http://www.amazon.com/Data-Analysis-Bayesian-Devinderjit-Siv... and can highly recommend it. The benefits of Bayesian reasoning can be grasped somewhat easily without a detailed knowledge of the math, but at the end of the day scientists and engineers still need to learn how to use it and write programs using it to do stuff! One review mentions the book dives right into the subject, it does; I believe Bayes' Theorem is shown on page 6. Some people find a brisk pace hard to follow, personally I like it since I can always fill in the gaps with other material if needed. (Like Jaynes, or http://uncertainty.stat.cmu.edu/ http://uncertainty.stat.cmu.edu/ )
- chrismealy 15y agoWhy is Bayes' theorem such a magnet for cranks? Is it because you make your prior anything and pretend it's partially true?
- russellallen 15y agoI don't know, but it does seem to be sometimes evangelised with more fervor than I'm comfortable with. Us v Them is always a suspicious narrative. Even in this article, stuff like: Finally, in 1983 the US Air Force sponsored a review of NASA's estimates of the probability of shuttle failure. NASA's estimate was 1 in 100,000. The contractor used Bayes and estimated the odds of rocket booster failure at 1 in 35. In 1986, Challenger exploded. makes me uneasy. The rhetoric is too clearly marked.
- wolf550e 15y agoThe "1 in 100,000" silliness was noticed by Feynman and has nothing to do with Bayes, it was just PHB-thinking. http://en.wikipedia.org/wiki/Rogers_Commission_Report#Role_of_Richard_Feynman http://en.wikipedia.org/wiki/Rogers_Commission_Report#Role_o...
- dagw 15y agoYea, that has nothing to do with Bayesian vs frequentist, and everything with massaging numbers to get the results that where expedient. I'm sure that the people at NASA would have been plenty smart enough to work out how to get the 1 in 100,000 result using Bayesian analysis if they really wanted to.
- taliesinb 15y agoThis made me laugh, because I'm working on a blogpost about how one can use Bayes to argue that humanity is about to go extinct. Seriously.
- gwern 15y ago'Seriously'?That seems plausible to me. It's not like nukes, asteroids, biowarfare, supervolcanoes, etc. have simply gone away.
- equark 15y agoWhile we're on the top of Bayes, does any Berkeley grad have a copy of Michael Jordan's forthcoming book?
- tylerneylon 15y agoIt has always bothered me that Baye's theorem is a named result, instead of just "the law of conditional probability," say. As soon as you formally define conditional probability, it is obvious. To be fair, though, I think he discovered it before probability theory was formalized, and I can see how it may have been non-obvious at the time. Sometimes the best progress makes challenging insights appear obvious in hindsight - nice work, Laplace!
- jorangreef 15y agoIt's interesting that the Reverend Bayes was resisted for so long, and not on objective grounds. We seem to have this tendency, even amongst the academic elite, to not always have the most honest of mental apparatus, especially where the matter at hand would require us to reverse position, or acknowledge more our ignorance, than our ability.
- ajuc 15y agoNot really the point of the article, but it always unnerves me, when people talk about Turing breaking Enigma, and not mention whole effort ( http://en.wikipedia.org/wiki/Cryptanalysis_of_the_Enigma http://en.wikipedia.org/wiki/Cryptanalysis_of_the_Enigma ). Enigma was broken by Polish mathemathicians in 1938, then improved by Germans, then broken again, Poles had given all their discoveries to France and to Great Brittain, they also invented first "Bomba" to that worked mechanically. It's complicated story, and these people deserve credit along with Turing. But till 1970 it was classified information, and to this day everybody talks only about Turing. Maybe it's my nationalism bias at work, but I'd prefer if the whole story was better known.
- jgrahamc 15y agoI agree. Also this article only makes it worse. It claims that (a) no one was working on Enigma before Turing and (b) Turing worked on it because he 'liked solitude'. WTF? Given that total bollocks in that part of the article I have a hard time reading the rest. Note that Bletchley Park does a very good job of talking about the Polish contribution to the cryptanalytic effort with examples of how the Polish attacks on Enigma were performed and a Polish memorial and they have a special annual Polish day. Also, if the author wanted to talk about Bayes Theorem and the WWII cryptanalytic effort there's a much nicer example from the Japanese codes: http://blog.jgc.org/2010/07/bayes-bletchley-jn-25-and-modern.html http://blog.jgc.org/2010/07/bayes-bletchley-jn-25-and-modern...
- JacobAldridge 15y agoFor those (possibly a small number on HN) not aware - I first encountered Bayes' Theorem via pg's articles on Spam - http://www.paulgraham.com/antispam.html http://www.paulgraham.com/antispam.html
- cellurl 15y agoI met Rudolph Kalman once...
- olalonde 15y agoIf you find the explanation too math heavy, this one might make more sense: http://commonsenseatheism.com/?p=13156 http://commonsenseatheism.com/?p=13156