9 ms·

Ask HN: Book which comprehensively covers probability theory?

53 points by abhikandoi2000 8y ago

My background is in computer science and I'd like to learn how probability theory is applied. I was reading the GraphSLAM paper to get a sense of the algorithms used for SLAM purposes in robots. While reading it, I realized that I have a tenuous grasp on probability theory, especially on topics like covariance, conditional probability and multivariate distributions (even things like what posterior probability represents). I'd like to rectify this and gain an intuitive understanding of the subject, since it is commonly used in numerous areas of engineering. I dislike books that introduce fully formed theorems with no derivation or proof of how they came into existence. Which comprehensive book(s) can I read?

NWChen 8y ago
Ch. 1-5 of Sheldon Ross's "A First Course in Probability" works for this purpose. Consider testing/hardening your knowledge with Mosteller's "Fifty Challenging Problems in Probability".
- abhikandoi2000 8y ago
  Thanks for this suggestion. It seems relevant. Ordered the book to read.
08-15 8y ago
E.T.Jaynes: "Probability Theory, The Logic Of Science"
- abhikandoi2000 8y ago
  I've started reading this book today. Looks quite comprehensive and replete with queer examples.
- probabill 8y ago
  Warning: Strongly opinionated and unconventional. Also doesn't comprehensively cover the topics you'd find in engineering papers. Still worth reading, but maybe not a good place to start.
  - abhikandoi2000 8y ago
    Point duly noted. Any suggestions?
papaf 8y ago
I recommend the probability chapters of 'Information Theory, Inference, and Learning Algorithms' [1]. I think this book is good because it starts off with balls in hats and goes on all the way to a Bayesian simulation of a Neuron and other subjects relevant for machine learning. It's available for free online, although most people I know end up buying the book[2]. [1] http://www.inference.org.uk/itila/ http://www.inference.org.uk/itila/ [2] http://www.inference.org.uk/itprnn/book.pdf http://www.inference.org.uk/itprnn/book.pdf
- yumraj 8y ago
  While clicking on the first link, came across this: http://www.inference.org.uk/itila/Potter.html http://www.inference.org.uk/itila/Potter.html Someone has a great sense of humor :)
rajekas 8y ago
Feller's two volume classic [1] has plenty of motivation - his exposition of combinatorics at the beginning of the first volume is a great introduction to that subject! However, it wasn't written with algorithms in mind. Venkatesh's more recent volume [2] is very well motivated and is better suited to modern engineering applications. Both have lots of exercises. [1] https://www.amazon.com/Introduction-Probability-Theory-Applications-Vol/dp/0471257087 https://www.amazon.com/Introduction-Probability-Theory-Appli... [2] https://www.amazon.com/Theory-Probability-Explorations-Applications/dp/1107024471 https://www.amazon.com/Theory-Probability-Explorations-Appli...
hackermailman 8y ago
https://www.youtube.com/playlist?list=PLm3J0oaFux3aafQm568blS9blxtA_EWQv https://www.youtube.com/playlist?list=PLm3J0oaFux3aafQm568bl... Some lectures on basic probability combined with some rigorous notes specific to CS http://www.cs.cmu.edu/~odonnell/papers/probability-and-computing-lecture-notes.pdf http://www.cs.cmu.edu/~odonnell/papers/probability-and-compu...
pedrodelfino 8y ago
I recommend "Introduction to Probability" by Joseph K Blitzstein and Jessica Hwang. This book is tough. Really tough. However, you are really going to do "deliberate practice" while having a go on the assignments. It bothers me that the author does not provide the answer to most of the questions. This is specially bad in this field of Mathematics (Probability). As an example, in calculus you can always plot the curve, the derivative and see if the result makes sense, in probability theory it is hard to have a simple and safe sanity check. You can watch all classes from Harvard here for free: https://www.youtube.com/watch?v=KbB0FjPg0mw&list=PL2SOU6wwxB0uwwH80KTQ6ht66KWxbzTIo https://www.youtube.com/watch?v=KbB0FjPg0mw&list=PL2SOU6wwxB... If you want, there is also a MOOC in EdX with the same material: https://www.edx.org/course/introduction-to-probability-0 https://www.edx.org/course/introduction-to-probability-0 Sheldon Ross' book is also a good one. It was mentioned here. I specially like the section on theoritical problems, only with exercises on proofs. Every chapter has one. However, Joe Blitzstein problems are more challenging and will train you more on intuition.
- pixelperfect 8y ago
  I second the recommendation for Blitzstein's book and lectures well as the frustration that more of the questions don't have solutions. I enjoyed reading it much more then Sheldon Ross' book, although both books have good problems.
Iwan-Zotow 8y ago
You could look at N.Zabaras courses on Statistical Computing https://www.zabaras.com/courses https://www.zabaras.com/courses I prefer 2017 version
wholien 8y ago
I've been working through "Probability - For the Enthusiastic Beginner" by David Morin. Very affordable for a textbook, clear instructions, solutions to exercises in the book is provided.

NWChen 8y ago

Ch. 1-5 of Sheldon Ross's "A First Course in Probability" works for this purpose. Consider testing/hardening your knowledge with Mosteller's "Fifty Challenging Problems in Probability".

abhikandoi2000 8y ago
Thanks for this suggestion. It seems relevant. Ordered the book to read.

08-15 8y ago

E.T.Jaynes: "Probability Theory, The Logic Of Science"

abhikandoi2000 8y ago
I've started reading this book today. Looks quite comprehensive and replete with queer examples.
probabill 8y ago
Warning: Strongly opinionated and unconventional. Also doesn't comprehensively cover the topics you'd find in engineering papers. Still worth reading, but maybe not a good place to start.
- abhikandoi2000 8y ago
  Point duly noted. Any suggestions?

papaf 8y ago

I recommend the probability chapters of 'Information Theory, Inference, and Learning Algorithms' [1]. I think this book is good because it starts off with balls in hats and goes on all the way to a Bayesian simulation of a Neuron and other subjects relevant for machine learning. It's available for free online, although most people I know end up buying the book[2]. [1] http://www.inference.org.uk/itila/ http://www.inference.org.uk/itila/ [2] http://www.inference.org.uk/itprnn/book.pdf http://www.inference.org.uk/itprnn/book.pdf

yumraj 8y ago
While clicking on the first link, came across this: http://www.inference.org.uk/itila/Potter.html http://www.inference.org.uk/itila/Potter.html Someone has a great sense of humor :)

rajekas 8y ago

Feller's two volume classic [1] has plenty of motivation - his exposition of combinatorics at the beginning of the first volume is a great introduction to that subject! However, it wasn't written with algorithms in mind. Venkatesh's more recent volume [2] is very well motivated and is better suited to modern engineering applications. Both have lots of exercises. [1] https://www.amazon.com/Introduction-Probability-Theory-Applications-Vol/dp/0471257087 https://www.amazon.com/Introduction-Probability-Theory-Appli... [2] https://www.amazon.com/Theory-Probability-Explorations-Applications/dp/1107024471 https://www.amazon.com/Theory-Probability-Explorations-Appli...

hackermailman 8y ago

https://www.youtube.com/playlist?list=PLm3J0oaFux3aafQm568blS9blxtA_EWQv https://www.youtube.com/playlist?list=PLm3J0oaFux3aafQm568bl... Some lectures on basic probability combined with some rigorous notes specific to CS http://www.cs.cmu.edu/~odonnell/papers/probability-and-computing-lecture-notes.pdf http://www.cs.cmu.edu/~odonnell/papers/probability-and-compu...

pedrodelfino 8y ago

I recommend "Introduction to Probability" by Joseph K Blitzstein and Jessica Hwang. This book is tough. Really tough. However, you are really going to do "deliberate practice" while having a go on the assignments. It bothers me that the author does not provide the answer to most of the questions. This is specially bad in this field of Mathematics (Probability). As an example, in calculus you can always plot the curve, the derivative and see if the result makes sense, in probability theory it is hard to have a simple and safe sanity check. You can watch all classes from Harvard here for free: https://www.youtube.com/watch?v=KbB0FjPg0mw&list=PL2SOU6wwxB0uwwH80KTQ6ht66KWxbzTIo https://www.youtube.com/watch?v=KbB0FjPg0mw&list=PL2SOU6wwxB... If you want, there is also a MOOC in EdX with the same material: https://www.edx.org/course/introduction-to-probability-0 https://www.edx.org/course/introduction-to-probability-0 Sheldon Ross' book is also a good one. It was mentioned here. I specially like the section on theoritical problems, only with exercises on proofs. Every chapter has one. However, Joe Blitzstein problems are more challenging and will train you more on intuition.

pixelperfect 8y ago
I second the recommendation for Blitzstein's book and lectures well as the frustration that more of the questions don't have solutions. I enjoyed reading it much more then Sheldon Ross' book, although both books have good problems.

Iwan-Zotow 8y ago

You could look at N.Zabaras courses on Statistical Computing https://www.zabaras.com/courses https://www.zabaras.com/courses I prefer 2017 version

wholien 8y ago

I've been working through "Probability - For the Enthusiastic Beginner" by David Morin. Very affordable for a textbook, clear instructions, solutions to exercises in the book is provided.