Probability Theory the Logic of Science is an excellent textbook on math and theory. Note the title even references cultus' point. Unfortunately it predates ma

Probability Theory the Logic of Science is an excellent textbook on math and theory. Note the title even references cultus' point. Unfortunately it predates many of the modern developments in methods / computation, but if you want to dive deep, I strongly recommend it. It takes the perspective of designing a reasoning robot to make the most effective decisions. Another resource, eg Stan's manual, can get you up and running on computation/inference. Your choice in computation tool should reflect the type and size of of problems you're interested in, and languages you're comfortable with. Stan has bindings for many scripting languages. R also offers Nimble, Python PyMC3 and Edward, and Julia has DynamicHMC and Turing. (EDIT: xcodevn has better Python recommendations: https://news.ycombinator.com/item?id=18213923 https://news.ycombinator.com/item?id=18213923 )