Inferential Statistics lets us make estimates about some quantity given a set of assumptions about how a sample was drawn. E.g., you might assume data is distri

Inferential Statistics lets us make estimates about some quantity given a set of assumptions about how a sample was drawn. E.g., you might assume data is distributed normally. From a theoretical standpoint, Machine Learning basically just explores what happens if you remove that assumption, and try to draw conclusions that apply no matter what the underlying distribution was. The book does build up that theory. You need to understand the Probably Approximately Correct framework (e.g., how many data points do you need to be 90% sure your algorithm will generalize with less than 5% generalization error) and VC dimension (basically a measure of how much different algorithms overfit.) It doesn't even introduce an algorithm until chapter 8! VC dimension is sort of the equivalent of O(n) style complexity for machine learning. You can use this framework to estimate the effectiveness of different algorithms from a theoretical standpoint given a certain amount of data. For example, Linear Regression's VC dimension grows as O(n) with the number of features, while Decision Trees grow as O(2^n) with depth. This means that Trees are much more likely to overfit on small data, but more likely to be find complex structure in larger datasets.