3 ms·

Looks coherent, informative and not too indulged with proving things with mathematical formulas. Problem is that the only way I could study it with the meticulo

by tekkk 8y ago

Looks coherent, informative and not too indulged with proving things with mathematical formulas. Problem is that the only way I could study it with the meticulousness it requires would be during a course. Studying it at my own leisure is a bit too much of a burden. However what irks me a little, as it did with my university's introductory machine learning course, is the direct jump to the algorithms while omitting explaining the basis for the underlying cause why it is the right thing to do. Perhaps it's presumed that the reader knows the basic statistics behind the idea that we can make assumptions of the data and predict values due to central limit theorem. Machine learning itself then can be seen as a derivate of that as more free-form, heuristic approach to cases where the central limit theorem can't be applied or is too restrictive. Am I speaking complete nonsense here? Please those wiser than me, tell me am I right?

SatvikBeri 8y ago
Inferential Statistics lets us make estimates about some quantity given a set of assumptions about how a sample was drawn. E.g., you might assume data is distributed normally. From a theoretical standpoint, Machine Learning basically just explores what happens if you remove that assumption, and try to draw conclusions that apply no matter what the underlying distribution was. The book does build up that theory. You need to understand the Probably Approximately Correct framework (e.g., how many data points do you need to be 90% sure your algorithm will generalize with less than 5% generalization error) and VC dimension (basically a measure of how much different algorithms overfit.) It doesn't even introduce an algorithm until chapter 8! VC dimension is sort of the equivalent of O(n) style complexity for machine learning. You can use this framework to estimate the effectiveness of different algorithms from a theoretical standpoint given a certain amount of data. For example, Linear Regression's VC dimension grows as O(n) with the number of features, while Decision Trees grow as O(2^n) with depth. This means that Trees are much more likely to overfit on small data, but more likely to be find complex structure in larger datasets.

by tekkk 8y ago

SatvikBeri 8y ago

Inferential Statistics lets us make estimates about some quantity given a set of assumptions about how a sample was drawn. E.g., you might assume data is distributed normally. From a theoretical standpoint, Machine Learning basically just explores what happens if you remove that assumption, and try to draw conclusions that apply no matter what the underlying distribution was. The book does build up that theory. You need to understand the Probably Approximately Correct framework (e.g., how many data points do you need to be 90% sure your algorithm will generalize with less than 5% generalization error) and VC dimension (basically a measure of how much different algorithms overfit.) It doesn't even introduce an algorithm until chapter 8! VC dimension is sort of the equivalent of O(n) style complexity for machine learning. You can use this framework to estimate the effectiveness of different algorithms from a theoretical standpoint given a certain amount of data. For example, Linear Regression's VC dimension grows as O(n) with the number of features, while Decision Trees grow as O(2^n) with depth. This means that Trees are much more likely to overfit on small data, but more likely to be find complex structure in larger datasets.