2 ms·
I have to agree with both points you make, but especially the second. The selection of topics, and their ordering, is very weird. Like why is there a whole chap
by mturmon 4y ago
I have to agree with both points you make, but especially the second. The selection of topics, and their ordering, is very weird. Like why is there a whole chapter (#4) on Markov Chains, sandwiched between a chapter on the SVD (#3) and an overview of Machine Learning (#5)? There has to be another way!
Most of the topics covered are interesting in their own way, but the pieces just do not add up -- it has the flavor of "every author contributes a chapter, ordered in round-robin style".
In particular: opening up with a whole chapter on the odd properties of high-dimensional spaces ("all the mass is at the edge", etc.) is interesting, but it doesn't work pedagogically. I've heard various talks, over the years, that present this family of results -- and it's nice to see so many of them collected here. But not as an introduction.
I'm doubtful that the subject of high-dimensional geometry (interesting though it is, our intuition in #dims = 3 does not scale correctly) provides frequent insight into why algorithms work or fail. (Happy to hear notable counterexamples on this.)
Name your favorite ML surprise: that weak learners can be converted into strong learners by boosting, that ANNs with 10^6 parameters can be trained with 10^4 data, that auto-encoder architectures work so well. If Chapter 1 is trying to shock the reader out of their complacency, that's probably where to start.
Also, this sentence from the last paragraph of the Introduction does not inspire confidence: "The term “almost surely” means with probability tending to one." Oops, that would be "equal to one."
- sfpotter 4y agoCouldn't agree more. Your comment about the weirdness of high-dimensional spaces is funny. I think this sort of stuff gets deployed for its high "Gee whiz!" factor. Not that it isn't interesting in its own right, but I fail to see what kind of a purchase it's going to give you on "data science"... The one example I know of re: insight is L1 regularization. The "spikiness" of the L1 unit ball in R^n for large n can be thought of as encouraging sparsity when using the L1 norm to regularize optimization problems.