4 ms·
The list is decent, but not exactly original. For people w/ a physics background, I would still recommend https://www.inference.org.uk/itprnn/book.pdf https://
by cdavid 6y ago
The list is decent, but not exactly original.
For people w/ a physics background, I would still recommend https://www.inference.org.uk/itprnn/book.pdf https://www.inference.org.uk/itprnn/book.pdf. Some of it is a bit obsolete, but then DL made a lot of stuff around generalization/overfitting somehow obsolete. It makes a lot of connection between different kind of approaches in ML, information theory, (Bayesian) statistics, and physics.
It is not a very good book if you only care about applications (in which case the Keras book, for beginner, or fastai/etc. are much more appropriate).
- activatedgeek 6y agoDavid MacKay was an absolute rockstar and this book is grossly underrated among beginners in machine learning. This should be THE complementary reference for anyone who uses the Bishop book. Both these books follow a philosophy which some people may not completely agree with, that from the "church of Bayes". My guess is that information theory went through its hype phase and much of the ideas are so pervasive across real systems that people forget how important those connections are. To tell you its importance, skim this work on information-theoretic probing [1]. I find this so satisfying. Its most famous alternative, linear probing, always felt inelegant. This paper experimentally shows how terribly linear probing fails. [1] https://arxiv.org/abs/2003.12298 https://arxiv.org/abs/2003.12298