8 ms·
The Elements of Statistical Learning [pdf]
- iamacyborg 6y agoI have the paper version of this but it goes way over my head. What should I read first to make sense of this?
- evanpw 6y ago"An Introduction to Statistical Learning" was written by two of the same authors, and is explicitly meant to be a lower-level introduction to the same ideas: https://statlearning.com/ISLR%20Seventh%20Printing.pdf https://statlearning.com/ISLR%20Seventh%20Printing.pdf
- iamacyborg 6y agoThank you!
- credit_guy 6y agoTry "Computer Age Statistical Inference" by Efron and Hastie. https://web.stanford.edu/~hastie/CASI_files/PDF/casi.pdf https://web.stanford.edu/~hastie/CASI_files/PDF/casi.pdf
- erdos4d 6y agoRather than a lower level intro, I'd recommend you bone up on linear algebra and multidimensional/vector calculus. The ideas in this book are not especially difficult to get but the math is right up front all the way through. If you can't actually read the equations and visualize what is being discussed, it will be quite difficult to ever make real progress with this book.
- vowelless 6y agoProbably the most recommended stats book out there. And for a reason. Certainly not an intro book. But, at some point, anyone interested in stats, ML, should go through it once.
- phonebucket 6y agoOf interest to people who like the look of this book is Bishop's Pattern Recognition and Machine Learning, also available freely and legally online: https://www.microsoft.com/en-us/research/people/cmbishop/ https://www.microsoft.com/en-us/research/people/cmbishop/
- blackbear_ 6y agoAnd to close the triad of machine learning bibles, don't forget Murphy's, which will apparently be extended soon! https://probml.github.io/pml-book/ https://probml.github.io/pml-book/
- bonoboTP 6y agoI also like Alpaydin's Intro to ML even though it's not as famous: https://mitpress.mit.edu/books/introduction-machine-learning-fourth-edition https://mitpress.mit.edu/books/introduction-machine-learning...
- st1x7 6y agoSo much of that book just goes over my head while I didn't have that problem with ESL. I don't know if it's Murphy's writing style or just the way he approaches the topic but I found his book significantly more difficult to process.
- 6y ago
- person_of_color 6y agoWhat is the equivalent book for Deep Learning?
- phonebucket 6y agoTo my mind: https://www.deeplearningbook.org/ https://www.deeplearningbook.org/
- blackbear_ 6y agoGoodfellow's [1] comes closest I'd say, but since deep learning is still moving very fast and lacks a solid formal grounding (unlike most other machine learning methods) it's not so easy to find a comprehensive book yet. [1] https://www.deeplearningbook.org/ https://www.deeplearningbook.org/
- mrfusion 6y agoWill that teach me transformers too?
- nmfisher 6y agoNo (at least, not the version I read a few years ago). But transformers are a specific neural network architecture, so I’d still recommend the book for the fundamentals around backpropagation, activation & loss functions, etc. Once you’re comfortable with neural networks (and the notation), the “Attention is all you need” paper is fairly accessible.
- mrfusion 6y ago> Once you’re comfortable with neural networks (and the notation) I think I am actually. And I read attention is all you need. (Well half of it) and it didn’t seem to delve into how they work.
- blackbear_ 6y agoIt really does, read section 3 carefully. As for why they work so well, that's still an open question.
- jll29 6y agoNot quite finished yet but coming soon: Speech & Language Processing (3rd ed.) https://web.stanford.edu/~jurafsky/slp3/ https://web.stanford.edu/~jurafsky/slp3/
- michaericalribo 6y agoThis is a great reference, but it’s pretty terse...it would be hard (IMO) to learn something for the first time from EoSL, though it’s great for having all the details and derivations. It’s just a very technical book, and though it is complete it is also quite inaccessible.
- thegginthesky 6y agoThat's why they wrote Introduction to Statistical Learning[0] and also a video series for the same book[1]. Both books and the video classes are a must for anyone working with Machine Learning and/or Statistics. [0] http://faculty.marshall.usc.edu/gareth-james/ISL/ http://faculty.marshall.usc.edu/gareth-james/ISL/ [1] https://www.youtube.com/watch?v=5N9V07EIfIg&list=PLOg0ngHtcqbPTlZzRHA2ocQZqB1D_qZ5V https://www.youtube.com/watch?v=5N9V07EIfIg&list=PLOg0ngHtcq...
- michaericalribo 6y agoDefinitely. And hell, EoSL is a _wide_ survey, and some of the content is only adjacent to ML. Eg, MCMC for Bayesian models: there are good applications in ML, and a short bit in EoSL about the Gibbs sampler, but there’s a massive parallel literature in the statistical inference world.
- jimsparkman 6y ago[0] returned a 404, but this appears to be substitute: https://statlearning.com/ https://statlearning.com/
- thegginthesky 6y agoHmm weird, it works for me. Thanks for the alternative link
- LaserPineapple 6y agoAnd a second edition of ISLR is "coming soon" [1]. [1] https://twitter.com/daniela_witten/status/1261693624439279616?s=19 https://twitter.com/daniela_witten/status/126169362443927961...
- madiyar 6y agoThis collection of Jupyter notebooks that reproduces graphics and implements algorithms from the book could be a nice supplementary resource. https://github.com/maitbayev/the-elements-of-statistical-learning https://github.com/maitbayev/the-elements-of-statistical-lea...
- bumby 6y agoDang, I was really hoping to find examples of the MCMC methods in Ch. 8. A strong point in the "Introduction to Statistical Learning" by the authors is that each chapter ends with example programs in R (albeit with a fair number of typos).
- scythmic_waves 6y agoTo echo some of the other comments here, this is a text that's really only appropriate for those who already have a graduate-level grasp of statistics. I love that it's freely available, but ESL is not an introductory text. For example, here's a screenshot from the introductory chapter (pg. 26): [1]. The authors expect you to already be familiar with matrix analysis applied to statistics. An Introduction to Statistical Learning (ISL) [2] is aimed at those with a high school level of math. [1] https://imgur.com/q0NeqdR https://imgur.com/q0NeqdR [2] https://statlearning.com/book.html https://statlearning.com/book.html
- melling 6y agoI’m up to Chapter 6 in ISLR https://github.com/melling/ISLR https://github.com/melling/ISLR Would Elements of Statistical Learning be my next book? I’ve seen the Bishop book highly recommended too, and it has been mentioned in this post. https://www.amazon.com/Pattern-Recognition-Learning-Information-Statistics/dp/0387310738 https://www.amazon.com/Pattern-Recognition-Learning-Informat...
- scythmic_waves 6y ago> Would Elements of Statistical Learning be my next book? Honestly no I don't think so. ESL is likely too advanced. I would use that screenshot I posted above as a litmus test. Do you understand that notation? The `E` with the subscript? And why they're using `trace[]`? If you do, then you can likely follow ESL. If not -- which would be understandable because even early undergrads likely can't -- then I say you shouldn't try and follow ISL up directly with ESL. It really is a graduate text. > I’ve seen the Bishop book highly recommended too, and it has been mentioned in this post. Bishop has a similar problem: [1]. I had to scroll to chapter 2 for this screenshot (pg. 83), but it really is par for the course. So, and this is totally my opinion here so YMMV, recommendations for foundational ML info tend to be wildly too advanced for the people seeking them out. I'm a math-y person. I really like learning about the math foundations of ML. But ML builds on a lot of other concepts and you can't just jump into the deep end. In my opinion, ML foundations should come at the end of a lengthy sequence of math and statistics courses. Students will just be too lost without them. I don't mean to be discouraging here. I think nearly anyone who's willing to put in the time can learn this stuff! But here's a more reasonable sequence I found on reddit a while back that would set someone up nicely for being able to follow ESL: [2]. Without the proper foundation, it's just too difficult to follow ESL or Bishop IMO. Last, I'll note that you don't need to understand the nitty-gritty of ML math to be an ML practitioner. In fact, I'd argue that taking the effort would be distracting because 1) a basic understanding (like you'd get from working through ISL) is probably good enough to start messing with libraries and 2) practitioners need a whole bunch of other knowledge (like general software skills and how to maintain ML datasets) that they also have to take the time to learn. [1] https://imgur.com/uXWZ6Bv https://imgur.com/uXWZ6Bv [2] https://www.reddit.com/r/learnmachinelearning/comments/ggpzk2/a_comprehensive_selfstudy_guide_for_the_math/ https://www.reddit.com/r/learnmachinelearning/comments/ggpzk...
- maximus411 6y agoThis page provides many step-by-step examples in both R and Python of the topics outlined in the ESL: https://www.statology.org/machine-learning-tutorials/ https://www.statology.org/machine-learning-tutorials/
- sischoel 6y agoIs there some survey/list somewhere which of these topics are still relevant today, might have a comeback in the future or are definitely a thing of the past? A lot of these topics where somehow mentioned in some of my ml courses at university but my professors never really bothered to put them in a bigger picture.
- the_duck 6y agoThe authors of this book also offer an excellent free online course that covers some of the same material. I found it incredibly well presented and easy to follow, without being superficial. https://online.stanford.edu/courses/sohs-ystatslearning-statistical-learning https://online.stanford.edu/courses/sohs-ystatslearning-stat...
- deleted 6y ago[deleted]