9 ms·
An Introduction to Statistical Learning with Applications in Python
- fensu 3y agoGood ol days of ML
- thumbuddy 3y agoI don't think most people realize this but the "old" stuff often works better, has less churn, and has far lower overhead costs for deployment than the "new" stuff. Depends on the domain and the goal.
- craigching 3y agoTo your point, I replaced an LSTM that required ~$100k of infrastructure with XGBoost that required no more infrastructure (we created and used the model at query time on existing infrastructure we already had for query loads) and only lost about 2% accuracy (LSTM: 98%, XGBoost: 96%). This was two years ago and it's still in use.
- randcraw 3y agoIt's also a LOT more explainable and interpretable.
- srvmshr 3y agoThis is an update to a very popular text which was originally in R. Professors Hastie & Tibshirani are leading educators in statistical learning. They also have a video course following these notes in Stanford Online. Very highly recommended if learning theoretical aspects of classical ML
- KRAKRISMOTT 3y agoAny plans for a Julia version?
- srvmshr 3y agoI actually like this book by Yoni Nazarathy https://statisticswithjulia.org/ https://statisticswithjulia.org/ They have a book on Mathematics of DL too which has several of the concepts covered here. (Disclosure: I am slightly biased towards this since I've known the author by online interactions)
- tndoan 3y agoHonestly, I dont think they will have Julia version soon. I converted all the code in this book to Julia sometimes ago: https://github.com/tndoan/ISLR.jl https://github.com/tndoan/ISLR.jl
- KRAKRISMOTT 3y agoAmazing work, any plans to update it for V2?
- deleted 3y ago[deleted]
- Exuma 3y agoFor anyone looking for the videos: https://www.youtube.com/playlist?list=PLoROMvodv4rOzrYsAxzQyHb8n_RWNuS1e https://www.youtube.com/playlist?list=PLoROMvodv4rOzrYsAxzQy...
- wklm 3y agois there a python variant of "The Elements of Statistical Learning"?
- disgruntledphd2 3y agoThere's no code in ESL so no Python version is necessary.
- wiz21c 3y agoI understand some may say "classical" ML but to me, those "few parameters" methods are very helpful in many cases and much easier to interpret than RNN :-)
- __rito__ 3y agoYou can download the whole book (legally) from here [pdf]: https://hastie.su.domains/ISLP/ISLP_website.pdf https://hastie.su.domains/ISLP/ISLP_website.pdf
- TheAlchemist 3y agoOff topic, but it's very interesting to observe the ratio of upvotes / comments. On any given chatGPT topic, there are hundreds of comments usually. Here, so far 100 upvotes, only 7 comments. The book looks great - and given the authors, it almost certainly is (I will buy it for sure). It makes me think though about the state of 'ML / AI / Data Science' - and the cynic part of me thinks that this upvotes / comments ratio kind of reflects the fact that most people interested in AI hype have not really touched a lot of underlying concepts and don't have any deeper understanding of maths / stats behind. PS. That being said, I didn't do a meaningful comment on the link topic neither.
- NeutralForest 3y agoI know about the book, I just think it's cool there's an actual Python version now so I upvote.
- __rito__ 3y agoWhat you are saying is true. This kind of books are key to getting started with Machine Learning/AI, and this particular book is a very good one. I started my ML journey with this book. There is a lot of hype around AI and it is going to be like the dotcom bubble. Unlike Crypto, AI has real uses right now and I am saying this not taking into account any LLM products. But there is also a lot of hype and wishful thinking, and this bubble is going to burst and hurt a lot of people. But that won't stop people making real money in the short term. Many hypers I know understand this well. But AI is here to stay. And even after the bubble bursts, there will be real uses of AI all around us.
- godelski 3y ago> and this bubble is going to burst and hurt a lot of people > But AI is here to stay. And even after the bubble bursts, there will be real uses of AI all around us. This is, as a ML researcher, exactly where I'm at (in belief). The utility is high, but so is the noise. Rather, the utility is sufficient. The danger of ML is not so much X-risk or malevolent AGI, but dumb ML being used inappropriately. And in general, that is using ML without understanding the limitations and having checks on it to ensure that when hallucinations happen that they don't cause major problems. But we're headed in a direction where we're becoming more reliant upon them and then once we have a few big issues with hallucinations the bubble will burst and can end up setting us back a lot in our progress to creating AGI. Previous winters were caused by lack of timely progress, but the next winter will happen because we shoot ourselves in the foot. Unfortunately, the more people you give guns to, the more likely this is to happen -- especially when there's no safety training or even acknowledgement of danger (or worse, the only discussion is about being shot by others).
- loehnsberg 3y agoISL is the best intro-level textbook of classic ML methods. It‘s theory-oriented yet simple enough to appeal to a wide audience of students (with basic knowledge in stats, linear algebra, and coding). Having the examples only in R was a pain when teaching with it while using Python. I hope they‘ll now turn this into a series of Jupyter notebooks and distribute it through Colab or similar.
- nemoniac 3y ago> I hope they‘ll now turn this into a series of Jupyter notebooks You can find them here: https://www.statlearning.com/resources-python https://www.statlearning.com/resources-python
- loehnsberg 3y agoThanks for sharing. That‘s exactly what I was looking for.
- isaacfung 3y agoChapter 13 comes in handy. Always wanted to learn the basics of testing but didn't want to take a full statistics course.
- semajian 3y agoA bit too wordy for me, I wish Landau was around to rewrite books like these, they'd be much shorter. Can we get a LLM to make it happen?
- rossdavidh 3y agoSure, as long as it's ok if it makes up stuff that isn't true a few times per chapter. :)
- graycat 3y agoMight consider, say, Sidney Siegel, N. John Castellan, Jr. 'Nonparametric Statistics for the Behavioral Sciences, Second Edition', ISBN 0-07-057357-3, McGraw-Hill, New York, 1988. So, "nonparametric" means make no assumptions about a probability distribution based on parameters. Or, call the material distribution-free. E.g., get to see about resampling plans -- tiny assumptions, really simple, darned cleaver, quite generally useful, especially appropriate for computing. Might use resampling to get more information from the data from "A - B" tests.
- laichzeit0 3y agoHuh? I have that book, and it's nothing like ISLR, at all. It's a good book, but ISLR covers topics such as gradient boosted trees, survival analysis, GLMs, etc. Nothing at all like the book you mentioned. If forced, you could say ISLR is more focused on prediction, not inference or hypothesis testing.
- graycat 3y agoFor statistical learning, the book I mentioned Nonparametric Statistics for the Behavioral Sciences, should make a good contribution to statistical learning. Some of the techniques are so robust, i.e., need such meager assumptions, that they should be especially welcome in automatically applied AI (artificial intelligence) applications. Actually the book ISLR, Introduction to Statistical Learning, does claim to cover "Resampling methods"
- defphysics 3y agoPersonally, I prefer Bishop (Pattern Recognition and Machine Learning) or Murphy (Probabilistic Machine Learning: An Introduction).
- dunefox 3y agoThose books are in a completely different ballpark.
- antegamisou 3y agoThose are what most universities around the world use for their introductory machine learning courses.
- dunefox 3y agoPress X to doubt.
- antegamisou 3y ago[1] CMU 10-701: Introduction to Machine Learning: Recommended Textbooks: Pattern Recognition and Machine Learning, Christopher Bishop Machine Learning: A probabilistic perspective, Kevin Murphy [2] University of Toronto CSC 311: Introduction to Machine Learning Suggested readings are optional; they are resources we recommend to help you understand the course material. All of the textbooks listed below are freely available online. Bishop = Pattern Recognition and Machine Learning, by Chris Bishop ESL = The Elements of Statistical Learning, by Hastie, Tibshirani, and Friedman. [3] EPFL CS-433 Machine Learning: Textbooks(not mandatory) Gilbert Strang, Linear Algebra and Learning from Data Christopher Bishop, Pattern Recognition and Machine Learning [4] University of Washington CSE 446: Machine Learning The required textbook for the course is: [Murphy] Machine Learning: A Probabilistic Perspective, Kevin Murphy. The following three texts are also excellent and their PDFs are available for free online. [B] Pattern Recognition and Machine Learning, Christopher Bishop. [HTF] The Elements of Statistical Learning: Data Mining, Inference, and Prediction, Trevor Hastie, Robert Tibshirani, Jerome Friedman. [5] Cornell University ECE4950: Machine Learning and Pattern Recognition Materials We will take materials from various sources. Some books are: Pattern Recognition and Machine Learning, Christopher Bishop Machine Learning: a Probabilistic Perspective, Kevin Murphy [6] Princeton University COS 324: Introduction to Machine Learning Optional Machine Learning Books [Murphy] Kevin Murphy, Machine Learning: A Probabilistic Perspective, MIT Press. [Bishop] Christopher M. Bishop, Pattern Recognition and Machine Learning, Springer. [7] ETH Zurich Introduction to Machine Learning (2023) Other Resources K. Murphy. Machine Learning: a Probabilistic Perspective. MIT Press, 2012. C. Bishop. Pattern Recognition and Machine Learning. Springer, 2007. [8] TUM (Technical University of Munich) Machine Learning This award-winning introductory Machine Learning lecture teaches the foundations of and concepts behind a wide range of common machine learning models. Literature Pattern Recognition and Machine Learning. Christopher Bishop. Springer-Verlag New York. 2006. Machine Learning: A Probabilistic Perspective. Kevin Murphy. MIT Press. 2012 [9] MIT Introduction To Machine Learning: Books: No textbook is required for this class, but students may find it helpful to purchase one of the following books. Bishop's book is much easier to read, whereas Murphy's book has substantially more depth and coverage (and is up to date). Machine Learning: a Probabilistic Perspective, by Kevin Murphy (2012). Pattern Recognition and Machine Learning, by Chris Bishop (2006). [10] UC Berkeley CS-194-10: Introduction to Machine Learning: Reading List (Preliminary Draft) The first two books are very helpful, and are available online, so those (in addition to AIMA) will be the primary sources. Bishop has a wide range of solid mathematical derivations, while Witten and Frank focus much more on the practical side of applied machine learning and on the Weka package (a Java library and interface for machine learning). Trevor Hastie, Rob Tibshirani, and Jerry Friedman, Elements of Statistical Learning, Second Edition, Springer, 2009. (Full pdf available for download.) Kevin P. Murphy, Machine Learning: A Probabilistic Perspective. Unpublished. Access information will be provided. Stuart Russell and Peter Norvig, Artificial Intelligence: A Modern Approach, Third Edition, Prentice Hall, 2010. Christopher Bishop, Pattern Recognition and Machine Learning, Springer, 2006. Ian Witten and Eibe Frank, Data Mining: Practical Machine Learning Tools and Techniques, Third Edition, Morgan Kaufmann, 2011. [1] https://www.cs.cmu.edu/~aarti/Class/10701_Spring21/ https://www.cs.cmu.edu/~aarti/Class/10701_Spring21/ [2] https://www.cs.toronto.edu/~rgrosse/courses/csc311_f21/ https://www.cs.toronto.edu/~rgrosse/courses/csc311_f21/ [3] https://www.epfl.ch/labs/mlo/machine-learning-cs-433/ https://www.epfl.ch/labs/mlo/machine-learning-cs-433/ [4] https://courses.cs.washington.edu/courses/cse446/23wi/textbooks/ https://courses.cs.washington.edu/courses/cse446/23wi/textbo... [5] https://people.ece.cornell.edu/acharya/teaching/ece4950s17/ece4950 https://people.ece.cornell.edu/acharya/teaching/ece4950s17/e... [6] https://www.cs.princeton.edu/courses/archive/spring19/cos324/ https://www.cs.princeton.edu/courses/archive/spring19/cos324... [7] https://las.inf.ethz.ch/teaching/introml-s23 https://las.inf.ethz.ch/teaching/introml-s23 [8] https://www.cs.cit.tum.de/en/daml/lehre/wintersemester-2022-23/machine-learning/ https://www.cs.cit.tum.de/en/daml/lehre/wintersemester-2022-... [9] https://people.csail.mit.edu/dsontag/courses/ml16/ https://people.csail.mit.edu/dsontag/courses/ml16/ [10] https://people.eecs.berkeley.edu/~russell/classes/cs194/f11/ https://people.eecs.berkeley.edu/~russell/classes/cs194/f11/
- cschmidt 3y agoThe python version is great news. I get asked fairly frequently to recommend an intro ML book. I would have suggested this, except they usually only knew python and not R. Now it a perfect first book!
- craigching 3y agoI used to introduce people new to machine learning with a python-converted version of ISL that I was developing. I never finished converting all of ISLR so this is very welcome!
- sabertoothed 3y agoHN hug of death. Download appears to be down.
- wodenokoto 3y agoI’ve been meaning to do a comparison of lab zero between the two. I’ve only had the chance to look over the Python lab for a few minutes, but compared to what I remember from the R labs, it is much, much more involved and longer. I know HN likes to complain about how difficult and confusing R is, but I think that it is an easier language for beginners or stat inclined people to start doing statistical work in. /two cents
- rossdavidh 3y agoPython is more natural for programmers learning statistics/ML. R is more natural for statisticians learning programming. Which is not too surprising, since those were the audiences each language was intended for. I think it's good to be accepting of both, and use the one that works better for a given task.
- verbify 3y agoI loved the youtube series that accompanied earlier versions of this book - https://www.youtube.com/watch?v=5N9V07EIfIg&list=PLOg0ngHtcqbPTlZzRHA2ocQZqB1D_qZ5V https://www.youtube.com/watch?v=5N9V07EIfIg&list=PLOg0ngHtcq... It can be watched without the book. The coding parts can be skipped. It has some insights missing in the book, and they've got an amazing mix of incredible technical talent and a great ability to distill and explain concepts.
- barrenko 3y agoIt's shocking how often it does not occur to me to check for video lectures of a great book.
- iaw 3y agoI've read through ISLR once and will likely go through it again at some point, this is a great text.
- kingkongjaffa 3y agoI have been using this and other resources to learn linear and logistic regression .