4 ms·

You can order the standard machine learning texts from most to least math-y, and least to most modern: - Pattern Recognition and Machine Learning (Bishop 2007)

by davmre 10y ago

You can order the standard machine learning texts from most to least math-y, and least to most modern: - Pattern Recognition and Machine Learning (Bishop 2007) - Machine Learning: A Probabilistic Perspective (Murphy 2012) - Deep Learning (Goodfellow, Bengio, Courville 2016) If you want cutting-edge material, read the Deep Learning book (which is still quite technical, though some of its content may be outdated in a few years). If you want timeless mathematical foundations very clearly presented, read Bishop. Murphy is a good middle ground. If you're self-teaching and have trouble focusing on a textbook for long periods, the Stanford CS229 lectures combined with Andrew Ng's course notes and assignments are probably the best resource. They are still quite rigorous and working through them will give you a solid foundation, after which you'll be more prepared to understand the deeper content in any of the texts above.

gtani 10y ago
Besides the DL book, there's other excellent texts that are freely available/open content on the web: - Elements statistical Learning, Hastie et al - Shalev-Shwartz and Ben-David: http://www.cs.huji.ac.il/~shais/UnderstandingMachineLearning/understanding-machine-learning-theory-algorithms.pdf http://www.cs.huji.ac.il/~shais/UnderstandingMachineLearning... - the late David MacKay's Info Theory - Bayesian Reasoning in ML, Barber - Hopcroft/Kannan (this is an older version, you can google latest: http://www.cs.cornell.edu/jeh/book112013.pdf http://www.cs.cornell.edu/jeh/book112013.pdf
- nilknarf 10y ago
  I didn't expect to see Hopcroft/Kannan in that list. I used it in a course taught by Kannan and back then it was called "Computer Science Theory for the Information Age". Apparently the book is now called "Foundations of Data Science". I always thought the old naming was terrible but compared to the new title it actually describes the content a lot better since the book is mainly about CS theory and mathematical foundations for them. It was one of my favorite courses but I would not classify it as ML book. [1]https://www.cs.cmu.edu/~venkatg/teaching/CStheory-infoage/ https://www.cs.cmu.edu/~venkatg/teaching/CStheory-infoage/ [2]https://www.cs.cornell.edu/jeh/book2016June9.pdf https://www.cs.cornell.edu/jeh/book2016June9.pdf
wjn0 10y ago
Where would you place "Elements of Statistical Learning" in relation to these, if you know?
- platz 10y ago
  You want ISLr (http://www-bcf.usc.edu/~gareth/ISL/ http://www-bcf.usc.edu/~gareth/ISL/), not ESL. ESL is the prototype for the former.
  - argonaut 10y ago
    Not sure why this is downvoted. ISLR is widely considered the "easier to read" / more pedagogical version of ESL. It's still mathematical but is closer to what the poster wants - a textbook that teaches them.
  - wjn0 10y ago
    Interesting, thanks - I wasn't aware of this text. But according to your link, ESL isn't a prototype of ISL, but a more 'advanced treatment'. The R applications in ISL seem like they might be very useful, though.
    - argonaut 10y ago
      ISLR was written after ESL to be an easier to read version. I guarantee you will have trouble if you try to teach yourself ML with ESL.
- davmre 10y ago
  I haven't read it in detail, but my impression is that it is mathy, like Bishop, but focuses more on 'classical' frequentist analysis, whereas Bishop takes a more open-ended Bayesian perspective and covers important machinery like graphical models and inference algorithms that I don't think are in ESL. As a researcher I tend to prefer the Bayesian perspective in Bishop, because it gives you a unifying framework for thinking about building your own models and learning algorithms. But lots of people seem to respect ESL and speak very highly about it. It's probably valuable if you are implementing one of the methods it covers and want to understand that specific method in great depth.
argonaut 10y ago
I hear people recommend textbooks a lot, and I honestly don't know why. Very few researchers I know learned machine learning through reading a textbook on their own. Furthermore, the first two textbooks are closer to reference books rather than actual pedagogical tutorials (I haven't read any single chapter in its entirety). The Deep Learning book assumes machine learning knowledge.
- Wazzymandias 10y ago
  What would you recommend instead for a beginner trying to get into ML?
  - cr0sh 10y ago
    I'll tell you how I started my journey: I took the Stanford ML Class in 2011 taught by Andrew Ng; ultimately, Coursera was born from it, and you can still find that class in their offerings: https://www.coursera.org/learn/machine-learning https://www.coursera.org/learn/machine-learning On a similar note, Udacity sprung up from the AI Class that ran at the same time (taught by Peter Norvig and Sebastian Thrun); Udacity has since added the class to their lineup (though at the time, they had trouble doing this - and so spawned the CS373 course): https://www.udacity.com/course/intro-to-artificial-intelligence--cs271 https://www.udacity.com/course/intro-to-artificial-intellige... https://www.udacity.com/course/artificial-intelligence-for-robotics--cs373 https://www.udacity.com/course/artificial-intelligence-for-r... I took the CS373 course later in 2012 (I had started the AI Class, but had to drop out due to personal issues at the time). Today I am currently taking Udacity's "Self-Driving Car Engineer" nanodegree program. But it all started with the ML Class. Prior to that, I had played around with things on my own, but nothing really made a whole lot of sense for me, because I lacked some of the basic insights, which the ML Class course gave to me. Primarily - and these are key (and if you don't have an idea about them, then you should study them first): 1. Machine learning uses a lot of tools based on and around probabilities and statistics. 2. Machine learning uses a good amount of linear algebra 3. Neural networks use a lot of matrix math (which is why they can be fast and scale - especially with GPUs and other multi-core systems) 4. If you want to go beyond the "black box" aspect of machine learning - brush up on your calculus (mainly derivatives). That last one is what I am currently struggling with and working through; while the course I am taking currently isn't stressing this part, I want to know more about what is going on "under the hood" so to speak. Right now, we are neck deep into learning TensorFlow (with Python); TensorFlow actually makes things pretty simple to create neural networks, but having the understanding of how forward and back-prop works (because in the ML Class we had to implement this using Octave - we didn't use a library) has been extremely helpful. Did I find the ML Class difficult? Yeah - I did. I hadn't touched linear algebra in 20+ years when I took the course, and I certainly hadn't any skills in probabilities (so, Kahn Academy and the like to the rescue). Even now, while things are a bit easier, I am still finding certain tasks and such challenging in this nanodegree course. But then, if you aren't challenged, you aren't learning.

by davmre 10y ago

gtani 10y ago

Besides the DL book, there's other excellent texts that are freely available/open content on the web: - Elements statistical Learning, Hastie et al - Shalev-Shwartz and Ben-David: http://www.cs.huji.ac.il/~shais/UnderstandingMachineLearning/understanding-machine-learning-theory-algorithms.pdf http://www.cs.huji.ac.il/~shais/UnderstandingMachineLearning... - the late David MacKay's Info Theory - Bayesian Reasoning in ML, Barber - Hopcroft/Kannan (this is an older version, you can google latest: http://www.cs.cornell.edu/jeh/book112013.pdf http://www.cs.cornell.edu/jeh/book112013.pdf

nilknarf 10y ago
I didn't expect to see Hopcroft/Kannan in that list. I used it in a course taught by Kannan and back then it was called "Computer Science Theory for the Information Age". Apparently the book is now called "Foundations of Data Science". I always thought the old naming was terrible but compared to the new title it actually describes the content a lot better since the book is mainly about CS theory and mathematical foundations for them. It was one of my favorite courses but I would not classify it as ML book. [1]https://www.cs.cmu.edu/~venkatg/teaching/CStheory-infoage/ https://www.cs.cmu.edu/~venkatg/teaching/CStheory-infoage/ [2]https://www.cs.cornell.edu/jeh/book2016June9.pdf https://www.cs.cornell.edu/jeh/book2016June9.pdf

wjn0 10y ago

Where would you place "Elements of Statistical Learning" in relation to these, if you know?

platz 10y ago
You want ISLr (http://www-bcf.usc.edu/~gareth/ISL/ http://www-bcf.usc.edu/~gareth/ISL/), not ESL. ESL is the prototype for the former.
- argonaut 10y ago
  Not sure why this is downvoted. ISLR is widely considered the "easier to read" / more pedagogical version of ESL. It's still mathematical but is closer to what the poster wants - a textbook that teaches them.
- wjn0 10y ago
  Interesting, thanks - I wasn't aware of this text. But according to your link, ESL isn't a prototype of ISL, but a more 'advanced treatment'. The R applications in ISL seem like they might be very useful, though.
  - argonaut 10y ago
    ISLR was written after ESL to be an easier to read version. I guarantee you will have trouble if you try to teach yourself ML with ESL.
davmre 10y ago
I haven't read it in detail, but my impression is that it is mathy, like Bishop, but focuses more on 'classical' frequentist analysis, whereas Bishop takes a more open-ended Bayesian perspective and covers important machinery like graphical models and inference algorithms that I don't think are in ESL. As a researcher I tend to prefer the Bayesian perspective in Bishop, because it gives you a unifying framework for thinking about building your own models and learning algorithms. But lots of people seem to respect ESL and speak very highly about it. It's probably valuable if you are implementing one of the methods it covers and want to understand that specific method in great depth.

argonaut 10y ago

I hear people recommend textbooks a lot, and I honestly don't know why. Very few researchers I know learned machine learning through reading a textbook on their own. Furthermore, the first two textbooks are closer to reference books rather than actual pedagogical tutorials (I haven't read any single chapter in its entirety). The Deep Learning book assumes machine learning knowledge.

Wazzymandias 10y ago
What would you recommend instead for a beginner trying to get into ML?
- cr0sh 10y ago
  I'll tell you how I started my journey: I took the Stanford ML Class in 2011 taught by Andrew Ng; ultimately, Coursera was born from it, and you can still find that class in their offerings: https://www.coursera.org/learn/machine-learning https://www.coursera.org/learn/machine-learning On a similar note, Udacity sprung up from the AI Class that ran at the same time (taught by Peter Norvig and Sebastian Thrun); Udacity has since added the class to their lineup (though at the time, they had trouble doing this - and so spawned the CS373 course): https://www.udacity.com/course/intro-to-artificial-intelligence--cs271 https://www.udacity.com/course/intro-to-artificial-intellige... https://www.udacity.com/course/artificial-intelligence-for-robotics--cs373 https://www.udacity.com/course/artificial-intelligence-for-r... I took the CS373 course later in 2012 (I had started the AI Class, but had to drop out due to personal issues at the time). Today I am currently taking Udacity's "Self-Driving Car Engineer" nanodegree program. But it all started with the ML Class. Prior to that, I had played around with things on my own, but nothing really made a whole lot of sense for me, because I lacked some of the basic insights, which the ML Class course gave to me. Primarily - and these are key (and if you don't have an idea about them, then you should study them first): 1. Machine learning uses a lot of tools based on and around probabilities and statistics. 2. Machine learning uses a good amount of linear algebra 3. Neural networks use a lot of matrix math (which is why they can be fast and scale - especially with GPUs and other multi-core systems) 4. If you want to go beyond the "black box" aspect of machine learning - brush up on your calculus (mainly derivatives). That last one is what I am currently struggling with and working through; while the course I am taking currently isn't stressing this part, I want to know more about what is going on "under the hood" so to speak. Right now, we are neck deep into learning TensorFlow (with Python); TensorFlow actually makes things pretty simple to create neural networks, but having the understanding of how forward and back-prop works (because in the ML Class we had to implement this using Octave - we didn't use a library) has been extremely helpful. Did I find the ML Class difficult? Yeah - I did. I hadn't touched linear algebra in 20+ years when I took the course, and I certainly hadn't any skills in probabilities (so, Kahn Academy and the like to the rescue). Even now, while things are a bit easier, I am still finding certain tasks and such challenging in this nanodegree course. But then, if you aren't challenged, you aren't learning.