7 ms·

Understanding Machine Learning: From Theory to Algorithms (2014)

202 points by subnaught 11y ago

enlightenedfool 11y ago
Glad to see another book to learn from for free. But my problem now is that there are so many books each with somewhat different approach and content for the same ML techniques. Not necessarily bad, but I get somewhat confused when trying to apply a method. EDIT: I guess the focus on the theory might help me.
Merkur 11y ago
great! thanx!
therobot24 11y ago
over 30 chapters and the only reference to graphical models is naive bayes and EM
wodenokoto 11y ago
It's pretty cool that the book is not only free, but they link to courses that uses it. If you speak Hebrew you can get two different professors take on how to teach the material in the book, as well as lecture notes from a total of 3 professors. That's pretty neat if there's a concept you are struggling with as a student!
michaelsbradley 11y ago
There's also the freely-accessible book A Course in Machine Learning: http://ciml.info/ http://ciml.info/
cdnsteve 11y ago
I feel like the barrier to machine learning for me, as I've seen in many tutorials and books and is an immediate discouragement, is the massive amount of math thrown in your face. Many of us didn't just graduate, need glasses and fall asleep at 8pm on the couch when the kids are in bed... Math is this distant fragment of memory buried under years of everything not Math. It feels like machine learning is only taught by academia but the majority of the audience is for practical use by the average developer wanting to play with it today.
- Omnipresent 11y ago
  I asked a related question [0] few weeks ago but didn't find a resource that talks about machine learning from real world example perspective. [0] https://news.ycombinator.com/item?id=10356874 https://news.ycombinator.com/item?id=10356874
- alfiedotwtf 11y ago
  The problem is that machine learning is applied probability and statistics. If you're interested, here's a gentle book that should give you enough background: http://www.amazon.com/Modern-Introduction-Probability-Statistics-Springer-ebook/dp/B00DZ0PKLG/ http://www.amazon.com/Modern-Introduction-Probability-Statis...
- davmre 11y ago
  Andrew Ng's Coursera ML course is supposed to be pretty accessible. I've also heard good things about Machine Learning for Hackers (http://www.amazon.com/Machine-Learning-Hackers-Drew-Conway/dp/1449303714 http://www.amazon.com/Machine-Learning-Hackers-Drew-Conway/d...). Ultimately, ML is a mathematical discipline. You can ask for a gentle approach that gets you to the foot of the mountain, but "if you want to learn about nature, to appreciate nature, it is necessary to understand the language that she speaks in." If you want to be more than an amateur, there's not much substitute for getting comfortable with math at the level of, say, Kevin Murphy's book. The good news is that the required math is fairly elementary - calculus, linear algebra, probability and statistics, all freshmen or maybe sophomore-level topics - so it shouldn't be beyond reach of a motivated developer able to set aside some time to learn. MOOCs and organizing study groups with friends/co-workers can help a lot here as well.
  - fapjacks 11y ago
    Yeah I'd like to second this. This course specifically is what opened the door for me.
  - subnaught 11y ago
    I'm currently taking Andrew Ng's Coursera course and I'd agree it's quite accessible. In fact, if you have a solid understanding of calculus and linear algebra, you might find it a bit slow at times.
Omnipresent 11y ago
Related: Foundations of Data Science: http://www.cs.cornell.edu/jeh/nosolutions90413.pdf http://www.cs.cornell.edu/jeh/nosolutions90413.pdf
alvern 11y ago
My question for someone that has an intermediate level of skill in machine learning, what's the best way to dip your toes in? (Udacity, coursera, edx, PDFs, Talking Machines podcast, etc)
- zintinio4 11y ago
  Find a problem to work on in a domain you find interesting. By reading published papers and trying to attack the problem, you'll be forced to pick up a lot of other knowledge not commonly discussed like: feature extraction and selection, dimensionality reduction, dealing with sparsity, common metrics for that problem, recent work, etc. I was forced to learn a massive amount in a short period of time for work, but I'd previously watched Andrew Ng's lectures, as well as majored in Math/CS. I can also generally recommend Hinton's NN lectures, Socher's Deep learning for NLP, Andrew Ng's Machine Learning, and a few books.
pinn42 11y ago
I'm creating general intelligence in Java multicore.
stevenmays 11y ago
I am considering taking Udacity's machine learning nanodegree [0] with zero machine learning background. It seems interesting. Any thoughts? [0] https://www.udacity.com/course/machine-learning-engineer-nanodegree--nd009 https://www.udacity.com/course/machine-learning-engineer-nan...
jjaredsimpson 11y ago
Top comment in another thread whines that nobody understands what the machinery of ML algorithms are really doing. Top comment here whines that math is hard.
pmelendez 11y ago
Just a curiosity: One of the authors also proposed Pegasos SVM [1] which is a nice online approximation to SVM and that can be written in 15 lines of code or so. http://ttic.uchicago.edu/~nati/Publications/PegasosMPB.pdf http://ttic.uchicago.edu/~nati/Publications/PegasosMPB.pdf
arbitrage314 11y ago
I'm a math geek, but I'm also a mostly self-taught data scientist. "The Elements of Statistical Learning" (https://web.stanford.edu/~hastie/local.ftp/Springer/OLD/ESLII_print4.pdf https://web.stanford.edu/~hastie/local.ftp/Springer/OLD/ESLI...) is far and away the best book I've seen. It took me hundreds of hours to get through it, but if you're looking to understand things at a pretty deep level, I'd say it's well-worth it. Even if you stop at chapter 3, you'll still know more than most people, and you'll have a great foundation. Hope this helps!
- gajjanag 11y ago
  I am a graduate student at MIT, and can second this recommendation. It is a fantastic book for machine learning and nothing else I have seen comes close.
  - nafizh 11y ago
    You meant ESL or UML?
    - lakeeffect 11y ago
      ESL. His post was an hour before the reader5000 uml post.
- reader5000 11y ago
  Having read significant chunks of both ESL and Understanding Machine Learning (albeit UML much more recently) I would argue that for many readers UML is superior. ESL pays short shrift to the computational complexity of learning whereas UML explicitly handles both statistical and computational complexity concerns. It doesnt matter how statistically pure your algorithm is if its running time scales exponentially with your data. All of UML's chapters are conceptually unified even when discussing different ML algorithms, with ESL being more of a grab-bag by chapter. Still, both high quality and free!
  - arbitrage314 11y ago
sagik 11y ago
Took the course at the Hebrew University. Awesome course. Here are the slides (English) and videos(Hebrew): http://www.cs.huji.ac.il/~shais/IML2014.html http://www.cs.huji.ac.il/~shais/IML2014.html
inglor 11y ago
I've read this book and warmly recommend it. It has a very pragmatic "no bullshit" approach and it's very mathematical and concise. The neural networks chapter is tiny (but that's ok - that's not the focus) and some of the questions are really hard - but overall I've really enjoyed it.

202 points by subnaught 11y ago

Glad to see another book to learn from for free. But my problem now is that there are so many books each with somewhat different approach and content for the same ML techniques. Not necessarily bad, but I get somewhat confused when trying to apply a method. EDIT: I guess the focus on the theory might help me.

Merkur 11y ago

great! thanx!

therobot24 11y ago

over 30 chapters and the only reference to graphical models is naive bayes and EM

wodenokoto 11y ago

It's pretty cool that the book is not only free, but they link to courses that uses it. If you speak Hebrew you can get two different professors take on how to teach the material in the book, as well as lecture notes from a total of 3 professors. That's pretty neat if there's a concept you are struggling with as a student!

michaelsbradley 11y ago

There's also the freely-accessible book A Course in Machine Learning: http://ciml.info/ http://ciml.info/

cdnsteve 11y ago

I feel like the barrier to machine learning for me, as I've seen in many tutorials and books and is an immediate discouragement, is the massive amount of math thrown in your face. Many of us didn't just graduate, need glasses and fall asleep at 8pm on the couch when the kids are in bed... Math is this distant fragment of memory buried under years of everything not Math. It feels like machine learning is only taught by academia but the majority of the audience is for practical use by the average developer wanting to play with it today.

Omnipresent 11y ago
I asked a related question [0] few weeks ago but didn't find a resource that talks about machine learning from real world example perspective. [0] https://news.ycombinator.com/item?id=10356874 https://news.ycombinator.com/item?id=10356874
alfiedotwtf 11y ago
The problem is that machine learning is applied probability and statistics. If you're interested, here's a gentle book that should give you enough background: http://www.amazon.com/Modern-Introduction-Probability-Statistics-Springer-ebook/dp/B00DZ0PKLG/ http://www.amazon.com/Modern-Introduction-Probability-Statis...
davmre 11y ago
Andrew Ng's Coursera ML course is supposed to be pretty accessible. I've also heard good things about Machine Learning for Hackers (http://www.amazon.com/Machine-Learning-Hackers-Drew-Conway/dp/1449303714 http://www.amazon.com/Machine-Learning-Hackers-Drew-Conway/d...). Ultimately, ML is a mathematical discipline. You can ask for a gentle approach that gets you to the foot of the mountain, but "if you want to learn about nature, to appreciate nature, it is necessary to understand the language that she speaks in." If you want to be more than an amateur, there's not much substitute for getting comfortable with math at the level of, say, Kevin Murphy's book. The good news is that the required math is fairly elementary - calculus, linear algebra, probability and statistics, all freshmen or maybe sophomore-level topics - so it shouldn't be beyond reach of a motivated developer able to set aside some time to learn. MOOCs and organizing study groups with friends/co-workers can help a lot here as well.
- fapjacks 11y ago
  Yeah I'd like to second this. This course specifically is what opened the door for me.
- subnaught 11y ago
  I'm currently taking Andrew Ng's Coursera course and I'd agree it's quite accessible. In fact, if you have a solid understanding of calculus and linear algebra, you might find it a bit slow at times.

Omnipresent 11y ago

Related: Foundations of Data Science: http://www.cs.cornell.edu/jeh/nosolutions90413.pdf http://www.cs.cornell.edu/jeh/nosolutions90413.pdf

alvern 11y ago

My question for someone that has an intermediate level of skill in machine learning, what's the best way to dip your toes in? (Udacity, coursera, edx, PDFs, Talking Machines podcast, etc)

zintinio4 11y ago
Find a problem to work on in a domain you find interesting. By reading published papers and trying to attack the problem, you'll be forced to pick up a lot of other knowledge not commonly discussed like: feature extraction and selection, dimensionality reduction, dealing with sparsity, common metrics for that problem, recent work, etc. I was forced to learn a massive amount in a short period of time for work, but I'd previously watched Andrew Ng's lectures, as well as majored in Math/CS. I can also generally recommend Hinton's NN lectures, Socher's Deep learning for NLP, Andrew Ng's Machine Learning, and a few books.

pinn42 11y ago

I'm creating general intelligence in Java multicore.

stevenmays 11y ago

I am considering taking Udacity's machine learning nanodegree [0] with zero machine learning background. It seems interesting. Any thoughts? [0] https://www.udacity.com/course/machine-learning-engineer-nanodegree--nd009 https://www.udacity.com/course/machine-learning-engineer-nan...

jjaredsimpson 11y ago

Top comment in another thread whines that nobody understands what the machinery of ML algorithms are really doing. Top comment here whines that math is hard.

pmelendez 11y ago

Just a curiosity: One of the authors also proposed Pegasos SVM [1] which is a nice online approximation to SVM and that can be written in 15 lines of code or so. http://ttic.uchicago.edu/~nati/Publications/PegasosMPB.pdf http://ttic.uchicago.edu/~nati/Publications/PegasosMPB.pdf

arbitrage314 11y ago

I'm a math geek, but I'm also a mostly self-taught data scientist. "The Elements of Statistical Learning" (https://web.stanford.edu/~hastie/local.ftp/Springer/OLD/ESLII_print4.pdf https://web.stanford.edu/~hastie/local.ftp/Springer/OLD/ESLI...) is far and away the best book I've seen. It took me hundreds of hours to get through it, but if you're looking to understand things at a pretty deep level, I'd say it's well-worth it. Even if you stop at chapter 3, you'll still know more than most people, and you'll have a great foundation. Hope this helps!

gajjanag 11y ago
I am a graduate student at MIT, and can second this recommendation. It is a fantastic book for machine learning and nothing else I have seen comes close.
- nafizh 11y ago
  You meant ESL or UML?
  - lakeeffect 11y ago
    ESL. His post was an hour before the reader5000 uml post.
reader5000 11y ago
Having read significant chunks of both ESL and Understanding Machine Learning (albeit UML much more recently) I would argue that for many readers UML is superior. ESL pays short shrift to the computational complexity of learning whereas UML explicitly handles both statistical and computational complexity concerns. It doesnt matter how statistically pure your algorithm is if its running time scales exponentially with your data. All of UML's chapters are conceptually unified even when discussing different ML algorithms, with ESL being more of a grab-bag by chapter. Still, both high quality and free!
- arbitrage314 11y ago

sagik 11y ago

Took the course at the Hebrew University. Awesome course. Here are the slides (English) and videos(Hebrew): http://www.cs.huji.ac.il/~shais/IML2014.html http://www.cs.huji.ac.il/~shais/IML2014.html

inglor 11y ago

I've read this book and warmly recommend it. It has a very pragmatic "no bullshit" approach and it's very mathematical and concise. The neural networks chapter is tiny (but that's ok - that's not the focus) and some of the questions are really hard - but overall I've really enjoyed it.