5 ms·

Ask HN: Machine Learning? Where to start?

6 points by swGooF 15y ago

What are the best books/websites to learn about machine learning? I have a solid background in math/cs/algorithms.

stollercyrus 15y ago
http://aima.cs.berkeley.edu/ http://aima.cs.berkeley.edu/ is pretty good Stuart J. Russell and Peter Norvig, 2002. Artificial Intelligence: A Modern Approach, 2nd edition, Prentice Hall. ISBN: 0137903952.
- swGooF 15y ago
  thanks
  - krisneuharth 15y ago
    See also: http://www.ai-class.com/ http://www.ai-class.com/
- ig1 15y ago
  As much as I like that book it's pretty dated now and doesn't cover a lot of the techniques that are used in modern data mining, etc.
clyfe 15y ago
I learned myself specifically SVM from here http://see.stanford.edu/see/courseinfo.aspx?coll=348ca38a-3a6d-4052-937d-cb017338d7b1 http://see.stanford.edu/see/courseinfo.aspx?coll=348ca38a-3a... (already had a good background in linear and abstract algebra).
_dps 15y ago
I recommend starting with Witten and Frank's "Data Mining: Practical Machine Learning Tools and Techniques", for two reasons: 1) your mathematical background will make it easy for you to spot the areas where they gloss over important theoretical details (e.g. they don't treat Mercer's theorem deeply, if at all if I recall correctly). This will keep you from making rookie mistakes like picking your own strange kernel functions for SVMs (all of this will make sense to you after reading W+F) 2) W+F is accompanied by Weka, which has decent-but-not-great implementations of a wide variety of algorithms in an open-source toolkit with a functional (though not particularly usable) GUI. You can be up-and-running on test problems from the the UCI Machine Learning repository in 20 minutes. Russell and Norvig is a good reference book, but my strong suggestion is to grab W+F, Weka, and some data from UCI's ML repo (http://www.ics.uci.edu/~mlearn/ http://www.ics.uci.edu/~mlearn/). Use Weka to run the standard algorithms from W+F on real data. After that things can get as complicated as you want. I'm personally betting that graphical models (see Koller's "Probabilistic Graphical Models") will continue to grow in importance. On the statistical side, I strongly recommend "Applied Linear Regression" by Weisberg to get a sense for the kinds of idea statisticians bring to the party (significance, parameter intervals, chained analysis like ANOVA). Hope that helps :-) Edit* spelling errors PS: Almost forgot MacKay's excellent "Information Theory, Inference, and Learning Algorithms" (free as pdf). It does a great job linking coding, ML, inference, and bayesian probability together. Read at least as far as the section linking K-means clustering to Expectation-Maximization over a Mixture of Gaussians, it's worth it.
- swGooF 15y ago
  Thanks, that is a detailed description. Great ideas to get me going.

6 points by swGooF 15y ago

What are the best books/websites to learn about machine learning? I have a solid background in math/cs/algorithms.

http://aima.cs.berkeley.edu/ http://aima.cs.berkeley.edu/ is pretty good Stuart J. Russell and Peter Norvig, 2002. Artificial Intelligence: A Modern Approach, 2nd edition, Prentice Hall. ISBN: 0137903952.

swGooF 15y ago
thanks
- krisneuharth 15y ago
  See also: http://www.ai-class.com/ http://www.ai-class.com/
ig1 15y ago
As much as I like that book it's pretty dated now and doesn't cover a lot of the techniques that are used in modern data mining, etc.

clyfe 15y ago

I learned myself specifically SVM from here http://see.stanford.edu/see/courseinfo.aspx?coll=348ca38a-3a6d-4052-937d-cb017338d7b1 http://see.stanford.edu/see/courseinfo.aspx?coll=348ca38a-3a... (already had a good background in linear and abstract algebra).

_dps 15y ago

I recommend starting with Witten and Frank's "Data Mining: Practical Machine Learning Tools and Techniques", for two reasons: 1) your mathematical background will make it easy for you to spot the areas where they gloss over important theoretical details (e.g. they don't treat Mercer's theorem deeply, if at all if I recall correctly). This will keep you from making rookie mistakes like picking your own strange kernel functions for SVMs (all of this will make sense to you after reading W+F) 2) W+F is accompanied by Weka, which has decent-but-not-great implementations of a wide variety of algorithms in an open-source toolkit with a functional (though not particularly usable) GUI. You can be up-and-running on test problems from the the UCI Machine Learning repository in 20 minutes. Russell and Norvig is a good reference book, but my strong suggestion is to grab W+F, Weka, and some data from UCI's ML repo (http://www.ics.uci.edu/~mlearn/ http://www.ics.uci.edu/~mlearn/). Use Weka to run the standard algorithms from W+F on real data. After that things can get as complicated as you want. I'm personally betting that graphical models (see Koller's "Probabilistic Graphical Models") will continue to grow in importance. On the statistical side, I strongly recommend "Applied Linear Regression" by Weisberg to get a sense for the kinds of idea statisticians bring to the party (significance, parameter intervals, chained analysis like ANOVA). Hope that helps :-) Edit* spelling errors PS: Almost forgot MacKay's excellent "Information Theory, Inference, and Learning Algorithms" (free as pdf). It does a great job linking coding, ML, inference, and bayesian probability together. Read at least as far as the section linking K-means clustering to Expectation-Maximization over a Mixture of Gaussians, it's worth it.

swGooF 15y ago
Thanks, that is a detailed description. Great ideas to get me going.