4 ms·

Good write-up, but IMO these kinds of posts are not useful for most people (despite there being many just like it) - the 30,000 foot view is too much informatio

by rm999 12y ago

Good write-up, but IMO these kinds of posts are not useful for most people (despite there being many just like it) - the 30,000 foot view is too much information for a beginner but too basic for someone who is knowledgeable in the field. In my ten years of being fairly immersed in machine learning there was maybe a one month period where an article like this would have been very useful, and that was a couple of years in after I had mastered the basics of the more useful techniques (like generalized linear models and SVMs) and was ready to learn what else was out there. Honestly, learning the high-level overview was not hard at that point. I think a solid machine learning book coupled with a chart like this (http://scikit-learn.org/stable/tutorial/machine_learning_map/ http://scikit-learn.org/stable/tutorial/machine_learning_map...) is the most useful way to approach machine learning.

catilac 12y ago
Which ML book would you recommend?
- maurycy 12y ago
  It depends on your math background, but I've found Yaser S. Abu-Mostafa's Learning From Data the best, although Christopher M. Bishop's Pattern Recognition and Machine Learning is the gold standard. ;-)
  - deleted 12y ago
    [deleted]
- maurits 12y ago
  The most popular choices seem to be: Machine Learning: a Probabilistic Perspective, by Murphy http://www.cs.ubc.ca/~murphyk/MLbook/ http://www.cs.ubc.ca/~murphyk/MLbook/ Pattern classification, by Duda et all http://www.amazon.com/Pattern-Classification-Pt-1-Richard-Duda/dp/0471056693 http://www.amazon.com/Pattern-Classification-Pt-1-Richard-Du... The Elements of Statistical Learning, by Hastie et all. It is free from Stanford. http://www-stat.stanford.edu/~tibs/ElemStatLearn http://www-stat.stanford.edu/~tibs/ElemStatLearn Mining of Massive Datasets, free from Stanford. http://infolab.stanford.edu/~ullman/mmds.html http://infolab.stanford.edu/~ullman/mmds.html Bayesian Reasoning and Machine Learning, by Barber, free available online. http://web4.cs.ucl.ac.uk/staff/D.Barber/pmwiki/pmwiki.php?n=Brml.HomePage http://web4.cs.ucl.ac.uk/staff/D.Barber/pmwiki/pmwiki.php?n=... Learning from data, by Abu-Mostafa. It comes with Caltech video lectures: http://work.caltech.edu/telecourse.html http://work.caltech.edu/telecourse.html Pattern Recognition and Machine Learning, by Bischop http://research.microsoft.com/en-us/um/people/cmbishop/prml/ http://research.microsoft.com/en-us/um/people/cmbishop/prml/ Also noteworthy Information Theory, Inference, and Learning Algorithms, by Mackay, free. http://www.inference.phy.cam.ac.uk/itprnn/book.html http://www.inference.phy.cam.ac.uk/itprnn/book.html Classification, Parameter Estimation and State Estimation, by van der Heijden. http://prtools.org http://prtools.org Computer Vision: Models, Learning, and Inference, by Prince, available for free http://www.computervisionmodels.com/ http://www.computervisionmodels.com/ Probabilistic Graphical Models, by Koller. Has an accompanying course on Coursera.
  - simplekoala 12y ago
    There was post on HN of a blog post link which contained a list of all free machine learning/data mining books. Wondering, if someone can post the link to it. I am unable to find it through search.
    - craigching 12y ago
      This one? http://christonard.com/12-free-data-mining-books/ http://christonard.com/12-free-data-mining-books/
      - gtani 12y ago
        or this which includes good backgrounders on lin.alg, probability and stat, http://www.reddit.com/r/MachineLearning/comments/1jeawf/machine_learning_books/ http://www.reddit.com/r/MachineLearning/comments/1jeawf/mach... Or http://www.electronicsforu.com/newelectronics/articles/hitscc.asp?id=12580&group_type=cool_stuff http://www.electronicsforu.com/newelectronics/articles/hitsc... this is an excellent review (but doesn't cover books by Mohri, Rostamizadeh, Talwalkar and Abu-Mostafa , Magdon-Ismail, Lin: http://www.amazon.com/review/R32N9EIEOMIPQU/ref=cm_cr_pr_perm?ie=UTF8&ASIN=0262018020 http://www.amazon.com/review/R32N9EIEOMIPQU/ref=cm_cr_pr_per...
      - simplekoala 12y ago
        yes, yes. Thanks!
  - syvlo 12y ago
    If you can afford it (both financially and regarding math background), Bishop is a really great choice. Almost everything you need to know is in it. I have it and just love it! But he goes quite deep in the mathematical explanations (which is a great point, there is no better way to learn and understand) meaning you have to be willing to work on your math for this book.
nostrademons 12y ago
This table of contents is beginning to seem very familiar (I've done Google's internal machine-learning course and looked at several other courses/tutorials on the net), but what I'm wondering is: how do you do the more basic operations of feature and parameter selection? When I've tried using machine learning in my daily work, I usually top out at "not quite good enough", largely because the features I end up choosing have a lot of noise in their data. Meanwhile, more experienced experts (those who have been doing this for 10+ years) produce much better results in much shorter time periods because they use totally off-the-wall features that I would never have considered. When they've presented at these classes, they've said that feature selection is really the vast majority of the work in practical machine learning. How do you develop an intuition about which features will be useful? Are there any resources that are more like "A practitioner's guide to machine learning", talking about the stuff that's useful in practice and not just in theory?
- nanidin 12y ago
  Unless you have a massive amount of features already, can't you just try throwing stuff at the wall until something sticks? And even then you could run PCA on your features to reduce dimensionality.
  - Chronic26 12y ago
    Deep learning.
- eshvk 12y ago
  > This table of contents is beginning to seem very familiar (I've done Google's internal machine-learning course and looked at several other courses/tutorials on the net), but what I'm wondering is: how do you do the more basic operations of feature and parameter selection? I have thought about this problem for a long time. Initially, I used to think it was a mere matter of finding clever Machine Learning algorithms that could auto-magically detect features for you. There are others who think it is a matter of throwing a bunch of subject matter experts into the problem and building feature vectors. I have worked with both philosophies of people. I think the answer lies somewhere in the middle. And it is a fucking hard problem. You need to do some clever algorithmic work. Maybe use a PCA or your favorite dimensionality reduction trick but also be clever enough (mathematically that is) to realize when PCA is brittle and when not to use it. You also need to have enough understanding of the subject to be able to ask the right questions from your subject matter experts. It is easy to get five people in a room and ask them to describe a song. But it is much harder to find out the right way of asking them: Is this song the kind of song that you would dance to while drunk on three beers in a podunk town in the Mid West? That requires experience to get that sort of intuition. The only thing I can recommend (pun unintended) is to keep solving these problems, focus on the domain(s) that you are deeply passionate about. Don't spread yourself too thin and make yourself to be a general purpose Machine Learner. That is the only way to get to building beautiful end to end products.
cf 12y ago
I have thought about this problem, and I think the better approach is something way more domain specific. Start with an application in mind, see which machine learning algorithms they are using and just try to get them working. Once you have something working, notice the strengths of the algorithms for that particular domain. There is usually some nice learning theory to explain that. Then move on to another domain and repeat. As you continue doing this patterns will emerge across the domains. This is usually the best time to start working through a machine learning textbook. When you take this approach you defer a chunk of the theory for practical examples, but can immediately start to apply the algorithms and have some results.

by rm999 12y ago

catilac 12y ago

Which ML book would you recommend?

maurycy 12y ago
It depends on your math background, but I've found Yaser S. Abu-Mostafa's Learning From Data the best, although Christopher M. Bishop's Pattern Recognition and Machine Learning is the gold standard. ;-)
- deleted 12y ago
  [deleted]
maurits 12y ago
The most popular choices seem to be: Machine Learning: a Probabilistic Perspective, by Murphy http://www.cs.ubc.ca/~murphyk/MLbook/ http://www.cs.ubc.ca/~murphyk/MLbook/ Pattern classification, by Duda et all http://www.amazon.com/Pattern-Classification-Pt-1-Richard-Duda/dp/0471056693 http://www.amazon.com/Pattern-Classification-Pt-1-Richard-Du... The Elements of Statistical Learning, by Hastie et all. It is free from Stanford. http://www-stat.stanford.edu/~tibs/ElemStatLearn http://www-stat.stanford.edu/~tibs/ElemStatLearn Mining of Massive Datasets, free from Stanford. http://infolab.stanford.edu/~ullman/mmds.html http://infolab.stanford.edu/~ullman/mmds.html Bayesian Reasoning and Machine Learning, by Barber, free available online. http://web4.cs.ucl.ac.uk/staff/D.Barber/pmwiki/pmwiki.php?n=Brml.HomePage http://web4.cs.ucl.ac.uk/staff/D.Barber/pmwiki/pmwiki.php?n=... Learning from data, by Abu-Mostafa. It comes with Caltech video lectures: http://work.caltech.edu/telecourse.html http://work.caltech.edu/telecourse.html Pattern Recognition and Machine Learning, by Bischop http://research.microsoft.com/en-us/um/people/cmbishop/prml/ http://research.microsoft.com/en-us/um/people/cmbishop/prml/ Also noteworthy Information Theory, Inference, and Learning Algorithms, by Mackay, free. http://www.inference.phy.cam.ac.uk/itprnn/book.html http://www.inference.phy.cam.ac.uk/itprnn/book.html Classification, Parameter Estimation and State Estimation, by van der Heijden. http://prtools.org http://prtools.org Computer Vision: Models, Learning, and Inference, by Prince, available for free http://www.computervisionmodels.com/ http://www.computervisionmodels.com/ Probabilistic Graphical Models, by Koller. Has an accompanying course on Coursera.
- simplekoala 12y ago
  There was post on HN of a blog post link which contained a list of all free machine learning/data mining books. Wondering, if someone can post the link to it. I am unable to find it through search.
  - craigching 12y ago
    This one? http://christonard.com/12-free-data-mining-books/ http://christonard.com/12-free-data-mining-books/
    - gtani 12y ago
      or this which includes good backgrounders on lin.alg, probability and stat, http://www.reddit.com/r/MachineLearning/comments/1jeawf/machine_learning_books/ http://www.reddit.com/r/MachineLearning/comments/1jeawf/mach... Or http://www.electronicsforu.com/newelectronics/articles/hitscc.asp?id=12580&group_type=cool_stuff http://www.electronicsforu.com/newelectronics/articles/hitsc... this is an excellent review (but doesn't cover books by Mohri, Rostamizadeh, Talwalkar and Abu-Mostafa , Magdon-Ismail, Lin: http://www.amazon.com/review/R32N9EIEOMIPQU/ref=cm_cr_pr_perm?ie=UTF8&ASIN=0262018020 http://www.amazon.com/review/R32N9EIEOMIPQU/ref=cm_cr_pr_per...
    - simplekoala 12y ago
      yes, yes. Thanks!
- syvlo 12y ago
  If you can afford it (both financially and regarding math background), Bishop is a really great choice. Almost everything you need to know is in it. I have it and just love it! But he goes quite deep in the mathematical explanations (which is a great point, there is no better way to learn and understand) meaning you have to be willing to work on your math for this book.

nostrademons 12y ago

This table of contents is beginning to seem very familiar (I've done Google's internal machine-learning course and looked at several other courses/tutorials on the net), but what I'm wondering is: how do you do the more basic operations of feature and parameter selection? When I've tried using machine learning in my daily work, I usually top out at "not quite good enough", largely because the features I end up choosing have a lot of noise in their data. Meanwhile, more experienced experts (those who have been doing this for 10+ years) produce much better results in much shorter time periods because they use totally off-the-wall features that I would never have considered. When they've presented at these classes, they've said that feature selection is really the vast majority of the work in practical machine learning. How do you develop an intuition about which features will be useful? Are there any resources that are more like "A practitioner's guide to machine learning", talking about the stuff that's useful in practice and not just in theory?

nanidin 12y ago
Unless you have a massive amount of features already, can't you just try throwing stuff at the wall until something sticks? And even then you could run PCA on your features to reduce dimensionality.
- Chronic26 12y ago
  Deep learning.
eshvk 12y ago
> This table of contents is beginning to seem very familiar (I've done Google's internal machine-learning course and looked at several other courses/tutorials on the net), but what I'm wondering is: how do you do the more basic operations of feature and parameter selection? I have thought about this problem for a long time. Initially, I used to think it was a mere matter of finding clever Machine Learning algorithms that could auto-magically detect features for you. There are others who think it is a matter of throwing a bunch of subject matter experts into the problem and building feature vectors. I have worked with both philosophies of people. I think the answer lies somewhere in the middle. And it is a fucking hard problem. You need to do some clever algorithmic work. Maybe use a PCA or your favorite dimensionality reduction trick but also be clever enough (mathematically that is) to realize when PCA is brittle and when not to use it. You also need to have enough understanding of the subject to be able to ask the right questions from your subject matter experts. It is easy to get five people in a room and ask them to describe a song. But it is much harder to find out the right way of asking them: Is this song the kind of song that you would dance to while drunk on three beers in a podunk town in the Mid West? That requires experience to get that sort of intuition. The only thing I can recommend (pun unintended) is to keep solving these problems, focus on the domain(s) that you are deeply passionate about. Don't spread yourself too thin and make yourself to be a general purpose Machine Learner. That is the only way to get to building beautiful end to end products.

cf 12y ago

I have thought about this problem, and I think the better approach is something way more domain specific. Start with an application in mind, see which machine learning algorithms they are using and just try to get them working. Once you have something working, notice the strengths of the algorithms for that particular domain. There is usually some nice learning theory to explain that. Then move on to another domain and repeat. As you continue doing this patterns will emerge across the domains. This is usually the best time to start working through a machine learning textbook. When you take this approach you defer a chunk of the theory for practical examples, but can immediately start to apply the algorithms and have some results.