7 ms·

Experts recommend Machine Learning books

150 points by dmonn 6y ago

horsemessiah 6y ago
State and Revolution is where I'd recommend people start for ML ;)
- troelsSteegin 6y ago
  I guess that [0] would emphasize unsupervised models? [0] https://marxistleninist.wordpress.com/2005/01/20/study-guide-the-state-and-revolution/ https://marxistleninist.wordpress.com/2005/01/20/study-guide...
- CinchWrench 6y ago
  Slightly Meta: I wish HN had a parallel community for these kind of silly posts. I use it as a worksafe site to take a break at work and sometimes jokes like this, while not being in the spirit of HN, are things I very much appreciate.
  - hndude 6y ago
    I believe its called reddit
    - mhh__ 6y ago
      The only difference I find between reddits and HN are the amount of memes and the delusions of grandeur.
  - inopinatus 6y ago
    Levity does seem inconsistently received, not least because it so often relies on shared cultural understanding. I was somewhat confounded by this until realising that timezone matters. Nevertheless I don’t think puns and wry observation are entirely out of place, and even surreal irony or absurdist tales can go over quite well; sometimes very well if they convey a parable. Satire and parody are probably the hardest to pull off. They’re often at someone’s expense, and that is frowned upon; even when they’re not intentionally so, someone with different points of reference may read it as such.
  - commandlinefan 6y ago
    I'm not sure I get it - is the joke that "ML" could stand for Marxist/Leninism?
    - horsemessiah 6y ago
      Yup!
inopinatus 6y ago
Slightly crestfallen that the “ML” here is machine learning and not the programming language. I still refer to my vintage paperback of L. C. Paulson’s ML for the Working Programmer from time to time.
- ru552 6y ago
  I still read NLP as Neuro-linguistic programming. Every. Time.
  - wenc 6y ago
    So I have a little disambiguation heuristic: Neuro-linguistic programming definitely had its day. It seems to have fallen out of fashion in the last 3 decades (it was really just a phenomenon in the 80s -- I remember those days) so it's quite likely that modern references to NLP don't refer to it. I usually read NLP as "Nonlinear Programming" (nonlinear optimization) which is the community I come from. This acronym is not widely used outside the community so if I'm not reading the optimization literature, I'm pretty sure NLP doesn't refer to it. Natural language processing is more in vogue these days, so that tends to be my default reading. The term itself seems to have existed for decades, so it's not like it came after the others but this is the most likely reading today.
    - LolWolf 6y ago
      Oh, for a second I thought the GP was referring to nonlinear programming. Then I remembered this is an ML thread. There are some nonzero number of papers referring to NLP (as in nonlinear programming) in ML, mostly for the purposes of constrained optimization, but I agree with your current breakdown.
melenaboija 6y ago
For NLP I would maybe add Speech and Language Processing From Dan Jurafsky, Available at: https://web.stanford.edu/~jurafsky/slp3/ https://web.stanford.edu/~jurafsky/slp3/
henrik_w 6y ago
For a survey of AI and ML I really liked "Artificial Intelligence – A Guide for Thinking Humans" by Melanie Mitchell. I've written a summary of it here: https://henrikwarne.com/2020/05/19/artificial-intelligence-a-guide-for-thinking-humans/ https://henrikwarne.com/2020/05/19/artificial-intelligence-a...
cdavid 6y ago
The list is decent, but not exactly original. For people w/ a physics background, I would still recommend https://www.inference.org.uk/itprnn/book.pdf https://www.inference.org.uk/itprnn/book.pdf. Some of it is a bit obsolete, but then DL made a lot of stuff around generalization/overfitting somehow obsolete. It makes a lot of connection between different kind of approaches in ML, information theory, (Bayesian) statistics, and physics. It is not a very good book if you only care about applications (in which case the Keras book, for beginner, or fastai/etc. are much more appropriate).
- activatedgeek 6y ago
  David MacKay was an absolute rockstar and this book is grossly underrated among beginners in machine learning. This should be THE complementary reference for anyone who uses the Bishop book. Both these books follow a philosophy which some people may not completely agree with, that from the "church of Bayes". My guess is that information theory went through its hype phase and much of the ideas are so pervasive across real systems that people forget how important those connections are. To tell you its importance, skim this work on information-theoretic probing [1]. I find this so satisfying. Its most famous alternative, linear probing, always felt inelegant. This paper experimentally shows how terribly linear probing fails. [1] https://arxiv.org/abs/2003.12298 https://arxiv.org/abs/2003.12298
newtohn99 6y ago
As a new grad (bachelors swe), is it worth it to jump on the ML hype train ? I see modelling is almost always only open to phds/masters. So is studying all that stuff just for being a MLE/ data engineer worth it, if you are already a software developer (full stack)?
- voidray 6y ago
  > I see modelling is almost always only open to phds/masters. I think this varies pretty widely based on employer, e.g. if you are at a smaller company (and you show interest and have the necessary skills) then you're much more likely to be able to contribute on the modeling side. It's easier to get there if you have an advanced degree, but definitely not necessary. That being said, IMHO book lists like this aren't very useful because there's no incentive to keep them short and realistic. Reading seminal papers and implementing them is a different learning philosophy, maybe, but lists like this are probably more feasible to complete: https://dennybritz.com/blog/deep-learning-most-important-ideas/ https://dennybritz.com/blog/deep-learning-most-important-ide...
  - dougmany 6y ago
    This is a great approach to learning. It explains how the field got to where it is and allows the reader to go as deep as they want.
- currymj 6y ago
  the field is getting pretty crowded. there are probably better things to do at this point for pure career advancement and money, for the amount of effort you would have to spend to get viable professional-level skills. i haven't touched the job market recently though. however, it remains intrinsically very interesting and can be fun to learn about. more so than most resume items.

150 points by dmonn 6y ago

horsemessiah 6y ago

State and Revolution is where I'd recommend people start for ML ;)

troelsSteegin 6y ago
I guess that [0] would emphasize unsupervised models? [0] https://marxistleninist.wordpress.com/2005/01/20/study-guide-the-state-and-revolution/ https://marxistleninist.wordpress.com/2005/01/20/study-guide...
CinchWrench 6y ago
Slightly Meta: I wish HN had a parallel community for these kind of silly posts. I use it as a worksafe site to take a break at work and sometimes jokes like this, while not being in the spirit of HN, are things I very much appreciate.
- hndude 6y ago
  I believe its called reddit
  - mhh__ 6y ago
    The only difference I find between reddits and HN are the amount of memes and the delusions of grandeur.
- inopinatus 6y ago
  Levity does seem inconsistently received, not least because it so often relies on shared cultural understanding. I was somewhat confounded by this until realising that timezone matters. Nevertheless I don’t think puns and wry observation are entirely out of place, and even surreal irony or absurdist tales can go over quite well; sometimes very well if they convey a parable. Satire and parody are probably the hardest to pull off. They’re often at someone’s expense, and that is frowned upon; even when they’re not intentionally so, someone with different points of reference may read it as such.
- commandlinefan 6y ago
  I'm not sure I get it - is the joke that "ML" could stand for Marxist/Leninism?
  - horsemessiah 6y ago
    Yup!

inopinatus 6y ago

Slightly crestfallen that the “ML” here is machine learning and not the programming language. I still refer to my vintage paperback of L. C. Paulson’s ML for the Working Programmer from time to time.

ru552 6y ago
I still read NLP as Neuro-linguistic programming. Every. Time.
- wenc 6y ago
  So I have a little disambiguation heuristic: Neuro-linguistic programming definitely had its day. It seems to have fallen out of fashion in the last 3 decades (it was really just a phenomenon in the 80s -- I remember those days) so it's quite likely that modern references to NLP don't refer to it. I usually read NLP as "Nonlinear Programming" (nonlinear optimization) which is the community I come from. This acronym is not widely used outside the community so if I'm not reading the optimization literature, I'm pretty sure NLP doesn't refer to it. Natural language processing is more in vogue these days, so that tends to be my default reading. The term itself seems to have existed for decades, so it's not like it came after the others but this is the most likely reading today.
  - LolWolf 6y ago
    Oh, for a second I thought the GP was referring to nonlinear programming. Then I remembered this is an ML thread. There are some nonzero number of papers referring to NLP (as in nonlinear programming) in ML, mostly for the purposes of constrained optimization, but I agree with your current breakdown.

melenaboija 6y ago

For NLP I would maybe add Speech and Language Processing From Dan Jurafsky, Available at: https://web.stanford.edu/~jurafsky/slp3/ https://web.stanford.edu/~jurafsky/slp3/

henrik_w 6y ago

For a survey of AI and ML I really liked "Artificial Intelligence – A Guide for Thinking Humans" by Melanie Mitchell. I've written a summary of it here: https://henrikwarne.com/2020/05/19/artificial-intelligence-a-guide-for-thinking-humans/ https://henrikwarne.com/2020/05/19/artificial-intelligence-a...

cdavid 6y ago

The list is decent, but not exactly original. For people w/ a physics background, I would still recommend https://www.inference.org.uk/itprnn/book.pdf https://www.inference.org.uk/itprnn/book.pdf. Some of it is a bit obsolete, but then DL made a lot of stuff around generalization/overfitting somehow obsolete. It makes a lot of connection between different kind of approaches in ML, information theory, (Bayesian) statistics, and physics. It is not a very good book if you only care about applications (in which case the Keras book, for beginner, or fastai/etc. are much more appropriate).

activatedgeek 6y ago
David MacKay was an absolute rockstar and this book is grossly underrated among beginners in machine learning. This should be THE complementary reference for anyone who uses the Bishop book. Both these books follow a philosophy which some people may not completely agree with, that from the "church of Bayes". My guess is that information theory went through its hype phase and much of the ideas are so pervasive across real systems that people forget how important those connections are. To tell you its importance, skim this work on information-theoretic probing [1]. I find this so satisfying. Its most famous alternative, linear probing, always felt inelegant. This paper experimentally shows how terribly linear probing fails. [1] https://arxiv.org/abs/2003.12298 https://arxiv.org/abs/2003.12298

newtohn99 6y ago

As a new grad (bachelors swe), is it worth it to jump on the ML hype train ? I see modelling is almost always only open to phds/masters. So is studying all that stuff just for being a MLE/ data engineer worth it, if you are already a software developer (full stack)?

voidray 6y ago
> I see modelling is almost always only open to phds/masters. I think this varies pretty widely based on employer, e.g. if you are at a smaller company (and you show interest and have the necessary skills) then you're much more likely to be able to contribute on the modeling side. It's easier to get there if you have an advanced degree, but definitely not necessary. That being said, IMHO book lists like this aren't very useful because there's no incentive to keep them short and realistic. Reading seminal papers and implementing them is a different learning philosophy, maybe, but lists like this are probably more feasible to complete: https://dennybritz.com/blog/deep-learning-most-important-ideas/ https://dennybritz.com/blog/deep-learning-most-important-ide...
- dougmany 6y ago
  This is a great approach to learning. It explains how the field got to where it is and allows the reader to go as deep as they want.
currymj 6y ago
the field is getting pretty crowded. there are probably better things to do at this point for pure career advancement and money, for the amount of effort you would have to spend to get viable professional-level skills. i haven't touched the job market recently though. however, it remains intrinsically very interesting and can be fun to learn about. more so than most resume items.