10 ms·
Machine Learning for Software Engineers
- BucketSort 10y ago4 hours on the front page of HN and no comments. Seems really weird. Nonetheless, this is a really good list of resources. I've tried making lists of resources like this in the past, but there are so many nicely put together resources like this. Now I have a list of list of resources.
- messel 10y agoWe need machine learning to learn about machine learning
- skriticos2 10y agoA DIY toolkit would be nice, that gives you various peaces that it explains and then let's you fit them together. Of course without a million dependencies, clean-room type display of concept. Anybody bored enough to build this?
- dolzenko 10y ago> This is my multi-month study plan for going from mobile developer (self-taught, no CS degree) to machine learning engineer. Wondering what could be the rationale? I mean is mobile-dev so critically different from ML?
- adamnemecek 10y agoFew subfields of CS are as hyped as ML.
- sn9 10y agoHype aside, it's still pretty useful and also pretty different from mobile development.
- sidlls 10y agoYes, they solve completely different problems. "Mobile-dev" is more or less application development for a target client, possibly with a client-server relationship. It may incorporate the products of a ML system, or any number of other systems, but is itself the same application development model we've had for 30 years, but with a mobile client. Machine Learning is applied computational statistics. It and its close cousin "data science" have become incredibly hyped the last few years. I suppose it's a matter of taste, but my view is that applying simple probability to a problem (e.g. "recommending" based on picking the thing most frequently voted up by other users and adjusting the recommendation as data comes in) isn't "machine learning" or "data science", but I've seen it called such by candidates I've interviewed and even one colleague. It's not that simple, and trivializing the terms just contributes to the hype.
- dolzenko 10y agoSorry, didn't make it clear, was wondering given all the difference why would one prefer one over the other? Seem like 'in the field' experience is not going to be very much different. Appears to me mobile can be even more fun.
- deleted 10y ago[deleted]
- rosspackard 10y agoI enjoy these lists that frequently pop up but I really wish for something better. Almost like a curriculum. I don't like that there are 20 different courses for X. Which one should I choose, in what order. Which ones have pre-requisite knowledge requirements, which ones don't. What programming languages does each one use? etc,etc,etc
- sn9 10y agoThose curricula exist. They're called university course offerings. Just look up a class you want to take, figure out what the prerequisites are (often listed in the course description or syllabus), and follow the chain of prerequisites until you hit a level that's appropriate. You can easily look do this with a few different universities and just compare their courses for a particular subject, find the most common textbooks, etc. And nowadays, this is even easier since you can find a MOOC or online course for basically any course in the DAG leading to nontrivial machine learning. (Programming languages should never be a factor, unless you're choosing between two otherwise equivalent offerings.)
- rosspackard 10y agoI shouldn't have used the word curriculum. I went to a top university and I know how frequently those prerequisite courses can be totally off base (my degree is chemical/biomolecular engineering). I used the term 'pre-requisite knowledge' because I would prefer a list of what ideas/concepts are needed to be fully understood before taking the course (ex/ instead of saying linear algebra is a prerequisite, say you need an understanding of solving linear equations using matrices but you don't need to understand linear spaces. Then you can focus your effort better and not take a 40 hour pre-req that you only need 4 hours from)
- sn9 10y agoThe idea that you might want to focus your effort on a tiny subset of a coherently structured course's contents seems short-sighted to me. The class you're currently looking at might only need that tiny subset, but what about classes you'll be interested in taking in the future? Do you really think it's more likely that that same tiny subset will be sufficient, or is it more likely that the utility of the class as a whole studied deeply rather than shallowly will allow you to tackle more advanced topics? The analogy to building a foundation is very apt and especially so for a subject as ubiquitous and illuminating as linear algebra. The course prerequisite list is, I think, usually a good balance of the right amount of information. You should usually prioritize understanding those subjects deeply if you want a high ROI thing to spend your time on.
- adamnemecek 10y agoTo be honest, this isn't the best list, it's a bit too blog heavy. I've started reading up on ML only recently but here are my recommendations. Note that I haven't went through all of them in entirety but they all seem useful. Note that a lot of them overlap to a large degree and that this list is more of a "choose your own adventure" than "you have to read all of these". Reqs: * Metacademy (http://metacademy.org http://metacademy.org) If you just want to check out what ML is about this is the best site. * Better Explained (https://betterexplained.com/ https://betterexplained.com/) if you need to brush up on some of the math * Introduction to Probability (https://smile.amazon.com/Introduction-Probability-Chapman-Statistical-Science/dp/1466575573 https://smile.amazon.com/Introduction-Probability-Chapman-St...) * Stanford EE263: Introduction to Linear Dynamical Systems (http://ee263.stanford.edu/ http://ee263.stanford.edu/) Beginner: * Andrew Ng's class (http://cs229.stanford.edu http://cs229.stanford.edu) * Python Machine Learning (https://smile.amazon.com/Python-Machine-Learning-Sebastian-Raschka/dp/1783555130 https://smile.amazon.com/Python-Machine-Learning-Sebastian-R...) * An Introduction to Statistical Learning (https://smile.amazon.com/Introduction-Statistical-Learning-Applications-Statistics/dp/1461471370/ https://smile.amazon.com/Introduction-Statistical-Learning-A...) Intermediate: * Pattern Recognition and Machine Learning (https://smile.amazon.com/Pattern-Recognition-Learning-Information-Statistics/dp/0387310738 https://smile.amazon.com/Pattern-Recognition-Learning-Inform...) * Machine Learning: A Probabilistic Perspective (https://smile.amazon.com/Machine-Learning-Probabilistic-Perspective-Computation/dp/0262018020 https://smile.amazon.com/Machine-Learning-Probabilistic-Pers...) * All of Statistics: A Concise Course in Statistical Inference (https://smile.amazon.com/gp/product/0387402721/ https://smile.amazon.com/gp/product/0387402721/) * Elements of Statistical Learning: Data Mining, Inference, and Prediction (https://smile.amazon.com/gp/product/0387848576 https://smile.amazon.com/gp/product/0387848576( * Stanford CS131 Computer vision (http://vision.stanford.edu/teaching/cs131_fall1617/ http://vision.stanford.edu/teaching/cs131_fall1617/) * Stanford CS231n Convolutional Neural Networks for Visual Recognition (http://cs231n.github.io/ http://cs231n.github.io/) * Convex Optimization (https://smile.amazon.com/Convex-Optimization-Stephen-Boyd/dp/0521833787 https://smile.amazon.com/Convex-Optimization-Stephen-Boyd/dp...) * Deep Learning (http://www.deeplearningbook.org/ http://www.deeplearningbook.org/ or https://smile.amazon.com/Deep-Learning-Adaptive-Computation-Machine/dp/0262035618/ https://smile.amazon.com/Deep-Learning-Adaptive-Computation-...) * Neural Networks and Deep Learning (http://neuralnetworksanddeeplearning.com/ http://neuralnetworksanddeeplearning.com/) Advanced: * Probabilistic Graphical Models: Principles and Techniques (https://smile.amazon.com/Probabilistic-Graphical-Models-Principles-Computation/dp/0262013193/ https://smile.amazon.com/Probabilistic-Graphical-Models-Prin...) I have also found that looking into probabilistic programming is helpful too. These resources are pretty good: * The Design and Implementation of Probabilistic Programming Languages (http://dippl.org http://dippl.org) * Practical Probabilistic Programming (https://smile.amazon.com/Practical-Probabilistic-Programming-Avi-Pfeffer/dp/1617292338 https://smile.amazon.com/Practical-Probabilistic-Programming...) The currently most popular ML frameworks are scikit-learn, Tensorflow, Theano and Keras.
- mtalantikite 10y agoI've been spending my free time getting myself familiar with ML and personally I'm focused on resharpening my math skills, reading papers from the "awesome deep learning papers" repo [1], playing with TensorFlow, and reading the Neural Networks and Deep Learning book [2]. I did my undergrad degree in math, so a lot of the math is just review for me, but ML seems to get fairly math heavy pretty quick. I would suggest anyone looking at ML to spend a good amount of time going through the backing math in addition to the CS parts, otherwise you might not develop much intuition for what's going on. [1] https://github.com/terryum/awesome-deep-learning-papers https://github.com/terryum/awesome-deep-learning-papers [2] http://neuralnetworksanddeeplearning.com/ http://neuralnetworksanddeeplearning.com/
- throwaway001231 10y agoIf you are serious about deep learning I would recommend this book instead: http://www.deeplearningbook.org http://www.deeplearningbook.org It is coming out next month, but you can view the entire book online.
- barleymash 10y agoThat dude is USA as heck.
- p1esk 10y agoI made similar lists 4 years ago. I quit my job, and applied to a PhD program in ML (still doing that). Looking back, I can offer the following advice if you goal is to become a ML Engineer: Foundation: Start with (re)learning math. Take those boring university level full length courses in calculus, linear algebra, and probability/statistics. No, unless you're a fresh STEM graduate, 10 page math refreshers won't do. If you're self-studying, make sure you do all exercises and take practice exams to test yourself. Tools: 1. (Re)learn C/C++, as well as a linear algebra library, such as Numpy or MATLAB. You will also have to learn parallel and distributed programming at some point (CUDA, MPI, OpenMP, etc). Next take a boring university level full length course on algorithms and data structures. 2. Get a book describing ML algorithms, and implement them yourself, first using plain C, then with MPI or CUDA, and finally using plain Numpy/MATLAB, or one of the low-level ML frameworks (Theano or TensorFlow). Application: Finally, start doing ML. Not learning about it, doing it. Choose an application that interests you (computer vision, NLP, speech recognition, etc), and start learning what you need to make something work). Focus on specific, practical tasks. If you don't have any particular application in mind, go to Kaggle, choose a competition, and read what models/tricks the winners used. Then jump right in and start competing. The first two requirements might take years to master, but if you skip them, you won't be able to do any serious work in ML, or even understand latest papers. You will be a script kiddie, not a hacker.
- chadgeidel 10y agoHonest question - C or C++? I thought the general language suggested was Python+NumPy along with R. Thanks!
- p1esk 10y agoYou should be equally comfortable with both C and C++.
- chadgeidel 10y agoSorry I wasn't clear - my question was why do you recommend C and C++ over Python? is it specifically because of the distributed/GPU programming aspect?
- hash-set 10y agoWhile it's nice to think that people with no background can succeed in Data Science, I'm not sure it is true. There are a lot of hacks who never did "science" coming out of the woodwork lately. It's a bit ugly.
- BickNowstrom 10y agoLearn Python stack: scipy, numpy, pandas, scikit-learn, jupyter, matplotlib/seaborn. Learn machine learning tools: XGBoost, Scikit-learn, Keras, Vowpal Wabbit. Do data science competitions: Kaggle, DrivenData, TopCoder, Numerai. Take these courses: https://www.coursera.org/learn/machine-learning https://www.coursera.org/learn/machine-learning http://work.caltech.edu/telecourse.html http://work.caltech.edu/telecourse.html Work on soft skills: Business, management, visualization, reporting. Do at least one real-life data science project: Open Data, Journalism, Pet project. Contribute to community: Create wrappers, open issues/pull reqs, write tutorials, write about projects. Read: FastML, /r/MachineLearning, Kaggle Forums, Arxiv Sanity Preserver. Implement: Recent papers, older algorithms, winning solutions. As a software engineer you have a major advantage for applied ML: You know how to code. AI is just Advanced Informatics. If you want to become a machine learning researcher... skip all this and start from scratch: a PhD. Else: Learn by doing. Only those who got burned by overfit, will know how to avoid it next time.
- azaydak 10y agoWhat would you guys recommend for someone like me who has a strong background and education in ML algorithms (a PhD in a similar field) and math however but has very little formal programming skills. When it comes to coding I usually find myself hacking together code with the intention to come back to it later but never do. I have never had a problem finding and keeping jobs but have always felt a little insecure when it comes to writing code with CS majors, even ones with just a few years out of college. Any good material to help get me up to speed on the CS part in particular with common ML problems?
- rosspackard 10y agoWhat kinds of things do you tend to be insecure about?
- azaydak 10y agoBasically, I'm confident that I can get any job done and working. However I am insecure or don't always know how to answering questions about how efficient / fast/ a block of code is. Also, code architecture and professional coding styles. And finally the sort of classical coding interview questions that most CS majors would learn in school but are not always relevant in daily use but seem to be asked often.
- rosspackard 10y agoThose are all goals that with a little work should be surmountable. When CS majors or programmers estimate how fast/efficient code is they generally are intuitively applying big 'O' notation or some similar concept. This can be gained from an intro level algorithms class. Or you could look up and read about the topic. The classical coding interview questions mostly stem from an intro algorithms and data structures class. Or you could get a book on interview prep and work through a couple a day. Finally, the coding style issue usually is ancillary knowledge and not found (or at least not done well) in typical CS curricular. I would suggest the book "Clean Code" by Robert Martin. It goes through code smells, commenting, general style, and is well written. So overall, you could look at Algorithms and Data Structures MOOC's. Also read a book on CS interview prep and writing maintainable code. Hope this helps a little. Good Luck!