7 ms·
I made similar lists 4 years ago. I quit my job, and applied to a PhD program in ML (still doing that). Looking back, I can offer the following advice if you g
by p1esk 10y ago
I made similar lists 4 years ago. I quit my job, and applied to a PhD program in ML (still doing that).
Looking back, I can offer the following advice if you goal is to become a ML Engineer:
Foundation:
Start with (re)learning math. Take those boring university level full length courses in calculus, linear algebra, and probability/statistics. No, unless you're a fresh STEM graduate, 10 page math refreshers won't do. If you're self-studying, make sure you do all exercises and take practice exams to test yourself.
Tools:
1. (Re)learn C/C++, as well as a linear algebra library, such as Numpy or MATLAB. You will also have to learn parallel and distributed programming at some point (CUDA, MPI, OpenMP, etc). Next take a boring university level full length course on algorithms and data structures.
2. Get a book describing ML algorithms, and implement them yourself, first using plain C, then with MPI or CUDA, and finally using plain Numpy/MATLAB, or one of the low-level ML frameworks (Theano or TensorFlow).
Application:
Finally, start doing ML. Not learning about it, doing it. Choose an application that interests you (computer vision, NLP, speech recognition, etc), and start learning what you need to make something work). Focus on specific, practical tasks. If you don't have any particular application in mind, go to Kaggle, choose a competition, and read what models/tricks the winners used. Then jump right in and start competing.
The first two requirements might take years to master, but if you skip them, you won't be able to do any serious work in ML, or even understand latest papers. You will be a script kiddie, not a hacker.
- chadgeidel 10y agoHonest question - C or C++? I thought the general language suggested was Python+NumPy along with R. Thanks!
- p1esk 10y agoYou should be equally comfortable with both C and C++.
- chadgeidel 10y agoSorry I wasn't clear - my question was why do you recommend C and C++ over Python? is it specifically because of the distributed/GPU programming aspect?
- dlss 10y agoI think his meaning was if you're looking to create new ML algorithms, rather than just apply the algorithms of others, you'll need C and C++. If you're just looking to apply the algorithms of others, or if you're 100% sure you'll only be (for example) using TensorFlow to develop new algorithms, Python should work great.
- p1esk 10y agoThat, and also sometimes writing something in C is the optimal solution (faster than Python, and simpler than trying to modify TensorFlow or Theano backend).
- sgk284 10y agoThe parent seems to be coming from a skewed background. Most of his suggestions are way too low level. It's like telling a software engineer they must first write an operating system before they can write a web app. Learn Theano, TensorFlow, and maybe Torch. They'll handle the low level details for you. You just have to know some math to use them. If you just want to dive in and be immediately productive, you'd be hard pressed to do better than sci-kit learn and keras (for machine learning / deep learning respectively)
- mamon 10y agoCUDA, MPI, C++ these are the deep internal layers most software engineers working on Machine Learning would never need to interact with. Please learn Python instead. R and Matlab (Octave) might also be helpful.
- p1esk 10y agoMy comment is intended for people who want to become ML Engineers, not software engineers curious about ML.
- kajecounterhack 10y agoNo, your comment is intended for people who want to become ML infrastructure engineers of a very specific type. If you don't know that stuff but know generally how ML processes work (measurements, models, data partitioning and cleaning), and you start getting to know the frameworks (e.g caffe/tensorflow/torch) you'll do just fine. Like, you can know what MapReduce is and how it works and just use abstractions over it, never having implemented it yourself, and you're still a data engineer. It's a myth that X engineer needs to know everything about field X. There are different levels of everything, and different ways to get there.
- RandomInteger4 10y agoOr better yet, learn C, C++, CUDA, MPI, Python, and R simultaneously.
- throwaway001231 10y agoI work full-time as an ML Engineer at a big tech company in Silicon Valley. I think most ML Engineers do not read papers and do not do low level programming. They mostly do feature engineering. Actually I feel like I forgot a lot of math since university and I would like to pick it up again. Does anyone have a resource they can recommend either for self-study or in person?
- p1esk 10y agoAnd yet you're here, looking for math refreshers :)
- throwaway001231 10y agoI personally don't do a lot of feature engineering and I do regularly read papers, I just don't think that I am typical.
- komali2 10y agoKhan Academy is good. Also http://patrickjmt.com/ http://patrickjmt.com/
- zuzoovn 10y agoDisclaimer: I am the author of this Top-down learning path. Could you please take a look and give me some advice? Or If you have any opportunities, please drop me a line.
- throwaway001231 10y agoMy advice would be to figure out what you want to do more concretely. Machine Learning / Data Science is a big field, if you have more concrete goal I think it would be easier.
- nulltype 10y agoI'm reading this one math book by Richard Hamming: https://www.amazon.com/Methods-Mathematics-Calculus-Probability-Statistics/dp/0486439453/ref=sr_1_1?s=books&ie=UTF8&qid=1478835492&sr=1-1&keywords=methods+of+mathematics https://www.amazon.com/Methods-Mathematics-Calculus-Probabil... Not sure if that's what you're looking for, but it seems to have all the stuff I forgot from college.
- otto_ortega 10y ago> 2. Get a book describing ML algorithms Which one would you recommend?
- p1esk 10y agoPretty much any book on ML will describe all basic algorithms in sufficient detail for you to implement them yourself. Don't waste time trying to find the best book.
- sidlls 10y agoYour comment is littered with advice that applies to machine learning research, not engineering. It is as applicable to machine learning as "learn deep data structures and algorithms" is to CRUD web app development for an internal "enterprise" application that will see simultaneous usage in the high dozens of users at best. In other words: it's only correct in a very narrow context. And you end with patronizing insults. Do you feel better?
- 0xdeadbeefbabe 10y agoSecurity folks do the same thing. I don't get it.
- sidlls 10y agoIt's pervasive in this industry, no matter the sub-discipline. That sort of crap doesn't happen in other, mature engineering fields. At least not to the extent it does in computing. It's as if there is some unaddressed need to be considered among the intellectual elite that festers and ferments into casual, ego driven aggression.
- guelo 10y agoYou must have not dealt with other engineering disciplines. At least in software engineering you don't have to pass exams and get a "Professional Engineer" certification from the government in order to get a job.
- vonmoltke 10y agoFirst, it's NCEES who creates and grades the PE certification, not the government. Second, with the exception of civil engineers other disciplines have no problems getting jobs without licenses. The license just opens more doors and makes finding work easier.
- sidlls 10y agoMy comment has nothing to do with PE certifications.
- jamez1 10y ago> Get a book describing ML algorithms, and implement them yourself, first using plain C, then with MPI or CUDA, and finally using plain Numpy/MATLAB, or one of the low-level ML frameworks (Theano or TensorFlow). You don't seem to understand what high-level and low-level mean, why on earth would you write something in C, and then re-write it in MATLAB? The point of the higher level languages is rapid development at the expense of performance. Re-writing code in a different language is quite a basic skill and not something to waste time perfecting. Instead the focus should be on understanding the language/systems themselves on a deeper level. No offense, but it sounds like you have a lot to learn before you should be giving out advice. Seriously you're talking about kaggle? Yet you don't make a single mention of computer science?
- p1esk 10y agoLet me give you an example. I first learned about neural networks from deeplearning.net tutorials. They use Theano, so when I got a convolutional network running, I was left wondering about two critical pieces: backpropagation, and the actual convolution operation. Theano completely hides them from a user. So I decided to implement a simple convnet using only Python and Numpy. That was when I realized that I had no idea how the backprop works for convnets. I'm guessing most people who only used Theano or similar framework to run convnets don't know how backprop works there. Next, I realized that the code I wrote was too slow for any practical purposes, and I decided to rewrite it in pure C. Well, actually I gave up half way and used Armadillo package (Numpy analog for C/C++), but the end result was my convnet became 5 times faster. It was still too slow, and I noticed that my code is using only one core out of 4 cores available on my computer, so I decided to parallelize it, using Cilk+. It was actually easier than I expected, and soon I got almost linear speedup using all 4 cores. The final exercise was to implement it with CUDA, and I got another factor of 5 speedup running it on my GPU. The benefit of this experience is enormous - I learned a lot about convnets, I learned a lot about tools and libraries useful for ML, and I learned a lot about making my code faster. p.s. I don't quite understand what you meant about Kaggle and CS. p.p.s. And yes, I do still have a lot to learn, but I wish someone gave me this advice when I was starting out.
- 10y ago