Given that the difference between any 2 ML implementations is going to be 2 things, everything else is going to be eliminated by a hyperparameter search. Neithe

Given that the difference between any 2 ML implementations is going to be 2 things, everything else is going to be eliminated by a hyperparameter search. Neither of these require more than basic mathematical knowledge. Certainly nothing that wouldn't be covered in an undergraduate statistics course. And of course, it's loads and loads of work. What makes the difference in success for machine learning projects (this is assuming you have some process to avoid screwups in place): 1) quality of how the data is input into the network. You can almost never "usefully" put raw data in front of a neural network. Call it "data represantation" (for instance, for comparing stock prices do a neural net DNN predictor on the prices. Now do the same on the deltas (today - yesterday). Works 1000x better (still not good enough, but very clear difference)) It may not compare to the feature engineering of SVMs, but it's very present (it kinda does imho, but people tend to get very defensive when suggesting that) 2) your cost/loss function. There are tricks like GANs which are simple in concept and avoid some of the issues, but even there you have your image comparator you're still going to need. You can massively improve image comparisons (e.g. mean square difference of the image scaled 4x4, times 1e12, plus same for 16x16 times 1e6, plus mean square difference at real res beats the crap out of just taking mean square difference. Very good results have also been obtained for MNIST by comparing sorted lists of black pixels, instead of actual images) Many things depend on your cost/loss function, and you especially have the eternal problem : I want it to improve in 10 different ways. How do I balance those well into a single number ? Robot should grab the teddy bear, and shouldn't hit anything else. Those are easy to balance, but how would you balance grabbing the teddy bear at all versus not damaging it ? It will make a huge difference in whether your neural net converges at all.