3 ms·
> So I am doubling down on ML/DL. The amount of free resources now available for learning machine learning/deep learning nowadays is robust and easy to compreh
by minimaxir 10y ago
> So I am doubling down on ML/DL.
The amount of free resources now available for learning machine learning/deep learning nowadays is robust and easy to comprehend. (indeed, Andrew Ng's Coursera class is very good). And running running ML code is even easier, with libraries like Tensorflow/Theano to abstract the ML gruntwork (and Keras to abstract the abstraction!)
I suspect that there may be machine learning knowledge crash, where the basics are repeated endlessly, but there is less unique, real world application of the knowledge learned. I've seen many Internet testimonials saying how "I followed an online tutorial and now I can classify handwritten digits, AI is the future!" The meme that Kaggle competitions are a metric of practical ML skill encourages budding ML enthusiasts to look at minimizing log-loss or maximizing accuracy without considering time/cost tradeoffs, which doesn't reflect real-world constraints.
Unfortunately, many successful real world applications of ML/DL are the ones not being instructed in tutorials as they are trade secrets (this is the case with "big data" literature, to my frustration). OpenAI is a good step toward transparency in the field, but that won't stop the ML-trivializing "this program can play Pong, AI is the future!" thought pieces (https://news.ycombinator.com/item?id=13256962 https://news.ycombinator.com/item?id=13256962).
- Dzugaru 10y agoPretty much this. In the real world you have to deal with - Too few labeled / garbage labeled data (70000 digits? How about only 1000 complex class objects?) - Obscure bugs in custom implementation (yeah my custom layer works and gradient is correct... or wait why it diverges after 10k iteration? hmm). - Timing/RAM constraints (it should segment an image under 10ms on Jetson TX1, well, good luck with GoogLeNet)
- epberry 10y agoNailed it. Neural net execution speed is so critical for may production systems and it's very difficult to hit the sweet spot on trade offs but I never hear about these issues in wild.
- Eridrus 10y ago> Nailed it. Neural net execution speed is so critical for may production systems and it's very difficult to hit the sweet spot on trade offs but I never hear about these issues in wild. That's probably because you're not listening. There's plenty of literature on scaling down neural nets to smaller devices because everyone knows it's an issue and can trivially get a smaller device, see techniques such as Quantization , Distillation, Pruning or architectures specifically designed for the task such as YOLO.