4 ms·

Deep Learning frees you from the need to do "feature engineering" and usually works much better than methods which require such process. I'd instead recommend e

by bitL 9y ago

Deep Learning frees you from the need to do "feature engineering" and usually works much better than methods which require such process. I'd instead recommend everyone to "dive deep" into deep learning and once they master it, get acquainted with classical methods that still might get used here and there. I understand it's difficult to let go of what you worked very hard to understand when you were studying ML, but such is life, "sunk cost fallacy" should not you blind from seeing 95% success rate of DL while observing paltry 60% success rate with SVM/HMM on the same problem. Just let it go.

peterhunt 9y ago
This might be true for CV and speech recognition and synthesis, but there are huge categories of problems (dare I say, the majority of industry use of ML) that are either working with time series data (which DL hasn’t had great success with) or must be highly explainable and tunable.
- fnl 9y ago
  Or you don't have millions of annotated examples to learn from, and no similar problem to transfer from...
  - agibsonccc 9y ago
    I both make most of my money from time series data and use deep learning and work with data with no labels. Here's a recent presentation I did on some of this work and a companion presentation I encourage people to read on how to use this effectively in production. While you are right that some feature engineering is needed, there's no reason DL can't be a part of your workflow. https://www.slideshare.net/agibsonccc/anomaly-detection-and-automatic-labeling-with-deep-learning https://www.slideshare.net/agibsonccc/anomaly-detection-and-... https://www.slideshare.net/pacoid/humanintheloop-a-design-pattern-for-managing-teams-that-leverage-ml https://www.slideshare.net/pacoid/humanintheloop-a-design-pa... For more of the basics, my book on deep learning might help as well (minimal math vs the standard text book): http://shop.oreilly.com/product/0636920035343.do http://shop.oreilly.com/product/0636920035343.do
    - 384028345 9y ago
      Thanks for the info! The book looks interesting. Do you have an opinion on the fast.ai and deeplearning.ai courses? I finally have some time to work through these and since the deeplearning.ai series starts on December 18th, I'm wondering which one to dive into since I can't tell from the outside how they compare.
      - agibsonccc 9y ago
        I would take both. deeplearning.ai focuses more on math fundamentals, fast.ai takes a more coding oriented approach. It also has 2 classes: a beginner and advanced one. I personally prefer the fast.ai approach.
      - bitL 9y ago
        Add Udacity's DLF ND to the mix and do all 3 of them, they are all a bit different. Udacity's one has the inventor of GANs doing lectures there, so it's pretty top notch as well.
      - fnl 9y ago
        While I agree with others that more is better, if you can take only one course, I strongly recommend taking Andrew Ng's. While it is true that you don't need to be able to design and understand 'nets from scratch to be able to use them, I agree with most of the brightest minds in DL that you won't get too far if you don't at least have an intuition for the math behind it. And Ng's course really only gives you that - an intuition. It does an excellent job at ensuring participants understand the bare minimum to do any kind of serious work. Learning ${your favorite framework}'s API will be a breeze if you understand the "why" already.
    - fnl 9y ago
      > While you are right that some feature engineering is needed, there's no reason DL can't be a part of your workflow. I (and, I believe, the earlier poster, too) never implied you can't use deep learning on such examples. What we (I think, both) were referring to was the claim that it would absolve you from feature engineering. (Which I understand you also refute.) > For more of the basics, my book on deep learning might help as well Congratulations on your book, I know how much hard work that is! Disclaimer: I make money with deep learning, too... ;-)

by bitL 9y ago

peterhunt 9y ago

This might be true for CV and speech recognition and synthesis, but there are huge categories of problems (dare I say, the majority of industry use of ML) that are either working with time series data (which DL hasn’t had great success with) or must be highly explainable and tunable.

fnl 9y ago
Or you don't have millions of annotated examples to learn from, and no similar problem to transfer from...
- agibsonccc 9y ago
  I both make most of my money from time series data and use deep learning and work with data with no labels. Here's a recent presentation I did on some of this work and a companion presentation I encourage people to read on how to use this effectively in production. While you are right that some feature engineering is needed, there's no reason DL can't be a part of your workflow. https://www.slideshare.net/agibsonccc/anomaly-detection-and-automatic-labeling-with-deep-learning https://www.slideshare.net/agibsonccc/anomaly-detection-and-... https://www.slideshare.net/pacoid/humanintheloop-a-design-pattern-for-managing-teams-that-leverage-ml https://www.slideshare.net/pacoid/humanintheloop-a-design-pa... For more of the basics, my book on deep learning might help as well (minimal math vs the standard text book): http://shop.oreilly.com/product/0636920035343.do http://shop.oreilly.com/product/0636920035343.do
  - 384028345 9y ago
    Thanks for the info! The book looks interesting. Do you have an opinion on the fast.ai and deeplearning.ai courses? I finally have some time to work through these and since the deeplearning.ai series starts on December 18th, I'm wondering which one to dive into since I can't tell from the outside how they compare.
    - agibsonccc 9y ago
      I would take both. deeplearning.ai focuses more on math fundamentals, fast.ai takes a more coding oriented approach. It also has 2 classes: a beginner and advanced one. I personally prefer the fast.ai approach.
    - bitL 9y ago
      Add Udacity's DLF ND to the mix and do all 3 of them, they are all a bit different. Udacity's one has the inventor of GANs doing lectures there, so it's pretty top notch as well.
    - fnl 9y ago
      While I agree with others that more is better, if you can take only one course, I strongly recommend taking Andrew Ng's. While it is true that you don't need to be able to design and understand 'nets from scratch to be able to use them, I agree with most of the brightest minds in DL that you won't get too far if you don't at least have an intuition for the math behind it. And Ng's course really only gives you that - an intuition. It does an excellent job at ensuring participants understand the bare minimum to do any kind of serious work. Learning ${your favorite framework}'s API will be a breeze if you understand the "why" already.
  - fnl 9y ago
    > While you are right that some feature engineering is needed, there's no reason DL can't be a part of your workflow. I (and, I believe, the earlier poster, too) never implied you can't use deep learning on such examples. What we (I think, both) were referring to was the claim that it would absolve you from feature engineering. (Which I understand you also refute.) > For more of the basics, my book on deep learning might help as well Congratulations on your book, I know how much hard work that is! Disclaimer: I make money with deep learning, too... ;-)