5 ms·
I've really enjoyed talking to Jacques about his lego project over the last few days, and I hope that it will lead to some additional learning materials on cour
by jph00 9y ago
I've really enjoyed talking to Jacques about his lego project over the last few days, and I hope that it will lead to some additional learning materials on course.fast.ai (which he kindly links to in his article) as a result. It's great to see such a thorough writeup of the whole process, which I think is much more interesting and useful than just showing the final result.
The key insight here, to me, is that deep learning saves a lot of time, as well as being more accurate. I hear very frequently people say "I'll just start with something simple - I'm not sure I even need deep learning"... then months later I see that they've built a complex and fragile feature engineering system and are having to maintain thousands of lines of code.
Every time I've heard from someone who has switched from a manual feature engineering approach to deep learning I've heard the same results as Jacques found in his lego sorter: dramatic improvements in accuracy, generally within a few days of work (sometimes even a few hours of work), with far less code to write and maintain. (This is in a fairly biased sample, since I've spent a lot of time with people in medical imaging over the past few years - but I've seen this in time series analysis, NLP, and other areas too.)
I know it's been trendy to hate on deep learning over the last year or so on HN, and I understand the reaction - we all react negatively to heavily hyped tech. And there's been a strong reaction from those who are heavily invested in SVMs/kernel methods, bayesian methods, etc to claim that deep learning isn't theoretically well grounded (which is not really that true any more, but is also beside the point for those that just want to get the best results for their project.)
I'd urge people that haven't really tried to built something with deep learning to have a go, and get your own experience before you come to conclusions.
- irq11 9y agoYou're seeing hate for deep learning on HN!?! I'd love to see something approaching critical thought! Most of the comments I see are silly fluff about the singularity and/or AGI and killer robots. But seriously: I think you're overstating the case. I've enjoyed watching the fast.ai videos, and I think deep learning is a clear choice for some areas right now. Many others, no. If you're working in a data-limited domain, in particular (i.e. most of them) there are probably better first choices. I know plenty of folks who have used deep learning to achieve no better than comparable results to the simpler methods they were using before. But yes, the initial engineering costs of a deep learning system have come down a lot further than I thought before I watched your videos.
- marze 9y agoIt is world changing tech. If skeptical, wait and see.
- minimaxir 9y ago> You're seeing hate for deep learning on HN!?! Most of the hate for deep learning on HN is usually against using it in titles as clickbait. (i.e. just simply taking a pretrained/predefined model and changing the source dataset without any optimization) The in-depth explanations of mechanics discussed in deep learning as noted in fast.ai courses and this submission do justify the use of the deep learning moniker, though.
- deepnotderp 9y agoA pre trained deep net can work very well in a data limited domain if there's sufficient overlap.
- irq11 9y agoYes, but that's true of any ML approach. Deep learning is great if you're dealing with a hugely dimensional problem, and you have the data to train the model. If one of those things is not true (and usually, one of those things is not true), you're better off starting simple.
- jacquesm 9y agoThat's an excellent point and one that kept me from trying the deep learning approach for a long time. But in the end the machine + rudimentary deep learning provided its own dataset and that really made it work. So even if I didn't have the data to train the model I had enough data to bootstrap the process and sometimes that's all you need.
- irq11 9y agoYes, agreed. Your use case was interesting to me in the way that you bootstrapped the problem. If only more domains were like that!
- ingend88 9y agoI've really enjoyed the Fast.ai videos and am in the process of using it to build some hardware/software solutions. Even 2 years ago, it was not possible to get started on a problem like this and solve it in a short time.
- stillmotion 9y ago> And there's been a strong reaction from those who are heavily invested in SVMs/kernel methods, bayesian methods, etc to claim that deep learning isn't theoretically well grounded (which is not really that true any more, but is also beside the point for those that just want to get the best results for their project.) Could you elaborate more on what has changed recently in the theoretical grounding of deep learning?
- jph00 9y agoIt's not so much that there's just one or two specific results, but rather that there's just far more researchers working on this now, and quite a few are working on more theoretical stuff. Sometimes, that theory results in really practical outcomes - a good example would be https://arxiv.org/abs/1701.07875 https://arxiv.org/abs/1701.07875 . Or another by the same researcher: https://arxiv.org/abs/1506.00059 https://arxiv.org/abs/1506.00059 . We've seen, in the last year or two, interesting results in nearly every area of theoretical research of deep learning, including generalization, optimization, generative modelling, bayesian models, and network architecture.
- gregw134 9y agoJeremy your fast.ai course was amazing. Any idea when/if there's going to be a part 2?
- deepnotderp 9y agoI can't recommend this comment enough! So many people give outdated advice with regards to deep learning like "it requires too much data and takes too much time" and "start with something theoretically grounded (tm) like an SVM". The truth is that an imagenet pre trained network will often work astonishingly well. The only other mark against deep learning is computational complexity, which I expect to go away soon (yours truly ;) )
- frozenport 9y agoExcept that the guys system doesn't work, at present. He will probably need both feature engineering and machine learning approaches.
- jacquesm 9y agoWhat gave you that impression? It works just fine for my use-case, and I will most likely be able to add more use-cases by virtue of being able to enrich the data from other data sets out there once I have the part identified. It is possible that at some point to get to the last .1% or so I may have to embed features that were engineered but right now there is no indication that that will be the case.
- scryder 9y ago>Every time I've heard from someone who has switched from a manual feature engineering approach to deep learning I've heard the same results ... dramatic improvements in accuracy, generally within a few days of work It bothers me how often otherwise rational individuals continue to be susceptible to survivorship bias. Of course you'd only hear about the successes, the failures are either embarrassed or know better than to tell you their story. The people who tried it for several months, saw accuracy cap out at something like 60-70% for processes that need something like >90% confidence to justify expenses, and which proceed to get ignominiously shifted into a different team (or fired) for building what the higher-ups view as a massive waste of money at your salary and the other engineers see as a data-guzzling black box. No, these guys aren't likely to tell their story. Or, at least THIS story. The story you instead get from these guys is how >After having had some fun times doing deep learning and data analysis, I'm excited get into the big new field of $THING_THAT_WAS_HIRING and all the other dross that modern vocal programmers use to mask any possible scent of failure and actual on-the-job difficulty experienced.
- jacquesm 9y agoThis goes pretty much for every tool though. Applying a tool to a job it isn't well suited to is always going to come out as a frustrating experience, be it a hammer or a piece of software. What I always try to do is to get a feel for the problem space trying different methods with as little investment as possible. That way you will - once you decide to go full power after a certain solution method - at least have a feeling that you are on the right path. Dogmatically trying to shoehorn every problem into the toolset that you know how to use is a way to stay reasonably productive but it rarely leads to optimal outcomes, sometimes you simply have to learn how to use a new tool in order to get to the maximum. I'm the last person to jump on new bandwagons, still have a dumb phone, don't use facebook and still run my own mailserver. Even so, when a tool has a significant and most importantly measurable advantage compared to the tools I'm already familiar with I'll adapt.
- mcv 9y agoI see plenty of stories about "$POPULAR_THING is not the solution and here's why". If it's true that HN has been hating on Deep Learning for the past year, there should be a market for articles explaining why or when it doesn't work. If someone started out using Deep Learning and then switched to something else, that seems like an excellent subject for an article.
- Macuyiko 9y agoGood points. How about you the following though: driven by all the hype surrounding deep learning and AI, consultants are currently jumping on this new thing they can sell to management without understanding much of the intricacies behind it, so even after facing many disillusions just a few years ago when the talk was all about big data, companies are now jumping towards investing in products promising some of that sweet deep learning magic. That's fine, but the thing is they once again plan to apply it on their badly maintained data warehouses (now often somewhere in their dusty hdfs stack) to build traditional predictive models. Customer churn, next best offer, propensity modeling, that sort of thing. I try to keep telling them that with structured data sets, and simple classification problems, a random forest or God forbid, a logistic regression model, would do just as well given that you spend some solid time in feature cleaning and preprocessing. Am I wrong in this setting? Perhaps I should also jump on the deep learning bandwagon and start selling 10 layer binary Classification networks :). Not disagreeing with you, but I do fear that many traditional industries will be jumping on board pretty soon without any of the actual use cases to warrant deep learning.
- deleted 9y ago[deleted]