4 ms·
The title is clickbait, but the problem with the religious fervor of machine learning/deep learning is that entrepreneurs/VC keep arguing that deep learning is
by minimaxir 10y ago
The title is clickbait, but the problem with the religious fervor of machine learning/deep learning is that entrepreneurs/VC keep arguing that deep learning is magic and can solve any problem if you just stack enough layers. (see also: https://www.reddit.com/r/ProgrammerHumor/comments/5si1f0/machine_learning_approaches/ https://www.reddit.com/r/ProgrammerHumor/comments/5si1f0/mac...)
Meanwhile, statistical methods for non-image/text data with identifiable features can often work better than neural networks, but they are not as sexy. (Good discussion on HN about this: https://news.ycombinator.com/item?id=13563892 https://news.ycombinator.com/item?id=13563892)
- wakkaflokka 10y agoI call it the Jimmy Johns method of machine learning - just keep stacking those layers.
- thearn4 10y agoFor whatever reason, statistics seems to be in the midst of a serious branding problem. Maybe traditional frameworks of inference are too conservative for non-researcher's tastes? I'm curious what Fisher, Neyman, and Pearson would say about the current state of the field. Especially considering how often they and other statisticians disagreed with each other throughout the 20th century.
- estro 10y agoI believe your branding hypothesis is correct, and all of this hype around neural nets and machine learning is creating a startup ecosystem ripe for market correction. I've heard a million times that most machine learning problems are solvable by regression, and I expect to hear that a million times more.
- CuriouslyC 10y agoRegression is great if your output is a linear function of your inputs. Neural networks don't have that limitation. For a long time people have assumed that linear functions are the rule and nonlinearity is the exception, but the fact that nonlinear predictors are trouncing linear ones in many domains seems to be proving that false. As for deep neural networks, in my opinion they are just an ensemble of neural networks, and ensemble methods have been shown to produce best-in-class results almost regardless of the base learning algorithm.
- fullshark 10y agoThe only real advantage linear modeling has is marginal effects can be determined easily. This matters for some business intelligence / decision making applications but if you want to build a model with as small errors as possible there's really no reason to restrict your model to be linear in the age of massive computing power.
- digitalzombie 10y agoWell it's also easy to model. NN needs lots of data from what I've read. Also the whole sport analytic prediction industry mostly uses linear regression... But next hype thing is medical data for them but they're super slow in adopting new thing. Industry seems boring if you're not into sport and into modeling in general not just linear regression.
- j7ake 10y agoRegression beats neural networks when you care about your model parameters.
- digitalzombie 10y ago> For whatever reason, statistics seems to be in the midst of a serious branding problem. Many of them don't know how to code. And the stat program of most of these universities doesn't require any real coding classes. The professor give you code templates and you tweak it to get the result. But the stat classes are really good to handle data. Data science in general like Chapman Hill and UC Irvine seems like a really wide range of stuff from data analyst, a lil programming, data visualization, etc... I don't feel like it goes deep enough in what Statistic does imo. Most are stuck with Tableau and SAS. Pharmacy companies uses SAS. Startup companies and tech companies uses Python and R. I'm speaking from one data point sample, my current stat grad program. My undergrad is comp sci and I did a few years in the tech industry before coming back to grad school.
- et-al 10y ago> I'm speaking from one data point sample, my current stat grad program. My undergrad is comp sci and I did a few years in the tech industry before coming back to grad school. If you don't mind sharing details; why'd you choose your current grad program?
- mattnewton 10y agoJust call it "statistical machine learning" or some other vacuous term and people who know enough will respect it (maybe after rolling their eyes a bit), and people who don't will be happy to hear you are using machine learning.
- zebrafish 10y agoI have no experience with the entrepreneur/VC side of this coin, but Data Scientists know the differences between traditional regression based ML and CNNs. We know that doing image recognition or NLP is best suited to nvidia/tensorflow/keras/deeplearning/buzzword/buzzword. We also know that SVMs & Random Forests & regression lines work just as well as they always have to make predictions based on your click-thru data. Maybe we should explain the cost/benefit of the buzzwords vs. the science?
- pif 10y agoYour comment sounds wise, but with all those acronyms and technical terms I can't be sure :-)
- logfromblammo 10y agoIt's simple. More buzzwords equals more money. So whatever it is you're really doing, just find some way to semi-plausibly attach the latest hot buzzwords to it. Most of the "dinosaurs" around here should know by now that you don't get paid to do what works; you get paid to do what the boss/customer wants. (And that's not always going to be exactly what they asked for, either.) As long as you continue to throw money at me, I'll continue to chase your wild geese, hunt your snipes, fish for your red herrings, and joust with your wind turbines. Most of us here can learn fast enough to always stay one or two steps ahead of the boss on whatever topic they might latch on to. And that becomes genuine experience if it somehow manages to graduate past the business fad phase.
- Eridrus 10y agoTo say something in defence of the deep learning religion: it's a very flexible framework for expressing lots of different ideas and the ability to train all those ideas jointly if you can frame them in terms of SGD updates is very appealing. And even if you don't want to use SGD to fit your model, the deep learning machinery can be reused for other things like bayesian inference, eg PyMC3 and Edward are built on Theano/TensorFlow respectively. There have even been papers showing differentiable trees integrated into the deep learning framework. There is work on how to input graphs. Maybe this is just my own desire for unifying frameworks shining through, but joint training seems to have real benefits and feature learning has clearly had a large impact on reinforcement learning. Maybe it's not the best tool for day to day data science work, but that tends towards what tools are accessible and good enough, rather than what is best.
- lottin 10y agoSexy is in the eye of the beholder. Seriously, I think the problem of ML is that it often goes against the so-called principle parsimony, which plays a critical role in statistical modelling. Especially in many areas where the whole point isn't strictly to predict future outcomes, but to understand how things work and to test hypotheses. Understanding leads to better prediction anyways. If you can predict something but you don't know why or how, I would take such predictions with a grain of salt.
- jackcosgrove 10y agoI think you have the cart before the horse regarding understanding and prediction. Prediction precedes understanding in importance and sometimes methodology, since if you can't predict outcomes correctly your understanding, however well fleshed-out, will be inaccurate. Interestingly, the parsimony principle is lent empirical weight by bad results obtained from overfitting. Machine learning enables us to multiply many more entities than we can using our conscious thought processes. The various image recognition models use many variables that enable results impossible with other techniques. It does not violate parsimony if you can obtain better results on a large data set. I can predict that if a car drives by it will not teleport to some other location, but will rather continue along its path. I've been able to predict this since I was a child. I did not need to study the laws of motion to do so. I would not take such a prediction with a grain of salt.
- kgwgk 10y agoIn many cases, an acceptable prediction plus understanding is better than a more accurate prediction alone. And I'd say the example in your last paragraph is an example of understanding, by the way, or you would not have been able to express it in abstract terms. (For a young child, an animal or any other simple mind it may be possible to get the prediction without understanding, I guess).
- randcraw 10y agoI think that's because the majority of folks using ML care only about discriminative ML, not generative. A black box ML technique that delivers low predictive errors is good enough. To those ends, parsimony is largely irrelevant. I suspect that others using ML, who are inclined to test hypotheses, induce cause, and discover unknowns (via generative ML), are far fewer. So it's hardly surprising that the eye of discriminative MLers dominates in defining 'sexiness'.
- russdill 10y agoOnce it's good enough to solve as many problems as VCs claim, you won't need anyone to program it :p
- cookiecaper 10y agoYeah, we just came across someone on HN trying to write a testing DSL with NLP. [0] He's apparently gotten funding for this. To be frank, the tech community has really been leaving a sour taste in my mouth the last few years. Everyone is anxious to bandwagon on these buzzwords that provide little to no benefit or are applicable only to a small segment of the tech population. In most cases, even a superficial understanding of the problem space should make it obvious that $This_Weeks_Sexy_Solution is a very bad fit. Do you need a server that persists data and is individually addressable? Then why are you using Kubernetes and Docker, which are still struggling to figure out these very basic things? k8s and Docker have very specific uses, but unless you're Google, they're probably not the right fit for your production environment right now. This phenomenon seemed to hit a critical mass with document databases and single-page apps, and it continues to iterate with every open-source release that comes out of Google or Facebook. Since Google released TensorFlow last year and the compounded hype of [much worse than advertised attempts at] conversational speech recognition in Siri and Alexa, the "machine learning" bandwagon is starting to try to edge into the spotlight, and it sounds like it's already a mandatory part of any VC pitch. Fortunately, machine learning is pretty hard and you get into hairy math practically right away, so I don't think the legs on this one will last as long. But we'll see. We'll at least have a lot of faux-ML going around and a lot of people making spurious claims on their resumes. The economic collisions that make Silicon Valley and the tech industry in general a hive for inexperienced and insecure youth are bearing some really interesting effects this way. How can a company that's not "blown about by every wind of [tech fad]" fully exploit its relative sanity for competitive advantage? [0] https://news.ycombinator.com/item?id=13572415 https://news.ycombinator.com/item?id=13572415 ; my reply downthread at https://news.ycombinator.com/item?id=13573978 https://news.ycombinator.com/item?id=13573978
- eldavido 10y agoI've noticed this and I think it's an expression of a more general human urge to feel more important / significant than you really are. It's the same urge that makes us all think we're better drivers than average, better looking than average, etc. It's the urge that make people move to Silicon Valley/Hollywood/etc., because everyone's going to "make it" even though statistically, that isn't possible. It's why every parent with the means pays through the nose to get their kids into Harvard, why SUV sales are high even though most people don't off-road, and why people buy technical outerwear for climbing, etc. if they never go near a mountain. Everyone has a constructed image of themselves and we're lying if we say we don't try to burnish it. I think using all this tech is an expression of the desire for status. The problem is, it really can be a time-consuming, frustrating, customer-data-losing, unsupportable, waste of resources. That's bad for business, bad for your health (spending time fixing shit rather than eating well or going to the gym), and it can make your friends and family irritated with you, too. I'm building a product right now on the most boring stack possible: python, postgres, pyramid. I've been there/done that on the whole sharded-MongoDB-Redis-Memcache-900 machines-AWS-Docker-startup-ops nightmare-use all the things (we actually used every single thing in that list at my last job). It was unreliable, a huge pain in the ass, a little bit fun, but ultimately quite counterproductive. These days, my bets are on things like the JVM, the .NET CLR, and Postgres. As Knish from Rounders would say, "My kids eat". Another thing to keep in mind: a lot of this is resume padding. Yes, this is completely unprofessional, reckless behavior, but it happens, many recruiters are keyword-driven, and as interviewers many of us are impressed with a keyword-laden resume. We do this to ourselves as an industry in how we hire, who we let speak at events, and who we praise. It's nobody's fault but our own.
- hasbroslasher 10y agoIt's worth noting that ML doesn't just mean neural networks - those just get the most hype. I'm also curious if you could give an example of traditional models (presumably logreg, SVM, decision forest/trees) outperforming DNN on classification tasks. I read your thread and agree with the points, but I'm curious to see if anyone's outlined examples of when DNN fail in particular (not necessarily RNN/CNN). Aside - I think it's really ironic for people with no technical know how to go on a talk show and say "learn ML or you're toast". First, the inertia of the world economy is much greater than 3 years. Second, it's unrealistic to ask every 7-11 cashier to go learn ML. Third, programming is a complex art and we're surely not 3 years away from the singularity. Most people I know who have some ML knowledge also realize how far we are from a computer program that can read code, read specs, and then decide how to implement or fix existing code. And last, even if all of the above were moot, I think it's more likely that we write a program that successfully helms a Fortune 500 company before we write one that produces or improves on software. The former could be more easily reduced to a game of chess while the latter requires some massive novel research into "creativity". Maybe it's Mark Cuban who should be learning ML!
- AndrewKemendo 10y ago>Most people I know who have some ML knowledge also realize how far we are from a computer program that can read code, read specs, and then decide how to implement or fix existing code. That's exactly the wrong way to think about it IMO. My vision of Machine Programming is that it's a fluid OS/UI that adapts application architecture to how people actually use them. So for example you have "Deep Excel" which is trained first to give the basic functionality of a spreadsheet, but as each person uses it, it gets data about most used/combined UI elements, follow on actions etc... that "rewire" through RL the UX to behave in the way that the user indicates it wants to behave. So each instance of the program running would be unique to each user and constantly changing to make it more adaptive to their needs. Network these in a deeper system and you can iterate on design and functionality of products way faster.
- nugator 10y agoSounds cool but have you ever given support to a non tech savvy Excel user? Now add the dimension of an ever changing UI.
- HeavenBanned 10y ago>can often work better than neural networks But what about neural networks of the future? Sure, _at the moment_ they might be not as good as current statistical methods, but surely their ceiling must be through the roof? Same goes for quantum computing and the like...