3 ms·

> "Using fancy tools like neural nets, boosting, and support vector machines without understanding basic statistics is like doing brain surgery before knowing h

• DavidSJ 10y ago
  Image classification has everything to do with statistics: you're guessing the probability distribution over the classes conditional on the input image vector; the model is trained through a process of statistical inference (using gradient descent).
  • argonaut 10y ago
    You have stated something that someone with a high school-level (e.g. superficial) understanding of probability/statistics would understand. Most research in neural networks requires only a very superficial level of statistics. There are certainly many areas of neural networks where statistics is important (more theoretical areas), but those don't form the core of the research field. Also, calling (stochastic) gradient descent a form of statistical inference, while technically correct, is a ridiculous stretching of the term. No researcher considers SGD to be a statistical inference algorithm.
• eli_gottlieb 10y ago
  >There are lots of neural network applications that have little to do with statistics (image recognition with convolutional neural networks for example). You're kidding, right? The most fundamental reasons that deep convnets work at all are statistical in nature.
  • niels_olson 10y ago
    Well, yes and no. It's like saying you need to understand Bernoulli's priciple before designing the SR-71. True, but 7 levels short of where you're working.
• visarga 10y ago
  > "Using fancy tools like neural nets, boosting, and support vector machines without understanding basic statistics is like doing brain surgery before knowing how to use a band-aid." Or like programming in C without also knowing assembly and compiler theory? Or flying a plane without having a degree in aerodynamics? I think we can extract a lot of use from high level frameworks that abstract away much of the gritty statistics and math. For most applications all we need is to have well behaved, well tested libraries and some basic intuition about how they work. Fortunately, in machine learning almost everything is a function from inputs X to outputs y, and we know how functions work from programming. It's easy to integrate in apps. The devil is in hyperparameter tuning, but we can get away with good initializations and some measure of web research. In time people will just use precomputed neural nets for standard tasks like Syntaxnet (text parsing), Inception (image classification) ore use web APIs to hosted services (less secure for sensitive data). We make those better, maybe fine tune them to our needs and get away with it 100x faster. There is also work in automated hyperparameter search. Machine learning could become a black box when they get good enough.
  • petewailes 10y ago
    Straw man going on here. The problem is, whilst you can get numbers out without understanding the statistics underneath, you can easily misinterpret that output if you don't understand it. Look at how p values are used in science journals for an example of poor stats knowledge affecting real life outputs. There are some things in life that do require you to do the requisite reading. Things based on stats fall into that camp.
• argonaut 10y ago
  I would agree with you and I also did research on neural networks when I was in college. I'm not sure if those downvoting you have actually done/seen neural network research. Deep learning is pretty engineering driven (as opposed to theory driven), right now. I have a pretty weak understanding of statistics and from my perspective it was very common for grad students to also have a weak understanding (of course, those working in statistical learning theory had a strong understanding). This is a pretty ordinary occurrence - it shouldn't be surprising - this is sort of like pointing out that many theoretical statisticians have poor coding skills.