4 ms·

> The current issue is that all of these models are black boxes Not quite. It'd be great if that was the only issue with neural network ML. A much bigger issue

> The current issue is that all of these models are black boxes Not quite. It'd be great if that was the only issue with neural network ML. A much bigger issue is that neural networks have extremely limited applicability. They're great for classification problems when you can have huge training datasets, but for many common problems they're useless. One obvious class of problems is time series prediction - extremely important in life and for business, and something neural networks are no good for.

throwaway3699 5y ago
Doesn't transfer learning level the playing field a bit? We already have excellent, off-the-shelf models for NLP, computer vision, ODT, etc... that just need fine-tuning for a particular business or domain problem. I think the 'huge training datasets' requirement is lessening every day. If you're doing something novel, then sure. I can see that.
- otabdeveloper4 5y ago
  Predicting time series isn't "novel". It's the oldest data science problem in the world. And ML can't solve it yet.
  - throwaway3699 5y ago
    Then go and use the right tool for the right job. I'm not suggesting neural networks are a panacea. I've worked with time-series problems recently and not used NN. NN not being good at one class of problems does not invalidate their usefulness for other classes, like your GP comment is suggesting.
    - otabdeveloper4 5y ago
      > ...like your GP comment is suggesting Really? What I literally wrote is "neural networks have extremely limited applicability". > I'm not suggesting neural networks are a panacea. That's good. Because ML practitioners are, even if they phrase it differently. (The idea is that since neural networks can fit curves then any problem can be solved by a neural network given enough layers and feature engineering grease.)
version_five 5y ago
There are many ML time series prediction techniques. And as a sibling points out, pretraining (not to mention various other low data methods) make big NN models useable even on small datasets.
- otabdeveloper4 5y ago
  > There are many ML time series prediction techniques. Yes, and they all give worse results than classic math and statistics approaches. This, in my opinion, is the worst part of ML of all. Instead of clearly stating the limits and boundary conditions of the algorithms you start hearing responses to the tune of "you're doing it wrong" and "wait until X magic pixie dust makes it usable", like you're being sold snake oil instead of algorithms research.

throwaway3699 5y ago

Doesn't transfer learning level the playing field a bit? We already have excellent, off-the-shelf models for NLP, computer vision, ODT, etc... that just need fine-tuning for a particular business or domain problem. I think the 'huge training datasets' requirement is lessening every day. If you're doing something novel, then sure. I can see that.

otabdeveloper4 5y ago
Predicting time series isn't "novel". It's the oldest data science problem in the world. And ML can't solve it yet.
- throwaway3699 5y ago
  Then go and use the right tool for the right job. I'm not suggesting neural networks are a panacea. I've worked with time-series problems recently and not used NN. NN not being good at one class of problems does not invalidate their usefulness for other classes, like your GP comment is suggesting.
  - otabdeveloper4 5y ago
    > ...like your GP comment is suggesting Really? What I literally wrote is "neural networks have extremely limited applicability". > I'm not suggesting neural networks are a panacea. That's good. Because ML practitioners are, even if they phrase it differently. (The idea is that since neural networks can fit curves then any problem can be solved by a neural network given enough layers and feature engineering grease.)

version_five 5y ago

There are many ML time series prediction techniques. And as a sibling points out, pretraining (not to mention various other low data methods) make big NN models useable even on small datasets.

otabdeveloper4 5y ago
> There are many ML time series prediction techniques. Yes, and they all give worse results than classic math and statistics approaches. This, in my opinion, is the worst part of ML of all. Instead of clearly stating the limits and boundary conditions of the algorithms you start hearing responses to the tune of "you're doing it wrong" and "wait until X magic pixie dust makes it usable", like you're being sold snake oil instead of algorithms research.