5 ms·

> This makes a good case for not learning machine learning three years ago. Have you looked at the monograph at all? This could have been written 5 to 10 years

> This makes a good case for not learning machine learning three years ago. Have you looked at the monograph at all? This could have been written 5 to 10 years ago, 15 years ago, maybe a few sections would have looked a little different. In fact, I like to recommend Mitchell's Machine Learning book (I think it was written in the 90s) as an introduction to people with a serious interest. There currently is a lot of hype going on for machine learning algorithms, because we see good progress in things like computer vision / pattern recognition. This kicks of a marketing machinery that really blurs the reality. In reality, we have an established body of methods and modelling techniques that are sufficient, because the available data is the bottleneck for prediction quality. The actual challenge is to come up with a valuable business proposition, not necessarily to build the predictive model.

dotancohen 9y ago
> Have you looked at the monograph at all? This could have been written 5 to 10 years ago The counterpoint is that a _lot_ could have been written 5 to 10 years ago, and only 10% of it would still be relevant. This is the relevant 10%.
- wirrbel 9y ago
  Looking at the table of contents, it is a subset of PRML [Bishop 2006]. And I suppose it is more than 10% PRML. And the things contained in PRML but not in the monograph are still relevant I'd say (Kalman filters, ...). [Bishop 2006] http://www.springer.com/de/book/9780387310732 http://www.springer.com/de/book/9780387310732
bitL 9y ago
I'd like to point out the fact that there is a split within Machine Learning into "classical" and "Deep Learning"-based one. While Deep Learning are just neural networks with Big Data on GPU, they obliterated many areas of classical ML. What you really want to learn is Deep (+Reinforcement) Learning, or what is coming known as differentiable programming. For classical ML, learn how to use Spark's MLlib and take Ng's Coursera course, that might be all you need from practical point of view.
- wirrbel 9y ago
  Deep learning has shown to have obliterated much of the computer vision area and other examples where classical approaches such as feature engineering are naturally hard. This comes at a price of a massive hunger for data of Deep Learning models to achieve an acceptable prediction accuracy. So with use cases like image recognition, etc. the relative performance of deep learning models is much better than with classical algorithms. In general (you'll probably find specific counterexamples for this, but as I said _in general_) the relative performance of deep learning models to classical machine learning may drop below that of classical models when you start looking at medium-sized data sets. And I can assure you, there are many data sets, with many applications for machine learning. And while the world talks about deep learning, I know of many companies, where random forests, suport vector machines or bayesian models have been running for years (which means, cross-validated by data unavailable during model development) with a prediction performance that a business can depend on. I agree with you that Reinforcement learning will as a technology become much more important in the next years. However, only if exploration is cheap enough. I expect Deep Reinforcement learning not to be the answer, at least not in its current state, but I can very well imagine that we will see more machine-learning-algorithm in the reinforcement-learning-loop experiments. I personally would hope to see more research in the bayesian reinforcement learning area.
  - bitL 9y ago
    I know, DL works only if you have massive datasets, DRL is even worse for the number of training episodes. Maybe you've heard about recent craziness of using GANs to generate training set fillers when your training sets are small, i.e. you have only 1,000 examples but need 10,000 for reasonable performance. Instead of gathering more examples, you use GAN to create believable training data, and it seems to be working quite well (i.e. a bump from 60% accuracy to 80% while bigger training dataset with real examples would bump you to 90%). What I observed is that many ML companies now run two pipelines in parallel, one based on Deep Learning and the other on classical ML, then cherry pick solutions that work best for the problem/scale they have.
    - wirrbel 9y ago

dotancohen 9y ago

> Have you looked at the monograph at all? This could have been written 5 to 10 years ago The counterpoint is that a _lot_ could have been written 5 to 10 years ago, and only 10% of it would still be relevant. This is the relevant 10%.

wirrbel 9y ago
Looking at the table of contents, it is a subset of PRML [Bishop 2006]. And I suppose it is more than 10% PRML. And the things contained in PRML but not in the monograph are still relevant I'd say (Kalman filters, ...). [Bishop 2006] http://www.springer.com/de/book/9780387310732 http://www.springer.com/de/book/9780387310732

bitL 9y ago

I'd like to point out the fact that there is a split within Machine Learning into "classical" and "Deep Learning"-based one. While Deep Learning are just neural networks with Big Data on GPU, they obliterated many areas of classical ML. What you really want to learn is Deep (+Reinforcement) Learning, or what is coming known as differentiable programming. For classical ML, learn how to use Spark's MLlib and take Ng's Coursera course, that might be all you need from practical point of view.

wirrbel 9y ago
Deep learning has shown to have obliterated much of the computer vision area and other examples where classical approaches such as feature engineering are naturally hard. This comes at a price of a massive hunger for data of Deep Learning models to achieve an acceptable prediction accuracy. So with use cases like image recognition, etc. the relative performance of deep learning models is much better than with classical algorithms. In general (you'll probably find specific counterexamples for this, but as I said _in general_) the relative performance of deep learning models to classical machine learning may drop below that of classical models when you start looking at medium-sized data sets. And I can assure you, there are many data sets, with many applications for machine learning. And while the world talks about deep learning, I know of many companies, where random forests, suport vector machines or bayesian models have been running for years (which means, cross-validated by data unavailable during model development) with a prediction performance that a business can depend on. I agree with you that Reinforcement learning will as a technology become much more important in the next years. However, only if exploration is cheap enough. I expect Deep Reinforcement learning not to be the answer, at least not in its current state, but I can very well imagine that we will see more machine-learning-algorithm in the reinforcement-learning-loop experiments. I personally would hope to see more research in the bayesian reinforcement learning area.
- bitL 9y ago
  I know, DL works only if you have massive datasets, DRL is even worse for the number of training episodes. Maybe you've heard about recent craziness of using GANs to generate training set fillers when your training sets are small, i.e. you have only 1,000 examples but need 10,000 for reasonable performance. Instead of gathering more examples, you use GAN to create believable training data, and it seems to be working quite well (i.e. a bump from 60% accuracy to 80% while bigger training dataset with real examples would bump you to 90%). What I observed is that many ML companies now run two pipelines in parallel, one based on Deep Learning and the other on classical ML, then cherry pick solutions that work best for the problem/scale they have.
  - wirrbel 9y ago