4 ms·

The problem isn’t that ML has gotten worse. It’s just as rigorous and far more powerful than it ever was. The problem is ML is hard, it hasn’t gotten orders-of-

The problem isn’t that ML has gotten worse. It’s just as rigorous and far more powerful than it ever was. The problem is ML is hard, it hasn’t gotten orders-of-magnitude easier to understand, and there’s enormous incentive now to pass off amateur understanding as complete. The real ML still happens — it’s just drowned out.

candiodari 8y ago
I think people are saying that that's something they're not happy with. The big methods in AI, like backprop, 1) work a LOT better for specific problems than statistics or statistical learning ever has (and at this point, I think we can safely say: ever will) 2) a lot of methods either can't be explained, or outright shouldn't work, according to statistical theory. The use of statistics in machine learning is limited to evaluating performance and individual element performance (and even that is tenuous at best in many cases). If you ask, say, why would an autoencoder, with an LSTM on it's compressed representation and Q-learning evaluation have somewhat decent performance on half the computer games humans ever designed ? Statistics will not be useful in formulating an answer. If you ask extremely valid questions, like "why would an LSTM predict anything ?". Statistics draws a blank. There is no good reason to assume an LSTM will ever converge (and on a truly random dataset, it won't, whereas statistical methods will still allow you to say something). I think there's 2 reasons for this 1) the "upper limit" of complexity a human can understand in a statistical model is lower than the upper limit a neural network can "understand". In statistics the human understanding is critical to getting to a valid model, in machine learning ... it is not. Meaning machine learning can learn relationships a human mind cannot. 2) There must be some fundamental property of the world we live in that matches neural network architecture. In order for backprop to work on real-world problems, it has to be the case that almost all real world phenomena are continuous, both "raw" and in the frequency domain. If this wasn't the case, machine learning would never be able to learn anything.
- p1esk 8y ago
  can't be explained, or outright shouldn't work, according to statistical theory When people invented a steam powered engine, some other people had probably said: "Modern physics can't explain how it works. It shouldn't work. It's too complicated". Then a few decades later physicists discover laws of thermodynamics.

candiodari 8y ago

I think people are saying that that's something they're not happy with. The big methods in AI, like backprop, 1) work a LOT better for specific problems than statistics or statistical learning ever has (and at this point, I think we can safely say: ever will) 2) a lot of methods either can't be explained, or outright shouldn't work, according to statistical theory. The use of statistics in machine learning is limited to evaluating performance and individual element performance (and even that is tenuous at best in many cases). If you ask, say, why would an autoencoder, with an LSTM on it's compressed representation and Q-learning evaluation have somewhat decent performance on half the computer games humans ever designed ? Statistics will not be useful in formulating an answer. If you ask extremely valid questions, like "why would an LSTM predict anything ?". Statistics draws a blank. There is no good reason to assume an LSTM will ever converge (and on a truly random dataset, it won't, whereas statistical methods will still allow you to say something). I think there's 2 reasons for this 1) the "upper limit" of complexity a human can understand in a statistical model is lower than the upper limit a neural network can "understand". In statistics the human understanding is critical to getting to a valid model, in machine learning ... it is not. Meaning machine learning can learn relationships a human mind cannot. 2) There must be some fundamental property of the world we live in that matches neural network architecture. In order for backprop to work on real-world problems, it has to be the case that almost all real world phenomena are continuous, both "raw" and in the frequency domain. If this wasn't the case, machine learning would never be able to learn anything.

p1esk 8y ago
can't be explained, or outright shouldn't work, according to statistical theory When people invented a steam powered engine, some other people had probably said: "Modern physics can't explain how it works. It shouldn't work. It's too complicated". Then a few decades later physicists discover laws of thermodynamics.