>> are now going to hit a new plateau in machine learning- or whether "this time it's different" I think it is actually both, yes we are going to hit a new pla

>> are now going to hit a new plateau in machine learning- or whether "this time it's different" I think it is actually both, yes we are going to hit a new plateau and yes this time it's different. It is different not because we have found something profoundly new, but because we are able to quickly, easily and successfully experiment with huge (deep), new neural network architectures and learning methodologies. This has become possible because a combination of factors that have come together towards the end of the 2000’s: e.g. much more computation power (GPGPUs), much more data available online, "simple" insigths such as progressive training of deep nets by stacking (auto encoder) layers, "Hey! Stochastic Gradient Descent works quite well actually!", Drop-out to improve generalisation capabilities, etc.. The great open source libraries such as TensorFlow, Theano and others make it even easier to do experiments. A framework like Keras even abstracts TensorFlow & Theano so you don’t have to worry what is used as deep learning framework. So we shifted to a much higher gear when it comes to machine learning research, and this will be like this for a while. Computing capabilities keep expanding: GPGPUs become ever faster for Deep Learning, but also Intel has the Xeon Phi Knights Landing with 72 cores and upcoming variants with Deep Learning specific instructions (Knights Mill). On the other hand we will definitely hit a plateau: 1) To make truly intelligent systems, we need to encode a lot of knowledge; knowledge that is common to us, but not at all to machines. Bootstrapping a general AI with human-like intelligence, will prove very difficult. I think such AIs need to develop just like children acquire knowledge and cognitively develop. The type of problems we encounter to achieve this are of a whole other type, for one, just imagine how much time this will take before we get this right! Imagine an AI that learns for a few years but fails to improve, can we reuse what it has learned in a new and better version of the AI? Will we capture all of its experiences to relearn a new version from scratch? 2) Apparently the human brain has a 100 billion neurons and trillions of connections, AlexNet (2012) has 650,000 neurons and 60 million parameters. We have grown the networks considerably since then, but compared to the complexity of the brain we have a (very, very) long way to go. 3) FLOPS/Watt : this is going to play an ever growing role in the success of AI. Our brain is incredibly efficient when it comes to energy use. We shouldn’t need a power plant for every robot we deliver to customers, right?