It's just that previously neural networks with more than 3 layers have been prohibitively expensive to train. Now that we've discovered some shortcut tricks and

It's just that previously neural networks with more than 3 layers have been prohibitively expensive to train. Now that we've discovered some shortcut tricks and got the training on to GPUs we can finally have those neural networks with lots of layers. Only the implementation details have changed.