Deep learning models are neural networks, but their recent popularization is due to a new method of building them incrementally, adding generative hidden layers

Deep learning models are neural networks, but their recent popularization is due to a new method of building them incrementally, adding generative hidden layers trained as autoencoders to extract representative features, until the final discriminative layer. The resulting model is still a neural network, which can be finetuned by gradient methods, though conventionally training the same model from scratch with a random initialization would not have worked.