I studied machine learning 20 years ago, and have only been a casual observer since. I hope someone else corrects me where I'm wrong. I'd say the big change wa

I studied machine learning 20 years ago, and have only been a casual observer since. I hope someone else corrects me where I'm wrong. I'd say the big change was first deep nets, mostly CNNs. 20 years ago, there was a lot of talk about fully connected hidden layers. And how they are equivalent to an ANN with one hidden layer, if you allow exponential scaling of the number of perceptrons. They took forever to converge. CNNs reduce the number of parameters, and hence the training by quite a lot. They also make it obvious that even if multi-hidden layer nets are equivalent theoretically, in practice, you can cheat a lot using convolutions, thereby making this theory pointless. Then came embeddings that segmented training into an embedding stage and a prediction stage, further reducing training time. RNNs never really took off as a name, and seems to have been replaced with LSTM, being a subset of the vague RNN term. Having a well-understood way of introducing memory is very useful for sequences, and makes scaling a model to sequences much cheaper. So yet another input size reducing thing. (I guess we're basically building Markov processes on top of ANNs now? When I was in school, it was an either-or choice.) On a meta-level, there are ensemble models, which is just taking many models, running them in parallel and combining the outputs. Simple idea, but apparently effective enough that people keep doing it. :) Then there was attention being added, which adds a stage where you find similarities between entities. This means the ANN doesn't have to figure clustering, which again speeds up convergence. (I think it also makes the models more modular, but I'm not sure.) Reinforcement learning (Q-learning from the late 1980s) arguably benefits tremendously with all the improvements to the ANN models, so I think time will tell how awesome this will be. For now, it seems we're all amazed by how much just the ANN (Q-function) can do on its own. All of this is enabled by vastly improved matrix arithmetic hardware, software libraries and RAM prices. (Aside from each of these techniques just reducing training time/size.) And then there's the inevitable hype bubble that just turns it all up to 11 for a while, because people with money want to invest and people without money want to pretend to know things. I wouldn't discount the availability of money as a "news" here, though.