Transformer is an evolution of large scale training of seq2seq. Diffusion models is an evolution of even older approach - training denoising autoencoders. It h

Transformer is an evolution of large scale training of seq2seq. Diffusion models is an evolution of even older approach - training denoising autoencoders. It had been a relatively gradual and accelerating progress, with number of people working in the field increasing exponentially, since 2010 or so, when Deep Learning on GPUs was popularized at NIPS by Theano. Tens of thousands people working together on Deep Learning. Many more on GPUs.