Sutton’s bitter lesson http://www.incompleteideas.net/IncIdeas/BitterLesson.html http://www.incompleteideas.net/IncIdeas/BitterLesson.html resonates. General l

Sutton’s bitter lesson http://www.incompleteideas.net/IncIdeas/BitterLesson.html http://www.incompleteideas.net/IncIdeas/BitterLesson.html resonates. General learning + search methods that scale with data + compute win out. Transformers in a sense is one of the most general algorithms able to tackle a whole bunch of domains from audio, video, images, language. I’m sure another better super algorithm will be invented but Transformer is currently what we have.