It seems amazing to me what the transformer can learn to SOTA levels, not just language but also images, video, code, math and proteins. Replacing so much handm

It seems amazing to me what the transformer can learn to SOTA levels, not just language but also images, video, code, math and proteins. Replacing so much handmade neural architecture with just one thing that does it all, that was an amazing step forward.