We've never had architectures that scale so effectively, unlocking new cognitive capabilities by just increasing parameters/exaflops/datasets without writing a

We've never had architectures that scale so effectively, unlocking new cognitive capabilities by just increasing parameters/exaflops/datasets without writing a lot more code or changing the architecture. Ilya Sutskever mentioned this in some interview, that transformers are the first with that property but probably won't be the last or best.