3 ms·
>On pure research they invented the transformer in 2017 (Vaswani) Transformers were invented in the early 90s: https://people.idsia.ch/~juergen/fast-weight-pro
by logicchains 3y ago
>On pure research they invented the transformer in 2017 (Vaswani)
Transformers were invented in the early 90s: https://people.idsia.ch/~juergen/fast-weight-programmer-1991-transformer.html https://people.idsia.ch/~juergen/fast-weight-programmer-1991... Google were just the first with the compute to scale them up.
- blovescoffee 3y agofwiw Schmidhuber is famous for claiming to have come up with virtually every architecture that's now useful/important.