5 ms·
Here in the thread he says: https://x.com/jacobaustin132/status/1886844724339675340 https://x.com/jacobaustin132/status/1886844724339675340 : `5 years ago, ther
by awongh 2y ago
Here in the thread he says: https://x.com/jacobaustin132/status/1886844724339675340 https://x.com/jacobaustin132/status/1886844724339675340 : `5 years ago, there were many ML architectures, but today, there is (mostly) only one [transformers].`
To what degree is this actually true, and what else is on the horizon that might become as popular as transformers?
- swyx 2y agoit's quite true. the convergence of all archs to transformers is well documented by karpathy. SSMs were once touted as transformer killers, but increasingly look like just optional supplements.