3 ms·
The Transformer architecture that powers GPT models is based on a now-famous Google Brain paper, Attention is All You Need. You're right that OpenAI has product
by Karrot_Kream 4y ago
The Transformer architecture that powers GPT models is based on a now-famous Google Brain paper, Attention is All You Need. You're right that OpenAI has productized this research much faster than Big Tech, but the research itself came from established R&D in Academia and Big Tech.
- nr2x 4y agoYes, but only one of the authors is still at google.
- pclmulqdq 4y agoWell, Google had to figure out how to cancel itself somehow. That's the only Google project, aside from search, that has had staying power.
- yreg 4y agoAnd Stable Diffusion is based on Google's Imagen paper.
- 6gvONxR4sf7o 4y agoThe diffusion papers most analogous to the transformer paper are from Berkeley, Stanford, and Freiburg. Imagen wasn’t foundational in the way you’re suggesting.