3 ms·
I’m sorry … but what?! As someone who worked on early LLM tech at Google hundreds of millions were poured into it. Do you think that paper just spontaneously ca
by tsunamifury 3y ago
I’m sorry … but what?! As someone who worked on early
LLM tech at Google hundreds of millions were poured into it. Do you think that paper just spontaneously came from pure theory?
- logicchains 3y agoMost of the core idea of transformers was invented in the early 1990s, including what at the time were termed Fast Weight Programmers and are formally equivalent to linearised self-attention: https://people.idsia.ch/~juergen/fast-weight-programmer-1991-transformer.html https://people.idsia.ch/~juergen/fast-weight-programmer-1991... . Google just had the hardware to actually run and experiment with large transformers.
- tsunamifury 3y agoJesus, the arrogance here. Are HNers this ridiculously out of Touch. That is the investment and it was no small price. JUST is carrying a ton of weight here.
- camjw 3y agoAt least you concede then that Google invested in the hardware.
- 7373737373 3y agoIt would be very funny if at some point someone stood up to Schmidhuber and told him they already created something he did, but yet another 30 years earlier.