4 ms·
The issue as I see it is that google invented transformers way before they were released publicly. Clearly there were not enough resources being spent on them w
by HDThoreaun 11mo ago
The issue as I see it is that google invented transformers way before they were released publicly. Clearly there were not enough resources being spent on them which is why the scale movement came about. Would google still be hoarding transformer based LLMs today without Ilya’s hype? Seems like a real possibility to me.
- godelski 11mo agoI'm not sure how you get there. The SIAYN movement happened after the AIAYN paper. The latter influenced the name of the former. So what do you mean by secret? It was published in a paper. That's public
- HDThoreaun 11mo agoNo one was releasing a product based on the paper though. Ilya had to go and raise a bunch of money for that to happen. Maybe I’m just more cynical and accepting of lying as the way things are done compared to you.