3 ms·
> Not necessarily? Well then, I expect we'll see a bunch of small projects that beat the big players any day now. I won't hold my breath.
by qarl 3y ago
> Not necessarily?
Well then, I expect we'll see a bunch of small projects that beat the big players any day now. I won't hold my breath.
- edgyquant 3y agoAgain you haven’t a clue of the point here. They were created as a research lab and instead after taking a paper created at Google they went closed source and are simply scaling that instead of working to create new and improved models that can actually run on reasonable hardware. Small transformers being able to beat the same models but scaled up is unrelated to anything being discussed and you just seem like a fanboy at this point
- smoldesu 3y agoYou're basically describing what happened to GPT-2 when T5-flan came out. Not to mention, the incumbent model at the time (BERT) was extremely competitive for it's smaller size. Pretty much nobody was talking about OpenAI's models back then because they were too large to be realistically useful. So yeah, I do actually anticipate smaller projects that devalue turtle-tower model scaling.
- qarl 3y agoCool. Send me a note when that happens.
- deleted 3y ago[deleted]