3 ms·
I understand your point. The gap in communication is we don't mean _literally_ no one _ever_ open-sourced models. I agree, that would be absurd. [1] Companies
by refulgentis 3y ago
I understand your point.
The gap in communication is we don't mean _literally_ no one _ever_ open-sourced models. I agree, that would be absurd. [1]
Companies, quite infamously and well-understood, _did_ hold back their "real" generative models, even from being available for pay.
Take a stab at a literal definition:
- post-GPT2 LLMs (ex. PALM, PALM2)
- art like DaLL-E, Imagen, Parti
Loosely, we had Disco Diffusion for art, and GPT-3 for LLMs, and then Dall-E, then Midjourney. That was over an _entire year_, and the floodgates on private ones didn't open till post SD/ChatGPT.
[1] thank you for the lengths you went to highlight the best over a considered span of time, I would have just said something snarky :)
[2] I did not realize FLAN was open-sourced a month before ChatGPT, that's fascinating: we're stretching a bit, beyond that, IMHO: the BERTs aren't recognizable as LLMs.
- smoldesu 3y agoAll good. I've also been working on LLMs since 2019-ish, so I wanted to toss a hat in the ring for the underrepresented transformer models. They were cool (eg. dumb), fast and worked better than they had any right to. In a lot of ways they are the ancestors of ChatGPT and Llama, so it's important to at least bring them into the discussion.
- deleted 3y ago[deleted]