3 ms·

> Was it the case that AI was already a thing and kept under wraps until someone pulled the trigger? There's transparent evidence that transformer-based LLMs e

> Was it the case that AI was already a thing and kept under wraps until someone pulled the trigger? There's transparent evidence that transformer-based LLMs existed before ChatGPT or Llama. There was originally GPT-2, which was not a public release but did get reverse engineered into GPT-Neo and GPT-J (among others). Besides that, there was BERT and T5 from Google, Longformers, XLNet, and dozens of other LLMs that predate the modern ones. > So what happened? OpenAI reduced the friction of using AI to a Google login. Other models existed, but were largely not perceived as powerful enough to compete with a hosted solution. The combination of GPT-3's power and relatively-cheap usage fees is what drove it's adoption. Local AI solutions were being developed before OpenAI stepped into the arena, and are continuing to get developed. Many FAANG companies are heavily invested in the openness of AI architectures.

filterfiber 3y ago
> There's transparent evidence that transformer-based LLMs existed before ChatGPT or Llama. Transformers were first published about in "Attention is all you need" in 2017 by google https://arxiv.org/abs/1706.03762 https://arxiv.org/abs/1706.03762 > There was originally GPT-2, which was not a public release but did get reverse engineered into GPT-Neo and GPT-J (among others). GPT 2 had a proper release in 2019 with the code, structure, checkpoints, etc. - https://openai.com/research/gpt-2-1-5b-release https://openai.com/research/gpt-2-1-5b-release > Besides that, there was BERT and T5 from Google, Longformers, XLNet, and dozens of other LLMs that predate the modern ones. Yes all the big players had various research models. We simply crossed the line into large-scale usefulness just recently (outside of classification/summerization), and like you said OpenAI was the first to really make it accessible to the public.

filterfiber 3y ago

> There's transparent evidence that transformer-based LLMs existed before ChatGPT or Llama. Transformers were first published about in "Attention is all you need" in 2017 by google https://arxiv.org/abs/1706.03762 https://arxiv.org/abs/1706.03762 > There was originally GPT-2, which was not a public release but did get reverse engineered into GPT-Neo and GPT-J (among others). GPT 2 had a proper release in 2019 with the code, structure, checkpoints, etc. - https://openai.com/research/gpt-2-1-5b-release https://openai.com/research/gpt-2-1-5b-release > Besides that, there was BERT and T5 from Google, Longformers, XLNet, and dozens of other LLMs that predate the modern ones. Yes all the big players had various research models. We simply crossed the line into large-scale usefulness just recently (outside of classification/summerization), and like you said OpenAI was the first to really make it accessible to the public.