3 ms·
> right now OpenAI has some kind of a monopoly in commercial large language models No? That's absolutely false. You can, using commodity hardware, host your ow
by smoldesu 4y ago
> right now OpenAI has some kind of a monopoly in commercial large language models
No? That's absolutely false. You can, using commodity hardware, host your own GPT-based text transformer. There is no "monopoly" just because OpenAI provides an API and hosts the stuff themselves. They have a "monopoly" on the research used to create the later versions of GPT, but that's no more of a "monopoly" than Apple's monopoly on the iOS source code.
Comparable products can be made and released right now. I presume Google and Microsoft are working on integrating AI into their software, but what's the rush? Nobody wants to integrate faulty AI in their software, especially if it's as consistently wrong as ChatGPT is. Tuning this stuff will take time, and making people trust AI will take something much more impressive than ChatGPT. And considering how expensive these larger AI models are to train, it's completely understandable why nobody wants to pour $12 million dollars into building an Infinite Liability Machine.
- behnamoh 4y agoTraining costs of large models are prohibitive. Only the big tech can afford to make insanely large models.
- smoldesu 4y agoYou don't need to. As I said, there are freely-available, pre-trained GPT models you can run on your own computer: https://huggingface.co/EleutherAI/gpt-neo-2.7B https://huggingface.co/EleutherAI/gpt-neo-2.7B No one has a monopoly on LLMs. You just can't run OpenAI's latest-and-greatest, because... why should you?
- behnamoh 4y agoGPT-3's performance is much much better than any open-source alternative that I've tried (Bloom, GPT-J, etc.) which is mostly due to parameter size. Like I said, OpenAI has the monopoly in "large" language models, but of course you can have smaller models.
- deleted 4y ago[deleted]
- muzani 4y agoGLaM, Gopher, LaMDA. Google alone has plenty of huge LLMs. ChatGPT is a sample of where AI currently is at, and many people were just taken by surprise because they didn't expect AI to be at the current level. OpenAI had to raise funds and interest, and got a home run.