3 ms·
In case there are still any doubts: NO ONE has any technological advantage when it comes to LLMs anymore. All that money that so many companies have burned to
by cs702 3y ago
In case there are still any doubts: NO ONE has any technological advantage when it comes to LLMs anymore.
All that money that so many companies have burned to train giant proprietary models is unlikely to see any payback.
Soon enough, more of those companies will realize it's in their best interest to open their models -- to gain mind-share, and to mess with competitors' plans.
First, it was LLaMA, with up to 65B params, opened against Meta's wishes.
Then, it was LLaMA 2, with up to 70B params, opened by Meta on purpose, to mess with Google's and Microsoft/OpenAI's plans.
Now, it's Falcon 180B. What comes next?
We live in interesting times.
- deleted 3y ago[deleted]
- ilaksh 3y agoThat's not true yet. The MoE of GPT-4 is not in open models is it? OpenAI has done extensive RLHF. If you compare the code generation ability of GPT-4 with Falcon 140B, it is clear that all of their efforts are still paying off. There is still a gap in efficiency and performance. I mean, the gap has definitely narrowed though. And I think it's true that they will have trouble recovering all of the billions. But at least some of it has been paid back already.
- cs702 3y agoWe're not in disagreement: It's the trend that matters here. The main advantage OpenAI has over competitors is no longer technology. It's developers, developers, developers. According to OpenAI, there are ~2 million developers worldwide using OpenAI's APIs to build applications.