3 ms·
I've had a feeling for a while that they've hit a plateau in terms of how much "intelligence" they can squeeze from all the text in the world. Many openai and o
by aster0id 3y ago
I've had a feeling for a while that they've hit a plateau in terms of how much "intelligence" they can squeeze from all the text in the world. Many openai and other researchers have publicly claimed that they need better data for better models. This to me feels like a desperate attempt to collect more data than they already are to improve their models.
- fullymanaged 3y agoYour feelings are dead wrong. First, they have not hit a plateau. If you're in any ways involved with AI research you'd know that there is an insane amount of low hanging fruit in terms of data (synthetic and real world), architectural improvements, loss function improvements and scaling. I also doubt their main motivation is collecting more data. Their motivation is directly competing with Google on their search business. The early success of Perplexity has shown that an answer engine built on top of LLMs is an improvement over a list of ranked web pages. OpenAI would be stupid to not go after that market, especially given the kind of mind-share they already have. It's clear that Google is stuck facing the innovator's dilemma. And this is not speculation. Sama said (in Lex's podcast) that taking on Google is something that he's very interested in. Coincidentally it also aligns with OpenAIs mission of making this tech benefit everyone.
- aster0id 3y agoI hope I am wrong. All I've seen in the past year are incrementally better models even from the top labs. Open source models are barely usable.