5 ms·
We're a long ways from "Peak LLM", if we will ever get there. If we are, indeed, in a virtuous cycle of LLMs building on each other, then we are actually in th
by ukuina 4y ago
We're a long ways from "Peak LLM", if we will ever get there.
If we are, indeed, in a virtuous cycle of LLMs building on each other, then we are actually in the knee of the curve before exponential increase in LLM capability.
An LLM that can access all other AI models (e.g., HuggingGPT) is not limited to the strengths and weaknesses of any one model. Declarations of "Peak LLM" or "LLMs can never be secured" are as laughable as statements like "Assembly can never be surpassed in abstraction".
- concerned_ 4y agoWill we ever break free of the 10,000 monkeys typing Shakespeare problem? 10,000 LLMs doesn't fix that
- sitkack 4y agoThat hasn't been determined http://incompleteideas.net/IncIdeas/BitterLesson.html http://incompleteideas.net/IncIdeas/BitterLesson.html Sparks of Artificial General Intelligence: Early experiments with GPT-4 https://arxiv.org/abs/2303.12712 https://arxiv.org/abs/2303.12712 LLMs exhibit emergent properties as they scale, we should assume the same will happen as we run divergent models in parallel. By asking a rhetorical question and then refuting a position that wasn't asked is a Straw Man, the reference to 10k monkeys is a false analogy, your 10k LLMs answer to the question no one asked is a hasty generalization. How have you shown that 10k LLMs won't fix straw-problem?
- Nevermark 4y agoI pasted the beginning of Hamlet into GPT-4 and it went on a run. So it seems that the chance of producing one of Shakespeare works no longer requires each work in the play to be randomly chosen in isolation, just enough correct word guesses to get the LLM into the groove. "ChatGPT, please generate 100 random words, then interpret them as the beginning of a literary work and complete the work." This is real progress. Many many monkeys may no longer be needed.
- misnome 4y agoJust imagine the amazing new colour I can make by mixing all these other ones together!
- ukuina 4y agoIt is closer to "Look at the amazing new tool I can make using all these other ones together!"
- soneca 4y ago> ”if we will ever get there.” What do you mean with this? There might never be a peak for something? It doesn’t make much sense to me, so I read it as a flag that your position is more faith-based (or “hope-based” for a less loaded word) than fact-based. I could be wrong in this interpretation of course, so the initial question in my comment is a genuine one.
- Taek 4y agoIt means LLMs might self-improve beyond the point where we can comprehend how intelligent they are. An LLM with 10x the capabilities of all human brains combined is indistinguishable to a human with one that has 100x the capabilities of all humanity combined, effectively making it possible for there to be "no peak"
- ihatepython 4y agoI don't think you understand how LLMs work
- Taek 4y agoI think you underestimate how intelligent LLMs are. If the training data only explains a certain concept in French, the LLM will nonetheless be able to tell you about that concept in any other language it has proficiency in. There's clearly a lot more going on under the hood than just a sophisticated markov chain.
- smackeyacky 4y agoHard no. LLMs devouring the output of LLMs will only result in noise. They already make up garbage and it's only going to get worse.