5 ms·
The LLMs are getting better that is for sure. But I think there is an inherent weakness in the way they are made that will prevent it from ever reaching a real
by tomohelix 4y ago
The LLMs are getting better that is for sure. But I think there is an inherent weakness in the way they are made that will prevent it from ever reaching a real AGI: it can't improve by itself.
Correct me if I am wrong, but these models and their weights are static after training. So while they can appear different depending on what you prompt them with, their core system never changes. In other words, they can't adapt. They can try their best, but that is it, they can never improve, evolve, learn from their mistake, internalize new knowledge, etc.
One of the best trait of life is adaptability. That was how we human spread everywhere on this planet and how even in the most extreme of environments, you can still find some bacteria living there. And these LLMs are inherently incapable of that.
- grensley 4y agoWhat's preventing an LLM from coding a better LLM?
- tomohelix 4y agoData contamination I think. Right now training heavily depends on human to guide it to what is right and what is wrong. If an AI try to train its successor, it will basically just make a copy of itself and nothing better can be made or taught. We are still better than an LLM, collectively. The best programmer is still miles ahead of GPT, the best writer can write something that make GPT look like a child learning the alphabet, etc. So we can still train it using the data from all of humanity. But if we aren't here, then the best an AI can train is a copy of itself. There are already discussions about where to get more authentic data to train the next generation of LLMs when so much of the internet might have been saturated with AI generated content.
- Buttons840 4y agoI'm imagining a Luddite using a LLM to generate terabytes of slightly wrong text to poison the water.
- geysersam 4y agoTransformer based LLMs were not developed by someone sitting down in deep though. They're a product of trial and error. LLMs are not particularly good at reasoning. They can't make experiments. They can't look up the information in their weights (because no text on the internet has that information). For these reasons they are not likely to be good at designing new LLMs.
- grensley 4y agoDo you think it's possible an LLM might be able to evaluate if another LLM is better than itself?
- Centigonal 4y agoit can definitely evaluate whether an LLM is better aligned than itself (for some narrow definition of "aligned") - this is the motivating principle behind Anthropic's Constitutional AI idea. "Better" is pretty vague, but "more capable" would be difficult, because I don't think ChatGPT has a good idea of its own capabilities.
- geysersam 4y agoIt's possible that a chatbot can be one part of the evaluation process for new chatbots. I'd bet they already are. But they can't be the only part. New models must be evaluated on many different kinds of tasks. And if current models fails at a task, how can they evaluate new models on that task accurately?
- ec109685 4y agoThey do learn from their mistakes while training though. That affects their weights, so yes it’s frozen now, but it wasn’t when the model was being built.
- tomohelix 4y agoThey learned until convergence, which basically is the point where more training doesn't seem to improve output anymore. At which point they call it completed. And that is the best it can ever be. It will never get better or be made to be better unless the entire system is scrapped and retrained. At any point though, an LLM that you can interact with is static. And that is my point. They can't improve once they are "born". And I do not see any feasible way for them to fix this as it is inherently a feature of the LLMs.
- skissane 4y agoChatGPT has a “knowledge-cutoff”. What if you continually fed new information (like a subscription to a news wire service) into the training process so the knowledge cutoff continually moved forward? LLMs do support additional training (fine-tuning). Why couldn’t that fine-tuning be a continual process, absorbing new information every day? What if OpenAI took all their records of public interaction with ChatGPT, and used that to fine-tune ChatGPT? In fact, given upvote/downvote and “regenerate response” allow users to give feedback on responses, maybe that is part of their plan?
- Yoric 4y ago> ChatGPT has a “knowledge-cutoff”. What if you continually fed new information (like a subscription to a news wire service) into the training process so the knowledge cutoff continually moved forward? > LLMs do support additional training (fine-tuning). Why couldn’t that fine-tuning be a continual process, absorbing new information every day? Definitely sounds possible. However, I suspect that it would be extremely expensive in terms of energy requirements: ChatGPT 3 training has already been evaluated to be equivalent to ~100k days of energy use of an average household in the west, iirc, and ChatGPT 4 training was undoubtedly worse. I don't know how much energy it costs to fine-tune a LLM. But I suspect that what you're proposing would involve a cost similar to training ChatGPT 4, except every few iterations of fine-tuning. > What if OpenAI took all their records of public interaction with ChatGPT, and used that to fine-tune ChatGPT? In fact, given upvote/downvote and “regenerate response” allow users to give feedback on responses, maybe that is part of their plan? Given the adversarial nature of Internet, it sounds like this would be used very quickly to poison ChatGPT. Plus, let's face it, there are good chances that we, as users of ChatGPT, are actually worse than the hand-curated sources that were picked to teach ChatGPT.
- dinobones 4y agoThey can learn from mistakes, within the context window. I often will correct GPT4 on a mistake it's made and then ask it to write me a new prompt from scratch that covers the edge case. GPT4 has a 32k token context window. But I see this number the like the 32mb of RAM of computers of old. With Flash Attention, the attention calculation is now basically O(n). So we could maybe someday see context windows in the billions of tokens, where "learning" is just looking back at a mistake in its past and avoiding it.
- tomohelix 4y agoGood point. I guess it is a matter of scale. Maybe when we get to something like billions of token windows or even more, it becomes "memories" for the AIs. And they can probably make better "judgements" using these memories to train next generation of AIs to internalize these improved logics without the need to have the context fed to them. Kinda similar to how we human do it too. It is certainly an exciting time that we are seeing so many parallels between silicon and meatbags. I didn't think I would live to see this.
- orbital-decay 4y agoThe main difference between ML in its current state and biological systems is the huge compute/energy asymmetry between training and inference. Biological systems have it too, but it's much less prominent, as they have neuroplasticity which allows them to learn on the fly. I guess the context window can be considered a (poor) substitute for the missing neuroplasticity, but it's really limited in what it can do.
- teaearlgraycold 4y agoHook an LLM up to an embeddings database and you’ve got something special. Tell it it can query the database and then give it back the highest ranked entries.
- janalsncm 4y agoThis is a textbook example of the “AI effect”, a special instance of moving the goalposts so common in the artificial intelligence domain that it has its own Wikipedia page. A few years ago the best chatbots basically just worked like a phone tree, trying to filter you into pre-written responses. Then GPT3 came along and was able to synthesize entire paragraphs of plausible text. ChatGPT was released only ~4 months ago and I was complaining that due to fundamental shortcomings it wouldn’t be able to play chess. Now it seems that GPT4 can. And that was a change over only 4 months. If the last hill to fight over is whether GPT4 can self improve, well, it’s only a matter of time until GPT4 is placed into some kind of self-improving CICD pipeline.
- luckingfof 4y ago[dead]
- tomohelix 4y agoTrust me I am one of the people absolutely baffled and pleasantly surprised by all the new things these LLMs can do. There are plenty of potential in this tech and I am not so scared to not acknowledge that. But what I am asking is a core feature of these models. The transformer is what make LLMs possible but it is by design static after it was trained. You can't simply install another pipeline or wrap it in a program and fix this problem. From what I understand, RNN isthe self improving type of AI and it was supplanted by the transformer type and one of the key differences between them is the ability to do learning from its own output. Transformer model sacrificed that for the ability to process a large amount of data in parallel. And that is by design. You can't change it as far as I know. I actually don't know that much so that is why I am asking if what I said was wrong. Also, maybe someone will make a new model merging both transformer and RNN. But what I am saying is right now, the type of LLMs we have probably can't learn no matter how sophisticated its training is or how many parameters it is built with.
- Buttons840 4y agoNeither RNNs nor transformers will change their parameters during a forward pass. In production GTP just does a forward pass through the network, but never calculates the parameter gradients, and this would be true if GTP was built on RNNs as well, and without those gradients the AI will not improve.
- BoorishBears 4y agoWhen people say this, I feel like it's useful to frame it in terms of what they think "real AGI" enables, since we don't even know how "real GI" works Before using ChatGPT I couldn't form a clear picture of that, but now it's clear that a lot of the more realistic things we imagined real AGI would enable, are going to be possible with LLMs. I mean, GPT4 can't improve itself... but it can improve your ability to utilize it. You can take a product idea that uses GPT4, feed it to GPT4 and GPT4 will help you write a frontend for it, come up with marketing campaigns, model personas and generate user stories. It can train other LLMs too. After they lowered the message cap, I had the bright idea to use it to generate several pages of fine tuning data for davinci to get better JSON responses. So far it's looking like a success. - It's not like 3.5 couldn't do all of this, but 4 already seems like an uncanny leap over 3.5 in that regard. And afaik we haven't established the ceiling on how much better LLMs will get as we simply spend more time training them, and compute simply gets faster.
- beepbooptheory 4y agoIsn't it even a little more fraught than that? I.e, the sophistication and worth of these models are directly relative to the crude and subjective human labor of feedback upon them. That is, the RLHF part?