3 ms·

What's preventing an LLM from coding a better LLM?

tomohelix 4y ago
Data contamination I think. Right now training heavily depends on human to guide it to what is right and what is wrong. If an AI try to train its successor, it will basically just make a copy of itself and nothing better can be made or taught. We are still better than an LLM, collectively. The best programmer is still miles ahead of GPT, the best writer can write something that make GPT look like a child learning the alphabet, etc. So we can still train it using the data from all of humanity. But if we aren't here, then the best an AI can train is a copy of itself. There are already discussions about where to get more authentic data to train the next generation of LLMs when so much of the internet might have been saturated with AI generated content.
- Buttons840 4y ago
  I'm imagining a Luddite using a LLM to generate terabytes of slightly wrong text to poison the water.
  - Yoric 4y ago
    You mean like $(FAMOUS BILLIONAIRE) who claimed he would develop an AI aligned with the values of $(POLITICAL PARTY)?
  - zamnos 4y ago
    One could setup a blog using GPT-2 to generate the text for that purpose.
geysersam 4y ago
Transformer based LLMs were not developed by someone sitting down in deep though. They're a product of trial and error. LLMs are not particularly good at reasoning. They can't make experiments. They can't look up the information in their weights (because no text on the internet has that information). For these reasons they are not likely to be good at designing new LLMs.
- grensley 4y ago
  Do you think it's possible an LLM might be able to evaluate if another LLM is better than itself?
  - Centigonal 4y ago
    it can definitely evaluate whether an LLM is better aligned than itself (for some narrow definition of "aligned") - this is the motivating principle behind Anthropic's Constitutional AI idea. "Better" is pretty vague, but "more capable" would be difficult, because I don't think ChatGPT has a good idea of its own capabilities.
  - geysersam 4y ago
    It's possible that a chatbot can be one part of the evaluation process for new chatbots. I'd bet they already are. But they can't be the only part. New models must be evaluated on many different kinds of tasks. And if current models fails at a task, how can they evaluate new models on that task accurately?

Data contamination I think. Right now training heavily depends on human to guide it to what is right and what is wrong. If an AI try to train its successor, it will basically just make a copy of itself and nothing better can be made or taught. We are still better than an LLM, collectively. The best programmer is still miles ahead of GPT, the best writer can write something that make GPT look like a child learning the alphabet, etc. So we can still train it using the data from all of humanity. But if we aren't here, then the best an AI can train is a copy of itself. There are already discussions about where to get more authentic data to train the next generation of LLMs when so much of the internet might have been saturated with AI generated content.

Buttons840 4y ago
I'm imagining a Luddite using a LLM to generate terabytes of slightly wrong text to poison the water.
- Yoric 4y ago
  You mean like $(FAMOUS BILLIONAIRE) who claimed he would develop an AI aligned with the values of $(POLITICAL PARTY)?
- zamnos 4y ago
  One could setup a blog using GPT-2 to generate the text for that purpose.

geysersam 4y ago

Transformer based LLMs were not developed by someone sitting down in deep though. They're a product of trial and error. LLMs are not particularly good at reasoning. They can't make experiments. They can't look up the information in their weights (because no text on the internet has that information). For these reasons they are not likely to be good at designing new LLMs.

grensley 4y ago
Do you think it's possible an LLM might be able to evaluate if another LLM is better than itself?
- Centigonal 4y ago
  it can definitely evaluate whether an LLM is better aligned than itself (for some narrow definition of "aligned") - this is the motivating principle behind Anthropic's Constitutional AI idea. "Better" is pretty vague, but "more capable" would be difficult, because I don't think ChatGPT has a good idea of its own capabilities.
- geysersam 4y ago
  It's possible that a chatbot can be one part of the evaluation process for new chatbots. I'd bet they already are. But they can't be the only part. New models must be evaluated on many different kinds of tasks. And if current models fails at a task, how can they evaluate new models on that task accurately?