3 ms·

> Despite all the hoopla, LLMs are not AGI or artifical brains - they are predict-next-word language models. By design they are not built for creativity, but ra

> Despite all the hoopla, LLMs are not AGI or artifical brains - they are predict-next-word language models. By design they are not built for creativity, but rather quite the opposite, they are designed to continue the input in the way best suggested by the training data - they are essentially built for recall, not creativity. This is just a completely base level of understanding of LLMs. How do you predict the next token with superhuman accuracy? Really think about how that is possible. If you think it's just stochastic parroting you are ngmi. >large language models have yet to produce a genuine breakthrough. The puzzle is why. I think you should really update on the fact that world class researchers are surprised by this. They understand something you don't and that is that it's clear these models build robust world models and that text prompts act as probes into those world models. The surprising part is that despite these sophisticated world models we can't seem to get unique insights out which almost surely already exist in those models. Even if all the model is capable of is memorizing text then just the sheer volume it has memorized should yield unique insights, no human can ever hope to hold this much text in their memory and then make connections between it. It's possible we just lack the prompt creativity to get these insights out but nevertheless there is something strange happening here.

HarHarVeryFunny 1y ago
> This is just a completely base level of understanding of LLMs. How do you predict the next token with superhuman accuracy? Really think about how that is possible. If you think it's just stochastic parroting you are ngmi. Yes, thank-you, I do understand how LLMs work. They learn a lot of generative rules from the training data, and will apply them in flexible fashion according to the context patterns they have learnt. You said stochastic parroting, not me. However, we're not discussing whether LLMs can be superhuman at tasks where they had the necessary training - we're discussing whether they are capable of creativity (and presumably not just the trivially obvious case of being able to apply their generative rules in any order - deductive closure, not stochastic parroting in the dumbest sense of that expression).
- vonneumannstan 1y ago
  >However, we're not discussing whether LLMs can be superhuman at tasks where they had the necessary training - we're discussing whether they are capable of creativity "Even if all the model is capable of is memorizing text then just the sheer volume it has memorized should yield unique insights, no human can ever hope to hold this much text in their memory and then make connections between it." Unless you think humans have magic meat then all we are really doing with "creativity" is connecting previously unconnected facts.
  - HarHarVeryFunny 1y ago
    > Unless you think humans have magic meat then all we are really doing with "creativity" is connecting previously unconnected facts. In the case of discovery and invention the "facts" being connected may not be things that were known before. An LLM is bound by it's training set. A human is not limited by what is currently known - they can explore (in directed or undirected fashion), learn, build hierarchies of new knowledge and understanding, etc.
  - HarHarVeryFunny 1y ago
    > "Even if all the model is capable of is memorizing text then just the sheer volume it has memorized should yield unique insights, no human can ever hope to hold this much text in their memory and then make connections between it." Yes, potentially, but the model has no curiosity or drive to do this (or anything else) by itself. All an LLM is built to do is predict. The only way to control the output and goad it into using the vast amount of knowledge that it has is by highly specific prompting. Basically it's only going to connect the dots if you tell what dots to connect, in which case it's the human being inventive, not the model. The model is trying to predict, so essentially if you want it to do something outside of the training set you're going to have to prompt it to do that. A human has curiosity (e.g. "what happens if I connect these dots .."), based on prediction failure and associated focus/etc - the innate desire to explore the unknown and therefore potentially learn. The model has none of that - it can't learn and has no curiosity. If the model's predictions are bad it will just hallucinate and generate garbage, perhaps "backtrack" and try again, and likely lead to context rot.
awongh 1y ago
Not just prompting, it also could be we haven't done the right kind of RLHF for these kinds of outputs?

HarHarVeryFunny 1y ago

> This is just a completely base level of understanding of LLMs. How do you predict the next token with superhuman accuracy? Really think about how that is possible. If you think it's just stochastic parroting you are ngmi. Yes, thank-you, I do understand how LLMs work. They learn a lot of generative rules from the training data, and will apply them in flexible fashion according to the context patterns they have learnt. You said stochastic parroting, not me. However, we're not discussing whether LLMs can be superhuman at tasks where they had the necessary training - we're discussing whether they are capable of creativity (and presumably not just the trivially obvious case of being able to apply their generative rules in any order - deductive closure, not stochastic parroting in the dumbest sense of that expression).

vonneumannstan 1y ago
>However, we're not discussing whether LLMs can be superhuman at tasks where they had the necessary training - we're discussing whether they are capable of creativity "Even if all the model is capable of is memorizing text then just the sheer volume it has memorized should yield unique insights, no human can ever hope to hold this much text in their memory and then make connections between it." Unless you think humans have magic meat then all we are really doing with "creativity" is connecting previously unconnected facts.
- HarHarVeryFunny 1y ago
  > Unless you think humans have magic meat then all we are really doing with "creativity" is connecting previously unconnected facts. In the case of discovery and invention the "facts" being connected may not be things that were known before. An LLM is bound by it's training set. A human is not limited by what is currently known - they can explore (in directed or undirected fashion), learn, build hierarchies of new knowledge and understanding, etc.
- HarHarVeryFunny 1y ago
  > "Even if all the model is capable of is memorizing text then just the sheer volume it has memorized should yield unique insights, no human can ever hope to hold this much text in their memory and then make connections between it." Yes, potentially, but the model has no curiosity or drive to do this (or anything else) by itself. All an LLM is built to do is predict. The only way to control the output and goad it into using the vast amount of knowledge that it has is by highly specific prompting. Basically it's only going to connect the dots if you tell what dots to connect, in which case it's the human being inventive, not the model. The model is trying to predict, so essentially if you want it to do something outside of the training set you're going to have to prompt it to do that. A human has curiosity (e.g. "what happens if I connect these dots .."), based on prediction failure and associated focus/etc - the innate desire to explore the unknown and therefore potentially learn. The model has none of that - it can't learn and has no curiosity. If the model's predictions are bad it will just hallucinate and generate garbage, perhaps "backtrack" and try again, and likely lead to context rot.

awongh 1y ago

Not just prompting, it also could be we haven't done the right kind of RLHF for these kinds of outputs?