4 ms·

> Everyone's trying to train GPT models to write code, but maybe we should be training them how to use a debugger. Though its a lot harder to map text generatio

> Everyone's trying to train GPT models to write code, but maybe we should be training them how to use a debugger. Though its a lot harder to map text generation to debugging... AI lay-person, web developer here. This hits at something that's been nagging at me for a few weeks. LLMs get labelled "intelligent", but they're "just" spitting out word/symbol patterns. They don't comprehend the meaning of the patterns (as best I have come to understand it from reading news). Deeper than this, "intelligence" isn't rooted in words. It's rooted in experience, logic and their interaction. The words are a side-effect; they're a way to label the experiences. We're not training LLMs with experiences, only a reference to experience had by others, which may or may not map to the experience of the interlocutor. The LLM has no experience and limited/no logic to wrestle those experiences into wisdom. Debugging is a "pursuit of wisdom" activity. Statistics, which underpin LLMs often identify truths (we wouldn't use them in science if they didn't), but they also hide truths. You need to clean data, take unusual slices of it, recombine it, and otherwise process it in order to uncover the true truths. I'm not sure LLMs can do that yet (if at all), and strongly doubt they can without _experience_ (sensory input, intention, failure etc).

Ygg2 3y ago
> LLMs get labelled "intelligent", but they're "just" spitting out word/symbol patterns. I think the word you are looking is knowledgeable. They have huge knowledge repository and can remix it in whatever way they please, but I don't think if you gave them a novel problem (outside their realm of knowledge) they could solve it.
- gaogao 3y ago
  The unicorn example from MSFT's Sparks of AGI paper, https://arxiv.org/abs/2303.12712 https://arxiv.org/abs/2303.12712, probably counts as a novel problem. As another example, ImageBlind, https://ai.facebook.com/blog/imagebind-six-modalities-binding-ai/ https://ai.facebook.com/blog/imagebind-six-modalities-bindin..., learned modality translations explicitly not in its training set such as text -> sound.
Version467 3y ago
> They don't comprehend the meaning of the patterns (as best I have come to understand it from reading news). It is not obvious to me that this is true. The argument against this is that a model has to develop an abstract understanding of the world in order to get better at predicting the next token. You'll find different opinions as to wether this happens with todays llm's, but it's definitely not a clear cut issue.
- ankurdhama 3y ago
  > The argument against this is that a model has to develop an abstract understanding of the world in order to get better at predicting the next token. I think they only develop abstract understanding about the relationships of the words/group of words.

Ygg2 3y ago

> LLMs get labelled "intelligent", but they're "just" spitting out word/symbol patterns. I think the word you are looking is knowledgeable. They have huge knowledge repository and can remix it in whatever way they please, but I don't think if you gave them a novel problem (outside their realm of knowledge) they could solve it.

gaogao 3y ago
The unicorn example from MSFT's Sparks of AGI paper, https://arxiv.org/abs/2303.12712 https://arxiv.org/abs/2303.12712, probably counts as a novel problem. As another example, ImageBlind, https://ai.facebook.com/blog/imagebind-six-modalities-binding-ai/ https://ai.facebook.com/blog/imagebind-six-modalities-bindin..., learned modality translations explicitly not in its training set such as text -> sound.

Version467 3y ago

> They don't comprehend the meaning of the patterns (as best I have come to understand it from reading news). It is not obvious to me that this is true. The argument against this is that a model has to develop an abstract understanding of the world in order to get better at predicting the next token. You'll find different opinions as to wether this happens with todays llm's, but it's definitely not a clear cut issue.

ankurdhama 3y ago
> The argument against this is that a model has to develop an abstract understanding of the world in order to get better at predicting the next token. I think they only develop abstract understanding about the relationships of the words/group of words.