3 ms·

That's easy: ask it anything, then "correct" it with some outrageous nonsense. It will apologize (as if to express regret), and say you're correct, and now the

by j0057 4y ago

That's easy: ask it anything, then "correct" it with some outrageous nonsense. It will apologize (as if to express regret), and say you're correct, and now the conversation is poisoned with whatever nonsense you fed it. All form and zero substance. We fall for it because normally the use of language is an expression of something, with ChatGPT language is just that, language, with no meaning. To me that proves knowing and reasoning happens on a deeper, more symbolic level and language is an expression of that, as are other things.

chaxor 4y ago
This isn't always true, especially not with gpt-4. Also, this isn't a proof really, or even evidence to show that it doesn't 'know' something. It appears to reason well about many tasks - specifically "under the hood" (reasoning not explicitly stated within the output provided). Yes, of course "it's just a language model" is spouted over and over and is sometimes true (though obviously not for gpt-4), but that statement does not provide any insight at all, and it certainly does not necessarily limit the capability of 'obtaining knowledge'.
- TuringTest 4y ago
  'Reasoning' (as in deriving new statements with precision, following logical inference rules) is precisely what the large language models can't do. It's better to think of this models as 'generating' chains of relevant words, where 'relevant' is defined by similarity of those areas of knowledge on which it has been trained, and which are "activated" as close to the topics in the prompt. Which is not at all dissimilar to how humans learn about a new topic, btw. This way, by "activating" concepts of areas of knowledge and finding words that are more likely than others to fit those concepts, the model is able to create texts following the constraints you instruct it with - such as poems that rhyme, or critical analysis of scientific articles. The most important point to be aware of is that this creation model is completely different to how automatic reasoning models create content, which is by having a formal representation of a knowledge domain and creating logical inferences that can be mathematically proven correct within the model. A reasoning model cannot lie, but it cannot create content beyond the logical implications of its premises; its quite the opposite of what language models do.

by j0057 4y ago

chaxor 4y ago

This isn't always true, especially not with gpt-4. Also, this isn't a proof really, or even evidence to show that it doesn't 'know' something. It appears to reason well about many tasks - specifically "under the hood" (reasoning not explicitly stated within the output provided). Yes, of course "it's just a language model" is spouted over and over and is sometimes true (though obviously not for gpt-4), but that statement does not provide any insight at all, and it certainly does not necessarily limit the capability of 'obtaining knowledge'.

TuringTest 4y ago
'Reasoning' (as in deriving new statements with precision, following logical inference rules) is precisely what the large language models can't do. It's better to think of this models as 'generating' chains of relevant words, where 'relevant' is defined by similarity of those areas of knowledge on which it has been trained, and which are "activated" as close to the topics in the prompt. Which is not at all dissimilar to how humans learn about a new topic, btw. This way, by "activating" concepts of areas of knowledge and finding words that are more likely than others to fit those concepts, the model is able to create texts following the constraints you instruct it with - such as poems that rhyme, or critical analysis of scientific articles. The most important point to be aware of is that this creation model is completely different to how automatic reasoning models create content, which is by having a formal representation of a knowledge domain and creating logical inferences that can be mathematically proven correct within the model. A reasoning model cannot lie, but it cannot create content beyond the logical implications of its premises; its quite the opposite of what language models do.