3 ms·
How does the failure to understand elementary math demonstrate that it's copy-pasting? If it were copy-pasting, wouldn't you expect it to be able to handle sim
by Imnimo 4y ago
How does the failure to understand elementary math demonstrate that it's copy-pasting? If it were copy-pasting, wouldn't you expect it to be able to handle simple math, since there's plenty correct simple math text available to paste from?
- ne0luddite 4y agoIf it does not have a 'model' for even simple arithmetic, then how exactly is it parsing/processing/generating/transforming code? Simply predicting and generating next token from a corpus of text (of code) is as good as copy pasting
- Imnimo 4y agoIt's not clear to me that the only way to parse/process/generate/transform code is to have a correct mathematical model of the code's behavior. Even human programmers sometimes analyze code based on an imprecise general understanding. If ChatGPT is able to perform these activities using only next-token prediction, I think we have to conclude that next-token prediction is in fact more powerful than mere copy-pasting.