3 ms·
If it does not have a 'model' for even simple arithmetic, then how exactly is it parsing/processing/generating/transforming code? Simply predicting and generati
by ne0luddite 4y ago
If it does not have a 'model' for even simple arithmetic, then how exactly is it parsing/processing/generating/transforming code? Simply predicting and generating next token from a corpus of text (of code) is as good as copy pasting
- Imnimo 4y agoIt's not clear to me that the only way to parse/process/generate/transform code is to have a correct mathematical model of the code's behavior. Even human programmers sometimes analyze code based on an imprecise general understanding. If ChatGPT is able to perform these activities using only next-token prediction, I think we have to conclude that next-token prediction is in fact more powerful than mere copy-pasting.