4 ms·
>all of this is to say that arithmetic, math is not highly encoded in language at all. How did you learn arithmetic then, if not by being shown numbers and equ
by coldtea 4y ago
>all of this is to say that arithmetic, math is not highly encoded in language at all.
How did you learn arithmetic then, if not by being shown numbers and equations, and having math rules and concepts explained to you through language?
- famouswaffles 4y agoI didn't say it's not there. or that you couldn't communicate the ideas of math/arithmetic with language. You can get GPT-3 (yes 3) to have 98.5% accuracy on addition arithmetic (even very large numbers) by..simply describing the algorithm of addition to performed on 2 numbers. https://arxiv.org/abs/2211.09066 https://arxiv.org/abs/2211.09066 The basic idea i'm communicating is that not everything that can be extracted from self supervisory token prediction can be extracted with the same ease. and it is very easy to see why math is one the higher difficulty things. It is extremely easy comparatively to infere "happy" from the sentence - John is smiling therefore he is ----. all the information you need to make that inference is packed tightly in the preceding words. it is not the same for arithmetic at all.
- andrewprock 4y agoIsn't a 98.5% accuracy quite low for a computer adding numbers?
- coldtea 4y agoIt's quite huge for an LLM AI not taught specifically to add numbers. This is not a specialized adding algorithm - it's a general purpose one. And probably better than most humans :-)
- andrewprock 4y agoBut it was taught specifically to add numbers. Unless my understanding of it's training corpus is way out of line.