3 ms·
In my experience, it can reason usably well but acts like it has dyscalculia. It'll set up a proper algorithm, step through it and trip over digits.
by FeepingCreature 3y ago
In my experience, it can reason usably well but acts like it has dyscalculia. It'll set up a proper algorithm, step through it and trip over digits.
- hutzlibu 3y ago"It'll set up a proper algorithm, step through it and trip over digits." Or it is just pretending to do so. And since it pretends, of course it trips over all small things as it does not understand them.
- FeepingCreature 3y agoI don't think you can pretend to properly describe and evaluate an algorithm, anymore than you can pretend to solve a riddle - the answer is either right or it isn't. And in this case, the shape of the answer is often right; it just makes ... ordinary errors. Ironically, the AI is a lot better at high-level thinking than correct calculation.
- hutzlibu 3y ago"Ironically, the AI is a lot better at high-level thinking than correct calculation." That would be actually a human like feature .. except I do not consider what LLMs are doing as thinking.
- throwaway4aday 3y agoYes and the reason it trips over digits is because those digits wind up being tokenized in ways that seem unexpected to us and would produce the same difficulty if we were presented with them: 100,000 + 987 - 1444 * 25,945.842 / 0.0042 becomes "100" one hundred "," comma "000" triple zero " +" space plus " 9" space nine "87" eighty seven " -" space minus " 14" space 14 "44" forty four " *" space times " 25" space twenty five "," comma "9" nine "45" forty five "." period "8" eight "42" forty two " /" space divide " 0" zero "." period "00" double zero "42" forty two Now imagine someone reading that to you over the phone once and asking you to do the math in your head and you aren't allowed to use paper and pencil and you have to get it right the first time.