3 ms·
Apple’s paper isn’t testing LLM reasoning. It’s testing the LLM’s social awareness — and their proclivity to assume good faith but errors in the input, rather
by zmgsabst 2y ago
Apple’s paper isn’t testing LLM reasoning.
It’s testing the LLM’s social awareness — and their proclivity to assume good faith but errors in the input, rather than comprehend the adversarial nature of math tests. My own testing indicates that the failures disappear if you warn 4o there may be deceptive language and 4o mini can spot its errors if prompted it was deceived.
It’s surprisingly hard to separate entities from their context — both in psychology and LLM testing.
- skybrian 2y agoInteresting if true. You should write up the results of your experiments.
- zmgsabst 2y agoI wrote up a preliminary whitepaper, but haven’t done a statistically sound test yet. I’ll freely admit this is just a competing theory when I’ve only tried a dozenish hand-written examples rather than hundreds by script. http://www.spacepowermonkey.com/publications/whitepapers/artificial-intelligence/ai-reasoning-ambiguity.html http://www.spacepowermonkey.com/publications/whitepapers/art...
- skybrian 2y agoAs previously shown with “thinking step by step”, changing the prompt is a cheap way to get better results. I think the benchmark they introduced will still be useful, though. Hopefully someone will re-run the benchmark with different prompts.
- zmgsabst 2y agoI’m not sure of the utility: Asking the LLM to do “reasoning” as a single intuitive step is fundamentally not assessing its potential to reason — and is unlike what we expect from humans where we insist they “show their work”. (As a math grader, I’ve penalized humans for doing what they’re asking of the LLM.) If we don’t assess human reasoning in the manner they’re using (zero work shown and not establishing clear context), then I don’t see why it would be useful to assess LLM reasoning in that manner. I think GSM-NoOp is a neat idea, but their testing methodology seems like a “gotcha” that wouldn’t be appropriate for human testing either.