10 ms·
I don’t think this is a particularly useful benchmark. It’s well known that LLMs are bad at math. The token based weighting can’t properly account for numbers
by SkyPuncher 3y ago
I don’t think this is a particularly useful benchmark.
It’s well known that LLMs are bad at math. The token based weighting can’t properly account for numbers that can vary wildly. Numbers are effectively wildcards in the LLM world.
- ben_w 3y agoSurely this is a "didn't read the question properly" problem rather than a "didn't maths right" problem? And that (understanding a natural language question) is the USP for LLMs.