4 ms·
You're using an LLM as if you don't understand how or why it works. Math is not what it's for. Here, look at what ChatGPT 4 does (for a case that hasn't been fe
by voiceblue 3y ago
You're using an LLM as if you don't understand how or why it works. Math is not what it's for. Here, look at what ChatGPT 4 does (for a case that hasn't been fed to it by a hundred other users yet):
What is the sum of odd numbers in this set: 12345654321, 123456543212, 123456543213, 12345654324? Output only the sum, no code.
The sum of the odd numbers in the set {12345654321, 123456543212, 123456543213, 12345654324} is 246913086434.
What does that prove? Only that I am using an LLM poorly and/or do not understand what it is. Using OpenChat-3.5 for what LLMs are actually good at (e.g. asking it for shell commands to perform certain operations, getting some general information about a topic) seems to work surprisingly well for a 7B model.
- sorokod 3y agoNot sure what you mean by 'using'. The prompt is from one of the benchmarks used in scoring LLM models - don't recall which. Here is another from BBH [1]: >>> (-1 + 2 + 9 * 5) = ... (-1 + 2 + 9 * 5) = 43 [1] https://github.com/suzgunmirac/BIG-Bench-Hard/tree/main https://github.com/suzgunmirac/BIG-Bench-Hard/tree/main
- GaggiX 3y agoThis is even worse than the other, just straight up use a calculator.
- freilanzer 3y agoThe use isn't the calculation, but the benchmark for its arithmetic capability.
- deleted 3y ago[deleted]