3 ms·
> saying LLMs can help with their accounting, help them close deals by crunching the numbers in seconds, write complex code for them etc etc. Why do you think
by stratos123 7mo ago
> saying LLMs can help with their accounting, help them close deals by crunching the numbers in seconds, write complex code for them etc etc.
Why do you think the results of this paper contradict these claims at all?
- grey-area 7mo agoA machine which confabulates and cannot count is not a good fit for accounting tasks. They’ll make all sorts of subtle errors which are difficult for humans to notice.
- stratos123 7mo agoThat wouldn't even necessarily be true if models really "couldn't count", since software exists - if an LLM is making an Excel spreadsheet rather than doing everything manually, it's both much harder for it to mess up and easier to notice and recover. It's even less true given that what this paper actually tests is "LLMs don't have a literally perfect accuracy when you make them do increasingly big problems with zero thinking". (Confabulation is IMO a much bigger problem, but it's unrelated to architecture - it's an artifact of how models are currently trained.)
- grey-area 7mo agoThey really can’t count, that’s not how they work at all. They don’t reason about maths they predict the most likely output for a given context. That’s sometimes useful but not at all the same thing.