7 ms·
Why don't LLMs ask for calculators?
- Terr_ 2y ago> The LLM has no self-reflection for the knowledge it knows and has no understanding of concepts beyond what can be assembled by patterns in language. My favorite framing: The LLM is just an ego-less extender of text documents. It is being iteratively run against movie script, which is usually incomplete and ending in: "User Says X, and Bot responds with..." Designers of these systems have--deliberately--tricked consumers into thinking they are talking to the LLM author, rather than supplying mad-libs dialogue for a User character that is the same fictional room as a Bot character. The Bot can only speak limitations which are story-appropriate for the character. It only says it's bad at math because lots of people have written lots of words saying the same thing. If you changed its name and description to Mathematician Dracula, it would have dialogue about how its awesome at math but can't handle sunlight, crucifixes, and garlic. This framing also explains how "prompt injection" and "hallucinations" 3 are not exceptional, but standard core behavior.
- tombert 2y agoUmm, they do though? When I use ChatGPT it will phone out to Wolfram Alpha to compute numbers and the like.
- HarHarVeryFunny 2y agoYeah, but that's tool use. It's not the LLM realizing it needs a calculator, but rather a tool use prompt (instruction) causing it to predict a tool invocation sequence that is then intercepted by the model-calling scaffolding. With reasoning models they could choose to do it slightly differently - just tell the model what tools are available, and how to invoke them, and let it decide when to use them in more flexible fashion, rather than rely on fine tuning to follow tool use instructions.
- Workaccount2 2y agoI don't know what happened but there was that time when GPT-4 could access wolfram alpha, and anytime you asked it something that was beyond the most basic math, it would automatically prompt wolfram for the answer.
- threatripper 2y agoThat was during the great plugin times. Haven't seen plugins for a while. Do they still exist?
- luma 2y agoThey can do exactly that, it's called Tool Use and nearly all modern models can handle it. For example, I have a consumer GPU that can run a R1 Qwen distill, which, when prompted for a large multiplication, will elect to write a python script to find the answer. This is a table stakes feature for even the open/free models today.
- deleted 2y ago[deleted]
- 13years 2y agoYou still can't get 100% reliability that would be necessary for certain problem domains. There are going to be some level of hallucination errors in the translation to the agent or code. If it is a complex problem, those will compound.
- zeroxfe 2y agoYou can't get 100% reliability from a human either.
- 13years 2y agoWhich is why humans use calculators. That is the key point being made secondary to the reliability. The LLM "knows" it is bad at Math. It knows the purpose of calculators. However, doesn't use this information to inform the user. It could also propose to the user it could write the answer using code. It doesn't do that either.
- lern_too_spel 2y agoIt has been prompted to give an answer to the user. If you prompted a human to give an answer, they would do the same thing. This example is awful.
- amelius 2y agoYes, but the takeaway message is that even laypeople can invent how LLMs should work.
- PaulHoule 2y agoMany LLMs, particularly, coding assistants, use "tools". Here is one with a calculator https://githubnext.com/projects/gpt4-with-calc/ https://githubnext.com/projects/gpt4-with-calc/ and another example https://www.pinecone.io/learn/series/langchain/langchain-tools/ https://www.pinecone.io/learn/series/langchain/langchain-too... LLMs often do a good job at mathy coding, for instance I told Copilot that "i want a python function that computes the collatz sequence for a given starting n and returns it as a list" def collatz_sequence(n): sequence = [n] while n != 1: if n % 2 == 0: n = n // 2 else: n = 3 * n + 1 sequence.append(n) return sequence which gives right answers, which I wouldn't count on copilot being able to do on its own.
- lblume 2y agoEspecially for a problem as well-known as this, expect code for it to be seen by the model during training at least a thousand times in different languages, flavors etc. This, on its own, is nothing new.
- PaulHoule 2y agoI wouldn't trust it to win a fight with the borrow checker in Rust but lots of simple and practical cases work, such as "use comprehensions to give me a sum of the square of all element divisible by three"? sum(x**2 for x in numbers if x % 3 == 0) and to do the same for a pandas series with pandas operators after asking it to inline something (numbers[numbers % 3 == 0] ** 2).sum() It's not a miracle, you have to go at it with some critical thinking and testing and it makes mistakes but so does Stack Overflow.
- lblume 2y agoIn the first case, I would literally type out the expression faster than thinking about expressing it in natural language.
- 13years 2y agoThat wasn't the point. The Math multiplication errors allow us to peer into the lack of reasoning across all domains. Yes, you can add tools. But hallucinations will still be there. Tools allow you to cut down on the steps the LLM has to perform. However, if you have a complex problem with many steps, there will be translation errors at some point coordinating the tools. Furthermore, if there is some other tool needed to get the result you need, the LLM isn't going to tell you. It will typically make up the result.
- behnamoh 2y agothis is copium, the author doesn't have a good grasp on LLMs. you can't simply "ask" a language model to see if they know they're bad at math and then conclude that the response actually reflects the knowledge encapsulated in the model... sigh...
- ThrowawayTestr 2y agoI just ask chatgpt to use a script to calculate an answer
- colonCapitalDee 2y agoClaude Sonnet 3.5 will often use JavaScript as calculator. It's not perfect when it comes to deciding whether it should write code, but that's easy to fix by prompting it with "Write some code to help you answer the question". The post is honestly quite strange. "When LLMs try and do math themselves they often get it wrong" and "LLMs don't use tools" are two entirely different claims! The first claim is true, the second claim is false, and yet the article uses the truth of the first claim as evidence for the second! This does not hold up at all.
- dr_dshiv 2y agoThe post is honestly very normal — many mid level intellectuals think this way and love these talk downs of LLMs. I don’t understand it for the life of me.
- exe34 2y ago> A simple observation of LLM behavior can tell us much more than might be apparent at first. I love the "I'm an expert, I ask simple questions and reveal profound truths" vibe.
- deleted 2y ago[deleted]
- enragedcacti 2y agoThe claim isn't "LLMs don't use tools", the author is saying that LLMs can't make reliable inferences regarding their own knowledge or capabilities which fundamentally limits their usefulness for many tasks. LLMs "know" that LLMs can't do math reliably, LLMs "know" that calculators can do math reliably, and yet LLMs generally just soldier on and try to do math themselves when asked. You can of course RL it or prompt it into writing javascript when it sees math but so far LLMs haven't been capable of generalizing the process of "I am bad at X" + "Thing is good at X" -> "I should ask for Thing to do X" unless that specific chain of thought is common in the training data. The solution so far has just been to throw more RL or carefully crafted synthetic data at it but its arguably more pavlovian than it is generalized learning. Someone could teach a dog to ring a bell that says "food" on it, and you could reasonably argue that it is using a tool. Will it then know to ring a bell that says "walk" when it wants to go outside?
- crancher 2y ago> Now, some might interject here and say we could, of course, train the LLM to ask for a calculator. However, that would not make them intelligent. Humans require no training at all for calculators, as they are such intuitive instruments. Does the author really believe humans are born with an innate knowledge of calculators and their use?
- MyOutfitIsVague 2y agoI remember having several days of lessons on how to use calculators in elementary, middle, and high school. They are "intuitive", but not all the functions of them are, and if you fully rely on intuition, you might expect them to do things like respect order of operations, which they very well might not.
- aithrowawaycomm 2y agoI think he means that you don't need to train a human to understand that a calculator is useful, and in particular when a problem is hard enough that you need to bust out a calculator. That sort of logic is self-apparent in humans, but struggles to be consistently evoked in LLMs. That said, I was using simple +*-/ calculators as a small child and I don't think I needed to be taught anything other than MC/MR. The tool is intuitive if you are familiar with formal written arithmetic (of course hunter-gatherers couldn't make sense of it).
- 13years 2y agoAs a kid of about 5 or 6 years old I used my first calculator with no instruction whatsoever. We are not talking about scientific calculators. Addition, Multiplication. It does not require training or instruction, just a minute of exploration.
- luma 2y agoYou did however need to be taught math first, you needed to learn how to pick things up, read numbers, interact with buttons, understand that a device might have an on and off state, and a zillion other things. It took about 5 or 6 years of training time to make that happen, and it was the result of parents, teachers, or others actively taking time to train you. That process didn’t involve parking you in a library at birth so you could just go figure it out. Author is simply being obtuse and presumably has some axe to grind or is just ignorant of how LLMs are trained. For example, LLMs don’t learn to chat from the data, they have to be instruct tuned to make that happen. Every LLM chatbot you’ve ever used had to have this extra training step. Further, this is the exact same training process that can also train for tool use. Trying to say “this should just happen from the data” is silly, it isn’t how any of this works. It’s not how you learned things, and it’s not how LLMs-as-chatbots work.
- scarface_74 2y agoThe paid version of ChatGPT has had a built in Python runtime for well over a year. The [>_] links to the Python code that was run. https://chatgpt.com/share/67b79516-9918-8010-897c-ba061a29847a https://chatgpt.com/share/67b79516-9918-8010-897c-ba061a2984...
- lblume 2y agoThe free version, at least once you are logged in, does as well. I don't pay OpenAI, and for the prompt "Calculate the difference between the two biggest primes less than the factorial of 20" it wrote the following code: import sympy # Calculate 20! factorial_20 = sympy.factorial(20) # Find the two largest primes less than 20! largest_prime = sympy.prevprime(factorial_20) second_largest_prime = sympy.prevprime(largest_prime) # Calculate the difference difference = largest_prime - second_largest_prime difference executed it, and produced the correct result, 40.
- karparov 2y agoAnd as you see in the responses here, most people miss the point, elect to patch over the aspects in which the lack of intelligence is glaring, and eventually the end product will be so hard to distinguish from actual intelligence that it's deemed "good enough". Is that bad? Idk. If you hoped that real AGI would eventually solve humanities biggest problems and questions, perhaps so. But if you want something that really really looks like AGI except to some nerds who still say "well actually", then it's gonna be good enough for most. And certainly sufficient for ending up the dystopia from that movie clip in the end.
- lern_too_spel 2y agoIf you don't give a human a calculator, they do the calculation in their head. The same with an LLM. If you give a human a calculator, they will use it. The same with an LLM. Both will say a calculator can do arithmetic better. I don't believe current LLMs are AGIs but this article's argument is a poor one.
- sega_sai 2y agoI am puzzled by the fact that the modern LLMs don't do multiplication in the same way humans do it, i.e. digit by digit. Surely they can write an algorithm for that, but why can't they perform it ?
- deleted 2y ago[deleted]
- deleted 2y ago[deleted]
- threeducks 2y agoTwo reasons: 1. LLMs "think" in terms of tokens, which usually are around 4 characters each. While humans only have to memorize 10x10 multiplication tables to perform multiplication of large numbers, LLMs have to memorize a 10000x10000 table, which is much more difficult. 2. LLMs can't "think in their head", so you have to make them spell out each step of the multiplication, just like (most) humans can't multiply huge numbers without intermediate steps. A simple way to demonstrate this is to ask an LLM for the birth year of a celebrity and then whether that number is even or odd. The answer will be correct almost every time. But if you ask whether the birth year of a celebrity is even or odd and forbid spelling out the year, the accuracy will be barely above 50 %.
- threatripper 2y agoCan't we tokenize numbers always as single digits and give the LLM an <thinking> scratchpad invisible to the user?
- threeducks 2y agoYes, but we could also give the LLM access to a Python interpreter and solve a much larger class of problems with correctness guarantees and around a billion times less compute.
- bobro 2y agoAre there a lot of examples written out of people talking through running that algorithm? I’d guess not.
- DrNosferatu 2y agoI’m surprised why LLMs don’t have in their system prompt a hard rule instructing that any numeric computations in particular, and any other computations in general must only be performed by tool use / running Python.
- deleted 2y ago[deleted]
- Sysreq2 2y agoA lot of people are talking about tool use and writing internal scripts, and yeah, that’s kind of an answer. Really though I think the author is highlighting that LLMs are not being used efficiently at the present moment. LLMs are great at certain tasks. Databases are better at certain tasks. Calculators too. While we could continually throw more and more compute at the problem, growing layers and injecting more data, wouldn’t it make more sense to just have an LLM call its own back-end calculator agent? When I ask it for obscure information maybe it should just pull from it’s own internal encyclopedia database. Let LLMs do what they do well, but let’s not forget the decades that brought us here. Even the smartest human still uses a calculator, so why doesn’t an AI? The fact that it writes its own JavaScript is flashy as hell but also completely unnecessary and error prone.
- 13years 2y ago> Really though I think the author is highlighting that LLMs are not being used efficiently at the present moment. Yes, that is a key point. It isn't to say they are useless tools, but that they aren't intelligent tools and that has significant meaning for what tasks we think they are appropriate for. Unfortunately, nearly everyone has misinterpreted the intent as showing LLMs can't use tools. The point is about how LLMs work differently than most think that they do.