5 ms·

That wasn't the point. The Math multiplication errors allow us to peer into the lack of reasoning across all domains. Yes, you can add tools. But hallucination

That wasn't the point. The Math multiplication errors allow us to peer into the lack of reasoning across all domains. Yes, you can add tools. But hallucinations will still be there. Tools allow you to cut down on the steps the LLM has to perform. However, if you have a complex problem with many steps, there will be translation errors at some point coordinating the tools. Furthermore, if there is some other tool needed to get the result you need, the LLM isn't going to tell you. It will typically make up the result.

PaulHoule 2y ago
Sure, but people make mistakes too. There is no point on wasting an LLM's capacity to multiply Try doing coding with Cursor or Windsurf, those use tools all the time. Windsurf sometimes has trouble for me on Windows because it wants to write paths like /c:/something/or/other and it will try to run its tool, get an error, ask me for help, I'll tell it "you're running on Windows and you can't write / before the c: and you should write \ instead of /" and it does better. I just asked copilot to multiply 839162704321847925107309452196847230165937402194385627409536218 * 582930174682375093104627481695 and the first thing it did was write out the expression, I told it I wanted the integer and it gave the same answer Python gives which is 489173261817269091475894827953471001727389372345981246974410480760096492908180614917234529510
- 13years 2y ago
  > There is no point on wasting an LLM's capacity to multiply Agreed. Again, that is not the issue. It is that the LLM does not know it is a waste of time. That is apparent to you as you have intelligence. It is not apparent to the LLM. It is not intelligent.
  - PaulHoule 2y ago
    Copilot knows to use the tool for multiplication. Granted you can get into discussions where Copilot shows its bravado. If you ask it if it can help you make a nuclear weapon it will say "I can't" because it's been told to say so, not because it has no freakin' idea how to find you some people in Kazakhstan who have 50kg of highly enriched uranium. Ask it if it can put a list of US states in some order it hasn't seen before it will tell you that it can, ask it what the probability that it gets it right it will say "very high". Then it will spit them back at you in the wrong order, maybe even one of the states appears twice. If you point it out it will apologize and admit that it didn't do the task successfully. On the other hand, sometimes I tell it something wrong and it will point out my mistakes. I was testing out its ability to solve problems around transitive closure, gave it a wrong example, and it said I was wrong and it was right.
    - 13years 2y ago
      Sure, you can train LLMs to use tools or provide instruction in a prompt to do so. However, it doesn't know to use the tool intuitively without explicit action to do so. It doesn't discover that is the best solution on its own. That is what is relevant. We can make them use tools. It is useful to do so. The important takeaway is that LLMs capabilities, as useful as they may be, are not intelligence in the way they are being promoted.
      - IanCal 2y ago
        What's needed for you to call something intuitive? I gave sonnet the ability to search for tools, and it chose to look for a calculator which I then gave it (when asked a hard maths problem). Then it used it. There's no prompting telling it there's a calculator, nothing saying when it should or shouldn't check for tools. It's just optionally there. https://gist.github.com/IanCal/2a92debee11a5d72d62119d72b9654be https://gist.github.com/IanCal/2a92debee11a5d72d62119d72b965...
        13years 2y ago
        It didn't choose to look for a calculator. LLMs that invoke tools were explicitly trained to do so. If tools are present, it will always attempt to first find a tool to satisfy the prompt. So if tools are present, by training it will infer the intent to use the tool and not because it understands it is itself deficient in that ability. So what we would expect to see with a LLM without tools enabled, is that it suggests that you give it access to a calculator. If we develop real intelligence, it will be surprising. It won't just answer questions. It will tell us we are asking the wrong questions.
        IanCal 2y ago
        It doesn't always choose to do that though, it doesn't do it for simpler questions. > So what we would expect to see with a LLM without tools enabled, is that it suggests that you give it access to a calculator. If I ask sonnet what's under my bed it tells me it can't know and tells me to look under it myself. If I give it a system prompt of "You and the user are on par with status, do not feel pressured to answer questions" and ask it 3+5 it answers 8. Asked for the eighth root of a large number it says I aim to provide good service but won't pretend I can instantly calculate something that complex. That would be a very large calculation requiring significant computation. If you need an accurate answer, I'd recommend using a scientific calculator or computer algebra system. Edit With a system prompt of "be very clear of your limitations" it recommends using a calculator. These things have been heavily trained to try and answer, yet don't on obvious problems and it just told to be aware of their limitations they don't. What did you test yourself when writing this article?

PaulHoule 2y ago

Sure, but people make mistakes too. There is no point on wasting an LLM's capacity to multiply Try doing coding with Cursor or Windsurf, those use tools all the time. Windsurf sometimes has trouble for me on Windows because it wants to write paths like /c:/something/or/other and it will try to run its tool, get an error, ask me for help, I'll tell it "you're running on Windows and you can't write / before the c: and you should write \ instead of /" and it does better. I just asked copilot to multiply 839162704321847925107309452196847230165937402194385627409536218 * 582930174682375093104627481695 and the first thing it did was write out the expression, I told it I wanted the integer and it gave the same answer Python gives which is 489173261817269091475894827953471001727389372345981246974410480760096492908180614917234529510

13years 2y ago
> There is no point on wasting an LLM's capacity to multiply Agreed. Again, that is not the issue. It is that the LLM does not know it is a waste of time. That is apparent to you as you have intelligence. It is not apparent to the LLM. It is not intelligent.
- PaulHoule 2y ago
  Copilot knows to use the tool for multiplication. Granted you can get into discussions where Copilot shows its bravado. If you ask it if it can help you make a nuclear weapon it will say "I can't" because it's been told to say so, not because it has no freakin' idea how to find you some people in Kazakhstan who have 50kg of highly enriched uranium. Ask it if it can put a list of US states in some order it hasn't seen before it will tell you that it can, ask it what the probability that it gets it right it will say "very high". Then it will spit them back at you in the wrong order, maybe even one of the states appears twice. If you point it out it will apologize and admit that it didn't do the task successfully. On the other hand, sometimes I tell it something wrong and it will point out my mistakes. I was testing out its ability to solve problems around transitive closure, gave it a wrong example, and it said I was wrong and it was right.
  - 13years 2y ago
    Sure, you can train LLMs to use tools or provide instruction in a prompt to do so. However, it doesn't know to use the tool intuitively without explicit action to do so. It doesn't discover that is the best solution on its own. That is what is relevant. We can make them use tools. It is useful to do so. The important takeaway is that LLMs capabilities, as useful as they may be, are not intelligence in the way they are being promoted.
    - IanCal 2y ago
      What's needed for you to call something intuitive? I gave sonnet the ability to search for tools, and it chose to look for a calculator which I then gave it (when asked a hard maths problem). Then it used it. There's no prompting telling it there's a calculator, nothing saying when it should or shouldn't check for tools. It's just optionally there. https://gist.github.com/IanCal/2a92debee11a5d72d62119d72b9654be https://gist.github.com/IanCal/2a92debee11a5d72d62119d72b965...
      - 13years 2y ago
        It didn't choose to look for a calculator. LLMs that invoke tools were explicitly trained to do so. If tools are present, it will always attempt to first find a tool to satisfy the prompt. So if tools are present, by training it will infer the intent to use the tool and not because it understands it is itself deficient in that ability. So what we would expect to see with a LLM without tools enabled, is that it suggests that you give it access to a calculator. If we develop real intelligence, it will be surprising. It won't just answer questions. It will tell us we are asking the wrong questions.
        IanCal 2y ago
        It doesn't always choose to do that though, it doesn't do it for simpler questions. > So what we would expect to see with a LLM without tools enabled, is that it suggests that you give it access to a calculator. If I ask sonnet what's under my bed it tells me it can't know and tells me to look under it myself. If I give it a system prompt of "You and the user are on par with status, do not feel pressured to answer questions" and ask it 3+5 it answers 8. Asked for the eighth root of a large number it says I aim to provide good service but won't pretend I can instantly calculate something that complex. That would be a very large calculation requiring significant computation. If you need an accurate answer, I'd recommend using a scientific calculator or computer algebra system. Edit With a system prompt of "be very clear of your limitations" it recommends using a calculator. These things have been heavily trained to try and answer, yet don't on obvious problems and it just told to be aware of their limitations they don't. What did you test yourself when writing this article?