13 ms·

Why is this a trick or somehow inferior to getting the AI model to be able to do it natively? Most humans also can’t reliably do complex arithmetic without the

Why is this a trick or somehow inferior to getting the AI model to be able to do it natively? Most humans also can’t reliably do complex arithmetic without the use of something like a calculator. And that’s no trick. We’ve built the modern world with such tools. Why should we fault AI for doing what we do? To me, training the AI use a calculator is not just a trick for hype, it’s exciting progress.

lanstin 3y ago
It would be exciting if the LLM knew it needed a calculator for certain things and went out and got it. If the human supervisors are pre-screening the input and massaging what the LLM is doing that is a sign we don't understand LLMs enough to engineer them precisely and can't count on them to be aware of their own limitations, which would seem to be a useful part of general intelligence.
- Spivak 3y ago
  It can if you let it, that's the whole premise of LangChain style reasoning and it works well enough. My dumb little personal chatbot knows it can access a Python REPL to carry out calculations and it does.
- CamperBob2 3y ago
  It would be exciting if the LLM knew it needed a calculator for certain things and went out and got it Isn't that what it does, when it writes a Python program to compute the answer to the user's question?
bufferoverflow 3y ago
Because if NN is smart enough, it should be able to do arithmetic flawlessly. Basic arithmetic doesn't even require that much intelligence, it's mostly attention to detail.
- janalsncm 3y ago
  Well it’s obviously not smart enough so the question is what do you do about it? Train another net that’s 1000x as big for 99% accuracy or hand it off to the lowly calculator which will get it right 100% of the time? And 1000x is just a guess. We have no scaling laws about this kind of thing. It could be a million. It could be 10.
  - bufferoverflow 3y ago
    I agree with you that we don't know if will take 10x or 1 million. We don't know if current LLM will scale at all. It might not be the way to AGI. But while we can delegate the math to the calculator, it's essentially sweeping the problem under the rug. It actually tells you your neural net is not very smart. We know for a fact that it was exposed to tons of math during training, and it still can't do even the most basic addition reliably, let alone multiplication or division. What we want is an actually smart network, not a dumb search engine that knows a billion factoids and quotes, and that hallucinates randomly.
michaelt 3y ago
By all means if it works to solve your problem, go ahead and do it. The reason some people have mixed feelings about this because of a historical observation - http://www.incompleteideas.net/IncIdeas/BitterLesson.html http://www.incompleteideas.net/IncIdeas/BitterLesson.html - that we humans often feel good about adding lots of hand-coded smarts to our ML systems reflecting our deep and brilliant personal insights. But it turns out just chucking loads of data and compute at the problem often works better. 20 years ago in machine vision you'd have an engineer choosing precisely which RGB values belonged to which segment, deciding if this was a case where a hough transform was appropriate, and insisting on a room with no windows because the sun moves and it's totally throwing off our calibration. In comparison, it turns out you can just give loads of examples to a huge model and it'll do a much better job. (Obviously there's an element of self-selection here - if you train an ML system for OCR, you compare it to tesseract and you find yours is worse, you probably don't release it. Or if you do, nobody pays attention to you)
- janalsncm 3y ago
  The reason we chucked loads of data at it was because we had no other options. If you wanted to write a function that classified a picture as a cat or a dog, good luck. With ML, you can learn such a function. That logic doesn’t extend to things we already know how to program computers to do. Arithmetic already works. We don’t need a neural net to also run the calculations or play a game of chess. We have specialized programs that are probably as good as we’re going to get in those specialized domains.
  - michaelt 3y ago
    > We don’t need a neural net to also run the calculations or play a game of chess. That's actually one of the specific examples from the link I mentioned:- > In computer chess, the methods that defeated the world champion, Kasparov, in 1997, were based on massive, deep search. At the time, this was looked upon with dismay by the majority of computer-chess researchers who had pursued methods that leveraged human understanding of the special structure of chess. When a simpler, search-based approach with special hardware and software proved vastly more effective, these human-knowledge-based chess researchers were not good losers. They said that ``brute force" search may have won this time, but it was not a general strategy, and anyway it was not how people played chess. These researchers wanted methods based on human input to win and were disappointed when they did not. While it's true that they didn't use an LLM specifically, it's still an example of chucking loads of compute at the problem instead of something more elegant and human-like. Of course, I agree that if you're looking for a good game of chess, Stockfish is a better choice than ChatGPT.

lanstin 3y ago

It would be exciting if the LLM knew it needed a calculator for certain things and went out and got it. If the human supervisors are pre-screening the input and massaging what the LLM is doing that is a sign we don't understand LLMs enough to engineer them precisely and can't count on them to be aware of their own limitations, which would seem to be a useful part of general intelligence.

Spivak 3y ago
It can if you let it, that's the whole premise of LangChain style reasoning and it works well enough. My dumb little personal chatbot knows it can access a Python REPL to carry out calculations and it does.
CamperBob2 3y ago
It would be exciting if the LLM knew it needed a calculator for certain things and went out and got it Isn't that what it does, when it writes a Python program to compute the answer to the user's question?

bufferoverflow 3y ago

Because if NN is smart enough, it should be able to do arithmetic flawlessly. Basic arithmetic doesn't even require that much intelligence, it's mostly attention to detail.

janalsncm 3y ago
Well it’s obviously not smart enough so the question is what do you do about it? Train another net that’s 1000x as big for 99% accuracy or hand it off to the lowly calculator which will get it right 100% of the time? And 1000x is just a guess. We have no scaling laws about this kind of thing. It could be a million. It could be 10.
- bufferoverflow 3y ago
  I agree with you that we don't know if will take 10x or 1 million. We don't know if current LLM will scale at all. It might not be the way to AGI. But while we can delegate the math to the calculator, it's essentially sweeping the problem under the rug. It actually tells you your neural net is not very smart. We know for a fact that it was exposed to tons of math during training, and it still can't do even the most basic addition reliably, let alone multiplication or division. What we want is an actually smart network, not a dumb search engine that knows a billion factoids and quotes, and that hallucinates randomly.

michaelt 3y ago

By all means if it works to solve your problem, go ahead and do it. The reason some people have mixed feelings about this because of a historical observation - http://www.incompleteideas.net/IncIdeas/BitterLesson.html http://www.incompleteideas.net/IncIdeas/BitterLesson.html - that we humans often feel good about adding lots of hand-coded smarts to our ML systems reflecting our deep and brilliant personal insights. But it turns out just chucking loads of data and compute at the problem often works better. 20 years ago in machine vision you'd have an engineer choosing precisely which RGB values belonged to which segment, deciding if this was a case where a hough transform was appropriate, and insisting on a room with no windows because the sun moves and it's totally throwing off our calibration. In comparison, it turns out you can just give loads of examples to a huge model and it'll do a much better job. (Obviously there's an element of self-selection here - if you train an ML system for OCR, you compare it to tesseract and you find yours is worse, you probably don't release it. Or if you do, nobody pays attention to you)

janalsncm 3y ago
The reason we chucked loads of data at it was because we had no other options. If you wanted to write a function that classified a picture as a cat or a dog, good luck. With ML, you can learn such a function. That logic doesn’t extend to things we already know how to program computers to do. Arithmetic already works. We don’t need a neural net to also run the calculations or play a game of chess. We have specialized programs that are probably as good as we’re going to get in those specialized domains.
- michaelt 3y ago
  > We don’t need a neural net to also run the calculations or play a game of chess. That's actually one of the specific examples from the link I mentioned:- > In computer chess, the methods that defeated the world champion, Kasparov, in 1997, were based on massive, deep search. At the time, this was looked upon with dismay by the majority of computer-chess researchers who had pursued methods that leveraged human understanding of the special structure of chess. When a simpler, search-based approach with special hardware and software proved vastly more effective, these human-knowledge-based chess researchers were not good losers. They said that ``brute force" search may have won this time, but it was not a general strategy, and anyway it was not how people played chess. These researchers wanted methods based on human input to win and were disappointed when they did not. While it's true that they didn't use an LLM specifically, it's still an example of chucking loads of compute at the problem instead of something more elegant and human-like. Of course, I agree that if you're looking for a good game of chess, Stockfish is a better choice than ChatGPT.