4 ms·

it's sort of a weakness inherent to LLM's... next word prediction isn't really supposed to be good at math. I doubt it will be ever 'cracked' with better LLM's

it's sort of a weakness inherent to LLM's... next word prediction isn't really supposed to be good at math. I doubt it will be ever 'cracked' with better LLM's, only multimodal ones with access to program execution and calculators.

xrd 3y ago
Agreed. But they do call it out here and I'm interested in understanding why.
foota 3y ago
There's been attempts to use different embeddings for numbers, which helps a lot. E.g., https://news.ycombinator.com/item?id=37936005 https://news.ycombinator.com/item?id=37936005
cjbprime 3y ago
> it's sort of a weakness inherent to LLM's... next word prediction isn't really supposed to be good at math. FWIW I don't agree with this in a theoretical sense. The reason LLMs can do as much as they can is because next token prediction attempts to infer a world model for the processes that generated each next-token in the training set. I don't see a reason that this would preclude learning arithmetic in order to better predict next tokens that require arithmetic. I'd guess that arithmetic will become suddenly reliable with one of the next significant (e.g. 2-5x) jumps in parameter count.
- justinl33 3y ago
  I disagree; 1. the parameters are there to make the model better at learning textual patterns, not arithmetic patterns. 2. Next token prediction is a terrible way to perform arithmetic. 3. Perhaps most importantly, the loss function does not incentivise being good at arithmetic at all. Any perceived arithmetic ability is just a textual coincidence. I agree that a sufficiently intelligent LLM, like 300+ IQ would have an excellent model of how multiplication works. It may even assist in finding new theorems, but a calculator will always be better at 926*725.
  - cjbprime 3y ago
    > 1. the parameters are there to make the model better at learning textual patterns, not arithmetic patterns. There's nothing "textual" about the tokens. They are arbitrary identifiers. There's nothing "textual" about transformers! The fact that e.g. GPT-4 can accept images as input, and that its textual performance improved as a result, and that transformers are also being used for text-to-speech models should have already communicated this. > 2. Next token prediction is a terrible way to perform arithmetic. This is just attempting to resolve our disagreement with pure assertion. It's certainly less efficient to use an artificial intelligence to do arithmetic. But whether it's efficient is a different question than how likely it is to be possible. > 3. Perhaps most importantly, the loss function does not incentivise being good at arithmetic at all. This is blatantly untrue. The same argument would suggest that LLMs can't do anything that wasn't exactly in their training set already. But they can.
    - justinl33 3y ago
      1. this is interesting. tokens are arbitrary identifiers... of text. It's called a 'text encoder' for a reason 2. it's a strong assertion but it is true. it's kind of inherent in the name; next word prediction. Why would you want a calculator to be making predictions? I agree that it's worth understanding whether it's possible, but your original point was that 'arithmetic will become suddenly reliable' - this is what I am disagreeing with. 3. this links to the above point - you are right: LLM's can already perform arithmetic, this could be easily proved by loading up gpt-2. but again, your original point was that LLM's will 'figure out' arithmetic - I don't believe they will. The loss function does not incentivize being good at arithmetic, it incentivizes making good predictions that are close to the true value. The loss function will not penalize an LLM who predict 'good' is the next word when it should have been 'great'. while it might penalize '2+2=3; since this sequence would be strongly represented in the training set, it's not going to penalize the model getting '1234*5678' one digit off - which is the problem.
      - eurekin 3y ago
        About 3., isn't that done specifically in order to not overfit?
jakderrida 3y ago
Annoyingly, Bard with Gemini will apparently use coding to answer every logical thinking question and get them all wrong. If you end with "Do not use code", it will get them all right.
- Kerbonut 3y ago
  ChatGPT-4 is starting to do this now all of the sudden, and it is also annoyingly returning erroneous information.
monkeydust 3y ago
You don't need multimodals to access tools such as calculator. Check out PandasAI or Langchain Agent workflow. Rather than workout 451 * 995 for example the llm constructs the pandas query, runs it, returns result to user. Works pretty well.
e12e 3y ago
Sure, llms don't do discreet math or logic directly. On the other hand, they write surprisingly good code. I'm guessing we'll see llms that do input>program(s)>run>summarize>output I'm not really disagreeing - I just think llms will do "more of the work" themselves by way of writing and running prolog programs, symbolic math (Julia etc) and running theorem provers.

xrd 3y ago

Agreed. But they do call it out here and I'm interested in understanding why.

foota 3y ago

There's been attempts to use different embeddings for numbers, which helps a lot. E.g., https://news.ycombinator.com/item?id=37936005 https://news.ycombinator.com/item?id=37936005

cjbprime 3y ago

> it's sort of a weakness inherent to LLM's... next word prediction isn't really supposed to be good at math. FWIW I don't agree with this in a theoretical sense. The reason LLMs can do as much as they can is because next token prediction attempts to infer a world model for the processes that generated each next-token in the training set. I don't see a reason that this would preclude learning arithmetic in order to better predict next tokens that require arithmetic. I'd guess that arithmetic will become suddenly reliable with one of the next significant (e.g. 2-5x) jumps in parameter count.

justinl33 3y ago
I disagree; 1. the parameters are there to make the model better at learning textual patterns, not arithmetic patterns. 2. Next token prediction is a terrible way to perform arithmetic. 3. Perhaps most importantly, the loss function does not incentivise being good at arithmetic at all. Any perceived arithmetic ability is just a textual coincidence. I agree that a sufficiently intelligent LLM, like 300+ IQ would have an excellent model of how multiplication works. It may even assist in finding new theorems, but a calculator will always be better at 926*725.
- cjbprime 3y ago
  > 1. the parameters are there to make the model better at learning textual patterns, not arithmetic patterns. There's nothing "textual" about the tokens. They are arbitrary identifiers. There's nothing "textual" about transformers! The fact that e.g. GPT-4 can accept images as input, and that its textual performance improved as a result, and that transformers are also being used for text-to-speech models should have already communicated this. > 2. Next token prediction is a terrible way to perform arithmetic. This is just attempting to resolve our disagreement with pure assertion. It's certainly less efficient to use an artificial intelligence to do arithmetic. But whether it's efficient is a different question than how likely it is to be possible. > 3. Perhaps most importantly, the loss function does not incentivise being good at arithmetic at all. This is blatantly untrue. The same argument would suggest that LLMs can't do anything that wasn't exactly in their training set already. But they can.
  - justinl33 3y ago
    1. this is interesting. tokens are arbitrary identifiers... of text. It's called a 'text encoder' for a reason 2. it's a strong assertion but it is true. it's kind of inherent in the name; next word prediction. Why would you want a calculator to be making predictions? I agree that it's worth understanding whether it's possible, but your original point was that 'arithmetic will become suddenly reliable' - this is what I am disagreeing with. 3. this links to the above point - you are right: LLM's can already perform arithmetic, this could be easily proved by loading up gpt-2. but again, your original point was that LLM's will 'figure out' arithmetic - I don't believe they will. The loss function does not incentivize being good at arithmetic, it incentivizes making good predictions that are close to the true value. The loss function will not penalize an LLM who predict 'good' is the next word when it should have been 'great'. while it might penalize '2+2=3; since this sequence would be strongly represented in the training set, it's not going to penalize the model getting '1234*5678' one digit off - which is the problem.
    - eurekin 3y ago
      About 3., isn't that done specifically in order to not overfit?

jakderrida 3y ago

Annoyingly, Bard with Gemini will apparently use coding to answer every logical thinking question and get them all wrong. If you end with "Do not use code", it will get them all right.

Kerbonut 3y ago
ChatGPT-4 is starting to do this now all of the sudden, and it is also annoyingly returning erroneous information.

monkeydust 3y ago

You don't need multimodals to access tools such as calculator. Check out PandasAI or Langchain Agent workflow. Rather than workout 451 * 995 for example the llm constructs the pandas query, runs it, returns result to user. Works pretty well.

e12e 3y ago

Sure, llms don't do discreet math or logic directly. On the other hand, they write surprisingly good code. I'm guessing we'll see llms that do input>program(s)>run>summarize>output I'm not really disagreeing - I just think llms will do "more of the work" themselves by way of writing and running prolog programs, symbolic math (Julia etc) and running theorem provers.