4 ms·

> 1. the parameters are there to make the model better at learning textual patterns, not arithmetic patterns. There's nothing "textual" about the tokens. They

> 1. the parameters are there to make the model better at learning textual patterns, not arithmetic patterns. There's nothing "textual" about the tokens. They are arbitrary identifiers. There's nothing "textual" about transformers! The fact that e.g. GPT-4 can accept images as input, and that its textual performance improved as a result, and that transformers are also being used for text-to-speech models should have already communicated this. > 2. Next token prediction is a terrible way to perform arithmetic. This is just attempting to resolve our disagreement with pure assertion. It's certainly less efficient to use an artificial intelligence to do arithmetic. But whether it's efficient is a different question than how likely it is to be possible. > 3. Perhaps most importantly, the loss function does not incentivise being good at arithmetic at all. This is blatantly untrue. The same argument would suggest that LLMs can't do anything that wasn't exactly in their training set already. But they can.

justinl33 3y ago
1. this is interesting. tokens are arbitrary identifiers... of text. It's called a 'text encoder' for a reason 2. it's a strong assertion but it is true. it's kind of inherent in the name; next word prediction. Why would you want a calculator to be making predictions? I agree that it's worth understanding whether it's possible, but your original point was that 'arithmetic will become suddenly reliable' - this is what I am disagreeing with. 3. this links to the above point - you are right: LLM's can already perform arithmetic, this could be easily proved by loading up gpt-2. but again, your original point was that LLM's will 'figure out' arithmetic - I don't believe they will. The loss function does not incentivize being good at arithmetic, it incentivizes making good predictions that are close to the true value. The loss function will not penalize an LLM who predict 'good' is the next word when it should have been 'great'. while it might penalize '2+2=3; since this sequence would be strongly represented in the training set, it's not going to penalize the model getting '1234*5678' one digit off - which is the problem.
- eurekin 3y ago
  About 3., isn't that done specifically in order to not overfit?
- cjbprime 3y ago
  LLMs are compression functions. An LLM that internalizes the rules of arithmetic will compress better than one that doesn't. That improvement will be measurable in the loss function, as it correctly predicts more of its training data that depends on arithmetic answers to predict the next word. > Why would you want a calculator to be making predictions? If the predictions are correct, why wouldn't I? You are objecting to the entire concept of LLMs at this point, there's nothing specific to arithmetic here.

justinl33 3y ago

1. this is interesting. tokens are arbitrary identifiers... of text. It's called a 'text encoder' for a reason 2. it's a strong assertion but it is true. it's kind of inherent in the name; next word prediction. Why would you want a calculator to be making predictions? I agree that it's worth understanding whether it's possible, but your original point was that 'arithmetic will become suddenly reliable' - this is what I am disagreeing with. 3. this links to the above point - you are right: LLM's can already perform arithmetic, this could be easily proved by loading up gpt-2. but again, your original point was that LLM's will 'figure out' arithmetic - I don't believe they will. The loss function does not incentivize being good at arithmetic, it incentivizes making good predictions that are close to the true value. The loss function will not penalize an LLM who predict 'good' is the next word when it should have been 'great'. while it might penalize '2+2=3; since this sequence would be strongly represented in the training set, it's not going to penalize the model getting '1234*5678' one digit off - which is the problem.

eurekin 3y ago
About 3., isn't that done specifically in order to not overfit?
cjbprime 3y ago
LLMs are compression functions. An LLM that internalizes the rules of arithmetic will compress better than one that doesn't. That improvement will be measurable in the loss function, as it correctly predicts more of its training data that depends on arithmetic answers to predict the next word. > Why would you want a calculator to be making predictions? If the predictions are correct, why wouldn't I? You are objecting to the entire concept of LLMs at this point, there's nothing specific to arithmetic here.