5 ms·

It's a non-deterministic language model, shouldn't we expect mediocre performance in math? It seems like the wrong tool for the job...

drdeca 1y ago
Deterministic is a special case of not-necessarily-deterministic.
CamperBob2 1y ago
We passed 'mediocre' a long time ago, but yes, it would be surprising if the same vocabulary representation is optimal for both verbal language and mathematical reasoning and computing. To the extent we've already found that to be the case, it's perhaps the weirdest part of this whole "paradigm shift."
rictic 1y ago
Models are deterministic, they're a mathematical function from sequences of tokens to probability distributions over the next token. Then a system samples from that distribution, typically with randomness, and there are some optimizations in running them that introduce randomness, but it's important to understand that the models themselves are not random.
- mgraczyk 1y ago
  This is only ideally true. From the perspective of the user of a large closed LLM, this isn't quite right because of non-associativity, experiments, unversioned changes, etc. It's best to assume that the relationship between input and output of an LLM is not deterministic, similar to something like using a Google search API.
  - ijk 1y ago
    And even on open LLMs, GPU instability can cause non-determinism. For performance reasons, determinism is seldom guaranteed in LLMs in general.
    - rar00 1y ago
      yep, even with greedy sampling and fixed system state, numerical instability is sufficient to make output sequences diverge when processing the same exact input
- geysersam 1y ago
  The LLMs are deterministic but they only return a probability distribution over following tokens. The tokens the user sees in the response are selected by some typically stochastic sampling procedure.
  - danielmarkbruce 1y ago
    Assuming decent data, it won't be stochastic sampling for many math operations/input combinations. When people suggest LLMs with tokenization could learn math, they aren't suggesting a small undertrained model trained on crappy data.
    - anonymoushn 1y ago
      I mean, this depends on your sampler. With temp=1 and sampling from the raw output distribution, setting aside numerics issues, these models output nonzero probability of every token at each position
      - danielmarkbruce 1y ago
        A large model well trained on good data will have logits so negative for something like "1+1=" -> 3 that they won't come up in practice unless you sample in a way to deliberately misuse the model.
currymj 1y ago
thanks to training data + this being a popular benchmark, they're pretty good at grinding through symbolic mathematical derivations, which is often useful if you want an explanation of a mathematical concept. there's not really a better tool for this job, except for "a textbook which answers the exact question you have". but from time to time, doing this does require doing arithmetic correctly (to correctly add two exponents or whatever). so it would be nice to be able to trust that. i imagine there are other uses for basic arithmetic too, QA applications over data that quotes statistics and such.
- agarren 1y ago
  > but from time to time, doing this does require doing arithmetic correctly (to correctly add two exponents or whatever). so it would be nice to be able to trust that. It sounds weird, but try writing your problem in LaTeX - I don’t know why, I’ve found a couple models to be incredibly capable at solving mathematical problems if you write them in LaTeX.

drdeca 1y ago

Deterministic is a special case of not-necessarily-deterministic.

CamperBob2 1y ago

We passed 'mediocre' a long time ago, but yes, it would be surprising if the same vocabulary representation is optimal for both verbal language and mathematical reasoning and computing. To the extent we've already found that to be the case, it's perhaps the weirdest part of this whole "paradigm shift."

rictic 1y ago

Models are deterministic, they're a mathematical function from sequences of tokens to probability distributions over the next token. Then a system samples from that distribution, typically with randomness, and there are some optimizations in running them that introduce randomness, but it's important to understand that the models themselves are not random.

mgraczyk 1y ago
This is only ideally true. From the perspective of the user of a large closed LLM, this isn't quite right because of non-associativity, experiments, unversioned changes, etc. It's best to assume that the relationship between input and output of an LLM is not deterministic, similar to something like using a Google search API.
- ijk 1y ago
  And even on open LLMs, GPU instability can cause non-determinism. For performance reasons, determinism is seldom guaranteed in LLMs in general.
  - rar00 1y ago
    yep, even with greedy sampling and fixed system state, numerical instability is sufficient to make output sequences diverge when processing the same exact input
geysersam 1y ago
The LLMs are deterministic but they only return a probability distribution over following tokens. The tokens the user sees in the response are selected by some typically stochastic sampling procedure.
- danielmarkbruce 1y ago
  Assuming decent data, it won't be stochastic sampling for many math operations/input combinations. When people suggest LLMs with tokenization could learn math, they aren't suggesting a small undertrained model trained on crappy data.
  - anonymoushn 1y ago
    I mean, this depends on your sampler. With temp=1 and sampling from the raw output distribution, setting aside numerics issues, these models output nonzero probability of every token at each position
    - danielmarkbruce 1y ago
      A large model well trained on good data will have logits so negative for something like "1+1=" -> 3 that they won't come up in practice unless you sample in a way to deliberately misuse the model.

currymj 1y ago

thanks to training data + this being a popular benchmark, they're pretty good at grinding through symbolic mathematical derivations, which is often useful if you want an explanation of a mathematical concept. there's not really a better tool for this job, except for "a textbook which answers the exact question you have". but from time to time, doing this does require doing arithmetic correctly (to correctly add two exponents or whatever). so it would be nice to be able to trust that. i imagine there are other uses for basic arithmetic too, QA applications over data that quotes statistics and such.

agarren 1y ago
> but from time to time, doing this does require doing arithmetic correctly (to correctly add two exponents or whatever). so it would be nice to be able to trust that. It sounds weird, but try writing your problem in LaTeX - I don’t know why, I’ve found a couple models to be incredibly capable at solving mathematical problems if you write them in LaTeX.