3 ms·

Language models aren't built for math. Their improvement/training cycles aren't sensitive to the exactness and rule-based nature of mathematical language, plus

Language models aren't built for math. Their improvement/training cycles aren't sensitive to the exactness and rule-based nature of mathematical language, plus there are probably a lot of bad/misleading examples of math in the source data. You'd have to be unrealistically pessimistic to call what GPT-3 and other huge language models produce "nonsense".

visarga 4y ago
It's not that they were not built for math, but more like verification is hard. But it's hard for humans as well. A large generative model + a fast verifier could do wonders. AlphaGo was built on that - the model can propose moves, but you can verify who won in the end. There are some code generation models that write their own tests as well, or use externally provided tests to verify their solutions. The DeepMind matrix multiplication algorithm was also "learning from verification" of generated solutions, because it's trivial to do that. In general verification remains an open problem.
- spywaregorilla 4y ago
  I disagree. It is that they were not built for math. While brain analogies are shittier than most people assume, this is like trying to do math in your head without being allowed to think through calculations.
  - visarga 4y ago
    Brains weren't built for math either, just for surviving. And the "trying to do math in your head" is true if you use naive question answering, but if you ask "step by step" or "chain of thought", or "supporting questions", any of them will allow for flexible time steps. There are some solutions called "Language Model Cascades" that compose language models calls to simulate arbitrary complex reasoning chains including recursion. There is no reason to think language models are unfit for math, they are fit for generating possible solutions that need to be verified somehow.
    - spywaregorilla 4y ago
      > Brains weren't built for math either, just for surviving. Brains were, however, built for language processing, in addition to many other tasks. > There is no reason to think language models are unfit for math, they are fit for generating possible solutions that need to be verified somehow. This is just a dumb idea though. Guess and check based on semantically well positioned answers in the ambiguity that is the embedding space until you find something that's not wrong is not the same thing as defining an algorithm and then executing it, which is how people do math. Sure, you could probably get it to a pretty good working state, but it seems pretty dumb to me. > There are some solutions called "Language Model Cascades" that compose language models calls to simulate arbitrary complex reasoning chains including recursion. If you're creating the reasoning chains yourself, you're arguably doing the hard part for the model and giving credit to the language part. If you're able to do get the model to define the chains, then you've already solved the hard part of the problem and could likely use something very different from language models altogether to greater effect.
      - visarga 4y ago
        > This is just a dumb idea though. True, but that's how "inspiration" works in humans as well: generate stupid ideas until you stumble upon a great one. That's why I said we only need verification. It's the artist-critic model, we got the artist we need the critic. Sometimes it's easy (in games, code, math) and other times we don't have a good way to verify.
        spywaregorilla 4y ago
        > True, but that's how "inspiration" works in humans as well: generate stupid ideas until you stumble upon a great one. I don't agree with this, but even if it were true, "inspiration" is not how we do math. > That's why I said we only need verification. It's the artist-critic model, we got the artist we need the critic. Sometimes it's easy (in games, code, math) and other times we don't have a good way to verify. That's not even how the typical language model works.

visarga 4y ago

It's not that they were not built for math, but more like verification is hard. But it's hard for humans as well. A large generative model + a fast verifier could do wonders. AlphaGo was built on that - the model can propose moves, but you can verify who won in the end. There are some code generation models that write their own tests as well, or use externally provided tests to verify their solutions. The DeepMind matrix multiplication algorithm was also "learning from verification" of generated solutions, because it's trivial to do that. In general verification remains an open problem.

spywaregorilla 4y ago
I disagree. It is that they were not built for math. While brain analogies are shittier than most people assume, this is like trying to do math in your head without being allowed to think through calculations.
- visarga 4y ago
  Brains weren't built for math either, just for surviving. And the "trying to do math in your head" is true if you use naive question answering, but if you ask "step by step" or "chain of thought", or "supporting questions", any of them will allow for flexible time steps. There are some solutions called "Language Model Cascades" that compose language models calls to simulate arbitrary complex reasoning chains including recursion. There is no reason to think language models are unfit for math, they are fit for generating possible solutions that need to be verified somehow.
  - spywaregorilla 4y ago
    > Brains weren't built for math either, just for surviving. Brains were, however, built for language processing, in addition to many other tasks. > There is no reason to think language models are unfit for math, they are fit for generating possible solutions that need to be verified somehow. This is just a dumb idea though. Guess and check based on semantically well positioned answers in the ambiguity that is the embedding space until you find something that's not wrong is not the same thing as defining an algorithm and then executing it, which is how people do math. Sure, you could probably get it to a pretty good working state, but it seems pretty dumb to me. > There are some solutions called "Language Model Cascades" that compose language models calls to simulate arbitrary complex reasoning chains including recursion. If you're creating the reasoning chains yourself, you're arguably doing the hard part for the model and giving credit to the language part. If you're able to do get the model to define the chains, then you've already solved the hard part of the problem and could likely use something very different from language models altogether to greater effect.
    - visarga 4y ago
      > This is just a dumb idea though. True, but that's how "inspiration" works in humans as well: generate stupid ideas until you stumble upon a great one. That's why I said we only need verification. It's the artist-critic model, we got the artist we need the critic. Sometimes it's easy (in games, code, math) and other times we don't have a good way to verify.
      - spywaregorilla 4y ago
        > True, but that's how "inspiration" works in humans as well: generate stupid ideas until you stumble upon a great one. I don't agree with this, but even if it were true, "inspiration" is not how we do math. > That's why I said we only need verification. It's the artist-critic model, we got the artist we need the critic. Sometimes it's easy (in games, code, math) and other times we don't have a good way to verify. That's not even how the typical language model works.