8 ms·

I think we should stop trying to quiz LLMs on mathematics, something for which they are explicitly not designed to do with their tokenized view of the world. As

I think we should stop trying to quiz LLMs on mathematics, something for which they are explicitly not designed to do with their tokenized view of the world. Ask GPT-4 to use its Wolfram plugin and it returns the answers quickly and correctly. Second, I think the code generation bit of this paper is blown out of proportion. The code can't be immediately injected into a codebase due to a formatting change (triple quotes). I'd be more interested in changes to the quality and performance of the code generated, not whether it can be easily copy/pasted from the page. I could not replicate that result (but could replicate the other math issues). I've also never seen the triple quotes in results so it's unclear if there was a temporary presentation bug.

sanxiyn 3y ago
Or OpenAI can stop being stupid and adopt LLaMA-like tokenization, which special cases numbers and tokenize them into individual digits.
- jinay 3y ago
  Are LLaMA-based models better at math because of this digit-based tokenization?
- bugglebeetle 3y ago
  Isn’t the tokenization tied to the training of the model?
  - colobas 3y ago
    The model learns an embedding table, where (roughly) each row is used as the model’s internal representation of each token. The numbers in that table are learned. What isn’t learned is the map from token (i.e. combination of characters/byte-pairs) to row-index in the embedding table. That is given by tokenization EDIT: removed redundant bit
ehnto 3y ago
I think knowing if the code can be used verbatim is actually the more important part practically speaking. That is the actually useful part. Quality is important to humans, because humans have to read it, but correctness is what people using ChatGPT for code actually need. So long as the quality and performance is good enough, then it will be useful. Performance is such a nuanced topic that you need very context aware devs anyway, and I think a general purpose LLM is never going to have that kind of awareness.
- staunton 3y ago
  > never going to have that kind of awareness Be careful with that goalpost, it might make sudden movements.
  - ineedasername 3y ago
    Yes, when ChatGPT went public using 3 I was like, “Hah, yeah, that can’t come close to writing the narrative portions in my work.” Then 4 came along and it was more like, “ooh, a basic first draft in 15 seconds? That I can work with.” Now, at least for some projects, I can give it a few rapid bullet points in incomplete sentences and have that first draft in seconds, after which I just need to tweak, add in tables and more detailed stats & results, etc. quite useful.
- gonzo41 3y ago
  GPT is very useful for the scenario of when you have a complex, dialect specific bit of SQL and need to turn it into an ORM query like sqlalchemy. It get's you about 90% the way there
xp84 3y ago
Seriously. GPT doing math is like using a 737 to drive around on the ground, or if you had the phone number of a prominent astrophysicist and you call him to do long division for you. Wtf is the point. We have computer things to do every math problem. It’s a waste of energy to use LLMs for it in my opinion.
- sheepscreek 3y ago
  A better example would be - calling your guitar lessons teacher for help on a statistics problem.
  - ben_w 3y ago
    Possibly more like instantiating the statistics problem by getting, say, every 3 members of an orchestra to represent a triplet of doors in the Monty Hall problem and then making a ball-park guess what the results are from a seat in the middle of the back row.
- lumost 3y ago
  I used GPT-4 to generate a non-cryptographic random 64 character string. It was faster to ask GPT-4 for the string than ask GPT-4 for the instructions to generate the string from my terminal. GPT-4 was faster than google.
  - ThrowawayTestr 3y ago
    Surely navigating to random.org is faster than typing a prompt.
  - oefnak 3y ago
    That definitely won't be random.
    - taneq 3y ago
      It's probably 4, that's pretty random.
deelly 3y ago
No, we definitely should continue to quiz LLMs on mathematics and absolutely any other topics. Otherwise how do we know and understand limitation of the system?
- muzani 3y ago
  We should also test its capabilities on cooking steak, flying rockets, and making love. Only then will we know if AI can be superior to humans on all things.
  - falcor84 3y ago
    I know you're being facetious, but I'd absolutely be in favor of having AI benchmarks for any and all of these.
    - muzani 3y ago
      I realized that halfway through typing that too. As well as the absurdity of trying to create AIs that exceed a human in all tasks. On a serious note, most of these AIs are bad at math but good at writing code for calculators. So what you'll be benchmarking is their ability to create and use tools.
  - ineedasername 3y ago
    Well, if there was an API hooked up to a webcam & 6-dof arm that would be an interesting task. (The steak)
    - BeFlatXIII 3y ago
      Lovemaking, too.
dontupvoteme 3y ago
Could not agree more. It doesn't understand what a number is, why is everyone trying to quiz it on maths instead of, perhaps, seeing how good it is at language tasks, or even foreign languages? I suspect it has gotten a lot worse in non-english since launch.
- delusional 3y ago
  it "understands" numbers the exact same amount that it understands words.
  - vermilingua 3y ago
    Well, no, because numbers are “words” that represent something that behaves in a very different way to words in a sentance.
- gwd 3y ago
  Why do you think this? FWIW I use GPT-4 regularly to explain Koine Greek from the New Testament to me; its ability there certainly hasn't diminished in the last two months.
- manojlds 3y ago
  The paper is talking about DELTA though. It used to do well and doesn't now.
dr_dshiv 3y ago
They should stop teaching math to kids for the same reason. Just give them a calculator! Or show us the evidence that math education improves cognitive abilities in other domains.
TeMPOraL 3y ago
In terms of evaluating LLMs, I'd argue quizzing them on maths is still better than the other thing people keep doing - quizzing them on facts and self-contradicting scenarios, hoping to get them to only either recall information perfectly, or answer with "I don't know". I'm not an AI/ML scientist, so I may be way off mark here, but everything I've read so far, and all my experience playing with GPT-3.5 and GPT-4, tell me that comparing performance of an LLM to that of a human is a category error, because the LLM isn't a good analogue of a whole human mind - but it's a very good analogue to human inner voice. The stream of consciousness. The whatever-it-is that surfaces your unconscious/subconscious thought process in form of words and sentences. The inner voice is fast, it's reactive. It generates thoughts that match the situation, whether they're correct or factually accurate or not. It's up to the conscious part of your mind to stop, refine, or recycle those thoughts. If you let it keep going, it'll give you thoughts based on what feels like should follow the thoughts that came before. And, unless you habituated responding to anything new with "I don't know" followed by ignoring the topic, the inner voice will start blurting answers to what looks like a question/problem statement; whether or not they'll make any sense, depends on your familiarity with the topic in question. Pretty much 1:1 what LLMs do. Now, this could all be noise, but I don't think so. I know not everyone has a distinct inner narrative (much like not everyone can visualize things in their mind - I can't), but many (most?) people do. The description of the "inner voice experience" I gave above is something I figured out over a decade ago - before LLMs or even deep learning were a thing, before I knew anything about the NLP beyond recognizing the term "Markov chain" is somehow related. Could my inner narration style be unique? Possibly, but given how advice to avoid connecting your inner voice directly with your vocal apparatus is deeply infused in culture and literature, I strongly suspect this is just how it works. All this to say: it is my hypothesis, so far corroborated by experience, that when you start feeding absurd amount of unlabeled text to a transformer model, letting it pick up on the structures encoded within, what you get is a close equivalent to our own inner voice - the part that deals with associations, not logic or data storage. You can't expect it to get good at performing arbitrary computation or recalling data with perfect fidelity, because it's structurally not what it's suited for. For humans, performing arbitrary calculations or perfect recall requires engaging a slower, more algorithmic thinking process (and/or external memory). That part is currently missing in the LLM-based AI systems we're playing with.
falcor84 3y ago
I would argue that ChatGPT is actually quite good at "mathematics", in the sense of helping me formulate a problem in an appropriate mathematical structure, and coming up with a good sequence of steps to solve / simplify it. Not perfect by any means, but not bad at all. What it is bad at is actually performing the steps accurately, but as others mentioned, that's where Wolfram and/or a code interpreter would come in.
tough 3y ago
Triple quotes sound like how you do markdown code blocks to me ``` code ```

sanxiyn 3y ago

Or OpenAI can stop being stupid and adopt LLaMA-like tokenization, which special cases numbers and tokenize them into individual digits.

jinay 3y ago
Are LLaMA-based models better at math because of this digit-based tokenization?
bugglebeetle 3y ago
Isn’t the tokenization tied to the training of the model?
- colobas 3y ago
  The model learns an embedding table, where (roughly) each row is used as the model’s internal representation of each token. The numbers in that table are learned. What isn’t learned is the map from token (i.e. combination of characters/byte-pairs) to row-index in the embedding table. That is given by tokenization EDIT: removed redundant bit

ehnto 3y ago

I think knowing if the code can be used verbatim is actually the more important part practically speaking. That is the actually useful part. Quality is important to humans, because humans have to read it, but correctness is what people using ChatGPT for code actually need. So long as the quality and performance is good enough, then it will be useful. Performance is such a nuanced topic that you need very context aware devs anyway, and I think a general purpose LLM is never going to have that kind of awareness.

staunton 3y ago
> never going to have that kind of awareness Be careful with that goalpost, it might make sudden movements.
- ineedasername 3y ago
  Yes, when ChatGPT went public using 3 I was like, “Hah, yeah, that can’t come close to writing the narrative portions in my work.” Then 4 came along and it was more like, “ooh, a basic first draft in 15 seconds? That I can work with.” Now, at least for some projects, I can give it a few rapid bullet points in incomplete sentences and have that first draft in seconds, after which I just need to tweak, add in tables and more detailed stats & results, etc. quite useful.
gonzo41 3y ago
GPT is very useful for the scenario of when you have a complex, dialect specific bit of SQL and need to turn it into an ORM query like sqlalchemy. It get's you about 90% the way there

xp84 3y ago

Seriously. GPT doing math is like using a 737 to drive around on the ground, or if you had the phone number of a prominent astrophysicist and you call him to do long division for you. Wtf is the point. We have computer things to do every math problem. It’s a waste of energy to use LLMs for it in my opinion.

sheepscreek 3y ago
A better example would be - calling your guitar lessons teacher for help on a statistics problem.
- ben_w 3y ago
  Possibly more like instantiating the statistics problem by getting, say, every 3 members of an orchestra to represent a triplet of doors in the Monty Hall problem and then making a ball-park guess what the results are from a seat in the middle of the back row.
lumost 3y ago
I used GPT-4 to generate a non-cryptographic random 64 character string. It was faster to ask GPT-4 for the string than ask GPT-4 for the instructions to generate the string from my terminal. GPT-4 was faster than google.
- ThrowawayTestr 3y ago
  Surely navigating to random.org is faster than typing a prompt.
- oefnak 3y ago
  That definitely won't be random.
  - taneq 3y ago
    It's probably 4, that's pretty random.

deelly 3y ago

No, we definitely should continue to quiz LLMs on mathematics and absolutely any other topics. Otherwise how do we know and understand limitation of the system?

muzani 3y ago
We should also test its capabilities on cooking steak, flying rockets, and making love. Only then will we know if AI can be superior to humans on all things.
- falcor84 3y ago
  I know you're being facetious, but I'd absolutely be in favor of having AI benchmarks for any and all of these.
  - muzani 3y ago
    I realized that halfway through typing that too. As well as the absurdity of trying to create AIs that exceed a human in all tasks. On a serious note, most of these AIs are bad at math but good at writing code for calculators. So what you'll be benchmarking is their ability to create and use tools.
- ineedasername 3y ago
  Well, if there was an API hooked up to a webcam & 6-dof arm that would be an interesting task. (The steak)
  - BeFlatXIII 3y ago
    Lovemaking, too.

dontupvoteme 3y ago

Could not agree more. It doesn't understand what a number is, why is everyone trying to quiz it on maths instead of, perhaps, seeing how good it is at language tasks, or even foreign languages? I suspect it has gotten a lot worse in non-english since launch.

delusional 3y ago
it "understands" numbers the exact same amount that it understands words.
- vermilingua 3y ago
  Well, no, because numbers are “words” that represent something that behaves in a very different way to words in a sentance.
gwd 3y ago
Why do you think this? FWIW I use GPT-4 regularly to explain Koine Greek from the New Testament to me; its ability there certainly hasn't diminished in the last two months.
manojlds 3y ago
The paper is talking about DELTA though. It used to do well and doesn't now.

dr_dshiv 3y ago

They should stop teaching math to kids for the same reason. Just give them a calculator! Or show us the evidence that math education improves cognitive abilities in other domains.

TeMPOraL 3y ago

In terms of evaluating LLMs, I'd argue quizzing them on maths is still better than the other thing people keep doing - quizzing them on facts and self-contradicting scenarios, hoping to get them to only either recall information perfectly, or answer with "I don't know". I'm not an AI/ML scientist, so I may be way off mark here, but everything I've read so far, and all my experience playing with GPT-3.5 and GPT-4, tell me that comparing performance of an LLM to that of a human is a category error, because the LLM isn't a good analogue of a whole human mind - but it's a very good analogue to human inner voice. The stream of consciousness. The whatever-it-is that surfaces your unconscious/subconscious thought process in form of words and sentences. The inner voice is fast, it's reactive. It generates thoughts that match the situation, whether they're correct or factually accurate or not. It's up to the conscious part of your mind to stop, refine, or recycle those thoughts. If you let it keep going, it'll give you thoughts based on what feels like should follow the thoughts that came before. And, unless you habituated responding to anything new with "I don't know" followed by ignoring the topic, the inner voice will start blurting answers to what looks like a question/problem statement; whether or not they'll make any sense, depends on your familiarity with the topic in question. Pretty much 1:1 what LLMs do. Now, this could all be noise, but I don't think so. I know not everyone has a distinct inner narrative (much like not everyone can visualize things in their mind - I can't), but many (most?) people do. The description of the "inner voice experience" I gave above is something I figured out over a decade ago - before LLMs or even deep learning were a thing, before I knew anything about the NLP beyond recognizing the term "Markov chain" is somehow related. Could my inner narration style be unique? Possibly, but given how advice to avoid connecting your inner voice directly with your vocal apparatus is deeply infused in culture and literature, I strongly suspect this is just how it works. All this to say: it is my hypothesis, so far corroborated by experience, that when you start feeding absurd amount of unlabeled text to a transformer model, letting it pick up on the structures encoded within, what you get is a close equivalent to our own inner voice - the part that deals with associations, not logic or data storage. You can't expect it to get good at performing arbitrary computation or recalling data with perfect fidelity, because it's structurally not what it's suited for. For humans, performing arbitrary calculations or perfect recall requires engaging a slower, more algorithmic thinking process (and/or external memory). That part is currently missing in the LLM-based AI systems we're playing with.

falcor84 3y ago

I would argue that ChatGPT is actually quite good at "mathematics", in the sense of helping me formulate a problem in an appropriate mathematical structure, and coming up with a good sequence of steps to solve / simplify it. Not perfect by any means, but not bad at all. What it is bad at is actually performing the steps accurately, but as others mentioned, that's where Wolfram and/or a code interpreter would come in.

tough 3y ago

Triple quotes sound like how you do markdown code blocks to me ``` code ```