4 ms·

I actually have a contrarian view: being able to do elementary math is not that important in the current stage. Yes, understanding elementary math is a cornerst

I actually have a contrarian view: being able to do elementary math is not that important in the current stage. Yes, understanding elementary math is a cornerstone for an AI to become more intelligent, but also let's be honest: LLMs are far from being AGIs and does not have common sense nor general ability to deduce or induct. If we accept such limitation of LLM, then focusing the mathematical understanding of an LLM appears to be incredibly boring.

slushy-chivalry 2y ago
if you sampled N random people on the street and asked them to solve this problem, what would the outcome be? would it be better than asking chatgpt N times? I wonder
- jiiam 2y ago
  I am deeply interested in this point of view of yours so I will be hijacking your reply to ask another question: is "better than asking a few random people on the street" the bar we should be setting? As far as mathematical thinking goes this doesn't seem an interesting metric at all. Do you believe that optimizing for this metric will indeed lead to reliable mathematical thinking? I am of the idea that LLMs are not suited to maths, but since I'm not an expert of the field I'm always looking for counterarguments. Of course we can always wait another couple of years and the question will be resolved.
  - jiggawatts 2y ago
    People compare a general intelligence against the yardstick of their own specialist skills. I’ve seen some truly absurd examples, like people complaining that it didn’t have the latest updates to some obscure research functional logic proof language that has maybe a hundred users globally! GPT 4 already has markedly superior English comprehension and basic logic than most people I interact with on a daily basis. It’s only outperformed by a handful of people, all of whom are “high achievers” such as entrepreneurs, professors, or consultants. I actively simplify my speech when talking to ordinary people to avoid overwhelming them. I don’t need to when instructing GPT.
    - deleted 2y ago
      [deleted]
  - grob-gambit 2y ago
    I don't have a counter argument. Not to be ironic but ChatGPT4o gives a better response to the question at hand than anything I have read in this thread: https://chatgpt.com/share/c10c540f-b9c2-4714-ae6b-77460b900be8 https://chatgpt.com/share/c10c540f-b9c2-4714-ae6b-77460b900b...
- hervature 2y ago
  HN: "Tesla needs to be 100x safer than the best human drivers!!!" Also HN: "ChatGPT just needs to spell its name."
  - fragmede 2y ago
    While words have power too, I'm not driving next to ChatGPT on the freeway where it's going to immediately kill or maim me if it hallucinates. Besides, only half of HN is all self-driving has to be 100x safer. the other half keeps bringing up the fact that Waymo is here and working, just not everywhere yet.
    - NBJack 2y ago
      Not as far as you know. Doctors: https://www.news-medical.net/news/20240424/Opportunities-and-limitations-of-using-a-large-language-model-to-respond-to-patient-messages.aspx#:~:text=A%20new%20study%20by%20investigators,draft%20replies%20to%20patient%20messages https://www.news-medical.net/news/20240424/Opportunities-and.... Lawyers: https://www.ft.com/content/2365b275-8b0b-4ae9-bc4a-15c6e0776d26 https://www.ft.com/content/2365b275-8b0b-4ae9-bc4a-15c6e0776... Or my fav: https://www.axios.com/2023/10/31/guardian-microsoft-generative-ai-poll-death https://www.axios.com/2023/10/31/guardian-microsoft-generati...
MagicMoonlight 2y ago
It’s important because solving a math problem requires you to actually understand something and follow deliberate steps. The fact that they can’t means they’re just a toy ultimately.
- gavindean90 2y ago
  No, I disagree. It is just deliberate steps. Understanding can greatly help you do the steps and remember which ones to do. Training math is likely hard because the corpus of training data is so much less because the computers themselves do our math as it relates to computers. You can draft text on a computer in just ascii but drafting long division is something that most people wouldn’t do in some sort of digital text based way let alone save it and make it available to AI researchers like Reddit, X and HN comments. I expect LLMs to be bad at math. That’s ok, they are bad because the computers themselves are so good at math.
  - waveBidder 2y ago
    you can easily generate reams and reams of long division text if you so choose, and i'd assume openai has done so to get to this point
- Shorel 2y ago
  No, hard disagree. It is not a toy, because language is not a toy. Language is a ridiculously powerful tool we have as humans, we just take it for granted because we are very good at it. What ChatGPT allows us is to evaluate the usefulness of language without all the other intellectual tools we have, like mathematics, logic, physics and self perception. So, the fact that a GPT model can do all those things while being just a language model is extraordinary. It is an idea serializer. Written language is not the idea, is just the serialization of the idea, the human thought. We serialize our ideas, and with a GPT we use the serialization to leverage some of the connections and behaviours of the idea, without actually understanding the meaning. This makes both tools, language for humans and GPT for computers, a powerful abstraction that allows us to not need to think about every detail because some of the logic in the ideas is covered by the serialization that language performs over the idea. I find all of this fascinating. Add the other parts of the human brain (a GPT is only the Broca's area analogue) to have real logic and physics and mathematics understanding, and the predictions about AI will be true.

slushy-chivalry 2y ago

if you sampled N random people on the street and asked them to solve this problem, what would the outcome be? would it be better than asking chatgpt N times? I wonder

jiiam 2y ago
I am deeply interested in this point of view of yours so I will be hijacking your reply to ask another question: is "better than asking a few random people on the street" the bar we should be setting? As far as mathematical thinking goes this doesn't seem an interesting metric at all. Do you believe that optimizing for this metric will indeed lead to reliable mathematical thinking? I am of the idea that LLMs are not suited to maths, but since I'm not an expert of the field I'm always looking for counterarguments. Of course we can always wait another couple of years and the question will be resolved.
- jiggawatts 2y ago
  People compare a general intelligence against the yardstick of their own specialist skills. I’ve seen some truly absurd examples, like people complaining that it didn’t have the latest updates to some obscure research functional logic proof language that has maybe a hundred users globally! GPT 4 already has markedly superior English comprehension and basic logic than most people I interact with on a daily basis. It’s only outperformed by a handful of people, all of whom are “high achievers” such as entrepreneurs, professors, or consultants. I actively simplify my speech when talking to ordinary people to avoid overwhelming them. I don’t need to when instructing GPT.
  - deleted 2y ago
    [deleted]
- grob-gambit 2y ago
  I don't have a counter argument. Not to be ironic but ChatGPT4o gives a better response to the question at hand than anything I have read in this thread: https://chatgpt.com/share/c10c540f-b9c2-4714-ae6b-77460b900be8 https://chatgpt.com/share/c10c540f-b9c2-4714-ae6b-77460b900b...
hervature 2y ago
HN: "Tesla needs to be 100x safer than the best human drivers!!!" Also HN: "ChatGPT just needs to spell its name."
- fragmede 2y ago
  While words have power too, I'm not driving next to ChatGPT on the freeway where it's going to immediately kill or maim me if it hallucinates. Besides, only half of HN is all self-driving has to be 100x safer. the other half keeps bringing up the fact that Waymo is here and working, just not everywhere yet.
  - NBJack 2y ago
    Not as far as you know. Doctors: https://www.news-medical.net/news/20240424/Opportunities-and-limitations-of-using-a-large-language-model-to-respond-to-patient-messages.aspx#:~:text=A%20new%20study%20by%20investigators,draft%20replies%20to%20patient%20messages https://www.news-medical.net/news/20240424/Opportunities-and.... Lawyers: https://www.ft.com/content/2365b275-8b0b-4ae9-bc4a-15c6e0776d26 https://www.ft.com/content/2365b275-8b0b-4ae9-bc4a-15c6e0776... Or my fav: https://www.axios.com/2023/10/31/guardian-microsoft-generative-ai-poll-death https://www.axios.com/2023/10/31/guardian-microsoft-generati...

MagicMoonlight 2y ago

It’s important because solving a math problem requires you to actually understand something and follow deliberate steps. The fact that they can’t means they’re just a toy ultimately.

gavindean90 2y ago
No, I disagree. It is just deliberate steps. Understanding can greatly help you do the steps and remember which ones to do. Training math is likely hard because the corpus of training data is so much less because the computers themselves do our math as it relates to computers. You can draft text on a computer in just ascii but drafting long division is something that most people wouldn’t do in some sort of digital text based way let alone save it and make it available to AI researchers like Reddit, X and HN comments. I expect LLMs to be bad at math. That’s ok, they are bad because the computers themselves are so good at math.
- waveBidder 2y ago
  you can easily generate reams and reams of long division text if you so choose, and i'd assume openai has done so to get to this point
Shorel 2y ago
No, hard disagree. It is not a toy, because language is not a toy. Language is a ridiculously powerful tool we have as humans, we just take it for granted because we are very good at it. What ChatGPT allows us is to evaluate the usefulness of language without all the other intellectual tools we have, like mathematics, logic, physics and self perception. So, the fact that a GPT model can do all those things while being just a language model is extraordinary. It is an idea serializer. Written language is not the idea, is just the serialization of the idea, the human thought. We serialize our ideas, and with a GPT we use the serialization to leverage some of the connections and behaviours of the idea, without actually understanding the meaning. This makes both tools, language for humans and GPT for computers, a powerful abstraction that allows us to not need to think about every detail because some of the logic in the ideas is covered by the serialization that language performs over the idea. I find all of this fascinating. Add the other parts of the human brain (a GPT is only the Broca's area analogue) to have real logic and physics and mathematics understanding, and the predictions about AI will be true.