4 ms·

ChatGPT is a natural language model, meaning it has been trained on vast amounts of text and thus is good at processing and outputting text back. To it, numbers

ChatGPT is a natural language model, meaning it has been trained on vast amounts of text and thus is good at processing and outputting text back. To it, numbers follow the rules of language, and not math, unlike for example a dedicated calculator app. Only thanks to seeing numbers in vast amount of text it was trained on, it is able to do common math relatively well, and anything uncommon very poorly.

zarzavat 4y ago
As pointed out by Yannic[0], ChatGPT is actually a source code model first, then they trained natural language model on top of that. Source code is still language but it has more math in it. But the truth is we don’t know. I personally wouldn’t be surprised if they do train it on a whole bunch of calculator output to boost its numerical reasoning. [0] https://youtu.be/0A8ljAkdFtg https://youtu.be/0A8ljAkdFtg 7:21
- joe_the_user 4y ago
  Could you include a link that isn't a video that leaves my eyes and ears bleeding? Plus from the video: "do you have a thorough idea what OpenAI is doing, neither do I" - cut to strobe light and rock music. Jeesh, can't see that as support.
  - Dylan16807 4y ago
    It's just a guy talking. > Plus from the video: "do you have a thorough idea what OpenAI is doing, neither do I" - cut to strobe light and rock music. Jeesh, can't see that as support. The hell are you talking about? There's no strobe there, he just goes to a slide with a white background, and the music is very quiet compared to his voice and not at all something I would call "rock".
    - ivanche 4y ago
      > It's just a guy talking. Maybe joe_the_user went overboard with calling it strobe light, but this is definitely not "just a guy talking". I don't know how people call this style of videos but it's very tiresome - constantly zooming in and out, full of sound effects (glass breaking, snaring, what not), changing backgrounds, changing lights, everything is just uncontrollably moving and shaking every 5-10 seconds or so.
      - Dylan16807 4y ago
        The sound effects are not loud though, and you don't have to look at the video to get the information. Some zooms aren't the kind of visual where "eyes bleeding" is a hyperbole of the actual problem, either. There are valid criticisms that can be made, but they are different from the criticisms joe_the_user actually made.
      - zarzavat 4y ago
        The special effects are done as a parody, as you might be able to tell from the fact that his background is a green screen with nothing on it. His usual viewers will be familiar with his “ML news” videos where the parody is way more in-your-face. His usual style of video is a rather dry screen capture of him annotating a PDF, so I guess he does it for some lighthearted relief.
- constantcrying 4y ago
  The whole approach of a language model makes it hard for it to understand mathematical operations. It generates probabalistically the next text segment that it thinks "fits". That approach does not work for mathematics and "learning" mathematical operations seems to be out if scope for language models, but that is somewhat of an inherent problem.
Telemakhos 4y ago
This will sound trite, but I wonder whether there's a lesson here to be learned about human education. We put a great deal of effort into educating our children in language and math, to the extent that those form the major branches of our standardized testing for college admissions tests (where they are still used). We treat them as separate subjects, perhaps for good reason, as we also do science and geography and art and music. Can ChatGPT's successes and shortcomings give us insight into how we should or should not educate our own progeny?
- abitnegative 4y ago
  Are you asking if we should just teach kids language and hope that they learn math, science, geography, art, and music as a side effect?
- tkgally 4y ago
  That doesn’t sound trite to me at all. ChatGTP raises major issues for the future of education in many areas, and discussions about how to deal with them have just begun.
  - 2devnull 4y ago
    It will obsolete essay writing or make cheating more of a problem. This may be good. Multiple choice tests will be favored over things like term papers, which are basically meaningless as of now. This will favor general intelligence (iq) at least until there are interface advances like supposedly neurolink or wearables, at which point iq will matter less and big five traits much more. That would be beautiful in many ways, but a major inflection point for humans; no more elites. A great leveling may reduce violence and oppression, and increase prosperity across the globe.
- ForHackernews 4y ago
  I doubt it. Humans are not ML models, and AI "neural networks" are not that much like a brain. I guess it can get philosophical quickly, but people are very quick to read too much into a fancy text-generation tool. We talk about it "learning" but it can't really learn anything, because it is not an entity with agency. It takes an input and responds with an output. I'm not saying it's not impressive, but it is manifestly nothing like a mind.

zarzavat 4y ago

As pointed out by Yannic[0], ChatGPT is actually a source code model first, then they trained natural language model on top of that. Source code is still language but it has more math in it. But the truth is we don’t know. I personally wouldn’t be surprised if they do train it on a whole bunch of calculator output to boost its numerical reasoning. [0] https://youtu.be/0A8ljAkdFtg https://youtu.be/0A8ljAkdFtg 7:21

joe_the_user 4y ago
Could you include a link that isn't a video that leaves my eyes and ears bleeding? Plus from the video: "do you have a thorough idea what OpenAI is doing, neither do I" - cut to strobe light and rock music. Jeesh, can't see that as support.
- Dylan16807 4y ago
  It's just a guy talking. > Plus from the video: "do you have a thorough idea what OpenAI is doing, neither do I" - cut to strobe light and rock music. Jeesh, can't see that as support. The hell are you talking about? There's no strobe there, he just goes to a slide with a white background, and the music is very quiet compared to his voice and not at all something I would call "rock".
  - ivanche 4y ago
    > It's just a guy talking. Maybe joe_the_user went overboard with calling it strobe light, but this is definitely not "just a guy talking". I don't know how people call this style of videos but it's very tiresome - constantly zooming in and out, full of sound effects (glass breaking, snaring, what not), changing backgrounds, changing lights, everything is just uncontrollably moving and shaking every 5-10 seconds or so.
    - Dylan16807 4y ago
      The sound effects are not loud though, and you don't have to look at the video to get the information. Some zooms aren't the kind of visual where "eyes bleeding" is a hyperbole of the actual problem, either. There are valid criticisms that can be made, but they are different from the criticisms joe_the_user actually made.
    - zarzavat 4y ago
      The special effects are done as a parody, as you might be able to tell from the fact that his background is a green screen with nothing on it. His usual viewers will be familiar with his “ML news” videos where the parody is way more in-your-face. His usual style of video is a rather dry screen capture of him annotating a PDF, so I guess he does it for some lighthearted relief.
constantcrying 4y ago
The whole approach of a language model makes it hard for it to understand mathematical operations. It generates probabalistically the next text segment that it thinks "fits". That approach does not work for mathematics and "learning" mathematical operations seems to be out if scope for language models, but that is somewhat of an inherent problem.

Telemakhos 4y ago

This will sound trite, but I wonder whether there's a lesson here to be learned about human education. We put a great deal of effort into educating our children in language and math, to the extent that those form the major branches of our standardized testing for college admissions tests (where they are still used). We treat them as separate subjects, perhaps for good reason, as we also do science and geography and art and music. Can ChatGPT's successes and shortcomings give us insight into how we should or should not educate our own progeny?

abitnegative 4y ago
Are you asking if we should just teach kids language and hope that they learn math, science, geography, art, and music as a side effect?
tkgally 4y ago
That doesn’t sound trite to me at all. ChatGTP raises major issues for the future of education in many areas, and discussions about how to deal with them have just begun.
- 2devnull 4y ago
  It will obsolete essay writing or make cheating more of a problem. This may be good. Multiple choice tests will be favored over things like term papers, which are basically meaningless as of now. This will favor general intelligence (iq) at least until there are interface advances like supposedly neurolink or wearables, at which point iq will matter less and big five traits much more. That would be beautiful in many ways, but a major inflection point for humans; no more elites. A great leveling may reduce violence and oppression, and increase prosperity across the globe.
ForHackernews 4y ago
I doubt it. Humans are not ML models, and AI "neural networks" are not that much like a brain. I guess it can get philosophical quickly, but people are very quick to read too much into a fancy text-generation tool. We talk about it "learning" but it can't really learn anything, because it is not an entity with agency. It takes an input and responds with an output. I'm not saying it's not impressive, but it is manifestly nothing like a mind.