5 ms·

I keep having this happen to a very small language I used to play around with. Now when I have forgotten how it works I thought I would try to learn it again wi

by Moru 3y ago

I keep having this happen to a very small language I used to play around with. Now when I have forgotten how it works I thought I would try to learn it again with ChatGPT and the results were pretty disappointing. It keeps using a mix of different languages, mostly C++ and Java it seems. And I hunt thrugh the documentation to find the real way to do it. It corrects itself but it's still wrong and it keeps going in circles. I guess I won't be learning that language again with help from the AI. It's just too small of a subject for the AI.

dingnuts 3y ago
This is a demonstrative example of why an LLM is NOT intelligent at all, and we need to stop calling these models AI. A real intelligence would be able to either put together the information needed and fill in gaps with research or it would be able to tell you that it does not know. "Hallucination" is yet another anthropomorphism of the model. A mind hallucinates; a model just produces incorrect results. LLMs are very cool but I'm really sick of the "intelligence" branding. If a person who didn't know about something just confidently made shit up, I wouldn't call them intelligent.
- pfortuny 3y ago
  I’ve just tries GPT (again, after months) on high level pseudomathematics and it blabbers on and on about nonsense…
  - cool_dude85 3y ago
    I dont know about the newer models but you used to be able to get authoritative sounding answers asking about "the fallacy of affirming the antecedent". So you don't need to be so high level.
    - pfortuny 3y ago
      Well, that is really good to know. I was just reassuring myself about its nonsense generation.
  - dagw 3y ago
    Terrance Tao, who is a pretty good mathematician, made a post [1] not too long ago about how he found ChatGPT to be at least slightly useful for solving a non-trivial math problem. His basic insight is to not ask for an answer directly, but to ask auxiliary questions and for suggestions of how to approach the problem, which eventually lead him down the right path. Basically how you would discuss the problem with a colleague how didn't know the correct answer of the top of their head either. [1] https://mathstodon.xyz/@tao/110601051375142142 https://mathstodon.xyz/@tao/110601051375142142
    - jprete 3y ago
      Frankly this sounds very close to rubber-ducking. I'm sure LLMs throw out some creative seeds, but they're much better at coming up with creative ideas than truth. I think people are too used to getting creative ideas from other people, where the ideas are at least superficially useful, to understand that they're dealing with the world's best improvising rubber duck.
- lynx23 3y ago
  What about "Fake it till you make it" then?
- stavros 3y ago
  Exactly. Humans are intelligent, and we never misremember anything, and we definitely never are sure we remember a thing correctly when, in fact, we don't. And I agree with you, I know people who make shit up about something they don't know about, and it goes "actual humans > LLMs > these people".
- Moru 3y ago
  I think this train has passed, AI has been accepted as the name and has now changed ownership to the public. This is how live languages work and we older people are supposed to be crying "But this is not what it means!" Well, now it does I'm affraid :-) I know we have had this conversation with our daughter several times and we have three languages to work with...
  - doublerabbit 3y ago
    Your correct. Its ML/LLM at heart but now accepted as to the word "cloud". Where all the "cloud" is, is an server in a datacentre with a fancy gui. I guess we've needed a new buzz world to keep the population excited.
    - Moru 3y ago
      Some younger guy was trying to explain that I used the word cluster wrong when talking about clustering pins on a map. Cluster was a collection of servers in a server room. He had never heard about k-means :-)
- aimor 3y ago
  I like to think of them as Artificial Idiots.
- sanitycheck 3y ago
  > If a person who didn't know about something just confidently made shit up, I wouldn't call them intelligent. It seems to be a good recipe for success in many fields (business, politics, art, apparently somewhat in social sciences if you make up the evidence for it too) so I'm not too sure about that!
  - MrVandemar 3y ago
    The poster said “intelligent” not “successful”. I can name several “successful” politicians I would never, ever call intelligent.
    - sanitycheck 3y ago
      I think you want "intelligent" to mean "ethical". Those politicians are definitely intelligent, they've devised a successful strategy to gain power. You could argue a successful sprinter doesn't need high intelligence, but politics is all about it.
      - MrVandemar 3y ago
        No, seriously, one of our PMs was clearly as dumb as a metaphor for something really dumb. Rhodes scholar too, which shows you what that's worth these days.
- QuesnayJr 3y ago
  I don't have a clear view on whether LLMs are intelligent or not, but I don't understand your argument at all. Why couldn't an intelligent agent exist that refuses to admit they don't know something, and just make stuff up? I've known people who come pretty close to this behavior.
- cbozeman 3y ago
  > If a person who didn't know about something just confidently made shit up, I wouldn't call them intelligent. Then you have a highly flawed understanding of humanity... to the point you should check out of tech for awhile and go read about psychology instead. Extremely intelligent people confidently make shit up when they don't know the answer all the time.

by Moru 3y ago

dingnuts 3y ago

This is a demonstrative example of why an LLM is NOT intelligent at all, and we need to stop calling these models AI. A real intelligence would be able to either put together the information needed and fill in gaps with research or it would be able to tell you that it does not know. "Hallucination" is yet another anthropomorphism of the model. A mind hallucinates; a model just produces incorrect results. LLMs are very cool but I'm really sick of the "intelligence" branding. If a person who didn't know about something just confidently made shit up, I wouldn't call them intelligent.

pfortuny 3y ago
I’ve just tries GPT (again, after months) on high level pseudomathematics and it blabbers on and on about nonsense…
- cool_dude85 3y ago
  I dont know about the newer models but you used to be able to get authoritative sounding answers asking about "the fallacy of affirming the antecedent". So you don't need to be so high level.
  - pfortuny 3y ago
    Well, that is really good to know. I was just reassuring myself about its nonsense generation.
- dagw 3y ago
  Terrance Tao, who is a pretty good mathematician, made a post [1] not too long ago about how he found ChatGPT to be at least slightly useful for solving a non-trivial math problem. His basic insight is to not ask for an answer directly, but to ask auxiliary questions and for suggestions of how to approach the problem, which eventually lead him down the right path. Basically how you would discuss the problem with a colleague how didn't know the correct answer of the top of their head either. [1] https://mathstodon.xyz/@tao/110601051375142142 https://mathstodon.xyz/@tao/110601051375142142
  - jprete 3y ago
    Frankly this sounds very close to rubber-ducking. I'm sure LLMs throw out some creative seeds, but they're much better at coming up with creative ideas than truth. I think people are too used to getting creative ideas from other people, where the ideas are at least superficially useful, to understand that they're dealing with the world's best improvising rubber duck.
lynx23 3y ago
What about "Fake it till you make it" then?
stavros 3y ago
Exactly. Humans are intelligent, and we never misremember anything, and we definitely never are sure we remember a thing correctly when, in fact, we don't. And I agree with you, I know people who make shit up about something they don't know about, and it goes "actual humans > LLMs > these people".
Moru 3y ago
I think this train has passed, AI has been accepted as the name and has now changed ownership to the public. This is how live languages work and we older people are supposed to be crying "But this is not what it means!" Well, now it does I'm affraid :-) I know we have had this conversation with our daughter several times and we have three languages to work with...
- doublerabbit 3y ago
  Your correct. Its ML/LLM at heart but now accepted as to the word "cloud". Where all the "cloud" is, is an server in a datacentre with a fancy gui. I guess we've needed a new buzz world to keep the population excited.
  - Moru 3y ago
    Some younger guy was trying to explain that I used the word cluster wrong when talking about clustering pins on a map. Cluster was a collection of servers in a server room. He had never heard about k-means :-)
aimor 3y ago
I like to think of them as Artificial Idiots.
sanitycheck 3y ago
> If a person who didn't know about something just confidently made shit up, I wouldn't call them intelligent. It seems to be a good recipe for success in many fields (business, politics, art, apparently somewhat in social sciences if you make up the evidence for it too) so I'm not too sure about that!
- MrVandemar 3y ago
  The poster said “intelligent” not “successful”. I can name several “successful” politicians I would never, ever call intelligent.
  - sanitycheck 3y ago
    I think you want "intelligent" to mean "ethical". Those politicians are definitely intelligent, they've devised a successful strategy to gain power. You could argue a successful sprinter doesn't need high intelligence, but politics is all about it.
    - MrVandemar 3y ago
      No, seriously, one of our PMs was clearly as dumb as a metaphor for something really dumb. Rhodes scholar too, which shows you what that's worth these days.
QuesnayJr 3y ago
I don't have a clear view on whether LLMs are intelligent or not, but I don't understand your argument at all. Why couldn't an intelligent agent exist that refuses to admit they don't know something, and just make stuff up? I've known people who come pretty close to this behavior.
cbozeman 3y ago
> If a person who didn't know about something just confidently made shit up, I wouldn't call them intelligent. Then you have a highly flawed understanding of humanity... to the point you should check out of tech for awhile and go read about psychology instead. Extremely intelligent people confidently make shit up when they don't know the answer all the time.