6 ms·

>People’s susceptibility to anthropomorphizing an even slightly convincing computer program has been known since ELIZA, one of the first chatbots, in 1966. It’s

>People’s susceptibility to anthropomorphizing an even slightly convincing computer program has been known since ELIZA, one of the first chatbots, in 1966. It’s called the ELIZA effect. I'm tired of these arguments. Very very few people are anthropomorphizing chatgpt.. very few. The majority of people both technical and non technical who have played with chatGPT in a non trivial way are aware of the chatbots limitations. It's like recognizing a crazy person on the street. Humans are well equipped for that. This argument characterizes the average person as some kind of stupid buffoon as if he/she can't tell chatGPT really screws shit up. Sure there are a few gullible outliers but as a generality his claim is simply completely false. Pretty much everyone and I mean everyone is aware about the limitations of LLMs. This is a weak and repeated trope that's being regurgitated as if the critics are LLMs themselves. Let me specify exactly what's going on. People who are afraid of/support AI are more speaking to the potential of AI. Why? Because just as much as this thing hallucinates about half the time it answers a complex question with an equally complex and correct answer and that answer in isolation is often indistinguishable or even at times superior to a humans answer. Yes we know it hallucinates and forgets shit. This is obvious, no need to readdress an obvious weakness that everyone is aware about. If critics want to have a real discussion then they seriously need to address the actual strengths and phenomenons of LLMs instead of repeatedly highlighting the obvious weaknesses. Because while we have somewhat of an explanation for the hallucinations we currently don't know how chatGPT was able to do something like this: https://www.engraved.blog/building-a-virtual-machine-inside/ https://www.engraved.blog/building-a-virtual-machine-inside/ Read to the end of you haven't seen this. The ending is what is quite unexplainable by experts. You can't just trivialize that entire post as if it was just a statistical phenomenon. There's obviously an alternative angle here.

gregjor 3y ago
Wrong. Too many people do anthropomorphize LLMs. And few people actually understand how LLMs work, or how to detect errors and hallucinations. Calling LLM autocomplete "AI" deliberately conflates the tech with sci-fi tropes that many people already confuse with reality. Numerous examples, from ELIZA to Watson, internet-based scams and frauds, full self-driving, to all of crypto demonstrate human credulity and gullibility, magnified if people think they will either profit or lose financially. The technology behind LLMs probably does have some interesting and valuable applications. But look at the hype and follow the money with a skeptical eye. If so-called AI does or will soon deliver real benefits then why all of the hype around it? Show, don't tell, and especially don't tell stories based on possible future advances or breakthroughs. Future editions of Extraordinary Popular Delusions and the Madness of Crowds will have long chapters about crypto and AI. If you think most people understand the tech and its limitations and don't act according to gullibility, ignorance, and greed that book should disabuse you.
- deltaninenine 3y ago
  [flagged]
  - happysadpanda2 3y ago
    While I believe much of what you argue is correct, I do unfortunately also believe that ultimately you are incorrect: For the non-techy part of the population, and certainly eve a percentage of the techy part of the population, there will be those who don't have an hour out of their daily lives to play around and discover the limits of chatgpt etc. Worse still, an even larger portion of that crowd, even if they have an hour to spare, just... won't. Playing around with an LLM just isn't something most people would find entertaining or enlightening enough to do out of their own volition. What I find more likely to happen, is that the members of above crowd will instead slowly, over time, get the occasional dose of exposure of an interaction with an LLM, and as long as the response yielded isn't batshit crazy, these people will likely develop a growing sense of confidence in these LLMs. At that point, it will likely be very difficult proving to them that the LLMs are far from perfect.
    - deltaninenine 3y ago
      That's just your opinion. I say we need to prove this out. If a significant portion of the tech and non techy population anthropomorphized LLMs to the point where they don't understand that LLMs hallucinate then surely some of those people exist on HN. If one of you readers is one such person who honestly has no idea what it means for chatGPT to "hallucinate" then let us know (and be honest, please don't troll). My bet is no one will respond with affirmation because the amount of people who don't get it is miniscule.
      - gregjor 3y ago
        You're arguing that something observed so often and consistently that it has had a name for decades -- the ELIZA Effect [1] -- doesn't actually happen often enough to care about. I have referred to ChatGPT hallucinations with multiple friends and family, some in tech and some not (like my parents and my kids), and with one exception none of them knew what I was talking about. Like most people they think computers can't make mistakes, so it follows logically (for them) that an (apparently) intelligent machine can't make mistakes, i.e. hallucinate. I have a couple of my own ChatGPT transcripts that include hallucinations and when I show those to people they say that I deliberately misled the AI, because how could it make a mistake? In my own experience, which includes people who work in the software field and people who don't, including a couple of friends who work with neural networks and LLMs, almost no one understands how LLMs work, or what limitations they might have, or what "hallucinate" means in the context of ChatGPT. Almost everyone I know is much more likely to believe AIs have already or will soon put them out of a job and start turning us into slaves or launching nuclear strikes, because that's the nonsense they get fed my the media. [1] https://en.wikipedia.org/wiki/ELIZA_effect https://en.wikipedia.org/wiki/ELIZA_effect
JdeBP 3y ago
It is ironic that you just anthropomorphized it yourself by using "it forgets".
- deltaninenine 3y ago
  When you delete things from your hard drive that fits the definition of making your computer forget something. Look up the definition of forget. It is not a human exclusive action. Therefore it is not "anthropomorphizing": https://www.merriam-webster.com/dictionary/forget https://www.merriam-webster.com/dictionary/forget
davidgerard 3y ago
co-author here: I warn you, I've got a paperclip and I'm not afraid to use it.
- deltaninenine 3y ago
  Don't actually understand your joke here.

gregjor 3y ago

Wrong. Too many people do anthropomorphize LLMs. And few people actually understand how LLMs work, or how to detect errors and hallucinations. Calling LLM autocomplete "AI" deliberately conflates the tech with sci-fi tropes that many people already confuse with reality. Numerous examples, from ELIZA to Watson, internet-based scams and frauds, full self-driving, to all of crypto demonstrate human credulity and gullibility, magnified if people think they will either profit or lose financially. The technology behind LLMs probably does have some interesting and valuable applications. But look at the hype and follow the money with a skeptical eye. If so-called AI does or will soon deliver real benefits then why all of the hype around it? Show, don't tell, and especially don't tell stories based on possible future advances or breakthroughs. Future editions of Extraordinary Popular Delusions and the Madness of Crowds will have long chapters about crypto and AI. If you think most people understand the tech and its limitations and don't act according to gullibility, ignorance, and greed that book should disabuse you.

deltaninenine 3y ago
[flagged]
- happysadpanda2 3y ago
  While I believe much of what you argue is correct, I do unfortunately also believe that ultimately you are incorrect: For the non-techy part of the population, and certainly eve a percentage of the techy part of the population, there will be those who don't have an hour out of their daily lives to play around and discover the limits of chatgpt etc. Worse still, an even larger portion of that crowd, even if they have an hour to spare, just... won't. Playing around with an LLM just isn't something most people would find entertaining or enlightening enough to do out of their own volition. What I find more likely to happen, is that the members of above crowd will instead slowly, over time, get the occasional dose of exposure of an interaction with an LLM, and as long as the response yielded isn't batshit crazy, these people will likely develop a growing sense of confidence in these LLMs. At that point, it will likely be very difficult proving to them that the LLMs are far from perfect.
  - deltaninenine 3y ago
    That's just your opinion. I say we need to prove this out. If a significant portion of the tech and non techy population anthropomorphized LLMs to the point where they don't understand that LLMs hallucinate then surely some of those people exist on HN. If one of you readers is one such person who honestly has no idea what it means for chatGPT to "hallucinate" then let us know (and be honest, please don't troll). My bet is no one will respond with affirmation because the amount of people who don't get it is miniscule.
    - gregjor 3y ago
      You're arguing that something observed so often and consistently that it has had a name for decades -- the ELIZA Effect [1] -- doesn't actually happen often enough to care about. I have referred to ChatGPT hallucinations with multiple friends and family, some in tech and some not (like my parents and my kids), and with one exception none of them knew what I was talking about. Like most people they think computers can't make mistakes, so it follows logically (for them) that an (apparently) intelligent machine can't make mistakes, i.e. hallucinate. I have a couple of my own ChatGPT transcripts that include hallucinations and when I show those to people they say that I deliberately misled the AI, because how could it make a mistake? In my own experience, which includes people who work in the software field and people who don't, including a couple of friends who work with neural networks and LLMs, almost no one understands how LLMs work, or what limitations they might have, or what "hallucinate" means in the context of ChatGPT. Almost everyone I know is much more likely to believe AIs have already or will soon put them out of a job and start turning us into slaves or launching nuclear strikes, because that's the nonsense they get fed my the media. [1] https://en.wikipedia.org/wiki/ELIZA_effect https://en.wikipedia.org/wiki/ELIZA_effect

JdeBP 3y ago

It is ironic that you just anthropomorphized it yourself by using "it forgets".

deltaninenine 3y ago
When you delete things from your hard drive that fits the definition of making your computer forget something. Look up the definition of forget. It is not a human exclusive action. Therefore it is not "anthropomorphizing": https://www.merriam-webster.com/dictionary/forget https://www.merriam-webster.com/dictionary/forget

davidgerard 3y ago

co-author here: I warn you, I've got a paperclip and I'm not afraid to use it.

deltaninenine 3y ago
Don't actually understand your joke here.