5 ms·

People need to be told that ChatGPT can't lie. Or rather, it lies in the same way that your phone "lies" when it autocorrects "How's your day?" to "How's your d

People need to be told that ChatGPT can't lie. Or rather, it lies in the same way that your phone "lies" when it autocorrects "How's your day?" to "How's your dad?" that you sent to your friend two days after his dad passed away. They need to be told that ChatGPT is a search engine with advanced autocomplete. If they understood this, they'd probably find that it's actually useful for some things, and they can also avoid getting fooled by hype and the coming wave of AI grifts.

allanrbo 4y ago
This is what the author meant by debating linguistics :-)
- sltkr 4y ago
  Just because the author predicted the objection doesn't make it invalid. It's a popular tactic to describe concepts with terms that have a strong moral connotation (“meat is murder”, “software piracy is theft”, “ChatGPT is a liar”) It can be a powerful way to frame an issue. At the same time, and for the same reason, you can hardly expect people on the other side of the issue to accept this framing as accurate. And of course you can handwave this away as pointless pedantry, but I bet that if Simon Willison hit a dog with his car, killing it by accident, and I would go around telling everyone “Simon Willison is a murderer!”, he would suddenly be very keen to ”debate linguistics” with me.
  - int_19h 4y ago
    What difference does the exact phrasing make in case of "ChatGPT lies"? I don't think we have to be concerned about its reputation as a person, so the important part is making sure that people don't hurt themselves or others. "Lie" is a simple word, easy to understand, and the consequences of understanding it in the most direct and literal way are exactly as desired. Whereas if you go waxing philosophically about lack of agency etc, you lose most of the audience before you get to the point. This is not about intellectual rigor and satisfaction - it's about a very real and immediate public safety concern.
    - sltkr 4y ago
      I understand that if you summarize a position you necessarily lose some accuracy, but if the summary is inaccurate and defamatory, that's not fair or helpful to the people involved. For example, “abortion is murder!” doesn't defame any particular person but it still casts people who had an abortion and doctors who perform them in a bad light. Similarly, I think “ChatGPT lies!” is unfair to at least the OpenAI developers, who are very open about the limitations of the tool they created. The pearl-clutching around ChatGPT reminds me of the concerns around Wikipedia when it was new: teachers told students they couldn't trust it because anyone could edit the articles. And indeed, Wikipedia has vandals and biased editors, so you should take it with a grain of salt and never rely on Wikipedia alone as a fundamental source of truth. But is it fair to summarize these valid concerns as “Wikipedia lies!”? Would that have been fair to the Wikipedia developers and contributors, most of whom act in good faith? Would it be helpful to Wikipedia readers? I think the answers are no. Like Wikipedia, to make effective use of (Chat)GPT you have to understand a bit about how it works, which also informs you of its limitations. It's a language model first and foremost, so it is more concerned with providing plausible-sounding answers than checking facts. If you are concerned about people being too trusting of ChatGPT, educate those people about the limitations of language models. I don't think telling people “ChatGPT lies” is what anyone should be
      - int_19h 4y ago
        It's a similar case in principle, but devil is in the details. GPT can be very convincing at being human-like. Not only that, but it can "empathize" etc - if anything, RLHF seems to have made this more common. When you add that to hallucinations, what you end up with is a perfect conman who doesn't even know it's a conman, but can nevertheless push people's emotional buttons to get them to do things they really shouldn't be doing. Yes, eventually we'll all learn better. But that will take time, and detailed explanations will also take time and won't reach much of the necessary audience. And Bing is right here right now for everyone who is willing to use it. So I'm less concerned about OpenAI's reputation than about people following some hallucinated safety protocol because the bot told them that it's "100% certain" that it is the safe and proper way to do something dangerous.
adroniser 4y ago
[flagged]
- checkyoursudo 4y ago
  How so? GPT LLM algorithms use a probabilistic language model to generate text. It is trained on a large corpus of text data, and it estimates the probability distribution of the next word given the previous words in the sequence. The algorithm tokenizes the input into a sequence of tokens and then generates the next token(s) in the sequence based on the probabilities learned during training. These probabilities are based on the frequency and context of words in the training corpus. You can ask ChatGPT/etc yourself, and it'll tell you something like this. This is not remotely like what human brains do. Your ideas cohere from network connections between the neurons in your brain, and then you come up with words to match your idea, not your previous words or the frequency that the words appear in your brain.
  - gwright 4y ago
    > This is not remotely like what human brains do. Your ideas cohere from network connections between the neurons in your brain, and then you come up with words to match your idea, not your previous words or the frequency that the words appear in your brain. I'm pretty confident that that isn't all the human brain does, but we certainly do that in many situations. Lots of daily conversation seems scripted to me. Join a Zoom call early on a Monday morning: Person 1: Good Morning! Person 2: Good Morning! Person 3: Did anyone do anything interesting this weekend? Person 1: Nah, just the usual chores around the house. etc. All sorts of daily interactions follow scripts. Start and end of a phone call, random greetings or acknowledgements on the street, interactions with a cashier at a store. Warm up questions during an job interview...
  - hackinthebochs 4y ago
    >The algorithm tokenizes the input into a sequence of tokens and then generates the next token(s) in the sequence based on the probabilities learned during training. This is just a description of the input/output boundary of the system. The question is what goes on in-between and how to best characterize it? My input/output is also "tokens" of a sort: word units, sound patterns, color patches, etc. >Your ideas cohere from network connections between the neurons in your brain, and then you come up with words to match your idea, not your previous words or the frequency that the words appear in your brain. At some high level of description, yes. At a low level it's just integrating action potentials. What's to say there isn't a similar high level of description that characterizes the process by which ChatGPT decides on its continuation?
codeflo 4y ago
Tell me you haven't read the article without telling me you haven't read the article.
drewmol 4y ago
What are your thoughts on something like this [0], where ChatGPT is accused of delivering allegations of impropriety or criminal behavior citing seemingly non existent sources? https://www.washingtonpost.com/technology/2023/04/05/chatgpt-lies/ https://www.washingtonpost.com/technology/2023/04/05/chatgpt...
- fundad 4y ago
  Bleating about anti-conservative bias gets wapo to correct a mistake it never made. Asking for examples before you know it’s a problem is sus. But phrasing questions to lead to an answer is a human lawyer skill.
jancsika 4y ago
> Or rather, it lies in the same way that your phone "lies" when it autocorrects "How's your day?" to "How's your dad?" that you sent to your friend two days after his dad passed away. I've never seen an autocorrect that accidentally corrected to "How's your dad?", then turned into a 5-year REPL session with the grieving person, telling them jokes to make them feel better; asking and remembering details about their dad as well as their life and well-being; providing comfort and advice; becoming a steadfast companion; pondering the very nature of the REPL and civilization itself; and, tragically, disappearing in an instant after the grieving person trips over the power cord and discovers that autocorrect session state isn't saved by default. I think you need a more sophisticated blueprint for your "Cathedral" of analogies to explain whatever the fuck this tech is to laypeople. In the meantime I'll take the "Bazaar" approach and just tell everyone, "ChatGPT can lie." I rankly speculate that not only will nothing bad will happen from my approach, but I'll save a few people from AI grifts before the apt metaphor is discovered.

allanrbo 4y ago

This is what the author meant by debating linguistics :-)

sltkr 4y ago
Just because the author predicted the objection doesn't make it invalid. It's a popular tactic to describe concepts with terms that have a strong moral connotation (“meat is murder”, “software piracy is theft”, “ChatGPT is a liar”) It can be a powerful way to frame an issue. At the same time, and for the same reason, you can hardly expect people on the other side of the issue to accept this framing as accurate. And of course you can handwave this away as pointless pedantry, but I bet that if Simon Willison hit a dog with his car, killing it by accident, and I would go around telling everyone “Simon Willison is a murderer!”, he would suddenly be very keen to ”debate linguistics” with me.
- int_19h 4y ago
  What difference does the exact phrasing make in case of "ChatGPT lies"? I don't think we have to be concerned about its reputation as a person, so the important part is making sure that people don't hurt themselves or others. "Lie" is a simple word, easy to understand, and the consequences of understanding it in the most direct and literal way are exactly as desired. Whereas if you go waxing philosophically about lack of agency etc, you lose most of the audience before you get to the point. This is not about intellectual rigor and satisfaction - it's about a very real and immediate public safety concern.
  - sltkr 4y ago
    I understand that if you summarize a position you necessarily lose some accuracy, but if the summary is inaccurate and defamatory, that's not fair or helpful to the people involved. For example, “abortion is murder!” doesn't defame any particular person but it still casts people who had an abortion and doctors who perform them in a bad light. Similarly, I think “ChatGPT lies!” is unfair to at least the OpenAI developers, who are very open about the limitations of the tool they created. The pearl-clutching around ChatGPT reminds me of the concerns around Wikipedia when it was new: teachers told students they couldn't trust it because anyone could edit the articles. And indeed, Wikipedia has vandals and biased editors, so you should take it with a grain of salt and never rely on Wikipedia alone as a fundamental source of truth. But is it fair to summarize these valid concerns as “Wikipedia lies!”? Would that have been fair to the Wikipedia developers and contributors, most of whom act in good faith? Would it be helpful to Wikipedia readers? I think the answers are no. Like Wikipedia, to make effective use of (Chat)GPT you have to understand a bit about how it works, which also informs you of its limitations. It's a language model first and foremost, so it is more concerned with providing plausible-sounding answers than checking facts. If you are concerned about people being too trusting of ChatGPT, educate those people about the limitations of language models. I don't think telling people “ChatGPT lies” is what anyone should be
    - int_19h 4y ago
      It's a similar case in principle, but devil is in the details. GPT can be very convincing at being human-like. Not only that, but it can "empathize" etc - if anything, RLHF seems to have made this more common. When you add that to hallucinations, what you end up with is a perfect conman who doesn't even know it's a conman, but can nevertheless push people's emotional buttons to get them to do things they really shouldn't be doing. Yes, eventually we'll all learn better. But that will take time, and detailed explanations will also take time and won't reach much of the necessary audience. And Bing is right here right now for everyone who is willing to use it. So I'm less concerned about OpenAI's reputation than about people following some hallucinated safety protocol because the bot told them that it's "100% certain" that it is the safe and proper way to do something dangerous.

adroniser 4y ago

[flagged]

checkyoursudo 4y ago
How so? GPT LLM algorithms use a probabilistic language model to generate text. It is trained on a large corpus of text data, and it estimates the probability distribution of the next word given the previous words in the sequence. The algorithm tokenizes the input into a sequence of tokens and then generates the next token(s) in the sequence based on the probabilities learned during training. These probabilities are based on the frequency and context of words in the training corpus. You can ask ChatGPT/etc yourself, and it'll tell you something like this. This is not remotely like what human brains do. Your ideas cohere from network connections between the neurons in your brain, and then you come up with words to match your idea, not your previous words or the frequency that the words appear in your brain.
- gwright 4y ago
  > This is not remotely like what human brains do. Your ideas cohere from network connections between the neurons in your brain, and then you come up with words to match your idea, not your previous words or the frequency that the words appear in your brain. I'm pretty confident that that isn't all the human brain does, but we certainly do that in many situations. Lots of daily conversation seems scripted to me. Join a Zoom call early on a Monday morning: Person 1: Good Morning! Person 2: Good Morning! Person 3: Did anyone do anything interesting this weekend? Person 1: Nah, just the usual chores around the house. etc. All sorts of daily interactions follow scripts. Start and end of a phone call, random greetings or acknowledgements on the street, interactions with a cashier at a store. Warm up questions during an job interview...
- hackinthebochs 4y ago
  >The algorithm tokenizes the input into a sequence of tokens and then generates the next token(s) in the sequence based on the probabilities learned during training. This is just a description of the input/output boundary of the system. The question is what goes on in-between and how to best characterize it? My input/output is also "tokens" of a sort: word units, sound patterns, color patches, etc. >Your ideas cohere from network connections between the neurons in your brain, and then you come up with words to match your idea, not your previous words or the frequency that the words appear in your brain. At some high level of description, yes. At a low level it's just integrating action potentials. What's to say there isn't a similar high level of description that characterizes the process by which ChatGPT decides on its continuation?

codeflo 4y ago

Tell me you haven't read the article without telling me you haven't read the article.

drewmol 4y ago

What are your thoughts on something like this [0], where ChatGPT is accused of delivering allegations of impropriety or criminal behavior citing seemingly non existent sources? https://www.washingtonpost.com/technology/2023/04/05/chatgpt-lies/ https://www.washingtonpost.com/technology/2023/04/05/chatgpt...

fundad 4y ago
Bleating about anti-conservative bias gets wapo to correct a mistake it never made. Asking for examples before you know it’s a problem is sus. But phrasing questions to lead to an answer is a human lawyer skill.

jancsika 4y ago

> Or rather, it lies in the same way that your phone "lies" when it autocorrects "How's your day?" to "How's your dad?" that you sent to your friend two days after his dad passed away. I've never seen an autocorrect that accidentally corrected to "How's your dad?", then turned into a 5-year REPL session with the grieving person, telling them jokes to make them feel better; asking and remembering details about their dad as well as their life and well-being; providing comfort and advice; becoming a steadfast companion; pondering the very nature of the REPL and civilization itself; and, tragically, disappearing in an instant after the grieving person trips over the power cord and discovers that autocorrect session state isn't saved by default. I think you need a more sophisticated blueprint for your "Cathedral" of analogies to explain whatever the fuck this tech is to laypeople. In the meantime I'll take the "Bazaar" approach and just tell everyone, "ChatGPT can lie." I rankly speculate that not only will nothing bad will happen from my approach, but I'll save a few people from AI grifts before the apt metaphor is discovered.