45 ms·

ChatGPT did not lie; it cannot lie. It was given a sequence of words and tasked with producing a subsequent sequence of words that satisfy with high probabilit

ChatGPT did not lie; it cannot lie. It was given a sequence of words and tasked with producing a subsequent sequence of words that satisfy with high probability the constraints of the model. It did that admirably. It's not its fault, or in my opinion OpenAI's fault, that the output is being misunderstood and misused by people who can't be bothered understanding it and project their own ideas of how it should function onto it.

grumple 3y ago
"It doesn't lie, it just generates lies and printed them to the screen!" I don't think there's a difference.
- wvenable 3y ago
  It's a text generator. You ask it generate something and it does. It produces only stories. Sometimes those stories are based on actual facts. This lawyer told it produce a defence story and it did just that.
- Bytewave81 3y ago
  To perhaps stir the "what do words really mean" argument, "lying" would generally imply some sort of conscious intent to bend or break the truth. A language model is not consciously making decisions about what to say, it is statistically choosing words which probabilistically sound "good" together.
  - coldtea 3y ago
    >A language model is not consciously making decisions about what to say Well, that is being doubted -- and by some of the biggest names in the field. Namely that it isn't "statistically choosing words which probabilistically sound good together". But that doing so is not already making a consciousness (even if basic) emerge. >it is statistically choosing words which probabilistically sound "good" together. That when we do speak (or lie), we do something much more nuanced, and not just do a higher level equivalent of the same thing, plus have the emergent illusion of consciousness, is also an idea thrown around.
    - mensetmanusman 3y ago
      "Well, that is being doubted -- and by some of the biggest names in the field." An appeal to authority is still a fallacy. We don't even have a way of proving if a person is experiencing consciousness, why would anyone expect we could agree if a machine is.
      - coldtea 3y ago
        >An appeal to authority is still a fallacy Which is neither here, nor there. I wasn't making a formal argument, I was stating a fact. Take it or leave it.
- raincole 3y ago
  Saying ChatGPT lies is like saying The Onion lies.
  - ceejayoz 3y ago
    The Onion (via its staff) intends to produce falsehoods. ChatGPT (nor its staff) does not.
- acka 3y ago
  Does a piece of software with a bug in it which causes it to produce incorrect output lie or is it simply a programming error? Did the programmer who wrote the buggy code lie? I don't think so.
- mod50ack 3y ago
  Lying implies an intention. ChatGPT doesn't have that. What ChatGPT definitely does do is generate falsehoods. It's a bullshitting machine. Sometimes the bullshit produces true responses. But ChatGPT has no epistemological basis for knowing truths; it just is trained to say stuff.
  - lo_zamoyski 3y ago
    And if you want to be pedantic, ChatGPT isn't even generating falsehoods. A falsehood requires propositional content and therefore intentionality, but ChatGPT doesn't have that. It merely generates strings that, when interpreted by a human being as English text, signify falsehoods.
    - mod50ack 3y ago
      Getting into the weeds, but I don't agree with this construal of what propositional content is or can be. (There is no single definition of "proposition" which has wide acceptance and specifies your condition here.) There is no similar way to assess truth outside of formalized mathematics, but the encoding of mathematical statements (think Gödel numbers) comes to mind; I don't think that the ability of the machine to understand propositions is necessary in order to make the propositions propositional; the system of ChatGPT is designed in order to return propositional content (albeit not ex nihilo, but according to the principles of its design) and this could be considered analogous to the encoding of arithmetical symbolic notation into an formally-described system. The difference is just that we happen to have a formal description of how some arithmetic systems operate, which we don't (and I would say can't) have for English. Mild throwback to my university days studying all of this!
- zdragnar 3y ago
  The difference is everything. It doesn't understand intent, it doesn't have a motivation. This is no different than what fiction authors, songwriters, poets and painters do. The fact that people assume what it produces must always be real because it is sometimes real is not its fault. That lies with the people who uncritically accept what they are told.
  - hoistbypetard 3y ago
    > That lies with the people who uncritically accept what they are told. That's partly true. Just as much fault lies with the people who market it as "intelligence" to those who uncritically accept what they are told.
    - williamcotton 3y ago
      This is displayed directly under the input prompt: ChatGPT may produce inaccurate information about people, places, or facts.
      - hoistbypetard 3y ago
        That's a good start. I think it needs to be embedded in the output.
- Dalewyn 3y ago
  An exceedingly complicated Autocomplete program, which an "AI" like ChatGPT is, does not have motives, does not know the concept of "lying" (nor any concept thereof), and simply does things as ordered by its user.
- japoco 3y ago
  Lying needs intent. ChatGPT does not think therefore it doesn’t lie in that sense.
  - ClumsyPilot 3y ago
    Thats like saying robots don't murder - they just kill
    - fallingknife 3y ago
      Which is actually a very good analogy. A lot of things can kill you, but only a human can be a murderer.
      - NotACop182 3y ago
        And pit bulls, but I digress. The debate gets lost in translation when we start having what do words mean debate.
      - joe_the_user 3y ago
        In movies and written fiction, "intelligent" robots, anthropomorphized animals, elves, dwarves and etc can all commit murder when given the attributes of humans. We don't have real things with all human attributes but we're getting closer and as we get close "needs to be a human" will get thinner as an explanation of what is or isn't human for an act of murder, deception and so-forth.
  - leereeves 3y ago
    Merriam-Webster gives two definitions for the verb "lie". The first requires intent, the second does not: > to create a false or misleading impression > Statistics sometimes lie. > The mirror never lies.
  - joe_the_user 3y ago
    This is an interesting discussion. The ideas of philosophy meet the practical meaning of words here. You can reasonably say a database doesn't lie. It's just a tool, everyone agrees it's a tool and if you get the wrong answer, most people would agree it's your fault for making the wrong query or using the wrong data. But the difference between ChatGPT and a database is ChatGPT will support it's assertions. It will say things that support it's position - not just fake references but an entire line of argument. Of course, all of this is simply duplicating/simulating for humans in discussions. You can call it is a "simulated lie" if you don't like the idea of it really lying. But I claim that in normal usage, people will take this as "real" lying and ultimately that functional meaning is what "higher" more philosophical will have to accept.
- icedchai 3y ago
  There is a difference. Is fiction a lie?
- kristjansson 3y ago
  Language generated without regard to its truth value is different than language that cares about its truth value. Ask Harry Frankfurt.
nemo44x 3y ago
Correct. ChatGPT is a bullshitter, not a liar. A bullshitter isn’t concerned with facts or truth or anything. A liar is concerned with concealing the truth. Bullshitters are actually probably worse than liars because at least liars live in the same reality as honest people.
- whitemary 3y ago
  Liars (humans) are often unconcerned with facts or truth either. More often these days, they are only concerned with profit. Are unconfirmed lies in the pursuit of mere profit somehow not lies?
  - nemo44x 3y ago
    This book [1] covers it. [1] https://en.m.wikipedia.org/wiki/On_Bullshit#:~:text=The%20liar%20is%20intentionally%20avoiding,the%20intention%20of%20doing%20so https://en.m.wikipedia.org/wiki/On_Bullshit#:~:text=The%20li....
    - whitemary 3y ago
      No, it does not. I have read the full text and it was garbage to my mind; another nonsense apology for liberal society's degeneracy.
      - nemo44x 3y ago
        I’m curious how it apologizes for liberal societies degeneracy? I’ve read it as well and appreciate the nuance in differentiating the concepts regardless of which type of system you are in. For instance, the concepts are relevant in a reactionary system just as much as a degenerate progressive one.
      - dendriti 3y ago
        The Right's devotion to truth and honesty in politics and society should serve as a beacon to us all.
- 3y ago
SantalBlush 3y ago
It is designed to give the illusion that it reasons the way a human does, which is why many people are using it. To blame the average user--who quite obviously doesn't understand how LLMs work--isn't fair, either. A lawyer, however, should have vetted a new piece of tech before using it in this way.
- taberiand 3y ago
  Personally I think there should be restricted access to these models to people who have passed a basic course on their use and behaviour. (I know that will never ever happen)
coldtea 3y ago
>ChatGPT did not lie; it cannot lie. If it lies like a duck, it is a lying duck.
- aaronblohowiak 3y ago
  Hallucinated not lied.
- olalonde 3y ago
  It doesn't lie like a duck. It unintentionally says falsehoods. Lying is intentional.
  - coldtea 3y ago
    That's irrelevant to whether it lies like a duck or not. The expression "if it X like a duck" means precisely that we should judge a thing to be a duck or not, based on it having the external appereance and outward activity of a duck, and ignoring any further subleties, intent, internal processes, qualia, and so on. In other words, "it lies like a duck" means: if it produces things that look like lies, it is lying, and we don't care how it got to produce them. So, Chat-GPT absolutely does "lie like a duck".
    - throwaway675309 3y ago
      Abductive reasoning aside, people are already anthropomorphizing GPT enough without bringing in a loaded word like "lying" which implies intent. Hallucinates is a far more accurate word.
      - wlonkly 3y ago
        What bothers me about "hallucinates" is the removal of agency. When a human is hallucinating, something is happening to them that is essentially out of their control and they are suffering the effects of it, unable to tell truth from fiction, a dysfunction that they will recover from. But that's not really what happens with ChatGPT. The model doesn't know truth from fiction in the first place, but the whole point of a useful LLM is that there is some level of control and consistency around the output. I've been using "bullshitting", because I think that's really what ChatGPT is demonstrating -- not a disconnection from reality, but not letting truth get in the way of a good story.
agnosticmantis 3y ago
Whether a statement is true or false doesn’t depend on the mechanism generating the statement. We should hold these models (or more realistically, their creators) to the same standard as humans. What do we do with a human that generates plausible-sounding sentences without regard for their truth? Let’s hold the creators of these models accountable, and everything will be better.
- MattGaiser 3y ago
  > What do we do with a human that generates plausible-sounding sentences without regard for their truth? Elect them as leaders?
  - endofreach 3y ago
    I always wondered about the weird typo when people write „GPT“ but substitute the „P“ and „T“ with an „O“ and „D“. Hail ChatGPT!
  - icedchai 3y ago
    ChatGPT is perfect for generating company mission statements, political rhetoric, and other forms of BS.
    - shagie 3y ago
      In Anathem there's the word: Bulshytt Bulshytt: Speech (typically but not necessarily commercial or political) that employs euphemism, convenient vagueness, numbing repetition, and other such rhetorical subterfuges to create the impression that something has been said. (there's quite a bit more about it to be said, though quoting it out of context loses much of the world building associated with it... and the raw quote is riddled with strange spellings that would have even more confusion)
- bobsmooth 3y ago
  >Let’s hold the creators of these models accountable, and everything will be better. Shall we hold Adobe responsible for people photoshopping their ex's face into porn as well?
flangola7 3y ago
ChatGPT isn't a legal entity but OpenAI is, and Altman has already recommend to Congress that coming regulations should make AI companies liable for produced text and be 230 exempt. I can see it already happening even without legislation, 230 shields liability from user-generated content but ChatGPT output isn't user generated. It's not even a recommendation algorithm steering you into other users' content telling why you should kill yourself - the company itself produced the content. If I was a judge or justice that would be cut and dry to me. Companies with AI models need to treat the models as if they were an employee. If your employee starts giving confidently bad legal advice to customers, you need to nip that in the bud or you're going to have a lot of problems.
- shagie 3y ago
  Why should OpenAI be more liable for a tool that they've created than any other tool creator where the tool is intentionally misused and warnings on the tool ignored? If I wrote text in Microsoft Word and in doing so, I had a typo in (for example) the name of a drug that Word corrected to something that was incorrect, is Microsoft liable for the use of autocorrect? If I was copying and pasting data into excel and some of it was interpreted as a date rather than some other data format resulting in an incorrect calculation that I didn't check at the end, is Microsoft again liable for that? At the bottom of the ChatGPT page, there's the text: ChatGPT may produce inaccurate information about people, places, or facts. If I can make an instance of Eliza say obscene or incorrect things, does that make the estate of Weizenbaum liable?
  - wang_li 3y ago
    >Why should OpenAI be more liable for a tool that they've created than any other tool creator where the tool is intentionally misused and warnings on the tool ignored? Because they provide a service, not a tool.
    - shagie 3y ago
      What is the service that OpenAI claims to provide? How is this different than Grammerly?
clnq 3y ago
This harks back to around 1999 when people would often blame computers for mistakes in their math, documents, reports, sworn filings, and so on. Then, a thousand different permutations of "computers don't make mistakes" or "computers are never wrong" became popular sayings. Large Language Models (LLMs) are never wrong, and they do not make mistakes. They are not fact machines. Their purpose is to abstract knowledge and to produce plausible language. GPT-4 is actually quite good at handling facts, yet it still hallucinates facts that are not common knowledge, such as legal ones. GPT-3.5, the original ChatGPT and the non-premium version, is less effective with even slightly obscure facts, like determining if a renowned person is a member of a particular organization. This is why we can't always have nice things. This is why AI must be carefully aligned to make it safe. Sooner or later, a lawyer might consider the plausible language produced by LLMs to be factual. Then, a politician might do the same, followed by a teacher, a therapist, a historian, or even a doctor. I thought the warnings about its tendency to hallucinate speech were clear — those warnings displayed the first time you open ChatGPT. To most people, I believe they were.
- RhysU 3y ago
  > Large Language Models (LLMs) are never wrong, and they do not make mistakes. I call B.S. If LLMs never made mistakes we wouldn't train them. Any random initialization would work.
  - taberiand 3y ago
    That's a different error context I think. It's a mistake if the model produces nonsense, because it's designed to produce realistic text. It's not a mistake if it produces non-factual information that looks realistic. And it fundamentally cannot always produce factual information, it doesn't have that capacity (but then, neither do humans and with the ability to source information this statement may well be obsolete soon enough) Though I wouldn't go so far as to say that the model cannot make mistakes - it clearly is susceptible to producing nonsense. I just think expecting it to always produce factual information is like using a hammer to cut wood and complaining the wood comes out all jagged
User23 3y ago
Right. Technically speaking ChatGPT bullshitted[1]. It can only bullshit. It is entirely indifferent to truth or falsehood and thus it can neither be honest nor lie. It is however an impressive bullshit generator. Even more impressively, a decent amount of the bullshit it generates is in fact true or otherwise correct. [1] using Frankfurt’s definition that it is communication that is completely indifferent to truth or falsehood.
whitemary 3y ago
> It was given a sequence of words and tasked with producing a subsequent sequence of words that satisfy with high probability the constraints of the model. This is exactly the sort of behavior that produces many of the lies that humans tell everyday. The "constraints of the model" are synonymous with the constraints of a person's knowledge of the world (which is their model).
smrtinsert 3y ago
Exactly. ChatGPT describes a universe recreated using probabilities. Caveat emptor.
daveguy 3y ago
Corollary: ChatGPT did not tell the truth; it cannot tell the truth.
daveguy 3y ago
Corollary: ChatGPT did not tell the truth; it cannot tell the truth. If you accept the premise of the parent post, then this is a natural corollary. I accept the premise of the parent post.
- zztop44 3y ago
  Or more precisely: the truth or falsity of ChatGPT’s statements is incidental to the process by which it determines which statements to make. Because unless you are using some personal definition of “tell the truth” then you must accept that ChatGPT often outputs statements which are demonstrably true.
  - daveguy 3y ago
    Yes, in this case "tell the truth" means knowingly tell the truth. Or knowingly lie. It just doesn't have a clue which is which.
croes 3y ago
>ChatGPT did not lie; it cannot lie. More important it can't tell the truth either. It produces the mostly likely series of words for the given prompt.
richardjam73 3y ago
The problem comes from people who call LLMs AIs. Then people who don't know how they work assume it is intelligent when it is not. I'm pretty sure that OpenAI is at fault in this by not informing users of the truth.
- sillysaurusx 3y ago
  Quite so. I had a visceral aversion to ML being rebranded AI. ML only very recently became anything resembling AI with ChatGPT. (Admittedly it’s so good as a coding assistant that it does feel like talking to Data from Star Trek though.)
quickthrower2 3y ago
Correct, it did not lie with intent. The best way to describe this in a “compared to human” way to describe it: is it is not mentally competent to answer questions
andrewfong 3y ago
Well, it sort of is OpenAI's fault that it presented the interface as a chat bot though. > It was given a sequence of words and tasked with producing a subsequent sequence of words that satisfy with high probability the constraints of the model. This is just autocorrect / autocomplete. And people are pretty good at understanding the limitations of generative text in that context (enough that "damn you autocorrect" is a thing). But for whatever reason, people assign more trust to conversational interfaces.
pms 3y ago
Their "own" ideas? Let me remind you that OpenAI released a report purposefully suggesting that GPT4 has relatively high IQ, passes a lot of college-level tests, and solves coding problems. Then it was revealed that there was training data contamination that led to good results in such tests [1], but GPT4 marketing received 10000 more attention than truth anyway. The popular belief is that using LLMs will give you a professional competitive advantage. Also, when we talk about the achievements of LLMs, then we anthropomorphize, but when we talk about their failures, then we don't anthropomorphize, i.e., "AI cannot lie"? Don't you see human biases drive AI hype? In my opinion, people clearly are confused and misled by marketing and this isn't the first time it's happening. For instance, people were confused for 40+ about global warming, among others due to greenwashing campaigns [2]. Is it ok to mislead in ads? Are we supposed to purposefully take advantage of others by keeping them confused to gain a competitive advantage? [1] https://twitter.com/cHHillee/status/1635790330854526981 https://twitter.com/cHHillee/status/1635790330854526981 [2] https://en.wikipedia.org/wiki/Global_Climate_Coalition https://en.wikipedia.org/wiki/Global_Climate_Coalition
- taberiand 3y ago
  The context is people who should know better, whose job it is to put the effort into understanding the tools they are using. Of course, I think these AI tools should require a basic educational course on their behaviour and operation before they can be used. But marketing nonsense is standard with everything; people have at least some responsibility for self education.
  - pms 3y ago
    > Marketing nonsense is standard Definitely, it will be standard, until we start pointing out lies.

grumple 3y ago

"It doesn't lie, it just generates lies and printed them to the screen!" I don't think there's a difference.

wvenable 3y ago
It's a text generator. You ask it generate something and it does. It produces only stories. Sometimes those stories are based on actual facts. This lawyer told it produce a defence story and it did just that.
Bytewave81 3y ago
To perhaps stir the "what do words really mean" argument, "lying" would generally imply some sort of conscious intent to bend or break the truth. A language model is not consciously making decisions about what to say, it is statistically choosing words which probabilistically sound "good" together.
- coldtea 3y ago
  >A language model is not consciously making decisions about what to say Well, that is being doubted -- and by some of the biggest names in the field. Namely that it isn't "statistically choosing words which probabilistically sound good together". But that doing so is not already making a consciousness (even if basic) emerge. >it is statistically choosing words which probabilistically sound "good" together. That when we do speak (or lie), we do something much more nuanced, and not just do a higher level equivalent of the same thing, plus have the emergent illusion of consciousness, is also an idea thrown around.
  - mensetmanusman 3y ago
    "Well, that is being doubted -- and by some of the biggest names in the field." An appeal to authority is still a fallacy. We don't even have a way of proving if a person is experiencing consciousness, why would anyone expect we could agree if a machine is.
    - coldtea 3y ago
      >An appeal to authority is still a fallacy Which is neither here, nor there. I wasn't making a formal argument, I was stating a fact. Take it or leave it.
raincole 3y ago
Saying ChatGPT lies is like saying The Onion lies.
- ceejayoz 3y ago
  The Onion (via its staff) intends to produce falsehoods. ChatGPT (nor its staff) does not.
acka 3y ago
Does a piece of software with a bug in it which causes it to produce incorrect output lie or is it simply a programming error? Did the programmer who wrote the buggy code lie? I don't think so.
mod50ack 3y ago
Lying implies an intention. ChatGPT doesn't have that. What ChatGPT definitely does do is generate falsehoods. It's a bullshitting machine. Sometimes the bullshit produces true responses. But ChatGPT has no epistemological basis for knowing truths; it just is trained to say stuff.
- lo_zamoyski 3y ago
  And if you want to be pedantic, ChatGPT isn't even generating falsehoods. A falsehood requires propositional content and therefore intentionality, but ChatGPT doesn't have that. It merely generates strings that, when interpreted by a human being as English text, signify falsehoods.
  - mod50ack 3y ago
    Getting into the weeds, but I don't agree with this construal of what propositional content is or can be. (There is no single definition of "proposition" which has wide acceptance and specifies your condition here.) There is no similar way to assess truth outside of formalized mathematics, but the encoding of mathematical statements (think Gödel numbers) comes to mind; I don't think that the ability of the machine to understand propositions is necessary in order to make the propositions propositional; the system of ChatGPT is designed in order to return propositional content (albeit not ex nihilo, but according to the principles of its design) and this could be considered analogous to the encoding of arithmetical symbolic notation into an formally-described system. The difference is just that we happen to have a formal description of how some arithmetic systems operate, which we don't (and I would say can't) have for English. Mild throwback to my university days studying all of this!
zdragnar 3y ago
The difference is everything. It doesn't understand intent, it doesn't have a motivation. This is no different than what fiction authors, songwriters, poets and painters do. The fact that people assume what it produces must always be real because it is sometimes real is not its fault. That lies with the people who uncritically accept what they are told.
- hoistbypetard 3y ago
  > That lies with the people who uncritically accept what they are told. That's partly true. Just as much fault lies with the people who market it as "intelligence" to those who uncritically accept what they are told.
  - williamcotton 3y ago
    This is displayed directly under the input prompt: ChatGPT may produce inaccurate information about people, places, or facts.
    - hoistbypetard 3y ago
      That's a good start. I think it needs to be embedded in the output.
Dalewyn 3y ago
An exceedingly complicated Autocomplete program, which an "AI" like ChatGPT is, does not have motives, does not know the concept of "lying" (nor any concept thereof), and simply does things as ordered by its user.
japoco 3y ago
Lying needs intent. ChatGPT does not think therefore it doesn’t lie in that sense.
- ClumsyPilot 3y ago
  Thats like saying robots don't murder - they just kill
  - fallingknife 3y ago
    Which is actually a very good analogy. A lot of things can kill you, but only a human can be a murderer.
    - NotACop182 3y ago
      And pit bulls, but I digress. The debate gets lost in translation when we start having what do words mean debate.
    - joe_the_user 3y ago
      In movies and written fiction, "intelligent" robots, anthropomorphized animals, elves, dwarves and etc can all commit murder when given the attributes of humans. We don't have real things with all human attributes but we're getting closer and as we get close "needs to be a human" will get thinner as an explanation of what is or isn't human for an act of murder, deception and so-forth.
- leereeves 3y ago
  Merriam-Webster gives two definitions for the verb "lie". The first requires intent, the second does not: > to create a false or misleading impression > Statistics sometimes lie. > The mirror never lies.
- joe_the_user 3y ago
  This is an interesting discussion. The ideas of philosophy meet the practical meaning of words here. You can reasonably say a database doesn't lie. It's just a tool, everyone agrees it's a tool and if you get the wrong answer, most people would agree it's your fault for making the wrong query or using the wrong data. But the difference between ChatGPT and a database is ChatGPT will support it's assertions. It will say things that support it's position - not just fake references but an entire line of argument. Of course, all of this is simply duplicating/simulating for humans in discussions. You can call it is a "simulated lie" if you don't like the idea of it really lying. But I claim that in normal usage, people will take this as "real" lying and ultimately that functional meaning is what "higher" more philosophical will have to accept.
icedchai 3y ago
There is a difference. Is fiction a lie?
kristjansson 3y ago
Language generated without regard to its truth value is different than language that cares about its truth value. Ask Harry Frankfurt.

nemo44x 3y ago

Correct. ChatGPT is a bullshitter, not a liar. A bullshitter isn’t concerned with facts or truth or anything. A liar is concerned with concealing the truth. Bullshitters are actually probably worse than liars because at least liars live in the same reality as honest people.

whitemary 3y ago
Liars (humans) are often unconcerned with facts or truth either. More often these days, they are only concerned with profit. Are unconfirmed lies in the pursuit of mere profit somehow not lies?
- nemo44x 3y ago
  This book [1] covers it. [1] https://en.m.wikipedia.org/wiki/On_Bullshit#:~:text=The%20liar%20is%20intentionally%20avoiding,the%20intention%20of%20doing%20so https://en.m.wikipedia.org/wiki/On_Bullshit#:~:text=The%20li....
  - whitemary 3y ago
    No, it does not. I have read the full text and it was garbage to my mind; another nonsense apology for liberal society's degeneracy.
    - nemo44x 3y ago
      I’m curious how it apologizes for liberal societies degeneracy? I’ve read it as well and appreciate the nuance in differentiating the concepts regardless of which type of system you are in. For instance, the concepts are relevant in a reactionary system just as much as a degenerate progressive one.
    - dendriti 3y ago
      The Right's devotion to truth and honesty in politics and society should serve as a beacon to us all.
3y ago

SantalBlush 3y ago

It is designed to give the illusion that it reasons the way a human does, which is why many people are using it. To blame the average user--who quite obviously doesn't understand how LLMs work--isn't fair, either. A lawyer, however, should have vetted a new piece of tech before using it in this way.

taberiand 3y ago
Personally I think there should be restricted access to these models to people who have passed a basic course on their use and behaviour. (I know that will never ever happen)

coldtea 3y ago

>ChatGPT did not lie; it cannot lie. If it lies like a duck, it is a lying duck.

aaronblohowiak 3y ago
Hallucinated not lied.
olalonde 3y ago
It doesn't lie like a duck. It unintentionally says falsehoods. Lying is intentional.
- coldtea 3y ago
  That's irrelevant to whether it lies like a duck or not. The expression "if it X like a duck" means precisely that we should judge a thing to be a duck or not, based on it having the external appereance and outward activity of a duck, and ignoring any further subleties, intent, internal processes, qualia, and so on. In other words, "it lies like a duck" means: if it produces things that look like lies, it is lying, and we don't care how it got to produce them. So, Chat-GPT absolutely does "lie like a duck".
  - throwaway675309 3y ago
    Abductive reasoning aside, people are already anthropomorphizing GPT enough without bringing in a loaded word like "lying" which implies intent. Hallucinates is a far more accurate word.
    - wlonkly 3y ago
      What bothers me about "hallucinates" is the removal of agency. When a human is hallucinating, something is happening to them that is essentially out of their control and they are suffering the effects of it, unable to tell truth from fiction, a dysfunction that they will recover from. But that's not really what happens with ChatGPT. The model doesn't know truth from fiction in the first place, but the whole point of a useful LLM is that there is some level of control and consistency around the output. I've been using "bullshitting", because I think that's really what ChatGPT is demonstrating -- not a disconnection from reality, but not letting truth get in the way of a good story.

agnosticmantis 3y ago

Whether a statement is true or false doesn’t depend on the mechanism generating the statement. We should hold these models (or more realistically, their creators) to the same standard as humans. What do we do with a human that generates plausible-sounding sentences without regard for their truth? Let’s hold the creators of these models accountable, and everything will be better.

MattGaiser 3y ago
> What do we do with a human that generates plausible-sounding sentences without regard for their truth? Elect them as leaders?
- endofreach 3y ago
  I always wondered about the weird typo when people write „GPT“ but substitute the „P“ and „T“ with an „O“ and „D“. Hail ChatGPT!
- icedchai 3y ago
  ChatGPT is perfect for generating company mission statements, political rhetoric, and other forms of BS.
  - shagie 3y ago
    In Anathem there's the word: Bulshytt Bulshytt: Speech (typically but not necessarily commercial or political) that employs euphemism, convenient vagueness, numbing repetition, and other such rhetorical subterfuges to create the impression that something has been said. (there's quite a bit more about it to be said, though quoting it out of context loses much of the world building associated with it... and the raw quote is riddled with strange spellings that would have even more confusion)
bobsmooth 3y ago
>Let’s hold the creators of these models accountable, and everything will be better. Shall we hold Adobe responsible for people photoshopping their ex's face into porn as well?

flangola7 3y ago

ChatGPT isn't a legal entity but OpenAI is, and Altman has already recommend to Congress that coming regulations should make AI companies liable for produced text and be 230 exempt. I can see it already happening even without legislation, 230 shields liability from user-generated content but ChatGPT output isn't user generated. It's not even a recommendation algorithm steering you into other users' content telling why you should kill yourself - the company itself produced the content. If I was a judge or justice that would be cut and dry to me. Companies with AI models need to treat the models as if they were an employee. If your employee starts giving confidently bad legal advice to customers, you need to nip that in the bud or you're going to have a lot of problems.

shagie 3y ago
Why should OpenAI be more liable for a tool that they've created than any other tool creator where the tool is intentionally misused and warnings on the tool ignored? If I wrote text in Microsoft Word and in doing so, I had a typo in (for example) the name of a drug that Word corrected to something that was incorrect, is Microsoft liable for the use of autocorrect? If I was copying and pasting data into excel and some of it was interpreted as a date rather than some other data format resulting in an incorrect calculation that I didn't check at the end, is Microsoft again liable for that? At the bottom of the ChatGPT page, there's the text: ChatGPT may produce inaccurate information about people, places, or facts. If I can make an instance of Eliza say obscene or incorrect things, does that make the estate of Weizenbaum liable?
- wang_li 3y ago
  >Why should OpenAI be more liable for a tool that they've created than any other tool creator where the tool is intentionally misused and warnings on the tool ignored? Because they provide a service, not a tool.
  - shagie 3y ago
    What is the service that OpenAI claims to provide? How is this different than Grammerly?

clnq 3y ago

This harks back to around 1999 when people would often blame computers for mistakes in their math, documents, reports, sworn filings, and so on. Then, a thousand different permutations of "computers don't make mistakes" or "computers are never wrong" became popular sayings. Large Language Models (LLMs) are never wrong, and they do not make mistakes. They are not fact machines. Their purpose is to abstract knowledge and to produce plausible language. GPT-4 is actually quite good at handling facts, yet it still hallucinates facts that are not common knowledge, such as legal ones. GPT-3.5, the original ChatGPT and the non-premium version, is less effective with even slightly obscure facts, like determining if a renowned person is a member of a particular organization. This is why we can't always have nice things. This is why AI must be carefully aligned to make it safe. Sooner or later, a lawyer might consider the plausible language produced by LLMs to be factual. Then, a politician might do the same, followed by a teacher, a therapist, a historian, or even a doctor. I thought the warnings about its tendency to hallucinate speech were clear — those warnings displayed the first time you open ChatGPT. To most people, I believe they were.

RhysU 3y ago
> Large Language Models (LLMs) are never wrong, and they do not make mistakes. I call B.S. If LLMs never made mistakes we wouldn't train them. Any random initialization would work.
- taberiand 3y ago
  That's a different error context I think. It's a mistake if the model produces nonsense, because it's designed to produce realistic text. It's not a mistake if it produces non-factual information that looks realistic. And it fundamentally cannot always produce factual information, it doesn't have that capacity (but then, neither do humans and with the ability to source information this statement may well be obsolete soon enough) Though I wouldn't go so far as to say that the model cannot make mistakes - it clearly is susceptible to producing nonsense. I just think expecting it to always produce factual information is like using a hammer to cut wood and complaining the wood comes out all jagged

User23 3y ago

Right. Technically speaking ChatGPT bullshitted[1]. It can only bullshit. It is entirely indifferent to truth or falsehood and thus it can neither be honest nor lie. It is however an impressive bullshit generator. Even more impressively, a decent amount of the bullshit it generates is in fact true or otherwise correct. [1] using Frankfurt’s definition that it is communication that is completely indifferent to truth or falsehood.

whitemary 3y ago

> It was given a sequence of words and tasked with producing a subsequent sequence of words that satisfy with high probability the constraints of the model. This is exactly the sort of behavior that produces many of the lies that humans tell everyday. The "constraints of the model" are synonymous with the constraints of a person's knowledge of the world (which is their model).

smrtinsert 3y ago

Exactly. ChatGPT describes a universe recreated using probabilities. Caveat emptor.

daveguy 3y ago

Corollary: ChatGPT did not tell the truth; it cannot tell the truth.

daveguy 3y ago

Corollary: ChatGPT did not tell the truth; it cannot tell the truth. If you accept the premise of the parent post, then this is a natural corollary. I accept the premise of the parent post.

zztop44 3y ago
Or more precisely: the truth or falsity of ChatGPT’s statements is incidental to the process by which it determines which statements to make. Because unless you are using some personal definition of “tell the truth” then you must accept that ChatGPT often outputs statements which are demonstrably true.
- daveguy 3y ago
  Yes, in this case "tell the truth" means knowingly tell the truth. Or knowingly lie. It just doesn't have a clue which is which.

croes 3y ago

>ChatGPT did not lie; it cannot lie. More important it can't tell the truth either. It produces the mostly likely series of words for the given prompt.

richardjam73 3y ago

The problem comes from people who call LLMs AIs. Then people who don't know how they work assume it is intelligent when it is not. I'm pretty sure that OpenAI is at fault in this by not informing users of the truth.

sillysaurusx 3y ago
Quite so. I had a visceral aversion to ML being rebranded AI. ML only very recently became anything resembling AI with ChatGPT. (Admittedly it’s so good as a coding assistant that it does feel like talking to Data from Star Trek though.)

quickthrower2 3y ago

Correct, it did not lie with intent. The best way to describe this in a “compared to human” way to describe it: is it is not mentally competent to answer questions

andrewfong 3y ago

Well, it sort of is OpenAI's fault that it presented the interface as a chat bot though. > It was given a sequence of words and tasked with producing a subsequent sequence of words that satisfy with high probability the constraints of the model. This is just autocorrect / autocomplete. And people are pretty good at understanding the limitations of generative text in that context (enough that "damn you autocorrect" is a thing). But for whatever reason, people assign more trust to conversational interfaces.

pms 3y ago

Their "own" ideas? Let me remind you that OpenAI released a report purposefully suggesting that GPT4 has relatively high IQ, passes a lot of college-level tests, and solves coding problems. Then it was revealed that there was training data contamination that led to good results in such tests [1], but GPT4 marketing received 10000 more attention than truth anyway. The popular belief is that using LLMs will give you a professional competitive advantage. Also, when we talk about the achievements of LLMs, then we anthropomorphize, but when we talk about their failures, then we don't anthropomorphize, i.e., "AI cannot lie"? Don't you see human biases drive AI hype? In my opinion, people clearly are confused and misled by marketing and this isn't the first time it's happening. For instance, people were confused for 40+ about global warming, among others due to greenwashing campaigns [2]. Is it ok to mislead in ads? Are we supposed to purposefully take advantage of others by keeping them confused to gain a competitive advantage? [1] https://twitter.com/cHHillee/status/1635790330854526981 https://twitter.com/cHHillee/status/1635790330854526981 [2] https://en.wikipedia.org/wiki/Global_Climate_Coalition https://en.wikipedia.org/wiki/Global_Climate_Coalition

taberiand 3y ago
The context is people who should know better, whose job it is to put the effort into understanding the tools they are using. Of course, I think these AI tools should require a basic educational course on their behaviour and operation before they can be used. But marketing nonsense is standard with everything; people have at least some responsibility for self education.
- pms 3y ago
  > Marketing nonsense is standard Definitely, it will be standard, until we start pointing out lies.