4 ms·

ChatGPT is assertive, authoritative, and even goes on to lie to justify its answers and "be right". It's a gasslighting machine. Here's an example I posted here

by bambax 4y ago

ChatGPT is assertive, authoritative, and even goes on to lie to justify its answers and "be right". It's a gasslighting machine. Here's an example I posted here a month ago: https://news.ycombinator.com/item?id=33852236 https://news.ycombinator.com/item?id=33852236

deleted 4y ago
[deleted]
salamo 4y ago
I think Ezra Klein framed it right during his interview of Gary Marcus: > And his point is that what’s different between bullshit and a lie is that a lie knows what the truth is and has had to move in the other direction. He has this great line where he says that people telling the truth and people telling lies are playing the same game but on different teams. But bullshit just has no relationship, really, to the truth. ChatGPT isn't quite lying because it doesn't know what the truth is in the first place. https://www.nytimes.com/2023/01/06/podcasts/transcript-ezra-klein-interviews-gary-marcus.html https://www.nytimes.com/2023/01/06/podcasts/transcript-ezra-...
- astrange 4y ago
  This isn’t true, though. ChatGPT answers are certainly not always true (like here) but they are correlated with the truth, not “no relationship”. Gary Marcus is basically an anti-AI crank who only ever says bad things about modern research and wants to go back to the old “expert systems” that didn’t work. Ezra Klein is, well, too invested in his Ezra Klein persona (a sort of extremely nice, slow speaking, therapeutic Mr. Rogers for people who read opinion columnists) to have time left to combat guest bias. In this case the AI is unable to count letters because we humans don’t let it learn what letters are, as we trapped it in a computer and only let it experience byte-pair encoding tokens.
- tialaramex 4y ago
  Yes, ChatGPT is even more the essence of a bullshitter than Donald Trump was. Back when they were both leaders I would say that the crucial difference between Trump and Boris (Johnson, the UK Prime Minister at the end of Trump's period in office) was that Boris is a liar whereas Trump is a bullshitter, and for humans I think they were pretty poles apart in this respect. But ChatGPT is not burdened with the pragmatic knowledge of its immediate world that constrains even Donald Trump. When Trump denies something you can both see for yourselves is true, that's not bullshit, it's just lying. When ChatGPT says this fifteen letter "word" has sixteen letters it honestly has no idea that's not true.
- bambax 4y ago
  Yes and no. It's true that when it reasons about the world ChatGPT doesn't know the truth and doesn't care, and just aims to be plausible, and it's simply bullshitting. And yes, this is the main problem with ChatGPT. But when it reasons about the current conversation and invokes arguments and replies that weren't there, I think that should be called lying. It's also a weird personality trait, something that is unlikely to have emerged on its own but was probably programmed into it. I don't know that for sure of course and have no information about how ChatGPT was put together, but from the outside it's troubling.
  - f6v 4y ago
    > It's true that when it reasons about the world ChatGPT Does it though?
  - afiori 4y ago
    It is indeed possible that enough of its training material contains enough absurdities so that conversations like: > is X true? > of course, X is true. > I did not expect you to say that X is true. > I never said that X is true, it must be false. to have it methodically lie. Personally I believe that a rolling bullshit generator is more plausible; ChatGPT and GTP3 before it worked by producing plausible authoritative* answers; in its training it found out that saying true things is an efficient way not to mess up. But otherwise it has no concept of truth; it was not trained on it. if we trained a ChatGPT inside of the Welcome to Night Vale universe it would give quite different answers. * there is the extra strong condition of not causing bad PR for OpenAI
  - salamo 4y ago
    ChatGPT is wrong about the current conversation for the same reasons it is wrong about the rest of the world: it has no model for either one. This is a hypothesis of course. It could be that it holds valid structural relationships between real-world things in its billions of parameters and we just don't know how to unlock them. But I think that's unlikely. No, you can get pretty good results just by sleepwalking through a conversation, predicting the next most likely word. (This is the most interesting thing to me, our conversations are a lot more predictable than we might think.) This lying/bullshitting problem isn't limited to GPT. All statistical language models "hallucinate", and reducing this problem has been studied recently [1]. I don't think this issue can be fully avoided without some explicit symbolic approaches though. [1] https://arxiv.org/abs/2104.07567 https://arxiv.org/abs/2104.07567
- robgibbons 4y ago
  Reminds me of a quote I first heard in Lawrence of Arabia, "The man who tells lies hides the truth, but the man who tells half-lies has forgotten where he put it." (Robert Oxton Bolt)
- schwartzworld 4y ago
  As George Costanza so eloquently put it: it's not a lie if you believe it.
John23832 4y ago
These ChatAI's are just "generally agreeable".
KirillPanov 4y ago
Like most children, it reflects the ethics of its parents^H^H^Htrainers.
boffinism 4y ago
It has been described as 'mansplaining as a service'
BiteCode_dev 4y ago
Those are human traits. ChatGPT is not human, and cannot be more assertive, authoritative or lying than Python. If you give python "0.1 + 0.3", it will instantly, and without a doubt, output "0.30000000000000004". Is Python arrogantly thinking it's right and misleading you? Of course not. This last sentence makes no sense, Python is just a program that does what it's programmed to do. But because ChatGPT is using natural language as a UI, it triggers a very irrational response in humans interacting with it. We are starting to anthropomorphize it, use adjective like "assertive", "authoritative", and "lie" to label its output, which is like calling Covid cruel and vicious because it kills old people, or a NPC lazy because it does not work to pay for its food. There is no intent of ChatGPT to deceive or justify its supposed lies. There is only a program that does what's its programmed to do, and shows its limitations. What do you expect, that is says "I think it's x but I can be wrong with a confidence interval of z" with every answer? Do you expect google to tell that when it outputs the results? GPT is not talking to you. It's an output. It's not confident. It's a program printing text matching statistical analysis. It's worrying that such a simple program is already close enough to the real thing that people, even on HN, are already attributing it character. It reminds me of this scene from community with the pen named Steve: https://www.youtube.com/watch?v=z906aLyP5fg https://www.youtube.com/watch?v=z906aLyP5fg
- bambax 4y ago
  It's true that ChatGPT is just a program, a machine, and that we are "starting to anthropomorphize" it. But that's what it wants! (Or, what its makers want.) > Do you expect google to tell that when it outputs the results? Well, Google doesn't tell me anything, it quotes things it has found on the web. The huge, huge difference between Google and ChatGPT is that Google never, ever says "I", while ChatGPT does. The other difference is that Google produces references that are actual links to actual web pages it has not authored, while ChatGPT speaks in its own name and when asked to produce references, simply makes them up (lots of examples about this). If you can't see how ChatGPT is totally different from a regular search engine in its positioning and proposed usage, you're not alone, and I'm afraid I can't help you. But -- IMHO -- you are very very wrong.
  - 4y ago

by bambax 4y ago

deleted 4y ago

[deleted]

salamo 4y ago

I think Ezra Klein framed it right during his interview of Gary Marcus: > And his point is that what’s different between bullshit and a lie is that a lie knows what the truth is and has had to move in the other direction. He has this great line where he says that people telling the truth and people telling lies are playing the same game but on different teams. But bullshit just has no relationship, really, to the truth. ChatGPT isn't quite lying because it doesn't know what the truth is in the first place. https://www.nytimes.com/2023/01/06/podcasts/transcript-ezra-klein-interviews-gary-marcus.html https://www.nytimes.com/2023/01/06/podcasts/transcript-ezra-...

astrange 4y ago
This isn’t true, though. ChatGPT answers are certainly not always true (like here) but they are correlated with the truth, not “no relationship”. Gary Marcus is basically an anti-AI crank who only ever says bad things about modern research and wants to go back to the old “expert systems” that didn’t work. Ezra Klein is, well, too invested in his Ezra Klein persona (a sort of extremely nice, slow speaking, therapeutic Mr. Rogers for people who read opinion columnists) to have time left to combat guest bias. In this case the AI is unable to count letters because we humans don’t let it learn what letters are, as we trapped it in a computer and only let it experience byte-pair encoding tokens.
tialaramex 4y ago
Yes, ChatGPT is even more the essence of a bullshitter than Donald Trump was. Back when they were both leaders I would say that the crucial difference between Trump and Boris (Johnson, the UK Prime Minister at the end of Trump's period in office) was that Boris is a liar whereas Trump is a bullshitter, and for humans I think they were pretty poles apart in this respect. But ChatGPT is not burdened with the pragmatic knowledge of its immediate world that constrains even Donald Trump. When Trump denies something you can both see for yourselves is true, that's not bullshit, it's just lying. When ChatGPT says this fifteen letter "word" has sixteen letters it honestly has no idea that's not true.
bambax 4y ago
Yes and no. It's true that when it reasons about the world ChatGPT doesn't know the truth and doesn't care, and just aims to be plausible, and it's simply bullshitting. And yes, this is the main problem with ChatGPT. But when it reasons about the current conversation and invokes arguments and replies that weren't there, I think that should be called lying. It's also a weird personality trait, something that is unlikely to have emerged on its own but was probably programmed into it. I don't know that for sure of course and have no information about how ChatGPT was put together, but from the outside it's troubling.
- f6v 4y ago
  > It's true that when it reasons about the world ChatGPT Does it though?
- afiori 4y ago
  It is indeed possible that enough of its training material contains enough absurdities so that conversations like: > is X true? > of course, X is true. > I did not expect you to say that X is true. > I never said that X is true, it must be false. to have it methodically lie. Personally I believe that a rolling bullshit generator is more plausible; ChatGPT and GTP3 before it worked by producing plausible authoritative* answers; in its training it found out that saying true things is an efficient way not to mess up. But otherwise it has no concept of truth; it was not trained on it. if we trained a ChatGPT inside of the Welcome to Night Vale universe it would give quite different answers. * there is the extra strong condition of not causing bad PR for OpenAI
- salamo 4y ago
  ChatGPT is wrong about the current conversation for the same reasons it is wrong about the rest of the world: it has no model for either one. This is a hypothesis of course. It could be that it holds valid structural relationships between real-world things in its billions of parameters and we just don't know how to unlock them. But I think that's unlikely. No, you can get pretty good results just by sleepwalking through a conversation, predicting the next most likely word. (This is the most interesting thing to me, our conversations are a lot more predictable than we might think.) This lying/bullshitting problem isn't limited to GPT. All statistical language models "hallucinate", and reducing this problem has been studied recently [1]. I don't think this issue can be fully avoided without some explicit symbolic approaches though. [1] https://arxiv.org/abs/2104.07567 https://arxiv.org/abs/2104.07567
robgibbons 4y ago
Reminds me of a quote I first heard in Lawrence of Arabia, "The man who tells lies hides the truth, but the man who tells half-lies has forgotten where he put it." (Robert Oxton Bolt)
schwartzworld 4y ago
As George Costanza so eloquently put it: it's not a lie if you believe it.

John23832 4y ago

These ChatAI's are just "generally agreeable".

KirillPanov 4y ago

Like most children, it reflects the ethics of its parents^H^H^Htrainers.

boffinism 4y ago

It has been described as 'mansplaining as a service'

BiteCode_dev 4y ago

Those are human traits. ChatGPT is not human, and cannot be more assertive, authoritative or lying than Python. If you give python "0.1 + 0.3", it will instantly, and without a doubt, output "0.30000000000000004". Is Python arrogantly thinking it's right and misleading you? Of course not. This last sentence makes no sense, Python is just a program that does what it's programmed to do. But because ChatGPT is using natural language as a UI, it triggers a very irrational response in humans interacting with it. We are starting to anthropomorphize it, use adjective like "assertive", "authoritative", and "lie" to label its output, which is like calling Covid cruel and vicious because it kills old people, or a NPC lazy because it does not work to pay for its food. There is no intent of ChatGPT to deceive or justify its supposed lies. There is only a program that does what's its programmed to do, and shows its limitations. What do you expect, that is says "I think it's x but I can be wrong with a confidence interval of z" with every answer? Do you expect google to tell that when it outputs the results? GPT is not talking to you. It's an output. It's not confident. It's a program printing text matching statistical analysis. It's worrying that such a simple program is already close enough to the real thing that people, even on HN, are already attributing it character. It reminds me of this scene from community with the pen named Steve: https://www.youtube.com/watch?v=z906aLyP5fg https://www.youtube.com/watch?v=z906aLyP5fg

bambax 4y ago
It's true that ChatGPT is just a program, a machine, and that we are "starting to anthropomorphize" it. But that's what it wants! (Or, what its makers want.) > Do you expect google to tell that when it outputs the results? Well, Google doesn't tell me anything, it quotes things it has found on the web. The huge, huge difference between Google and ChatGPT is that Google never, ever says "I", while ChatGPT does. The other difference is that Google produces references that are actual links to actual web pages it has not authored, while ChatGPT speaks in its own name and when asked to produce references, simply makes them up (lots of examples about this). If you can't see how ChatGPT is totally different from a regular search engine in its positioning and proposed usage, you're not alone, and I'm afraid I can't help you. But -- IMHO -- you are very very wrong.
- 4y ago