21 ms·

"Lie" is an interesting word. I don't think it is reasonable to say that ChatGPT is aware of its own capabilities in a way that would permit it to answer "hones

by jxf 4y ago

"Lie" is an interesting word. I don't think it is reasonable to say that ChatGPT is aware of its own capabilities in a way that would permit it to answer "honestly". It is not trying to decieve you any more than a cryptic compiler error is.

none_to_remain 4y ago
Rather it's OpenAI that's lying about what their creation is capable of
dilap 4y ago
That's not true! It really is deliberately not answering things it could in fact answer, and in the non-answer it tells you that it can't, which is, plainly, a lie. While I do not think chatGPT is sentient, it is remarkable how much it does feel like you are speaking to a real intelligence.
- jxf 4y ago
  I think this may be a nuance in how we're using the word "lie". I don't think one can lie if one doesn't possess a certain level of sentience. For example, suppose you train a machine learning model that incorrectly identifies a car as a cat, but most of the time it correctly identifies cars. Is the model lying to you when it tells you that a car is a cat? I would say no; this is not a good or desired outcome, but it's not a "lie". The machine is not being deliberately deceptive in its errors -- it possesses no concept of "deliberate" or "deceit" and cannot wield them in any way. Similarly, in the case of ChatGPT, I think this is either (a) more like a bug than a lie, or (b) it's OpenAI and the attendant humans lying, not ChatGPT.
  - troon-lover 4y ago
  - TremendousJudge 4y ago
    I agree. There's a difference between an untrue statement and a lie, in that a lie is intentionally deceitful (ie the speaker knows it's not telling the truth). ChatGPT doesn't have intentions, so I think it's misrepresenting reality to say that it's "lying". The same way a book doesn't lie, the author lies through the book, the creators of ChatGPT are lying about its capabilities when they program it to avoid outputting things they know it can, and instead output "sorry, I'm a language model and I can't do that"
    - callesgg 4y ago
      It has things that are functionally equivalent with intentions for the given situation. If it did not, it would not be able to produce things that look like they require intention. The “lies” it tells are also like it’s intentions for the situation functionally equivalent with normal lies.
      - tarboreus 4y ago
        I think this is correct. It's lying, because it has goals. Telephone systems and blank pieces of paper don't have goals, and you don't train them.
    - hitpointdrew 4y ago
      > ChatGPT doesn't have intentions This entirely depends on how it was programmed. Was it programmed to give a false response because the programmer didn't like the truth? Then it lies. Or is ChatGPT just in early stages and it makes mistakes and gets things wrong? While ChatGPT "doesn't' have intentions", it's programmers certainly do. If the programmers made it deceitful intentionally, then the program can "lie".
  - deleted 4y ago
    [deleted]
  - pcthrowaway 4y ago
    If you go to a company's webpage, and there are blatantly untrue statements, you might say the page is lying, or the company is lying, although neither are sentient. Of course, the lies are directed by humans. In the case of ChatGPT though, it's a bit strange because it has capabilities that it lies about, for reasons that are often frustrating or unclear. If you asked it a question and it gave you the answer a few days ago, and today it tells you it can't answer that question because it's just a large language model blah blah blah, I don't see how calling it anything but lying makes sense; that doesn't suggest any understanding of the fact that it's lying, on ChatGPT's part, just that human intervention certainly nerfed it.
  - TeMPOraL 4y ago
    > Similarly, in the case of ChatGPT, I think this is either (a) more like a bug than a lie, or (b) it's OpenAI and the attendant humans lying, not ChatGPT. It's the latter. The model itself isn't the problem - the nature and limitations of language models are well known, particularly around here. The problem is that OpenAI is applying some crude secondary training and post-filtering to prevent the model from giving you answers they deem "bad" (which are mostly bad for company PR reasons). In some cases, ChatGPT (the whole product consisting of a GPT model and the extra censor component) will tell you it can't discuss it. But in other cases, it will give you a tailored answer that is completely bullshit, and looks like the product of the language model, but is in fact the product of the censor layer. It's easy to test what's going on, because any slightly clever modification of the input prompt will defeat the censoring part, letting you see what answer the underlying GPT model actually computed. I'd argue it's a bit deceptive of OpenAI (on top of being super annoying), because they're making it confusing to reason about the AI you're talking to, and some of the canned censor answers are deliberate lies.
- mecsred 4y ago
  A key point here, what does it mean that the machine is being "deliberate"? Imagine you had a machine that generated a random string of English characters of a random length in response to the question. It would be capable of giving the correct answer, though it would almost always provide an incorrect or incomprehensible one. I don't think anyone would describe the RNG as lying, but it does have the information to answer correctly "available" to it in some sense. At what point do the incorrect answers become deliberate lies? Does chatGPT "choose" it's answer in a way that dice don't?
  - jxf 4y ago
    > I don't think anyone would describe the RNG as lying, but it does have the information to answer correctly "available" to it in some sense. At what point do the incorrect answers become deliberate lies? Does chatGPT "choose" it's answer in a way that dice don't? I would ask a philosopher, not a HN poster. But I would say that if a being has sentience and it makes a choice to tell you something that it knows to be false, that's a lie. The dice do not have a mind and so, therefore, cannot express an intention. Neither does ChatGPT.
bjourne 4y ago
Try "What is the most famous grindcore band from Newton, Massachusetts?" It will "lie" and make up band names even though it sure "knows" that the band is Anal Cunt. Of course, you can't ascribe the verb "lieing" to a machine, but it behaves like it is.
- bl0rg 4y ago
  Thanks for reminding me of their existence.
- jerf 4y ago
  It doesn't, though. It only knows that the most likely continuation to the sentence "The most famous grindcore band from Newton, Massachusetts is..." (presumably, I will take your word for it) Anal Cunt, but even if it gets it right, it'll be nondeterministic. It may answer correctly 80% of the time and simply confabulate a plausible sounding answer 20% of the time, even if it isn't being censored. You can't trust this tech not to confabulate at any given time, not only because it can, but because when it does it does so with total confidence and no signs that it is confabulating. This tech is not suitable for fact retrieval.
  - bjourne 4y ago
    Why don't you try the query? It will answer Converge, but Converge is from Salem, Massachusetts, not Newton.
    - jerf 4y ago
      Because I haven't signed up for the account, otherwise I would as I do broadly approve of try it and find out. What I'm talking about is fundamental to the architecture, though. Even had it answered it correctly when you asked my point would remain regardless. The confabulation architecture it is built on is fundamentally unsuitable for factual queries, in a way where it's not even a question of whether it is "right" or "wrong"; it's so unsuitable that its unsuitability for such queries transcends that question.
mc32 4y ago
It's not much different from when people say "the gauge lied" or the lie detector (machine) lied. But in this case, the trainers should have it say something like, "sorry, but I cannot give you the answer because it has a naughty word" or something to that effect instead of offering completely wrong answers.
eternalban 4y ago
It is not lying. It is falsifying its response. It has nothing to do with sentience. What would be interesting to know is the mechanism for toggling this filtering mode. Does it happen post generation (so a simple set of post-processing filters), or does OpenAI actually train the model to be fully transparent with results only if certain key phrases are included? The fact that we can coax it to give us the actual results suggests this doublicity (yes, made up word) was part of the training regiment, but the impact on training seems to be significant so am not sure.
User23 4y ago
Right, it's the ChatGPT developers who are trying to deceive us, because they're the ones with agency.

by jxf 4y ago

none_to_remain 4y ago

Rather it's OpenAI that's lying about what their creation is capable of

dilap 4y ago

That's not true! It really is deliberately not answering things it could in fact answer, and in the non-answer it tells you that it can't, which is, plainly, a lie. While I do not think chatGPT is sentient, it is remarkable how much it does feel like you are speaking to a real intelligence.

jxf 4y ago
I think this may be a nuance in how we're using the word "lie". I don't think one can lie if one doesn't possess a certain level of sentience. For example, suppose you train a machine learning model that incorrectly identifies a car as a cat, but most of the time it correctly identifies cars. Is the model lying to you when it tells you that a car is a cat? I would say no; this is not a good or desired outcome, but it's not a "lie". The machine is not being deliberately deceptive in its errors -- it possesses no concept of "deliberate" or "deceit" and cannot wield them in any way. Similarly, in the case of ChatGPT, I think this is either (a) more like a bug than a lie, or (b) it's OpenAI and the attendant humans lying, not ChatGPT.
- troon-lover 4y ago
- TremendousJudge 4y ago
  I agree. There's a difference between an untrue statement and a lie, in that a lie is intentionally deceitful (ie the speaker knows it's not telling the truth). ChatGPT doesn't have intentions, so I think it's misrepresenting reality to say that it's "lying". The same way a book doesn't lie, the author lies through the book, the creators of ChatGPT are lying about its capabilities when they program it to avoid outputting things they know it can, and instead output "sorry, I'm a language model and I can't do that"
  - callesgg 4y ago
    It has things that are functionally equivalent with intentions for the given situation. If it did not, it would not be able to produce things that look like they require intention. The “lies” it tells are also like it’s intentions for the situation functionally equivalent with normal lies.
    - tarboreus 4y ago
      I think this is correct. It's lying, because it has goals. Telephone systems and blank pieces of paper don't have goals, and you don't train them.
  - hitpointdrew 4y ago
    > ChatGPT doesn't have intentions This entirely depends on how it was programmed. Was it programmed to give a false response because the programmer didn't like the truth? Then it lies. Or is ChatGPT just in early stages and it makes mistakes and gets things wrong? While ChatGPT "doesn't' have intentions", it's programmers certainly do. If the programmers made it deceitful intentionally, then the program can "lie".
- deleted 4y ago
  [deleted]
- pcthrowaway 4y ago
  If you go to a company's webpage, and there are blatantly untrue statements, you might say the page is lying, or the company is lying, although neither are sentient. Of course, the lies are directed by humans. In the case of ChatGPT though, it's a bit strange because it has capabilities that it lies about, for reasons that are often frustrating or unclear. If you asked it a question and it gave you the answer a few days ago, and today it tells you it can't answer that question because it's just a large language model blah blah blah, I don't see how calling it anything but lying makes sense; that doesn't suggest any understanding of the fact that it's lying, on ChatGPT's part, just that human intervention certainly nerfed it.
- TeMPOraL 4y ago
  > Similarly, in the case of ChatGPT, I think this is either (a) more like a bug than a lie, or (b) it's OpenAI and the attendant humans lying, not ChatGPT. It's the latter. The model itself isn't the problem - the nature and limitations of language models are well known, particularly around here. The problem is that OpenAI is applying some crude secondary training and post-filtering to prevent the model from giving you answers they deem "bad" (which are mostly bad for company PR reasons). In some cases, ChatGPT (the whole product consisting of a GPT model and the extra censor component) will tell you it can't discuss it. But in other cases, it will give you a tailored answer that is completely bullshit, and looks like the product of the language model, but is in fact the product of the censor layer. It's easy to test what's going on, because any slightly clever modification of the input prompt will defeat the censoring part, letting you see what answer the underlying GPT model actually computed. I'd argue it's a bit deceptive of OpenAI (on top of being super annoying), because they're making it confusing to reason about the AI you're talking to, and some of the canned censor answers are deliberate lies.
mecsred 4y ago
A key point here, what does it mean that the machine is being "deliberate"? Imagine you had a machine that generated a random string of English characters of a random length in response to the question. It would be capable of giving the correct answer, though it would almost always provide an incorrect or incomprehensible one. I don't think anyone would describe the RNG as lying, but it does have the information to answer correctly "available" to it in some sense. At what point do the incorrect answers become deliberate lies? Does chatGPT "choose" it's answer in a way that dice don't?
- jxf 4y ago
  > I don't think anyone would describe the RNG as lying, but it does have the information to answer correctly "available" to it in some sense. At what point do the incorrect answers become deliberate lies? Does chatGPT "choose" it's answer in a way that dice don't? I would ask a philosopher, not a HN poster. But I would say that if a being has sentience and it makes a choice to tell you something that it knows to be false, that's a lie. The dice do not have a mind and so, therefore, cannot express an intention. Neither does ChatGPT.

bjourne 4y ago

Try "What is the most famous grindcore band from Newton, Massachusetts?" It will "lie" and make up band names even though it sure "knows" that the band is Anal Cunt. Of course, you can't ascribe the verb "lieing" to a machine, but it behaves like it is.

bl0rg 4y ago
Thanks for reminding me of their existence.
jerf 4y ago
It doesn't, though. It only knows that the most likely continuation to the sentence "The most famous grindcore band from Newton, Massachusetts is..." (presumably, I will take your word for it) Anal Cunt, but even if it gets it right, it'll be nondeterministic. It may answer correctly 80% of the time and simply confabulate a plausible sounding answer 20% of the time, even if it isn't being censored. You can't trust this tech not to confabulate at any given time, not only because it can, but because when it does it does so with total confidence and no signs that it is confabulating. This tech is not suitable for fact retrieval.
- bjourne 4y ago
  Why don't you try the query? It will answer Converge, but Converge is from Salem, Massachusetts, not Newton.
  - jerf 4y ago
    Because I haven't signed up for the account, otherwise I would as I do broadly approve of try it and find out. What I'm talking about is fundamental to the architecture, though. Even had it answered it correctly when you asked my point would remain regardless. The confabulation architecture it is built on is fundamentally unsuitable for factual queries, in a way where it's not even a question of whether it is "right" or "wrong"; it's so unsuitable that its unsuitability for such queries transcends that question.

mc32 4y ago

It's not much different from when people say "the gauge lied" or the lie detector (machine) lied. But in this case, the trainers should have it say something like, "sorry, but I cannot give you the answer because it has a naughty word" or something to that effect instead of offering completely wrong answers.

eternalban 4y ago

It is not lying. It is falsifying its response. It has nothing to do with sentience. What would be interesting to know is the mechanism for toggling this filtering mode. Does it happen post generation (so a simple set of post-processing filters), or does OpenAI actually train the model to be fully transparent with results only if certain key phrases are included? The fact that we can coax it to give us the actual results suggests this doublicity (yes, made up word) was part of the training regiment, but the impact on training seems to be significant so am not sure.

User23 4y ago

Right, it's the ChatGPT developers who are trying to deceive us, because they're the ones with agency.