6 ms·

Exactly, i talked to ChatGPT and it gave me a lot of wrong information in an authorative tone. I consider it dangerous as-is.

by Tepix 4y ago

bentt 4y ago
It's only dangerous if you consider it authoritative. Informative and authoritative are different. It can expose you to terms you've never heard which you can then do further research on. This alone has been valuable for me so far.
adamsmith143 4y ago
Turns out humans do this all the time and they actually have real power.
- Tepix 4y ago
  Yes, just look at the twitter thread, most people are not even noticing that the answers are wrong.
- jerf 4y ago
  Yes they do, and I do not deny the power of human's ability to confidently spew nonsense. However, humans do have some known failure cases that help us detect that. For instance, pressing the human on a couple of details will generally show up all but the very best bullshit artists; there is a limit to how fast humans can make crap up. Some of us are decent at the con-game aspects but it isn't too hard to poke through this limit on how fast they can make stuff up. Computers can confabulate at full speed for gigabytes at a time. Personally, I consider any GPT or GPT-like technology unsuitable for any application in which truth is important. Full stop. The technology fundamentally, in its foundation, does not have any concept of truth, and there is no obvious way to add one, either after the fact or in its foundation. (Not saying there isn't one, period, but it certainly isn't the sort of thing you can just throw a couple of interns at and get a good start on.) "The statistically-most likely conclusion of this sentence" isn't even a poor approximation of truth... it's just plain unrelated. That is not what truth is. At least not with any currently even remotely feasible definition of "statistically most likely" converted into math sufficient to be implementable. And I don't even mean "truth" from a metaphysical point of view; I mean it in a more engineering sense. I wouldn't set one of these up to do my customer support either. AI Dungeon is about the epitome of the technology, in my opinion, and generalized entertainment from playing with a good text mangler. It really isn't good for much else.
  - visarga 4y ago
    > Personally, I consider any GPT or GPT like technology unsuitable for any application in which truth is important . Full stop. The technology fundamentally, in its foundation, does not have any concept of truth I think you got it all wrong. Not all GPT-3 tasks are "closed-book". If you can fit in the context a piece of information, then GPT-3 will take it into consideration. That means you can do a search, get the documents into the prompt, and then ask your questions. It will reference the text and give you grounded answers. Of course you still need to vet the sources of information you use, if you give it false information into the context, it will give wrong answers.
    - jerf 4y ago
      I don't think you're right. Even if you add "correct" context, and in many of these cases "I can locate correct context" already means the GPT-tech isn't adding much, GPT still as absolutely no guard rails stopping it from confabulating. It might confabulate something else, but it still confabulating. Fundamentally, GPT is a technology for building convincing confabulations, and we hope that if we keep pounding on it and making it bigger we can get those confabulations to converge on reality. I do not mean this as an insult, I mean it as a reasonable description of the underlying technology. This is, fundamentally, not a sane way to build most of the systems I see people trying to build with it. AI Dungeon is a good use because the whole point of AI Dungeon is to confabulate at scale. This works with the strengths of GPT-like tech (technically, "transformer-based tech" is probably a closer term but nobody knows what that is).
    - mannykannot 4y ago
      This hangs on what it means to "take it into consideration." If you gave me new information, I would attempt to see it in context, evaluate its relevance, and either update my positions accordingly or explain why I do not see it making a difference. If I saw difficulties doing this, I would ask for clarification, explaining what it was that was seemed difficult or unclear. As far as I can tell, there is no reason to think that the way GPT-3 generates its responses could possibly result in this happening - even the basic ability of correctly inferring corollaries from a collection of facts seems beyond what those methods could deliver, except insofar as the syntax of their expression matches common patterns in the corpus of human language use. And the empirical results so far, while being impressive and thought-provoking in many ways, support this skepticism.
- majormajor 4y ago
  So if you're suggestion is that it's ok if computers are as unreliable as humans, what's the point of computers, then?
  - rafaelero 4y ago
    The point is that they get better and they don't need to be perfect.
    - majormajor 4y ago
      Nobody's shown a way yet to teach a computer how to tell bullshit from facts and filter out the bullshit in it's regurgitation/hallucination text creation stuff. So until that happens, all you've done is let people put bullshit-spewing humans in more places. People already know not to necessarily trust humans, now they'll (re)learn that about computer generated text. (It's actually probably not clear to everyone what's computer-generated text and human-generated text, so more likely, specific places that rely on this will just be seen as untrustworthy. "Create more untrustworthy sources of text" is... underwhelming, honestly.)
      - rafaelero 4y ago
        > Nobody's shown a way yet to teach a computer how to tell bullshit from facts and filter out the bullshit in it's regurgitation/hallucination text creation stuff. And yet they keep improving at every iteration. Also, keep in mind that this objection will exist even if these AI get near omniscience. People disagree with facts all the time, usually for political motives. Therefore your type of criticism won't ever be settled.
        adamsmith143 4y ago
        I've said this before but these people are going to be shouting that 'the AI doesn't really understand the world' right up until the moment a nanobot swam dissolves them into goop for processing.
MollyRealized 4y ago
Actually, the name of the entity is ChatGTP. It is stands for General Translation Protocol, referencing translation from the AI code and source information into a more generally understandable English language. (joking here)
- MollyRealized 4y ago
  A little annoyed that no one picked up on the fact that I was riffing on "it gave me a lot of wrong information in an authorative tone".
seydor 4y ago
Ah, they 've been Galactica'ed already
rafaelero 4y ago
- Spivak 4y ago
  I mean it's not like it's dangerous on its own, but if you're like "Hey GPT how do I put out a grease fire?" and it replies "Pour water on it" and you believe it then you're in for a bad time. So I mean I guess you're technically right, it's not dangerous so long as you have 0% confidence in anything it says and consider it entertainment. But what would-be scrappy Google competitor is gonna do that? The thing that makes it particularly insidious is that it's going to be right a lot, but being right means nothing when there's nothing to go off of to figure out what case you're in. If you actually had no idea when the Berlin Wall fell and it spit out 1987 how would you disprove it? Probably go ask a search engine.
  - rafaelero 4y ago
    I don't see the danger you are afraid of. The same artifacts you are proposing (skepticism, verification) should already be put in place with any pubic expert.
    - macintux 4y ago
      Humans will generally either provide a confidence level in their answers, or if they’re consistently wrong, you’ll learn to disregard them. If a computer is right every time you’ve asked a question, then gives you the wrong answer in an emergency like a grease fire, it’s hard to have a defense against that. If you were asking your best friend, you’d have some sense of how accurate they tend to be, and they’d probably say something like “if I remember correctly” or “I think” so you’ll have a warning that they could easily be wrong.
      - rafaelero 4y ago
        If the AI is correct 90% of the time, you can be reasonably sure it will be correct next time. That's a rational expectation. If you are at a high stake situation, then even a 1% rate of false positive is too high and you should definitely apply some verifications. Again, I don't see the danger.
uoaei 4y ago
It would fit right in here on HN.
- WandaVision 4y ago
  No. Unlike ChatGPT HN has error correcting system build in it. Like this comment.
  - Tepix 4y ago
    Well you can tell ChatGPT that it made a mistake and it will acknowledge it but only poorly correct itself in my case.
neonate 4y ago
What wrong information did it give you?
- knorker 4y ago
  Not parent commenter, but it told me 1093575151355318117 is not prime, but the product of 3, 5, 7, 11, 13, 17, 19, 23, 29, 31, 37, 41, 43, 47, 53, 59, 61, 67, 71, 73, 79, 83, 89, 97, and 101. But 116431182179248680450031658440253681535 is not 1093575151355318117. There are some other math problems where it will confidently do step by step and give you nonsense. Edit: and see https://news.ycombinator.com/item?id=33818443 https://news.ycombinator.com/item?id=33818443 and https://twitter.com/dgbrazales/status/1598265067086442496 https://twitter.com/dgbrazales/status/1598265067086442496
- Tepix 4y ago
  Just one example, talking about the board game Othello it gave some completely bogus facts about the board.
healthapiguy 4y ago
Google also gives plenty of wrong information
- baandam 4y ago

Exactly, i talked to ChatGPT and it gave me a lot of wrong information in an authorative tone. I consider it dangerous as-is.

by Tepix 4y ago

Exactly, i talked to ChatGPT and it gave me a lot of wrong information in an authorative tone. I consider it dangerous as-is.

bentt 4y ago

It's only dangerous if you consider it authoritative. Informative and authoritative are different. It can expose you to terms you've never heard which you can then do further research on. This alone has been valuable for me so far.

adamsmith143 4y ago

Turns out humans do this all the time and they actually have real power.

Tepix 4y ago
Yes, just look at the twitter thread, most people are not even noticing that the answers are wrong.
jerf 4y ago
Yes they do, and I do not deny the power of human's ability to confidently spew nonsense. However, humans do have some known failure cases that help us detect that. For instance, pressing the human on a couple of details will generally show up all but the very best bullshit artists; there is a limit to how fast humans can make crap up. Some of us are decent at the con-game aspects but it isn't too hard to poke through this limit on how fast they can make stuff up. Computers can confabulate at full speed for gigabytes at a time. Personally, I consider any GPT or GPT-like technology unsuitable for any application in which truth is important. Full stop. The technology fundamentally, in its foundation, does not have any concept of truth, and there is no obvious way to add one, either after the fact or in its foundation. (Not saying there isn't one, period, but it certainly isn't the sort of thing you can just throw a couple of interns at and get a good start on.) "The statistically-most likely conclusion of this sentence" isn't even a poor approximation of truth... it's just plain unrelated. That is not what truth is. At least not with any currently even remotely feasible definition of "statistically most likely" converted into math sufficient to be implementable. And I don't even mean "truth" from a metaphysical point of view; I mean it in a more engineering sense. I wouldn't set one of these up to do my customer support either. AI Dungeon is about the epitome of the technology, in my opinion, and generalized entertainment from playing with a good text mangler. It really isn't good for much else.
- visarga 4y ago
  > Personally, I consider any GPT or GPT like technology unsuitable for any application in which truth is important . Full stop. The technology fundamentally, in its foundation, does not have any concept of truth I think you got it all wrong. Not all GPT-3 tasks are "closed-book". If you can fit in the context a piece of information, then GPT-3 will take it into consideration. That means you can do a search, get the documents into the prompt, and then ask your questions. It will reference the text and give you grounded answers. Of course you still need to vet the sources of information you use, if you give it false information into the context, it will give wrong answers.
  - jerf 4y ago
    I don't think you're right. Even if you add "correct" context, and in many of these cases "I can locate correct context" already means the GPT-tech isn't adding much, GPT still as absolutely no guard rails stopping it from confabulating. It might confabulate something else, but it still confabulating. Fundamentally, GPT is a technology for building convincing confabulations, and we hope that if we keep pounding on it and making it bigger we can get those confabulations to converge on reality. I do not mean this as an insult, I mean it as a reasonable description of the underlying technology. This is, fundamentally, not a sane way to build most of the systems I see people trying to build with it. AI Dungeon is a good use because the whole point of AI Dungeon is to confabulate at scale. This works with the strengths of GPT-like tech (technically, "transformer-based tech" is probably a closer term but nobody knows what that is).
  - mannykannot 4y ago
    This hangs on what it means to "take it into consideration." If you gave me new information, I would attempt to see it in context, evaluate its relevance, and either update my positions accordingly or explain why I do not see it making a difference. If I saw difficulties doing this, I would ask for clarification, explaining what it was that was seemed difficult or unclear. As far as I can tell, there is no reason to think that the way GPT-3 generates its responses could possibly result in this happening - even the basic ability of correctly inferring corollaries from a collection of facts seems beyond what those methods could deliver, except insofar as the syntax of their expression matches common patterns in the corpus of human language use. And the empirical results so far, while being impressive and thought-provoking in many ways, support this skepticism.
majormajor 4y ago
So if you're suggestion is that it's ok if computers are as unreliable as humans, what's the point of computers, then?
- rafaelero 4y ago
  The point is that they get better and they don't need to be perfect.
  - majormajor 4y ago
    Nobody's shown a way yet to teach a computer how to tell bullshit from facts and filter out the bullshit in it's regurgitation/hallucination text creation stuff. So until that happens, all you've done is let people put bullshit-spewing humans in more places. People already know not to necessarily trust humans, now they'll (re)learn that about computer generated text. (It's actually probably not clear to everyone what's computer-generated text and human-generated text, so more likely, specific places that rely on this will just be seen as untrustworthy. "Create more untrustworthy sources of text" is... underwhelming, honestly.)
    - rafaelero 4y ago
      > Nobody's shown a way yet to teach a computer how to tell bullshit from facts and filter out the bullshit in it's regurgitation/hallucination text creation stuff. And yet they keep improving at every iteration. Also, keep in mind that this objection will exist even if these AI get near omniscience. People disagree with facts all the time, usually for political motives. Therefore your type of criticism won't ever be settled.
      - adamsmith143 4y ago
        I've said this before but these people are going to be shouting that 'the AI doesn't really understand the world' right up until the moment a nanobot swam dissolves them into goop for processing.

MollyRealized 4y ago

Actually, the name of the entity is ChatGTP. It is stands for General Translation Protocol, referencing translation from the AI code and source information into a more generally understandable English language. (joking here)

MollyRealized 4y ago
A little annoyed that no one picked up on the fact that I was riffing on "it gave me a lot of wrong information in an authorative tone".

seydor 4y ago

Ah, they 've been Galactica'ed already

rafaelero 4y ago

Spivak 4y ago
I mean it's not like it's dangerous on its own, but if you're like "Hey GPT how do I put out a grease fire?" and it replies "Pour water on it" and you believe it then you're in for a bad time. So I mean I guess you're technically right, it's not dangerous so long as you have 0% confidence in anything it says and consider it entertainment. But what would-be scrappy Google competitor is gonna do that? The thing that makes it particularly insidious is that it's going to be right a lot, but being right means nothing when there's nothing to go off of to figure out what case you're in. If you actually had no idea when the Berlin Wall fell and it spit out 1987 how would you disprove it? Probably go ask a search engine.
- rafaelero 4y ago
  I don't see the danger you are afraid of. The same artifacts you are proposing (skepticism, verification) should already be put in place with any pubic expert.
  - macintux 4y ago
    Humans will generally either provide a confidence level in their answers, or if they’re consistently wrong, you’ll learn to disregard them. If a computer is right every time you’ve asked a question, then gives you the wrong answer in an emergency like a grease fire, it’s hard to have a defense against that. If you were asking your best friend, you’d have some sense of how accurate they tend to be, and they’d probably say something like “if I remember correctly” or “I think” so you’ll have a warning that they could easily be wrong.
    - rafaelero 4y ago
      If the AI is correct 90% of the time, you can be reasonably sure it will be correct next time. That's a rational expectation. If you are at a high stake situation, then even a 1% rate of false positive is too high and you should definitely apply some verifications. Again, I don't see the danger.

uoaei 4y ago

It would fit right in here on HN.

WandaVision 4y ago
No. Unlike ChatGPT HN has error correcting system build in it. Like this comment.
- Tepix 4y ago
  Well you can tell ChatGPT that it made a mistake and it will acknowledge it but only poorly correct itself in my case.

neonate 4y ago

What wrong information did it give you?

knorker 4y ago
Not parent commenter, but it told me 1093575151355318117 is not prime, but the product of 3, 5, 7, 11, 13, 17, 19, 23, 29, 31, 37, 41, 43, 47, 53, 59, 61, 67, 71, 73, 79, 83, 89, 97, and 101. But 116431182179248680450031658440253681535 is not 1093575151355318117. There are some other math problems where it will confidently do step by step and give you nonsense. Edit: and see https://news.ycombinator.com/item?id=33818443 https://news.ycombinator.com/item?id=33818443 and https://twitter.com/dgbrazales/status/1598265067086442496 https://twitter.com/dgbrazales/status/1598265067086442496
Tepix 4y ago
Just one example, talking about the board game Othello it gave some completely bogus facts about the board.

healthapiguy 4y ago

Google also gives plenty of wrong information

baandam 4y ago