4 ms·

Of course, my technical friends very quickly found the edge cases, getting it to contradict itself, etc. OK, I'm a technical person but I asked the chatbot in

Of course, my technical friends very quickly found the edge cases, getting it to contradict itself, etc. OK, I'm a technical person but I asked the chatbot in the article broad questions that were difficult but not "tricky" ("what's a good race to play for a Druid in DnD", "Compare Kerouac's On The Road to his Desolation Angels" and got a reasonable summary of search plus answers that were straight-up false). Maybe your "nontechnical" friend weren't able to notice that the thing's output of misinformation but seems like more of a problem, not less. Also, ChatGPT in particular seems to go to pains to say it's not conscious and that's actually a good thing. These chatBots can be useful search summarizers making their limits clear (like github navigator). They're noxious if they instill a delusion of their consciousness in people and I don't think you should be so happy about fooling your friends. Every new technology has initially had cases where people could be deluded into think it was magic but those instances can't be taken as proof of that magic or as bragging rights.

nl 4y ago
Yes, this "truthfulness" problem is the real problem with all these "generative search" products. Forget nudes in generated images. This is the real ethics issue!
- gamegoblin 4y ago
  You can somewhat detect BS by getting the model to also output the log-probability of its token selection. See https://twitter.com/goodside/status/1581151528949141504 https://twitter.com/goodside/status/1581151528949141504 for examples.
  - nl 4y ago
    I don't think that's going to work. Probability measure for "Trump is the present President of the US" is likely the very high. It's still untrue.
    - andai 4y ago
      GPT-3 training data cuts off in October 2019. Not sure if they updated it since last year.
      - freediver 4y ago
        Updating it doesn't make this kind of problem away, unless they figure out a way to have real time updates to the model (could happen)
        Breza 4y ago
        You wouldn't even need the model to be trained in real time. I'd love to see OpenAI buy Wolfram Research. WolframAlpha has managed to integrate tons of external data into a natural language interface. ChatGPT already knows when to insert placeholders, such as "$XX.XX" or "[city name]" when it doesn't know a specific bit of information. Combining the two could be very powerful. You could have data that's far more current than what's possible by retraining a large model.
motoxpro 4y ago
You're missing that a large number of people don't go into it "trying to break it"
- joe_the_user 4y ago
  I didn't go into it trying to break or trick it. The only thing tricky about the questions I asked was that I knew the answer to them. I don't think it's necessary dumber than the first page of a Google search but it's certainly not more well informed than that. But it certainly seems smart, which is actually a bit problematic.
  - oefrha 4y ago
    It’s actually not that different from chatting to the know-it-all type of Internet rando: they can talk to you about anything and seem knowledgeable on all of them, but go into a topic you actually know about and you realize they’re just making shit up or regurgitating myths they read somewhere. You can find that kind of user on HN.
  - andai 4y ago
    Yeah this is my main concern about GPT-3, there's no truth-fiction slider, and it will often slip complete fabrications into the output, making it dangerous to rely on for real world information. Which is really a shame, because it does actually give great output most of the time.
    - dingclancy 4y ago
      I have never seen a human made website with a truth-fiction slider. The answers can be straight up false and scary, but it is no different from other publications out there. Even with the most credible news sources, it is still up to the person reading it to sense the BS.
    - deadpannini 4y ago
      Why is this a special concern about GPT-3? I cannot think of an institution, entity, or tool about which those statements are not true. Replace "GPT-3" with "Hacker News posters" "Wikipedia", or "News broadcasts" to create three more 100%-accurate paragraphs.

nl 4y ago

Yes, this "truthfulness" problem is the real problem with all these "generative search" products. Forget nudes in generated images. This is the real ethics issue!

gamegoblin 4y ago
You can somewhat detect BS by getting the model to also output the log-probability of its token selection. See https://twitter.com/goodside/status/1581151528949141504 https://twitter.com/goodside/status/1581151528949141504 for examples.
- nl 4y ago
  I don't think that's going to work. Probability measure for "Trump is the present President of the US" is likely the very high. It's still untrue.
  - andai 4y ago
    GPT-3 training data cuts off in October 2019. Not sure if they updated it since last year.
    - freediver 4y ago
      Updating it doesn't make this kind of problem away, unless they figure out a way to have real time updates to the model (could happen)
      - Breza 4y ago
        You wouldn't even need the model to be trained in real time. I'd love to see OpenAI buy Wolfram Research. WolframAlpha has managed to integrate tons of external data into a natural language interface. ChatGPT already knows when to insert placeholders, such as "$XX.XX" or "[city name]" when it doesn't know a specific bit of information. Combining the two could be very powerful. You could have data that's far more current than what's possible by retraining a large model.

motoxpro 4y ago

You're missing that a large number of people don't go into it "trying to break it"

joe_the_user 4y ago
I didn't go into it trying to break or trick it. The only thing tricky about the questions I asked was that I knew the answer to them. I don't think it's necessary dumber than the first page of a Google search but it's certainly not more well informed than that. But it certainly seems smart, which is actually a bit problematic.
- oefrha 4y ago
  It’s actually not that different from chatting to the know-it-all type of Internet rando: they can talk to you about anything and seem knowledgeable on all of them, but go into a topic you actually know about and you realize they’re just making shit up or regurgitating myths they read somewhere. You can find that kind of user on HN.
- andai 4y ago
  Yeah this is my main concern about GPT-3, there's no truth-fiction slider, and it will often slip complete fabrications into the output, making it dangerous to rely on for real world information. Which is really a shame, because it does actually give great output most of the time.
  - dingclancy 4y ago
    I have never seen a human made website with a truth-fiction slider. The answers can be straight up false and scary, but it is no different from other publications out there. Even with the most credible news sources, it is still up to the person reading it to sense the BS.
  - deadpannini 4y ago
    Why is this a special concern about GPT-3? I cannot think of an institution, entity, or tool about which those statements are not true. Replace "GPT-3" with "Hacker News posters" "Wikipedia", or "News broadcasts" to create three more 100%-accurate paragraphs.