4 ms·

I wouldn't even mind so much if the answers were right. The problem is that a lot of them are totally wrong, but completely reasonable- and plausible-sounding,

I wouldn't even mind so much if the answers were right. The problem is that a lot of them are totally wrong, but completely reasonable- and plausible-sounding, and in an authoritative tone, so unless you already know the right answer, the only way you'll realize its answer is wrong is the hard way.

gtirloni 4y ago
The StackOverflow mods have a lot of knowledge about closing questions based on really small details that go against their rules. I'm sure they will do well spotting AI-generated answers.
- luckylion 4y ago
  I doubt it. They've had problems with cheap automated answers for years where bots would essentially search for the question on SO and then copy an answer from another question verbatim. The answers were rarely useful because questions happen to be different even though they use the same keywords. Not only did they never bother to block that, they also didn't mind it and wanted to rely on the community down-voting those answers instead of at least blocking the bot -- and that's with a trivial check (identical answer already in DB). With something that's AI-generated, there's no chance. And with the general quality of many of the answers, there's no way to tell apart wrong answers from AI or humans.
ravenstine 4y ago
What you said may sum up the current state of AI. People's minds are continually being blown, but will there be a realization that these AIs are specialized to provide specious output and nothing else? There's a canyon of difference between something that sounds correct and a thing that is actually correct.
- bambax 4y ago
  Exactly. ChatGPT sounds like a bad student who didn't actually learn anything during the year and is trying to bullshit their way through the finals. Or a politician, maybe. It's formalizing the worst traits of humanity.
- bediger4000 4y ago
  Yes. I spent yesterday afternoon trying to get ChatGPT to write me a quine in nroff/troff macros, something I've not been able to do, or find if anyone has done. The generated quines look like they'd work, but don't. Same with an M4 macro processor quine - looks maybe correct, doesn't work. It did generate a Go quine.
  - YeGoblynQueenne 4y ago
    >> It did generate a Go quine. Because there are several of them on the web: https://duckduckgo.com/?t=ffcm&q=Go+quine&atb=v344-1&ia=web https://duckduckgo.com/?t=ffcm&q=Go+quine&atb=v344-1&ia=web
bambax 4y ago
That's the worst part. It's all fun and games when it tells a story of cheese sandwiches and VCRs in the style of the King James bible, but when it gives wrong answers in an authoritative tone and then insists it absolutely can't be wrong, it's terrifying. I don't understand what good could come of this. Or at least make it detect what is fiction and what isn't. People are treating it like it's Wikipedia, but it's not. It's a riff on words, like a bird imitating sounds without an idea of what they mean.
- convexfunction 4y ago
  Just imagine the hell we'd be in if people could give wrong answers in an authoritative tone and then insist they absolutely can't be wrong!
  - bambax 4y ago
    For some reason, we assume what comes out of a computer is more trustworthy than what people say. We think computers are transparent, reliable, idempotent and don't have an agenda. Even more so if we call it "intelligent"... But ChatGPT is a bullshit machine, and that much is new.
    - convexfunction 4y ago
      Who's "we"? :) To the extent this perception exists -- and I don't think "came from a computer" falls within the top 5 actually effective methods of laundering bullshit nowadays, though maybe it used to -- you might expect that it gets crushed into dust as the public gets more exposure to high-profile counterexamples. And, wait, isn't the concern usually that people read AI-generated content and trust it but don't think it came from a computer?
    - tpmoney 4y ago
      At least the good part of the answers being on stack overflow then is, like they used to say "On the internet no one knows you're a dog". So whether the answer came from ChatGPT or an aggressively overconfident fool, a wrong answer should get the same downvotes regardless, and a correct answer should get the same up votes. Probably the two biggest issues with ChatGPT being used to provide answers is whether it's wrong often enough to start swinging the experience of the site negative, and more importantly that some people are getting fake internet points unfairly.

gtirloni 4y ago

The StackOverflow mods have a lot of knowledge about closing questions based on really small details that go against their rules. I'm sure they will do well spotting AI-generated answers.

luckylion 4y ago
I doubt it. They've had problems with cheap automated answers for years where bots would essentially search for the question on SO and then copy an answer from another question verbatim. The answers were rarely useful because questions happen to be different even though they use the same keywords. Not only did they never bother to block that, they also didn't mind it and wanted to rely on the community down-voting those answers instead of at least blocking the bot -- and that's with a trivial check (identical answer already in DB). With something that's AI-generated, there's no chance. And with the general quality of many of the answers, there's no way to tell apart wrong answers from AI or humans.

ravenstine 4y ago

What you said may sum up the current state of AI. People's minds are continually being blown, but will there be a realization that these AIs are specialized to provide specious output and nothing else? There's a canyon of difference between something that sounds correct and a thing that is actually correct.

bambax 4y ago
Exactly. ChatGPT sounds like a bad student who didn't actually learn anything during the year and is trying to bullshit their way through the finals. Or a politician, maybe. It's formalizing the worst traits of humanity.
bediger4000 4y ago
Yes. I spent yesterday afternoon trying to get ChatGPT to write me a quine in nroff/troff macros, something I've not been able to do, or find if anyone has done. The generated quines look like they'd work, but don't. Same with an M4 macro processor quine - looks maybe correct, doesn't work. It did generate a Go quine.
- YeGoblynQueenne 4y ago
  >> It did generate a Go quine. Because there are several of them on the web: https://duckduckgo.com/?t=ffcm&q=Go+quine&atb=v344-1&ia=web https://duckduckgo.com/?t=ffcm&q=Go+quine&atb=v344-1&ia=web

bambax 4y ago

That's the worst part. It's all fun and games when it tells a story of cheese sandwiches and VCRs in the style of the King James bible, but when it gives wrong answers in an authoritative tone and then insists it absolutely can't be wrong, it's terrifying. I don't understand what good could come of this. Or at least make it detect what is fiction and what isn't. People are treating it like it's Wikipedia, but it's not. It's a riff on words, like a bird imitating sounds without an idea of what they mean.

convexfunction 4y ago
Just imagine the hell we'd be in if people could give wrong answers in an authoritative tone and then insist they absolutely can't be wrong!
- bambax 4y ago
  For some reason, we assume what comes out of a computer is more trustworthy than what people say. We think computers are transparent, reliable, idempotent and don't have an agenda. Even more so if we call it "intelligent"... But ChatGPT is a bullshit machine, and that much is new.
  - convexfunction 4y ago
    Who's "we"? :) To the extent this perception exists -- and I don't think "came from a computer" falls within the top 5 actually effective methods of laundering bullshit nowadays, though maybe it used to -- you might expect that it gets crushed into dust as the public gets more exposure to high-profile counterexamples. And, wait, isn't the concern usually that people read AI-generated content and trust it but don't think it came from a computer?
  - tpmoney 4y ago
    At least the good part of the answers being on stack overflow then is, like they used to say "On the internet no one knows you're a dog". So whether the answer came from ChatGPT or an aggressively overconfident fool, a wrong answer should get the same downvotes regardless, and a correct answer should get the same up votes. Probably the two biggest issues with ChatGPT being used to provide answers is whether it's wrong often enough to start swinging the experience of the site negative, and more importantly that some people are getting fake internet points unfairly.