3 ms·

Seems like a massive non-event. Why should it matter if an otherwise good answer came from an LLM? Presumably no one is bothered if a human author is aided by

by nomilk 3y ago

Seems like a massive non-event. Why should it matter if an otherwise good answer came from an LLM? Presumably no one is bothered if a human author is aided by a search engine, tech blogs, their own intuition and experimentation, or some guesswork - so long as they submit a quality answer; so why should it be any different for answers aided by LLMs - are they supposed to be automatically worse? The sole determinate of whether an answer should be allowed is its quality (i.e. usefulness to people accessing the site who are trying to solve their programming problems). An answer's origins are irrelevant (except for the purpose of attribution). StackOverflow's voting system already does an excellent job of getting the best answers to the top and down-voted ones to the bottom. Further, high-rep users have the ability to close/delete very bad answers. Protesting LLMs seems like not much more of an extension on the idea of protesting autocorrect.

deleted 3y ago
[deleted]
vintermann 3y ago
> Why should it matter if an otherwise good answer came from an LLM? Because good as LLMs may be at writing good answers, they're even better at writing good-looking answers. That's pretty much what they're been trained to do, and actually good answers only appear as a fortunate side effect. Obviously, this makes moderators' jobs a lot harder. I can understand that they get frustrated. It also allows automated climbing in SO's gamified status hierarchy, which makes it even more dysfunctional and threatens those currently on top of it. So they want to be able to ban posts for looking automated. It's just that whether you decide manually, or with an automated tool of your own, you're going to have a huge false positive rate and quite possibly unfortunate biases in it as well. (Very recently there was a paper showing how automated LM detectors were great at sussing out non-native English speakers and deciding they were bots).
- wilg 3y ago
  I agree with the rest of your comment, but it's framed as a retort but doesn't really work as one. The person your responding to already said "otherwise good answer" which excludes a wrong but "good-looking" one.
  - JenrHywy 3y ago
    Moderators generally aren't checking for correctness - who would have that sort of time?
    - komadori 3y ago
      I would expect the moderators to have some subject knowledge and often it doesn't take much to spot that an answer is blatantly fantastical as LLMs sometimes are. The finer points of some API might need actual checking, but a lot of things you'll know on sight.
  - vintermann 3y ago
    Point is, you don't know which are wrong unless you do the work. The moderator has to do the actual work, but the LM (or rather, the user running an LM bot on his account) gets the credit. They asked why it is a problem, and I tried to explain why it is in practice.
fzeroracer 3y ago
I have to keep posting this in every thread on this topic I guess. LLMs have an extremely high noise to value ratio; it is easy and quick for a user to generate an incorrect but nicely worded post and it is easy for said users to do so across multiple posts instantly. Additionally, there is no value in allowing for users to post answers by LLMs because if someone wanted that, they could've just asked the LLM themselves. A great example of this problem is people submitting bullshit PRs to repos, which has become worse with the advent of LLM popularity [1]. This means reviewers need to spend a lot more time sussing out if it's AI generated or not which takes away time from reviewing actual, valuable PRs. Or even stuff that's bad but generated by an actual human, because there's potential for actual knowledge transfer there. [1] https://mastodon.social/@danluu/110335983520055904 https://mastodon.social/@danluu/110335983520055904
KronisLV 3y ago
> Why should it matter if an otherwise good answer came from an LLM? I think the problem is that quite a bit of the time the answers won't be good, in addition to it being extremely easy to generate a large volume of content, which makes moderation difficult. That's more or less what happened with Clarkesworld: https://www.npr.org/2023/02/24/1159286436/ai-chatbot-chatgpt-magazine-clarkesworld-artificial-intelligence https://www.npr.org/2023/02/24/1159286436/ai-chatbot-chatgpt... > "By the time we closed on the 20th, around noon, we had received 700 legitimate submissions and 500 machine-written ones," he said. Imagine there suddenly being 2x more content to moderate, maybe 5x or 10x. And many of the completely false answers being written in a confident tone, yet still with hallucinations like referencing methods that don't exist and so on. That said, it's not like SO has any problem making everyone already have a hard time, with famously nitpicky moderation that scares people away.
alexott 3y ago
I stumbled this week on few answers on a topic that I’m following, and although they were really nicely looking, correctly formatted, etc. - they were absolutely wrong - ChatGPT was hallucinating, showing calls to functions that aren’t in specific SDKs, etc. And I’ve noticed that such answers are added to old questions too, adding more noise :-(

by nomilk 3y ago

deleted 3y ago

[deleted]

vintermann 3y ago

> Why should it matter if an otherwise good answer came from an LLM? Because good as LLMs may be at writing good answers, they're even better at writing good-looking answers. That's pretty much what they're been trained to do, and actually good answers only appear as a fortunate side effect. Obviously, this makes moderators' jobs a lot harder. I can understand that they get frustrated. It also allows automated climbing in SO's gamified status hierarchy, which makes it even more dysfunctional and threatens those currently on top of it. So they want to be able to ban posts for looking automated. It's just that whether you decide manually, or with an automated tool of your own, you're going to have a huge false positive rate and quite possibly unfortunate biases in it as well. (Very recently there was a paper showing how automated LM detectors were great at sussing out non-native English speakers and deciding they were bots).

wilg 3y ago
I agree with the rest of your comment, but it's framed as a retort but doesn't really work as one. The person your responding to already said "otherwise good answer" which excludes a wrong but "good-looking" one.
- JenrHywy 3y ago
  Moderators generally aren't checking for correctness - who would have that sort of time?
  - komadori 3y ago
    I would expect the moderators to have some subject knowledge and often it doesn't take much to spot that an answer is blatantly fantastical as LLMs sometimes are. The finer points of some API might need actual checking, but a lot of things you'll know on sight.
- vintermann 3y ago
  Point is, you don't know which are wrong unless you do the work. The moderator has to do the actual work, but the LM (or rather, the user running an LM bot on his account) gets the credit. They asked why it is a problem, and I tried to explain why it is in practice.

fzeroracer 3y ago

I have to keep posting this in every thread on this topic I guess. LLMs have an extremely high noise to value ratio; it is easy and quick for a user to generate an incorrect but nicely worded post and it is easy for said users to do so across multiple posts instantly. Additionally, there is no value in allowing for users to post answers by LLMs because if someone wanted that, they could've just asked the LLM themselves. A great example of this problem is people submitting bullshit PRs to repos, which has become worse with the advent of LLM popularity [1]. This means reviewers need to spend a lot more time sussing out if it's AI generated or not which takes away time from reviewing actual, valuable PRs. Or even stuff that's bad but generated by an actual human, because there's potential for actual knowledge transfer there. [1] https://mastodon.social/@danluu/110335983520055904 https://mastodon.social/@danluu/110335983520055904

KronisLV 3y ago

> Why should it matter if an otherwise good answer came from an LLM? I think the problem is that quite a bit of the time the answers won't be good, in addition to it being extremely easy to generate a large volume of content, which makes moderation difficult. That's more or less what happened with Clarkesworld: https://www.npr.org/2023/02/24/1159286436/ai-chatbot-chatgpt-magazine-clarkesworld-artificial-intelligence https://www.npr.org/2023/02/24/1159286436/ai-chatbot-chatgpt... > "By the time we closed on the 20th, around noon, we had received 700 legitimate submissions and 500 machine-written ones," he said. Imagine there suddenly being 2x more content to moderate, maybe 5x or 10x. And many of the completely false answers being written in a confident tone, yet still with hallucinations like referencing methods that don't exist and so on. That said, it's not like SO has any problem making everyone already have a hard time, with famously nitpicky moderation that scares people away.

alexott 3y ago

I stumbled this week on few answers on a topic that I’m following, and although they were really nicely looking, correctly formatted, etc. - they were absolutely wrong - ChatGPT was hallucinating, showing calls to functions that aren’t in specific SDKs, etc. And I’ve noticed that such answers are added to old questions too, adding more noise :-(