5 ms·

The fact something is or isn't AI doesn't require independent moderation procedures. Either the content has an answer to the question or it doesn't.

by Zetice 3y ago

fzeroracer 3y ago
If that's your base criteria, why not remove humans from the equation entirely and just turn Stack Overflow into purely AI driven? It doesn't matter if the answer is correct, only the speed and formatting of the answer.
- DougBTX 3y ago
  That’s a more realistic option than you’d think. What is the point of a software Q&A site anyway, why not just read the docs? In my experience, Q&A is useful because it summarises and consolidates disparate information into a concise response to a prompt (sorry, I mean to a Question!) which is... exactly what a chat AI does. Isn’t chat AI an existential threat to SO, even if AI is banned there? Certainly I find chat AI better in many cases, and if I don’t like the answers, just going to the docs is the next step, not a generic Google search with SO in the SERP.
  - never_inline 3y ago
    > In my experience, Q&A is useful because it summarises and consolidates disparate information into a concise response to a prompt (sorry, I mean to a Question!) which is... exactly what a chat AI does There is knowledge you will never find in documentation. 50% of places where SO has been helpful to me are not "consolidating documentation". They are solutions to obscure bugs, Useful APIs which are not documented with any example, and low-level logic / high-level design solutions.
wtallis 3y ago
If something doesn't answer the question, then it should be removed by moderators. But it shouldn't stop there: junk generated by AI is going to be mass-produced, and anyone posting incorrect AI-generated answers is probably going to be a repeat offender. Taking proactive action against such users who are operating so far outside the purpose of the site is very likely necessary to have any hope of maintaining a reasonable signal to noise ratio.
- omegabravo 3y ago
  probably, or is? The offence here is spamming, AI is orthogonal. Users spamming should be given a timeout.
  - Eisenstein 3y ago
    > The offence here is spamming, AI is orthogonal. You can make broad classifications for an act (it wasn't murder, it was negligent discharge of a firearm), but that doesn't preclude the fact that it is also something else (shooting in the direction of a person without regard to their life and killing them is murder).
    - omegabravo 3y ago
      Your analogy isn't a great fit, because the distinction between the two carries different penalties. In this scenario, spamming with or without AI would carry the same penalty. The benefit is that you don't need to determine if AI was involved, since it doesn't matter.
  - andrewaylett 3y ago
    Consider as an analogy, safety while driving. Driving dangerously is fairly uniformly proscribed, but we also have specific prohibitions on various mechanisms by which people drive dangerously. It's much easier to prove that someone was intoxicated or using a mobile phone, rather than needing to show in each case that the intoxication was dangerous or that the mobile phone use was distracting and the distraction was dangerous. Similarly, use of LLMs to generate and post large quantities of text from a short prompt is inherently spamming. The user could provide exactly the same value by posting their prompt. If the prompt isn't valuable, the output won't be either, as anyone with a copy of the prompt could generate equivalent output for themselves if they wished to.
    - andrewaylett 3y ago
      As an addendum, to not (I hope) distract from the point about spam -- I don't at all object to using an LLM for inspiration, or editing, or summarisation. So long as there's no claim that the output contains more information than the input. Any statement of fact in the output should be present in the prompt or validated by a human before publishing, or it's suspect. And if it's published without disclosing the lack of validation, it's unethical.
rtpg 3y ago
Incorrect answers are not what moderators are moderating. A super easy way to make answers that look right even if they are wrong, for people trying to game SO's point system, is a perfect way for there to be just a huge amount of extra work for mods without ~any benefit. There are definitely alternatives to the unilateral ban (Thinking about how, like, chess.com does bans based on people cheating), but saying "AI content is qualitatively no different" ignores the bigger ecosystem problem from having the average answer quality simply go down
ohgodplsno 3y ago
Taking aside the fact that the AI generated answers are awfully wrong most of the time, this line of thinking also leads to an absolutely dreadful state of internet where everything is AI generated crap. This is made worse by the fact that StackOverflow has points, badges and everything. People buy and sell stars on github to increase their chances of being hired somewhere. People absolutely will game the shit out of StackOverflow, not caring for a single second if it actually makes it a better place if it means they get shiny badges and get to put "top commenter on the C# subject on SO (588100 points)" on their resume.
- janekm 3y ago
  Giving moderators the power to arbitrarily ban users they deem to have produced content by AI is not likely to work for long though. Perhaps today it's still feasible to note the "vibe" for GPT, but that's not likely to be possible for long (it's probably an artefact of the RLHF process anyway). Then what? You'll end up with essentially arbitrary application of the ban hammer based on the moderators' "feelings". I certainly don't have the answers (and let's face it it's likely going to be a problem here too).
  - ohgodplsno 3y ago
    >You'll end up with essentially arbitrary application of the ban hammer based on the moderators' "feelings". So, like the current organisation of things, that works quite well? You put entirely too much value on moderators being perfect. Moderators have been biased forever, have banned for no reason forever. It's a website, you can live with that (or without it). If they're unreasonable, move away, find another place or create another place. If you can't move away, deal with it, create an alt, don't get caught doing the same thing that got you banned, and that'll be it.
  - alwaysbeconsing 3y ago
    > is not likely to work for long though. This is true, and it's why the policy promulgated by the moderators was explicitly temporary. It was a response to an acute problem to give everyone time to figure out how to handle it as a chronic condition: https://meta.stackoverflow.com/questions/421831/temporary-policy-chatgpt-is-banned https://meta.stackoverflow.com/questions/421831/temporary-po...
pbhjpbhj 3y ago
The problem is that moderation does not necessarily check the solution. A moderator can't acquire the specific version of hardware the questioner had, recreate the failure conditions, then test the solution. Instead, a moderator looks to see if an answer seems reasonable and fits the correct syntax and site-specific rules. LLMs are great at producing bullshit that looks convincing. Essentially, one has to trust responders to provide answers to some extent. Untrustworthy responders can use generators to bullshit their way into acquiring trust.
paganel 3y ago
Because AI doesn't answer the question, there's nothing "volitional" in it, it's just a (computationally smart) rehash of past answers, hoping that it will all fit together somehow.
- pwdisswordfishc 3y ago
  > there's nothing "volitional" in it > hoping Hmmmm
  - paganel 3y ago
    I know, no matter how much I try (and I do try) the anthropomorphization creeps through from the most unexpected places. I personally hate it, as I hate all this related AI-discourse.
jabiko 3y ago
[dead]
TheRealDunkirk 3y ago
I think you're getting to the heart of the behind-the-scenes business decision: Let AI responses proliferate on the platform -- heck, maybe even allow a bot to submit the questions to various LLM's and post the response as a normal user -- then make the volunteer community figure out whether the response was good or not, without letting the mods interfere in the process.
- MauranKilom 3y ago
  ...the mods are part of the volunteer community, and their role largely already is more about keeping the bad apples out than establishing whether a particular response is good or not. Where do you get the idea from that the mods are incapable of this while "the volunteer community" is (or why do you think that's what SE thinks)?
  - TheRealDunkirk 3y ago
    I was implying that I think SE is doing this because it’s the only thing I can think of that would explain what appears to be an insane change of policy. How else would you explain it? What else would be the goal except to test AI responses in the wild as a sort of meta experiment? I’m just trying to connect the dots here.

The fact something is or isn't AI doesn't require independent moderation procedures. Either the content has an answer to the question or it doesn't.

by Zetice 3y ago

The fact something is or isn't AI doesn't require independent moderation procedures. Either the content has an answer to the question or it doesn't.

fzeroracer 3y ago

If that's your base criteria, why not remove humans from the equation entirely and just turn Stack Overflow into purely AI driven? It doesn't matter if the answer is correct, only the speed and formatting of the answer.

DougBTX 3y ago
That’s a more realistic option than you’d think. What is the point of a software Q&A site anyway, why not just read the docs? In my experience, Q&A is useful because it summarises and consolidates disparate information into a concise response to a prompt (sorry, I mean to a Question!) which is... exactly what a chat AI does. Isn’t chat AI an existential threat to SO, even if AI is banned there? Certainly I find chat AI better in many cases, and if I don’t like the answers, just going to the docs is the next step, not a generic Google search with SO in the SERP.
- never_inline 3y ago
  > In my experience, Q&A is useful because it summarises and consolidates disparate information into a concise response to a prompt (sorry, I mean to a Question!) which is... exactly what a chat AI does There is knowledge you will never find in documentation. 50% of places where SO has been helpful to me are not "consolidating documentation". They are solutions to obscure bugs, Useful APIs which are not documented with any example, and low-level logic / high-level design solutions.

wtallis 3y ago

If something doesn't answer the question, then it should be removed by moderators. But it shouldn't stop there: junk generated by AI is going to be mass-produced, and anyone posting incorrect AI-generated answers is probably going to be a repeat offender. Taking proactive action against such users who are operating so far outside the purpose of the site is very likely necessary to have any hope of maintaining a reasonable signal to noise ratio.

omegabravo 3y ago
probably, or is? The offence here is spamming, AI is orthogonal. Users spamming should be given a timeout.
- Eisenstein 3y ago
  > The offence here is spamming, AI is orthogonal. You can make broad classifications for an act (it wasn't murder, it was negligent discharge of a firearm), but that doesn't preclude the fact that it is also something else (shooting in the direction of a person without regard to their life and killing them is murder).
  - omegabravo 3y ago
    Your analogy isn't a great fit, because the distinction between the two carries different penalties. In this scenario, spamming with or without AI would carry the same penalty. The benefit is that you don't need to determine if AI was involved, since it doesn't matter.
- andrewaylett 3y ago
  Consider as an analogy, safety while driving. Driving dangerously is fairly uniformly proscribed, but we also have specific prohibitions on various mechanisms by which people drive dangerously. It's much easier to prove that someone was intoxicated or using a mobile phone, rather than needing to show in each case that the intoxication was dangerous or that the mobile phone use was distracting and the distraction was dangerous. Similarly, use of LLMs to generate and post large quantities of text from a short prompt is inherently spamming. The user could provide exactly the same value by posting their prompt. If the prompt isn't valuable, the output won't be either, as anyone with a copy of the prompt could generate equivalent output for themselves if they wished to.
  - andrewaylett 3y ago
    As an addendum, to not (I hope) distract from the point about spam -- I don't at all object to using an LLM for inspiration, or editing, or summarisation. So long as there's no claim that the output contains more information than the input. Any statement of fact in the output should be present in the prompt or validated by a human before publishing, or it's suspect. And if it's published without disclosing the lack of validation, it's unethical.

rtpg 3y ago

Incorrect answers are not what moderators are moderating. A super easy way to make answers that look right even if they are wrong, for people trying to game SO's point system, is a perfect way for there to be just a huge amount of extra work for mods without ~any benefit. There are definitely alternatives to the unilateral ban (Thinking about how, like, chess.com does bans based on people cheating), but saying "AI content is qualitatively no different" ignores the bigger ecosystem problem from having the average answer quality simply go down

ohgodplsno 3y ago

Taking aside the fact that the AI generated answers are awfully wrong most of the time, this line of thinking also leads to an absolutely dreadful state of internet where everything is AI generated crap. This is made worse by the fact that StackOverflow has points, badges and everything. People buy and sell stars on github to increase their chances of being hired somewhere. People absolutely will game the shit out of StackOverflow, not caring for a single second if it actually makes it a better place if it means they get shiny badges and get to put "top commenter on the C# subject on SO (588100 points)" on their resume.

janekm 3y ago
Giving moderators the power to arbitrarily ban users they deem to have produced content by AI is not likely to work for long though. Perhaps today it's still feasible to note the "vibe" for GPT, but that's not likely to be possible for long (it's probably an artefact of the RLHF process anyway). Then what? You'll end up with essentially arbitrary application of the ban hammer based on the moderators' "feelings". I certainly don't have the answers (and let's face it it's likely going to be a problem here too).
- ohgodplsno 3y ago
  >You'll end up with essentially arbitrary application of the ban hammer based on the moderators' "feelings". So, like the current organisation of things, that works quite well? You put entirely too much value on moderators being perfect. Moderators have been biased forever, have banned for no reason forever. It's a website, you can live with that (or without it). If they're unreasonable, move away, find another place or create another place. If you can't move away, deal with it, create an alt, don't get caught doing the same thing that got you banned, and that'll be it.
- alwaysbeconsing 3y ago
  > is not likely to work for long though. This is true, and it's why the policy promulgated by the moderators was explicitly temporary. It was a response to an acute problem to give everyone time to figure out how to handle it as a chronic condition: https://meta.stackoverflow.com/questions/421831/temporary-policy-chatgpt-is-banned https://meta.stackoverflow.com/questions/421831/temporary-po...

pbhjpbhj 3y ago

The problem is that moderation does not necessarily check the solution. A moderator can't acquire the specific version of hardware the questioner had, recreate the failure conditions, then test the solution. Instead, a moderator looks to see if an answer seems reasonable and fits the correct syntax and site-specific rules. LLMs are great at producing bullshit that looks convincing. Essentially, one has to trust responders to provide answers to some extent. Untrustworthy responders can use generators to bullshit their way into acquiring trust.

paganel 3y ago

Because AI doesn't answer the question, there's nothing "volitional" in it, it's just a (computationally smart) rehash of past answers, hoping that it will all fit together somehow.

pwdisswordfishc 3y ago
> there's nothing "volitional" in it > hoping Hmmmm
- paganel 3y ago
  I know, no matter how much I try (and I do try) the anthropomorphization creeps through from the most unexpected places. I personally hate it, as I hate all this related AI-discourse.

jabiko 3y ago

[dead]

TheRealDunkirk 3y ago

I think you're getting to the heart of the behind-the-scenes business decision: Let AI responses proliferate on the platform -- heck, maybe even allow a bot to submit the questions to various LLM's and post the response as a normal user -- then make the volunteer community figure out whether the response was good or not, without letting the mods interfere in the process.

MauranKilom 3y ago
...the mods are part of the volunteer community, and their role largely already is more about keeping the bad apples out than establishing whether a particular response is good or not. Where do you get the idea from that the mods are incapable of this while "the volunteer community" is (or why do you think that's what SE thinks)?
- TheRealDunkirk 3y ago
  I was implying that I think SE is doing this because it’s the only thing I can think of that would explain what appears to be an insane change of policy. How else would you explain it? What else would be the goal except to test AI responses in the wild as a sort of meta experiment? I’m just trying to connect the dots here.