6 ms·

No, the new policy means that in almost all cases you cannot moderate content you consider AI-authored at all. This means moderators cannot delete those posts n

No, the new policy means that in almost all cases you cannot moderate content you consider AI-authored at all. This means moderators cannot delete those posts nor suspend the users for this specific reason. The result is pretty much that AI-generated content is essentially allowed as it cannot be effectively moderated. Even though many sites still have an official policy that disallows it. Disclaimer: I'm a mod on a small SE site, though I have not acted as a mod on any AI-generated content.

LewisVerstappen 3y ago
How do you know if content is AI generated? If mods could decree a piece of content as AI generated (and delete it) willy nilly, then that would be far worse IMO.
- mwint 3y ago
  As a common example, you see someone who asks a question in the format: “helo plz can help w cod, is broke has error” And then the next day posts four answers in perfect English to four different topics, with that GPT “vibe”. You can’t reliably detect generated content in a vacuum, but Stack Overflow is a very metadata-rich environment for moderators.
  - cja 3y ago
    As a user of SO, etc. I don't care how they wrote the answer. Is it a good answer? Does it help me? That's what I'm interested in. Why censor good answers?
    - minitech 3y ago
      They’re usually not good answers, and they’re subject to the bullshit asymmetry principle.
      - Marazan 3y ago
        This is a vital comment that people need to understand. It is _effort_ to moderate content, to read answers and see if they are legitimate. It is almost zero effort to chatgpt out "answers". The people providing the ChatGPT answers _do not care_ about them, they are only looking to pad their Rep. It is an attack against the very core of the StackExchange system.
        egeozcan 3y ago
        Why do people want to pad their rep? I have nearly 14K "rep" on SO, never had any positive effect on my career whatsoever. Also, theoretically (never happened), if someone mentioned their exceptional SE profile on their resume or their cover letter, I'd for sure ask them some details from the topics they suggest to be experts on. Will they also bring ChatGPT to the interview? That'd be fun to watch.
        selcuka 3y ago
        Stack Exchange accounts are sold and bought, so apparently there is an appeal.
        oriolid 3y ago
        If something has a score, people will compete in it just for the satisfaction of moving up on the leaderboard.
        yeputons 3y ago
        > Will they also bring ChatGPT to the interview? That'd be fun to watch. It is already done, as I've heard. The previous iteration was to pay your lookalike to pass the interview for you. Or just pay anyone and blame lack of camera on "technical difficulties".
        shagie 3y ago
        > Why do people want to pad their rep? I have nearly 14K "rep" on SO, never had any positive effect on my career whatsoever. If you are an applicant to a position where there are 5000 resumes submitted and the people doing the filter want a quick and easy numerical ranking of them - provide your Stack Overflow account. At that point, they can look up your rep and pick the best ones based on that. This doesn't happen as much in US based companies as there are other metrics that they can use to filter candidates rather than SO rep. However, if you are in India and applying to a consultancy and every resume and transcript are very similar to the point of not being able to distinguish between them - SO rep provides a very easy way to rank and filter applicants. Unfortunately, I can't verify this. I've only heard it second hand but it does make sense and helps explain why I occasionally get SO account and stats on resumes from contractors.
    - pbhjpbhj 3y ago
      Why turn SO into a cache of LLM responses, why not just ask an LLM if you want answers and don't care if those answers work?
      - ChrisLTD 3y ago
        Bingo. Stack Exchange could just call an LLM API when a question is asked and show the response. That wouldn’t be near as valuable as their verified index of answers.
        arsome 3y ago
        However, having knowledgeable people upvote/downvote/correct the AI answer could prove quite valuable.
    - tremon 3y ago
      Can you verify answers faster than GPT can generate them?
- fabian2k 3y ago
  I'm not the ideal person to ask here. But in general the output of specific AI tools like ChatGPT produces certain patterns that are quite noticeable. And you have to keep in mind that you usually have more than a single post available in these cases. The pattern of posting content is also a signal by itself. So if e.g. a user posts a dozen long answers within 10 minutes, and they all have characteristics of ChatGPT, that would be a pretty good signal.
  - LewisVerstappen 3y ago
    > So if e.g. a user posts a dozen long answers within 10 minutes, and they all have characteristics of ChatGPT, that would be a pretty good signal. Ah yeah, the frequency of posts definitely makes sense. > output of specific AI tools like ChatGPT produces certain patterns that are quite noticeable I would agree with you for gpt 3.5, but I don't think this is the case for GPT-4 (I've spent several hundred hours using GPT-4 for various tasks - mainly related to coding & learning random subjects).
    - fabian2k 3y ago
      In this case it certainly matters that ChatGPT is free. There are just many more people using that than paying for GPT-4 access.
      - pointlessone 3y ago
        Still, if the answer happens to be factually correct is it an issue? Say, a person has an answer but English is not their native language and they manage to stir ChatGPT into writing a good answer. Would we prefer to have that posted instead of keeping a question hanging without an answer at all? The only issue I can see with AI use is the rate of new content generation. Recent models are quite OK at giving a decent answer. SO is not a pinnacle of exceptionally well thought out answers from people either. There are great detailed and well sourced answers but more often then not you get an incomplete, outdated or even just plain wrong answers. Bespoke artisanal hand-crafted ethically sourced answers from fully organic free range humans that still lead to stack overflows and misalignments elements on webpages.
- lionkor 3y ago
  Somehow people are missing the obvious: Answers that say "Source: ChatGPT" or similar. I have seen plenty of those, and of course they are hot garbage and get downvoted
notatoad 3y ago
but you can still moderate it for all the normal quality issues, it has to be on-topic and truthful and properly sourced. it's only if it's a good answer, but you think it might have been ai generated that you can't moderate it? i'm really not seeing the problem here.
- JdeBP 3y ago
  That's because you've entirely missed the part where the diamond moderators report that in private they were given far stricter rules than were let on in public, which disallowed this sort of weaselling.
Zetice 3y ago
The fact something is or isn't AI doesn't require independent moderation procedures. Either the content has an answer to the question or it doesn't.
- fzeroracer 3y ago
  If that's your base criteria, why not remove humans from the equation entirely and just turn Stack Overflow into purely AI driven? It doesn't matter if the answer is correct, only the speed and formatting of the answer.
  - DougBTX 3y ago
    That’s a more realistic option than you’d think. What is the point of a software Q&A site anyway, why not just read the docs? In my experience, Q&A is useful because it summarises and consolidates disparate information into a concise response to a prompt (sorry, I mean to a Question!) which is... exactly what a chat AI does. Isn’t chat AI an existential threat to SO, even if AI is banned there? Certainly I find chat AI better in many cases, and if I don’t like the answers, just going to the docs is the next step, not a generic Google search with SO in the SERP.
    - never_inline 3y ago
      > In my experience, Q&A is useful because it summarises and consolidates disparate information into a concise response to a prompt (sorry, I mean to a Question!) which is... exactly what a chat AI does There is knowledge you will never find in documentation. 50% of places where SO has been helpful to me are not "consolidating documentation". They are solutions to obscure bugs, Useful APIs which are not documented with any example, and low-level logic / high-level design solutions.
- wtallis 3y ago
  If something doesn't answer the question, then it should be removed by moderators. But it shouldn't stop there: junk generated by AI is going to be mass-produced, and anyone posting incorrect AI-generated answers is probably going to be a repeat offender. Taking proactive action against such users who are operating so far outside the purpose of the site is very likely necessary to have any hope of maintaining a reasonable signal to noise ratio.

LewisVerstappen 3y ago

How do you know if content is AI generated? If mods could decree a piece of content as AI generated (and delete it) willy nilly, then that would be far worse IMO.

mwint 3y ago
As a common example, you see someone who asks a question in the format: “helo plz can help w cod, is broke has error” And then the next day posts four answers in perfect English to four different topics, with that GPT “vibe”. You can’t reliably detect generated content in a vacuum, but Stack Overflow is a very metadata-rich environment for moderators.
- cja 3y ago
  As a user of SO, etc. I don't care how they wrote the answer. Is it a good answer? Does it help me? That's what I'm interested in. Why censor good answers?
  - minitech 3y ago
    They’re usually not good answers, and they’re subject to the bullshit asymmetry principle.
    - Marazan 3y ago
      This is a vital comment that people need to understand. It is _effort_ to moderate content, to read answers and see if they are legitimate. It is almost zero effort to chatgpt out "answers". The people providing the ChatGPT answers _do not care_ about them, they are only looking to pad their Rep. It is an attack against the very core of the StackExchange system.
      - egeozcan 3y ago
        Why do people want to pad their rep? I have nearly 14K "rep" on SO, never had any positive effect on my career whatsoever. Also, theoretically (never happened), if someone mentioned their exceptional SE profile on their resume or their cover letter, I'd for sure ask them some details from the topics they suggest to be experts on. Will they also bring ChatGPT to the interview? That'd be fun to watch.
        selcuka 3y ago
        Stack Exchange accounts are sold and bought, so apparently there is an appeal.
        oriolid 3y ago
        If something has a score, people will compete in it just for the satisfaction of moving up on the leaderboard.
        yeputons 3y ago
        > Will they also bring ChatGPT to the interview? That'd be fun to watch. It is already done, as I've heard. The previous iteration was to pay your lookalike to pass the interview for you. Or just pay anyone and blame lack of camera on "technical difficulties".
        shagie 3y ago
        > Why do people want to pad their rep? I have nearly 14K "rep" on SO, never had any positive effect on my career whatsoever. If you are an applicant to a position where there are 5000 resumes submitted and the people doing the filter want a quick and easy numerical ranking of them - provide your Stack Overflow account. At that point, they can look up your rep and pick the best ones based on that. This doesn't happen as much in US based companies as there are other metrics that they can use to filter candidates rather than SO rep. However, if you are in India and applying to a consultancy and every resume and transcript are very similar to the point of not being able to distinguish between them - SO rep provides a very easy way to rank and filter applicants. Unfortunately, I can't verify this. I've only heard it second hand but it does make sense and helps explain why I occasionally get SO account and stats on resumes from contractors.
  - pbhjpbhj 3y ago
    Why turn SO into a cache of LLM responses, why not just ask an LLM if you want answers and don't care if those answers work?
    - ChrisLTD 3y ago
      Bingo. Stack Exchange could just call an LLM API when a question is asked and show the response. That wouldn’t be near as valuable as their verified index of answers.
      - arsome 3y ago
        However, having knowledgeable people upvote/downvote/correct the AI answer could prove quite valuable.
  - tremon 3y ago
    Can you verify answers faster than GPT can generate them?
fabian2k 3y ago
I'm not the ideal person to ask here. But in general the output of specific AI tools like ChatGPT produces certain patterns that are quite noticeable. And you have to keep in mind that you usually have more than a single post available in these cases. The pattern of posting content is also a signal by itself. So if e.g. a user posts a dozen long answers within 10 minutes, and they all have characteristics of ChatGPT, that would be a pretty good signal.
- LewisVerstappen 3y ago
  > So if e.g. a user posts a dozen long answers within 10 minutes, and they all have characteristics of ChatGPT, that would be a pretty good signal. Ah yeah, the frequency of posts definitely makes sense. > output of specific AI tools like ChatGPT produces certain patterns that are quite noticeable I would agree with you for gpt 3.5, but I don't think this is the case for GPT-4 (I've spent several hundred hours using GPT-4 for various tasks - mainly related to coding & learning random subjects).
  - fabian2k 3y ago
    In this case it certainly matters that ChatGPT is free. There are just many more people using that than paying for GPT-4 access.
    - pointlessone 3y ago
      Still, if the answer happens to be factually correct is it an issue? Say, a person has an answer but English is not their native language and they manage to stir ChatGPT into writing a good answer. Would we prefer to have that posted instead of keeping a question hanging without an answer at all? The only issue I can see with AI use is the rate of new content generation. Recent models are quite OK at giving a decent answer. SO is not a pinnacle of exceptionally well thought out answers from people either. There are great detailed and well sourced answers but more often then not you get an incomplete, outdated or even just plain wrong answers. Bespoke artisanal hand-crafted ethically sourced answers from fully organic free range humans that still lead to stack overflows and misalignments elements on webpages.
lionkor 3y ago
Somehow people are missing the obvious: Answers that say "Source: ChatGPT" or similar. I have seen plenty of those, and of course they are hot garbage and get downvoted

notatoad 3y ago

but you can still moderate it for all the normal quality issues, it has to be on-topic and truthful and properly sourced. it's only if it's a good answer, but you think it might have been ai generated that you can't moderate it? i'm really not seeing the problem here.

JdeBP 3y ago
That's because you've entirely missed the part where the diamond moderators report that in private they were given far stricter rules than were let on in public, which disallowed this sort of weaselling.

Zetice 3y ago

The fact something is or isn't AI doesn't require independent moderation procedures. Either the content has an answer to the question or it doesn't.

fzeroracer 3y ago
If that's your base criteria, why not remove humans from the equation entirely and just turn Stack Overflow into purely AI driven? It doesn't matter if the answer is correct, only the speed and formatting of the answer.
- DougBTX 3y ago
  That’s a more realistic option than you’d think. What is the point of a software Q&A site anyway, why not just read the docs? In my experience, Q&A is useful because it summarises and consolidates disparate information into a concise response to a prompt (sorry, I mean to a Question!) which is... exactly what a chat AI does. Isn’t chat AI an existential threat to SO, even if AI is banned there? Certainly I find chat AI better in many cases, and if I don’t like the answers, just going to the docs is the next step, not a generic Google search with SO in the SERP.
  - never_inline 3y ago
    > In my experience, Q&A is useful because it summarises and consolidates disparate information into a concise response to a prompt (sorry, I mean to a Question!) which is... exactly what a chat AI does There is knowledge you will never find in documentation. 50% of places where SO has been helpful to me are not "consolidating documentation". They are solutions to obscure bugs, Useful APIs which are not documented with any example, and low-level logic / high-level design solutions.
wtallis 3y ago
If something doesn't answer the question, then it should be removed by moderators. But it shouldn't stop there: junk generated by AI is going to be mass-produced, and anyone posting incorrect AI-generated answers is probably going to be a repeat offender. Taking proactive action against such users who are operating so far outside the purpose of the site is very likely necessary to have any hope of maintaining a reasonable signal to noise ratio.