8 ms·

How's the Stack Overflow strike going?

64 points by jlericson 3y ago

nailer 3y ago
Top 0.05% here. I still haven’t seen any written evidence for the policy change being protested (the protest site says they are protesting a “near-total prohibition on moderating AI-generated content”, it seems incredible that StackOverflow would ever prohibit moderation of any type of content) and when I’ve asked I’ve been told it was discussed in secret so nobody can prove it. What I suspect is that somebody has asked the site to use automatic GPT removal tools and the site has said they don’t work, which is true.
- creativenolo 3y ago
  Yeah, given ChatGPT is most likely trained on SO a simple test would be to use the automatic tools on all of the pre-ChatGPT SO. My bet is, much of pre-ChatGPT SO would be removed.
- MikeDelta 3y ago
  A teacher thought he figured it out and asked ChatGPT if it wrote the text that his students handed in. ChatGPT said yes for every piece of work, so he failed the whole class. Apparently it didn't write it. Turns out ChatGPT thinks it wrote a lot of other things like the email that the teacher wrote to the class to sack them, when one of the students asked ChatGPT as proof of fiction. Not sure how it finally ended. https://news.ycombinator.com/item?id=35963163 https://news.ycombinator.com/item?id=35963163
fabian2k 3y ago
https://meta.stackexchange.com/questions/389928/gpt-on-the-platform-data-actions-and-outcomes https://meta.stackexchange.com/questions/389928/gpt-on-the-p... This post shows the data that SE finally provided, though only after the strike was already happening. So far the community is not convinced. It looks like there is something happening there that reduced the number of people that provide multiple answers more than expected, but not everybody is convinced it is the mod actions on ChatGPT posts. Another issue might be that this only explains a part of the decline in questions and answers. So even if this part is solved, the larger problem of declining engagement remains.
- albert_e 3y ago
  Slightly off topic possibly -- but I personally felt overwhelmed in recent months due to the daily onslaught of new GPT/LLM/GenAI related news. I work in a related tech area and I have been meaning to do two things (1) prusue recertifications on a couple of professional certs that are up for renewal later this year and (2) get hands-on experience with some of the LLM and GenAI tech that is coming out so I can either broaden my professional skills to include them or at least benefit from knowing those tools thrugh inproved productivity/creativity. What actually happened is that I achieved much less -- virtually zero -- tangible progress on both these goals. I think some of my general interests and pursuits have similarly suffered due to this indecisiveness and overload. I may just be blaming some external factors for my laziness or ADHD but I cant escape the feeling that I am in a more confusing world now. Why am I saying this here -- Some of the trends in user behaviour that these massive tech-oriented community sites are seeing over this period could have external factors like those I described above. Maybe people are being drawn elsewhere. Maybe people are overwhelmend, confused, or doing something else, maybe participating more in SD / MidJourney Discords. Assuming that their website and traffic is a closed box and only the variables they change will affect the outcomes would be naiive.
- starball-tgz 3y ago
  that "something" is pretty obvious: people are asking their questions to ChatGPT. Fewer questions -> fewer answers. I've done queries into the Stack Exchange network's public data dump and put my analysis here: https://meta.stackexchange.com/a/389960/997587 https://meta.stackexchange.com/a/389960/997587
dvt 3y ago
Stack Overflow has absolutely been destroyed by the moderators, so I couldn't care less about their "strike." I was (and still am) in the top ~%0.80 of users[1] but no longer contribute to the site (I stopped ~6 years ago) because of the moderators (a few in particular, but I won't name any names as some of them are here on HN). It has been an absolute shitshow of closing questions that shouldn't be closed, anally-retentive nitpicks which intimidate new users, the essential nuking of the community wiki (even prior to the official deprecation), bad answers being upvoted, good answers being deleted, and so on. The whole "community moderator" thing ended up being a popularity contest where typical nitwitted social climbers ended up injecting themselves in every single minor conflict on the site just to score visibility points come community voting time. On top of this, SO is also dying as it has no real viable way of cleaning up or deprecating old answers, and if new ones are asked, they are closed in favor of the old (outdated) ones. Slowly, reddit and language forums/mailing lists are becoming more and more valuable as Stack Overflow becomes more and more of a trash heap. It sucks because I really really loved Stack Overflow, but it just broke my heart one too many times. [1] https://stackoverflow.com/users/243613/david-titarenco https://stackoverflow.com/users/243613/david-titarenco
- muppetman 3y ago
  I'll say. I'm too scared to use it because every legit question I've seen is shut down by a mod with bonkers reasoning
  - _nalply 3y ago
    No need. ChatGPT to the rescue. The answer quality is about the same but for different reasons. It's a hit or miss for both. But! With ChatGPT there is no social pressure exerted by humans. It's only you and the machine. It's liberating.
    - joebiden2 3y ago
      Don't know why this gets downvoted. It is simply true. In most cases, ChatGPT4 gets it right. You have to check anyway.
ftxbro 3y ago
I'm on the internet a lot but I don't know, I feel like it would be almost embarrassing to be an unpaid moderator of a site like that. Then to 'strike' like they are flexing their unpaid power on some company that doesn't care about them ... I don't know it seems like a lack of self respect. Why don't they quit moderating and do something else, it would seem more dignified.
- emodendroket 3y ago
  The entire company is structured around unpaid labor so they might care if it all stops.
deeviant 3y ago
I have not used SO once since chatgpt 4 came out, not once. I'm sure there are cases there SO has better responses than GPT, but frankly, they are rare, and the ability to query and shape chatGPTs responses basically obsoletes the SO model overnight. So the SO CEO is absolutely correct to pivot to AI before that ship sinks.
- bombolo 3y ago
  Well chatgpt is trained on SO. If people stop using SO, chatgpt will not be able to respond about newer languages/frameworks.
  - deeviant 3y ago
    Chatgpt is trained on the internet, off which SO is part of it. It's also trained off interactions with it's users, which would eventually take over sources like SO.
    - bombolo 3y ago
      You think people will start to teach chatgpt when it can't answer?
- AnonCoward42 3y ago
  Isn't ChatGPT also using SO? If SO would stop and there would be no replacement, then the answers might get worse. My take is that these two currently complement. In my opinion the issue with SO and with most social media is the obsession with metrics. It would be nice if they took more of an HN approach to metrics (votes particularly).
  - arp242 3y ago
    It's pretty neat when you get upvotes on answers you made months or years ago; a substantial number of them represent someone being helped out by your answer. I think that's pretty neat. For years the open source tool I spent the most time on was one I only very occasionally used myself, but got a lot of feedback and questions. Because this was a non-technical tool most of it was over email, and translated in fairly few "stars" (although I did migrate to GitHub quite late). All of this makes it rather different from voting on HN.
gregsher88 3y ago
I submitted a question regarding an obscure realmd bug and subsequently submitted the answer (which took about 24 hours to arrive at). The answer was nitpicked over a tag, downvoted, and finally removed. I would not be surprised if someone in the world is currently struggling to resolve the same realmd issue as it is not documented anywhere else online. That was the last time I will contribute to Stack Overflow.
- deleted 3y ago
  [deleted]
- eyelidlessness 3y ago
  That’s pretty par for the course for the last few years, and I don’t even bother looking there for answers much anymore for the same reason.
- jsnell 3y ago
  Does this have something to do with the strike? (E.g. you think the moderation is still ongoing despite the supposed strike, or worse than before?) Or is this just a generic SO complaint?
  - ftxbro 3y ago
    I feel like you are itching to get his hacker news answer nitpicked over its relevance, downvoted, and finally removed.
    - deleted 3y ago
      [deleted]
    - jsnell 3y ago
      I mean, yes. Wouldn't it be pretty nice for the top comment to be relevant to the article rather than the 500th rerun of the same SO moderation complaints?
      - ricardo81 3y ago
nomilk 3y ago
Seems like a massive non-event. Why should it matter if an otherwise good answer came from an LLM? Presumably no one is bothered if a human author is aided by a search engine, tech blogs, their own intuition and experimentation, or some guesswork - so long as they submit a quality answer; so why should it be any different for answers aided by LLMs - are they supposed to be automatically worse? The sole determinate of whether an answer should be allowed is its quality (i.e. usefulness to people accessing the site who are trying to solve their programming problems). An answer's origins are irrelevant (except for the purpose of attribution). StackOverflow's voting system already does an excellent job of getting the best answers to the top and down-voted ones to the bottom. Further, high-rep users have the ability to close/delete very bad answers. Protesting LLMs seems like not much more of an extension on the idea of protesting autocorrect.
- deleted 3y ago
  [deleted]
- vintermann 3y ago
  > Why should it matter if an otherwise good answer came from an LLM? Because good as LLMs may be at writing good answers, they're even better at writing good-looking answers. That's pretty much what they're been trained to do, and actually good answers only appear as a fortunate side effect. Obviously, this makes moderators' jobs a lot harder. I can understand that they get frustrated. It also allows automated climbing in SO's gamified status hierarchy, which makes it even more dysfunctional and threatens those currently on top of it. So they want to be able to ban posts for looking automated. It's just that whether you decide manually, or with an automated tool of your own, you're going to have a huge false positive rate and quite possibly unfortunate biases in it as well. (Very recently there was a paper showing how automated LM detectors were great at sussing out non-native English speakers and deciding they were bots).
  - wilg 3y ago
    I agree with the rest of your comment, but it's framed as a retort but doesn't really work as one. The person your responding to already said "otherwise good answer" which excludes a wrong but "good-looking" one.
londgine 3y ago
This article has some useful information, but I read the petition first, and it was very hard to understand what they were talking about. My tl;dr is: 1: chatgpt became an easy way to provide mostly correct answers that sound completely correct. 2: mods got annoyed that many incorrect answers were being published, so they made a bot that would identify and delete answers written by chatgpt. 3: As anyone who asked a question to stack overflow, and had it removed as a duplicate, knows, the mod's bots have a very high false positive rate. 4: The admins didn't like that answers from actual humans were being deleted that often, so they forbid the mods from autodeleting based on their tool. 5: some mods are angry at that decision, so they won't be moderating until the admins change that policy. In this situation (as opposed to the Reddit situation), I am all for the admins. I am unable to ask any question in stack overflow without it being removed as a duplicate (even if I clearly state why the other answer is irrelevant to my question). If the AI detecting bot has even a remotely similar false positive rate as the duplication bot, then I am glad that it is not allowed. This would discourage many people from not only asking questions, but also answering.
- fabian2k 3y ago
  There is no bot. This is entirely about human moderators acting on posts they consider AI-generated.
mbork_pl 3y ago
To provide a different perspective: I very rarely asked questions on SO, but I was a pretty active member (submitting both questions and answers) at tex.stackexchange.com a few years ago. It was a very, very nice and welcoming community (and probably still is, it's just that I moved on and don't participate that much anymore for personal/professional reasons). I am almost sure the same goes for emacs.stackexchange.com (which I sometimes read) and a few other sites in the "family". So I would not conflate SO with the wider SE set of communities.
- xcdzvyn 3y ago
  Aviation Stack Exchange is lovely too! :)
- JdeBP 3y ago
  It's important to remember that there are 181 Stack Exchange sites. It's important to not paint them with the same brush. It's important to remember, conversely, that this is a general strike, and that diamond moderators from the 180 others are involved. It's also important to remember that the 180 other sites are suffering from ChatGPT-created content as well. Sometimes it's better, in that there's almost no-one copying machine-written answers in the first place. Sometimes it's worse, in that now the brakes are very much off. The SQA stackexchange site, for example, turns out to have 2 fairly prolific ChatGPT copy-and-pasters that have been discussed on its Meta site, and who have histories of soliciting business for their respective companies in answers. Since the inception of the diamond moderator strike, they have gone into out-and-out open competition, posting ChatGPT answers competing with each other to years-old questions that actually have human-written answers of long standing. The RetroComputing stackexchange site, for example, appears to have no copies and pastes of ChatGPT content at all over the past few months.
- Fomite 3y ago
  I eventually got burnt out of answering there, but I found Academia StackExchange pleasant, and CrossValidated is in my dissertation acknowledgements.
dmingod666 3y ago
StackOverflow is toxic and if I don't have to deal with that it's a plus in my book. Too much drama on for my liking. Finding content is harder too, go through a Google search and 5-6 tabs before you reach an answer, ChatGPT is my first go-to tool for solving common stuff and stuff like syntax reference.
bandrami 3y ago
SE has been more or less useless at least for me for about a decade, mostly because of completely overzealous de-duping moderation. It doesn't help that I'm mostly on Server Fault, where small differences in versions and combinations of services can make a huge difference in what the right answer is. But if the question is "how to get nginx to pull auth from two different databases" the answer can be very different based on 1. the version of nginx you're talking about and 2. the specific databases you are using. But a mod sees "nginx auth multiple db" and deletes it as a duplicate.
- munimbiz 3y ago
  [flagged]
- starball-tgz 3y ago
  In those cases where things are version sensitive, it's useful to put version info in the title. That way _everyone_ will know what version the question is about- people searching for answers, and people trying to curate the knowledge-base.
dataengineer56 3y ago
Stack Overflow seems mostly self-moderating to me. The vast majority of moderator actions are on posts that have already been downvoted and reported by regular users. SO could automate this human intervention. A moderator already isn't required for a post to be closed, it just takes 3 close votes from normal users. So then what is the purpose of the mods? They seem to prefer spending their time generating a constant stream of metadrama that has little to do with SO itself. See Monica, the code of conduct changes, modifying the design of the upvote/downvote buttons, Chat GPT... I think SO needs to scrap the mods and replace them with a small number of professional mods who they pay.
JdeBP 3y ago
It has been interesting seeing the effects. * The most apparent immediate effect was on Ask Ubuntu, where the diamond moderators no longer handling spam with their tools that can delete it immediately meant that the ordinary moderators had to handle it the longer and harder way with accumulated votes to close. There was a lot more off-topic advertising staying visible for longer almost as soon as the strike began. The difference was noticeable on the first day. As I write this 5 out of the top 10 active Q&As are egregious spam. This is higher visibility than before the strike. * The effects at SuperUser and AskDifferent have been less prominent than at Ask Ubuntu. * As I have observed elsewhere, the brakes are off at SQA stackexchange, with the 2 prolific ChatGPT regurgitators now going into open competition with each other. * Conversely, although there are localized cases like SQA stackexchange, there doesn't appear to have been a widespread gold rush to post machine-written answers now that it is public that they aren't going to be zapped and their posters aren't going to be suspended. There are several things that clearly a lot of people commenting here don't know: * There is a difference between a moderator and a diamond moderator on Stack Exchange. ** Pretty much all of the grumbles in the past 2 hours of this discussion have nothing to do with diamond moderators, yet it is the diamond moderators that have the access to the confidential moderators-only discussion forum, that were subject to the secret policy directive sent out therein, that can issue account suspensions, and who are the major participants in the strike. ** Everyone with even a modicum of points is a moderator. This point has been long-touted in the introduction pages to the sites, and was a selling-point of the system in the first place. Ironically, diamond moderators often don't partake in the actions that are being once again grumbled about in a Hacker News submission that has the words "Stack Overflow" in its headline, because adding the diamond moderator flag to their accounts turns their tools into supervotes, and they actually get feedback and advice to not do the things that ordinary people do. If you've seen comments on a question or downvotes, or votes to close as duplicates, these are from the hundreds of thousands of people with moderation capabilities on the various sites, almost always not from the people who have gone on strike here. ** Ironically, the people who have gone on strike did so because they were given secret instructions that, had they enforced them, would have made it seem that they weren't following the published policies and were not doing their jobs. * There are 181 Stack Exchange sites. This is a general strike. ** Some of the people involved, particularly the ones from Academia stackexchange, object to ChatGPT in very strong terms indeed. One Academia SE person in discussions has repeatedly made the point that questions and answers in that arena are often life-changing, and machine-written answers have far more frequent potential to be disastrous. ** Yes, jlericson is also at fault for misrepresenting things, calling this a "Stack Overflow strike" when it is more than that. It's a catchy click-bait title. But although it garners the attention of more people than "Stack Exchange" probably would, it conversely hides the fact that some of the smaller SE sites have different problems with ChatGPT. "But the code worked after I fixed the obvious problems! So how is this answer a problem?" is a meaningless counterargument to machine-written answer objections on many of the sites, which have nothing to do with computer code.
bazoom42 3y ago
I’m surprised by the negativity towards SO. I’m relieved when google gives me a SO hit for a question, because in my experience it has the best answers, often better than official documentation. I’ve asked about 25 questions and never had one closed. A few didn’t get a useful answer but most gave very helpful answers. In one case an answer wrote 50 lines of code which would have taken me days to figure out. Maybe my experience is uncommon?

64 points by jlericson 3y ago

Top 0.05% here. I still haven’t seen any written evidence for the policy change being protested (the protest site says they are protesting a “near-total prohibition on moderating AI-generated content”, it seems incredible that StackOverflow would ever prohibit moderation of any type of content) and when I’ve asked I’ve been told it was discussed in secret so nobody can prove it. What I suspect is that somebody has asked the site to use automatic GPT removal tools and the site has said they don’t work, which is true.

creativenolo 3y ago
Yeah, given ChatGPT is most likely trained on SO a simple test would be to use the automatic tools on all of the pre-ChatGPT SO. My bet is, much of pre-ChatGPT SO would be removed.
MikeDelta 3y ago
A teacher thought he figured it out and asked ChatGPT if it wrote the text that his students handed in. ChatGPT said yes for every piece of work, so he failed the whole class. Apparently it didn't write it. Turns out ChatGPT thinks it wrote a lot of other things like the email that the teacher wrote to the class to sack them, when one of the students asked ChatGPT as proof of fiction. Not sure how it finally ended. https://news.ycombinator.com/item?id=35963163 https://news.ycombinator.com/item?id=35963163

fabian2k 3y ago

https://meta.stackexchange.com/questions/389928/gpt-on-the-platform-data-actions-and-outcomes https://meta.stackexchange.com/questions/389928/gpt-on-the-p... This post shows the data that SE finally provided, though only after the strike was already happening. So far the community is not convinced. It looks like there is something happening there that reduced the number of people that provide multiple answers more than expected, but not everybody is convinced it is the mod actions on ChatGPT posts. Another issue might be that this only explains a part of the decline in questions and answers. So even if this part is solved, the larger problem of declining engagement remains.

albert_e 3y ago
Slightly off topic possibly -- but I personally felt overwhelmed in recent months due to the daily onslaught of new GPT/LLM/GenAI related news. I work in a related tech area and I have been meaning to do two things (1) prusue recertifications on a couple of professional certs that are up for renewal later this year and (2) get hands-on experience with some of the LLM and GenAI tech that is coming out so I can either broaden my professional skills to include them or at least benefit from knowing those tools thrugh inproved productivity/creativity. What actually happened is that I achieved much less -- virtually zero -- tangible progress on both these goals. I think some of my general interests and pursuits have similarly suffered due to this indecisiveness and overload. I may just be blaming some external factors for my laziness or ADHD but I cant escape the feeling that I am in a more confusing world now. Why am I saying this here -- Some of the trends in user behaviour that these massive tech-oriented community sites are seeing over this period could have external factors like those I described above. Maybe people are being drawn elsewhere. Maybe people are overwhelmend, confused, or doing something else, maybe participating more in SD / MidJourney Discords. Assuming that their website and traffic is a closed box and only the variables they change will affect the outcomes would be naiive.
starball-tgz 3y ago
that "something" is pretty obvious: people are asking their questions to ChatGPT. Fewer questions -> fewer answers. I've done queries into the Stack Exchange network's public data dump and put my analysis here: https://meta.stackexchange.com/a/389960/997587 https://meta.stackexchange.com/a/389960/997587

dvt 3y ago

Stack Overflow has absolutely been destroyed by the moderators, so I couldn't care less about their "strike." I was (and still am) in the top ~%0.80 of users[1] but no longer contribute to the site (I stopped ~6 years ago) because of the moderators (a few in particular, but I won't name any names as some of them are here on HN). It has been an absolute shitshow of closing questions that shouldn't be closed, anally-retentive nitpicks which intimidate new users, the essential nuking of the community wiki (even prior to the official deprecation), bad answers being upvoted, good answers being deleted, and so on. The whole "community moderator" thing ended up being a popularity contest where typical nitwitted social climbers ended up injecting themselves in every single minor conflict on the site just to score visibility points come community voting time. On top of this, SO is also dying as it has no real viable way of cleaning up or deprecating old answers, and if new ones are asked, they are closed in favor of the old (outdated) ones. Slowly, reddit and language forums/mailing lists are becoming more and more valuable as Stack Overflow becomes more and more of a trash heap. It sucks because I really really loved Stack Overflow, but it just broke my heart one too many times. [1] https://stackoverflow.com/users/243613/david-titarenco https://stackoverflow.com/users/243613/david-titarenco

muppetman 3y ago
I'll say. I'm too scared to use it because every legit question I've seen is shut down by a mod with bonkers reasoning
- _nalply 3y ago
  No need. ChatGPT to the rescue. The answer quality is about the same but for different reasons. It's a hit or miss for both. But! With ChatGPT there is no social pressure exerted by humans. It's only you and the machine. It's liberating.
  - joebiden2 3y ago
    Don't know why this gets downvoted. It is simply true. In most cases, ChatGPT4 gets it right. You have to check anyway.

ftxbro 3y ago

I'm on the internet a lot but I don't know, I feel like it would be almost embarrassing to be an unpaid moderator of a site like that. Then to 'strike' like they are flexing their unpaid power on some company that doesn't care about them ... I don't know it seems like a lack of self respect. Why don't they quit moderating and do something else, it would seem more dignified.

emodendroket 3y ago
The entire company is structured around unpaid labor so they might care if it all stops.

deeviant 3y ago

I have not used SO once since chatgpt 4 came out, not once. I'm sure there are cases there SO has better responses than GPT, but frankly, they are rare, and the ability to query and shape chatGPTs responses basically obsoletes the SO model overnight. So the SO CEO is absolutely correct to pivot to AI before that ship sinks.

bombolo 3y ago
Well chatgpt is trained on SO. If people stop using SO, chatgpt will not be able to respond about newer languages/frameworks.
- deeviant 3y ago
  Chatgpt is trained on the internet, off which SO is part of it. It's also trained off interactions with it's users, which would eventually take over sources like SO.
  - bombolo 3y ago
    You think people will start to teach chatgpt when it can't answer?
AnonCoward42 3y ago
Isn't ChatGPT also using SO? If SO would stop and there would be no replacement, then the answers might get worse. My take is that these two currently complement. In my opinion the issue with SO and with most social media is the obsession with metrics. It would be nice if they took more of an HN approach to metrics (votes particularly).
- arp242 3y ago
  It's pretty neat when you get upvotes on answers you made months or years ago; a substantial number of them represent someone being helped out by your answer. I think that's pretty neat. For years the open source tool I spent the most time on was one I only very occasionally used myself, but got a lot of feedback and questions. Because this was a non-technical tool most of it was over email, and translated in fairly few "stars" (although I did migrate to GitHub quite late). All of this makes it rather different from voting on HN.

gregsher88 3y ago

I submitted a question regarding an obscure realmd bug and subsequently submitted the answer (which took about 24 hours to arrive at). The answer was nitpicked over a tag, downvoted, and finally removed. I would not be surprised if someone in the world is currently struggling to resolve the same realmd issue as it is not documented anywhere else online. That was the last time I will contribute to Stack Overflow.

deleted 3y ago
[deleted]
eyelidlessness 3y ago
That’s pretty par for the course for the last few years, and I don’t even bother looking there for answers much anymore for the same reason.
jsnell 3y ago
Does this have something to do with the strike? (E.g. you think the moderation is still ongoing despite the supposed strike, or worse than before?) Or is this just a generic SO complaint?
- ftxbro 3y ago
  I feel like you are itching to get his hacker news answer nitpicked over its relevance, downvoted, and finally removed.
  - deleted 3y ago
    [deleted]
  - jsnell 3y ago
    I mean, yes. Wouldn't it be pretty nice for the top comment to be relevant to the article rather than the 500th rerun of the same SO moderation complaints?
    - ricardo81 3y ago

nomilk 3y ago

Seems like a massive non-event. Why should it matter if an otherwise good answer came from an LLM? Presumably no one is bothered if a human author is aided by a search engine, tech blogs, their own intuition and experimentation, or some guesswork - so long as they submit a quality answer; so why should it be any different for answers aided by LLMs - are they supposed to be automatically worse? The sole determinate of whether an answer should be allowed is its quality (i.e. usefulness to people accessing the site who are trying to solve their programming problems). An answer's origins are irrelevant (except for the purpose of attribution). StackOverflow's voting system already does an excellent job of getting the best answers to the top and down-voted ones to the bottom. Further, high-rep users have the ability to close/delete very bad answers. Protesting LLMs seems like not much more of an extension on the idea of protesting autocorrect.

deleted 3y ago
[deleted]
vintermann 3y ago
> Why should it matter if an otherwise good answer came from an LLM? Because good as LLMs may be at writing good answers, they're even better at writing good-looking answers. That's pretty much what they're been trained to do, and actually good answers only appear as a fortunate side effect. Obviously, this makes moderators' jobs a lot harder. I can understand that they get frustrated. It also allows automated climbing in SO's gamified status hierarchy, which makes it even more dysfunctional and threatens those currently on top of it. So they want to be able to ban posts for looking automated. It's just that whether you decide manually, or with an automated tool of your own, you're going to have a huge false positive rate and quite possibly unfortunate biases in it as well. (Very recently there was a paper showing how automated LM detectors were great at sussing out non-native English speakers and deciding they were bots).
- wilg 3y ago
  I agree with the rest of your comment, but it's framed as a retort but doesn't really work as one. The person your responding to already said "otherwise good answer" which excludes a wrong but "good-looking" one.

londgine 3y ago

This article has some useful information, but I read the petition first, and it was very hard to understand what they were talking about. My tl;dr is: 1: chatgpt became an easy way to provide mostly correct answers that sound completely correct. 2: mods got annoyed that many incorrect answers were being published, so they made a bot that would identify and delete answers written by chatgpt. 3: As anyone who asked a question to stack overflow, and had it removed as a duplicate, knows, the mod's bots have a very high false positive rate. 4: The admins didn't like that answers from actual humans were being deleted that often, so they forbid the mods from autodeleting based on their tool. 5: some mods are angry at that decision, so they won't be moderating until the admins change that policy. In this situation (as opposed to the Reddit situation), I am all for the admins. I am unable to ask any question in stack overflow without it being removed as a duplicate (even if I clearly state why the other answer is irrelevant to my question). If the AI detecting bot has even a remotely similar false positive rate as the duplication bot, then I am glad that it is not allowed. This would discourage many people from not only asking questions, but also answering.

fabian2k 3y ago
There is no bot. This is entirely about human moderators acting on posts they consider AI-generated.

mbork_pl 3y ago

To provide a different perspective: I very rarely asked questions on SO, but I was a pretty active member (submitting both questions and answers) at tex.stackexchange.com a few years ago. It was a very, very nice and welcoming community (and probably still is, it's just that I moved on and don't participate that much anymore for personal/professional reasons). I am almost sure the same goes for emacs.stackexchange.com (which I sometimes read) and a few other sites in the "family". So I would not conflate SO with the wider SE set of communities.

xcdzvyn 3y ago
Aviation Stack Exchange is lovely too! :)
JdeBP 3y ago
It's important to remember that there are 181 Stack Exchange sites. It's important to not paint them with the same brush. It's important to remember, conversely, that this is a general strike, and that diamond moderators from the 180 others are involved. It's also important to remember that the 180 other sites are suffering from ChatGPT-created content as well. Sometimes it's better, in that there's almost no-one copying machine-written answers in the first place. Sometimes it's worse, in that now the brakes are very much off. The SQA stackexchange site, for example, turns out to have 2 fairly prolific ChatGPT copy-and-pasters that have been discussed on its Meta site, and who have histories of soliciting business for their respective companies in answers. Since the inception of the diamond moderator strike, they have gone into out-and-out open competition, posting ChatGPT answers competing with each other to years-old questions that actually have human-written answers of long standing. The RetroComputing stackexchange site, for example, appears to have no copies and pastes of ChatGPT content at all over the past few months.
Fomite 3y ago
I eventually got burnt out of answering there, but I found Academia StackExchange pleasant, and CrossValidated is in my dissertation acknowledgements.

dmingod666 3y ago

StackOverflow is toxic and if I don't have to deal with that it's a plus in my book. Too much drama on for my liking. Finding content is harder too, go through a Google search and 5-6 tabs before you reach an answer, ChatGPT is my first go-to tool for solving common stuff and stuff like syntax reference.

bandrami 3y ago

SE has been more or less useless at least for me for about a decade, mostly because of completely overzealous de-duping moderation. It doesn't help that I'm mostly on Server Fault, where small differences in versions and combinations of services can make a huge difference in what the right answer is. But if the question is "how to get nginx to pull auth from two different databases" the answer can be very different based on 1. the version of nginx you're talking about and 2. the specific databases you are using. But a mod sees "nginx auth multiple db" and deletes it as a duplicate.

munimbiz 3y ago
[flagged]
starball-tgz 3y ago
In those cases where things are version sensitive, it's useful to put version info in the title. That way _everyone_ will know what version the question is about- people searching for answers, and people trying to curate the knowledge-base.

dataengineer56 3y ago

Stack Overflow seems mostly self-moderating to me. The vast majority of moderator actions are on posts that have already been downvoted and reported by regular users. SO could automate this human intervention. A moderator already isn't required for a post to be closed, it just takes 3 close votes from normal users. So then what is the purpose of the mods? They seem to prefer spending their time generating a constant stream of metadrama that has little to do with SO itself. See Monica, the code of conduct changes, modifying the design of the upvote/downvote buttons, Chat GPT... I think SO needs to scrap the mods and replace them with a small number of professional mods who they pay.

JdeBP 3y ago

It has been interesting seeing the effects. * The most apparent immediate effect was on Ask Ubuntu, where the diamond moderators no longer handling spam with their tools that can delete it immediately meant that the ordinary moderators had to handle it the longer and harder way with accumulated votes to close. There was a lot more off-topic advertising staying visible for longer almost as soon as the strike began. The difference was noticeable on the first day. As I write this 5 out of the top 10 active Q&As are egregious spam. This is higher visibility than before the strike. * The effects at SuperUser and AskDifferent have been less prominent than at Ask Ubuntu. * As I have observed elsewhere, the brakes are off at SQA stackexchange, with the 2 prolific ChatGPT regurgitators now going into open competition with each other. * Conversely, although there are localized cases like SQA stackexchange, there doesn't appear to have been a widespread gold rush to post machine-written answers now that it is public that they aren't going to be zapped and their posters aren't going to be suspended. There are several things that clearly a lot of people commenting here don't know: * There is a difference between a moderator and a diamond moderator on Stack Exchange. ** Pretty much all of the grumbles in the past 2 hours of this discussion have nothing to do with diamond moderators, yet it is the diamond moderators that have the access to the confidential moderators-only discussion forum, that were subject to the secret policy directive sent out therein, that can issue account suspensions, and who are the major participants in the strike. ** Everyone with even a modicum of points is a moderator. This point has been long-touted in the introduction pages to the sites, and was a selling-point of the system in the first place. Ironically, diamond moderators often don't partake in the actions that are being once again grumbled about in a Hacker News submission that has the words "Stack Overflow" in its headline, because adding the diamond moderator flag to their accounts turns their tools into supervotes, and they actually get feedback and advice to not do the things that ordinary people do. If you've seen comments on a question or downvotes, or votes to close as duplicates, these are from the hundreds of thousands of people with moderation capabilities on the various sites, almost always not from the people who have gone on strike here. ** Ironically, the people who have gone on strike did so because they were given secret instructions that, had they enforced them, would have made it seem that they weren't following the published policies and were not doing their jobs. * There are 181 Stack Exchange sites. This is a general strike. ** Some of the people involved, particularly the ones from Academia stackexchange, object to ChatGPT in very strong terms indeed. One Academia SE person in discussions has repeatedly made the point that questions and answers in that arena are often life-changing, and machine-written answers have far more frequent potential to be disastrous. ** Yes, jlericson is also at fault for misrepresenting things, calling this a "Stack Overflow strike" when it is more than that. It's a catchy click-bait title. But although it garners the attention of more people than "Stack Exchange" probably would, it conversely hides the fact that some of the smaller SE sites have different problems with ChatGPT. "But the code worked after I fixed the obvious problems! So how is this answer a problem?" is a meaningless counterargument to machine-written answer objections on many of the sites, which have nothing to do with computer code.

bazoom42 3y ago

I’m surprised by the negativity towards SO. I’m relieved when google gives me a SO hit for a question, because in my experience it has the best answers, often better than official documentation. I’ve asked about 25 questions and never had one closed. A few didn’t get a useful answer but most gave very helpful answers. In one case an answer wrote 50 lines of code which would have taken me days to figure out. Maybe my experience is uncommon?