9 ms·

StackOverflow petition to allow removing AI generated content

127 points by miohtama 3y ago

deepnotderp 3y ago
“Let’s ban computers on a forum about computers”
- _jab 3y ago
  I mean, would you want HN to be completely overrun by bots?
  - eimrine 3y ago
    HN is for chatting but SO is for storing answers. Why moderators can not de-hallicinate the answers? Probably it needs some time before SO will became the only source of autogenerated answers on SO.
    - EGreg 3y ago
      Why can’t moderators on HN deal with a torrent of AI generated comment? Mission has been f**ing accomplished and now we will be enjoyin the xkcd dystopia: https://xkcd.com/810/ https://xkcd.com/810/
      - pixl97 3y ago
        Imagine the AI dystopia being like the entire Earth. We are only about a few shovel fulls deep. But don't worry, tech companies are spending billions of dollars on tools to dig even faster. Buckle up, because it's going to be a wild ride.
  - cameronfraser 3y ago
    depends if the answers are better than the ones by humans
  - digitalsushi 3y ago
    that's fascinating. if i am here as a technical resource, i dont care if i am here to make connections and maybe even, every 10 years, a new friend, then absolutely not. so the challenge to the reply to my post, is to say something that disproves i come here for connections and friends.
  - ldehaan 3y ago
    [dead]
- JdeBP 3y ago
  There are 181 Stack Exchange sites. There are fewer than 10 that I have found, in years of using the place, that are about computers.
noirscape 3y ago
Not surprised - the universal issue with AI generated content is that it's near impossible to curate (not helped by what we'll call... a certain eagerness of its advocates to share the output of these tools) and often is wrong in imperceptibly tiny ways.
isoprophlex 3y ago
I don't understand why you would, for the forseeable future, want to allow AI generated content on SO. Not with the current state of the art in generative AI. If I want an AI generated answer, with all the pros and cons specific to LLMs, I'll just open chatgpt or turn on copilot... SO answers (used to be) in an entirely different league in terms of trustworthiness. I'm totally on board with this moderator strike. Edit: it appears there is a very high false positive rate, making it difficult to distinguish between some answers. So, there's that to consider...
- lordnacho 3y ago
  If someone has already put in the right prompt to get the best answer, you will now not have to do that. Also, SO isn't just about getting the right answer, it's also about finding the pros and cons of each proposed solution.
- rafark 3y ago
  If the code works just fine, why remove it? For small algorithms it should be good enough.
  - NotTheDr01ds 3y ago
    "Code working" isn't necessarily black-and-white. For a new user (the one asking the question), the code may appear to solve the problem, but may have corner-cases or even security risks. That's entirely possible with user-generated code as well, of course, but GPT/AI allows it to be produced at a much higher rate, with the person who posted the answer often not being capable of (or not caring to) validate or correct it.
    - Kuinox 3y ago
      Yes, and SO already have plentiful of sample code that appear to solve the problem, but have huge flaws.
- rich_sasha 3y ago
  There could still be value, if ChatGPT is right 60% of the time, there is still value in someone filtering that down to the correct answers.
koheripbal 3y ago
Why? If the answers are good, leave them.
- isoprophlex 3y ago
  That's the thing, A) they might not be, B) once we start mixing human answers with AI on SO, it's value as a data source for future LLM train runs will decrease. The tech is moving at breakneck pace, and in general I'm loving it. But let's not hurry the adoption of AI too much
  - 8note 3y ago
    For B, SO still has voting on answers as well as acceptance of answers It's not as valuable as raw text input, but it's still marked up and classified
    - jatins 3y ago
      > SO still has voting on answers We call it RLHF these days
    - shagie 3y ago
      Consider the answer to the question "how often do people down vote incorrect information?" While SO still has voting, the value of a vote in terms of identifying correct information has fallen off substantially in the past several years. Poorly written questions and incorrect answers are more likely to get an upvote for trying than a down vote for what's actually written.
- kristopolous 3y ago
  Because the system isn't confirming the answers. Let's assume we have the final form AGI. If it doesn't confirm the claims are accurate then it falls under the same flaw as humans who don't confirm. If it's a bunch of untested, unconfirmed results that are statistically pretty decent, this doesn't really reduce the amount of work. That is the same misplaced enthusiasm that developers tend to do when they fetishize new tools or languages which actually make their job more complicated and time consuming but occupy the time in a way that gives illusory optics to the contrary. The fact that most devs get fooled by such productivity illusions are why 10x engineer are a real thing.
dlivingston 3y ago
What's the motivation for Stack Overflow to allow AI-generated content in the first place? These AI models were likely heavily trained on SO data, so any LLM-based answer is merely a regurgitation of thousands of human answers before it. In addition, asking a question on Stack Overflow and having some LLM respond seems to me like the equivalent of asking GPT-X.Y directly, albeit with extra steps.
- cosmolev 3y ago
  There is a possibility that everything we output is merely a regurgitation of thousands of human answers, ideas, and thoughts we have encountered before. Including this comment of yours. And mine.
  - endisneigh 3y ago
    I hate this sort of retort since it’s fundamentally meaningless. All atoms that will ever exist were in the singularity and exploded during the big bang to encompass all of the universe, and so we are all just moving along. Ok, so what.
    - deleted 3y ago
      [deleted]
    - Ukv 3y ago
      The point of the retort is precisely to point out the meaninglessness when "regurgitation" is extended to include systems with some degree of generalization/extrapolation/discovery capabilities. The expectation set on AI seems to be to break information theory, "it's not original because it can only draw a horse (more accurately than a random guess) due to having trained on information about a horse" or so on.
  - dvt 3y ago
    There's this new trend of what I'll dub techno-nihilism, which is essentially a counterargument to the stochastic parrot argument. The former being: well what if WE are stochastic parrots, after all that's how we learn, right? Well yes, but actually no. It's trivially false because ChatGPT was trained on something (in this case, Stack Overflow), which, in turn was trained on something else (maybe a book), and so on. So knowledge, imagination, and genuine creativity must exist somewehere down that chain. Everything can't be just repeating what was learned prior ad infinitum, or we'd have nothing new. Ironically, even the development of large language models is an exercise in creativity.
softwaredoug 3y ago
Can answers be edited by the community to correct any hallucinations? Or is there just an outright ban on removing AI content?
- Ukv 3y ago
  There's no ban on removing AI content as far as I'm aware - admins just upped the evidence requirement for determining a post as AI-generated, seemingly in response to high false positive rates of hunches and current detectors: > We recently performed a set of analyses on the current approach to AI-generated content moderation. The conclusions of these analyses strongly indicate to us that AI-generated content is not being properly identified across the network, and that the potential for false-positives is very high. > In order to help mitigate the issue, we've asked moderators to apply a very strict standard of evidence to determining whether a post is AI-authored when deciding to suspend a user. > We've reminded moderators that suspensions (and typically mod messages as well) are for real, verifiable malfeasance only, and should not be enacted on the basis of hunches, guesses, intuition, or unverified heuristics https://meta.stackexchange.com/questions/389582/what-is-the-network-policy-regarding-ai-generated-content https://meta.stackexchange.com/questions/389582/what-is-the-...
  - fabian2k 3y ago
    This is essentially a ban on removing AI content, the cases where it is still allowed to remove the posts are exceedingly rare. This is rather confusing as the public policy doesn't contain those details. I'm a moderator on a small SE site, so I have seen the internal communication between SE and the mods on this.
    - Ukv 3y ago
      > This is essentially a ban on removing AI content I think framing it as "a ban on removing AI content" is wrong/misleading and the reason for softwaredoug's question about whether you'd even be allowed to edit low-quality hallucinations. To my understanding you can still remove/edit/ban for all the same reasons as normal content, just that the additional reason (removing for inherently being AI generated) is now significantly stricter. > the cases where it is still allowed to remove the posts are exceedingly rare. Not aware of the internal guidelines unless there's something you're allowed to share, but generally I could understand the admins' desire to err towards high-precision over high-recall.
EGreg 3y ago
So it has been only a couple months and AI spam is causing problems for submissions in communities from StackOverflow to Sci-Fi publishing to photo contests. The AIs can overrun all the CAPTCHAs too. So, this is just the beginning. What will happen next year on HN? Anyone can deploy a swarm of bots
- dlivingston 3y ago
  With generative AI (text, images, videos), I'm of the opinion that very soon -- if not now -- every"one" on the internet should be assumed a bot until proven otherwise. In other words, it's now trivial to fake a digital persona, including "selfies", "personalities", and more. How do you know I'm not an LLM?
- lamp987 3y ago
  dead internet will become real
  - pixl97 3y ago
    It was a misnomer calling it 'dead internet theory', Thoughty2 correctly renames it 'dead internet prophecy'.
skilled 3y ago
It is most certainly a predicament. If people feel like they can freely post ChatGPT content, it adds enormous strain on making sure that the solution provided actually works. This then has to be done by moderators (with experience) or other users who land on the site. If it is the latter, people might get frustrated that answers are wrong, devaluing SO's value overall. In a nutshell, allowing AI in its current iteration will create more problems when it comes to moderation, content quality, and the sanity of sites users - pretty much. Quite a lot of variables to find common ground on this one, but I think StackOverflow should not allow AI content for the time being.
- aarong11 3y ago
  I mean people post wrong answers to stackoverflow all the time. They generally get downvoted.
- pixl97 3y ago
  So, prove it's AI content, that's going to be the hard part with a ton of false negatives. I look at it the other way, AI answers should be fine where tools like Code Interpreter are in use and they can provide a verifiable solution. At the end of the day people copying from SO are generally looking for something that works, and the problems of this being best practice can be fought out by the people down thread.
PeterStuer 3y ago
Who cares about provenance? If the answer is usefull it is. Don't think the "humans" post bogus "quick google search" answers on SO for rep farming? Have you accessed the site in the last 5 years?
w0ts0n 3y ago
We talk about job disruption, GPT has SO's number. They should have never shut their job board. They have painted themselves into a tiny corner now and OpenAI + others are coming for their lunch.
- gtirloni 3y ago
  LLMs will usually lag a lot though. Often a human has more up to date knowledge. Computers being such an active and ever changing field, that matters. OpenAI should want people to keep contributing original answers to feed its AI.
crazygringo 3y ago
Either answers are good or not. It doesn't matter if they're generated by a 13-year-old in their bedroom, someone studying CS at university, a well-respected IC at a top tech company... or an AI. If answers are good, keep them. If they're bad, downvote them. If they're redundant or off-topic or gibberish, delete them. And to those asking why you would ever want AI-generated content on StackOverflow when you could just go to ChatGPT/Copilot/etc... it's because of all of the commentary. People are giving different answers, arguing the pros and cons, pointing out errors... all of the discussion around a StackOverflow question is usually just as valuable as any given answer, if not more so. Of course, I understand that AI-powered accounts that can ask/answer hundreds of questions a minute are a problem simply because moderation can't keep up with them. But StackOverflow already has a lot of limits just for human accounts -- lots of actions you can't take until you've contributed certain amounts of value. Just extend these protections to do things like rate-limiting and so forth, to ensure that the normal ratio of content submitted vs. moderated stays constant and manageable.
- raman162 3y ago
  Agreed. I have found myself sometimes asking Chat-GPT for guidance related to obscure error messages and occasionally its more useful than google or stack overflow.
  - rafark 3y ago
    When it doesn’t hallucinate, which in my case it’s most of the time.
    - NotTheDr01ds 3y ago
      Sure, but if you try it out, you pretty quickly realize it's a hallucination. Unfortunately the type of GPT content we're now getting on Stack Overflow and its sibling sites is mostly unvalidated GPT hallucinations.
- nologic01 3y ago
  > If answers are good, keep them. If they're bad, downvote them. If they're redundant or off-topic or gibberish, delete them. Yeah, lets us keep providing free labor to help train somebody else's models, improve somebody else's infrastructure so that they can even more effectively dominate.
deltree7 3y ago
[dead]
mypastself 3y ago
The current title is “StackOverflow petition to allow remove AI generated content”, which sounds… uh… AI-generated.
softwaredoug 3y ago
Before jumping to conclusions, be sure to check out this context: https://meta.stackexchange.com/questions/389582/what-is-the-network-policy-regarding-ai-generated-content https://meta.stackexchange.com/questions/389582/what-is-the-... TL; DR - the policy is “we can’t tell if something is AI generated, so hard to justify removal on just that basis.” > We recently performed a set of analyses on the current approach to AI-generated content moderation. The conclusions of these analyses strongly indicate to us that AI-generated content is not being properly identified across the network, and that the potential for false-positives is very high. Through no fault of moderators' own, we also suspect that there have been biases for or against residents of specific countries as a potential result of the heuristics being applied to these posts. Finally, internal evidence strongly suggests that the overapplication of suspensions for AI-generated content may be turning away a large number of legitimate contributors to the site.
- minimaxir 3y ago
  This follows from the recent case where Texas A&M students were falsely accused of using ChatGPT and subsequently failed, which caused a ton of negative publicity: https://www.rollingstone.com/culture/culture-features/texas-am-chatgpt-ai-professor-flunks-students-false-claims-1234736601/ https://www.rollingstone.com/culture/culture-features/texas-...
- whatyesaid 3y ago
  I would've thought the problem is bots that are mass solving instead of human + ChatGPT. If a bot mass solves you can identify it by posting frequency and quality over time.
- fabian2k 3y ago
  SE has not provided the data here, and what they communicated is a mess. Some of their main arguments are rather dubious and didn't convince the mods. Keep in mind that moderators have more than just the pure content of the answers available. This is usually not about a single post, but many of them.
tortoise_in 3y ago
I would go with only human code should exist.
- pixl97 3y ago
  And how do you prove it?
raincole 3y ago
I'd propose a different solution. For every single new question, auto-generate an answer with GPT and mark it as AI-generated.
- Me1000 3y ago
  At least the generated responses won’t start by asking me “why in the world would you want to do that?”
  - mediumsmart 3y ago
    You nailed it.
whatyesaid 3y ago
AI generated can be somewhat sniffed out today but I'm not sure how long that will even last. Usually you can tell when the writing is great but the points made are trash or miss the point. I personally wouldn't find it helpful. I'd say let the question poster and visitor decide if they want to see AI generated stuff. But that does rely on AI answers being marked as such or detected.
Veen 3y ago
People here are arguing about whether it’s ok to have AI-generated content on StackOverflow or not. But it seems to me that’s not the issue. The real issue is that people suck at identifying AI content, and so do AI detectors. So moderation based on that identification is obviously going to be unfair and inaccurate. Moderators are removing perfectly acceptable human-written answers based on their spurious intuition that it’s AI generated, and they are striking because they want to go on doing so. It doesn’t matter how confident moderators are that they can identity AI content when the correlation between their confidence and reality is so low.
- wordcloudsare 3y ago
  Any one tool is bad at detecting AI but combining both human intuition and multiple automatic tools can get a very accurate result. Moderators already only make suspensions when multiple systems agree. Not to mention the best AI might be hard to distinguish but most AI content is quite obvious (when reading carefully) because how bad it is. It can trick a casual reader but not a experienced moderator who knows about the subject matter.
lukev 3y ago
This just makes obvious sense. If someone wants a GPT-generated answer, they can use ChatGPT themselves. If I'm bothering to post my question to a forum, the social contract is that I'd like a response from a human.
slowhadoken 3y ago
ChatGPT isn’t AI, I really wish people would stop calling it that. It tricks people into thinking something magical is happening. There is still only human generated content and ChatGPT scraped it.
- delecti 3y ago
  "AI" is a moving target. As soon as techniques under the banner of "AI" start being used, they get a specific name and people start saying "that's not AI". For now, LLMs "are AI", just like current text-to-image models (Stable Diffusion, Midjourney, etc). There's no winning a terminology war single-handed.
- pixl97 3y ago
  And this folks is the AI-Effect in actions. ChatGPT is AI. It is a human level general intelligence, no, and we're likely very lucky for that. Humanity is going to have to accept that the words we used to define AI were far too imprecise and don't reflect how development of AI is actually occurring. If you want to argue the language of it, then demand we make new, testable, and scientific definitions for the capabilities of computer intelligence. Any less, and all we'll hear is the sound of scraping goal posts as they get moved further and further.
adamgordonbell 3y ago
Anyone around who moderators on SO? Is there a general sense of alienation from corporate SO? My understanding is that in the early days, a lot of the devs at SO were actually recruited from the SO and Meta moderation userbase but probably that doesn't scale. Example: Ben: And then, April 29th, I was, again, sitting in the agency, doing my thing, and I also had my personal email open, and I suddenly got an email from Jeff Atwood. But it said, “Who should work for Stack Overflow?” That was the subject line, and then the body of the message said, “You should,” and that was basically it. And that was kind of like this jaw-dropping thing, because from that moment on, it was this, “Okay, maybe the skill that I have is actually marketable,” kind of moment. https://corecursive.com/stack-overflow/ https://corecursive.com/stack-overflow/
- wordcloudsare 3y ago
  Not a mod but mods have complained publicly about the sense of isolation for years
- NobodyNada 3y ago
  I used to be heavily active in curating SO, and yes, there is an incredibly strong sense of alienation from the corporation (which is what drove me away). In the old days, most of the staff, from devs to management to the CEO, were active users of the site and hung out on Meta and in chat. They were easy to reach and happy to answer questions; and whenever they made major changes to the site they would go to Meta to ask for feedback and adjust their plans accordingly. There was a noticeable shift in this dynamic starting around ~2016. Around this time, the company stopped focusing development effort on the core site functionality, and instead prioritized side products & attempts to monetize the site (most of which failed). Feature requests on Meta were almost completely ignored, and site features that had been on previously-announced timelines/roadmaps were never delivered. But the community was still as strong as ever; and so people started implementing these missing features themselves in the form of bots and userscripts. This was the "golden age" of moderation bots, and a really fun time to be a part of the community -- power users ran heavily modified frontends that could display all kinds of additional information and automate repetitive actions, and integrate with bots to do things that Stack Exchange's systems were bad at -- like flagging spam and low-quality posts; identifying plagiarism; and detecting flamewars in comments. As the company grew, they hired a ton of middle management who were not active participants in the site, and largely did not care about the day-to-day. This was alright when they left us alone, but around 2018-2019 they began to take an openly hostile stance towards the "power users". Here's an excellent post from that time summarizing the general sentiment: https://meta.stackexchange.com/questions/331513 https://meta.stackexchange.com/questions/331513 The short version is: the company began blaming power users for things like the site's "unwelcoming" reputation (which is really a symptom of the site's outdated and opaque moderation tooling, and power users had been clamoring for better tools for years). They began a pattern of rolling out features and UI changes that took major steps backwards in usability and accessibility -- and due to all the negative feedback these changes received on Meta, they announced staff would no longer participate on Meta because it was "too negative". A high-level manager famously quipped that the opinion of Meta was not relevant as it represented "0.015%" of Stack Exchange's userbase -- despite the fact that that 0.015% was responsible for the majority of content & moderation activity contributed to the site. In late 2019 it got a whole lot worse when, in rapid succession: 1) the company updated the site terms-of-service to illegally change the license of user-submitted content, and 2) an employee abruptly revoked a volunteer's moderation privileges without due process, and then went to news outlets making false accusations about that user's behavior (https://meta.stackexchange.com/questions/333965 https://meta.stackexchange.com/questions/333965). Shortly after that, Stack Overflow fired several well-loved and highly respected staff moderators, for undisclosed reasons (https://meta.stackexchange.com/questions/342039/ https://meta.stackexchange.com/questions/342039/). A lot of people, including myself, left the community in the wake of this. Since then -- at least from my outsider perspective of checking in once in a while to see what's going on -- it seemed for a while like the company was learning from its mistakes.They apologized for ignoring Meta, began asking for and listening to community feedback once again, created new policies to protect volunteers from the kind of abuse that happened in 2019, and began implementing some of those long-ignored and long-overdue feature requests. But in 2021 the company got bought out by a VC firm that is even more aggressive about trying to monetize the site; they started cranking up advertising and pushing generally unwanted side products, but they mostly left the community alone. That brings us to generative AI. As soon as ChatGPT came out, a deluge of users began copy/pasting Stack Overflow questions into ChatGPT and copy/pasting its answers into the answer box (usually with no editing or fact-checking effort). General consensus among the community seems to be that ChatGPT produces wrong or unhelpful information to an unacceptable degree, and that allowing machine-generated content on Stack Overflow defeats the purpose of the site (you go to ChatGPT if you want answers from a machine, but you go to SO if you want answers from a human). The staff supported this consensus and made it official policy -- but at the same time the CEO kept making rambling blog posts about how "AI is the future about Stack Overflow" and launching sweeping initiatives within the company to do...AI related things? Nobody really knows what he's talking about. That all leads up to the events that triggered strike: out of nowhere, the company suddenly announced a few days ago that it was overruling previous community consensus and prohibiting users from deleting content on the basis of being AI-generated.
moffkalast 3y ago
And they can tell it's AI generated... how?
- anaganisk 3y ago
  Possible AI generated, closed.
z3t4 3y ago
It's an ongoing joke about spam bots, SEO spam, etc, that the day they actually post helpful and useful content it's mission fucking accomplished. With "AI" we are one step closer.
JdeBP 3y ago
The thing that comes to mind when I hear about volunteers withholding their efforts is the "reinstate Monica" protest. I haven't counted how many years it has been, but there are still people with "reinstate Monica" in their Stack Exchange profiles to this day.
cosmolev 3y ago
I would be more afraid of AI generated questions rather than answers.
DoingIsLearning 3y ago
One problem with this is that future Models will probably be trained with data that includes a subset of AI generated data. As this ratio grows the need to classify AI/human becomes critical, otherwise all these models will probably degrade over time.
Dowwie 3y ago
Some people use their StackOverflow karma for social credit and portfolio. Their karma won't have the same value as it did before AI-generated responses were allowed. I think that comments should be approved but zero karma applied for providing the correct response using AI. There won't be an incentive to provide AI generated responses, but they won't be rejected either.
throw060223 3y ago
Oh StackOverflow. I forgot it existed since ChatGPT was released. Yeah.. surely a 2013 reply from a reputation farmer suggesting a dirty jQuery solution will be better, since it was made by a human. LOL
ldehaan 3y ago
[dead]

127 points by miohtama 3y ago

deepnotderp 3y ago

“Let’s ban computers on a forum about computers”

_jab 3y ago
I mean, would you want HN to be completely overrun by bots?
- eimrine 3y ago
  HN is for chatting but SO is for storing answers. Why moderators can not de-hallicinate the answers? Probably it needs some time before SO will became the only source of autogenerated answers on SO.
  - EGreg 3y ago
    Why can’t moderators on HN deal with a torrent of AI generated comment? Mission has been f**ing accomplished and now we will be enjoyin the xkcd dystopia: https://xkcd.com/810/ https://xkcd.com/810/
    - pixl97 3y ago
      Imagine the AI dystopia being like the entire Earth. We are only about a few shovel fulls deep. But don't worry, tech companies are spending billions of dollars on tools to dig even faster. Buckle up, because it's going to be a wild ride.
- cameronfraser 3y ago
  depends if the answers are better than the ones by humans
- digitalsushi 3y ago
  that's fascinating. if i am here as a technical resource, i dont care if i am here to make connections and maybe even, every 10 years, a new friend, then absolutely not. so the challenge to the reply to my post, is to say something that disproves i come here for connections and friends.
- ldehaan 3y ago
  [dead]
JdeBP 3y ago
There are 181 Stack Exchange sites. There are fewer than 10 that I have found, in years of using the place, that are about computers.

noirscape 3y ago

Not surprised - the universal issue with AI generated content is that it's near impossible to curate (not helped by what we'll call... a certain eagerness of its advocates to share the output of these tools) and often is wrong in imperceptibly tiny ways.

isoprophlex 3y ago

I don't understand why you would, for the forseeable future, want to allow AI generated content on SO. Not with the current state of the art in generative AI. If I want an AI generated answer, with all the pros and cons specific to LLMs, I'll just open chatgpt or turn on copilot... SO answers (used to be) in an entirely different league in terms of trustworthiness. I'm totally on board with this moderator strike. Edit: it appears there is a very high false positive rate, making it difficult to distinguish between some answers. So, there's that to consider...

lordnacho 3y ago
If someone has already put in the right prompt to get the best answer, you will now not have to do that. Also, SO isn't just about getting the right answer, it's also about finding the pros and cons of each proposed solution.
rafark 3y ago
If the code works just fine, why remove it? For small algorithms it should be good enough.
- NotTheDr01ds 3y ago
  "Code working" isn't necessarily black-and-white. For a new user (the one asking the question), the code may appear to solve the problem, but may have corner-cases or even security risks. That's entirely possible with user-generated code as well, of course, but GPT/AI allows it to be produced at a much higher rate, with the person who posted the answer often not being capable of (or not caring to) validate or correct it.
  - Kuinox 3y ago
    Yes, and SO already have plentiful of sample code that appear to solve the problem, but have huge flaws.
rich_sasha 3y ago
There could still be value, if ChatGPT is right 60% of the time, there is still value in someone filtering that down to the correct answers.

koheripbal 3y ago

Why? If the answers are good, leave them.

isoprophlex 3y ago
That's the thing, A) they might not be, B) once we start mixing human answers with AI on SO, it's value as a data source for future LLM train runs will decrease. The tech is moving at breakneck pace, and in general I'm loving it. But let's not hurry the adoption of AI too much
- 8note 3y ago
  For B, SO still has voting on answers as well as acceptance of answers It's not as valuable as raw text input, but it's still marked up and classified
  - jatins 3y ago
    > SO still has voting on answers We call it RLHF these days
  - shagie 3y ago
    Consider the answer to the question "how often do people down vote incorrect information?" While SO still has voting, the value of a vote in terms of identifying correct information has fallen off substantially in the past several years. Poorly written questions and incorrect answers are more likely to get an upvote for trying than a down vote for what's actually written.
kristopolous 3y ago
Because the system isn't confirming the answers. Let's assume we have the final form AGI. If it doesn't confirm the claims are accurate then it falls under the same flaw as humans who don't confirm. If it's a bunch of untested, unconfirmed results that are statistically pretty decent, this doesn't really reduce the amount of work. That is the same misplaced enthusiasm that developers tend to do when they fetishize new tools or languages which actually make their job more complicated and time consuming but occupy the time in a way that gives illusory optics to the contrary. The fact that most devs get fooled by such productivity illusions are why 10x engineer are a real thing.

dlivingston 3y ago

What's the motivation for Stack Overflow to allow AI-generated content in the first place? These AI models were likely heavily trained on SO data, so any LLM-based answer is merely a regurgitation of thousands of human answers before it. In addition, asking a question on Stack Overflow and having some LLM respond seems to me like the equivalent of asking GPT-X.Y directly, albeit with extra steps.

cosmolev 3y ago
There is a possibility that everything we output is merely a regurgitation of thousands of human answers, ideas, and thoughts we have encountered before. Including this comment of yours. And mine.
- endisneigh 3y ago
  I hate this sort of retort since it’s fundamentally meaningless. All atoms that will ever exist were in the singularity and exploded during the big bang to encompass all of the universe, and so we are all just moving along. Ok, so what.
  - deleted 3y ago
    [deleted]
  - Ukv 3y ago
    The point of the retort is precisely to point out the meaninglessness when "regurgitation" is extended to include systems with some degree of generalization/extrapolation/discovery capabilities. The expectation set on AI seems to be to break information theory, "it's not original because it can only draw a horse (more accurately than a random guess) due to having trained on information about a horse" or so on.
- dvt 3y ago
  There's this new trend of what I'll dub techno-nihilism, which is essentially a counterargument to the stochastic parrot argument. The former being: well what if WE are stochastic parrots, after all that's how we learn, right? Well yes, but actually no. It's trivially false because ChatGPT was trained on something (in this case, Stack Overflow), which, in turn was trained on something else (maybe a book), and so on. So knowledge, imagination, and genuine creativity must exist somewehere down that chain. Everything can't be just repeating what was learned prior ad infinitum, or we'd have nothing new. Ironically, even the development of large language models is an exercise in creativity.

softwaredoug 3y ago

Can answers be edited by the community to correct any hallucinations? Or is there just an outright ban on removing AI content?

Ukv 3y ago
There's no ban on removing AI content as far as I'm aware - admins just upped the evidence requirement for determining a post as AI-generated, seemingly in response to high false positive rates of hunches and current detectors: > We recently performed a set of analyses on the current approach to AI-generated content moderation. The conclusions of these analyses strongly indicate to us that AI-generated content is not being properly identified across the network, and that the potential for false-positives is very high. > In order to help mitigate the issue, we've asked moderators to apply a very strict standard of evidence to determining whether a post is AI-authored when deciding to suspend a user. > We've reminded moderators that suspensions (and typically mod messages as well) are for real, verifiable malfeasance only, and should not be enacted on the basis of hunches, guesses, intuition, or unverified heuristics https://meta.stackexchange.com/questions/389582/what-is-the-network-policy-regarding-ai-generated-content https://meta.stackexchange.com/questions/389582/what-is-the-...
- fabian2k 3y ago
  This is essentially a ban on removing AI content, the cases where it is still allowed to remove the posts are exceedingly rare. This is rather confusing as the public policy doesn't contain those details. I'm a moderator on a small SE site, so I have seen the internal communication between SE and the mods on this.
  - Ukv 3y ago
    > This is essentially a ban on removing AI content I think framing it as "a ban on removing AI content" is wrong/misleading and the reason for softwaredoug's question about whether you'd even be allowed to edit low-quality hallucinations. To my understanding you can still remove/edit/ban for all the same reasons as normal content, just that the additional reason (removing for inherently being AI generated) is now significantly stricter. > the cases where it is still allowed to remove the posts are exceedingly rare. Not aware of the internal guidelines unless there's something you're allowed to share, but generally I could understand the admins' desire to err towards high-precision over high-recall.

EGreg 3y ago

So it has been only a couple months and AI spam is causing problems for submissions in communities from StackOverflow to Sci-Fi publishing to photo contests. The AIs can overrun all the CAPTCHAs too. So, this is just the beginning. What will happen next year on HN? Anyone can deploy a swarm of bots

dlivingston 3y ago
With generative AI (text, images, videos), I'm of the opinion that very soon -- if not now -- every"one" on the internet should be assumed a bot until proven otherwise. In other words, it's now trivial to fake a digital persona, including "selfies", "personalities", and more. How do you know I'm not an LLM?
lamp987 3y ago
dead internet will become real
- pixl97 3y ago
  It was a misnomer calling it 'dead internet theory', Thoughty2 correctly renames it 'dead internet prophecy'.

skilled 3y ago

It is most certainly a predicament. If people feel like they can freely post ChatGPT content, it adds enormous strain on making sure that the solution provided actually works. This then has to be done by moderators (with experience) or other users who land on the site. If it is the latter, people might get frustrated that answers are wrong, devaluing SO's value overall. In a nutshell, allowing AI in its current iteration will create more problems when it comes to moderation, content quality, and the sanity of sites users - pretty much. Quite a lot of variables to find common ground on this one, but I think StackOverflow should not allow AI content for the time being.

aarong11 3y ago
I mean people post wrong answers to stackoverflow all the time. They generally get downvoted.
pixl97 3y ago
So, prove it's AI content, that's going to be the hard part with a ton of false negatives. I look at it the other way, AI answers should be fine where tools like Code Interpreter are in use and they can provide a verifiable solution. At the end of the day people copying from SO are generally looking for something that works, and the problems of this being best practice can be fought out by the people down thread.

PeterStuer 3y ago

Who cares about provenance? If the answer is usefull it is. Don't think the "humans" post bogus "quick google search" answers on SO for rep farming? Have you accessed the site in the last 5 years?

w0ts0n 3y ago

We talk about job disruption, GPT has SO's number. They should have never shut their job board. They have painted themselves into a tiny corner now and OpenAI + others are coming for their lunch.

gtirloni 3y ago
LLMs will usually lag a lot though. Often a human has more up to date knowledge. Computers being such an active and ever changing field, that matters. OpenAI should want people to keep contributing original answers to feed its AI.

crazygringo 3y ago

Either answers are good or not. It doesn't matter if they're generated by a 13-year-old in their bedroom, someone studying CS at university, a well-respected IC at a top tech company... or an AI. If answers are good, keep them. If they're bad, downvote them. If they're redundant or off-topic or gibberish, delete them. And to those asking why you would ever want AI-generated content on StackOverflow when you could just go to ChatGPT/Copilot/etc... it's because of all of the commentary. People are giving different answers, arguing the pros and cons, pointing out errors... all of the discussion around a StackOverflow question is usually just as valuable as any given answer, if not more so. Of course, I understand that AI-powered accounts that can ask/answer hundreds of questions a minute are a problem simply because moderation can't keep up with them. But StackOverflow already has a lot of limits just for human accounts -- lots of actions you can't take until you've contributed certain amounts of value. Just extend these protections to do things like rate-limiting and so forth, to ensure that the normal ratio of content submitted vs. moderated stays constant and manageable.

raman162 3y ago
Agreed. I have found myself sometimes asking Chat-GPT for guidance related to obscure error messages and occasionally its more useful than google or stack overflow.
- rafark 3y ago
  When it doesn’t hallucinate, which in my case it’s most of the time.
  - NotTheDr01ds 3y ago
    Sure, but if you try it out, you pretty quickly realize it's a hallucination. Unfortunately the type of GPT content we're now getting on Stack Overflow and its sibling sites is mostly unvalidated GPT hallucinations.
nologic01 3y ago
> If answers are good, keep them. If they're bad, downvote them. If they're redundant or off-topic or gibberish, delete them. Yeah, lets us keep providing free labor to help train somebody else's models, improve somebody else's infrastructure so that they can even more effectively dominate.

deltree7 3y ago

[dead]

mypastself 3y ago

The current title is “StackOverflow petition to allow remove AI generated content”, which sounds… uh… AI-generated.

softwaredoug 3y ago

Before jumping to conclusions, be sure to check out this context: https://meta.stackexchange.com/questions/389582/what-is-the-network-policy-regarding-ai-generated-content https://meta.stackexchange.com/questions/389582/what-is-the-... TL; DR - the policy is “we can’t tell if something is AI generated, so hard to justify removal on just that basis.” > We recently performed a set of analyses on the current approach to AI-generated content moderation. The conclusions of these analyses strongly indicate to us that AI-generated content is not being properly identified across the network, and that the potential for false-positives is very high. Through no fault of moderators' own, we also suspect that there have been biases for or against residents of specific countries as a potential result of the heuristics being applied to these posts. Finally, internal evidence strongly suggests that the overapplication of suspensions for AI-generated content may be turning away a large number of legitimate contributors to the site.

minimaxir 3y ago
This follows from the recent case where Texas A&M students were falsely accused of using ChatGPT and subsequently failed, which caused a ton of negative publicity: https://www.rollingstone.com/culture/culture-features/texas-am-chatgpt-ai-professor-flunks-students-false-claims-1234736601/ https://www.rollingstone.com/culture/culture-features/texas-...
whatyesaid 3y ago
I would've thought the problem is bots that are mass solving instead of human + ChatGPT. If a bot mass solves you can identify it by posting frequency and quality over time.
fabian2k 3y ago
SE has not provided the data here, and what they communicated is a mess. Some of their main arguments are rather dubious and didn't convince the mods. Keep in mind that moderators have more than just the pure content of the answers available. This is usually not about a single post, but many of them.

tortoise_in 3y ago

I would go with only human code should exist.

pixl97 3y ago
And how do you prove it?

raincole 3y ago

I'd propose a different solution. For every single new question, auto-generate an answer with GPT and mark it as AI-generated.

Me1000 3y ago
At least the generated responses won’t start by asking me “why in the world would you want to do that?”
- mediumsmart 3y ago
  You nailed it.

whatyesaid 3y ago

AI generated can be somewhat sniffed out today but I'm not sure how long that will even last. Usually you can tell when the writing is great but the points made are trash or miss the point. I personally wouldn't find it helpful. I'd say let the question poster and visitor decide if they want to see AI generated stuff. But that does rely on AI answers being marked as such or detected.

Veen 3y ago

People here are arguing about whether it’s ok to have AI-generated content on StackOverflow or not. But it seems to me that’s not the issue. The real issue is that people suck at identifying AI content, and so do AI detectors. So moderation based on that identification is obviously going to be unfair and inaccurate. Moderators are removing perfectly acceptable human-written answers based on their spurious intuition that it’s AI generated, and they are striking because they want to go on doing so. It doesn’t matter how confident moderators are that they can identity AI content when the correlation between their confidence and reality is so low.

wordcloudsare 3y ago
Any one tool is bad at detecting AI but combining both human intuition and multiple automatic tools can get a very accurate result. Moderators already only make suspensions when multiple systems agree. Not to mention the best AI might be hard to distinguish but most AI content is quite obvious (when reading carefully) because how bad it is. It can trick a casual reader but not a experienced moderator who knows about the subject matter.

lukev 3y ago

This just makes obvious sense. If someone wants a GPT-generated answer, they can use ChatGPT themselves. If I'm bothering to post my question to a forum, the social contract is that I'd like a response from a human.

slowhadoken 3y ago

ChatGPT isn’t AI, I really wish people would stop calling it that. It tricks people into thinking something magical is happening. There is still only human generated content and ChatGPT scraped it.

delecti 3y ago
"AI" is a moving target. As soon as techniques under the banner of "AI" start being used, they get a specific name and people start saying "that's not AI". For now, LLMs "are AI", just like current text-to-image models (Stable Diffusion, Midjourney, etc). There's no winning a terminology war single-handed.
pixl97 3y ago
And this folks is the AI-Effect in actions. ChatGPT is AI. It is a human level general intelligence, no, and we're likely very lucky for that. Humanity is going to have to accept that the words we used to define AI were far too imprecise and don't reflect how development of AI is actually occurring. If you want to argue the language of it, then demand we make new, testable, and scientific definitions for the capabilities of computer intelligence. Any less, and all we'll hear is the sound of scraping goal posts as they get moved further and further.

adamgordonbell 3y ago

Anyone around who moderators on SO? Is there a general sense of alienation from corporate SO? My understanding is that in the early days, a lot of the devs at SO were actually recruited from the SO and Meta moderation userbase but probably that doesn't scale. Example: Ben: And then, April 29th, I was, again, sitting in the agency, doing my thing, and I also had my personal email open, and I suddenly got an email from Jeff Atwood. But it said, “Who should work for Stack Overflow?” That was the subject line, and then the body of the message said, “You should,” and that was basically it. And that was kind of like this jaw-dropping thing, because from that moment on, it was this, “Okay, maybe the skill that I have is actually marketable,” kind of moment. https://corecursive.com/stack-overflow/ https://corecursive.com/stack-overflow/

wordcloudsare 3y ago
Not a mod but mods have complained publicly about the sense of isolation for years
NobodyNada 3y ago
I used to be heavily active in curating SO, and yes, there is an incredibly strong sense of alienation from the corporation (which is what drove me away). In the old days, most of the staff, from devs to management to the CEO, were active users of the site and hung out on Meta and in chat. They were easy to reach and happy to answer questions; and whenever they made major changes to the site they would go to Meta to ask for feedback and adjust their plans accordingly. There was a noticeable shift in this dynamic starting around ~2016. Around this time, the company stopped focusing development effort on the core site functionality, and instead prioritized side products & attempts to monetize the site (most of which failed). Feature requests on Meta were almost completely ignored, and site features that had been on previously-announced timelines/roadmaps were never delivered. But the community was still as strong as ever; and so people started implementing these missing features themselves in the form of bots and userscripts. This was the "golden age" of moderation bots, and a really fun time to be a part of the community -- power users ran heavily modified frontends that could display all kinds of additional information and automate repetitive actions, and integrate with bots to do things that Stack Exchange's systems were bad at -- like flagging spam and low-quality posts; identifying plagiarism; and detecting flamewars in comments. As the company grew, they hired a ton of middle management who were not active participants in the site, and largely did not care about the day-to-day. This was alright when they left us alone, but around 2018-2019 they began to take an openly hostile stance towards the "power users". Here's an excellent post from that time summarizing the general sentiment: https://meta.stackexchange.com/questions/331513 https://meta.stackexchange.com/questions/331513 The short version is: the company began blaming power users for things like the site's "unwelcoming" reputation (which is really a symptom of the site's outdated and opaque moderation tooling, and power users had been clamoring for better tools for years). They began a pattern of rolling out features and UI changes that took major steps backwards in usability and accessibility -- and due to all the negative feedback these changes received on Meta, they announced staff would no longer participate on Meta because it was "too negative". A high-level manager famously quipped that the opinion of Meta was not relevant as it represented "0.015%" of Stack Exchange's userbase -- despite the fact that that 0.015% was responsible for the majority of content & moderation activity contributed to the site. In late 2019 it got a whole lot worse when, in rapid succession: 1) the company updated the site terms-of-service to illegally change the license of user-submitted content, and 2) an employee abruptly revoked a volunteer's moderation privileges without due process, and then went to news outlets making false accusations about that user's behavior (https://meta.stackexchange.com/questions/333965 https://meta.stackexchange.com/questions/333965). Shortly after that, Stack Overflow fired several well-loved and highly respected staff moderators, for undisclosed reasons (https://meta.stackexchange.com/questions/342039/ https://meta.stackexchange.com/questions/342039/). A lot of people, including myself, left the community in the wake of this. Since then -- at least from my outsider perspective of checking in once in a while to see what's going on -- it seemed for a while like the company was learning from its mistakes.They apologized for ignoring Meta, began asking for and listening to community feedback once again, created new policies to protect volunteers from the kind of abuse that happened in 2019, and began implementing some of those long-ignored and long-overdue feature requests. But in 2021 the company got bought out by a VC firm that is even more aggressive about trying to monetize the site; they started cranking up advertising and pushing generally unwanted side products, but they mostly left the community alone. That brings us to generative AI. As soon as ChatGPT came out, a deluge of users began copy/pasting Stack Overflow questions into ChatGPT and copy/pasting its answers into the answer box (usually with no editing or fact-checking effort). General consensus among the community seems to be that ChatGPT produces wrong or unhelpful information to an unacceptable degree, and that allowing machine-generated content on Stack Overflow defeats the purpose of the site (you go to ChatGPT if you want answers from a machine, but you go to SO if you want answers from a human). The staff supported this consensus and made it official policy -- but at the same time the CEO kept making rambling blog posts about how "AI is the future about Stack Overflow" and launching sweeping initiatives within the company to do...AI related things? Nobody really knows what he's talking about. That all leads up to the events that triggered strike: out of nowhere, the company suddenly announced a few days ago that it was overruling previous community consensus and prohibiting users from deleting content on the basis of being AI-generated.

moffkalast 3y ago

And they can tell it's AI generated... how?

anaganisk 3y ago
Possible AI generated, closed.

z3t4 3y ago

It's an ongoing joke about spam bots, SEO spam, etc, that the day they actually post helpful and useful content it's mission fucking accomplished. With "AI" we are one step closer.

JdeBP 3y ago

The thing that comes to mind when I hear about volunteers withholding their efforts is the "reinstate Monica" protest. I haven't counted how many years it has been, but there are still people with "reinstate Monica" in their Stack Exchange profiles to this day.

cosmolev 3y ago

I would be more afraid of AI generated questions rather than answers.

DoingIsLearning 3y ago

One problem with this is that future Models will probably be trained with data that includes a subset of AI generated data. As this ratio grows the need to classify AI/human becomes critical, otherwise all these models will probably degrade over time.

Dowwie 3y ago

Some people use their StackOverflow karma for social credit and portfolio. Their karma won't have the same value as it did before AI-generated responses were allowed. I think that comments should be approved but zero karma applied for providing the correct response using AI. There won't be an incentive to provide AI generated responses, but they won't be rejected either.

throw060223 3y ago

Oh StackOverflow. I forgot it existed since ChatGPT was released. Yeah.. surely a 2013 reply from a reputation farmer suggesting a dirty jQuery solution will be better, since it was made by a human. LOL

ldehaan 3y ago

[dead]