12 ms·
Not a big surprise once LLMs came along: stack overflow developed some pretty unpleasant traits over time. Everything from legitimate questions being closed fo
by porcoda 10mo ago
Not a big surprise once LLMs came along: stack overflow developed some pretty unpleasant traits over time. Everything from legitimate questions being closed for no good reason (or being labeled a duplicate even though they often weren’t), out of date answers that never get updated as tech changes, to a generally toxic and condescending culture amongst the top answerers. For all their flaws, LLMs are so much better.
- palata 10mo agoAgreed. I personally stopped contributing to StackOverflow before LLMs, because of the toxic moderation. Now with LLMs, I can't remember the last time I visited StackOverflow.
- eurekin 10mo agoSame here. I just didn't want to expend energy racing trigger happy mods. It was so odd, to this day remember vividly how they cleanup their arguments once proven wrong on the closing vote. Literally minutes before it would the close threshold.
- lysace 10mo agoGen 0: expertsexchange.com, later experts-exchange.com (1996) Gen 1: stackoverflow.com (2008) Gen 2: chatgpt.com (2022, sort of)
- econ 10mo agoYahoo answers Google answers And the horrific Quora
- lysace 10mo agoNone of those worked as programming tools. I really miss Google Answers though, with the bounties. Random example: http://answers.google.com/answers/threadview/id/762357.html http://answers.google.com/answers/threadview/id/762357.html It's remarkable how similar in style the answers are to what we all know from e.g. chatgpt.
- dotancohen 10mo agoIt's remarkable only in the sense that you can see where the LLMs were trained from.
- pvab3 10mo agoThat would be such a useful service today
- NewsaHackO 10mo ago>Gen 0: expertsexchange.com No way.
- lysace 10mo agoIt was a more innocent time. Proof: https://web.archive.org/web/19990429180417/http://www.expertsexchange.com/ https://web.archive.org/web/19990429180417/http://www.expert...
- Alupis 10mo agoPeople in this thread are missing another key component in the decline of StackOverflow - the more experienced you become, the less useful it is. The harder the problem, the less engagement it gets. People who spend hours working on your issue are rewarded with a single upvote. Meanwhile, "how do I concat a string" gets dozens or hundreds of upvotes. The incentive/reward structure punished experienced folks with challenging/novel questions. Pair that with the toxic moderation and trigger-happy close-votes, you get a zombie community with little new useful content.
- encom 10mo ago>toxic moderation and trigger-happy close-votes >zombie community Like Reddit post 2015.
- deleted 10mo ago[deleted]
- fabian2k 10mo agoStack Overflow moderation is very transparent compared to whatever Reddit considers moderation. For programming my main problem with Reddit is that the quality of posts is very low compared to SO. It's not quite comparable because the more subjective questions are not allowed on SO, but there's a lot of advice on Reddit that I would consider harmful (often in the direction of adding many more libraries than most people should).
- deleted 10mo ago[deleted]
- bmacho 10mo ago> People in this thread are missing another key component in the decline of StackOverflow - the more experienced you become, the less useful it is. This is literally not true. The rate you learn and encounter new things depends on many things: you, your mood, your energy etc. But not on the amount of your experience. > The harder the problem, the less engagement it gets. People who spend hours working on your issue are rewarded with a single upvote. This is true, but not relevant, I don't think many people care. Some might, but not many.
- baby 10mo agoThe same is true for reddit imo, it became impossible to post anything to a subreddit way before LLMs
- why-o-why 10mo agoAnd you can't delete your post when you realize how awful it was years later! That anti-information sticks around for ages. Even worse when there are bad answers attached to it, too.
- Ajedi32 10mo agoIf you're talking about deleting questions, that's because deleting the question would delete everyone's answers that they potentially worked very hard on and which others might find useful. If you think the answers are bad you can always post your own competing answer.
- conorcleary 10mo ago"A Human commented at ##:##pm" "An AI Bot commented at..." "A suspected AI Bot commented at..." "An unconfirmed Human commented at..."
- why-o-why 10mo agoya but you assume someone worked hard on the answer. there are alot of times when you get garbage top to bottom.
- what 10mo agoYou can’t delete anything here either… so make sure you don’t say anything awful.
- why-o-why 10mo agocreate a new account every few weeks and don't forget to mix you you'er writin' style to fakeout stylometrics. its all against the rules but i disagree with HN terms. internet points don't mean crapola to me. but i like dropping in here every now and then to chit caht. i should have the right to be anonymous and non-deidentifiable here and speak freely. of IP address ---are--- tracked here and you can easily be shadowbanned. but i don't say anything awful, but i am naturally an asshat and i just can't seem to change my spots. 90% of the time i'm ok, but 10% i'm just a raving tool.
- xp84 10mo agoThe dumbest part of SO is how the accepted answer would often be bad, and sometimes someone had posted a better answer after the fact, and if the all-powerful moderators had the power to update it, they sure never did. Likewise, there were often better insights in comments. Apparently if you have the right mod powers, you can just edit an answer (such as the accepted one) to make it correct, but that always struck me as a bizarre feature, to put words in other people’s mouths. I think overall SO took the gamification, and the “internet points” idea, way too far. As a professional, I don’t care about Reddit Karma or the SO score or my HN karma. I just wanted answers that are correct, and a place to discuss anything that’s actually interesting. I did value SO once as part of the tedious process of attempting to get some technical problem solved, as it was the best option we had, but I definitely haven’t been there since 2023. RIP.
- halapro 10mo ago> took gamification way too far I disagree, I always thought it SO did a great job with it. The only part I would have done differently would be to cap the earnable points per answer. @rndusr124 shouldn't have moderation powers just because his one and only 2009 answer got 3589 upvotes.
- loeg 10mo agoLong before LLMs. Setting aside peak-COVID as a weird aberration, question volume has been in decline since 2014 or maybe 2016.
- echelon 10mo agoStack Overflow would still have a vibrant community if it weren't for the toxic community. Imagine a non-toxic Stack Overflow replacement that operated as an LLM + Wiki (CC-licensed) with a community to curate it. That seems like the sublime optimal solution that combines both AI and expertise. Use LLMs to get public-facing answers, and the community can fix things up. No over-moderation for "duplicates" or other SO heavy-handed moderation memes. Someone could ask a question, an LLM could take a first stab at an answer. The author could correct it or ask further questions, and then the community could fill in when it goes off the rails or can't answer. You would be able to see which questions were too long-tail or difficult for the AI to answer, and humans could jump in to patch things up. This could be gamified with points. This would serve as fantastic LLM training material for local LLMs. The authors of the site could put in a clause saying that "training is allowed as long as you publish your weights + model". Someone please build this. Edit: Removed "LLMs did not kill Stack Overflow." first sentence as suggested. Perhaps that wasn't entirely accurate, and the rest of the argument stands better on its own legs.
- deleted 10mo ago[deleted]
- MPSimmons 10mo agoThe fact that they basically stopped the ability to ask 'soft' questions without a definite answer made it very frustrating. There's no definitive answer to a question about best practices, but you can't ask people to share their experiences or recommendations.
- banana_giraffe 10mo agoThey actually added some new question categories a while ago [1] "Troubleshooting / Debugging" is meant for the traditional questions, "Tooling recommendation", "Best practices", and "General advice / Other" are meant for the soft sort of questions. I have no clue what the engagement is on these sort of categories, though. It feels like a fix for a problem that started years ago, and by this point, I don't really know if there's much hope in bringing back the community they've worked so hard to scare away. It's pretty telling just how much the people that are left hate this new feature. [1] https://meta.stackoverflow.com/questions/435293/opinion-based-questions-alpha-experiment-on-stack-overflow https://meta.stackoverflow.com/questions/435293/opinion-base...
- p-e-w 10mo agoIndeed. StackOverflow was by far the most unpleasant website that I have regularly interacted with. Sometimes, just seeing how users were treated there (even in Q&A threads that I wasn’t involved in at all) disturbed me so much it was actually interfering with my work. I’m so, so glad that I can now just ask an AI to get the same (or better) answers, without having to wade through the barely restrained hate on that site.
- hdgvhicv 10mo ago> For all their flaws, LLMs are so much better. For now. They still need to be enshitted.
- fooker 10mo agoYeah just wait for the ads
- esafak 10mo agoModels are check-pointed. You can save one you like and use it forever.
- stavros 10mo agoDoesn't help when the ads are a layer above the model.
- esafak 10mo agoThere are open source models you yourself or a trusted third party can run. No ads.
- shermantanktop 10mo agoYou can save an open source + open weights model, which is frozen in time. That’s still very useful for some things but lacks knowledge of current data. So we’ll end up with a choice of low-performing stale models or high-performing enshittified models which know about more current information.
- esafak 10mo agoOpen source models get updated all the time. You'd only be a few months behind.
- shermantanktop 10mo agoDirect enshittification is intentional and wouldn’t affect open models. Indirect pollution via AI slop in the input and the same content manipulation mechanisms as SEO hacking is still a threat for open models.
- antisthenes 10mo agoYep, LLMs are perfect for the "quick buy annoying to answer 500 times" questions about writing a short script, or configuring something, or using the right combination of command line parameters. Quicker than searching the entirety of Google results and none of the attitude.
- jonahss 10mo agoIt seemed to me that pre-llm, google had stopped surfacing stackoverflow answers in search results.
- wood_spirit 10mo agoMy memory is there were a spate of SO scraping sites that google would surface above SO and google just would not zap. It would have been super trivial to fix but google didn’t. My pet theory was that google were getting doubleclick revenue from the scrapers so had incentives to let them scrape and to promote them in search results.
- xp84 10mo agoI remember those too! There were seemingly thousands of them! Reminds me of my most black-hat project — a Wikipedia proxy with 2 Adsense ads injected into the page. It made me like $20-25 a month for a year or so but sadly (nah, perfectly fairly) Google got wise to it.
- 20after4 10mo agoI'm actually surprised it was only ~$20 a month.
- xp84 10mo agoHonestly, if I'd gotten in earlier I bet it would have made more. I also made zero attempts to SEO it -- there were no links to it anywhere else on the Internet, so it would have been in the very first tranche of useless duplicative spam sites of that type to be cleaned up. It was up from like 2010-2013 or so.
- eigenspace 10mo agoBecause nobody was clicking on them
- adrum 10mo agoI also wonder if GitHub Discussions was also a (minor) contributing factor to the decline. I recall myself using GitHub Discussions more and more when it came to repo specific issues. The timeline also matches: https://github.blog/changelog/2020-12-08-github-discussions-now-available-as-a-public-beta/ https://github.blog/changelog/2020-12-08-github-discussions-... https://github.blog/news-insights/product-news/github-discussions-out-of-beta/ https://github.blog/news-insights/product-news/github-discus...
- dom96 10mo agoDo we have any stats for the number of GitHub discussions created each month to compare to this?
- secondcoming 10mo agoHow can we be sure that LLMs won't start giving stale answers?
- mark-r 10mo agoWe can't. I don't think the LLMs themselves can recognize when an answer is stale. They could if contradicting data was available, but their very existence suppresses the contradictory data.
- zahlman 10mo agoLLMs don't experience the world, so they have no reason a priori to know what is or isn't truthful in the training data. (Not to mention the confabulation. Making up API method names is natural when your model of the world is that the method names you've seen are examples and you have no reason to consider them an exhaustive listing.)
- Someone1234 10mo agoThey still use the official documentation/examples, public Github Repos, and your own code which are all more likely to be evergreen. SO was definitely a massive training advantage before LLMs matured though.
- g947o 10mo agoThey will, but model updates and competition help solve the problem. If people find that Claude consistently gives better/more relevant answers over GPT, for example, people will choose the better model. The worst thing with Q/A sites isn't they don't work. It's that they there are no alternatives to stackoverflow. Some of the most upvoted answers on stackoverflow prove that it can work well in many cases, but too bad most other times it doesn't.
- Cloudef 10mo agoLLMs are just statistics, eventually they kill themselves with feedback loop by consuming their own farts (literally)
- nautilus12 10mo agoThe irony is that the LLMs are trained on stack overflow and should inherit a lot of those traits and errors.
- Bratmon 10mo agoYeah, but they don't inherit their rules and attitude. Really, if we could apply some RLHF to the Stack Overflow community, it would be doing a lot better.
- ivewonyoung 10mo ago>For all their flaws, LLMs are so much better But LLMs get their answers from StackOverflow and similar places being used as the source material. As those start getting outdated because of lack of activity, LLMs won't have the source material to answer questions properly.
- badthingfactory 10mo agoStackOverflow answers are outdated. Every time I end up on that site these days, I find myself reading answers from 12 years ago that are no longer relevant.
- shawn_w 10mo agoI see plenty of old answers that are still very relevant. Suppose it depends on what language/tech tags you follow.
- zahlman 10mo agoThere have been many times I have seen someone complain on the meta site about answers being old and outdated, and then they give specific examples, and I go check them out and they're actually still perfectly valid.
- terminalshort 10mo agoNow they can read the documentation and code in the repo directly and answer based on that.
- croes 10mo agoSO had answers that you couldn't find in the documentation and were you can't look in the source code. If everything would be well documentated SO wouldn't have being as big as it was in the first place.
- nutjob2 10mo agoI think the industry is quickly moving to syntheticly derived knowledge, or custom/systematic knowledge production from humans.
- dmezzetti 10mo agoThis change was happening well before LLMs. People were tired of being yelled at and treated poorly. A cautionary tale for many of these types of tech platforms, this one included.
- bborud 10mo agoI suppose all sites that have a voting component run the risk of becoming unpleasant. Hacker News, and we who frequent it, ought to have that in mind.
- shermantanktop 10mo agodang and the other HN moderators do a heroic job to set the tone, which has second- and third-order effects on behavior.
- immibis 10mo ago[flagged]
- cellularmitosis 10mo agoI think it has more to do with the fact that when you offer zero salary for moderators, you have to take what you can get, and it ain't good. I don't really see a connection to the voting mechanic.
- immibis 10mo agoIt's also disconnected incentives. SO users get numbers to go up by taking moderation actions so of course they do that. Also you literally get banned from reviewing questions if you don't flag enough of them to be closed. These are incentives put in place by the SO company intentionally. It's not like only slimy people get to use moderator tools like on Reddit, since you need a lot of reputation points you get by having questions and answers voted up. It's more like (1) you select people who write surface-level-good answers since that's what's upvoted, and they moderate with a similar attitude and (2) once you have access to moderator tools you're forced to conform with (1) or your access is revoked, and (3) the company is completely incompetent and doesn't give a shit about any of this.
- bborud 10mo agoWhy do you think it makes a difference if they are paid or not? Or more to the point: what are you saying? That people have different standards when paid? That lack of remuneration justifies poor effort? Isn’t that a very transactional view of human interaction? Are we that transactional? Do we want this? We’re talking about how communities can become toxic. How we humans sometimes create an environment that is at odds with our intentions. Or at least what we outwardly claim to be our intentions. I think it is a bit sad when people feel they have to be compensated to not let a community deteriorate.
- jstummbillig 10mo agoThat depends on what you mean by "came along". If you mean "once that everyone got around to the idea that LLMs were going to be good at this thing" then sure, but it was not long ago that the majority of people around here were very skeptical of the idea that LLMs would ever be any good at coding.
- rustystump 10mo agoI dont think this is true. People were skeptical of agi / better than human coding which is not the case. As a matter of fact i think searching docs was one of the first manor uses of llms before code.
- nutjob2 10mo agoThat's because there has been rapid improvement by LLMs. Their tendency to bullshit is still an issue, but if one maintains a healthy skepticism and uses a bit of logic it can be managed. The problematic uses are where they are used without any real supervision. Enabling human learning is a natural strength for LLMs and works fine since learning tends to be multifaceted and the information received tends to be put to a test as a part of the process.
- make3 10mo agoWhat you're arguing about is the field completely changing over 3 years; it's nothing, as a time for everyone to change their minds. LLMs were not productified in a meaningful way before ChatGPT in 2022 (companies had sufficiently strong LLMs, but RLHF didn't exist to make them "PR-safe"). Then we basically just had to wait for LLM companies to copy Perplexity and add search engines everywhere (RAG already existed, but I guess it was not realistic to RAG the whole internet), and they became useful enough to replace StackOverflow.
- yigalirani 10mo agoall true, but i still find myself ask questions there after llm gave wrong answers and wasted my time
- Salgat 10mo agoSeemed like for every other question, I received unsolicited advice telling me how I shouldn't be doing it this way, only for me to have to explain why I wanted to do it this way (with silence from them).
- zero_shift 10mo agoOh I love that game! (At least I think it's a game) You ask how to do X. Member M asks why you want to do X. Because you want to do Y. Well!? why do you want to do Y?? Because Y is on T and you can't do K so you need a Z Well! Well! Why do you even use Z?? Clearly J is the way it is now recommended! Because Z doesn't work on a FIPS environment. ... Can you help me? ... I just spent 15 minutes explaining X, Y and Z. Do you have any help? ...(crickets)
- hsbauauvhabzb 10mo agoTbf the problem there is probably FIPS more than anything else.
- immibis 10mo agoIf someone is paying you to implement a security vulnerability and you've told them and you don't have liability, you just do it. That's how capitalism works. You do whatever people give you money for.
- hsbauauvhabzb 10mo agoI wasn’t referring to vulnerabilities, I was referring to arbitrary silly security theatre controls. But id hate to deal with you professionally. Gross.
- stirfish 10mo agoHow do I add a second spout to this can? ... Well, the pump at the gas station doesn't fit in my car, but they sold me a can with a spout that fits in my car. ... It's tedious to fill the can a dozen times when I just want to fill up my gas tank. Can you help me or not? ... I understand, but I already bought the can. I don't need the "perfect" way to fill a gas tank, I just want to go home.
- gfhifd42 10mo agoIt doesn't have anything to do with LLMs. It has to do with shifting one's focus from doing good things to making money. Joel did that, and SO failed because of it. Joel promised the answering community he wouldn't sell SO out from under them, but then he did. And so the toxicity at the top trickled down into the community. Those with integrity left the community and only toxic, selfcentered people remained to destroy what was left in effort to salvage what little there was left for themselves. Mods didn't dupe questions to help the community. They did it to keep their own answers at the top on the rankings.
- oofbey 10mo agoHow did Joel sell out? Curious as I’m not aware of any monetary changes. I watched Joel several times support completely brain dead policies in the meta discussions which really set the rules and tone. So my respect there is low.
- gfhifd42 10mo agoHe and Jeff made it abundantly clear their mission was to destroy the sex change site because that site was immoral for monetizing the benevolence of the community who answered the questions. "Knowledge should be free" they said. "You shouldn't make money off stuff like this," they said. Plenty of links and backstory in my other comments.
- oofbey 10mo agoThey literally sold: https://arstechnica.com/gadgets/2021/06/stack-overflow-sold-to-tech-investor-prosus-for-1-8-billion/ https://arstechnica.com/gadgets/2021/06/stack-overflow-sold-...
- ChrisMarshallNY 10mo agoOh yeah. My favorite feature of LLMs, is the only dumb question, is the one I don't ask. I guess someone could train an LLM to be spiteful and nasty, but that would only be for entertainment.
- fragmede 10mo agoIf you say the wrong thing to grok, it will go off on you. It's quite entertaining!
- vivzkestrel 10mo agonot only stackoverflow, but also reddit.com/r/aws reddit.com/r/docker reddit.com/r/postgresql all 3 of them have extremely toxic communities. ask a question and get downvoted instantly! Noo!! your job is to actually upvote the question to maximize exposure for the algorithm unless it is a really really stupid question that a google search could fix
- deleted 10mo ago[deleted]
- zahlman 10mo ago> legitimate questions being closed for no good reason They are closed for good reasons. People just have their own ideas about what the reasons should be. Those reasons make sense according to others' ideas about what they'd like Stack Overflow to be, but they are completely wrong for the site's actual goals and purposes. The close reasons are well documented (https://meta.stackoverflow.com/questions/417476 https://meta.stackoverflow.com/questions/417476) and well considered, having been exhaustively discussed over many years. > or being labeled a duplicate even though they often weren’t I have seen so many people complain about this. It is vanishingly rare that I actually agree with them. In the large majority of cases it is comically obvious to me that the closure was correct. For example, there have been many complaints in the Python tag that were on the level of "why did you close my question as a duplicate of how to do X with a list? I clearly asked how to do it with a tuple!" (for values of X where you do it the same way.) > a generally toxic and condescending culture amongst the top answerers. On the contrary, the top answerers are the ones who will be happy to copy and paste answers to your question and ignore site policy, to the constant vexation of curators like myself trying to keep the site clean and useful (as a searchable resource) for everyone. > For all their flaws, LLMs are so much better. I actually completely agree that people who prefer to ask LLMs should ask LLMs. The experience of directly asking (an LLM) and getting personalized help is explicitly the exact thing that Stack Overflow was created to get away from (i.e., the traditional discussion forum experience, where experts eventually get tired of seeing the same common issues all the time and all the same failures to describe a problem clearly, and where third parties struggle to find a useful answer in the middle of along discussion).
- abanana 10mo agoYou seem to have filled this thread with a huge number of posts that try to justify SO's actions. Over and over, these justifications are along the lines of "this is our mission", "read our policy", "understand us". Often, doing what your users want leads to success. Stamping authority over your users, and giving out a constant air of "we know better than all of you", drives them away. And when it's continually emphasized publicly (rather than just inside a marketing department) that the "mission" and the "policy" are infinitely more important than what your users are asking for, that's a pretty quick route to failure. When you're completely embedded in a culture, you don't have the ability to see it through the eyes of the majority on the outside. I would suggest that some of your replies here - trying to deny the toxicity and condescension - are clearly showing this.