8 ms·
The Fall of StackOverflow: A Data-Driven Analysis
- DaiPlusPlus 2y agoThe linked site seems overloaded (it’s popular on Reddit right now) - is it the same article as this one? https://en.numere.org/home/blog/the-fall-of-stackoverflow https://en.numere.org/home/blog/the-fall-of-stackoverflow …if so, it was published a year ago: https://www.linkedin.com/posts/erik-h%C3%A4nel-6a44151a6_numere-the-fall-of-stackoverflow-activity-7096759154903498752-Uqsx https://www.linkedin.com/posts/erik-h%C3%A4nel-6a44151a6_num... *EDIT:* The linked site (with an ironic… or apt domain-name) finally loaded for me - it is a different article. Interesting to compare conclusions though: 2023 article: > In summary, it appears that the biggest contributor to "StackOverflow's fall" is not one of the two LLMs GPT3 or GPT4, but rather the market power of Google's algorithm. The latter's changes in May 2022 caused a significant drop in traffic that is much more significant than the effect of GPT. However, since traffic also provides advertising revenue, this can be a big problem in the future. […] Of course, this also reflects the previous analysis: most of the generally valid questions have already been answered (and their answers can also be generated by GPT). What remains are the few questions that are very specific. 2024 article: > StackOverflow's decline was a result of issues within its model and execution, which predated ChatGPT's introduction. While ChatGPT may have accelerated this decline, it was not the sole cause. There are likely many factors contributing to StackOverflow's struggles, but AI alone cannot account for the entirety of its challenges. They both kinda agree - if you squint a bit.
- byteface 2y agoYou're right. For me personally, Discord was killing Stackoverflow about a year earlier. Waiting a week for a reply on Stackoverflow was becoming dull while discord live chat rooms were buzzing with instant feedback. Even as copilot first arrived it wasn't a scratch on asking an room of professionals at realtime. But now chatGPT has become a bit more fledged you can see why it would dent other platforms. I've often thought about this as how would you train future bots if all we do is talk with them now?. Where will they read their answers?
- DaiPlusPlus 2y agoWhenever I go on Discord the chats are so busy my message flies up off the screen in seconds - how does it work for you?
- byteface 2y agoTo be fair I haven't used it for about a year. As I recall I joined about 40 rooms though and not all were busy. But I think in general if your question is esoteric you would have to peel of into a room with someone with the domain specific knowledge. Some problems require a wordy description of the problem and that wont fly well in a live chat. I was really just after verification that my solution was optimal. But sometimes there just isn't a better way.
- nottorp 2y agoHoly $DEITY. Discord is a -ing chat platform. They haven't fixed anything for you. The various communities that run their own discord based chats may or may not have fixed things for you.
- frereubu 2y agoI haven't used Discord, but isn't the issue there that the answer gets quickly buried? It might work well for one person getting an answer, but the huge value of SO is that the answers stick around and turn up pretty well in searches.
- dspillett 2y ago> I haven't used Discord, but isn't the issue there that the answer gets quickly buried? Yes. This is one of the key reasons behind creating SO and why it worked so well at first. As well as non-current content in some sources getting buried or otherwise being hard to find, or simply expiring (individual forums going offline), the other issue was the distribution of people willing+able to answer questions around the many disparate forums was inefficient – for all the talk of “the Internet should be as decentralised as possible” no one has found a better answer for this than a bit of centralisation with open licensing (SO wouldn't have attracted as many of the better people answering if their work was going to be locked in rather than covered by something like CC-BY-SA). The solution that is taking over a bit ATM is Chatty Glorified Predictive Text and its friends, because this gets around the latency issues (you get a much faster answer, even if an incorrect one that means you need to rephrase the question several times) and licensing issues (though IMO the morality of that is rather dubious, the legality of it is still being argued in a number of places, but nether question of legality nor morality is going to stop it happening going forward).
- jl6 2y ago> most of the generally valid questions have already been answered My experience was that SO excelled at answering basic questions with definitive answers, but anything creeping outside that sweet spot would be poorly understood by both answerers and moderators. It’s still a useful resource for that historical record of basic answers, but it feels like complex, niche or subtle questions are not welcome.
- seb1204 2y agoSo where to go with such questions? I have one that I am not sure if it is a bug, a user issue or an android issue or a phone/hardware issue.
- vintagedave 2y agoExactly! I don't know where to go either. Once upon a time, SO was the place, but they stopped that with site policy changes around 2012 or so. A lot more became off-topic, and I found that the value I got from it declined right from that time. The view count sharply dropping off around 2013 mirrors (to me) the increasingly more narrowly defined scope of the site's acceptable content than what it was when it started.
- CannisterFlux 2y agoI went back and checked my SO account history. I stopped posting answers regularly at the end of 2011. I can't remember now what drama caused me to stop, but in 2011 I was posting 2-3 answers per day, then in 2012 I didn't post anything until mid-year, then about a post or two per year, and my last post was over 6 years ago. Digging through my local notes and journals as to why I stopped, I have an unpublished draft blog post from 2012 that has some examples of bad questions from 2012, and I later added https://meta.stackoverflow.com/questions/252506 https://meta.stackoverflow.com/questions/252506 to it as something that resonated with me - "Why did I even help this guy? I have just been wasting my time". I must have been burned out on the whole thing by that point, but it was 10+ years ago and I've not even visited SO in years. I do remember they added a tier of moderation around this time, and I had sufficient points to have access to that. Every time I visited the site it showed a red dot on the stuff-to-do icon, encouraging me to review a queue of bad questions. I'm pretty sure that was when I decided to stop contributing, because the site felt like it was more focused on the tedious bureaucracy of it all than the technical stuff.
- deleted 2y ago[deleted]
- hoseja 2y agoThe problem with volunteer moderation is, the "volunteers" eventually want to be paid in other, corrosive ways.
- dspillett 2y agoThe article is currently not responding so I can't see if this is covered, but a bigger problem is the volunteer answerers. Some left when the site was sold, many remained with a wait-and-see stance and are going due to licensing changes (https://meta.stackexchange.com/questions/401324/announcing-a-change-to-the-data-dump-process https://meta.stackexchange.com/questions/401324/announcing-a...). Obviously people asking LLMs instead is an important factor, as are unverified (often not entirely correct) answers from those using LLMs to gain Internet points.
- DaiPlusPlus 2y ago> Some left when the site was sold Que? What was sold - by whom and to who? StackExchange is still its own company - and owns StackOverflow - doesn’t it?
- DaiPlusPlus 2y agoThe diamond-symbol moderators are masochists or flagellants. Fortunately community-moderation works well - for the most-part - on SO/SE; not to say SE is anything like Reddit (e.g. the impotent protests over “Monica” which feels like it happened over a decade ago - and people still reference that episode in their display-names). If I ever become a moderator on SO it’ll be because I want to bolster my resume (I did get to interview at BlueOrigin on the basis of my SO karma score - so it has value; certainly more than LinkedIn, ew)
- babo 2y agoWeb archive link: https://web.archive.org/web/20240828071621/https://pdftranslate.ai/blog/stackoverflow-fall https://web.archive.org/web/20240828071621/https://pdftransl...
- 2-3-7-43-1807 2y agojust yesterday i wanted to ask a question about average data usage of macbooks on askdifferent (SO sibling in the stackexchange family for mac topics) ... of course the question was removed within half an hour for being too fuzzy. tried it on the macos subreddit ... couldn't even post it there for too low karma ... posted it on r/apple ... still waiting for moderator approval. it's not the first time that it feels like it's becoming more and more difficult to ask a simple question on the internet. and i have 20k points on SO - was using it a lot many years ago - i know the drill.
- sshine 2y agoThe asymmetry between questioners and answerers might be increasing. I get quality answers on IRC. Most other places seem flooded.
- DaiPlusPlus 2y ago> average data usage of macbooks With all sincerity, I honestly don’t know what this is supposed to mean.
- 2-3-7-43-1807 2y agowell, i didn't post the full question - right? b/c i'm not expecting an answer here ... but to be a little more clear: data usage refers to up- and download.
- 2y ago
- boxed 2y agoI have a different explanation: Diminishing returns. A 100% successful StackOverflow SHOULD show a tapering off of questions. This is exactly what we should expect from perfect success. Now, I don't think perfect success is what we see, but a system where a question can be answered once and then re-read many times should absolutely not keep new question counts high, that would be a sign of a massive failure. The view counts is harder to explain. I would guess it can be partially explained by search engines inlining stackoverflow answers into the search results so people DO get answers from SO, but don't actually go to the site.
- cen4 2y agoYup an Involution.
- vintagedave 2y agoTechnology does change though and a decent number of questions I see on SO should have different answers now than when the official answer was accepted. Answers tapering off is true but I think is also due to it being difficult to get new, different answers to old questions.
- layer8 2y agoNew answers can be added and upvoted. Maybe there should be a mechanism for older upvotes to expire.
- boxed 2y agoThere does indeed need to be a way to make sure newer information is weighted properly. But this is quite a minor issue compared to the great mass of questions. I mean, statistically it's a minor issue. It's a big issue for certain domains like Swift where tons of the answers are comically out of date.
- dataengineer56 2y agoI find StackOverflow interesting because there is (in my opinion) a ginormous gap between how the volunteer moderators feel that the site should operate vs how its users actually use it. I think it would be almost impossible for a new user to ask a well-received question these days. Instead of striving to improve the user experience then the moderators spend their time bickering over meta issues. Many haven't asked or answered a question in the last five years.
- DaiPlusPlus 2y agoOut of curiosity, have you seen the new “Staging Ground” feature?
- lawgimenez 2y agoI was one of the first testers of Staging Ground back in late 2022, if I remember they stopped it mid-2023. Now, I'm surprised they revived SG. Let's just say I'm not a fan of the idea or implementation.
- dataengineer56 2y agoYes in my opinion it's an absolute disaster that does nothing to help the new user experience. 1. Post a question as a new user 2. It goes into Staging Grounds instead of the main site, so there's no way to get an answer 3. Watch as a handful of powerusers flag it and demand changes, with a combination of user-written and automated comments 4. Make some of the recommended changes 5. It gets closed anyway and never makes it to the main site To me it seems like a forum that effectively sanctions the bullying of new users. I'd like to see Staging Grounds success statistics.
- tempaway4575144 2y agoI have a theory that if the homepage didn't show 'Featured On Meta' the reputation of the site would be much higher because we wouldn't all be getting a window into the bickering and nitpicking. The bickering and nitpicking is far too prominent on the homepage
- mike_hearn 2y ago> The Views Count chart reveals an even clearer trend of constant decline starting in 2013, highlighting significant issues with the model StackOverflow operated under. This is the raw data used for this observation. This seems to be wrong. According to the graph StackOverflow has nearly no views at all, which can't be right. The link goes to a query that selects votes not views, and does indeed show an enormous decline. But it's possible that people still view the site, they just don't vote.
- vegabook 2y agoIndeed. But one possibility is that these are rate of change charts and that that crucial fact was omitted.
- cybersquare 2y agoThat's because the latest count is on the order of 3M views while the scale of the graph is 1B views, or 0.003% of the full scale. It's dramatic but not wrong if you believe SO's own data. Take a look at the raw data linked in the article.
- gklitz 2y agoThat just has got to be wrong. I refuse to believe that we had a significant proportion of the world population viewing StackOverFlow answers in 2011, either the counting is wrong or the dating or aggregation is wrong. It simply cannot be true. I know a lot of HN readers live in the tech-mindset-bubble, but it simply was never the case that any answers on stack overflow rivaled GangNam styles popularity at its peak breakthrough.
- cybersquare 2y agoThis is number of views, not number of unique users. It also likely includes views by bots. Notice how the viewership drops dramatically around 2014 once you could start downloading the database instead of scraping the site.
- mike_hearn 2y ago
- WithinReason 2y agoThat graph shows a negative value for number of views this August (-1751)
- vegabook 2y agoyeah but hang on. Why are these charts going down? Is it because questions and answers are actively being _removed_? Or is this a first-derivative of [answer/question] count ie the rate of change? In which case a) this should be clearly stated ("questions added per period x"), and also, it wouldn't be surprising that the charts go down because the knowledge is already there and so fewer questions/answers are needed, even without AI. While there is indeed a big decline post-GPT et al, the decline started a lot earlier than general availability of AI.
- physicsguy 2y agoStackOverflow is one of the most hostile sites I've ever tried to use. I vividly remember asking a question before was different in a subtle but important way from existing ones (which I'd seen) and it got marked as a duplicate.
- klez 2y agoCan you please link your question?
- matsemann 2y agoMy experience from asking about that here earlier is that they seldom can. Their well-worded and perfect questions maybe weren't that good in retrospect when they dug it up again?
- nottorp 2y agoI've seen many questions like the OP describes with my own eyes though. Marked as duplicate. With helpful answers saying "this is a duplicate of X..." in spite of the question stating they don't do X but something that sounds like X. Also your answer sounds like your average SO moderator's :)
- klez 2y agoWhat people who only use stackoverflow superficially don't get is that stackoverflow is not a community, but a collection of communities that coalesce around major tags. I, for example, frequent almost exclusively the Java tag and I can assure you that the question we mark as duplicates the most are very few: - How to compare strings in java (because people keep using == instead of `.equals()` - Why doesn't this class/array change when I pass it to a method (because java is pass-by-value, not pass-by-reference)[1] - What does "cannot find symbol" mean (where the duplicate target helpfully explain in what cases this could happen, including, but not limited to "you declared it inside a loop so it's not visible outside) [2] - Why my scanner skips reading a line in this loop (because every time you use `nextFoo` where `Foo` is not `Line` you need to read the newline as well before continuing)[3] The other sort of questions we tend to close are the one easily answerable by learning any basic tutorial, the ones that tell us "my code doesn't work" without showing their code, the ones that just dump their homework and expect us to write a solution. And casual user rarely see this questions because we usually also delete them, but I can assure that most of our moderation time is spent handling this kind of stuff, not closing as duplicate. On the other hand, one kind of question that is not well received but IMHO should not lead to closing or downvoting (but it does) is "I want to do this weird and non-idiomatic thing. How do I do it}". Unless, of course, it's an actual XY problem, in which case it helps if the person asking the question states why they want to do such a thing. All of this to say, your experience with the site will vary depending on what technology you're asking about. And "your answer sounds like your average SO moderator's" yes, it does, because we use the site a lot and we know what we're talking about :) [0] https://stackoverflow.com/questions/513832/how-do-i-compare-strings-in-java https://stackoverflow.com/questions/513832/how-do-i-compare-... [1] https://stackoverflow.com/questions/40480/is-java-pass-by-reference-or-pass-by-value https://stackoverflow.com/questions/40480/is-java-pass-by-re... [2] https://stackoverflow.com/questions/25706216/what-does-a-cannot-find-symbol-or-cannot-resolve-symbol-error-mean https://stackoverflow.com/questions/25706216/what-does-a-can... [3] https://stackoverflow.com/questions/13102045/scanner-is-skipping-nextline-after-using-next-or-nextfoo https://stackoverflow.com/questions/13102045/scanner-is-skip...
- okaleniuk 2y agoAround 2013, StackOverflow had started to turn into a history museum. There were still some niches where there was a bit of life but the rest become too much "do not touch the exhibits" to be viable.
- ryu2k2 2y agoA few years back SO used to be the common top result for my search queries regarding tech and development. I'm not sure when the shift started, but these days it's always a list of content farm links with maybe a link to SO somewhere in between. The result for me was that I started to rely much more on documentation and writing down notes of stuff I found myself looking up a lot. I never used SO directly for looking up stuff, always relied on search engines, and as they declined in quality I just stopped visiting SO almost alltogether. I imagine I'm not the only one with this story. Considering users also abandoned the site for LLM assistans and the strict moderation on there, I'm not surprised it is in heavy decline.
- boxed 2y agoI think you just described the fall of Google, not anything to do with SO directly.
- Ayesh 2y agoI think StackOverflow/StackExchange has served their purpose, and are now rightfully in a decline. There are some niches sites in the network like bicycling, English, Mathematica, Academia, etc. They contain extremely high quality answers. But for programming, it's nowadays much more easier on GitHub and relevant issue trackers. I think it's good, because there are many questions in those sites that contain often outdated answers. Some lucky questions get a comment there saying the answer is outdated, but there are many outdated answers. Perhaps, JavaScript has a lot to do with this. JS evolves at a much faster rate than sites like StackOverflow can keep up.
- caporaltito 2y agoWhat a peculiar take, the whole of it
- alecco 2y agoMaybe some post-StackOverflow site could do much better if split into communities. It doesn't make much sense to pile together highly popular topics (e.g. JS or Java a few years ago) with hard topics (e.g. low-level optimizations or difficult languages like Haskell). Or maybe even remove gamification altogether. This is happening already with Discord communities but it's not searchable or usable and the format is not good. StackOverflow (and mods on a power trip) betrayed the community. I hope something fills the void. I'd be happy to help or even throw a few bucks in. Or at least contribute. Maybe this time not VC-backed nonsense.
- alecco 2y agoOther problems with StackOverflow: - gamification problems: e.g. a race to answer instead of thinking it through, downvoting competing answers - the accepted answer sometimes is incorrect (but sounds good enough to OP) - balance between repeat or vague questions and over-zealous mods blocking - need of better mod vetting (ideally, paid) - question quality Some of these could be helped with tools. For example have a small repro and test answers in a VM. And have small ML helpers to fill in the spaces and link to other questions or official docs. Maybe all answers should be wiki-style (cooperative). Maybe questions could be first in a staging area and meta-evaluated if it's good, dupe, or too vague. And it could also be improved by the community. And some mechanism to avoid Eternal September!
- layer8 2y ago> Maybe all answers should be wiki-style (cooperative). It’s valuable to read different perspectives, and even conflicting opinions, and have them attributed. Having only a cooperative answer would lead to edit wars and would drown out minority perspectives.
- alecco 2y agoAbsolutely. I didn't mean all answers combined into one, just that all answers are wiki. Like today in SO.
- Etheryte 2y agoI feel like this article overlooks a very simple idea at the heart of what Stack Overflow is: there are only so many questions you can ask about e.g. how to write a CSS selector before all the others are duplicates. This is not necessarily a bad thing, the whole idea of Stack Overflow is to have answers to general questions, not to everyone's individual question. A lot of people, especially newcomers, don't understand this though, and then get frustrated when their question gets closed as a dupe. Closing dupes is a feature not a bug, just like all roads lead to Rome, you want your questions to build a linked list to a good canonical question with a quality answer. If the whole site is riddled with a hundred thousand questions that were caused by a typo then everyone would get drowned in the noise. In other words, as far as I'm concerned this is working as intended, Stack Overflow is a QA site, not tech support.
- lkrubner 2y ago"the whole idea of Stack Overflow is to have answers to general questions, not to everyone's individual question" But why? I ran a site called WPQuestions.com for 7 years, it was where people paid between $10 to $50 to get fast answers for their WordPress questions. I called it "micro consulting" in that the consultants were answering small questions for small fees. While I get that StackOverflow wanted to be free, I wonder why it didn't commit to being more of a Helpdesk for people in need. Why have general answers, when people have such an urgent need for specific answers to their specific questions? An incredibly wasted opportunity. Even more so, with AI able to answer all of the general questions, the only avenue open to non-AI companies is answering very specific questions that are heavily context dependent.
- Etheryte 2y agoAnswering everyone's specific personal questions is geared towards beginners and doesn't add any value to others. E.g. fixing your typo doesn't help the next guy understand why their code doesn't work. Explaining the general case is timeless and valuable to all skill levels. There are plenty of sites that do what you describe, but there's a reason Stack Overflow became widely known and used instead: the QA is applicable to everyone working in that space, not just that one person who asked about their specific case.
- code_biologist 2y agoThe views chart is wildly wrong and misleading in the context presented. The linked query [1] returns aggregate views by question creation date — when it should be showing aggregate views by view date. It's not surprising that earlier questions have racked up a lot more views. [1] https://data.stackexchange.com/stackoverflow/revision/1858209/2267130/ https://data.stackexchange.com/stackoverflow/revision/185820...
- Sammi 2y agoI don't understand the difference you are describing, but the data you link to shows a very similar graph to the OP. Or at least I can't see what the difference is. Could you elaborate?
- josh-sematic 2y agoIf a question was created in 2013 and viewed by somebody in 2020, that moved the line higher in 2013, not 2020. A more useful view would be one where it contributed to 2020. In other words, the graph shown answers “how often were questions viewed, dependent on when they were written.” The point the post is trying to make would benefit more from “how many total views were there in a given year.”
- dagelf 2y agoAh... the graph of "how many real programmers are left". Behold this X post, and cringe as you slowly die... https://x.com/dagelf/status/1827622956849078618 https://x.com/dagelf/status/1827622956849078618
- nunobrito 2y agoYeah, that site where is difficult to place an answer and you get criticized often by derranged people. Never participated there again.
- m11a 2y agoI think some contributing factors are: 1) a lot of questions have been answered. How often do you ask something that is answered by a 2010-2014 answer very well? IME, often 2) programming isn’t as niche as it used to be. these days, a lot of SO is just asking for free labour on very niche questions like an exception someone is getting. which was never the point of SO there’s a wealth of answers on SO historically, which are great. But I don’t remember the last time I used SO to answer a specific question for a recent software tool, or relied on any post-2021 question/answer
- SebFender 2y agoA great example of where rules took over usage.
- dagelf 2y ago... another factor... don't underestimate the power of Google's algorithm to subvert. Don't have to look far for examples. What's the source of most of their traffic? SO is still the best evergreen forum for a bunch of things. Not to mention, how much dumber would today's almost-not-dumb AI's be without its data?! Not everyone should measure their "health" in terms of foot traffic. "The best things on the internet can't be measured?" ... a modern counterpart to "can't buy me love?"
- nottorp 2y ago> 2. Heavy moderation discouraged people from asking questions. ... or answering questions. If you answer with anything but something ready to copy/paste the moderators will frown upon it at best. No "teach the man how to fish" answers. To rephrase in 2024 terms: Before LLMs SO's moderation discouraged questions that a modern LLM couldn't answer (anything going into more than syntax) and also discouraged answers that are beyond what a modern LLM can generate (i.e. code ready to copy/paste, possibly wrong because neither the LLM nor the reputation chaser bothered to at least try to compile it). But let's not blame just the moderation. Karma whores that answer every new question with a mildly related copy/paste within seconds have been a thing ever since SO grew up.
- ivanjermakov 2y agoMy problem with SO (does not affect other Stack Exchange sites) is that the more experience I gain in programming, the more I find more efficient ways to solve my problem (read the spec/source code), therefore less use cases I have for SO. Additionally, isn't the goal of a site like SO is to finally answer every programming question? It is not an insurmountable task if you disallow opinionated questions and be strict about duplicates. I feel like it serves its purpose well and will be a valuable resource for years to come.