3 ms·

I think the mistake you might be making here is that some level of routine engagement and mentorship is needed to sustain the community. If I'm an expert, but I

I think the mistake you might be making here is that some level of routine engagement and mentorship is needed to sustain the community. If I'm an expert, but I can only say something useful once every six months when some super-niche question drops, I'm just gonna stop visiting the site because of the negative feedback loop of going there and not finding anything to do 99% of the time. The other extreme - drowning in "noob" questions - is also harmful, which is why you end up with FAQs and rules. But there are human dynamics that underpin all these communities, and LLMs are undoubtedly destroying the incentives to participate.

svara 2y ago
But that's really not how it is - LLMs have their uses, obviously, but they really fall apart as soon as you're trying to do something even just slightly off the beaten path. As such, they're certainly still useful for answering questions for which StackOverflow has a dozen duplicates, and that may very well be a net positive for StackOverflow. Here's an example of something that came up in my work the other day - fiddling with prompts and checking the responses is ultimately more work than just writing the code from scratch. GPT4 and Sonnet 3.5 never came close to a correct response, even though the code always looks sort of "directionally correct". https://chatgpt.com/share/ba85ced9-63d6-4fb3-8dc1-b8efbaa58fa5 https://chatgpt.com/share/ba85ced9-63d6-4fb3-8dc1-b8efbaa58f... And that's despite this almost certainly being somewhere on Github in some form.
JimDabell 2y ago
> If I'm an expert, but I can only say something useful once every six months when some super-niche question drops > The other extreme - drowning in "noob" questions - is also harmful, which is why you end up with FAQs and rules. This isn’t reflective of what actually happened with Stack Overflow though. These are not equal dangers. There were plenty of reasonable questions, then it became overwhelmed by people who didn’t want to read tutorials or documentation but instead wanted other people to do their work for them or to have the documentation looked up and recited to them by a person not a search engine. Stack Overflow didn’t much care because it’s increased engagement, but the expert users absolutely did care and started to leave. “There aren’t enough interesting questions” is not a danger. “Experts don’t want to be a flesh interface to a search engine” is not only a danger but has already been harming the community for a long time. LLMs have the chance to reverse this damage, and they can do this by selectively reducing the volume of the harmful engagement. So I am rejecting the notion that declining activity necessarily means degraded community.

svara 2y ago

But that's really not how it is - LLMs have their uses, obviously, but they really fall apart as soon as you're trying to do something even just slightly off the beaten path. As such, they're certainly still useful for answering questions for which StackOverflow has a dozen duplicates, and that may very well be a net positive for StackOverflow. Here's an example of something that came up in my work the other day - fiddling with prompts and checking the responses is ultimately more work than just writing the code from scratch. GPT4 and Sonnet 3.5 never came close to a correct response, even though the code always looks sort of "directionally correct". https://chatgpt.com/share/ba85ced9-63d6-4fb3-8dc1-b8efbaa58fa5 https://chatgpt.com/share/ba85ced9-63d6-4fb3-8dc1-b8efbaa58f... And that's despite this almost certainly being somewhere on Github in some form.

JimDabell 2y ago

> If I'm an expert, but I can only say something useful once every six months when some super-niche question drops > The other extreme - drowning in "noob" questions - is also harmful, which is why you end up with FAQs and rules. This isn’t reflective of what actually happened with Stack Overflow though. These are not equal dangers. There were plenty of reasonable questions, then it became overwhelmed by people who didn’t want to read tutorials or documentation but instead wanted other people to do their work for them or to have the documentation looked up and recited to them by a person not a search engine. Stack Overflow didn’t much care because it’s increased engagement, but the expert users absolutely did care and started to leave. “There aren’t enough interesting questions” is not a danger. “Experts don’t want to be a flesh interface to a search engine” is not only a danger but has already been harming the community for a long time. LLMs have the chance to reverse this damage, and they can do this by selectively reducing the volume of the harmful engagement. So I am rejecting the notion that declining activity necessarily means degraded community.