6 ms·

The evolution of expert communities

125 points by zdw 3y ago

jw_cook 3y ago
Ouch, this is painfully accurate. Although I think communities can go in a few different directions after stage 3. Another common outcome seems to be: arriving at an imperfect but reasonable compromise between the needs of new users and core contributors, which keeps the community relatively stable but maybe not quite as fun as it used to be.
FappleEater 3y ago
[flagged]
codetrotter 3y ago
Some of the comments mention LLMs. And they talked about LLMs having inaccuracies. OpenAI recently announced the ability to create custom versions of ChatGPT that combine instructions, extra knowledge, and skills. https://openai.com/blog/introducing-gpts https://openai.com/blog/introducing-gpts I could see some of the mentioned expert communities start to build on top of that to offer newcomers a customized chat with domain knowledge.
- zdw 3y ago
  If this parent post is the quality of LLM generated content, the forum will go nowhere - it's not adding anything to the conversation.
  - codetrotter 3y ago
    The point is that the problem is this: Experts want to provide material for newcomers. Newcomers don’t want to read the material. Using specially crafted material by the experts with an LLM can provide a way of delivering the expert knowledge to the newcomers in a format they can readily consume. Thus keeping the newcomers from flooding the forums with the same old basic questions over and over again.
    - zdw 3y ago
      Your argument seems to be that newcomers would rather use a interactive chat system than search the archives. As LLMs can hallucinate bad and false answers that often skip nuance, this would not be helpful in many cases, and possibly actively harmful. You're solving for experts not being bothered, but not for new folks being helped. A big part of learning is learning how to learn. Trying to skip that step with lying "AI" is fools gold.
    - Exoristos 3y ago
      There's no way I'm tediously interrogating some wordy half-witted chatbot when I could search the forums.
      - vasco 3y ago
        If when you arrive at a forum you use the search, the LLM isn't for you, and you're in the top 1% forum users.
    - therein 3y ago
      Back in the day and even today forums will throttle and captcha the search endpoint because it is ripe for abuse and computationally intensive. But nobody bats an eye as we throw clouds of GPUs to provide a chatbot experience.
h2odragon 3y ago
Once a community reaches the stable state, where maintenance of the library and maybe a "industry news" outlet is all thats actively needed most of the time; a meta-community for those few people needed to do that can foster long term stability. Often IRC channels in my experience. When those small groups blow up the stable platform fragments and you get pointless project forks (one of which stagnates); and so on. Many of the participants in those groups take them far too seriously and are overzealous about their opinions (thats what led them to make the sacrifices that put them in that group to begin with). There's strong echos in the shapes of the histories of cults, guilds and specialized military units.
wslh 3y ago
Isn't something special about HN? I think so. First, it has survived for long. I don't know in the future but I was here since 2009 and it is a community that always delight me. Not saying that you cannot find issues in HN but it continues to provide a lot of value. I can compare with other communities that I participate. For example, some subreddits were incredible but "suddenly" something happened that changed the rules of the game and you find yourself as an alien when before you felt at home. Most probably dang, the moderator, [1] adds a lot of value here? [1] https://news.ycombinator.com/item?id=34920400 https://news.ycombinator.com/item?id=34920400 | https://archive.is/czYVG https://archive.is/czYVG
- matheusmoreira 3y ago
  HN is special because of the people who participate in it. I see lots of actual hackers here. Lots of programmers, many employed at the big corporations, it gets to the point people joke about HN being their Google support channel. Lots of entrepeneurs. Lots of people from many other fields. Here you find comments from people who worked on stuff like Oracle database describing what that's like. The creator of an algorithm shows up to explain it. No idea if such things happen on other sites.
  - baq 3y ago
    This also happened on Reddit before it decided to IPO thus enshittify to the gills. Maybe it still happens, but I stopped going there.
  - stuartjohnson12 3y ago
    I made a post asking something about Django and the creator of Django showed up and answered. The fact that's possible here when Django is a massive framework adopted by millions is such a great signal - the density of expertise on Hacker News is just remarkable.
- keyle 3y ago
  What keeps me coming back to HN is the value of the comments. When people actually add insight, whatever the quality of the article highlighted almost becomes secondary in value. That's why I recommend upvoting valuable comment "TIL", and actively sending down the yolo/kneejerks/meh/me2 comments that don't add value/different perspective, to keep the comment section valuable for the next gal/guy. If I don't agree, I tend to leave it alone... I value someone not agreeing if they have a point and bring a different perspective. It's ok to disagree, but always attack the argument, not the person! That's a healthy debate.
matheusmoreira 3y ago
Perhaps elitism is a natural human reaction to the so called deluges. Do communities always need new users? Must growth be unlimited and eternal? Maybe there's an optimal number of people for every community.
- Barrin92 3y ago
  you don't really need growth at all but what you do need is definitely fresh blood from time to time. When the average age starts to tick up you're going to have a problem at some point. Very important and not much discussed issue in a lot of OSS projects I think. Remarkable amount of leaders and core maintainers that keep getting older and lack of people in their 20s and 30s being elevated into positions of responsibility or even joining.
- WJW 3y ago
  Communities that wish to keep existing do need new users, yes. Existing users die, lose interest or leave in other ways and if they do not get replaced by new users, the end of the community is inevitable.
esafak 3y ago
This insightful analysis describes a natural tendency, but we don't have to leave communities to their own devices. Look at the various mechanisms incorporated into Slashdot, Wikipedia, and StackExchange to address these problems. Their design is what set them apart and above forums, whose quality was more sensitive to that of their moderators. The next step is to devise mechanisms to counteract the decay. Does anyone know any literature on this subject? It could be about physical expert communities.
- kibwen 3y ago
  If you want a sustainable community, then you need new members to counteract the inevitable loss of existing members. To get new members, you first need to help them find your community somehow, then you need to give them a reason to stay, then you need to have a process where the old guard can pass the torch to the new. For "helping them find your community", that means that you need to not be precious about the platform that you're currently on; you need to be where young people are. That means somehow convincing the old guard to be willing to change platforms to accommodate the possibility of new members. In practice this probably just means forking your community onto every new platform and convincing a sustainable core of experienced users to populate the new venue, leaving the rest behind; but forking small communities often means death as they lose the critical mass of users necessary for long-term appeal. For "giving them a reason to stay", that means that your community has to be welcoming to new users. But also, since new users will probably be young-ish, the topic of the community itself needs to be something that young people have a reason to care about. If you're a community of Victrola and vintage phonograph enthusiasts, I'm afraid you're not going to appeal to the youth no matter what you do; if your topic isn't perennially cool, then you might have already lost the battle for longevity. For "having a process to pass the torch", this means you need at least a little bit of a formal structure. If you have a forum, then who runs it, who sets the rules, who enforces the rules? If the answer is "one random guy", and then that random guy gets run over by a zeppelin, then you've lost. Basically, creating sustainable, long-lasting, intergenerational communities is extremely difficult. Anything that you come up with to solve the problem is going to start looking like either a religion or a bureaucracy.
  - 3y ago
ordu 3y ago
> Stage 3: […] seasoned members develop lengthy FAQs and create strict, multi-page posting guidelines. Yeah... Old times... We did just that on our forum. But it was worse than that. There were several subforums and each one had its own guideline and a couple of FAQs. When I think of it now I can't believe how stupid we were. We scrapped them eventually (after five years or so), but it was too late. Though I do not think it was the reason why forum died. Just world had changed. > Schisms and purges within the administrator community commonly happen at this point, too. It was more fun for us. We had a very agnostic administrator. So being moderators we fought wars, banned each other, and then appealed to the admin. We lived through three "wars of moderators". I was banned for month in one of them. And at one point I banned another moderator. It was so much fun. I can't find such a place today. People are too serious about their precious communities.
gsocb 3y ago
It's extremely hard to evolve anything significant in a single generation. History of science shows things take thousands of years. What speed up we get from tech and tools hides how unfit our own brains are for many problems people think they can handle. Theory of Bounded Rationality goes into it quite well. Also the Explore - Exploit tradeoff applies here. Depending on the type of Problems the group is solving different members are in explore or exploit mode. Some problems you need a whole lot of explorers and for others you don't. Welcoming anyone and everyone into a group without awareness of the current groups composition, what mode individuals are in, and what the real problems being solved are, is like watching what happens to a theatre company after they have finished a long run of a particular play.
wayfinder 3y ago
The issue is that a lot of expert topics are stagnant. The community dies because the topic is already dead. i.e. I follow woodworking but the communities are stale because there is rarely anything new. Woodworking is about old as a human topic can get. It’s just new memes now. Hacker News is lively because the computer industry is brand spanking new plus HN also covers non-computer topics that are constantly changing. I follow skateboarding and there’s always something new and always something to chat about.
- npn 3y ago
  > i.e. I follow woodworking but the communities are stale because there is rarely anything new. Woodworking is about old as a human topic can get. It’s just new memes now. wait what? have you guys already discuss all thousands type of woods, each with its own characteristics, application and unique treatments? you also discussed in deep all the exotic tools the far east has been using for thousand of years? and all the wood joinery used by Chinese, Japanese and Vietnamese people? there are like hundreds of those which with different usage. are you saying that your community has already discussed all of that already? that's really awesome!
  - wayfinder 3y ago
    I mean the things you listed aren’t going to generate a lot of discussion topics. You might make a general thread about x86 opcodes and it would run its course but you wouldn’t make a new thread for every opcode in x86 because you’d merc everyone with boredom. Communities are for general discussion. If you desire a thorough look of all sorts of Asian (or Western) joinery, you’d specifically look for a book or website, maybe asking a forum for pointers, but the forum itself ain’t the place. Plus at some point, because none of what you mentioned is new, it will eventually be discussed. Then death.
    - deleted 3y ago
      [deleted]
    - MichaelZuo 3y ago
      It seems doubtful that the pool of novel woodworking topics has run dry. Let's say someone wants to build an entire house out of solid ebony, that could generate hundreds of hours of interesting discussion.
neilv 3y ago
A variation is when the community limits its growth (such as by being invite-only), and eventually transitions directly from stage 1 to stage 5. In one real-world example of 1->5, I thought that the transition to stage 5 was because growth was limited too much: that stage 5 happens when insufficient new people keep joining. But I haven't seen enough examples to generalize that much.
smelendez 3y ago
This happens in offline communities dedicated to learning too. It’s common to see professional conferences where newer people go to lots of panel sessions and more experienced people hang out in the hallways (and bars). University and job training programs are kind of all about this process. When you connect with your old study group you’re probably sharing memes and memories, not expertise. The transition is just more explicit because usually at some point you graduate or otherwise move on.
mattlondon 3y ago
Correction: The evolution of Reddit communities A lot of this does not apply to non-reddit - e.g. IRC or even here on HN or he'll the internet in general. It is a shame Reddit became the cesspit of the internet - it was great before subreddits were a thing. As soon as you give random people power in the form of moderation/admin rights (e.g. subreddits, or Wikipedia), 95% of the time they go on a power trip, the drama/politics start, and things start to go to shit. Ultimately the communities die when the community wants to talk about X but the vitriolic power-crazed moderators are screaming at their computer screen about how their community is only for Y and are banning people here and deleting posts there. Turns out if you shutdown conversation, there is no more conversation. Who knew!?
- phreeza 3y ago
  In my experience this definitely also applies to non-reddit communities such as phpBB-style forums.
linuxdeveloper 3y ago
[flagged]
self_awareness 3y ago
I'm not sure if this blog post is particulary insightful. Doesn't everything work like that, not only "expert communities"? Like: - Entertainment - more people watching, it means incentive to appeal to a wider audience, it often means it has to be more simple, but this is a contrast to what passionates expect, - User interface - more people using an app means UI needs to be streamlined and be less complicated (can't use keyboard anymore, we need to use touch UI now), but this is a contrast to what power users expect, - "Expert" communities - more people in a forum, this means less complicated subjects, so "experts" can't grow, they are only expected to give all the time. Maybe I didn't understand the blog post, but I see it as "the water is wet" content.
neontomo 3y ago
Related concept on the life-cycle and eventual death of any sub-culture: https://meaningness.com/geeks-mops-sociopaths https://meaningness.com/geeks-mops-sociopaths It's a bit bleak and gatekeeping but it raises some interesting questions on how subculture eventually turns into a capital-making machine.

125 points by zdw 3y ago

Ouch, this is painfully accurate. Although I think communities can go in a few different directions after stage 3. Another common outcome seems to be: arriving at an imperfect but reasonable compromise between the needs of new users and core contributors, which keeps the community relatively stable but maybe not quite as fun as it used to be.

FappleEater 3y ago

[flagged]

codetrotter 3y ago

Some of the comments mention LLMs. And they talked about LLMs having inaccuracies. OpenAI recently announced the ability to create custom versions of ChatGPT that combine instructions, extra knowledge, and skills. https://openai.com/blog/introducing-gpts https://openai.com/blog/introducing-gpts I could see some of the mentioned expert communities start to build on top of that to offer newcomers a customized chat with domain knowledge.

zdw 3y ago
If this parent post is the quality of LLM generated content, the forum will go nowhere - it's not adding anything to the conversation.
- codetrotter 3y ago
  The point is that the problem is this: Experts want to provide material for newcomers. Newcomers don’t want to read the material. Using specially crafted material by the experts with an LLM can provide a way of delivering the expert knowledge to the newcomers in a format they can readily consume. Thus keeping the newcomers from flooding the forums with the same old basic questions over and over again.
  - zdw 3y ago
    Your argument seems to be that newcomers would rather use a interactive chat system than search the archives. As LLMs can hallucinate bad and false answers that often skip nuance, this would not be helpful in many cases, and possibly actively harmful. You're solving for experts not being bothered, but not for new folks being helped. A big part of learning is learning how to learn. Trying to skip that step with lying "AI" is fools gold.
  - Exoristos 3y ago
    There's no way I'm tediously interrogating some wordy half-witted chatbot when I could search the forums.
    - vasco 3y ago
      If when you arrive at a forum you use the search, the LLM isn't for you, and you're in the top 1% forum users.
  - therein 3y ago
    Back in the day and even today forums will throttle and captcha the search endpoint because it is ripe for abuse and computationally intensive. But nobody bats an eye as we throw clouds of GPUs to provide a chatbot experience.

h2odragon 3y ago

Once a community reaches the stable state, where maintenance of the library and maybe a "industry news" outlet is all thats actively needed most of the time; a meta-community for those few people needed to do that can foster long term stability. Often IRC channels in my experience. When those small groups blow up the stable platform fragments and you get pointless project forks (one of which stagnates); and so on. Many of the participants in those groups take them far too seriously and are overzealous about their opinions (thats what led them to make the sacrifices that put them in that group to begin with). There's strong echos in the shapes of the histories of cults, guilds and specialized military units.

wslh 3y ago

Isn't something special about HN? I think so. First, it has survived for long. I don't know in the future but I was here since 2009 and it is a community that always delight me. Not saying that you cannot find issues in HN but it continues to provide a lot of value. I can compare with other communities that I participate. For example, some subreddits were incredible but "suddenly" something happened that changed the rules of the game and you find yourself as an alien when before you felt at home. Most probably dang, the moderator, [1] adds a lot of value here? [1] https://news.ycombinator.com/item?id=34920400 https://news.ycombinator.com/item?id=34920400 | https://archive.is/czYVG https://archive.is/czYVG

matheusmoreira 3y ago
HN is special because of the people who participate in it. I see lots of actual hackers here. Lots of programmers, many employed at the big corporations, it gets to the point people joke about HN being their Google support channel. Lots of entrepeneurs. Lots of people from many other fields. Here you find comments from people who worked on stuff like Oracle database describing what that's like. The creator of an algorithm shows up to explain it. No idea if such things happen on other sites.
- baq 3y ago
  This also happened on Reddit before it decided to IPO thus enshittify to the gills. Maybe it still happens, but I stopped going there.
- stuartjohnson12 3y ago
  I made a post asking something about Django and the creator of Django showed up and answered. The fact that's possible here when Django is a massive framework adopted by millions is such a great signal - the density of expertise on Hacker News is just remarkable.
keyle 3y ago
What keeps me coming back to HN is the value of the comments. When people actually add insight, whatever the quality of the article highlighted almost becomes secondary in value. That's why I recommend upvoting valuable comment "TIL", and actively sending down the yolo/kneejerks/meh/me2 comments that don't add value/different perspective, to keep the comment section valuable for the next gal/guy. If I don't agree, I tend to leave it alone... I value someone not agreeing if they have a point and bring a different perspective. It's ok to disagree, but always attack the argument, not the person! That's a healthy debate.

matheusmoreira 3y ago

Perhaps elitism is a natural human reaction to the so called deluges. Do communities always need new users? Must growth be unlimited and eternal? Maybe there's an optimal number of people for every community.

Barrin92 3y ago
you don't really need growth at all but what you do need is definitely fresh blood from time to time. When the average age starts to tick up you're going to have a problem at some point. Very important and not much discussed issue in a lot of OSS projects I think. Remarkable amount of leaders and core maintainers that keep getting older and lack of people in their 20s and 30s being elevated into positions of responsibility or even joining.
WJW 3y ago
Communities that wish to keep existing do need new users, yes. Existing users die, lose interest or leave in other ways and if they do not get replaced by new users, the end of the community is inevitable.

esafak 3y ago

This insightful analysis describes a natural tendency, but we don't have to leave communities to their own devices. Look at the various mechanisms incorporated into Slashdot, Wikipedia, and StackExchange to address these problems. Their design is what set them apart and above forums, whose quality was more sensitive to that of their moderators. The next step is to devise mechanisms to counteract the decay. Does anyone know any literature on this subject? It could be about physical expert communities.

kibwen 3y ago
If you want a sustainable community, then you need new members to counteract the inevitable loss of existing members. To get new members, you first need to help them find your community somehow, then you need to give them a reason to stay, then you need to have a process where the old guard can pass the torch to the new. For "helping them find your community", that means that you need to not be precious about the platform that you're currently on; you need to be where young people are. That means somehow convincing the old guard to be willing to change platforms to accommodate the possibility of new members. In practice this probably just means forking your community onto every new platform and convincing a sustainable core of experienced users to populate the new venue, leaving the rest behind; but forking small communities often means death as they lose the critical mass of users necessary for long-term appeal. For "giving them a reason to stay", that means that your community has to be welcoming to new users. But also, since new users will probably be young-ish, the topic of the community itself needs to be something that young people have a reason to care about. If you're a community of Victrola and vintage phonograph enthusiasts, I'm afraid you're not going to appeal to the youth no matter what you do; if your topic isn't perennially cool, then you might have already lost the battle for longevity. For "having a process to pass the torch", this means you need at least a little bit of a formal structure. If you have a forum, then who runs it, who sets the rules, who enforces the rules? If the answer is "one random guy", and then that random guy gets run over by a zeppelin, then you've lost. Basically, creating sustainable, long-lasting, intergenerational communities is extremely difficult. Anything that you come up with to solve the problem is going to start looking like either a religion or a bureaucracy.
- 3y ago

ordu 3y ago

> Stage 3: […] seasoned members develop lengthy FAQs and create strict, multi-page posting guidelines. Yeah... Old times... We did just that on our forum. But it was worse than that. There were several subforums and each one had its own guideline and a couple of FAQs. When I think of it now I can't believe how stupid we were. We scrapped them eventually (after five years or so), but it was too late. Though I do not think it was the reason why forum died. Just world had changed. > Schisms and purges within the administrator community commonly happen at this point, too. It was more fun for us. We had a very agnostic administrator. So being moderators we fought wars, banned each other, and then appealed to the admin. We lived through three "wars of moderators". I was banned for month in one of them. And at one point I banned another moderator. It was so much fun. I can't find such a place today. People are too serious about their precious communities.

gsocb 3y ago

It's extremely hard to evolve anything significant in a single generation. History of science shows things take thousands of years. What speed up we get from tech and tools hides how unfit our own brains are for many problems people think they can handle. Theory of Bounded Rationality goes into it quite well. Also the Explore - Exploit tradeoff applies here. Depending on the type of Problems the group is solving different members are in explore or exploit mode. Some problems you need a whole lot of explorers and for others you don't. Welcoming anyone and everyone into a group without awareness of the current groups composition, what mode individuals are in, and what the real problems being solved are, is like watching what happens to a theatre company after they have finished a long run of a particular play.

wayfinder 3y ago

The issue is that a lot of expert topics are stagnant. The community dies because the topic is already dead. i.e. I follow woodworking but the communities are stale because there is rarely anything new. Woodworking is about old as a human topic can get. It’s just new memes now. Hacker News is lively because the computer industry is brand spanking new plus HN also covers non-computer topics that are constantly changing. I follow skateboarding and there’s always something new and always something to chat about.

npn 3y ago
> i.e. I follow woodworking but the communities are stale because there is rarely anything new. Woodworking is about old as a human topic can get. It’s just new memes now. wait what? have you guys already discuss all thousands type of woods, each with its own characteristics, application and unique treatments? you also discussed in deep all the exotic tools the far east has been using for thousand of years? and all the wood joinery used by Chinese, Japanese and Vietnamese people? there are like hundreds of those which with different usage. are you saying that your community has already discussed all of that already? that's really awesome!
- wayfinder 3y ago
  I mean the things you listed aren’t going to generate a lot of discussion topics. You might make a general thread about x86 opcodes and it would run its course but you wouldn’t make a new thread for every opcode in x86 because you’d merc everyone with boredom. Communities are for general discussion. If you desire a thorough look of all sorts of Asian (or Western) joinery, you’d specifically look for a book or website, maybe asking a forum for pointers, but the forum itself ain’t the place. Plus at some point, because none of what you mentioned is new, it will eventually be discussed. Then death.
  - deleted 3y ago
    [deleted]
  - MichaelZuo 3y ago
    It seems doubtful that the pool of novel woodworking topics has run dry. Let's say someone wants to build an entire house out of solid ebony, that could generate hundreds of hours of interesting discussion.

neilv 3y ago

A variation is when the community limits its growth (such as by being invite-only), and eventually transitions directly from stage 1 to stage 5. In one real-world example of 1->5, I thought that the transition to stage 5 was because growth was limited too much: that stage 5 happens when insufficient new people keep joining. But I haven't seen enough examples to generalize that much.

smelendez 3y ago

This happens in offline communities dedicated to learning too. It’s common to see professional conferences where newer people go to lots of panel sessions and more experienced people hang out in the hallways (and bars). University and job training programs are kind of all about this process. When you connect with your old study group you’re probably sharing memes and memories, not expertise. The transition is just more explicit because usually at some point you graduate or otherwise move on.

mattlondon 3y ago

Correction: The evolution of Reddit communities A lot of this does not apply to non-reddit - e.g. IRC or even here on HN or he'll the internet in general. It is a shame Reddit became the cesspit of the internet - it was great before subreddits were a thing. As soon as you give random people power in the form of moderation/admin rights (e.g. subreddits, or Wikipedia), 95% of the time they go on a power trip, the drama/politics start, and things start to go to shit. Ultimately the communities die when the community wants to talk about X but the vitriolic power-crazed moderators are screaming at their computer screen about how their community is only for Y and are banning people here and deleting posts there. Turns out if you shutdown conversation, there is no more conversation. Who knew!?

phreeza 3y ago
In my experience this definitely also applies to non-reddit communities such as phpBB-style forums.

linuxdeveloper 3y ago

[flagged]

self_awareness 3y ago

I'm not sure if this blog post is particulary insightful. Doesn't everything work like that, not only "expert communities"? Like: - Entertainment - more people watching, it means incentive to appeal to a wider audience, it often means it has to be more simple, but this is a contrast to what passionates expect, - User interface - more people using an app means UI needs to be streamlined and be less complicated (can't use keyboard anymore, we need to use touch UI now), but this is a contrast to what power users expect, - "Expert" communities - more people in a forum, this means less complicated subjects, so "experts" can't grow, they are only expected to give all the time. Maybe I didn't understand the blog post, but I see it as "the water is wet" content.

neontomo 3y ago

Related concept on the life-cycle and eventual death of any sub-culture: https://meaningness.com/geeks-mops-sociopaths https://meaningness.com/geeks-mops-sociopaths It's a bit bleak and gatekeeping but it raises some interesting questions on how subculture eventually turns into a capital-making machine.