7 ms·

> Something important changed between the 90s and today. If you look at most open source projects now, the distribution of who’s doing the work versus who’s sim

> Something important changed between the 90s and today. If you look at most open source projects now, the distribution of who’s doing the work versus who’s simply there is skewed dramatically: it’s common to see projects where 95% of the work is done by a nucleus of people, perhaps even a single developer, with a long tail of “contributors” Any evidence this has actually changed between the 90s and now? I have a strong suspicion that it's always been this way. There are exceptional projects where the work is more distributed (is Linux an example?), but my suspicion is that they were exceptional in the 90s and still are, and that all along most projects have this "dramatic skew" as described. For software of now, you can look at github. For software of the 90s, 2000s... I wonder if anyone has tried to research this? I definitely don't see any reason to assume it was different without evidence.

z3rgl1ng 6y ago
+1, [Citation Needed].
- z3rgl1ng 6y ago
  Christ, every reply to this thread is an anecdote. Please back up assertions with _some_ data point..
bergie 6y ago
Totally anecdotal, but I feel open source projects were much more community oriented in the 90s and the 00s. But after that, it seems they’re either driven by a company, or an individual. That would roughly conform to the article’s timeline of GitHub adoption.
- jrochkind1 6y ago
  By "community-oriented", do you mean that the distribution of commits was such that more people were making more commits, instead of "95% of the work is done by a nucleus of people"? I'm not sure if those are the same thing, I'm just talking about the distribution of work.
- gowld 6y ago
  Everything has more "individuals" now, since everyone can get a soapbox/megaphone, from YouTube to GitHub to Twitter to all the rest. And it's possible to monetize that work now, with ads and micro-patrons, which is a huge drain on free and open source, mitigated only (partially) by the increased ability for open sourcerers to collaborate and that nowadays we're drowing in software and don't really need much more new software. You don't need to join a team to get noticed (and on the ugly side, even though having a team helps, it's better for the Personal Brand to make that team invisible)
- nerdponx 6y ago
  Counter-anecdote: most of the small open-source projects I use only have a small number of active contributors because there is a small number of people who actively try to contribute. Projects that stop accepting PRs tend to be pronounced "dead" after just a few months, but there isn't always someone able/willing to fork it and keep the project going. Maybe there are more projects to contribute to than before, so each project gets a smaller % of the "potential contributor" pool.
nitrogen 6y ago
For software of the 90s, 2000s Check Sourceforge, Freshmeat.net archives, IRC, old anonymous FTP mirrors, etc.
Spivak 6y ago
Users of software care about their own workflows, their pet-features, and the bugs they hit. Some are willing to put up the work to patch in those features or fix those bugs and upstream them. Fewer still are people who are invested enough to dedicate themselves to a project as a part-time job. I don't think this dynamic has changed since the dawn of open source.
coldtea 6y ago
>Any evidence this has actually changed between the 90s and now? Experience with the relevant communities (Gnome, KDE, and others, etc) in the 90s and 00s. One reason for the change since then is that nowadays most things are more feature complete so low hanging fruits are not there (higher cost to entry for newcomers and fewer things to do if you're not "up there" as a developer), plus you need to "compete" or "collaborate" with a number of paid developers from companies like RedHat who handle most of the project and steer its direction wherever the company likes. Plus FOSS just isn't as glamorous as it used to be: The year of "Linux of the Desktop" (which as a vision meant mass adoption of Linux to win over Windows in desktops and laptops, not the "but Linux is fine for desktop use today, I've used it for years" or "but billions of Android phones use Linux behind the scenes" we settle'd for) never came. "Big" companies stopped caring about Linux except as an server thing (e.g. we had Easel, Corel, and others competing for desktop Linux, even IBM and others had an interest, whereas VA Linux was one of the biggest IPO stories). There are much less stories about the sexiness of FOSS or some FOSS future, and hackers changing everything working together, etc, in mainstream (not industry) media, etc.
- gowld 6y ago
  Linux on the "fleet" desktop almost happened (some companies and schools do it), but then webapps got big and ChromeOS swept in. Now the only reason to make a non-ChromeOS "webtop" fleet is if your organization is explicitly Google-free for competitive/legal/political reasons.
- Lichtso 6y ago
  Keep in mind that Linux is not the only free software project. Take a look at Blender (for example) which is turning the scene upside down. And, on the last point: Was there ever any mainstream media coverage of FOSS?
- dalbasal 6y ago
  You could call it the facebook effects nominally. The web is/was the key open technology of this era. Collaborative efforts had a lot of successes. Wikipedia turned out to be a good encyclopedia, which seemed laughable/impossible initially. Linux, obviously. The www itself. Open, collaborative, organized outside of a corporate body. These were big, impressive wins. AOL and others who tried to compete with openness head on failed. FB succeeded. Or rather, around that time a lot of these wins started being accumulated within a more standard corporate frame. FB controls who sees what and doesn't share. Twitter is a company that controls a protocol. Android is, philosophically and business-wise the anti-thesis to the goals you mentioned. "Hackers changing everything working together" is now stuff that HR talks about... entirely coopted.
jdbernard 6y ago
Another component, I think, is the difference between being able to work on whatever you want vs. having to follow a roadmap dictated by project leadership. Other commenters have touched on this (pointing out the decline of community orientation, rise of corporate oversight, or the fading glamour, for example), but no one has identified it directly. The article points out that we collectively re-examined Brook's rule, going from "adding more engineers to a project makes it ship later" to "if you need to get more work done, go attract more contributors." I think the evidence shows it's more nuanced. If you have a large amount of potential tasks and a large pool of potential contributors, and little strong preference for which tasks get done (OSS in the 90's maybe) then highly self-motivated people can pick up work they find interesting or which are useful for them. You can then avoid the exponential explosion of communication/coordination overhead by empowering those individual contributors to make their own decisions about how that work should be done. This is how OSS can scale to hundreds of contributors. Once you introduce some sort of top-down planning that dictates which tasks are important, the blessed approaches, strict coding patterns etc. then you demotivate a large number of contributors who don't share the top-down priorities and you have to pay the exponential overhead cost of distributed coordination to enforce the top-down priorities.
stonogo 6y ago
It used to be you could email a patch to someone and that patch would get reviewed and committed. Since then there has sprung up a fetish for bureaucratic process, and tooling has arisen to foster that, such that the barrier to entry is significantly higher now than it was in the 90s. All you needed was a text editor and an email account. Now you need accounts on whatever code forge is involved, and you didn't write enough unit tests, and you failed to properly format your pull request, and what do you mean you're not an expert in whatever version control we've adopted? This pull request sat in an open state because you didn't tag it appropriately. The CI system was updated and your code wasn't covered in the automated integration test, please resubmit after making minor punctuation changes in a metadata file. Over time, this stuff filters out contributors and leaves a core of people who are comfortable with the structure of the engineering process for that specific project. It's not a bad thing that engineering standards are higher, but it's no longer a world of people 'scratching an itch' -- there's yak shaving to be done, and the number of people willing to shave that yak has always been smaller than the number of people who are not. I don't think the author is right about things having been harder in the old days led to a higher proportion of the internet being qualified to contribute. I think it's much harder now to patch software upstream than it ever has been.
- nerdponx 6y ago
  In my experience, bureaucracy is mostly uncorrelated with the willingness of a project to accept a PR from a new contributor. Homebrew has very strict contribution requirements but has a huge number of contributors and is generally easy to contribute to. Meanwhile there are lots of little projects that go dead for lack of interest by the sole maintainer in reviewing and merging patches. Also this: Since then there has sprung up a fetish for bureaucratic process, and tooling has arisen to foster that, such that the barrier to entry is significantly higher now than it was in the 90s. All you needed was a text editor and an email account. Now you need accounts on whatever code forge is involved, and you didn't write enough unit tests, and you failed to properly format your pull request, and what do you mean you're not an expert in whatever version control we've adopted? This pull request sat in an open state because you didn't tag it appropriately. The CI system was updated and your code wasn't covered in the automated integration test, please resubmit after making minor punctuation changes in a metadata file. Doesn't really make sense to me. How many projects have you actually had this problem with? I don't think the author is right about things having been harder in the old days led to a higher proportion of the internet being qualified to contribute. I think it's much harder now to patch software upstream than it ever has been. For some people, using Git and Github is a barrier to entry. For others, submitting a patch to a mailing list is a barrier to entry.

z3rgl1ng 6y ago

+1, [Citation Needed].

z3rgl1ng 6y ago
Christ, every reply to this thread is an anecdote. Please back up assertions with _some_ data point..

bergie 6y ago

Totally anecdotal, but I feel open source projects were much more community oriented in the 90s and the 00s. But after that, it seems they’re either driven by a company, or an individual. That would roughly conform to the article’s timeline of GitHub adoption.

jrochkind1 6y ago
By "community-oriented", do you mean that the distribution of commits was such that more people were making more commits, instead of "95% of the work is done by a nucleus of people"? I'm not sure if those are the same thing, I'm just talking about the distribution of work.
gowld 6y ago
Everything has more "individuals" now, since everyone can get a soapbox/megaphone, from YouTube to GitHub to Twitter to all the rest. And it's possible to monetize that work now, with ads and micro-patrons, which is a huge drain on free and open source, mitigated only (partially) by the increased ability for open sourcerers to collaborate and that nowadays we're drowing in software and don't really need much more new software. You don't need to join a team to get noticed (and on the ugly side, even though having a team helps, it's better for the Personal Brand to make that team invisible)
nerdponx 6y ago
Counter-anecdote: most of the small open-source projects I use only have a small number of active contributors because there is a small number of people who actively try to contribute. Projects that stop accepting PRs tend to be pronounced "dead" after just a few months, but there isn't always someone able/willing to fork it and keep the project going. Maybe there are more projects to contribute to than before, so each project gets a smaller % of the "potential contributor" pool.

nitrogen 6y ago

For software of the 90s, 2000s Check Sourceforge, Freshmeat.net archives, IRC, old anonymous FTP mirrors, etc.

Spivak 6y ago

Users of software care about their own workflows, their pet-features, and the bugs they hit. Some are willing to put up the work to patch in those features or fix those bugs and upstream them. Fewer still are people who are invested enough to dedicate themselves to a project as a part-time job. I don't think this dynamic has changed since the dawn of open source.

coldtea 6y ago

>Any evidence this has actually changed between the 90s and now? Experience with the relevant communities (Gnome, KDE, and others, etc) in the 90s and 00s. One reason for the change since then is that nowadays most things are more feature complete so low hanging fruits are not there (higher cost to entry for newcomers and fewer things to do if you're not "up there" as a developer), plus you need to "compete" or "collaborate" with a number of paid developers from companies like RedHat who handle most of the project and steer its direction wherever the company likes. Plus FOSS just isn't as glamorous as it used to be: The year of "Linux of the Desktop" (which as a vision meant mass adoption of Linux to win over Windows in desktops and laptops, not the "but Linux is fine for desktop use today, I've used it for years" or "but billions of Android phones use Linux behind the scenes" we settle'd for) never came. "Big" companies stopped caring about Linux except as an server thing (e.g. we had Easel, Corel, and others competing for desktop Linux, even IBM and others had an interest, whereas VA Linux was one of the biggest IPO stories). There are much less stories about the sexiness of FOSS or some FOSS future, and hackers changing everything working together, etc, in mainstream (not industry) media, etc.

gowld 6y ago
Linux on the "fleet" desktop almost happened (some companies and schools do it), but then webapps got big and ChromeOS swept in. Now the only reason to make a non-ChromeOS "webtop" fleet is if your organization is explicitly Google-free for competitive/legal/political reasons.
Lichtso 6y ago
Keep in mind that Linux is not the only free software project. Take a look at Blender (for example) which is turning the scene upside down. And, on the last point: Was there ever any mainstream media coverage of FOSS?
dalbasal 6y ago
You could call it the facebook effects nominally. The web is/was the key open technology of this era. Collaborative efforts had a lot of successes. Wikipedia turned out to be a good encyclopedia, which seemed laughable/impossible initially. Linux, obviously. The www itself. Open, collaborative, organized outside of a corporate body. These were big, impressive wins. AOL and others who tried to compete with openness head on failed. FB succeeded. Or rather, around that time a lot of these wins started being accumulated within a more standard corporate frame. FB controls who sees what and doesn't share. Twitter is a company that controls a protocol. Android is, philosophically and business-wise the anti-thesis to the goals you mentioned. "Hackers changing everything working together" is now stuff that HR talks about... entirely coopted.

jdbernard 6y ago

Another component, I think, is the difference between being able to work on whatever you want vs. having to follow a roadmap dictated by project leadership. Other commenters have touched on this (pointing out the decline of community orientation, rise of corporate oversight, or the fading glamour, for example), but no one has identified it directly. The article points out that we collectively re-examined Brook's rule, going from "adding more engineers to a project makes it ship later" to "if you need to get more work done, go attract more contributors." I think the evidence shows it's more nuanced. If you have a large amount of potential tasks and a large pool of potential contributors, and little strong preference for which tasks get done (OSS in the 90's maybe) then highly self-motivated people can pick up work they find interesting or which are useful for them. You can then avoid the exponential explosion of communication/coordination overhead by empowering those individual contributors to make their own decisions about how that work should be done. This is how OSS can scale to hundreds of contributors. Once you introduce some sort of top-down planning that dictates which tasks are important, the blessed approaches, strict coding patterns etc. then you demotivate a large number of contributors who don't share the top-down priorities and you have to pay the exponential overhead cost of distributed coordination to enforce the top-down priorities.

stonogo 6y ago

It used to be you could email a patch to someone and that patch would get reviewed and committed. Since then there has sprung up a fetish for bureaucratic process, and tooling has arisen to foster that, such that the barrier to entry is significantly higher now than it was in the 90s. All you needed was a text editor and an email account. Now you need accounts on whatever code forge is involved, and you didn't write enough unit tests, and you failed to properly format your pull request, and what do you mean you're not an expert in whatever version control we've adopted? This pull request sat in an open state because you didn't tag it appropriately. The CI system was updated and your code wasn't covered in the automated integration test, please resubmit after making minor punctuation changes in a metadata file. Over time, this stuff filters out contributors and leaves a core of people who are comfortable with the structure of the engineering process for that specific project. It's not a bad thing that engineering standards are higher, but it's no longer a world of people 'scratching an itch' -- there's yak shaving to be done, and the number of people willing to shave that yak has always been smaller than the number of people who are not. I don't think the author is right about things having been harder in the old days led to a higher proportion of the internet being qualified to contribute. I think it's much harder now to patch software upstream than it ever has been.

nerdponx 6y ago
In my experience, bureaucracy is mostly uncorrelated with the willingness of a project to accept a PR from a new contributor. Homebrew has very strict contribution requirements but has a huge number of contributors and is generally easy to contribute to. Meanwhile there are lots of little projects that go dead for lack of interest by the sole maintainer in reviewing and merging patches. Also this: Since then there has sprung up a fetish for bureaucratic process, and tooling has arisen to foster that, such that the barrier to entry is significantly higher now than it was in the 90s. All you needed was a text editor and an email account. Now you need accounts on whatever code forge is involved, and you didn't write enough unit tests, and you failed to properly format your pull request, and what do you mean you're not an expert in whatever version control we've adopted? This pull request sat in an open state because you didn't tag it appropriately. The CI system was updated and your code wasn't covered in the automated integration test, please resubmit after making minor punctuation changes in a metadata file. Doesn't really make sense to me. How many projects have you actually had this problem with? I don't think the author is right about things having been harder in the old days led to a higher proportion of the internet being qualified to contribute. I think it's much harder now to patch software upstream than it ever has been. For some people, using Git and Github is a barrier to entry. For others, submitting a patch to a mailing list is a barrier to entry.