4 ms·

The Fastest Growing Open Source Project?

28 points by maximesalomon 13y ago

mkal_tsr 13y ago
How can they say they're the fastest growing when Popcorn Time nearly doubled the unique contributors (95-ish to 40ish) in nearly the same time? Seems rather disingenuous to include PopcornTime clearly showing it's much younger than CodeCombat, showing at the same age of being open-source it has nearly double the unique contributors, then claiming it's growing slower than CodeCombat. Don't get me wrong, what CodeCombat is doing is great and all the well wishes to them, but I don't think they're interpreting the data very fairly.
- stonogo 13y ago
  The Fastest-Growing Open Source Project As Determined By Our Incredibly Specific and Sometimes Downright Strange Metrics
  - nwinter 13y ago
    Unique contributors since open source launch is strange?
    - stonogo 13y ago
      "Unique contributors within inital 86-day window of project upload to Github" is not equivalent to "unique contributors since open-source launch."
      - schmatz 13y ago
        You're right, that's true. It's not the ideal metric, but it's one that is realistic to compute with limited time and resources. To analyze that statistic using a growth metric like "largest 30 day average contributor gain" across such a gigantic dataset, consisting of hundreds of millions of rows and tens of millions of unique repositories, requires resources and engineer time which few people possess. We certainly don't want to divert too much of either of those things away from development of CodeCombat just for a fun data science experiment!
        stonogo 13y ago
        So you narrowed the problem set down to one that was sufficiently effortless, then uploaded a blog post declaring yourself world champions. I'm not sure that makes anything better. "Fastest-growing open-source project, as far as we care" is a perfectly fine headline. What you've got now is disingenuous and baselessly self-congratulatory.
- schmatz 13y ago
  Hi mkal_tsr, you have a valid point there! Popcorn Time is awesome, and if it continues (unlike the other Popcorn Time repository) it will eventually pass CodeCombat. However, it might also experience some event or plateau, like Pullup did, that will stop its growth. A really interesting way to analyze the GitHub Timeline would be to rank repositories based on sustained average growth rate. We chose a fast, (sort of) cheap, and computationally easy way to run these statistics, but there are a myriad of other ways to process the same dataset, which we'd definitely be interested in seeing.
dalek2point3 13y ago
erm, and also the fact that MANY MANY open source projects are not on github? Linux, apache, android, some of the most influential projects are missing. This should be titled, "the fastest growing open source project by people who are not too important to be hosting on Github".
- schmatz 13y ago
  Actually, the Linux kernel, Android, and Apache are all on GitHub, and were analyzed (though the latter two are mirrors, so I guess not purely on GitHub.) For projects like Linux, while it may have 3500+ contributors, it got that way over 23 years. We also made the assumption that due to the social nature of GitHub, its meteoric growth, and the visibility that it brings open source projects, the fastest growing open source projects are on GitHub. We presume this assumption is correct, but don't have the data (or even know where we could get the data) to verify it.
dalek2point3 13y ago
Also see: http://xkcd.com/1102/ http://xkcd.com/1102/
AndyNemmity 13y ago
I was pretty confident I knew that openstack was the fastest growing open source project. They didn't show up in the answers, so maybe the criteria was such that it was cut out?
- schmatz 13y ago
  Looking at OpenStack, it does look like an aggregate of all of the organization's repositories would be definitely the fastest growing. However, we looked at individual repositories, so that's why it's not on the graph. It would be interesting see the growth of the big open source organizations (GNU vs Apache vs OpenStack etc.)
  - kiallmacinnes 13y ago
    OpenStack kinda has a single repo - every commit to the core projects triggers a submodule update in https://github.com/openstack/openstack https://github.com/openstack/openstack - and there are LOTS of projects which aren't included in this. For example CI and Developer tooling etc. > openstack$ git count -pm -n3 > 2014-03-01 2608 > 2014-02-01 2545 > 2014-01-01 2173 That looks OpenStack has 7326 commits in the previous 3 months. > codecombat$ git count -pm -n3 > 2014-03-01 642 > 2014-02-01 426 > 2014-01-01 427 The same tool gives me 1495 commits in the previous 3 months for CodeCombat. (The tool I used is https://github.com/moskytw/git-count https://github.com/moskytw/git-count )
    - kiallmacinnes 13y ago
      Replying to my own comment feels odd.. whatever ;) If you're interested in numbers[1] shows the commits currently running through the OpenStack CI system at this very moment in time.. I'm counting 125 when I look at the page. In the last 24 hours, OpenStack's CI system has peaked at just under 1000 jobs ran in an hour, an eyeballed average looks like to be 450 jobs every hour for the last 24 hours. For build slaves over the same period, It looks like about 750 Jenkins slaves. Oh and - OpenStack has a strict 1 change == 1 commit policy. So "Fix typo in previous commit" never happens .. The actually reduces the overall number of commits ;) [1]: http://status.openstack.org/zuul/ http://status.openstack.org/zuul/
chriseppstein 13y ago
Navel gazing is the best way to convince yourself that things are going great and to give you that boost to keep working on the project. At some point you will probably look back at this and think it was nothing to brag about, but it doesn't matter because you're having fun and enjoying the feedback loop between you and your users -- this is a good sign.
KevinBongart 13y ago
Worth noting: there's a Code Combat challenge on ChallengePost: http://codecombat.challengepost.com http://codecombat.challengepost.com You have 50 more days to build a working parser and help others learn to code in new languages playing CodeCombat

28 points by maximesalomon 13y ago

How can they say they're the fastest growing when Popcorn Time nearly doubled the unique contributors (95-ish to 40ish) in nearly the same time? Seems rather disingenuous to include PopcornTime clearly showing it's much younger than CodeCombat, showing at the same age of being open-source it has nearly double the unique contributors, then claiming it's growing slower than CodeCombat. Don't get me wrong, what CodeCombat is doing is great and all the well wishes to them, but I don't think they're interpreting the data very fairly.

stonogo 13y ago
The Fastest-Growing Open Source Project As Determined By Our Incredibly Specific and Sometimes Downright Strange Metrics
- nwinter 13y ago
  Unique contributors since open source launch is strange?
  - stonogo 13y ago
    "Unique contributors within inital 86-day window of project upload to Github" is not equivalent to "unique contributors since open-source launch."
    - schmatz 13y ago
      You're right, that's true. It's not the ideal metric, but it's one that is realistic to compute with limited time and resources. To analyze that statistic using a growth metric like "largest 30 day average contributor gain" across such a gigantic dataset, consisting of hundreds of millions of rows and tens of millions of unique repositories, requires resources and engineer time which few people possess. We certainly don't want to divert too much of either of those things away from development of CodeCombat just for a fun data science experiment!
      - stonogo 13y ago
        So you narrowed the problem set down to one that was sufficiently effortless, then uploaded a blog post declaring yourself world champions. I'm not sure that makes anything better. "Fastest-growing open-source project, as far as we care" is a perfectly fine headline. What you've got now is disingenuous and baselessly self-congratulatory.
schmatz 13y ago
Hi mkal_tsr, you have a valid point there! Popcorn Time is awesome, and if it continues (unlike the other Popcorn Time repository) it will eventually pass CodeCombat. However, it might also experience some event or plateau, like Pullup did, that will stop its growth. A really interesting way to analyze the GitHub Timeline would be to rank repositories based on sustained average growth rate. We chose a fast, (sort of) cheap, and computationally easy way to run these statistics, but there are a myriad of other ways to process the same dataset, which we'd definitely be interested in seeing.

dalek2point3 13y ago

erm, and also the fact that MANY MANY open source projects are not on github? Linux, apache, android, some of the most influential projects are missing. This should be titled, "the fastest growing open source project by people who are not too important to be hosting on Github".

schmatz 13y ago
Actually, the Linux kernel, Android, and Apache are all on GitHub, and were analyzed (though the latter two are mirrors, so I guess not purely on GitHub.) For projects like Linux, while it may have 3500+ contributors, it got that way over 23 years. We also made the assumption that due to the social nature of GitHub, its meteoric growth, and the visibility that it brings open source projects, the fastest growing open source projects are on GitHub. We presume this assumption is correct, but don't have the data (or even know where we could get the data) to verify it.

dalek2point3 13y ago

Also see: http://xkcd.com/1102/ http://xkcd.com/1102/

AndyNemmity 13y ago

I was pretty confident I knew that openstack was the fastest growing open source project. They didn't show up in the answers, so maybe the criteria was such that it was cut out?

schmatz 13y ago
Looking at OpenStack, it does look like an aggregate of all of the organization's repositories would be definitely the fastest growing. However, we looked at individual repositories, so that's why it's not on the graph. It would be interesting see the growth of the big open source organizations (GNU vs Apache vs OpenStack etc.)
- kiallmacinnes 13y ago
  OpenStack kinda has a single repo - every commit to the core projects triggers a submodule update in https://github.com/openstack/openstack https://github.com/openstack/openstack - and there are LOTS of projects which aren't included in this. For example CI and Developer tooling etc. > openstack$ git count -pm -n3 > 2014-03-01 2608 > 2014-02-01 2545 > 2014-01-01 2173 That looks OpenStack has 7326 commits in the previous 3 months. > codecombat$ git count -pm -n3 > 2014-03-01 642 > 2014-02-01 426 > 2014-01-01 427 The same tool gives me 1495 commits in the previous 3 months for CodeCombat. (The tool I used is https://github.com/moskytw/git-count https://github.com/moskytw/git-count )
  - kiallmacinnes 13y ago
    Replying to my own comment feels odd.. whatever ;) If you're interested in numbers[1] shows the commits currently running through the OpenStack CI system at this very moment in time.. I'm counting 125 when I look at the page. In the last 24 hours, OpenStack's CI system has peaked at just under 1000 jobs ran in an hour, an eyeballed average looks like to be 450 jobs every hour for the last 24 hours. For build slaves over the same period, It looks like about 750 Jenkins slaves. Oh and - OpenStack has a strict 1 change == 1 commit policy. So "Fix typo in previous commit" never happens .. The actually reduces the overall number of commits ;) [1]: http://status.openstack.org/zuul/ http://status.openstack.org/zuul/

chriseppstein 13y ago

Navel gazing is the best way to convince yourself that things are going great and to give you that boost to keep working on the project. At some point you will probably look back at this and think it was nothing to brag about, but it doesn't matter because you're having fun and enjoying the feedback loop between you and your users -- this is a good sign.

KevinBongart 13y ago

Worth noting: there's a Code Combat challenge on ChallengePost: http://codecombat.challengepost.com http://codecombat.challengepost.com You have 50 more days to build a working parser and help others learn to code in new languages playing CodeCombat