5 ms·

It's what I call an "AHA" piece of statistics. Lots of data right there, and nicely visualised at that, only what it actually means is unfathomable without kno

by V-2 12y ago

It's what I call an "AHA" piece of statistics. Lots of data right there, and nicely visualised at that, only what it actually means is unfathomable without knowing any broader context. For instance: C++ has the greatest number of opened issues per repository, then comes Rust, then Scala. All right. Does it indicate that they're more tricky than others and hence more bug reports? Or perhaps that projects written in these languages are under more intense scrutiny? Or that people watching these repositories are just more eager to step up and file an issue instead of sulking in silence (a trait of programming culture surrounding these languages)? And so on, and so on. Or it could be one in case of C++, another in case of Rust - since they differ under so many aspects. Wide field for wildest speculations, but no meaningful correlations identified.

steveklabnik 12y ago
Rust is one of the heavier users of Issues. I've started a pretty huge re categorization and triage effort, and when I started, we had around 2200 open. Earlier today I got us to 1899. Part of this is that we tracked things like 'it would be nice to have a library to do things like X,' and in preparation for 1.0, we're moving those over to our RFC repo instead, so that the main project has only bugs and internal enhancements.
- V-2 12y ago
  Yes, I imagine, because it's a relatively new language. In case of other ones the reasons are probably of entirely different nature.
- steveklabnik 12y ago
  ... I guess I should also mention that not all of them are these kinds of issues, there's also lots of stuff, like in any tracker, that got fixed somewhere, but the issue was never closed. Or where a library that used to exist no longer does. Etc. Triage is drudge work, but really valuable for any big project.
nly 12y ago
> C++ has the greatest number of opened issues per repository, then comes Rust, then Scala. All right. > Does it indicate that they're more tricky than others and hence more bug reports? They're all static languages. In fact, so are C# and Java (the next two of the top 5). Two of these (C++ and Rust) are also in the top 5 for pushes per repo. C++ being top overall. Imho, some of these stats needs to be scaled with respect to LOC. The push/issue ratio might be good as well.
burntsushi 12y ago
Yes, context is key! In addition to what steveklabnik said about Rust, the other interesting bit of context is that the deluge of breaking changes[1] has resulted in a very interesting phenomenon: everyone chips in and submits PRs to library dependencies that haven't been updated by their authors yet. The pace is so fast that most of the time, in my case, it's just a matter of "I haven't gotten home from work yet and there are already 3 PRs against my repos with fixes for the latest Rust nightly." As a result, many of my Rust projects have an oddly disproportionate number of contributors/pull requests relative to other projects I have that are more popular. [1] - Breaking changes have simmered down a lot in the last few weeks. We still have one more semi-big one in front of us, but hopefully smooth sailing from there...
- madsravn 12y ago
  Exactly. And I feel this graph gives an idea of what the tendencies for a language is, and then you can research the context. You can glimpse a lot of fun things from this graph.
_pmf_ 12y ago
> Does it indicate that they're more tricky than others and hence more bug reports? Or that these projects are actually used.
amirmc 12y ago
Indeed. This is a nice example of data porn, in that we can pore over graphs and trends and make lots of speculations (which will conveniently confirm our pre-existing biases). It's visualisation, not analysis so we should take it with a large pinch of salt. Having said that, I do think the visualisation is beautiful and there are definitely useful things that could be drawn from the data if someone were willing to do the extra work. However, I'm not sure I have much faith in the data quality e.g. some of my repos are considered 'CSS' just because I've added some boilerplate from elsewhere.
amyjess 12y ago
IIRC, another thing that skews the numbers is that GitHub pages and the like count as JavaScript code.
- vorg 12y ago
  And 10-line Gradle build files count as Groovy code.
3JPLW 12y ago
What's interesting to me is that there seems to be an inverse correlation between pushes per repository and the number of forks. Are forks counted as individual repositories? That would be a boring explanation.
mVChr 12y ago
For a scientific paper, you are correct. For practical or business purposes, this is a nice bit of incomplete information to help make a decision. I want to take a serious, time-invested dive into a new statically compiled language, but which one should I pick? An old die-hard or the new-hotness? I could make a guess from reading the docs and such, but I'd also want to know community activity and support. This is a handy chart for getting a sense of that. Or, I'm a business owner who just hired my first engineer. He's saying the backend should definitely be built in Groovy, or maybe he'd be willing to do Scala or something else, but definitely Groovy, yeah, Groovy man. I might be able to get a better idea of which would be beneficial for my long-term business prospects (hiring more engineers, etc) by checking out a chart like this as I might not have time to do real in-depth research. As a scientist you require complete, sound and accurate statistical data. As a business practice (this site is about start-ups, no?) you need to be comfortable making serious and important decisions based off of incomplete and possibly inaccurate information because making fast decisions is often paramount. You can and will always make other fast decisions later and decide whether it's worth the effort to course-correct if you need to due to new and more accurate information. This is maybe too deep an analysis of a fun little infographic, but as a former professional poker player who made a living off of incomplete information you got my cockles up.
- V-2 12y ago
  I'd also want to know community activity and support. This is a handy chart for getting a sense of that. The entire chart? Wouldn't the first column be sufficient? Number of repositories gives you some idea about language popularity. Well, kind of: there's bias of hype here. Obviously choices behind open-sourced projects on GitHub aren't representative for the industry. It's the software's world avant-garde, if anything. And even so, that's just one parameter out of five, and it can be very well be considered in isolation from all the rest. I wouldn't make business decisions based, for instance, on the average number of open issues. Because it's an outcome of many different variables. So how would you know what it means? Is high good? Is high bad? Interrelations between data - shown by this clever chart - are even more mysterious. TeX has a very high number of pushes per repository (second best), while there's fairly few repositories, and they are rarely forked. At the same time R has low number of pushes (second lowest of all), whereas it wins in the "new forks per repository" category (#1). What do you make of that, businesswise?

by V-2 12y ago

steveklabnik 12y ago

Rust is one of the heavier users of Issues. I've started a pretty huge re categorization and triage effort, and when I started, we had around 2200 open. Earlier today I got us to 1899. Part of this is that we tracked things like 'it would be nice to have a library to do things like X,' and in preparation for 1.0, we're moving those over to our RFC repo instead, so that the main project has only bugs and internal enhancements.

V-2 12y ago
Yes, I imagine, because it's a relatively new language. In case of other ones the reasons are probably of entirely different nature.
steveklabnik 12y ago
... I guess I should also mention that not all of them are these kinds of issues, there's also lots of stuff, like in any tracker, that got fixed somewhere, but the issue was never closed. Or where a library that used to exist no longer does. Etc. Triage is drudge work, but really valuable for any big project.

nly 12y ago

> C++ has the greatest number of opened issues per repository, then comes Rust, then Scala. All right. > Does it indicate that they're more tricky than others and hence more bug reports? They're all static languages. In fact, so are C# and Java (the next two of the top 5). Two of these (C++ and Rust) are also in the top 5 for pushes per repo. C++ being top overall. Imho, some of these stats needs to be scaled with respect to LOC. The push/issue ratio might be good as well.

burntsushi 12y ago

Yes, context is key! In addition to what steveklabnik said about Rust, the other interesting bit of context is that the deluge of breaking changes[1] has resulted in a very interesting phenomenon: everyone chips in and submits PRs to library dependencies that haven't been updated by their authors yet. The pace is so fast that most of the time, in my case, it's just a matter of "I haven't gotten home from work yet and there are already 3 PRs against my repos with fixes for the latest Rust nightly." As a result, many of my Rust projects have an oddly disproportionate number of contributors/pull requests relative to other projects I have that are more popular. [1] - Breaking changes have simmered down a lot in the last few weeks. We still have one more semi-big one in front of us, but hopefully smooth sailing from there...

madsravn 12y ago
Exactly. And I feel this graph gives an idea of what the tendencies for a language is, and then you can research the context. You can glimpse a lot of fun things from this graph.

_pmf_ 12y ago

> Does it indicate that they're more tricky than others and hence more bug reports? Or that these projects are actually used.

amirmc 12y ago

Indeed. This is a nice example of data porn, in that we can pore over graphs and trends and make lots of speculations (which will conveniently confirm our pre-existing biases). It's visualisation, not analysis so we should take it with a large pinch of salt. Having said that, I do think the visualisation is beautiful and there are definitely useful things that could be drawn from the data if someone were willing to do the extra work. However, I'm not sure I have much faith in the data quality e.g. some of my repos are considered 'CSS' just because I've added some boilerplate from elsewhere.

amyjess 12y ago

IIRC, another thing that skews the numbers is that GitHub pages and the like count as JavaScript code.

vorg 12y ago
And 10-line Gradle build files count as Groovy code.

3JPLW 12y ago

What's interesting to me is that there seems to be an inverse correlation between pushes per repository and the number of forks. Are forks counted as individual repositories? That would be a boring explanation.

mVChr 12y ago

For a scientific paper, you are correct. For practical or business purposes, this is a nice bit of incomplete information to help make a decision. I want to take a serious, time-invested dive into a new statically compiled language, but which one should I pick? An old die-hard or the new-hotness? I could make a guess from reading the docs and such, but I'd also want to know community activity and support. This is a handy chart for getting a sense of that. Or, I'm a business owner who just hired my first engineer. He's saying the backend should definitely be built in Groovy, or maybe he'd be willing to do Scala or something else, but definitely Groovy, yeah, Groovy man. I might be able to get a better idea of which would be beneficial for my long-term business prospects (hiring more engineers, etc) by checking out a chart like this as I might not have time to do real in-depth research. As a scientist you require complete, sound and accurate statistical data. As a business practice (this site is about start-ups, no?) you need to be comfortable making serious and important decisions based off of incomplete and possibly inaccurate information because making fast decisions is often paramount. You can and will always make other fast decisions later and decide whether it's worth the effort to course-correct if you need to due to new and more accurate information. This is maybe too deep an analysis of a fun little infographic, but as a former professional poker player who made a living off of incomplete information you got my cockles up.

V-2 12y ago
I'd also want to know community activity and support. This is a handy chart for getting a sense of that. The entire chart? Wouldn't the first column be sufficient? Number of repositories gives you some idea about language popularity. Well, kind of: there's bias of hype here. Obviously choices behind open-sourced projects on GitHub aren't representative for the industry. It's the software's world avant-garde, if anything. And even so, that's just one parameter out of five, and it can be very well be considered in isolation from all the rest. I wouldn't make business decisions based, for instance, on the average number of open issues. Because it's an outcome of many different variables. So how would you know what it means? Is high good? Is high bad? Interrelations between data - shown by this clever chart - are even more mysterious. TeX has a very high number of pushes per repository (second best), while there's fairly few repositories, and they are rarely forked. At the same time R has low number of pushes (second lowest of all), whereas it wins in the "new forks per repository" category (#1). What do you make of that, businesswise?