4 ms·

Doesn't work. "When a measure becomes a target, it ceases to be a good measure." You can totally build something that is a decent metric, but as soon as you cr

Doesn't work. "When a measure becomes a target, it ceases to be a good measure." You can totally build something that is a decent metric, but as soon as you create incentives to "game it" (optimize for the metric not the actual goal you're trying to measure), it will be gamed, and creating metrics that are resistant to that is nearly impossible in most cases.

acover 5y ago
Edit: Nvm, I misunderstood the initial problem and my solution doesn't address it. Why not just keep commits as a metric, then estimate the quality of those commits by sampling. If the organization appears to be gaming the stats then flag it as such.
- simonh 5y ago
  Because that would be a huge time sink for the maintainers as well. They’re the only ones qualified to evaluate quality and they have better things to do with the time it would take. This problem is created by these companies and it’s their problem to solve.
- jart 5y ago
  I tend to look on commit count as a negative when reviewing open source projects. TeX is probably the highest quality open source project and it's has had maybe 15 commits since I was born. Bash has had about 15 commits in the last five years. Then there's everyone else who's playing to win the Github game with 500 stars and 5,000 commits. That level of activity might impress normal people but it doesn't impress me.
  - sdesol 5y ago
    > I tend to look on commit count as a negative when reviewing open source projects. I do agree that tracking commit counts isn't very useful but tracking commit frequency, in my opinion, is quite useful. I'm obviously biased but I use commit frequency to tell me how fast a project is moving and what is the investment. Take the following for example: https://public-001.gitsense.com/insights/github/repos?p=impact&r=github%3Amicrosoft%2Fvscode&v=repo https://public-001.gitsense.com/insights/github/repos?p=impa... by breaking down how frequently contributors commit on a daily basis, you can get a very good sense of investment and speed. In the case of the vscode project, Microsoft is investing a lot of resources into vscode and it is evolving at an extremely fast pace. Software metrics in general isn't the issue, it's the lack of context around them that is the issue. Disclaimer: I'm the creator of the tool that I'm linking to

acover 5y ago

Edit: Nvm, I misunderstood the initial problem and my solution doesn't address it. Why not just keep commits as a metric, then estimate the quality of those commits by sampling. If the organization appears to be gaming the stats then flag it as such.

simonh 5y ago
Because that would be a huge time sink for the maintainers as well. They’re the only ones qualified to evaluate quality and they have better things to do with the time it would take. This problem is created by these companies and it’s their problem to solve.
jart 5y ago
I tend to look on commit count as a negative when reviewing open source projects. TeX is probably the highest quality open source project and it's has had maybe 15 commits since I was born. Bash has had about 15 commits in the last five years. Then there's everyone else who's playing to win the Github game with 500 stars and 5,000 commits. That level of activity might impress normal people but it doesn't impress me.
- sdesol 5y ago
  > I tend to look on commit count as a negative when reviewing open source projects. I do agree that tracking commit counts isn't very useful but tracking commit frequency, in my opinion, is quite useful. I'm obviously biased but I use commit frequency to tell me how fast a project is moving and what is the investment. Take the following for example: https://public-001.gitsense.com/insights/github/repos?p=impact&r=github%3Amicrosoft%2Fvscode&v=repo https://public-001.gitsense.com/insights/github/repos?p=impa... by breaking down how frequently contributors commit on a daily basis, you can get a very good sense of investment and speed. In the case of the vscode project, Microsoft is investing a lot of resources into vscode and it is evolving at an extremely fast pace. Software metrics in general isn't the issue, it's the lack of context around them that is the issue. Disclaimer: I'm the creator of the tool that I'm linking to