3 ms·
Hello! I have done this sort of research internally at companies and have coordinated with the world’s leading researchers in developer productivity for a long
by mkanat 2y ago
Hello! I have done this sort of research internally at companies and have coordinated with the world’s leading researchers in developer productivity for a long time.
Myself and every other leading researcher in the field that I have spoken to have serious concerns about this supposed “research.” For example, you can see a post here from Margaret Anne-Storey, who is arguably the world’s leading researcher in developer productivity:
https://www.linkedin.com/posts/margaret-anne-storey-8419462_ghost-engineers-a-relic-of-the-past-in-developer-activity-7271246185674203137-O6NI https://www.linkedin.com/posts/margaret-anne-storey-8419462_...
I share her concerns. I have asked Yegor multiple times when he plans to publish his research methodology for review by others and he has not even answered my question. That is, he hasn’t even said “it’s under review and we will publish it by _____.” He’s certainly argued with me about the validity of his results, but suspiciously has not actually published any research methodology as far as I have been able to find.
This is very concerning, because I have worked with this type of data set extensively. I am possibly one of the world’s leading experts in using and analyzing this type of data, and I can tell you that it is fraught with data anomalies of the sort that would easily cause an incorrect result here. For example, you might believe all developers are working in GitHub, but then it turns out the company has 10% of developers contributing to some other source control system. Or one of your GitHub orgs isn’t instrumented correctly. Or some of those people aren’t actually software engineers even though they have a title that _looks_ like that. The list goes on and on, and these are _real_ problems that happen in the data at _every_ company.
Im frankly shocked that the Washington Post or any other credible media organization is publishing this without any verification or background on the validity of the research.
I am fully willing to believe that some engineers don’t do work, or do very little. I’ve seen it. But if you’re going to claim that you did research for a University, with the University’s name on it, I expect things like a published paper, understanding the margin of error on your published numbers, etc. especially in a field that is so prone to errors in the data.