3 ms·

Types are controversial because we can't measure engineer productivity. Full stop. I see people on here arguing that they are faster one way or the other, yet w

Types are controversial because we can't measure engineer productivity. Full stop. I see people on here arguing that they are faster one way or the other, yet we have absolutely no empirical evidence for such a claim. What makes it more complex is that "fighting with types" often takes one out of the flow in a different way than usual progamming challenges. Since it's unmeasurable, we cannot see the effect of productivity relative to team size, feature churn, product timeline, unit test discipline, etc. For example, I'm completely convinced types add significant productivity increase to any team of more than two programmers. Likewise types are a net productivity improvement to software that has to be added to multiple times in a year. If it's a one-off write and throw away, or tiny team, it probably doesn't help as much. Also if it's a big team, but with highly siloed developers, likely doesn't add as much value. I think fighting with types FEELS slower than it is, like how dealing with a car that doesn't start immediately or a street with a lot of stop signs feels slower than just getting out and walking, despite the empirical evidence to the contrary. However, these beliefs of mine are BACKED BY ZERO EVIDENCE. Just let that sink in. We all are arguing about a topic that inherently cannot be measured. We should be trying to tackle the measurement issue first, rather than just keep yelling about chocolate vs vanilla forever. When all we have to go on is feelings, we get people arguing which is better: cars or bicycles, without any discussion or facts about top speed or total distance. The cyclist is always going to talk about wind in their hair feeling like they are going so fast, or how it slows them down to look for gas stations and just stand there pumping gas when they could be making progress pedaling. And to stretch the metaphor even further, there are plenty of environments when a sedan is slower than a mountain bike, and plenty where they are the same. I suspect all this is true with types, yet we have absolutely no way to know. For all we know, the "collective wisdom" of types might be exactly backwards, because basing engineering decisions on feelings rarely correlates to empirical results.

fulafel 6y ago
I think from an empirical mindset we have to grant that the lack of evidence probably means there is no significant advantage. Because there have been lots of studies, we can't simply say we don't know if there's an effect, or we don't know how to measure it.
- JackMorgan 6y ago
  Don't conflate the inability to measure productivity with the conclusion that for this one issue the differences can't be that large. That's like saying, "just because we can't measure distance or velocity, rockets and cars probably are similar speeds, or it depends on the driver's personality." It's empirically unsolved how productive an engineer even is, therefore all other discussions are 100% subjective. We don't know if there's an effect, because we have no way at all to measure it without costs so staggeringly large no one will ever try. We'd need similar engineers, doing all the same practices, with the same stories, and just one difference. But then the engineers would need to have been selected to be similar skill and speed previously, which would require doing other projects with all fixed practices and languages to measure developer skill relative to peers. This also doesn't take into account team cooperation and morale, creative thinking around problem solving, etc. It's a bummer, but without some breakthrough in AI research allowing developers to be measured based on the code they write, it's seeming more and more like an unsolvable problem.
adnzzzzZ 6y ago
I personally believe that productivity differences depend mostly on the programmer's personality. Certain types of personalities will be more productive with statically typed systems, others with more dynamically typed ones. If this is the case then the measurement problem becomes one of personality assessment rather than anything else, and even when we manage to get this measured correctly, because it's a personality issue, we'll still have people arguing cars or bicycles without facts being able to help at all (except to point that it's mostly a matter of preference so the discussion is pointless).
boyakasha 6y ago
Exactly. There is a lot of post-hoc rationalisation happening, along the lines of: > I like language/technology X. Why? Well I'm obviously a super smart and rational person, so my preferences could only be based on facts and logic. Therefore everything I like must be objectively correct. QED. Anecdotally, the more someone prides themselves on being intelligent and logical, the less skeptical they are of their unsubstantiated emotional judgements. Despite primarily working in dynlangs, I would guess that type systems affect developer productivity somewhere in the range of +-50%. While I have pulled that number directly from my ass, I think that if the effect size was larger then there wouldn't be much of a debate. If a company could swap languages for any given team and get 3x more functionality, or get the same work done for 1/3rd of the cost, we would know about it already. And to be clear, 3x productivity is achievable, but not by simply switching to Haskell.
- JackMorgan 6y ago
  Keep in mind though, the statement that "and get 3x the functionality" <- right there, that's what is unmeasurable. We can't know. We have no unit of measurement, so no one could possibly know. A practice could be 100x faster, but because software is rarely as important as marketing, sales, and luck, we'd never know. A dev team that's 100x slower could still win, and it probably happens regularly. It's the essential issue with all tech discussions. We assume that "hand waving" someone would notice. But without any unit of measurement, I posit that no one ever will. There are so many network effects that change the success of a software project that is unlikely that even in another 50 years we will have consensus.

fulafel 6y ago

I think from an empirical mindset we have to grant that the lack of evidence probably means there is no significant advantage. Because there have been lots of studies, we can't simply say we don't know if there's an effect, or we don't know how to measure it.

JackMorgan 6y ago
Don't conflate the inability to measure productivity with the conclusion that for this one issue the differences can't be that large. That's like saying, "just because we can't measure distance or velocity, rockets and cars probably are similar speeds, or it depends on the driver's personality." It's empirically unsolved how productive an engineer even is, therefore all other discussions are 100% subjective. We don't know if there's an effect, because we have no way at all to measure it without costs so staggeringly large no one will ever try. We'd need similar engineers, doing all the same practices, with the same stories, and just one difference. But then the engineers would need to have been selected to be similar skill and speed previously, which would require doing other projects with all fixed practices and languages to measure developer skill relative to peers. This also doesn't take into account team cooperation and morale, creative thinking around problem solving, etc. It's a bummer, but without some breakthrough in AI research allowing developers to be measured based on the code they write, it's seeming more and more like an unsolvable problem.

adnzzzzZ 6y ago

I personally believe that productivity differences depend mostly on the programmer's personality. Certain types of personalities will be more productive with statically typed systems, others with more dynamically typed ones. If this is the case then the measurement problem becomes one of personality assessment rather than anything else, and even when we manage to get this measured correctly, because it's a personality issue, we'll still have people arguing cars or bicycles without facts being able to help at all (except to point that it's mostly a matter of preference so the discussion is pointless).

boyakasha 6y ago

Exactly. There is a lot of post-hoc rationalisation happening, along the lines of: > I like language/technology X. Why? Well I'm obviously a super smart and rational person, so my preferences could only be based on facts and logic. Therefore everything I like must be objectively correct. QED. Anecdotally, the more someone prides themselves on being intelligent and logical, the less skeptical they are of their unsubstantiated emotional judgements. Despite primarily working in dynlangs, I would guess that type systems affect developer productivity somewhere in the range of +-50%. While I have pulled that number directly from my ass, I think that if the effect size was larger then there wouldn't be much of a debate. If a company could swap languages for any given team and get 3x more functionality, or get the same work done for 1/3rd of the cost, we would know about it already. And to be clear, 3x productivity is achievable, but not by simply switching to Haskell.

JackMorgan 6y ago
Keep in mind though, the statement that "and get 3x the functionality" <- right there, that's what is unmeasurable. We can't know. We have no unit of measurement, so no one could possibly know. A practice could be 100x faster, but because software is rarely as important as marketing, sales, and luck, we'd never know. A dev team that's 100x slower could still win, and it probably happens regularly. It's the essential issue with all tech discussions. We assume that "hand waving" someone would notice. But without any unit of measurement, I posit that no one ever will. There are so many network effects that change the success of a software project that is unlikely that even in another 50 years we will have consensus.