3 ms·

It's near impossible to objectively measure anything when it comes to programming as projects are neither repeatable nor comparable and test subjects are never

It's near impossible to objectively measure anything when it comes to programming as projects are neither repeatable nor comparable and test subjects are never blank slates. I have read quite a few studies, and I always come away thinking that the methods used are inadequate to support any conclusions at all. But does that mean there is no causal effect of programming language choice? I don't know and I'm not making any claims either way. I was talking about abstraction in one person's mind vs abstraction shared between a group a people. And what I said could actually be the reason why the choice of programming language has little effect (if that is the case)

pron 7y ago
> But does that mean there is no causal effect of programming language choice? I don't know and I'm not making any claims either way. I'm also not making definitive claims, but we can't ignore observations, either. While small effects can appear or disappear depending on methodology, big effects are easy to find and hard to hide, especially in an environment with strong selection pressures. Therefore, the most likely explanation to why no big effect has been detected -- either in studies or in industry -- is that there isn't one, and that should at least be enough to make it the working hypothesis. So before explaining effects that have yet to be detected we should establish that they exist at all, something we have so far failed to do. There may well be a small effect, and there could still be a large effect, although the chance of that is small unless we have a really good explanation to why we haven't found it.
- fauigerzigerk 7y ago
  Here are some reasons to doubt this working hypothesis: 1) It is inconsistent with the personal experience of most developers. Few would claim that C is just as productive and safe as some garbage collected language (provided that both are suitable for the task at hand), or that a SQL group by statement is not more productive to write than the equivalent procedural code. 2) The methods used to study the subject are completely unfit for purpose. 3) It is inconsistent with the few results that seem at least somewhat well supported because they are simple enough to measure, such as the constant bug count per line of code.
  - pron 7y ago
    > It is inconsistent with the personal experience of most developers. I disagree. I've been programming for about 30 years now, and most reports I've received -- both from programmers and managers -- are that language doesn't matter; certainly not much. Carefully selected personal "feeling" report are worthless (especially in biased forums), as that would produce even more evidence that homeopathy is effective than that programming languages make a difference. > Few would claim that C is just as productive and safe as some garbage collected language I agree, but that's where the many caveats come in. First, my argument is more one of diminishing returns (following Brooks's theory). I.e., languages make an increasingly small difference. C may have been a big improvement over assemby, and Java may have been a smaller but still quite big improvement over C, but now differences are quite small. Indeed, the effect of GC and memory safety vs C was one that was almost immediately detected by industry (once mature, performant, etc.) and triggered a huge shift. We do not see such shifts now. Companies move among mainstream languages, among non-mainstream languages, and between the two classes in a way that does not suggest a big effect at all. > or that a SQL group by statement is not more productive to write than the equivalent procedural code. That's a completely different matter. I don't think anyone believes that writing a 1MLOC program in SQL is easier than writing a similar one in Java or Python, even if the Python/Java one is 1MLOC, and the SQL one is 200KLOC. When the program is small, there can be large differences, but that's a completely different problem (and also explained by theory). > The methods used to study the subject are completely unfit for purpose. I disagree, but it doesn't matter. Big effects are easy to find and hard to hide. And in an environment with strong selective pressures, they're found even when no studies at all are conducted. > It is inconsistent with the few results that seem at least somewhat well supported because they are simple enough to measure, such as the constant bug count per line of code. It is not. First, the difference in LOC is not as big as PL fans claim (it can be big for small programs, not large ones). Second, you may want to review those results.
    - fauigerzigerk 7y ago
      >I agree, but that's where the many caveats come in. First, my argument is more one of diminishing returns (following Brooks's theory). I.e., languages make an increasingly small difference. C may have been a big improvement over assemby, and Java may have been a smaller but still quite big improvement over C, but now differences are quite small. I don't necessarily disagree, but I'm a bit skeptical about the way in which you use the concept of diminishing returns. If I'm not mistaken then Brooks used diminishing returns in relation to headcount. And this is in fact how economists use the term as well - adding quantitatively more of one production factor. But that doesn't apply to qualitative changes like introducing new production methods and tools, which is what programming languages are. New programming languages are not simply "more programming language". If we agree that there have been observable effects of new programming languages in the past, then we cannot exclude the possibility of the same happening in the future. >First, the difference in LOC is not as big as PL fans claim What PL fans claim is a bit of a fluffy benchmark, but there are undoubtedly significant differences in LOC, certainly significant enough to make an economic difference. >I've been programming for about 30 years now... Shockingly, me too :)

pron 7y ago

> But does that mean there is no causal effect of programming language choice? I don't know and I'm not making any claims either way. I'm also not making definitive claims, but we can't ignore observations, either. While small effects can appear or disappear depending on methodology, big effects are easy to find and hard to hide, especially in an environment with strong selection pressures. Therefore, the most likely explanation to why no big effect has been detected -- either in studies or in industry -- is that there isn't one, and that should at least be enough to make it the working hypothesis. So before explaining effects that have yet to be detected we should establish that they exist at all, something we have so far failed to do. There may well be a small effect, and there could still be a large effect, although the chance of that is small unless we have a really good explanation to why we haven't found it.

fauigerzigerk 7y ago
Here are some reasons to doubt this working hypothesis: 1) It is inconsistent with the personal experience of most developers. Few would claim that C is just as productive and safe as some garbage collected language (provided that both are suitable for the task at hand), or that a SQL group by statement is not more productive to write than the equivalent procedural code. 2) The methods used to study the subject are completely unfit for purpose. 3) It is inconsistent with the few results that seem at least somewhat well supported because they are simple enough to measure, such as the constant bug count per line of code.
- pron 7y ago
  > It is inconsistent with the personal experience of most developers. I disagree. I've been programming for about 30 years now, and most reports I've received -- both from programmers and managers -- are that language doesn't matter; certainly not much. Carefully selected personal "feeling" report are worthless (especially in biased forums), as that would produce even more evidence that homeopathy is effective than that programming languages make a difference. > Few would claim that C is just as productive and safe as some garbage collected language I agree, but that's where the many caveats come in. First, my argument is more one of diminishing returns (following Brooks's theory). I.e., languages make an increasingly small difference. C may have been a big improvement over assemby, and Java may have been a smaller but still quite big improvement over C, but now differences are quite small. Indeed, the effect of GC and memory safety vs C was one that was almost immediately detected by industry (once mature, performant, etc.) and triggered a huge shift. We do not see such shifts now. Companies move among mainstream languages, among non-mainstream languages, and between the two classes in a way that does not suggest a big effect at all. > or that a SQL group by statement is not more productive to write than the equivalent procedural code. That's a completely different matter. I don't think anyone believes that writing a 1MLOC program in SQL is easier than writing a similar one in Java or Python, even if the Python/Java one is 1MLOC, and the SQL one is 200KLOC. When the program is small, there can be large differences, but that's a completely different problem (and also explained by theory). > The methods used to study the subject are completely unfit for purpose. I disagree, but it doesn't matter. Big effects are easy to find and hard to hide. And in an environment with strong selective pressures, they're found even when no studies at all are conducted. > It is inconsistent with the few results that seem at least somewhat well supported because they are simple enough to measure, such as the constant bug count per line of code. It is not. First, the difference in LOC is not as big as PL fans claim (it can be big for small programs, not large ones). Second, you may want to review those results.
  - fauigerzigerk 7y ago
    >I agree, but that's where the many caveats come in. First, my argument is more one of diminishing returns (following Brooks's theory). I.e., languages make an increasingly small difference. C may have been a big improvement over assemby, and Java may have been a smaller but still quite big improvement over C, but now differences are quite small. I don't necessarily disagree, but I'm a bit skeptical about the way in which you use the concept of diminishing returns. If I'm not mistaken then Brooks used diminishing returns in relation to headcount. And this is in fact how economists use the term as well - adding quantitatively more of one production factor. But that doesn't apply to qualitative changes like introducing new production methods and tools, which is what programming languages are. New programming languages are not simply "more programming language". If we agree that there have been observable effects of new programming languages in the past, then we cannot exclude the possibility of the same happening in the future. >First, the difference in LOC is not as big as PL fans claim What PL fans claim is a bit of a fluffy benchmark, but there are undoubtedly significant differences in LOC, certainly significant enough to make an economic difference. >I've been programming for about 30 years now... Shockingly, me too :)