4 ms·

My own take of this: - This study tries to enhance the original work by better metrics (e.g. confusion with C/C++), using better statistics (e.g. uncertainty).

My own take of this: - This study tries to enhance the original work by better metrics (e.g. confusion with C/C++), using better statistics (e.g. uncertainty). - Automatically evaluating the quality of a code is hard and time-consuming. They had to use peer review of labellisation. - Most of the old claims are not confirmed. - Only 2 languages in this study had both a significant impact (impact coeff) and enough fiability (p-value): Clojure and Haskell. But the impact is still small, and there may be other influencing factors. - They propose several ways to further enhance their work (e.g. taking regression tests into account).

msluyter 7y ago
"... and there may be other influencing factors." Selection bias? Consider the type of person likely to be attracted to languages like Haskell or Clojure. (Note, didn't read the original study so I don't know the selection criteria. Were code samples in different languages from the same programmers compared, or were there different sets of programmers per language? My comment really only applies to the latter case.)
- sidlls 7y ago
  What is the type of person likely to be attracted to languages like Haskell or Clojure?
  - maxkwallace 7y ago
    I can think of selection bias effects both in-favor of more defects and against it. On one hand, Haskell or Clojure probably attract more academic types who have a reputation for (consistent with my experience) on-average worse-quality code. On other hand, the barrier-to-entry for these languages is higher so they tend to be programmed by more experienced folks. I assume that people pick functional languages in part because they believe the functional paradigm leads to higher-quality code, so the sample probably cares more about code quality to begin with.
    - danielscrubs 7y ago
      You can’t win against people’s insecurities. They went to school, they where not bashed in the head with hammers.
twanvl 7y ago
> - Only 2 languages in this study had both a significant impact (impact coeff) and enough fiability (p-value): Clojure and Haskell. But the impact is still small, and there may be other influencing factors. They also found a significant positive association between C++ and bug.
JamesBarney 7y ago
And should be weighed against of the difficulty of trying to staff a team full of Haskell developers :).
sidlls 7y ago
They found a statistically significant correlation: not a causal relationship and the correlation was small. Using "significant" without those qualifications presents a misleading picture of the results.

msluyter 7y ago

"... and there may be other influencing factors." Selection bias? Consider the type of person likely to be attracted to languages like Haskell or Clojure. (Note, didn't read the original study so I don't know the selection criteria. Were code samples in different languages from the same programmers compared, or were there different sets of programmers per language? My comment really only applies to the latter case.)

sidlls 7y ago
What is the type of person likely to be attracted to languages like Haskell or Clojure?
- maxkwallace 7y ago
  I can think of selection bias effects both in-favor of more defects and against it. On one hand, Haskell or Clojure probably attract more academic types who have a reputation for (consistent with my experience) on-average worse-quality code. On other hand, the barrier-to-entry for these languages is higher so they tend to be programmed by more experienced folks. I assume that people pick functional languages in part because they believe the functional paradigm leads to higher-quality code, so the sample probably cares more about code quality to begin with.
  - danielscrubs 7y ago
    You can’t win against people’s insecurities. They went to school, they where not bashed in the head with hammers.

twanvl 7y ago

> - Only 2 languages in this study had both a significant impact (impact coeff) and enough fiability (p-value): Clojure and Haskell. But the impact is still small, and there may be other influencing factors. They also found a significant positive association between C++ and bug.

JamesBarney 7y ago

And should be weighed against of the difficulty of trying to staff a team full of Haskell developers :).

sidlls 7y ago

They found a statistically significant correlation: not a causal relationship and the correlation was small. Using "significant" without those qualifications presents a misleading picture of the results.