3 ms·

Python and Ruby come out with quite opposite results. There is absolutely no reason why those languages, on their own, should lead to wildly different error rat

Python and Ruby come out with quite opposite results. There is absolutely no reason why those languages, on their own, should lead to wildly different error rates. Both dynamically typed, general purpose, very equivalent languages. So if this study seems intuitively like a fools-errand, that would be your smoking gun. The study admits the effects are small. The one clear conclusion is an obvious one: "Result 4: Defect types are strongly associated with languages; Some defect type like memory error, concurrency errors also depend on language primitives. Language matters more for specific categories than it does for defects overall." Does Clojure have an inherently lower defect rate than C++? It's arguably an interesting question to ask, but looking at the Python and Ruby results makes it clear, to my anyway, that we'll have to stick to arguing about it over beers, rather than finding the answer in data.

tinco 11y ago
The Ruby ecosystem has very strong emphasis on tests and test driven development. Perhaps bugs in Ruby simply often are fixed before they're committed because the test suites have better coverage. Also I think Ruby developers are more likely to be professional (as in paid and educated to be programmers), whereas python developers might (slightly more) often be mathematicians, scientists or sysadmins. As is noted in the paper, all other factors that were controlled for were strongly dominant. That means who you are and what you are doing is much more important than what language you do it in.
voidlogic 11y ago
>Python and Ruby come out with quite opposite results. There is absolutely no reason why those languages, on their own, should lead to wildly different error rates. Both dynamically typed, general purpose, very equivalent languages. So if this study seems intuitively like a fools-errand, that would be your smoking gun. I'm not saying this is necessarily the case, but your analysis doesn't leave room for the possibility that the demographics of organizations or users of different languages may vary greatly.

tinco 11y ago

The Ruby ecosystem has very strong emphasis on tests and test driven development. Perhaps bugs in Ruby simply often are fixed before they're committed because the test suites have better coverage. Also I think Ruby developers are more likely to be professional (as in paid and educated to be programmers), whereas python developers might (slightly more) often be mathematicians, scientists or sysadmins. As is noted in the paper, all other factors that were controlled for were strongly dominant. That means who you are and what you are doing is much more important than what language you do it in.

voidlogic 11y ago

>Python and Ruby come out with quite opposite results. There is absolutely no reason why those languages, on their own, should lead to wildly different error rates. Both dynamically typed, general purpose, very equivalent languages. So if this study seems intuitively like a fools-errand, that would be your smoking gun. I'm not saying this is necessarily the case, but your analysis doesn't leave room for the possibility that the demographics of organizations or users of different languages may vary greatly.