6 ms·

Seems like they're just doing associations, and not controlling for a lot of factors. For example, you can come up with some narratives to explain a lot of thes

Seems like they're just doing associations, and not controlling for a lot of factors. For example, you can come up with some narratives to explain a lot of these findings: * Python, JS, PHP, Java, C++ are all very common first languages; I'd imagine the average experience level for these is dragged down a bit compared to others like Golang, Haskell, or Scala. One of the recent Stack Overflow Developer surveys found Golang use to be strongly correlated with age, and since I'm assuming bug count would negatively correlate with age you'd see a confounding factor here. * Ruby's community has an obsessive culture towards unit testing, that could impact bug levels. * The trade-offs that make C and C++ attractive such as memory performance are double-edged swords and will naturally correlate with more bugs. For example, I work a fair bit in embedded systems which use MISRA C out of necessity; I've also worked in large scale data processing which chooses C++ for its memory mapping performance over Java or Go. If "number of defects" is your primary measure then you're ignoring some of the reasons languages are chosen. Sometimes a language that minimizes bugs is not a viable option. * I'd guess there is a skill correlation here: an engineer who is willing to seek out a specific less-used language and learn to use it effectively is probably more likely (not guaranteed) to do a better job than one who is just going with the default choice. It'd be very interesting to see a regression done that accounts for these factors.

cestith 7y ago
That would be interesting. I also have a feeling it would be incredibly difficult to accurately control for social differences in language communities.
- robbrit 7y ago
  Yeah you'd have to take them one-by-one and see if there is some proxy if you can't measure the behaviour directly. For example, with testing you could use code-coverage or even a binary variable for the existence of tests - not perfect, but would give better results than no control at all.
- shados 7y ago
  Definitely. Heck, in .NET's early days, there was a distinct difference in quality of the software and of the team using VB.net vs C#, when they were 99% the same language (they had some features unique to each, but they were very few). But C# was used a lot in new projects or by people familiar with Java while VB.net was used in VB6 migrations or teams coming from VB6. That had a drastic effect on the end result, but the language had very little to do with it.
  - moogly 7y ago
    The VB shops I've seen code from tended to really like `On Error Resume Next`, too... Sure, I've seen global try/catch blocks swallowing all errors in C#/C++ code too, but definitely not as prevalent in my anecdata.
    - cestith 7y ago
      Some developers seem to be more along the lines of "On Error Resumé Next".
JamesBarney 7y ago
JS also has more ways to shoot yourself in the foot than any other language I've worked with.
- stcredzero 7y ago
  PHP? C?
  - robbrit 7y ago
    Always remember to turn on warnings in PHP. I remember the joys of tracking down a mistyped variable name that PHP just instantiated and gave a null value to, which was then implicitly converted to empty string when the variable was used.
  - JamesBarney 7y ago
    Last time I worked with those languages was in college 10 years ago so I didn't know portion of shooting myself in the foot was my fault versus the language.
  - besulzbach 7y ago
    I agree that JS is not substantially "worse" than some other languages. Despite these three languages giving far too much freedom to the programmers (which some might even like), high-quality code can be written in all three of them, as long as intelligent and strict coding guidelines are thoroughly followed.
    - JamesBarney 7y ago
      I always think of languages as a solution space, and some points in the space possessing bugs while others don't. A perfect language(impossible) would make it so you could write any correct version of a program in the language but not any incorrect version. And language constraints like static typing have the disadvantage that you're preventing a bunch of correct possible solutions while closing off incorrect solutions. Ideally closing off many more incorrect solutions than correct ones increasing your chance of landing in a correction solution. And of course lots of out of band(non-language) processes and tools that influence this like unit testing/code reviews/strict coding guidelines. But it's always nicer when it's prevented at the language level as opposed to out of band.
stcredzero 7y ago
Ruby's community has an obsessive culture towards unit testing, that could impact bug levels. Like Smalltalk, they're forced to do this to catch the errors that would've been caught by type annotation. So then, could the argument be made that less stringent type annotation hurts code quality by removing effort from unit testing?
- robbrit 7y ago
  > they're forced to do this Figuratively speaking. None of the languages they test enforce any unit testing. What I'm suggesting is that since the Ruby community tends to be more obsessive about testing, that might lead to an improvement in code quality when comparing against other similar languages like PHP or JS which are less obsessive about testing. It's something you might have to control for to see the impact. Type systems are a completely different factor; you'd have to control for that variable too to determine the impact of it. My experience with both systems makes me predict that static typing would improve code quality, but if you want to actually be scientific you'd want to add that factor to the statistical analysis to see if the data support that hypothesis. Anecdotes like personal experience don't count.
  - stcredzero 7y ago
    Figuratively speaking. None of the languages they test enforce any unit testing. The trivial type-mistake "Unhandled Exceptions" in Smalltalk would leak out more often than was comfortable, if the project didn't have a good means of testing to catch those. I suspect that has something to do with the Ruby community being more obsessive about testing.
    - igouy 7y ago
      > "Unhandled Exceptions" in Smalltalk ? MessageNotUnderstood
      - stcredzero 7y ago
        Right. This, and many others, would result in an "Unhandled Exception" notifier dialog in many versions of Smalltalk. I used to work for the company acting as the vendor of 2 (3?) commercial Smalltalks, also doing consulting work for others.
scott_s 7y ago
I think you're making the same confusion a lot of other commenters have made, specifically responding to the original study as opposed to this failed replication of that study: https://news.ycombinator.com/item?id=20105150 https://news.ycombinator.com/item?id=20105150
mcguire 7y ago
Additionally, take a look at Fig. 8, "linus C". Those points scream "your data is bad".
pron 7y ago
It doesn't matter. They are looking at effects relative to the total effect of language choice, which they report as less than 1%. Instead of looking at these secondary effects, the main practical result is that language choice hardly matters.

cestith 7y ago

That would be interesting. I also have a feeling it would be incredibly difficult to accurately control for social differences in language communities.

robbrit 7y ago
Yeah you'd have to take them one-by-one and see if there is some proxy if you can't measure the behaviour directly. For example, with testing you could use code-coverage or even a binary variable for the existence of tests - not perfect, but would give better results than no control at all.
shados 7y ago
Definitely. Heck, in .NET's early days, there was a distinct difference in quality of the software and of the team using VB.net vs C#, when they were 99% the same language (they had some features unique to each, but they were very few). But C# was used a lot in new projects or by people familiar with Java while VB.net was used in VB6 migrations or teams coming from VB6. That had a drastic effect on the end result, but the language had very little to do with it.
- moogly 7y ago
  The VB shops I've seen code from tended to really like `On Error Resume Next`, too... Sure, I've seen global try/catch blocks swallowing all errors in C#/C++ code too, but definitely not as prevalent in my anecdata.
  - cestith 7y ago
    Some developers seem to be more along the lines of "On Error Resumé Next".

JamesBarney 7y ago

JS also has more ways to shoot yourself in the foot than any other language I've worked with.

stcredzero 7y ago
PHP? C?
- robbrit 7y ago
  Always remember to turn on warnings in PHP. I remember the joys of tracking down a mistyped variable name that PHP just instantiated and gave a null value to, which was then implicitly converted to empty string when the variable was used.
- JamesBarney 7y ago
  Last time I worked with those languages was in college 10 years ago so I didn't know portion of shooting myself in the foot was my fault versus the language.
- besulzbach 7y ago
  I agree that JS is not substantially "worse" than some other languages. Despite these three languages giving far too much freedom to the programmers (which some might even like), high-quality code can be written in all three of them, as long as intelligent and strict coding guidelines are thoroughly followed.
  - JamesBarney 7y ago
    I always think of languages as a solution space, and some points in the space possessing bugs while others don't. A perfect language(impossible) would make it so you could write any correct version of a program in the language but not any incorrect version. And language constraints like static typing have the disadvantage that you're preventing a bunch of correct possible solutions while closing off incorrect solutions. Ideally closing off many more incorrect solutions than correct ones increasing your chance of landing in a correction solution. And of course lots of out of band(non-language) processes and tools that influence this like unit testing/code reviews/strict coding guidelines. But it's always nicer when it's prevented at the language level as opposed to out of band.

stcredzero 7y ago

Ruby's community has an obsessive culture towards unit testing, that could impact bug levels. Like Smalltalk, they're forced to do this to catch the errors that would've been caught by type annotation. So then, could the argument be made that less stringent type annotation hurts code quality by removing effort from unit testing?

robbrit 7y ago
> they're forced to do this Figuratively speaking. None of the languages they test enforce any unit testing. What I'm suggesting is that since the Ruby community tends to be more obsessive about testing, that might lead to an improvement in code quality when comparing against other similar languages like PHP or JS which are less obsessive about testing. It's something you might have to control for to see the impact. Type systems are a completely different factor; you'd have to control for that variable too to determine the impact of it. My experience with both systems makes me predict that static typing would improve code quality, but if you want to actually be scientific you'd want to add that factor to the statistical analysis to see if the data support that hypothesis. Anecdotes like personal experience don't count.
- stcredzero 7y ago
  Figuratively speaking. None of the languages they test enforce any unit testing. The trivial type-mistake "Unhandled Exceptions" in Smalltalk would leak out more often than was comfortable, if the project didn't have a good means of testing to catch those. I suspect that has something to do with the Ruby community being more obsessive about testing.
  - igouy 7y ago
    > "Unhandled Exceptions" in Smalltalk ? MessageNotUnderstood
    - stcredzero 7y ago
      Right. This, and many others, would result in an "Unhandled Exception" notifier dialog in many versions of Smalltalk. I used to work for the company acting as the vendor of 2 (3?) commercial Smalltalks, also doing consulting work for others.

scott_s 7y ago

I think you're making the same confusion a lot of other commenters have made, specifically responding to the original study as opposed to this failed replication of that study: https://news.ycombinator.com/item?id=20105150 https://news.ycombinator.com/item?id=20105150

mcguire 7y ago

Additionally, take a look at Fig. 8, "linus C". Those points scream "your data is bad".

pron 7y ago

It doesn't matter. They are looking at effects relative to the total effect of language choice, which they report as less than 1%. Instead of looking at these secondary effects, the main practical result is that language choice hardly matters.