7 ms·

This premise of many bugs being "type errors" is overblown and certainly very recent, for i've never heard such a claim on the internet until only recently (1 o

This premise of many bugs being "type errors" is overblown and certainly very recent, for i've never heard such a claim on the internet until only recently (1 or two years ago). And the world was using dynamic languages for decades on the internet (old ASP, old Actionscript, old JS, PHP4 anyone?) If the 15% of your bugs are simple type errors, then I'd suspect the quality and experience of your development team. The big white elephant in the room that javascript users don't want to acknowledge. Mind you, static type checks are nice stuff, but to claim that 15% of bugs are type bugs, is another thing altogether. I speak from experience, having managed development teams for years. Junior coders need (and deserve) to be trained first doing stuff without hurry to allow for mistake, not used immediately or thrown into critical (or rushed) projects.

mannykannot 9y ago
> I've never heard such a claim on the internet until only recently. Knowledge progresses. You can choose to come along or be left behind. > If the 15% of your bugs are simple type errors, then I'd suspect the quality and experience of your development team. This is an empirical study, carefully constructed to be conservative in its estimation of the achievable benefits of what even these limited type systems could achieve in practice. No amount of sophistry about what can be called a type bug can alter that. If you think the code sample was unrepresentative, perhaps you could arrange for the code of the projects you have managed to be analysed by the same methods.
- flavio81 9y ago
  >Knowledge progresses. You can choose to come along or be left behind. Dynamic weakly typed languages have been in use on the internet for more than 10 years and i've listed them above, so we're not talking about new stuff or new paradigms. If you, with your overtly smug comment above, want to imply that you are not "left behind" in terms of "knowledge" of the state of the art in programming, you shouldn't be clinging to javascript transpilers but using truly modern tools like Haskell, Clojure/Clojurescript, Racket, Julia and Lisp/Parenscript. Because knowledge progresses, and you shouldn't be left behind.
  - mannykannot 9y ago
    Indeed I have moved on - I'm not using Javascript transpilers. Maybe it was an unreasonably smug reply, for which I apologize, and I hope it hasn't distracted from the main point that the value of this empirical study is not diminished by specious arguments about what constitutes a type error.
    - flavio81 9y ago
      >the value of this empirical study The problem is the sample that this study is looking at. As i wrote above, GitHub is simply a big container of code for literally everybody, so judging kind of bugs produced by looking at what it's inside the Github repositories is akin to judging how many gramatical errors book authors make by taking a look at all the MS word documents stored in Dropbox. PS: No worries, and thank you for being friendly. :)
      - mannykannot 9y ago
        That's a fair point, and I would like to see this sort of study extended to professional code, and more complete type systems. Unfortunately, there are only a few languages and type systems in which this sort of comparison can be made. I am, by temperament, inclined to to believe that typing can improve quality, but so far, empirical studies have not made much of a case for that, and I put empirical evidence above even my own opinions.
adamnemecek 9y ago
It follows from Curry-Howard isomorphism.
- deleted 9y ago
  [deleted]
- kybernetikos 9y ago
  Can you say more clearly what it is that you're saying follows from the Curry-Howard Correspondence? My understanding is that the proof that your program is isomorphic to is not necessarily a proof of anything interesting in the domain of your program, which means that the practical consequences of Curry-Howard for day to day programmers is pretty limited. This is very definitely the case for most type systems practically used today (including the one that typescript uses).
  - adamnemecek 9y ago
    It means that you encode some of your logic as types which can be statically checked. Ergo your logic can be statically checked.
zbentley 9y ago
Depends on what you mean by type errors. For example, NPEs in Java are technically a "type error", in that your code is mishandling data due to insufficiently specified types, and I'd have no trouble believing that 15% of Java bugs are NPEs or otherwise null-related. If you extend the definition of type errors to include nulling issues, I'd have no trouble believing that TS reduces the bug rate by that much or more, since TypeScript lets you have non-nullable types. Of course, you could argue that Java devs are just as lacking in quality and experience as JS/TS devs, but that starts to sound suspiciously like No True Scotsman/"real programmers do it in x86 assembly".
tveita 9y ago
> If the 15% of your bugs are simple type errors, then I'd suspect the quality and experience of your development team. IME everyone writes bugs, but simple type bugs rarely make it past testing if you have full test coverage. It would be interesting to know what the typical test coverage in the relevant projects looks like.
rbehrends 9y ago
One problem here is that Javascript eats a lot of type errors silently. $ node > 1+[] '1' > 1-[] 1 > 1*null 0 > 1*undefined NaN > x = {} {} > 1+x.a NaN This is not something that other dynamically typed languages necessarily do.
- flavio81 9y ago
  This is not something that other dynamically typed languages necessarily do Yes, I agree, and this is because Javascript is very weakly typed, and sadly, no tool like Typescript will go so far in counteracting this, because the runtime itself (JS) is still weakly typed. So... no real substitute for good old runtime testing, i'd say.
  - neurotrace 9y ago
    Except that a strong static type system wouldn't allow you to write code like this in the first place. You wouldn't hit these issues at run-time because you caught them at compile-time.
    - mikewhy 9y ago
      Say you have an API endpoint: router.post('/users', (req, res) => { const body = req.body as MyEndpointInterface }) If someone posts to that endpoint with the wrong interface, you're still going to have problems. I'm not sure of any language that would catch such a thing at compile-time and not need some sort of test.
- mazelife 9y ago
  Before I even read the article, I wondered how may of the bugs detected would be due to JavaScript's implicit type coercion rather than specifically dynamic vs. static typing issues. Sadly the article doesn't seems to say, but I thought it telling that the one example they gave: function addNumbers(x, y) { return x + y } console.log(addNumbers(3, "0")) ...of a bug that type annotations can detect has nothing to do with dynamic typing! As you point out there are plenty of dynamically typed languages (e.g. Python) where this would raise an exception. Now you could make an argument that in a static vs. dynamic context yo'd be talking about a compile-time vs. a run-time error, but strictly speaking, the bug here (a function returning a nonsensical value) has nothing to do w/ the kind of type checking performed. Static/dynamic typing and strong/weak typing are orthogonal issues and it's a pet peeve of mind when people sort of muddle them together as they do in this article. FWIW, I think there are decent arguments to be made on both sides of static/dynamic divide, but weakly-typed languages have always just felt like a terrible idea to me that allows careless thinking about data types to creep in all to easily.
marcosdumay 9y ago
Javascript is written around the expectation that developers are flawless, having total knowledge of their codebase and never making any mistake. Looks like there exist people that share that expectation, that's news to me. > And the world was using dynamic languages for decades on the internet (old ASP, old Actionscript, old JS, PHP4 anyone?) Are you claiming that that code has a low bug density? Those languages are famous for low quality code.
- flavio81 9y ago
  Please recall that my claim is for simple type errors. PHP SQL injection attacks, for example, were caused not by type errors but simply by a lack of enough education on how to safely use SQL (i.e. by using prepared statements, etc.) No javascript or PHP system will prevent those. And that's my point, the really serious, time-consuming, havoc-creating bugs are the ones no static type check will save you from.
  - Jyaif 9y ago
    > PHP SQL injection attacks, for example, were caused not by type errors Great example: SQL injection are catchable with types. Instead of using "String", whatever risks getting SQL injection attacks should only accept "EscapedString".
  - marcosdumay 9y ago
    Besides flavio81's comment about SQL injection, Django has a very similar system for avoiding XSS too, where strings are automatically quoted when displaying, and only SafeString objects can create HTML tags.
skybrian 9y ago
The study showed that average programs on Github really do have these errors. Maybe not on your teams, but they are definitely out there. Also, I am not sure how you could know whether these bugs have happened on you teams unless you were specifically classifying bugs by root cause? When the cause of a bug is a simple type error, programmers don't generally tell their manager, they just say it's fixed.
jjuel 9y ago
The true beauty of TS is that you can use it however you want. You want your TS file to be vanilla JS go ahead. You want to use the type system go ahead. There doesn't seem to be much of a downside to me except for the developers that falsely think a type system will slow them down.
Raphmedia 9y ago
I'm going to echo this. Ever since I've been working with JS (8+ years) type errors have only been a small handful of the bugs I've seen. Even 15% seem too high.
lmm 9y ago
> This premise of many bugs being "type errors" is overblown and certainly very recent, for i've never heard such a claim on the internet until only recently (1 or two years ago). Not true, I've been seeing it since at least 2004, and making it from not so long after that. > the world was using dynamic languages for decades on the internet (old ASP, old Actionscript, old JS, PHP4 anyone?) All of which were notoriously buggy. > If the 15% of your bugs are simple type errors, then I'd suspect the quality and experience of your development team. I struggle to think of a bug that isn't a type error. Almost all bugs^ boil down to "the programmer thought this was this, but it was actually that". As you gain experience with using static typing, you see how you can use it to eliminate all your bugs, and the ones that slip through turn out to be cases where you should have made a type distinction but didn't. ^ implementation bugs - I'm not counting cases where the program does what was asked for but that turns out to not be what was wanted
- jbergens 9y ago
  I think that when you have a type system that catches all bugs the type system itself is probably Turing complete. Writing the types in such a system may very well be programming. The more advanced systems are already too hard to use too be helpful for common business people which means they don't help much in reducing bugs from misunderstanding the specs. Just my $0.02.
  - lmm 9y ago
    > I think that when you have a type system that catches all bugs the type system itself is probably Turing complete. Writing the types in such a system may very well be programming. It is programming (though the best way to do it is in a system that's distinctively not Turing-complete - look at what Idris does), but it's all verified at compile time. > The more advanced systems are already too hard to use too be helpful for common business people which means they don't help much in reducing bugs from misunderstanding the specs Not my experience. The type systems make it much easier to express your actual business logic in straight-through code that looks like what it's meant to do, shunting off the awkward cross-cutting concerns to be resolved ambiently in a way that's safe, but doesn't get in the way right now.
camus2 9y ago
> This premise of many bugs being "type errors" is overblown and certainly very recent It really depends on whether your language is strongly typed or weakly typed. If it is strongly typed, then the runtime will catch type errors better, for instance if you try to add a string to a integer, it will yield an error. With Javascript it will yield nothing so bugs with accumulate and sometimes silently until the source of the bug becomes difficult to trace. Same for object properties. Let's say you expect foo.prop and you write foo.props . A strongly typed language would complain, JavaScript won't. Some will argue :"Well duh, that's what testing is for". To an certain extent, and it might not catch type coercion bugs. Furthermore code bases with 100% code coverage are rare these days ... > I speak from experience, having managed development teams for years. Junior coders need (and deserve) to be trained first doing stuff without hurry to allow for mistake, not used immediately or thrown into critical (or rushed) projects. Juniors needs good tools in order to eliminate potential bugs. Static typing is one of that tool.
- flavio81 9y ago
  >Juniors needs good tools in order to eliminate potential bugs. Static typing is one of that tool. Yes, I agree fully with you, it is excellent for training. My only gripe is against that "15%" claim and its implications.
rasz 9y ago
I stumble on randomly getting integer/float/string in surprising places at least once per week in JS. Store integer in localstorage, you will randomly receive float back in chrome.
yawaramin 9y ago
You speak from experience, the paper speaks from 95% confidence level statistical significance from a seemingly well-designed and very conservative study. Who should I trust...?
- flavio81 9y ago
  >from a seemingly well-designed and very conservative study I'd dispute that. As i mentioned above, the sample is "javascript github projects". Github is used, at least in my country, often as scratchpad for bootcamp studies (in Javascript of course). This means in a given month, 200+ repositories created of code made by total beginners. Is this representative of your development team?
  - yawaramin 9y ago
    Yes, but consider this: how many scratchpad GitHub projects have actual bugfix PRs that are linked to filed and resolved bug tickets? The projects in this study were selected exactly on this basis. Further, the authors realise that GitHub projects may be biased: > GitHub may not be representative of projects, since proprietary projects tend not to use it. And, they overcome this bias by sampling, from all GitHub projects: > To construct a list of bugs we could uniformly sample, we took a snapshot of all publicly available JavaScript projects on GitHub, with their closed issue reports. We uniformly selected a closed and linked issue, using the procedure described above and stopped sampling when we reached 400 bugs. When you sample from all available JavaScript GitHub projects, you have a very high probability of overcoming representativeness bias.

mannykannot 9y ago

> I've never heard such a claim on the internet until only recently. Knowledge progresses. You can choose to come along or be left behind. > If the 15% of your bugs are simple type errors, then I'd suspect the quality and experience of your development team. This is an empirical study, carefully constructed to be conservative in its estimation of the achievable benefits of what even these limited type systems could achieve in practice. No amount of sophistry about what can be called a type bug can alter that. If you think the code sample was unrepresentative, perhaps you could arrange for the code of the projects you have managed to be analysed by the same methods.

flavio81 9y ago
>Knowledge progresses. You can choose to come along or be left behind. Dynamic weakly typed languages have been in use on the internet for more than 10 years and i've listed them above, so we're not talking about new stuff or new paradigms. If you, with your overtly smug comment above, want to imply that you are not "left behind" in terms of "knowledge" of the state of the art in programming, you shouldn't be clinging to javascript transpilers but using truly modern tools like Haskell, Clojure/Clojurescript, Racket, Julia and Lisp/Parenscript. Because knowledge progresses, and you shouldn't be left behind.
- mannykannot 9y ago
  Indeed I have moved on - I'm not using Javascript transpilers. Maybe it was an unreasonably smug reply, for which I apologize, and I hope it hasn't distracted from the main point that the value of this empirical study is not diminished by specious arguments about what constitutes a type error.
  - flavio81 9y ago
    >the value of this empirical study The problem is the sample that this study is looking at. As i wrote above, GitHub is simply a big container of code for literally everybody, so judging kind of bugs produced by looking at what it's inside the Github repositories is akin to judging how many gramatical errors book authors make by taking a look at all the MS word documents stored in Dropbox. PS: No worries, and thank you for being friendly. :)
    - mannykannot 9y ago
      That's a fair point, and I would like to see this sort of study extended to professional code, and more complete type systems. Unfortunately, there are only a few languages and type systems in which this sort of comparison can be made. I am, by temperament, inclined to to believe that typing can improve quality, but so far, empirical studies have not made much of a case for that, and I put empirical evidence above even my own opinions.

adamnemecek 9y ago

It follows from Curry-Howard isomorphism.

deleted 9y ago
[deleted]
kybernetikos 9y ago
Can you say more clearly what it is that you're saying follows from the Curry-Howard Correspondence? My understanding is that the proof that your program is isomorphic to is not necessarily a proof of anything interesting in the domain of your program, which means that the practical consequences of Curry-Howard for day to day programmers is pretty limited. This is very definitely the case for most type systems practically used today (including the one that typescript uses).
- adamnemecek 9y ago
  It means that you encode some of your logic as types which can be statically checked. Ergo your logic can be statically checked.

zbentley 9y ago

Depends on what you mean by type errors. For example, NPEs in Java are technically a "type error", in that your code is mishandling data due to insufficiently specified types, and I'd have no trouble believing that 15% of Java bugs are NPEs or otherwise null-related. If you extend the definition of type errors to include nulling issues, I'd have no trouble believing that TS reduces the bug rate by that much or more, since TypeScript lets you have non-nullable types. Of course, you could argue that Java devs are just as lacking in quality and experience as JS/TS devs, but that starts to sound suspiciously like No True Scotsman/"real programmers do it in x86 assembly".

tveita 9y ago

> If the 15% of your bugs are simple type errors, then I'd suspect the quality and experience of your development team. IME everyone writes bugs, but simple type bugs rarely make it past testing if you have full test coverage. It would be interesting to know what the typical test coverage in the relevant projects looks like.

rbehrends 9y ago

One problem here is that Javascript eats a lot of type errors silently. $ node > 1+[] '1' > 1-[] 1 > 1*null 0 > 1*undefined NaN > x = {} {} > 1+x.a NaN This is not something that other dynamically typed languages necessarily do.

flavio81 9y ago
This is not something that other dynamically typed languages necessarily do Yes, I agree, and this is because Javascript is very weakly typed, and sadly, no tool like Typescript will go so far in counteracting this, because the runtime itself (JS) is still weakly typed. So... no real substitute for good old runtime testing, i'd say.
- neurotrace 9y ago
  Except that a strong static type system wouldn't allow you to write code like this in the first place. You wouldn't hit these issues at run-time because you caught them at compile-time.
  - mikewhy 9y ago
    Say you have an API endpoint: router.post('/users', (req, res) => { const body = req.body as MyEndpointInterface }) If someone posts to that endpoint with the wrong interface, you're still going to have problems. I'm not sure of any language that would catch such a thing at compile-time and not need some sort of test.
mazelife 9y ago
Before I even read the article, I wondered how may of the bugs detected would be due to JavaScript's implicit type coercion rather than specifically dynamic vs. static typing issues. Sadly the article doesn't seems to say, but I thought it telling that the one example they gave: function addNumbers(x, y) { return x + y } console.log(addNumbers(3, "0")) ...of a bug that type annotations can detect has nothing to do with dynamic typing! As you point out there are plenty of dynamically typed languages (e.g. Python) where this would raise an exception. Now you could make an argument that in a static vs. dynamic context yo'd be talking about a compile-time vs. a run-time error, but strictly speaking, the bug here (a function returning a nonsensical value) has nothing to do w/ the kind of type checking performed. Static/dynamic typing and strong/weak typing are orthogonal issues and it's a pet peeve of mind when people sort of muddle them together as they do in this article. FWIW, I think there are decent arguments to be made on both sides of static/dynamic divide, but weakly-typed languages have always just felt like a terrible idea to me that allows careless thinking about data types to creep in all to easily.

marcosdumay 9y ago

Javascript is written around the expectation that developers are flawless, having total knowledge of their codebase and never making any mistake. Looks like there exist people that share that expectation, that's news to me. > And the world was using dynamic languages for decades on the internet (old ASP, old Actionscript, old JS, PHP4 anyone?) Are you claiming that that code has a low bug density? Those languages are famous for low quality code.

flavio81 9y ago
Please recall that my claim is for simple type errors. PHP SQL injection attacks, for example, were caused not by type errors but simply by a lack of enough education on how to safely use SQL (i.e. by using prepared statements, etc.) No javascript or PHP system will prevent those. And that's my point, the really serious, time-consuming, havoc-creating bugs are the ones no static type check will save you from.
- Jyaif 9y ago
  > PHP SQL injection attacks, for example, were caused not by type errors Great example: SQL injection are catchable with types. Instead of using "String", whatever risks getting SQL injection attacks should only accept "EscapedString".
- marcosdumay 9y ago
  Besides flavio81's comment about SQL injection, Django has a very similar system for avoiding XSS too, where strings are automatically quoted when displaying, and only SafeString objects can create HTML tags.

skybrian 9y ago

The study showed that average programs on Github really do have these errors. Maybe not on your teams, but they are definitely out there. Also, I am not sure how you could know whether these bugs have happened on you teams unless you were specifically classifying bugs by root cause? When the cause of a bug is a simple type error, programmers don't generally tell their manager, they just say it's fixed.

jjuel 9y ago

The true beauty of TS is that you can use it however you want. You want your TS file to be vanilla JS go ahead. You want to use the type system go ahead. There doesn't seem to be much of a downside to me except for the developers that falsely think a type system will slow them down.

Raphmedia 9y ago

I'm going to echo this. Ever since I've been working with JS (8+ years) type errors have only been a small handful of the bugs I've seen. Even 15% seem too high.

lmm 9y ago

> This premise of many bugs being "type errors" is overblown and certainly very recent, for i've never heard such a claim on the internet until only recently (1 or two years ago). Not true, I've been seeing it since at least 2004, and making it from not so long after that. > the world was using dynamic languages for decades on the internet (old ASP, old Actionscript, old JS, PHP4 anyone?) All of which were notoriously buggy. > If the 15% of your bugs are simple type errors, then I'd suspect the quality and experience of your development team. I struggle to think of a bug that isn't a type error. Almost all bugs^ boil down to "the programmer thought this was this, but it was actually that". As you gain experience with using static typing, you see how you can use it to eliminate all your bugs, and the ones that slip through turn out to be cases where you should have made a type distinction but didn't. ^ implementation bugs - I'm not counting cases where the program does what was asked for but that turns out to not be what was wanted

jbergens 9y ago
I think that when you have a type system that catches all bugs the type system itself is probably Turing complete. Writing the types in such a system may very well be programming. The more advanced systems are already too hard to use too be helpful for common business people which means they don't help much in reducing bugs from misunderstanding the specs. Just my $0.02.
- lmm 9y ago
  > I think that when you have a type system that catches all bugs the type system itself is probably Turing complete. Writing the types in such a system may very well be programming. It is programming (though the best way to do it is in a system that's distinctively not Turing-complete - look at what Idris does), but it's all verified at compile time. > The more advanced systems are already too hard to use too be helpful for common business people which means they don't help much in reducing bugs from misunderstanding the specs Not my experience. The type systems make it much easier to express your actual business logic in straight-through code that looks like what it's meant to do, shunting off the awkward cross-cutting concerns to be resolved ambiently in a way that's safe, but doesn't get in the way right now.

camus2 9y ago

> This premise of many bugs being "type errors" is overblown and certainly very recent It really depends on whether your language is strongly typed or weakly typed. If it is strongly typed, then the runtime will catch type errors better, for instance if you try to add a string to a integer, it will yield an error. With Javascript it will yield nothing so bugs with accumulate and sometimes silently until the source of the bug becomes difficult to trace. Same for object properties. Let's say you expect foo.prop and you write foo.props . A strongly typed language would complain, JavaScript won't. Some will argue :"Well duh, that's what testing is for". To an certain extent, and it might not catch type coercion bugs. Furthermore code bases with 100% code coverage are rare these days ... > I speak from experience, having managed development teams for years. Junior coders need (and deserve) to be trained first doing stuff without hurry to allow for mistake, not used immediately or thrown into critical (or rushed) projects. Juniors needs good tools in order to eliminate potential bugs. Static typing is one of that tool.

flavio81 9y ago
>Juniors needs good tools in order to eliminate potential bugs. Static typing is one of that tool. Yes, I agree fully with you, it is excellent for training. My only gripe is against that "15%" claim and its implications.

rasz 9y ago

I stumble on randomly getting integer/float/string in surprising places at least once per week in JS. Store integer in localstorage, you will randomly receive float back in chrome.

yawaramin 9y ago

You speak from experience, the paper speaks from 95% confidence level statistical significance from a seemingly well-designed and very conservative study. Who should I trust...?

flavio81 9y ago
>from a seemingly well-designed and very conservative study I'd dispute that. As i mentioned above, the sample is "javascript github projects". Github is used, at least in my country, often as scratchpad for bootcamp studies (in Javascript of course). This means in a given month, 200+ repositories created of code made by total beginners. Is this representative of your development team?
- yawaramin 9y ago
  Yes, but consider this: how many scratchpad GitHub projects have actual bugfix PRs that are linked to filed and resolved bug tickets? The projects in this study were selected exactly on this basis. Further, the authors realise that GitHub projects may be biased: > GitHub may not be representative of projects, since proprietary projects tend not to use it. And, they overcome this bias by sampling, from all GitHub projects: > To construct a list of bugs we could uniformly sample, we took a snapshot of all publicly available JavaScript projects on GitHub, with their closed issue reports. We uniformly selected a closed and linked issue, using the procedure described above and stopped sampling when we reached 400 bugs. When you sample from all available JavaScript GitHub projects, you have a very high probability of overcoming representativeness bias.