4 ms·

I've got a great bug prediction tool: ghc, the haskell compiler. Whenever I use a value of the wrong type my program won't compile. In my code it is always a bu

by T_S_ 15y ago

I've got a great bug prediction tool: ghc, the haskell compiler. Whenever I use a value of the wrong type my program won't compile. In my code it is always a bug when that happens. (Some smarter people can write non-buggy code the compiler doesn't like.) Sometimes I can trick ghc into using the wrong type. Like when I use "String" to mean "File". That's when all the bugs show up as run-time errors. But I should know a string is not a file. And hopefully somebody writes a module that knows what a "File" really is and stops me from writing buggy code if I use it.

nl 15y ago
Type errors are only one type of bug.
- _delirium 15y ago
  True, but the whole point of designing modern type systems is to attempt to catch as many common bugs in the static-check phase as possible (while still being decidable), rather than purely checking datatypes (int, string, etc.) in the classical sense. I don't think it's too unreasonable to compare it to empirical bug prediction: they're a rationalist vs. an empiricist approach to statistically predicting where bugs probably lie, with some various tradeoffs.
  - nl 15y ago
    I'm not sure what your point is. Yes, type systems can catch a class of bugs. Google - like most of the software development world - has a lot of people who believe in type systems completely. That's why they have tools like the Closure JS compiler[1], which provides type-checking for Javascript, and GWT[2], which produces (un-typesafe) Javascript/CSS/HTML from (mostly)typesafe Java. Using methods of reducing bugs (such as type safety) is orthogonal to producing systems that predict where bugs will occur. [1] http://code.google.com/closure/compiler/ http://code.google.com/closure/compiler/ [2] http://code.google.com/webtoolkit/ http://code.google.com/webtoolkit/
    - _delirium 15y ago
      Your last statement's the part I disagree with, though it depends on what you mean by "orthogonal". If you mean that they're two different approaches that can coexist, then yes. But I don't think they target orthogonal classes of bugs. In both cases, the goal is to employ some algorithmic, decidable method at compile-time to predict whether a given piece of code is "correct" or "incorrect", trading off the possibility of a false positive or false negative. Each approach does better or worse in different cases, but I think in a manner that either isn't orthogonal, or at least isn't obviously orthogonal (if "orthogonal" is meant in any strong sense, rather than just "two different approaches"). And I think for any one, we can at least in principle ask whether the other one could've done it: e.g., if we're noticing a lot of bugs of a certain sort, could the type-checker have caught that? The ambition, at least, of static type systems is to render accurate bug-prediction impossible, because every bug statically predictable at compile-time will be a type error.
      - nl 15y ago
        >The ambition, at least, of static type systems is to render accurate bug-prediction impossible, because every bug statically predictable at compile-time will be a type error. I might not use Haskell, but I do understand what a type system does, and why it is important. BUT, not all bugs are statically predictable at compile-time. Take things like cross-browser compatibility - something like GWT goes a long way to reducing bugs with that, but no type system will protect you from a new bug in a new browser you need to work around. (Edit: by orthogonal I meant "statistically independent". Given a piece of code written in a type safe language, this method will predict bugs independently of a type system.)
        chii 15y ago
        so your thesis is that a bug prediction system ought to be able to predict bugs that are not possible to statically predict? I suppose that is a reasonable argument. i think different people in this argument are arguing about different things.
      - jdp23 15y ago
- Rickasaurus 15y ago
  Type Systems are good at catching more than just type errors. For example, booleans are much better represented in the type system.
srean 15y ago
I am not surprised that you are being downvoted. In my experience, one may question the superiority of dynamic typing at HN only to the detriment of ones karma. Well, its not HN alone, dynamic languages are certainly popular and quite persuasively championed. Now I belong to the "had been persuaded before, but now I am not so sure" category. More so after discovering absolutely stupid typos and compile time checkable errors in long running Python code that I had written. It really sucks when your code bails out after 16 hrs of number crunching because of some silly error on your part. I used to think, "who makes type errors of all things?" it turns out that that idiot is me. I am not claiming that dynamic languages are bad, or that checked languages are always good, but I do benefit from checked code. A better programmer perhaps wouldn't benefit as much. Rather than implementing a type checking system badly and in an ad-hoc manner, I now lean more towards compile time checked languages, especially for pieces of code that I expect will run for a long time. I think the sweet spot would be optionally type-checked / type-inferred languages, for example Cython, Common Lisp. Strongly type-checked languages _tend_ to be all or nothing. EDIT: @nl yeah! absolutely, they arent by any means a bug predicting algo.
- nl 15y ago
  Type systems are great. Other things that are great are unit tests, automatic code inspection tools and code reviews. They aren't the same as bug prediction systems, though.
- tedunangst 15y ago
  I think, more accurately, one may rave about the irrelevant merits of static typing to the detriment of one's karma. Here's a bug from the real world: the login form doesn't render correctly in IE. When someone implements IE's layout algorithm as a haskell type, I'll start paying attention.
  - srean 15y ago
    That there exist errors that cannot be type-checked makes type-checking irrelevant ? I dont quite see why exhaustiveness is a pre-requisite for relevance or attention. Is that a thumb rule you use for other things as well ? I am not sure if type-checking even aspires to be exhaustive. As long as it can detect my stupid and costly mistakes at a cost that is cheaper than the cost of the mistakes, I am happy. But I can understand that for people who do not make costly and compile time checkable mistakes, it might not be worth it. @tedunangst Got you. Your second paragraph in the previous comment threw me off. It does read like you consider type-checking unworthy of attention because it cannot catch all possible errors. Agree with your comment about people overstating their case. I think the noisiest of both the camps (if one can call it that) are equally guilty.
jdp23 15y ago
Defect detection != defect prediction
Lewisham 15y ago
Hi, I'm Chris, author of the blog/work. While I love me some type systems, I worry you're thinking a bit too code-centric. For example, let's say I describe to you a really hard problem, and you come up with a solution (that type checks!), and then we launch to users and they find a use case we never thought of, and our code suddenly breaks. I am sure this has happened to you. The problem isn't at the code level (that's just the manifestation), but our current understanding of the problem. IMHO, once you reach a certain level of competency, a sizable minority, if not majority, of bugs that make their way into a source control system (particularly one with code review, like Google's) are not line-by-line problems, but wider problems with actually understanding the task at hand and all that encompasses. When you work on the complexity of problems that Google does, you're tending towards a very large chunk of bugs being members of this class. However, we don't have good tools (yet) to have a computer properly check the semantic meaning of our code. Bug prediction sits as a sort of baby step. It's the computer making a best-effort guess of where issues will be. The algorithm designed here is saying "Look this file has been a struggle. It's probably going to continue being a struggle, so we need to focus attention when we change anything to it."
- deleted 15y ago
  [deleted]
- T_S_ 15y ago
  Apologies for the perception of trollishness in my comment. The intent was merely to reflect my own personal quest to write better code. Obviously data mining has its place in a shop that produces mountains of code like Google. Not to mention other techniques like unit testing, etc.
JabavuAdams 15y ago
These are trivial kinds of errors. The big fuckups are invariably more subtle and people / requirements oriented. When all you've got is a hammer ....

by T_S_ 15y ago

nl 15y ago

Type errors are only one type of bug.

_delirium 15y ago
True, but the whole point of designing modern type systems is to attempt to catch as many common bugs in the static-check phase as possible (while still being decidable), rather than purely checking datatypes (int, string, etc.) in the classical sense. I don't think it's too unreasonable to compare it to empirical bug prediction: they're a rationalist vs. an empiricist approach to statistically predicting where bugs probably lie, with some various tradeoffs.
- nl 15y ago
  I'm not sure what your point is. Yes, type systems can catch a class of bugs. Google - like most of the software development world - has a lot of people who believe in type systems completely. That's why they have tools like the Closure JS compiler[1], which provides type-checking for Javascript, and GWT[2], which produces (un-typesafe) Javascript/CSS/HTML from (mostly)typesafe Java. Using methods of reducing bugs (such as type safety) is orthogonal to producing systems that predict where bugs will occur. [1] http://code.google.com/closure/compiler/ http://code.google.com/closure/compiler/ [2] http://code.google.com/webtoolkit/ http://code.google.com/webtoolkit/
  - _delirium 15y ago
    Your last statement's the part I disagree with, though it depends on what you mean by "orthogonal". If you mean that they're two different approaches that can coexist, then yes. But I don't think they target orthogonal classes of bugs. In both cases, the goal is to employ some algorithmic, decidable method at compile-time to predict whether a given piece of code is "correct" or "incorrect", trading off the possibility of a false positive or false negative. Each approach does better or worse in different cases, but I think in a manner that either isn't orthogonal, or at least isn't obviously orthogonal (if "orthogonal" is meant in any strong sense, rather than just "two different approaches"). And I think for any one, we can at least in principle ask whether the other one could've done it: e.g., if we're noticing a lot of bugs of a certain sort, could the type-checker have caught that? The ambition, at least, of static type systems is to render accurate bug-prediction impossible, because every bug statically predictable at compile-time will be a type error.
    - nl 15y ago
      >The ambition, at least, of static type systems is to render accurate bug-prediction impossible, because every bug statically predictable at compile-time will be a type error. I might not use Haskell, but I do understand what a type system does, and why it is important. BUT, not all bugs are statically predictable at compile-time. Take things like cross-browser compatibility - something like GWT goes a long way to reducing bugs with that, but no type system will protect you from a new bug in a new browser you need to work around. (Edit: by orthogonal I meant "statistically independent". Given a piece of code written in a type safe language, this method will predict bugs independently of a type system.)
      - chii 15y ago
        so your thesis is that a bug prediction system ought to be able to predict bugs that are not possible to statically predict? I suppose that is a reasonable argument. i think different people in this argument are arguing about different things.
    - jdp23 15y ago
Rickasaurus 15y ago
Type Systems are good at catching more than just type errors. For example, booleans are much better represented in the type system.

srean 15y ago

I am not surprised that you are being downvoted. In my experience, one may question the superiority of dynamic typing at HN only to the detriment of ones karma. Well, its not HN alone, dynamic languages are certainly popular and quite persuasively championed. Now I belong to the "had been persuaded before, but now I am not so sure" category. More so after discovering absolutely stupid typos and compile time checkable errors in long running Python code that I had written. It really sucks when your code bails out after 16 hrs of number crunching because of some silly error on your part. I used to think, "who makes type errors of all things?" it turns out that that idiot is me. I am not claiming that dynamic languages are bad, or that checked languages are always good, but I do benefit from checked code. A better programmer perhaps wouldn't benefit as much. Rather than implementing a type checking system badly and in an ad-hoc manner, I now lean more towards compile time checked languages, especially for pieces of code that I expect will run for a long time. I think the sweet spot would be optionally type-checked / type-inferred languages, for example Cython, Common Lisp. Strongly type-checked languages _tend_ to be all or nothing. EDIT: @nl yeah! absolutely, they arent by any means a bug predicting algo.

nl 15y ago
Type systems are great. Other things that are great are unit tests, automatic code inspection tools and code reviews. They aren't the same as bug prediction systems, though.
tedunangst 15y ago
I think, more accurately, one may rave about the irrelevant merits of static typing to the detriment of one's karma. Here's a bug from the real world: the login form doesn't render correctly in IE. When someone implements IE's layout algorithm as a haskell type, I'll start paying attention.
- srean 15y ago
  That there exist errors that cannot be type-checked makes type-checking irrelevant ? I dont quite see why exhaustiveness is a pre-requisite for relevance or attention. Is that a thumb rule you use for other things as well ? I am not sure if type-checking even aspires to be exhaustive. As long as it can detect my stupid and costly mistakes at a cost that is cheaper than the cost of the mistakes, I am happy. But I can understand that for people who do not make costly and compile time checkable mistakes, it might not be worth it. @tedunangst Got you. Your second paragraph in the previous comment threw me off. It does read like you consider type-checking unworthy of attention because it cannot catch all possible errors. Agree with your comment about people overstating their case. I think the noisiest of both the camps (if one can call it that) are equally guilty.

jdp23 15y ago

Defect detection != defect prediction

Lewisham 15y ago

Hi, I'm Chris, author of the blog/work. While I love me some type systems, I worry you're thinking a bit too code-centric. For example, let's say I describe to you a really hard problem, and you come up with a solution (that type checks!), and then we launch to users and they find a use case we never thought of, and our code suddenly breaks. I am sure this has happened to you. The problem isn't at the code level (that's just the manifestation), but our current understanding of the problem. IMHO, once you reach a certain level of competency, a sizable minority, if not majority, of bugs that make their way into a source control system (particularly one with code review, like Google's) are not line-by-line problems, but wider problems with actually understanding the task at hand and all that encompasses. When you work on the complexity of problems that Google does, you're tending towards a very large chunk of bugs being members of this class. However, we don't have good tools (yet) to have a computer properly check the semantic meaning of our code. Bug prediction sits as a sort of baby step. It's the computer making a best-effort guess of where issues will be. The algorithm designed here is saying "Look this file has been a struggle. It's probably going to continue being a struggle, so we need to focus attention when we change anything to it."

deleted 15y ago
[deleted]
T_S_ 15y ago
Apologies for the perception of trollishness in my comment. The intent was merely to reflect my own personal quest to write better code. Obviously data mining has its place in a shop that produces mountains of code like Google. Not to mention other techniques like unit testing, etc.

JabavuAdams 15y ago

These are trivial kinds of errors. The big fuckups are invariably more subtle and people / requirements oriented. When all you've got is a hammer ....