4 ms·

Of course in production software type errors are rare. They're generally the first errors that pop up during debugging or testing. Which means the primary benef

Of course in production software type errors are rare. They're generally the first errors that pop up during debugging or testing. Which means the primary benefit of a basic static type system is speeding up the feedback loop of finding all your type errors, and reducing the need for testing. The funny thing about that blog post is that the top 25 bugs "don't look like" type errors to the author, but almost all of them are type errors given some type system and type definition...and not just theoretical, but by existing languages and usages. Buffer overflow, for example, isn't a "type error" according to C, but it is according to Rust. As type systems become stronger and more advanced, more errors become type errors, which means you'll only recognize them as type errors if you use languages and constructs that make them type errors.

mpweiher 10y ago
> Of course in production software type errors are rare. Not according to static-typing fundamentalists. > They're generally the first errors that pop up during debugging or testing. Exactly, and dynamic languages tend to be highly interactive, whereas most languages with highly evolved static type systems tend to have very slow compilers. So the idea that the compiler for that language gives you useful feedback before your dynamic system gives you feedback from actually running the code is at best dubious. > ...but almost all of them are type errors I think you are confusing type errors with modeling errors, but that's very common.
- uglycoyote 10y ago
  Hi... Static typing fundamentalist here. > So the idea that the compiler for that language gives you useful feedback before your dynamic system gives you feedback from actually running the code is at best dubious. It depends highly on how long it takes to run the code. I work on large pieces of software both static and dynamic which can take a long time to restart and test after changing the code. (far longer than the turnaround time for a static-language change-and-recompile turnaround). In the dynamic systems it frequently takes all day to make what should be a simple change because the turnaround time to test for simple errors is so lengthy and it requires multiple tries just to get it to "compile" (I use this word in a loose sense since there's no compiler but I say "compile" meaning types are correct, I'm calling functions that actually exist, accessing data members that actually exist, free of usages of undefined variables etc.). Not only does it take far longer to purge all of the errors from the newly written code in dynamic languages but I also find that when working with dynamic code that I didn't write myself, the lack of explorability (either manually or IDE-driven) hinders understanding of the code and therefore writing new code involves much more guesswork, which means I write far more incorrect code in the dynamic language. Once the code gets to the point where it is "production ready" it's usually free of these sort of errors -- whether it is written in a dynamic or static language. Otherwise, it would fail and by definition would not meet the label of "production ready". So I would tend to agree with the grandparent's statement that "in production software type errors are rare". The type of software that I work on though tends to be in house software that's a bit more bleeding edge -- it's constantly being adapted to be used in new situations, needing to be changed, and never quite meeting the label "production ready" in that it's not something that QA would approve of and we would ship to other users. For this kind of software, I would say that the parts written in dynamic languages have far more latent type errors than the static code. These a little landmines that don't occur in the everyday use of the software but users step on all the time when they hit edge cases that no coder had enough imagination to test for. In my own experience, writing code in a dynamic language can be faster than in a static language, but this usually falls apart around the point where the code gets to the point of being more than will fit on one monitor screen, or more than will fit in one person's head at any time. That's the point where in the dynamic language you start having to guess things, and in the static language the computer helps you avoid the guesswork. I would disagree with your point about slow compilers, because for smallish codebases static compilers are practically instantaneous anyhow, and for large ones the cost of compilation in still much smaller than the cost of one testing iteration of the application.
  - mpweiher 10y ago
    > Hi... Static typing fundamentalist here. Nice to meet you :-) > long time to restart and test after changing the code. So one of the reasons for having dynamically typed languages is that they allow you to modify the code without restarting the "program". In some non-trivial senses, there are Smalltalk images that have been running non-stop since ~1976 and have been transformed from running on 16 bit Altos to 64 bit x86s and various other machines. The reason being dynamic is important for that sort of capability is that allowing temporary inconsistencies allows you to incrementally get from point a to point b, and is also important for exploratory programming. While I don't know your circumstances, it sounds like you are paying the cost of a dynamic approach without reaping the benefits. That probably sucks. I guess the important point here is that dynamic approaches enable certain capabilities. That's why they're useful. A lot of the criticism I see seems to boil down to "take a static language and the development practices associated with that and just strip away the types". That's not useful. That said, there are obviously areas where one approach works better and other areas where another approach is more appropriate. For example, it could be that the structure of your programs/data is such that you couldn't do an incremental adjustment without restarting even if you had the technical capability (it sounds like you don't currently have that capability, correct me if I am wrong) Your reasoning actually sounds very non-fundamentalist. > [faster dynamic] usually falls apart around the point > where the code gets to the point of being more than > will fit on one monitor screen, Static types have a now well-documented positive effect on comprehensibility of foreign codebases, and this typically includes the code I wrote yesterday ;-). It's not a safety effect, though, and apparently it works even if the type-system is not sound and therefore not safe. > I would disagree with your point about slow compilers > because for smallish codebases static compilers are > practically instantaneous anyhow, and for large ones > the cost of compilation in still much smaller than > the cost of one testing iteration of the application. Have to disagree with you there. In my experience with two such languages, Swift and Scala, compile times can be (and typically are) painful even for fairly small code-bases. Heck even C++ is pretty awful in that regard. And as to restarting, see my previous comments.
    - uglycoyote 10y ago
      Thanks for the interesting reply. I haven't tried swift or Scala, but my experience is that I mostly work in C++ (in console video games). Even though there are thousands of files and millions of lines of code, the time it takes to recompile after a small change is typically 5-10 seconds which is nothing compared to the time it takes to fix the mistake if i type the wrong thing, so the static compilation step is very welcome in this case. Compiling can take minutes with certain kinds of changes (e.g. change the signature of a widely used function that is included in many other files) but that's also generally a case where the longer compilation time is welcome because it tells my all of the myriad of call sites where the function is now being used incorrectly and needs to be fixed. I have a lot of C# experience and generally find compilation times to be a non-issue there even on large projects. It was interesting what you said about the advantage of dynamic languages is making changes while the program is still running. How does this work in the face of code that crashes? Do you just build some kind of well-armoured inner loop with lots of exception handling that you never have to change, and then all of your changes are in modules which are can fail without taking the system down? We have a python-based "pipeline" which manages the building of all of our game assets and takes a long time to restart since it has a database of gigabytes of information that it needs to reload on startup. And changing the code involves a restart, so the cost of getting the code wrong is a long wait. I know that python does allow for reloading of modules, but the architect of this system has told me that it would be a bad idea due to possible data layout inconsistencies between the new code and the resident data, whereas reloading and deserializing the information is safer.
- dllthomas 10y ago
  > So the idea that the compiler for that language gives you useful feedback before your dynamic system gives you feedback from actually running the code is at best dubious. Working in Haskell, I get useful feedback about code before I write it! I find this very conspicuously missing when I work in Python. I can't ask "what's the shape of what I need to put here?"
  - thinkpad20 10y ago
    I write quite a lot in both Haskell and Python. In python, I pretty much never find myself wondering what shape of object I need to put somewhere, because the type system is so simple and the language is imperative, so there is rarely a need for any of the kind of elaborate highly intricate types that one sees in Haskell. That plus having a simple debugger like pdb and easy access to print statements etc tend to make it a breeze. On the other hand, writing Haskell without a type system (if it were even possible) would be a complete nightmare, because of the very complex types (monad transformers, lenses, higher order functions, etc) required to express the same kind of computations that are easy to express in an imperative language. I definitely think Haskell's type system is a huge asset to writing correct programs, but I'm not sure that this particular strength is one that I would echo.
    - dllthomas 10y ago
      > In python, I pretty much never find myself wondering what shape of object I need to put somewhere, because the type system is so simple and the language is imperative, so there is rarely a need for any of the kind of elaborate highly intricate types that one sees in Haskell. It doesn't take "elaborate types" - just the question of "what interfaces does this need to support?" is a hard one to answer if you don't already have correct (and sufficiently detailed) docs. "This needs a duck... does it need to walk? quack? both? fly?" It's possible that the Python code bases I've worked in have been poor that I need to ask these questions, but I don't know how I would have improved them. > That plus having a simple debugger like pdb and easy access to print statements etc tend to make it a breeze. But in order for pdb to work, I have to have already written the code. It's inherently "push based", whereas in GHCi (or with typed-holes) I can "pull". (Which isn't to say that sometimes "push based" questions aren't the ones I want to ask, or that it wouldn't sometimes be nice to have a better debugger in Haskell.)
- saosebastiao 10y ago
  > Not according to static-typing fundamentalists Not in production software, no. Because even the shittiest dynamic language software still gets run once or twice before it gets put into production. Type errors are extremely common during development, and that is where static types shine. > Exactly, and dynamic languages tend to be highly interactive, whereas most languages with highly evolved static type systems tend to have very slow compilers. So the idea that the compiler for that language gives you useful feedback before your dynamic system gives you feedback from actually running the code is at best dubious. My extremely slow scala compiler can compile 50k lines from scratch faster than a boilerplate rails project can even start up. And I never compile from scratch during development, because I have incremental compilation. Even as slow as the compiler is, it is never as slow as compiling+running code of similar complexity in a repl, and it's not even close. > I think you are confusing type errors with modeling errors, but that's very common. Nope. Type errors. Just because they aren't called type errors in one language doesn't mean they are not type errors in another language with a different type system.
  - mpweiher 10y ago
    > Nope. Type errors. I see your "nope" and raise you a "nope, not at all". A lot of these errors have relevant information modeled as strings. Your static type checker is only going to tell you that these are two strings, and yes, they happen to be compatible. Congratulations! Oh, you meant that these things (like SQL statements) should not be represented as strings? And that that modeling should include semantically distinct entities for user strings (need quoting) and SQL syntax (not quotes)? That's modeling, whether you do static type checking or not.
    - saosebastiao 10y ago
      You mean it's impossible to have compile time type safety of sql queries, and that the compiler can't tell the difference between an SQL string and any other string? You're wrong. http://underscore.io/blog/posts/2015/05/28/typechecking-sql.html http://underscore.io/blog/posts/2015/05/28/typechecking-sql....
- rdnetto 10y ago
  > I think you are confusing type errors with modeling errors, but that's very common. How would you distinguish between them? Strong/expressive type systems encode the modelling in a way which is mechanically verifiable. e.g. by not permitting negative or fractional numbers to be submitted to a function which indexes an array.

mpweiher 10y ago

> Of course in production software type errors are rare. Not according to static-typing fundamentalists. > They're generally the first errors that pop up during debugging or testing. Exactly, and dynamic languages tend to be highly interactive, whereas most languages with highly evolved static type systems tend to have very slow compilers. So the idea that the compiler for that language gives you useful feedback before your dynamic system gives you feedback from actually running the code is at best dubious. > ...but almost all of them are type errors I think you are confusing type errors with modeling errors, but that's very common.

uglycoyote 10y ago
Hi... Static typing fundamentalist here. > So the idea that the compiler for that language gives you useful feedback before your dynamic system gives you feedback from actually running the code is at best dubious. It depends highly on how long it takes to run the code. I work on large pieces of software both static and dynamic which can take a long time to restart and test after changing the code. (far longer than the turnaround time for a static-language change-and-recompile turnaround). In the dynamic systems it frequently takes all day to make what should be a simple change because the turnaround time to test for simple errors is so lengthy and it requires multiple tries just to get it to "compile" (I use this word in a loose sense since there's no compiler but I say "compile" meaning types are correct, I'm calling functions that actually exist, accessing data members that actually exist, free of usages of undefined variables etc.). Not only does it take far longer to purge all of the errors from the newly written code in dynamic languages but I also find that when working with dynamic code that I didn't write myself, the lack of explorability (either manually or IDE-driven) hinders understanding of the code and therefore writing new code involves much more guesswork, which means I write far more incorrect code in the dynamic language. Once the code gets to the point where it is "production ready" it's usually free of these sort of errors -- whether it is written in a dynamic or static language. Otherwise, it would fail and by definition would not meet the label of "production ready". So I would tend to agree with the grandparent's statement that "in production software type errors are rare". The type of software that I work on though tends to be in house software that's a bit more bleeding edge -- it's constantly being adapted to be used in new situations, needing to be changed, and never quite meeting the label "production ready" in that it's not something that QA would approve of and we would ship to other users. For this kind of software, I would say that the parts written in dynamic languages have far more latent type errors than the static code. These a little landmines that don't occur in the everyday use of the software but users step on all the time when they hit edge cases that no coder had enough imagination to test for. In my own experience, writing code in a dynamic language can be faster than in a static language, but this usually falls apart around the point where the code gets to the point of being more than will fit on one monitor screen, or more than will fit in one person's head at any time. That's the point where in the dynamic language you start having to guess things, and in the static language the computer helps you avoid the guesswork. I would disagree with your point about slow compilers, because for smallish codebases static compilers are practically instantaneous anyhow, and for large ones the cost of compilation in still much smaller than the cost of one testing iteration of the application.
- mpweiher 10y ago
  > Hi... Static typing fundamentalist here. Nice to meet you :-) > long time to restart and test after changing the code. So one of the reasons for having dynamically typed languages is that they allow you to modify the code without restarting the "program". In some non-trivial senses, there are Smalltalk images that have been running non-stop since ~1976 and have been transformed from running on 16 bit Altos to 64 bit x86s and various other machines. The reason being dynamic is important for that sort of capability is that allowing temporary inconsistencies allows you to incrementally get from point a to point b, and is also important for exploratory programming. While I don't know your circumstances, it sounds like you are paying the cost of a dynamic approach without reaping the benefits. That probably sucks. I guess the important point here is that dynamic approaches enable certain capabilities. That's why they're useful. A lot of the criticism I see seems to boil down to "take a static language and the development practices associated with that and just strip away the types". That's not useful. That said, there are obviously areas where one approach works better and other areas where another approach is more appropriate. For example, it could be that the structure of your programs/data is such that you couldn't do an incremental adjustment without restarting even if you had the technical capability (it sounds like you don't currently have that capability, correct me if I am wrong) Your reasoning actually sounds very non-fundamentalist. > [faster dynamic] usually falls apart around the point > where the code gets to the point of being more than > will fit on one monitor screen, Static types have a now well-documented positive effect on comprehensibility of foreign codebases, and this typically includes the code I wrote yesterday ;-). It's not a safety effect, though, and apparently it works even if the type-system is not sound and therefore not safe. > I would disagree with your point about slow compilers > because for smallish codebases static compilers are > practically instantaneous anyhow, and for large ones > the cost of compilation in still much smaller than > the cost of one testing iteration of the application. Have to disagree with you there. In my experience with two such languages, Swift and Scala, compile times can be (and typically are) painful even for fairly small code-bases. Heck even C++ is pretty awful in that regard. And as to restarting, see my previous comments.
  - uglycoyote 10y ago
    Thanks for the interesting reply. I haven't tried swift or Scala, but my experience is that I mostly work in C++ (in console video games). Even though there are thousands of files and millions of lines of code, the time it takes to recompile after a small change is typically 5-10 seconds which is nothing compared to the time it takes to fix the mistake if i type the wrong thing, so the static compilation step is very welcome in this case. Compiling can take minutes with certain kinds of changes (e.g. change the signature of a widely used function that is included in many other files) but that's also generally a case where the longer compilation time is welcome because it tells my all of the myriad of call sites where the function is now being used incorrectly and needs to be fixed. I have a lot of C# experience and generally find compilation times to be a non-issue there even on large projects. It was interesting what you said about the advantage of dynamic languages is making changes while the program is still running. How does this work in the face of code that crashes? Do you just build some kind of well-armoured inner loop with lots of exception handling that you never have to change, and then all of your changes are in modules which are can fail without taking the system down? We have a python-based "pipeline" which manages the building of all of our game assets and takes a long time to restart since it has a database of gigabytes of information that it needs to reload on startup. And changing the code involves a restart, so the cost of getting the code wrong is a long wait. I know that python does allow for reloading of modules, but the architect of this system has told me that it would be a bad idea due to possible data layout inconsistencies between the new code and the resident data, whereas reloading and deserializing the information is safer.
dllthomas 10y ago
> So the idea that the compiler for that language gives you useful feedback before your dynamic system gives you feedback from actually running the code is at best dubious. Working in Haskell, I get useful feedback about code before I write it! I find this very conspicuously missing when I work in Python. I can't ask "what's the shape of what I need to put here?"
- thinkpad20 10y ago
  I write quite a lot in both Haskell and Python. In python, I pretty much never find myself wondering what shape of object I need to put somewhere, because the type system is so simple and the language is imperative, so there is rarely a need for any of the kind of elaborate highly intricate types that one sees in Haskell. That plus having a simple debugger like pdb and easy access to print statements etc tend to make it a breeze. On the other hand, writing Haskell without a type system (if it were even possible) would be a complete nightmare, because of the very complex types (monad transformers, lenses, higher order functions, etc) required to express the same kind of computations that are easy to express in an imperative language. I definitely think Haskell's type system is a huge asset to writing correct programs, but I'm not sure that this particular strength is one that I would echo.
  - dllthomas 10y ago
    > In python, I pretty much never find myself wondering what shape of object I need to put somewhere, because the type system is so simple and the language is imperative, so there is rarely a need for any of the kind of elaborate highly intricate types that one sees in Haskell. It doesn't take "elaborate types" - just the question of "what interfaces does this need to support?" is a hard one to answer if you don't already have correct (and sufficiently detailed) docs. "This needs a duck... does it need to walk? quack? both? fly?" It's possible that the Python code bases I've worked in have been poor that I need to ask these questions, but I don't know how I would have improved them. > That plus having a simple debugger like pdb and easy access to print statements etc tend to make it a breeze. But in order for pdb to work, I have to have already written the code. It's inherently "push based", whereas in GHCi (or with typed-holes) I can "pull". (Which isn't to say that sometimes "push based" questions aren't the ones I want to ask, or that it wouldn't sometimes be nice to have a better debugger in Haskell.)
saosebastiao 10y ago
> Not according to static-typing fundamentalists Not in production software, no. Because even the shittiest dynamic language software still gets run once or twice before it gets put into production. Type errors are extremely common during development, and that is where static types shine. > Exactly, and dynamic languages tend to be highly interactive, whereas most languages with highly evolved static type systems tend to have very slow compilers. So the idea that the compiler for that language gives you useful feedback before your dynamic system gives you feedback from actually running the code is at best dubious. My extremely slow scala compiler can compile 50k lines from scratch faster than a boilerplate rails project can even start up. And I never compile from scratch during development, because I have incremental compilation. Even as slow as the compiler is, it is never as slow as compiling+running code of similar complexity in a repl, and it's not even close. > I think you are confusing type errors with modeling errors, but that's very common. Nope. Type errors. Just because they aren't called type errors in one language doesn't mean they are not type errors in another language with a different type system.
- mpweiher 10y ago
  > Nope. Type errors. I see your "nope" and raise you a "nope, not at all". A lot of these errors have relevant information modeled as strings. Your static type checker is only going to tell you that these are two strings, and yes, they happen to be compatible. Congratulations! Oh, you meant that these things (like SQL statements) should not be represented as strings? And that that modeling should include semantically distinct entities for user strings (need quoting) and SQL syntax (not quotes)? That's modeling, whether you do static type checking or not.
  - saosebastiao 10y ago
    You mean it's impossible to have compile time type safety of sql queries, and that the compiler can't tell the difference between an SQL string and any other string? You're wrong. http://underscore.io/blog/posts/2015/05/28/typechecking-sql.html http://underscore.io/blog/posts/2015/05/28/typechecking-sql....
rdnetto 10y ago
> I think you are confusing type errors with modeling errors, but that's very common. How would you distinguish between them? Strong/expressive type systems encode the modelling in a way which is mechanically verifiable. e.g. by not permitting negative or fractional numbers to be submitted to a function which indexes an array.