5 ms·

[[citation needed]]. "Everyone knows" that type systems reduce bugs, but actual evidence for that is pretty weak. https://danluu.com/empirical-pl/ https://danl

by atdt 6y ago

[[citation needed]]. "Everyone knows" that type systems reduce bugs, but actual evidence for that is pretty weak. https://danluu.com/empirical-pl/ https://danluu.com/empirical-pl/

sethammons 6y ago
That study calls out significant issues. I can go from personal experience. Porting python to Go. In python, we had to have tests dedicated to ensuring the right types were handled the right way. Is it a string or an array passed in? Welp, need a new test to cover that. Whole classes of tests go away with more advanced type systems. Those tests were needed to prevent those classes of bugs. So better type system == removing of whole class of bugs.
- sixstringtheory 6y ago
  Came here to say I've had this exact same debate. We were writing long Python test suites to check types. I constantly (and playfully, I don't care to die on this hill) pointed out this is fully automatable with a static typing system. I remember many times being told to just read thousands of lines of uncommented tests in order to understand some thing I was trying to debug. (Don't get me started on Python stack traces!)
  - viraptor 6y ago
    I've started applying `mypy --strict` to my new python code. It caught so many issues already it's definitely worth it. Mostly in cases of uncommon error handling, some in code which was tested but under wrong assumptions and would actually fail on production. (This is a problem with "you can test types approach")
    - SnowflakeOnIce 6y ago
      Yes, mypy is terrific, and has changed the way I write Python programs. Taking this gradual typing tool a step further, you can run your tests with the typeguard plugin for pytest, which will dynamically check the types of most values, pointing out places where your types are a lie. https://typeguard.readthedocs.io/en/latest/userguide.html#using-the-pytest-plugin https://typeguard.readthedocs.io/en/latest/userguide.html#us...
      - viraptor 6y ago
        That's amazing. Thank you for mentioning this.
- pfdietz 6y ago
  But, do you really save tests? You have to test your code anyway. Does a type system buy you anything more in code that's going to be heavily tested for other reasons?
  - kenhwang 6y ago
    So many tests it's not even funny. A huge chunk of tests in dynamic/softly typed languages just end up being input/output validations where in a strongly typed language you don't bother since you know the compiler would refuse to compile it.
    - pfdietz 6y ago
      Except if you are comprehensively testing your code for things other than what's caught by type checking, you get those validations for free. In my view, type checking lulls one into thinking the code is better than it actually is, just because it compiles. But it doesn't test the algorithms. And testing the algorithms exercises the code enough to reveal the shallow type errors.
      - kenhwang 6y ago
        Positive vs negative testing. Checking the algorithm is generally positive testing. You still should have negative testing, and in a strong type system language the compiler can be leveraged to do much more of that for you. But yes, there's a lull with type checking that makes people believe only negative testing is sufficient, which is as foolish as thinking only positive testing is sufficient.
- waheoo 6y ago
  This is where the type system debate breaks down. What you point out is a developer error. Mixing variable types in a dynamically typed language is, like generics, a code smell. Using a dynamically typed language is all about not having to babysit the compiler, its not about playing dumb about types. If you design your functions in a way that that function will never receive an array, you dont have to test for it. And if some idiot tries one day, it should error very loudly that something is wrong. Too many abuse dynamic typing to the point they feel safer offloading this headwork to a compiler. That is, they get the compiler to make sure nobody will ever send an array into that function that takes a string. All generics do is bring back the lack of typesafety and all the manual typechecking that comes with muddying generalizing the parameter typeset. I actually dont hate generics, they are super powerful, but I question every usage as suspicious. I also wish people would remember human brains arent all the same. We have different ways of thinking about problems. Some people need to have a compiler to busy work, some people need to do busy work while they think. I just wish people shitting on dynamic typed languages would remember they came to solve the mess and monotomy that arose from typesafety. Someone mentioned they felt go was going backwards from rust, i think both languages are massive steps backward from python.
  - icholy 6y ago
    > If you design your functions in a way that that function will never receive an array, you dont have to test for it. And if some idiot tries one day, it should error very loudly that something is wrong. Then you've moved it from a test to a runtime assertion. You're still re-implementing the type-checker, poorly.
    - bonestormii_ 6y ago
      And on top of this, it's usually not "some idiot" calling the function from a REPL. It's "some idiot's code", which may not be executed until the program is running in production somewhere if it is rare. "Oh, someone uploaded THAT kind of file? Well, let me see... AttributeError: Type SomeObject does not have attribute append. Here's a list of 47 functions that were called between the action you took and this error being raised. None of them guarantee the types going in or out, so you should probably get going waking through all this logic. You're welcome! -Python <3, XOXOXO"
- dragonwriter 6y ago
  > In python, we had to have tests dedicated to ensuring the right types were handled the right way. Is it a string or an array passed in? Welp, need a new test to cover that def foo(arg: str) -> int vs. def foo(arg: List[Union[Set[int],Optional[Dict[str, Any]]]]) -> str But...Python has a static type checker with a more expressive type system than Go. Also, IIRC, better type inference.
Polylactic_acid 6y ago
The evidence is pretty clear to anyone who writes in an untyped language. I can't count the number of bugs that I see in ruby code at work that would simply not exist in rust. The biggest one being unexpected nils.

by atdt 6y ago

[[citation needed]]. "Everyone knows" that type systems reduce bugs, but actual evidence for that is pretty weak. https://danluu.com/empirical-pl/ https://danluu.com/empirical-pl/

sethammons 6y ago

That study calls out significant issues. I can go from personal experience. Porting python to Go. In python, we had to have tests dedicated to ensuring the right types were handled the right way. Is it a string or an array passed in? Welp, need a new test to cover that. Whole classes of tests go away with more advanced type systems. Those tests were needed to prevent those classes of bugs. So better type system == removing of whole class of bugs.

sixstringtheory 6y ago
Came here to say I've had this exact same debate. We were writing long Python test suites to check types. I constantly (and playfully, I don't care to die on this hill) pointed out this is fully automatable with a static typing system. I remember many times being told to just read thousands of lines of uncommented tests in order to understand some thing I was trying to debug. (Don't get me started on Python stack traces!)
- viraptor 6y ago
  I've started applying `mypy --strict` to my new python code. It caught so many issues already it's definitely worth it. Mostly in cases of uncommon error handling, some in code which was tested but under wrong assumptions and would actually fail on production. (This is a problem with "you can test types approach")
  - SnowflakeOnIce 6y ago
    Yes, mypy is terrific, and has changed the way I write Python programs. Taking this gradual typing tool a step further, you can run your tests with the typeguard plugin for pytest, which will dynamically check the types of most values, pointing out places where your types are a lie. https://typeguard.readthedocs.io/en/latest/userguide.html#using-the-pytest-plugin https://typeguard.readthedocs.io/en/latest/userguide.html#us...
    - viraptor 6y ago
      That's amazing. Thank you for mentioning this.
pfdietz 6y ago
But, do you really save tests? You have to test your code anyway. Does a type system buy you anything more in code that's going to be heavily tested for other reasons?
- kenhwang 6y ago
  So many tests it's not even funny. A huge chunk of tests in dynamic/softly typed languages just end up being input/output validations where in a strongly typed language you don't bother since you know the compiler would refuse to compile it.
  - pfdietz 6y ago
    Except if you are comprehensively testing your code for things other than what's caught by type checking, you get those validations for free. In my view, type checking lulls one into thinking the code is better than it actually is, just because it compiles. But it doesn't test the algorithms. And testing the algorithms exercises the code enough to reveal the shallow type errors.
    - kenhwang 6y ago
      Positive vs negative testing. Checking the algorithm is generally positive testing. You still should have negative testing, and in a strong type system language the compiler can be leveraged to do much more of that for you. But yes, there's a lull with type checking that makes people believe only negative testing is sufficient, which is as foolish as thinking only positive testing is sufficient.
waheoo 6y ago
This is where the type system debate breaks down. What you point out is a developer error. Mixing variable types in a dynamically typed language is, like generics, a code smell. Using a dynamically typed language is all about not having to babysit the compiler, its not about playing dumb about types. If you design your functions in a way that that function will never receive an array, you dont have to test for it. And if some idiot tries one day, it should error very loudly that something is wrong. Too many abuse dynamic typing to the point they feel safer offloading this headwork to a compiler. That is, they get the compiler to make sure nobody will ever send an array into that function that takes a string. All generics do is bring back the lack of typesafety and all the manual typechecking that comes with muddying generalizing the parameter typeset. I actually dont hate generics, they are super powerful, but I question every usage as suspicious. I also wish people would remember human brains arent all the same. We have different ways of thinking about problems. Some people need to have a compiler to busy work, some people need to do busy work while they think. I just wish people shitting on dynamic typed languages would remember they came to solve the mess and monotomy that arose from typesafety. Someone mentioned they felt go was going backwards from rust, i think both languages are massive steps backward from python.
- icholy 6y ago
  > If you design your functions in a way that that function will never receive an array, you dont have to test for it. And if some idiot tries one day, it should error very loudly that something is wrong. Then you've moved it from a test to a runtime assertion. You're still re-implementing the type-checker, poorly.
  - bonestormii_ 6y ago
    And on top of this, it's usually not "some idiot" calling the function from a REPL. It's "some idiot's code", which may not be executed until the program is running in production somewhere if it is rare. "Oh, someone uploaded THAT kind of file? Well, let me see... AttributeError: Type SomeObject does not have attribute append. Here's a list of 47 functions that were called between the action you took and this error being raised. None of them guarantee the types going in or out, so you should probably get going waking through all this logic. You're welcome! -Python <3, XOXOXO"
dragonwriter 6y ago
> In python, we had to have tests dedicated to ensuring the right types were handled the right way. Is it a string or an array passed in? Welp, need a new test to cover that def foo(arg: str) -> int vs. def foo(arg: List[Union[Set[int],Optional[Dict[str, Any]]]]) -> str But...Python has a static type checker with a more expressive type system than Go. Also, IIRC, better type inference.

Polylactic_acid 6y ago

The evidence is pretty clear to anyone who writes in an untyped language. I can't count the number of bugs that I see in ruby code at work that would simply not exist in rust. The biggest one being unexpected nils.