3 ms·

> most TDD effort could be replaced by a very simple type system I'm not convinced that's true. When writing Python, even before tools like mypy, most of my t

by moeris 5y ago

> most TDD effort could be replaced by a very simple type system I'm not convinced that's true. When writing Python, even before tools like mypy, most of my tests weren't assertions about types. They involved types implicitly, and so functioned as a form of type checking. But most of my tests were about invariants and specific oracles For example, I don't need to check that a result is a `str` if I'm checking that it starts with "my-prefix". I'd be much more inclined to say that a simple contract system could replace most unit tests.

Jensson 5y ago
The big difference is the amount of inputs you have to test for, dynamic languages can fail in infinitely more ways. In a compile time type checked language you know what types can be passed to the function so the amount of possible execution paths is much lower. and in many cases the type checking itself is enough for you to be confident that the code works. In dynamic languages you always need a test that executed the code as even small things like typos will cause the code to fail at runtime.
- pydry 5y ago
  I tend to find that "wrong type" accounts for a vanishingly small % of production bugs in average python programs. The only exceptions I experienced this on were on A) projects that had zero tests and B) projects that insisted on passing around dicts and primitives rather than classes. If the project developed a mimimal level of testing that one might tentatively argue that you'd want anyway to validate behavior and made half an attempt to use OO this class of bug seemed to dry up pretty quickly. I've also seen Java developers switch to python and write entirely untested code and then get frustrated that they got type errors at runtime that the Java compiler used to catch for them. I wouldnt exactly say I felt sympathetic to their plight.
  - zppln 5y ago
    This is pretty much my experience as well, including the part about Java programmers.
  - dllthomas 5y ago
    > "wrong type" I find that people have very different understandings of what "wrong type" means. If you restrict yourself to places where python or js actually mention type in the error messages, that misses a whole lot of things that basic use of a simple type system would catch, much less skilled use of a sophisticated type system. There's an example I give - admittedly tangential given that it does not replace (and could not be replaced by) unit tests, but hopefully giving some insight into the range of what's possible: I once used C's type system to catch "you called this function from the wrong thread" at compile time, converting subtle runtime failures or meaningful runtime overhead into "you can't call this here, dummy" before running anything. This was hugely helpful as requirements evolved and I found myself moving large pieces of functionality between threads. I don't think this clearly resolves the broader discussion; static type checking is a tool and its applicability may well depend on the problems at hand, the language in question, and the developers involved; I think how it should be applied certainly depends on these things. But I think it is too often dismissed based on an overly narrow understanding of what a "type error" can be.
    - pydry 5y ago
      I use asserts in python in a similar manner to the way you used types in C. Combined with tests it's a highly flexible way of "locking down" behaviors that should be impossible.
dwohnitmok 5y ago
> most of my tests weren't assertions about types. They involved types implicitly, and so functioned as a form of type checking. But most of my tests were about invariants and specific oracles While it's true that most type systems are not capable of completely replacing tests (and for those where they might be possible, you might not want to), it's also important to point out that the real gains from types aren't from "oh make sure that this input is actually an integer and not a boolean." If you have the following three features, which a lot of type systems have, you can get a ton of mileage far beyond simple "wrong shape" errors. 1. Exhaustivity checking of different type variants (e.g. things like `type NonBooleanLogic = True or False or Unknown` whether that's implemented through sealed subclassing or sum types) when you write logic that switches against different variants (i.e. switch(myNonBooleanLogicResult) on { True -> // Do something False -> // Do something // Typechecker errors out and tells you that you've failed to cover the Unknown case } ). 2. Ability to cheaply make new types 3. Ability to make certain type constructors and functions private For example how do I ensure that all data passing through our codebase that makes it into our database is sanitized? Make a `Sanitized` type with a private constructor which can only be constructed in the function that actually performs sanitization. Then make the input type to the function that writes to the database `Sanitized` and hide the non-sanitized input function as a private function that can't be accessed anywhere else. Tada now you have very high confidence that all data going into your database is sanitized, no matter how many code paths in your codebase you have that ultimately write to the database. By making the constructor private, you've ensured that all those code paths must've called the sanitization function somewhere (and if you make `Sanitized` immutable you've also ensured that it cannot be tampered with after being sanitized). Likewise if I had a bunch of code that consumed and produced `NonBooleanLogic`s and I wanted to add a new value to `NonBooleanLogic` such as `NeitherTrueNorFalse` , then I don't need to go up to various coworker and ask about all the places in various modules that I should change to make sure that they all process this new `NeitherTrueNorFalse` value correctly. I just add it to `NonBooleanLogic` and the typechecker plays the role of my coworkers, going around and pointing out every part in the codebase that requires my attention. There's a bunch of these simple tricks that seem inconsequential on their own, but taken together greatly increase confidence when refactoring code or adding new features to a preexisting codebase that you've done everything correctly, more so than what "oh just make sure this integer isn't a boolean" would seem to entail.
drewcoo 5y ago
Contracts are typically for higher order behaviors than unit tests target. They're for the component/service functional granularity. Contract testing is great, though. And given that the contracts are about behaviors, they're a great candidate for BDD. (But please hold the Cucumber!)

by moeris 5y ago

Jensson 5y ago

The big difference is the amount of inputs you have to test for, dynamic languages can fail in infinitely more ways. In a compile time type checked language you know what types can be passed to the function so the amount of possible execution paths is much lower. and in many cases the type checking itself is enough for you to be confident that the code works. In dynamic languages you always need a test that executed the code as even small things like typos will cause the code to fail at runtime.

pydry 5y ago
I tend to find that "wrong type" accounts for a vanishingly small % of production bugs in average python programs. The only exceptions I experienced this on were on A) projects that had zero tests and B) projects that insisted on passing around dicts and primitives rather than classes. If the project developed a mimimal level of testing that one might tentatively argue that you'd want anyway to validate behavior and made half an attempt to use OO this class of bug seemed to dry up pretty quickly. I've also seen Java developers switch to python and write entirely untested code and then get frustrated that they got type errors at runtime that the Java compiler used to catch for them. I wouldnt exactly say I felt sympathetic to their plight.
- zppln 5y ago
  This is pretty much my experience as well, including the part about Java programmers.
- dllthomas 5y ago
  > "wrong type" I find that people have very different understandings of what "wrong type" means. If you restrict yourself to places where python or js actually mention type in the error messages, that misses a whole lot of things that basic use of a simple type system would catch, much less skilled use of a sophisticated type system. There's an example I give - admittedly tangential given that it does not replace (and could not be replaced by) unit tests, but hopefully giving some insight into the range of what's possible: I once used C's type system to catch "you called this function from the wrong thread" at compile time, converting subtle runtime failures or meaningful runtime overhead into "you can't call this here, dummy" before running anything. This was hugely helpful as requirements evolved and I found myself moving large pieces of functionality between threads. I don't think this clearly resolves the broader discussion; static type checking is a tool and its applicability may well depend on the problems at hand, the language in question, and the developers involved; I think how it should be applied certainly depends on these things. But I think it is too often dismissed based on an overly narrow understanding of what a "type error" can be.
  - pydry 5y ago
    I use asserts in python in a similar manner to the way you used types in C. Combined with tests it's a highly flexible way of "locking down" behaviors that should be impossible.

dwohnitmok 5y ago

> most of my tests weren't assertions about types. They involved types implicitly, and so functioned as a form of type checking. But most of my tests were about invariants and specific oracles While it's true that most type systems are not capable of completely replacing tests (and for those where they might be possible, you might not want to), it's also important to point out that the real gains from types aren't from "oh make sure that this input is actually an integer and not a boolean." If you have the following three features, which a lot of type systems have, you can get a ton of mileage far beyond simple "wrong shape" errors. 1. Exhaustivity checking of different type variants (e.g. things like `type NonBooleanLogic = True or False or Unknown` whether that's implemented through sealed subclassing or sum types) when you write logic that switches against different variants (i.e. switch(myNonBooleanLogicResult) on { True -> // Do something False -> // Do something // Typechecker errors out and tells you that you've failed to cover the Unknown case } ). 2. Ability to cheaply make new types 3. Ability to make certain type constructors and functions private For example how do I ensure that all data passing through our codebase that makes it into our database is sanitized? Make a `Sanitized` type with a private constructor which can only be constructed in the function that actually performs sanitization. Then make the input type to the function that writes to the database `Sanitized` and hide the non-sanitized input function as a private function that can't be accessed anywhere else. Tada now you have very high confidence that all data going into your database is sanitized, no matter how many code paths in your codebase you have that ultimately write to the database. By making the constructor private, you've ensured that all those code paths must've called the sanitization function somewhere (and if you make `Sanitized` immutable you've also ensured that it cannot be tampered with after being sanitized). Likewise if I had a bunch of code that consumed and produced `NonBooleanLogic`s and I wanted to add a new value to `NonBooleanLogic` such as `NeitherTrueNorFalse` , then I don't need to go up to various coworker and ask about all the places in various modules that I should change to make sure that they all process this new `NeitherTrueNorFalse` value correctly. I just add it to `NonBooleanLogic` and the typechecker plays the role of my coworkers, going around and pointing out every part in the codebase that requires my attention. There's a bunch of these simple tricks that seem inconsequential on their own, but taken together greatly increase confidence when refactoring code or adding new features to a preexisting codebase that you've done everything correctly, more so than what "oh just make sure this integer isn't a boolean" would seem to entail.

drewcoo 5y ago

Contracts are typically for higher order behaviors than unit tests target. They're for the component/service functional granularity. Contract testing is great, though. And given that the contracts are about behaviors, they're a great candidate for BDD. (But please hold the Cucumber!)