4 ms·

Well designed Static Types are better than Unit Tests. They only fail because they are hard, because c++ is hard, because our present cultural reality is optimi

Well designed Static Types are better than Unit Tests. They only fail because they are hard, because c++ is hard, because our present cultural reality is optimized for Time to Market, because we dont teach computer science to kids in elementary school and because if it works, it works, we ship it and move on. I propose that Unit Tests have arisen in popularity solely because of the rise of Python, as a band aid to the complexity that rises as an untyped code base gets large. Matt Godbolt does a great job giving an overview of the power of static types to ensure correctness in this C++ On Sea Talk [1]. I've worked in places where people evangelize Unit Tests. But it is certainly true that no one really code reviews the Unit Tests that hard. People generally skim the tests file to see how many tests there are and give it the thumbs up. But then they seriously review the feature code. If TDD is to truly followed, shouldnt we review the tests more thoroughly than the implementation? Does anyone actually do that? My basic assertion is that, in practice, no one _really_ follows TDD. I think it's more of an easy political win. If you evangelize TDD, you look like a "real programmer" who cares about Stability and Reliability, but are you actually following through with that? Are you rejecting pull requests because some test case isn't present? Are you primarily reviewing the tests above the implementation? Is it actually feasible to test every code path in an untyped language? I would propose that it is not. That correctness can only come through the compiler, through a static type system. 1: https://www.youtube.com/watch?v=nLSm3Haxz0I https://www.youtube.com/watch?v=nLSm3Haxz0I

packetlost 6y ago
There's more than 1 class of problems that unit tests solve though. I'd still want pretty good testing over my business logic heavy code. I'd also like to point out that Python's typing module + mypy is, while still not perfect, quite good as a tacked-on type system. Also I believe unit tests as a concept first gained popularity in the Java ecosystem, not Python, which kind of negates the assertion that they are popularized because of Python's typing shortcomings.
- orwin 6y ago
  No, it was from the smalltalk guys, who later designed Eclisped and came to Java. But it was from smalltalk and never caught up until PHP, Ruby and python came strong.
arunix 6y ago
I remember reading Kent Beck's TDD book many years ago. Most of the example code is in Java. There's one chapter that uses Python, but that's just to show how a TDD framework could be implemented in a language which (at that time) didn't have one.
pmcollins 6y ago
i feel like uncle bob’s opinion on static types arose when he became an aficionado of clojure, which doesn’t have them.
- Jach 6y ago
  You can actually find his opinion forming even earlier, e.g. here's something from 2003: https://www.artima.com/weblogs/viewpost.jsp?thread=4639 https://www.artima.com/weblogs/viewpost.jsp?thread=4639 His opinions do change though, if slowly. Perhaps in 10 years he'll try Common Lisp.
  - nightowl_games 6y ago
    He mentions Python as a correlated factor in his love of unit tests. He also reveals how 'trendy' his mindset and strong opinions are.
jakelazaroff 6y ago
This is basically the other extreme. Why must we pick a side? Static types are useful. Unit tests are useful. One of my projects involves a bunch of graphics code. A lot of tricky math and special cases. When I was trying to refactor that code, I added a bunch of unit tests to make sure images would continue to be rendered correctly. And they prevented me from shipping a bunch of rendering bugs! Bugs that the type system would have let slip through, because the function signatures didn't change.
- nightowl_games 6y ago
  > This is basically the other extreme. Why must we pick a side? Static types are useful. Unit tests are useful. I did not say that Unit Tests are not useful. I am merely proposing that they are less useful than Static Typing and hypothesizing on why TDD is common, and commenting on my experience with TDD evangelists. I agree that Unit Tests are useful and valuable.
- commandlinefan 6y ago
  > Static types are useful. Unit tests are useful. And unit tests with static types are even more useful still!
D13Fd 6y ago
Static types don't tell you if you broke something when you changed the code.
- nightowl_games 6y ago
  That's a very broad statement. There are classes of bugs that can be detected by the compiler, and there are classes of bugs that cannot. Both of these statements are also true for Unit Tests.
- mhh__ 6y ago
  That depends on the type system, although in practice encoding this information isn't easy I readily admit.
- rovolo 6y ago
  They tell you if you broke a class of things, and I think that class of issues is pretty broad: * when you remove a method from a class, no user of that class calls that method * when you change the return type of a method, all users of that method are updated to handling the new type * in type systems with nullability, they make sure all users handle null values, and they make sure methods with non-null return types never return a null value There are functional changes which can break your code which won't be caught by the type system, but there are a lot of details which the type system will make sure you check when you make changes.
- gherkinnn 6y ago
  Not entirely. Types let me know whenever I try to connect a water pipe to the electricity. Or when I forget to wire up the grounding. What it can't tell me is that the lights go on when I flip the switch.
- orwin 6y ago
  -Warning -Werror -Wall -Wextra will tell you plenty. And you can add -Wconversion if you want to be clear on int conversion, and maybe -WShadow if you want to be sure you code is easily understandable. I did TDD five time. Twice with python, one with Go, onw with Rust and once with Common lisp. It was usefull with python, not great with Go and especially Common lisp (i mean, debugging while the code is held is better than debugging with tests, and Common lisp is a really good language to prototype, and TDD takes that out). And Miserable with Rust. With rust i'll test functionality and unsafe code, but i won't ever do TDD again, even if you paid me FU money.
novembermike 6y ago
TDD is more of a Java thing and it has a lot to do with the lack of a good type system in Java. TDD essentially lets you create a class that encapsulates a type and then enforce the contract for that type through the tests. Modern type systems tend to just let you do it directly so the unit tests are a little less useful.
nightowl_games 6y ago
Another factor is the complexity of the tests. If you really want to cover as much of the cases as possible in your tests, you end up all this bespoke mocking, spying, argument capturing and asserting stuff. Your test suite tends to be completely designed around some third party mocking library. True TDD evangelists will tell you that you will learn to write your feature code to enable the usage of the mocking library to test it. That is giving a lot of power to your mocking library. It is a substantial effort to onboard your team to the unit testing framework. It is a substantial effort to read the unit tests and review them. At a lot of shops, it's an unspoken fact that the tests are more complex than the feature code. That the tests take longer to write than the feature code. And that frankly no one really cares if you just skip a whole class of tests because you couldnt figure Mockito out. The test suite ends up being huge. The code inside the test suite only makes sense to a small subset of the developers and the large majority of the test suite never actually produces value. We've all seen the teams that ignore their failing test suite. We've all seen the team that spends a sprint trying to speed up their test suite. I cant prove it, I dont have data, but in my opinion, these giant, unwieldy test suites are largely a by product of the lack of strong typing in javascript & python. The thing that I hold near to my heart on this topic is that no system is better than having an architectural leader on your team who can answer questions and guide the team. Our task is to create higher level languages that allow a single leader to review more code, because each code review is smaller. By creating "bolted on" systems like Unit Tests that produce physically more code, we are wasting the architects time. Over time, I believe we will move towards code that is trivially verifiable on sight, and we will move away from code that is complex and hard to manually verify.
nightowl_games 6y ago
Adding this John Blow youtube video where I feel he illustrates a good aspect of type systems. https://www.youtube.com/watch?v=2J-HIh3kXCQ https://www.youtube.com/watch?v=2J-HIh3kXCQ

packetlost 6y ago

There's more than 1 class of problems that unit tests solve though. I'd still want pretty good testing over my business logic heavy code. I'd also like to point out that Python's typing module + mypy is, while still not perfect, quite good as a tacked-on type system. Also I believe unit tests as a concept first gained popularity in the Java ecosystem, not Python, which kind of negates the assertion that they are popularized because of Python's typing shortcomings.

orwin 6y ago
No, it was from the smalltalk guys, who later designed Eclisped and came to Java. But it was from smalltalk and never caught up until PHP, Ruby and python came strong.

arunix 6y ago

I remember reading Kent Beck's TDD book many years ago. Most of the example code is in Java. There's one chapter that uses Python, but that's just to show how a TDD framework could be implemented in a language which (at that time) didn't have one.

pmcollins 6y ago

i feel like uncle bob’s opinion on static types arose when he became an aficionado of clojure, which doesn’t have them.

Jach 6y ago
You can actually find his opinion forming even earlier, e.g. here's something from 2003: https://www.artima.com/weblogs/viewpost.jsp?thread=4639 https://www.artima.com/weblogs/viewpost.jsp?thread=4639 His opinions do change though, if slowly. Perhaps in 10 years he'll try Common Lisp.
- nightowl_games 6y ago
  He mentions Python as a correlated factor in his love of unit tests. He also reveals how 'trendy' his mindset and strong opinions are.

jakelazaroff 6y ago

This is basically the other extreme. Why must we pick a side? Static types are useful. Unit tests are useful. One of my projects involves a bunch of graphics code. A lot of tricky math and special cases. When I was trying to refactor that code, I added a bunch of unit tests to make sure images would continue to be rendered correctly. And they prevented me from shipping a bunch of rendering bugs! Bugs that the type system would have let slip through, because the function signatures didn't change.

nightowl_games 6y ago
> This is basically the other extreme. Why must we pick a side? Static types are useful. Unit tests are useful. I did not say that Unit Tests are not useful. I am merely proposing that they are less useful than Static Typing and hypothesizing on why TDD is common, and commenting on my experience with TDD evangelists. I agree that Unit Tests are useful and valuable.
commandlinefan 6y ago
> Static types are useful. Unit tests are useful. And unit tests with static types are even more useful still!

D13Fd 6y ago

Static types don't tell you if you broke something when you changed the code.

nightowl_games 6y ago
That's a very broad statement. There are classes of bugs that can be detected by the compiler, and there are classes of bugs that cannot. Both of these statements are also true for Unit Tests.
mhh__ 6y ago
That depends on the type system, although in practice encoding this information isn't easy I readily admit.
rovolo 6y ago
They tell you if you broke a class of things, and I think that class of issues is pretty broad: * when you remove a method from a class, no user of that class calls that method * when you change the return type of a method, all users of that method are updated to handling the new type * in type systems with nullability, they make sure all users handle null values, and they make sure methods with non-null return types never return a null value There are functional changes which can break your code which won't be caught by the type system, but there are a lot of details which the type system will make sure you check when you make changes.
gherkinnn 6y ago
Not entirely. Types let me know whenever I try to connect a water pipe to the electricity. Or when I forget to wire up the grounding. What it can't tell me is that the lights go on when I flip the switch.
orwin 6y ago
-Warning -Werror -Wall -Wextra will tell you plenty. And you can add -Wconversion if you want to be clear on int conversion, and maybe -WShadow if you want to be sure you code is easily understandable. I did TDD five time. Twice with python, one with Go, onw with Rust and once with Common lisp. It was usefull with python, not great with Go and especially Common lisp (i mean, debugging while the code is held is better than debugging with tests, and Common lisp is a really good language to prototype, and TDD takes that out). And Miserable with Rust. With rust i'll test functionality and unsafe code, but i won't ever do TDD again, even if you paid me FU money.

novembermike 6y ago

TDD is more of a Java thing and it has a lot to do with the lack of a good type system in Java. TDD essentially lets you create a class that encapsulates a type and then enforce the contract for that type through the tests. Modern type systems tend to just let you do it directly so the unit tests are a little less useful.

nightowl_games 6y ago

Another factor is the complexity of the tests. If you really want to cover as much of the cases as possible in your tests, you end up all this bespoke mocking, spying, argument capturing and asserting stuff. Your test suite tends to be completely designed around some third party mocking library. True TDD evangelists will tell you that you will learn to write your feature code to enable the usage of the mocking library to test it. That is giving a lot of power to your mocking library. It is a substantial effort to onboard your team to the unit testing framework. It is a substantial effort to read the unit tests and review them. At a lot of shops, it's an unspoken fact that the tests are more complex than the feature code. That the tests take longer to write than the feature code. And that frankly no one really cares if you just skip a whole class of tests because you couldnt figure Mockito out. The test suite ends up being huge. The code inside the test suite only makes sense to a small subset of the developers and the large majority of the test suite never actually produces value. We've all seen the teams that ignore their failing test suite. We've all seen the team that spends a sprint trying to speed up their test suite. I cant prove it, I dont have data, but in my opinion, these giant, unwieldy test suites are largely a by product of the lack of strong typing in javascript & python. The thing that I hold near to my heart on this topic is that no system is better than having an architectural leader on your team who can answer questions and guide the team. Our task is to create higher level languages that allow a single leader to review more code, because each code review is smaller. By creating "bolted on" systems like Unit Tests that produce physically more code, we are wasting the architects time. Over time, I believe we will move towards code that is trivially verifiable on sight, and we will move away from code that is complex and hard to manually verify.

nightowl_games 6y ago

Adding this John Blow youtube video where I feel he illustrates a good aspect of type systems. https://www.youtube.com/watch?v=2J-HIh3kXCQ https://www.youtube.com/watch?v=2J-HIh3kXCQ