The problem is that sophisticated type systems only catch a subset of the bugs that a unit test can catch. For example, let's say I'm adding the ability to transfer funds from one account to the other in a banking application. I want to display a warning when the amount of money being transferred is over a certain percentage (let's say 95%) of the funds in the account. That's pretty easy to do in a unit test: create mock account, call the transferFunds() method, and verify that the warning is triggered when the value being transferred is over 95% of the amount in the account.
How would I do that with a type system?
I think the point is that if that was written in JS you would have a test to ensure your method behaves correctly with characters passed in, or strings, or just nil(/nul?) Whereas in Java et Al, you don't need any of those tests, you just make your parameter some sort of numeric and that just isn't a problem any more.
At the risk of veering off topic, I'm genuinely curious what kind of bug that unit test is designed to catch?
Given that transferFunds shows a warning over that 95% threshold, what type of change is going to break that logic? Some sort of botched code rewrite where values doesn't retain their meanings? That batchMode shouldn't trigger the warning? That race condition between the check and the actual transfer?
Most of the unit testing I see is more like test-of-defintions (is fullName still 80 characters wide and doesn't accept null bytes?) which confuses me because a definition can only be correct or incorrect in context.
I accept the usefulness in dynamic languages in the absence of types. My Personal preference is for tests to be a bit higher level (does login with a username containing null still cause an exception?) but I have come to terms with the fact that few people agree with me across multiple organizations, so there must be some point to this trivial testing that is completely lost on me.
> Some sort of botched code rewrite where values doesn't retain their meanings?
Yes, exactly. It's supposed to catch the rewrite where somebody turns (a * 100) / b > 95 into (a / b) * 100 > 95 and suddenly, the code doesn't work anymore.
Showing a warning is full scenario - it has a whole UX. If this warning is important enough to have a PM, a UX designer, and be translated into 20 languages, it's important enough for the engineer to make sure it actually shows up when its supposed to.
> My Personal preference is for tests to be a bit higher level (does login with a username containing null still cause an exception?).
These tests are great, but what if username has a bunch of validations on it?
For example, it's reasonable to think a username field might be validated with:
- Must be required
- Within 1-32 characters that match a certain pattern (let's say a regex to limit it to lowercase letters, numbers and -)
- Must not be a blacklisted word (admin, administrator, etc.)
- Must be unique (enforced with a database index)
Pretty standard stuff. Are you going to write 5 integration tests for this? 4 to test each validation and then the success case? These would be tests that exercise your entire web framework's routing stack from request to response (ie. the user visiting a /register URL and then submitting the form).
Personally I would not. I would write 1 unit test for each of those things (4 unhappy cases where I assert a specific validation error for each invalid input and 1 happy case where with valid input I expect 0 validation errors). In this case, the "unit" would likely be a `register_user` function that accepts params as input and either aborts with validation errors, or succeeds by writing the record to the DB.
Then, for an integration test I would have 2 tests. One to make sure with invalid input I end up with some type of error displayed in the HTML response (it doesn't matter which one), and another test with the success case to make sure things work when they should (such as the user is registered and a new record was created in the DB).
So I end up with a tiny bit of overlap in tests. Technically the unit test for the success case doesn't need to be there since the integration test covers it but I usually include it for the sake of completeness because it's usually like 4 lines of code to make that test but I'm not 100% opposed to someone saying it should be left out.
This is very much not the kind of bug that the parent was talking about. He's talking about stuff like passing an int value to a (supposedly) string parameter. Basic type stuff.
I also agree, as years ago I wrote a large amount of javascript which had type assertions pretty much everywhere. They didn't take long to write but I'm very glad I did them, and I'd rather have the language do it to save me time, clutter and maintenance.
Lax typing is a real cost. I've been working on some SQL I inherited and even SQL's not-too-bad typing allowed the original coder to mix types where I wouldn't, and create potential runtime errors eg. to assign the contents of a 64-bit integer field to an 32-bit integer field. That's legal and gives no warning. It's also suddenly a runtime error when it exceeds 2^31 (and there are other possible consequences such as screwing up the optimiser). I'd much rather it forced me to match types exactly, or explicitly cast.
If anyone does have this case it's two distinct functions in my mind the first function does the comparison and returns the warning
You can then assert and type against the warning function, the printing function you could only assert against with output buffering or something like that but in my mind that's less likely to go wrong anyway
Keep your type system simple, custom types beyond in built primitives cause head aches imo
With dependent types: https://en.m.wikipedia.org/wiki/Dependent_type https://en.m.wikipedia.org/wiki/Dependent_type
While I agree that this is hard to implement with a type system, I never want to see this implemented as a unit test.
What you describe is a real use case that I personally want to have tested on a real system with all the stuff in place (but test database instead of the real one). It has nothing to do with one unit you can test on its own.
And the crucial problem here is that you tie your testing code on the inner workings of transferFunds, when you are mocking stuff.
> and verify that the warning is triggered when the value being transferred is over 95% of the amount in the account.
Please also note that you cannot verify this property using testing. You can only verify that this works for one particular set of values (or a big but finite set of values). Testing never can verify that something always works. That's the wrong tool for this job.
With TDD, writing unit tests accompanies writing code - which in itself necessities thinking in terms of testability, which is a good thing making the program's components more separable. Regarding the former example, mocking only a single part where it connects to other parts of the system (for example some interface through which it sends warnings to users) is easy and makes the resulting code better (eg, easy to replace the notification manager).
> With TDD, writing unit tests accompanies writing code - which in itself necessities thinking in terms of testability
I, as a user of your software, only care about the feature, and that the feature works. I don't care about testability, whatever that means. And since I case about features, I think this entails that features have to be tested end to end, and not some unrelated classes ("mocks") that are not even used in the final product.
> is easy and makes the resulting code better (eg, easy to replace the notification manager).
It might or it might not. I have seen too much code which was written way too complex which a lot of unnecessary classes and interfaces, just for the sake of "testability". However, some features did not work every once in a while.
If you have the need for different notification backends, go ahead. Even if you need a notification manager (whatever this is) for this. But don't make code more complicated (sometimes called "test-induced design damage") without need.
To my mind, a unit test should test a unit: something which functions independently, and which is interacted with through an abstraction layer. Type systems are good tools for clarifying and catching bugs around these abstraction layers.
Maybe my understanding is wrong, but what you described doesn't seem like a unit test. I think the appropriate term for a test like this at the junction of UI, business logic and code-level triggers is 'feature test', 'integration test', or maybe even just 'test'.
No mainstream type system can do that so GP would be fine with a unit test here.
You could use Coq, Isabelle, or Lean. Their type systems are powerful enough to allow such checks.
The parent never claimed that a type system means bug free code, or replaces all unit tests. However, it does mean you need less tests.
In a typed, compiled language you get some pretty nice guarantees just by getting your code compiled. In a dynamic language, you have no way to know if your code even runs until you have 100% code coverage.