5 ms·

> The point of the SO post is that, given a limited development time, they chose to focus on performance and sacrifice testability. I used to think that this

> The point of the SO post is that, given a limited development time, they chose to focus on performance and sacrifice testability. I used to think that this was necessarily a choice for a long time, in fact I only very recently started writing almost TDD (not completely, there's things that I will still write code first, but no longer do I leave it untested, and I also don't see a problem in writing contained parts of code and add the tests after when finishing that "contained part"), but this isn't necessarily a choice. Tests help in many ways, and I learned this in doing any project, large or small for any amount of time larger than 1 month (or maybe 2 weeks). And sometimes it's not even the correctness of the program that is the most important, in the sense that you can write correct code for the most part correct and legible without tests. What I've found it really helps is when you have to tear down a prior specified behaviour/request/spec, or update it, or integrate it with other things. Tests at least tell you that "all these assumptions you made and wrote in the past 3 months while writing this code still hold true", and this is important. Of course they don't cover all things and I don't think you should test to infinity but, the thing is, once they're written and not overly brittle (UI's for instance are complex to test correctly because it's easy to make them overly brittle if there's any churn going in there) and you find new bugs or over-sighted assumptions you can add them to your test suite, so they keep accumulating value. The other thing I noticed is very helpful is when you realise that you can't easily write tests for something. This is usually a sign that your code isn't as organised/designed as it should be, has coupling and/or other accidental complexity that is not required (there's exceptions where the problem domain and flows required are complex enough to make testing complex also ofc). The time dimension and lee-way when writing code were before my main gripes with testing regularly but sincerely I think that my code with tests is much better the more I practice it and I can't see much/any slow-down from before in implementing things, and certainly the opposite after any significant time has passed or the code base has grown large enough to cover different requirements.

josephg 6y ago
Something people rarely mention with tests is that their value has a Pareto distribution. Most of the value of a test suite will be captured by a small number of tests. (And inversely, most of your tests will never discover any bugs). The upshot of this is that you will get disproportionate value from the first few tests you add to a project. I think its very rare to have a long lived project in which the first 10 tests aren't worth writing. The next 100 tests? That's a much harder question.
- hnedeotes 6y ago
  Actually I agree with you, and also forgot to mention that tests themselves because of being code should be cleaned up just like with regular code and tests simplified whenever possible. I think the important question is what to be tested. I have written, even not that long ago, unit tests for a function that all passed in isolation and actually forgot to write a simple test with the context it would be called (which was very simple as well, but I just didn't), all isolated tests were good but the actual functionality was still broken. I haven't been doing it extensively for a long time but for instance, if you have an user action that should update a record, write a job to be executed, then this should be tested. It's easy and does very small assumptions. You don't need to test how the record is updated or the job inserted, just that it happens. You might want to test that all failures provide correct error messages, but that is not as important as at least having the baseline of, if this action that is central happens both these things happen as well.
tsimionescu 6y ago
> What I've found it really helps is when you have to tear down a prior specified behaviour/request/spec, or update it, or integrate it with other things. Honestly, in my experience, the opposite is true of unit tests. I've never been able to make a significant change to a piece of code that was thoroughly unit tested without having to more or less re-write the unit tests from scratch. However, integration tests and whole-project automation do help tremendously in what you are describing.
- hnedeotes 6y ago
  I'm not completely convinced of exhaustive unit testing either as an end in itself. I also haven't been writing tests regularly for that long but I did extensively test quite a large surface with a game I'm writing for a long time in the past months but there it's also not exactly unit tests what I did and more verifying that actions produced the expected results and, where it was easy, also making sure some obvious side-effects were not part of the result. Besides that I also haven't been thrown into codebases where I had to refactor in anger someone else's code or tests, I imagine it can get very frustrating as well when overdone or done badly (and not saying I wouldn't do it badly either - I think it can be easy to overdo unit testing without any relevant gains). I guess what I'm leaning towards in my own practice though is thinking of unit tests as trash-able. That is, if you want to, using them to drive the design and verify along the way the small parts that comprise whatever end API (in the library sense not REST, even if it's just part of your "code" and not a library in itself) you're designing it's ok, if that makes sense to your development style, but once you have the API in place the important thing is that the API itself is kept under test. At that point if the unit tests become a burden they should be trash-able, as long as the API - I imagine what you mean by integration? - is kept tested. I don't know if this flies in practice if you're working in an environment that requires unit testing for everything and what not or if it's actually better to loose the time and re-write them - annoying as it might be, they usually indicate that something is broken if they're failing which can be better than being oblivious. All of this though must also be regarded in terms of what is the "unit" we're referring here to and I also think different languages and ways of organising programs can change a lot the way one sees units, apis, etc. Some languages you have more of a domain like organisation where you set up APIs to "reach" to the underlying stuff, that probably means a unit in those languages. Others that are inching more towards a "lower level" generate what looks like much smaller unit definitions - which would probably be very annoying to test extensively - sometimes looks like people talk about unit testing and one is using ft and the other m without having defined the unit of measure.

josephg 6y ago

Something people rarely mention with tests is that their value has a Pareto distribution. Most of the value of a test suite will be captured by a small number of tests. (And inversely, most of your tests will never discover any bugs). The upshot of this is that you will get disproportionate value from the first few tests you add to a project. I think its very rare to have a long lived project in which the first 10 tests aren't worth writing. The next 100 tests? That's a much harder question.

hnedeotes 6y ago
Actually I agree with you, and also forgot to mention that tests themselves because of being code should be cleaned up just like with regular code and tests simplified whenever possible. I think the important question is what to be tested. I have written, even not that long ago, unit tests for a function that all passed in isolation and actually forgot to write a simple test with the context it would be called (which was very simple as well, but I just didn't), all isolated tests were good but the actual functionality was still broken. I haven't been doing it extensively for a long time but for instance, if you have an user action that should update a record, write a job to be executed, then this should be tested. It's easy and does very small assumptions. You don't need to test how the record is updated or the job inserted, just that it happens. You might want to test that all failures provide correct error messages, but that is not as important as at least having the baseline of, if this action that is central happens both these things happen as well.

tsimionescu 6y ago

> What I've found it really helps is when you have to tear down a prior specified behaviour/request/spec, or update it, or integrate it with other things. Honestly, in my experience, the opposite is true of unit tests. I've never been able to make a significant change to a piece of code that was thoroughly unit tested without having to more or less re-write the unit tests from scratch. However, integration tests and whole-project automation do help tremendously in what you are describing.

hnedeotes 6y ago
I'm not completely convinced of exhaustive unit testing either as an end in itself. I also haven't been writing tests regularly for that long but I did extensively test quite a large surface with a game I'm writing for a long time in the past months but there it's also not exactly unit tests what I did and more verifying that actions produced the expected results and, where it was easy, also making sure some obvious side-effects were not part of the result. Besides that I also haven't been thrown into codebases where I had to refactor in anger someone else's code or tests, I imagine it can get very frustrating as well when overdone or done badly (and not saying I wouldn't do it badly either - I think it can be easy to overdo unit testing without any relevant gains). I guess what I'm leaning towards in my own practice though is thinking of unit tests as trash-able. That is, if you want to, using them to drive the design and verify along the way the small parts that comprise whatever end API (in the library sense not REST, even if it's just part of your "code" and not a library in itself) you're designing it's ok, if that makes sense to your development style, but once you have the API in place the important thing is that the API itself is kept under test. At that point if the unit tests become a burden they should be trash-able, as long as the API - I imagine what you mean by integration? - is kept tested. I don't know if this flies in practice if you're working in an environment that requires unit testing for everything and what not or if it's actually better to loose the time and re-write them - annoying as it might be, they usually indicate that something is broken if they're failing which can be better than being oblivious. All of this though must also be regarded in terms of what is the "unit" we're referring here to and I also think different languages and ways of organising programs can change a lot the way one sees units, apis, etc. Some languages you have more of a domain like organisation where you set up APIs to "reach" to the underlying stuff, that probably means a unit in those languages. Others that are inching more towards a "lower level" generate what looks like much smaller unit definitions - which would probably be very annoying to test extensively - sometimes looks like people talk about unit testing and one is using ft and the other m without having defined the unit of measure.