6 ms·
I generally try to keep my HN comments positive, but this is total Bullshit, yes, with a capital 'B'. Unit tests are not albotrosses around the neck of your co
by bluejekyll 10y ago
I generally try to keep my HN comments positive, but this is total Bullshit, yes, with a capital 'B'.
Unit tests are not albotrosses around the neck of your code, they are proof that the work that you just did is correct and you can move on. After that they become proof that any refactor of your code was correct, or if the test fails and doesn't make sense, that the expectations of your test were incorrect. When you go to connect things up after that, and they don't work, you can look at the tests to verify that at least the units of code are working properly.
I am no TDD fan, but I do believe that writing your code in a why that makes it easy to test generally also improves the API and design of the entire system. If it's unit testable, then it has decent separation of concerns, if not, then there may be something wrong (and yes this applies to all situations). I use this methodology for client/server interactions as well where I can run the client code in one thread and the server in another, with no sockets, to simulate their functioning together (thus abstracting out an entire area of potential fault that can be tested in isolation from network issues).
The article/paper raises good points about making sure that the tests are not just being written for the sake of code-coverage, but to say they are useless is just sloppy. Utilize the testing pyramid [1], if you adhere properly to that, everything about your system will be better.
I have a serious question, given that this was written by a consultant, is it possible that tests get in the way of completing a project in a timely manner, thus causing a conflict of interest in terms of testing?
[1] - http://martinfowler.com/bliki/TestPyramid.html http://martinfowler.com/bliki/TestPyramid.html
- je42 10y agoYes, that.
- partycoder 10y agoAnd they are also "executable documentation". If you as a developer want to know how to use something, unit tests can be of help. Then, writing tests for the sake of code coverage, is like judging take a driving license exam and being judged on the streets you visited or how much gas you used. Tests need to be written with logic verification in mind. I am deeply sad this article got upvoted. People should have a sense of responsibility when giving visibility to stuff like this.
- skybrian 10y agoHave you seen people reading tests much in practice? I've heard the "executable documentation" line but my general impression is that people usually read the production code. They'll only look at the tests when they fail or they need to modify them.
- codazzo 10y agoThat's assuming the code even makes sense. Have you ever been looking at some code, having no precise idea what it does, with no commit history to give an explanation? That's when unit tests would be useful documentation.
- wallacoloo 10y agoSorry if this is what you were specifying when saying "executable documentation", but have you seen the way Rust handles documentation tests [1]? The idea is to provide documentation via comments before a function definition, and inside this documentation you can provide example uses. The test-runner automatically extracts these examples and verifies them. In this way, your documentation and tests are always in agreement with each other, and a developer can be aware of expected behavior by reading either the code comments or the auto-generated html docs. I'm rather fond of this system, personally. 1: https://doc.rust-lang.org/book/testing.html#documentation-tests https://doc.rust-lang.org/book/testing.html#documentation-te...
- agumonkey 10y agoThanks, you got me testing rustlang testing facilities. Very enjoyable. Makes you want to do low level programming 10x more than with c/cpp.
- mdpopescu 10y agoIf I start working on a new project and they have tests, I always start there. Heck, if I resume working on an old project of mine, I start with the tests.
- audunw 10y ago
- bbcbasic 10y ago> but to say they are useless is just sloppy or link bait
- abritinthebay 10y ago> they are proof that the work that you just did is correct and you can move on Well... they should be. The number of Tests I've seen that are functionally useless is far too high. Because what you test is much more important. I know I'm not imparting new wisdom here but if your test can't survive a refactor it's probably a) far too fragile b) poorly written and c) testing the wrong thing. I actually would not be surprised if a large percentage of Unit Tests are useless, other than coverage stats, but I agree with you that this article is very full of hyperbole to the point of ruining any point it might have had.
- bluejekyll 10y agoYes, I totally agree with this. I've worked in large orgs, where there were QA engineers that needed work, would come and ask, what tests they could write. My response was always, "well, I think I handled all the positive use cases, so you can add negative test cases", which I've never really seen a huge point in (though that depends on the interface contract).
- vigilant 10y agoSystem level integration tests are also proof that the work we did is correct and we can move on. And it is better proof than unit tests, because we aren't mocking everything. And they are far more likely to survive a large and risky refactor than unit tests. Don't confusing testing in general with unit testing. Just use the right tool for the job. If your unit tests aren't catching a material number of bugs compared the the effort spent, compared to other testing methods, then don't do them. Unit tests have benefits such as quicker execution time, etc. - but that has to be weighed against cost.
- srj 10y agoSystem level integration tests also tend to be more flaky. Both unit and integration tests are useful. Unit tests are also pretty quick to write once you have the mocking setup.
- kitsune_ 10y agoThat's one of the dirty little secrets with end to end tests that almost no one talks about. You will probably spend more time running after ghosts in the machine than finding actual bugs.
- Arnt 10y agoI've had that experience too. But also the opposite. I've worked both on code where writing the tests was more effort than the code, and on code where writing the tests was easy, quick and helpful. The latter makes sense, after all a good test is straightline code, zero ifs, zero loops. But the former? I think the key is that mocking should be used sparingly, but without hesitation.
- crdoconnor 10y agoPlenty of people talk about it: http://googletesting.blogspot.co.uk/2015/04/just-say-no-to-more-end-to-end-tests.html http://googletesting.blogspot.co.uk/2015/04/just-say-no-to-m... The dirty little secret that nobody talks about is the cause of the ghosts: poorly engineered tests.
- jghn 10y agoTests are not proofs, types provide proofs. Tests demonstrate that the exact conditions being tested actually work. That's not a bad thing but when working with static typing and robust type systems the need for unit testing goes down tremendously compared to dynamic languages.
- bluejekyll 10y agoWhile I agree with you to an extent, I'm a huge Rust fan, longtime C/C++ and Java engineer, I disagree that a strongly typed language allows you to not write unit tests. Unit tests verify logic, the compilers only guarantee syntax. Which yes, handles a huge set of issues that in a dynamic language would require 100% test coverage, you still need to check the logic of the code. Thus, unit tests are still a huge benefit. One reason I love Rust is the build in #[test] feature. Builtin tests from the ground up, awesome.
- jghn 10y agoI didn't say not write tests but that it reduces the volume of tests needed for largely the reasons you stated
- rubber_duck 10y agoTypes provide semantic information as well so checking verifies semantics as well (extent depends on the strength of the type system). Saying you're a Rust fan and then saying compilers only guarantee syntax when the entire point of that language is to have a constrained subset in which types guarantee no data races/leaks at compile time doesn't make sense to me.
- bluejekyll 10y agoI didn't mean to imply that rust doesn't provide more than just syntax. You are correct that rust provides a huge number of guarantees around semantics, but honestly, I think it's not a traditional compiler, it provides a ton of features that you don't find in C compilers for example. Rust is the greatest thing since sliced bread, my comment was only about statically typed and compiled languages in general. I think you read more into my comment than I intended.
- justinlardinois 10y ago> given that this was written by a consultant, is it possible that tests get in the way of completing a project in a timely manner, thus causing a conflict of interest in terms of testing? That's not really what conflict of interest means, but it's certainly possible that his being in that position gives him a different set of priorities than, say, a project manager, and he's working backwards to come up with an argument that justifies his opinion.
- bluejekyll 10y agoPerhaps it would have been better to phrase that as 'competing priorities', but you got the point.
- tracker1 10y agoActually, unit testing only ensures two things. 1. Tha you've at least looked at and/or considered every piece of code at least twice. Two, it adds friction when writing complex code that isn't modular or a functional composition. It is not a guarantee of anything beyond that. However, having something close to automated deployment means that you should have very close to 100% coverage if only to ensure the above two things. One thing I've rallied against at several points, more so in scripted environments is DI/IoC, it's often not needed, and there are often simpler solutions than what is generally used. The benefits are being able to have multiple targets for an interface, and being able to more cleanly unit test some systems in a strongly typed compiled language. All of that said, I don't always go for thorough testing, but I try to write code in such a way that it's more modular, and would be easier to test, should the need arise.
- will_hughes 10y ago> The benefits are being able to have multiple targets for an interface, and being able to more cleanly unit test some systems in a strongly typed compiled language. The benefit for DI/IOC for me is mainly not having to worry about how to compose the system. If some new module I'm writing needs a particular dependency, then I just ask for that in my constructor. I neither need to know nor care how to set it up or initialise it. That's taken care of by the container - either automatically (convention based registration) or by the maintainer of FooService. This makes it dead easy to create smaller integration/test harnesses: run the normal container build step, and then resolve an instance of my class, out it pops with all the dependencies handled.
- skybrian 10y agoIt gets voted up because there's a grain of truth. Some (usually inexperienced) programmers really do write a lot of mock-heavy, boilerplate tests that don't verify any behavior of importance. Maybe you haven't seen it. But keep in mind that consultants get called in when something has gone wrong. They tend not to see clean codebases by teams that are working well.
- 49531 10y agoI've also found unit testing to be a fairly good way of documenting code. A unit test will often times shine light on an angle traditional documentation doesn't always cover.
- olalonde 10y ago> I have a serious question, given that this was written by a consultant, is it possible that tests get in the way of completing a project in a timely manner, thus causing a conflict of interest in terms of testing? Doubt so, he works for a software testing consultancy. They've been around since 1994 apparently. http://rbcs-us.com/ http://rbcs-us.com/
- wyager 10y ago>they are proof that the work that you just did is correct Nope. They may be evidence, but they aren't proof. Proof comes from formal systems like type systems.
- Someone 10y ago"Unit tests are [...] proof that the work that you just did is correct and you can move on." Unit tests only can proof your software to be buggy. Even for the extremely simple "two, a function that, given no arguments, returns the number 2", your unit test can't verify that to be _always_ true. It may fail to return on Tuesdays (https://bugs.launchpad.net/ubuntu/+source/file/+bug/248619 https://bugs.launchpad.net/ubuntu/+source/file/+bug/248619), internally use a web service that doesn't work on leap days (http://www.wired.com/insights/2012/02/leap-day-azure-outage/ http://www.wired.com/insights/2012/02/leap-day-azure-outage/), other calls may overwrite the 'constant' it returns (http://programmers.stackexchange.com/questions/254799/ever-change-the-value-of-4-how-did-this-come-into-hayes-thomas-quiz http://programmers.stackexchange.com/questions/254799/ever-c...), etc.
- mdpopescu 10y agoYou can verify it to always be true as long as the dependencies (which you've mocked) hold. If they don't... if that's a possibility then you might need some error recovery somewhere, not necessarily in this particular unit.
- sangnoir 10y ago> Even for the extremely simple "two, a function that, given no arguments, returns the number 2", your unit test can't verify that to be _always_ true That is true, I don't think anyone argues against that point. What unit tests do, is verify that under the code works under specific conditions (as defined in the test's set up). When unit tests fail under those specific conditions, then you know there is a problem. Another thing I like about unit tests is that it is much easier (and faster) to test different combinations of conditions
- majewsky 10y agoIn functional programming languages, the type of "two" would indicate whether it has side effects. If not, then "two() == 2" is conclusive proof.
- powera 10y agoYeah, anyone that invokes Turing machines to claim unittests don't work has completely missed the point of testing their code. Perhaps if the article was talking about "formal verifications of code" it would bear some resemblance to good advice.
- yarou 10y ago>Unit tests are not albotrosses around the neck of your code, they are proof that the work that you just did is correct and you can move on. IMHO, this is a failing of the type system. If you have functional purity, you don't have a need for unit tests, as they are blended into the code you write. Functional purity gives you the power to formally verify the correctness of your program.
- kartan 10y ago> Unit tests are not albatrosses around the neck of your code All code is a liability. The less code usually the better. KISS principle. > writing your code in a why that makes it easy to test generally also improves the API and design of the entire system You can get the same result applying SOLID principles, without the cost of maintaining extra code. The presence of unit testing doesn't guarantees easy to test code either (and this is the worse case scenenario, bad designed code with a lot of test coverage that is broken in each refactor). > is it possible that tests get in the way of completing a project in a timely manner? Are you dismissing this as a point? Time is a resource and writing unit tests adds to that cost. Unit testing needs to: * Make you faster because there are less bugs in the long run. and/or * Prevent money loss caused by bugs compared to the money lost caused by the opportunity cost of writing unit tests instead of new features. Also when writing unit tests developers are less prone to refactoring when requirements change as it will require to refactor also all the unit test affected. You can agree or disagree with the points int he article, but it is NOT just "bullshit".
- jaked89 10y agoIt's not a proof that your work is correct. It's a proof that a few cases, out of potentially millions, or billions, work.
- twa927 10y ago> If it's unit testable, then it has decent separation of concerns But then the separation of concepts is driven by the testability, not the problem per se. It means that there's possibly a lot of complexity just for supporting testing.
- hvidgaard 10y agoI'm not a strict TDD follower, yet a byproduct of my development process leaves me with unittests. They give me confidence that the code I just wrote behaves the way I want it, and deleting them after that just ensure that refactoring isn't on the table without a significant time investment. The drawback of this, is that the test suit grow fast, and need to me groomed pretty often, but it's a fair price to pay and in the end it's faster for me, and produce far less bugs.
- blub 10y agoI don't belive that you read the document. You are arguing against points which have not been made in the article or which are more nuanced than one would believe by reading posts such as yours. The article is very reasonable, logical and a good counterpoint to the countless TDD anecdotes which obsessively focus on unit-tests to the detriment of any other types of testing or design practices.
- LnxPrgr3 10y agoI don't ever unconditionally trust code, but my faith rises: * a little, if it passes a decent unit and/or integration test suite; * a fair bit more, if it survives a round with humans who didn't write it but did set out to break it; * quite a bit more, once it's been in production for a while without complaints tracing back to it; * gradually approaching, but never reaching, perfect trust the longer it survives exposure to the world. A lot of bugs will make it past step 1--even through tests with 100% code coverage. Hopefully no one takes them as anything near proof of correctness.