11 ms·

Ok let's stop writing UT and see what happens. Wait... we've already tried that, and we know the result pretty well: 1. In dynamic languages, simple type misma

• misja111 10y ago
  There is a difference between unit tests and integration/ system/ component tests. I think the author is talking about pure unit tests, i.e. the ones that test only one specific piece of code and mock everything else away. Too fine grained unit tests actually make refactoring more difficult, because a refactor typically touches so much code that they have to be rewritten. But component tests do provide the safety net that you had in mind when you wrote your reaction.
• zmmmmm 10y ago
  > 3. No one dares to refactor anything ==> Code rottens ==> maintenance hell. In my experience, with enough poorly written unit tests and a hard line policy on unit test coverage (which often results in poorly written tests), the unit tests often turn out to be as much a barrier to code maintenance as the regular code is. I'm an advocate of judicious unit testing, but taking it too far is worse than not doing it at all if you have integration or system / functional tests IMHO.
  • dzhiurgis 10y ago
    Sounds like my life these days. Been given some smelly code (2k loc) that I cannot fully comprehend, so I just went out and naively refactored it. Code was covered without assertions, so I insisted spending some time into writing at least a few while everyone else just asked me to test. And of course once in production tons of bugs started to appear. I still believe I can write system tests though.
  • brianwawok 10y ago
    This is the real trick. Unit tests encourage you to maintain abstractions and layers. If your abstractions are good it works well. But if you need to change abstractions it now becomes a much harder problem. One nice part of more system level tests than unit tests. Hi ahead ans refactor a bunch. If the same inputs still give the same outputs you should be golden still.
    • jacquesm 10y ago
      > If the same inputs still give the same outputs you should be golden still. That's dangerously subtly wrong. It only holds if your inputs include both positive and negative cases. If you only push positive cases through your code and it still gives you the same outputs then it might be broken in subtle but non-obvious ways, aka happy-path testing. Better be very careful with this.
  • brightball 10y ago
    You can refactor anything if you are conscious of backwards compatibility.
• ploxiln 10y ago
  I think there's some arguing past each other in discussion of this. You've seen some terrible programmers who don't write unit tests, and I (and others) have seen terrible programmers that do write (terrible) unit tests. Tests that mock everything and test nothing. Tests that assert bad implementation details but not important results. Huge volumes of terrible tests that are a huge burden. I agree, having some good tests is very good. Obviously. But I disagree with the hypothetical statement "someone writing a lot of tests is always doing a better thing than someone writing a lot fewer tests". No silver bullet and all that. Also annoyed by "agile methodologies" and "best practices" in general. Good sense and taste really can't be turned into a list of rules you can blindly apply anywhere. I suggest reading http://alistair.cockburn.us/Characterizing+people+as+non-linear,+first-order+components+in+software+development/v/slim http://alistair.cockburn.us/Characterizing+people+as+non-lin...
  • adrianratnapala 10y ago
    You are correct that writing lots of tests doesn't imply good programming. But lets consider the logical converse: does good programming imply lots of testing (not necessarily unit tests)? I say yes. Of course this is a big, complicated, world; it probably contains good programs without tests. But in my usual experience, the moment a programmer - even a good one - neglects testing, bad stuff starts creeping into the code.
  • deleted 10y ago
    [deleted]
• im_down_w_otp 10y ago
  Unit tests are mostly useless. If one wants their tests to be anything more than a happy path and sad path that the developer thought up (combined with an accretion of sad paths that were found later as bugs), then one would do themselves a tremendous favor by adopting property-based testing and mutation testing. Doing so forces you to define several constraint-based models for your program, and then layering mutation testing over the top helps you figure out how good your tests actually are.
  • deleted 10y ago
    [deleted]
  • CraigJPerry 10y ago
    >> Unit tests are mostly useless The speed difference though, integration tests just take too long to be able to completely replace unit tests.
    • brianwawok 10y ago
      It depends. Both on project size and how your tests parallelize. If you can integration test an entire project in 30 seconds, that is fast enough. Fast enough to have it in a loop for by the book TDD? No but I'm not sure by the book TDD adds any value. You mash test once before committing and if all is good you go ahead.
      • sitkack 10y ago
        With a decent hierarchical decomposition, integration testing at component/module/subsystem boundary achieves both good coverage and low latency test results. I worked on a big ETL database migration tool in Python and it had a top level integration test with the smallest possible subset of data, like 500k end to end. Then each subsystem (it was architected like a nanopass compiler) had a more complex set of integration tests that only checked that pass but more rigorously. The whole project (10k lines of code) had under 10 unit tests that double checked tricky low level primitives. Every test was independent and could be run in parallel, my longest test was just under a minute, my shortest was about 1 sec. The shortest tests were run on module import ensuring both an easy affordance to testing and a tight feedback loop with no unit tests to impede refactoring.
• mdpopescu 10y ago
  Speaking of (3): I have a bunch of newbies I'm mentoring. Every time I explain about unit tests, I show them a class I wrote that had 37 lines altogether. After I finished working on that project, a new team took over. They didn't write unit tests (or maintain the existing ones). A month later, that class had a method with 1500 lines. It's been around a year since development has basically frozen on that project.
  • mattmanser 10y ago
    What has the number of lines got to do with the unit tests? That functionality had to go somewhere, no matter how badly written it was. Unit tests don't magically save you from code bloat. Also I work on 3 code bases written by other people with zero unit tests, I refactor them all the time. Other developers have refactored them. Not having unit test is not a big barrier to refactoring.
    • zephyrfalcon 10y ago
      > Unit tests don't magically save you from code bloat. No, but the idea is that such a method would be hard to test, considering it (presumably) does so much. So in order to test it properly, you are encouraged to break it up into smaller methods. (Whether somebody actually does that is a different story, of course...)
  • Const-me 10y ago
    Are you saying the requirement to unit tests is the main reason why those developers don’t write 1500-lines methods?
    • mdpopescu 10y ago
      I'm saying that they would be prevented from writing those things if they also had to test them. Since they're not required to test their methods, they're free to write code that's pretty much impossible to reason about.
      • 10y ago
• gaius 10y ago
  In OCaml (or any similar language) 95% of what you "unit test" for in Javascript just goes away. If people cared about software quality, they would use strongly typed languages before even thinking about "unit tests". What people actually want is a) to hack it out quickly and b) to have a lot of busywork to do as a form of job security. Hence 50,000 lines of "unit tests" per 10,000 lines of application...
  • mdpopescu 10y ago
    I have never met anyone who doesn't care about code quality yet writes unit tests...
    • TickleSteve 10y ago
      then you have never worked in anything but a small focussed team. The majority of development is performed by people in corporate environments, 9->5. They simply know that they "have to" write unit tests. No reasoning is performed, simply churn out the code to satisfy the boss and walk out at 5pm.
      • mdpopescu 10y ago
        Sorry, you are correct; what I wrote was simply stupid. I have, in fact, encountered one method with a cyclomatic complexity of 20+, which had a single test (it verified that the result was not None). I just tried to put that code behind me and yes, those "tests" were there because they were mandatory. Blech.
  • sametmax 10y ago
    Here is a list of basic errors in Python. Only builtin ones, I'm not talking about ones from other libs. And I'm talking of error categories, as we all know one exception can be used in a lot of different contexts: BaseException +-- SystemExit +-- KeyboardInterrupt +-- GeneratorExit +-- Exception +-- StopIteration +-- StopAsyncIteration +-- ArithmeticError | +-- FloatingPointError | +-- OverflowError | +-- ZeroDivisionError +-- AssertionError +-- AttributeError +-- BufferError +-- EOFError +-- ImportError +-- LookupError | +-- IndexError | +-- KeyError +-- MemoryError +-- NameError | +-- UnboundLocalError +-- OSError | +-- BlockingIOError | +-- ChildProcessError | +-- ConnectionError | | +-- BrokenPipeError | | +-- ConnectionAbortedError | | +-- ConnectionRefusedError | | +-- ConnectionResetError | +-- FileExistsError | +-- FileNotFoundError | +-- InterruptedError | +-- IsADirectoryError | +-- NotADirectoryError | +-- PermissionError | +-- ProcessLookupError | +-- TimeoutError +-- ReferenceError +-- RuntimeError | +-- NotImplementedError | +-- RecursionError +-- SyntaxError | +-- IndentationError | +-- TabError +-- SystemError +-- TypeError +-- ValueError | +-- UnicodeError | +-- UnicodeDecodeError | +-- UnicodeEncodeError | +-- UnicodeTranslateError 52 exceptions. Now, on this list, how many can be linked to language typing ? +-- TypeError +-- ValueError | +-- UnicodeError | +-- UnicodeDecodeError | +-- UnicodeEncodeError | +-- UnicodeTranslateError +-- AttributeError And ALL those errors can also be triggered with something a type system would no catch : runtime class generation, unexpexted user input, corrupted database, wrong data header, etc) So basically, relying only on typing will cover about 10% of the error types, most of which are caught with linters such as flake8 and a code intelligence tools such as Jedi. You are making a very weak case.
• greenimpala 10y ago
  Agree!
• TeMPOraL 10y ago
  What. > Ok let's stop writing UT and see what happens. Wait... we've already tried that, and we know the result pretty well: Stuff still doesn't work and doesn't get delivered on time. That's all we know, and unit tests didn't change that. > Yes these are bugs that should have been caught by a compiler had we had one. WHAT? What kind of dynamic languages are you working with? > 2. There's no documentation as to how something should work, or what functionality a module is trying to express. Unit tests - at least those made in TDD style - tend to obfuscate that; they usually reflect the implementation instead of the interface anyway. > 3. No one dares to refactor anything ==> Code rottens ==> maintenance hell. You seem to know very timid programmers. Or the kind who think tests are their safety net, so they don't bother to actually read the code they're changing and understand what it does. > 5. When something does break in those "top level system tests", no one has a clue where to look, as all the building blocks of our system are now considered equally broken. Look at the crash log. Or think for just a second instead of saying "it's broken" and giving up. > 6. It's scary to reuse existing modules, as no one knows if they work or not, outside the scope of the specific system with which they were previously tested. Hence re-invention, code duplication, and yet another maintenance hell. Re-invention, code duplication, etc. have nothing to do with tests (hell, most unit tests I've seen have quite a lot of code duplication in them). It's not scary to reuse existing modules without tests, it just requires to... use them. Pretty much all the points you listed are examples of people shutting down their brains and hoping someone else's unit tests will think for them. This is a very bad approach to programming. It's the wrong kind of laziness.
  • oblio 10y ago
    > You seem to know very timid programmers. Or the kind who think tests are their safety net, so they don't bother to actually read the code they're changing and understand what it does. Muhahahaha! Sometimes it's almost impossible to understand a piece of code, especially if that code is old and has had comb-overs from a lot of developers. Unit tests at least might give some hints to what's happening.
  • tigershark 10y ago
    Pretty much all your answers show that you never worked in a big enough project and that you don't understand unit tests. When working in a codebase with adequate test coverage your bugs are more likely to be caught. Please, don't tell me that you don't need tests because you don't introduce bugs. I heard this argument multiple times in the last 10 years, and it was always proved untrue. The point number 2 simply proves that you don't know what a proper unit test looks like if you say that a unit test obfuscates the documentation. The point number 3 just confirms what I thought, that you believe that you don't need tests because you don't introduce bugs. The most interesting part is that you think that other people that write unit tests are lazy. Very interesting position, I would say that whoever tries to refactor a big chunk of legacy code without adding an appropriate coverage is certainly lazy and worse overconfident and probably never did anything similar in the past. The point number 5 proves that you never, ever, worked in a big project if you think that it's enough to look at the logs to find a bug. It's common knowledge nowadays that earlier a bug is caught the less effort it takes to fix it. Finally I think that should be clear that the very bad and toxic approach to programming is not the one that involves writing unit tests. This approach tries to formalise the requirements in a set of scenarios, describing each condition that must be satisfied for a given event/input. Your approach seems focused on just writing code, without a proper design phase, without understanding completely the requirements with a close feedback cycle with the users and with a complete disregard of existing functionality that you think you can write better because you are a better programmer given that you don't need unit tests. In my opinion not writing tests is laziness.
• crdoconnor 10y ago
  >Did I fail to mention something? Yep. Integration tests under "top level system test" and above "tightly coupled unit tests" are not only excellent at catching bugs in integration code, they don't require hours of tedious mocking and don't have to be completely rewritten when you refactor the code base. >No one dares to refactor anything Is a problem on heavily unit tested code. Unit tested = tests that are tightly coupled to the code = refactoring turns the tests red whether or not anything was broken = people prefer leave the code alone.
  • gonvaled 10y ago
    Don't agree. Refactoring does not mean that you need to refactor the tests, but that you will be testing other pieces. That is, when you refactor the old pieces are still there (in the libraries), and their tests still work, but you are maybe not using them anymore higher up in the hierarchy.
• ryanbrunner 10y ago
  Unit tests as documentation in my experience is only true in a very minor, almost trivial sense, and for reasons that are very much in line with the subject of the original article. For the most part, unit tests aren't organized around what a unit is supposed to do from a use case perspective, but more about how it happens to be implemented today. The article refers to this with the inability of most developers to indicate which business requirement would fail if the test failed. When you're attempting to understand or refactor something where the testing for the most part tests the implementation of this particular unit rather than it's purpose, the tests themselves offer little in either documentation or refactoring confidence than the implementation itself. What's worse, making your code "more testable" often compounds this effect. As code is taken to an extreme level of modularity, and unit isolation is enforced rigidly in tests, the tests often do nothing other than specifically test every line of the implementation of the unit (i.e. method 'foo' on class 'Bar' calls method 'a' on class 'B' and passes it's results to method 'x' on class 'Y', returning that result)
  • duncanawoods 10y ago
    IMHO "unit tests as documentation" isn't quite the right description. Its more like "guaranteed to work example code and tutorials". One example is where there might be preconditions for calling a function. This type of information is necessary to know but can be hard to work out even with rich documentation. Without documentation, you are going to be reading a lot of code to try and reverse-engineer the internals. With unit tests, you can do a quick "find all references", see how the method is called within different tests cases and be confident that the setup actions in the given test fixtures will work for you.
    • brianwawok 10y ago
      If a method has preconditions, say X cannot be null, Y must be positive, and X > Y, you could test this in a unit test. But honestly make it the top 3 lines of the method with Asserts. Living with the code is going to be generally better than living off in some test. You can read it and see it very easily.
• ahoka 10y ago
  Oh god, you are describing my previous project. Especially 3. and 5., which makes the most tiny bit of changes take months and endless amount of money and manpower.
• jakub_g 10y ago
  There's one insidious thing about no tests when dealing with dynamic languages: when you have no tests, no one knows what data looks like (typical code: `bigJson = getBigJsonFromServer()`). Is 'foobar' an array or string? What kind of string is it exactly? Can it be null, or it will be "unknown"? Put two years of cowboy coding on top and you're screwed. Of course you can create unit tests with fake data, though IMO it's a bad practice. I think it's best to have unit tests working on real production-like data (in addition to have edge-casey-data tested as well).
• pmelendez 10y ago
  >2. There's no documentation as to how something should work, or what functionality a module is trying to express. Unit tests of complex code is often more complex and obfuscated than the code itself, hence its usefulness as documentation is very limited. >3. No one dares to refactor anything ==> Code rottens ==> maintenance hell. >4. Bugs are caught by costly human beings (often used to execute "system level tests") instead of pieces of code. Lack of unit tests doesn't necessarily means lack of automated tests. Functional tests when applicable, are far more useful in my opinion than unit tests.
  • _asummers 10y ago
    > Unit tests of complex code is often more complex and obfuscated than the code itself, hence its usefulness as documentation is very limited. I've found that expressing unit tests in the language of the problem domain and never stressing implementation details wherever possible is the ONLY way to prevent this from happening. Avoid mocking and test doubles where possible, but sometimes they're the right tool. The tests should be treated as production code that gets boyscout rule'd as people go into it for whatever reason.
    • pmelendez 10y ago
      >"Avoid mocking and test doubles where possible, but sometimes they're the right tool. The tests should be treated as production code that gets boyscout rule'd as people go into it for whatever reason." I agree, but a lot of people (at least in my environment) would play the "that's not an unit test" card, which is fine to me, we should care about the least intrusive way to automate code tests regardless of the philosophy behind it.
  • tigershark 10y ago
    Only bad unit tests are obfuscated. In the good ones even the users can easily understand what the test is doing and can even add new test cases.
• mannykannot 10y ago
  4. Bugs are caught by costly human beings (often used to execute "system level tests") instead of pieces of code. The author went to some length to point out (correctly) that test automation is an orthogonal issue. Reading beyond the title, we see the issue is that an obsession with unit tests, and especially an obsession that is focused entirely on a measure as simplistic as code coverage, tends to result in a sub-optimal use of development resources (which are always limited.) A redundant or otherwise unhelpful test (of any sort) does not gain value by being run automatically.