5 ms·

> I would definitely choose a comprehensive suite of automated unit tests over a comprehensive suite of end-to-end/system tests any day of the week. Huh, after

• jameskilton 12y ago
  No-one ever said that integration tests and unit tests are mutually exclusive. Integration test the happy paths, small integration suite that proves everything works together. Unit test the individual units of functionality or behavior, whichever you're more comfortable with. Unit tests aren't supposed to tell you that the whole system works. Unit tests exist for two main reasons: Confidence that you can refactor your code without breaking functionality, and proving that your components are doing what they're supposed to do. Yes you technically can accomplish this with just an integration test suite but you will find refactoring bugs much harder to track down because the test failures aren't telling you exactly what went wrong. You can never have full confidence in a system with just one suite of tests. You really need multiple suites. And when you TDD it all the way down (Integration test first, then work through the stack TDD-ing each part, when done Integration test and all unit tests should pass, and feature is done!) it's not a chore but actually quite enjoyable.
  • brazzy 12y ago
    The problem is that refactoring definitely can also cause components to not do what they're supposed to do outside the happy path.
    • jameskilton 12y ago
      Then you're not refactoring. Refactoring is defined as changing how the code is implemented without affecting the functionality / behavior of said code. Also, Unit tests are where you ensure the non-happy-paths are functional (error handling, input robustness, etc).
      • Retric 12y ago
        That's a ridiculous definition, that's like suggesting coding is turning requirements into bug free computer code. Refracting is attempting to replace working code with code better suited to your long term goals.
        • jameskilton 12y ago
          I don't see how your definition of refactoring and mine differ.
          • Retric 12y ago
            In your definition if you introduce or remove a bug your not refactoring. More basically for things like simulations or graphics you may trade accuracy for speed so your code may behave slightly differently, but be much faster. The important bit with refactoring is the goal is not fixing a specific bug, however realistically for any sufficiently large change bugs can and will be both added and removed. Edit: "Typically, refactoring applies a series of standardised basic micro-refactorings, each of which is (usually) a tiny change in a computer program's source code that either preserves the behaviour of the software, or at least does not modify its conformance to functional requirements." http://en.wikipedia.org/wiki/Code_refactoring http://en.wikipedia.org/wiki/Code_refactoring
  • dreamfactory2 12y ago
    > TDD it all the way down (Integration test first, then work through the stack TDD-ing each part, when done Integration test and all unit tests should pass, and feature is done!) Interesting. I think this is probably right and also the reverse of what most people try to do (they start with units and work up)
    • ryanjshaw 12y ago
      I find this approach has a number of advantages: - it forces you to flesh out your conceptual/paper design in a deliverables-oriented way and makes it obvious where the gaps are, which gives you a good idea of how ready you are to wrap up design and start development, and in turn can be used to create your work breakdown structure and task prioritisation; sometimes it can result in identifying simplifications upfront (saving you time down the line) - when you have a good idea of the scope and complexity of a solution like this, you're less likely to waste time on the less important stuff - if you're the tech lead on a team, giving your developers the high-level components and tests means they're more likely to get things right (or at least limit the damage they do when they get things wrong) Without component-level unit tests or design, you can easily dive into a solution without thinking it through and really regret it.
  • falcolas 12y ago
    In my experience, if your integration testing is only happy path testing, you're doing integration testing wrong. Integration testing should, at a minimum, include: * Combinatorial testing * i.e. testing all possible input combinations for every pair of two inputs * Fuzz testing * Limit testing * Success path testing * Failure case testing Integration testing will discover more component interaction bugs than unit testing alone, and doing only happy path testing is doing it wrong. On the other hand, I wholly believe that testing done by the developer alone will never catch as many bugs as testing done by both the developer and a QA engineer to the mix will. They think differently than we do, and as such are adept at finding our blind spots.
    • jameskilton 12y ago
      We may have a wording definition issue here. I've heard multiple definitions of "integration tests". I usually use them where some people use "acceptance tests", aka, top-down be-a-user type testing (selenium in Firefox, for example). Sorry if there was any confusion there. Agreed having a QA team also go through the app is great for finding things that tests can't catch, like usability errors, or crazy edge cases you didn't think about, etc. I have gotten to work with a good QA team and it's amazing the things they find.
• MaybiusStrip 12y ago
  I think it depends how tightly your business logic and your view are bound and how well your business logic is abstracted. I use Angular at work and 90% of the bugs we get can be caught by a unit test. On a rare occasion something will be mis-labeled on the markup and the two-way-binding "fails" silently, but usually our bugs are errors in the logic. Whenever you fix a bug in your code, try to cover it with a unit test. If you can't, ask yourself whether it might be because the code is poorly abstracted. I think there are definitely varying degrees of value in unit tests. Your example is definitely not super valuable, but unit tests on higher level methods can save a lot of trouble. When I code I want to rest assured that the different pieces I am using work. I want assurance that this class I'm about to use has an API that does exactly what it says. If I want to change modify the class, I want to see exactly what assumptions I'm breaking that users of this class might be making, and I want to be able to do this in under a second. This can have a drastic impact on developer velocity. I don't know of any other way of doing this than unit tests.
  • meric 12y ago
    Also it's much less tedious when developing a web app to test your code using unit tests, rather than doing curl commands and/or actually launching the frontend to test the API manually and trying to glean errors from the web inspector.
  • matwood 12y ago
    Whenever you fix a bug in your code, try to cover it with a unit test. If you can't, ask yourself whether it might be because the code is poorly abstracted. This, big time. Whenever I get a bug report I first write a test that fails because of the bug. Only then can I a) be sure I understand the bug and why it happened and b) be sure I fixed the bug and finally c) prevent a regression in the future. Even for people who are not TDD or big on tests in general can make great use out of writing tests to verify and fix bugs.
• sp332 12y ago
  Unit test are more useful for refactoring, making sure the system works the same before & after.
• switch007 12y ago
  >I've found little value from unit tests. Not even when re-factoring? Refactoring code without unit tests makes me extremely anxious and I try to avoid it.
  • columbo 12y ago
    Yeah it all depends; unit tests can make refactoring even more difficult (especially if you heavily rely on mocking). I've done major refactoring (moving from one language to another) and the testing was done entirely on the rest api. With several thousand tests I felt very confident that the user experience and expected behavior remain unchanged. Honestly I don't hate unit testing and do I use it on a regular basis, the article was rather heavy handed.
  • twic 12y ago
    Refactoring code without tests is definitely scary. But those tests don't need to be unit tests. (caveat: i understand "unit test" to mean a test which tests a single class, or sometimes a very small number of classes, with any collaborators replaced with stubs or mocks) In my experience, bigger tests - what my colleagues call integration tests, which might involve 3-30 classes and often the database - give me a lot more confidence in my changes. Tests are only a useful safety net for changes which are entirely confined to the thing they test. If you're changing the way some particular method is implemented, say from looping over a collection to mapping a lambda over it, then the change is confined to that class, and a unit test can help. But if you're changing the method's contract, say from taking a collection of objects to taking one at a time, then the change takes in that class and its clients, and the unit test is useless - it tests that the method does something that it should no longer do! Yes, you can rewrite the unit test, but that does nothing to reassure you that the overall behaviour hasn't changed. You're starting from scratch. The thing is, in my experience, valuable refactorings tend to take in more than one class. Sometimes much more than one class. I want to be able to make those refactorings with a safety net. Unit tests can't give me that.
• lmm 12y ago
  Does deleting products ever break? I wouldn't bother with any tests, unit or functional, for a code path that doesn't have any logic in it - if it compiled then it's almost certainly correct. Where testing is useful is when there's complex logic. And such logic is much easier to test at the unit level.
  • pfg 12y ago
    I can imagine deleting products may be failing due to some UI change. Maybe a button isn't visible in certain IE versions. Maybe the JS code executing the backend request is failing in some browsers. That's where e2e tests are useful.
    • lmm 12y ago
      Those are purely failures in the UI layer. To the extent that "unit testing" a web frontend is possible, you could catch those problems with a pure unit test that mocked out the backend, it wouldn't have to be an end-to-end test.
  • jameshart 12y ago
    In most real systems, deletes of core business objects like 'product' are really only soft deletes; there are caching layers; deleting things has knock-on consequences for other things (if you delete a product when an instance of it is in a customer's shopping basket, what happens?) - it doesn't seem illogical to want to test that when you 'delete' a product, it disappears from the product catalog... but then you'd want to do other tests to verify it remains in a shopping basket, and is still visible in purchase history, and so on.
    • lmm 12y ago
      Fair enough - but in that case we're really not talking about the kind of test the grandparent described, the one that looks like p = productDao.create("My New Product") assert true p.delete()
      • 12y ago
• brown9-2 12y ago
  Typically this would be answered in a unit test like this: This would be a really bad way to test all of "create a user, assign the user as an administrator, log off of the root admin, log in as the new user, create a product, delete a product, get the product list and verify the product isn't on the list". You would be ignoring 80% of the flow. The goal of unit testing is to test each piece in isolation and to provide confidence that when you have to assemble a lot of little pieces of code to form a larger piece of functionality, that each of those little pieces is working as it's contract says it should. If your integration test containing 8 steps failed, would you know which piece of the puzzle caused the failure from the integration test report itself? Of course, you could go the route of testing both small pieces of functionality and also testing large pieces (the integration tests), so that you can have overlapping levels of confidence.
• opendais 12y ago
  I agree with you. In my experience, the value in unit tests is in refactoring. I'd only write them as I refactor [to confirm the functionality matches before/after the refactor]. Most of the modifications we do to common libraries, etc. involve higher level rewrites that would break the unit tests anyway and the only way to confirm things still function is integration tests. :/
• briantakita 12y ago
  > I've gone the salesforce route, instead of using selenium all my "unit tests" are executed against the rest api. I've taken the same route for apis and for client side testing. On my current project, I'm creating a rich client. I have an extensive jasmine test suite with edge case and race condition testing. I created a library called jasmine-flow, to organize the tests into flows. This allows edge case testing while reducing duplicate test setup. In my case test suite time was reduced by 10x. https://github.com/btakita/jasmine-flow https://github.com/btakita/jasmine-flow