7 ms·

Look at it this way: you must be testing code as you write it anyway. There's really no other sane way to do it. You make a change, you load the page and see th

by ef4 15y ago

Look at it this way: you must be testing code as you write it anyway. There's really no other sane way to do it. You make a change, you load the page and see that your change worked, or you call your new function from an interactive interpreter. Smart automated testing just takes all that extra test work you're already doing and saves it as you go along. No need to try to invent extra things to test. You just test what you would have tested anyway by hand.

Iv 15y ago
You think like that until you hit your first serious regressions and discover it has been in the code base for several months and that the person responsible for it has left. I used to work at a company where automated testing was sending you emails about what your commits broke. It does help at improving code quality.
harel 15y ago
But then you'd still do the manual test after you complete your code. Nobody (I hope) codes blind hoping it would work or caught later by a test suite. Test suits don't reveal everything. Only what you tested for.
- fragsworth 15y ago
  The process goes more like this: 1. Write some code 2. Test it 3. Debug your code 4. Test it 5. Debug your code 6. Test it ... 39. Debug your code 40. Test it; now it finally seems to work If all your "test it" steps are being done manually, you're being very inefficient. A good unit test can actually make development go by faster with the added bonus of defending your code from changes down the road that might screw things up.
  - niccl 15y ago
    That's all fine unless you're exploring a solution space. Then the overhead of writing tests which are thrown away _in entirety_ is outrageous. AFAIK TDD really only works if you're either a) prepared to waste a huge amount of time writing tests which will later be completely redundant or b) working to a very clear set of requirements with tools and frameworks you already know intimately. </Rant>
    - pimeys 15y ago
      Of course you have to know your tools. I usually start by doing test cases, then writing the real features while spork and whatchr evaluates my new code every time I save. I rarely even open a browser, it's the final thing I do when my tests are green. It doesn't slow me down, and I can be sure that my feature is there and works even when somebody refactors our software.
    - sfev 15y ago
      Sure, but what you're describing is prototyping. Prototype all you want without tests, but once you've settled on a design & are ready to make it production-ready you should spend time at least writing unit tests for your work or (ideally) take what you've learned & re-apply to a clean design written in a test-first manner. You might think this is a waste of time, but putting code into production without tests is going to give you more trouble in the long run.
    - viscanti 15y ago
      I find I spend significantly more time refactoring/maintaining code than I spend writing exploratory code. It's silly to write tests for prototype work, but once you're actually close to having a working prototype, tests help. Having decent test coverage saves so much more time when refactoring/maintaining. TDD isn't "THE" way, but test coverage helps. It's not fun (at least not for me), but it's less aggravating than breaking something 6 months down the road in some non-obvious way. I'm human, so I assume I'll screw something up eventually. Having test coverage helps keep me from shooting myself in the foot later.
    - mattbriggs 15y ago
      in TDD what you are talking about is called a "spike", just write a bunch of code to try out assumptions and find a direction to go that you are reasonably sure is a good one.
    - peeters 15y ago
      Different levels of tests work here. I usually start with a very high-level test and then as I implement I do unit tests once I have a reasonably high confidence that the units are a good design. You should often be able to at least create an automated acceptance test for what you're doing (e.g., "as a user I want to click this button and see XYZ"). This is usually extremely decoupled from the implementation so it should survive refactoring. So then do your exploratory code, get the test passing, and then refactor, introducing lower-level tests. If that doesn't seem doable you might be taking on a task that doesn't have a good set of requirements. Writing code without any concrete use case in mind is fun and all but that kind of code should usually be limited to a prototyping sandbox.
  - Retric 15y ago
    That's mostly a beginners problem, I know several people where it's mostly: 1. Write a lot of code 2. Test it 3. It works! 4. Test it 5. It works! 6. Test it 7. It works! ... 15. Test it 16. It works! 17. Are you done? TDD in no way makes 17 any clear, because every test they thought of before writing the code works more or less the first time. And that's the core problem with testing, for a solid developer what fails is has nothing to do with the code it's always a question of edge cases they did not think. (Wait, some sales people are their own managers and outside consultants at the same time? well just bob) You can force these people to write tests, but it really does just slow them down.
    - mattbriggs 15y ago
      I have never seen any half way decent developer write code for more then a few minutes without some sort of feedback, automated test suite or not. You are right that it will work more or less, but they work out the "less" part of that statement sooner rather then later. In my experience, that is a place you get to over time, only the people out of college write code for multiple hours straight, then debug everything afterwards. That is actually the primary goal of tdd, to free you from the more mundane aspects of the code/run/debug loop. The secondary goal is to give you a good base for changing the code later and finding out what broke, again without a ton of manual actions. But as useful as that is (and it is extremely useful), it doesn't hold a candle to the first benefit.
      - fragsworth 15y ago
        That's the point I was trying to make. The main benefit comes from eliminating the "run/debug" part of the "code/run/debug" loop. It then just becomes "code/test" where "test" takes all of a couple seconds each time.
        alextgordon 15y ago
        But if you don't run it yourself, how do you know it's working? All you know is that the tests pass. Tests can easily do more harm than good if you let them give you a false sense of security.
- haar 15y ago
  Even if you test manually what you just changed, in a relatively complex codebase how can you guarantee that your changes haven't broken behaviour in a separate yet related aspect of the system? This is what I find the major advantage of a comprehensive test-suite to be, I don't have to worry as much about breaking any part of the system as a whole - if my suite passes, then I know everything I've worked on so far works, not just the bit I think I changed.
  - vishgm 15y ago
    so true!!
  - anarchotroll 15y ago
    You can never actually guarantee anything even if you have tests. The tests will only increase the likelihood of catching regressions.
- Oompa 15y ago
  Then you didn't write good enough tests. I have deployed code that thousands of customers see without manually testing it. If my tests are green, I'm confident in deploying my code.
  - djcapelis 15y ago
    What tests your tests?
    - Oompa 15y ago
      No one. That's how much faith I had in that test.
debacle 15y ago
Smart automated testing sounds amazing, until you realize that it's not smart at all. The dumb computer that you're ordering to do your bidding is the same dumb computer that is going to be running your tests, and chances are the programmer is invariant as well. In short, good programmers need unit tests less and bad programmers will write bad tests. You can't fix a personnel problem with technology.
- deleted 15y ago
  [deleted]
- anarchotroll 15y ago
  Exactly. The computer will just repeat what you tell it to do. There is the chance that you will tell it wrong (a bug in your test code), and the chance that what you told it is not true anymore. An automated test basically saves you the work of doing the same thing over an over again as you develop. But we must keep in mind that maintaining test code has a cost. Automated testing is not a holy grail and it isn't useful 100% of the time. You should carefully decide what code is worth testing via automation. Unit tests in general don't catch any regressions, they only help you develop. Functional tests might be useful, but only if you are testing something that is not likely to change much. E.g.: It is not worth to automate testing the UI if you are going to completely redesign it next month. Manual testing can actually be cheaper than maintaining test code.

by ef4 15y ago

Iv 15y ago

You think like that until you hit your first serious regressions and discover it has been in the code base for several months and that the person responsible for it has left. I used to work at a company where automated testing was sending you emails about what your commits broke. It does help at improving code quality.

harel 15y ago

But then you'd still do the manual test after you complete your code. Nobody (I hope) codes blind hoping it would work or caught later by a test suite. Test suits don't reveal everything. Only what you tested for.

fragsworth 15y ago
The process goes more like this: 1. Write some code 2. Test it 3. Debug your code 4. Test it 5. Debug your code 6. Test it ... 39. Debug your code 40. Test it; now it finally seems to work If all your "test it" steps are being done manually, you're being very inefficient. A good unit test can actually make development go by faster with the added bonus of defending your code from changes down the road that might screw things up.
- niccl 15y ago
  That's all fine unless you're exploring a solution space. Then the overhead of writing tests which are thrown away _in entirety_ is outrageous. AFAIK TDD really only works if you're either a) prepared to waste a huge amount of time writing tests which will later be completely redundant or b) working to a very clear set of requirements with tools and frameworks you already know intimately. </Rant>
  - pimeys 15y ago
    Of course you have to know your tools. I usually start by doing test cases, then writing the real features while spork and whatchr evaluates my new code every time I save. I rarely even open a browser, it's the final thing I do when my tests are green. It doesn't slow me down, and I can be sure that my feature is there and works even when somebody refactors our software.
  - sfev 15y ago
    Sure, but what you're describing is prototyping. Prototype all you want without tests, but once you've settled on a design & are ready to make it production-ready you should spend time at least writing unit tests for your work or (ideally) take what you've learned & re-apply to a clean design written in a test-first manner. You might think this is a waste of time, but putting code into production without tests is going to give you more trouble in the long run.
  - viscanti 15y ago
    I find I spend significantly more time refactoring/maintaining code than I spend writing exploratory code. It's silly to write tests for prototype work, but once you're actually close to having a working prototype, tests help. Having decent test coverage saves so much more time when refactoring/maintaining. TDD isn't "THE" way, but test coverage helps. It's not fun (at least not for me), but it's less aggravating than breaking something 6 months down the road in some non-obvious way. I'm human, so I assume I'll screw something up eventually. Having test coverage helps keep me from shooting myself in the foot later.
  - mattbriggs 15y ago
    in TDD what you are talking about is called a "spike", just write a bunch of code to try out assumptions and find a direction to go that you are reasonably sure is a good one.
  - peeters 15y ago
    Different levels of tests work here. I usually start with a very high-level test and then as I implement I do unit tests once I have a reasonably high confidence that the units are a good design. You should often be able to at least create an automated acceptance test for what you're doing (e.g., "as a user I want to click this button and see XYZ"). This is usually extremely decoupled from the implementation so it should survive refactoring. So then do your exploratory code, get the test passing, and then refactor, introducing lower-level tests. If that doesn't seem doable you might be taking on a task that doesn't have a good set of requirements. Writing code without any concrete use case in mind is fun and all but that kind of code should usually be limited to a prototyping sandbox.
- Retric 15y ago
  That's mostly a beginners problem, I know several people where it's mostly: 1. Write a lot of code 2. Test it 3. It works! 4. Test it 5. It works! 6. Test it 7. It works! ... 15. Test it 16. It works! 17. Are you done? TDD in no way makes 17 any clear, because every test they thought of before writing the code works more or less the first time. And that's the core problem with testing, for a solid developer what fails is has nothing to do with the code it's always a question of edge cases they did not think. (Wait, some sales people are their own managers and outside consultants at the same time? well just bob) You can force these people to write tests, but it really does just slow them down.
  - mattbriggs 15y ago
    I have never seen any half way decent developer write code for more then a few minutes without some sort of feedback, automated test suite or not. You are right that it will work more or less, but they work out the "less" part of that statement sooner rather then later. In my experience, that is a place you get to over time, only the people out of college write code for multiple hours straight, then debug everything afterwards. That is actually the primary goal of tdd, to free you from the more mundane aspects of the code/run/debug loop. The secondary goal is to give you a good base for changing the code later and finding out what broke, again without a ton of manual actions. But as useful as that is (and it is extremely useful), it doesn't hold a candle to the first benefit.
    - fragsworth 15y ago
      That's the point I was trying to make. The main benefit comes from eliminating the "run/debug" part of the "code/run/debug" loop. It then just becomes "code/test" where "test" takes all of a couple seconds each time.
      - alextgordon 15y ago
        But if you don't run it yourself, how do you know it's working? All you know is that the tests pass. Tests can easily do more harm than good if you let them give you a false sense of security.
haar 15y ago
Even if you test manually what you just changed, in a relatively complex codebase how can you guarantee that your changes haven't broken behaviour in a separate yet related aspect of the system? This is what I find the major advantage of a comprehensive test-suite to be, I don't have to worry as much about breaking any part of the system as a whole - if my suite passes, then I know everything I've worked on so far works, not just the bit I think I changed.
- vishgm 15y ago
  so true!!
- anarchotroll 15y ago
  You can never actually guarantee anything even if you have tests. The tests will only increase the likelihood of catching regressions.
Oompa 15y ago
Then you didn't write good enough tests. I have deployed code that thousands of customers see without manually testing it. If my tests are green, I'm confident in deploying my code.
- djcapelis 15y ago
  What tests your tests?
  - Oompa 15y ago
    No one. That's how much faith I had in that test.

debacle 15y ago

Smart automated testing sounds amazing, until you realize that it's not smart at all. The dumb computer that you're ordering to do your bidding is the same dumb computer that is going to be running your tests, and chances are the programmer is invariant as well. In short, good programmers need unit tests less and bad programmers will write bad tests. You can't fix a personnel problem with technology.

deleted 15y ago
[deleted]
anarchotroll 15y ago
Exactly. The computer will just repeat what you tell it to do. There is the chance that you will tell it wrong (a bug in your test code), and the chance that what you told it is not true anymore. An automated test basically saves you the work of doing the same thing over an over again as you develop. But we must keep in mind that maintaining test code has a cost. Automated testing is not a holy grail and it isn't useful 100% of the time. You should carefully decide what code is worth testing via automation. Unit tests in general don't catch any regressions, they only help you develop. Functional tests might be useful, but only if you are testing something that is not likely to change much. E.g.: It is not worth to automate testing the UI if you are going to completely redesign it next month. Manual testing can actually be cheaper than maintaining test code.