3 ms·

I like to do integration tests and just match expected output.

by tty7 11y ago

iofj 11y ago
This is brilliant advice. This whole unit test madness needs to stop. This post advocates: 1) unit test everything 2) complain you can't change anything without breaking a lot 3) complain that the program doesn't work even when the unit tests say "it should". Well that's pretty much the expected result, isn't it ? Of course unit tests break if you change a method, even if the resulting program works better. And a built program doesn't work if all unit tests pass ? Well that's just not what a unit test tests ... so of course it doesn't. An alternative tactic could be: 1) understand the problem domain. Writing an administrative system ? You should be a better accountant than the worst accountant that works in that department. If a feature request comes in, and you can't answer "why do they want this", it's time to spend some time in their team. 2) get an integration test. Something that emulates the software doing what it's meant to do at a large scale with as many real components as possible. E.g. an administration system should just be asked to run an administration, and have the result checked. The result should be checked in the sense that it should match what the accounting handbook says it should say, NOT if the code worked as expected. You should try to do realistic things. E.g. have the test read the production database, and then enter the first 50000 customers into the test in a random order, constantly doing things like getting a customer balance in between. Insert 10 test customers and see how their accounts evolve and if this matches business rules. Check if the total system still balances out to zero at every point. Have it load the system, demanding it performs at a minimal acceptable level. 3) change things. 4) find out what breaks on a functional level. Get a report that when adding the current customer list one at a time to the software, 5 weren't in the database after entering them. Fix. 5) Run the integration test again. It passes. 6) Confidently go to your boss and declare this software will work. You can say that, because you just have demonstrated that it works. Some bad programmers hate this, because it finds problems unit tests won't ever find, and won't tell you where the problem is. They don't help programmers that don't understand what the program is doing. They don't directly report why things are happening. They won't let developers get away with "I just changed the part I understand", and don't let them mark the bug as fixed after that. Entering a new customer takes 50s after your field addition change ? Integration tests will find this, and create an easily debuggable situation. Integration tests will catch that code that is multiple directories and modules apart, maybe even separated by an RPC won't cooperate. Integration tests, in short, tell you that the program does what it's built to do.
- louhike 11y ago
  The point of unit tests is to give you confidence into making some changes without breaking the behavior of the methods and without having to do integration test or user tests. The advantage of unit tests is they are quick to do. Their purpose is different than integration tests. You are not supposed to deliver an application just because unit tests pass.
  - iofj 11y ago
    The point is not to tell you that you should never have unit tests. They can be useful (though rarely). I'm not suggesting that if you write a new datastructure that you shouldn't have a few tests that only really exercise one or a few methods. They do not guarantee the program does what it's supposed to do. "without having to do integration test or user tests" doesn't exist. It's fiction. It leads to all the bad places to be in software development. It leads to you explaining to a manager or PM that "it matches the requirements, I don't accept that you have a problem", when the software just lost your firm a million dollars. Software development in reality : you do not get accurate requirements. Your design does not match reality (and the 1st version is so far off it's not even internally consistent: it likely can't be built at all, never mind perform it's intended function). Getting better at software development, after a while, stops being about improving your ability to collect requirements. It does not make your v1 designs better. You simply learn to deal with change, to provide good places for as-yet-unknown changes in your designs, to go out weekly and ask for changes in requirements, knowing that either your design can accommodate them or you'd rather hear them sooner than later.
Ace17 11y ago
The problem is, Integrated tests are a scam. https://www.youtube.com/watch?v=VDfX44fZoMc https://www.youtube.com/watch?v=VDfX44fZoMc

I like to do integration tests and just match expected output.

by tty7 11y ago

I like to do integration tests and just match expected output.

iofj 11y ago

This is brilliant advice. This whole unit test madness needs to stop. This post advocates: 1) unit test everything 2) complain you can't change anything without breaking a lot 3) complain that the program doesn't work even when the unit tests say "it should". Well that's pretty much the expected result, isn't it ? Of course unit tests break if you change a method, even if the resulting program works better. And a built program doesn't work if all unit tests pass ? Well that's just not what a unit test tests ... so of course it doesn't. An alternative tactic could be: 1) understand the problem domain. Writing an administrative system ? You should be a better accountant than the worst accountant that works in that department. If a feature request comes in, and you can't answer "why do they want this", it's time to spend some time in their team. 2) get an integration test. Something that emulates the software doing what it's meant to do at a large scale with as many real components as possible. E.g. an administration system should just be asked to run an administration, and have the result checked. The result should be checked in the sense that it should match what the accounting handbook says it should say, NOT if the code worked as expected. You should try to do realistic things. E.g. have the test read the production database, and then enter the first 50000 customers into the test in a random order, constantly doing things like getting a customer balance in between. Insert 10 test customers and see how their accounts evolve and if this matches business rules. Check if the total system still balances out to zero at every point. Have it load the system, demanding it performs at a minimal acceptable level. 3) change things. 4) find out what breaks on a functional level. Get a report that when adding the current customer list one at a time to the software, 5 weren't in the database after entering them. Fix. 5) Run the integration test again. It passes. 6) Confidently go to your boss and declare this software will work. You can say that, because you just have demonstrated that it works. Some bad programmers hate this, because it finds problems unit tests won't ever find, and won't tell you where the problem is. They don't help programmers that don't understand what the program is doing. They don't directly report why things are happening. They won't let developers get away with "I just changed the part I understand", and don't let them mark the bug as fixed after that. Entering a new customer takes 50s after your field addition change ? Integration tests will find this, and create an easily debuggable situation. Integration tests will catch that code that is multiple directories and modules apart, maybe even separated by an RPC won't cooperate. Integration tests, in short, tell you that the program does what it's built to do.

louhike 11y ago
The point of unit tests is to give you confidence into making some changes without breaking the behavior of the methods and without having to do integration test or user tests. The advantage of unit tests is they are quick to do. Their purpose is different than integration tests. You are not supposed to deliver an application just because unit tests pass.
- iofj 11y ago
  The point is not to tell you that you should never have unit tests. They can be useful (though rarely). I'm not suggesting that if you write a new datastructure that you shouldn't have a few tests that only really exercise one or a few methods. They do not guarantee the program does what it's supposed to do. "without having to do integration test or user tests" doesn't exist. It's fiction. It leads to all the bad places to be in software development. It leads to you explaining to a manager or PM that "it matches the requirements, I don't accept that you have a problem", when the software just lost your firm a million dollars. Software development in reality : you do not get accurate requirements. Your design does not match reality (and the 1st version is so far off it's not even internally consistent: it likely can't be built at all, never mind perform it's intended function). Getting better at software development, after a while, stops being about improving your ability to collect requirements. It does not make your v1 designs better. You simply learn to deal with change, to provide good places for as-yet-unknown changes in your designs, to go out weekly and ask for changes in requirements, knowing that either your design can accommodate them or you'd rather hear them sooner than later.

Ace17 11y ago

The problem is, Integrated tests are a scam. https://www.youtube.com/watch?v=VDfX44fZoMc https://www.youtube.com/watch?v=VDfX44fZoMc