4 ms·

the problem isn't that my tests are bad, the problem is that it's commonplace to adapt new interfaces in a constantly evolving codebase. when your unit tests we

by choxi 15y ago

the problem isn't that my tests are bad, the problem is that it's commonplace to adapt new interfaces in a constantly evolving codebase. when your unit tests were built for one interface, you have to change them when you change the interface. for example, the interface to enroll in a course used to be an association between an "enrollment" and the "course". when we added "courses has_many sessions", you enrolled in "courses" via "sessions". you have to rewrite all your enrollment specs to respect that new interface now, regardless of if they were shitty or well-written. to me, that's a maintenance cost that isn't worth it at a startup given how rapidly you have to change those internal interfaces.

gnaritas 15y ago
You're mixing up interface and implementation. You should be able to change how you implement the association without changing the interface or breaking the test; unless your test knows too much about the internals of the implementation, in which case it's a poorly written test.
- _delirium 15y ago
  That's the OO theory, but interfaces do frequently change in any codebase that isn't mature, because it's not always clear up front what the interface actually should be, and if you haven't committed to supporting it (e.g. because it's an externally exposed API that can't be dropped), it can be better to change your 6-month-old interface than keep one you realized wasn't great.
  - jaggederest 15y ago
    Good tests help you change either the interface or the implementation separately. Trying to change both at once is madness, in my opinion, and almost impossible even with good tests.
jaggederest 15y ago
> you have to rewrite all your enrollment specs to respect that new interface now, regardless of if they were shitty or well-written. No, not if you're doing it correctly. Again, that's not a unit test, it's a functional test. Unit tests test really small blocks of code. If you're crossing class boundaries, generally, it's not really a unit test. Let me show you what I mean with tests I wrote: https://github.com/newrelic/rpm/blob/master/test/new_relic/agent/error_collector/notice_error_test.rb https://github.com/newrelic/rpm/blob/master/test/new_relic/a... Those are unit tests (not beautiful ones, written to refactor, but I digress) - they mock cross-object (and even cross method, in some cases) and they really focus on individual paths through the code. If you change a method, only tests relating to that method fail. Edit: also, "the problem isn't that my tests are bad," is a poor assumption to start with - you're assuming a priori something that we're discussing here, which is that you're complaining about problems resulting from not testing correctly. "Bad" is a loaded word in any case - I'm not sure there's a test I've seen that couldn't be improved.
- gfodor 15y ago
  And, if you decide to rename any of the methods on the classes you've mocked out here, your unit tests will continue to pass despite the fact your implementation is now full of bugs. And, if you do rename the methods or class you've mocked, you now have to update every single test for any class coupled with the changed method. I've never understood endo-testing/mock objects in environments where the compiler cannot check your mocked interfaces. I also don't understand how people can aruge that you shouldn't be testing against the implementation and then say in the next breath you should be mocking out every single method call the internal implementation makes explicitly. You're just setting yourself to get lots and lots of green tests on code that will explode as soon as it hits production. Whenever I've done aggressive mock object based testing I soon have zero confidence in my tests because I get burned due to the fact that the mock objects eventually start asserting that the wrong behavior is right and my code explodes when integrated. (And yes, I know the excuse here is that you should then write integration tests and functional tests too. But seriously now, how many tests are you going to end up writing for your 100 line Ruby class before you decide you're going overboard in the name of purity?) Better to instead just write it so instead of worrying about no obvious deficiencies there are obviously no deficiencies (avoid side effects, state, extra coupling), and write some functional tests just to be safe. Yes, those ones that actually hit the database and test the interaction between multiple classes that TDD advocates loathe because they are so slow and impure. Slow they may be, but at least I know they're testing the code that's going to run on my servers. I'd rather have 10 tests break when I change one class that are easy to fix than have zero tests break when I change one class and let broken code get to production.

by choxi 15y ago

gnaritas 15y ago

You're mixing up interface and implementation. You should be able to change how you implement the association without changing the interface or breaking the test; unless your test knows too much about the internals of the implementation, in which case it's a poorly written test.

_delirium 15y ago
That's the OO theory, but interfaces do frequently change in any codebase that isn't mature, because it's not always clear up front what the interface actually should be, and if you haven't committed to supporting it (e.g. because it's an externally exposed API that can't be dropped), it can be better to change your 6-month-old interface than keep one you realized wasn't great.
- jaggederest 15y ago
  Good tests help you change either the interface or the implementation separately. Trying to change both at once is madness, in my opinion, and almost impossible even with good tests.

jaggederest 15y ago

> you have to rewrite all your enrollment specs to respect that new interface now, regardless of if they were shitty or well-written. No, not if you're doing it correctly. Again, that's not a unit test, it's a functional test. Unit tests test really small blocks of code. If you're crossing class boundaries, generally, it's not really a unit test. Let me show you what I mean with tests I wrote: https://github.com/newrelic/rpm/blob/master/test/new_relic/agent/error_collector/notice_error_test.rb https://github.com/newrelic/rpm/blob/master/test/new_relic/a... Those are unit tests (not beautiful ones, written to refactor, but I digress) - they mock cross-object (and even cross method, in some cases) and they really focus on individual paths through the code. If you change a method, only tests relating to that method fail. Edit: also, "the problem isn't that my tests are bad," is a poor assumption to start with - you're assuming a priori something that we're discussing here, which is that you're complaining about problems resulting from not testing correctly. "Bad" is a loaded word in any case - I'm not sure there's a test I've seen that couldn't be improved.

gfodor 15y ago
And, if you decide to rename any of the methods on the classes you've mocked out here, your unit tests will continue to pass despite the fact your implementation is now full of bugs. And, if you do rename the methods or class you've mocked, you now have to update every single test for any class coupled with the changed method. I've never understood endo-testing/mock objects in environments where the compiler cannot check your mocked interfaces. I also don't understand how people can aruge that you shouldn't be testing against the implementation and then say in the next breath you should be mocking out every single method call the internal implementation makes explicitly. You're just setting yourself to get lots and lots of green tests on code that will explode as soon as it hits production. Whenever I've done aggressive mock object based testing I soon have zero confidence in my tests because I get burned due to the fact that the mock objects eventually start asserting that the wrong behavior is right and my code explodes when integrated. (And yes, I know the excuse here is that you should then write integration tests and functional tests too. But seriously now, how many tests are you going to end up writing for your 100 line Ruby class before you decide you're going overboard in the name of purity?) Better to instead just write it so instead of worrying about no obvious deficiencies there are obviously no deficiencies (avoid side effects, state, extra coupling), and write some functional tests just to be safe. Yes, those ones that actually hit the database and test the interaction between multiple classes that TDD advocates loathe because they are so slow and impure. Slow they may be, but at least I know they're testing the code that's going to run on my servers. I'd rather have 10 tests break when I change one class that are easy to fix than have zero tests break when I change one class and let broken code get to production.