3 ms·

I haven't experienced that problem. I've experienced the problem everyone else seems to have, too: Too many unit tests makes things brittle over time. One small

I haven't experienced that problem. I've experienced the problem everyone else seems to have, too: Too many unit tests makes things brittle over time. One small change breaks a bunch of stuff in some mechanical way. Perhaps a required parameter got added to a function, or perhaps a class got too big and needed to be split up. Either way, you sometimes end up with a slew of tests that don't compile. Testing at a higher level should help, because the tests are more coupled to the behavior and less coupled to the implementation details. To the extent that you've achieved that, tests breaking due to a change shouldn't be spurious, because the breakage is due to a legitimate change in behavior that needs to be accounted for in the tests' expectations. Wild guess: The difference in experience here can be explained by the first three words of the article. A contractor who isn't sticking around on any one project for long isn't going to see consequences of their design decisions when they take many years to incubate. And a contractor who's seen this at "no less than six different companies" probably hasn't been a long-timer at many of them. Unit testing has only been de rigueur for 20 years, tops, so, even assuming those were the only six companies, I think it's safe to assume a maximum of 3 years per company. Subtract a year or two from the incubation time because it'll take a while to comprehensively revamp a test suite. A lot of Shakespeare's plays seem to end on a happier note, too, if you always leave the theatre right after the third or fourth act.

rectang 6y ago
The best ROI I get when writing tests is from thorough unit testing of the most fundamental, most heavily used low-level components. That's because components like "StreamReader" or "Lock" or "Response" tend not to change much over time. In contrast, higher-level tests often break because the high-level components the test change frequently in response to business needs. I don't get this meme that unit tests are "brittle". Are the interfaces to your low-level components changing all the time? Perhaps they are not written with "loose coupling" in mind and are exposing needlessly fragile and complex APIs?
- mumblemumble 6y ago
  It's frequently the domain model objects themselves changing that causes all the spurious breakage. This is not nearly as much a problem in dynamic languages, especially if you tend to do data-level programming instead of defining a lot of custom types for your domain modeling. And I suspect that this is exactly why classical TDD was first popularized among Java developers, while the London school first gained traction in the Ruby and Python communities.
- exclipy 6y ago
  Clearly, the original proponents of unit testing were thinking about low level components like stream readers and binary trees. The world has moved on now and most programmers are duct taping parts together so unit testing no longer has a place in this world. But the dogma remains.
  - rectang 6y ago
    > The world has moved on now and most programmers are duct taping parts together so unit testing no longer has a place in this world. Even on business-domain level components there are often simple routines which can and should be tested. Larger routines should be built up from smaller ones. Divide-and-conquer is a fundamental engineering principle. If there are no such small-enough-to-be-tested routines, then the code is not being written with testing in mind. If all your functions are hundreds of lines long, then yeah it's hard to write unit tests!
    - exclipy 6y ago
      Occasionally, yes. But if you're job is to convert JSON to DOM nodes using React, then no. In this world, what's more likely to go wrong is that one element overlaps another when zoomed to 122% on IE 11.
  - mumblemumble 6y ago
    The earliest proponents I know of - the ones writing XP books back in the '90s - had a very flexible definition of "unit" and assumed that you would vary the scale according to your needs. So some units might be low level components like data structures, but others might be large modules that included a lot of moving parts, up to and including actual databases. The world has moved on now, and "agile" no longer means "use your own best judgment." It's become a carefully catalogued list of rules and prescriptions and definitions and dictums. Which, come to think of it, is an excellent example of memetic natural selection. "Use your own best judgment" is a meme that is uniquely poorly adapted to survival in an ecosystem that consists largely of people jabbering back and forth on the Internet like we are right now. You can't base a rollicking good fun argument or a strongly worded blog post on something like that.
- ensiferum 6y ago
  I agree. Your unit tests should focus on testing individual pieces, the fundamental building blocks (classes and functions) that you have in your system. The changes should be isolated so when a class API changes only those unit tests would need to change. I find the key thing here to be separation of concerns. Aim for testable classes and functions and that will make you also thing about the depencies. For example I have class implementing NNT protocol. Initially one could think that this class needs to deal with IO or even worse with socket based data transfer. But if you think about it doesn't. The api can be based on an idea of getting a buffer full of data in and getting buffer Full of data out. Class coupled with IO/socket functionality = coupled hard to test without its trivial to unit test. (Reduced functionality class only maintains the protocol state doesn't concern itself where the data comes or goes) Reference. https://github.com/ensisoft/newsflash-plus/blob/master/engine/session.h https://github.com/ensisoft/newsflash-plus/blob/master/engin...

rectang 6y ago

The best ROI I get when writing tests is from thorough unit testing of the most fundamental, most heavily used low-level components. That's because components like "StreamReader" or "Lock" or "Response" tend not to change much over time. In contrast, higher-level tests often break because the high-level components the test change frequently in response to business needs. I don't get this meme that unit tests are "brittle". Are the interfaces to your low-level components changing all the time? Perhaps they are not written with "loose coupling" in mind and are exposing needlessly fragile and complex APIs?

mumblemumble 6y ago
It's frequently the domain model objects themselves changing that causes all the spurious breakage. This is not nearly as much a problem in dynamic languages, especially if you tend to do data-level programming instead of defining a lot of custom types for your domain modeling. And I suspect that this is exactly why classical TDD was first popularized among Java developers, while the London school first gained traction in the Ruby and Python communities.
exclipy 6y ago
Clearly, the original proponents of unit testing were thinking about low level components like stream readers and binary trees. The world has moved on now and most programmers are duct taping parts together so unit testing no longer has a place in this world. But the dogma remains.
- rectang 6y ago
  > The world has moved on now and most programmers are duct taping parts together so unit testing no longer has a place in this world. Even on business-domain level components there are often simple routines which can and should be tested. Larger routines should be built up from smaller ones. Divide-and-conquer is a fundamental engineering principle. If there are no such small-enough-to-be-tested routines, then the code is not being written with testing in mind. If all your functions are hundreds of lines long, then yeah it's hard to write unit tests!
  - exclipy 6y ago
    Occasionally, yes. But if you're job is to convert JSON to DOM nodes using React, then no. In this world, what's more likely to go wrong is that one element overlaps another when zoomed to 122% on IE 11.
- mumblemumble 6y ago
  The earliest proponents I know of - the ones writing XP books back in the '90s - had a very flexible definition of "unit" and assumed that you would vary the scale according to your needs. So some units might be low level components like data structures, but others might be large modules that included a lot of moving parts, up to and including actual databases. The world has moved on now, and "agile" no longer means "use your own best judgment." It's become a carefully catalogued list of rules and prescriptions and definitions and dictums. Which, come to think of it, is an excellent example of memetic natural selection. "Use your own best judgment" is a meme that is uniquely poorly adapted to survival in an ecosystem that consists largely of people jabbering back and forth on the Internet like we are right now. You can't base a rollicking good fun argument or a strongly worded blog post on something like that.
ensiferum 6y ago
I agree. Your unit tests should focus on testing individual pieces, the fundamental building blocks (classes and functions) that you have in your system. The changes should be isolated so when a class API changes only those unit tests would need to change. I find the key thing here to be separation of concerns. Aim for testable classes and functions and that will make you also thing about the depencies. For example I have class implementing NNT protocol. Initially one could think that this class needs to deal with IO or even worse with socket based data transfer. But if you think about it doesn't. The api can be based on an idea of getting a buffer full of data in and getting buffer Full of data out. Class coupled with IO/socket functionality = coupled hard to test without its trivial to unit test. (Reduced functionality class only maintains the protocol state doesn't concern itself where the data comes or goes) Reference. https://github.com/ensisoft/newsflash-plus/blob/master/engine/session.h https://github.com/ensisoft/newsflash-plus/blob/master/engin...