6 ms·

I must be one of the few who think this sort of unit-testing is necessary to a certain extent, but in practice seems rather to lend itself to causing terrible c

by sidlls 10y ago

I must be one of the few who think this sort of unit-testing is necessary to a certain extent, but in practice seems rather to lend itself to causing terrible code smells all over the place: things like small one-liner functions for conditionals and basic arithmetic that are designed with mock- and test-ability in mind (rather than whether or not they make the code maintainable or correct), factoring code specifically to increase "code coverage" of the tests, etc. I think integration and other coarse-grained tests are far more useful more often, and using asserts (which will cause these coarse-grained tests to fail, too) is a better pattern than "unit test all the things".

jwilliams 10y ago
I've seen this in practice too. Lots of unit tests for small things that actually make the overall codebase brittle. I think it comes from a pretty decent practice of coding from the tests outward. However I'd argue the long term value of many of those tests is minimal.
- adrianratnapala 10y ago
  Yes. I write lots of unit tests, but they are mostly part of the coding process: I find they are superior to the manual testing I would have to do otherwise every time I add new code. But as long term protections against regressions, such tests are less valuable than more coarse-grained integration tests.
deleted 10y ago
[deleted]
cytzol 10y ago
> things like small one-liner functions for conditionals and basic arithmetic that are designed with mock- and test-ability in mind I've done this before. It's very easy to want to make all your function/method calls work the same. To stop, I applied the following rule whenever I had to choose between making a function static or mockable: does this function's name completely describe its implementation? Small methods like "StringUtils::isEmptyOrWhitespace" pass this test: it's just going to check if the string is empty or only contains whitespace, and that doesn't need to have its implementation swapped out. On the other side, "ImageUtils::detectFacesInImage" goes into no detail about how it does the face detection, and definitely needs an interface to test against. That said, it all depends. You might allow your users to pick which characters count as whitespace, I don't know. > I think integration and other coarse-grained tests are far more useful more often! Yes! Integration tests are the "main" tests. Unit tests are secondary. Use them when you need to debug why your integration tests fail.
JamesBarney 10y ago
I've heard these change to design tradeoffs that occur to increase testability, test induced design damage. Enterprise craftsman I believe
- sidlls 10y ago
  This sort of rigid adherence to a particular paradigm isn't restricted to enterprise developers. I've seen this sort of thing in practice (or attempted) from colleagues in data pipelines for machine learning and other very much not-enterprise settings.
  - adrianratnapala 10y ago
    I find such religious devotion rules common among scientists who have to code even though they are not programmers by trade. Religious overcommenting etc. Whatever rule they happened to read in their FORTRAN-for-beginners book. ML people are probably more knowledgeable, but their job is also often more like science than traditional software engineering.
    - sidlls 10y ago
      That doesn't match my experience (as a former academic scientist who moved into software development). The governing rule for us was always "publish or perish." Programming was a means (exploring a problem) to an end (advancing knowledge and publishing the results). I, at least, never came across the jargon for testing, maintenance, etc. until I moved out of academia and into this industry.
      - adrianratnapala 10y ago
        Oh I agree, scientists are far less inundated by software "best practices" than professionals. It's just that those which they have heard of, they often stick to religiously. Personally I prefer the results of these scientists to much of what I see from real software engineers. It might be wreck, but it is only 1000 lines of wreck -- and not a sophisticated wreck created with powerful modern tools.
BeetleB 10y ago
>but in practice seems rather to lend itself to causing terrible code smells all over the place I'm working on a code base of about half a million lines at work. We have lots of functional/integration tests, but no unit tests, and the code (C++) is not designed for it. To add unit tests, we would have to rearchitect a lot of the code. Last year, as a side project, I took some of our code, rearchitected it, and added unit tests as a proof of concept. Since I knew I would have to convince management (essentially justifying a lot of churn to add these tests), I had a simple rule: Every change I make to add unit tests should have some value even if we never add unit tests. This helped me focus on code changes that I could objectively say improved the code quality, and helped me reject changes that were "just for unit testing".
ptero 10y ago
You are not alone -- to me, unit tests only make sense when there already exists a well designed set of higher level tests. Otherwise, unit tests produce a large quantity of outputs that does not really help understand the core of the problem.
curyous 10y ago
Great to see this view, it meshes with my experience. Making your code testable does terrible things to the design.
pooktrain 10y ago
Trying to cover your state space with coarse-grained tests in database-backed applications can lead to slow tests suites that are laborious to set up/tear down, which is a barrier to fast build times in CI. This harms productivity both while writing tests and in deployment. J.B Rainsberger says it best: https://www.youtube.com/watch?v=VDfX44fZoMc https://www.youtube.com/watch?v=VDfX44fZoMc
- crdoconnor 10y ago
  I've heard a lot of people say this but I don't find it to be true. Realistic and loosely coupled tests that require minimal maintenance improve my productivity much, much more than a fast build time does. JD Rainsberger's proscriptions in particular sound like a recipe for creating a massive amount of test code, a huge maintenance headache and tests that are not at all realistic.
  - pooktrain 10y ago
    I'm not sure what you mean by "Realistic" tests. Perhaps you mean integrated tests that exercise, say, the entire stack from handling the HTTP request down to the database, if we're talking about a REST API. If you don't care much about covering large portions of the state space and your tolerance for defects is high, then I'm sure your claim about improved productivity is true. My tolerance for defects is low, which compels me to try and cover the state space more - happy paths, unhappy paths, etc. I've tried doing this with integrated tests, and found it to be very difficult to do in a timely manner. When I switched to using DI, coding to interfaces, and using mocks across architectural boundaries, my ability to cover the state space went up, and my time to deliver code with few defects went down. I think this is because often the code I care about exercising is limited to a small section of the call stack. In an integrated test, I have to set up too much state and have the machine do too much irrelevant work just to verify this small portion of the code. With DI, interfaces, and mocks, I can isolate just this behavior much more quickly. That leaves coupling and maintenance. This is a valid concern in the mockist style, but I haven't found it to be a limiting factor in maintaining my code. I think it comes down to testing the right things. If you're using mocks to verify a contract between two objects, and you want to change that contract, naturally your tests for the contract will break. In that case, I think it's fine to just delete them and write new tests that establish your contract. This kind of approach isn't right in all situations, but mockist style testing has helped my write some stable code that would have been a real pain to test due to external dependencies otherwise. EDIT: That is not to say that your tolerance for defects is high, but rather that productivity is relative to your goals. Sometimes the time spent preventing defects and keeping maintainability high isn't worth it if the repercussions don't translate to dollars and cents.

by sidlls 10y ago

jwilliams 10y ago

I've seen this in practice too. Lots of unit tests for small things that actually make the overall codebase brittle. I think it comes from a pretty decent practice of coding from the tests outward. However I'd argue the long term value of many of those tests is minimal.

adrianratnapala 10y ago
Yes. I write lots of unit tests, but they are mostly part of the coding process: I find they are superior to the manual testing I would have to do otherwise every time I add new code. But as long term protections against regressions, such tests are less valuable than more coarse-grained integration tests.

deleted 10y ago

[deleted]

cytzol 10y ago

> things like small one-liner functions for conditionals and basic arithmetic that are designed with mock- and test-ability in mind I've done this before. It's very easy to want to make all your function/method calls work the same. To stop, I applied the following rule whenever I had to choose between making a function static or mockable: does this function's name completely describe its implementation? Small methods like "StringUtils::isEmptyOrWhitespace" pass this test: it's just going to check if the string is empty or only contains whitespace, and that doesn't need to have its implementation swapped out. On the other side, "ImageUtils::detectFacesInImage" goes into no detail about how it does the face detection, and definitely needs an interface to test against. That said, it all depends. You might allow your users to pick which characters count as whitespace, I don't know. > I think integration and other coarse-grained tests are far more useful more often! Yes! Integration tests are the "main" tests. Unit tests are secondary. Use them when you need to debug why your integration tests fail.

JamesBarney 10y ago

I've heard these change to design tradeoffs that occur to increase testability, test induced design damage. Enterprise craftsman I believe

sidlls 10y ago
This sort of rigid adherence to a particular paradigm isn't restricted to enterprise developers. I've seen this sort of thing in practice (or attempted) from colleagues in data pipelines for machine learning and other very much not-enterprise settings.
- adrianratnapala 10y ago
  I find such religious devotion rules common among scientists who have to code even though they are not programmers by trade. Religious overcommenting etc. Whatever rule they happened to read in their FORTRAN-for-beginners book. ML people are probably more knowledgeable, but their job is also often more like science than traditional software engineering.
  - sidlls 10y ago
    That doesn't match my experience (as a former academic scientist who moved into software development). The governing rule for us was always "publish or perish." Programming was a means (exploring a problem) to an end (advancing knowledge and publishing the results). I, at least, never came across the jargon for testing, maintenance, etc. until I moved out of academia and into this industry.
    - adrianratnapala 10y ago
      Oh I agree, scientists are far less inundated by software "best practices" than professionals. It's just that those which they have heard of, they often stick to religiously. Personally I prefer the results of these scientists to much of what I see from real software engineers. It might be wreck, but it is only 1000 lines of wreck -- and not a sophisticated wreck created with powerful modern tools.

BeetleB 10y ago

>but in practice seems rather to lend itself to causing terrible code smells all over the place I'm working on a code base of about half a million lines at work. We have lots of functional/integration tests, but no unit tests, and the code (C++) is not designed for it. To add unit tests, we would have to rearchitect a lot of the code. Last year, as a side project, I took some of our code, rearchitected it, and added unit tests as a proof of concept. Since I knew I would have to convince management (essentially justifying a lot of churn to add these tests), I had a simple rule: Every change I make to add unit tests should have some value even if we never add unit tests. This helped me focus on code changes that I could objectively say improved the code quality, and helped me reject changes that were "just for unit testing".

ptero 10y ago

You are not alone -- to me, unit tests only make sense when there already exists a well designed set of higher level tests. Otherwise, unit tests produce a large quantity of outputs that does not really help understand the core of the problem.

curyous 10y ago

Great to see this view, it meshes with my experience. Making your code testable does terrible things to the design.

pooktrain 10y ago

Trying to cover your state space with coarse-grained tests in database-backed applications can lead to slow tests suites that are laborious to set up/tear down, which is a barrier to fast build times in CI. This harms productivity both while writing tests and in deployment. J.B Rainsberger says it best: https://www.youtube.com/watch?v=VDfX44fZoMc https://www.youtube.com/watch?v=VDfX44fZoMc

crdoconnor 10y ago
I've heard a lot of people say this but I don't find it to be true. Realistic and loosely coupled tests that require minimal maintenance improve my productivity much, much more than a fast build time does. JD Rainsberger's proscriptions in particular sound like a recipe for creating a massive amount of test code, a huge maintenance headache and tests that are not at all realistic.
- pooktrain 10y ago
  I'm not sure what you mean by "Realistic" tests. Perhaps you mean integrated tests that exercise, say, the entire stack from handling the HTTP request down to the database, if we're talking about a REST API. If you don't care much about covering large portions of the state space and your tolerance for defects is high, then I'm sure your claim about improved productivity is true. My tolerance for defects is low, which compels me to try and cover the state space more - happy paths, unhappy paths, etc. I've tried doing this with integrated tests, and found it to be very difficult to do in a timely manner. When I switched to using DI, coding to interfaces, and using mocks across architectural boundaries, my ability to cover the state space went up, and my time to deliver code with few defects went down. I think this is because often the code I care about exercising is limited to a small section of the call stack. In an integrated test, I have to set up too much state and have the machine do too much irrelevant work just to verify this small portion of the code. With DI, interfaces, and mocks, I can isolate just this behavior much more quickly. That leaves coupling and maintenance. This is a valid concern in the mockist style, but I haven't found it to be a limiting factor in maintaining my code. I think it comes down to testing the right things. If you're using mocks to verify a contract between two objects, and you want to change that contract, naturally your tests for the contract will break. In that case, I think it's fine to just delete them and write new tests that establish your contract. This kind of approach isn't right in all situations, but mockist style testing has helped my write some stable code that would have been a real pain to test due to external dependencies otherwise. EDIT: That is not to say that your tolerance for defects is high, but rather that productivity is relative to your goals. Sometimes the time spent preventing defects and keeping maintainability high isn't worth it if the repercussions don't translate to dollars and cents.