4 ms·

I think part of what you are getting at here also points to differences in what people mean by “unit test”. It’s always possible to write a test case that cove

I think part of what you are getting at here also points to differences in what people mean by “unit test”. It’s always possible to write a test case that covers a new high-level functional requirement as you understand it; part of the skill of test-first (disclaimer - I use this approach sometimes but not religiously and don’t consider myself a master at this) is identifying the best next test to write. But a lot of people cast “unit test” as “test for each method on a class” which is too low-level and coupled to the implementation; if you are writing those sort of UTs then in some sense you are doing gradient descent with a too-small step size. There is no appreciable gradient to move down; adding a new test for a small method doesn’t always get you closer to adding the next meaningful bit of functionality. When I have done best with TDD is when I start with what most would call “functional tests” and test the behaviors, which is isomorphic to the design process of working with stakeholders to think through all the ways the product should react to inputs. I think the early TDD guys like Kent Beck probably assumed you are sitting next to a stakeholder so that you can rapidly iterate on those business/product/domain questions as you proceed. There is no “upfront spec” in agile, the process of growing an implementation leads you to the next product question to ask.

merlincorey 4y ago
> But a lot of people cast “unit test” as “test for each method on a class” which is too low-level and coupled to the implementation; if you are writing those sort of UTs then in some sense you are doing gradient descent with a too-small step size. There is no appreciable gradient to move down; adding a new test for a small method doesn’t always get you closer to adding the next meaningful bit of functionality. In my experience, the best time to do "test for each method on a class" or "test for each function in a module" is when the component in question is a low level component in the system that must be relied upon for correctness by higher level parts of the system. Similarly, in my experience, it is often a waste of effort and time to do such thorough low level unit testing on higher level components composed of multiple lower level components. In those cases, I find it's much better to write unit tests at the highest level possible (i.e. checking `module.top_level_super_function()` inputs produce expected outputs or side effects)
- remexre 4y ago
  > is when the component in question is a low level component in the system that must be relied upon for correctness by higher level parts of the system. And then, property tests are more likely to be possible, and IMO should be preferred!
  - ENGNR 4y ago
    TTRD test driven re-design?
    - knicholes 4y ago
      TDRD
      - dwattttt 4y ago
        The first acronym cleverly demonstrates the redesign
mkl95 4y ago
> But a lot of people cast “unit test” as “test for each method on a class” which is too low-level and coupled to the implementation; Those tests suit a project that applies the open-closed principle strictly, such as libraries / packages that will rarely be modified directly and will mostly be used by "clients" as their building blocks. They don't suit a spaghetti monolith with dozens of leaky APIs that change on every sprint. The harsh truth is that in the industry you are more likely to work with spaghetti code than with stable packages. "TDD done right" is a pipe dream for the average engineer.
P5fRxh5kUvp2th 4y ago
The above poster used 'TDD', not 'unit test', they are not the same thing. You can (and often should!) have a suite of unit tests, but you can choose to write them after the fact, and after the fact means after most of the exploration is done. I think if most people stopped thinking of unit tests as a correctness mechanism and instead thought of them as a regression mechanism unit tests as a whole would be a lot better off.
- adhesive_wombat 4y ago
  Also as an dependency canary: when your low level object tests start demanding access to databases and config files and networking, it's time for a think. Also a passing unit test always provides up-to-date implicit documentation on how to use the tested code.
- geophile 4y ago
  None of this either/or reasoning is correct, in my experience. In practice, I write tests both before and after implementation, for different reasons. In practice, my tests both test correctness, and of course they also work as regression tests. Writing before the fact allows you to test your mental model of the interface, unspoiled by having the implementation fresh in your mind. (Not entirely, since you probably have some implementation ideas very early on.) Writing tests after the fact is what you must do to explore 1) weak points that occur to you as you implement, and 2) bugs. After-the-fact testing also allows you to hone in on vagueness in the spec, which may show up as (1) or (2).
  - P5fRxh5kUvp2th 4y ago
    There's nothing binary about the observation that unit test and TDD are different sets.
- savolai 4y ago
  Hi. I got curious. Do you mean ”regression detection mehanism”? Could you elaborate? Thanks.
commandlinefan 4y ago
> a lot of people cast “unit test” as “test for each method on a class” which is too low-level Definitely agree with you here - I've seen people dogmatically write unit tests for getter and setter methods at which point I have a hard time believing they're not just fucking with me. However, there's a "sweet spot" in between writing unit tests on every single function and writing "unit tests" that don't run without a live database and a few configuration files in specific locations, which (in my experience) is more common when you ask a mediocre programmer to try to write some tests.
- icedchai 4y ago
  I'm having flashbacks to a previous workplace. I was literally asked to write unit tests for getters and setters. I complained they were used elsewhere in the code, and therefore tested indirectly anyway. Nope, my PR would not be "approved" until I tested every getter and setter. I think I lasted there about 6 months.
slevcom 4y ago
This is well said. I always suspect that many people who have a hard time relating to TDD already have experience writing these class & method oriented tests. So they understandably struggle with trying to figure out how to write them before writing the code. Thinking about tests in terms of product features is how it clicked for me. That being said, as another poster above mentioned, using TDD for unstable or exploratory features is often unproductive. But that’s because tests for uncertain features are often unproductive, regardless if you wrote them before or after. I once spent months trying to invent a new product using TDD. I was constantly deleting tests because I was constantly changing features. Even worse, I found myself resisting changing features that needed changing because I was attached to the work I had done to test them. I eventually gave up. I still use TDD all the time, but not when I’m exploring new ideas.
- majikandy 4y ago
  I do the same. But almost always wish I had done TDD. The times I can bring myself to git reset —-hard after I have finished exploring and then TDD it in, the code benefits. Often though I can’t bring myself to do it and I retro fit in a few tests and reorder the commit history :)

merlincorey 4y ago

> But a lot of people cast “unit test” as “test for each method on a class” which is too low-level and coupled to the implementation; if you are writing those sort of UTs then in some sense you are doing gradient descent with a too-small step size. There is no appreciable gradient to move down; adding a new test for a small method doesn’t always get you closer to adding the next meaningful bit of functionality. In my experience, the best time to do "test for each method on a class" or "test for each function in a module" is when the component in question is a low level component in the system that must be relied upon for correctness by higher level parts of the system. Similarly, in my experience, it is often a waste of effort and time to do such thorough low level unit testing on higher level components composed of multiple lower level components. In those cases, I find it's much better to write unit tests at the highest level possible (i.e. checking `module.top_level_super_function()` inputs produce expected outputs or side effects)

remexre 4y ago
> is when the component in question is a low level component in the system that must be relied upon for correctness by higher level parts of the system. And then, property tests are more likely to be possible, and IMO should be preferred!
- ENGNR 4y ago
  TTRD test driven re-design?
  - knicholes 4y ago
    TDRD
    - dwattttt 4y ago
      The first acronym cleverly demonstrates the redesign

mkl95 4y ago

> But a lot of people cast “unit test” as “test for each method on a class” which is too low-level and coupled to the implementation; Those tests suit a project that applies the open-closed principle strictly, such as libraries / packages that will rarely be modified directly and will mostly be used by "clients" as their building blocks. They don't suit a spaghetti monolith with dozens of leaky APIs that change on every sprint. The harsh truth is that in the industry you are more likely to work with spaghetti code than with stable packages. "TDD done right" is a pipe dream for the average engineer.

P5fRxh5kUvp2th 4y ago

The above poster used 'TDD', not 'unit test', they are not the same thing. You can (and often should!) have a suite of unit tests, but you can choose to write them after the fact, and after the fact means after most of the exploration is done. I think if most people stopped thinking of unit tests as a correctness mechanism and instead thought of them as a regression mechanism unit tests as a whole would be a lot better off.

adhesive_wombat 4y ago
Also as an dependency canary: when your low level object tests start demanding access to databases and config files and networking, it's time for a think. Also a passing unit test always provides up-to-date implicit documentation on how to use the tested code.
geophile 4y ago
None of this either/or reasoning is correct, in my experience. In practice, I write tests both before and after implementation, for different reasons. In practice, my tests both test correctness, and of course they also work as regression tests. Writing before the fact allows you to test your mental model of the interface, unspoiled by having the implementation fresh in your mind. (Not entirely, since you probably have some implementation ideas very early on.) Writing tests after the fact is what you must do to explore 1) weak points that occur to you as you implement, and 2) bugs. After-the-fact testing also allows you to hone in on vagueness in the spec, which may show up as (1) or (2).
- P5fRxh5kUvp2th 4y ago
  There's nothing binary about the observation that unit test and TDD are different sets.
savolai 4y ago
Hi. I got curious. Do you mean ”regression detection mehanism”? Could you elaborate? Thanks.

commandlinefan 4y ago

> a lot of people cast “unit test” as “test for each method on a class” which is too low-level Definitely agree with you here - I've seen people dogmatically write unit tests for getter and setter methods at which point I have a hard time believing they're not just fucking with me. However, there's a "sweet spot" in between writing unit tests on every single function and writing "unit tests" that don't run without a live database and a few configuration files in specific locations, which (in my experience) is more common when you ask a mediocre programmer to try to write some tests.

icedchai 4y ago
I'm having flashbacks to a previous workplace. I was literally asked to write unit tests for getters and setters. I complained they were used elsewhere in the code, and therefore tested indirectly anyway. Nope, my PR would not be "approved" until I tested every getter and setter. I think I lasted there about 6 months.

slevcom 4y ago

This is well said. I always suspect that many people who have a hard time relating to TDD already have experience writing these class & method oriented tests. So they understandably struggle with trying to figure out how to write them before writing the code. Thinking about tests in terms of product features is how it clicked for me. That being said, as another poster above mentioned, using TDD for unstable or exploratory features is often unproductive. But that’s because tests for uncertain features are often unproductive, regardless if you wrote them before or after. I once spent months trying to invent a new product using TDD. I was constantly deleting tests because I was constantly changing features. Even worse, I found myself resisting changing features that needed changing because I was attached to the work I had done to test them. I eventually gave up. I still use TDD all the time, but not when I’m exploring new ideas.

majikandy 4y ago
I do the same. But almost always wish I had done TDD. The times I can bring myself to git reset —-hard after I have finished exploring and then TDD it in, the code benefits. Often though I can’t bring myself to do it and I retro fit in a few tests and reorder the commit history :)