3 ms·

Allow me to even more strongly disagree. I've not yet seen a situation where strongly-tested private code would have helped, although I routinely come across c

Allow me to even more strongly disagree. I've not yet seen a situation where strongly-tested private code would have helped, although I routinely come across cases where too many tests get directly in the way of a refactor or bugfix that _would_ improve the program. These days, I'm of the opinion that you should test at the highest level possible (if testing a web service, for example, at the level of the actual public API to the service), thoroughly define the behaviours expected for those APIs, and leave the insides of the service entirely untested, unless you have _very_ good reason to. (Critical code, payment code, hard-to-test code, etc.) This ensures you have the important behaviours tested, but leaves you free to modify implementation details as necessary without needing to alter or rewrite entire swathes of test code that suddenly starts failing because you decided to break up or consolidate an inner method.

zem 3y ago
contrariwise, i have never come across a case where having tests for private functions/methods has hindered refactoring or bugfixing. that feels more like undue coupling than undue testing to me. if i have to alter or delete swaths of code i just delete the associated tests as well, and write new tests for the new implementation. perhaps relatedly, i like to build my code from the bottom up, and make sure each layer is solid so that i can use it to construct the next layer. there is often not a public api at all until i am well into the project, so for me that would involve writing fair bit of code with no tests at all simply because it was all private code. i do use the "public methods on private helper objects" pattern a lot, but there is a fair mix of times where it's the best way to write the code and times where it's just to keep the test framework happy.
seer 3y ago
this is so true. I go even further - for the browser level tests, avoid using “testid” or css classes or anything that is “implementation”, but rely in your test on solely things that the user can read / interact with. So don’t “Press button with id “generate”, but the button that says “save” inside the content element titled “generation”. This way any refactoring work would not require test changes (as it should) and any change of the visible ui / workflow to the user would require an adjustment to the test. This is a style that I learned from ruby’s integration testing framework “capybara”, and have been replicating it wherever I can since. A nice bonus is that if you switch rendering technologies, you can reuse the tests (like react native for example).
- jabradoodle 3y ago
  > avoid using “testid” > This way any refactoring work would not require test changes This is the purpose of a testid, your letting people know removing it can/will break tests. (Especially e2e tests that might not only be ran by you/ might not live alongside the code) > and any change of the visible ui / workflow to the user would require an adjustment to the test. I really don't follow this reasoning; why would I want my tests to fail when the Accept button is renamed to Agree?
diarrhea 3y ago
Agree. The core downside to this approach is pinpointing failure points. A large integration test failing can have many causes. In the same vein, technically, detailed unit tests can immediately discover these. However, in reality, having such a tight, well defined, correct web of unit tests such that any failure can be immediately traced to individual methods is unlikely and an immense amount of work.
- thom 3y ago
  An isolated failing integration test implies a better unit test could exist somewhere. But as you point out, you still probably came out ahead by not writing _all_ the possible unit tests you could have.
josephg 3y ago
Too many unit tests can slow down refactoring. But it can still be worthwhile. Eg: I’ve done some deep algorithmic work on CRDTs lately and my code has a lot of internal parts which are all quite fiddly to implement correctly. For example, I’m using a custom btree with some unusual features as an internal data structure. Btrees are famously difficult to implement correctly, so my btree has both unit tests and fuzz tests to make sure it does what I expect. Having that test suite makes integration tests easier to triage, since I know any failures in my integration tests probably don’t come from the btree. And some btree code paths are probably rarely or never executed by my integration tests. But I still want that code to be correct. Testing it in isolation is the best way. They’re a lot of other small pieces that I test like this - like saving & loading code, my graph traversal utility code (for comparing versions semantically) and so on. Low level unit testing can find bugs in utility modules while you write the module itself, not later (when you start using the module in some other code).

zem 3y ago

contrariwise, i have never come across a case where having tests for private functions/methods has hindered refactoring or bugfixing. that feels more like undue coupling than undue testing to me. if i have to alter or delete swaths of code i just delete the associated tests as well, and write new tests for the new implementation. perhaps relatedly, i like to build my code from the bottom up, and make sure each layer is solid so that i can use it to construct the next layer. there is often not a public api at all until i am well into the project, so for me that would involve writing fair bit of code with no tests at all simply because it was all private code. i do use the "public methods on private helper objects" pattern a lot, but there is a fair mix of times where it's the best way to write the code and times where it's just to keep the test framework happy.

seer 3y ago

this is so true. I go even further - for the browser level tests, avoid using “testid” or css classes or anything that is “implementation”, but rely in your test on solely things that the user can read / interact with. So don’t “Press button with id “generate”, but the button that says “save” inside the content element titled “generation”. This way any refactoring work would not require test changes (as it should) and any change of the visible ui / workflow to the user would require an adjustment to the test. This is a style that I learned from ruby’s integration testing framework “capybara”, and have been replicating it wherever I can since. A nice bonus is that if you switch rendering technologies, you can reuse the tests (like react native for example).

jabradoodle 3y ago
> avoid using “testid” > This way any refactoring work would not require test changes This is the purpose of a testid, your letting people know removing it can/will break tests. (Especially e2e tests that might not only be ran by you/ might not live alongside the code) > and any change of the visible ui / workflow to the user would require an adjustment to the test. I really don't follow this reasoning; why would I want my tests to fail when the Accept button is renamed to Agree?

diarrhea 3y ago

Agree. The core downside to this approach is pinpointing failure points. A large integration test failing can have many causes. In the same vein, technically, detailed unit tests can immediately discover these. However, in reality, having such a tight, well defined, correct web of unit tests such that any failure can be immediately traced to individual methods is unlikely and an immense amount of work.

thom 3y ago
An isolated failing integration test implies a better unit test could exist somewhere. But as you point out, you still probably came out ahead by not writing _all_ the possible unit tests you could have.

josephg 3y ago

Too many unit tests can slow down refactoring. But it can still be worthwhile. Eg: I’ve done some deep algorithmic work on CRDTs lately and my code has a lot of internal parts which are all quite fiddly to implement correctly. For example, I’m using a custom btree with some unusual features as an internal data structure. Btrees are famously difficult to implement correctly, so my btree has both unit tests and fuzz tests to make sure it does what I expect. Having that test suite makes integration tests easier to triage, since I know any failures in my integration tests probably don’t come from the btree. And some btree code paths are probably rarely or never executed by my integration tests. But I still want that code to be correct. Testing it in isolation is the best way. They’re a lot of other small pieces that I test like this - like saving & loading code, my graph traversal utility code (for comparing versions semantically) and so on. Low level unit testing can find bugs in utility modules while you write the module itself, not later (when you start using the module in some other code).