3 ms·

As the author points out, "What constitutes a unit? Is it a function? A class?" If you test to a function, or a class, then your tests imply that function or c

As the author points out, "What constitutes a unit? Is it a function? A class?" If you test to a function, or a class, then your tests imply that function or class must be present, with that specific API. In my experience, most people write unit tests for internal implementation details that are irrelevant to the business domain. They end up inhibiting code change rather than encouraging change, because anything changes often require re-evaluating each failing test to see if it was meaningful in the first place - and if 100s of tests are no longer meaningful, it's easy to skip the couple of tests which are true regression tests. As the author writes, full end-to-end tests "are often slow, cumbersome, hard to debug and tend to be flaky." Instead, find the internal interfaces which tied to the business logic ("something that delivers value to a paying client"), and write the tests to that. You can use unit test frameworks for that sort of functional testing.

wakawaka28 3y ago
>If you test to a function, or a class, then your tests imply that function or class must be present, with that specific API. So what? Every implementation will have some basic structure. That structure can be modified if needed. The point of a unit test is not to posit that any particular thing exists, but that the things that do exist actually work. >In my experience, most people write unit tests for internal implementation details that are irrelevant to the business domain. They end up inhibiting code change rather than encouraging change, because anything changes often require re-evaluating each failing test to see if it was meaningful in the first place - and if 100s of tests are no longer meaningful, it's easy to skip the couple of tests which are true regression tests. If the internal implementation is not observable without a bunch of other bullshit, these tests can help instill confidence that the stuff actually works. If you don't test, or test the overall system, it takes much longer. There is such a thing as a pointless test but it is far more common in my experience to have stuff that isn't covered at all by tests. If your biggest problem is that you have to delete some tests that you made obsolete, that's perfectly ok. >As the author writes, full end-to-end tests "are often slow, cumbersome, hard to debug and tend to be flaky." Those types of tests are not unit tests. >Instead, find the internal interfaces which tied to the business logic ("something that delivers value to a paying client"), and write the tests to that. You can use unit test frameworks for that sort of functional testing. It isn't only the business logic that needs to be tested. Anything that is cumbersome to test "enough" in the overall system ought to be unit tested. At work I'm faced with a series of structures that are cumbersome to test in isolation and in totality. If I had unit tests I could make changes at least 3x faster.
- eesmith 3y ago
  > Every implementation will have some basic structure. Setting aside functional vs OO paradigms, even if you have a simple helper function like: def removeprefix(s, prefix): if s.startswith(prefix): return s[len(prefix):] return s do you write tests for it? Or do you write tests for the higher level routines which call it? I think most write tests for the function. If you write tests for the function then you hinder future refactoring from removeprefix(s, prefix) to s.removeprefix(prefix) once you switch to a Python which implements s.removeprefix (3.9, I think?) For example, in red-green-refactor TDD, you are not supposed to change the tests when you refactor. What you've ended up with are tests for the specific structure, and not the general goals. > If you don't test, or test the overall system, it takes much longer. Good thing neither I nor the linked-to author makes either of those arguments. > it is far more common in my experience to have stuff that isn't covered at all by tests Which is why I use coverage-based method to identify what need more tests. Coverage is a powerful tool, and easily misused. > If your biggest problem is that you have to delete some tests that you made obsolete, that's perfectly ok. The biggest problem is that you decide to not refactor because there are so many tests to delete, and you have to figure out if the failing test really is okay to delete (because it was implementation specific) vs. one that needs to be updated (eg, because it handles a regression case). > Those types of tests are not unit tests Correct! And no one said they were. > At work I'm faced with a ... That's fine. You do you.
  - wakawaka28 3y ago
    >do you write tests for it? Or do you write tests for the higher level routines which call it? I think most write tests for the function. I try to not test things that are that simple. But I wouldn't fault anyone for writing a small number of test cases for it. >For example, in red-green-refactor TDD, you are not supposed to change the tests when you refactor. > >What you've ended up with are tests for the specific structure, and not the general goals. I'm not familiar with this methodology. But if your refactoring requires changing the interfaces then it must require changing the tests. It may be ideal to have such rigid interfaces in some cases, but I don't think you have to be so committed to one methodology. Big changes require big testing. >If you write tests for the function then you hinder future refactoring from removeprefix(s, prefix) to s.removeprefix(prefix) once you switch to a Python which implements s.removeprefix (3.9, I think?) How? Is the old code supposed to stop working because a new function was added? You aren't obligated to delete perfectly working and tested code. And if the new version of Python breaks it somehow, the tests will tell you quickly compared to finding out in a bigger test. >Which is why I use coverage-based method to identify what need more tests. Coverage is a powerful tool, and easily misused. Coverage is fine. But I mean, coverage is for people who already committed to testing every line of code. Not every project has unit tests in the first place so coverage is a moot point. >The biggest problem is that you decide to not refactor because there are so many tests to delete, and you have to figure out if the failing test really is okay to delete (because it was implementation specific) vs. one that needs to be updated (eg, because it handles a regression case). This work is very easy compared to fixing the real product in most cases. If you didn't write a bunch of frivolous tests it isn't that much work. >>Those types of tests are not unit tests > >Correct! And no one said they were. Well this is a discussion about unit tests. If you want to talk about other types of tests you need to be clear about that. >That's fine. You do you. I was just pointing out that unit tests are not always feasible to do. But I wish they were. The stuff I see at work was not designed by me, so it's not my fault it is infeasible to test at least.

wakawaka28 3y ago

>If you test to a function, or a class, then your tests imply that function or class must be present, with that specific API. So what? Every implementation will have some basic structure. That structure can be modified if needed. The point of a unit test is not to posit that any particular thing exists, but that the things that do exist actually work. >In my experience, most people write unit tests for internal implementation details that are irrelevant to the business domain. They end up inhibiting code change rather than encouraging change, because anything changes often require re-evaluating each failing test to see if it was meaningful in the first place - and if 100s of tests are no longer meaningful, it's easy to skip the couple of tests which are true regression tests. If the internal implementation is not observable without a bunch of other bullshit, these tests can help instill confidence that the stuff actually works. If you don't test, or test the overall system, it takes much longer. There is such a thing as a pointless test but it is far more common in my experience to have stuff that isn't covered at all by tests. If your biggest problem is that you have to delete some tests that you made obsolete, that's perfectly ok. >As the author writes, full end-to-end tests "are often slow, cumbersome, hard to debug and tend to be flaky." Those types of tests are not unit tests. >Instead, find the internal interfaces which tied to the business logic ("something that delivers value to a paying client"), and write the tests to that. You can use unit test frameworks for that sort of functional testing. It isn't only the business logic that needs to be tested. Anything that is cumbersome to test "enough" in the overall system ought to be unit tested. At work I'm faced with a series of structures that are cumbersome to test in isolation and in totality. If I had unit tests I could make changes at least 3x faster.

eesmith 3y ago
> Every implementation will have some basic structure. Setting aside functional vs OO paradigms, even if you have a simple helper function like: def removeprefix(s, prefix): if s.startswith(prefix): return s[len(prefix):] return s do you write tests for it? Or do you write tests for the higher level routines which call it? I think most write tests for the function. If you write tests for the function then you hinder future refactoring from removeprefix(s, prefix) to s.removeprefix(prefix) once you switch to a Python which implements s.removeprefix (3.9, I think?) For example, in red-green-refactor TDD, you are not supposed to change the tests when you refactor. What you've ended up with are tests for the specific structure, and not the general goals. > If you don't test, or test the overall system, it takes much longer. Good thing neither I nor the linked-to author makes either of those arguments. > it is far more common in my experience to have stuff that isn't covered at all by tests Which is why I use coverage-based method to identify what need more tests. Coverage is a powerful tool, and easily misused. > If your biggest problem is that you have to delete some tests that you made obsolete, that's perfectly ok. The biggest problem is that you decide to not refactor because there are so many tests to delete, and you have to figure out if the failing test really is okay to delete (because it was implementation specific) vs. one that needs to be updated (eg, because it handles a regression case). > Those types of tests are not unit tests Correct! And no one said they were. > At work I'm faced with a ... That's fine. You do you.
- wakawaka28 3y ago
  >do you write tests for it? Or do you write tests for the higher level routines which call it? I think most write tests for the function. I try to not test things that are that simple. But I wouldn't fault anyone for writing a small number of test cases for it. >For example, in red-green-refactor TDD, you are not supposed to change the tests when you refactor. > >What you've ended up with are tests for the specific structure, and not the general goals. I'm not familiar with this methodology. But if your refactoring requires changing the interfaces then it must require changing the tests. It may be ideal to have such rigid interfaces in some cases, but I don't think you have to be so committed to one methodology. Big changes require big testing. >If you write tests for the function then you hinder future refactoring from removeprefix(s, prefix) to s.removeprefix(prefix) once you switch to a Python which implements s.removeprefix (3.9, I think?) How? Is the old code supposed to stop working because a new function was added? You aren't obligated to delete perfectly working and tested code. And if the new version of Python breaks it somehow, the tests will tell you quickly compared to finding out in a bigger test. >Which is why I use coverage-based method to identify what need more tests. Coverage is a powerful tool, and easily misused. Coverage is fine. But I mean, coverage is for people who already committed to testing every line of code. Not every project has unit tests in the first place so coverage is a moot point. >The biggest problem is that you decide to not refactor because there are so many tests to delete, and you have to figure out if the failing test really is okay to delete (because it was implementation specific) vs. one that needs to be updated (eg, because it handles a regression case). This work is very easy compared to fixing the real product in most cases. If you didn't write a bunch of frivolous tests it isn't that much work. >>Those types of tests are not unit tests > >Correct! And no one said they were. Well this is a discussion about unit tests. If you want to talk about other types of tests you need to be clear about that. >That's fine. You do you. I was just pointing out that unit tests are not always feasible to do. But I wish they were. The stuff I see at work was not designed by me, so it's not my fault it is infeasible to test at least.