3 ms·

How do tests written after you've changed the code prove the state of the system prior to the change? Or put another way, what technique do you use to ensure t

by BrianHV 17y ago

How do tests written after you've changed the code prove the state of the system prior to the change? Or put another way, what technique do you use to ensure the test is exercising what you think it is? I understand what you mean about inertia. If I'm not sure what I want an API to look like, I'll sometimes mess around with it without tests, then discard the work and rewrite with tests. I suppose post-hoc testing could be equivalent, but I find I usually write the code better the second time.

dalke 17y ago
I think I pointed that out in my essay when I discussed how one limitation with TDD is that tests must fail, then change the code to pass the test. This doesn't let you add tests which are expected to pass but which are used to assess the validity of the code. One way is make the tests fail, such as by changing the comparison from the expected value to something else, or by forcing an exception at the end of the code. Once the test fails then you know the test is being executed against the expected code, so then change it to make it pass. This is done without modifying the code. It's similar to what you would have to do if you depend on a third-party package which might be buggy. Code coverage is another way to check if the test is actually exercising the code paths you think it should. And yes, in some cases I will insert failures into the code itself, to convince myself that the test is also working, but that's only one of several possibilities. While you make only a single iteration when developing APIs, I find that I many more than that to get the API to feel right, and that API development needs feedback based on implementation. I'll end up with a bunch of semi-tested but reasonable code, which I then test through unit tests, guided by my experience and assisted through code coverage (either manual or automated). It is much easier to make this code work right than it would be to start from scratch. I'm also not sure that I would say "post hoc", since it really is a mixture of different types and quality of tests all throughout the development process.
- BrianHV 17y ago
  Thanks for this. I'm going to start paying attention to times when I want to change the API but feel inhibited by the tests. I haven't noticed it happening in the past, but it's possible I dismissed the idea of changing the API quickly enough that it just flew past me.
- joevandyk 17y ago
  When the code needs additional functionality, then yes, I'd say that TDD indicates you should create a failing test case. But if you are adding additional test cases to prove that edge cases work correctly (and no code changes need to happen), then I don't see why you'd need to have a failing test case, and I don't know how you could argue for it.
  - dalke 17y ago
    "I don't see why you'd need to have a failing test case". It's going to depend on the case, but I've had enough times in Python where I've written (via copy & paste) a new test case and forgotten to change the name, so the new test replaces the old, that I slightly wary of trusting that new test code which seems to pass, actually did run. On the other hand, if I have two parallel lists, one with input parameters and the other with expected results, and I add a new test case to those lists, then I'm not going to worry so much about the new test passing by accident because the likelihood of making an silent error there is quite small.
wgj 17y ago
Under TDD you write tests not after you've changed code, but before you have written any code at all. Under TDD, all tests initially fail, as there is no code yet to allow the test case to succeed. You then write the code necessary to allow the tests to succeed. This is why tests in TDD are commonly referred to as a 'spec.' They serve as the actual specification for what to write next. My take on it is that this is useful when implementing a well-known spec, like a communications protocol. It is less useful in an iterative design where the assumptions of the spec may not hold true at all later.

by BrianHV 17y ago

dalke 17y ago

I think I pointed that out in my essay when I discussed how one limitation with TDD is that tests must fail, then change the code to pass the test. This doesn't let you add tests which are expected to pass but which are used to assess the validity of the code. One way is make the tests fail, such as by changing the comparison from the expected value to something else, or by forcing an exception at the end of the code. Once the test fails then you know the test is being executed against the expected code, so then change it to make it pass. This is done without modifying the code. It's similar to what you would have to do if you depend on a third-party package which might be buggy. Code coverage is another way to check if the test is actually exercising the code paths you think it should. And yes, in some cases I will insert failures into the code itself, to convince myself that the test is also working, but that's only one of several possibilities. While you make only a single iteration when developing APIs, I find that I many more than that to get the API to feel right, and that API development needs feedback based on implementation. I'll end up with a bunch of semi-tested but reasonable code, which I then test through unit tests, guided by my experience and assisted through code coverage (either manual or automated). It is much easier to make this code work right than it would be to start from scratch. I'm also not sure that I would say "post hoc", since it really is a mixture of different types and quality of tests all throughout the development process.

BrianHV 17y ago
Thanks for this. I'm going to start paying attention to times when I want to change the API but feel inhibited by the tests. I haven't noticed it happening in the past, but it's possible I dismissed the idea of changing the API quickly enough that it just flew past me.
joevandyk 17y ago
When the code needs additional functionality, then yes, I'd say that TDD indicates you should create a failing test case. But if you are adding additional test cases to prove that edge cases work correctly (and no code changes need to happen), then I don't see why you'd need to have a failing test case, and I don't know how you could argue for it.
- dalke 17y ago
  "I don't see why you'd need to have a failing test case". It's going to depend on the case, but I've had enough times in Python where I've written (via copy & paste) a new test case and forgotten to change the name, so the new test replaces the old, that I slightly wary of trusting that new test code which seems to pass, actually did run. On the other hand, if I have two parallel lists, one with input parameters and the other with expected results, and I add a new test case to those lists, then I'm not going to worry so much about the new test passing by accident because the likelihood of making an silent error there is quite small.

wgj 17y ago

Under TDD you write tests not after you've changed code, but before you have written any code at all. Under TDD, all tests initially fail, as there is no code yet to allow the test case to succeed. You then write the code necessary to allow the tests to succeed. This is why tests in TDD are commonly referred to as a 'spec.' They serve as the actual specification for what to write next. My take on it is that this is useful when implementing a well-known spec, like a communications protocol. It is less useful in an iterative design where the assumptions of the spec may not hold true at all later.