3 ms·

> When fixing a bug, add a failing test first, as a separate commit. That way it becomes easy to verify for anyone that the test indeed fails without the follow

> When fixing a bug, add a failing test first, as a separate commit. That way it becomes easy to verify for anyone that the test indeed fails without the follow up fix. I would genuinely like to know in what world this is useful advice. Maybe in personal projects or small teams or companies with a maniacal lead? I just can’t imagine someone in a sufficiently large organization (which most companies hope to be someday) goes sifting through feature commits to get to the bottom of a failing test.

pjbster 3y ago
I think the failing test is meant to help with code review. The commit would be rebased out prior to merging into main.
akoboldfrying 3y ago
I don't really understand your reaction. Adding a test that fails before and passes after the fix is good practice (do you disagree?), and putting that in a separate commit is a trivial effort (do you disagree?) It's basically the lowest-friction way I know to do the part of TDD that counts.
- chris_wot 3y ago
  Yeah, but it kills a git bisect run
  - akoboldfrying 3y ago
    How so? Adding 1 commit adds at most 1 (but usually 0) git bisect steps.
    - chris_wot 3y ago
      Yes, but if you run git bisect and then try to run your unit tests, a failure that was intentional will tell the bisect to potentially divide the commits at the wrong point. You'd need to either exclude the test (probably not a great idea) or mark the commits as skippable. Also not a great idea.
      - akoboldfrying 3y ago
        Are you suggesting using the unit tests at the HEAD selected by the current git bisect step as the binary signal for bisect? Because if so, I agree that a commit that introduces a failing test will break bisect (there will be a pass->fail transition AND a fail->pass transition, and bisecting requires there to be at most one such transition). But even more importantly: Without the separate commit containing only the failing test, every commit will pass. That is, bisect will have no useful signal at all. I think that can't be what you're suggesting, so could you clarify what bisect process you have in mind?
        chris_wot 3y ago
        Sure, there have been occassions where on a particular architecture or there is a particular set of circumstances where a test will succeed on a particular system but will fail elsewhere. If you try to bisect such a situation, you'll not be able to run git bisect run on your tests because of the failing test. I'm not against a failing test being committed, but I believe it isn't hugely productive. The test failing is surely a signal only to the developer who wrote the test that it failed? Or perhaps it should be failed on a seperate branch and then merged to the master branch, which is pristine. Every organisation has there own workflow, I suppose. Edit: a crucial bit of info I didn't note was that you keep developing and add a lot more commits before it is noticed.
F-W-M 3y ago
In one of the best teams I worked with we started doing that on our own. I still do it (not always, but often) on my projects.
hmeh 3y ago
I don’t think it is. By all means add a failing test, but I’d tend to commit it along with the code that makes it pass. This makes the commit cohesive. If I must commit it as work in progress I would, but I’d probably amend it once I had the test passing.
computerfriend 3y ago
I sift through commits to get to the bottom of failing tests.

pjbster 3y ago

I think the failing test is meant to help with code review. The commit would be rebased out prior to merging into main.

akoboldfrying 3y ago

I don't really understand your reaction. Adding a test that fails before and passes after the fix is good practice (do you disagree?), and putting that in a separate commit is a trivial effort (do you disagree?) It's basically the lowest-friction way I know to do the part of TDD that counts.

chris_wot 3y ago
Yeah, but it kills a git bisect run
- akoboldfrying 3y ago
  How so? Adding 1 commit adds at most 1 (but usually 0) git bisect steps.
  - chris_wot 3y ago
    Yes, but if you run git bisect and then try to run your unit tests, a failure that was intentional will tell the bisect to potentially divide the commits at the wrong point. You'd need to either exclude the test (probably not a great idea) or mark the commits as skippable. Also not a great idea.
    - akoboldfrying 3y ago
      Are you suggesting using the unit tests at the HEAD selected by the current git bisect step as the binary signal for bisect? Because if so, I agree that a commit that introduces a failing test will break bisect (there will be a pass->fail transition AND a fail->pass transition, and bisecting requires there to be at most one such transition). But even more importantly: Without the separate commit containing only the failing test, every commit will pass. That is, bisect will have no useful signal at all. I think that can't be what you're suggesting, so could you clarify what bisect process you have in mind?
      - chris_wot 3y ago
        Sure, there have been occassions where on a particular architecture or there is a particular set of circumstances where a test will succeed on a particular system but will fail elsewhere. If you try to bisect such a situation, you'll not be able to run git bisect run on your tests because of the failing test. I'm not against a failing test being committed, but I believe it isn't hugely productive. The test failing is surely a signal only to the developer who wrote the test that it failed? Or perhaps it should be failed on a seperate branch and then merged to the master branch, which is pristine. Every organisation has there own workflow, I suppose. Edit: a crucial bit of info I didn't note was that you keep developing and add a lot more commits before it is noticed.

F-W-M 3y ago

In one of the best teams I worked with we started doing that on our own. I still do it (not always, but often) on my projects.

hmeh 3y ago

I don’t think it is. By all means add a failing test, but I’d tend to commit it along with the code that makes it pass. This makes the commit cohesive. If I must commit it as work in progress I would, but I’d probably amend it once I had the test passing.

computerfriend 3y ago

I sift through commits to get to the bottom of failing tests.