5 ms·

100%. Apparently relatively few have experienced the power of `git bisect run`. More than once I've had to explain that no, this isn't just some idiosyncratic a

100%. Apparently relatively few have experienced the power of `git bisect run`. More than once I've had to explain that no, this isn't just some idiosyncratic aesthetic preference, there's a reason your version control tooling is designed the way it is.

enchiridion 6y ago
I'm afraid I might be one of those people. Care to share? Always looking to level up!
- renewiltord 6y ago
  If you have a branch which always builds and tests green, you can add a new test, choose a red commit and a green commit and run `git bisect` in a loop to automatically get to the commit that caused the behaviour in that test to regress.
  - renox 6y ago
    Did just that last week.. But for some reason, each time I share the result of a git bisect some thinks that I'm a wizard and instead of learning how to do it themselves they ask me later.. Thanks but its git bisect the magic tool not me (writing the shell script did take me some times though).
- nominatronic 6y ago
  Say someone finds a bug in your software. They're running release 4.1.2. They'd just upgraded from 2.6.4, which is three years old, where what they're doing used to work. You put together a test case and sure enough, it fails on 4.1.2 but passes on 2.6.4. But when did it get broken, and how? The software is complex and you can't see an obvious problem. And looking at the history, there's 4000 commits between those two tagged releases. It would take forever to test them all. You can do something smarter, though. You pick a point somewhere in the middle. Maybe it's around version 3.2.0. You test that version, and it passes. So - assuming the bug only got introduced once - the problematic commit lies somewhere between 3.2.0 and 4.1.2. You pick a version between those, and repeat the process. Rather than testing 4000 different versions, you only have to test about log2(4000) = ~12 versions of the code. The git bisect command is designed to help you with this. If you run the following commands: $ git bisect start $ git checkout v3.2.0 $ <run test, which passes> $ git bisect good $ git checkout v4.1.2 $ <run test, which fails> $ git bisect bad ...then git bisect will take over, choose the commit half way between those, check it out for you and prompt you to run your tests and report the result with 'git bisect good' or 'git bisect bad'. Then depending on the result, it will choose the next commit to test, and repeat the process until it can tell you exactly which commit introduced the bug. But maybe your test takes a while to run. Even with fewer intermediate versions to test, you're going to spend all day running tests, waiting for the results and telling git bisect what to do next. So you automate your test into a little script. If the test passes, it exits with a result of zero (success). If it fails, it returns non-zero. And then you run: $ git bisect run ./my-test-script and go do something else with your day. When you come back, it will have automatically found the commit that introduced the error. Magic! But there's a cost. For this to work, you need discipline from day one. You can't have commits in the history that say "WIP, changed some stuff, not finished yet" and others that fix things up later in the branch. You need to make sure each commit is a fully self-contained change that leaves the code in a working, testable state. If you have a small number of individual commits at which the code won't build cleanly, or which break testing for whatever reason, you can work around them with 'git bisect skip' - and you can implement this in your test script with a special return code. But if this happens too much, the whole approach becomes unmanageable.
  - Aeolun 6y ago
    So if we squash every merge request we end up with the desired behavior as well.
    - lstamour 6y ago
      Sort of. The only issue with merges in GitHub tends to be that merge or squashed commit itself isn’t tested before being committed. It’s always possible, though in practice this should be quite rare, that the merge or squash commit produces a different result than any of the individual commits did. I suppose you could do something like a pre-push hook that would run tests before sharing your code with everyone else, though I haven’t looked recently at what options are available with GitHub. Edit: I stand somewhat corrected, seems most CI systems actually test the merged code - https://github.com/actions/checkout/issues/15#issuecomment-524107344 https://github.com/actions/checkout/issues/15#issuecomment-5... which I presume includes manually merged scenarios also. That said, they don’t appear to test squashed commits, under the assumption I suppose that any series of sequential changes will always cleanly squash with upstream as long as there are no merge conflicts. When you use squash-and-merge approaches to GitHub PRs, you have to make sure you automatically delete old branches, as the connection between branch and commit only lives within the GitHub PR at that point (and maybe in a commit comment somewhere, not sure). But it’s not as explicit as merges which include the name of the branch in the merge commit.
      - bJGVygG7MQVF8c 6y ago
        > seems most CI systems actually test the merged code The thrust of your point is right, though. CI tests the merged squash-commit after it's already on master, so you can still get into states where you have a failing build on master. That risk is mitigated somewhat if you block the merging of out-of-date PR branches but this isn't supported on all the platforms.
  - enchiridion 6y ago
    Thank you for the detailed response!
XorNot 6y ago
Higher level tooling doesn't prioritize this. AFAIK neither Gitlab nor Github have an actual option in their CI implementations for "test every commit before allowing merge". And both of those are everywhere. There's also nothing out there which enforces practice - i.e. new code must be covered by some amount of test case (I'll settle for "it is executed in some way as part of build"). These are all things that could be done, but they're not priorities to the big players evidently, and good luck to me trying to get any organization to commit to putting that tooling in or running something custom that does it.
- bJGVygG7MQVF8c 6y ago
  Higher level tooling doesn't prioritize this. That's an excellent point. It's deep in the DNA of GitHub, unfortunately. I'm always reminded of this contrast: > Look, I don’t give two shits about my Git commits. [...] > From GitHub’s perspective, individual commits become less valuable because the atomic unit is the pull request. Utter Disregard for Git Commit History https://zachholman.com/posts/git-commit-history/ https://zachholman.com/posts/git-commit-history/ (The problem with this "unit of change" theorizing is obvious, especially considering GH only recently added a squash-merge feature.) vs. Torvalds on GitHub PRs: > Git comes with a nice pull-request generation module, but github instead decided to replace it with their own totally inferior version. As a result, I consider github useless for these kinds of things. It's fine for hosting, but the pull requests and the online commit editing, are just pure garbage. https://github.com/torvalds/linux/pull/17#issuecomment-5654674 https://github.com/torvalds/linux/pull/17#issuecomment-56546...
  - hinkley 6y ago
    Commit squashing is a scourge. I sometimes go the other way; rewrite my history as a set of independent refactors.
- Aeolun 6y ago
  Why do we want to test every commit? Does it matter if you broke something if you end up fixing it later?
ajuc 6y ago
Learning git bisect starts being worth it if you have more than let's say 8 commits between the last good version and first bad version, and usually you don't (in small projects at least).
- bJGVygG7MQVF8c 6y ago
  > Learning git bisect starts being worth it Worth what? The time invested to learn it? It takes 15 minutes to learn, conservatively. More charitably, if you mean the time and energy required to modify one's habits: they're good habits Brent. > if you have more than let's say 8 commits between the last good version and first bad version, and usually you don't How anyone could even hypothetically arrive at this result is beyond me.
  - cerberusss 6y ago
    > How anyone could even hypothetically arrive at this result is beyond me Well, for a handful of commits, it's very easy to do a "manual binary search" for the commit that caused a bug. Not the OP but that's how I read the comment.
  - ajuc 6y ago
    > How anyone could even hypothetically arrive at this result is beyond me. Most of the time bugs are fixed shortly after they are introduced. So there's few commits between the last known good version and first known bad version. So people can intuitively check some version in between and do bisect that way, exploiting their knowledge to cut some steps. For example if there's 8 commits but 5 of them only change modules that have nothing to do with the bug. Also if these 8 commits are very small you can treat them as a whole and look at all their changes at once when searching for the bug (and git blame to see the reason for change in particular line). It's a great tool when I need it, but 99% of the time I don't.
    - bJGVygG7MQVF8c 6y ago
      Thanks for clarifying. > So people can intuitively check some version in between and do bisect that way, exploiting their knowledge to cut some steps If you have a hypothesis you hold with enough weight to validate, bisecting in general (let along git bisect) isn't even in the picture yet. The strategy really only enters the picture when you've exhausted any strong hunches and need to debug more systematically. > and do bisect that way In context you're not actually describing bisecting. But taking 'bisect' here as 'bisect': git automates bisecting through the commit history to a greater (with `run`) or lesser (without it) extent. Once you've paid that 15-minute fixed cost of familiarization, `git bisect` is strictly less time- and cognitive-energy-intensive than the manual alternative.

enchiridion 6y ago

I'm afraid I might be one of those people. Care to share? Always looking to level up!

renewiltord 6y ago
If you have a branch which always builds and tests green, you can add a new test, choose a red commit and a green commit and run `git bisect` in a loop to automatically get to the commit that caused the behaviour in that test to regress.
- renox 6y ago
  Did just that last week.. But for some reason, each time I share the result of a git bisect some thinks that I'm a wizard and instead of learning how to do it themselves they ask me later.. Thanks but its git bisect the magic tool not me (writing the shell script did take me some times though).
nominatronic 6y ago
Say someone finds a bug in your software. They're running release 4.1.2. They'd just upgraded from 2.6.4, which is three years old, where what they're doing used to work. You put together a test case and sure enough, it fails on 4.1.2 but passes on 2.6.4. But when did it get broken, and how? The software is complex and you can't see an obvious problem. And looking at the history, there's 4000 commits between those two tagged releases. It would take forever to test them all. You can do something smarter, though. You pick a point somewhere in the middle. Maybe it's around version 3.2.0. You test that version, and it passes. So - assuming the bug only got introduced once - the problematic commit lies somewhere between 3.2.0 and 4.1.2. You pick a version between those, and repeat the process. Rather than testing 4000 different versions, you only have to test about log2(4000) = ~12 versions of the code. The git bisect command is designed to help you with this. If you run the following commands: $ git bisect start $ git checkout v3.2.0 $ <run test, which passes> $ git bisect good $ git checkout v4.1.2 $ <run test, which fails> $ git bisect bad ...then git bisect will take over, choose the commit half way between those, check it out for you and prompt you to run your tests and report the result with 'git bisect good' or 'git bisect bad'. Then depending on the result, it will choose the next commit to test, and repeat the process until it can tell you exactly which commit introduced the bug. But maybe your test takes a while to run. Even with fewer intermediate versions to test, you're going to spend all day running tests, waiting for the results and telling git bisect what to do next. So you automate your test into a little script. If the test passes, it exits with a result of zero (success). If it fails, it returns non-zero. And then you run: $ git bisect run ./my-test-script and go do something else with your day. When you come back, it will have automatically found the commit that introduced the error. Magic! But there's a cost. For this to work, you need discipline from day one. You can't have commits in the history that say "WIP, changed some stuff, not finished yet" and others that fix things up later in the branch. You need to make sure each commit is a fully self-contained change that leaves the code in a working, testable state. If you have a small number of individual commits at which the code won't build cleanly, or which break testing for whatever reason, you can work around them with 'git bisect skip' - and you can implement this in your test script with a special return code. But if this happens too much, the whole approach becomes unmanageable.
- Aeolun 6y ago
  So if we squash every merge request we end up with the desired behavior as well.
  - lstamour 6y ago
    Sort of. The only issue with merges in GitHub tends to be that merge or squashed commit itself isn’t tested before being committed. It’s always possible, though in practice this should be quite rare, that the merge or squash commit produces a different result than any of the individual commits did. I suppose you could do something like a pre-push hook that would run tests before sharing your code with everyone else, though I haven’t looked recently at what options are available with GitHub. Edit: I stand somewhat corrected, seems most CI systems actually test the merged code - https://github.com/actions/checkout/issues/15#issuecomment-524107344 https://github.com/actions/checkout/issues/15#issuecomment-5... which I presume includes manually merged scenarios also. That said, they don’t appear to test squashed commits, under the assumption I suppose that any series of sequential changes will always cleanly squash with upstream as long as there are no merge conflicts. When you use squash-and-merge approaches to GitHub PRs, you have to make sure you automatically delete old branches, as the connection between branch and commit only lives within the GitHub PR at that point (and maybe in a commit comment somewhere, not sure). But it’s not as explicit as merges which include the name of the branch in the merge commit.
    - bJGVygG7MQVF8c 6y ago
      > seems most CI systems actually test the merged code The thrust of your point is right, though. CI tests the merged squash-commit after it's already on master, so you can still get into states where you have a failing build on master. That risk is mitigated somewhat if you block the merging of out-of-date PR branches but this isn't supported on all the platforms.
- enchiridion 6y ago
  Thank you for the detailed response!

XorNot 6y ago

Higher level tooling doesn't prioritize this. AFAIK neither Gitlab nor Github have an actual option in their CI implementations for "test every commit before allowing merge". And both of those are everywhere. There's also nothing out there which enforces practice - i.e. new code must be covered by some amount of test case (I'll settle for "it is executed in some way as part of build"). These are all things that could be done, but they're not priorities to the big players evidently, and good luck to me trying to get any organization to commit to putting that tooling in or running something custom that does it.

bJGVygG7MQVF8c 6y ago
Higher level tooling doesn't prioritize this. That's an excellent point. It's deep in the DNA of GitHub, unfortunately. I'm always reminded of this contrast: > Look, I don’t give two shits about my Git commits. [...] > From GitHub’s perspective, individual commits become less valuable because the atomic unit is the pull request. Utter Disregard for Git Commit History https://zachholman.com/posts/git-commit-history/ https://zachholman.com/posts/git-commit-history/ (The problem with this "unit of change" theorizing is obvious, especially considering GH only recently added a squash-merge feature.) vs. Torvalds on GitHub PRs: > Git comes with a nice pull-request generation module, but github instead decided to replace it with their own totally inferior version. As a result, I consider github useless for these kinds of things. It's fine for hosting, but the pull requests and the online commit editing, are just pure garbage. https://github.com/torvalds/linux/pull/17#issuecomment-5654674 https://github.com/torvalds/linux/pull/17#issuecomment-56546...
- hinkley 6y ago
  Commit squashing is a scourge. I sometimes go the other way; rewrite my history as a set of independent refactors.
Aeolun 6y ago
Why do we want to test every commit? Does it matter if you broke something if you end up fixing it later?

ajuc 6y ago

Learning git bisect starts being worth it if you have more than let's say 8 commits between the last good version and first bad version, and usually you don't (in small projects at least).

bJGVygG7MQVF8c 6y ago
> Learning git bisect starts being worth it Worth what? The time invested to learn it? It takes 15 minutes to learn, conservatively. More charitably, if you mean the time and energy required to modify one's habits: they're good habits Brent. > if you have more than let's say 8 commits between the last good version and first bad version, and usually you don't How anyone could even hypothetically arrive at this result is beyond me.
- cerberusss 6y ago
  > How anyone could even hypothetically arrive at this result is beyond me Well, for a handful of commits, it's very easy to do a "manual binary search" for the commit that caused a bug. Not the OP but that's how I read the comment.
- ajuc 6y ago
  > How anyone could even hypothetically arrive at this result is beyond me. Most of the time bugs are fixed shortly after they are introduced. So there's few commits between the last known good version and first known bad version. So people can intuitively check some version in between and do bisect that way, exploiting their knowledge to cut some steps. For example if there's 8 commits but 5 of them only change modules that have nothing to do with the bug. Also if these 8 commits are very small you can treat them as a whole and look at all their changes at once when searching for the bug (and git blame to see the reason for change in particular line). It's a great tool when I need it, but 99% of the time I don't.
  - bJGVygG7MQVF8c 6y ago
    Thanks for clarifying. > So people can intuitively check some version in between and do bisect that way, exploiting their knowledge to cut some steps If you have a hypothesis you hold with enough weight to validate, bisecting in general (let along git bisect) isn't even in the picture yet. The strategy really only enters the picture when you've exhausted any strong hunches and need to debug more systematically. > and do bisect that way In context you're not actually describing bisecting. But taking 'bisect' here as 'bisect': git automates bisecting through the commit history to a greater (with `run`) or lesser (without it) extent. Once you've paid that 15-minute fixed cost of familiarization, `git bisect` is strictly less time- and cognitive-energy-intensive than the manual alternative.