4 ms·

Do people here have examples of some bugs for which they had to resort to VCS history to find the cause? I'm struggling to picture a single bug in my whole care

by thom 6y ago

Do people here have examples of some bugs for which they had to resort to VCS history to find the cause? I'm struggling to picture a single bug in my whole career where this would have been quicker than just following the logic of the code. If there's information in commit messages that isn't evident in the code itself, that seems a terrible way to live.

pizza234 6y ago
As wrote in another comment, bisecting (which is for me a significant tool) relies on history (specifically, a granular one). However, it also must a disciplined history.
- thom 6y ago
  What's an example of a bug you've had to bisect for recently? Forgive me, it just seems like such a last resort thing.
  - LocalPCGuy 6y ago
    I don't bisect regularly, but when I do, it's usually trying to figure out what the original reason for introducing the code that is problematic is. The entire point is on projects big enough, you may not be able to just "follow the logic" enough to know that your fix the to apparent bug isn't re-introducing some regression that was fixed previously. So bisect to me, is a way to figure out where and why the code was changed the way that it was, so I have a better understanding of the changes I can make going forward. That doesn't mean the change made in the past was correct and must be maintained (obviously something is broken), or that it might be that the obvious solution based on following the code logic is correct. But that doesn't mean I wasted time making sure I best understand the reasoning behind the changes made. But this is also why I don't resort to it very often, because it isn't necessary in all cases (I'd even say it isn't necessary in most cases). Bisect also allows you to see other changes made in the commit in question, and around that commit, so you get a better overall picture of the logic.
    - thom 6y ago
      If you've got a codebase where the intent of the code isn't clear, where you can't track bugs, and where you can't fix them without being confident you're not breaking other stuff, surely those are all fundamental problems worth addressing?
      - JoshuaDavid 6y ago
        Yes, but those things take time to address and in the meantime the need to make changes that address more immediate needs doesn't go away. Bisect specifically helps if you've got an easily reproducible bug in a hairy part of the codebase and you know that bug could be arbitrarily old but is on a path that recently started getting exercised much more often. I use it maybe once a month but when I do it is very nice to have that context of how the bug happened and any other places that might need to be fixed.
        thom 6y ago
        Sure, but at that point there's a whole raft of practices that seem more important than 'having a nice commit history'.
        JoshuaDavid 6y ago
        Bisect doesn't require a nice commit history, just a granular one (well, it doesn't require a granular one either but it's a bit less helpful if it lets you identify the problem commit as the +18000/-7000 LOC "Overhaul payment system <7 paragraph description of new payment system>" than if it drops you on the +3/-3 LOC "fix encoding bug <no description of what the bug is or what the fix does>") Edit: I should also add that if you use github, squash-merging pull requests is fine because you can pull down and check out the pull request branch and run bisect on that. Or more generally if you don't delete history and keep a record of what commits were squashed to make your giant commit, your friendly local maintenance programmer might grumble slightly but will still be able to work effectively. Just please avoid erasing history entirely.
  - tharkun__ 6y ago
    I can give you an example though I have to be sort of vague for obvious reasons too. In fact, I have been bisecting to find the cause of a bug 3 times in the past 3 to 4 weeks. Customer Service reported a problem with something that my team is responsible for. I knew almost for certain that we didn't break it. I had a vague idea that another team might have broken it by a recent-ish change to a different service that we both rely on but aren't the maintainers of (we have a monorepo). In fact they made multiple changes to it recently (they put me on the PR for awareness). I don't know the code for that service but it was really easy to check for the occurrence of the bug (literally two clicks in the UI to reproduce or not reproduce). Instead of trying to understand a service that I don't own, which would require mental concentration, I randomly checked out a commit from 4 weeks ago, tested => bug not reproducible and started a bisect. I was actually gonna be stuck in some meetings for the next few hours but bisecting is mechanical, so I was able to just do it on the side (deploying is a little bit of waiting in between, you gotta refresh etc.) and a little later I was able to just paste a very neutral "XYZ is the first bad commit" into the ticket. Totally cuts down on the 'drama' too as you're not simply 'accusing' someone else of causing an issue in _your_ part of the application. It's just there, very neutral, there's the commit that caused it. Git told me!
    - thom 6y ago
      But that just sounds like an _incredible_ amount of effort and even at the end you still don't actually know the cause of the bug. Surely you just raise it to someone who owns the code in question? It sounds like you know who they are already?
      - tharkun__ 6y ago
        Until you have done it it may sound like an incredible amount of effort but in fact it isn't at all. Your reality might be different, but we also deal with different time zones. In this particular example the other team was in a different time zone and already gone for the day. The easiest and most efficient way was a 'dumb' bisect. If you _only_ raise it to the team in question, you have nothing in hand but suspicions. Their natural reaction is most probably going to be "No can't be us, must be something/someone else" and you've wasted an entire day waiting for their response that was predictable anyway and you've also just increased both their and your own irritation level. Their's because they probably feel like you're trying to blame them for something they feel is not their fault. Your own because you've just wasted a day trying to solve this and meanwhile customer service thinks you caused a regression. A bisect that literally took zero brain cycles and told me which commit caused it before I was even halfway done with my meetings allowed me to let them know exactly which commit caused it without a doubt (unless your company is so broken that they wouldn't even believe the bisect output :)). This also gives them very valuable information because instead of them now trying to have to figure out which of their changes maybe caused this thing I'm 'accusing' them of, they have a single commit, which is directly tied to a single ticket. In this particular case, the developer in question actually knew right away from seeing the commit message/ticket number what particular part of their change was the cause of the issue, without even having to dig through the code.
- calpaterson 6y ago
  For example when a bug has messed up data and you want to know when it was introduced. Then you know how far you have to go back to correct it. Other times knowing how it was introduced can give you a sense of what the change the original author was trying to make.
detaro 6y ago
As parent said, history is not just commit messages. Trivial example from yesterday: "this worked last week... what did they even touch inbetween?", look at the most relevant commit diff, context makes it obvious someone just accidentally deleted a line too many while replacing a code block. For more complex stuff, it's helpful for "what were they trying to do with this line", "what's the requirements document they worked off when they wrote this", "which other versions are likely to have this bug too", "Is this new and I can go ask whoever wrote this about it or is it years old", ...
NateEag 6y ago
What follows is a summary of a time bisect saved a team I was on from a lot of pain. It's been a long time so I may have some details wrong, but the point should be clear. Years ago I worked at an e-commerce shop. One fine day we discovered that the checkout process had been broken in a very particular corner case at some point in the past several months. No one had any idea how it had happened. Most of the team was trying to work out what could have possibly gone wrong by staring intently at code in their IDEs and debuggers while stepping through reproductions. I thought, "everyone's already doing that - I'll try a bisect, since we have a reliable reproduction for this inscrutable weirdness." It took maybe twenty minutes before I could point to a < twenty-line change. The author of said change was flummoxed at the breakage, as it was nowhere near the actual checkout code, but a few minutes of the whole team staring at it revealed a classic piece of PHP spaghetti insanity that had managed to break the checkout process (I no longer recall the specific issue - might've been global variable name collision). I think it would have taken hours before we got to an answer without bisect. Granted, it only worked so well because of good commit hygiene (which I had trained everyone to use when I introduced git and code review to that team), but it showed quite clearly how useful good commit hygiene is when paired with bisect.
u801e 6y ago
> Do people here have examples of some bugs for which they had to resort to VCS history to find the cause? A more common scenario is to run git blame on the files you plan to modify to see what commits were responsible for them. If the message is detailed enough, you can avoid introducing a regression because you altered logic introduced by a commit to fix a bug. > If there's information in commit messages that isn't evident in the code itself, that seems a terrible way to live. Good commit messages include why a change was made, which usually isn't really reflected in the code itself. Comments may be outdated if the code is changed without updating the comments, but the git blame output for a line of code is timeless because the associated between that commit and line of code isn't lost until the line itself is updated.
Noumenon72 6y ago
* Commit that added line was fixing something unrelated; deduced line was duplicated in merge resolution. * Line was comparing two different enums for equality and failing; commit history showed one enum had been an int and the commit message that changed it explained why the author thought the comparison was safe anyway, so we could ask why that wasn't true Plus even when the _cause_ is in the code, I constantly check "what's the history of this function?" while trying to figure out whether the code was always intended to do this crazy thing or whether it's been morphed by recent changes. I have "Show selection history" mapped to Ctrl+Shift+G+H in PyCharm.
ThrustVectoring 6y ago
I've used version control history to verify that a mysterious bug was not caused by any changes in the codebase. Instead, we were set up to automatically incorporate minor version changes in dependencies, one of which improperly had a breaking change. This class of bug is impossible to find by following the logic of the code because it's not in the code.
david_allison 6y ago
"had to" is a stretch, but as the first example from GitHub: https://github.com/ankidroid/Anki-Android/issues/7781 https://github.com/ankidroid/Anki-Android/issues/7781 There's a lot of value in bisecting to find the exact commit, moving from "this is probably the commit that broke it" to "this is exactly the commit which broke it" Typical workflow: * branch off master * Write a test which passes at some point in the past, and fails on HEAD * rebase the test into the branch at the point in the past, so bisect works * git bisect to pinpoint the bug With a non-squashed and a history where each commit builds, `git bisect` points to the exact failing commit and makes the investigation trivial An additional advantage is that you're not wasting time investigating: the unit test can be used as a regression test

by thom 6y ago

pizza234 6y ago

As wrote in another comment, bisecting (which is for me a significant tool) relies on history (specifically, a granular one). However, it also must a disciplined history.

thom 6y ago
What's an example of a bug you've had to bisect for recently? Forgive me, it just seems like such a last resort thing.
- LocalPCGuy 6y ago
  I don't bisect regularly, but when I do, it's usually trying to figure out what the original reason for introducing the code that is problematic is. The entire point is on projects big enough, you may not be able to just "follow the logic" enough to know that your fix the to apparent bug isn't re-introducing some regression that was fixed previously. So bisect to me, is a way to figure out where and why the code was changed the way that it was, so I have a better understanding of the changes I can make going forward. That doesn't mean the change made in the past was correct and must be maintained (obviously something is broken), or that it might be that the obvious solution based on following the code logic is correct. But that doesn't mean I wasted time making sure I best understand the reasoning behind the changes made. But this is also why I don't resort to it very often, because it isn't necessary in all cases (I'd even say it isn't necessary in most cases). Bisect also allows you to see other changes made in the commit in question, and around that commit, so you get a better overall picture of the logic.
  - thom 6y ago
    If you've got a codebase where the intent of the code isn't clear, where you can't track bugs, and where you can't fix them without being confident you're not breaking other stuff, surely those are all fundamental problems worth addressing?
    - JoshuaDavid 6y ago
      Yes, but those things take time to address and in the meantime the need to make changes that address more immediate needs doesn't go away. Bisect specifically helps if you've got an easily reproducible bug in a hairy part of the codebase and you know that bug could be arbitrarily old but is on a path that recently started getting exercised much more often. I use it maybe once a month but when I do it is very nice to have that context of how the bug happened and any other places that might need to be fixed.
      - thom 6y ago
        Sure, but at that point there's a whole raft of practices that seem more important than 'having a nice commit history'.
        JoshuaDavid 6y ago
        Bisect doesn't require a nice commit history, just a granular one (well, it doesn't require a granular one either but it's a bit less helpful if it lets you identify the problem commit as the +18000/-7000 LOC "Overhaul payment system <7 paragraph description of new payment system>" than if it drops you on the +3/-3 LOC "fix encoding bug <no description of what the bug is or what the fix does>") Edit: I should also add that if you use github, squash-merging pull requests is fine because you can pull down and check out the pull request branch and run bisect on that. Or more generally if you don't delete history and keep a record of what commits were squashed to make your giant commit, your friendly local maintenance programmer might grumble slightly but will still be able to work effectively. Just please avoid erasing history entirely.
- tharkun__ 6y ago
  I can give you an example though I have to be sort of vague for obvious reasons too. In fact, I have been bisecting to find the cause of a bug 3 times in the past 3 to 4 weeks. Customer Service reported a problem with something that my team is responsible for. I knew almost for certain that we didn't break it. I had a vague idea that another team might have broken it by a recent-ish change to a different service that we both rely on but aren't the maintainers of (we have a monorepo). In fact they made multiple changes to it recently (they put me on the PR for awareness). I don't know the code for that service but it was really easy to check for the occurrence of the bug (literally two clicks in the UI to reproduce or not reproduce). Instead of trying to understand a service that I don't own, which would require mental concentration, I randomly checked out a commit from 4 weeks ago, tested => bug not reproducible and started a bisect. I was actually gonna be stuck in some meetings for the next few hours but bisecting is mechanical, so I was able to just do it on the side (deploying is a little bit of waiting in between, you gotta refresh etc.) and a little later I was able to just paste a very neutral "XYZ is the first bad commit" into the ticket. Totally cuts down on the 'drama' too as you're not simply 'accusing' someone else of causing an issue in _your_ part of the application. It's just there, very neutral, there's the commit that caused it. Git told me!
  - thom 6y ago
    But that just sounds like an _incredible_ amount of effort and even at the end you still don't actually know the cause of the bug. Surely you just raise it to someone who owns the code in question? It sounds like you know who they are already?
    - tharkun__ 6y ago
      Until you have done it it may sound like an incredible amount of effort but in fact it isn't at all. Your reality might be different, but we also deal with different time zones. In this particular example the other team was in a different time zone and already gone for the day. The easiest and most efficient way was a 'dumb' bisect. If you _only_ raise it to the team in question, you have nothing in hand but suspicions. Their natural reaction is most probably going to be "No can't be us, must be something/someone else" and you've wasted an entire day waiting for their response that was predictable anyway and you've also just increased both their and your own irritation level. Their's because they probably feel like you're trying to blame them for something they feel is not their fault. Your own because you've just wasted a day trying to solve this and meanwhile customer service thinks you caused a regression. A bisect that literally took zero brain cycles and told me which commit caused it before I was even halfway done with my meetings allowed me to let them know exactly which commit caused it without a doubt (unless your company is so broken that they wouldn't even believe the bisect output :)). This also gives them very valuable information because instead of them now trying to have to figure out which of their changes maybe caused this thing I'm 'accusing' them of, they have a single commit, which is directly tied to a single ticket. In this particular case, the developer in question actually knew right away from seeing the commit message/ticket number what particular part of their change was the cause of the issue, without even having to dig through the code.
calpaterson 6y ago
For example when a bug has messed up data and you want to know when it was introduced. Then you know how far you have to go back to correct it. Other times knowing how it was introduced can give you a sense of what the change the original author was trying to make.

detaro 6y ago

As parent said, history is not just commit messages. Trivial example from yesterday: "this worked last week... what did they even touch inbetween?", look at the most relevant commit diff, context makes it obvious someone just accidentally deleted a line too many while replacing a code block. For more complex stuff, it's helpful for "what were they trying to do with this line", "what's the requirements document they worked off when they wrote this", "which other versions are likely to have this bug too", "Is this new and I can go ask whoever wrote this about it or is it years old", ...

NateEag 6y ago

What follows is a summary of a time bisect saved a team I was on from a lot of pain. It's been a long time so I may have some details wrong, but the point should be clear. Years ago I worked at an e-commerce shop. One fine day we discovered that the checkout process had been broken in a very particular corner case at some point in the past several months. No one had any idea how it had happened. Most of the team was trying to work out what could have possibly gone wrong by staring intently at code in their IDEs and debuggers while stepping through reproductions. I thought, "everyone's already doing that - I'll try a bisect, since we have a reliable reproduction for this inscrutable weirdness." It took maybe twenty minutes before I could point to a < twenty-line change. The author of said change was flummoxed at the breakage, as it was nowhere near the actual checkout code, but a few minutes of the whole team staring at it revealed a classic piece of PHP spaghetti insanity that had managed to break the checkout process (I no longer recall the specific issue - might've been global variable name collision). I think it would have taken hours before we got to an answer without bisect. Granted, it only worked so well because of good commit hygiene (which I had trained everyone to use when I introduced git and code review to that team), but it showed quite clearly how useful good commit hygiene is when paired with bisect.

u801e 6y ago

> Do people here have examples of some bugs for which they had to resort to VCS history to find the cause? A more common scenario is to run git blame on the files you plan to modify to see what commits were responsible for them. If the message is detailed enough, you can avoid introducing a regression because you altered logic introduced by a commit to fix a bug. > If there's information in commit messages that isn't evident in the code itself, that seems a terrible way to live. Good commit messages include why a change was made, which usually isn't really reflected in the code itself. Comments may be outdated if the code is changed without updating the comments, but the git blame output for a line of code is timeless because the associated between that commit and line of code isn't lost until the line itself is updated.

Noumenon72 6y ago

* Commit that added line was fixing something unrelated; deduced line was duplicated in merge resolution. * Line was comparing two different enums for equality and failing; commit history showed one enum had been an int and the commit message that changed it explained why the author thought the comparison was safe anyway, so we could ask why that wasn't true Plus even when the _cause_ is in the code, I constantly check "what's the history of this function?" while trying to figure out whether the code was always intended to do this crazy thing or whether it's been morphed by recent changes. I have "Show selection history" mapped to Ctrl+Shift+G+H in PyCharm.

ThrustVectoring 6y ago

I've used version control history to verify that a mysterious bug was not caused by any changes in the codebase. Instead, we were set up to automatically incorporate minor version changes in dependencies, one of which improperly had a breaking change. This class of bug is impossible to find by following the logic of the code because it's not in the code.

david_allison 6y ago

"had to" is a stretch, but as the first example from GitHub: https://github.com/ankidroid/Anki-Android/issues/7781 https://github.com/ankidroid/Anki-Android/issues/7781 There's a lot of value in bisecting to find the exact commit, moving from "this is probably the commit that broke it" to "this is exactly the commit which broke it" Typical workflow: * branch off master * Write a test which passes at some point in the past, and fails on HEAD * rebase the test into the branch at the point in the past, so bisect works * git bisect to pinpoint the bug With a non-squashed and a history where each commit builds, `git bisect` points to the exact failing commit and makes the investigation trivial An additional advantage is that you're not wasting time investigating: the unit test can be used as a regression test