3 ms·

The sad reality is that testing and reviewing will only get us to defect density of one defect per 1000 lines of code changed - a nice round number for a clarit

by thesz 4y ago

The sad reality is that testing and reviewing will only get us to defect density of one defect per 1000 lines of code changed - a nice round number for a clarity of discussion (and we are lucky here, see below). E.g., if patch changes or adds 300 lines of code, which is not unusual amount of code, the probability of defect is 1/3. My changeset right now contains 320 files and 3200 lines added or changed (a fix for long standing defect in design). It is not green in testing, but you get the idea: it is quite probably I will introduce some defect into main branch. Three against one that I will and it'll pass review. Usually, private branches are get deleted after merge happen. In the end, I will have a situation where there is a defect in the main branch and there is no way to bisect it narrower than to squashed commit. What should I do with 3200 changed lines in 320 files when there is a defect? [1] https://stackoverflow.com/questions/4049958/embedded-software-defect-rate https://stackoverflow.com/questions/4049958/embedded-softwar... The page above hints that pre-review defect density for C/C++ code is 4.5 defects per 100 lines of code. After review it is 0.82 defects per 100 LOC, or 8.2 defects per 1000 lines of code. My big patch will introduce 25 defects that'll pass review. Even 10 fold decrease in defect density after testing (haha) will not significantly reduce probability of the introduction of a single defect.

mafuy 4y ago
I don't follow. Either you can break your big commit down into pieces that can be applied one by one - then do that using history rewriting (and testing each part!). Or that's not possible, e.g. because it's a global architecture change or something that is already atomar. Then there is nothing to bisect anyway - there is nothing that can be done, no matter which tool you use and no matter if you squash or not.
- thesz 4y ago
  When commit is squashed and rebased upon main branch there is no pieces to bisect. Having private branches also does not always work. For example, there was a purge of unnecessary branches in our main repository. If I use a fork of main repository (and I do), my private branches are also not readily available to the team.
  - int_19h 4y ago
    You don't have to squash it all down to one commit. Many people do, but it's a choice they're making.
    - thesz 4y ago
      There are protocols to merge and squashing was required in two of four github-based teams I've worked with. The choice is done for you. It is not yours or theirs often enough.
      - int_19h 4y ago
        I've seen that, and even successfully fought against it once. I've also been on teams that don't treat it as a religious issue in the first place, and people rebase or merge from their private forks as they see fit. Personally, I think that the only bar that is necessary is that commit history should remain readable - which is subjective, but that's why we have code reviews. In any case, this is not an issue with the tool, but rather with the religion that formed around it, which is often the case with software tooling. One thing that helps a lot with a pragmatic consensus is to ensure that all members of the team have personal experience debugging old code that they didn't write or previously review.

by thesz 4y ago

mafuy 4y ago

I don't follow. Either you can break your big commit down into pieces that can be applied one by one - then do that using history rewriting (and testing each part!). Or that's not possible, e.g. because it's a global architecture change or something that is already atomar. Then there is nothing to bisect anyway - there is nothing that can be done, no matter which tool you use and no matter if you squash or not.

thesz 4y ago
When commit is squashed and rebased upon main branch there is no pieces to bisect. Having private branches also does not always work. For example, there was a purge of unnecessary branches in our main repository. If I use a fork of main repository (and I do), my private branches are also not readily available to the team.
- int_19h 4y ago
  You don't have to squash it all down to one commit. Many people do, but it's a choice they're making.
  - thesz 4y ago
    There are protocols to merge and squashing was required in two of four github-based teams I've worked with. The choice is done for you. It is not yours or theirs often enough.
    - int_19h 4y ago
      I've seen that, and even successfully fought against it once. I've also been on teams that don't treat it as a religious issue in the first place, and people rebase or merge from their private forks as they see fit. Personally, I think that the only bar that is necessary is that commit history should remain readable - which is subjective, but that's why we have code reviews. In any case, this is not an issue with the tool, but rather with the religion that formed around it, which is often the case with software tooling. One thing that helps a lot with a pragmatic consensus is to ensure that all members of the team have personal experience debugging old code that they didn't write or previously review.