3 ms·

> All those big changes introduce commits that make git bisect generally slower. Bisection search is log2(n) so doubling the number of commits should only add

> All those big changes introduce commits that make git bisect generally slower. Bisection search is log2(n) so doubling the number of commits should only add one more bisection step, yes? > Which might be awful if you also have some C code to recompile at every step of bisecting. That reminds me, I've got to try out ccache (https://ccache.dev/ https://ccache.dev/ ) for my project. My full compile is one minute, but the three files that take longest to compile rarely change.

bombolo 4y ago
> Bisection search is log2(n) so doubling the number of commits should only add one more bisection step, yes? And testing 1 extra step could only add a 1 hour build more, yes?
- eesmith 4y ago
  It could, certainly. But 1) you don't have one black commit for every non-black commit, do you? Because the general best practice is to do like kuu suggested and have a specific black version as part of the development environment, with a pre-commit hook to ensure no random formatting gets introduced. 2) assuming 500 commits in your bisection, that's, what, about 9 compilations you'll need to do, so it will take you 9 hours to run. So even with a black commit after every human commit, that yes, 1 hour more, but it's also only 11% longer. Even with only 10 non-black commits and 10 black commits, your average bisect time will only increase from 3.6 hours to 4.6 hours, or 30% longer. I'm curious to know what project you are on with 1 hour build times and the regular need for hours-long bisection search, but where there isn't a common Python dev environment with a specific black version. Are you using ccache and/or distributed builds? If not, why isn't there a clear economic justification for improving your build environment? I mean, I assume developers need to build and test before commit, which means each commit is already taking an hour. Isn't that a waste of expensive developer time? And, I assume it's not the black formatting changes which result in hour-long builds. If they do, could you explain how?
  - bombolo 4y ago
    As I said in other comments, if you try to force contributors to reproduce exactly your local setup, you will be left with no contributors. Which is why you set up a CI to run the tests… because people will most likely not. As for build times, it was an extreme example. But even an extra step taking 5 extra minutes is very annoying to me…
    - geysersam 4y ago
      > if you try to force contributors to reproduce exactly your local setup python -m venv venv pip install -r requirements.txt Do you consider that imposing? I assumed that was standard. Don't basically all Python projects in existence use something like it?
    - eesmith 4y ago
      > if you try to force contributors to reproduce exactly your local setup, you will be left with no contributors. ... That's not been my experience. To the contrary, having a requirements.txt means your contributors are more likely to have a working environment, as when your package depends on package X but the contributor has a 5-year-old buggy version of X, doesn't realize it, and it causes your program to do the wrong thing. In any case, your argument only makes sense if no one on the project uses black or other code formatter. Even if you alone use it, odds are good that most of your collaborator's commits will need to be reformatted. > .. an extra step taking 5 extra minutes ... How do black reformatting changes cause an extra 5 minutes? What Python code base with only a couple of contributors and no need for a requirements.txt takes 5+ minutes to byte-compile and package the Python code, and why? Adding 5 minutes to you build means your bisections are taking at least an hour, so it seems like focusing on black changes is the wrong place to look.
      - bombolo 4y ago
        > How do black reformatting changes cause an extra 5 minutes? Did you even read my comments? Black reformatting causes more steps in bisecting. It's quite easy that a test suite takes 5+ minutes.
sgtnoodle 4y ago
Perhaps the poster meant that the contents of the commits themselves make bisection slower? By touching a lot of files unnecessarily, incremental build systems have to do more work than otherwise.
- bombolo 4y ago
  Both… and version jumps in formatting tool basically will touch every single file.
  - makapuf 4y ago
    Every file that changed because rules changed, which shouldn't be frequent, I don't remember black changed radically since its creation, do you have an example of some widespread syntax change ?
    - bombolo 4y ago
      Just take a codebase and run black from different ubuntu releases. The funny thing is that if you run the versions backwards you will NOT obtain identical files with what you started with.
- eesmith 4y ago
  Could be? I've not heard of that problem, but I don't know everything. I'm not used to Python code (which is all that black touches) as being notably slow to build, nor am I used to incremental build systems for Python byte compilation. And I expect in a project with 2 developers which is big enough for things to be slow, then most of the files will be unchanged semantically speaking, only swapping back and forth between two syntactically re-blackened representations, so wouldn't an caching build system be able to cache both forms? (NB: I said "caching build system" because an incremental build system which expects time linear order, wouldn't be that helpful in bisection, which jumps back-and-forth through the commits.)

bombolo 4y ago

> Bisection search is log2(n) so doubling the number of commits should only add one more bisection step, yes? And testing 1 extra step could only add a 1 hour build more, yes?

eesmith 4y ago
It could, certainly. But 1) you don't have one black commit for every non-black commit, do you? Because the general best practice is to do like kuu suggested and have a specific black version as part of the development environment, with a pre-commit hook to ensure no random formatting gets introduced. 2) assuming 500 commits in your bisection, that's, what, about 9 compilations you'll need to do, so it will take you 9 hours to run. So even with a black commit after every human commit, that yes, 1 hour more, but it's also only 11% longer. Even with only 10 non-black commits and 10 black commits, your average bisect time will only increase from 3.6 hours to 4.6 hours, or 30% longer. I'm curious to know what project you are on with 1 hour build times and the regular need for hours-long bisection search, but where there isn't a common Python dev environment with a specific black version. Are you using ccache and/or distributed builds? If not, why isn't there a clear economic justification for improving your build environment? I mean, I assume developers need to build and test before commit, which means each commit is already taking an hour. Isn't that a waste of expensive developer time? And, I assume it's not the black formatting changes which result in hour-long builds. If they do, could you explain how?
- bombolo 4y ago
  As I said in other comments, if you try to force contributors to reproduce exactly your local setup, you will be left with no contributors. Which is why you set up a CI to run the tests… because people will most likely not. As for build times, it was an extreme example. But even an extra step taking 5 extra minutes is very annoying to me…
  - geysersam 4y ago
    > if you try to force contributors to reproduce exactly your local setup python -m venv venv pip install -r requirements.txt Do you consider that imposing? I assumed that was standard. Don't basically all Python projects in existence use something like it?
  - eesmith 4y ago
    > if you try to force contributors to reproduce exactly your local setup, you will be left with no contributors. ... That's not been my experience. To the contrary, having a requirements.txt means your contributors are more likely to have a working environment, as when your package depends on package X but the contributor has a 5-year-old buggy version of X, doesn't realize it, and it causes your program to do the wrong thing. In any case, your argument only makes sense if no one on the project uses black or other code formatter. Even if you alone use it, odds are good that most of your collaborator's commits will need to be reformatted. > .. an extra step taking 5 extra minutes ... How do black reformatting changes cause an extra 5 minutes? What Python code base with only a couple of contributors and no need for a requirements.txt takes 5+ minutes to byte-compile and package the Python code, and why? Adding 5 minutes to you build means your bisections are taking at least an hour, so it seems like focusing on black changes is the wrong place to look.
    - bombolo 4y ago
      > How do black reformatting changes cause an extra 5 minutes? Did you even read my comments? Black reformatting causes more steps in bisecting. It's quite easy that a test suite takes 5+ minutes.

sgtnoodle 4y ago

Perhaps the poster meant that the contents of the commits themselves make bisection slower? By touching a lot of files unnecessarily, incremental build systems have to do more work than otherwise.

bombolo 4y ago
Both… and version jumps in formatting tool basically will touch every single file.
- makapuf 4y ago
  Every file that changed because rules changed, which shouldn't be frequent, I don't remember black changed radically since its creation, do you have an example of some widespread syntax change ?
  - bombolo 4y ago
    Just take a codebase and run black from different ubuntu releases. The funny thing is that if you run the versions backwards you will NOT obtain identical files with what you started with.
eesmith 4y ago
Could be? I've not heard of that problem, but I don't know everything. I'm not used to Python code (which is all that black touches) as being notably slow to build, nor am I used to incremental build systems for Python byte compilation. And I expect in a project with 2 developers which is big enough for things to be slow, then most of the files will be unchanged semantically speaking, only swapping back and forth between two syntactically re-blackened representations, so wouldn't an caching build system be able to cache both forms? (NB: I said "caching build system" because an incremental build system which expects time linear order, wouldn't be that helpful in bisection, which jumps back-and-forth through the commits.)