3 ms·
> But that is basically just saying your existing way to bisect is not fully reliable and you fear using rebase would make it quantitatively noticeably worse. W
by lmm 6y ago
> But that is basically just saying your existing way to bisect is not fully reliable and you fear using rebase would make it quantitatively noticeably worse. What I am saying is it can and should be fully reliable, and this is not theoretical either.
What you're proposing doesn't actually improve matters; you don't gain anything (other than saving a bit of machine time) by only testing known-compiling commits. At the end of the day the bisect lands you on either a single commit or a string of commits (in the case where some commits don't compile), and that's the diff that you have to go through manually; the whole point of bisect is to make that diff as small as possible. If you limit your bisect to testing changes from the history of master (one way or another), you guarantee that you'll land a diff that's a complete feature branch; if you allow the bisect to go through every commits then you have a decent chance of landing on a much smaller diff.
> But I am arguing from plenty of real-life experience with a workflow, which is incidentally rebase-based (but the same guarantees would hold just as well if I moved it to a purely merge based approach) where I a) know that every "merge" to master has been fully tested before it lands b) can reliably bisect over ~years of commits with a simple git alias, without having to ever manually bisect skip bad commits[+]. And by construction, not because I hope that devs will generally have tested their commits sufficiently manually or are diligent about fixing merge conflicts in a way that does not result in intermittent broken commits. So whilst several of your other reasons are perfectly valid (long-lived branches), you can maybe see why this one is a bit amusing to me.
I've got plenty of real-life experience with plenty of different workflows, thank you very much. If you decide you want a history-of-master history, there are plenty of ways to get that (I'd argue squash-merging is the lowest-overhead way to do it). But the result of that is you get a much less useful bisect than a workflow where you have small commits on feature branches and use merges.
> It's not impractical at all, in fact if your company pays me for it I'm more than happy to set it up for you :)
I don't work for that company any more, but bear in mind this was for a codebase with 500 developers where builds took around 1.5 hours. We looked at building speculative batches of PRs together but decided the costs were higher than the benefits; master occasionally got broken and we fixed it when it happened.
Fundamentally a broken master is always a problem you can have, because flaky tests happen (and, as you mention, meta-level problems can happen). It's good to minimise the times when master is broken, but it's not realistic to assume you can avoid it entirely, so your workflow should be able to handle having the occasional isolated broken commit in the history of master.
- patrec 6y ago> What you're proposing doesn't actually improve matters; you don't gain anything (other than saving a bit of machine time) by only testing known-compiling commits. You're not being quite honest here: of course just knowing if something compiles at all, whilst nice, is not that massive a difference (because you can fairly easily script an automated bisect that will skip non-compiling commits by itself). But that's much weaker than what I actually said: knowing that the commit has passed your (1.5 h) CI, and making it so that all your merge commits on master will have this property. Since you apparently haven't tried it, maybe you should be a bit more careful about dismissing the utility out of hand? In the absence of an automatic way to avoid bad commits, I have in the past given up on using bisect to track something down on multiple instances because it was just too much overhead to work out what commits needed to be skipped; if you hit a commit that's broken in a more subtle fashion than "does not compile" but that CI would have found it's not always cheap to work out if you hit the regression or some unrelated temporary breakage. And of course there are plenty of benefits unrelated to bisecting. You are also making a several implicit assumptions which are unlikely to hold: it is not true that typically none of the utility of bisect manifests before you have identified the precise lines of code that caused an issue and understood them. If you have some acute problem in production, being able to reliably and quickly locate the feature branch that introduced it is very valuable (for getting the right people to look at the problem or even shipping some work around before the problem is fully diagnosed). Often certainly a recent deployment is to blame. But sometimes a deployment can exercise something that regressed much earlier and trigger a data corruption that is not noticed immediately. Also, of course once you found some problematic merge there is no reason you wouldn't then continue to use bisect to find the precise commit in the feature branch! Since the cost of dealing with potentially bad commits inside the relevant (ex-)feature branch is much smaller than the cost of dealing with potentially bad commits everywhere on the tree before you have found the right branch (for several obvious reasons), that's still a big win. And yeah it also beats just squash committing and dealing with a single monster diff. > I've got plenty of real-life experience with plenty of different workflows, thank you very much. I'm not denigrating your experience. What I'm stating that I have good reasons to be sceptical of part of your rationale for disliking rebasing ("it will mess up bisecting") because whatever other things you might have done better and more ergonomically in your previous development workflows than I have in mine, I get the pretty clear impression I have experienced some affordances around bisecting in particular that I value which you haven't. > I don't work for that company any more, but bear in mind this was for a codebase with 500 developers where builds took around 1.5 hours. I don't think this is beyond what's doable, but you can tell me if you attempted/considered and rejected what I propose below (and if so why it was not workable). You basically need two things: The most vital one being 1. a merge queue that's processed in order (so CI doesn't race against a moving master as you described). This you will need even with a dozen developers and 10 minute CI runs. 2. at that scale, speculative batching as you mentioned. It'll probably need to be reasonably intelligent as well so you can always merge a sizeable batch every 1.5h with high likelihood, even if you assume that (say) on average 2% of your open PRs will break on merge into master/being combined into a batch. For example, run several alternative batches in parallel, run tests in an intelligent and code change dependent manner to maximize the chances of early failure detection and so on and so forth. Assuming each developer lands something on master every two days on average, you'd have to deal with 250 merges a day. Say you can run about 5 sequential CI runs per work day, you'd need to test pretty large batches of ~50PRs per batch, which is more than I have experience with. Since you will only batch stuff that passed branch CI, and disregarding flakes for the moment, a batch should only fail because of an incompatibility of PRs within the same batch or against some very recent addition to master. So you can probably get the failure rate into the low single digit percentages, at which you'd need to do run enough alternative batches in parallel to cope with one or two bad PRs per batch. That still seems feasible if you put batches together intelligently, although of course things get exponentially worse as failure rate increases (already over 1k ways to omit two PRs from a batch of 50; whereas running 50 variations of a batch with 1 PR omitted in parallel is still cost-neutral compared to not batching). > Fundamentally a broken master is always a problem you can have, because flaky tests happen (and, as you mention, meta-level problems can happen). It's good to minimise the times when master is broken, but it's not realistic to assume you can avoid it entirely, so your workflow should be able to handle having the occasional isolated broken commit in the history of master. Sure. But minimizing those bad commits (and probably even marking them after the discovery, e.g. via git-notes) pays off.