6 ms·
At Sun Microsystems, Inc., (RIP) we have many "gates" (repos) that made up Solaris. Cross-gate development was somewhat more involved, but still not bad. Basi
by cryptonector 9y ago
At Sun Microsystems, Inc., (RIP) we have many "gates" (repos) that made up Solaris. Cross-gate development was somewhat more involved, but still not bad. Basically: you installed the latest build of all of Solaris, then updated the bits from your clones of the gates in question. Still, a single repo is great if it can scale, and GVFS sounds great!
But that's not what I came in to say.
I came in to describe the rebase (not merge!) workflow we used at Sun, which I recommend to anyone running a project the size of Solaris (or larger, in the case of Windows), or, really, even to much smaller projects.
For single-developer projects, you just rebased onto the latest upstream periodically (and finally just before pushing).
For larger projects, the project would run their own upstream that developers would use. The project would periodically rebase onto the latest upstream. Developers would periodically rebase onto their upstream: the project's repo.
The result was clean, linear history in the master repository. By and large one never cared about intra-project history, though project repos were archived anyways so that where one needed to dig through project-internal history ("did they try a different alternative and found it didn't work well?"), one could.
I strongly recommend rebase workflows over merge workflows. In particular, I recommend it to Microsoft.
- holtalanm 9y agowe use a rebase workflow in git at my current employer, and it is amazing. previous employer used a merge workflow (primarily because we didnt understand git very well at the time), and there were merge conflicts all the time when pulling new changes down or merging new changes in. It was a headache to say the least. As the integration manager for one project, I usually spent the better part of an hour just going through the pull requests and merge conflicts from the previous day. I managed a team that was on the other side of the world, so there were always new changes when I started working in the morning.
- cryptonector 9y agoYes! One of the most important advantages of a rebase workflow is that you can see immediately what upstream commits your conflict with, as opposed to some massive merge you have to go chasing branch history to figure out the semantics of the change in question. "Amazing" is right. Sun was doing rebases in the 90s, and it never looked back.
- dom0 9y agoMy exact experience (in the context of "merging upwards"). Large merges are a huge pain to do, and are basically impossible to review, too.
- cryptonector 9y agoYes! Reviewing huge merges is infeasible. Besides, most CR tools are awful at capturing history, especially in multi-repo systems. So rebasing and keeping history clean and linear is a huge win there. Though, of course, rebasing is a win in general, even if you happen to have an awesome CR tool (a unicorn I've yet to run into).
- cryptonector 9y agoAnother thing is that keeping your unpushed commits "on top" is a great aid in general (e.g., it makes it trivial to answer what haven't I pushed here yet?"), but also is the source of rebasing's conflict resolution power. Because you're unpushed commits are on top, it's easy to isolate each set of merge conflicts (since you're going commit by commit) and to find the source of the conflicts upstream (with log/blame tools, without having to chase branch and merge histories).
- chocolatebunny 9y agoWe use a rebase workflow when working with third party source code. We keep all third party code on a main git branch and we create new branches off the main branch as we rebase our changes from third party code version to version.
- cryptonector 9y agoWhy wouldn't you use it for your own source?
- e40 9y agoCan you describe it in a little more detail? Do you still use branches? If so, for what? For different versions?
- cryptonector 9y agoGreat question. We basically had a single branch per repo, and every repo other than the master one was a fork (ala github). But that was dictated by the limitations of the VCS we used (Teamware) before the advent of Hg and git. So "branches" were just a developer's or project's private playgrounds. When done you pushed to the master (or abandoned the "branch"). Project branches got archived though. In a git world what this means is that you can have all the branches you want, and you can even push them to the master repo if that's ok with its maintainers, or else keep them in your forks (ala github) or in a repo meant for archival. But! There is only one true repo/branch, and that's the master branch in the master repo, and there are no merge commits in there. For developers working on large projects the workflow went like this: - clone the project repo/branch - work and commit, pulling --rebase periodically - push to the project repo/branch - when the project repo/branch rebases onto a newer upstream the developer has to rebase their downstream onto the rebased project repo/branch Project techleads or gatekeepers (larger projects could have someone be gatekeeper but not techlead) would be responsible for rebasing the project onto the latest upstream. To simplify things the upstream did a bi-weekly "release" (for internal purposes) that projects would rebase onto on either a bi-weekly or monthly schedule. This minimizes the number of rebases to do periodically. When the project nears the dev complete time, the project will start rebasing more frequently. For very large projects the upstream repo would close to all other developers so that the project could rebase, build, test, and push without having to rinse and repeat. (Elsewhere I've seen uni-repo systems where there is no closing of the upstream for large projects. There a push might have to restart many times because of other pushes finishing before it. This is a terrible problem. But manually having to "close" a repo is a pain too. I think that one could automate the process of prioritizing pushes so as to minimize the restarts.)
- nikital 9y agoA pain I have with rebase workflow is that it creates untested commits (because diffs were blindly applied to a new version of the code). If I rebase 100 commits, some of the commits will be subtly broken. How do you deal with that?
- cryptonector 9y agoWith git rebase you can in fact build and test each commit. That's what the 'exec' directive is for (among other things) in rebase scripts! Basically, if you pick a commit, and in the next line exec make && make check (or whatever) then that build & test command will run with the workspace HEAD at that commit. Add such an exec after every pick/squash/fixup and you'll build and test every commit.
- lsaferite 9y agoOr you could use the "-x" parameter to execute something between every rebase.
- ninkendo 9y agoThis is why, in git workflows with rebases, it's a good idea to create merge commits anyway, even if the master branch can fast-forwarded. That way, looking at the history, you know what commits are stable/tested by looking at merge commits. Others that were brought in since the last merge commit can be considered intermediary commits that don't need to be individually tested. (Of course, there's also the rebase-and-squash workflow which I've personally never used, but it accomplishes the same thing by erasing any intermediary history altogether.)
- cryptonector 9y agoI've toyed with the idea of using merge commits to record sets of commits as being... atoms. Suppose you want to push regression tests first, then bug fixes, but both together: this is useful for showing that the test catches the bug and the bug fix fixes it. But now you need to document that they go together, in case they need to be reverted, or cherry-picked onto release maintenance branches. I think branch push history is really something that should be a first-class feature. I could live with using merge commits (or otherwise empty-commits) to achieve this, but I'll be filtering them from history most of the time!
- emmelaich 9y agoAre you saying that you use Git instead of Mercurial these days? Not necessarily implied by you; just checking.
- cryptonector 9y agoMe personally? Yes, I use git whenever I can. I still have to use Mercurial for some things. I don't know what Oracle does nowadays with the gates that make up Solaris. My guess is that they still have a hodge podge, with some gates using git, some Mercurial, and some Teamware still. But that's just a guess. For all I know they may have done what Microsoft did and gone with a single repo for the whole thing.
- deleted 9y ago[deleted]
- emmelaich 9y agoThis is old but I meant Sun/Oracle/Java. I remember watching the process to choose a new VCS. Mercurial is a fine choice but not the best choice. Even over the duration of the decision process the world was clearly moving overwhelmingly to Git.
- rdubz 9y agoA problem with rebase workflows that I don't see addressed (here or in the replies) is: if I have, say, 20 local commits and am rebasing them on top of some upstream, I have to fix conflicts up to 20 times; in general I will have to stop to fix conflicts at least as many times as I would have to while merging (namely 0 or 1 times). Moreover, resolution work during a rebase creates a fake history that does not reflect how the work was actually done, which is antithetical to the spirit of version control, in a sense. A result of this is the loss of any ability to distinguish between bugs introduced in the original code (pre-rebase) vs. bugs introduced while resolving conflicts (which are arguably more likely in the rebase case since the total amount of conflict-resolving can be greater). It comes down to Resolution Work is Real Work: your code is different before and after resolution (possibly in ways you didn't intend!), and rebasing to keep the illusion of a total ordering of commits is a bit of an outdated/misuse of abstractions we now have available that can understand projects' evolution in a more sophisticated way. I was a dedicated rebaser for many years but have since decided that merging is superior, though we're still at the early stages of having sufficient tooling and awareness to properly leverage the more powerful "merge" abstraction, imho.
- ferdterguson 9y agoIMO, VC comes down not to tracking what was actually done, but to creating snapshots of logical steps that are reasonable to roll back to and git bisect with.
- cryptonector 9y agoAnd cherry-pick onto release maintenance branches.
- cryptonector 9y agoWell, git rerere helps here, though, honestly, this never happens to me even when I have 20 commits. Also, this is what you want, as it makes your commits easier to understand by others. Otherwise, with thousands of developers your merge graph is going to be a pile of incomprehensible spaghetti, and good luck cherry-picking commits into old release patch branches! Ah, right, that's another reason to rebase: because your history is clean, linear, and merge-free, it makes it easier to pick commits from the mainline into release maintenance branches. The "fake history" argument is no good. Who wants to see your "fix typo" commits if you never pushed code that needed them in the first place? I truly don't care how you worked your commits. I only care about the end result. Besides, if you have thousands of developers, each on a branch, each merging, then the upstream history will have an incomprehensible (i.e., _useless_) merge graph. History needs to be useful to those who will need it. Keep it clean to make it easier on them. Rebase _is_ the "more powerful merge abstraction", IMO.
- richardwhiuk 9y agoWhy is a clean linear history desirable? It's not reflective of how the product was built? Is it just for some naive desire of purity?
- ubolonton_ 9y agoIt's easier to see commits of a branch grouped together in most history viewers. Even though sorting commits topologically can help, most history viewers don't support that option. When there is an undesired behavior that is hard to reason about, git-bisect can be used to determine the commit that first introduced it. With a normal merge, it will point to the merge commit, because it was the first time the 2 branches interacted. With a rebase, git bisect will point to one of the rebased commits, each of which already interacted with the branch coming before. Resolving conflicts in a big merge commit vs in small rebased commits is like resolving conflicts in a distributed system by comparing only the final states, vs inspecting at the actual sequences of changes.
- cryptonector 9y agoWho cares about how a product's sub-projects were put together? What one should care about is how those sub-projects were put together into a final product. To be sure, the sub-projects' internal history can be archived, but it needn't pollute the upstream's history. Analogously: who cares how you think? Aside from psychologists and such, that is. We care about what you say, write, do.
- cryptonector 9y agoA trip through "git rebase" search results at HN sure is cringe-inducing. So many people fail to get it.