35 ms·

Trunk-Based Development

199 points by solarengineer 10y ago

hashkb 10y ago
The suggestion that feature branches hinder CD is just untrue, at least in my experience. To me, this seems like a silly restriction to put on such a critical tool (the scm) which is designed to support unlimited flexibility.
- nhumrich 10y ago
  You must not understand what CD really means then, because full CD is impossible with feature branches. CD means every commit gets built and delivered all the way prod if it passes all tests. The key word is "continuous" as in, every single commit. Branch based means you only deliver once you merge. Merging is a manual step, so your not doing full-CD. The merge is basically your "trigger" and your doing "delivery when I am ready"
  - geocar 10y ago
    CD is not impossible with feature branches. My live system looks up the branch name in the URL or in a (signed) cookie and checks out the appropriate branch then and there. This is possible because my development system is integrated into my live system.
    - nhumrich 10y ago
      So your delivering all commits on all branches to prod? That sounds hairy
      - geocar 10y ago
        How many commits do you do in a day? The hardest part is figuring out how to do this without a map.
        nhumrich 10y ago
        On a big team, potentially 100's. But if your delivering all commits regardless of branches, why even have branches?
        geocar 10y ago
        Because they're still useful? A branch is a useful tool for the developers who know their development process includes pushing to and pulling from branch "xyz" while they develop whatever feature they're tasked to develop. That they don't solve every problem doesn't bother me.
    - perlgeek 10y ago
      If push to two feature branches, A and B, which one lands in production? both? if so, how? or the last one you pushed? Or the one that finished its build later?
      - geocar 10y ago
        My specific implementation uses the git commit id. Only the tool for signing cookies actually knows about branches.
  - joneholland 10y ago
    We deploy every commit in master and use short lived feature branches and pull requests. I would still consider this continuous delivery.
    - hashkb 10y ago
      Yes, this is what I think of as CD. Hiding work that isn't ready (or work that may never make it in front of users) behind a flag on prod sounds like extra work for no gain, to me.
      - nhumrich 10y ago
        The gain is early integration. If you have more than one branch going on at the same time, they can diverge quite significantly, and merging can cause bugs, or sometimes more work. If you only ever have one feature branch at a time, you might as well be doing trunk based with manual deploys to prod.
        hashkb 10y ago
        Rebase early and often and this isn't an issue. Some see this as a chore, but I view it as taking advantage of the tool's power. You resolve the same set of conflicts as you would merging, but (as you say) earlier. Edit: I don't think anyone was trying to imply a single allowed feature branch at a time; we can probably all easily agree that's basically no different than working on master.
        nhumrich 10y ago
        Yes, rebase early and often works great. But if your branch includes a major refactor, your going to find yourself editing others code to get your rebase to work. That just increases your own scope. But yes, rebase often is often a very viable strategy. In practice though, I find very few that actually do it because it becomes painful when your branch lives too long simply because business isn't ready to ship something
        notarealdoctor 10y ago
  - hashkb 10y ago
    Anyway the article mentions Continuous Delivery, not Continuous Deployment. Neither means what you say, according to a Puppet Labs blog post, but the latter sometimes does to some people. But that's not what the article says, and so it's not what I'm countering in my comment.
  - jessaustin 10y ago
    So write the tests first; isn't that what we're supposed to do anyway? That way the branch doesn't move to prod on commit, because it isn't passing the tests yet. BTW "commit" is also a "manual step".
    - nhumrich 10y ago
      I'm kind of lost by your comment. > Write tests first What does this have to do with TBD vs branch based? Should be that way either way. In fact TBD breaks down if you don't write your tests. > A branch doesn't move to prod on a commit Are you saying your shipping branches to prod regardless as to if it's on trunk or not? Or perhaps your whole comment is just saying how TBD is supposed to work and your agreeing with me?
      - jessaustin 10y ago
        Ah, never mind, I was confused.
  - hamandcheese 10y ago
    How does code review fit in to such a workflow? That's also a manual step, but one I and many others consider good and necessary.
    - nhumrich 10y ago
      According to the article, and my personal experience, you don't commit directly to trunk. Trunk based means that you don't "collaborate" on a branch. So you write your code based off of trunk, once it's ready you commit in a branch, simply for purpose of code review. It gets reviewed, and probably runs an automated build. Once both pass, you merge and it goes through your deployment pipeline. Yes the merge/code review are manual, but your not merging to say, "I'm ready for this feature to be on prod now" your simply saying it looks good. The biggest difference is that if developer B wants to use your new method, he has to wait for it to be on trunk rather than branching off of your branch, etc.
      - gaastonsr 10y ago
        It really is the same. He very much could branched off your branch and once merged just start syncing with master. And I see that better than waiting for your PR to be merged.
      - biehl 10y ago
        So, like squashed feature branches?
  - notarealdoctor 10y ago
    > CD means every commit gets built and delivered all the way prod if it passes all tests. I always wondered why a company would allow code to go straight to production, live, affecting customers, without a human giving the OK to release.
    - dpark 10y ago
      Because you should be able to automate that OK. There is generally no verification that a human can do on a computer system that cannot be automated. The only question is whether you're willing to invest enough to fully automate the go/no go decision.
      - alkonaut 10y ago
        So the person that has an opportunity to misunderstand the specification (the developer) is given the task of writing the tests to OK them, even though the test is based on the same misunderstanding? That's terrifying. After 15 years in the same field I still don't have half the business knowledge as those writing the specs I implement. I would never ever want a nontrivial feature I wrote to hit a customer without manual testing by an expert in the area.
        dpark 10y ago
        Where do you work that software engineers write code and then hand off their code to non-software-engineer "experts" for testing? That sounds like a really broken process.
        alkonaut 10y ago
        I implement a program used by structural engineers. So basically the user is a structural engineer and I'm a software engineer. They typically find nuances of program behavior that I never thought of because I'm not an expert in structural engineering. I think the same would be true if I made a trading platform, an x-ray machine UI or whatever. When the expert uses it to do what they are experts in, they will invariably find issues (bugs, omissions, simple improvements) that weren't obvious to begin with. One can argue that if the spec was 100% perfect then I can always test it myself and hopefully even do so with automated tests - but I have never seen a spec like that (perhaps more importantly - if you have in house "end user like testers" for expert software then it's likely more economical to have expert testing and iterate than spend the time on more detailed upfront specs)
  - clusmore 10y ago
    Why do we have this obsession with religiously following methodologies rather than focusing on the goals they attempt to achieve? >CD means every commit gets built and delivered all the way prod if it passes all tests. The key word is "continuous" as in, every single commit. In my experience one of two things happen with this approach. Either the functionality is hidden behind flags and intentionally doesn't impact prod, in which case there's no point in it being there. Or you only commit when the feature is complete, which is effectively the same "delivery when I am ready" as with merges. The goal of CD is not to push every commit to prod -- this is merely the process. If the goal achieved by the process of pushing every commit to prod can be achieved via a different process, we shouldn't be so quick to criticise the other process. We should instead discuss the tradeoffs between the two, because in reality there is no best option, only tradeoffs.
  - UK-AL 10y ago
    What do you think unpushed/uncommited changes on your local machine. They are basically feature branches.
joneholland 10y ago
Trunk based development works well on any scm that sucks at branching. Perforce, subversion etc. But, on git, using GitHub-Flow is far superior. The two poster children for TBD do not use git. Don't cargo cult their process without understanding the unique problems they have that you don't. Edit: downvotes on HN? This isn't Reddit, and I'm advocating github flow, not git flow. GitHub flow is trunk based development but with all work in feature branches that live for less than s day or two, instead of branch by abstraction in the core code. That's it!
- nhumrich 10y ago
  Who are your two poster children? Because in my mind it's Amazon and Google, who both use git
  - joneholland 10y ago
    In the article it was google (perforce) and Facebook (mercurial with hacks). Both are massive mono repos. At Amazon they were mostly perforce when I was there. Granted that was a few years back.
    - nhumrich 10y ago
      I would say the bigger the repo becomes (in terms of # of collaborators) the more important trunk based becomes regardless of SCM choice. Mainly because merging many branches for many teams becomes very tedious and hairy. By the time you resolve all the merge conflicts, someone else will commit a merge and you'll have to resolve conflicts all over again.
      - joneholland 10y ago
        I agree, which is also why I think monorepos are an anti pattern.
        hamandcheese 10y ago
chvid 10y ago
This is what I do when I can get away with it. Suitable for smallish teams. All developers sit in the same branch; changes and conflicts are visible (and resolved) immediately. All work takes place there including bug fixes. When we branch out for release; the focus is on disabling unwanted features. Actual development work + "hardening" is done in the main trunk before the branch.
- paul_h 10y ago
  Small teams like Google's 25000 devs and QA automators? :)
  - chvid 10y ago
    I think it can work well with maybe 10-20 people. Depends on how well you software is modularised.
    - paul_h 10y ago
      So I'm saying given Google do Trunk Based Development (https://trunkbaseddevelopment.com/game-changers/#google-sharing-their-trunk-usage-2016 https://trunkbaseddevelopment.com/game-changers/#google-shar...) with that many committers, any size team can do it.
    - ithkuil 10y ago
      I assure you it works well with 25000 people, I've worked in such an environment. It seems to me that in the various threads I read here that there seems to be a bit of misunderstanding. If you work on short lived feature branches and merge them to trunk/master and other people branch new feature branches from trunk/master and so on you might be just doing TBD without calling it that way. Many things are just common sense consequences of that, e.g. if other people are spawning their feature branches from trunk/master; then you don't want to merge something that breaks trunk/master, otherwise you'd be making the life for your colleagues harder, since now they cannot know whether the code they added breaks some tests or they are broken because somebody else broke them. The feature-flag thing comes into play only when you want to break down a larger feature into smaller branches, each landing in trunk/master before continuing the next step. This get more important the more people can actually make changes on the same part of the codebase that you big feature is going to touch. If it's only you that touches a section of your code base (and you can do it because it's well modularized), you won't feel much pain of making your feature branch last longer. However, TBD shows its strengths precisely when the team grows; when you no longer can make the assumption that only you will work on a given piece of code.
geocar 10y ago
I described something that seems very similar years ago[1] (in response to git-flow), however I've since come up with something better: I think knowing how to get things live is important, and the entire development process should reflect that -- too many people have a "code cut" that takes days or even weeks, and that's despite using the latest and greatest CI tools. A lot of the problems may sound familiar: How do we test something? How do we know it's good enough? We have a UAT step, but it doesn't find all the problems, so do we need more tests? To be more careful? My approach[2] turns the entire problem into a software problem, and it's proving very successful to me (faster request turnaround, fewer problem reports post-deploy, and so on). Being able to select a "git branch name" for a specific user and get acceptance is powerful. [1]: https://news.ycombinator.com/item?id=6125964 https://news.ycombinator.com/item?id=6125964 [2]: https://news.ycombinator.com/item?id=11190540 https://news.ycombinator.com/item?id=11190540
rkaregaran 10y ago
Puppet Labs’ recently published State of DevOps (2016) report specifically calls out trunk-based development as a leading indicator of high performing organizations [1]. [1] https://puppet.com/resources/white-paper/2016-state-of-devops-report https://puppet.com/resources/white-paper/2016-state-of-devop...
- hashkb 10y ago
  Amount of info required to download free report makes it not free. Can you quote?
  - shawn-butler 10y ago
    I think this is the relevant portion for which you are looking: "We found that having branches or forks with very short lifetimes (less than a day) before being merged into trunk, and less than three active branches in total, are important aspects of continuous delivery, and all contribute to higher performance. So does merging code into trunk or master on a daily basis. Teams that don’t have code freeze periods (when people can’t merge code or pull requests) also achieve higher performance."
  - Tepix 10y ago
    Try this link: https://puppet.com/system/files/2016-06/2016%20State%20of%20DevOps%20Report_0.pdf?download=1 https://puppet.com/system/files/2016-06/2016%20State%20of%20...
  - nhumrich 10y ago
    All you have to do is enter an email address. It doesn't even have to be a real email.
    - Sir_Substance 10y ago
      It's pretty rude to ask every visitor to a free report to hand over contact details. What are you planning to do with all those contact details? I can only think of plans that range from annoying to malicious. None that are good. Don't normalize rude behavior. Conditionally free is not free.
vemv 10y ago
> Trunk Based Development is a key enabler of Continuous Integration, and by extension Continuous Delivery. Quite false. How do you expect developers to take you seriously when you essentially say "you can't properly do CI/CD with your current approach"? I sure am enjoying CI/CD right now. Trunk-based development seems to rely heavily on feature flags, which are a huge source of complexity and inconsistency. Even when useful, they're liabilities; not assets. git-flow (and variations) is simple, enables consistent codebase state, makes no assumptions about your codebase/infrastructure, and allows you to know the exact state of production/staging. Finally, as it happens so often with open source projects and initiatives, first thing I should see is a big "Why". Why do I want this? Why are the alternatives inferior?
- hashkb 10y ago
  I'm on your side but my comment (yours is better) faced opposition so: They redefine CD and then say nothing else is it, and you aren't doing it.
  - mbrock 10y ago
    It seems like many people are unaware that "continuous integration" since its inception has meant pretty much the same thing as "trunk based development." Wikipedia's definition: "In software engineering, continuous integration (CI) is the practice of merging all developer working copies to a shared mainline several times a day."
    - hashkb 10y ago
      That sounds like it allows for feature branches to me. Edit: anyway, isn't that extremely pedantic? I've known literally hundreds of companies that say "we are using CI" when what they mean is "master is continuously tested + deployed"... is there a name for that we should all start using? I'm not trying to diminish the awesomeness of "real" CD that some of you out there appear to be doing and proud of preserving the term for.
      - 10y ago
revelation 10y ago
No wonder they call it trunk, it's "branches in SVN are pure suck" elevated to some sort of grand vision. Must have been feeling left out with all the cool kids doing Git.
mpdehaan2 10y ago
Developing on master is something I favor, with tagging for releases, though it makes sense to create release branches if you are delivering software packages (that others will install) and not just deploying it to .com websites. Hotfixes can then be applied to both the maintaince branch and master.
whyileft 10y ago
If you need to hide feature work behind a flag on prod you're probably doing it wrong. There are a few times when you need to, but features can in most cases be written in a way that they can live inside production before complete. And learning to work like that also has the benefit of aggressively fighting features that drag on forever. There are definitely exceptions, but 9 times out of 10 learning to develop features in a way that they can be continuously integrated to production will dramatically reduce features dragging on, decrease bug heavy feature launches and increase velocity. EDIT: The downvoting system on HN optimizes for older, over the hill developers as is shown here once again.
- notarealdoctor 10y ago
  > There are a few times when you need to, but features can in most cases be written in a way that they can live inside production before complete. That should never, ever, ever be done.
  - nhumrich 10y ago
    Why not? Loads of companies do this. Almost all the big ones I know of do this.
  - paul_h 10y ago
    Really? http://techcrunch.com/2011/05/30/facebook-source-code http://techcrunch.com/2011/05/30/facebook-source-code
- nhumrich 10y ago
  I would say for backend you can get away without feature flags most of the time. But for the frontend, it's almost impossible to ship a new incomplete feature without feature flags
  - whyileft 10y ago
    That's actually a very valid point. I was not thinking as much about HTML/CSS as I haven't been involved in a lot of large scale frontend refactors.
ZenoArrow 10y ago
Correct me if I'm wrong but isn't this just a form of centralised version control? If this is a better approach for many teams, perhaps this an indication that DVCS weren't the silver bullet they were hyped up to be. Aside from that, in my opinion the whole centralised version control vs. decentralised version control, and all the variants thereof, miss the bigger issue. Keeping track of the history of a file, and the authors of the changes, is trivial for any version control system. The real challenge is in resolving merge conflicts, and it's the low level of sophistication in merge tools that is the real bottleneck here. That's why I think tools like Semantic Merge are far more important to a development workflow than SVN vs Git vs Mercurial. It relies on the merge tool understanding the structure of code rather than treating it as an ordinary text file. Similar tools could be built for any language that offers a 'compiler as a service' (such as RLS for Rust). https://www.semanticmerge.com/ https://www.semanticmerge.com/
- paul_h 10y ago
  SemanticMerge mentioned on the site: https://trunkbaseddevelopment.com/game-changers/#plasticscm-s-semantic-merge-2013 https://trunkbaseddevelopment.com/game-changers/#plasticscm-.... It got it's name after I blogged about it too - http://paulhammant.com/2013/02/14/features-i-would-love-source-control-tools-to-have/ http://paulhammant.com/2013/02/14/features-i-would-love-sour... - and am still owed a beer.
  - ZenoArrow 10y ago
    Thanks for the links, I'm glad you identified this as an issue and inspired further development of a solution (though I'm surprised you're owed a beer for such an obvious name, must've taken all of about 5 seconds to think of that one ;-) ). Also found this article in the comments of your blog to be interesting: http://blog.plasticscm.com/2013/04/put-your-hands-on-programming-language.html?m=1 http://blog.plasticscm.com/2013/04/put-your-hands-on-program... This article you linked to was interesting too, is this a common workflow on CI solutions now?: https://blog.snap-ci.com/blog/2013/11/07/automatic-branch-tracking-and-integration/ https://blog.snap-ci.com/blog/2013/11/07/automatic-branch-tr...
paul_h 10y ago
One of the authors here. Ask questions :)
- neandrake 10y ago
  Paul Hammant? Thank you for your blogs and thanks for putting this site together, they're very informative. I work at a company that has been doing "Cascade" model for close to a decade [0], very similar to TBD but not quite the same. In addition to having a mainline trunk repo which everyone's work is pushed to, we have started utilizing Phabricator in the past few years -- it enables us to have short-running feature branches which aren't public to the upstream repository but can be shared if necessary and used for review (I'm not sure how similar this might be to GitHub flow? We use Mercurial). [0] http://paulhammant.com/2013/12/04/what_is_your_branching_model/ http://paulhammant.com/2013/12/04/what_is_your_branching_mod...
teolemon 10y ago
What about Trump based developments ? (eg Trump Tower, Trump Resorts...)
kosma 10y ago
This workflow, like all others, is just a formalization of some reality. The reality is that many organizations and teams run on nothing but trunk - and it more-or-less works on them. Some teams work by building blocks separately and then joining them together; other prefer to hammer away on a problem all together. Both approaches work. What worries me are emotional claims that the other approach is fundamentally flawed, or that some git workflow is The One True Way. I've seen feature branches get stuck in review for weeks. I've seen trunk-only developers not knowing how to merge a branch back to trunk. I've seen trunks brought into a state of disarray and never fixed again. I've seen svn-only devs get completely bogged down and confused by a git wofkflow. Trunk-only is a reality. Not a reality some of us would want to live it - but one that nevertheless exists.
- paul_h 10y ago
  https://trunkbaseddevelopment.com/youre-doing-it-wrong/#merely-naming-a-branch-trunk https://trunkbaseddevelopment.com/youre-doing-it-wrong/#mere... Subversion having that TBT thing made it easy for people to respond 'yes we're doing trunk based development' when they're not
- crdoconnor 10y ago
  I would hesitate to proscribe a "one true way" as well but the longer I develop the more convinced I am that a having branch lives longer than a few days indicates a serious underlying problem (usually a lack of test coverage or faith in said tests).
  - brianwawok 10y ago
    How do you support a 3 year LTS branch for enterprise customers without long lived branches?
    - lloeki 10y ago
      IIUC your use case this is not the kind of branch that gets merged back, it's more of "another trunk" whose version is largely frozen and that may get backports and fixes.
whyileft 10y ago
Yup, as the other comment stated, this is par for the course in modern quality software development. Its actually rather shocking to me to see my comment get downvoted. The quality of the HN audience is declining dramatically. Feels like all we have is a bunch of web agency lifers at this point.
- macintux 10y ago
  I didn't know "CD" stand for "continuous denigration".
- grzm 10y ago
  The quality of the HN audience is declining dramatically Your account is 30 days old. On what basis are you making this remark?
- icebraining 10y ago
  Feature flags are used by essentially every major software company. Hell, the browser you're using to post this has quite a few. A bunch of major software companies use them heavily as well. You may disagree and think they're terrible, but calling their users just "web agency lifers" is objectively incorrect.
  - deleted 10y ago
    [deleted]
- dang 10y ago
  A microcode assembler has been in the top 5 for hours now. I suspect we're ok. It's tempting to draw general conclusions about HN from specific things you don't like, but that's sample bias. We detached this subthread from https://news.ycombinator.com/item?id=13514804 https://news.ycombinator.com/item?id=13514804 and marked it off-topic.
cat199 10y ago
With a sentence like: " It has been a lesser known branching model of choice since the mid-nineties, and considered tactically since the eighties. " Given: The Release Engineering of 4.3BSD https://docs.freebsd.org/44doc/papers/releng.html https://docs.freebsd.org/44doc/papers/releng.html 1989 concerning a key software system and part of software history, released in 1986, which states: " For the 4.3BSD release, certain users had permission to modify the master copy of the system source directly. ... The development phase continues until CSRG decides that it is appropriate to make a release. " ,that the rcs tools of the time basically didn't even support branches natively, and this model is followed to this day by the direct descendants of this project I'm prone to think that.. well. Hmm.
cygned 10y ago
We currently use short-lived feature branches, merged via Pull Requests (+ review / automated testing) into the main development line. This way, we can communicate changes in a detailed manner before they are added to the product and make sure there is no unfished or bad code in the main branch. (The dev team is small, 5-8 devs). I don't see (yet?), what benefits TBD would provide in such a setup.
- einrealist 10y ago
  With TBD you can reduce the overhead of coordination between committers. Edit for clarification: "Branching in code", á la Feature Toggles is much better, because everything around it can be automated. With CVS branches, you shift that process into an earlier development stage where you need more human collaboration. So TBD can shift the focus of collaboration to things that matter more: the code.
  - icebraining 10y ago
    Why can't you automatically merge all the branches for testing? Sure, sometimes it won't cleanly merge, but then again, features behind toggles won't always work well together.
    - qwer 10y ago
      I once wished this were possible too, but in reality it isn't. With n branches there are n! permutations to test. That's a lot of infrastructure to maintain and computing power to spend and you still don't end up with a single team-blessed deployment artifact that can automatically be promoted for manual testing or release. Don't get me wrong, feature-toggles are indeed a pain in the ass, but in my experience cherry-picking branches to merge really is much much worse.
      - icebraining 10y ago
        And with n feature flags, aren't there also n! permutations to test, that is, each combination of enabled/disabled? If you say no, because you just test features as they get manually enabled, why can't you do the same by manually selecting branches to be merged?
Animats 10y ago
"The core requirement of Continuous Integration that all team members commit to trunk at least once every 24 hours." Continuous integration is a means to an end, not an end in itself.
- jfim 10y ago
  Indeed. There's a wide spectrum of software complexity and while committing to trunk every day might be reasonable for lower complexity projects, it's definitely not for higher complexity ones. Having such a requirement also means that people can't undertake major refactorings/rewrites of significant subsystems, leading to long term tech debt.
  - dpark 10y ago
    In my experience, the "refactor the world in a separate branch" strategy is always a giant mess. The initial work is perhaps simpler but the final merge is always horrible and the full impact of breaks isn't found until the merge is committed and you're stuck in "fix forward" mode because reverting is an unacceptable choice due to the cost of merging. I'm not sure the overall cost to the team wouldn't be lower if these sorts of rewrites were done in place. (Also it's not really a refactoring if it's so big you can't do it in branch.)
    - shubb 10y ago
      Not disagreeing, but a way to avoid this is to pull changes from the main branch into the development branch every day, and write your new subsystem 'beside' the old one until it is ready to go. If you do it like this, it's not much easier for the branched developers than working on the trunk - it's still a moving codebase - but it does mean that if you cancel the rewrite there isn't any pollution to the main branch.
      - dpark 10y ago
        The scenario you called out, where the rewriting work might be abandoned/cancelled, is the only scenario I can think that makes a separate branch actually better. For cases where you're confident that the work will not be cancelled, you have the same amount of work to push the rewrite back into the main branch constantly as you do to constantly keep the child branch in sync. i.e. If you can pull from the main branch into the rewrite branch and build/test successfully, you can do the same in reverse. If you cannot pull from main into rewrite and build/test successfully, you're just postponing the pain to the end when you'll dump it on the rest of the team when you finally merge back and break everything in the main branch.
antoncohen 10y ago
I think some people are getting confused about terminology between Trunk Based Development[1], Git Flow[2], and GitHub Flow[3]. TBD and GitHub Flow are nearly identical, basically if you use TBD on GitHub you use GitHub Flow. Git Flow (not GitHub Flow) is a model that is all about cutting releases, and supporting bug fixes to releases. Git Flow is nonsensical when doing Continuous Delivery[4] of a web application. If you are delivering web app code to production every day, it doesn't make sense to cut versioned releases, with for example semantic versioning, instead you just release the code that has passed tests. It also doesn't make sense to have hotfix branches, because there is only one version in production[5], and you release every day, so there is no need to backport a fix to older versions, or do anything special to get a fix out quickly. Trunk Based Development has branches. If you are working with a team you pretty much never want to commit straight to master. The difference is that in TBD the branches are short-lived and only exist for code review purposes. This is what TBD would look like with GitHub and GitHub "forks". git remote -a origin git@github.com:<your company>/<repo>.git (fetch) origin git@github.com:<your company>/<repo>.git (push) <github username> git@github.com:<github username>/<repo>.git (fetch) <github username> git@github.com:<github username>/<repo>.git (push) git checkout master git pull # make sure you are starting with the latest master git checkout -b my-cool-feature # create a branch for you change vim <file> git add <file> git status # check that all expected code is staged git diff HEAD # review your changes git commit # write a meaningful commit message git fetch # get the latest changes from origin git rebase origin/master # put your changes on top of the latest changes from master git push <github username> my-cool-feature:my-cool-feature # push to a branch on your "fork" # Open a PR to merge from <github username>/my-cool-feature to origin/master # Once the PR is approved, use the GitHub UI to merge the PR into master git checkout master git pull # pull down your code that just merged to master git branch -d my-cool-feature # delete your feature branch [1] https://trunkbaseddevelopment.com/ https://trunkbaseddevelopment.com/ [2] http://nvie.com/posts/a-successful-git-branching-model/ http://nvie.com/posts/a-successful-git-branching-model/ [3] https://guides.github.com/introduction/flow/ https://guides.github.com/introduction/flow/ [4] I'm defining CD to mean delivering to production at least once a day. [5] Unlike on-premises software, with a SaaS product you only have two versions: (1) what is in production, and (2) what is going out to production (in staging, canary, etc.).
ungzd 10y ago
TLDR: instead of using diffs of changed lines in branches, use 'if' statements in code to make Enterprise Continuous Integration gods happy.
- UK-AL 10y ago
  The advantages of feature branches is that you can turn them on and off easily without trying merge in, or revert and dealing with the resultant conflicts this can cause. Feature switches also have more advanced features like being able targeting specific users, if you want a gradual rollout. We use both feature branches and feature switches, since branches short be very short lived. However large areas of development require months.
paulddraper 10y ago
> Google do Trunk Based Development and they have 25000 developers and QA automators in that trunk For some stuff, but Is Android trunk-only? No. Is V8 trunk-only? No. Is Chrome trunk-only? No. Those have release branches. --- Some software works with trunk-only paradigm, some does not. A rule of thumb is whether your customer-facing distribution is SaaS/web-based. If you're making Google Search, you do it in a trunk. If you're making Google Chrome, you don't.
- paul_h 10y ago
  > Those have release branches. Some Trunk Based Development teams do "branch for release" https://trunkbaseddevelopment.com/branch-for-release/ https://trunkbaseddevelopment.com/branch-for-release/ and some do "release from trunk" https://trunkbaseddevelopment.com/release-from-trunk/ https://trunkbaseddevelopment.com/release-from-trunk/ Google gave numbers themselves - 95% of their devs are in one big trunk (formerly perforce, but re-written in-house in 2012). It's true that the open source facing teams are outside that, and have processes that are less lock-step than what they do in-house. Android - Samsung, on each release (maybe not anymore), used to check all the composite parts of Android into one big trunk for themselves. Back into their Perforce. > If you're making Google Chrome, you don't. There's nothing stopping the Chrome team from doing Trunk Based Development on Github, and accepting unsolicited pull requests like any other team.
  - paulddraper 10y ago
    You can have long lived release branches in "trunk-based development"?!? In that case, I guess I haven't ever seen a branching strategy that wasn't "trunk-based." What would that even be?
    - qwer 10y ago
      Anything involving long-lived feature branches especially ones with multiple collaborators, eg git-flow.
      - 10y ago
z3t4 10y ago
I read Drunk-Based Development and was a bit disappointed. I can get awesome ideas while very tired and usually write them down, but a few weeks later, even though I was super exited when I wrote it down, it no longer makes any sense ...
- lucideer 10y ago
  I read Trump-Based Development...
ricotijsen 10y ago
I read that as "Trump-based development"
xorcist 10y ago
If there was indeed a superior way of developing software, why doesn't anyone take this knowledge and out-compete for example Linux? It has literally hundreds of branches of all kinds (stable branches, feature branches, test branches), this should be easy pickings! Or out-compete Google or Microsoft which, contrary to whats said in this article, use branches and probably in all their big projects. Sorry, I don't buy it. I suspect that if you are religiously against branches you will just have to re-invent them by some other name.
stcredzero 10y ago
The Envy configuration management system for Smalltalk worked like this. As you were working, you could see if someone had changed code "nearby" (same class, same category, same protocol) and you could review it and merge it in. Perforce provides something with the same effect. If you periodically get the most recent revision, you can see files with collisions in your change sets, then merge them in.
alkonaut 10y ago
In subversion the only way I can possible develop with sanity is in an "unstable trunk". Branch off releases N weeks before shipping, and have only the requirement that trunk passes automated tests while release branches are manually tested. Obviously you don't release more than say once every month or two, but that's more than enough for most. I'm honestly not sure if the site is satire? No one develops in long lived feature branches in svn (rather, everyone does, but most only try it once before they realize the pain of trunk based dev is much smaller than that of svn merging).
- sydd 10y ago
  If you release only a month or 2 you are doing something really wrong (or need to work based on some insane spec). Read the basics about CI and CD, read the clean code book, read about modern organisation practices (e.g. Github releases several times a day). Releasing every month or so is (thank God) a relic of the old waterfall development times.
  - rabidrat 10y ago
    I would say this must be satire but you seem sincere. You are promoting a thin slice of modern "best practices" which are not the only way to do software development. And it doesn't sound like you've ever worked on a major software project if you think that a month is a prohibitively long development timeframe. A mid-size architectural refactor of a large codebase can and maybe should take a month. When you get into the 200k+ SLOC range, no one person understands the entire thing, and most of us don't want to 'move fast and break things' when a little forethought, communication, and analysis can avoid it. I think this 'growth at all costs mentality' is ruining a lot more than our software, too.
  - diek 10y ago
    No, they're not. Is IntelliJ "doing it wrong" to release every three months[1] ? Absolutely not. A release can change things about a product, and you want to ensure you are pushing those changes (UI, behavior, plugin APIs) at a pace that matches the users. You don't want to be pushing changes to a plugin API every day. The surface area of stability that has to be maintained at that point is insane. In the interim, you should definitely be able to build off master. Your CICD server should build and run tests on every commit to master. Then a release gets a tag in version control and you should be able to issue bug fix releases for only that specific version. Telling customers, "There is a critical security bug in version X, but in order to fix that bug you need to upgrade completely to version X+2" is not an acceptable answer. Note that the description above works best in my experience for products that push a release artifact (like IntelliJ) versus a service like GitHub. Release management approaches will vary based on the product. [1] https://www.jetbrains.com/idea/download/previous.html https://www.jetbrains.com/idea/download/previous.html
jwatte 10y ago
Trunk only goes well with continuous deployment. If each increment in deployment is small enough, then you know exactly when things break and rollbacks are easy. The 3-2-1 system where you develop for 3 months, then merge for 2 months, then debug in production for 1 month, is a real drag, and doesn't lend itself to quick feedback at all. For a software looking for product market fit with ultra short cycles, I would absolutely choose trunk only. For engineering projects in areas where cycles are months, or years, more traditional branching makes more sense.
cwills 10y ago
We have a smallish team working on web apps and use a trunk-based-development branching strategy. Not really to do with helping CI/CD. The advantage we see is that new developers can start developing as soon as they clone the repository, without needing to switch to the 'development' branch. This is particularly useful when on boarding new developers who are inexperienced with Git. I found that when using Git Flow that the (stable) master branch frequently ends up out-of-date when developers are required to remember to merge into it releases into it. Diagram of our branching strategy: https://gist.github.com/CameronWills/abf9e307669b1005c88ef82e55102cd0 https://gist.github.com/CameronWills/abf9e307669b1005c88ef82...
- paul_h 10y ago
  I looked at your diagram. I don;t think you're doing trunk based development.

199 points by solarengineer 10y ago

The suggestion that feature branches hinder CD is just untrue, at least in my experience. To me, this seems like a silly restriction to put on such a critical tool (the scm) which is designed to support unlimited flexibility.

nhumrich 10y ago
You must not understand what CD really means then, because full CD is impossible with feature branches. CD means every commit gets built and delivered all the way prod if it passes all tests. The key word is "continuous" as in, every single commit. Branch based means you only deliver once you merge. Merging is a manual step, so your not doing full-CD. The merge is basically your "trigger" and your doing "delivery when I am ready"
- geocar 10y ago
  CD is not impossible with feature branches. My live system looks up the branch name in the URL or in a (signed) cookie and checks out the appropriate branch then and there. This is possible because my development system is integrated into my live system.
  - nhumrich 10y ago
    So your delivering all commits on all branches to prod? That sounds hairy
    - geocar 10y ago
      How many commits do you do in a day? The hardest part is figuring out how to do this without a map.
      - nhumrich 10y ago
        On a big team, potentially 100's. But if your delivering all commits regardless of branches, why even have branches?
        geocar 10y ago
        Because they're still useful? A branch is a useful tool for the developers who know their development process includes pushing to and pulling from branch "xyz" while they develop whatever feature they're tasked to develop. That they don't solve every problem doesn't bother me.
  - perlgeek 10y ago
    If push to two feature branches, A and B, which one lands in production? both? if so, how? or the last one you pushed? Or the one that finished its build later?
    - geocar 10y ago
      My specific implementation uses the git commit id. Only the tool for signing cookies actually knows about branches.
- joneholland 10y ago
  We deploy every commit in master and use short lived feature branches and pull requests. I would still consider this continuous delivery.
  - hashkb 10y ago
    Yes, this is what I think of as CD. Hiding work that isn't ready (or work that may never make it in front of users) behind a flag on prod sounds like extra work for no gain, to me.
    - nhumrich 10y ago
      The gain is early integration. If you have more than one branch going on at the same time, they can diverge quite significantly, and merging can cause bugs, or sometimes more work. If you only ever have one feature branch at a time, you might as well be doing trunk based with manual deploys to prod.
      - hashkb 10y ago
        Rebase early and often and this isn't an issue. Some see this as a chore, but I view it as taking advantage of the tool's power. You resolve the same set of conflicts as you would merging, but (as you say) earlier. Edit: I don't think anyone was trying to imply a single allowed feature branch at a time; we can probably all easily agree that's basically no different than working on master.
        nhumrich 10y ago
        Yes, rebase early and often works great. But if your branch includes a major refactor, your going to find yourself editing others code to get your rebase to work. That just increases your own scope. But yes, rebase often is often a very viable strategy. In practice though, I find very few that actually do it because it becomes painful when your branch lives too long simply because business isn't ready to ship something
        notarealdoctor 10y ago
- hashkb 10y ago
  Anyway the article mentions Continuous Delivery, not Continuous Deployment. Neither means what you say, according to a Puppet Labs blog post, but the latter sometimes does to some people. But that's not what the article says, and so it's not what I'm countering in my comment.
- jessaustin 10y ago
  So write the tests first; isn't that what we're supposed to do anyway? That way the branch doesn't move to prod on commit, because it isn't passing the tests yet. BTW "commit" is also a "manual step".
  - nhumrich 10y ago
    I'm kind of lost by your comment. > Write tests first What does this have to do with TBD vs branch based? Should be that way either way. In fact TBD breaks down if you don't write your tests. > A branch doesn't move to prod on a commit Are you saying your shipping branches to prod regardless as to if it's on trunk or not? Or perhaps your whole comment is just saying how TBD is supposed to work and your agreeing with me?
    - jessaustin 10y ago
      Ah, never mind, I was confused.
- hamandcheese 10y ago
  How does code review fit in to such a workflow? That's also a manual step, but one I and many others consider good and necessary.
  - nhumrich 10y ago
    According to the article, and my personal experience, you don't commit directly to trunk. Trunk based means that you don't "collaborate" on a branch. So you write your code based off of trunk, once it's ready you commit in a branch, simply for purpose of code review. It gets reviewed, and probably runs an automated build. Once both pass, you merge and it goes through your deployment pipeline. Yes the merge/code review are manual, but your not merging to say, "I'm ready for this feature to be on prod now" your simply saying it looks good. The biggest difference is that if developer B wants to use your new method, he has to wait for it to be on trunk rather than branching off of your branch, etc.
    - gaastonsr 10y ago
      It really is the same. He very much could branched off your branch and once merged just start syncing with master. And I see that better than waiting for your PR to be merged.
    - biehl 10y ago
      So, like squashed feature branches?
- notarealdoctor 10y ago
  > CD means every commit gets built and delivered all the way prod if it passes all tests. I always wondered why a company would allow code to go straight to production, live, affecting customers, without a human giving the OK to release.
  - dpark 10y ago
    Because you should be able to automate that OK. There is generally no verification that a human can do on a computer system that cannot be automated. The only question is whether you're willing to invest enough to fully automate the go/no go decision.
    - alkonaut 10y ago
      So the person that has an opportunity to misunderstand the specification (the developer) is given the task of writing the tests to OK them, even though the test is based on the same misunderstanding? That's terrifying. After 15 years in the same field I still don't have half the business knowledge as those writing the specs I implement. I would never ever want a nontrivial feature I wrote to hit a customer without manual testing by an expert in the area.
      - dpark 10y ago
        Where do you work that software engineers write code and then hand off their code to non-software-engineer "experts" for testing? That sounds like a really broken process.
        alkonaut 10y ago
        I implement a program used by structural engineers. So basically the user is a structural engineer and I'm a software engineer. They typically find nuances of program behavior that I never thought of because I'm not an expert in structural engineering. I think the same would be true if I made a trading platform, an x-ray machine UI or whatever. When the expert uses it to do what they are experts in, they will invariably find issues (bugs, omissions, simple improvements) that weren't obvious to begin with. One can argue that if the spec was 100% perfect then I can always test it myself and hopefully even do so with automated tests - but I have never seen a spec like that (perhaps more importantly - if you have in house "end user like testers" for expert software then it's likely more economical to have expert testing and iterate than spend the time on more detailed upfront specs)
- clusmore 10y ago
  Why do we have this obsession with religiously following methodologies rather than focusing on the goals they attempt to achieve? >CD means every commit gets built and delivered all the way prod if it passes all tests. The key word is "continuous" as in, every single commit. In my experience one of two things happen with this approach. Either the functionality is hidden behind flags and intentionally doesn't impact prod, in which case there's no point in it being there. Or you only commit when the feature is complete, which is effectively the same "delivery when I am ready" as with merges. The goal of CD is not to push every commit to prod -- this is merely the process. If the goal achieved by the process of pushing every commit to prod can be achieved via a different process, we shouldn't be so quick to criticise the other process. We should instead discuss the tradeoffs between the two, because in reality there is no best option, only tradeoffs.
- UK-AL 10y ago
  What do you think unpushed/uncommited changes on your local machine. They are basically feature branches.

joneholland 10y ago

Trunk based development works well on any scm that sucks at branching. Perforce, subversion etc. But, on git, using GitHub-Flow is far superior. The two poster children for TBD do not use git. Don't cargo cult their process without understanding the unique problems they have that you don't. Edit: downvotes on HN? This isn't Reddit, and I'm advocating github flow, not git flow. GitHub flow is trunk based development but with all work in feature branches that live for less than s day or two, instead of branch by abstraction in the core code. That's it!

nhumrich 10y ago
Who are your two poster children? Because in my mind it's Amazon and Google, who both use git
- joneholland 10y ago
  In the article it was google (perforce) and Facebook (mercurial with hacks). Both are massive mono repos. At Amazon they were mostly perforce when I was there. Granted that was a few years back.
  - nhumrich 10y ago
    I would say the bigger the repo becomes (in terms of # of collaborators) the more important trunk based becomes regardless of SCM choice. Mainly because merging many branches for many teams becomes very tedious and hairy. By the time you resolve all the merge conflicts, someone else will commit a merge and you'll have to resolve conflicts all over again.
    - joneholland 10y ago
      I agree, which is also why I think monorepos are an anti pattern.
      - hamandcheese 10y ago

chvid 10y ago

This is what I do when I can get away with it. Suitable for smallish teams. All developers sit in the same branch; changes and conflicts are visible (and resolved) immediately. All work takes place there including bug fixes. When we branch out for release; the focus is on disabling unwanted features. Actual development work + "hardening" is done in the main trunk before the branch.

paul_h 10y ago
Small teams like Google's 25000 devs and QA automators? :)
- chvid 10y ago
  I think it can work well with maybe 10-20 people. Depends on how well you software is modularised.
  - paul_h 10y ago
    So I'm saying given Google do Trunk Based Development (https://trunkbaseddevelopment.com/game-changers/#google-sharing-their-trunk-usage-2016 https://trunkbaseddevelopment.com/game-changers/#google-shar...) with that many committers, any size team can do it.
  - ithkuil 10y ago
    I assure you it works well with 25000 people, I've worked in such an environment. It seems to me that in the various threads I read here that there seems to be a bit of misunderstanding. If you work on short lived feature branches and merge them to trunk/master and other people branch new feature branches from trunk/master and so on you might be just doing TBD without calling it that way. Many things are just common sense consequences of that, e.g. if other people are spawning their feature branches from trunk/master; then you don't want to merge something that breaks trunk/master, otherwise you'd be making the life for your colleagues harder, since now they cannot know whether the code they added breaks some tests or they are broken because somebody else broke them. The feature-flag thing comes into play only when you want to break down a larger feature into smaller branches, each landing in trunk/master before continuing the next step. This get more important the more people can actually make changes on the same part of the codebase that you big feature is going to touch. If it's only you that touches a section of your code base (and you can do it because it's well modularized), you won't feel much pain of making your feature branch last longer. However, TBD shows its strengths precisely when the team grows; when you no longer can make the assumption that only you will work on a given piece of code.

geocar 10y ago

I described something that seems very similar years ago[1] (in response to git-flow), however I've since come up with something better: I think knowing how to get things live is important, and the entire development process should reflect that -- too many people have a "code cut" that takes days or even weeks, and that's despite using the latest and greatest CI tools. A lot of the problems may sound familiar: How do we test something? How do we know it's good enough? We have a UAT step, but it doesn't find all the problems, so do we need more tests? To be more careful? My approach[2] turns the entire problem into a software problem, and it's proving very successful to me (faster request turnaround, fewer problem reports post-deploy, and so on). Being able to select a "git branch name" for a specific user and get acceptance is powerful. [1]: https://news.ycombinator.com/item?id=6125964 https://news.ycombinator.com/item?id=6125964 [2]: https://news.ycombinator.com/item?id=11190540 https://news.ycombinator.com/item?id=11190540

rkaregaran 10y ago

Puppet Labs’ recently published State of DevOps (2016) report specifically calls out trunk-based development as a leading indicator of high performing organizations [1]. [1] https://puppet.com/resources/white-paper/2016-state-of-devops-report https://puppet.com/resources/white-paper/2016-state-of-devop...

hashkb 10y ago
Amount of info required to download free report makes it not free. Can you quote?
- shawn-butler 10y ago
  I think this is the relevant portion for which you are looking: "We found that having branches or forks with very short lifetimes (less than a day) before being merged into trunk, and less than three active branches in total, are important aspects of continuous delivery, and all contribute to higher performance. So does merging code into trunk or master on a daily basis. Teams that don’t have code freeze periods (when people can’t merge code or pull requests) also achieve higher performance."
- Tepix 10y ago
  Try this link: https://puppet.com/system/files/2016-06/2016%20State%20of%20DevOps%20Report_0.pdf?download=1 https://puppet.com/system/files/2016-06/2016%20State%20of%20...
- nhumrich 10y ago
  All you have to do is enter an email address. It doesn't even have to be a real email.
  - Sir_Substance 10y ago
    It's pretty rude to ask every visitor to a free report to hand over contact details. What are you planning to do with all those contact details? I can only think of plans that range from annoying to malicious. None that are good. Don't normalize rude behavior. Conditionally free is not free.

vemv 10y ago

> Trunk Based Development is a key enabler of Continuous Integration, and by extension Continuous Delivery. Quite false. How do you expect developers to take you seriously when you essentially say "you can't properly do CI/CD with your current approach"? I sure am enjoying CI/CD right now. Trunk-based development seems to rely heavily on feature flags, which are a huge source of complexity and inconsistency. Even when useful, they're liabilities; not assets. git-flow (and variations) is simple, enables consistent codebase state, makes no assumptions about your codebase/infrastructure, and allows you to know the exact state of production/staging. Finally, as it happens so often with open source projects and initiatives, first thing I should see is a big "Why". Why do I want this? Why are the alternatives inferior?

hashkb 10y ago
I'm on your side but my comment (yours is better) faced opposition so: They redefine CD and then say nothing else is it, and you aren't doing it.
- mbrock 10y ago
  It seems like many people are unaware that "continuous integration" since its inception has meant pretty much the same thing as "trunk based development." Wikipedia's definition: "In software engineering, continuous integration (CI) is the practice of merging all developer working copies to a shared mainline several times a day."
  - hashkb 10y ago
    That sounds like it allows for feature branches to me. Edit: anyway, isn't that extremely pedantic? I've known literally hundreds of companies that say "we are using CI" when what they mean is "master is continuously tested + deployed"... is there a name for that we should all start using? I'm not trying to diminish the awesomeness of "real" CD that some of you out there appear to be doing and proud of preserving the term for.
    - 10y ago

revelation 10y ago

No wonder they call it trunk, it's "branches in SVN are pure suck" elevated to some sort of grand vision. Must have been feeling left out with all the cool kids doing Git.

mpdehaan2 10y ago

Developing on master is something I favor, with tagging for releases, though it makes sense to create release branches if you are delivering software packages (that others will install) and not just deploying it to .com websites. Hotfixes can then be applied to both the maintaince branch and master.

whyileft 10y ago

If you need to hide feature work behind a flag on prod you're probably doing it wrong. There are a few times when you need to, but features can in most cases be written in a way that they can live inside production before complete. And learning to work like that also has the benefit of aggressively fighting features that drag on forever. There are definitely exceptions, but 9 times out of 10 learning to develop features in a way that they can be continuously integrated to production will dramatically reduce features dragging on, decrease bug heavy feature launches and increase velocity. EDIT: The downvoting system on HN optimizes for older, over the hill developers as is shown here once again.

notarealdoctor 10y ago
> There are a few times when you need to, but features can in most cases be written in a way that they can live inside production before complete. That should never, ever, ever be done.
- nhumrich 10y ago
  Why not? Loads of companies do this. Almost all the big ones I know of do this.
- paul_h 10y ago
  Really? http://techcrunch.com/2011/05/30/facebook-source-code http://techcrunch.com/2011/05/30/facebook-source-code
nhumrich 10y ago
I would say for backend you can get away without feature flags most of the time. But for the frontend, it's almost impossible to ship a new incomplete feature without feature flags
- whyileft 10y ago
  That's actually a very valid point. I was not thinking as much about HTML/CSS as I haven't been involved in a lot of large scale frontend refactors.

ZenoArrow 10y ago

Correct me if I'm wrong but isn't this just a form of centralised version control? If this is a better approach for many teams, perhaps this an indication that DVCS weren't the silver bullet they were hyped up to be. Aside from that, in my opinion the whole centralised version control vs. decentralised version control, and all the variants thereof, miss the bigger issue. Keeping track of the history of a file, and the authors of the changes, is trivial for any version control system. The real challenge is in resolving merge conflicts, and it's the low level of sophistication in merge tools that is the real bottleneck here. That's why I think tools like Semantic Merge are far more important to a development workflow than SVN vs Git vs Mercurial. It relies on the merge tool understanding the structure of code rather than treating it as an ordinary text file. Similar tools could be built for any language that offers a 'compiler as a service' (such as RLS for Rust). https://www.semanticmerge.com/ https://www.semanticmerge.com/

paul_h 10y ago
SemanticMerge mentioned on the site: https://trunkbaseddevelopment.com/game-changers/#plasticscm-s-semantic-merge-2013 https://trunkbaseddevelopment.com/game-changers/#plasticscm-.... It got it's name after I blogged about it too - http://paulhammant.com/2013/02/14/features-i-would-love-source-control-tools-to-have/ http://paulhammant.com/2013/02/14/features-i-would-love-sour... - and am still owed a beer.
- ZenoArrow 10y ago
  Thanks for the links, I'm glad you identified this as an issue and inspired further development of a solution (though I'm surprised you're owed a beer for such an obvious name, must've taken all of about 5 seconds to think of that one ;-) ). Also found this article in the comments of your blog to be interesting: http://blog.plasticscm.com/2013/04/put-your-hands-on-programming-language.html?m=1 http://blog.plasticscm.com/2013/04/put-your-hands-on-program... This article you linked to was interesting too, is this a common workflow on CI solutions now?: https://blog.snap-ci.com/blog/2013/11/07/automatic-branch-tracking-and-integration/ https://blog.snap-ci.com/blog/2013/11/07/automatic-branch-tr...

paul_h 10y ago

One of the authors here. Ask questions :)

neandrake 10y ago
Paul Hammant? Thank you for your blogs and thanks for putting this site together, they're very informative. I work at a company that has been doing "Cascade" model for close to a decade [0], very similar to TBD but not quite the same. In addition to having a mainline trunk repo which everyone's work is pushed to, we have started utilizing Phabricator in the past few years -- it enables us to have short-running feature branches which aren't public to the upstream repository but can be shared if necessary and used for review (I'm not sure how similar this might be to GitHub flow? We use Mercurial). [0] http://paulhammant.com/2013/12/04/what_is_your_branching_model/ http://paulhammant.com/2013/12/04/what_is_your_branching_mod...

teolemon 10y ago

What about Trump based developments ? (eg Trump Tower, Trump Resorts...)

kosma 10y ago

This workflow, like all others, is just a formalization of some reality. The reality is that many organizations and teams run on nothing but trunk - and it more-or-less works on them. Some teams work by building blocks separately and then joining them together; other prefer to hammer away on a problem all together. Both approaches work. What worries me are emotional claims that the other approach is fundamentally flawed, or that some git workflow is The One True Way. I've seen feature branches get stuck in review for weeks. I've seen trunk-only developers not knowing how to merge a branch back to trunk. I've seen trunks brought into a state of disarray and never fixed again. I've seen svn-only devs get completely bogged down and confused by a git wofkflow. Trunk-only is a reality. Not a reality some of us would want to live it - but one that nevertheless exists.

paul_h 10y ago
https://trunkbaseddevelopment.com/youre-doing-it-wrong/#merely-naming-a-branch-trunk https://trunkbaseddevelopment.com/youre-doing-it-wrong/#mere... Subversion having that TBT thing made it easy for people to respond 'yes we're doing trunk based development' when they're not
crdoconnor 10y ago
I would hesitate to proscribe a "one true way" as well but the longer I develop the more convinced I am that a having branch lives longer than a few days indicates a serious underlying problem (usually a lack of test coverage or faith in said tests).
- brianwawok 10y ago
  How do you support a 3 year LTS branch for enterprise customers without long lived branches?
  - lloeki 10y ago
    IIUC your use case this is not the kind of branch that gets merged back, it's more of "another trunk" whose version is largely frozen and that may get backports and fixes.

whyileft 10y ago

Yup, as the other comment stated, this is par for the course in modern quality software development. Its actually rather shocking to me to see my comment get downvoted. The quality of the HN audience is declining dramatically. Feels like all we have is a bunch of web agency lifers at this point.

macintux 10y ago
I didn't know "CD" stand for "continuous denigration".
grzm 10y ago
The quality of the HN audience is declining dramatically Your account is 30 days old. On what basis are you making this remark?
icebraining 10y ago
Feature flags are used by essentially every major software company. Hell, the browser you're using to post this has quite a few. A bunch of major software companies use them heavily as well. You may disagree and think they're terrible, but calling their users just "web agency lifers" is objectively incorrect.
- deleted 10y ago
  [deleted]
dang 10y ago
A microcode assembler has been in the top 5 for hours now. I suspect we're ok. It's tempting to draw general conclusions about HN from specific things you don't like, but that's sample bias. We detached this subthread from https://news.ycombinator.com/item?id=13514804 https://news.ycombinator.com/item?id=13514804 and marked it off-topic.

cat199 10y ago

With a sentence like: " It has been a lesser known branching model of choice since the mid-nineties, and considered tactically since the eighties. " Given: The Release Engineering of 4.3BSD https://docs.freebsd.org/44doc/papers/releng.html https://docs.freebsd.org/44doc/papers/releng.html 1989 concerning a key software system and part of software history, released in 1986, which states: " For the 4.3BSD release, certain users had permission to modify the master copy of the system source directly. ... The development phase continues until CSRG decides that it is appropriate to make a release. " ,that the rcs tools of the time basically didn't even support branches natively, and this model is followed to this day by the direct descendants of this project I'm prone to think that.. well. Hmm.

cygned 10y ago

We currently use short-lived feature branches, merged via Pull Requests (+ review / automated testing) into the main development line. This way, we can communicate changes in a detailed manner before they are added to the product and make sure there is no unfished or bad code in the main branch. (The dev team is small, 5-8 devs). I don't see (yet?), what benefits TBD would provide in such a setup.

einrealist 10y ago
With TBD you can reduce the overhead of coordination between committers. Edit for clarification: "Branching in code", á la Feature Toggles is much better, because everything around it can be automated. With CVS branches, you shift that process into an earlier development stage where you need more human collaboration. So TBD can shift the focus of collaboration to things that matter more: the code.
- icebraining 10y ago
  Why can't you automatically merge all the branches for testing? Sure, sometimes it won't cleanly merge, but then again, features behind toggles won't always work well together.
  - qwer 10y ago
    I once wished this were possible too, but in reality it isn't. With n branches there are n! permutations to test. That's a lot of infrastructure to maintain and computing power to spend and you still don't end up with a single team-blessed deployment artifact that can automatically be promoted for manual testing or release. Don't get me wrong, feature-toggles are indeed a pain in the ass, but in my experience cherry-picking branches to merge really is much much worse.
    - icebraining 10y ago
      And with n feature flags, aren't there also n! permutations to test, that is, each combination of enabled/disabled? If you say no, because you just test features as they get manually enabled, why can't you do the same by manually selecting branches to be merged?

Animats 10y ago

"The core requirement of Continuous Integration that all team members commit to trunk at least once every 24 hours." Continuous integration is a means to an end, not an end in itself.

jfim 10y ago
Indeed. There's a wide spectrum of software complexity and while committing to trunk every day might be reasonable for lower complexity projects, it's definitely not for higher complexity ones. Having such a requirement also means that people can't undertake major refactorings/rewrites of significant subsystems, leading to long term tech debt.
- dpark 10y ago
  In my experience, the "refactor the world in a separate branch" strategy is always a giant mess. The initial work is perhaps simpler but the final merge is always horrible and the full impact of breaks isn't found until the merge is committed and you're stuck in "fix forward" mode because reverting is an unacceptable choice due to the cost of merging. I'm not sure the overall cost to the team wouldn't be lower if these sorts of rewrites were done in place. (Also it's not really a refactoring if it's so big you can't do it in branch.)
  - shubb 10y ago
    Not disagreeing, but a way to avoid this is to pull changes from the main branch into the development branch every day, and write your new subsystem 'beside' the old one until it is ready to go. If you do it like this, it's not much easier for the branched developers than working on the trunk - it's still a moving codebase - but it does mean that if you cancel the rewrite there isn't any pollution to the main branch.
    - dpark 10y ago
      The scenario you called out, where the rewriting work might be abandoned/cancelled, is the only scenario I can think that makes a separate branch actually better. For cases where you're confident that the work will not be cancelled, you have the same amount of work to push the rewrite back into the main branch constantly as you do to constantly keep the child branch in sync. i.e. If you can pull from the main branch into the rewrite branch and build/test successfully, you can do the same in reverse. If you cannot pull from main into rewrite and build/test successfully, you're just postponing the pain to the end when you'll dump it on the rest of the team when you finally merge back and break everything in the main branch.

antoncohen 10y ago

I think some people are getting confused about terminology between Trunk Based Development[1], Git Flow[2], and GitHub Flow[3]. TBD and GitHub Flow are nearly identical, basically if you use TBD on GitHub you use GitHub Flow. Git Flow (not GitHub Flow) is a model that is all about cutting releases, and supporting bug fixes to releases. Git Flow is nonsensical when doing Continuous Delivery[4] of a web application. If you are delivering web app code to production every day, it doesn't make sense to cut versioned releases, with for example semantic versioning, instead you just release the code that has passed tests. It also doesn't make sense to have hotfix branches, because there is only one version in production[5], and you release every day, so there is no need to backport a fix to older versions, or do anything special to get a fix out quickly. Trunk Based Development has branches. If you are working with a team you pretty much never want to commit straight to master. The difference is that in TBD the branches are short-lived and only exist for code review purposes. This is what TBD would look like with GitHub and GitHub "forks". git remote -a origin git@github.com:<your company>/<repo>.git (fetch) origin git@github.com:<your company>/<repo>.git (push) <github username> git@github.com:<github username>/<repo>.git (fetch) <github username> git@github.com:<github username>/<repo>.git (push) git checkout master git pull # make sure you are starting with the latest master git checkout -b my-cool-feature # create a branch for you change vim <file> git add <file> git status # check that all expected code is staged git diff HEAD # review your changes git commit # write a meaningful commit message git fetch # get the latest changes from origin git rebase origin/master # put your changes on top of the latest changes from master git push <github username> my-cool-feature:my-cool-feature # push to a branch on your "fork" # Open a PR to merge from <github username>/my-cool-feature to origin/master # Once the PR is approved, use the GitHub UI to merge the PR into master git checkout master git pull # pull down your code that just merged to master git branch -d my-cool-feature # delete your feature branch [1] https://trunkbaseddevelopment.com/ https://trunkbaseddevelopment.com/ [2] http://nvie.com/posts/a-successful-git-branching-model/ http://nvie.com/posts/a-successful-git-branching-model/ [3] https://guides.github.com/introduction/flow/ https://guides.github.com/introduction/flow/ [4] I'm defining CD to mean delivering to production at least once a day. [5] Unlike on-premises software, with a SaaS product you only have two versions: (1) what is in production, and (2) what is going out to production (in staging, canary, etc.).

ungzd 10y ago

TLDR: instead of using diffs of changed lines in branches, use 'if' statements in code to make Enterprise Continuous Integration gods happy.

UK-AL 10y ago
The advantages of feature branches is that you can turn them on and off easily without trying merge in, or revert and dealing with the resultant conflicts this can cause. Feature switches also have more advanced features like being able targeting specific users, if you want a gradual rollout. We use both feature branches and feature switches, since branches short be very short lived. However large areas of development require months.

paulddraper 10y ago

> Google do Trunk Based Development and they have 25000 developers and QA automators in that trunk For some stuff, but Is Android trunk-only? No. Is V8 trunk-only? No. Is Chrome trunk-only? No. Those have release branches. --- Some software works with trunk-only paradigm, some does not. A rule of thumb is whether your customer-facing distribution is SaaS/web-based. If you're making Google Search, you do it in a trunk. If you're making Google Chrome, you don't.

paul_h 10y ago
> Those have release branches. Some Trunk Based Development teams do "branch for release" https://trunkbaseddevelopment.com/branch-for-release/ https://trunkbaseddevelopment.com/branch-for-release/ and some do "release from trunk" https://trunkbaseddevelopment.com/release-from-trunk/ https://trunkbaseddevelopment.com/release-from-trunk/ Google gave numbers themselves - 95% of their devs are in one big trunk (formerly perforce, but re-written in-house in 2012). It's true that the open source facing teams are outside that, and have processes that are less lock-step than what they do in-house. Android - Samsung, on each release (maybe not anymore), used to check all the composite parts of Android into one big trunk for themselves. Back into their Perforce. > If you're making Google Chrome, you don't. There's nothing stopping the Chrome team from doing Trunk Based Development on Github, and accepting unsolicited pull requests like any other team.
- paulddraper 10y ago
  You can have long lived release branches in "trunk-based development"?!? In that case, I guess I haven't ever seen a branching strategy that wasn't "trunk-based." What would that even be?
  - qwer 10y ago
    Anything involving long-lived feature branches especially ones with multiple collaborators, eg git-flow.
    - 10y ago

z3t4 10y ago

I read Drunk-Based Development and was a bit disappointed. I can get awesome ideas while very tired and usually write them down, but a few weeks later, even though I was super exited when I wrote it down, it no longer makes any sense ...

lucideer 10y ago
I read Trump-Based Development...

ricotijsen 10y ago

I read that as "Trump-based development"

xorcist 10y ago

If there was indeed a superior way of developing software, why doesn't anyone take this knowledge and out-compete for example Linux? It has literally hundreds of branches of all kinds (stable branches, feature branches, test branches), this should be easy pickings! Or out-compete Google or Microsoft which, contrary to whats said in this article, use branches and probably in all their big projects. Sorry, I don't buy it. I suspect that if you are religiously against branches you will just have to re-invent them by some other name.

stcredzero 10y ago

The Envy configuration management system for Smalltalk worked like this. As you were working, you could see if someone had changed code "nearby" (same class, same category, same protocol) and you could review it and merge it in. Perforce provides something with the same effect. If you periodically get the most recent revision, you can see files with collisions in your change sets, then merge them in.

alkonaut 10y ago

In subversion the only way I can possible develop with sanity is in an "unstable trunk". Branch off releases N weeks before shipping, and have only the requirement that trunk passes automated tests while release branches are manually tested. Obviously you don't release more than say once every month or two, but that's more than enough for most. I'm honestly not sure if the site is satire? No one develops in long lived feature branches in svn (rather, everyone does, but most only try it once before they realize the pain of trunk based dev is much smaller than that of svn merging).

sydd 10y ago
If you release only a month or 2 you are doing something really wrong (or need to work based on some insane spec). Read the basics about CI and CD, read the clean code book, read about modern organisation practices (e.g. Github releases several times a day). Releasing every month or so is (thank God) a relic of the old waterfall development times.
- rabidrat 10y ago
  I would say this must be satire but you seem sincere. You are promoting a thin slice of modern "best practices" which are not the only way to do software development. And it doesn't sound like you've ever worked on a major software project if you think that a month is a prohibitively long development timeframe. A mid-size architectural refactor of a large codebase can and maybe should take a month. When you get into the 200k+ SLOC range, no one person understands the entire thing, and most of us don't want to 'move fast and break things' when a little forethought, communication, and analysis can avoid it. I think this 'growth at all costs mentality' is ruining a lot more than our software, too.
- diek 10y ago
  No, they're not. Is IntelliJ "doing it wrong" to release every three months[1] ? Absolutely not. A release can change things about a product, and you want to ensure you are pushing those changes (UI, behavior, plugin APIs) at a pace that matches the users. You don't want to be pushing changes to a plugin API every day. The surface area of stability that has to be maintained at that point is insane. In the interim, you should definitely be able to build off master. Your CICD server should build and run tests on every commit to master. Then a release gets a tag in version control and you should be able to issue bug fix releases for only that specific version. Telling customers, "There is a critical security bug in version X, but in order to fix that bug you need to upgrade completely to version X+2" is not an acceptable answer. Note that the description above works best in my experience for products that push a release artifact (like IntelliJ) versus a service like GitHub. Release management approaches will vary based on the product. [1] https://www.jetbrains.com/idea/download/previous.html https://www.jetbrains.com/idea/download/previous.html

jwatte 10y ago

Trunk only goes well with continuous deployment. If each increment in deployment is small enough, then you know exactly when things break and rollbacks are easy. The 3-2-1 system where you develop for 3 months, then merge for 2 months, then debug in production for 1 month, is a real drag, and doesn't lend itself to quick feedback at all. For a software looking for product market fit with ultra short cycles, I would absolutely choose trunk only. For engineering projects in areas where cycles are months, or years, more traditional branching makes more sense.

cwills 10y ago

We have a smallish team working on web apps and use a trunk-based-development branching strategy. Not really to do with helping CI/CD. The advantage we see is that new developers can start developing as soon as they clone the repository, without needing to switch to the 'development' branch. This is particularly useful when on boarding new developers who are inexperienced with Git. I found that when using Git Flow that the (stable) master branch frequently ends up out-of-date when developers are required to remember to merge into it releases into it. Diagram of our branching strategy: https://gist.github.com/CameronWills/abf9e307669b1005c88ef82e55102cd0 https://gist.github.com/CameronWills/abf9e307669b1005c88ef82...

paul_h 10y ago
I looked at your diagram. I don;t think you're doing trunk based development.