6 ms·

The benefits of trunk-based development

51 points by gpi 1y ago

Ozzie_osman 1y ago
What they are describing as "Trunk-Based Development" seems a little extreme, and frankly, I'm not sure how achievable it is. Generally, when I've seen/implemented TBD it has involved Pull Requests and code reviews (they seem to imply it shouldn't). With their definition, I think a few issues arise: * Even if you think you can have high quality without PRs, in many industries, it's part of security/compliance. * They say that PRs undermine shared ownership, when actually, PRs done right can enable shared ownership (yes, I can't just merge whatever I want, but also, I get to help ensure no one merges whatever they want). They also say that "no branches are created". In reality, I don't see how you do any modern development without branches. If anything, contributors need a local branch that is then merged to the trunk (unless, somehow, everyone is live-editing the main branch?). The idea of TBD is just to have smaller, short-lived branches that are frequently merged into main (very doable with pull requests).
- ainiriand 1y ago
  Perhaps I can shed some light into your questions as I work on TBD since 3 years now. In our case there are no Pr's, we push to master, the test suite is triggered, the dev environment receives the new code. It moves to other environments if the tests pass. We release to master using a canary instance that gets only a small share of the traffic, if our metrics for success fail then we rollback and start a hotfix. Yes you can have as many branches you want in your local but in the remote there is only one.
  - coolgoose 1y ago
    And how is that different than just doing a pr, and merging to trunk? It feels like you are just canceling a lot of pipeline tests all the time, assuming there's a cadence of 10+ pushes per day.
  - MichaelNolan 1y ago
    > In our case there are no Pr's, we push to master Just to clarify, are you saying there is no code review where another engineer looks at your change before you merge to main? I’ve never seen a TBD project work that way. I can’t imagine how it would work for anything but the smallest teams.
    - ainiriand 1y ago
      In our case we are 3 devs.
      - SenHeng 1y ago
        I have been in projects where I am the sole dev, and I still create PRs and self review them before merging. I do this because I am human and make mistakes.
  - Disposal8433 1y ago
    > we push to master, the test suite is triggered If you have a bug due to the last commit, don't you have to write fixes all the time? How do you review the code?
    - JimDabell 1y ago
      You make sure it works first. Are you opening PRs with bugs “all the time”? Why? Nothing is ever bulletproof, but bugs shouldn’t be the norm in new code.
      - ansgri 1y ago
        That requires the ability to fully test it locally, which unfortunately is uncommon in decent-sized systems.
- Disposal8433 1y ago
  It is indeed extreme and very limited to writing web applications in a specific context. That is not how I have been working for the past 20 years, and his solution would be destructive in my daily job. One of his previous article (https://thinkinglabs.io/articles/2022/05/30/on-the-evilness-of-feature-branching-the-problems.html https://thinkinglabs.io/articles/2022/05/30/on-the-evilness-...) takes it one step further. It seems that everyone works on everything at once and people step on each other toes all the time. They would definitely improve by using the feature branches that they shun. Last but not least, he's talking about quality, high-trust, or on-demand build which are topics unrelated to a branching strategy. Quality is done with local tools like linters and CI to prevent bad merges, and on-demand build can be done with tags in any branching strategy. The whole article is confusing actually. > everyone is working on a single mainline It's the same for every strategy. And TBD mentions release branches, so... he's wrong in the very first sentence. I do like TBD though because it's a clean way to handle release and feature branches, but that's it.
  - JimDabell 1y ago
    > It is indeed extreme and very limited to writing web applications in a specific context It’s not. I’ve used TBD for mobile apps and it’s been great. I do prefer short-lived branches with PRs (which is another form of TBD), but it’s wrong to say that their approach is limited to web apps in a specific context.
    - Disposal8433 1y ago
      It definitely is. He's saying that "we uncover more problems sooner" which can only be done on systems such as web sites where the tests and their results are done very fast. Where I work we can have weeks of testing to make sure that every feature has not introduced a bug. I'm not criticizing TBD which I like, I criticize his thoughts that it's related to TBD where in reality the whole article talks about his specific job and the organization of his team, which again is unrelated to any branching strategy.
      - JimDabell 1y ago
- Izkata 1y ago
  > If anything, contributors need a local branch that is then merged to the trunk (unless, somehow, everyone is live-editing the main branch?). The term originates in the svn or earlier era (hence "trunk"), when "commit" and "push" were the same operation. So there wasn't really such a thing as a local branch, just uncommitted changes. And svn was able to commit changes to any files that no one else had committed to yet, it wasn't whole-repo versioning like git that required you to be fully up-to-date before pushing.
safety1st 1y ago
I'm a big fan of trunk based development in the Churchillian sense. It's the worst form of development, except for all the others. I don't think it's for every project, or even every feature in a given project. But there's a niche it works best for, and that niche happens to represent a massive percentage of all software engineering. The niche is, you're building a web app. It's some shitty Web 8.0 version of a Craigslist or something. No one wants to pay for formalized testing. No one really wants to spend a lot of time on quality. Arguably it's not that important to users. The year is Enshittified 2030 and they're lucky if they can even log into their online banking, let alone slide into someone's DMs on your platform. What's the solution? TBH it's not tripling down on good engineering process. (Don't get me wrong, good engineering process absolutely has a role, I'm merely suggesting that as you move from flight control software to online classifieds to your neighbor's recipe blog, that role diminishes in favor of less formal feedback.) The solution is radically forcing the code into the light of day in front of people who can say or do something about the biggest problems. The minor bugs will probably slip through this net, but the really nasty ones WILL get worked out if people on your team have to deal with them first, followed by an 'insider' circle of users, finally followed by the general public. That translates to certain buzzwords actually representing great ideas in this type of development, such as: feature flags, CI, pilot rollouts, and... trunk-based development. Just get the code/feature out in front of a couple people who matter, as early as the day it was written, and keep expanding until you're public.
- urban_winter 1y ago
  The idea that you can apply lower-quality engineering practices in some systems is totally wrong, I think. Good engineering practices will help you retain good developers, will allow you to maintain a sustainable development pace and will prevent you annoying your users so much that they stop using your system. I can't imagine any commercial project where this doesn't matter. Isn't this just another representation of the fallacy that it's possible to deliver faster by cutting quality? That's the kind of thing I expect to hear from ignorant stakeholders who know nothing about developing software, not on HN.
  - IanCal 1y ago
bjornsing 1y ago
Is it really true that Google has no feature branches? Sounds unlikely to me. How do you do code reviews for example if everything goes straight to main?
- aoeusnth1 1y ago
  Yes, Google uses trunk based development for almost everything. Even Android is now on trunk based development (previously a branchy mess) and heavily uses sets of feature flags for promotion / demotion through quality lifecycles. I’m not sure why TBD should have anything to do with code reviews. You aren’t able to submit to main without first uploading a change or stack of changes to code reviews, then you submit the changes after they are approved. That has absolutely nothing to do with your branching model. Just think of it as a rebase request instead of a pull request. PR was always the wrong terminology anyway.
  - Izkata 1y ago
    > Just think of it as a rebase request instead of a pull request. PR was always the wrong terminology anyway. I kinda feel like your terminology is mixed up... You start off describing branches, but then say "pull" request and that it was the wrong terminology? A pull request is requesting a pull from a separate forked repo. A merge request is requesting a merge from a branch within the same repo. And: > You aren’t able to submit to main without first uploading a change or stack of changes to code reviews, then you submit the changes after they are approved. How is this functionally different from a merge request? It seems like manually doing the exact same thing.
  - bjornsing 1y ago
    > You aren’t able to submit to main without first uploading a change or stack of changes to code reviews, then you submit the changes after they are approved. Well… On a conceptual level I’d call that a feature branch.
osigurdson 1y ago
TBD as described here is what everyone basically used to do in the olds with SVN (because branches were so hard). PRs are objectively better, just have to complete them on a regular basis. The article conflates PRs with long running branches.
- ethagnawl 1y ago
  > because branches were so hard They were also slow because, if memory serves, creating a branch involved creating a logical copy everything (files and history) instead of just tracking the delta using pointers/copy-on-write/whatever-clever-thing locally. It was on the order of tens of minutes in one project I worked on and, needless to say, we stopped doing it. Merging the branch was also (as noted) a headache when this all ran in reverse. Merge conflicts were also much more common, though I can't remember why just now -- presumably because synchronizing changes was equally slow and painful, rebasing was not an option and people couldn't be bothered. It's hard to emphasize just what a leap forward Git and Mercurial were. In addition to quick and cheap branches, you could also commit changes while working offline! The distributed nature of these tools was also revelatory and is underutilized in the modern age.
  - osigurdson 1y ago
    I don't remember it being particularly slow on a large code base. It was tricky however, I can't remember the details but it was only ever done if absolutely needed. So, according to this article the bad old days of svn are actually what they want.
devjab 1y ago
After a couple of decades in the Danish software industry I've come to accept that there is only trunk-based development (COME AT ME!). Feature branches, set releases and so on are fake safety gaps that people add when they don't trust themselves and/or their developers. These days I view it similar to someone who won't deploy changes on fridays because they fear for their weekend. If you're too afraid to deploy on a friday, then you should not be allowed to deploy on a monday. Similarily, if you're not brave enough to push changes to production, then your changes aren't ready. Of course I don't actually think there is a religion on how to do these things, but I've never seen any particular quality added by people doing quarterly releases from a feature branch. For the most part I think this part at the very beginning of the article explains a lot of the reason as to why that is: > When we do trunk-based development, the WIP we commit gets used, before any actual user sees it, by our whole team. In my eyes all of it comes down to management much more than deployment strategies. The disadvantage of feature branching is that it's very easy for management to cut costs on testing. The disadvantage of trunk-based development is that it requires functional teams of people who can actually work together and accept the "norms".
- urban_winter 1y ago
  Our team has thorough, high quality, automated tests. In the past 5 years we have never released such a badly broken system that emergency remedial work was required. We still don't release on Fridays. It's just common sense, risk management and (most importantly) respect for the developers home and family time.
  - devjab 1y ago
    I'd argue that it depends on your users. From a business perspective, it's a lot safer for us to release on fridays because failures would not affect a lot of people over the weekend. Of course I do live in a country where you'd have to agree to fix it over the weekend, and if you do, you'd be compensated with new days off + money.
- tsimionescu 1y ago
  Having literally no branches might work for tiny projects, but none of this is the case for any decently-sized software. If you want to release a piece of software to a customer who will be using it as-is for some time (which is one possible business model), then you need a version of your software where all features that the customer sees are working well. To do that, you need to fix even minor bugs in those features without adding any (visible) new features to the product. However, this doesn't scale. You can't pause all development of new features while someone is re-aligning a few UI elements and cleaning up error messages that were too confusing. So, you have to create a split in the software: one part is getting ready to ship, another is work in progress for future releases. Now, you can create this split using git branches, or using feature flags. So yes, trunk based development can work, you can absolutely build up the new experimental features into the software you're about to ship, but you'll have to hide them behind a feature flag that is disabled - a runtime branch. Or, you can have a release branch where only bug fixes are allowed, and submit major new work on the main branch. There are pros and cons to either solution, but whatever you do, you need stable vs unstable branches somewhere. And, of course, sometimes you'll have to release hotfixes, and there you won't get away without a true SCM branch, since you can't just give your customers your latest and greatest software just to patch a security issue in libcurl for a 2 year old release.
JimDabell 1y ago
> Note that the Pull Requests were introduced by the Open Source Software community. It allows core teams to maintain a system and accept contributions from the outside world. The outside world is generally perceived as low-trust. Therefore, we want to verify those contributions. Pull Requests are pretty helpful in that regard. However, in a closed-source corporate setting, the Pull Request model essentially indicates that the team owns the codebase, but it is not allowed to contribute. I think this is a very lopsided way of looking at pull requests. They are about a lot more than just trust. Reviewing and being reviewed is a great way of learning from colleagues, making common practices gel in a team, and keeping up to date with changes to the codebase. It’s not just a barrier. > money stuck into the system. It is stuck because the organisation invested considerably in creating all this code on parallel branches. However, as long as this code is not merged into mainline, deployed in production and released to users, it does not generate any revenue. Therefore, it is money stuck in the system. But, because we have less WIP, we create less inventory. As such, we have less invested money stuck in the system. I’ve been telling people that work in flight should be minimised for a long time, but this is the first time I’ve heard the inventory metaphor and I like it. It’s a similar metaphor to tech debt.
- mock-possum 1y ago
  My rule is if I’ve had a WIP branch for more than a day without making progress on it, I ask one of my coworkers if they’d like to tag in on it, or my boss to see if he feels like looking at it with me. It’s actually a really hard thing to do, there’s this inertia to reaching out that I hate feeling mired in - but every single time I’ve done it, it’s been okay. It’s really the times when I haven’t reached, in retrospect, when things have not gone okay.
- tucnak 1y ago
  How is Gerrit doing these days; is it any good for breaking away from GitHub? I'm looking for a git/mailing-list solution a small business could use on-prem but not any of that Gitlab nonsense (to throw people off-balance, if you will)
  - mikepurvis 1y ago
bytefish 1y ago
A decade in software development taught me: it always depends. It’s never a good idea to speak in absolutes. Your development workflow depends on your product, on your team, their skills, on your organization and so on. As with everything you need to carefully evaluate, if Trunk-based development is a good fit for your product, your team and your organizational structure. Sometimes it fits well, sometimes it’s not a good fit. I think a lot of people dismiss gitflow and its feature-branched model as too complicated. Yes, it feels so slow, right? But it gives inexperienced teams a very clear and documented workflow, especially so, if you need to support multiple versions of your product in the wild. It clearly dictates how to do bugfixes and hotfixes, that need to be done in multiple releases of your product. Could you achieve the same with Trunk-based development? Maybe? Could an inexperienced team come up with it on their own? I don’t think so. See, it always depends.
fjfaase 1y ago
After having worked four years with trunk based development and enjoying it very much, I worked on a project were it was required to have one branch per feature for review purposes. It caused lots of delays, temporary work arounds, merge conflicts, moving changes between branches, bugs being introduced, due to reviews being delayed. I got the impression it made development twice as slow and did not improve code quality.
oc1 1y ago
It's funny as i get older to see how all these patterns just repeat themselves. When i started out i worked in a team that used subversion and it was by default trunk-based development flow. no code reviews except the single big changeset from all devs. it was how things were done back then by many teams. Then there was a shift towards feature-request-based development with the rise of git and github. trunk-based development was now seen as "unprofessional" in corporate. totally fine and i was there to teach this mind shift. while the feature-branch-based flow solved some problems it introduced many more like stall branches and out-of-sync code. this became the default as most devs simply weren't aware or had the discipline to merge early and cut their "tickets" to an appropriate size. mostly for compliance reasons in corporate and big teams i would still advise feature-request-based flow. but as another colleague noted here - it slows you down and if it's a one or two people shop / product you will reap more benefits by doing basic trunk-based flow.
bob1029 1y ago
If the whole team is available in one physical office, a centralized model like perforce can run circles around the various git flows. You typically see this in places like game development. Being able to lock files centrally is a massive advantage as long as you have to be at the office to do it. This forces conflict resolution in a much more direct and immediate way than the passive aggressive PR submissions that are eventually reviewed for conflicts after all parties have already burned untold amounts of energy on potentially unmergable code.
stpedgwdgfhgdd 1y ago
It probably depeds on the kind of software you create, but in our case: When i add non-trivial code I want others of my team to review it first. When it is on trunk, it gets less easier to refactor as others already might use it. In this sense using a MR is really good way to get some quality checks before it goes to main and needs to be supported (in most cases) indefinitely. Next, we have system tests that i really want to avoid breaking on main and getting EVERYONE stuck. The problem arises if I keep that branch for more than a week. Merge conflicts get harder, the financial investment is not leveraged, etc etc. The problem is with long lived branches. This is something you should avoid, you can take it to the extreme, no branches. But extreme is almost never good. PS CC is really good at rebasing and resolving merge conflicts.
- _kidlike 1y ago
  I think the article misunderstood the debate... the debate is (or at least was) about gitflow, and specifically about having git branches per environment. So there are git branches for dev, staging, etc, all the way to prod. And git changes become deployed on the respective branch. Trunk based development is about having a single master (sorry, "main"), and generating deployable artifacts from there, and then the remaining environments have only deployments of versioned artifacts. Unless I'm missing some new debate about the value of PullRequests, but that sounds extreme.
crinkly 1y ago
I wonder how all these absolutist methodologies really work when you take subjectivity and our general inability to measure anything reliably away? I say this because we've tried all of them after someone read an article on a blog saying it'd solve all our problems. And we always have the same problems afterwards despite people promoting that it solved them. Sometimes even worse ones. The root cause in my company is we hire the lowest cost outsourcing outfit we can find, which hires inexperienced juniors and pretends they are senior and has a huge staff turnover because who the hell wants to work like that for more than 12 months. But of course changing the repo structure and process will compensate for that. My observation is that a good engineering team will self-organise a methodology that works. The methodology is unimportant compared to getting a good engineering team. I am fortunate that I work in a small niche pocket in the organisation that has good engineers and has resisted normalising with the process churn of the rest of the organisation. We run a "dark ages" team really. We were on SVN until 2020-ish, because, well it worked fine for us. Now we're on github on commandment but it is used like SVN. We have light feature branches. Master is always stable. Releases are tagged off master. At no point do we use any other github features not even PRs. CI is makefiles. Push to ECR/S3 by human. Deployment is makefiles. Comms is email because Slack is too distracting and demands immediacy. Issue tracking is a 17 year old JIRA project with a default workflow. We have the lowest defect rate, the lowest time to market and highest ROI per head in the entire org and we don't have or need a platform/sre/devops team supporting us. We are a disliked because we didn't do anything for over 10 years other than deliver software.
- djfivyvusn 1y ago
  Not well - Posted from my 45 minute long standup
  - crinkly 1y ago
    My commiserations. We don't do standups. Drop 15 mins in whoever's calendar and use the phone works. Yes we're that backwards! Every alternate Friday lunch time we meet up in person though.
    - 1y ago
zabil 1y ago
Used trunk-based dev a lot. Not a fan. It sounds good on paper, but in practice, it gets messy fast and code quality is hard to maintain, especially when teams are remote. What actually works is trunk-based deployments — keep main always deployable, and ship from there. Simple. PRs, are underrated. They’re great for sharing context. You get inline comments, CI runs, you can test stuff in isolation by spinning up infra, and teammates actually see what’s changing. Stacked diffs make juggling multiple PRs manageable. And yeah, PR reviews can slow you down, but honestly, I think that's a plus. Slowing down just enough to have another human look at the code pays off more often than not.
- aoeusnth1 1y ago
  I don’t understand why PRs are being discussed here at all – doesn’t anyone serious use trunk based development with code reviews? How is that different from a PR? Seems like we need to disentangle the discussion of whether code review is good from whether feature branches vs trunk based dev are good. Those are completely unrelated questions to me. I think there are a lot of interesting questions about using feature flags (a baby branch) vs actual branches. Personally I’m pro flags and anti branches, after a lot of experience in developer tools and CI.
  - JimDabell 1y ago
    There are different flavours of TBD. Some use short-lived feature branches, some commit directly to master. The article advocates for the latter. PRs basically have a hard-requirement on branches (or equivalent, like fork), because the code that is being requested to pull needs to be available somewhere. The article also advocates for not using pull requests. However people who take this position also often advocate for post-merge code review. And more advocate for pair programming or mob programming, which they consider to represent code review as well. So branchless TBD isn’t incompatible with code review, just code review as it is commonly practiced.
  - globular-toast 1y ago
    Definitely feature flags are the way to go. Having those code paths integrated early doesn't cost much and makes it basically free when you want to "merge" the feature into default. This does require your software to have a decent architecture such that feature flags aren't littering every part of your codebase, though. Ideally you want something like a whole module/plugin being enabled/disabled right at the entrypoint of your program/module. But this also pays dividends in the long run.
evrennetwork 1y ago
[flagged]
globular-toast 1y ago
Had to look up what trunk-based development is. It's where you have only one eternal integration branch, possibly longer lived release branches and short-lived feature branches. Yes, this is the best way. Having multiple eternal branches like git flow is complete nonsense. Just don't do it. You do not need anything so ridiculous. I joined one team that used eternal branches to deploy to various environments (staging, prod etc). What a nightmare! You could just leave a commit out of prod and be none the wiser. With a single integration branch you either do continuous deployment to prod or you use tags to mark releases. No need for another branch. Release branches are there if you need to patch things but don't want to release everything on master. You just tag a release on the release branch. I disagree with the linked article that says DVCS has made more branching common. Branching was never difficult, merging was. Git in particular made short lived branches a thing because now they are cheap enough to merge every day.
nineplay 1y ago
> When we do trunk-based development, the WIP we commit gets used, before any actual user sees it, by our whole team. I think this is where I'm stuck. I rarely work in projects where my team can 'use' the code in any meaningful way. When I wrote code for accountant my team wasn't going to be able to tell if the wash sale calculations went wonky.
ozim 1y ago
OK it is faster as article claims but I want to see the tickets, team composition, how long they work together, team dynamics, assholes blocking PRs vs assholes pushing stuff to trunk „because they know better”. Amount of work that was done and amount of bug reports and anyone can write that their customers are happy. If they worked previously on PRs some numbers how that was going compared to what they do now. For me it is just fluff and I wasted time reading it.
Frieren 1y ago
> Note that the Pull Requests were introduced by the Open Source Software community. "Pull requests" by that name maybe. But code reviews existed way before that. As soon as code was written, programmers started to ask for peer reviews of their code. - Design and code inspections to reduce errors in program development. 31 December 1976. https://ieeexplore.ieee.org/document/5388086 https://ieeexplore.ieee.org/document/5388086 But I guess that today's programmers think that software engineering was created in 2010.
luismedel 1y ago
The article title isn't misleading at all: they list the (subjective) benefits without listing any of its disadvantages. Sure, that list is useful for some trivial projects. But good luck selling that to some regulated companies where basic risk assessment is a must. Tell a financial institution you'll _never_ fuck their money transfer pipeline, because you don't do code reviews, you "deliver value" even on Fridays and your methodology avoids all sort of errors or malicious modifications. IMHO, trunk based development removes some safety. Software development is a complex task and we all risk to break things even on a good day. You're not smarter by removing safety from the process.
jFriedensreich 1y ago
I cannot imagine any advantage of this extreme description of trunk based development over trunk based development with commit stack workflows. You dont have long lived wip but you have draft commits that can mature and be worked on in parallel (also critical for coding agents) constantly auto rebasing on each other. i cannot count the number of bugs that i found before something landed on trunk just because i let “finished” work mature in draft for a bit while i work on the next task. this is especially important if the features have overlap or include ui changes that could have glitches undetectable by tests. similarly you don’t have prs but you have review stacks that are tested and evaluated in any amount of detail required by the context BEFORE they become the new trunk tip. (only fast forward, no merge commits)
HelloNurse 1y ago
> With trunk-based development teams, managing technical debt becomes a non-issue. It just happens. Unless it doesn't happen. Issues that are dealt with automatically are those that would break the build, or make tests or linting checks fail; everything else can simply accumulate without anyone "wasting time" on it. Technical debt is often recognized retroactively (e.g. there is a new foreign customer and not enough time to write thousands of neglected translated labels and messages).
WolfOliver 1y ago
I don't get this trust discussion. I don't trust myself not to do a mistake, why should I trust somebody else. Also - while pair programming is great - it does not replace a code review. When you do pair programming you start working on the problem at the same time, you get tiered and annoyed of the problem at the same time. You want to go to lunch break at the same time. All reasons to slip in a sloppy solution and convince each other it is actually ok. While doing code review you get a fresh perspective on the matter.
yfw 1y ago
My experience is you have to juggle a lot of flags and need to sometimes pay more to keep your changes isolated. Theres also the issue of how you test these in your dev env, since you can usually only have a few permutations active
bitbasher 1y ago
I once worked at a company (4 engineers including the CTO at the time). The CTO urged everyone to push to master, and called me out sarcastically asking, "Why don't you push to master?" I responded, "Because that's stupid." I'm not very eloquent and I don't work there anymore. They are currently valued over a billion dollars (for now).
barbazoo 1y ago
> With trunk-based development, we naturally commit (and push) more frequently to mainline. If we commit repeatedly to a remote mainline, we produce more frequent builds. If we produce more frequent builds, we uncover more problems sooner, allowing us to fix them when they are still small and easy to fix. This enables us to Build Quality Into the product instead of testing quality later, hence leading to better quality. Strawman, cicd can be set up such that branches build and test too. I’m having a hard time figuring out what they mean by trunk based development here. They’re too deep in, you gotta zoom out a bit and tell folks what the problem is.

51 points by gpi 1y ago

What they are describing as "Trunk-Based Development" seems a little extreme, and frankly, I'm not sure how achievable it is. Generally, when I've seen/implemented TBD it has involved Pull Requests and code reviews (they seem to imply it shouldn't). With their definition, I think a few issues arise: * Even if you think you can have high quality without PRs, in many industries, it's part of security/compliance. * They say that PRs undermine shared ownership, when actually, PRs done right can enable shared ownership (yes, I can't just merge whatever I want, but also, I get to help ensure no one merges whatever they want). They also say that "no branches are created". In reality, I don't see how you do any modern development without branches. If anything, contributors need a local branch that is then merged to the trunk (unless, somehow, everyone is live-editing the main branch?). The idea of TBD is just to have smaller, short-lived branches that are frequently merged into main (very doable with pull requests).

ainiriand 1y ago
Perhaps I can shed some light into your questions as I work on TBD since 3 years now. In our case there are no Pr's, we push to master, the test suite is triggered, the dev environment receives the new code. It moves to other environments if the tests pass. We release to master using a canary instance that gets only a small share of the traffic, if our metrics for success fail then we rollback and start a hotfix. Yes you can have as many branches you want in your local but in the remote there is only one.
- coolgoose 1y ago
  And how is that different than just doing a pr, and merging to trunk? It feels like you are just canceling a lot of pipeline tests all the time, assuming there's a cadence of 10+ pushes per day.
- MichaelNolan 1y ago
  > In our case there are no Pr's, we push to master Just to clarify, are you saying there is no code review where another engineer looks at your change before you merge to main? I’ve never seen a TBD project work that way. I can’t imagine how it would work for anything but the smallest teams.
  - ainiriand 1y ago
    In our case we are 3 devs.
    - SenHeng 1y ago
      I have been in projects where I am the sole dev, and I still create PRs and self review them before merging. I do this because I am human and make mistakes.
- Disposal8433 1y ago
  > we push to master, the test suite is triggered If you have a bug due to the last commit, don't you have to write fixes all the time? How do you review the code?
  - JimDabell 1y ago
    You make sure it works first. Are you opening PRs with bugs “all the time”? Why? Nothing is ever bulletproof, but bugs shouldn’t be the norm in new code.
    - ansgri 1y ago
      That requires the ability to fully test it locally, which unfortunately is uncommon in decent-sized systems.
Disposal8433 1y ago
It is indeed extreme and very limited to writing web applications in a specific context. That is not how I have been working for the past 20 years, and his solution would be destructive in my daily job. One of his previous article (https://thinkinglabs.io/articles/2022/05/30/on-the-evilness-of-feature-branching-the-problems.html https://thinkinglabs.io/articles/2022/05/30/on-the-evilness-...) takes it one step further. It seems that everyone works on everything at once and people step on each other toes all the time. They would definitely improve by using the feature branches that they shun. Last but not least, he's talking about quality, high-trust, or on-demand build which are topics unrelated to a branching strategy. Quality is done with local tools like linters and CI to prevent bad merges, and on-demand build can be done with tags in any branching strategy. The whole article is confusing actually. > everyone is working on a single mainline It's the same for every strategy. And TBD mentions release branches, so... he's wrong in the very first sentence. I do like TBD though because it's a clean way to handle release and feature branches, but that's it.
- JimDabell 1y ago
  > It is indeed extreme and very limited to writing web applications in a specific context It’s not. I’ve used TBD for mobile apps and it’s been great. I do prefer short-lived branches with PRs (which is another form of TBD), but it’s wrong to say that their approach is limited to web apps in a specific context.
  - Disposal8433 1y ago
    It definitely is. He's saying that "we uncover more problems sooner" which can only be done on systems such as web sites where the tests and their results are done very fast. Where I work we can have weeks of testing to make sure that every feature has not introduced a bug. I'm not criticizing TBD which I like, I criticize his thoughts that it's related to TBD where in reality the whole article talks about his specific job and the organization of his team, which again is unrelated to any branching strategy.
    - JimDabell 1y ago
Izkata 1y ago
> If anything, contributors need a local branch that is then merged to the trunk (unless, somehow, everyone is live-editing the main branch?). The term originates in the svn or earlier era (hence "trunk"), when "commit" and "push" were the same operation. So there wasn't really such a thing as a local branch, just uncommitted changes. And svn was able to commit changes to any files that no one else had committed to yet, it wasn't whole-repo versioning like git that required you to be fully up-to-date before pushing.

safety1st 1y ago

I'm a big fan of trunk based development in the Churchillian sense. It's the worst form of development, except for all the others. I don't think it's for every project, or even every feature in a given project. But there's a niche it works best for, and that niche happens to represent a massive percentage of all software engineering. The niche is, you're building a web app. It's some shitty Web 8.0 version of a Craigslist or something. No one wants to pay for formalized testing. No one really wants to spend a lot of time on quality. Arguably it's not that important to users. The year is Enshittified 2030 and they're lucky if they can even log into their online banking, let alone slide into someone's DMs on your platform. What's the solution? TBH it's not tripling down on good engineering process. (Don't get me wrong, good engineering process absolutely has a role, I'm merely suggesting that as you move from flight control software to online classifieds to your neighbor's recipe blog, that role diminishes in favor of less formal feedback.) The solution is radically forcing the code into the light of day in front of people who can say or do something about the biggest problems. The minor bugs will probably slip through this net, but the really nasty ones WILL get worked out if people on your team have to deal with them first, followed by an 'insider' circle of users, finally followed by the general public. That translates to certain buzzwords actually representing great ideas in this type of development, such as: feature flags, CI, pilot rollouts, and... trunk-based development. Just get the code/feature out in front of a couple people who matter, as early as the day it was written, and keep expanding until you're public.

urban_winter 1y ago
The idea that you can apply lower-quality engineering practices in some systems is totally wrong, I think. Good engineering practices will help you retain good developers, will allow you to maintain a sustainable development pace and will prevent you annoying your users so much that they stop using your system. I can't imagine any commercial project where this doesn't matter. Isn't this just another representation of the fallacy that it's possible to deliver faster by cutting quality? That's the kind of thing I expect to hear from ignorant stakeholders who know nothing about developing software, not on HN.
- IanCal 1y ago

bjornsing 1y ago

Is it really true that Google has no feature branches? Sounds unlikely to me. How do you do code reviews for example if everything goes straight to main?

aoeusnth1 1y ago
Yes, Google uses trunk based development for almost everything. Even Android is now on trunk based development (previously a branchy mess) and heavily uses sets of feature flags for promotion / demotion through quality lifecycles. I’m not sure why TBD should have anything to do with code reviews. You aren’t able to submit to main without first uploading a change or stack of changes to code reviews, then you submit the changes after they are approved. That has absolutely nothing to do with your branching model. Just think of it as a rebase request instead of a pull request. PR was always the wrong terminology anyway.
- Izkata 1y ago
  > Just think of it as a rebase request instead of a pull request. PR was always the wrong terminology anyway. I kinda feel like your terminology is mixed up... You start off describing branches, but then say "pull" request and that it was the wrong terminology? A pull request is requesting a pull from a separate forked repo. A merge request is requesting a merge from a branch within the same repo. And: > You aren’t able to submit to main without first uploading a change or stack of changes to code reviews, then you submit the changes after they are approved. How is this functionally different from a merge request? It seems like manually doing the exact same thing.
- bjornsing 1y ago
  > You aren’t able to submit to main without first uploading a change or stack of changes to code reviews, then you submit the changes after they are approved. Well… On a conceptual level I’d call that a feature branch.

osigurdson 1y ago

TBD as described here is what everyone basically used to do in the olds with SVN (because branches were so hard). PRs are objectively better, just have to complete them on a regular basis. The article conflates PRs with long running branches.

ethagnawl 1y ago
> because branches were so hard They were also slow because, if memory serves, creating a branch involved creating a logical copy everything (files and history) instead of just tracking the delta using pointers/copy-on-write/whatever-clever-thing locally. It was on the order of tens of minutes in one project I worked on and, needless to say, we stopped doing it. Merging the branch was also (as noted) a headache when this all ran in reverse. Merge conflicts were also much more common, though I can't remember why just now -- presumably because synchronizing changes was equally slow and painful, rebasing was not an option and people couldn't be bothered. It's hard to emphasize just what a leap forward Git and Mercurial were. In addition to quick and cheap branches, you could also commit changes while working offline! The distributed nature of these tools was also revelatory and is underutilized in the modern age.
- osigurdson 1y ago
  I don't remember it being particularly slow on a large code base. It was tricky however, I can't remember the details but it was only ever done if absolutely needed. So, according to this article the bad old days of svn are actually what they want.

devjab 1y ago

After a couple of decades in the Danish software industry I've come to accept that there is only trunk-based development (COME AT ME!). Feature branches, set releases and so on are fake safety gaps that people add when they don't trust themselves and/or their developers. These days I view it similar to someone who won't deploy changes on fridays because they fear for their weekend. If you're too afraid to deploy on a friday, then you should not be allowed to deploy on a monday. Similarily, if you're not brave enough to push changes to production, then your changes aren't ready. Of course I don't actually think there is a religion on how to do these things, but I've never seen any particular quality added by people doing quarterly releases from a feature branch. For the most part I think this part at the very beginning of the article explains a lot of the reason as to why that is: > When we do trunk-based development, the WIP we commit gets used, before any actual user sees it, by our whole team. In my eyes all of it comes down to management much more than deployment strategies. The disadvantage of feature branching is that it's very easy for management to cut costs on testing. The disadvantage of trunk-based development is that it requires functional teams of people who can actually work together and accept the "norms".

urban_winter 1y ago
Our team has thorough, high quality, automated tests. In the past 5 years we have never released such a badly broken system that emergency remedial work was required. We still don't release on Fridays. It's just common sense, risk management and (most importantly) respect for the developers home and family time.
- devjab 1y ago
  I'd argue that it depends on your users. From a business perspective, it's a lot safer for us to release on fridays because failures would not affect a lot of people over the weekend. Of course I do live in a country where you'd have to agree to fix it over the weekend, and if you do, you'd be compensated with new days off + money.
tsimionescu 1y ago
Having literally no branches might work for tiny projects, but none of this is the case for any decently-sized software. If you want to release a piece of software to a customer who will be using it as-is for some time (which is one possible business model), then you need a version of your software where all features that the customer sees are working well. To do that, you need to fix even minor bugs in those features without adding any (visible) new features to the product. However, this doesn't scale. You can't pause all development of new features while someone is re-aligning a few UI elements and cleaning up error messages that were too confusing. So, you have to create a split in the software: one part is getting ready to ship, another is work in progress for future releases. Now, you can create this split using git branches, or using feature flags. So yes, trunk based development can work, you can absolutely build up the new experimental features into the software you're about to ship, but you'll have to hide them behind a feature flag that is disabled - a runtime branch. Or, you can have a release branch where only bug fixes are allowed, and submit major new work on the main branch. There are pros and cons to either solution, but whatever you do, you need stable vs unstable branches somewhere. And, of course, sometimes you'll have to release hotfixes, and there you won't get away without a true SCM branch, since you can't just give your customers your latest and greatest software just to patch a security issue in libcurl for a 2 year old release.

JimDabell 1y ago

> Note that the Pull Requests were introduced by the Open Source Software community. It allows core teams to maintain a system and accept contributions from the outside world. The outside world is generally perceived as low-trust. Therefore, we want to verify those contributions. Pull Requests are pretty helpful in that regard. However, in a closed-source corporate setting, the Pull Request model essentially indicates that the team owns the codebase, but it is not allowed to contribute. I think this is a very lopsided way of looking at pull requests. They are about a lot more than just trust. Reviewing and being reviewed is a great way of learning from colleagues, making common practices gel in a team, and keeping up to date with changes to the codebase. It’s not just a barrier. > money stuck into the system. It is stuck because the organisation invested considerably in creating all this code on parallel branches. However, as long as this code is not merged into mainline, deployed in production and released to users, it does not generate any revenue. Therefore, it is money stuck in the system. But, because we have less WIP, we create less inventory. As such, we have less invested money stuck in the system. I’ve been telling people that work in flight should be minimised for a long time, but this is the first time I’ve heard the inventory metaphor and I like it. It’s a similar metaphor to tech debt.

mock-possum 1y ago
My rule is if I’ve had a WIP branch for more than a day without making progress on it, I ask one of my coworkers if they’d like to tag in on it, or my boss to see if he feels like looking at it with me. It’s actually a really hard thing to do, there’s this inertia to reaching out that I hate feeling mired in - but every single time I’ve done it, it’s been okay. It’s really the times when I haven’t reached, in retrospect, when things have not gone okay.
tucnak 1y ago
How is Gerrit doing these days; is it any good for breaking away from GitHub? I'm looking for a git/mailing-list solution a small business could use on-prem but not any of that Gitlab nonsense (to throw people off-balance, if you will)
- mikepurvis 1y ago

bytefish 1y ago

A decade in software development taught me: it always depends. It’s never a good idea to speak in absolutes. Your development workflow depends on your product, on your team, their skills, on your organization and so on. As with everything you need to carefully evaluate, if Trunk-based development is a good fit for your product, your team and your organizational structure. Sometimes it fits well, sometimes it’s not a good fit. I think a lot of people dismiss gitflow and its feature-branched model as too complicated. Yes, it feels so slow, right? But it gives inexperienced teams a very clear and documented workflow, especially so, if you need to support multiple versions of your product in the wild. It clearly dictates how to do bugfixes and hotfixes, that need to be done in multiple releases of your product. Could you achieve the same with Trunk-based development? Maybe? Could an inexperienced team come up with it on their own? I don’t think so. See, it always depends.

fjfaase 1y ago

After having worked four years with trunk based development and enjoying it very much, I worked on a project were it was required to have one branch per feature for review purposes. It caused lots of delays, temporary work arounds, merge conflicts, moving changes between branches, bugs being introduced, due to reviews being delayed. I got the impression it made development twice as slow and did not improve code quality.

oc1 1y ago

It's funny as i get older to see how all these patterns just repeat themselves. When i started out i worked in a team that used subversion and it was by default trunk-based development flow. no code reviews except the single big changeset from all devs. it was how things were done back then by many teams. Then there was a shift towards feature-request-based development with the rise of git and github. trunk-based development was now seen as "unprofessional" in corporate. totally fine and i was there to teach this mind shift. while the feature-branch-based flow solved some problems it introduced many more like stall branches and out-of-sync code. this became the default as most devs simply weren't aware or had the discipline to merge early and cut their "tickets" to an appropriate size. mostly for compliance reasons in corporate and big teams i would still advise feature-request-based flow. but as another colleague noted here - it slows you down and if it's a one or two people shop / product you will reap more benefits by doing basic trunk-based flow.

bob1029 1y ago

If the whole team is available in one physical office, a centralized model like perforce can run circles around the various git flows. You typically see this in places like game development. Being able to lock files centrally is a massive advantage as long as you have to be at the office to do it. This forces conflict resolution in a much more direct and immediate way than the passive aggressive PR submissions that are eventually reviewed for conflicts after all parties have already burned untold amounts of energy on potentially unmergable code.

stpedgwdgfhgdd 1y ago

It probably depeds on the kind of software you create, but in our case: When i add non-trivial code I want others of my team to review it first. When it is on trunk, it gets less easier to refactor as others already might use it. In this sense using a MR is really good way to get some quality checks before it goes to main and needs to be supported (in most cases) indefinitely. Next, we have system tests that i really want to avoid breaking on main and getting EVERYONE stuck. The problem arises if I keep that branch for more than a week. Merge conflicts get harder, the financial investment is not leveraged, etc etc. The problem is with long lived branches. This is something you should avoid, you can take it to the extreme, no branches. But extreme is almost never good. PS CC is really good at rebasing and resolving merge conflicts.

_kidlike 1y ago
I think the article misunderstood the debate... the debate is (or at least was) about gitflow, and specifically about having git branches per environment. So there are git branches for dev, staging, etc, all the way to prod. And git changes become deployed on the respective branch. Trunk based development is about having a single master (sorry, "main"), and generating deployable artifacts from there, and then the remaining environments have only deployments of versioned artifacts. Unless I'm missing some new debate about the value of PullRequests, but that sounds extreme.

crinkly 1y ago

I wonder how all these absolutist methodologies really work when you take subjectivity and our general inability to measure anything reliably away? I say this because we've tried all of them after someone read an article on a blog saying it'd solve all our problems. And we always have the same problems afterwards despite people promoting that it solved them. Sometimes even worse ones. The root cause in my company is we hire the lowest cost outsourcing outfit we can find, which hires inexperienced juniors and pretends they are senior and has a huge staff turnover because who the hell wants to work like that for more than 12 months. But of course changing the repo structure and process will compensate for that. My observation is that a good engineering team will self-organise a methodology that works. The methodology is unimportant compared to getting a good engineering team. I am fortunate that I work in a small niche pocket in the organisation that has good engineers and has resisted normalising with the process churn of the rest of the organisation. We run a "dark ages" team really. We were on SVN until 2020-ish, because, well it worked fine for us. Now we're on github on commandment but it is used like SVN. We have light feature branches. Master is always stable. Releases are tagged off master. At no point do we use any other github features not even PRs. CI is makefiles. Push to ECR/S3 by human. Deployment is makefiles. Comms is email because Slack is too distracting and demands immediacy. Issue tracking is a 17 year old JIRA project with a default workflow. We have the lowest defect rate, the lowest time to market and highest ROI per head in the entire org and we don't have or need a platform/sre/devops team supporting us. We are a disliked because we didn't do anything for over 10 years other than deliver software.

djfivyvusn 1y ago
Not well - Posted from my 45 minute long standup
- crinkly 1y ago
  My commiserations. We don't do standups. Drop 15 mins in whoever's calendar and use the phone works. Yes we're that backwards! Every alternate Friday lunch time we meet up in person though.
  - 1y ago

zabil 1y ago

Used trunk-based dev a lot. Not a fan. It sounds good on paper, but in practice, it gets messy fast and code quality is hard to maintain, especially when teams are remote. What actually works is trunk-based deployments — keep main always deployable, and ship from there. Simple. PRs, are underrated. They’re great for sharing context. You get inline comments, CI runs, you can test stuff in isolation by spinning up infra, and teammates actually see what’s changing. Stacked diffs make juggling multiple PRs manageable. And yeah, PR reviews can slow you down, but honestly, I think that's a plus. Slowing down just enough to have another human look at the code pays off more often than not.

aoeusnth1 1y ago
I don’t understand why PRs are being discussed here at all – doesn’t anyone serious use trunk based development with code reviews? How is that different from a PR? Seems like we need to disentangle the discussion of whether code review is good from whether feature branches vs trunk based dev are good. Those are completely unrelated questions to me. I think there are a lot of interesting questions about using feature flags (a baby branch) vs actual branches. Personally I’m pro flags and anti branches, after a lot of experience in developer tools and CI.
- JimDabell 1y ago
  There are different flavours of TBD. Some use short-lived feature branches, some commit directly to master. The article advocates for the latter. PRs basically have a hard-requirement on branches (or equivalent, like fork), because the code that is being requested to pull needs to be available somewhere. The article also advocates for not using pull requests. However people who take this position also often advocate for post-merge code review. And more advocate for pair programming or mob programming, which they consider to represent code review as well. So branchless TBD isn’t incompatible with code review, just code review as it is commonly practiced.
- globular-toast 1y ago
  Definitely feature flags are the way to go. Having those code paths integrated early doesn't cost much and makes it basically free when you want to "merge" the feature into default. This does require your software to have a decent architecture such that feature flags aren't littering every part of your codebase, though. Ideally you want something like a whole module/plugin being enabled/disabled right at the entrypoint of your program/module. But this also pays dividends in the long run.

evrennetwork 1y ago

[flagged]

globular-toast 1y ago

Had to look up what trunk-based development is. It's where you have only one eternal integration branch, possibly longer lived release branches and short-lived feature branches. Yes, this is the best way. Having multiple eternal branches like git flow is complete nonsense. Just don't do it. You do not need anything so ridiculous. I joined one team that used eternal branches to deploy to various environments (staging, prod etc). What a nightmare! You could just leave a commit out of prod and be none the wiser. With a single integration branch you either do continuous deployment to prod or you use tags to mark releases. No need for another branch. Release branches are there if you need to patch things but don't want to release everything on master. You just tag a release on the release branch. I disagree with the linked article that says DVCS has made more branching common. Branching was never difficult, merging was. Git in particular made short lived branches a thing because now they are cheap enough to merge every day.

nineplay 1y ago

> When we do trunk-based development, the WIP we commit gets used, before any actual user sees it, by our whole team. I think this is where I'm stuck. I rarely work in projects where my team can 'use' the code in any meaningful way. When I wrote code for accountant my team wasn't going to be able to tell if the wash sale calculations went wonky.

ozim 1y ago

OK it is faster as article claims but I want to see the tickets, team composition, how long they work together, team dynamics, assholes blocking PRs vs assholes pushing stuff to trunk „because they know better”. Amount of work that was done and amount of bug reports and anyone can write that their customers are happy. If they worked previously on PRs some numbers how that was going compared to what they do now. For me it is just fluff and I wasted time reading it.

Frieren 1y ago

> Note that the Pull Requests were introduced by the Open Source Software community. "Pull requests" by that name maybe. But code reviews existed way before that. As soon as code was written, programmers started to ask for peer reviews of their code. - Design and code inspections to reduce errors in program development. 31 December 1976. https://ieeexplore.ieee.org/document/5388086 https://ieeexplore.ieee.org/document/5388086 But I guess that today's programmers think that software engineering was created in 2010.

luismedel 1y ago

The article title isn't misleading at all: they list the (subjective) benefits without listing any of its disadvantages. Sure, that list is useful for some trivial projects. But good luck selling that to some regulated companies where basic risk assessment is a must. Tell a financial institution you'll _never_ fuck their money transfer pipeline, because you don't do code reviews, you "deliver value" even on Fridays and your methodology avoids all sort of errors or malicious modifications. IMHO, trunk based development removes some safety. Software development is a complex task and we all risk to break things even on a good day. You're not smarter by removing safety from the process.

jFriedensreich 1y ago

I cannot imagine any advantage of this extreme description of trunk based development over trunk based development with commit stack workflows. You dont have long lived wip but you have draft commits that can mature and be worked on in parallel (also critical for coding agents) constantly auto rebasing on each other. i cannot count the number of bugs that i found before something landed on trunk just because i let “finished” work mature in draft for a bit while i work on the next task. this is especially important if the features have overlap or include ui changes that could have glitches undetectable by tests. similarly you don’t have prs but you have review stacks that are tested and evaluated in any amount of detail required by the context BEFORE they become the new trunk tip. (only fast forward, no merge commits)

HelloNurse 1y ago

> With trunk-based development teams, managing technical debt becomes a non-issue. It just happens. Unless it doesn't happen. Issues that are dealt with automatically are those that would break the build, or make tests or linting checks fail; everything else can simply accumulate without anyone "wasting time" on it. Technical debt is often recognized retroactively (e.g. there is a new foreign customer and not enough time to write thousands of neglected translated labels and messages).

WolfOliver 1y ago

I don't get this trust discussion. I don't trust myself not to do a mistake, why should I trust somebody else. Also - while pair programming is great - it does not replace a code review. When you do pair programming you start working on the problem at the same time, you get tiered and annoyed of the problem at the same time. You want to go to lunch break at the same time. All reasons to slip in a sloppy solution and convince each other it is actually ok. While doing code review you get a fresh perspective on the matter.

yfw 1y ago

My experience is you have to juggle a lot of flags and need to sometimes pay more to keep your changes isolated. Theres also the issue of how you test these in your dev env, since you can usually only have a few permutations active

bitbasher 1y ago

I once worked at a company (4 engineers including the CTO at the time). The CTO urged everyone to push to master, and called me out sarcastically asking, "Why don't you push to master?" I responded, "Because that's stupid." I'm not very eloquent and I don't work there anymore. They are currently valued over a billion dollars (for now).

barbazoo 1y ago

> With trunk-based development, we naturally commit (and push) more frequently to mainline. If we commit repeatedly to a remote mainline, we produce more frequent builds. If we produce more frequent builds, we uncover more problems sooner, allowing us to fix them when they are still small and easy to fix. This enables us to Build Quality Into the product instead of testing quality later, hence leading to better quality. Strawman, cicd can be set up such that branches build and test too. I’m having a hard time figuring out what they mean by trunk based development here. They’re too deep in, you gotta zoom out a bit and tell folks what the problem is.