8 ms·

I am a huge monorepo supporter, including "no development branches". However there's a big difference between development and releases. You still want to be ab

I am a huge monorepo supporter, including "no development branches". However there's a big difference between development and releases. You still want to be able to cut stable releases that allow for cherrypicks for example, especially so in a monorepo. Atomic changes are mostly a lie when talking about cross API functions, i.e. frontend talking to a backend. You should always define some kind of stable API.

giancarlostoro 9mo ago
I like keeping old branches but a lot of places ditch them, never understood why. I also dislike git squash, it means you have to make a brand new branch for your next PR, waste of time when I should be able to pull down master / dev / main / whatever and merge it into my working branch. I guess this is another reason I prefer the forking approach of github, let devs have their own sandbox and their own branches, and let them get their work done, they will PR when its ready.
- eddd-ddde 9mo ago
  I'm very fortunate to not have to use PR style forges at work (branch based, that is). Instead each commit is its own unit of code to review, test, and merge individually. I never touch branches anymore since I also use JJ locally.
  - catlifeonmars 9mo ago
    What is JJ?
    - Denvercoder9 9mo ago
      https://github.com/jj-vcs/jj https://github.com/jj-vcs/jj
- sallveburrpi 9mo ago
  squash results in a cleaner commit history. at least that’s why we mandate it at my work. not everyone feels the same about it I guess
  - Faaak 9mo ago
    What about separate, atomic, commits? Are they squashed too? Makes reverting a fix harder without impacting the rest, no?
    - ezfe 9mo ago
      PRs should be atomic, if they need to be separated for reverting, they should be multiple PRs.
  - Denvercoder9 9mo ago
    Squashing only results in a cleaner commit history if you're making a mess of the history on your branches. If you're structuring the commit history on your branches logically, squashing just throws information away.
    - TheGRS 9mo ago
      Not everyone develops and commits the same way and mandating squashing is a much simpler management task than training up everyone to commit in a similar manner.
      - esafak 9mo ago
        Besides, they probably shouldn't make PR commits atomic, but do so as often as needed. It's a good way to avoid losing work. This is in tension with leaving behind clean commits, and squashing resolves it.
        gbear605 9mo ago
        The solution there is to make your commit history clean by rebasing it. I often end my day with a “partial changes done” commit and then the next day I’ll rebase it into several commits, or merge some of the changes into earlier commits. Even if we squash it into main later, it’s helpful for reviewing.
- normie3000 9mo ago
  > you have to make a brand new branch for your next PR Is there overhead to creating a branch?
lorey 9mo ago
Very interesting points. Would you mind sharing a few examples of when cherry-picking is necessary and why atomic changes are a lie? I'm using a monorepo for my company across 3+ products and so far we're deploying from stable release to stable release without any issues.
- GeneralMayhem 9mo ago
  Do you take down all of your projects and then bring them back up at the new version? If not, then you have times at which the change is only partially complete.
  - rezonant 9mo ago
    Nah, these days the new thing is Vibe Deployments, just ship the change and pray.
  - awesome_dude 9mo ago
    People that Blue Green are doing that, aren't they? Canary/Incremental, not so much
    - Denvercoder9 9mo ago
      Blue/green might allow you to do (approximately) atomic deploys for one service, but it doesn't allow you to do an atomic deploy of the clients of that service as well.
      - nkmnz 9mo ago
        Why that? In a very simple case, all services of a monorepo run on a single VM. Spin up new VM, deploy new code, verify, switch routing. Obviously, this doesn't work with humongous systems, but the idea can be expanded upon: make sure that components only communicate with compatible versions of other components. And don't break the database schema in a backward-incompatible way.
        9mo ago
djhedges 9mo ago
We use a mono repo and feature flag new features which gives us the deployment control timing.
- odie5533 9mo ago
  What do you use for feature flags?
  - emptysea 9mo ago
    Not OP, but I think building feature flags yourself really isn’t hard and worth doing. It’s such an important component that I wouldn’t want to depend on a third party
    - abustamam 9mo ago
      I agree, but it's hard to get the nuances right. It's easy to roll out a feature to half of your user base. It's a bit harder to roll a feature out to half of users who are in a certain region, and have the flag be sticky on them. We use Unleash at work, which is open source, and it works pretty well.
    - schrodinger 9mo ago
      I generally agree, but see some more nuance. I think feature-flagging is an overloaded term that can mean two things. First, my philosophy is that long-lived feature branches are bad, and lead to pain and risk once complete and need to be merged. Instead, prefer to work in small, incremental PRs that are quickly merged to main but dormant in production. This ensures the team is aware of the developing feature and cannot break your in-progress code (e.g. with a large refactor). This usage of "feature flags" is simple enough that it's fine and maybe even preferable to build yourself. It could be as simple as env vars or a config file. -- However, feature flagging may also refer to deploying two variants of completed code for A/B testing or just an incremental rollout. This requires the ability to expose different code paths to selected users and measure the impact. This sort of tooling is more difficult to build. It's not impossible, but comparatively complex because it probably needs to be adjustable easily without releases (i.e. requires a persistence layer) and by non-engineers (i.e. requires an admin UI). This becomes a product, and unless it's core to your business, it's probably better to pick something off the shelf. Something I learned later in my career is that measuring the impact is actually a separate responsibility. Product metrics should be reported on anyway, and this is merely adding the ability to tag requests or other units of work with the variants applied, and slice your reporting on it. It's probably better not to build this either, unless you have a niche requirement not served by the market. -- These are clearly two use cases, but share the overloaded term "feature flag": 1. Maintaining unfinished code in `main` without exposing it to users, which is far superior than long-lived feature branches but requires the ability to toggle. 2. Choosing which completed features to show to users to guide your product development. (2) is likely better served by something off the shelf. And although they're orthogonal use cases, sometimes the same tool can support both. But if you only need (1), I wouldn't invest in a complex tool that's designed to support (2)—which I think is where I agree with you :)
RaftPeople 9mo ago
> including "no development branches" Can you explain this comment? Are you saying to develop directly in the main branch? How do you manage the various time scales and complexity scales of changes? Task/project length can vary from hours to years and dependencies can range from single systems to many different systems, internal and external.
- eddd-ddde 9mo ago
  Yeah, all new commits are merged to main. The complexity comes from releases. Suppose you have a good commit 123 were all your tests pass for some project, you cut a release, and deploy it. Then development continues until commit 234, but your service is still at 123. Some critical bug is found, and fixed in commit 235. You can't just redeploy at 235 since the in-between may include development of new features that aren't ready, so you just cherry pick the fix to your release. It's branches in a way, but _only_ release branches. The only valid operations are creating new releases from head, or applying cherrypicks to existing releases.
  - imiric 9mo ago
    I don't see how you're avoiding development branches. Surely while a change is in development the author doesn't simply push to main. Otherwise concurrent development, and any code review process—assuming you have one—would be too impractical. So you can say that you have short-lived development branches that are always rebased on main. Along with the release branch and cherry-pick process, the workflow you describe is quite common.
    - ozozozd 9mo ago
      Their dev branch is _the_ development branch. They don’t do code reviews or any sort of parallel development. They’re under the impression that “releases are complex and this is how they avoid it” but they just moved the complexity and sacrificed things like parallel work, code reviews, reverts of whole features.
      - 9mo ago

giancarlostoro 9mo ago

I like keeping old branches but a lot of places ditch them, never understood why. I also dislike git squash, it means you have to make a brand new branch for your next PR, waste of time when I should be able to pull down master / dev / main / whatever and merge it into my working branch. I guess this is another reason I prefer the forking approach of github, let devs have their own sandbox and their own branches, and let them get their work done, they will PR when its ready.

eddd-ddde 9mo ago
I'm very fortunate to not have to use PR style forges at work (branch based, that is). Instead each commit is its own unit of code to review, test, and merge individually. I never touch branches anymore since I also use JJ locally.
- catlifeonmars 9mo ago
  What is JJ?
  - Denvercoder9 9mo ago
    https://github.com/jj-vcs/jj https://github.com/jj-vcs/jj
sallveburrpi 9mo ago
squash results in a cleaner commit history. at least that’s why we mandate it at my work. not everyone feels the same about it I guess
- Faaak 9mo ago
  What about separate, atomic, commits? Are they squashed too? Makes reverting a fix harder without impacting the rest, no?
  - ezfe 9mo ago
    PRs should be atomic, if they need to be separated for reverting, they should be multiple PRs.
- Denvercoder9 9mo ago
  Squashing only results in a cleaner commit history if you're making a mess of the history on your branches. If you're structuring the commit history on your branches logically, squashing just throws information away.
  - TheGRS 9mo ago
    Not everyone develops and commits the same way and mandating squashing is a much simpler management task than training up everyone to commit in a similar manner.
    - esafak 9mo ago
      Besides, they probably shouldn't make PR commits atomic, but do so as often as needed. It's a good way to avoid losing work. This is in tension with leaving behind clean commits, and squashing resolves it.
      - gbear605 9mo ago
        The solution there is to make your commit history clean by rebasing it. I often end my day with a “partial changes done” commit and then the next day I’ll rebase it into several commits, or merge some of the changes into earlier commits. Even if we squash it into main later, it’s helpful for reviewing.
normie3000 9mo ago
> you have to make a brand new branch for your next PR Is there overhead to creating a branch?

lorey 9mo ago

Very interesting points. Would you mind sharing a few examples of when cherry-picking is necessary and why atomic changes are a lie? I'm using a monorepo for my company across 3+ products and so far we're deploying from stable release to stable release without any issues.

GeneralMayhem 9mo ago
Do you take down all of your projects and then bring them back up at the new version? If not, then you have times at which the change is only partially complete.
- rezonant 9mo ago
  Nah, these days the new thing is Vibe Deployments, just ship the change and pray.
- awesome_dude 9mo ago
  People that Blue Green are doing that, aren't they? Canary/Incremental, not so much
  - Denvercoder9 9mo ago
    Blue/green might allow you to do (approximately) atomic deploys for one service, but it doesn't allow you to do an atomic deploy of the clients of that service as well.
    - nkmnz 9mo ago
      Why that? In a very simple case, all services of a monorepo run on a single VM. Spin up new VM, deploy new code, verify, switch routing. Obviously, this doesn't work with humongous systems, but the idea can be expanded upon: make sure that components only communicate with compatible versions of other components. And don't break the database schema in a backward-incompatible way.
      - 9mo ago

djhedges 9mo ago

We use a mono repo and feature flag new features which gives us the deployment control timing.

odie5533 9mo ago
What do you use for feature flags?
- emptysea 9mo ago
  Not OP, but I think building feature flags yourself really isn’t hard and worth doing. It’s such an important component that I wouldn’t want to depend on a third party
  - abustamam 9mo ago
    I agree, but it's hard to get the nuances right. It's easy to roll out a feature to half of your user base. It's a bit harder to roll a feature out to half of users who are in a certain region, and have the flag be sticky on them. We use Unleash at work, which is open source, and it works pretty well.
  - schrodinger 9mo ago
    I generally agree, but see some more nuance. I think feature-flagging is an overloaded term that can mean two things. First, my philosophy is that long-lived feature branches are bad, and lead to pain and risk once complete and need to be merged. Instead, prefer to work in small, incremental PRs that are quickly merged to main but dormant in production. This ensures the team is aware of the developing feature and cannot break your in-progress code (e.g. with a large refactor). This usage of "feature flags" is simple enough that it's fine and maybe even preferable to build yourself. It could be as simple as env vars or a config file. -- However, feature flagging may also refer to deploying two variants of completed code for A/B testing or just an incremental rollout. This requires the ability to expose different code paths to selected users and measure the impact. This sort of tooling is more difficult to build. It's not impossible, but comparatively complex because it probably needs to be adjustable easily without releases (i.e. requires a persistence layer) and by non-engineers (i.e. requires an admin UI). This becomes a product, and unless it's core to your business, it's probably better to pick something off the shelf. Something I learned later in my career is that measuring the impact is actually a separate responsibility. Product metrics should be reported on anyway, and this is merely adding the ability to tag requests or other units of work with the variants applied, and slice your reporting on it. It's probably better not to build this either, unless you have a niche requirement not served by the market. -- These are clearly two use cases, but share the overloaded term "feature flag": 1. Maintaining unfinished code in `main` without exposing it to users, which is far superior than long-lived feature branches but requires the ability to toggle. 2. Choosing which completed features to show to users to guide your product development. (2) is likely better served by something off the shelf. And although they're orthogonal use cases, sometimes the same tool can support both. But if you only need (1), I wouldn't invest in a complex tool that's designed to support (2)—which I think is where I agree with you :)

RaftPeople 9mo ago

> including "no development branches" Can you explain this comment? Are you saying to develop directly in the main branch? How do you manage the various time scales and complexity scales of changes? Task/project length can vary from hours to years and dependencies can range from single systems to many different systems, internal and external.

eddd-ddde 9mo ago
Yeah, all new commits are merged to main. The complexity comes from releases. Suppose you have a good commit 123 were all your tests pass for some project, you cut a release, and deploy it. Then development continues until commit 234, but your service is still at 123. Some critical bug is found, and fixed in commit 235. You can't just redeploy at 235 since the in-between may include development of new features that aren't ready, so you just cherry pick the fix to your release. It's branches in a way, but _only_ release branches. The only valid operations are creating new releases from head, or applying cherrypicks to existing releases.
- imiric 9mo ago
  I don't see how you're avoiding development branches. Surely while a change is in development the author doesn't simply push to main. Otherwise concurrent development, and any code review process—assuming you have one—would be too impractical. So you can say that you have short-lived development branches that are always rebased on main. Along with the release branch and cherry-pick process, the workflow you describe is quite common.
  - ozozozd 9mo ago
    Their dev branch is _the_ development branch. They don’t do code reviews or any sort of parallel development. They’re under the impression that “releases are complex and this is how they avoid it” but they just moved the complexity and sacrificed things like parallel work, code reviews, reverts of whole features.
    - 9mo ago