4 ms·

We have repository systems built for centralized atomic updates, and giant monorepos, like SVN. Question is why are we trying to have Git do this, which was exp

by slver 5y ago

We have repository systems built for centralized atomic updates, and giant monorepos, like SVN. Question is why are we trying to have Git do this, which was explicitly designed with the exact opposite goal? Is this an attempt to do SVN in Git so we get to keep the benefits of the former, and the cool buzzword-factor of the latter? I don't know. Also when I try to think about reasons to have atomic cross-project changes, my mind keeps drawing negative examples, such as another team changing the code on your project, is that a good practice? Not really. Well unless all projects are owned by the same team, it'll happen in a monorepo. Atomic updates not scaling beyond certain technical level is often a good thing, because they also don't scale on human and organizational level.

alexhutcheson 5y ago
1. You determine that a library used by a sizable fraction of the code in your entire org has a problem that’s critical to fix (maybe a security issue, or maybe the change could just save millions of dollars in compute resources, etc.), but the fix requires updating the use of that library in ~30 call sites spread across the codebases of ~10 different teams. 2. You create a PR that fixes the code and the problematic call sites in a single commit. It gets merged and you’re done. In the multi-repo world, you need to instead: 1. Add conditional branching in your library so that it supports both the old behavior and new behavior. This could be an experiment flag, a new method DoSomethingV2, a new constructor arg, etc. Depending on how you do this, you might dramatically increase the number of call sites that need to be modified. 2. Either wait for all the problematic clients to update to the new version of your library, or create PRs to manually bump their version. Whoops - turns out a couple of them were on a very old version, and the upgrade is non-trivial. Now that’s your problem to resolve before you proceed. 3. Create PRs to modify the calling code in every repo that includes problematic calls, and follow up with 10 different reviewers to get them merged. 4. If you still have the stamina, go through steps 1-3 again to clean up the conditional logic you added to your library in step 1. Basically, if code calls libraries that exist in different repos, then making backwards-incompatible changes to those libraries becomes extremely expensive. This is bad, because sometimes backwards-incompatible changes would have very high value. If the numbers from my example were higher (e.g. 1000 call sites across 100 teams), then the library maintainer in a monorepo would probably still want to use a feature flag or similar to avoid trying to merge a commit that affects 1000 files in one go. However, the library maintainer’s job is still dramatically easier, because they don’t have to deal with 100 individual repos, and they don’t need to do anything to ensure that everyone is using the latest version of their library.
- slver 5y ago
  Your monorepo scenario makes the following unlikely assumptions: 1. A critical security/performance fix has no other recourse than breaking the interface compatibility of a library. Far more common scenario is this can be fixed in the implementation without BC breaks (otherwise systems like semver wouldn't make sense). 2. The person maintaining the library knows the codebases of 10 teams better than the those 10 teams, so that person can patch their projects better and faster than the actual teams. As a library maintainer, you know the interface of your library. But that's merely the "how" on the other end of those 30 call sites. You don't know the "why". You can easily break their projects, despite your code compiles just fine. So that'd be reckless of an approach. Also your multi-repo scenario is artificially contrived. No, you don't need conditional branching and all this nonsense. In the common scenario, you just push a patch that maintains BC and tell the teams to update and that's it. And if you do have BC breaks, then: 1. Push a major version with the BC breaks and the fix. 2. Push a patch version deprecating that release and telling developers to update. That's it. You don't need all this nonsense you listed.
  - howinteresting 5y ago
    Semver is a second-rate coping mechanism for when better coordination systems don't exist.
    - slver 5y ago
      Patching the code of 10 projects you don't maintain isn't an example of a "coordination system". It's an example of avoiding having one. In multithreading this would be basically mutable shared state with no coordination. Every thread sees everything, and is free to mutate any of it at any point. Which as we all know is a best practice in multithreading /s
      - howinteresting 5y ago
        The same code can have multiple overlapping sets of maintainers. For example, one team can be responsible for business logic while another team can manage core abstractions shared by many product teams. Yet another team may be responsible for upgrading to newer toolchains and language features. They'll all want to touch the same code but make different, roughly orthogonal changes to it. Semver provides just a few bits of information, not nearly enough to cover the whole gamut of shared and distributed responsibility. The comparison with multithreading is not really valid, since monorepos typically linearize history.
howinteresting 5y ago
Of course I want people who care about modernizing code to come in and modernize my code (such as upgrades to newer language versions). Why should the burden be distributed when it can be concentrated among experts? I leverage type systems and write tests to catch any mistakes they might make.
pyrale 5y ago
The philosophy differences between git and svn aren't limited to one centralized repo vs. multiple repos. With git, you get a local stage/history which lets you rework/reorder your commits for clarity before pushing. It also allows for more options to resolve conflicts, although this increased ability has brought its own problems.
- pcl 5y ago
  I also find git’s branching model to be far superior to subversion’s. I assume that this is due to its design center around distributed repositories and patch files. But it’s still there even when using a central repo and a monorepo structure.

by slver 5y ago

alexhutcheson 5y ago

1. You determine that a library used by a sizable fraction of the code in your entire org has a problem that’s critical to fix (maybe a security issue, or maybe the change could just save millions of dollars in compute resources, etc.), but the fix requires updating the use of that library in ~30 call sites spread across the codebases of ~10 different teams. 2. You create a PR that fixes the code and the problematic call sites in a single commit. It gets merged and you’re done. In the multi-repo world, you need to instead: 1. Add conditional branching in your library so that it supports both the old behavior and new behavior. This could be an experiment flag, a new method DoSomethingV2, a new constructor arg, etc. Depending on how you do this, you might dramatically increase the number of call sites that need to be modified. 2. Either wait for all the problematic clients to update to the new version of your library, or create PRs to manually bump their version. Whoops - turns out a couple of them were on a very old version, and the upgrade is non-trivial. Now that’s your problem to resolve before you proceed. 3. Create PRs to modify the calling code in every repo that includes problematic calls, and follow up with 10 different reviewers to get them merged. 4. If you still have the stamina, go through steps 1-3 again to clean up the conditional logic you added to your library in step 1. Basically, if code calls libraries that exist in different repos, then making backwards-incompatible changes to those libraries becomes extremely expensive. This is bad, because sometimes backwards-incompatible changes would have very high value. If the numbers from my example were higher (e.g. 1000 call sites across 100 teams), then the library maintainer in a monorepo would probably still want to use a feature flag or similar to avoid trying to merge a commit that affects 1000 files in one go. However, the library maintainer’s job is still dramatically easier, because they don’t have to deal with 100 individual repos, and they don’t need to do anything to ensure that everyone is using the latest version of their library.

slver 5y ago
Your monorepo scenario makes the following unlikely assumptions: 1. A critical security/performance fix has no other recourse than breaking the interface compatibility of a library. Far more common scenario is this can be fixed in the implementation without BC breaks (otherwise systems like semver wouldn't make sense). 2. The person maintaining the library knows the codebases of 10 teams better than the those 10 teams, so that person can patch their projects better and faster than the actual teams. As a library maintainer, you know the interface of your library. But that's merely the "how" on the other end of those 30 call sites. You don't know the "why". You can easily break their projects, despite your code compiles just fine. So that'd be reckless of an approach. Also your multi-repo scenario is artificially contrived. No, you don't need conditional branching and all this nonsense. In the common scenario, you just push a patch that maintains BC and tell the teams to update and that's it. And if you do have BC breaks, then: 1. Push a major version with the BC breaks and the fix. 2. Push a patch version deprecating that release and telling developers to update. That's it. You don't need all this nonsense you listed.
- howinteresting 5y ago
  Semver is a second-rate coping mechanism for when better coordination systems don't exist.
  - slver 5y ago
    Patching the code of 10 projects you don't maintain isn't an example of a "coordination system". It's an example of avoiding having one. In multithreading this would be basically mutable shared state with no coordination. Every thread sees everything, and is free to mutate any of it at any point. Which as we all know is a best practice in multithreading /s
    - howinteresting 5y ago
      The same code can have multiple overlapping sets of maintainers. For example, one team can be responsible for business logic while another team can manage core abstractions shared by many product teams. Yet another team may be responsible for upgrading to newer toolchains and language features. They'll all want to touch the same code but make different, roughly orthogonal changes to it. Semver provides just a few bits of information, not nearly enough to cover the whole gamut of shared and distributed responsibility. The comparison with multithreading is not really valid, since monorepos typically linearize history.

howinteresting 5y ago

Of course I want people who care about modernizing code to come in and modernize my code (such as upgrades to newer language versions). Why should the burden be distributed when it can be concentrated among experts? I leverage type systems and write tests to catch any mistakes they might make.

pyrale 5y ago

The philosophy differences between git and svn aren't limited to one centralized repo vs. multiple repos. With git, you get a local stage/history which lets you rework/reorder your commits for clarity before pushing. It also allows for more options to resolve conflicts, although this increased ability has brought its own problems.

pcl 5y ago
I also find git’s branching model to be far superior to subversion’s. I assume that this is due to its design center around distributed repositories and patch files. But it’s still there even when using a central repo and a monorepo structure.