5 ms·

I’ve never understood the debate between mono and multi repos. With the right tooling, the line seems to vanish and you just have folders anyway. Each repo may

I’ve never understood the debate between mono and multi repos. With the right tooling, the line seems to vanish and you just have folders anyway. Each repo may have their own policies and permissions, which is the biggest reason I see to keep them separate, but again the distinction still seems little more than a folder. Am I missing something?

ergothus 8y ago
The arguments against circle around unnecessary coupling. The arguments for revolve around necessary coupling. If a change triggers a change in multiple repos, it is easier to review those changes for completeness and administer them (commits/rollbacks to the code) in sync if the are in the same repo. My own jury is still out as to which argument is more valid, I don't have enough experience with mono repos yet.
- jimmaswell 8y ago
  > If a change triggers a change in multiple repos, it is easier to review those changes for completeness and administer them (commits/rollbacks to the code) in sync if the are in the same repo. This sounds like what submodules are for. A and B depend on C and C is kept in its own repo, referenced as a submodule by A and B.
perfmode 8y ago
When you think of monorepo, think of 1000 daily engineers and upward of 10,000 commits per day in the repo. Think of running a company the size of google or Facebook off of a single repository. Of course you can have monorepo at small scales. However, it is at the larger scales where differences become apparent and trade-offs become non-trivial.
- azhenley 8y ago
  That perspective does change things a bit :) All their eggs are literally in one basket!
- justinwp 8y ago
  We made the switch at less than 10 engineers and would never go back.
- busterarm 8y ago
  Even at a 1/5th of that scale the problems become obvious.
weberc2 8y ago
In my case, I'm struggling to find good tooling to support monorepos. We're running a microservice architecture, but our CI is triggered by GitHub web hooks. Currently we're either doing a full build of everything on each webhook event or we're doing some error-prone git-diffing to try to make sure we're only rebuilding when necessary. I've looked at tools like Buck and Bazel, but they seem really heavy for our ~30 person engineering team and they also seem to have odd ways of doing things (no support for pulling from a package repository, vendoring is assumed to be vendored--which incidentally the author of this blog post characterizes as a Bad Thing). Folks who are using monorepos successfully--what tooling do you use to solve these problems?
- porlune 8y ago
  I'm working to solve the same issue with our CI/CD pipeline. Have you tried checking for version differences between your existing images and the services in your monorepo as an indicator of which services to rebuild(instead of using a git-difference)?
  - weberc2 8y ago
    I'm not sure what you're proposing? Hashing the inputs to the build function and associating that with an image and using that checksum to determine if there's a change? If so, that doesn't seem to solve the problem of reliably identifying inputs to include in the checksum in the first place, which is a big part of the problem. I imagine we could do the build in a container and use the Docker build cache to identify what things need to be rebuilt, but that doesn't seem like it would scale well since afaik there's no good way to share the cache across multiple nodes.
- justinwp 8y ago
  We have been using Bazel in 20 dev python org. I think most of us have been happy with the transition while collaboration has improved and onboarding has become much easier. The biggest pain point is the python part and not having the resources to vendor everything but the Bazel team is working to fix this albeit very slowly. We get around some of these nonhermetic issues with python and dependencies by using a consistent docker based dev environment for everyone. Only takes a few minutes bazel build/test/run anything in our repo. EDIT: We run Bazel in drone as part of our CI/CD system and this is probably the most unoptimized part given how they do not work that well together. Bazel query allows us to due targeted builds/tests based upon the git changes and dependents of those changes.
sytse 8y ago
I agree tooling can resolve much of the difference. I think the tooling for monorepos as currently more advanced although it tends to be company specific. At GitLab we're working on a feature to group merge requests so people can get closer to an atomic commit across repos. This will help the multi repo users.
- azhenley 8y ago
  Good point! I haven't looked into the tools available to small companies and just assumed open source solutions existed. As an intern at Microsoft a few years ago I extended an internal tool that provides abstractions such that it can work seamlessly across repos using different version control systems.
coldtea 8y ago
>I’ve never understood the debate between mono and multi repos. With the right tooling, the line seems to vanish and you just have folders anyway. First, that's assuming the "right tooling" exists. Microsoft, Google, and FB have all wrote custom tooling or custom extensions to tooling (e.g. to version control tools) to fit their monorepos and get acceptable performance. And all three have more resources to throw at their problems compared to a 10, 100 or 400 person company. Second, this is a little like "C and Haskell are both turing complete, so what's the big deal with adopting either?".
- seanmcdirmid 8y ago
  The first point is valid. No one seems interested in releasing such tech publicly. The second point misunderstands the point being made. It is more like saying you don’t care if your program is running on X86 or ARM because the compiler/interpreter abstracts that detail away from you anyways.
  - Redoubts 8y ago
    > because the compiler/interpreter abstracts that detail away This assumes the first point though.
    - seanmcdirmid 8y ago
      The people who are using the tech aren’t bothered by the first point.
- aidenn0 8y ago
  MS Google and FB also have much larger monorepos than a 10, 100 or 400 person company (which is a point TFA makes). Also, if you want to make an atomic commit between two git repositories, it's possible to do it, but you are at some point no longer working with git, you are working with some VCS that is building on top of git.
shados 8y ago
The benefits and drawbacks of both approaches is mostly: -what do existing tools do for you by default -what does creating artificial boundaries around projects does to your org (this is purely social/psychological). In both cases there are good and bad things, but given enough tooling and enough process, they do end up exactly the same. In practice you don't have enough time to write an infinite amount of tooling to achieve this, and the psychological aspect if the repository as a boundary is extremely strong. (disclosure, Im personally for having many repos in a medium to large organization, because I think those boundaries are good).
mncharity 8y ago
> I’ve never understood the debate between mono and multi repos. With the right tooling, the line seems to vanish [...] Am I missing something? The space of development styles has high diversity. Tools are opinionated and constraining, with limited coverage of the space. Tools depending on tools exacerbates this. People underestimate the diversity. People have limited awareness of the characteristics of their own style that permit a particular tooling to work. People have even less awareness of how their tooling would degrade with an altered style. People similarly underestimate the cost and limitations of using additional tooling and workarounds to adapt tools to a different style. And so, the "debate". So git is perfect, doing everything the way you want. And git is a joke toy, badly solving a tiny fraction of what you need from a VCS. And who could think otherwise? The pair of OP blog posts repeatedly illustrate this dynamic. (In javascript/node land...) For solo personal projects, where extensive custom tooling can easily be a rathole, I generally use a mix of mono- and multirepos. And shift code between them based on tool ecosystem friction.
qznc 8y ago
With a monorepo you can do an atomic commit over everything. Without you can not.
- mlthoughts2018 8y ago
  This is a negative quality of monorepos.

ergothus 8y ago

The arguments against circle around unnecessary coupling. The arguments for revolve around necessary coupling. If a change triggers a change in multiple repos, it is easier to review those changes for completeness and administer them (commits/rollbacks to the code) in sync if the are in the same repo. My own jury is still out as to which argument is more valid, I don't have enough experience with mono repos yet.

jimmaswell 8y ago
> If a change triggers a change in multiple repos, it is easier to review those changes for completeness and administer them (commits/rollbacks to the code) in sync if the are in the same repo. This sounds like what submodules are for. A and B depend on C and C is kept in its own repo, referenced as a submodule by A and B.

perfmode 8y ago

When you think of monorepo, think of 1000 daily engineers and upward of 10,000 commits per day in the repo. Think of running a company the size of google or Facebook off of a single repository. Of course you can have monorepo at small scales. However, it is at the larger scales where differences become apparent and trade-offs become non-trivial.

azhenley 8y ago
That perspective does change things a bit :) All their eggs are literally in one basket!
justinwp 8y ago
We made the switch at less than 10 engineers and would never go back.
busterarm 8y ago
Even at a 1/5th of that scale the problems become obvious.

weberc2 8y ago

In my case, I'm struggling to find good tooling to support monorepos. We're running a microservice architecture, but our CI is triggered by GitHub web hooks. Currently we're either doing a full build of everything on each webhook event or we're doing some error-prone git-diffing to try to make sure we're only rebuilding when necessary. I've looked at tools like Buck and Bazel, but they seem really heavy for our ~30 person engineering team and they also seem to have odd ways of doing things (no support for pulling from a package repository, vendoring is assumed to be vendored--which incidentally the author of this blog post characterizes as a Bad Thing). Folks who are using monorepos successfully--what tooling do you use to solve these problems?

porlune 8y ago
I'm working to solve the same issue with our CI/CD pipeline. Have you tried checking for version differences between your existing images and the services in your monorepo as an indicator of which services to rebuild(instead of using a git-difference)?
- weberc2 8y ago
  I'm not sure what you're proposing? Hashing the inputs to the build function and associating that with an image and using that checksum to determine if there's a change? If so, that doesn't seem to solve the problem of reliably identifying inputs to include in the checksum in the first place, which is a big part of the problem. I imagine we could do the build in a container and use the Docker build cache to identify what things need to be rebuilt, but that doesn't seem like it would scale well since afaik there's no good way to share the cache across multiple nodes.
justinwp 8y ago
We have been using Bazel in 20 dev python org. I think most of us have been happy with the transition while collaboration has improved and onboarding has become much easier. The biggest pain point is the python part and not having the resources to vendor everything but the Bazel team is working to fix this albeit very slowly. We get around some of these nonhermetic issues with python and dependencies by using a consistent docker based dev environment for everyone. Only takes a few minutes bazel build/test/run anything in our repo. EDIT: We run Bazel in drone as part of our CI/CD system and this is probably the most unoptimized part given how they do not work that well together. Bazel query allows us to due targeted builds/tests based upon the git changes and dependents of those changes.

sytse 8y ago

I agree tooling can resolve much of the difference. I think the tooling for monorepos as currently more advanced although it tends to be company specific. At GitLab we're working on a feature to group merge requests so people can get closer to an atomic commit across repos. This will help the multi repo users.

azhenley 8y ago
Good point! I haven't looked into the tools available to small companies and just assumed open source solutions existed. As an intern at Microsoft a few years ago I extended an internal tool that provides abstractions such that it can work seamlessly across repos using different version control systems.

coldtea 8y ago

>I’ve never understood the debate between mono and multi repos. With the right tooling, the line seems to vanish and you just have folders anyway. First, that's assuming the "right tooling" exists. Microsoft, Google, and FB have all wrote custom tooling or custom extensions to tooling (e.g. to version control tools) to fit their monorepos and get acceptable performance. And all three have more resources to throw at their problems compared to a 10, 100 or 400 person company. Second, this is a little like "C and Haskell are both turing complete, so what's the big deal with adopting either?".

seanmcdirmid 8y ago
The first point is valid. No one seems interested in releasing such tech publicly. The second point misunderstands the point being made. It is more like saying you don’t care if your program is running on X86 or ARM because the compiler/interpreter abstracts that detail away from you anyways.
- Redoubts 8y ago
  > because the compiler/interpreter abstracts that detail away This assumes the first point though.
  - seanmcdirmid 8y ago
    The people who are using the tech aren’t bothered by the first point.
aidenn0 8y ago
MS Google and FB also have much larger monorepos than a 10, 100 or 400 person company (which is a point TFA makes). Also, if you want to make an atomic commit between two git repositories, it's possible to do it, but you are at some point no longer working with git, you are working with some VCS that is building on top of git.

shados 8y ago

The benefits and drawbacks of both approaches is mostly: -what do existing tools do for you by default -what does creating artificial boundaries around projects does to your org (this is purely social/psychological). In both cases there are good and bad things, but given enough tooling and enough process, they do end up exactly the same. In practice you don't have enough time to write an infinite amount of tooling to achieve this, and the psychological aspect if the repository as a boundary is extremely strong. (disclosure, Im personally for having many repos in a medium to large organization, because I think those boundaries are good).

mncharity 8y ago

> I’ve never understood the debate between mono and multi repos. With the right tooling, the line seems to vanish [...] Am I missing something? The space of development styles has high diversity. Tools are opinionated and constraining, with limited coverage of the space. Tools depending on tools exacerbates this. People underestimate the diversity. People have limited awareness of the characteristics of their own style that permit a particular tooling to work. People have even less awareness of how their tooling would degrade with an altered style. People similarly underestimate the cost and limitations of using additional tooling and workarounds to adapt tools to a different style. And so, the "debate". So git is perfect, doing everything the way you want. And git is a joke toy, badly solving a tiny fraction of what you need from a VCS. And who could think otherwise? The pair of OP blog posts repeatedly illustrate this dynamic. (In javascript/node land...) For solo personal projects, where extensive custom tooling can easily be a rathole, I generally use a mix of mono- and multirepos. And shift code between them based on tool ecosystem friction.

qznc 8y ago

With a monorepo you can do an atomic commit over everything. Without you can not.

mlthoughts2018 8y ago
This is a negative quality of monorepos.