4 ms·

From the arguments I've seen, the top highlights of single vs multi repo: Single repo enables company-wide code sharing and refactoring with the least amount o

by jasode 9y ago

From the arguments I've seen, the top highlights of single vs multi repo: Single repo enables company-wide code sharing and refactoring with the least amount of friction. E.g. A programmer can modify "foobar(x,y,z)" to "foobar2(z,y,x)" and update all the corporate-wide client code that calls it. This ensures all teams are always using the same code instead of Team A on helperutils_v5.2.3 and Team B on helperutils_v6.1.0 which are incompatible with each other. The biggest downside seems to be slow performance for syncing huge repos. This was one of the motivations for Microsofts GVFS for git. It enables sparse downloads. If there are good arguments for multi-repos that are not related to performance which override the benefits of company-wide consistency of code, it would be good to discuss them. (I'm talking about companies like MS/Google/Facebook and not consulting firms with a different repo for each customer project.)

pulse7 9y ago
Company-wide code sharing is not always desired, especially if you want to protect IP.
- klodolph 9y ago
  Monorepo doesn't imply read permissions for the entire repo for everyone in the company.
  - codesnik 9y ago
    then again, point about "requesting access rights" as the downside of the multirepo is mute.
    - klodolph 9y ago
      From a broader perspective, most of these complaints are complaints about specific tooling (Git, Perforce, whatever build system or dependency management you use) rather than about multirepo or monorepo approaches.
    - pc86 9y ago
      Moot?
  - lloeki 9y ago
    The natural namespacing of multirepo makes this (and many other things) much simpler and sometimes even almost zero effort through the repo-handling tool (like Git{Hub,Lab}). Over here we use both and make a call based on the project and team that will have to work on it. For a couple of projects we even went through mono->multi->mono (and kept the whole git history without history rewriting thanks to git-write-tree and git-commit-tree, see [0]). [0]: https://gist.github.com/lloeki/37044e3f28812ce4ced0d9924a65172f https://gist.github.com/lloeki/37044e3f28812ce4ced0d9924a651... PS: our history looks like this (yes, multiple starts of history, which as valid a git DAG as anything but throws some GUI tools off): P1 *---*---*---*---*---*---*---*---*---*---* \ /| P1 split --*---*---*------ | /| P2 *---*---*---*---*---*-- | / P3 *-------*---*---*
    - klodolph 9y ago
      That history graph looks pretty familiar :-) In our case, it went from app, to app with supporting library, the library got extracted into a module and used with multiple apps, tools and new libraries were added that got shared between multiple apps, and pretty soon it felt like everything was sized MxN. We ended up combining all of the source code into one Git repo and stashing the media assets in SVN. One of the primary driving factors here was the ability to refactor code between modules more easily. Code would show up in an app or two, and then get refactored into a common library.
  - TheAceOfHearts 9y ago
    Based on what I've heard from people at Facebook and Google, they both grant engineers access to most code. I've read Apple doesn't provide full access. Personally, I'd find it a bit frustrating. It can be really interesting to poke around and learn how some internals or backend stuff works.
xorcist 9y ago
> enables company-wide code sharing and refactoring It does, but primarily it enables the tooling around that to be so much more integrated. Indexing the repo becomes possible, and fast searches, find-usage, inter-project code review becomes so much more frictionless. Build and testing systems end up having inter-project dependencies all the time. You probably already have them. It's like the difference between a big database with a hundred tables and a hundred small databases. Go with the the big database every time, until you have reason not to, in which case you have a lot more knowledge about your requirements.
kasey_junk 9y ago
Company wide code sharing is also why you wouldn't want a monorepo. By having it in a monorepo you require a coordination effort on all changes even if it would be ok for people to be using different versions.
- njs12345 9y ago
  There are ways of doing this in a monorepo (by copying code to a new target for instance) - the difference is that you don't need to pay the price of versioning with every change.
  - kasey_junk 9y ago
    Of course. Thats why this is one of those holy war things. Its trade offs all the way down.
lmm 9y ago
The repo should be the unit of versioning/releasing, just for convenience - that way bisection is easier and release tags for different units are in different places. (As an extreme example, if you had two completely distinct projects with no code in common, you wouldn't want them to be in the same repo would you?) Now that could mean a monorepo, if you're a company with one product that's released as a single unit. Or it could mean multi-repo if you have distinct components with their own release cycles. IMO that comes down to Conway's law in reverse: it's best to structure your components in a way that reflects your organization's communication structures. If you're in constant communication then moving from `foobar` to `foobar2` is a "just do it" scenario; if you're in distinct teams don't talk to each other much then having this happen via an explicit release cycle, deprecation period and so on is better.
codesnik 9y ago
isn't that (monorepo) approach essentially forces you to have one release and synchronized release schedule? What if you have rolling release&
- pc86 9y ago
  I am pretty solidly in the multirepo camp more often than not, but nothing about monorepo forces synchronized release schedules as long as you are branching appropriately.
- deleted 9y ago
  [deleted]
awinder 9y ago
Off the top of my head: 1. Security. It boggles the mind that companies with large source trees would let their entire source tree get cloned by a single developer. Maybe this isn’t really an issue but seems like you’re creating unnecessary surface area when the iOS dev team can walk around with the machine learning groups code. 2. Dev aesthetics / onboarding. There’s a complexity cost to sharing all of that code. If I join a company and the first step is spending (weeks? months?) getting comfortable understanding the universe of this gigantic source tree, instead of being able to safely understand a small piece of it, that can lead to frustration and it’s certainly not going to be productive. I’ve never worked in a massive, well put together monorepo, so maybe these issues are solved in other ways. But it seems like your point #1 is a trade-off – being able to call code from anywhere in the same repo is powerful, but it’s also capable of breeding a ton of complexity, so please mind the mere mortals who will follow one day.
- jakobegger 9y ago
  I think that being able to look at the source of everything is such a big productivity advantage that the risk of an employee stealing your source is negligible in comparison.
- Walkman 9y ago
  Neither will help if you employ bad people or the company has a bad culture. You should trust every developer.
  - xref 9y ago
    Even if you trust every developer implicitly that's not a reason to forego the Principle of Least Privilege
    - Klathmon 9y ago
      I can't imagine working somewhere that I'm not allowed to even see all the code.
bryanlarsen 9y ago
The primary reason for multi-repo in my experience is when you have to share a code base with an external team: contractors, customers, open source, et cetera. Certainly you can do that with a mono-repo, but if tools & processes have to work without the whole mono-repo, it may be easy to go all-in on multi-repos.
catern 9y ago
Another problem is that a single repo for all the code you run is impossible for all but the biggest companies (Google). Most of the code that you run is open source dependencies, like the Python interpreter, GNU C library, and Linux kernel. You could import everything, including outside dependencies, into a single repo and build an OS/application image from that repo. That is effectively what Google does. But that requires a lot of careful attention to pulling in updates of external code, and requires a particularly excellent build and dependency manager (aka a package manager). It is also huge enough to run into serious performance problems no matter how you try to fix it. So more practically, your monorepo is just for your internal code. Which means if you ever want to fix a bug in any outside libraries, or make some of your code available as a separate library for the outside world, you need to work with multiple repos. So why not just be multi-repo from the start? Or rather, have multiple monorepos, one per project or group.

by jasode 9y ago

pulse7 9y ago

Company-wide code sharing is not always desired, especially if you want to protect IP.

klodolph 9y ago
Monorepo doesn't imply read permissions for the entire repo for everyone in the company.
- codesnik 9y ago
  then again, point about "requesting access rights" as the downside of the multirepo is mute.
  - klodolph 9y ago
    From a broader perspective, most of these complaints are complaints about specific tooling (Git, Perforce, whatever build system or dependency management you use) rather than about multirepo or monorepo approaches.
  - pc86 9y ago
    Moot?
- lloeki 9y ago
  The natural namespacing of multirepo makes this (and many other things) much simpler and sometimes even almost zero effort through the repo-handling tool (like Git{Hub,Lab}). Over here we use both and make a call based on the project and team that will have to work on it. For a couple of projects we even went through mono->multi->mono (and kept the whole git history without history rewriting thanks to git-write-tree and git-commit-tree, see [0]). [0]: https://gist.github.com/lloeki/37044e3f28812ce4ced0d9924a65172f https://gist.github.com/lloeki/37044e3f28812ce4ced0d9924a651... PS: our history looks like this (yes, multiple starts of history, which as valid a git DAG as anything but throws some GUI tools off): P1 *---*---*---*---*---*---*---*---*---*---* \ /| P1 split --*---*---*------ | /| P2 *---*---*---*---*---*-- | / P3 *-------*---*---*
  - klodolph 9y ago
    That history graph looks pretty familiar :-) In our case, it went from app, to app with supporting library, the library got extracted into a module and used with multiple apps, tools and new libraries were added that got shared between multiple apps, and pretty soon it felt like everything was sized MxN. We ended up combining all of the source code into one Git repo and stashing the media assets in SVN. One of the primary driving factors here was the ability to refactor code between modules more easily. Code would show up in an app or two, and then get refactored into a common library.
- TheAceOfHearts 9y ago
  Based on what I've heard from people at Facebook and Google, they both grant engineers access to most code. I've read Apple doesn't provide full access. Personally, I'd find it a bit frustrating. It can be really interesting to poke around and learn how some internals or backend stuff works.

xorcist 9y ago

> enables company-wide code sharing and refactoring It does, but primarily it enables the tooling around that to be so much more integrated. Indexing the repo becomes possible, and fast searches, find-usage, inter-project code review becomes so much more frictionless. Build and testing systems end up having inter-project dependencies all the time. You probably already have them. It's like the difference between a big database with a hundred tables and a hundred small databases. Go with the the big database every time, until you have reason not to, in which case you have a lot more knowledge about your requirements.

kasey_junk 9y ago

Company wide code sharing is also why you wouldn't want a monorepo. By having it in a monorepo you require a coordination effort on all changes even if it would be ok for people to be using different versions.

njs12345 9y ago
There are ways of doing this in a monorepo (by copying code to a new target for instance) - the difference is that you don't need to pay the price of versioning with every change.
- kasey_junk 9y ago
  Of course. Thats why this is one of those holy war things. Its trade offs all the way down.

lmm 9y ago

The repo should be the unit of versioning/releasing, just for convenience - that way bisection is easier and release tags for different units are in different places. (As an extreme example, if you had two completely distinct projects with no code in common, you wouldn't want them to be in the same repo would you?) Now that could mean a monorepo, if you're a company with one product that's released as a single unit. Or it could mean multi-repo if you have distinct components with their own release cycles. IMO that comes down to Conway's law in reverse: it's best to structure your components in a way that reflects your organization's communication structures. If you're in constant communication then moving from `foobar` to `foobar2` is a "just do it" scenario; if you're in distinct teams don't talk to each other much then having this happen via an explicit release cycle, deprecation period and so on is better.

codesnik 9y ago

isn't that (monorepo) approach essentially forces you to have one release and synchronized release schedule? What if you have rolling release&

pc86 9y ago
I am pretty solidly in the multirepo camp more often than not, but nothing about monorepo forces synchronized release schedules as long as you are branching appropriately.
deleted 9y ago
[deleted]

awinder 9y ago

Off the top of my head: 1. Security. It boggles the mind that companies with large source trees would let their entire source tree get cloned by a single developer. Maybe this isn’t really an issue but seems like you’re creating unnecessary surface area when the iOS dev team can walk around with the machine learning groups code. 2. Dev aesthetics / onboarding. There’s a complexity cost to sharing all of that code. If I join a company and the first step is spending (weeks? months?) getting comfortable understanding the universe of this gigantic source tree, instead of being able to safely understand a small piece of it, that can lead to frustration and it’s certainly not going to be productive. I’ve never worked in a massive, well put together monorepo, so maybe these issues are solved in other ways. But it seems like your point #1 is a trade-off – being able to call code from anywhere in the same repo is powerful, but it’s also capable of breeding a ton of complexity, so please mind the mere mortals who will follow one day.

jakobegger 9y ago
I think that being able to look at the source of everything is such a big productivity advantage that the risk of an employee stealing your source is negligible in comparison.
Walkman 9y ago
Neither will help if you employ bad people or the company has a bad culture. You should trust every developer.
- xref 9y ago
  Even if you trust every developer implicitly that's not a reason to forego the Principle of Least Privilege
  - Klathmon 9y ago
    I can't imagine working somewhere that I'm not allowed to even see all the code.

bryanlarsen 9y ago

The primary reason for multi-repo in my experience is when you have to share a code base with an external team: contractors, customers, open source, et cetera. Certainly you can do that with a mono-repo, but if tools & processes have to work without the whole mono-repo, it may be easy to go all-in on multi-repos.

catern 9y ago

Another problem is that a single repo for all the code you run is impossible for all but the biggest companies (Google). Most of the code that you run is open source dependencies, like the Python interpreter, GNU C library, and Linux kernel. You could import everything, including outside dependencies, into a single repo and build an OS/application image from that repo. That is effectively what Google does. But that requires a lot of careful attention to pulling in updates of external code, and requires a particularly excellent build and dependency manager (aka a package manager). It is also huge enough to run into serious performance problems no matter how you try to fix it. So more practically, your monorepo is just for your internal code. Which means if you ever want to fix a bug in any outside libraries, or make some of your code available as a separate library for the outside world, you need to work with multiple repos. So why not just be multi-repo from the start? Or rather, have multiple monorepos, one per project or group.