4 ms·

At least google and meta are heavy into monorepos, I'm really curious what company is using a _repo per service_. That's insane.

dewey 2y ago
It's almost never a good idea to get inspired by what Google / Meta / Huge Company is doing as most of the times you don't have their problems and they have custom toolings and teams making everything work on that scale.
- CharlieDigital 2y ago
  In this case, I'd say it's the opposite: monorepo as an approach works amazingly well for small teams all the ways up to huge orgs (with the right tooling to support it). The difference is that past a certain level of complexity, the org will most certainly need specialized tooling to support massive codebases to make CI/CD (build, test, deploy, etc.) times sane. On the other hand, multi-repos may work for massive orgs, but is always going to add friction for small orgs.
  - dewey 2y ago
    In this case I wasn't even referring to mono repo or not, but more about the idea of taking inspiration from very large companies for your own not-large-company problems.
  - influx 2y ago
    I’ve used one of the Meta monorepos (yeah there’s not just one!) and it’s super painful at that scale.
    - CRConrad 2y ago
      > ...one of the Meta monorepos (yeah there’s not just one! Eeh... Might be quite meta-, but really doesn't feel very mono-.
- aleksiy123 2y ago
  I feel like this has been repeated so much now that peoples takeaway is that you shouldn't adopt anything from large companies as a small company by default. And thats simply not true. The point here is to understand what are the problems that are being solved, understand if they are similar to yours, and make a decision based on wether the tradeoffs are a good fit for you. Not necessarily disagreeing with you but I just feel the pendulum on this statement has swung to far to the other side now.
  - CRConrad 2y ago
    > peoples takeaway is that you shouldn't adopt anything from large companies as a small company by default. And thats simply not true. Yes it is. You said so yourself: > The point here is to understand what are the problems that are being solved, understand if they are similar to yours, and make a decision based on wether the tradeoffs are a good fit for you. That's you agreeing: “understand and make a decision” is pretty much the exact opposite of “adopt anything by default” — which peoples takeaway is that you shouldn't do — isn't it? So you're agreeing with the general takeaway, not arguing against it. I think this pendulum has stopped swinging and come to a rest. And pendulums (pendula? penduli?) always stop pointing straight down, midway between the extremes: You shouldn't mindlessly reject something just because it comes from a big company, but certainly not blindly adopt it just because it does, either. But, just as you said, understand if it fits for you and then decide based on that. And hey, on this particular issue, it seems some big companies do it one way, and others the opposite — so maybe the (non-existent) “average big company” is halfway between the extremes, too. (Whatever that means; one-and-a-half repository? Or half an infinity of repositories?)
pc86 2y ago
It can make sense when you have a huge team of devs and different teams responsible for everything where you may be on multiple teams, and nobody is exactly responsible for all the same set of services you are. Depending on the security/access provisioning culture of the org, "taking half a day to manually grant access to the repos so-and-so needs access to" may actually be an easier sell than "give everyone access to all our code." If you just have 20-30 devs and everyone is pretty silo'd (e.g. frontend or backend, data or API, etc) having 75 repos for your stuff is just silly.
- eddd-ddde 2y ago
  Even then, what are the benefits? I understand that sometimes there are conditions that require more repos, like open source components, collaboration with other companies, etc. but when everything is under your control you can basically treat any directory as its own "repository".
bobnamob 2y ago
Amazon uses "repo per service" and it is semi insane, but Brazil (the big ol' internal build system) and Coral (the internal service framework) make it "workable". As someone who worked in the dev tooling org, getting teams to keep their deps up to date was a nightmare.
- bluGill 2y ago
  Monorepo and multi repo both have their own need for teams to work on dev tooling when the project gets large.
jgtrosh 2y ago
My team implemented (and reimplemented!) a project using one repo per module. I think the main benefit was ensuring enough separation of concern due to the burden of changing multiple parts together. I managed to reduce something like 10 repos down to 3... Work in progress.
- tpm 2y ago
  > burden of changing multiple parts together Then you are adapting your project to the properties of code repository. I don't see that as a benefit.
wrs 2y ago
I worked at a Fortune 1 company that used one repo per release for a certain major software component.
- biorach 2y ago
  was that as insane as it sounds?
- seadan83 2y ago
  Did that work out well at all? Any silver lining? My first thought is: "branches" & "tags" - wow... Would branches/tags have just been easier to work with? Were they working with multiple services in a multi-repo? Seems like a cross-product explosion of repos. Did that configuration inhibit releases, or was the process cumbersome but just smooth because it was so rote?
  - wrs 2y ago
    It was a venerable on-prem application done in classic three-tier architecture (VB.NET client, app server, and database). It was deployed on a regular basis to thousands of locations (one deploy per location) and was critical to a business with 11-digit revenue. So yeah, cumbersome, but established, and huge downside risk to messing with the status quo. It was basically Git applied on top of an existing “copy the source” release process.
psoundy 2y ago
Have you heard of OpenShift 4? Self-hosted Kubernetes by Red Hat. Every little piece of the control plane is its own 'operator' (basically a microservice) and every operator is developed in its own repo. A github search for 'operator' in the openshift org has 178 results: https://github.com/orgs/openshift/repositories?language=&q=operator&sort=&type=all https://github.com/orgs/openshift/repositories?language=&q=o... Not all are repos hosting one or more microservices, but most appear to be. Best of luck ensuring consistency and quality across so many repos.
- adra 2y ago
  It's just as easy? When you have a monorepo with 5 million lines of code, you're only going to focus on the part of the code you care about and forget the rest. Same with 50 repos of 100,000 loc. Enforcing standards means actually having org level mandates around acceptable development standards, and it's enforced using tools. Those tools should be just as easily run on one monorepo than 50+ distributed repositories, nay?
  - psoundy 2y ago
    Even in the best case of what you are describing, how are these tools configured and their configuration maintained except via PRs to the repos in question? For every such change, N PRs having to be proposed, reviewed and merged. And all this without considering the common need (in a healthy project at least) to make cross-cutting changes with similar friction around landing a change across repos. If you wanted to, sure, applying enough time and money could make it work. I like to think that those resources might be better spent, though.
deleted 2y ago
[deleted]
marcosdumay 2y ago
Compared to all the issues with keeping a service running, pushing code to a different repo is trivial. If you think that's insane, the insanity is definitively not on the repository separation.
LaserToy 2y ago
Amazon

dewey 2y ago

It's almost never a good idea to get inspired by what Google / Meta / Huge Company is doing as most of the times you don't have their problems and they have custom toolings and teams making everything work on that scale.

CharlieDigital 2y ago
In this case, I'd say it's the opposite: monorepo as an approach works amazingly well for small teams all the ways up to huge orgs (with the right tooling to support it). The difference is that past a certain level of complexity, the org will most certainly need specialized tooling to support massive codebases to make CI/CD (build, test, deploy, etc.) times sane. On the other hand, multi-repos may work for massive orgs, but is always going to add friction for small orgs.
- dewey 2y ago
  In this case I wasn't even referring to mono repo or not, but more about the idea of taking inspiration from very large companies for your own not-large-company problems.
- influx 2y ago
  I’ve used one of the Meta monorepos (yeah there’s not just one!) and it’s super painful at that scale.
  - CRConrad 2y ago
    > ...one of the Meta monorepos (yeah there’s not just one! Eeh... Might be quite meta-, but really doesn't feel very mono-.
aleksiy123 2y ago
I feel like this has been repeated so much now that peoples takeaway is that you shouldn't adopt anything from large companies as a small company by default. And thats simply not true. The point here is to understand what are the problems that are being solved, understand if they are similar to yours, and make a decision based on wether the tradeoffs are a good fit for you. Not necessarily disagreeing with you but I just feel the pendulum on this statement has swung to far to the other side now.
- CRConrad 2y ago
  > peoples takeaway is that you shouldn't adopt anything from large companies as a small company by default. And thats simply not true. Yes it is. You said so yourself: > The point here is to understand what are the problems that are being solved, understand if they are similar to yours, and make a decision based on wether the tradeoffs are a good fit for you. That's you agreeing: “understand and make a decision” is pretty much the exact opposite of “adopt anything by default” — which peoples takeaway is that you shouldn't do — isn't it? So you're agreeing with the general takeaway, not arguing against it. I think this pendulum has stopped swinging and come to a rest. And pendulums (pendula? penduli?) always stop pointing straight down, midway between the extremes: You shouldn't mindlessly reject something just because it comes from a big company, but certainly not blindly adopt it just because it does, either. But, just as you said, understand if it fits for you and then decide based on that. And hey, on this particular issue, it seems some big companies do it one way, and others the opposite — so maybe the (non-existent) “average big company” is halfway between the extremes, too. (Whatever that means; one-and-a-half repository? Or half an infinity of repositories?)

pc86 2y ago

It can make sense when you have a huge team of devs and different teams responsible for everything where you may be on multiple teams, and nobody is exactly responsible for all the same set of services you are. Depending on the security/access provisioning culture of the org, "taking half a day to manually grant access to the repos so-and-so needs access to" may actually be an easier sell than "give everyone access to all our code." If you just have 20-30 devs and everyone is pretty silo'd (e.g. frontend or backend, data or API, etc) having 75 repos for your stuff is just silly.

eddd-ddde 2y ago
Even then, what are the benefits? I understand that sometimes there are conditions that require more repos, like open source components, collaboration with other companies, etc. but when everything is under your control you can basically treat any directory as its own "repository".

bobnamob 2y ago

Amazon uses "repo per service" and it is semi insane, but Brazil (the big ol' internal build system) and Coral (the internal service framework) make it "workable". As someone who worked in the dev tooling org, getting teams to keep their deps up to date was a nightmare.

bluGill 2y ago
Monorepo and multi repo both have their own need for teams to work on dev tooling when the project gets large.

jgtrosh 2y ago

My team implemented (and reimplemented!) a project using one repo per module. I think the main benefit was ensuring enough separation of concern due to the burden of changing multiple parts together. I managed to reduce something like 10 repos down to 3... Work in progress.

tpm 2y ago
> burden of changing multiple parts together Then you are adapting your project to the properties of code repository. I don't see that as a benefit.

wrs 2y ago

I worked at a Fortune 1 company that used one repo per release for a certain major software component.

biorach 2y ago
was that as insane as it sounds?
seadan83 2y ago
Did that work out well at all? Any silver lining? My first thought is: "branches" & "tags" - wow... Would branches/tags have just been easier to work with? Were they working with multiple services in a multi-repo? Seems like a cross-product explosion of repos. Did that configuration inhibit releases, or was the process cumbersome but just smooth because it was so rote?
- wrs 2y ago
  It was a venerable on-prem application done in classic three-tier architecture (VB.NET client, app server, and database). It was deployed on a regular basis to thousands of locations (one deploy per location) and was critical to a business with 11-digit revenue. So yeah, cumbersome, but established, and huge downside risk to messing with the status quo. It was basically Git applied on top of an existing “copy the source” release process.

psoundy 2y ago

Have you heard of OpenShift 4? Self-hosted Kubernetes by Red Hat. Every little piece of the control plane is its own 'operator' (basically a microservice) and every operator is developed in its own repo. A github search for 'operator' in the openshift org has 178 results: https://github.com/orgs/openshift/repositories?language=&q=operator&sort=&type=all https://github.com/orgs/openshift/repositories?language=&q=o... Not all are repos hosting one or more microservices, but most appear to be. Best of luck ensuring consistency and quality across so many repos.

adra 2y ago
It's just as easy? When you have a monorepo with 5 million lines of code, you're only going to focus on the part of the code you care about and forget the rest. Same with 50 repos of 100,000 loc. Enforcing standards means actually having org level mandates around acceptable development standards, and it's enforced using tools. Those tools should be just as easily run on one monorepo than 50+ distributed repositories, nay?
- psoundy 2y ago
  Even in the best case of what you are describing, how are these tools configured and their configuration maintained except via PRs to the repos in question? For every such change, N PRs having to be proposed, reviewed and merged. And all this without considering the common need (in a healthy project at least) to make cross-cutting changes with similar friction around landing a change across repos. If you wanted to, sure, applying enough time and money could make it work. I like to think that those resources might be better spent, though.

deleted 2y ago

[deleted]

marcosdumay 2y ago

Compared to all the issues with keeping a service running, pushing code to a different repo is trivial. If you think that's insane, the insanity is definitively not on the repository separation.

LaserToy 2y ago

Amazon