4 ms·

Is there a point in having a monorepo if you're all in on the microservices approach? I'm a big microservices skeptic, but as far as I understand it, the benefi

Is there a point in having a monorepo if you're all in on the microservices approach? I'm a big microservices skeptic, but as far as I understand it, the benefit of microservices is independence of change & deployment enforced by solid API contracts—don't you give that all up when you use a monorepo? What does "Monorepo with microservices" give you that a normal monolithic backend doesn't? (Obviously e.g. an image resizer or something else completely decoupled from your business logic should be a separate service / repo anyway—my point is more along the lines of "If something shares code, shouldn't it share a deployment strategy?")

radlad 5y ago
Monorepo with microservices gives you the ability to scale and perform SRE-type maintenance at a granular level. Teams maintain responsibility for their service, but are more easily able to refactor, share code, pull dependencies like GraphQL schemas into the frontend, etc. across many services.
- nightpool 5y ago
  So basically each team has to reinvent devops from the ground up, and staff their own on call rotation, instead of having a centralized devops function that provides a stable platform? That sounds horrendous. Although, that said, I can at least see the benefits of the "1 service per team" methodology, where you have a dedicated team that's independently responsible for updating their service. I'm more used to associating "microservices" with the model where a single team is managing 5 or 6 interacting services, and the benefits there seem much smaller.
  - radlad 5y ago
    > That sounds horrendous. Different teams can make their own decisions, but as a developer on a team that ran our own SRE, I found it came with many advantages. Specifically, we saw very little downtime, and when outages did occur were very prepared to fix it as we knew the exact state of our services (code, infrastructure, recent changes and deploys.) Additionally, we had very good logging and metrics because we knew what we'd want to have in the event of a problem. And I'm not sure what you mean "from the ground up." We were able to share a lot of Ansible playbooks, frameworks, and our entire observability stack across all teams. But I think you may also be missing the rest of my post. This is only one possible advantage. Even if the team doesn't perform their own SRE, these services can be scaled independently - both in terms of infrastructure and codebase - even while sharing code (including things like protocol data structures, auth schemes, etc.) A service that receives 1 SAML Response from an IdP (identity provider) per day per user may not need the same resources as a dashboard that exposes all SP (service providers) to a user many times a day. And an administration panel for this has still different needs. Yet, all of these services may communicate with each other.
echelon 5y ago
> Is there a point in having a monorepo if you're all in on the microservices approach? Monorepos are excellent for microservices. - You can update the protobuf service graph (all strongly typed) easily and make sure all the code changes are compatible. You still have to release in a sensible order to make sure the APIs are talking in an expected way, but this at least ensures that the code agrees. - You can address library vulns and upgrades all at once for everything. Everything can get the new gRPC release at the same time. Instead of having app owners be on the hook for this, a central team can manage these important upgrades and provide assistance / pairing for only the most complex situations. - If you're the one working on a very large library migration, you can rebase daily against the entire fleet of microservices and not manage N-many code changes in N-many repos. This makes huge efforts much easier. Bonus: you can land incrementally across everything. - If you're the one scoping out one of these "big changes", you can statically find all of the code you'll impact or need to understand. This is such an amazing win. No more hunting for repos and grepping for code in hundreds of undiscovered places. - Once a vuln is fixed, you can tell all apps to deploy after SHA X to fix VULN Y. This is such an easy thing for app owners to do. - You can collect common service library code in a central place (eg. internal Guava, auth tools, i18n, etc). Such packages are easy to share and reuse. All of your internal code is "vendored" essentially, but you can choose to depend on only the things you need. A monorepo only feels heavy if you depend on all the things (or your tooling doesn't support git or build operations - you seriously have to staff a monorepo team). - Other teams can easily discover and read your code. Monorepos are the best possible way to go as long as you have the tooling to support it. They fall over and become a burden if they're not seriously staffed. When they work, they really work.
- echelon 5y ago
  Should have also mentioned: all those changes to cross cutting library code will trigger builds and tests of the dependent services. You can find out at once what breaks. It's a superpower.
  - jyounker 5y ago
    ^^^^^ This ^^^^^
rsj_hn 5y ago
Yeah, I've seen it used to allow teams to use consistent frameworks and libraries across many different microservices. Think of authentication, DB clients, logging, webservers, grpc/http service front ends, uptime oracle -- there's lots of cross cutting concerns that are shared code among many microservices. So the next thing you decide to do is create some microservice framework that bundles all that stuff in and allow your microservice team to write some business logic on top. But now 99% of your executables are in this microservice framework that everyone is using, and that's the point where a lot of companies go the monorepo route. Actually most companies do some mix -- have a lot of stuff in a big repo and then other smaller repos alongside that.

radlad 5y ago

Monorepo with microservices gives you the ability to scale and perform SRE-type maintenance at a granular level. Teams maintain responsibility for their service, but are more easily able to refactor, share code, pull dependencies like GraphQL schemas into the frontend, etc. across many services.

nightpool 5y ago
So basically each team has to reinvent devops from the ground up, and staff their own on call rotation, instead of having a centralized devops function that provides a stable platform? That sounds horrendous. Although, that said, I can at least see the benefits of the "1 service per team" methodology, where you have a dedicated team that's independently responsible for updating their service. I'm more used to associating "microservices" with the model where a single team is managing 5 or 6 interacting services, and the benefits there seem much smaller.
- radlad 5y ago
  > That sounds horrendous. Different teams can make their own decisions, but as a developer on a team that ran our own SRE, I found it came with many advantages. Specifically, we saw very little downtime, and when outages did occur were very prepared to fix it as we knew the exact state of our services (code, infrastructure, recent changes and deploys.) Additionally, we had very good logging and metrics because we knew what we'd want to have in the event of a problem. And I'm not sure what you mean "from the ground up." We were able to share a lot of Ansible playbooks, frameworks, and our entire observability stack across all teams. But I think you may also be missing the rest of my post. This is only one possible advantage. Even if the team doesn't perform their own SRE, these services can be scaled independently - both in terms of infrastructure and codebase - even while sharing code (including things like protocol data structures, auth schemes, etc.) A service that receives 1 SAML Response from an IdP (identity provider) per day per user may not need the same resources as a dashboard that exposes all SP (service providers) to a user many times a day. And an administration panel for this has still different needs. Yet, all of these services may communicate with each other.

echelon 5y ago

> Is there a point in having a monorepo if you're all in on the microservices approach? Monorepos are excellent for microservices. - You can update the protobuf service graph (all strongly typed) easily and make sure all the code changes are compatible. You still have to release in a sensible order to make sure the APIs are talking in an expected way, but this at least ensures that the code agrees. - You can address library vulns and upgrades all at once for everything. Everything can get the new gRPC release at the same time. Instead of having app owners be on the hook for this, a central team can manage these important upgrades and provide assistance / pairing for only the most complex situations. - If you're the one working on a very large library migration, you can rebase daily against the entire fleet of microservices and not manage N-many code changes in N-many repos. This makes huge efforts much easier. Bonus: you can land incrementally across everything. - If you're the one scoping out one of these "big changes", you can statically find all of the code you'll impact or need to understand. This is such an amazing win. No more hunting for repos and grepping for code in hundreds of undiscovered places. - Once a vuln is fixed, you can tell all apps to deploy after SHA X to fix VULN Y. This is such an easy thing for app owners to do. - You can collect common service library code in a central place (eg. internal Guava, auth tools, i18n, etc). Such packages are easy to share and reuse. All of your internal code is "vendored" essentially, but you can choose to depend on only the things you need. A monorepo only feels heavy if you depend on all the things (or your tooling doesn't support git or build operations - you seriously have to staff a monorepo team). - Other teams can easily discover and read your code. Monorepos are the best possible way to go as long as you have the tooling to support it. They fall over and become a burden if they're not seriously staffed. When they work, they really work.

echelon 5y ago
Should have also mentioned: all those changes to cross cutting library code will trigger builds and tests of the dependent services. You can find out at once what breaks. It's a superpower.
- jyounker 5y ago
  ^^^^^ This ^^^^^

rsj_hn 5y ago

Yeah, I've seen it used to allow teams to use consistent frameworks and libraries across many different microservices. Think of authentication, DB clients, logging, webservers, grpc/http service front ends, uptime oracle -- there's lots of cross cutting concerns that are shared code among many microservices. So the next thing you decide to do is create some microservice framework that bundles all that stuff in and allow your microservice team to write some business logic on top. But now 99% of your executables are in this microservice framework that everyone is using, and that's the point where a lot of companies go the monorepo route. Actually most companies do some mix -- have a lot of stuff in a big repo and then other smaller repos alongside that.