5 ms·

So a monolithic codebase makes it easier to make an organization wide change. Microservices make it easier to have people work and ship in independent teams. Th

by sytse 11y ago

So a monolithic codebase makes it easier to make an organization wide change. Microservices make it easier to have people work and ship in independent teams. The interesting thing is that your can have have microservices with a monolithic codebase (as Google and Facebook are comprised of many services). But you can also have a monolithic service with many codebases (like our GitLab that uses 800+ gems that live in separate codebases). And of course you can have a monolithic codebase with a monolithic service (a simple php app). And you can have microservices with diverse codebases (like all the hipsters are doing). I'm wondering if microservices force you to coordinate via the codebase just like using many codebases force you to coordinate via the monolithic service. Does the coordination has to happen somewhere? I wonder if early adopters of microservices in many codebases (SoundCloud) are experiencing coordination problems trying to change services.

scrollaway 11y ago
I'd be interested on an analysis of the meta-differences between those, but I think one of the main ones is a monolithic codebase makes it massively harder to open source components. I'm sure that, working on GitLab, you can agree with that - if GitLab were a massive, monolithic codebase and you wanted to open source specific parts of it, it'd be a huge pain.
- sytse 11y ago
  Indeed, if you have a monolithic codebase releasing your code for other people use is much harder. In git it is possible to create a separate repository for it (using subtree and filter-branch). But the harder part is versioning and release management. At Google they avoid the cost of versioning their services and the release management around that, but this makes it really hard to have the outside world use their code. Maybe this is hard anyway since their code probably depends on a lot of services common to Google (GFS, BigTable). This is probably why they can't release Borg/Omega but have to make Kubernetes instead.
sytse 11y ago
I think that advantages mentioned in the presentation about the monolithic codebase can be achieved if you have one source code server that is open to everyone (for example GitLab with most projects set to internal). Some of the tools will be easier to write for one repository than iterating over many, but that seems solvable. The biggest advantage seems to be that when you are an author of a dependency you can propose upgrades to all services that use your application. It is not clear to me but it seems that for small changes you can just force that change on the code owners. This ensures that the dependency author incurs the cost of a change (as is done for API changes in the Linux kernel) and that you do not need to version the API of the dependency. Interestingly Google recently started marking API's private by default. So they are moving in the direction of explicit API management. As soon as you work with people that are outside your control (as is common in open source) you would need to version the API as well in my opinion.
durin42 11y ago
Google has tons of services internally that talk via RPC. The monolithic repo means that it's much easier to hunt down and find people that are (say) using an outdated RPC method and help them fix their code. (Just one example of how it's useful even when things are mostly services.)
- Touche 11y ago
  This discourages you from ever making breaking changes to an API. On the face of it that sounds good but sometimes you do have to make breaking changes. My guess is that there are many duplicate projects in the Google code base for when a breaking change is needed. This is a way to sidestep the problem.
  - rictic 11y ago
    As someone who's responsible for migrating the users of a large breaking change (~300 projects), it does discourage you from making a breaking change. I'm not convinced that this is a bad thing though. Breaking changes can be both expensive and tempting. By making the author of such a change feel the cost of it you prevent the migration work from becoming an externality. It also encourages writing tools for automating this stuff. A big part of the motivation for polyup (https://github.com/PolymerLabs/polyup https://github.com/PolymerLabs/polyup) was the fact that we'd be responsible for getting people to upgrade.
  - tuckerman 11y ago
    I think sometimes it's actually the opposite. I was able to make a breaking change to an API and update all of the callers to use the new one in a single commit. Tests would run on all the clients and, since in my workspace the server was updated simultaneously, I could be more sure it would work.
    - Touche 11y ago
      That's fine for small breaking changes like an API being renamed, but some times breaking changes require actually refactoring code which is hard/impossible to do without intimate knowledge of a codebase. Think python 2->3 or Angular 1->2. These types of changes do happen, and I bet they happen at Google. I don't think anyone is rewriting a downstream app when they make these changes. Most likely they are doing something like forking the library and renaming it, which is just another form of versioning.

by sytse 11y ago

scrollaway 11y ago

I'd be interested on an analysis of the meta-differences between those, but I think one of the main ones is a monolithic codebase makes it massively harder to open source components. I'm sure that, working on GitLab, you can agree with that - if GitLab were a massive, monolithic codebase and you wanted to open source specific parts of it, it'd be a huge pain.

sytse 11y ago
Indeed, if you have a monolithic codebase releasing your code for other people use is much harder. In git it is possible to create a separate repository for it (using subtree and filter-branch). But the harder part is versioning and release management. At Google they avoid the cost of versioning their services and the release management around that, but this makes it really hard to have the outside world use their code. Maybe this is hard anyway since their code probably depends on a lot of services common to Google (GFS, BigTable). This is probably why they can't release Borg/Omega but have to make Kubernetes instead.

sytse 11y ago

I think that advantages mentioned in the presentation about the monolithic codebase can be achieved if you have one source code server that is open to everyone (for example GitLab with most projects set to internal). Some of the tools will be easier to write for one repository than iterating over many, but that seems solvable. The biggest advantage seems to be that when you are an author of a dependency you can propose upgrades to all services that use your application. It is not clear to me but it seems that for small changes you can just force that change on the code owners. This ensures that the dependency author incurs the cost of a change (as is done for API changes in the Linux kernel) and that you do not need to version the API of the dependency. Interestingly Google recently started marking API's private by default. So they are moving in the direction of explicit API management. As soon as you work with people that are outside your control (as is common in open source) you would need to version the API as well in my opinion.

durin42 11y ago

Google has tons of services internally that talk via RPC. The monolithic repo means that it's much easier to hunt down and find people that are (say) using an outdated RPC method and help them fix their code. (Just one example of how it's useful even when things are mostly services.)

Touche 11y ago
This discourages you from ever making breaking changes to an API. On the face of it that sounds good but sometimes you do have to make breaking changes. My guess is that there are many duplicate projects in the Google code base for when a breaking change is needed. This is a way to sidestep the problem.
- rictic 11y ago
  As someone who's responsible for migrating the users of a large breaking change (~300 projects), it does discourage you from making a breaking change. I'm not convinced that this is a bad thing though. Breaking changes can be both expensive and tempting. By making the author of such a change feel the cost of it you prevent the migration work from becoming an externality. It also encourages writing tools for automating this stuff. A big part of the motivation for polyup (https://github.com/PolymerLabs/polyup https://github.com/PolymerLabs/polyup) was the fact that we'd be responsible for getting people to upgrade.
- tuckerman 11y ago
  I think sometimes it's actually the opposite. I was able to make a breaking change to an API and update all of the callers to use the new one in a single commit. Tests would run on all the clients and, since in my workspace the server was updated simultaneously, I could be more sure it would work.
  - Touche 11y ago
    That's fine for small breaking changes like an API being renamed, but some times breaking changes require actually refactoring code which is hard/impossible to do without intimate knowledge of a codebase. Think python 2->3 or Angular 1->2. These types of changes do happen, and I bet they happen at Google. I don't think anyone is rewriting a downstream app when they make these changes. Most likely they are doing something like forking the library and renaming it, which is just another form of versioning.