5 ms·

See, this is where your argument broke down for me. Once you’ve decided there is some library of common code, and assuming you factor out that code into another

See, this is where your argument broke down for me. Once you’ve decided there is some library of common code, and assuming you factor out that code into another repo, you’ve just lost your ability to easily make breaking changes to the common code, which is something trivially easy to do in a monorepo. Why would you want that? It seems to me that if you have multiple projects sharing a base of common code then a monorepo is clearly superior.

flukus 8y ago
> you’ve just lost your ability to easily make breaking changes to the common code I haven't lost anything, I've gained the ability to make breaking changes because I don't have to update everything that breaks all at once. I don't have to do it at all because that's the job of the team responsible. With a monorepo what happens when their are 17 projects using the common code and I'm not familiar with 16 of them? Do I have to dive into the code of all 16 and fix them?
- vkou 8y ago
  You can soft-deprecate the old code path, and communicate that warnings will turn into errors at some future date. You can send your pull request to the affected team leads, and request that they approve it, once they make changes on their end. I mean, the alternative is that you have 17 different projects, each using one of five different versions of the common code. Heaven forbid one of them makes an incorrect assumption about another. Getting 17 different teams to dance together around a breaking change is always going to be hard.
  - u801e 8y ago
    If the common code is a versioned package, then each of the 17 different projects could update their code to handle breaking changes in the common package independently and update the version dependency after thorough testing.
    - vkou 8y ago
      You can have versioned packages inside a mono-repository, too, though. /common_libs/foo_lib_v1.13/, /common_libs/foo_lib_v1.14/, etc.
      - flukus 8y ago
        By that point your creating micro-repositories in your mono-repository and getting the worst of both worlds.
        vkou 8y ago
        No, you're getting the best of both worlds, because it's incredibly clear to the infrastructure maintainers what version everyone's on, whether or not the old version can be safely deprecated, who is responsible for deprecating it, etc.
- robert-brown 8y ago
  Yes, that's what you do. And you commit the change when all the tests pass.
  - flukus 8y ago
    And that's insane, people don't scale like that. It's harder enough keeping your head around one large project let alone every project a company has that you might have to jump into at any point.
    - joshuamorton 8y ago
      What exactly makes automated refactoring difficult?
    - ebikelaw 8y ago
      You're arguing for the nonexistence of something that obviously exists. There are tens of thousands of engineers at Google working in this manner on one of the largest codebases ever assembled.
- brazzledazzle 8y ago
  I don't have a horse in this race but if you have, say, a security issue and that needs to propagate downstream where does your responsibility end in that situation? Do you try to track down the dependencies and open issues in their trackers? Or maybe a more common problem is a change to a library that consumes an API that's changing so updating the library has a drop dead date.
  - shados 8y ago
    As a data point, we put the dependency graphs in a database at build time. When we have an emergency and need to push a library update to thousands of repos, we make the change, then trigger builds for all the dependents. We don't auto deploy (too risky), but we use the data we have to start nagging the owner of all the repositories to tell them they have to deploy asap. Since all projects are very small, they build and deploy very, very quickly (a few minutes at most for the big ones).
  - flukus 8y ago
    > I don't have a horse in this race but if you have, say, a security issue and that needs to propagate downstream where does your responsibility end in that situation? This is an issue that needs to be managed, from the systems I've seen it tends to be managed poorly, that's in both monoish repos and multi-repo setups as well as everyone using third party packages. I don't think committing everything to trunk is a good way to resolve it though, they only upside to this approach is that it might force you to resolve it. What I have to deal with much more frequently is the opposite problem, we have an urgent update that will break several things but has to be deployed for one dependent binary ASAP and fixing the rest of the universe first is not an option. Worst case it might create some security issues, something that should be a breaking change getting kludged into a new breaking change but still being broken.
- ebikelaw 8y ago
  At Google yes, you would be expected to fix everybody. This will also give you exposure to client use cases and form a good basis for arguing your change is safe, necessary, and correct. The idea that clients can run on the old library forever is a nightmare, especially for security-relevant changes. When I see a binary built yesterday I want it to contain yesterday’s code, not a copy of libpng 0.1 from 1996.
- cpeterso 8y ago
  Someone will need to update those 16 unfamiliar projects, whether it's you or those projects' owners. From what I hear, Google's process is that the developer making the breaking change has to either update the other projects, or coordinate with the project owners, before they can check in the breaking change. It helps that Google has standard build and test systems for all their projects in the one monorepo.
- tiglionabbit 8y ago
  What you're proposing goes a step beyond multiple repos and into package versioning. That is one viable workflow: Make a change to the common code and publish it as a new package version while allowing all existing code to continue to use the old package. Then, migrate other projects to the newer version of the dependency one by one. Allowing multiple versions of the same code to exist in production at once adds complexity. It's a trade-off. Also, if you're doing this with code that is ultimately webpacked to run in a web browser and you don't pay attention to the full tree of dependencies you're working with, there's a chance you end up loading two versions of the same library into a single web page, increasing the page weight and possibly causing incompatibilities in event handling. Google prefers to simply have one master version of the entire company at a time. I've spent a lot of time wondering which solution is the best and I'm still not sure.
  - shados 8y ago
    > Also, if you're doing this with code that is ultimately webpacked to run in a web browser and you don't pay attention to the full tree of dependencies you're working with, there's a chance you end up loading two versions of the same library into a single web page, increasing the page weight and possibly causing incompatibilities in event handling. You probably should have a way to visualize bundle size increases in PRs easily, so that this becomes obvious. Alternatively, some package managers like Yarn let you flatten the dependency tree, forcing you to pick one version of everything. Even with a monorepo, since you'll likely be using 3rd party dependencies, it's always an interesting exercise because of how hard NPM makes this: getting to a point where you only have 1 version of every 3rd party package can be very, very hard as some combinations of libs are mutually exclusive.
  - slavik81 8y ago
    It almost certainly depends on the company. Consider General Electic's microwave oven firmware and hydroelectric control station software. Both might actually share some code. Maybe they both use GEEventLoop or GEHardwareButton or something. But there's no reason to be concerned about having different versions in production at once. I don't think there's a universal answer to your question.
- Too 8y ago
  If you are making a breaking change to a public interface you maintain, wouldn't you want to know how that interface is being used first, before justifying such a major breaking change? Not just change it for the sake of your own libraries internal convenience and hope that users of the library adopt. Since you know how the api is supposed to be used and all its best practices, understanding how to change the parts of the 16 projects using it should be quite easy, you shouldn't have to dig into the domain knowledge for all those projects.
not_kurt_godel 8y ago
> you’ve just lost your ability to easily make breaking changes to the common code It should be hard to make breaking changes in common code. Even 'trivial' breaking changes seem to have a way of breaking things even when they shouldn't. If you need to make a breaking change to common code, the proper way to do it is add the new functionality separately, deprecate the old functionality (i.e. with javadoc so it gets called out explicitly by the IDE), and incorporate it 1-by-1 into consumers until none are using the deprecated version anymore.
- lclarkmichalek 8y ago
  And you can do that in a monorepo. But realistically, there are plenty of trivial breaking changes (renaming Foo to FooX) that don't warrant that effort, and so usually don't get done outside of monorepos. You should apply care when making breaking changes. Having it be hard is a separate issue - I'd say distractions from multi repo tooling would introduce more risks overall. Having a unified CI system in a monorepo is really nice.

flukus 8y ago

> you’ve just lost your ability to easily make breaking changes to the common code I haven't lost anything, I've gained the ability to make breaking changes because I don't have to update everything that breaks all at once. I don't have to do it at all because that's the job of the team responsible. With a monorepo what happens when their are 17 projects using the common code and I'm not familiar with 16 of them? Do I have to dive into the code of all 16 and fix them?

vkou 8y ago
You can soft-deprecate the old code path, and communicate that warnings will turn into errors at some future date. You can send your pull request to the affected team leads, and request that they approve it, once they make changes on their end. I mean, the alternative is that you have 17 different projects, each using one of five different versions of the common code. Heaven forbid one of them makes an incorrect assumption about another. Getting 17 different teams to dance together around a breaking change is always going to be hard.
- u801e 8y ago
  If the common code is a versioned package, then each of the 17 different projects could update their code to handle breaking changes in the common package independently and update the version dependency after thorough testing.
  - vkou 8y ago
    You can have versioned packages inside a mono-repository, too, though. /common_libs/foo_lib_v1.13/, /common_libs/foo_lib_v1.14/, etc.
    - flukus 8y ago
      By that point your creating micro-repositories in your mono-repository and getting the worst of both worlds.
      - vkou 8y ago
        No, you're getting the best of both worlds, because it's incredibly clear to the infrastructure maintainers what version everyone's on, whether or not the old version can be safely deprecated, who is responsible for deprecating it, etc.
robert-brown 8y ago
Yes, that's what you do. And you commit the change when all the tests pass.
- flukus 8y ago
  And that's insane, people don't scale like that. It's harder enough keeping your head around one large project let alone every project a company has that you might have to jump into at any point.
  - joshuamorton 8y ago
    What exactly makes automated refactoring difficult?
  - ebikelaw 8y ago
    You're arguing for the nonexistence of something that obviously exists. There are tens of thousands of engineers at Google working in this manner on one of the largest codebases ever assembled.
brazzledazzle 8y ago
I don't have a horse in this race but if you have, say, a security issue and that needs to propagate downstream where does your responsibility end in that situation? Do you try to track down the dependencies and open issues in their trackers? Or maybe a more common problem is a change to a library that consumes an API that's changing so updating the library has a drop dead date.
- shados 8y ago
  As a data point, we put the dependency graphs in a database at build time. When we have an emergency and need to push a library update to thousands of repos, we make the change, then trigger builds for all the dependents. We don't auto deploy (too risky), but we use the data we have to start nagging the owner of all the repositories to tell them they have to deploy asap. Since all projects are very small, they build and deploy very, very quickly (a few minutes at most for the big ones).
- flukus 8y ago
  > I don't have a horse in this race but if you have, say, a security issue and that needs to propagate downstream where does your responsibility end in that situation? This is an issue that needs to be managed, from the systems I've seen it tends to be managed poorly, that's in both monoish repos and multi-repo setups as well as everyone using third party packages. I don't think committing everything to trunk is a good way to resolve it though, they only upside to this approach is that it might force you to resolve it. What I have to deal with much more frequently is the opposite problem, we have an urgent update that will break several things but has to be deployed for one dependent binary ASAP and fixing the rest of the universe first is not an option. Worst case it might create some security issues, something that should be a breaking change getting kludged into a new breaking change but still being broken.
ebikelaw 8y ago
At Google yes, you would be expected to fix everybody. This will also give you exposure to client use cases and form a good basis for arguing your change is safe, necessary, and correct. The idea that clients can run on the old library forever is a nightmare, especially for security-relevant changes. When I see a binary built yesterday I want it to contain yesterday’s code, not a copy of libpng 0.1 from 1996.
cpeterso 8y ago
Someone will need to update those 16 unfamiliar projects, whether it's you or those projects' owners. From what I hear, Google's process is that the developer making the breaking change has to either update the other projects, or coordinate with the project owners, before they can check in the breaking change. It helps that Google has standard build and test systems for all their projects in the one monorepo.
tiglionabbit 8y ago
What you're proposing goes a step beyond multiple repos and into package versioning. That is one viable workflow: Make a change to the common code and publish it as a new package version while allowing all existing code to continue to use the old package. Then, migrate other projects to the newer version of the dependency one by one. Allowing multiple versions of the same code to exist in production at once adds complexity. It's a trade-off. Also, if you're doing this with code that is ultimately webpacked to run in a web browser and you don't pay attention to the full tree of dependencies you're working with, there's a chance you end up loading two versions of the same library into a single web page, increasing the page weight and possibly causing incompatibilities in event handling. Google prefers to simply have one master version of the entire company at a time. I've spent a lot of time wondering which solution is the best and I'm still not sure.
- shados 8y ago
  > Also, if you're doing this with code that is ultimately webpacked to run in a web browser and you don't pay attention to the full tree of dependencies you're working with, there's a chance you end up loading two versions of the same library into a single web page, increasing the page weight and possibly causing incompatibilities in event handling. You probably should have a way to visualize bundle size increases in PRs easily, so that this becomes obvious. Alternatively, some package managers like Yarn let you flatten the dependency tree, forcing you to pick one version of everything. Even with a monorepo, since you'll likely be using 3rd party dependencies, it's always an interesting exercise because of how hard NPM makes this: getting to a point where you only have 1 version of every 3rd party package can be very, very hard as some combinations of libs are mutually exclusive.
- slavik81 8y ago
  It almost certainly depends on the company. Consider General Electic's microwave oven firmware and hydroelectric control station software. Both might actually share some code. Maybe they both use GEEventLoop or GEHardwareButton or something. But there's no reason to be concerned about having different versions in production at once. I don't think there's a universal answer to your question.
Too 8y ago
If you are making a breaking change to a public interface you maintain, wouldn't you want to know how that interface is being used first, before justifying such a major breaking change? Not just change it for the sake of your own libraries internal convenience and hope that users of the library adopt. Since you know how the api is supposed to be used and all its best practices, understanding how to change the parts of the 16 projects using it should be quite easy, you shouldn't have to dig into the domain knowledge for all those projects.

not_kurt_godel 8y ago

> you’ve just lost your ability to easily make breaking changes to the common code It should be hard to make breaking changes in common code. Even 'trivial' breaking changes seem to have a way of breaking things even when they shouldn't. If you need to make a breaking change to common code, the proper way to do it is add the new functionality separately, deprecate the old functionality (i.e. with javadoc so it gets called out explicitly by the IDE), and incorporate it 1-by-1 into consumers until none are using the deprecated version anymore.

lclarkmichalek 8y ago
And you can do that in a monorepo. But realistically, there are plenty of trivial breaking changes (renaming Foo to FooX) that don't warrant that effort, and so usually don't get done outside of monorepos. You should apply care when making breaking changes. Having it be hard is a separate issue - I'd say distractions from multi repo tooling would introduce more risks overall. Having a unified CI system in a monorepo is really nice.