8 ms·

At least the author gave us the courtesy of italicizing his broken assumption from the outset of the post. > Because, at scale, a monorepo must solve every pro

At least the author gave us the courtesy of italicizing his broken assumption from the outset of the post. > Because, at scale, a monorepo must solve every problem that a polyrepo must solve, with the downside of encouraging tight coupling, and the additional herculean effort of tackling VCS scalability. Right. But you have to get to "scale" first (as it relates to VCSs). Most companies don't. Even if they're successful. Introducing polyrepos front loads the scaling problems for no reason whatsoever. A giant waste of time. Checkmate! I didn't even need a snarky poll. The irony of that poll is that it clearly demonstrates his zealotry, not other people's.

adrianN 8y ago
You can't split monorepos after the fact, at least not without immense costs. You can always just put all your small repos into a big one.
- btschaegg 8y ago
  I think there's a nuance to this that should be pointed out: Monorepos allow you to do very bad hacks (I need this other component over there; let me just put in a Symlink. Done.). And if people can, they will use those hacks. If you split your repo up from the get go, the worst thing you can get that you'll have to assemble multiple distinct, well-encapsulated (in terms of project structure) things into one. In Git, that could lead to multiple root commits, but that's about it.
  - wcdolphin 8y ago
    No. The worst case is that the engineering team spent more time working on “well encapsulated projects” than on the most important project for their business and are all now out of jobs. Most companies don’t fail because of tech debt. And certainly not because of version control tech debt.
    - 0xFACEFEED 8y ago
      Exactly. Whenever I see an engineer take a hardline position (eg: "no monorepos you zealots!") I always ask myself: is this person just annoyed? Most of the time they're just annoyed. One side effect of every successful business are annoyed worker ants that are sick of dealing with growth problems. I've been there. I know how annoying it can be. Personally I've found comfort in embracing the chaos and learning to manage it responsibly. No dogma. No absolutes. Know how to do monorepos well. Know how to do polyrepos well. Learn the pitfalls of both. Don't assume other people are stupid zealots.
      - rkangel 8y ago
        I agree. Any article (like the linked one) that states one side of a case as an absolute without giving any exceptions or caveats is going to be greeted by me with scepticism. Particularly as he keeps mentioning 3 large engineering organisations that disagree with him.
    - btschaegg 8y ago
      Not exactly. (At least small) companies can go out of business because of bugs. And one great way to "achieve" said bugs are implicit dependencies hidden from developers that didn't introduce them. > The worst case is that the engineering team spent more time working on “well encapsulated projects” than on the most important project for their business I'm not really sure how I should read this. Don't you use your repos to solve business problems? Why should that change because of the repo layout?
      - wsy 8y ago
        If you do a poly-repo approach from the start, and have dependencies between repos, you need to introduce component versioning from the start. Component versioning doesn't solve any business problems, but requires engineering effort.
      - yowlingcat 8y ago
        Small businesses are not going to go out of business because of bugs unless those bugs aren't addressed. They'll go out of business because of poor sales and product management. Different things.
  - jogjayr 8y ago
    > Monorepos allow you to do very bad hacks (I need this other component over there; let me just put in a Symlink. Done.). Why would you put in a symlink? You could just provide a path to the actual component and import it into your project. > the worst thing you can get that you'll have to assemble multiple distinct, well-encapsulated (in terms of project structure) things into one When you have multiple repos, you also have multiple versions and releases of things. Now every team has the following options: 1. Backport critical fixes to every version still in use (hard to scale) 2. Publish a deprecation policy, aggressively deprecate older releases, and ignore pleading and begging from any team that fails to upgrade (infeasible - there'll always be a team that can't upgrade at that moment for some reason) You also have to solve the conflicting diamond dependency problem. This is when libfoo and libbar both depend on different versions of libbarfoodep. It's even more fun in Java because everything compiles and builds, but fails at runtime. So now you have to add rules and checks to your entire dependency tree - some package managers have this (Maven), others don't (npm IIRC).
    - jhasse 8y ago
      > Why would you put in a symlink? You could just provide a path to the actual component and import it into your project. Where do I need to put the path again? Ah what the heck, I'll just add a symlink inside a folder that's already somewhere in the build definitions.
      - xorcist 8y ago
        No VCS will ever protect you from crappy code though.
        jhasse 8y ago
        That's true, but we should at least make it as hard as possible to write crappy code ;)
      - fnord123 8y ago
        What language and build tool is this that you're using? I don't know anyone who has abused Maven or Cargo or Go like this. And I don't imagine Visual Studio Solutions for C# are used like this. Is there an underlying disagreement based on JS/Ruby/Python scriptish coding (which creaks when a lot of developers work on it) vs C and C++ (which have astonishingly bad build system stories) vs big-iron languages that don't sweat when in a monorepo.
  - Xylakant 8y ago
    > Monorepos allow you to do very bad hacks (I need this other component over there; let me just put in a Symlink. Done.). I've seen that with polyrepos as well: The entire project would require you to clone the individual repos into a specific directory structure so that things would work (no, not even submodules).
- 0xFACEFEED 8y ago
  > You can't split monorepos after the fact, at least not without immense costs. Sure you can. The difficulty of doing so depends on many (many) factors. If your team does their job well then the costs won't be immense. It might be annoying, but not that hard. Speaking in absolutes or platitudes solves nothing. Sometimes monorepos make sense. Sometimes polyrepos make sense. It's entirely dependent on what your company does.
  - adrianN 8y ago
    Of course, if everybody is very diligent in keeping things in the monorepo distinct and independent, then it's easy to split it later on. But relying on constant diligence doesn't work out in the long run in my experience.
    - Lio 8y ago
      It's not like polyrepos solve the long term diligence problem though. E.g. if relying solely on a package manager to to keep coupled things in lock step, you need make sure that version numbers are kept up to date for every little change made to every library. You can easily end up with a situation where someone in another team makes a small change but doesn't change the lib version number. That's a people issue but it does happen. You can get round that by using a repo SHA but now you two things to keep up to date for every library. Like wise you'll have to be diligent in versioning APIs. Anecdotally I've found it easier to keep things in lock step when in a single repo and using a single pull request for each story than I have where separate teams have to keep separate repos in sync. Both work but the monorepo approach worked better for the projects I've worked on. It just lead to less moving parts and more repeatability when there's a single SHA to watch. I also have been luck enough that I haven't worked on a project so large that we couldn't build a monorepo on a single machine with "normal" build tooling.
- jogjayr 8y ago
  > You can always just put all your small repos into a big one. It's not quite as simple as that. You'll need to avoid rebuilding the entire repo for every change - using something like Bazel. This means your build tooling has to be replaced entirely, which is a non-trivial task and not something your devops/release engineering team will thank you for. For any 3rd party libraries used by your projects you need to either ingest those projects into your monorepo and update forever. Or keep npm/maven/pip/gem/whatever around just for managing 3rd party dependencies (+ whatever system you use to front the main language package registries, because of course you're not talking directly to NPM/Maven central are you? What if they go down or do a leftpad?). I think either system - monorepos or polyrepos - works fine; just pick one and stick with it. Monorepos will probably give you better velocity starting out. Past a certain size, which most software shops will never hit, the already-available tools lend themselves better to polyrepo. And more devs know polyrepo tools (eg. Jenkins) than the corresponding monorepo solution (eg. Bazel). Things might swing in favor of monorepo on the VCS side if Twitter/Google/FB ever open-source their stuff.
  - adrianN 8y ago
    Eh, if you small repos build separately before you put them into one big repository, they'll build separately after, even if you just have a Makefile in each. If building your software depended on building its parts first, you already have the tooling to do so.
    - jogjayr 8y ago
      This really does not work for all languages. Speaking from personal experience, trying to monorepo multiple JS projects managed by NPM, it's more work than that.
      - deleted 8y ago
        [deleted]
      - adrianN 8y ago
        I know neiher JS nor NPM. Could you elaborate a bit? What's the difference between having your code in folders A B C, each their own repo, and having your code in folders A B C, subfolders of some monorepo? Does NPM try to do something clever with your VCS?
- tome 8y ago
  It's funny you say that, because the currently top-voted comment says exactly the opposite: https://news.ycombinator.com/item?id=18811368 https://news.ycombinator.com/item?id=18811368
- rauhl 8y ago
  > You can't split monorepos after the fact, at least not without immense costs. That has not been my experience at all. At a previous employer, we did exactly that with a multi-language library. In fairness, having multiple languages enforced fairly good directory structure in our single repo. But isn’t that the real point: good structure makes life easier, period. The thing is, going into a project you often don’t know what the right structures are yet. Creating a new repo for each component you think you need ossifies those choices, making it far more difficult to walk back on them later on (first because you may not even see the architectural mistake, second because the maintainers of that component will have an investment in its existence). > You can always just put all your small repos into a big one. In my experience, that’s harder, precisely because over time so much tooling has been built into each repo to manage builds, images, deployment &c. I’ve worked in monorepos & I’ve worked in multrepos, and so far my experience has been that monorepos enable faster velocity and more-maintainable software. I’ve not (yet) worked at Google- or Facebook-scale, though, and I’m completely open to the idea that at that scale a team really does need lots and lots of repos, and tooling to stitch them all together.
outsomnia 8y ago
The author talks about proponents of monorepos, but I thought when I read it: actually they are victims of monorepos trying to explain to themselves as much as anyone why they choose to suffer with them. (Actual reason: for $$$). Nobody would choose to drag around every historical afterthought in the development sequence of long forgotten software going back three decades that no longer builds with current tools, just so they can work on a small library off in a corner. Software is getting written and added to these monorepos at a much faster rate than hardware and networks are able to hide the bloat-upon-bloat growth of them.
- zbentley 8y ago
  Bloat and the failure to remove broken/bad legacy code have nothing to do with monorepos. "Hey, what does this server do?" "No idea; it hasn't been touched in years. What's deployed to it?" "Some 'foobar-ng' thing, never heard of that. Says it was last updated 5 years ago. Pull up our source repo for that package, will you?" "Hang on, we've got like 30 services with names containing 'foobar', let me find that one . . . oh god. You don't wanna know." "Fuck it, I'm just gonna shut this server down and remove that ancient, dead, busted package." "The main billing system just broke! What did you do?!"
- marcinzm 8y ago
  >Nobody would choose to drag around every historical afterthought in the development sequence of long forgotten software going back three decades that no longer builds with current tools, just so they can work on a small library off in a corner. If it doesn't work then it should be deleted. If it's still running somewhere then it should be maintained. Presumably you have a CI system so the monorepo actually requires everything in it to build. In my experience, it's polyrepos that allow for dead and un-maintained code to just sit there for eternity. You forget about that unused repo right until the moment the service it deploys to (if you can track down that dependency) needs an update or goes down. Monorepos can more easily force system wide CI that checks for broken dependencies or other issues.
  - bluGill 8y ago
    That old code worked 30 years ago... We have a closet in our office with a computer with Windows XP no service packs, and whatever compiler was used to build at the time. If we every have to release an update for whatever we were shipping we can do so. (assuming that computer still boots after all these years...) In the embedded world supporting software for 30 years is not unheard of. We avoid it, but it is in the back of our mind that someday we might have to release an update. Fortunately 30 years ago nothing was internet connected, we are worried that we might be releasing security updates for our current products 50 years from now...

adrianN 8y ago

You can't split monorepos after the fact, at least not without immense costs. You can always just put all your small repos into a big one.

btschaegg 8y ago
I think there's a nuance to this that should be pointed out: Monorepos allow you to do very bad hacks (I need this other component over there; let me just put in a Symlink. Done.). And if people can, they will use those hacks. If you split your repo up from the get go, the worst thing you can get that you'll have to assemble multiple distinct, well-encapsulated (in terms of project structure) things into one. In Git, that could lead to multiple root commits, but that's about it.
- wcdolphin 8y ago
  No. The worst case is that the engineering team spent more time working on “well encapsulated projects” than on the most important project for their business and are all now out of jobs. Most companies don’t fail because of tech debt. And certainly not because of version control tech debt.
  - 0xFACEFEED 8y ago
    Exactly. Whenever I see an engineer take a hardline position (eg: "no monorepos you zealots!") I always ask myself: is this person just annoyed? Most of the time they're just annoyed. One side effect of every successful business are annoyed worker ants that are sick of dealing with growth problems. I've been there. I know how annoying it can be. Personally I've found comfort in embracing the chaos and learning to manage it responsibly. No dogma. No absolutes. Know how to do monorepos well. Know how to do polyrepos well. Learn the pitfalls of both. Don't assume other people are stupid zealots.
    - rkangel 8y ago
      I agree. Any article (like the linked one) that states one side of a case as an absolute without giving any exceptions or caveats is going to be greeted by me with scepticism. Particularly as he keeps mentioning 3 large engineering organisations that disagree with him.
  - btschaegg 8y ago
    Not exactly. (At least small) companies can go out of business because of bugs. And one great way to "achieve" said bugs are implicit dependencies hidden from developers that didn't introduce them. > The worst case is that the engineering team spent more time working on “well encapsulated projects” than on the most important project for their business I'm not really sure how I should read this. Don't you use your repos to solve business problems? Why should that change because of the repo layout?
    - wsy 8y ago
      If you do a poly-repo approach from the start, and have dependencies between repos, you need to introduce component versioning from the start. Component versioning doesn't solve any business problems, but requires engineering effort.
    - yowlingcat 8y ago
      Small businesses are not going to go out of business because of bugs unless those bugs aren't addressed. They'll go out of business because of poor sales and product management. Different things.
- jogjayr 8y ago
  > Monorepos allow you to do very bad hacks (I need this other component over there; let me just put in a Symlink. Done.). Why would you put in a symlink? You could just provide a path to the actual component and import it into your project. > the worst thing you can get that you'll have to assemble multiple distinct, well-encapsulated (in terms of project structure) things into one When you have multiple repos, you also have multiple versions and releases of things. Now every team has the following options: 1. Backport critical fixes to every version still in use (hard to scale) 2. Publish a deprecation policy, aggressively deprecate older releases, and ignore pleading and begging from any team that fails to upgrade (infeasible - there'll always be a team that can't upgrade at that moment for some reason) You also have to solve the conflicting diamond dependency problem. This is when libfoo and libbar both depend on different versions of libbarfoodep. It's even more fun in Java because everything compiles and builds, but fails at runtime. So now you have to add rules and checks to your entire dependency tree - some package managers have this (Maven), others don't (npm IIRC).
  - jhasse 8y ago
    > Why would you put in a symlink? You could just provide a path to the actual component and import it into your project. Where do I need to put the path again? Ah what the heck, I'll just add a symlink inside a folder that's already somewhere in the build definitions.
    - xorcist 8y ago
      No VCS will ever protect you from crappy code though.
      - jhasse 8y ago
        That's true, but we should at least make it as hard as possible to write crappy code ;)
    - fnord123 8y ago
      What language and build tool is this that you're using? I don't know anyone who has abused Maven or Cargo or Go like this. And I don't imagine Visual Studio Solutions for C# are used like this. Is there an underlying disagreement based on JS/Ruby/Python scriptish coding (which creaks when a lot of developers work on it) vs C and C++ (which have astonishingly bad build system stories) vs big-iron languages that don't sweat when in a monorepo.
- Xylakant 8y ago
  > Monorepos allow you to do very bad hacks (I need this other component over there; let me just put in a Symlink. Done.). I've seen that with polyrepos as well: The entire project would require you to clone the individual repos into a specific directory structure so that things would work (no, not even submodules).
0xFACEFEED 8y ago
> You can't split monorepos after the fact, at least not without immense costs. Sure you can. The difficulty of doing so depends on many (many) factors. If your team does their job well then the costs won't be immense. It might be annoying, but not that hard. Speaking in absolutes or platitudes solves nothing. Sometimes monorepos make sense. Sometimes polyrepos make sense. It's entirely dependent on what your company does.
- adrianN 8y ago
  Of course, if everybody is very diligent in keeping things in the monorepo distinct and independent, then it's easy to split it later on. But relying on constant diligence doesn't work out in the long run in my experience.
  - Lio 8y ago
    It's not like polyrepos solve the long term diligence problem though. E.g. if relying solely on a package manager to to keep coupled things in lock step, you need make sure that version numbers are kept up to date for every little change made to every library. You can easily end up with a situation where someone in another team makes a small change but doesn't change the lib version number. That's a people issue but it does happen. You can get round that by using a repo SHA but now you two things to keep up to date for every library. Like wise you'll have to be diligent in versioning APIs. Anecdotally I've found it easier to keep things in lock step when in a single repo and using a single pull request for each story than I have where separate teams have to keep separate repos in sync. Both work but the monorepo approach worked better for the projects I've worked on. It just lead to less moving parts and more repeatability when there's a single SHA to watch. I also have been luck enough that I haven't worked on a project so large that we couldn't build a monorepo on a single machine with "normal" build tooling.
jogjayr 8y ago
> You can always just put all your small repos into a big one. It's not quite as simple as that. You'll need to avoid rebuilding the entire repo for every change - using something like Bazel. This means your build tooling has to be replaced entirely, which is a non-trivial task and not something your devops/release engineering team will thank you for. For any 3rd party libraries used by your projects you need to either ingest those projects into your monorepo and update forever. Or keep npm/maven/pip/gem/whatever around just for managing 3rd party dependencies (+ whatever system you use to front the main language package registries, because of course you're not talking directly to NPM/Maven central are you? What if they go down or do a leftpad?). I think either system - monorepos or polyrepos - works fine; just pick one and stick with it. Monorepos will probably give you better velocity starting out. Past a certain size, which most software shops will never hit, the already-available tools lend themselves better to polyrepo. And more devs know polyrepo tools (eg. Jenkins) than the corresponding monorepo solution (eg. Bazel). Things might swing in favor of monorepo on the VCS side if Twitter/Google/FB ever open-source their stuff.
- adrianN 8y ago
  Eh, if you small repos build separately before you put them into one big repository, they'll build separately after, even if you just have a Makefile in each. If building your software depended on building its parts first, you already have the tooling to do so.
  - jogjayr 8y ago
    This really does not work for all languages. Speaking from personal experience, trying to monorepo multiple JS projects managed by NPM, it's more work than that.
    - deleted 8y ago
      [deleted]
    - adrianN 8y ago
      I know neiher JS nor NPM. Could you elaborate a bit? What's the difference between having your code in folders A B C, each their own repo, and having your code in folders A B C, subfolders of some monorepo? Does NPM try to do something clever with your VCS?
tome 8y ago
It's funny you say that, because the currently top-voted comment says exactly the opposite: https://news.ycombinator.com/item?id=18811368 https://news.ycombinator.com/item?id=18811368
rauhl 8y ago
> You can't split monorepos after the fact, at least not without immense costs. That has not been my experience at all. At a previous employer, we did exactly that with a multi-language library. In fairness, having multiple languages enforced fairly good directory structure in our single repo. But isn’t that the real point: good structure makes life easier, period. The thing is, going into a project you often don’t know what the right structures are yet. Creating a new repo for each component you think you need ossifies those choices, making it far more difficult to walk back on them later on (first because you may not even see the architectural mistake, second because the maintainers of that component will have an investment in its existence). > You can always just put all your small repos into a big one. In my experience, that’s harder, precisely because over time so much tooling has been built into each repo to manage builds, images, deployment &c. I’ve worked in monorepos & I’ve worked in multrepos, and so far my experience has been that monorepos enable faster velocity and more-maintainable software. I’ve not (yet) worked at Google- or Facebook-scale, though, and I’m completely open to the idea that at that scale a team really does need lots and lots of repos, and tooling to stitch them all together.

outsomnia 8y ago

The author talks about proponents of monorepos, but I thought when I read it: actually they are victims of monorepos trying to explain to themselves as much as anyone why they choose to suffer with them. (Actual reason: for $$$). Nobody would choose to drag around every historical afterthought in the development sequence of long forgotten software going back three decades that no longer builds with current tools, just so they can work on a small library off in a corner. Software is getting written and added to these monorepos at a much faster rate than hardware and networks are able to hide the bloat-upon-bloat growth of them.

zbentley 8y ago
Bloat and the failure to remove broken/bad legacy code have nothing to do with monorepos. "Hey, what does this server do?" "No idea; it hasn't been touched in years. What's deployed to it?" "Some 'foobar-ng' thing, never heard of that. Says it was last updated 5 years ago. Pull up our source repo for that package, will you?" "Hang on, we've got like 30 services with names containing 'foobar', let me find that one . . . oh god. You don't wanna know." "Fuck it, I'm just gonna shut this server down and remove that ancient, dead, busted package." "The main billing system just broke! What did you do?!"
marcinzm 8y ago
>Nobody would choose to drag around every historical afterthought in the development sequence of long forgotten software going back three decades that no longer builds with current tools, just so they can work on a small library off in a corner. If it doesn't work then it should be deleted. If it's still running somewhere then it should be maintained. Presumably you have a CI system so the monorepo actually requires everything in it to build. In my experience, it's polyrepos that allow for dead and un-maintained code to just sit there for eternity. You forget about that unused repo right until the moment the service it deploys to (if you can track down that dependency) needs an update or goes down. Monorepos can more easily force system wide CI that checks for broken dependencies or other issues.
- bluGill 8y ago
  That old code worked 30 years ago... We have a closet in our office with a computer with Windows XP no service packs, and whatever compiler was used to build at the time. If we every have to release an update for whatever we were shipping we can do so. (assuming that computer still boots after all these years...) In the embedded world supporting software for 30 years is not unheard of. We avoid it, but it is in the back of our mind that someday we might have to release an update. Fortunately 30 years ago nothing was internet connected, we are worried that we might be releasing security updates for our current products 50 years from now...