4 ms·

Can someone explain the rationale behind switching to a monorepo? I just don't get it. Does it mean also having a single unified production environment build?

• whydid 4y ago
  How many people are in your organization?
  • nsm 4y ago
    People seem to conflate a monorepo with having everything else the same as well, just because that's how Google and other BigCos do it. You don't have to. At some level a monorepo is just a way to stick all your code in one giant directory and manage it all under one VCS repository. You could still do separate build tools per project, separate vendoring if you really wanted and so on. However you may find that being able to simply import other first party code by path instead of doing some cross repo dependency process is a massive win. Edit: apologies, this was meant to be a reply to the top level comment.
    • Galanwe 4y ago
      > However you may find that being able to simply import other first party code by path instead of doing some cross repo dependency process is a massive win. I don't quite get the benefits of "depending by path". The engineering seems huge, you now have to create a magic meta build and testing system so that only individual components that changed are rebuilt/tested. That seems like a scaling nightmare at best. Also, dependency management is hardly an issue on most modern stacks anyway. Javascript, rust, python, etc, all have private package hosting tools that are trivial to deploy.
    • CrimsonRain 4y ago
      Import by path for monorepo is an anti pattern. It is exactly what another gp complained about: turning monorepo into soft dependant monolith. https://news.ycombinator.com/item?id=34048245 https://news.ycombinator.com/item?id=34048245
• fsociety 4y ago
  At large companies with developer experience teams, it can be nice. You setup a singular repo (or even several), use a cross-language and cross-platform build system, and you can get a lot of gains. You can make sweeping code mods too and update your customer’s libraries for them. It makes working in all the languages mostly consistent, and provides a nice platform for optimizations like only testing code which changed. I think the mistake is smaller companies adopting it without understanding the large amount of investment it requires. If you have a mono-repo mostly in one language, and you hire someone who is going to work in another language then you could be in for a world of pain. Unless you resource the language support.
• lhorie 4y ago
  I maintain the web monorepo at Uber, so I think I can give some context. Monorepos allows us to centralize important dependency upgrades. E.g. fixing log4j vulns is a lot easier when you can patch everything simultaneously. Same for tzdata (2022g gave very little heads up) Auditing for npm supply chain attacks was a lot simpler in monorepo than microrepos. Etc. Monolithic version control doesn't have to mean monolithic everything. Our web projects can be deployed independently of each other, and we leverage tool like yarn workspace focus and bazel for granular installs and builds/tests/etc. It doesn't have to mean monoversions either. We support multiple version of libraries, though we prefer coalescing them as much as possible to facilitate effort centralization. Finding out that your library change will break downstreams before you land the change is a feature. We had microrepos before and the main problem is that to this day I still get some random team coming to me for help w/ some rediscovered 7 year old repo that doesn't even build anymore cus lockfiles weren't a thing back then. At a large enough org, you'll inevitably see the full spectrum of team quality, from the really good teams to the one intern/contractor getting thrown into the deep end of some unloved ancient thing. You want a common denominator that lets you do things like patch vulns in unstaffed projects. I've done fairly large migrations both to and from monorepos. Each has pros and cons. For us and companies like Google, monorepos work well with our organization model. For others it may not.
  • jsbdkdbcjcndkd 4y ago
    in 7 years someone will discovery a random subdirectory that doesn't build because wormholes weren't a thing back then ;) monorepos solve none of the problems you listed. you are just basking in the short lived light after a big refactoring. only thing monorepo does is make it easier to update some shared code and have all code which uses it run tests beforr pushing the new lib version. with many repos (micro repos is false speach to justify monorepos) you first publish the shared lib and then find out downstream failures. anyone using monorepo in a way you can't build a small pieceocally is doing it very wrong.
  • Galanwe 4y ago
    Thanks for your answer. How does CI work in practice? how do you avoid rebuilding and retesting the whole repo at every change? That would be an insane waste of resource and time.
• charcircuit 4y ago
  >Does it mean also having a single unified production environment build? No, the way you store the source code for projects is independent to how you ship them. >Not having to update dependencies globally has been a feature as well. For a large upgrade where a bot can't fix what will break it is typical to introduce it and have both versions in the monorepo at the same time. You still migrate each project one at a time and eventually remove the old version. Of course you could never finish migrating and have to support both versions at the same time. >I found that if the structuring of projects in repositories make sense I don't see how that makes any less sense than putting projects into folders. >The deployment seems like a nightmare too, having to update the whole stack at once because you then have no idea which individual service changed between releases. If your landing page is edited there is no reason that you should be deploying a new version of your mobile app. Deploying everything overran every commit is a tooling issue. >Not to mention I really don't want people to spend time migrating project X - that does its work perfectly without issue since 5 years - to the latest version of LibFooBar just because project Y wants it. Then pay the cost of supporting two versions of LibFooBar in your monorepo. You can completely emulate what you do with multirepo with a monorepo. A monorepo gives you extra things like a single revision that can let you see the state of everything from when a build was made or making it easy to depend on the latest version of libraries without having to constantly bump it either manually or via bots.
• allanrbo 4y ago
  A monorepo is indeed not necessarily the one true best way for everyone. But it is _a_ way that works well for certain large corps, such as Google, that can dedicate teams to solve challenges that indeed come with it. Some advantages off the top of my head: - Let's say I am getting an error in production, and it was built from version X of the repo. In 3 seconds I am navigating the source tree of all of my tens of thousands of dependencies at that exact version. - While developing, doing experiments, or reproing a bug, I can trivially make temporary changes to any dependency. It's a zero effort thing, so I often jump into any dependency without hesitating. For example for adding some extra logging. - Step debugging into code from any library dependency is trivial. - Making changes to a library let me use the build and test system to find if it breaks any user. Because the build and test system is completely consistent across the monorepo, I can easily dig into any user's code, fix it, and run their tests with my changed library. The "Software Engineering At Google" book page discusses more pros and cons: https://abseil.io/resources/swe-book/html/ch16.html#version_control_at_google https://abseil.io/resources/swe-book/html/ch16.html#version_...
• Groxx 4y ago
  It's much more of a political tool than a technical one. Every benefit (minus atomic commits*) can be had immediately on micro-repos with a for-loop to do the monorepo-thing in each one. If bazel is what you want, great, use bazel! In microrepos! Want consistent dependencies? Enforce them! HEAD must build? Wonderful, nothing's stopping you from doing that! It's all solvable, and quite easily. What you actually want is reliability and consistency. A monorepo gives you one political entity, with a clear adoption path, to argue with when enforcing those kinds of requirements. So they're much more likely to actually be achieved. In a big company, that may be worthwhile... but oh boy are the downsides large, and the only way to deal with them is massive eng effort and money. * Atomic commits are a completely false promise. Your code on multiple machines doesn't change execution atomically, and being able to split a breaking change is a good thing, and massively harder to support in monorepos - it lets you adopt changes gradually, rather than forcing it on everything at once. You know, the same thing that every safe-change-practice handbook says you should do. Except in monorepos apparently.
  • rtpg 4y ago
    Monorepo lets me have a _single_ Github PR with the changes across 4 projects in front of me. To replicate what you're talking about in a multirepo setup, you have to introduce changesets in 4 different places, then merge them in the right order, make a bunch of tiny releases, update commit hashes to point to the right one, and do a lot of other bookkeeping because in this house we keep our projects separated.
    • Galanwe 4y ago
      Indeed, the small additional bookkeeping is the price to pay. I'm quite used to it, so I don't even see it anymore. Is that all? These 5 less minutes of bookkeeping are the killer feature of monorepos? Because, as far as I see it, there's an insane amount of engineering to make a monorepo work even at small scale. Are these 5 minutes per multi-project PR worth it? I wouldn't be surprised these 5 minutes are largely offsets solely by the additional hours of CI testing time introduced by having to run tests on the whole monorepo at each commit instead of just the project that changed.
• SergeAx 4y ago
  The rationale is that Google, Meta, Microsoft and other successfull companies are running monorepos. Ergo, if you want your company to be successfull — you have to employ monorepo too. It's that simple!
  • stitched2gethr 4y ago
    I'm not sure it is. Some of us have evaluated several approaches based on their merits and tried to makes decisions that are best for our specific organization.