5 ms·

These benefits do exist, but drawbacks still exist too. The "constantly updating, constantly recompiling" problem can get pretty bad when there are many develop

These benefits do exist, but drawbacks still exist too. The "constantly updating, constantly recompiling" problem can get pretty bad when there are many developers. Developers either lose time recompiling more than they should need to, or keep working in out-of-date trees and dealing with trickier merges. Multiply by the number of developers in a large org, and that can be a lot of lost productivity. The thing I don't see discussed often enough, in all of the "monorepos are good/bad" debates, is that they require a certain discipline. When one commit can span multiple components, it's easy to add or ignore bad coupling between those components. That contributes to the rebuild treadmill for everyone, and is bad design in a bunch of other ways as well. IDEs make it worse, because they obscure the boundaries between components, but everyone ends up using an IDE because their predecessors made it very difficult to follow the flow of control across components otherwise. So you end up with one big ball of mud. With separate repos this is discouraged, because violating the rules of good design causes immediate pain. With a monorepo the pain must come in the form of stronger reviews, to discourage bad coupling even though it's easy. Unfortunately, the very same companies that have embraced monorepos have also abandoned separation of concerns as a design principle. :( ETA: please read bunderbunder's comment (https://news.ycombinator.com/item?id=19796960 https://news.ycombinator.com/item?id=19796960) as well. It's an insightful take on some of the same issues.

Waterluvian 8y ago
I like your comment about commits spanning components. Having many repos forces my team to think about how to stage changes that span many repos.
- jsight 8y ago
  TBH, I'm not convinced that a great effort to think about "how to stage changes that span many repos" is always (or maybe even often) a benefit. It seems like the simpler approach of just making it easy to make multimodule changes is a benefit of the monorepo.
  - notacoward 8y ago
    Having to stage changes across multiple repos, and minimize dependencies between them, can definitely create more work up front. The point is that it lessens the recurring cost (not just across time but across many developers) of having those spurious dependencies in the code forever. Code spends more time being maintained than being written, so it's the recurring cost that dominates long-term productivity.
  - bluGill 8y ago
    That depends on your deployment model. Google is live at head, with no support for past releases. In the web world this is reasonable: they control the servers and thus can force releases. I don't work in that world though, there are times where I have to support customers on older releases who refuse to upgrade (when the customer is spending millions of dollars per year you can't ignore their rules no matter who stupid you think they are). Thus I need to think about those factors anyway.
    - jsolson 8y ago
      Google still has the notion of release branches. They're point-in-time snapshots of the monorepo (copy on write), plus the occasional cherry-pick from HEAD. Development happens at HEAD is the important part.
jbwyme 8y ago
This is a concern with monorepo but you can still enforce the boundaries programmatically. For example you can add some simple, custom lint rules that prevent cross-boundary imports. Another thing you can do is have your CI only check out specific directories when running tests using something like git’s sparse-checkout.
ceronman 8y ago
There is available tooling to help you with that, not only has to be discipline. A monorepo oriented build tool such as Bazel will make sure that you don't compile more than needed. This is extremely efficient. In fact, in my company with a few hundred projects we tried it and it was 10 times faster than our previous Maven based workflow. These tools also help you with dependency management so you keep your code organized. The killer feature is that you finally have true Continuous Integration because the build tool will figure out any dependencies of the code that you change, rebuilding it and testing to make sure that nothing is broken.
- notacoward 8y ago
  > A monorepo oriented build tool such as Bazel will make sure that you don't compile more than needed. Not at all true in my experience. I use Buck, which is very similar. It constantly recompiles stuff it doesn't need to by any standard. I've seriously considered avoiding Mercurial bookmarks in favor of manual patch management to see if I can avoid triggering the cases where it gets stupid. Maybe it's a problem specific to Buck, but my conversations with peers at other companies suggest not (and only a tiny percentage of developers at any of these companies get a choice of which tool to use). The other issue is that "more than needed" is subject to more than one interpretation. There's "more than needed" as code was actually written, and there's "more than needed" as code should have been written. Tooling can help with the first but not the second. If developers are sloppy about how their header files are structured, then a change to one that's widely used will make it "necessary" to recompile a lot of code - including code that didn't actually use the part that changed. Oddly enough, though, that "necessary" recompilation wouldn't occur if those same header files were split up. (BTW some languages make good design more difficult by requiring private implementation details in public header files, but that's a separate conversation.) Tooling can optimize what it can see, but what it sees might already be a tangle of dependencies added only because a monorepo made it easy.
  - rkangel 8y ago
    It sounds like it might be worth trying Bazel (I haven't used either). The parent made a suggestion of a specific tool with one anecdatum about performance, and your argument is that a different tool doesn't do the same thing.
closeparen 8y ago
Your language and/or build system should provide visibility controls to limit what can be coupled. Package-privacy exists for a reason. Yes you can post a PR that relaxes someone else’s control but they can block it.
dfabulich 8y ago
In my experience, the first time the team tries to separate concerns, they don't necessarily fully understand the problem, and build out the wrong separations. Multirepo teams live with the wrong modularity boundaries longer than they should, because fixing the design ("violating the rules," when they're bad rules) causes immediate pain. In monorepo it's easier to do the refactor in an atomic commit. It's all part of the sandboxing cycle. https://xkcd.com/2044/ https://xkcd.com/2044/

Waterluvian 8y ago

I like your comment about commits spanning components. Having many repos forces my team to think about how to stage changes that span many repos.

jsight 8y ago
TBH, I'm not convinced that a great effort to think about "how to stage changes that span many repos" is always (or maybe even often) a benefit. It seems like the simpler approach of just making it easy to make multimodule changes is a benefit of the monorepo.
- notacoward 8y ago
  Having to stage changes across multiple repos, and minimize dependencies between them, can definitely create more work up front. The point is that it lessens the recurring cost (not just across time but across many developers) of having those spurious dependencies in the code forever. Code spends more time being maintained than being written, so it's the recurring cost that dominates long-term productivity.
- bluGill 8y ago
  That depends on your deployment model. Google is live at head, with no support for past releases. In the web world this is reasonable: they control the servers and thus can force releases. I don't work in that world though, there are times where I have to support customers on older releases who refuse to upgrade (when the customer is spending millions of dollars per year you can't ignore their rules no matter who stupid you think they are). Thus I need to think about those factors anyway.
  - jsolson 8y ago
    Google still has the notion of release branches. They're point-in-time snapshots of the monorepo (copy on write), plus the occasional cherry-pick from HEAD. Development happens at HEAD is the important part.

jbwyme 8y ago

This is a concern with monorepo but you can still enforce the boundaries programmatically. For example you can add some simple, custom lint rules that prevent cross-boundary imports. Another thing you can do is have your CI only check out specific directories when running tests using something like git’s sparse-checkout.

ceronman 8y ago

There is available tooling to help you with that, not only has to be discipline. A monorepo oriented build tool such as Bazel will make sure that you don't compile more than needed. This is extremely efficient. In fact, in my company with a few hundred projects we tried it and it was 10 times faster than our previous Maven based workflow. These tools also help you with dependency management so you keep your code organized. The killer feature is that you finally have true Continuous Integration because the build tool will figure out any dependencies of the code that you change, rebuilding it and testing to make sure that nothing is broken.

notacoward 8y ago
> A monorepo oriented build tool such as Bazel will make sure that you don't compile more than needed. Not at all true in my experience. I use Buck, which is very similar. It constantly recompiles stuff it doesn't need to by any standard. I've seriously considered avoiding Mercurial bookmarks in favor of manual patch management to see if I can avoid triggering the cases where it gets stupid. Maybe it's a problem specific to Buck, but my conversations with peers at other companies suggest not (and only a tiny percentage of developers at any of these companies get a choice of which tool to use). The other issue is that "more than needed" is subject to more than one interpretation. There's "more than needed" as code was actually written, and there's "more than needed" as code should have been written. Tooling can help with the first but not the second. If developers are sloppy about how their header files are structured, then a change to one that's widely used will make it "necessary" to recompile a lot of code - including code that didn't actually use the part that changed. Oddly enough, though, that "necessary" recompilation wouldn't occur if those same header files were split up. (BTW some languages make good design more difficult by requiring private implementation details in public header files, but that's a separate conversation.) Tooling can optimize what it can see, but what it sees might already be a tangle of dependencies added only because a monorepo made it easy.
- rkangel 8y ago
  It sounds like it might be worth trying Bazel (I haven't used either). The parent made a suggestion of a specific tool with one anecdatum about performance, and your argument is that a different tool doesn't do the same thing.

closeparen 8y ago

Your language and/or build system should provide visibility controls to limit what can be coupled. Package-privacy exists for a reason. Yes you can post a PR that relaxes someone else’s control but they can block it.

dfabulich 8y ago

In my experience, the first time the team tries to separate concerns, they don't necessarily fully understand the problem, and build out the wrong separations. Multirepo teams live with the wrong modularity boundaries longer than they should, because fixing the design ("violating the rules," when they're bad rules) causes immediate pain. In monorepo it's easier to do the refactor in an atomic commit. It's all part of the sandboxing cycle. https://xkcd.com/2044/ https://xkcd.com/2044/