5 ms·

My Thoughts on Monorepo

37 points by shekhargulati 6y ago

davidjade 6y ago
Stopped reading when less than 10 seconds in a sign up pop up covered the article.
- Rumudiez 6y ago
  No popup for me in Firefox + uBlock Origin. Have you tried a reputable ad/tracking blocker?
  - davidjade 6y ago
    Yes, I am also using Firefox with uBlock Origin locked down - no 3rd-party scripts, 3rd-party no frames, etc... Local network also has a pi-hole. I should clarify that it was an overlay style popup - not an actual browser window popup.
    - lokedhs 6y ago
      I also didn't get a popup. I then tried after disabling Ublock and Privacy Badger in a private browsing window to ensure that no cookies affected the result. After doing this, I still did not get any popup.
justicezyx 6y ago
The disadvantages are more like "it's not affordable" ``` Monorepos could slow down developers because of slow build times, poor tooling, and merge conflicts. Most developers still in 2020 struggle to cleanly merge code. Git is slow for projects with large numbers of files and history. There is cognitive overhead involved as developers have to get comfortable with a much larger code base than they would have with multirepo setup. To do monorepo well require investment in tooling that most organization non-tech leadership will fail to understand ``` Monorepo is clearly more desirable than distributed repo. But it's not a cheap solution.
- 908B64B197 6y ago
  > Most developers still in 2020 struggle to cleanly merge code Sounds like the hiring bar is too low.
  - deathanatos 6y ago
    We regularly¹ have unclean merges at my company, and I consider my coworkers competent, and well above "the bar". The biggest problem we have with clean merges is that nearly every CI system & code review tool out there doesn't test the final product — the merge commit — until after it is merged. GitHub follows this flow by default, and usually even with more CI tooling on the side. Something like bors-ng is what one has to adopt. (And we're struggling to do this for our monorepo due to other points the article touches on.) While it is possible (if difficult) in git, some VCS systems struggle even harder with this: Perforce and derivatives, for example, simply can't, by design, perform atomic test & commit. ¹like once a month, or so, over the whole company
    - 908B64B197 6y ago
      Could this be fixed by using a better branching strategy?
    - michael1999 6y ago
      This is my experience too. Traditional ci hygiene requires that the build runs clean on the developer machine before publishing a change. But once the build time crosses 10 minutes, people resent waiting and start to run test subsets, and count on the ci server to catch breakage. Once that becomes normal, breakage is inevitable. And if commit tempo is higher than a developer can finish a test-run, then it is impossible. Solutions include patch queues, pr builds, etc. but those are all more complicated than a single stream on head.
    - gct 6y ago
      merge into a stabilize branch and test that...
erik_seaberg 6y ago
Atomic commits are a red herring because deployment isn’t atomic or irreversible. You still have to think about how the system stays healthy while a new build is halfway deployed. A deprecated API still needs to work until you’re certain you will never roll back any consumer and start calling it again.
- rav 6y ago
  At least with a monorepo you only have to worry about deploy ordering. With a multirepo you have to worry about code review and merge order before you get to worry about the deploy order. I would like to be able to review the library API change together with the changes in the app that need the new API. At my work we either review library changes in isolation, or we skip the review on library changes (if the author deems them trivial) and only do review on the main app changes, but that means we often miss opportunities for making the library API fit the use-cases as well as they could.
cousin_it 6y ago
Well informed article, thank you. In practice I'm pretty happy with a huge monorepo, but have a somewhat surprising problem with it: it makes fine-grained code reuse too easy. People end up adding tons of dependencies, and building anything requires building ten thousand targets at head. (It seems the npm ecosystem ran into the same problem from another angle, they also made fine-grained reuse too easy and ended up with huge dependency trees.) In theory, the solution is using larger libraries with less frequent releases, but then we run into lag and the diamond problem again. I don't know how to square this circle.
- ithkuil 6y ago
  That's why bazel has visibility rules. You have to explicitly mark things public or express which package can import which package. If used judiciously it can help keep things in check or at least have the teams involved be aware of the maintenance burden.
Glavnokoman 6y ago
Having tried both worlds. If a company offers me a job and says they develop in a monorepo I am not going to tell them to f*ck off. I would just ask for twice the salary I would agree to work for in a multirepo company.

37 points by shekhargulati 6y ago

davidjade 6y ago

Stopped reading when less than 10 seconds in a sign up pop up covered the article.

Rumudiez 6y ago
No popup for me in Firefox + uBlock Origin. Have you tried a reputable ad/tracking blocker?
- davidjade 6y ago
  Yes, I am also using Firefox with uBlock Origin locked down - no 3rd-party scripts, 3rd-party no frames, etc... Local network also has a pi-hole. I should clarify that it was an overlay style popup - not an actual browser window popup.
  - lokedhs 6y ago
    I also didn't get a popup. I then tried after disabling Ublock and Privacy Badger in a private browsing window to ensure that no cookies affected the result. After doing this, I still did not get any popup.

justicezyx 6y ago

The disadvantages are more like "it's not affordable" ``` Monorepos could slow down developers because of slow build times, poor tooling, and merge conflicts. Most developers still in 2020 struggle to cleanly merge code. Git is slow for projects with large numbers of files and history. There is cognitive overhead involved as developers have to get comfortable with a much larger code base than they would have with multirepo setup. To do monorepo well require investment in tooling that most organization non-tech leadership will fail to understand ``` Monorepo is clearly more desirable than distributed repo. But it's not a cheap solution.

908B64B197 6y ago
> Most developers still in 2020 struggle to cleanly merge code Sounds like the hiring bar is too low.
- deathanatos 6y ago
  We regularly¹ have unclean merges at my company, and I consider my coworkers competent, and well above "the bar". The biggest problem we have with clean merges is that nearly every CI system & code review tool out there doesn't test the final product — the merge commit — until after it is merged. GitHub follows this flow by default, and usually even with more CI tooling on the side. Something like bors-ng is what one has to adopt. (And we're struggling to do this for our monorepo due to other points the article touches on.) While it is possible (if difficult) in git, some VCS systems struggle even harder with this: Perforce and derivatives, for example, simply can't, by design, perform atomic test & commit. ¹like once a month, or so, over the whole company
  - 908B64B197 6y ago
    Could this be fixed by using a better branching strategy?
  - michael1999 6y ago
    This is my experience too. Traditional ci hygiene requires that the build runs clean on the developer machine before publishing a change. But once the build time crosses 10 minutes, people resent waiting and start to run test subsets, and count on the ci server to catch breakage. Once that becomes normal, breakage is inevitable. And if commit tempo is higher than a developer can finish a test-run, then it is impossible. Solutions include patch queues, pr builds, etc. but those are all more complicated than a single stream on head.
  - gct 6y ago
    merge into a stabilize branch and test that...

erik_seaberg 6y ago

Atomic commits are a red herring because deployment isn’t atomic or irreversible. You still have to think about how the system stays healthy while a new build is halfway deployed. A deprecated API still needs to work until you’re certain you will never roll back any consumer and start calling it again.

rav 6y ago
At least with a monorepo you only have to worry about deploy ordering. With a multirepo you have to worry about code review and merge order before you get to worry about the deploy order. I would like to be able to review the library API change together with the changes in the app that need the new API. At my work we either review library changes in isolation, or we skip the review on library changes (if the author deems them trivial) and only do review on the main app changes, but that means we often miss opportunities for making the library API fit the use-cases as well as they could.

cousin_it 6y ago

Well informed article, thank you. In practice I'm pretty happy with a huge monorepo, but have a somewhat surprising problem with it: it makes fine-grained code reuse too easy. People end up adding tons of dependencies, and building anything requires building ten thousand targets at head. (It seems the npm ecosystem ran into the same problem from another angle, they also made fine-grained reuse too easy and ended up with huge dependency trees.) In theory, the solution is using larger libraries with less frequent releases, but then we run into lag and the diamond problem again. I don't know how to square this circle.

ithkuil 6y ago
That's why bazel has visibility rules. You have to explicitly mark things public or express which package can import which package. If used judiciously it can help keep things in check or at least have the teams involved be aware of the maintenance burden.

Glavnokoman 6y ago

Having tried both worlds. If a company offers me a job and says they develop in a monorepo I am not going to tell them to f*ck off. I would just ask for twice the salary I would agree to work for in a multirepo company.