4 ms·

I love the concept of monorepos. With the proper tooling and support I think they can be force multipliers for high functioning engineering organizations. Howev

by zaphar 5y ago

I love the concept of monorepos. With the proper tooling and support I think they can be force multipliers for high functioning engineering organizations. However almost none of the publicly available tooling for CI/CD, IDEs, Source code hosting, or any of the other force multipliers out there support it properly, so for most organizations I think it's a wash. It's a shame really since if they did support monorepos better or even just stop assuming too much about the repository then a whole lot of things could be unlocked for engineering teams. As it stands though going the monorepo route means you'll be building a lot of bespoke tooling to support it.

MuffinFlavored 5y ago
> With the proper tooling and support Could you elaborate/provide some examples?
- Jensson 5y ago
  Google is the main example, their monorepo setup works and scales really well. It is basically a remote filesystem directory that automatically checks out the files you need to build your project and not other files.
  - akvadrako 5y ago
    Why would that be needed except for huge codebases at megacorps?
    - pxue 5y ago
      no one needs this except maybe google. even then if they started today with git maybe they wouldn't have used a monorepo. same as pointing to netflix and arguing you need 100+ microservices. no one needs this, maybe except netflix
      - zaphar 5y ago
        Using a monorepo was a deliberate choice at Google. They modified their source control solution to support a monorepo every time it failed to scale sufficiently. If they had started with Git those same people would probably have modified Git to make it work as a monorepo. There are specific economies of scale Google gets with a monorepo that help it keep up with the sheer amount of cruft and technical debt they generate everyday. Google writes a lot of code and ensuring that security fixes get applied or legacy deprecated APIs get removed is for some people a full time job. Having a monorepo makes that job actually achievable rather than impossible. Source: I worked there and was familiar with many of the engineers working on the tooling for the monorepo.
- zaphar 5y ago
  In a monorepo you usually only want to build portions of the codebase at any given time in your CI/CD. Most of the CI/CD tooling out there are accidentally hostile to this. As just one example it is often difficult to impossible to rebuild your code when a dependency changes. Build systems often don't expose this information usefully, so you'll find yourself either hand rolling something that looks at git changes and tries to infer what should be rebuilt and what shouldn't. Or you just rebuild everything in the entire repo every night. Or you can adopt Bazel everywhere which can be a hard sell in many companies but does give you better tools. Another example: Most source code hosting products don't give good support for path based permissions which you need in order to ensure that folks with the right context review code before it is merged in. Which pushes you to segregate your repositories along ownership and responsibility lines.

by zaphar 5y ago

MuffinFlavored 5y ago

> With the proper tooling and support Could you elaborate/provide some examples?

Jensson 5y ago
Google is the main example, their monorepo setup works and scales really well. It is basically a remote filesystem directory that automatically checks out the files you need to build your project and not other files.
- akvadrako 5y ago
  Why would that be needed except for huge codebases at megacorps?
  - pxue 5y ago
    no one needs this except maybe google. even then if they started today with git maybe they wouldn't have used a monorepo. same as pointing to netflix and arguing you need 100+ microservices. no one needs this, maybe except netflix
    - zaphar 5y ago
      Using a monorepo was a deliberate choice at Google. They modified their source control solution to support a monorepo every time it failed to scale sufficiently. If they had started with Git those same people would probably have modified Git to make it work as a monorepo. There are specific economies of scale Google gets with a monorepo that help it keep up with the sheer amount of cruft and technical debt they generate everyday. Google writes a lot of code and ensuring that security fixes get applied or legacy deprecated APIs get removed is for some people a full time job. Having a monorepo makes that job actually achievable rather than impossible. Source: I worked there and was familiar with many of the engineers working on the tooling for the monorepo.
zaphar 5y ago
In a monorepo you usually only want to build portions of the codebase at any given time in your CI/CD. Most of the CI/CD tooling out there are accidentally hostile to this. As just one example it is often difficult to impossible to rebuild your code when a dependency changes. Build systems often don't expose this information usefully, so you'll find yourself either hand rolling something that looks at git changes and tries to infer what should be rebuilt and what shouldn't. Or you just rebuild everything in the entire repo every night. Or you can adopt Bazel everywhere which can be a hard sell in many companies but does give you better tools. Another example: Most source code hosting products don't give good support for path based permissions which you need in order to ensure that folks with the right context review code before it is merged in. Which pushes you to segregate your repositories along ownership and responsibility lines.