3 ms·

Airbnb has a version of this internally that was pretty awesome called Evergreen, based on the Uber paper as well. I wish more companies would open source thei

by esprehn 2mo ago

Airbnb has a version of this internally that was pretty awesome called Evergreen, based on the Uber paper as well. I wish more companies would open source their monorepo infra. A well done monorepo is a huge force multiplier on a large organization, but the OSS world is lacking a lot of the infra so everyone starts from a painful place and works up or has a bad impression of monorepos. Google's Piper is another example where open sourcing or selling it would have done wonders for the industry. In an age of agents landing code very quickly piper has scaled very well since it was already at unimaginable commit velocity, meanwhile everyone else is trying to rebuild source control to keep up.

esafak 2mo ago
Keeping Master Green at Scale https://dl.acm.org/doi/10.1145/3302424.3303970 https://dl.acm.org/doi/10.1145/3302424.3303970
kyrra 2mo ago
The problem with many of these tools is they are built on lots of other tech, like S3 dev called out: https://x.com/haipingfu/status/2084880266858995990 https://x.com/haipingfu/status/2084880266858995990 Piper is built on lots of other tech like spanner and chubby and many others. And some of those have specific hardware requirements that only exist in Google datacenters. Untangling that web of tech is neigh impossible (or not something worth the cost to leadership).
- fragmede 2mo ago
  Why is that a problem? They sell Spanner on GCP. Sell Piper as well! It's obvious as shit but Google doesn't know what it's doing anymore. They should have bought GitHub, not Microsoft, and then we wouldn't have had the problems with scaling GitHub either.
  - kyrra 2mo ago
    Piper only works with CitC. Which is effectively a virtual file system when mounted locally. And when you have a monorepo that big, standard tools that want to scan an entire directory will stop working. Every part of the developer tool chain has to change to deal with code base that big. I agree that Google should have bought github. The tech they built to scale Google Code I think would have helped a ton with GitHub.
    - fragmede 2mo ago
      They sell and support Google Drive on Mac and windows, and the web. Selling a VFS layer is entirely within their capabilities. Blaze/Bazel has already been exported. You're right that there's a whole stack, but if Facebook can make jj happen, Google could make hg happen.
twic 2mo ago
> A well done monorepo is a huge force multiplier on a large organization How so? I work at a company which uses a monorepo, and I haven't seen any upside to it yet. We have a tools team that's invested a vast amount of work in it. Still seems strictly worse than a 'normal' polyrepo setup. I haven't understood why so many people are so enthusiastic about it.
- fastball 2mo ago
  Do you do any cross workspace/repo work, or are you mostly constrained to a single namespace?
  - malfist 2mo ago
    Not op but I do cross repo work, but it's rightfully separate PRs as its separate services, separate contracts and separate deployments. Pretending a monorepo cross service PRs are contiguous is a recipe for deployment race conditions
    - fastball 2mo ago
      Is it one or the other? You either need to keep everything in separate PRs or you need to pretend that services are contiguous where deployment order doesn't matter?
- IshKebab 2mo ago
  1. No submodules. They suck. They don't work with worktrees. They're a pain to work with. 2. Cross-project changes become trivial instead of nightmarish. 3. Testing becomes tractable. Make a change in a submodule? Good luck testing that it doesn't break any of the other repos that depend on it. You essentially turn its API into a fully public API, which introduces a ton of extra work (if you do it right, which nobody does).
mupuff1234 2mo ago
I don't understand how come monorepos never just got "solved", and why git didn't expand in that direction. I switched from a company with a monorepo to one without, and it just feels like going back to the stone age.
- fcarraldo 2mo ago
  I work at a monorepo company and any time someone gets to work on a project that necessitates working outside of the monorepo, it's a night-and-day improvement. Tools, especially open source ones (linters, static analysis, scanning, LSPs, IDEs, etc) are not built for monorepos, and with AI Agents working in the monorepo results in an enormous increase in input tokens as the agents are constantly trying to grep this giant source tree. I'm sure it's possible that we're doing the monorepo thing wrong, but I'm genuinely curious what the upside is that you're experiencing? Or are these drawbacks unique to our implementation?
  - plumeria 2mo ago
    > Tools, especially open source ones (linters, static analysis, scanning, LSPs, IDEs, etc) are not built for monorepos, and with AI Agents working in the monorepo results in an enormous increase in input tokens as the agents are constantly trying to grep this giant source tree. I've been thinking that one could dynamically patch .claude/settings.json (or its equivalent for other agents) to allow reads/writes only to the active app/package being edited and its dependencies (other packages/apps).
    - fcarraldo 2mo ago
      You can, if you have a trusted build graph. I know that “cone-shaped” checkout tools like this are common in monorepo environments, but unfortunately there aren’t any maintained open source implementations that I’m aware of.
    - handfuloflight 2mo ago
      This is a great idea.

by esprehn 2mo ago

esafak 2mo ago

Keeping Master Green at Scale https://dl.acm.org/doi/10.1145/3302424.3303970 https://dl.acm.org/doi/10.1145/3302424.3303970

kyrra 2mo ago

The problem with many of these tools is they are built on lots of other tech, like S3 dev called out: https://x.com/haipingfu/status/2084880266858995990 https://x.com/haipingfu/status/2084880266858995990 Piper is built on lots of other tech like spanner and chubby and many others. And some of those have specific hardware requirements that only exist in Google datacenters. Untangling that web of tech is neigh impossible (or not something worth the cost to leadership).

fragmede 2mo ago
Why is that a problem? They sell Spanner on GCP. Sell Piper as well! It's obvious as shit but Google doesn't know what it's doing anymore. They should have bought GitHub, not Microsoft, and then we wouldn't have had the problems with scaling GitHub either.
- kyrra 2mo ago
  Piper only works with CitC. Which is effectively a virtual file system when mounted locally. And when you have a monorepo that big, standard tools that want to scan an entire directory will stop working. Every part of the developer tool chain has to change to deal with code base that big. I agree that Google should have bought github. The tech they built to scale Google Code I think would have helped a ton with GitHub.
  - fragmede 2mo ago
    They sell and support Google Drive on Mac and windows, and the web. Selling a VFS layer is entirely within their capabilities. Blaze/Bazel has already been exported. You're right that there's a whole stack, but if Facebook can make jj happen, Google could make hg happen.

twic 2mo ago

> A well done monorepo is a huge force multiplier on a large organization How so? I work at a company which uses a monorepo, and I haven't seen any upside to it yet. We have a tools team that's invested a vast amount of work in it. Still seems strictly worse than a 'normal' polyrepo setup. I haven't understood why so many people are so enthusiastic about it.

fastball 2mo ago
Do you do any cross workspace/repo work, or are you mostly constrained to a single namespace?
- malfist 2mo ago
  Not op but I do cross repo work, but it's rightfully separate PRs as its separate services, separate contracts and separate deployments. Pretending a monorepo cross service PRs are contiguous is a recipe for deployment race conditions
  - fastball 2mo ago
    Is it one or the other? You either need to keep everything in separate PRs or you need to pretend that services are contiguous where deployment order doesn't matter?
IshKebab 2mo ago
1. No submodules. They suck. They don't work with worktrees. They're a pain to work with. 2. Cross-project changes become trivial instead of nightmarish. 3. Testing becomes tractable. Make a change in a submodule? Good luck testing that it doesn't break any of the other repos that depend on it. You essentially turn its API into a fully public API, which introduces a ton of extra work (if you do it right, which nobody does).

mupuff1234 2mo ago

I don't understand how come monorepos never just got "solved", and why git didn't expand in that direction. I switched from a company with a monorepo to one without, and it just feels like going back to the stone age.

fcarraldo 2mo ago
I work at a monorepo company and any time someone gets to work on a project that necessitates working outside of the monorepo, it's a night-and-day improvement. Tools, especially open source ones (linters, static analysis, scanning, LSPs, IDEs, etc) are not built for monorepos, and with AI Agents working in the monorepo results in an enormous increase in input tokens as the agents are constantly trying to grep this giant source tree. I'm sure it's possible that we're doing the monorepo thing wrong, but I'm genuinely curious what the upside is that you're experiencing? Or are these drawbacks unique to our implementation?
- plumeria 2mo ago
  > Tools, especially open source ones (linters, static analysis, scanning, LSPs, IDEs, etc) are not built for monorepos, and with AI Agents working in the monorepo results in an enormous increase in input tokens as the agents are constantly trying to grep this giant source tree. I've been thinking that one could dynamically patch .claude/settings.json (or its equivalent for other agents) to allow reads/writes only to the active app/package being edited and its dependencies (other packages/apps).
  - fcarraldo 2mo ago
    You can, if you have a trusted build graph. I know that “cone-shaped” checkout tools like this are common in monorepo environments, but unfortunately there aren’t any maintained open source implementations that I’m aware of.
  - handfuloflight 2mo ago
    This is a great idea.