5 ms·

So the motivation behind this project primarily is humongous monorepos that git can't handle? Are such huge monorepos a good idea in the first place? The idea b

So the motivation behind this project primarily is humongous monorepos that git can't handle? Are such huge monorepos a good idea in the first place? The idea behind the monorepo is that you don't have to wait for changes in a dependency to integrate — as soon as the change is checked into the dependency it is available to you. But this also means it is easy for a commit into a dependency to break a lot of downstream projects as it is not possible for the person maintaining the dependency to test all the dependent projects, unless it is acceptable to rely solely on test automation. It would be better for the owners of the dependent projects to decide when to take the new version of the dependency, after thorough testing.

MajimasEyepatch 2y ago
Basically all of the big tech companies run on giant monorepos.
jkaptur 2y ago
The idea is that it is acceptable to rely on test automation, and that teams should apply the "Beyonce Rule" - if you like it then you should've put a test on it.
- hinkley 2y ago
  If your tests contain 100% of your requirements then you can rewrite the code. If not then you’re fucked.
- zeroonetwothree 2y ago
  Test coverage at Meta is pretty far off from this
  - breadwinner 2y ago
    And that's true in most companies. So then the question is, how acceptable is it to cause a break in some dependent project by making a change in a dependency. That will depend on the company and the type of product. For a consumer product company like Meta this is probably ok, but for an enterprise company that makes mission-critical software it may be less ok.
    - deleted 2y ago
      [deleted]
  - amethyst 2y ago
    And yet, as someone working on core language infra, we apply exactly that sort of ideal when making changes. If a diff doesn't break any tests, then it's "safe" to land, and if something does indeed break afterwards, then it's the broken team's responsibility to fix forward or otherwise provide proof that it's a big enough problem to roll back. If we end up in SEV review for a change, and there were no broken tests on the diff, then there are going to be some hard questions for the team that didn't write tests. Ie, tests aren't mandatory, but if you aren't writing tests, it's your responsibility when someone else's change breaks your project.
    - breadwinner 2y ago
      Tests are hard for UI components. Even when the web page has all the expected elements, the appearance may be broken. At least for UI projects, your approach will fail.
oftenwrong 2y ago
These massive companies do employ a significant reliance on test automation. This approach is paired with sophisticated build tools and supporting infrastructure to make it possible to meaningfully test proposed changes, and ensure that they do not break any downstream projects. edit: This comment I made recently is relevant, since it speaks to the motivations for using a monorepo and trunk-based development within a business: https://news.ycombinator.com/item?id=41293123 https://news.ycombinator.com/item?id=41293123
munificent 2y ago
> But this also means it is easy for a commit into a dependency to break a lot of downstream projects as it is not possible for the person maintaining the dependency to test all the dependent projects, unless it is acceptable to rely solely on test automation. You can invert this sentence to imagine an alternate world: If you are able to rely completely on test automation, then it becomes possible for a person to commit a change to a dependency without breaking all downstream projects. Companies with successful monorepos live in that alternate world. Yes, it requires heavy investment in automated testing: you need a culture where every engineer writes tests for all code they write and edit all the time, you need good automated test tooling, and you need to pay for the servers to run the tests all the time. But if you have that, you get to avoid the hellscape of bit rotting dependencies that no one can touch because too much of the world sits on top of it. I used to work at EA, which at the time had a separate Perforce repo for every game. There was a big push to reuse libraries across multiple games, but game teams had control over when they updated those libraries. I spent six months of my life doing nothing but integrating upstream changes in some libraries into various game repos that were using them. Not entirely coincidentally, I left EA not too long after.
xpe 2y ago
> Are such huge monorepos a good idea in the first place? In just one question, the comment above has done an impressive trifecta; it (1) ventures off-topic; (2) invites a holy war; (3) nerd-snipes. So, let's see how this train wreck unfolds.
mhh__ 2y ago
> Are such huge monorepos a good idea Yes. Internal dependencies, automation, buildkite (i.e. let's run other projects CI against our changes), etc are all basically a scam that only makes "sense" in certain open source projects.

MajimasEyepatch 2y ago

Basically all of the big tech companies run on giant monorepos.

jkaptur 2y ago

The idea is that it is acceptable to rely on test automation, and that teams should apply the "Beyonce Rule" - if you like it then you should've put a test on it.

hinkley 2y ago
If your tests contain 100% of your requirements then you can rewrite the code. If not then you’re fucked.
zeroonetwothree 2y ago
Test coverage at Meta is pretty far off from this
- breadwinner 2y ago
  And that's true in most companies. So then the question is, how acceptable is it to cause a break in some dependent project by making a change in a dependency. That will depend on the company and the type of product. For a consumer product company like Meta this is probably ok, but for an enterprise company that makes mission-critical software it may be less ok.
  - deleted 2y ago
    [deleted]
- amethyst 2y ago
  And yet, as someone working on core language infra, we apply exactly that sort of ideal when making changes. If a diff doesn't break any tests, then it's "safe" to land, and if something does indeed break afterwards, then it's the broken team's responsibility to fix forward or otherwise provide proof that it's a big enough problem to roll back. If we end up in SEV review for a change, and there were no broken tests on the diff, then there are going to be some hard questions for the team that didn't write tests. Ie, tests aren't mandatory, but if you aren't writing tests, it's your responsibility when someone else's change breaks your project.
  - breadwinner 2y ago
    Tests are hard for UI components. Even when the web page has all the expected elements, the appearance may be broken. At least for UI projects, your approach will fail.

oftenwrong 2y ago

These massive companies do employ a significant reliance on test automation. This approach is paired with sophisticated build tools and supporting infrastructure to make it possible to meaningfully test proposed changes, and ensure that they do not break any downstream projects. edit: This comment I made recently is relevant, since it speaks to the motivations for using a monorepo and trunk-based development within a business: https://news.ycombinator.com/item?id=41293123 https://news.ycombinator.com/item?id=41293123

munificent 2y ago

> But this also means it is easy for a commit into a dependency to break a lot of downstream projects as it is not possible for the person maintaining the dependency to test all the dependent projects, unless it is acceptable to rely solely on test automation. You can invert this sentence to imagine an alternate world: If you are able to rely completely on test automation, then it becomes possible for a person to commit a change to a dependency without breaking all downstream projects. Companies with successful monorepos live in that alternate world. Yes, it requires heavy investment in automated testing: you need a culture where every engineer writes tests for all code they write and edit all the time, you need good automated test tooling, and you need to pay for the servers to run the tests all the time. But if you have that, you get to avoid the hellscape of bit rotting dependencies that no one can touch because too much of the world sits on top of it. I used to work at EA, which at the time had a separate Perforce repo for every game. There was a big push to reuse libraries across multiple games, but game teams had control over when they updated those libraries. I spent six months of my life doing nothing but integrating upstream changes in some libraries into various game repos that were using them. Not entirely coincidentally, I left EA not too long after.

xpe 2y ago

> Are such huge monorepos a good idea in the first place? In just one question, the comment above has done an impressive trifecta; it (1) ventures off-topic; (2) invites a holy war; (3) nerd-snipes. So, let's see how this train wreck unfolds.

mhh__ 2y ago

> Are such huge monorepos a good idea Yes. Internal dependencies, automation, buildkite (i.e. let's run other projects CI against our changes), etc are all basically a scam that only makes "sense" in certain open source projects.