4 ms·

Both monorepo or "micro repo" end up falling apart at scale without some devops work involved. Either will work if you only have a few dozen projects. Neither w

by shados 8y ago

Both monorepo or "micro repo" end up falling apart at scale without some devops work involved. Either will work if you only have a few dozen projects. Neither will work once you hit 10s of millions of lines of code. But people seem to forget that it wasn't that long ago that git didn't exist, making multiple repos was a pain in the butt. Managing multiple repos locally was hell. Monorepos were the norm. Then as the state of version control ramped up, and making repos became easy, and having so much code in one repo had performance issues (overnight CVS/SourceSafe/SVN pull on your first day at work anyone? Branches that take hours to create?), people started making repos per project. The micro-service fad made that a no-brainer. Now, for companies like Facebook and Google, or really any company that wrote code before the modern days and has a non-trivial amount of it, switching was not exactly a simple matter. So they just poured their energy into making the monorepo work. They're not the only ones to do it either (though not everyone has to do it at Google, Facebook or Microsoft scale, obviously, so its a bit easier for most). And so it works. And then people forget how to make distributed repos work and claim things like "omg I have to make 1 PR per repo when making breaking changes!", as if it was a big deal or it wasn't a solved problem.

hobls 8y ago
Absolutely! At some point you must invest in your tools. (Early, in my opinion.) I think the clarification I’d offer is that in the age of GitHub the “standard” model is multiple repos, so you’re actually giving up some tooling if you just shove everything in a single repo. (I’m also not sure I’d generally categorize tools work as as “dev ops,” though I can certainly see how they end up intertwined.)
- lclarkmichalek 8y ago
  I've seen hardly any tools to manage dependencies across multiple repos. Modifying multiple repos at the same time isn't an issue I see many resources devoted to, and managing those cross repo versions is almost never done well. In comparison, both buck and bazel offer pretty mature monorepo management tooling. On the VCS front, you can take native git/HG a long way.
fcarraldo 8y ago
People also seem to forget that "Monorepo" or (many) "Microrepos" is not a binary choice. You can have both tiny repositories which do a single thing and large repositories that consist of many projects. It's totally cool to have both, assuming your team can be trusted to make the appropriate choices as they create new projects. > And then people forget how to make distributed repos work and claim things like "omg I have to make 1 PR per repo when making breaking changes!", as if it was a big deal or it wasn't a solved problem. Is this a solved problem? I typically do make one PR per repo to resolve breaking changes, though it's certainly not a big deal. Still, if there's an easier way, I'd love to hear about it!
- shados 8y ago
  > Is this a solved problem I don't mean that it's magical, just that it's not particularly sorcery. Instead of making a breaking change, add new method, deprecate old method. Update projects, then get rid of old deprecated method. Because they're distinct you can do this one by one so some project can reap the benefits without having to wait until all the problems are solved. Some people in this thread act like its freagin impossible. Avoiding breaking changes in APIs or proper deprecation strategies is an art everyone developing software should know: sooner or later they'll have to contribute to an open source project or have to make a more complicated breaking change or SOMETHING and will have to deal with it. Even if they use a monorepo. And when it happens you don't want it to be the first time anyone deals with it.
  - iamforreal 8y ago
    You can also just have tooling to find every reference of a function and then refactoring all at once, sending it out in a single pr, but that's a bit more advanced
  - w_t_payne 8y ago
    We have about 400 repos in a team of about 20 developers. We do have extensive tooling to help coordinate all of these, but configuration management is still by far the biggest engineering challenge that we face. I don't recall having such issues when I was working with Subversion and Perforce. On the other hand, not everything was rosy in the 'good old days': MS Source Safe was (by far) the worst VCS experience that I have ever had.
ec109685 8y ago
Microsoft went from multiple smaller repos for windows to one large one: https://blogs.msdn.microsoft.com/bharry/2017/05/24/the-largest-git-repo-on-the-planet/ https://blogs.msdn.microsoft.com/bharry/2017/05/24/the-large...
AmericanChopper 8y ago
>Both monorepo or "micro repo" end up falling apart at scale without some devops work involved Wouldn’t any project fall apart without devops work?
- shados 8y ago
  yeah, that's my point.

by shados 8y ago

hobls 8y ago

Absolutely! At some point you must invest in your tools. (Early, in my opinion.) I think the clarification I’d offer is that in the age of GitHub the “standard” model is multiple repos, so you’re actually giving up some tooling if you just shove everything in a single repo. (I’m also not sure I’d generally categorize tools work as as “dev ops,” though I can certainly see how they end up intertwined.)

lclarkmichalek 8y ago
I've seen hardly any tools to manage dependencies across multiple repos. Modifying multiple repos at the same time isn't an issue I see many resources devoted to, and managing those cross repo versions is almost never done well. In comparison, both buck and bazel offer pretty mature monorepo management tooling. On the VCS front, you can take native git/HG a long way.

fcarraldo 8y ago

People also seem to forget that "Monorepo" or (many) "Microrepos" is not a binary choice. You can have both tiny repositories which do a single thing and large repositories that consist of many projects. It's totally cool to have both, assuming your team can be trusted to make the appropriate choices as they create new projects. > And then people forget how to make distributed repos work and claim things like "omg I have to make 1 PR per repo when making breaking changes!", as if it was a big deal or it wasn't a solved problem. Is this a solved problem? I typically do make one PR per repo to resolve breaking changes, though it's certainly not a big deal. Still, if there's an easier way, I'd love to hear about it!

shados 8y ago
> Is this a solved problem I don't mean that it's magical, just that it's not particularly sorcery. Instead of making a breaking change, add new method, deprecate old method. Update projects, then get rid of old deprecated method. Because they're distinct you can do this one by one so some project can reap the benefits without having to wait until all the problems are solved. Some people in this thread act like its freagin impossible. Avoiding breaking changes in APIs or proper deprecation strategies is an art everyone developing software should know: sooner or later they'll have to contribute to an open source project or have to make a more complicated breaking change or SOMETHING and will have to deal with it. Even if they use a monorepo. And when it happens you don't want it to be the first time anyone deals with it.
- iamforreal 8y ago
  You can also just have tooling to find every reference of a function and then refactoring all at once, sending it out in a single pr, but that's a bit more advanced
- w_t_payne 8y ago
  We have about 400 repos in a team of about 20 developers. We do have extensive tooling to help coordinate all of these, but configuration management is still by far the biggest engineering challenge that we face. I don't recall having such issues when I was working with Subversion and Perforce. On the other hand, not everything was rosy in the 'good old days': MS Source Safe was (by far) the worst VCS experience that I have ever had.

ec109685 8y ago

Microsoft went from multiple smaller repos for windows to one large one: https://blogs.msdn.microsoft.com/bharry/2017/05/24/the-largest-git-repo-on-the-planet/ https://blogs.msdn.microsoft.com/bharry/2017/05/24/the-large...

AmericanChopper 8y ago

>Both monorepo or "micro repo" end up falling apart at scale without some devops work involved Wouldn’t any project fall apart without devops work?

shados 8y ago
yeah, that's my point.