5 ms·

Google and Facebook are the largest examples of monorepos that I'm aware of. Here is a write-up about it overall, though you don't need to be convinced :) http

by Tushon 9y ago

Google and Facebook are the largest examples of monorepos that I'm aware of. Here is a write-up about it overall, though you don't need to be convinced :) https://danluu.com/monorepo/ https://danluu.com/monorepo/

bluejekyll 9y ago
Both Google and Facebook are also dealing with such large repos that they've needed to start either customizing or building their own SCM's. MS started the GitVFS project to do a similar thing to suit their needs. Most people aren't at that scale, but IMO, many benefits people get from monorepos you also get by using GitHub/Gitlab with master projects mapping in git repos via sub-modules. Anyway, it really sucks to work on extremely large monorepos when you don't have access to the same resources as Google and Facebook. For this reason I'm personally always hesitant to recommend monorepos as the be-all-end-all.
- muxator 9y ago
  A good number of engineers from facebbok and google are active developing on mercurial, even if it gets comparatively less attention from the big public. Facebook also maintains a set of custom extension for it [0], and there is an interesting talk about the reasons beyond their choice [1]. [0] https://phab.mercurial-scm.org/diffusion/FBHGX/ https://phab.mercurial-scm.org/diffusion/FBHGX/ [1] https://m.youtube.com/watch?v=gOVD-DrUpwQ https://m.youtube.com/watch?v=gOVD-DrUpwQ
  - lclarkmichalek 9y ago
    There is also Mononoke, a HG server being built in Rust: https://github.com/facebookexperimental/mononoke https://github.com/facebookexperimental/mononoke (work for FB)
    - muxator 9y ago
      That's really interesting. I've updated wikipedia mentioning this.
- davedx 9y ago
  > it really sucks to work on extremely large monorepos What kind of scale are we talking, and what issues do you get?
  - joshribakoff 9y ago
    I combined 50 repos into a mono repo. It sucks because git is very slow and the logs are noisy but having 50 repos sucks worse. Deploys also take forever but at least they're atomic now. The real issue was the previous devs copy and pasting codebases because hey heard microservices were web scale. Overall I'm happy with the tradeoff it's allowed me to unify the look and feel of the platform a lot easier by just doing platform wide find and replace, and also revert across the board if stuff goes wrong. It's also allowed me to start removing duplicate code bases. Instead of doing 50 repos for 50 apps, a better idea is to do layers. One repo for the backend. One for the frontend. One for the cache layer, etc. you start separating out your electron app from your web app into separate repos and you'll find you need even more repos for shared UI elements. It can easily lead to copy pasting or worse an explosion of repos. You don't want to tell your new hires "welcome to ABC corp. as your first task clone 90 repos and npm install each one". If you're going to do it at least write a script to set the whole build up in one go. Also keep in mind the tech giants mostly used monoliths up until thousands of employees before refactoring to microservices. For auth you should probably have every app validate tokens by going out over the network to the auth microservice. This way you can easily switch, for example from JWT to sessions, in one place.
  - bluejekyll 9y ago
    600k+ files at 12gigs of Repo (without history). I've been trying to work on what option we haven't to get off our old SCM. Right now Git is potentially too slow, and that's just the local system problem. Git LFS works decently well large files. I've explored lots of different options, and hope to look at mercurial at some point, but am not hopeful.
- corpMaverick 9y ago
  The fact that Google and Facebook go to such lengths means that there is a big benefit on using the monorepo. Or perhaps they are stuck with it. > For this reason I'm personally always hesitant to recommend monorepos as the be-all-end-all. May be not a monorepo, but at least trying avoid having too many repos.
  - bluejekyll 9y ago
    Again, I go back to the sub-module approach with some small amount of automation built around that. Managing sub-modules kinda sucks, but if you use it as a reference to all of the products in production and as a synchronization point for delivery, then it gives you all the benefits of the monorepo, without the performance issues.
fapjacks 9y ago
Uber as well.

by Tushon 9y ago

bluejekyll 9y ago

Both Google and Facebook are also dealing with such large repos that they've needed to start either customizing or building their own SCM's. MS started the GitVFS project to do a similar thing to suit their needs. Most people aren't at that scale, but IMO, many benefits people get from monorepos you also get by using GitHub/Gitlab with master projects mapping in git repos via sub-modules. Anyway, it really sucks to work on extremely large monorepos when you don't have access to the same resources as Google and Facebook. For this reason I'm personally always hesitant to recommend monorepos as the be-all-end-all.

muxator 9y ago
A good number of engineers from facebbok and google are active developing on mercurial, even if it gets comparatively less attention from the big public. Facebook also maintains a set of custom extension for it [0], and there is an interesting talk about the reasons beyond their choice [1]. [0] https://phab.mercurial-scm.org/diffusion/FBHGX/ https://phab.mercurial-scm.org/diffusion/FBHGX/ [1] https://m.youtube.com/watch?v=gOVD-DrUpwQ https://m.youtube.com/watch?v=gOVD-DrUpwQ
- lclarkmichalek 9y ago
  There is also Mononoke, a HG server being built in Rust: https://github.com/facebookexperimental/mononoke https://github.com/facebookexperimental/mononoke (work for FB)
  - muxator 9y ago
    That's really interesting. I've updated wikipedia mentioning this.
davedx 9y ago
> it really sucks to work on extremely large monorepos What kind of scale are we talking, and what issues do you get?
- joshribakoff 9y ago
  I combined 50 repos into a mono repo. It sucks because git is very slow and the logs are noisy but having 50 repos sucks worse. Deploys also take forever but at least they're atomic now. The real issue was the previous devs copy and pasting codebases because hey heard microservices were web scale. Overall I'm happy with the tradeoff it's allowed me to unify the look and feel of the platform a lot easier by just doing platform wide find and replace, and also revert across the board if stuff goes wrong. It's also allowed me to start removing duplicate code bases. Instead of doing 50 repos for 50 apps, a better idea is to do layers. One repo for the backend. One for the frontend. One for the cache layer, etc. you start separating out your electron app from your web app into separate repos and you'll find you need even more repos for shared UI elements. It can easily lead to copy pasting or worse an explosion of repos. You don't want to tell your new hires "welcome to ABC corp. as your first task clone 90 repos and npm install each one". If you're going to do it at least write a script to set the whole build up in one go. Also keep in mind the tech giants mostly used monoliths up until thousands of employees before refactoring to microservices. For auth you should probably have every app validate tokens by going out over the network to the auth microservice. This way you can easily switch, for example from JWT to sessions, in one place.
- bluejekyll 9y ago
  600k+ files at 12gigs of Repo (without history). I've been trying to work on what option we haven't to get off our old SCM. Right now Git is potentially too slow, and that's just the local system problem. Git LFS works decently well large files. I've explored lots of different options, and hope to look at mercurial at some point, but am not hopeful.
corpMaverick 9y ago
The fact that Google and Facebook go to such lengths means that there is a big benefit on using the monorepo. Or perhaps they are stuck with it. > For this reason I'm personally always hesitant to recommend monorepos as the be-all-end-all. May be not a monorepo, but at least trying avoid having too many repos.
- bluejekyll 9y ago
  Again, I go back to the sub-module approach with some small amount of automation built around that. Managing sub-modules kinda sucks, but if you use it as a reference to all of the products in production and as a synchronization point for delivery, then it gives you all the benefits of the monorepo, without the performance issues.

fapjacks 9y ago

Uber as well.