7 ms·

The trick is that Google have their own VCS, build tooling, automated refactoring tools, etc etc, specifically designed to deal with their monorepo. Nobody else

The trick is that Google have their own VCS, build tooling, automated refactoring tools, etc etc, specifically designed to deal with their monorepo. Nobody else has that - we're stuck with git and a complex landscape of tools for managing code in ad-hoc ways. As a result, with the tools we have, many repos is better than a monorepo - but perhaps if we had those tools, for some cases, a monorepo might be better than many repos. Note that even where Google are forced to use git (e.g. Android, Chrome) they use a many-repo approach.

w_t_payne 9y ago
I have written my own tooling. It is doable. Just time-consuming. -- It took me about a year (2015) of my (limited) spare time.
user5994461 9y ago
Google has that, Facebook has that, Microsoft partially has that, some investment banks have that too. Everywhere I've seen mono repo, mono repo was better than multi repo. They all built special tooling and have dedicated teams to support it.
- perfmode 9y ago
  I wish someone would create an open-source vcs that supports mono-repos at scale out of the box.
  - albertzeyer 9y ago
    Microsoft has released some tools to support that for Git. https://blogs.msdn.microsoft.com/bharry/2017/05/24/the-largest-git-repo-on-the-planet/ https://blogs.msdn.microsoft.com/bharry/2017/05/24/the-large... https://blogs.msdn.microsoft.com/devops/2017/05/30/optimizing-git-beyond-gvfs/ https://blogs.msdn.microsoft.com/devops/2017/05/30/optimizin...
  - qznc 9y ago
    Why is the problem with Subversion?
    - hennsen 9y ago
      Are you proposing a subversion-based monorepo?
      - gmueckl 9y ago
        We get a lot of mileage out of Subversion as a monorepo. It certainly works better than people like to give it credit for.
      - qznc 9y ago
        Well, it seems to fit the requirements better than git. Obviously, subversion is not used much. I would like to hear some experience reports what is the problem with it.
        gmueckl 9y ago
        We have two issues with Subversion: - it requires a certain discipline: we need branching in our workflow and this is handled mostly by convention in a subversion repository. We have "branches" that were created by less careful colleagues by copying subdirectories of trunk to the branches folder. - all the tooling developers fled to work on making git bearable. It seems that there is good money in sugarcoating got and none in making good tools for Subversion (awareness of branches in Jenkins, decent code review...). We have a budget, but that does not compensate for the lead that git has in that regard. Other than that, subversion fits our needs. It just works.
        hennsen 9y ago
        Subversion is not used much anymore - just in case you entered the industry after this. Subversion has been used in basically each and every open source project as a replacement for the previously most used CVS. Subversion was better tjan cvs, but still bad in many aspects, slow synchronization and bad branching and merging support come to my mind. Because of these shortcomings and because of the idea of decentralized versioning coming up, many systems like git, mercurial, and others came up then, and git seems to be the most successful of these by now
  - supergreg 9y ago
    Maybe we could look at the problem from the other side. Create tools to manage multi-repos like if they were a single mono-repo. A docke-compose for git.
    - zeckalpha 9y ago
      https://myrepos.branchable.com https://myrepos.branchable.com
    - monocasa 9y ago
      It's sort of intrinsically a pain since then you lose atomic, cross project commits.
    - zifnab06 9y ago
      Android does this with a tool named repo. It tires into gerrit and you can treat the whole Android project as one repo.
- hello_there 9y ago
  What special tooling is required to deal with a monorepo that is not required for multi repo?
  - user5994461 9y ago
    Must have: Tooling that can interact on a file or sub directory level. Git cannot do that. Should have: Access control to view and change file on a subdirectory basis. Everyone can see the repo so you can't permissions users per repo anymore. It's optional but these companies have that. Recommended: Global search tools, global refactoring tools, global linting that can identify file types automatically and apply sane rules, unit test checks and on commit checks available out of the box for everything and that run remotely quickly, etc... It's regular tooling that every development company should have, but only big companies with mono repos have it. It's not that the tooling is needed to deal with the mono repo, it's that the tools are great and you want them. But they can't be implemented in a multi repo setup. Think of it. How could you have a global search tool in a multi repo setup? Most likely, you can't even identify what repo exists inside the company. Makes me realize. If I ever go back to another tech company, the shit tooling is gonna make me cry.
    - lclarkmichalek 9y ago
      > Must have: Tooling that can interact on a file or sub directory level. Git cannot do that. I mean, when you get big, sure. But until you're big, git is fine. Working at fb, I don't use some crazy invocation to replace `hg log -- ./subdir`, I just do `hg log -- ./subdir`. Sparse checkouts are useful, but their necessity is based on your scale - the bigger you are, the more you need them. Most companies aren't big enough to need them. > Should have: Access control to view and change file on a subdirectory basis. Everyone can see the repo so you can't permissions users per repo anymore. It's optional but these companies have that. Depends on your culture (and regulatory requirements). I prefer companies where anyone can modify anyone's code. > Recommended: Global search tools, global refactoring tools, global linting that can identify file types automatically and apply sane rules, unit test checks and on commit checks available out of the box for everything and that run remotely quickly, etc... I'd bump this up to `should have`. The power of a monorepo is being able to modify a lib that is used by everyone in the company, and have all of the dependencies recursively tested. Global search is required, but until you're big, ripgrep will probably be fine (and after that you just dump it into elasticsearch).
- robrenaud 9y ago
  Facebook is painfully non mono repo.
  - sah2ed 9y ago
    Dan Abramov's comment elsewhere in this thread says otherwise: "I work at Facebook, and can confirm we keep all code in a monorepo". https://news.ycombinator.com/item?id=15893184 https://news.ycombinator.com/item?id=15893184
    - robrenaud 9y ago
      Unless something drastic changed in the last year, I really doubt it. There is the fb frontend, the backend, the offline batch processing repo, and the instagram frontend repo. I think the phone apps have their own repos too? It was a giant mess, especially when you had to make changes that spanned repos, like introducing a new backend API and then depending on it, or changing logging formats.
fishywang 9y ago
> Note that even where Google are forced to use git (e.g. Android, Chrome) they use a many-repo approach. Google uses many-repo approach for Android and Chrome because you cannot fit everything in a single git repo (well you can, but it will be a pain in the ass to work on that repo). Git is just not designed for huge repos. Google is also working on tools to make the many-repo of Android or Chrome work like a monorepo.

w_t_payne 9y ago

I have written my own tooling. It is doable. Just time-consuming. -- It took me about a year (2015) of my (limited) spare time.

user5994461 9y ago

Google has that, Facebook has that, Microsoft partially has that, some investment banks have that too. Everywhere I've seen mono repo, mono repo was better than multi repo. They all built special tooling and have dedicated teams to support it.

perfmode 9y ago
I wish someone would create an open-source vcs that supports mono-repos at scale out of the box.
- albertzeyer 9y ago
  Microsoft has released some tools to support that for Git. https://blogs.msdn.microsoft.com/bharry/2017/05/24/the-largest-git-repo-on-the-planet/ https://blogs.msdn.microsoft.com/bharry/2017/05/24/the-large... https://blogs.msdn.microsoft.com/devops/2017/05/30/optimizing-git-beyond-gvfs/ https://blogs.msdn.microsoft.com/devops/2017/05/30/optimizin...
- qznc 9y ago
  Why is the problem with Subversion?
  - hennsen 9y ago
    Are you proposing a subversion-based monorepo?
    - gmueckl 9y ago
      We get a lot of mileage out of Subversion as a monorepo. It certainly works better than people like to give it credit for.
    - qznc 9y ago
      Well, it seems to fit the requirements better than git. Obviously, subversion is not used much. I would like to hear some experience reports what is the problem with it.
      - gmueckl 9y ago
        We have two issues with Subversion: - it requires a certain discipline: we need branching in our workflow and this is handled mostly by convention in a subversion repository. We have "branches" that were created by less careful colleagues by copying subdirectories of trunk to the branches folder. - all the tooling developers fled to work on making git bearable. It seems that there is good money in sugarcoating got and none in making good tools for Subversion (awareness of branches in Jenkins, decent code review...). We have a budget, but that does not compensate for the lead that git has in that regard. Other than that, subversion fits our needs. It just works.
      - hennsen 9y ago
        Subversion is not used much anymore - just in case you entered the industry after this. Subversion has been used in basically each and every open source project as a replacement for the previously most used CVS. Subversion was better tjan cvs, but still bad in many aspects, slow synchronization and bad branching and merging support come to my mind. Because of these shortcomings and because of the idea of decentralized versioning coming up, many systems like git, mercurial, and others came up then, and git seems to be the most successful of these by now
- supergreg 9y ago
  Maybe we could look at the problem from the other side. Create tools to manage multi-repos like if they were a single mono-repo. A docke-compose for git.
  - zeckalpha 9y ago
    https://myrepos.branchable.com https://myrepos.branchable.com
  - monocasa 9y ago
    It's sort of intrinsically a pain since then you lose atomic, cross project commits.
  - zifnab06 9y ago
    Android does this with a tool named repo. It tires into gerrit and you can treat the whole Android project as one repo.
hello_there 9y ago
What special tooling is required to deal with a monorepo that is not required for multi repo?
- user5994461 9y ago
  Must have: Tooling that can interact on a file or sub directory level. Git cannot do that. Should have: Access control to view and change file on a subdirectory basis. Everyone can see the repo so you can't permissions users per repo anymore. It's optional but these companies have that. Recommended: Global search tools, global refactoring tools, global linting that can identify file types automatically and apply sane rules, unit test checks and on commit checks available out of the box for everything and that run remotely quickly, etc... It's regular tooling that every development company should have, but only big companies with mono repos have it. It's not that the tooling is needed to deal with the mono repo, it's that the tools are great and you want them. But they can't be implemented in a multi repo setup. Think of it. How could you have a global search tool in a multi repo setup? Most likely, you can't even identify what repo exists inside the company. Makes me realize. If I ever go back to another tech company, the shit tooling is gonna make me cry.
  - lclarkmichalek 9y ago
    > Must have: Tooling that can interact on a file or sub directory level. Git cannot do that. I mean, when you get big, sure. But until you're big, git is fine. Working at fb, I don't use some crazy invocation to replace `hg log -- ./subdir`, I just do `hg log -- ./subdir`. Sparse checkouts are useful, but their necessity is based on your scale - the bigger you are, the more you need them. Most companies aren't big enough to need them. > Should have: Access control to view and change file on a subdirectory basis. Everyone can see the repo so you can't permissions users per repo anymore. It's optional but these companies have that. Depends on your culture (and regulatory requirements). I prefer companies where anyone can modify anyone's code. > Recommended: Global search tools, global refactoring tools, global linting that can identify file types automatically and apply sane rules, unit test checks and on commit checks available out of the box for everything and that run remotely quickly, etc... I'd bump this up to `should have`. The power of a monorepo is being able to modify a lib that is used by everyone in the company, and have all of the dependencies recursively tested. Global search is required, but until you're big, ripgrep will probably be fine (and after that you just dump it into elasticsearch).
robrenaud 9y ago
Facebook is painfully non mono repo.
- sah2ed 9y ago
  Dan Abramov's comment elsewhere in this thread says otherwise: "I work at Facebook, and can confirm we keep all code in a monorepo". https://news.ycombinator.com/item?id=15893184 https://news.ycombinator.com/item?id=15893184
  - robrenaud 9y ago
    Unless something drastic changed in the last year, I really doubt it. There is the fb frontend, the backend, the offline batch processing repo, and the instagram frontend repo. I think the phone apps have their own repos too? It was a giant mess, especially when you had to make changes that spanned repos, like introducing a new backend API and then depending on it, or changing logging formats.

fishywang 9y ago

> Note that even where Google are forced to use git (e.g. Android, Chrome) they use a many-repo approach. Google uses many-repo approach for Android and Chrome because you cannot fit everything in a single git repo (well you can, but it will be a pain in the ass to work on that repo). Git is just not designed for huge repos. Google is also working on tools to make the many-repo of Android or Chrome work like a monorepo.