8 ms·

I know this is slightly tangental but I'm always a little shocked that Facebook (and I think Google to some extent) have massive mono repositories. The benefit

by agentgt 10y ago

I know this is slightly tangental but I'm always a little shocked that Facebook (and I think Google to some extent) have massive mono repositories. The benefits of having one repository do not seem to be worth the serious performance issues as well as potential coupling that can happen with a gigantic code base as well also making much more difficult to OSS certain parts. e.g. why doesn't FB use dependency management (or binary package management aka maven, npm, etc)? That is have multiple repositories and cut releases. Build tools to help cut and manage the releases and dependency graph of the releases. There are even plugins and shell scripts that will make Mercurial act like a mono repository for many small repositories (I use it for our own Java Maven code base). I must be missing some killer features and would love to see it in action (FB major repository).

steveklabnik 10y ago
I haven't worked on a monorepo, but I will say that Facebook _does_ use at least some of these tools, apparently: enough so that they created a whole replacement for the npm client, yarn. From talking to some companies that use monorepos that want to use Rust, they _do_ use these kinds of tools, but want support for it in the tool. Firefox, for example, is pretty much a monorepo, and we built tooling in Cargo to help. I think that part of it is that "monorepo" can mean slightly different things. Does it only contain your companies' code, or also your dependencies? Do you modify those dependencies in-repo?
- agentgt 10y ago
  Yeah I'm not even sure I understand what FB's monorepository is so it is probably unfair critique. I can definitely see an "app" (Firefox) as a monorepo but not giant web companies (particularly with the surge of microservices). BTW Cargo is fantastic (and I'm fairly picky on package management / build tools).
gcp 10y ago
OSSing certain parts is certainly an issue, but what do you mean with "potential coupling" being an issue? That's exactly the advantage, right?
- agentgt 10y ago
  I suppose you could make the argument the other way but my thoughts are that by having the code all together it maybe confusing what is public API (ie interfaces / contracts) and what is not. I know personally that I have had to move components into separate projects (not necessarily repositories but compile units) to avoid developers from accessing things they shouldn't (e.g. access the database directly instead of going through a different layer). Separate repositories would be enforcing it to a greater extent. But like I said earlier I suppose I could see it the other way (ie visibility and thus prevention).
  - barkingcat 10y ago
    I don't use version control systems to any extent but it seems like your developer problem is an issue of team-coherence and code review, and not a matter that your vcs should/need to enforce. Even in a distributed team like linux kernel development, code maintainers have the ability to say "this is how we (as core maintainers) want you to access certain data structure. If we catch you doing anything else illegally, your code won't be accepted, and we will keep rejecting your code until you adhere to our coding style and api design."
    - agentgt 10y ago
      While I appreciate the "code review should be improved" it is fairly easy to accidentally import something you should not have in many IDEs (the merge the class path) as well as other accidents. These issues along with many others including code formatting can be automated. Separate projects allow less namespace collision accidents. You could achieve all of this with monorepo but requires proper tooling.
jcranmer 10y ago
Anyone who is old enough to remember the pain of VCSes like CVS and RCS will note that the number one feature touted to move to any other VCS is invariably "atomic changesets"--all the necessary changes to files are listed in a single changeset. Monorepos are nothing less than remembering the value of those atomic changesets. An example of utility of monorepos is things like automation--if you change, for example, how you publish packages, and you need to maintain multiple stable branches, it is immensely useful to keep the automation steps in the same repository as code. If you don't, your automation repository then looks like if version < 31: step_a() step_b() step_c() elif version < 35: step_a() step_bv2() step_d() else: ... which quickly grows unmaintainable at scale.
- brazzledazzle 10y ago
  I can only imagine that the only way to scale that would to make whoever owns the repo responsible for automation as well. A dedicated team can provide the tooling/frameworks (basically a "golden path") but if/how would need to be up to the team. I think in a general sense this is the approach Netflix takes to services. That said I have no idea how Netflix does source control, only that they use BitBucket/Stash on-prem and that supports both mercurial and git.
- agentgt 10y ago
  If you do it right your automation steps are just another package dependency that is versioned (I know because this is how we do it). We have a single bash script that will auto update itself (think akin to home brew albeit in bash). The only immensely dangerous thing that can happen is if you drop your package repository or change formats of the repository which rarely happens.
cmrdporcupine 10y ago
Been at Google for coming up 5 years and I don't think I'm giving away any secrets when I say don't recall there being any performance issues with the mono repository. And coupling is what is explicitly being sought, not rejected. The whole point is to build everything off head, keep head sane at all times, and avoid version dependency hell.
- pierrebai 10y ago
  Then maybe you can answer a few questions: - Do you have zero external dependencies on 3rd party library not own by Google? These must still be managed anyway? How do you deal with project X is not ready to move to external library version N but project Y needs version N? - What about release branches? If you need to intergrate a bug-fix in a sub-system, the magic mono-repo now means merging a fix is harder as it may depend on other unrelated changes all over teh repo? The laissez-faire attitude of not having to care about details in HEAD would seem to bite back in release branches.
  - cmrdporcupine 10y ago
    I am not sure how much of that I can answer in detail, sorry. But yes we have working strategies for dealing with both situations.
    - tonfa 10y ago
      http://m.cacm.acm.org/magazines/2016/7/204032-why-google-stores-billions-of-lines-of-code-in-a-single-repository/fulltext?mobile=true http://m.cacm.acm.org/magazines/2016/7/204032-why-google-sto... has a lot of answers (was published last summer).
  - z3t4 10y ago
    I think its insane not to have dependencies and third party code in your scm. Although it seems facebook plan to move those to a package manager due to performance problems and libs touching ten thousand files in a path release.
masklinn 10y ago
> e.g. why doesn't FB use dependency management (or binary package management aka maven, npm, etc)? That is have multiple repositories and cut releases. Build tools to help cut and manage the releases and dependency graph of the releases. Because it's a pain in the ass and a monorepo means you don't have to deal with that crap for internal code. It's especially important for refactorings or API changes where you can atomically perform a change at once rather than have to wait for the changes to trickle down across hundreds of repositories over days or weeks. > potential coupling that can happen with a gigantic code base The coupling is a feature not a bug. Decoupling is a mean not an end, and it has costs. If you don't need it you'd rather not pay it. Same as generic collections, if you don't need them there's nothing wrong with specific ones, quite the opposite.
avar 10y ago
As someone who works at another company with a big monorepo, briefly: * Ability to change an API and all its users at the same time. * Circular dependencies become a non-issue in a lot of cases where the would be if you vendor your dependencies. * Even if you vendor your dependencies hunting for bugs is a lot easier, your bisect of a bug in a library will just come down the commit that upgraded it from 1.0 to 2.0 without a monorepo, with a monorepo you can bisect anything down to a specific commit. * Code discoverability / mass edits are a lot easier, and integrate seamlessly into a lot more tools than if you need to manage multiple checkouts etc.
- hinkley 10y ago
  The big thing I miss about Subversion is that you could check out one subtree. On the biggest SVN project I worked on, we built 5 binaries from the same code tree, but only the leads and a couple of senior devs that worked on crosscutting concerns (ie, the people who would do the mass edits you mention) had the entire tree checked out. Everyone else had just the one or two modules they were working on. The second biggest, we used just enough of a module system to enforce modularity by having a single code repo but separate compilation units. Your circular dependencies between subprojects would show up at compile time (or at least, on a clean build, as our CI machine did). Stopped a lot of obscure runtime issues and made people think about what they were trying to do.
  - tux1968 10y ago
    This post is about Mercurial with which i'm not familiar, but in Git you can checkout a subtree since version 1.7[1]. It's commonly called "sparse checkout". [1] http://jasonkarns.com/blog/subdirectory-checkouts-with-git-sparse-checkout/ http://jasonkarns.com/blog/subdirectory-checkouts-with-git-s...
    - glandium 10y ago
      But you still need to clone the whole thing. Sparse clones are theoretically possible, but aren't implemented AFAIK.
the_duke 10y ago
When you split your libraries into separate projects, you have to start versioning them. And every update to the code requires other teams to then update their library/app to use the new version, and apps depending on that dependency... you get my drift. You end up with a complicated dependency headache which hurts productivity. With a monorepo, you can statically identify all places where your library is used and update those automatically with tooling, or manually. You can also monitor where and how a piece of code is used across the company, etc. If you then ensure that only commits get accepted that pass the relevant tests, you end up with a sane and working HEAD that's always up to date. Of course there are lot's of drawbacks as well. But Google and FB seem to have concluded that this is the better approach for them.
- groovy2shoes 10y ago
  > When you split your libraries into separate projects, you have to start versioning them. Or you can have your tooling use the convenient built-in versioning provided by the VCS. It's not called a "version control system" for nothing. > And every update to the code requires other teams to then update their library/app to use the new version, and apps depending on that dependency... you get my drift. This happens regardless of whether the codebase is organized as a monorepo or not. I've been subjected to monorepos at my previous and current employers, and we run into difficulties with this all the time. At least once a quarter, I get an e-mail from someone telling me that they updated such-and-such, and now the build fails in one of my projects. So I have to sit there and figure out wth they changed, why it's causing my code to fail the build, and how to fix it. Only once was the problem actually in my code (my Makefile, actually, which failed to bind some variable that was expected by the build system but wasn't documented anywhere, and of course the build system spits out a worthless error message, but I digress). > You end up with a complicated dependency headache which hurts productivity. Again, monorepos aren't immune from this, nor do polyrepos inherently suffer from it. > With a monorepo, you can statically identify all places where your library is used and update those automatically with tooling, or manually. You can also monitor where and how a piece of code is used across the company, etc. There is literally nothing preventing this from being doable with a polyrepo. In the case of updates, you may need to have the updated repos checked out, but it's not like you can only ever have one repo checked out at a time. > If you then ensure that only commits get accepted that pass the relevant tests, you end up with a sane and working HEAD that's always up to date. I've never worked anywhere on any project that disallowed commits that didn't pass the tests. It's typically been up to the committer to ensure that what they commit is acceptable. Furthermore, while my employers thus far have all been customers of AccuRev or Perforce (whose only real feature beyond what Subversion offers is that they cost lots of money), and have not personally had the pleasure of working with a DVCS at my day job, this workflow of "allow only test-passing commits" is wholly contrary to the "many small commits in quick succession" workflow afforded by DVCSs, which I contend is one of the biggest productivity boosts afforded by DVCSs (along with cheap branches and network unnecessity) and one of their main draws. And even so, it's still rather easy to maintain a sane and working HEAD with a DVCS (thanks to the cheap branches if nothing else). If you find that you can't do anything you've listed here without a monorepo, that's a tooling issue rather than an organizational one. Frankly, I've never seen a large monorepo I felt was justified. Every single one was just baggage held over from a time long past when a single repo still made sense for the codebase.
adrianratnapala 10y ago
Regarding performance: these mono-repo tools work differently and are optimised for their workload. Regarding why the hell you would do it: In a corporate setting you tend to throw all kinds of st into your repo. In my workplace that includes, network configurations, 200 MB binary tools, usernames (and who knows, passwords too probably). We do this because we want all the different things, including external dependencies and "ops stuff" to be syncronised. That would be totally insane in the open source world where you are releasing to the public -- but it seems corporations can make it work internally.
jordigh 10y ago
This is a FAQ: http://danluu.com/monorepo/ http://danluu.com/monorepo/
warbiscuit 10y ago
Aside from all the other technical bits, one killer feature for my company is TortoiseHG (http://tortoisehg.bitbucket.org/ http://tortoisehg.bitbucket.org/). Unlike a number of the other Tortoise-* projects (which don't seem related actually), TortoiseHG is a really great cross-platform gui, written in python & qt -- runs just about everywhere, and makes working with mercurial uniformly easy. The fact that instead of "hg ci" I can type "thg ci" and get a commit window with cherry-picking and meld integration right off the bat is really powerful. I've not seen any good git guis that come close to it's featureset.
- rjbwork 10y ago
  Yeah, THg is the absolute best DVCS GUI client I've had the pleasure of using. Everything else is awful and confusing in comparison IMO. I even use it for Git with hggit sometimes.
smegel 10y ago
Maybe Docker is the solution. Your "working state" is now in an image (maybe a Docker file that checks out particular revisions of different repos), so putting everything into one giant VC repo is not necessary. A versioned Dockerfile that says "this is how we built SystemX at version 1.5" is much better than doing it in Git, as it also covers how the underlying server was built, while a giant git repo might have every application you need at the correct version, but won't say anything about the server it is deployed to.
- jahewson 10y ago
  The advantage of a monorepo is that you have atomic commits. What you're suggesting does not provide that.
jahewson 10y ago
> There are even plugins and shell scripts that will make Mercurial act like a mono repository for many small repositories Seems kinda backwards to me, why not project portions of a monorepo to behave like individual repositories? That way you get atomic commits too.
- agentgt 10y ago
  > Seems kinda backwards to me, why not project portions of a monorepo to behave like individual repositories? That way you get atomic commits too. But on the other hand you now need custom scripts to filter out other projects that are completely irrelevant to you. Just the .hgtags file alone must be a 10k line nightmare... or maybe the big boys just don't deploy/release often. It is a lot more tooling than I think you might think. Linux and Firefox having a monorepository is not the same as a whole company as big as google and facebook having a single repository for all of their projects. I'm not even sure if these companies really are doing that. The use of monorepository could just mean a single place of storage but maybe they do allow a couple of trees (or maybe not).
amluto 10y ago
As an open-source example, the Linux kernel is a mono repository. It makes refactoring easier, and it makes it much easier to drop support for old (or not so old) unused features.
- agentgt 10y ago
  The Linux Kernel and Firefox (mentioned earlier) are not really good examples. I would put those in a single repository as I would imagine most people would. But a whole company as big as FB and Google using a single tree seems like an incoherent nightmare without proper (somewhat proprietary) tooling (filtering of logs, branches, tags, etc).

by agentgt 10y ago

steveklabnik 10y ago

I haven't worked on a monorepo, but I will say that Facebook _does_ use at least some of these tools, apparently: enough so that they created a whole replacement for the npm client, yarn. From talking to some companies that use monorepos that want to use Rust, they _do_ use these kinds of tools, but want support for it in the tool. Firefox, for example, is pretty much a monorepo, and we built tooling in Cargo to help. I think that part of it is that "monorepo" can mean slightly different things. Does it only contain your companies' code, or also your dependencies? Do you modify those dependencies in-repo?

agentgt 10y ago
Yeah I'm not even sure I understand what FB's monorepository is so it is probably unfair critique. I can definitely see an "app" (Firefox) as a monorepo but not giant web companies (particularly with the surge of microservices). BTW Cargo is fantastic (and I'm fairly picky on package management / build tools).

gcp 10y ago

OSSing certain parts is certainly an issue, but what do you mean with "potential coupling" being an issue? That's exactly the advantage, right?

agentgt 10y ago
I suppose you could make the argument the other way but my thoughts are that by having the code all together it maybe confusing what is public API (ie interfaces / contracts) and what is not. I know personally that I have had to move components into separate projects (not necessarily repositories but compile units) to avoid developers from accessing things they shouldn't (e.g. access the database directly instead of going through a different layer). Separate repositories would be enforcing it to a greater extent. But like I said earlier I suppose I could see it the other way (ie visibility and thus prevention).
- barkingcat 10y ago
  I don't use version control systems to any extent but it seems like your developer problem is an issue of team-coherence and code review, and not a matter that your vcs should/need to enforce. Even in a distributed team like linux kernel development, code maintainers have the ability to say "this is how we (as core maintainers) want you to access certain data structure. If we catch you doing anything else illegally, your code won't be accepted, and we will keep rejecting your code until you adhere to our coding style and api design."
  - agentgt 10y ago
    While I appreciate the "code review should be improved" it is fairly easy to accidentally import something you should not have in many IDEs (the merge the class path) as well as other accidents. These issues along with many others including code formatting can be automated. Separate projects allow less namespace collision accidents. You could achieve all of this with monorepo but requires proper tooling.

jcranmer 10y ago

Anyone who is old enough to remember the pain of VCSes like CVS and RCS will note that the number one feature touted to move to any other VCS is invariably "atomic changesets"--all the necessary changes to files are listed in a single changeset. Monorepos are nothing less than remembering the value of those atomic changesets. An example of utility of monorepos is things like automation--if you change, for example, how you publish packages, and you need to maintain multiple stable branches, it is immensely useful to keep the automation steps in the same repository as code. If you don't, your automation repository then looks like if version < 31: step_a() step_b() step_c() elif version < 35: step_a() step_bv2() step_d() else: ... which quickly grows unmaintainable at scale.

brazzledazzle 10y ago
I can only imagine that the only way to scale that would to make whoever owns the repo responsible for automation as well. A dedicated team can provide the tooling/frameworks (basically a "golden path") but if/how would need to be up to the team. I think in a general sense this is the approach Netflix takes to services. That said I have no idea how Netflix does source control, only that they use BitBucket/Stash on-prem and that supports both mercurial and git.
agentgt 10y ago
If you do it right your automation steps are just another package dependency that is versioned (I know because this is how we do it). We have a single bash script that will auto update itself (think akin to home brew albeit in bash). The only immensely dangerous thing that can happen is if you drop your package repository or change formats of the repository which rarely happens.

cmrdporcupine 10y ago

Been at Google for coming up 5 years and I don't think I'm giving away any secrets when I say don't recall there being any performance issues with the mono repository. And coupling is what is explicitly being sought, not rejected. The whole point is to build everything off head, keep head sane at all times, and avoid version dependency hell.

pierrebai 10y ago
Then maybe you can answer a few questions: - Do you have zero external dependencies on 3rd party library not own by Google? These must still be managed anyway? How do you deal with project X is not ready to move to external library version N but project Y needs version N? - What about release branches? If you need to intergrate a bug-fix in a sub-system, the magic mono-repo now means merging a fix is harder as it may depend on other unrelated changes all over teh repo? The laissez-faire attitude of not having to care about details in HEAD would seem to bite back in release branches.
- cmrdporcupine 10y ago
  I am not sure how much of that I can answer in detail, sorry. But yes we have working strategies for dealing with both situations.
  - tonfa 10y ago
    http://m.cacm.acm.org/magazines/2016/7/204032-why-google-stores-billions-of-lines-of-code-in-a-single-repository/fulltext?mobile=true http://m.cacm.acm.org/magazines/2016/7/204032-why-google-sto... has a lot of answers (was published last summer).
- z3t4 10y ago
  I think its insane not to have dependencies and third party code in your scm. Although it seems facebook plan to move those to a package manager due to performance problems and libs touching ten thousand files in a path release.

masklinn 10y ago

> e.g. why doesn't FB use dependency management (or binary package management aka maven, npm, etc)? That is have multiple repositories and cut releases. Build tools to help cut and manage the releases and dependency graph of the releases. Because it's a pain in the ass and a monorepo means you don't have to deal with that crap for internal code. It's especially important for refactorings or API changes where you can atomically perform a change at once rather than have to wait for the changes to trickle down across hundreds of repositories over days or weeks. > potential coupling that can happen with a gigantic code base The coupling is a feature not a bug. Decoupling is a mean not an end, and it has costs. If you don't need it you'd rather not pay it. Same as generic collections, if you don't need them there's nothing wrong with specific ones, quite the opposite.

avar 10y ago

As someone who works at another company with a big monorepo, briefly: * Ability to change an API and all its users at the same time. * Circular dependencies become a non-issue in a lot of cases where the would be if you vendor your dependencies. * Even if you vendor your dependencies hunting for bugs is a lot easier, your bisect of a bug in a library will just come down the commit that upgraded it from 1.0 to 2.0 without a monorepo, with a monorepo you can bisect anything down to a specific commit. * Code discoverability / mass edits are a lot easier, and integrate seamlessly into a lot more tools than if you need to manage multiple checkouts etc.

hinkley 10y ago
The big thing I miss about Subversion is that you could check out one subtree. On the biggest SVN project I worked on, we built 5 binaries from the same code tree, but only the leads and a couple of senior devs that worked on crosscutting concerns (ie, the people who would do the mass edits you mention) had the entire tree checked out. Everyone else had just the one or two modules they were working on. The second biggest, we used just enough of a module system to enforce modularity by having a single code repo but separate compilation units. Your circular dependencies between subprojects would show up at compile time (or at least, on a clean build, as our CI machine did). Stopped a lot of obscure runtime issues and made people think about what they were trying to do.
- tux1968 10y ago
  This post is about Mercurial with which i'm not familiar, but in Git you can checkout a subtree since version 1.7[1]. It's commonly called "sparse checkout". [1] http://jasonkarns.com/blog/subdirectory-checkouts-with-git-sparse-checkout/ http://jasonkarns.com/blog/subdirectory-checkouts-with-git-s...
  - glandium 10y ago
    But you still need to clone the whole thing. Sparse clones are theoretically possible, but aren't implemented AFAIK.

the_duke 10y ago

When you split your libraries into separate projects, you have to start versioning them. And every update to the code requires other teams to then update their library/app to use the new version, and apps depending on that dependency... you get my drift. You end up with a complicated dependency headache which hurts productivity. With a monorepo, you can statically identify all places where your library is used and update those automatically with tooling, or manually. You can also monitor where and how a piece of code is used across the company, etc. If you then ensure that only commits get accepted that pass the relevant tests, you end up with a sane and working HEAD that's always up to date. Of course there are lot's of drawbacks as well. But Google and FB seem to have concluded that this is the better approach for them.

groovy2shoes 10y ago
> When you split your libraries into separate projects, you have to start versioning them. Or you can have your tooling use the convenient built-in versioning provided by the VCS. It's not called a "version control system" for nothing. > And every update to the code requires other teams to then update their library/app to use the new version, and apps depending on that dependency... you get my drift. This happens regardless of whether the codebase is organized as a monorepo or not. I've been subjected to monorepos at my previous and current employers, and we run into difficulties with this all the time. At least once a quarter, I get an e-mail from someone telling me that they updated such-and-such, and now the build fails in one of my projects. So I have to sit there and figure out wth they changed, why it's causing my code to fail the build, and how to fix it. Only once was the problem actually in my code (my Makefile, actually, which failed to bind some variable that was expected by the build system but wasn't documented anywhere, and of course the build system spits out a worthless error message, but I digress). > You end up with a complicated dependency headache which hurts productivity. Again, monorepos aren't immune from this, nor do polyrepos inherently suffer from it. > With a monorepo, you can statically identify all places where your library is used and update those automatically with tooling, or manually. You can also monitor where and how a piece of code is used across the company, etc. There is literally nothing preventing this from being doable with a polyrepo. In the case of updates, you may need to have the updated repos checked out, but it's not like you can only ever have one repo checked out at a time. > If you then ensure that only commits get accepted that pass the relevant tests, you end up with a sane and working HEAD that's always up to date. I've never worked anywhere on any project that disallowed commits that didn't pass the tests. It's typically been up to the committer to ensure that what they commit is acceptable. Furthermore, while my employers thus far have all been customers of AccuRev or Perforce (whose only real feature beyond what Subversion offers is that they cost lots of money), and have not personally had the pleasure of working with a DVCS at my day job, this workflow of "allow only test-passing commits" is wholly contrary to the "many small commits in quick succession" workflow afforded by DVCSs, which I contend is one of the biggest productivity boosts afforded by DVCSs (along with cheap branches and network unnecessity) and one of their main draws. And even so, it's still rather easy to maintain a sane and working HEAD with a DVCS (thanks to the cheap branches if nothing else). If you find that you can't do anything you've listed here without a monorepo, that's a tooling issue rather than an organizational one. Frankly, I've never seen a large monorepo I felt was justified. Every single one was just baggage held over from a time long past when a single repo still made sense for the codebase.

adrianratnapala 10y ago

Regarding performance: these mono-repo tools work differently and are optimised for their workload. Regarding why the hell you would do it: In a corporate setting you tend to throw all kinds of st into your repo. In my workplace that includes, network configurations, 200 MB binary tools, usernames (and who knows, passwords too probably). We do this because we want all the different things, including external dependencies and "ops stuff" to be syncronised. That would be totally insane in the open source world where you are releasing to the public -- but it seems corporations can make it work internally.

jordigh 10y ago

This is a FAQ: http://danluu.com/monorepo/ http://danluu.com/monorepo/

warbiscuit 10y ago

Aside from all the other technical bits, one killer feature for my company is TortoiseHG (http://tortoisehg.bitbucket.org/ http://tortoisehg.bitbucket.org/). Unlike a number of the other Tortoise-* projects (which don't seem related actually), TortoiseHG is a really great cross-platform gui, written in python & qt -- runs just about everywhere, and makes working with mercurial uniformly easy. The fact that instead of "hg ci" I can type "thg ci" and get a commit window with cherry-picking and meld integration right off the bat is really powerful. I've not seen any good git guis that come close to it's featureset.

rjbwork 10y ago
Yeah, THg is the absolute best DVCS GUI client I've had the pleasure of using. Everything else is awful and confusing in comparison IMO. I even use it for Git with hggit sometimes.

smegel 10y ago

Maybe Docker is the solution. Your "working state" is now in an image (maybe a Docker file that checks out particular revisions of different repos), so putting everything into one giant VC repo is not necessary. A versioned Dockerfile that says "this is how we built SystemX at version 1.5" is much better than doing it in Git, as it also covers how the underlying server was built, while a giant git repo might have every application you need at the correct version, but won't say anything about the server it is deployed to.

jahewson 10y ago
The advantage of a monorepo is that you have atomic commits. What you're suggesting does not provide that.

jahewson 10y ago

> There are even plugins and shell scripts that will make Mercurial act like a mono repository for many small repositories Seems kinda backwards to me, why not project portions of a monorepo to behave like individual repositories? That way you get atomic commits too.

agentgt 10y ago
> Seems kinda backwards to me, why not project portions of a monorepo to behave like individual repositories? That way you get atomic commits too. But on the other hand you now need custom scripts to filter out other projects that are completely irrelevant to you. Just the .hgtags file alone must be a 10k line nightmare... or maybe the big boys just don't deploy/release often. It is a lot more tooling than I think you might think. Linux and Firefox having a monorepository is not the same as a whole company as big as google and facebook having a single repository for all of their projects. I'm not even sure if these companies really are doing that. The use of monorepository could just mean a single place of storage but maybe they do allow a couple of trees (or maybe not).

amluto 10y ago

As an open-source example, the Linux kernel is a mono repository. It makes refactoring easier, and it makes it much easier to drop support for old (or not so old) unused features.

agentgt 10y ago
The Linux Kernel and Firefox (mentioned earlier) are not really good examples. I would put those in a single repository as I would imagine most people would. But a whole company as big as FB and Google using a single tree seems like an incoherent nightmare without proper (somewhat proprietary) tooling (filtering of logs, branches, tags, etc).