4 ms·

Having all code in a single repository increases developer productivity by lowering the barrier to change. You can make a single atomic commit in one repository

Having all code in a single repository increases developer productivity by lowering the barrier to change. You can make a single atomic commit in one repository as opposed to N commits in M repositories. This is much, much easier than dealing with subrepos, repo sync, etc. Unified repos scales well up to a certain point before troubles arise. e.g. fully distributed VCS starts to break down when you have hundreds of MB and people with slow internet connections. Large projects like the Linux kernel and Firefox are beyond this point. You also have implementation details such as Git's repacks and garbage collection that introduce performance issues. Facebook is a magnitude past where troubles begin. The fact they control the workstations and can throw fast disks, CPU, memory, and 1 gbps+ links at the problem has bought them time. Facebook made the determination that preserving a unified repository (and thus preserving developer productivity) was more important than dealing with the limitation of existing tools. So, they set out to improve one VCS system: Mercurial (https://code.facebook.com/posts/218678814984400/scaling-mercurial-at-facebook/ https://code.facebook.com/posts/218678814984400/scaling-merc...). They are effectively leveraging the extensibility of Mercurial to turn it from a fully distributed VCS to one that supports shallow clones (remotefilelog extension) and can leverage filesystem watching primitives to make I/O operations fast (hgwatchman) and more. Unlike compiled tools (like Git), Facebook doesn't have to wait for upstream to accept possibly-controversial and difficult-to-land enhancements or maintain a forked Git distribution. They can write Mercurial extensions and monkeypatch the core of Mercurial (written in Python) to prove out ideas and they can upstream patches and extensions to benefit everybody. Mercurial is happily accepting their patches and every Mercurial user is better off because of Facebook. Furthermore, Mercurial's extensibility makes it a perfect complement to a tailored and well-oiled development workflow. You can write Mercurial extensions that provide deep integration with existing tools and systems. See http://gregoryszorc.com/blog/2013/11/08/using-mercurial-to-query-mozilla-metadata/ http://gregoryszorc.com/blog/2013/11/08/using-mercurial-to-q.... There are many compelling reasons why you would want to choose Mercurial over other solutions. Those reasons are even more compelling in corporate environments (such as Facebook) where the network effect of Git + GitHub (IMO the foremost reason to use Git) doesn't significantly factor into your decision.

leccine 12y ago
Hello there, have you heard of service oriented architecture? You must be joking to justify a single repository with "easier to change". Your problem is that the code base must be tightly coupled if splitting the services out to different repos is not possible and you need to contribute to multiple repositories to get something done. I would say, the biggest change in Amazon's architecture was moving over to the service oriented way and it was worth the effort. Developers are forced to separate different functions to separate services and they are in charge of that service. If it goes down their are getting the alerts. All of the services are using 250ms timeouts so there is no cascading effect when a services goes down. The web page consists of few thousand service calls and it degrades gracefully. Facebook obviously have some tech depth that they need to fix. Using stupid design justified with some random crap that does not even make sense is not really acceptable (at least for me).
- indygreg2 12y ago
  SOA isn't a magic bullet. What if multiple services are utilizing a shared library? For each service to be independent in the way I think you are advocating for, you would need multiple copies of that shared library (either via separate copies in separate repos or a shared copy via something like subrepos). Multiple copies leads to copies getting out of sync. You (likely) lose the ability to perform a single atomic commit. Furthermore, you've increased the barrier to change (and to move fast) by introducing uncertainty. Are Service X and Service Y using the latest/greatest version of the library? Why did my change to this library break Service Z? Oh, it's because Service Z lags 3 versions behind on this library and can't talk with my new version. Unified repositories help eliminate the sync problem and make a whole class of problems that are detrimental to productivity and moving fast go away. Facebook isn't alone in making this decision. I believe Google maintains a large Perforce repository for the same reasons.
  - nuser 12y ago
    The advantages you ascribe to monolithic repos aren't due to monolithic repos, they're due to comprehensive tests. The multiple copies of a shared library argument is straight-up nonsense, because in the multiple repo scenario, there would be (one or more) repos of shared libraries. There wouldn't be copying. Unless the devs were morons. Builds in multiple repo environments are clearly identifiable, it's just by a combination of SHAs instead of a single SHA. In practice, this is a non-issue. Version clashes happen in every scenario. Unified repos create a horrific dependency hell, because it's impossible to have one service using version X of a lib while another service uses a version Y. Instead, if you want to update from X to Y, the entire codebase needs to get the update, no matter whether it needs it or not. It's a boondoggle. These decisions, as near as I can tell, are not made because they're good or bad, they're essentially arbitrary, related to what the first few engineers did. If it starts as a giant ball of code, it will always be a giant ball of code. If it starts out well-organized, it might remain well-organized.

leccine 12y ago

Hello there, have you heard of service oriented architecture? You must be joking to justify a single repository with "easier to change". Your problem is that the code base must be tightly coupled if splitting the services out to different repos is not possible and you need to contribute to multiple repositories to get something done. I would say, the biggest change in Amazon's architecture was moving over to the service oriented way and it was worth the effort. Developers are forced to separate different functions to separate services and they are in charge of that service. If it goes down their are getting the alerts. All of the services are using 250ms timeouts so there is no cascading effect when a services goes down. The web page consists of few thousand service calls and it degrades gracefully. Facebook obviously have some tech depth that they need to fix. Using stupid design justified with some random crap that does not even make sense is not really acceptable (at least for me).

indygreg2 12y ago
SOA isn't a magic bullet. What if multiple services are utilizing a shared library? For each service to be independent in the way I think you are advocating for, you would need multiple copies of that shared library (either via separate copies in separate repos or a shared copy via something like subrepos). Multiple copies leads to copies getting out of sync. You (likely) lose the ability to perform a single atomic commit. Furthermore, you've increased the barrier to change (and to move fast) by introducing uncertainty. Are Service X and Service Y using the latest/greatest version of the library? Why did my change to this library break Service Z? Oh, it's because Service Z lags 3 versions behind on this library and can't talk with my new version. Unified repositories help eliminate the sync problem and make a whole class of problems that are detrimental to productivity and moving fast go away. Facebook isn't alone in making this decision. I believe Google maintains a large Perforce repository for the same reasons.
- nuser 12y ago
  The advantages you ascribe to monolithic repos aren't due to monolithic repos, they're due to comprehensive tests. The multiple copies of a shared library argument is straight-up nonsense, because in the multiple repo scenario, there would be (one or more) repos of shared libraries. There wouldn't be copying. Unless the devs were morons. Builds in multiple repo environments are clearly identifiable, it's just by a combination of SHAs instead of a single SHA. In practice, this is a non-issue. Version clashes happen in every scenario. Unified repos create a horrific dependency hell, because it's impossible to have one service using version X of a lib while another service uses a version Y. Instead, if you want to update from X to Y, the entire codebase needs to get the update, no matter whether it needs it or not. It's a boondoggle. These decisions, as near as I can tell, are not made because they're good or bad, they're essentially arbitrary, related to what the first few engineers did. If it starts as a giant ball of code, it will always be a giant ball of code. If it starts out well-organized, it might remain well-organized.