26 ms·

Why did Facebook choose Mercurial over Git?

by mikevm 9y ago

michel-slm 9y ago
from a previous blog post: "Our engineers were comfortable with Git and we preferred to stay with a familiar tool, so we took a long, hard look at improving it to work at scale. After much deliberation, we concluded that Git's internals would be difficult to work with for an ambitious scaling project." https://code.facebook.com/posts/218678814984400/scaling-mercurial-at-facebook/ https://code.facebook.com/posts/218678814984400/scaling-merc...
- sanxiyn 9y ago
  Note that Microsoft arrived at the opposite conclusion and wrote GVFS: https://github.com/Microsoft/GVFS https://github.com/Microsoft/GVFS
  - deleted 9y ago
    [deleted]
  - krupan 9y ago
    Reading that it sounds like they reached the same conclusion, that they didn't want to mess with git's internals. They wrote a filesystem to run underneath git.
    - xpaulbettsx 9y ago
      If you look at that same repo, they're trying a similar approach, https://github.com/facebookexperimental/eden https://github.com/facebookexperimental/eden
    - daemin 9y ago
      Hearing from friends that work there, the RTT for the underlying filesystem adds quite a lot of time for daily operations, especially if they are not working on the west coast of the USA. It was said a pull takes 45 minutes. This leads me to believe that to handle large repos and files within a tool like git, its internals should be changed a bit so that there's not as many file accesses that need to be done (fstat, read, write). Also for certain operations to be batched together to better hide the latency involved in global communications.
      - kyrra 9y ago
        If you do it right, it will eventually work. Google does something similar with fuse and their mono repo (CITC is magic). https://m.cacm.acm.org/magazines/2016/7/204032-why-google-stores-billions-of-lines-of-code-in-a-single-repository/fulltext https://m.cacm.acm.org/magazines/2016/7/204032-why-google-st...
      - alexeiz 9y ago
        If you're working with a Git repo on a remote filesystem, you're doing it wrong. Git is not designed for remote filesystems. It relies on certain file operations to be extremely fast. So Git works best with local filesystems. With Git, you want to clone the entire repository locally and work with it locally. That's the idea behind the distributed version control: every committer has the entire copy of the repository. With a remote filesystem you're effectively centralizing your repository.
        daemin 9y ago
        Yeah, the git metadata is on a remote file server, that's the entire point of the Microsoft file system extensions for running a huge git monorepo. The code and assets you are working on is local, but the history and other metadata elements are stored remote and lazy loaded. So if you're remote file server is relatively close it doesn't matter too much and the lag is not noticeable, but if it's across the country or across the world...
  - kyrra 9y ago
    There was some interesting answers around this in the original thread for GVFS: https://news.ycombinator.com/item?id=15725497 https://news.ycombinator.com/item?id=15725497 The Microsoft PM for GVFS commented there.
  - OskarS 9y ago
    As a game dev in a small studio that runs on git, I'm so exited to see where GitVFS goes, and especially since the announcement they're working on macOS drivers. We're pushing git to it's limit, and it would really nice if we wouldn't have to switch to another system (git is mostly great, and we're all good at using it, even the artists). GitVFS seemingly solves all our problems in a really slick way.
pjmlp 9y ago
Probably because they prefer to use sane SCM systems.
- hiccuphippo 9y ago
  What is insane with the other ones?
  - pjmlp 9y ago
    Do you really need to ask regarding git, given the amount of tutorials how to rescue the working state back? https://xkcd.com/1597/ https://xkcd.com/1597/
    - peatmoss 9y ago
      I’ve taught an intro to git class a number of times to scientists wanting to build some basic software development skills (through Software Carpentry: https://software-carpentry.org/ https://software-carpentry.org/). It can be easy to forget that the cognitive overhead of version control is pretty extreme. Looking at git through beginners’ eyes is telling. And our curriculum basically gets people to a baseline proficiency with change, add, and commit plus pushing, pulling, and a little merging. In other words, scratching the surface. Scientists then start asking all the reasonable questions about restoring work under various scenarios, and whether they should store large amounts of data with their code. It’s in those moments where I start asking myself hard questions about git, its usability, and its architecture. Subversion at least had binary diffs and reasonable support for big files. Git may have won the day, but I cannot help but think that it’s a pretty major compromise (even a retrograde step) compared to what could have / should have been.
      - bachmeier 9y ago
        In my experience the most confusing thing for beginners is adding new files, committing, and pushing in separate steps. It also doesn't help that Google returns links to the "right way" to use Git - meaning advanced large enterprise usage.

Why did Facebook choose Mercurial over Git?

by mikevm 9y ago

Why did Facebook choose Mercurial over Git?

from a previous blog post: "Our engineers were comfortable with Git and we preferred to stay with a familiar tool, so we took a long, hard look at improving it to work at scale. After much deliberation, we concluded that Git's internals would be difficult to work with for an ambitious scaling project." https://code.facebook.com/posts/218678814984400/scaling-mercurial-at-facebook/ https://code.facebook.com/posts/218678814984400/scaling-merc...

sanxiyn 9y ago
Note that Microsoft arrived at the opposite conclusion and wrote GVFS: https://github.com/Microsoft/GVFS https://github.com/Microsoft/GVFS
- deleted 9y ago
  [deleted]
- krupan 9y ago
  Reading that it sounds like they reached the same conclusion, that they didn't want to mess with git's internals. They wrote a filesystem to run underneath git.
  - xpaulbettsx 9y ago
    If you look at that same repo, they're trying a similar approach, https://github.com/facebookexperimental/eden https://github.com/facebookexperimental/eden
  - daemin 9y ago
    Hearing from friends that work there, the RTT for the underlying filesystem adds quite a lot of time for daily operations, especially if they are not working on the west coast of the USA. It was said a pull takes 45 minutes. This leads me to believe that to handle large repos and files within a tool like git, its internals should be changed a bit so that there's not as many file accesses that need to be done (fstat, read, write). Also for certain operations to be batched together to better hide the latency involved in global communications.
    - kyrra 9y ago
      If you do it right, it will eventually work. Google does something similar with fuse and their mono repo (CITC is magic). https://m.cacm.acm.org/magazines/2016/7/204032-why-google-stores-billions-of-lines-of-code-in-a-single-repository/fulltext https://m.cacm.acm.org/magazines/2016/7/204032-why-google-st...
    - alexeiz 9y ago
      If you're working with a Git repo on a remote filesystem, you're doing it wrong. Git is not designed for remote filesystems. It relies on certain file operations to be extremely fast. So Git works best with local filesystems. With Git, you want to clone the entire repository locally and work with it locally. That's the idea behind the distributed version control: every committer has the entire copy of the repository. With a remote filesystem you're effectively centralizing your repository.
      - daemin 9y ago
        Yeah, the git metadata is on a remote file server, that's the entire point of the Microsoft file system extensions for running a huge git monorepo. The code and assets you are working on is local, but the history and other metadata elements are stored remote and lazy loaded. So if you're remote file server is relatively close it doesn't matter too much and the lag is not noticeable, but if it's across the country or across the world...
- kyrra 9y ago
  There was some interesting answers around this in the original thread for GVFS: https://news.ycombinator.com/item?id=15725497 https://news.ycombinator.com/item?id=15725497 The Microsoft PM for GVFS commented there.
- OskarS 9y ago
  As a game dev in a small studio that runs on git, I'm so exited to see where GitVFS goes, and especially since the announcement they're working on macOS drivers. We're pushing git to it's limit, and it would really nice if we wouldn't have to switch to another system (git is mostly great, and we're all good at using it, even the artists). GitVFS seemingly solves all our problems in a really slick way.

pjmlp 9y ago

Probably because they prefer to use sane SCM systems.

hiccuphippo 9y ago
What is insane with the other ones?
- pjmlp 9y ago
  Do you really need to ask regarding git, given the amount of tutorials how to rescue the working state back? https://xkcd.com/1597/ https://xkcd.com/1597/
  - peatmoss 9y ago
    I’ve taught an intro to git class a number of times to scientists wanting to build some basic software development skills (through Software Carpentry: https://software-carpentry.org/ https://software-carpentry.org/). It can be easy to forget that the cognitive overhead of version control is pretty extreme. Looking at git through beginners’ eyes is telling. And our curriculum basically gets people to a baseline proficiency with change, add, and commit plus pushing, pulling, and a little merging. In other words, scratching the surface. Scientists then start asking all the reasonable questions about restoring work under various scenarios, and whether they should store large amounts of data with their code. It’s in those moments where I start asking myself hard questions about git, its usability, and its architecture. Subversion at least had binary diffs and reasonable support for big files. Git may have won the day, but I cannot help but think that it’s a pretty major compromise (even a retrograde step) compared to what could have / should have been.
    - bachmeier 9y ago
      In my experience the most confusing thing for beginners is adding new files, committing, and pushing in separate steps. It also doesn't help that Google returns links to the "right way" to use Git - meaning advanced large enterprise usage.