4 ms·

Whenever the Git vs Mercurial topic comes up I think this thread [1] about Facebook having massive slowdowns with their large codebase using Git is quite intere

Whenever the Git vs Mercurial topic comes up I think this thread [1] about Facebook having massive slowdowns with their large codebase using Git is quite interesting. [1]: http://thread.gmane.org/gmane.comp.version-control.git/189776 http://thread.gmane.org/gmane.comp.version-control.git/18977...

the_mitsuhiko 13y ago
How is this related to mercurial?
- ngoldbaum 13y ago
  Facebook uses mercurial internally.
  - the_mitsuhiko 13y ago
    Source? That linked article is from when facebook used subversion.
    - tonfa 13y ago
      I don't know what they use in production, but Matt (mercurial project leader) currently works for FB: http://mercurial.selenic.com/wiki/mpm http://mercurial.selenic.com/wiki/mpm FB hired quite a few mercurial hacker, they host Mercurial sprints and have a big attendance from FB devs (e.g. London sprint): http://mercurial.selenic.com/wiki/2.6sprint http://mercurial.selenic.com/wiki/2.6sprint
    - ezquerra 13y ago
      The definitely do. They migrated their repositories to mercurial a while ago. There are plenty of mentions of this fact on mercurial's development mailing list (mercurial-devel@selenic.com). Facebook is a big user and backer of mercurial. I attended the last mercurial sprint in Facebook's London office (which was great, BTW). Facebook will also host the next mercurial sprint in New York. They have recently hired Matt Mackall, mercurial's creator. Several other mercurial core developers also work for Facebook and are paid to work on improving mercurial as their main job. I believe at some point they considered both git and mercurial as their new VCS. They have some huge repositories (hundreds of thousands if not millions of commits) and a huge amount of people accessing those repositories. I think they found some scalability issues with git's performance with repos of that scale (http://comments.gmane.org/gmane.comp.version-control.git/189776 http://comments.gmane.org/gmane.comp.version-control.git/189...). Apparently it was easier for them to improve mercurial's performance, perhaps because mercurial is written in python with some performance sensitive parts written in C. Over the last year they have made a lot of progress and mercurial's performance on huge repositories is now even better than it used to be.
    - edogawaconan 13y ago
      A bit of searching: http://www.reddit.com/r/IAmA/comments/1nl9at/i_am_a_member_of_facebooks_hhvm_team_a_c_and_d/ccjubpq http://www.reddit.com/r/IAmA/comments/1nl9at/i_am_a_member_o...
- garenp 13y ago
  Facebook tried to use git but ran into some scalability limits due to gits design (frequent index updates), whereas mercurial is append-only and doesn't have this problem. Hence, facebook has some legitimate technical reasons to not use git -- contrary to the common opinion that git is always "fast".
kyrra 13y ago
Thanks for the link. Interesting to see how slow it gets with such a large codebase. From what I could tell, people seem to think the project should be split into a number of smaller projects to fix the issue. This post brings up one of the major issues with DVCS, if you have a large single repo, it just doesn't play well with git/hg. This is even more apparently if you have a lot of binary objects.
- jlgreco 13y ago
  If you try to hammer a nail with a crowbar, you are going to experience some difficulty. Large projects like Android get along swimmingly on git because they have structured themselves such that git is the right tool for the job. If Android insisted on having one single git repo, despite that not being a git best practice, they would have trouble too. Structure things right and a git based system will scale far beyond what centralized monolithic repos like perforce can handle. If your codebase is growing properly big, splitting your code is something that you'll need to do eventually anyway.
  - taeric 13y ago
    This sounds strange, seeing as that the kernel is a single repository. Right? I would expect it to compete in size with most any other project. That not the case?
    - jlgreco 13y ago
      The kernel isn't actually that large. It should be sub-GB for the foreseeable future iirc. Single git repos get stretched to their limits when companies try to put 10-20 years worth of sourcecode from every single project they ever had (often with large binary files for testing or whatever) into a single repo because they are trying to use it like they used perforce. We're talking anything from several gigabytes up to the unimaginably large.
      - taeric 13y ago
        I was thinking more in terms of total lines of code. Is the "best practice" to archive code after a while? Would make some sense, but I'm not sure how that would work. Everyone has to make the switch at once, right?

the_mitsuhiko 13y ago

How is this related to mercurial?

ngoldbaum 13y ago
Facebook uses mercurial internally.
- the_mitsuhiko 13y ago
  Source? That linked article is from when facebook used subversion.
  - tonfa 13y ago
    I don't know what they use in production, but Matt (mercurial project leader) currently works for FB: http://mercurial.selenic.com/wiki/mpm http://mercurial.selenic.com/wiki/mpm FB hired quite a few mercurial hacker, they host Mercurial sprints and have a big attendance from FB devs (e.g. London sprint): http://mercurial.selenic.com/wiki/2.6sprint http://mercurial.selenic.com/wiki/2.6sprint
  - ezquerra 13y ago
    The definitely do. They migrated their repositories to mercurial a while ago. There are plenty of mentions of this fact on mercurial's development mailing list (mercurial-devel@selenic.com). Facebook is a big user and backer of mercurial. I attended the last mercurial sprint in Facebook's London office (which was great, BTW). Facebook will also host the next mercurial sprint in New York. They have recently hired Matt Mackall, mercurial's creator. Several other mercurial core developers also work for Facebook and are paid to work on improving mercurial as their main job. I believe at some point they considered both git and mercurial as their new VCS. They have some huge repositories (hundreds of thousands if not millions of commits) and a huge amount of people accessing those repositories. I think they found some scalability issues with git's performance with repos of that scale (http://comments.gmane.org/gmane.comp.version-control.git/189776 http://comments.gmane.org/gmane.comp.version-control.git/189...). Apparently it was easier for them to improve mercurial's performance, perhaps because mercurial is written in python with some performance sensitive parts written in C. Over the last year they have made a lot of progress and mercurial's performance on huge repositories is now even better than it used to be.
  - edogawaconan 13y ago
    A bit of searching: http://www.reddit.com/r/IAmA/comments/1nl9at/i_am_a_member_of_facebooks_hhvm_team_a_c_and_d/ccjubpq http://www.reddit.com/r/IAmA/comments/1nl9at/i_am_a_member_o...
garenp 13y ago
Facebook tried to use git but ran into some scalability limits due to gits design (frequent index updates), whereas mercurial is append-only and doesn't have this problem. Hence, facebook has some legitimate technical reasons to not use git -- contrary to the common opinion that git is always "fast".

kyrra 13y ago

Thanks for the link. Interesting to see how slow it gets with such a large codebase. From what I could tell, people seem to think the project should be split into a number of smaller projects to fix the issue. This post brings up one of the major issues with DVCS, if you have a large single repo, it just doesn't play well with git/hg. This is even more apparently if you have a lot of binary objects.

jlgreco 13y ago
If you try to hammer a nail with a crowbar, you are going to experience some difficulty. Large projects like Android get along swimmingly on git because they have structured themselves such that git is the right tool for the job. If Android insisted on having one single git repo, despite that not being a git best practice, they would have trouble too. Structure things right and a git based system will scale far beyond what centralized monolithic repos like perforce can handle. If your codebase is growing properly big, splitting your code is something that you'll need to do eventually anyway.
- taeric 13y ago
  This sounds strange, seeing as that the kernel is a single repository. Right? I would expect it to compete in size with most any other project. That not the case?
  - jlgreco 13y ago
    The kernel isn't actually that large. It should be sub-GB for the foreseeable future iirc. Single git repos get stretched to their limits when companies try to put 10-20 years worth of sourcecode from every single project they ever had (often with large binary files for testing or whatever) into a single repo because they are trying to use it like they used perforce. We're talking anything from several gigabytes up to the unimaginably large.
    - taeric 13y ago
      I was thinking more in terms of total lines of code. Is the "best practice" to archive code after a while? Would make some sense, but I'm not sure how that would work. Everyone has to make the switch at once, right?