5 ms·

Huh, fascinating. git was initially created for the Linux kernel development, and I haven't heard of any issues there. Offhand I would have said, as a codebase,

Huh, fascinating. git was initially created for the Linux kernel development, and I haven't heard of any issues there. Offhand I would have said, as a codebase, the Linux kernel would be larger and more complex than facebook, but I don't have a great sense of everything involved in both cases. So what's the story here: kernel developers put up with longer git times, the kernel is better organized, the scope of facebook is more massive even than the linux kernel, or there's some inherent design in git that works better for kernel work than web work?

cbs 15y ago
From the sounds of it facebook has a really, really big ball of highly coupled code.
- joelhaasnoot 15y ago
  Sounds like PHP. Oh wait, it is.
  - jonursenbach 15y ago
    Not really relevant.
    - joelhaasnoot 15y ago
      While it may not be, I do find it an every day battle to keep my PHP wel 'styled'. Sure, PHP is the first language I learnt, and I do use a framework, but sometimes a long method is easier in the short run than writing good model functions. And I have models, but mostly for the ORM and they're all completely interconnected. PHP makes me lazy, fast
      - mise 15y ago
        It's for this type of thinking that I've been looking at Rails recently.
        mise 15y ago
        If downvoting me, I would appreciate at least if you explain yourself :)
        mise 15y ago
        Funny.
      - philjohn 15y ago
        That's pretty much the same in any language - if you don't have the discipline to keep your code in a good state in one, why would you suddenly gain that discipline in another?
- haberman 15y ago
  Not necessarily. In the open-source world, when you want to change an API, you have to either add the change as a new API (leaving the existing API intact) or break backward compatibility and maintain parallel versions, gradually migrating users off of the old version. Both of these options are a huge pain, and have a direct cost (larger API surface or parallel maintenance/migration efforts). When your entire repo and all callers are in the same code-base, you have a much more attractive option: change the API and all callers in a single changelist. You've now cleaned up your API without incurring any of the costs of the two open-source options. This is why it can be nice, even if you have a bunch of nicely structured components, to have all code in a single repository.
  - deleted 15y ago
    [deleted]
marginalboy 15y ago
It isn't surprising if Facebook has a large, highly coupled code base. Given their reputation for tight timelines and maverick-advocacy, I'm continually surprised the thing works at all.
- nbm 15y ago
  I wouldn't say that a large repository implies that the code is highly-coupled. There are advantages for keeping certain code together in a single repo. Being able to easily discover users of functions of a library, being able to "transactionally" commit an update to a library (or set of libraries) and the code that uses it, being able to do code review over changes of code in various places, being able to discover if someone else has solved this problem before, and so forth. If you only have your project and its libraries checked out, you don't serendipitously discover things in other projects. As mentioned in this talk on how Facebook worked on visualizing interdependence between modules to drive down coupling at https://www.facebook.com/note.php?note_id=10150187460703920 https://www.facebook.com/note.php?note_id=10150187460703920 , there are at least 10k modules with clear dependency information in a front-end repo, and the situation probably is a lot better now that they have that information-dense representation to work from (I don't work on the PHP/www side of things, I spend most of my time in back-end and operations repos).
  - SomeCallMeTim 15y ago
    None of what you mention here precludes breaking up the code into many smaller repositories, and then having them all linked together in one super-repository. Then tags at the super-repository level can record the exact state of all submodules. It's not about not checking the other modules out; you can make this the standard behavior, sure. Instead it's about having git manage reasonable sized blocks of the code base.
    - nbm 15y ago
      I'm not sure this would work - if an operation needs you to stat() a file for every file in the repo (for example), whether it is 10k files in 1 repo or 1k files each in 10 repos will probably just as bad?
zargon 15y ago
The linux kernel has an order of magnitude fewer files than Facebook's test repository (25,000 in version 2.6.27, according to http://www.schoenitzer.de/lks/lks_en.html http://www.schoenitzer.de/lks/lks_en.html) and only 9 million lines of text. This is on the largish side for a single project, but if Facebook likes to keep all their work in single repo then it isn't too difficult to go way beyond those stats. Think of keeping all GNU projects in a single repo.
aidenn0 15y ago
The linux kernel is several orders of magnitude smaller. They are talking about 1.3 million files totalling nearly 10GB for the working tree. My kernel checkout has 39 thousand files totaling 489MB.

cbs 15y ago

From the sounds of it facebook has a really, really big ball of highly coupled code.

joelhaasnoot 15y ago
Sounds like PHP. Oh wait, it is.
- jonursenbach 15y ago
  Not really relevant.
  - joelhaasnoot 15y ago
    While it may not be, I do find it an every day battle to keep my PHP wel 'styled'. Sure, PHP is the first language I learnt, and I do use a framework, but sometimes a long method is easier in the short run than writing good model functions. And I have models, but mostly for the ORM and they're all completely interconnected. PHP makes me lazy, fast
    - mise 15y ago
      It's for this type of thinking that I've been looking at Rails recently.
      - mise 15y ago
        If downvoting me, I would appreciate at least if you explain yourself :)
        mise 15y ago
        Funny.
    - philjohn 15y ago
      That's pretty much the same in any language - if you don't have the discipline to keep your code in a good state in one, why would you suddenly gain that discipline in another?
haberman 15y ago
Not necessarily. In the open-source world, when you want to change an API, you have to either add the change as a new API (leaving the existing API intact) or break backward compatibility and maintain parallel versions, gradually migrating users off of the old version. Both of these options are a huge pain, and have a direct cost (larger API surface or parallel maintenance/migration efforts). When your entire repo and all callers are in the same code-base, you have a much more attractive option: change the API and all callers in a single changelist. You've now cleaned up your API without incurring any of the costs of the two open-source options. This is why it can be nice, even if you have a bunch of nicely structured components, to have all code in a single repository.
- deleted 15y ago
  [deleted]

marginalboy 15y ago

It isn't surprising if Facebook has a large, highly coupled code base. Given their reputation for tight timelines and maverick-advocacy, I'm continually surprised the thing works at all.

nbm 15y ago
I wouldn't say that a large repository implies that the code is highly-coupled. There are advantages for keeping certain code together in a single repo. Being able to easily discover users of functions of a library, being able to "transactionally" commit an update to a library (or set of libraries) and the code that uses it, being able to do code review over changes of code in various places, being able to discover if someone else has solved this problem before, and so forth. If you only have your project and its libraries checked out, you don't serendipitously discover things in other projects. As mentioned in this talk on how Facebook worked on visualizing interdependence between modules to drive down coupling at https://www.facebook.com/note.php?note_id=10150187460703920 https://www.facebook.com/note.php?note_id=10150187460703920 , there are at least 10k modules with clear dependency information in a front-end repo, and the situation probably is a lot better now that they have that information-dense representation to work from (I don't work on the PHP/www side of things, I spend most of my time in back-end and operations repos).
- SomeCallMeTim 15y ago
  None of what you mention here precludes breaking up the code into many smaller repositories, and then having them all linked together in one super-repository. Then tags at the super-repository level can record the exact state of all submodules. It's not about not checking the other modules out; you can make this the standard behavior, sure. Instead it's about having git manage reasonable sized blocks of the code base.
  - nbm 15y ago
    I'm not sure this would work - if an operation needs you to stat() a file for every file in the repo (for example), whether it is 10k files in 1 repo or 1k files each in 10 repos will probably just as bad?

zargon 15y ago

The linux kernel has an order of magnitude fewer files than Facebook's test repository (25,000 in version 2.6.27, according to http://www.schoenitzer.de/lks/lks_en.html http://www.schoenitzer.de/lks/lks_en.html) and only 9 million lines of text. This is on the largish side for a single project, but if Facebook likes to keep all their work in single repo then it isn't too difficult to go way beyond those stats. Think of keeping all GNU projects in a single repo.

aidenn0 15y ago

The linux kernel is several orders of magnitude smaller. They are talking about 1.3 million files totalling nearly 10GB for the working tree. My kernel checkout has 39 thousand files totaling 489MB.